pax_global_header00006660000000000000000000000064130345011050014502gustar00rootroot0000000000000052 comment=d8a08185e9f3f2d14edf5ade2d64af152a58ea91 weboob-1.2/000077500000000000000000000000001303450110500126215ustar00rootroot00000000000000weboob-1.2/.ci/000077500000000000000000000000001303450110500132725ustar00rootroot00000000000000weboob-1.2/.ci/requirements.txt000066400000000000000000000001271303450110500165560ustar00rootroot00000000000000coverage==4.2 flake8==3.2.1 mock==2.0.0 nose==1.3.7 pyflakes==1.3.0 xunitparser==1.3.3 weboob-1.2/.ci/requirements_modules.txt000066400000000000000000000000431303450110500203030ustar00rootroot00000000000000BeautifulSoup html2text simplejson weboob-1.2/.gitignore000066400000000000000000000002171303450110500146110ustar00rootroot00000000000000*_ui.py *.pyc *.swp Session.vim build dist *.egg-info *~ tags docs/source/api modules/modules.list /localconfig *.idea/ *.DS_Store *.coverage* weboob-1.2/.gitlab-ci.yml000066400000000000000000000010611303450110500152530ustar00rootroot00000000000000image: "python:2.7" before_script: - "pip install -r .ci/requirements.txt" - "REQUIREMENTS=$(mktemp) && python setup.py requirements > ${REQUIREMENTS} && pip install -r ${REQUIREMENTS} && rm ${REQUIREMENTS}" - "pip install -r .ci/requirements_modules.txt" build: stage: "build" script: - "./tools/local_install.sh ~/bin" lint: stage: "test" script: - "./tools/pyflakes.sh" - "./tools/weboob_lint.sh" unittests: stage: "test" script: - "./tools/run_tests.sh" doc: stage: "deploy" script: - "cd ./docs && make html" weboob-1.2/.mailmap000066400000000000000000000032041303450110500142410ustar00rootroot00000000000000Romain Bignon Romain Bignon Juke Juke Julien Hebert Clément Schreiner Noé Rubinstein Johann Broudin Florent Fourcot Florent Florent Fourcot Florent Fourcot Florent Fourcot ffourcot Florent Fourcot Florent Fourcot Nicolas Duhamel Gabriel Kerneis Christophe Lampin Kitof Ahmed Boussadia Ahmed Bousadia Simon Murail smurail Vincent Paredes Jean-Philippe Dutrève Jean-Philippe Dutrève Jean-Philippe Dutreve Baptiste Delpey bdelpey Samuel Loury Vincent Ardisson Benjamin CARTON Bezleputh weboob-1.2/AUTHORS000066400000000000000000000245531303450110500137020ustar00rootroot00000000000000Weboob is a free software written by: Romain Bignon * Core team; * Qt Applications; * Maintainer of too many modules. Laurent Bachelier * Core team; * Citelis, Pastealacon, Pastebin, PayPal, RadioFrance and YouTube modules maintainer. Florent Fourcot * Release manager; * Maintainer of too many modules. Benjamin Carton * Core team; * Maintainer of too many modules: Julien Hebert * Developer of masstransit; * Inrocks, LeFigaro, Minutes20 and Transilien modules maintainer. Noe Rubinstein * Bototo, Eatmanga, MangaFox, MangaHere, MangaReader, Mangatoshokan and Simplyreadit modules maintainer. Julien Veyssier * Suboob, Cineoob, QCineoob, Cookboob, QCookboob, Booblyrics and QBooblyrics developer; * CreditMutuel, Geolocip, Ipinfodb, Tvsubtitle, Piratebay, Podnapisi, Attilasub, Opensubtitles, Kickass, Btmon, Imdb, Allocine, Cpasbien, Marmiton, 750g, Allrecipes, CuisineAZ, T411, Parolesnet, Paroles2chansons, Parolesmusique and Parolesmania modules maintainer. Christophe Benz * Bouygues, INA, SFR and Youtube modules maintainer. * Ex-member of the core team. Vincent A. * funmooc, github, guerrillamail, imgur, lutim, mailinator, ovs, pariskiwi, pixtoilelibre, quvi, sueurdemetal, unsee modules maintainer; * Many patches on too many code parts. * Port of qt applications to qt5. Baptiste Delpey * Many fixes and improvement on bank and bill modules. * Author of CapBankTransfer API. Lucas Verney (Phyks) * Author of gitlab-ci part. * Fixes on many tests. * Fixes on some modules. Edouard Lambert * Many fixes and improvement of bank modules. Roger Philibert * Ehentai, Izneo, JacquieEtMichel, OKC and Youjizz modules maintainer. Nicolas Duhamel * BP, CanalPlus and Orange modules maintainer. Clément Schreiner * MediaWiki and Newsfeed modules maintainer. François Revol * Europarl, GDCVault, linuxjobs and Vimeo modules maintainer. * Canaplus and qvideoob fixes * Patchs for Haiku support * Patchs on dailymotion and arte module. * Ambassador of weboob in magazine and conferences. Pierre Mazière * RockRadio module maintainer; * lcl ex-maintener; * BNPorc fixes. Xavier Guerrin * Author of the CrAgr module. Gabriel Kerneis * Boursorama module maintainer. Oleg Plakhotniuk * Maintainer of Walls Fargo and CitiBank; * Improvements on the PayPal module; * Various fixes; * Several tools. Vicnet * lacentrale module maintainer. Kevin Pouget * CreditCooperatif module maintainer. Gilles Quenot * Fortuneo module maintainer. Mathieu Jourdan * GdfSuez module maintainer. Benjamin Drieu * TricTracTV module maintainer. Jeremy Monnet * Opacwebaloes module maintainer. Lord * CappedTV module maintainer. Lucien Loiseau * GoogleTranslate module maintainer. Christophe Lampin * HelloBank, ameli and amelipro modules maintainer. * Patchs on cragr and freemobile Sébastien Monel * QHandJoob application. Alexandre Lissy * JVMalin module maintainer. Arno Renevier * Weather module maintainer. Matthieu Weber * Unpronounceable and torrentz modules maintainer. * Fix on formatter encoding. * Fixes on t411, btdigg, and more. * Improvement of encoding output on boobank. Jocelyn Jaubert * SocieteGenerale module maintainer. Alexandre Morignot * residentadivsor module maintainer. * Documentation to write a module with Browser2 * Adapt munin scripts to python3 Johann Broudin * CMB module maintainer. * Fix on virginradio. Guilhem Bonnefille * RMLL module maintainer. Cedric Defortis * MeteoFrance module maintainer. P4ncake * Vice module maintainer. Matthieu Rakotojaona * Btdigg module maintainer. François D. * ArretsurImages module maintainer. Christophe Gouiran * EDF module maintainer. Cédric Félizard * KiwiBank module maintainer. * Support of currencies in boobank. Hervé Werner * Velib module maintainer. Thomas Lecavelier * Patches on radioob. * nihonnooto module maintainer. Benjamin Bouvier * Number26 maintener. Clément Calmels * Patches on NoLifeTV. Gabriel Serme * Support of two-factor authentication for Boursorama. Jean-Philippe Dutrève * Fixes on bank modules. Vincent Texier * Fixes on YouJizz and Dailymotion. Ahmed Boussadia * Fixes on OKC module. John Morrow * OFX formatter in boobank. Mathieu Lordon * Fixes on bank modules. Erwan Jahier * Patches on BP. Yann Rouillard * Add kids.dailymotion.com for dailymotion Raphaël Rigo * Patch for poleemploi * support for credit card history in boursorama theo * Ergonomics enhancements. Willy Villard * Patches on modules. Alexandre Flament * Script contrib/downloadboob. BohwaZ * Script contrib/report_accounts.sh. Grmbl Frechneu * Anonymizer script. Jérôme Poisson * Patches on webcontentedit. Laurent Dufréchou * Windows support. Fabien Grumelard * Fixes in FranceTelevision. leto * AuM anti-spam updates. Vincent Paredes * Changes in Repl. * Many fixes and patchs on banking modules. Richard Genoud * Fixes on CrArg. Jean-Christophe Dubacq * Fix on FreeMobile. Lucas Nussbaum * Fix on Credit Mutuel. Jerem * Fix on BP authentication. Camille Baldock * Fixes on BP. Alexandre Franke * Contribution to boobank. Loic Bontonou * Fix on arretsurimages module. Olivier Schwander * Fix XDG support. Luc Didry * Fixes on LCL. Jean-Benoist Leger * Fix on Dailymotion. Stefano Zacchiroli * Patch on the build system. Adrien Kunysz * Fixes on setup.py. Michael Scherer * Contribution on weboob-config. Damien Cassou * Minor fixes on housing modules. ianux * Patches on videoob. Laurent George * Fixes on AuM. Romain Garbage * Fixes for FreeBSD. Pierre-Louis Bonicoli * Fix on bash completion script. * Fixes on documentation Philippe Fremy * Fixes for Windows. Antoine * QVideoob fixes. Ryan Nowakowski * Import error. David Jeanneteau * Fix on ING and icon of ING. Juliette Stehlé * Enable LDD and PEL accounts on Boursorama. Etienne Carriere * Fix on dailymotion Simon Murail * Contribution to bank capability and browser2. * Add deepcopy to BaseObject Alexandre Bonhomme * Fix and tests on several modules Camille Dehecq * Contributions on weboob-generic munin script Samuel Loury * Fix on the documentation Caram Dache * Patch on boobank OFX export Mickaël Thomas * Patch to use gpg by default rather than gpg James Galt * Many fixes and improvement on bank and bill modules. Simon Lipp * Add url attribute to BaseObject, and fill it. * Tapatalk and openedx maintener. Yannick Heinrich * Documentation for installation statement on OSX Roland Mas * imm-o-matic script author Jean Walrave * Fix on bill and bank modules. Jonathan Schmidt * Fix on bill and bank modules. ZeHiro * Add support for banque de Savoir on banquepopulaire. Edouard Lefebvre du Prey * Patch on browser. * Add italian translation for dates. Julien Danjou * Fix on seloger. Edmond Van Overtveldt Julia Leven Jérémy Bossut Valentin Lefils * Tests for the browser Discover how to contribute to this great software: http://dev.weboob.org/guides/contribute weboob-1.2/CONTRIBUTING.md000066400000000000000000000025761303450110500150640ustar00rootroot00000000000000How to contribute ================= Write a patch ------------- Help yourself with the `documentation `_. Find an opened issue on `this website `_, or write your own bugfix or feature. Then, once it is necessary, commit with:: $ git commit -a Do not forget to write a helpful commit message. Check your patch ---------------- You can run these scripts to be sure your patch doesn't break anything:: $ tools/pyflakes.sh $ tools/weboob_lint.sh $ tools/run_tests.sh yourmodulename # or without yourmodulename to test everything Perhaps you should also write or fix tests. These tests are automatically run by `Gitlab CI `_ at each commit and merge requests. Create a merge request or send a patch -------------------------------------- The easiest way to send your patch is to create a fork on `the Weboob Gitlab `_ and create a merge request from there. This way, the code review process is easier and continuous integration is run automatically (see previous section). If you prefer good old email patches, just use :: $ git format-patch -n -s origin Then, send them with this command:: $ git send-email --to=weboob@weboob.org *.patch You can also send the files by yourself if you haven't any configured MTA on your system. weboob-1.2/COPYING000066400000000000000000001033301303450110500136540ustar00rootroot00000000000000 GNU AFFERO GENERAL PUBLIC LICENSE Version 3, 19 November 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software. A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public. The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version. An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU Affero General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Remote Network Interaction; Use with the GNU General Public License. Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements. You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see . weboob-1.2/ChangeLog000066400000000000000000005753721303450110500144160ustar00rootroot00000000000000Weboob 1.2 (2017-01-08) General * Add inheritance for modules, browsers and pages * Add gitlab-ci integration * Move all Qt applications to Qt5 * New application QBooblyrics * New urls for source repositories: https://git.weboob.org/ General: removed modules * Remove btdigg module, website is dead General: new modules * New module afer https://adherent.gie-afer.fr (CapBank) * New module amundi https://www.amundi-ee.com (CapBank) * New module caels https://www.ca-els.com/ (CapBank) * New module creditdunordpee https://salaries.pee.credit-du-nord.fr (CapBank) * New module number26 https://n26.com (CapBank) * New module spirica https://www.spirica.fr/ (CapBank) * New module yomoni https://www.yomoni.fr/ (CapBank) * New module centquatre https://billetterie.104.fr (CapCalendarEvent) * New module materielnet http://www.materiel.net/ (CapDocument) * New module onlinenet https://www.online.net (CapDocument) * New module trainline https://www.trainline.fr/ (CapDocument) * New module linuxjobs https://www.linuxjobs.fr/ (CapJob) * New module manpower http://www.manpower.fr/ (CapJob) * New module lyricsdotcom http://www.lyrics.com/ (CapLyrics) * New module lyricsmode http://www.lyricsmode.com/ (CapLyrics) * New module lyricsplanet http://www.lyricsplanet.com/ (CapLyrics) * New module paroles2chansons http://www.paroles2chansons.com/ (CapLyrics) * New module openedx (CapMessages) * New module tapatakl (CapMessages) * New module jirafeau https://jirafeau.net/ (CapPaste) * New module cpasbien http://www.cpasbien.cm/ (CapTorrent) * New module torrentz https://torrentz2.eu/ (CapTorrent) * New module pornhub http://www.pornhub.com (CapVideo) General: core * Add BackendsCall.stop() method to stop jobs * Add a 'requests' system to replace the primitive callbacks system * Check version before to run update request in repositories * Fix core unittests and some pyflakes warnings * Make as much modules unittests as possible run without backend * Add WebNip.load_or_install_module which does not install module (for compatibility with abstract objects) * Some fixes for windows compatibility * Force keyring generation with gpg1 Documentation * Fix some typo in browser documentation * Clarify absurl behavior * Don't add applications/index in index.rst (empty page) * Don't use :any: special keywork for old system compatibility * Update favicon path * Add contribution guide * Rewrite Weboob doc guides according to latest changes in CI and modules Browser * Detect downgrade from HTTPS to HTTP and raise exception * TableElement now handles regexp * Do not store page url if there is no current location * Use urllib3 embedded in requests first * Update firefox ESR profile to the last version * Keep order of POST data * Add XLSPage, PDFPage and GWTPage classes * Catch TooManyRedirects after reloading cookies from storage * Do not use (and remove) global logger setting 'save_responses' to know if we save responses or not * Fix some invalid encoding names (#2602, #3522) * Introde CacheMixin class to store responses * Store the request object in Form class (allow to modify it on the fly) * Add xpath funtions star-with and ends-with * Introduce PartialHTMLPage for pages wit multi-roots Browser: URL * Fix a crash when a pattern use parameters with the same name than in another pattern * Add unit test for URL with multiple patterns and same parameters subset * Add an option to send headers in go() method Filters * RawText: can now optionally retrieves text from sub-elements * Date/Datetime: add fuzzy option * Async: read result.page from loaded_page * TableCell: can now iterate on elements * Add Upper and Capitalize filters * Add Currency filter Elements * DictElement: Handle sub-dictionaries * ItemElement: Ignore items if the condition fails * ListElement: check condition before iteration Old browser * Allow monkey patch to disable SSL certificate check * Better SSL support (import more recent TLS versions if available) * Fix https downgrade detection Exceptions * Add ActionNeeded exception * Add NoAccountsException * Move ModuleInstallError to Exceptions module Tools * Introduce LimitedLRUDict to not cache too much values * Add a WeboobEncoder class * Add get_pdf_rows helper Tools: Backend * Save responses if _debug_dir backend configuration is set * Add optional 'klass' parameter for Module.create_backend Tools: bank * Add tools about IBANs and French ribs Tools: date * Add Italian translation * Fix date translation for values with \b Tools: formatters * Fix TableFormatter with lists of different lengths * Protect separator contained into values for CSV output (#2614) * Simplify CSV formatter with a call to csv python module * Fix use of -O parameter for csv formatter * Set ";" as csv default separator (#3378) Tools: tests * Add skip_without_config decorator * Add is_backend_configured method to BackendTest class Tools: ValueBackendPassword * When callback returns None, convert to empty string Tools: VirtKeyboard * Add parameterizable output code separator Capabilities * Add an url attribute to BaseObject * Add many currencies (turkish lira, romania new leu, colombian peso, etc) * Fix few __repr__/__str__ returning unicode * Add BoolField class Capabilities: Bank * Add description for 'coming' field on Account objects * Add PEE, PEA, PERP, PERCO, RSP, ARTICLE_83 and more account types * Add CARD_SUMMARY and DEFERRED_CARD transaction types * Add portfolio_share field to Investment * New fields for Investment to handle currencies * Add diff_percent field on Investment * Don't crash on empty raw values * Split transfers related methods to a new capability: CapBankTransfer * Update recipient object * Deep rework of transfer API, change modules and applications Capabilities: Contact * Add Advisor class, used in many bank modules Capabilities: Document / Bill * Major (and API breaking) change (all modules are updated) * Update all Bill modules to use url field instead of private field * Rename field "deadline" to "duedate" * Add field "income" to Bill Capabilities: File * Add license field Capabilities: Housing * Add url in HousePhoto __repr__ method * Add price_per_meter field Capabilities: Message * Create a genericnewspaper module with no capability that will be extended by other modules Capabilities: Image * Move Thunbmail class to Image capability * Fix many modules to use Thumbnail instead of BaseImage Capabilities: Lyrics * Remove useless constructor for SongLyrics objects Capabilities: PriceComparison * Add Price not found error Capabilities: Recipe * Do not crash if picture_url is an empty string * Handle decimal quantities in krecipe exports Applications: weboob script * Add "update" command Applications: console * Use self.do instead od self.weboob.do in get_object_list * Strip left and right condition (we now support rules lines like --condition 'amount > 2000') * Disable BACKEND@issues.weboob.org addresses (to much SPAM) Applications: all Qt applications * Use new-style signals and slot * Add Qt5 package in our tools * Port all applications to Qt5 * Add "wait" parameter to stop method Applications: boobank * Display sub-totals for each currency (when several currencies are detected) * Fix error display in budgea command * Increase timeout of budgea command * Do not take of loan accounts in total * Display loan accounts label in blue * Use encoding aware custom print() * New advisor command with custom formatter Applications: comparoob * Add defaultcount management * Improve products management Applications: flatboob * Improve display if Area is not found Applications: monboob * Add full thread history in headers (fix some bad detection one MUA) Applications: videoob * Correctly handle non complete url in m3u8 Applications: qcineoob * Sort torrent search result list by seeders * Display backend icon for each search results * Set tab text max size and add tooltip * Add keyboard shortcut (alt+left) to go back * Tab text is now updated on actions Applications: qcookboob * Display backend icon for search results * Add keyboard shortcut (alt+left) to go back * Better tab text/tooltip management * Fix bug when opening in a new tab, bad is was given Applications: qflatboob * Load all fields in widget list (help to rank adverts) * Display price per meter ratio Tools: boilerplate * Improve arg signature of methods generated by CapRecipe * Make u8 idempotent * Don't add failing test by default Tools * local_run: fix script when PYTHONPATH is not set * run_tests.sh: fix wrong exist status when specifying a single backend Packaging * Add a command to get the requirements with setup.py * Add cssselect to the list of requirements * Ensure that requests version is >= 2.0 Contrib * Boobot: add tasks management * Boobot: fix compatibility with iso-8859-15 messages * Boobank_indicator: do not include loan account in total * Add imm-o-matic script Modules: 750g * Update module to use new website * Do not crash when there is only one or no result Modules: adecco * Update module to use new website * Skip test if backend is not configured * Add default configuration for tests Modules: agendaculturel * Website migrated from http to https * Skip test if backend is not configured * Add default configuration for tests Modules: agendadulibre * Skip test if backend is not configured * Add default configuration for tests Modules: allocine * Update favicon Modules: allrecipes * Uses API instead to parse website Modules: amazon * Handle prices that occur twice on the page (#2057) * Fixed login after website update (#2057) * Another discount type (#2057) * Added 'Free Shipping Promo' bill detailisation keyword. (#2057) * Force label to document * Fixing amazon : sometimes tva or other amounts arent available * Use new login link id Modules: amazonstorecard * Improved stability (retry on error) (#2115) * Major website update (#2115) * Parse recent transactions (#2115) * Handle amounts more than ,000 (#2617) * Parse payment due date; allow transactions with empty id (#2617) Modules: americanexpress * Remove unused constants * Rewrite to browser2 and remove old code * Fix date parsing * Update dateguesser range to one year * Handle account suspended page * List multiple accounts * Don't report canceled accounts without history * Fix div id with single card Modules: ameli * Update module after site changes Modules: amelipro * Fix tests calls Modules: apec * Fix the way experience and contact fields are filled * JSON api format has been updated * JSON call got updated * Handle accentuated characters in search * Add default configuration for tests Modules: apivie * No more fake ids on apivie transactions Modules: arte * Do not crash while trying to handle a bad url * Handle an other type of web page in cinema.arte * Fix Debian bug #831609, correctly handle m3u8 url * Many little fixes * Handle arte creative * There are some duplicate videos in creative pages * Fix arte cinema (module was not able to find video url anymore) * Add default configuration for tests Modules: aum * Remove duplicate entry key in AUM module Modules: axabanque * Fix no history on card * Handle ACCOUNT_TYPES * Fix accounts typing * Port to browser2 and adding investments * Don't crash in case of customer error * Check when user need to change password * Fix problems when checking error * Fix asv not being fetch and portfolio * Fix bug going on agipi page * Get account IBANs and fix typing * Fix investments on market account * Fix accounts from tabs and more investments * Fix url change on invest * Fix handling pel type * Support new login and some changes * Fix: password can be only digits * Support the new website for investments * The login can now be an email * Accounts numbers may be written differently * New URL for accounts list * Some pages return overlapping accounts, skip duplicates Modules: banqueaccord * Support new login page Modules: banquepopulaire * Handling currency for cards * Adding currency for card * Handle crash on natixis website * Removing balance information in raw * Fix missing label in history * Set Account.iban to NotAvailable if user hasn't subscribed to online documents * Fixing banquepopulaire : No IBAN for checking account * Fix wrong account type * Fixing account type for "Fructi Selection Vie" * Adding more life insurance support * No IBAN on investment and life insurance * Add support for Banque de Savoie. * Check number in raw instead of label * Handle different label * Investments on savings account * Port to browser2 the easy way (without filters and so on) * Fix problem with unicode cookies * Test unavailable service on all pages * Raise NotImplementedError for new natixis pages * Use absurl for regular pages after having visited market * Update navigation token even on error pages * Fix crash on some pages without token Modules: barclays * Support investments on life insurance accounts * Updated cards handling * Fix account type detection to fix card linking * Move to browser2 and support IBAN on pdf Modules: bforbank * Adding pagination for bforbank * Don't crash on no IBAN for checking account * Fix reading when IBAN is not available * Add a favicon * Use spirica module for life insurance * Implement investments/history for MARKET accounts with lcl Modules: blablacar * Fix departure date regexp * Fix parsing date * Fix: site got updated * Add a favicon Modules: bnporc * Don't crash without simplejson (support other json modules) * Add support PEA liquidity account type * Support one more page to refresh password * Get a year of history * Correctly decode IBANs * Rebuild IBAN only if it isn't already one * Catch wrongpass * Print temporary password to allow rescue connection on website * Fix: site changed * Adapt to support CapBankTransfer * Try harder to connect on the website * Add url to renew password * Fix BNPEnterprise: add leading zero when IBAN key is < 10 * New website for BNPEnterprise, rebuild all module * Fix bnp enterprise: libelle not loaded for some reason, so take the type libelle on this case * Fix BNPEnterprise: swith date and rdate, move Account import * Fix CleanDecimal not always working * Catch change password page and raise ActionNeeded * Prevent date overlap in history search * Transfer fix on bnp * Categories on recipients * Handle transfer unavailable on fetching IBAN and recipients * Raise BrowserUnavailable on 1001 error code * Add transfer.account_balance * Add advisor support * Add a cache for account_list * Don't overwrite browser.page with something which can be a Response * Use response.json() to parse response, to prevent encoding issues Modules: boursorama * Better fall back on old website * Skip first visit page before falling back on old webiste * Better newwebsite url * Fix unitprice parsing * Support of Bourso new website and use Browser2 * Add support of loan accounts * Fixing class for transaction label changed * Handling coming card * Add a condition to prevent crashs * Fix label parsing * Update regexp for TYPE_ORDER * Fix cards transactions link * Handle wrong pin code * Fix loan, market, valuation etc .. * Two-step-auth: fix request if browser has been shutdown * Detect when PIN tries limit has been reached * Fix checking of errors * Handling when no informations is avalaible in investment * Fix in case there are divs into accounts list * Handle new bourso card ID * handle blocked page1 * Raise wrongpass on blocking secret infos update page * Handling new info profil page * Take custom-id if regular ids are not present * Fix same transactions on different accounts * Fix cards rdate parsing * Handle ord accounts * Fix pea history parsing * Fix some nav issues * More accurate url and valuation calculation * Adding incident page * Fixing wrong inheritance * Handle no link on market history * Fix history parsing * Parse transactions label back again * Try to be easier with bourso website * Do not iter coming on li and market accounts * Add a cache for accounts * Handle new history page type * Increase HTTP timeout, the website can be slow * Better typing to avoid crash * Do not crash on ClientError * Try to update balances of accounts before parsing * Try to fix XMLSyntaxError * Don't rash when no id are available (on some foreign accounts) * Catch XMLSyntaxError * Fix virtkeyboard * Don't call need_login twice * Fix page not being handled by the right class * Date might be missing * Fix unauthorized on virtkeyboard with old stored cookies * Fix cookie delation after going on login page * Fix card transactions request * Fix skiping not boursorama's accounts * Fix card accounts ids * Handle more cases where client need to log in on website * Handle more login page * relog on random disconnection * Fix coming parsing * Add IBAN page * Fix market accounts transactions date parsing * Fix useless nav and wrong webid leading to wrong IBAN * Fix transaction amount parsing * Fix crash on parsing amounts * Handle url with correct page * Type more loan accounts to prevent crash on history * Exclude deffered_card transactions of checking accounts * Add new param to login * Fix crash on expert derive account Modules: bouygues * Fixing bouygues module which works with b&you * Fix date parsing * Fixing bouygues module multiple accounts * Don't crash for accounts without document * For some reason, bills might be duplicated on website * Fix: https is now mandatory for documents URL * Fix: sometimes download document crash when reference is not specified Modules: bp * Handling wrong website selection * Add wrongpass page * Fix duplicate accounts with better id parsing * Fix regexp to get accounts ids * Support CapBankTransfer new API * Accounts are now in EUR by default * Handle deferred card * Accurate messages if users are on wrong website (between pro and perso websites) * Don't crash when no accounts are available * Port to browser2 (the easy way) * List loan accounts * More support of transfer (double auth not handled) * Fix pro browser working with StatesMixin * Reimbursed loans have an empty amount string * Parse life insurance investments and history * Fetch history for retirement contracts * Skip loan account with "unavailable data" * Fix BrowserIncorrectPassword with par/pro mix * Fix statesmixin on expired session * Handle internal recip not in accounts list * Add a cache for accounts_list * Fix a crash on unavailable pages * Skip more useless LOAN accounts * Add more regexp for transaction types * Add advisor support * Catch no accouts Modules: bred * Cut password to the first 8th char * Fetch IBAN only for checking accounts * Transactions typing * Do not yield not real market accounts * Fix duplicate id accounts Modules: caissedepargne * Fix pagination on history * Fix investments on multiple market accounts * Fix come back on synthesis * Handle loans caisse depargne * Fixing when not loan account is present * Better check if we are on wrong page * Detect and bypass useless message page * Fixing website failed redirecting * Handle delayed debit card and cleaning module * Browser handles technical error page (instead of code in iter_* functions) * Add certhash * Fix account valuation_diff parsing * Do not yield anything on market cgu * Guess IBAN only if this is a correct RIB * Fix login * Fix encoding error * Fix when there are multiple choices of spaces (between pro and personal) * Remove LoginPage and do only direct requests * Better wrong pass handling * Fixing can't leave credit page on old website * Fix navigation on history * Catch wrongpass * Life insurance might not be available for consultation * Adding lifeinsurance's history * Fix website being awful * Fix infinite loop on no accounts * Only keep relevant wrongpass (and return BrowserUnavailable for other errors) * Fallback on pro domain when loggin on personal fails if pro exists * Invest date might be unavailable * Better handle of real no accounts * Read multiple cb tabs * Bump to browser2 * Fix: sometime link is a list, so fix this * Fix: an 'href' who takes himself for an 'a' tag * Fix: on cenet website, sometime a year is not available, so try every months * Fix: on garbage page sometime link is not available, due to an ActionNeeded * Fix: if garbage go_back_link is not available, skip this step * Remove deprecated exception, use new ones * Add support of advisor Modules: carrefourbanque * Manage more accounts, investments * Fix portfolio on some modules Modules: cic * Improve account typing * Update cic sources from credit mutuel * Support CapBankTransfer new API * Cic now inherits from creditmutuel * Adding advisor for cm/cic Modules: citibank * Fixed login after website update. (#2098) * Switching to javascript library * Switching to new base url * When parsing a date in a statement, try both leap and non-leap years (#2098) * Handle negative and positive account balance (#2098) Modules: cmb * Handling agricultor website * No more crash on no account * Bump to browser2 and more (more accounts, investments, coming...) * Use AbstractBrowser instead of symlinks to cmso Modules: cmso * Fix account type * Fix wrongpass detection * Don't raise BrowserUnavailable when no savings accounts * Don't crash when no accounts are available in pro website * Manage multiple areas * Don't crash without accounts * Index were not unique * Don't crash when no IBAN for checking account * Support cmmc website * cmb/cmso/cmmc: support loans * Return on subscription before switching area to prevent 500 * Handle 500 when no history on market account * Have useful account ids * More history is now recovered * Add a forgotten return in iter_history * Getting a year of history * Use open instead of random async * Better id on market accounts * Fix crash when id is not found on market accounts * Handle multiarea on cmso pro * Handle no account in area * Fix regexp for some account ids * Start fetching investments * Fill 'areas' attribute outside of do_login (to work with blackbox) * Fix investments with non investment accounts * cmso pro: prevent infinite loop Modules: colisprive * Add a favicon Modules: colissimo * Get full history Modules: cragr * Fix broken perimeters * Get rid of not required assert * Account type on label before section title * Update label and balance on market websit * Fix bad home url and no account link for market accounts * Fix when cards and market accounts * Do not globally crash if we can't go to the market page * Fix current_perimeter and no link for market * Fix guessing of market links (use real hostname, not proxy's one) * Fix investment parsing and add vdate * Type and history of ESPE INTEG * Fix cards history navigation * Fix PEL history * Fixing cragr investment * Fix regression with cards navigation * Handle pagination on cragr CardsPage * Fix invest parsing * Do not crash on cards on some weird cases * Raise wrongpass on cgu * Fix investment parsing * Correctly handle the first connection page * Fix bad detection of current perimeter * Truncate passwords to first six chars * Remove trailling chars in isin * Fix encoding in investment code * Don't try to detect perimeter on not handled page * Handle perimeters having the same name * Handling currency on multiple card * Truncate username to the first 11 chars * Add new DAV TIGERE accounts * Fix infinite loop on wrong pass * Fix cards * Fix transactions parsing (new column) * Add account CPTEXCPRO * Fix multiple cards accounts * Update ordering date in get_history, it's useless and the link might be dead. * Fix transactions parsing on some accounts type * Fix bad isin * Handle life insurance on bgpi website * Fix some lifeinsurance * Adding pagination on accounts page * Fix parsing of unitprice on some market accounts * Fix portfolio_share on bgpi invest * Fix crash when no investments in BGPI * Fix listing card with pagination * Fix get of IBAN * Adapt to new CapBankTransfer API * Don't try to read history on market accounts * Fix detail unavailable on market accounts * Fix parsing of valuation_date of market invest * Seems like cragr normandie is not using new login method anymore * Update webid of deferred card to avoid duplicates * Fix sublabel parsing, sometimes there's no sublabel parent Modules: creditcooperatif * Add coming card page * Limit password to 12 chars * Fix xpath issue * Fix crash on technical error on website * Add IbanPage support * Fix crash on notavailable iban page Modules: creditdunord * Add bank SMC support * Add bank nuger support * Fix transactions parsing on rhonealpe * Unify regular and pro accounts cards handling * Fix deferred_card typing * Dectection of wrongpass * Fix duplicate accounts * Skip accounts without details * Fix bad isin like FR0010411884 * Handle no account * Move to browser2 * Handle loans on pro website * In iter_* methods, raise AccountNotFound if account is not found, instead of crash * Fix navigation on website to list loan accounts * Fix parsing of transactions without value date * Fix parsing of euro funds * Add loan typing * Fix navigation on website to get transactions * Fix creditdunord transaction date parsing * Some dates are not available instead of a parse error Modules: creditmutuel * Fix history and investments of life insurance accounts * Fix life insurance * Fix coming for cards * Fix when unitprice not available * Set default balance value to NotAvailable * Handle fleet cards * handle page to detect new website * Fix date parsing for cards * Raise skipitem instead of parse error * Fix amount commission of non euro fleets transactions * Handle hidden parts of labels * Fix isin parsing * Adding IBAN for new website * Parse life insurance's transactions dates as French dates; * Get IBAN based id * Handle pagination * Fix crash on more transactions * Only use last card differed for coming * Support cb diff * Adding cards to creditmutuel's new website * Fix setting deleted to transactions * Fix parsing of differred_date * Don't crash on empty city field * Use new CapBankTransfer API * Fix id and IBAN of accounts on old website but displayed on new accounts page * Fix date parsing for coming * Handle real no accounts on cic * Fix currency on lifeinsurance accounts * Fix crash when there is no bank accounts * Fix card names and set coming balances * Fix multiple month of history on cb differed * Correctly handle lifeinsurance accounts on cic * Handle newhome page to detect new website * Fix original amount of card transactions on new website * Get now about 6 months of history on new website * Handle loan history page * Handle page and fix crash on history on some accounts * Handle another user page * Fix card history link * Fix transfer on the new website * Add support of creditmut pro and add transfer category * Fix regexp for URL of subbank * Fix finding account and recipient in init_transfer * Skip wrong IBAN instead of crash * Limit transfer label to 27 char instead of raising error * Add cards on new website with tiers page * Add transfer.account_balance * Don't crash if a transfer has no label * Truncate label checking * Strip transfer.label to compare with bank return * Detect ActionNeeded after login * Prefer None accounts_list over missing attribute * Skip debit card * Add support of life insurance on new website * Add advisor * Parse balance and currency of cards in some cases * Raise wrongpass with error messages Modules: cuisineaz * Update to the new website site changed Modules: dailymotion * Fix module and bump to browser2 * Get video from page url Modules: delubac * Detect bad password * Add basic account types * Get IBAN * Fix transactions parsing Modules: dhl * Use dhl.com's JSON API * Add support for both DHL Express and Deutsche Post DHL * show URL in event description Modules: dlfp * Populate url attribute messages and thread objects * Always fill the parent attribute * Disable wiki tests when the backend is not configured Modules: dresdenwetter * Skip test if backend is not configured Modules: edf * Bump module to Browser 2 and fix parsing * Add edfpro website * Add a favicon * Always set vat attribute to to NotAvailable * Remove bad tests in edfpro folder * Ignore duplicates Modules: entreparticuliers * Fix parsing : site changed * Correctly handle source * Fix url field filling * Support new price_per_meter attribute of CapHousing * Fix bug when response is empty * Add a 100km default rayon value * Do not crash if area is unknown * Add a favicon Modules: explorimmo * Support new price_per_meter attribute of CapHousing * Fix bug in town names' encoding Modules: feedly * Update google login state * Remove need_login decorator for get_unread_feed Modules: fortuneo * Update certhash (#2550) * Detect wrong passwords * Fix duplicate of accounts * Don't try to get IBAN for checking account (token sent by SMS to customers) * Replace BrowserIncorrectPassword with ActionNeeded on renew password * Port to browser2 * Limit max time site wants us to wait in async requests * Fix separators cleaning for decimals Modules: francetelevisions * Ignore duplicate in videos search result Modules: freemobile * Fix subscription listing (site changed) * Fix sub._login parsing * Fix error when two bills happen on the same day * Id of details are not uniques anymore, so take build a new custom one Modules: funmooc * Base url of the website has changed * Fix module and add logged tests Modules: ganassurances * Add pagination to ganassurances/groupama module * Handle history and coming of another account type Modules: github * Port to browser2 * Shorten some long lines + fill url of Issue * Few fixes and tests * Use CacheMixin new class Modules: googletranslate * Bump to browser2 Modules: groupamaes * Fix bug when there is no credit anymore on PEE / fix account typing * Handle investment * Fix groupamas amount parsing * Fix crash when parsing investments * Fix portfolio_share range * Improve investment management (quantities of values of each funds are divided by availibility date) * Fix amount of withdrawal (Sadly withdrawal are not incomes) * Add a label in order to detect negative amount in transactions Modules: happn * Fix issues with new kind of ids Modules: hsbc * Fix account type * Add more history with pagination * Cache accounts and fetch IBANs * Fix accounts typing * Fix rib navigation * Fix currency of accounts * Auto delete card summary * Handle when website duplicates tr list * Raise not error implemented on life insurance for history * Add more account types to avoid crash Modules: ideel * New way of handling free shipping (#2146) Modules: ilmatieteenlaitos * Adapted to site changes Modules: imdb * Update imdb favicon to make it more different from the allocine one Modules: imgur * Implement gallery search and CapImage * Add date attribute Modules: ina * Fix duration parsing Modules: indeed * Search for words in the entire advert, not only in the title * Indeed now uses https Modules: ing * Use CleanDecimal to simplify parsing * Add history for life insurance accounts * Fix ing 500 error on bill download * Fix investment on multiple line * Handle blank transaction label * Loop on coming from market website to avoid 500 errors * Fix missing some investments * Increase requests timeout to 60 seconds * Fix id of life insurance * Fix investment on life inssurence and history on all accounts * Fix name of Account._id field * Fix navigation on the new life insurance pages * Refresh account link * Quick fix navigation * Ahistory to some asv and more * Get IBANs for accounts * Transactions detail might be unavailable * Some fixes on market accounts * Use new CapBankTransfer API * Only go on IBAN page for valid account * Fix transfer method signature * Catch action needed * Seems like response headers of download might change * Don't raise ActionNeeded for a check reception notice * Disable CapTransfer since it's broken * New login for asv page * SSL errors due to a bad certificate, so create it manually and fix a false https * Avoid re-login after lifeinsurance and market pages * Before many operations, try to go on main website first Modules: inrocks * Bump to browser2 / inherits from genericnewspaper module Modules: ipinfodb * Bump to browser2 Modules: itella * Update to use JSON API * Handle missing arrival date * Use status from JSON instead of guessing Modules: jcvelaux * Add api_key backend option * Fix str-unicode warnings and port to browser2 Modules: jvmalin * Add a favicon Modules: lacentrale * Bump to browser2 * Module connot handle products without criteria * Improve products management Modules: lcl * Remove SSL hacks (no need to RC4 anymore) * Add individual loans to lcl * Fix loans post bourse deconnection, fix loans regex * Fix unique id on loans lcl * Add certhash * Use valuation for BoursePage objects, not balance * Raise wronpass if the password is not a digit * Change label of transactions and yield card_summary * Retry read of amounts (hack to fix broken HTML) * Don't return 0 as account ID * Fix bad isin like FR0000120404 AC * BUMP LCL entreprise browser to bump2 * Fix isins like FR12345435 ABC * Code of investments might be unavailable * Rework of transactions type parsing * Don't crash when tbody is missing on market accounts table * Add a new account type * Get more history on several account types * Fix matching asv accounts * Keep check type on labels * Fix life insurance matching from site to site * Fix lifeinsurance * Check if the IBAN is for the current account * Fix crash if there is no IBAN * Catch password expired * Fix duplicate id on lcl pro * Fixing transactions are in multiple account * Fix params of history navigation when only one account * Add transfer support and use new api * Ignore HTTP error 500 on history * Have a default accounts_list to be able to bypass login * Add transfer.account_balance * Fix investment unitvalue * Fix iter recipients on loans * Handle loan not available * Fix lcl espace pro and enterprise login * Adding advisor for lcl * Add more regexp for transaction types Modules: ldlc * Better way to get subid * Add a favicon Modules: leboncoin * Adapt to new website design * Fix incomplete url in pagination * Website now uses https * Fix description field filling * Support price_per_meter new attribute * Fix typo for poitou charentes region * Add url attribute on objects built from list * Fix cost for items in rent search page Modules: lefigaro * Bump to browser2 / inherits from genercinewspaper Modules: liberation * Bump to browser2 / inherits from genericnewspaper Modules: logicimmo * Retrieve a bigger photo * Do not crash when area is unknown * Support price_per_meter new attribute * Fix town search Modules: lutim * Fix tests Modules: mailinator * Port to apibrowser Modules: mangafox * Add a favicon Modules: mangago * Add a favicon Modules: mangareader * Add a favicon Modules: marmiton * Always retrieve title Modules: mediawiki * Implement CapImage for searching images * Use https by default * Fix image search Modules: minutes20 * Bump to browser2 / inherits from genericnewspaper Modules: monster * Fix: site changed Modules: myhabit * Update after website has changed (#2615) Modules: nettokom * Add a favicon Modules: newsfeed * Populate url attribute for messages Modules: okc * Revert "do not remove old threads anymore" introduced in 1.1 Modules: oney * Fix login * Verify SSL/TLS certificate again * Raise wrongpass when password is not 10 chars Modules: opensubtitles * Fix opensubtitles small site change Modules: orange * Fix orange module which works with sosh * Add label for bills * Support multi account and more * Remove old url Modules: ovh * Fix login, remove API calls (fixed several times, OVH login page often change) * Add a favicon Modules: pap * Remove dots from cost * Support new price_per_meter attribute of CapHousing * Update to the new website Modules: pariskiwi * Fix module and prefer json mediawiki api Modules: parolesmania * Fix song search * Site changed, search artist corrected * Adapt module to not use SongLyrics construtor anymore * Bump to browser2 * Artist search: remove page calls to browser methods Modules: parolesmusique * Site changed, search song and artist corrected * Adapt module to not use SongLyrics construtor anymore * Bump to browser2 * Artist search: remove page calls to browser methods * [parolesmusique] artist search : remove page calls to browser methods, browser now does what it should : browsing Modules: parolesnet * Adapt module to not use SongLyrics construtor anymore * Bump to browser2 * Artist search: remove page calls to browser methods * Fix parolesnet, iter artist lyrics : just consider first song list to avoid ID duplication Modules: pastealacon * Fix test_spam test * Force site encoding, it is iso8859-15, not utf-8 * Fix post test when backend is not configured Modules: paypal * Update account_type at the beginning of get_accounts method * Fix crash on old website * Fix parsing of converted amount * Add english word to skipped transactions * Fix part converted amount for some transactions * Bypass new anti-parsing protection of the website * Support multiple accounts with different currencies * Get data-ads-challenge-url attribute (anti-scrapping protection again) * Catch 500 on detail and skip transaction * Fix detection of account currency in weird case * Consider that tr is valid if transactionStatus not present * Fix currency detection * Better detection of currency * No crash on securisation step * Fix domain of detail navigation * Fix transactions detail link * Fix parsing of converted amount * Fix crash on transaction conversion * Skip transactions labeled 'Bill to' * Display a warning when a currency is not found and ignore account * Fix account_type detection * Handle more home page to detect account type * Fix paypal payback * Handle more login page * Add home page paypal * Skip fake transactions * Handle another wrongpass page * Exclude draft transactions * Account detection on accountPage too * Fix value seperator detection on perso accounts * New fixes for login issues * Raise unicode message in BrowserIncorrectPassword exception * Use json data for transactions parsing * Pagination and some improvements * Problems with amount and decimals * Skip empty transactions * Set Iban as NotAvailable for all Paypal accounts * Handle english transactionStatus * Skip broken transactions * Exclude fake transactions (for english website) * Skip suspended transactions * Handle textual amount is integer * Exclude fake transactions * On old website, try to go on the new one * Skip 'Annulation des frais de PayPal' transactions * Paypal commission are now net minus gross * Detect ActionNeeded * Computation commision if it is not null * Add one more login parameter * Parse secondary transaction based on label Modules: piratebay * Ported to browser2 * Handle fuzzy dates Modules: poivy * Add a favicon Modules: popolemploi * Fix search: site changed Modules: presseurop * Bump to browser2 / inherits from genericnewspaper Modules: prixcarburants * Bump to browser2 / handle date of update * Fix parsing date (#2606) * Adapt module for new comparoob API * Website behaviour changed (#2634) * Set default configuration for tests * Update url to new website Modules: radiofrance * Fix: all websites are updated * Fix franceinter * Fix france culture podcasts * Handle selections and podcast from francetvinfo Modules: regionsjob * Handle case advert is coming from another website * Add a place filter Modules: residentadvisor * Add a favicon Modules: s2e * Fetch history on the right account * Remove id on transactions: it's not really an id * Fix capeasi login page and condition on fetching transactions * Using website with more details * Fix transactions investments valuation parsing * Fix parsing of accounts and history pages * Add currency support, add new account types * Don't crash when the customer has no account * Catch another real no accounts * Handle multientreprise * Fix transaction amount sum * Add new SWISS account type * Handle maintenance page * Add new hashes for virtual keyboard * Detect more no accounts * Handling weird wrongpass * Add a favicon * Add AMF code to investments Modules: seloger * Fix xpath for looking into cities * Retrieve all cities * Fix typo for photos * Do not crash if area is unknown * Add support of new price_per_meter field for CapHousing * Fix pagination Modules: senscritique * Fix: site got updated * Fix: json url changed * Fix parsing date Modules: societegenerale * Add cookie before login * Fix account type * Handling some wrongpass * Bump SGPE browser to browser2 * Add rdate to lifeinsurance for the sake of sorting * Do not yield canceled transactions * Raise wrongpass instead of crash for non-digit passwords * Handling multispace on sogepro * New soge entreprise browser * Add new type for transaction * Add deffered card detection * Website limit login to 8 char * History pagination * Catch login forbidden * Fix xpath on invest * Handle the two different parsing cases on invest * Catch wrongpass * Investments are now more detailed * Add advisor * Switch to browser2 * Fix pagination in life insurance history and use Form class * Add transfer support * Fix ix account id being different in sending panel and list * Handle renew password and raise ActionNeeded Modules: sueurdemetal * Fix fillobj method Modules: t411 * Site url changed * Get get_torrent_file() to work * Add tests Modules: taz * Bump to browser2 / inherits from genericnewspaper Modules: transilien * Fix: site changed Modules: twitter * Fix: site changed Modules: ups * Use https Modules: vicsec * Website got updated (#2177) * Case fix (#2134) Modules: vicseccard * Website updated (#2134) Modules: vimeo * datePublished attribute was renamed to uploadDate * Handle JSON format changes * Fix categories videos in order to use API Modules: virginradio * Correct a bug occuring when the StreamTitle is empty Modules: wellsfargo * Handle webpages with no transaction activity (#2123) * Try to get a statement multiple times (#2123) * Handle SMS security code, leap and non-leap years (#2618) Modules: wordreference * Fix module for new website * Update tests Modules: yahoo * Bump to browser2 Modules: youjizz * Fix parsing of video urls * Support mp4 videos * Use now https, but not for all videos (sometimes not available with https) Modules: youtube * Use googleapi module instead of gdata which is obsolete Weboob 1.1 (2016-02-11) General * Last release with python 2.6 support * New CapShop capability (#1662) * New shopoob application * Add new 'weboob' script to find applications * New APIBrowser class * Add a contrib script to help forking weboob * New WEBOOB_DATADIR environment variable General: removed modules * Removed grooveshark module: website is closed * remove module hellobank, use bnporc instead General: new modules * New Amazon Store Card banking module (CapBank) (#1698) * New bforbank module https://www.bforbank.com (CapBank) * New kiwibank module https://www.kiwibank.co.nz (CapBank) * New s2e (Employee Savings Plans) Support for Esalia, Capeasi, "BNP Paribas epargne & Retraite Entreprises" and "HSBC Epargne et Retraite en Entreprise" (CapBank) * New Victoria's Secret store credit card module (CapBank) (#1732) * New agendaculturel module http://www.agendaculturel.fr (CapCalendar) * New residentadvisor module https://www.residentadvisor.net (CapCalendar) * New happn module https://www.happn.com (CapDating) * New freegeoip module https://freegeoip.net (CapGeolocIp) * New ipapi module http://ip-api.com (CapGeolocIp) * New entreparticuliers module http://www.entreparticuliers.com (CapHousing) * New explorimmmo module http://www.explorimmo.com (CapHousing) * New logicimmo module http://www.logic-immo.com (CapHousing) * New virginradio module http://www.virginradio.fr (CapRadio) * New Ideel module http://www.ideel.com (CapShop) (#1733) * New MyHabit module https://www.myhabit.com (CapShop) (#1734) * New Victoria's Secret module https://www.victoriassecret.com (CapShop) (#1731) * New blablacar module https://www.blablacar.fr (CapTravel) * New dhl module http://nolp.dhl.de (CapParcel) * New dpd module https://tracking.dpd.de (CapParcel) * New gls module https://gls-group.eu (CapParcel) * New itella module http://www.itella.fi (CapParcel) * New funmooc module https://www.france-universite-numerique-mooc.fr (CapVideo) * New rmll module http://video.rmll.info (CapVideo) * New vine module https://vine.co (CapVideo) * New ilmatieteenlaitos module http://ilmatieteenlaitos.fi (CapWeather) * New ldlc module https://secure.ldlc.com and https://secure.ldlc-pro.com (CapBill) * New t411 module https://www.t411.in (CapTorrent) Core * Repositories update: display a message when all modules are up-to-date * Log full backtrace on module build error * Update windows installation (#1668) * Better detection of gpgv.exe location * Fix issues when using NamedTemporaryFile on windows * Add a datadir option, handle WEBOOB_DATADIR * Use gpg by default rather than gpgv for signature verification * Create empty temporary GPG homedir * Exclude backend using instance_name instead of module_name * Try to be consistent in variable name between backends and modules * Convert StopIteration for PEP 479 Doc * Account() constructor does not take an id * Fix symlink creation in setup.rst * Use browser2 in module documentation * Add few docstrings in core and browser Filters * Add Eval filter, for arithmetic expressions * Add Slugify filter * Add a JSValue filter to look for literals * Add Decode filter, to decore urlencoded strings * CleanText: add Unicode normalization * Regexp: nth param can now be '*' to find all * Attr: fix error message * CleanText: fix paramter name "children" * DateTime: handle TypeError exception raised by parse_date * Duration: correct '_regexp' attribut name * Dict: several improvements * Join: improve it to work like join command * Join: add ,ewMone, addBefpre, addAfter parameters * Async: if AsyncLoad parameter is None, do not try to open it * CleanDecimal: convert INT, float and long (and not only str) * Attr: fix an error message * Dict: handle TypeError in parsing * Time: Update regexp to catch more values Browser * Add comments on Form class * New unit tests * New api for content, encoding, and document building * Add ChecksumPage class * Cleaner way to get rid of BOM for CsvPage * Fix wrong encoding detection logic * Detect XML encoding * Enable passing the path to a CA file to Browser.VERIFY * Add an asset method to get absolute path of module local files * Browser.VERIFY can now be a module local file * Fix compatibility with python 2.6 in HTMLPage (#1651) * TableElement: fix count columns when there is an attribute "colspan" * Do not display urllib3 warning if there is no TLS certificate verification * Add an IPhone profile * Add a GoogleBot profile * Add method Browser.set_profile * DomainBrowser: ability to give the baseurl in constructor * Add LoginBrowser.do_logout() method * Handle "Refresh" meta tag in HTMLPage, with a REFRESH_MAX option * Add an absurl method to Page * Fix crash when using HEAD on a page-matching URL * Add BrowserQuestion exception (ask user to change config when raised) * Add 'replace' xpatch function in HTMLPage * Introduce new DictElement class * Add some JSON helpers * Strip data on CsvPage * Better control of HTTP referrer * Allow changing the Session class * Pass async/sync callbacks in the same way * Bump Firefox version to next ESR (38) * Fix documentation: use HTMLPage instead of Page * Add StatesMixin class to store state of a browser * Change worker pool size after session init * Allow to force method in URL.go (pass argument to location()) * Support using id in HTMLPage.get_form() * Rename async parameter to is_async Old Browser * Do not use PROTOCOL_SSLv3 as it is deprecated, use bad named PROTOCOL_SSLv23 instead * Add BaseBrowser.get_page method (don't change the location) * Add a StateBrowser to store cookies in storage * CSVParser: catch exceptions * Disable lowsslcheck if there is a proxy, as ssl doesn't deal with it Capabilities * Add the possibility to define Field type argument as a string * Create enum. Use it in calendar and housing (code factorization) * Add a way to instantiate an object from a dict of its fields values * Add russian ruble and XOF currencies * Add SGD, BRL, JPY and MXN in currencies Capabilities: Audio * Move specific audio filters from capabilities/audio.py to tools/capabilities/audio/audio.py * Add specific filters to create audio objects id * Do not require an id in constructors Capabilities: Bank * Fix StatusField to be a BaseObject * Change quantity type to decimal number * Better explanation for Investment class * Introduce new type Account.TYPE_LIKE_INSURANCE * Add credit limit, payment due date and amount for credit card bank accounts (#1717) * Add IBAN properties * Add card number and commission in Transaction * Add original_amount and original_currency in Transaction * Add country in Transaction * Add Transaction.investments and Investment.vdate attributes * Add Account.valuation_diff attribute * Add Account.number attribute Capabilities: Bill * Inherits Detail and Bill from Currency * Remove Bill.idparent attribute * Add Bill.vat attribute Capabilities: Calendar * Add timezone parameter to calendar event in order order do get the good date/hours in ical exports * Add a field 'summary' to Query object * Add ticket status in BaseCalendarEvent * Add a ticket field to Query * Add BaseCalendarEvent.timezone attribute * Add new categories Capabilities: Cinema * A person may have now several roles in a movie Capabilities: Collection * Do not require an params in constructor Capabilities: Contact * Set default value of Contact.profile to OrderedDict() Capabilities: Dating * Fix API to store optimization name in object ID Capabilities: Gallery * Correct method name: s/search_gallery/search_galleries/ Capabilities: GeolocIp * Add osmlink field Capabilities: Housing * Support new type SHARING Capabilities: Messages * Remove _Message and _Thread and use strings in Field instead Capabilities: Recipe * Do not force to use id in object constructor Capabilities: Torrent * Remove name mandatory parameter in constructor Capabilities: Thumbnail * Thumbnail now inherits of _BaseImage Capabilities: Weather * Do not require an id in constructor * Fix default value for date * Cosmetics in temperature formatting Applications * Backport check_output function for python 2.6 * Remove wconio dependency on win32 platform * Allow ResultsCondition to compare timedelta * Encode properly output in console.py (#1673) * Fix BrowserSSLError catch in ConsoleApplication (#1702) * Handle attribute error when stdout does not habe encoding attribute * If backend name is already taken, suggest the first available one with the same base * Formatters: use weboob.tools.misc.guess_encoding instead of utf-8 * Formatters: manage tty colummns while displaying text * Use os.devnull and binary flag for windows compatibility Applications: REPL * Fix a crash when ReplApplication.do() is called with a function * Create a message for 403 errors Applications: boobank * Format: ISIN is a 12 letter word * Format integer for quantity * Sometines the full code is not available. Display description instead * Add a command 'Budgea' to export accounts and transactions * Fix investment display when some fields are empty * Identify credit card for each transaction in OFX export Applications: boobill * Use do_ls to cache objects in REPL * Add binary mode for writing. Fix file downloading on windows Applications: boobcoming * Fix crash without end_date (#1667) * Add search by summary * Print ticket status * Add search by ticket status * Handle error correctly in do_attends * Add a load command * Improve ical export (end lines with '\r\n', as RFC says) * Add timezone in ical export * Display price as float * Improve date display * Do not crash during export of an unknown event Applications: cineoob * Give roles informations in persons_in_common and movies_in_common * Support several roles in the same movie for an actor * Movies in common are now sorted by release year * Fix problem with stream parameter in request.get in python-requests v2.5 (#1683) Applications: flatboob * Display backend in city choice Applications: geolooc * Fill OpenStreetMap link Applications: havedate * Store optimization name in object ID Applications: parceloob * Clean cached objects before getting new ones Applications: qcineoob * Add 'view' and 'view in new tab' buttons for all search results * Add 'view thumbnail' for movie and person search results * Factorisation of events in search results * Persons in common implemented in movie page * Movies in common implemented in person page * Support several roles in the same movie for an actor * Better display of search results, spaces added * Increase biography field size * Movies in common are now sorted by release year * Better user input validation for same-person movies in common Applications: qcookboob * Implement tab navigation (#1290) * Fix bug when thumbnail_url is empty Applications: radioob * Can now download full album * Add file in search subcommand Applications: videoob * Fix the problem with streamed calls on requests (#1683) * Display download subcommand in debug mode Applications: weboorrent * Fix formatter for info command Applications: weboob-config * Improve info command to allow json formatting * Put all config details in a dictionnary * Format capabilities as JSON array * Add installation status in modules subcommand * Prompt user to accept an untrusted keyring (#771) * Add module_name param in add_backend to allow command line interaction * Update help of "add" command Tools * Move guess_encoding into weboob.tools.misc * Add a Javascript class * Avoid crash on UnicodeDecodeError with retry decorator Tools: bash-completion * Hide error (#1671) Tools: date * Supporting french month abbreviation without dot * Extend date_max_bump default value to 31 days in LinearDateGuesser and ChaoticDateGuesser * Add some french translation * Parse french date with DMY order by default Tools: virtkeyboard * Fix not logical arguments order * Dump of tiles: save full image too * Allow to receive a list in get_symbol_code() method * Support pollution in coords attributes Contrib * Add a gtk appindicator for boobank * Remove bashisms in report_accounts.sh * weboob-generic: fix object listing when tomonitore is not set Contrib: downloadboob * Print link message only when needed * Fix new file detection * Support download of m3u8 files * Allow passing config and section filter on command line * Replace call to 'fill_video' by 'fillobj' Modules: 750g * 750g is now working using browser2 (#1706) * Fix: Only select the first thumbnail image * Fix: Site changed * Improve instructions parsing Modules: adecco * Bump to browser2 Modules: agendadulibre * Add missing favicons * Adapt for timezone support * Do not crash when json included in webpage is not valid * Correctly retrieve start and end date Modules: allocine * Remove excessive quote_plus call on pattern to make multi-word search work again * Adapt for more informations in persons_in_common and movies_in_common * Fix: accents were causing problems in search * Handle videos from allocine * Handles showtimelist in allocine * Fix: bug while playing videos from external url * Fix: syntax error on allocine * Adapt for timezone support * Max rallocine rate is 5 * Fix unicode warning Modules: allrecipes * Adapt to new version of website * bump to browser2 Modules: amazon * Skip "In transit" orders. (#1684) * New discount: "Lightning Deal" (#1684) * Generate payment transaction if there aren't any (#1684) * Order charges xpath fix (#1684) * Amazon module: skip 'Not yet shipped' orders (#1699) * Amazon module is updated according to the website change (#1715) * Skip "On the way" orders at Amazon (#1716) * Support new version of Amazon order details header (#1721) * Retry Amazon requests after ReadTimeout exception * handle Timeout exception as ReadTimeout has been introduced in recent versions of requests * Add French Support and Shopoob compatibilities * Add french translations * Retry ConnectionError as well (#1742) * pep8 fixes * Support for a new payment layout and priceless items (#1764) * Scrape the updated website (#1808) * Update after site changes * New status to ignore * More shippings and discounts (#1853) * Retry getting an order page (#1876) * amazon bills * Fix: work with iter_resource, and use ISO currency codes instead of symbols Modules: ameli * Update after sites changes, and upgrade to new browser * Handle authentication errors * PaymentDetailsPage: use a simpler URL * Site was updated * details isn't implemented: remove dummy code * ameli: prevents crashes when no bills are available Modules: amelipro * Update after sites changes, and upgrade to new browser * Fix decimal bug Modules: americanexpress * Handling no amount on account * Add default date to beginning_debit_date * Handle new americanexpress site but only if there is only one card * Increase timedelta to guess real date * Handle US account * Amex is dumber and dumber ( increase time delta because amex have hard time to put the date in a good interval) * Fix parsing of rdate * Fixing card type * Removing useless id Modules: apec * Use https instead of http * Adapt to the new version of the website and use browser2 Modules: arretsurimages * Fix: get videos titles in h1 (#1447) * Fix: correctly implement search and add a test (#1691) Modules: arte * Fix bug id was not well filled in info command and title was not filled * Adapt to browser2 * Fix display bug during newly install * Default value must be key, not label * Fix bug when urls are not found * Fix bug when date is not available * Handle special characters in search * Fix arte concert (a fields has been removed fron json) * Handle special cases in json parsing * Do not raise 404 error while searching a video using an other site url * Fix program menu management * Fill id using a better value for arte cinema Modules: audioaddict * Fix description data removal from audioaddict API * Rename SKY FM to Radio Tunes * Add Fresca Radio Modules: aum * Search query string: replace %(lat)s and %(lng)s with position in my profile * Fix parsing of geo position Modules: axabanque * New login pages on axabanques (#1709, #1711) * account.id is now only paramNumCompte if there is not paramNumContrat * Handle expired password * Handling banned account * Do not try to recover transaction from DAT account * axabanque fix duplicate accounts id * Handling duplicate id for account using Jsp Id * Stop parsing history of investment accounts * Fixing random number formating * Add error page Modules: banqueaccord * Fix inconsistent history on loan accounts if there is no init date * Labels of login fields changed Modules: banquepopulaire * Support new authentication on some region websites * Fix login on new website (needs upper case login) * Fix crash for new cards with no debited history * Break the new protection * Support full list of accounts (#1360) * Fix crash in a particular case of navigation * Supporting new account page * Parsing last connection date correctly * Fix navigation when passable unavailable page is displayed before login * Support case where user may accept TERMS_OF_USE * No history if account has no prev_balance * Apply alsace and lorrainechampagne fusion in bpalc in banquepopulaire module * Add Investment accounts * Correctly return an iterator even if this isn't an investment account * Fix crash on empty field banquepopulaire investment * Add life insurance investment * Skip message pages to get_investment on banquepopulaire * Fix a crash on iter_accounts after an investment on banquepopulaire * Remove stored 'token' attribute from browser and fix navigation on banquepopulaire website * Correctly get token only when available * Handle error page to avoid global crashes * Fix to get token * Adding support for IBAN * Fix getting tokens on some pages * Attempt to fix cgu validation * Adding loan * Add unavailable page * Fixing useless id * Ignore \0 bytes in documents Modules: barclays * Add AssurancePage on which we don't support get_history * Fix barclays navigation * Handle suspended accounts * Fix bug when card is first account * Support more account types * Sorting transaction before returning * Fixing page payement to foreign currency * Remove print * Remove bad transactions ids Modules: biplan * Site changed * Detect invalid events * Do not crash when no time can be found * Add timezone / fix encoding and hours detection * Fix parsing error on start date Modules: bnporc * Fix to get transactions when they are ordered by type * Get IBAN of accounts * Do not try to get IBAN for delegated accounts * Add investments to bnporc module * Handle corner case * Move old code for old site in a new folder * Add support for new bnp site * Move new browser into pp/ * Parse transaction labels, and fix account's coming * bnp history not supported on life insurance * Investment not implemented for entreprise browser * bnp investment fix * Handle error message after login * bnp type life insurance on capitalisation accounts * Do not get investments from PEA Espèces * Handling account without history * bnp history always return an iterator * New bnp entreprise certhash * Switch to new website and several fixes * Correctly set IDs * Raise BrowserIncorrectPassword when user should use hellobank instead * Raise IncorrectPassword after 100 connections * WIP on new company bnp website * Adapt module to use Browser2 * Add company browser * Fall back on old website for pro accounts * Add iban field for company accounts * New certificate * Fix compatibility with python2.6 * Fix concerning banqueprivee subdomain * Handle website unavailable for history * Better handle of expired password * rename pp folder to ppold * Fix PEP8 E713 * Support lifeinsurances on new website * Add authentification error code * Skip transaction without date * Adding support iban for some pro account * Automatically update password to the same one when the 100 connections threshold is reached * Type more accounts to get history and transactions investments on those * Revert fall back on oldwebsite for pro accounts * Add a page to renew password * Support checking type * Adding loan for crédit immobilier * Calculate IBAN key if messing * Add Hellobank support * Use shorter id * Revert "[bnporc] use shorter id" * Handle investment and history on market accounts * Fix crash if bnp throws 500 HTTP status code * New connection threshold page url for hellobank * Valution_diff on market accounts * Add number to iter_accounts * Fixing bad id for coming transaction * Prevent us from crashing when bnp crashes Modules: boursorama * Add investment * Support market investments * Fix duplicate entry on card account * Cleaner way to iterate * Better explanation for Investment class * Fix website changes (virtual keyboard, accounts list) (#1696) * Ignore moneycenter accounts * Minor fix * Introduce new type Account.TYPE_LIKE_INSURANCE * Use two_authentication with BrowserToBeContinued instead of raw_input * Use StateBrowser, s/BrowserToBeContinued/BrowserQuestion/ and coding style fixes * Remove text after card number if any * Raising incorrecte password when profil is incomplete * Fix CC history for boursorama * Update hash of boursorama's certificate * Fix duplicate accounts * bourso new investment page and fix parsing * Fix crash when there is more data in the cell * Fix PEP8 E713 * Fix parsing of unit price * Fix investment link parsing * Ignore immediate debit cards * Investments on lifeinsurance transactions * Fix pagination of lifeinsurance history * Ignore promotional fake accounts * Fix parsing some special market accounts * Valuation diff on lifeinsurance accounts * Fixing bad id transactions * Do not use useless id * Fixing false id in transactions * Fix parsing of investment label * Fall back on old website if we are on the new one Modules: bouygues * Rewrite backend using browser2 (#1970 and #1978) * Add support for bill on bouygues * Avoid useless login * Handle website modifications * Home page uses https (#2510) Modules: bp * Support professional accounts * Update upstream certificate hash (#1702) * Handling when website have an error * Fix parsing of virtual keyboard url * Add a page to pro website * Catch a no online banking service at login * Set type of accounts on professional accounts * Identify credit card for each transaction * Update to new certificate * Adding IBAN support for banque postal pro * Add a regexp on withdrawal labels * Parse IBAN for personal accounts * Do not crash if there is no IBAN for an account * Iban in unicode * Fix for newer pdfminer versions * Fixing bad account type on pro website * Sorting transactions before returning them * Fix for new API * Only call initialize if needed * Fix get_pages usage for new API * Fix newapi import * Fixing fake id for transactions * Fixing pdf parse with new api * Fixing no RIB Modules: bred * Support new website of bred * Fix compatibility with Decimal of python2.6 (#1685) * The 'categorie' key isn't always in json * Detection of disabled accounts * Correctly strip currency code * Correctly deal with cookies storage * Add type Account.TYPE_DEPOSIT * Catch technical error at login * Fix duplicates accounts * Fix PEP8 E713 * Get IBAN on accounts * Dd certash * Handle multiple universes * Oops fix mistake Modules: caissedepargne * Fix getting card accounts on old website * Set type on card transactions * Support Banque BCP which uses same website than caissedepargne * Do not force TLSv1 anymore as in some cases it doesn't work * Investment on lifeinsurance and market account * Add balance on NUANCE * Fix IndexError related to investment feature * Handling website error for market place * Fixing index error when checking if we are on error page * Fix BrowserUnavailable when browser is on market pages * Iban on cassedepargne accounts * Add new certhash * Ignoring CB transaction aggregation * Try to type more accounts * Adding support for saving type * Prevent crash on market * Get more investments * Valuation_diff on market accounts * Fix crash if there are several market accounts * Fix not authorized to read market accounts * Fix accounts parsing on banquedelareunion * Adding compte titre type * Fixing leaving investment space Modules: canalplus * Correctly fix the problem with streamed calls on requests (#1683) Modules: carrefourbanque * Fix accounts parsing * Fix 404 on history and now support "compte livret" * Fix when account balance is negative Modules: cci * Update for Join filter changes * Site changes Modules: chronopost * Status guessing Modules: cic * Handle new certificate * Supporting mouvements.html and situation_financiere.html * Fix list index out of range on iter_accounts * Rewrite module to browser2 (based on creditmutuel) * Cic can have letter in login * Add IBAN on cic * Fixing cic account with "Contre-valeur" Modules: citelis * Certificate hash change Modules: citibank * Do not wait for transactions if there's none. (#1661) * Credit limit, payment due date and amount for Citibank. (#1719) * Use V8 for login (#1743) * Relogin when the website returns garbage instead of a PDF. (#1820) * Site was updated. Closes (#1820) * Parse thousands correctly in the statements. (#1984) * Backport check_output for 2.6 Modules: cmb * Fix detection of authentication fail * Add deposit type in CMB module * Fixing when login is obsolete * Fixing cmb with some pro module * Quick fix for multiple space. Modules: cmso * Add pro browser to cmso bank module * Correctly raise instance of BrowserIncorrectPassword * Correctly handle 500 error during login * Fix parsing of market accounts and factorization of the Transaction class * Fix website changes on auth page * Go to subscription page before get accounts for some users * Use a local instance of LinearDateGuesser instead of global one * Support pagination of transactions * Fix: coma are used as decimal separators * Correctly handle expired sessions * Fixing duplicate entry * Handle double site type on the pro side * Clean + blackbox compatibility on cmso pro Modules: colisprive * Rewrite for browser2 * Fix history parsing Modules: cragr * Fix parsing of labels with new crédit agricole website * Fix parsing of raw transaction text * Handle useless pages to prevent being redirected to mobile version * Get IBAN on accounts * Detect connection errors * Add new labels to market type * Fix account list parsing, skip irrelevant account * Fix getting history for card accounts * Fix import of Account * Add tdbgestion page * The type of PEL accounts is now SAVINGS instead of MARKET. * Refactor * Support life insurance accounts * Handle sessionSAG parameter * Support market accounts * Handling bad investement parsing * Handling bad value for investement * Fix parsing on cragr investment * Fix av investment parsing * Do not return NotAvailable if value == 0.0 * Support loan accounts * Fix double clean of investment amounts * Add account types * DAV PEA is a checking account * Handle new login navigation on some sub websites * Fix incorrect password on new login * Add page to fix login access * Fixing categorization * Cragr perimeters * Adding new saving account name * Fix infinite loop when a crash occurs on perimters * Fix when some perimeters are crashing * Fix broken perimeters * Fix in case of CGU poping when changing perimeters * Fix card accounts balance parsing * Fix mistake Modules: creditcooperatif * Update regexp to support SEPA transfers * Fix exception in some cases * Fix regexp for external transfer * Handle investment * Correctly return an iterator even if this isn't an investment account * Add attribute _inv to pro accounts Modules: creditdunord * Handle the case when the creditdunord CB label is "OPERATION SUR CARTE" (#1712) * Remove spaces from IDs * Handling account with no history * Handling no details for this account * Fix crash on accounts which doesn't support history * Handle duplicate accounts * courtois: we now skip transactions without date even on pro accounts * Fixing wrong id when no details for an account * Trying to avoid duplicate accounts * Fix courtois we now always return iter for investment * Handle IBAN * Fix auth fail url * Add VirtualKeybord auth * Change lifeinsurance accounts type * Removing false id for transactions * Fix life insurance accounts and add valuation_diff * Raise BrowserIncorrectPassword is password isn't digits * Fixing history for account * Fixing pro account * Fix assert for password * Avoids infinite loop in case of fail login Modules: creditmutuel * Handle new useless pages * Updat transfer method * Save and load state of browser into storage * Handle refresh on empty page * Handling already logged * Handling html page * /cmmabn/fr/ is a valid login page * If do_login() is called, force go on the login page * Handle refresh on LoginPage too * Handle disconnection * Fix accounts parsing * Fix parsing of 'soft' parts of account names * Fix parsing of balances if accounts page has been configured by user to display balance in several currencies * Clear cookies before to login * Better form detection * Increase timeout * Clean transactions label * Parse contactless payments ("PSC") transactions * Handling orignal currency * Using TableCell to handle dynamic transaction list * Ignore ELEVE CARTE line * Add account types patterns * Add type pattern for young accounts * Investment for market accounts * Fix crash when trying to find reale amount of market accounts * Fix portefeuille account parsing * Add IBAN on creditmutuel accounts * Fix parsing market accounts * Fix parsing of unitprice * Fix skipping market summary fake account * Fix parsing valuation of a market account * get more iban * Fix parsing isin codes in special cases * Handle new website * Redirect page added * Fix label * Fix parsing label of accounts on old websites * Fix wrong valuation on market accounts if several are present * Valuation_diff on market accounts and new balance for PEA * Account typing * Fix blackbox compatibility * Fetch more transactions * Fix detection if transactions are coming or not * Fix when last_debit is None * New accounts type * Life insurance on old website * Fill the rdate field on life insurances history Modules: cuisineaz * Site changed / rewritten using browser 2 Modules: dailymotion * Fix dailymotion "search" return bug message * Fix to get video url Modules: delubac * Rewrite module with browser2 on new website * Handle accounts and transactions * Website require a digit password Modules: ovh * Adding bill for ovh Modules: edf * Correctly set amount and currency on bills Modules: ehentai * original_title does not exist anymore * There is no more description available * Update cardinality field name * Use Thumbnail object Modules: feedly * Favicon transparency and remove JPEG artifacts * Use new DictElement object * Improve tests * Iimprove login management * Fix login * Do not crash when there is no title in article Modules: fortuneo * Detection of accounts type * Support investments detail * Add page to browser: pea account type * Introduce new type Account.TYPE_LIKE_INSURANCE * Update changed certificate * Fix fortuneo module now support "compte especes" * Fix support of PEA accounts * Fixing empty performance * Fixing 2nd detail line who was considered like investment * Fixing bad int values on investment * Fetch history of life insurances * compte-espaces is a AccountHistory page * Correctly set type of market accounts * Fix savings account type * Fixing closed life insurance account on fortuneo * Raise IncorrectPassword when there is new id * Throwing error when an sms is required to access information * Switching error to browser incorrect password * Fixing listing account * Fix crash when parsing accounts without balance * Fix navigation * Support deferred debit cards * Set type of card transactions * Fix new pass required detection Modules: francetelevisions * Fix "search" always return empty list * Correctly fix the problem with streamed calls on requests (#1683) * Remove "latest" feature * Fix pluzz's search (1700) * Add option to get video from an url * Handle lastest replay videos * Fix bug search date in bad balise * Fix bug: we can only search one time * Use new DictElement object * Fix: site changed (#1855) * Fix: site changed again (#1868) * Update tests * Fix bug while getting video from some url * Fix error when video url is none * Handle url from francetvinfo website Modules: freemobile * Correctly handle wrong credentials * Fix parsing of bills urls * Force UTF-8 encoding with the standard mechanism * Enable bills filtering for multi-account * Fix subscription date parsing for multi-account * Fix forfait name parsing for multi-account Modules: ganassurances * Rewrite ganassurance with browser2 and fix auth and history * Add missing date column title for cards history * Fixing statefull website with account's link changing * Fixing coming date with an other label * Change AccountsPage xpath to only get personal finances accounts * SSL is not supported anymore * Add changing pass/id page Modules: googletranslate * Fix code used by Google Translate for Greek Modules: groupamaes * Use a local instance of LinearDateGuesser instead of global one * Should now always find accounts * Fix : site changed Modules: hellobank * Fix permissions, encoding and copyrights * Do not try to get IBAN from life insurance accounts * Do not try to get history of life insurance accounts * Allow transaction parsing across multiple pages * remove module, use bnporc instead Modules: hsbc * Handling when website is gone away * Fixing shared accounts_list * Fix new list of accounts and new URLs * No history on loan accounts * Handling duplicate account id (#2114) * Submit two useless form to mimic browser * Fix date parsing * Add coming for cards * Do not use fake id for transactions from hsbc * Allow hsbc user to have more than 8 length password * Fixing credit is positive on some account Modules: hybride * Fix: site changed * Fix: Date parsing * Fix: site changed again * Fix bug when there is no end date * Iimprove end_date management * Update to support timezone * Use Decode filter to get the good id * Simplify id Modules: imdb * Fix: python datetime is locale-dependent * Fix: 'N/A' value in movie release date Modules: imgur * Port to browser2 * Implement CapGallery Modules: ina * Bump to browser2 * Handle CapAudio * Find video from url Modules: ing * Coding style * Change selection of keyboard (more generic) * Fix navigation after a transfer * Remove hack to convert decimal to int * Fix crash if there is no ':' in IDs * Update regexp * Allow birthday to be entered with separators * Support of ASV accounts * Fix switch between market and checking websites * ASV accounts don't have history * Fix detection of errors in page * Get valuation information in several locations * NotImplemented exception for history on asv and clean code * As ING website sucks, re-login to retry getting investments list * Introduce new type Account.TYPE_LIKE_INSURANCE * Fix website change * Be sure we are logout before login * Store "where" in ing browser * Fix bad login with cookie cache * Fix parsing of euro founds * Add a kind of life insurance account name * Fix parsing of ISIN code for market accounts * Fix switches between market and bank websites * Euro funds have diff equal to 0 * Fixing not avalaible value * Handling netissima page * Fix set of Investment.unitprice for market accounts * Correctly get subscriptions even if previous page was on market * Fix finding jid on some pages * Load ISIN code from details page if not available in the list * Fixing empty page in some case in investment without pl in URL * Prevent useless request since there is no history on asv accounts * Handling +infinity (best investment ever) * Fix ISIN code on asv * Do not slugify investments without ISIN codes * Fix ing bills * Parse bills dates * Skip summary bills * Fix isin on US investment * Fix when valuation is not available Modules: inrocks * Url changed * Fix url matching * Do not try to parse premium news pages Modules: kickass * New domain name * Adaptation to new browser, iter_torrents mysteriously not working * kickass now works with new browser Modules: lcl * Website change, rewrite with browser2 * Ignore summary lines in history tables * Fix stop parsing not transactions lines * Certificate updated * Fetch detail of transactions and use it in Transaction.raw * Change timeout to 30 seconds * Call default Browser.deinit() * Do not change Transaction.raw if there is no detail * Adding pagination support for lcl * Fix async bug with cards * Fixing RC4 and cert for pp browser * New certificate for LCL Espace Pro * Browser doesn't inherit 'object', so 'super' doesn't work * Handling account can't use online service when login * Fixing bad login when session expire * New transaction detail page * New certificate for lcl entreprise * Fix date parsing bug on transaction label 0/07/2015 * Wrongpass instead of BrowserUnavailable * Fix PEP8 E713 * Fix contracts selection routing * Remove validate method * Handle one case of bad login/pass * Handle investment * Fix for clients with no market investment accounts * Fix crash on asv accounts * Fix parsing of life insurance accounts with special chars in owner name * Fixing when we have some additional information * Disconnecting from bourse portal before returning the account list * Fix bugs on av accounts, double ids and disc from detail * Fix mistake in disc page * Fix life insurance accounts ids * Fix parsing of market accounts * Fix crash on enterprise websites * Adding iban info when available * Fix pro accounts iban * No permission page to fix crash * Fix crash in case life insurances are unavailables * Support a new account type * Fix parsing of some transactions on pro accounts * Adding saving account * Handling IBAN single account * Adding checking account * Fixing payement wording * Adding SAVING type for account * Handle home page so the module is blackbox compatible * Adding account type * Change page to handle specific behavior Modules: leboncoin * Add missing favicons * Fix bug while parsing date * Fix bug when area is unknown * Add regions support * Fix error when there is no price and improve description display * Improve cities display in search * Fix search city using zipcode * Fix bug when user do not choose any city * Do not crash if a select does not exist in the form * Fix title parsing in search result * Get main photo when there is no carousel * Fix bug in costs retricton for rent type * Fix bug due to other backends cities * Fix site changed * Fix regex in id detection Modules: lutim * Upgrade to Browser2 Modules: mailinator * Fix module as the api changed Modules: mareeinfo * Add missing favicons Modules: marmiton * marmiton is now working using browser2 (#1706) * Fix bugs in marmiton * xpath change (#2117) Modules: mediawiki * Update tests to use "Project:Sandbox" page * Port to browser2 * Remove useless stuff Modules: meteofrance * Adapt to browser2 * Use new DictElement object * Allow to iter forecasts using town name in addition to city id * Site changed Modules: monster * Fix and adapt to browser2 * Improve page parsing * Adapt for new Join filter * Handle adverts coming from partner websites Modules: okc * Correctly detect when login fails * Handle logged page * Make all attributes of optim start with an underscore * Fix parsing of profiles * Way used to vote has changed * Fix encoding of urls * Fix parsing of thread title * Rewrite okc module with browser2 and less ugly code * Skip empty essay fields * Add status and orientation fields * Do not remove old threads anymore * Iter_unread_messages: do not visite read threads Modules: oney * Add new hashes for virtkeyboard * Change get_balance xpath * Adding hash for virtkeyboard * Handle multiple cards Modules: orange * Adapt to website changes (#1828) * Add https urls * Update maintainer * Supporting bill capability for orange Modules: pap * Adapt to new DictElement object * Fix bug in parsing date * Fix : site changed Modules: pastealacon * Fix: url changed * Works with other pastebin sites * Revert "pastealacon: work with other pastebin sites" Modules: paypal * Handling when user can't acces account balance on main account * Supporting new paypal website * Fixing using new account page version when we are already on old account page * Handling when currency is not in right place * Fixing primary account display twice * Fixing wrong id on account * Checking website version after login * Fix crash on new accounts list (#1654) * Fix selection of table containing last file requests to be less ambiguous * When balance is zero, paypal add a class to the div * Handling new website account for individual * Paypal fix for KeyError: 'counterparty' * We now ignore transactions with no 'displayAmount' * Fix name of paypal accounts (display currency instead of balance) * Take care of transaction's net amounts (#1694) * Fix warning with adding a new home page * Handling home page on new private website * Fix paypal new activity URL * Fix paypal * Use locale.format to format amount * Do not use locale.setlocale to parse amount * Fix history on paypal, stop parsing suspended recurring payment * Get converted amount for foreign currencies * Fix crash IndexError: list index out of range (#1747) * Remove unused code, fix timeout of history fetch * We don't try to parse transactions without currencyCode anymore * New cert + fix conversion warning * Handling useless captcha page * New paypal certificate * Execute javascript convert() function on token * Paypal handle bad login/password * Add paypal new page to fix crash on history of some accounts * Fix crash on old website and inform users * Change detection of pro or perso account_type * Increase DEFAULT_TIMEOUT for accounts with a lot of transactions with foreign currency * Fix call of ads_token convert function * Fix timeout on accounts with huge transactions number * Fixing timeout for some accounts * Handling timeout when fetching old history * Fix transactions parsing * Get real amount accounting paypal's commissions * Fixing captcha * Handling promo page for new paypal application * Rewrite paypal with browser2 and fix login * Add a way to detect login errors * Pro accounts transactions amount is now returned without commission * Ignoring refound offer for paypal * Fixing paypal auth challenge * Do not supply floats to Decimal constructor * Fix login * Detect a login error * Fix parsing accounts without available balance * All personal accounts seems to be on new api. * Parse more transactions not in account currency * Try to get the activity 3 times * Discard another transaction type * Fix regexp parsing converted amount of transactions * Fix mistake of converted amount parsing * Handling payement from another bank * Fix parse_transaction return * Using original wording for payement from another bank and setting type * Check if there is a associated transaction for "Achat de" * Fixing bad return * Include archived transactions * Fix pro accounts transactions amounts not in account currency * Fix parsing converted amount from other currency * Fixing paypal using different token for conversion * Removing commande à from transaction list * English key word to skip transactions * Fcking paypal Modules: piratebay * Use new domain names Modules: playme * Add a profile walker to automatically send won challenges * Implement get_account_status() * Fix warnings * Answer only to 5 questions * Fix challenges with users who have already challenged you * Fix crash when there is no contact * Use Module.create_browser to correctly give all standard arguments (proxy, logger, etc.) Modules: poivy * Fix low balance listing Modules: popolemploi * Fix search and upgrade to browser2 * Website uses https * Site changed Modules: presseurop * Fix body selectors Modules: radiofrance * Site changed / backend rewritten using browser2 * Manage selection podcasts / improve testing * Handle podcasts * Fix indentation typo * Minor fixes * Cleanly display live in repl * Handle new francebleu website, retrieve the good player url when default player is the video player * Fix 404 errors in selection Modules: razibus * Increase timeout * Support timezone Modules: redmine * Fix crash when parsing custom fields Modules: regionsjob * Fix: site changed * Fix parsing when advert is a picture * Update to new Join filter * Do not crash while parsing an external advert Modules: sachsen * Fix parsing of flow level * Improve test for sachsen module * Partial fix of new website * Fix history with new website Modules: seloger * Remove useless condition * Fix bug on pagination * Use new DictElement object * Handle special characters (#1883) * Fix json parsing Modules: senscritique * Skip test when StopIteration is raised * Improve module code * Fix date parsing * Add Mangas missing channel * Fix bug when end_date is asked and not found * Use set_profile(Firefox()) instead of _setup_session(Firefox()) * Remove id lists * Fix channels selection * Site changed (we cannot choose channels packages anymore) Modules: societegenerale * Correctly detect when the website is unavailable * Page parsing for saving accounts * Update upstream certificate hash * Update certificate hashes * Credit card is no longer in coming * Switch rdate/date and little fixes * Update hash of soge enterprise certificate * Fix duplicate accounts on cards accounts * Adding IBAN support for pro and entreprise accounts * Setting new certificat hash * Fix greediness of a transaction pattern to avoid crash * Update certificate * Testing account true id for card * Check ssl certificate after browser construction * Support market account * Support life insurance account (Investment) * Refactor * Support life insurance account (Transaction) * Fix type of 'PEA Espèces' account * Detect and log unknown type account * Do not iterate over all available transaction * Move set_date into LifeInsuranceHistory, and do not return to transactions list * Do not store NotAvailable amounts * Do not crash on pro account * Fix investment valuation * Fixing unsuported link account * Adding PEA Especes to unsuported link * Dd an url to the crashing ones * New ssl certificate * Add reinitpassword page to prevent crash. * Fix parsing and pagination of market investments * Try to type more accounts * Adding iban number * Try to fix weird crash * Fix navigation without id transactions * Fixing fake id for investment * Fixing account type Modules: tinder * Fix warnings * Fix crash when a contact has been removed * Use APIBrowser * Do not crash when the limitation rate is reached * fix login on tinder: need to find the cookie in js * Avoid aborting profiles walker when queue is empty * Fix PEP8 E713 * use Module.create_browser to correctly give all standard arguments (proxy, logger, etc.) Modules: transilien * Fix _regexp name in RoadMapDuration filter * Fix ambiguous departure/arrival in roadmap * Use new DictElement object * Fix incorrect roadmap duration parsing (#2089) Modules: twitter * Fix: site changed * Exclude non-tweets * Fix: trendy tweets (site changed) * Fix search pagination * Fix comments parsing * Fix condition to correctly check if user is logged Modules: vimeo * There is no latest category on vimeo (#1692) * Allow to browse categories and channels * Bump to https and handle site changes * isSafeForFamily field is not always filled + correctly handle its value Modules: vlille * Fix encoding error Modules: wellsfargo * Force TLS v1.0 for Wells Fargo module. (#1647) * Credit limit, payment due date and amount for Wells Fargo bank (#1720) * Scrape minimum credit card payment and due date (#1809) * Update to new login security checks (#1999) * backport check_output for 2.6 Modules: wordreference * [wordreference] Adapt to browser2 Modules: youjizz * Min pattern size for search is 3 characters * Use UserError instead of ValueError * Videos are now hosted on another domain Modules: youporn * 'a' and 'i' pattern are not supported (404 error) * Rewrite with Browser 2 * Use UserError instead of ValueError * get_video fill the id Modules: youtube * Fix parsing of js signature (#1695) * Backported changes from youtube-dl * Fix call to _extract_signature_function() Weboob 1.0 (2014-10-17) General * New agendadulibre module http://www.agendadulibre.org/ (CapCalendarEvent) * New Citibank module https://online.citibank.com (CapBank) (#1642) * New leboncoin module http://www.leboncoin.fr/ (CapHousing) * New mareeinfo module http://maree.info/ (CapGauge) * New playme module http://goplayme.com/ (CapMessages, CapMessagesPost, CapDating) * New razibus module http://razibus.net/ (CapCalendarEvent) API Big-Bang * Rename BaseBackend to Module * Rename BACKEND to MODULE * Rename backend.py to module.py * Rename BaseApplication to Application * Rename CapBase to Capability * Rename BasePage to Page * Rename BaseBrowser to Browser * Move CleanHTML to html filters * Remove * imports in filters * Move weboob.tools.browser2 to weboob.browser * Move weboob.tools.exceptions to weboob.exceptions * Move weboob.tools.browser to weboob.deprecated.browser * Move weboob.tools.parsers to weboob.deprecated.browser.parsers * Move weboob.tools.mech to weboob.deprecated.mech * Remove the "backend" result in do() calls Core * Catch the proper exception for missing icon * Replace usage of os.mknod() by os.open(O_CREAT) * Use the print() function everywhere * WebNip.iter_backends takes a new optional parameter 'module' * Add __getitem__ on WebNip to get a loaded backend by name * Create PrintProgress class instead of using IProgress as default one * Allow to load a module with config=None * A lot of pep8 fixes Capabilities * Let get_currency guess US$ means USD * Prevent mess when copying BaseObject instances Capabilities: bank * Add Investment.description field * Add Emirati Dirham AEB currency Capabilities: calendar * Add Conference event category Capabilities: parcel * Add parcelnotfound exception Capabilities: housing * Add and handle in flatboob house_types field * Add and handle in leboncoin a new house type: UNKNOWN * Adding a url field in housing capability and management of it in flatboob Applications * Add a new debug level (-dd option) * Add a " LIMIT " keyword in conditions * Centralize encoding guesses, default to UTF-8 (#1352) * Use class attributes as much as possible for application output * Define std* in the proper class * Handle datetime in condition argument * os.isatty is now forbidden (as stream.fileno() is not implemented by StringIO) * logging: Output to stderr, not stdout * logging: better colors Applications: repl * When getting an object, if at least one is found, display errors but correctly return the found object Applications: boobmsg * Fix "show" for threads Applications: flatboob * Ask for query.type in flatboob * Add load command * Fix bug type_of_good does not exist anymore Applications: Qflatboob * Manage count to avoid problems during pagination Applications: pastoob * Add an option to set a custom file encoding Applications: parceloob * Catch parcelnotfound by untracking Applicatitons: traveloob * Fix: crash if departure time is not available Applications: videoob * Set non verbose mode for wget when downloading m3u8 (fix #1643) Applications: weboobcfg * Return correct exit status code for enable and disable commands Applications: webcontentedit * Better checks for vim usage Browser * Add a way to asynchronously handle requests and pages * Backporting mergin_hook to support hook's requests in wheezy * HTMLPage checks the inner charset and parse again document if it is not the same than Content-Type HTTP header * Add a trivial android profile * Add has-class xpath function Browser: filters * Add debug informations * Raise ParseError only with None/NotAvailable/NotLoaded values, not with empty strings * Add a way to customize sign handling for CleanDecimal * Regexp: let template be a callable * Add some javascript dedicated filters * Add an nth parameter to Regexp filter * Add __str__ to _Filters Browser: elements * handle_loaders into AbstractElement * Ability to select an ItemElement DeprecatedBrowser * Fix: certificate check on servers which don't allow SSLv3 Documentation * Update to the new API * Show base classes in documentation Tools * American amount to decimal conversion (ref #1641) * PDF decompression function (ref #1641) * Regexp-based tokenizer (ref #1641) Tools: html2text * Use the class if possible Tools: make_man * Copyright on top of file Tools: newsfeed * No need for workaround with feedparser>=5.1 Tools: tests * Allow changing modules path and adding to PYTHONPATH Tools: pyflakes * Add test to prevent usage of prints in modules * Detect deprecated has_key function Tools: values * Ability to set value to an empty string if it is available in choices Packaging: setup * Add futures, avoid Py2-only libs under Py3 * Use Python3-compatible syntax in debpydep * Add ignore dirs for flake8 Contrib: boobot * Add a check_twitter method Contrib: videoobmc * Force relative imports Contrib: weboob-generic (munin script) * Add category option Modules: alloresto * Fix: website changes (enable https and fix the form xpath) Modules: arretsurimages * Fix: site changed Modules: aum * Remove useless features of module that don't work anymore * Enable https * Import exceptions from core Modules: banqueaccord * Support canceled transactions * Increase timeout because of slow website Modules: biplan * Use the Python SkipTest if possible Modules: boursorama * Remove prints Modules: bred * Limit length of password * Remove lot of old code and keep card transactions in separate card accounts * Translating accnum description Modules: carrefourbanque * Do not try to parse useless accounts (closes #1432) * Fix: login form is now the second form on the page Modules: cic * Fix: new certificate hash * Set an unique id Modules: cmso * Fix: parsing of transaction amounts (strip nbsp) * Fix: parsing of huge account balances Modules: colissimo * Fix: return the real error message, not "label" * Raise ParcelNotFound in colissimo * Return the fullid of not found parcel * Upgrade to browser2 Modules: cragr * Remove prints * Add a regexp for checking password Modules: creditcooperatif * Add unique id to creditcooperatif (perso) * Update regexps * Use find object * Upgrade to browser2 (perso) Modules: creditmutuel * Fix: do not lock browser2 anymore (#1635) Modules: dresdenwetter * Add the debug decorator to dresdenwetter filter Modules: europarl * Remove prints Modules: feedly * Use the Python SkipTest if possible * Fix: unicode warning Modules: fortuneo * Do exactly the same thing than js to always get accounts list Modules: gazelle * Fix: infinite loop on fail login, and fix error message lookup Modules: gdcvault * Remove prints Modules: grooveshark * Fix: bug when Year field is empty in grooveshark json * Use the Python SkipTest if possible Modules: hds * Convert to browser2 and fix it Modules: hellobank * Remove prints Modules: hybride * Use the Python SkipTest if possible Modules: imgur * Restrict URL to imgur domains Modules: ing * Fix: add an Index for some accounts... * Add a test to detect loops in the history * Fix: testing of saving accounts * Fix: crash on coming operations * Add loggedPage on bourse.ingdirect.fr * Add a @ckeck_bourse decorator for a clean redirect Modules: kickass * Fix: parsing of torrent titles Modules: lacentrale * Fix: deprecated has_key Modules: lcl * Always raise instances of NotImplementedError Modules: minutes20 * Fix: parsing insolite pages Modules: nettokom * Add tests Modules: okc * Remove prints Modules: oney * Add a favicon * Add missing symbols for the virtual keyboard * Fix: do not crash on months with no transactions Modules: ouifm * Fix: new radio names Modules: ovs * Force relative import Modules: pap * Adapt to browser2 * Exclude adverts from other websites * Fix: image retrieving Modules: pastebin * Fix: crash on spam page Modules: paypal * Use AmericanTransaction.decimal_amount in PayPal module. Part of #1641 Modules: quvi * Force relative import Modules: seloger * Adapt to browser2 * Fix: pagination * Fix: obj filling Modules: societegenerale * Remove prints * PIL is a global requirement, remove the check Modules: tinder * Fix: auth on tinder by correctly set the User-Agent header Modules: transilien * Fix: crash on late departures Modules: twitter * Fix storage system * Fix purge system * Do not import Browser1 exception Modules: unsee * Restrict URL to unsee domains Modules: vlille * Better description Modules: wellsfargo * Fix: compatibility with old versions of mechanize * Add a favicon * Rewrite Wells Fargo with browser2 (closes #1624) * Improved Wells Fargo module stability. * Use AmericanTransaction.decimal_amount, closest_date, decompress_pdf and ReTokenizer in WellsFargo module. Part of #1641 Modules: youjizz * Fix: fillobj on video thumbnail Modules: youtube * Update part of the js interpreter Weboob 0.j (2014-09-03) General: * New module: feedly (CapMessages) * New module: oney (CapBank) * New module: twitter (CapMessages) * New module: wellsfargo (CapBank) (#1430) Core * Rename CapBaseObject to BaseObject (#1424) * Rename ICap to Cap (#1424) * Ability to use weboob.function as alias to weboob.do('function') (#1425) Core: repositories * Fix HTTP error handling for browser2 * Use ConfigParser in priority with python2 (#1393) * Load browser only when needed Capabilities * Move DateField/TimeField/Delta out of BaseObject * Add LBP to currencies * Add documentation on object constants Capabilities: audio * Add Playlist and Album classes Capabilities: audiostream * Fix: get_audiostream does not have pattern name (#1626) Capabilities: dating * Add iter_new_contacts method Capabilities: files * Fix repr() and str() on File-based objects Capabilities: image * Remove data field in to_dict method to avoid json crash during conversion Capabilities: messages * Remove required items in Message constructor Capabilities: travel * Do not require in id in constructor Applications * Remove default import of browser1 * Import debug modules only when needed Applications: console * Remove the import of SSL exceptions * Add the default value displayed "upper" in aliases (#1319) * Allows shortcuts for modules (#881) * Use shortcut of id in interactive mode (#881) Applications: REPL * Allow to browse subfolders with ls * Change formatter when it cannot handle all selected fields * Introduce the DISPLAYED_FIELDS in formatter * Set fields in a consistant way with do() * Introduce parse_fields function * Use fullid parameter for CapObjects * Correct multiple language error * Move format_collection from repl to ifromatter * Remove the 'inspect' command Applications: formatters * Remove the '*' special fields in formatter * Table and Json formatters can write output to a file now (#1412) * Handle format_collection with JSON formatter Applications: boobank * Do not crash if the account type isn't in list (#1542) * Write the account currency in ofx output Applications: radioob * Manage Albums and Playlists * Fix: bug when a radio id contains a dot Applications: QHaveDate * Add tab to send queries Applications: videoob * Improve m3u8 management in download Browser1 * Introduce local exception for SSL errors * Only load FirefoxCookies as needed * Update Firefox versions to latest ESR Browser2 * Add more specialized exceptions * Allow setting query string params on build_url * Matching content with url using is_here * Ability to override the flush() method * Allow for a custom element finder * Add CSV pages * Do not crash if total_seconds() is not implemented * Fix documentation of nr parameter * Update Firefox versions to latest ESR * Add support for forms with multiple "submit" elements * Allow more flexibility for the submit button parameter Browser2: ListElement * Move ItemListTable-Element outside of page.py Browser2: filters * Overload & and | operators to chain filters (#1426) * Split filters in several files * Fix filters doctest * Force unicode * New RawText filter * New Base filter * New Type filter * Date: use default value for empty input * Date: properly handle defaults that are not datetimes * MultiFilter: allow for a default argument * Dict: manage default * Dict: ability to use Dict['a']['b']['c'] instead of Dict('a/b/c') (#1426) * CleanHTML: manage basestring * CleanDecimal: possibility to set custom separators * CleanDecimal: set replace_dots default value to False * CleanDecimal: do not crash with inputs like NotAvailable * CleanText: handle the non-breaking space thanks to the re.UNICODE flag * CleanText: add an option to keep (but normalize) newlines * CleanText: \t is always in \s so no need to add it * CleanText: add tests * CleanText: fix re flags usage for Python 2.6 (#1444) * Env: add support for a default Documentation * New Home Page * Add a "How to contribute" page * Add logo/favicon * Set more customizations * Add instruction for developers missing the first steps (#868) * Define backends/modules * Add local_run in documentation for developers * Fix many docstring issues * Change module documentation to learn browser2 (#1451) * Add __repr__ on NotAvailable, NotLoaded and _NO_DEFAULT constants to be more readable on doc * Import several pages from the wiki * Add documentation to report a bug (#873) Tools: AmericanTransaction * Add a transaction amounts cleaner helper for american banks Tools: captcha * Refactor VirtKeyboard class * Add a margin attribut * Add a grid based virtual keyboard Tools: date * Add more french dates translations * Class methods to convert date[time] objects Tools: genericArticle * Fix unicode warning Tools: make_man * Tell that it was generated automatically Tools: pyflakes * Fix: call of pyflakes on Archlinux (#1404) Tools: test * Fix: call of test.py (#1403) Tools: yaml * Represent weboob date[time] objects as timestamps Misc: local_run script * Allow customizing where the modules are Misc: setup * Support python3 (#1417 #1418 #1419) * Add prettytable in dependencies (#929) * Configure isort and flake8 Misc: Windows Installer * Remove some files * Fix bugs in windows installer scripts Contrib: munin * Rename generic-munin to weboob-munin * Move all scripts in the same folder * Encode and decode ID's in weboob-munin Contrib: boobot * Add command %delquote * Fix: %searchquote on unicode strings Contrib: XBMC/Kodi * Add a xbmc/Kodi plugin that interracts with videoob Modules: arte * Fix: Do not crash if 'VDA' fields is missing in json * Use M3U8 format instead of HBBTV * Fill video.url with NotAvailable if url is not found * Improve tests * Improve video quality choice * Handle arte podcasts * Add tests for program categories Modules: aum * Implement iter_new_contacts Modules: banquepopulaire * Strip displayed balance at end of transaction labels * Display check number in label (#1027) * Fix: remove spaces in IDs (#1368) * Support loan payment type Modules: biplan * Handle summer holiday in tests Modules: bnporc * Update order regexp * Fix: transfer regexp * Remove space in ids Modules: boursorama * Some English fixes in comments * Add new certificat hash Modules: bp * Fix: new login image for virtkeyboard Modules: bred * Handle space in account number * Switch configuration description strings to unicode Modules: caissedepargne * Force use of TLSv1 on lowsslcheck as the web server support of SSLv3 is broken Modules: colissimo * Fix: New API key for collisimo (#1617) Modules: cragr * Order transactions by date to prevent LinearDateGuesser to be duped by the f*cking website Modules: creditmutuel * Fix: set of debit date for card transactions Modules: dailymotion * Fix: dailymotion mplayer error "No stream found to handle url" * Fix: use https for test Modules: francetelevisions * Use filters as classes in chain (refs #1426) Modules: freemobile * Some English fixes in comments * Fix date of subscriptions when next month as less days than excepted (#1347) Modules: gdcvault * Remove unused import of ControlNotFoundError Modules: grooveshark * Update to match Album and Playlist management in radioob * Display users playlists only when split_path length is 0 * Fix: catch exception when id is not an integer Modules: hellobank * Get default account name if the custom name is empty Modules: hybride * Fix: handle summer holiday in tests Modules: imdb * Some English fixes in comments * Use omdbapi instead of imdbapi * Fix: site changed Modules: ina * Fix: bad characters in titles (double encoded unicode) Modules: ing * Some English fixes in comments * Remove the index on ing for pagination * Support coming operations * Fix: parsing of 'tomorrow' transaction dates Modules: izneo * Fix: bug in page list Modules: lcl * Handle lcl pro https://professionnels.secure.lcl.fr * New certificate * Add transaction patterns Modules: leclercmobile * Fix: do not crash if balance is not available Modules: lefigaro * Remove dead code Modules: meteofrance * Fix: site changed (#1390) * Fix: call the url that retrieve all the search results (#1431) * Raise an exception if forecast param is not a city id (#1433) Modules: opensubtitles * Some English fixes in comments * Fix: site changed (#1295) Modules: pastealacon * Convert to Browser2 (#674) * Use specialized Browser exception Modules: pastebin * Convert to browser2 * overload & and | operators to chain filters (refs #1426) * Handle limit exceeded warning * Fix: crash with Base() and filter chaining Modules: paypal * Get more transactions on paypal (#1405) * Retrieve all transactions from the history, merchant and regular account support (#1406) * Paypal transactions history fetching with adaptive steps (#1406) * Checking if tr contains text * Make Paypal module use AmericanTransaction helper. * Fix: empty amount. (#1415) * Support french dates for last CSV request * Ignore canceled transactions Modules: popolemploi * Fix: site is now only availbe using https Modules: presseurop * Presseurop is back! (named now voxeurop) Modules: radiofrance * Fix: FIP radio does not work (#1449) Modules: sachsen * Set the datetime to NotAvailable by default Modules: senscritique * Fix: bug in network selection * Fix: set channels and programs parameters in get_event Modules: societegenerale * Fix: certificate changed * Fix: certificate updated (#1414) Modules: sueurdemetal * Fix: broken module due to departments containing letters Modules: tinder * Update recs only when needed * Fix attribute type Modules: transilien * Adapt to browser2 * Fix: site changed (#938) Modules: vimeo * Fix: site changed * Adapt to browser2 * Enable search and tests (#1082) * Catch HttpNotFound errors Modules: youjizz * Overload & and | operators to chain filters (refs #1426) * Use filters as classes in chain (refs #1426) Modules: youtube * Fix: Youtube mplayer error "No stream found to handle url" * Fix: is_logged function does not work (#1423) * Backport some youtube-dl changes (#1422) Weboob 0.i (2014-05-19) General * New Browser: browser2 is here! See below * First steps in python3 support * New GPG key for laurentb (see https://laurent.bachelier.name/gpgtransition.txt) * New module: btdigg (CapTorrent) * New module: Mailinator (CapMessage) * New module: guerrillamail (CapMessage) * New module: SensCritique (CapCalendar) * New module: imgur (CapPaste) * New module: unsee (CapPaste) * New module: lut.im (CapPaste) * New module: pixtoilelibre (CapPaste) * New module: ColisPrive (CapParcel) * New module: AlloResto (CapBank) * New module: LaCentrale (CapPriceComparison) * New module: Groupamaes (CapBank) * New module: Tinder (CapDating) Core * Import pkg_resources only when needed, to prevent a hard useless dependency Core: browser2 * Add a new browser based on python-requests * Full support of SSL * Add the option _proxy_ssl for backends * New management of Html FORMs * New URL class * Better debugging (save HTTP requests/responses) * Simple support of pagination (@pagination decorator) * Introduce ListElement and ItemElement (parser helper) * Add a lot of Filters for ItemElement * Add TableElement, to help the parsing of a table * 24 modules are already using browser2! * And a lot more... Core: browser1 * Unify no_login and nologin * Fix call of logger (can be unset) * Move exceptions in weboob.tools.exceptions (compatibility with browser2) Core: BackendCalls * Rewrite BackendCalls with queues Core: repositories * Make GPG ignore local settings Core: configuration * Support for password retrieval from any cli tools * Prompt user during configuration of a backend Core: ManPages * Escape more '-' Core: Parsers * Add a xml parser * Remove unused/obsolete parsers Core: DateTools * Add ChaoticDateGuesser Applications: base * Add version information in Debug * catch ^C on global bcall errors handler * Allow to use condition on the fullid * Do not display warning message if -n or count is set Applications: console * Fix: masked parameter wasn't considered * Fix: password encoding issues * Display backend@issues.weboob.org and not backend.EMAIL Applications: media_player * Add a play_proxy option Applications: Qt * Add keywords in .desktop files (#1356) Applications: REPL * Exclude backends that do not implement the required method (#1325) * New obj_to_filename() method (from videoob) * Do not fetch the collections when not needed * Fix selecting a field in non-interactive mode * Use a fast path for ls if -U is set * Add documentation of the "ls -d" option * Gather collections having the same path * Override comp_obj from repl in order to sort by start_date Applications: formatters * Add a json formatter that works on a line level (json_line) Applications: yamlconfig * Allow a default equal to None Application: boobcoming * Add documentation * Add icon * Use ls/cd commands to navigate into categories * Improve collection management Application: boobsize * Add icon * Display null values * Fix hints in not found error Application: boobtracker * Support colors for issues * Create/edit tickets in a text editor * Fix email address regexp * Fix crash in interactive mode * New options --tracker, --priority, --start, --due Application: parceloob * Remove trailing \n * Ignore status of not loaded backends * Always untrack if the id is in the storage Application: pastoob * Paste binary files with "post_bin FILENAME" and "get_bin ID" * Add 'info' command Application: Qhavedate * Ignore error when notes are not implemented on contacts Application: radioob * Fix audio objects search * Pick up first available stream when radio provides a playlist (#1345) Application: translaboob * Fix a typo in the documentation Application: videoob * Braces-enclosed tags are replaced with data * Play/info: ability to give several videos * Add command 'videoob playlist download' * Add command 'videoob playlist play' * Move obj_to_filename() into ReplApplication * Use wget to download m3u8 videos Application: webcontentedit * add a -r parameter to get command to precise revision id Application: wetboobs * Remove ICapGauge Capabilities: Base * Change currencies integer constants to ISO code strings * Do not require an id, because of ItemElement which constructs objects without arguments * Introduce find_object Capabilities: CapWeather * Fix: zero temperature can not be displayed (#1333) * Accept date objects for the forecast Capabilities: CapCalendar * Raise NotImplementedError on "attends" method * Add TELE category Capabilities: CapBugTracker * Fix some conversion warnings Capabilities: CapBank * Add Account.TYPE_CARD * Add TransactionsElement and TransactionElement (helpers with browser2) Capabilities: CapContact * Factorize aum's Contact.get_text() and boobmsg formatter Capabilities: CapGeoloc * Remove unused ipaddr field Capabilities: CapMessage * Avoid warnings in GenericNewspaperBackend and GenericNewsPage Capabilities: CapPast * Show a user-friendly error message for CapNotFound Capabilities: CapTracker * Add trackers and priorities fields to Project * Add start, due, tracker and priority fields to Issue Modules: banquepopulaire, bnporc, bp, cic, cragr, creditdunord, creditmutuel, ing, lcl, societegenerale * Improvements on transaction and account types detection Module: AmericanExpress * Fix wrong date guessing * Increase timedelta to 90 * Check if a card is valid * Set card balance to Account.balance instead of Account.coming Module: Apec * Updates reflecting site changes Module: Apivie * Fix getting an account from list * Fix parsing of negative transactions Module: Arte * Encode UTF-8 strings in search * Fix naming of collections * ArteLive: new website * Improve the parsing of dates * Fix bug when VDA field not found in json file * Fix problem on extension Module: AxaBanque * Adding support for sub accounts * Fix parsing of accounts without amount Module: Banque Accord * Add icon * Fix parsing of negative amount * Fix detection of card pages * Fix virtual keyboard * Add a hash for symbol 5 on virtkeymap * Adding support for Leroy Merlin * Support LOAN accounts * Upgrade to browser2 Module: Banque Populaire * Fix: parsing of accounts (site changed) * Fix: parsing of accounts on specific regions * If an account type is not supported, raise NotImplementedError Module: Biplan * Fix bug while parsing start/end time * Updates reflecting site changes * Upgrade to browser2 Module: Bnporc * New certificate * Fix encoding issues * Disable unused parameter rotating_password * Change the domain to entreprises.bnpparibas.net for BNPEnterprise Module: Boursorama * Fix crash when there is no ID on transaction * Handle card history Module: BP * New certificate * List market account (#1298) * An account type must not be None Module: Bred * Support dispobank * New certificate Module: CanalPlus * Fix browsing categories (site changed) Module: Carrefour Banque * Upgrade to browser2 * Fix: Site changed Module: CCI * Upgrade to browser2 Module: Citelis * New icon * Fix: detection of login errors Module: cmb * Change a function call to avoid useless None * Upgrade to browser2 Module: cmso * Setting encoding to iso Module: Colissimo * Add icon * Fix license Module: Cragr * Fix parsing dates in labels (compatibility with Perigord) Module: Credit Du Nord * If type is not found, set TYPE_UNKNOWN instead of None * Add "Banque Rhône-Alpes" Module: Credit Mutuel * New icon * Upgrade to browser2 * Fix parsing of user private accounts * Fix crash on unknown accounts types Module: cuisineaz * Fix cuisineaz preparation and cooking time (site changed) Module: Dailymotion * Fix empty fields in dailymotion plugin * Change extension from flv to mp4 * New icon * Add support for videos available at kids.dailymotion.com * Fix the search (site changed) Module: Delubac * Support LCR * Fix navigation Module: DLFP * Fix logout (site changed) Module: DresdenWetter * Upgrade to browser2 Module: Fortuneo * New certificate * Do not override rdate with operation date Module: France Televisions * Upgrade to browser2 Module: Freemobile * Fix date of subscriptions when next month has less days than expected (#1347) * Upgrade to browser2 Module: googletranslate * Fix : site is now only available using https Module: grooveshark * Fix: no more field AvgRate (site changed) Module: HSBC * Fix: support of the SecureKey * Upgrade to browser2 Module: hybride * Upgrade to browser2 Module: INA * Site changed Module: indeed * Upgrade to browser2 Module: ING * Update regexps used to detect categories (site changed) * Upgrade to browser2 * Support pagination in iter_bills * Do not crash on invest objects when quantities are >999 Module: Kickass * Fix title getter (site changed) Module: LeclercMobile * Do not crash if the subscription is closed Module: LeFigaro * Fix: site changed * Fix the tests Module: LCL * New maintener * Support changes in login screens * Get more history * Remove logging checking on the contract page Module: Liberation * Add icon Module: lolix * Fix the size of the icon Module: mediawiki * Fix encoding error Module: MeteoFrance * Strip the text before to return it Module: OKC * Add the visit to a profile * Add a profile walker * First sent message is configurable * Do not crash on removed contacts * Do not crash on 'like' messages Module: pastealacon * Upgrade to browser2 Module: Paypal * Certificate changed (three times) Module: piratebay * New domain (thepiratebay.se) Module: Pole Emploi * Updates reflecting site changes * Fix parsing of dates * Fix bug in url encoding * Fix place selection Module: Poivy * Upgrade to browser2 * Support pagination for the history Module: Redmine * Add Issue.fields attribute to support custom fields * Support new versions of redmine * Use the right method to get project * If a category does not exist, try to create it * Fix compatibility with redmine 2.4 * Support start/end/tracker/priority * Fix finding the control to add a note Module: RegionsJob * Add icon * Upgrade to browser2 Module: sachsen * Upgrade to browser2 Module: Société Générale * Display card as accounts (#1362) Module: Sueurdemetal * Add date_end parameter in get_concerts_date call * Fix date and category filters Module: voyagessncf * If station ID is not found, try to look for it as a name * Forge a random IP source address to avoid redirection to other website (#1327) Module: vlille * Upgrade to browser2 Module: Weather * Fix: there are not always ten days of forecast (#1343) Module: youjizz * Handle videos longer than 59 minutes * Support pagination * Upgrade to browser2 Module: youtube * Fix: site changed * Fix: unexpected argument to BrokenPageError Tools: boilerplate * Add 'cap' recipe to prepare methods * Upgrade to browser2 Tools: local_install * Run tools/stale_pyc.py before local installation Tools: pyflakes * Ignore 'except' in a comment Misc: boobot * Add quoting commands * Fix unicode issues Misc: Windows Installer * Better script to generate the .exe * Better management for 64 bits systems Weboob 0.h (2014-01-08) General * New application: boobcoming * New application: boobsize * New capability: CapCalendarEvent * New capability: CapFile * New capability: CapAudio * New capability: CapAudioStream * New capability: CapImage * New module: Apivie (CapBank) * New module: Audioaddict (CapRadio) * New module: Banqueaccord (CapBank) * New module: Biplan (CapCalendarEvent) * New module: CCI (CapJob) * New module: Collisimo (CapParcel) * New module: EDF (CapBill) (#1296) * New module: GitHub (CapBugTracker) * New module: Indeed (CapJob) * New module: Jacquieetmichel (CapVideo) * New module: jcvelaux (CapGauge) (#376) * New module: JVMalin.fr (CapTravel) * New module: Liberation (CapMessage) * New module: lhybride.org (CapCalendarEvent) * New module: Monster (CapJob) * New module: Nectarine Demoscene Music (CapRadio) * New module: Nihon no Oto (CapRadio) * New module: OVS (CapMessage, CapContact) * New module: Pariskiwi (CapCalendarEvent) * New module: Poivy (CapBill) * New module: Quvi (CapVideo) * New module: RegionsJob (CapJob) * New module: Somafm (CapRadio) * New module: Sueurdemetal (CapCalendarEvent) * New module: VoyagesSNCF (CapTravel) * Removed module: ecrans (see new liberation module) * Removed module: Isohunt (site closed) Core * Add class WebNip (Web [in] Non Interactive Programs) * Modify/factorize/add CapVideo, CapAudio, CapImage, CapFile Core: application * The "condition" system has been rewritten (and moved from bcall to reply applications) (#1289) (#1288) * repl: add <,> and | operators in conditions * repl: support date in conditions (#1308) * repl: catch error in condition evaluation * repl: sort results of "ls" * repl: add -U option to not sort result with "ls" * repl: add documentation for conditions in manpages * repl: exclude backends which does not implement 'method' in get_object * repl: restrict to 40 results in ls by default * repl: accept 0 and negative values for count in interactive mode * Base: hide FormFieldConversionWarning * JSON: fix export format (#1294) * JSON: encode all subobjects CapBank * Add Invstment in __all__ * Use raw label to calculate unique_id CapGallery * Add iter_gallery_images method CapJob * Change API: add advanced_search_job CapTravel * Add parameter 'date' to ICapTravel.iter_station_departures * Add fields to Departure (arrival_time, price, currency) CapVideo * Inherit from CapImage CapWeather * Fix htmltable display for wetboobs forecast command (#1318) Application: boobank * Truncate label in formatter 'account_list' * Add OFX export Application: boobill * Catch some errors (#1303) * Fix: documentation erros Application: handjoob * Fix: display of results * Better ls command Application: Qhandjoob * Adapt to ICapJob update Application: parceloob * Do not show "None", but an empty string Application: pastoob * Add help for "infinite" max_age Application: radioob * Let user choose his Stream to use from Radio instance (#1291) * Use CapRadio * Add support for download and streaming of BaseAudio objects * Add support for playlist of BaseAudio objects Application: traveloob * Add (colored) formatters Application: videoob * Add playlist support * Sanitize default download filename * Fix: do not return None objects on get_object * Support curl for downloading * Allow to download in a folder with generated filename Application: wetboobs * Depreciate ICapGauge commands (#1284) Tools: Application * Add mpv to default players Tools: GenericBackend * Make a real link in signature Tools: Browser * Move the test on self.insecure to catch direct calls Tools: Value * Order Choices Module: adecco * Fix: site changed Module: allocine * Implementation of the new API mechanism Module: ameli * Fix: site changes * Better handle of login Module: amelipro * Fix listing of bills Module: apec * Fix: site changed (get job offers) * Use html2text to fill description content Module: arte * Add liveweb.arte.tv url management * Update to use arte API * Fix: fully fill Artevideo in get_video method (#1312) Module: arretsurimage * Prevent to catch all urls Module: Banquepopulaire * Get list of all accounts * Fix: do not crash if there is no operations * Fix: do not crash when there is no full list page * Support loan accounts * Support accounts with empty balance * Fix: do not crash with market accounts * Fix: new authentication * Fix: sometimes it is not possible to sort by value date in history * Fix: crash on special account histories * Support defered cards * Fix: getting history on some special workflow * Fix: crash when there is the coming column with empty value Module: Barclays * New favicon Module: Bnporc * Fix: update BNP certificate fingerprint * Add transaction patterns * Support new virtual keyboard * Fix: typo in label, s/Profesionnels/Professionnels Module: Boursorama * Fix: crash when there is no link on an opening account * Remove dead code Module: BP * New Virtualkeyboard Module: Bred * New configuration option: force the selection of account * Fix: selection of accounts when there are both personal and enterprise ones * Fix: set the right debit date on card transactions * Fix: crash on page to select account Module: Caissedepargne * New certificat Module: Carrefourbannque * Removed broken regexp for login parameter. Module: CIC and Creditmutuel * Backport fixes from creditmutuel * Correctly get the accounting amount * Support coming transactions * Do not try to find coming transactions for inappropriate account * Set the right debit date on card transactions * Fix parsing card debit date when there is a link in text * Support multi-cards pages Module: CMB * Fix: parsing accounts when there are mandated ones Module: Citelis * New certificate Module: Chronopost * Do not set location to "None" if empty Module: Cragr * Do not follow spam links * Fix: parsing of card accounts in particular cases * Fix: correctly set cards' Transaction.date to the debit date * Fix: crash when there are only three columns in card transactions listi * Fix: account IDs can contain anything else than digits Module: CreditCooperatif * New favicon Module: Creditdunord * Support enterprise accounts * Fix: parameters on request to get professionnal accounts history * Support SEPA Order parsing * Fix: pagination on professional website Module: Dailymotion * Fix: dailymotion thumbmail retrieving (#1310) * Fix: found the video URL * Fix: try to get the full JSON data * Fix: bugs in video pages (#1320) Module: Delubac * New favicon Module: Dresdenwetter * New favicon Module: francetelevisions * Fix: downloading in non-interactive mode * Fix: KeyError: 'data-date' (site changed) (#1282) Module: Grooveshark * Force mp3 extension * Many code style fixes * Manage albums search * Add user playlist management * Declare user_id as an instance variable * Fix: unitialized property user_id * Fix: NoneType return value when user is not logged * Fix: typo in test * Adapt test for logged and non logged users Module: hellobank * Fix decimal parsing of account values Module: Imdb * Fix: site changed * Add some tests Module: Leclercmobile * New favicon Module: Mangago * Fix: GenericComicReaderTest import path Module: MeteoFrance * Fix: parsing of pages, site changed Module: LCL * Fix: detection of authentication errors * Support new contracts selection page * New certificate * Fix: remove all of the agency parameter (#1313) Module: Leclercmobile * Better balance information * Remove "votre solde" in history * Strip labels Module: Lolix * Fill search job method in lolix to avoid error message Module: Nettokom * Add get_balance method * Fix: encoding warnings Module: NolifeTV * Use mobile.nolife-tv.com instead of online.nolife-tv.com * Add theme/type entries * Allow anonymous surfing Module: popolemploi * Improve deep search Module: Prixcarburant * Fix sites changes (#1031) Module: ING * Use raw parser for TitrePage * Add LDD support * Get history of titre account * Do not crash if there is no operation * Raise UseError on empty reason for transfer (#1315) Module: ipinfodb * Fix: form changed from GET to POST Module: SFR * Fix: site changed * Add some tests Module: Societegenerale * Fix: parsing of cards with undefined payment date * Add transaction patterns * Fix: conversion warnings (#1304) * Fix: support new authentication system Module: TVSubtitles * Add some tests Module: Vlille * Improve method _get_sensor_by_id * Scrap webpage instead of provided xml (get more informations) Module: Wordreference * Fix: site changed Module: Youjizz * Fix: get real thumbnail Module: Youtube * Add a new YouTube video URL * Set YOUTUBE_MAX_START_INDEX to 500 Contrib: downloadboob * Fix: call to ICapVideo.search_videos (API changed) (#1301) * Fix: print help instead of crash Contrib: generic-munin * Add get_object_list command * Fix: do not crash if one value is NotAvailable Contrib: kmymoney * Fix dead lock with krosspython > 4.8 * Add a Weboob configuration tab in account configuration window * Restrict download to N transactions by updating the history Misc: packaging * Get absolute path without readlink (compatibility with BSD systems) * Add windows installer * Use more common syntax (refs #1299) * Better detection of Pillow Misc: tools * local_run: do not erase existing PYTHONPATH Weboob 0.g (2013-08-03) General * New application: handjoob * New application: parceloob * New application: qhandjoob * New capabilitie: CapJob * New capabilitie: CapParcel * New module: Citélis (CapBank) * New module: Banque Delubac (CapBank) * New module: Hello Bank! (CapBank) (#1276) * New module: Ameli (CapBill) * New module: AmeliPro (CapBill) * New module: GDF SUEZ DolceVita (CapBill) * New module: V'Lille (CapGauge) * New module: Adecco (CapJob) * New module: Apec (CapJob) * New module: Lolix (CapJob) * New module: Pôle Emploi (CapJob) * New module: Paroles.net (CapLyrics) * New module: Chronopost (CapParcel) * New module: UPS (CapParcel) * New module: Allrecipes (CapRecipe) * New module: Supertoinette (CapRecipe) * New module: podnapisi (CapSubtitle) * New module: Arrêt sur Images (CapVideo) * New module: Grooveshark (CapVideo) * New tool: stale_pyc.py * New tool: certhash.py Core * Change '_backend' setting to '_module' (#789) * Remove all compatibily glue for python 2.5 (#806) * Remove some from __future__, forbid 2.5 in setup * Do not crash when repository is invalid (#1281) * Explicit module load errors * Fix: import Pillow in core * Fix: create containing directory of backend configuration * Fix: better English sentence when Weboob refuses to start * Fix: crash if backend name contains unicode chars Core: applications * Display a message when more results are available (#1038) * Default behavior is now to unlimit results, except for explicit commands (searches or history) (#1139) * Remove useless calls to ReplApplication.flush() (#812) * Add colors in console applications * console: display collections before objects with ls command * console: fix error if there are private attributes on backend * console: better handling of "tiny" choices, allow forcing * console: Automatically create a storage if STORAGE class attribute is not empty * repl: add a new caps parameter to get_object * repl: get_object supports backend's services returning lists * repl: Invoking 'help' command from shell display only list of commands * Fix: printing of DictObj Core: browser * Add a cache for DNS requests * Implicitly convert unicode objects to str for form values Core: capabilities * Move Currency from CapBank to CapBase * Fix: problem with strftime on date.year<1900 CapBase * add method to convert currency value to text CapBank * do not crash if date in label regexp isn't valid * add Investment object and iter_investment method * add Transaction.unique_id * more help for date fields * introduce vdate * FrenchTransaction.parse takes a new optional argument 'vdate' CapBill * add attributes to Bill (#1202) * add new attributes to Detail (#1203) CapBook * remove unused parameter 'pattern' in iter_books CapGauge * add address field to GaugeSensor CapMessage * default type of threads is IS_THREADS CapRecipe * handle interval number of persons * fix comparison to None CapSubtitle * add 'ext' attribut (extension of file) CapWeather * always call base constructor of CapBaseObject to prevent shared fields Core: documentation * Fix typo and wordwrap in INSTALL * Add manpages for new applications Core: tools * Do not try to fill None * All string values are Unicode * Allow overriding delay and tries in decorated function arguments * Add parse_french_date in weboob.tools.date Application: boobank * In interactive mode, 'transfer' commands asks confirmation (#1077) * Add parameter END_DATE to 'history' and 'coming' commands (#1150) * Add investment command * Avoid blank lines between entries in (pretty_)qif formatting * Better error messages * Prevent encoding issues in __repr__ of Transactions * Fix: do not flush twice Application: booblyrics * Fix: search command needs 2 arguments Application: boobmsg * Fix: display of messages list in non interactive mode Application: cineoob * Add caps parameter to get_object (#1280) * Filmographies and castings restricted to movie/person backend * Add language abbreviations in subtitle search help * Small bug fixes Application: comparoob * Add missing docstrings to commands Application: cookboob * Handle interval number of persons * Better nb_person interval display * Check if preparation time is available * Strip newlines from descriptions in search results Application: flatboob * Add command help * Do not use limit for cities search Application: qcineoob * Check if short description is available * Encode filename when download * Add tabs navigation * ctrl+l to get focus on search edit * Do not go on middle click opened tab * Remove useless print Application: qcookboob * Handle interval number of persons * Better nb_person interval display Application: qvideoob * Use fillobj instead of get_video to complete all fields Application: suboob * Rename command 'getfile' to 'download' * Save subtitles in a better file name (#1278) * Print filename where subtitle is saved (#1279) * Add language abbreviations in subtitle search help * Do not show URL if empty Application: webcontentedit * Use NamedTemporaryFile(delete=False) instead of mkstemp Application: weboobcfg * Remove 'backends' command from help * Fix: do not crash if a module cannot be loaded Application: weboorrents * Catch empty seeders and leechers Application: wetboobs * Fix: gauges command parsing Module: 750g * Strip accents in search URL * Fix: comments and accents striping Module: allocine * Fix: gender detection Module: americanexpress * Fix: parsing of coming value * Fix: parsing of April month Module: aum * Support new fields * Logging to see profile (because of craps) * Unescape description of profiles * Disable warning when unable to get public profile from website * Fix: encoding errors * Fix: typo on 'style' field Module: arte * Add support of ArteLive * Fix: parsing on German and English pages * Fix: id is now prefixed with "--" * Fix: prevent module from catching all URLs Module: attilasub * Add attribute 'ext' Module: axabanque * Detect when there is no operations Module: banquepopulaire * Support new authentication on some regions * Sort entries by value date * Add new URL for homepage * Store vdate * Fix: go on professional accounts list page Module: bnporc * Add support of personal accounts on professional website * Add enterprise website support * Support when sometimes 'coming' value is '-' * Prevent navigation conflicts during accounts iteration * Add regexp for payback of card transactions Module: bp * Fix: support new authentication Module: boursorama * Add two factor authentication for boursorama module * Add history for saving accounts * Add PEL support * Fix: parsing on new website * Fix: new certificat * Fix: support new website Module: bred * Supports new page which asks for a code from a card * Fix: login on accounts with several subaccounts * Fix: new certificat Module: caissedepargne * Support professional website * Add support of deferred debit cards * Be more robust on logout * Clear cookies during login * Fix: detection on login failed * Fix: navigation * Fix: avoid crashes if some useless fields are missing in global form * Fix: support new version of website * Fix: import of ControlNotFound on old versions of mechanize * Fix: navigation in card accounts history Module: cmb * Use new-style classes (coding style) Module: cic * Supports new page which asks for a code from a card Module: cragr * Add support for credit cards * Ignore accounts without balance * Add new regions * Support special login URLs on some regions * Use new-style classes (coding style) * Support account where name contains the owner * Do not raise BrowserIncorrectPassword when the website displays an useless "urgent message" on login * Better detection of useful label * Fix: use web site instead of mobile website for ca-centrest, and probably others (#1035) * Fix: an account owner name can start with Mle Module: creditcooperatif * Support coming transactions on pro accounts Module: creditdunord * Fix: navigation in history calls * Fix: attach cards to the good account * Fix: encoding error to get history for accounts with special chars in name * Fix: navigation for deferred cards * Fix: crash if the date of a transaction is empty Module: creditmutuel * Fix: listing of recipients in transfer Module: cuisineaz * Temporary fix of person number range * Fix: comments and accents striping Module: dailymotion * Fix: parsing of authors Module: dlfp * Be compliant with the dlfp 1st april joke * Fix: get new threads even if there is no comment Module: dresdenwetter * Add tests * Fix: no more snow sensor Module: freemobile * Add some tests * Fix: history when internationals calls option is not subscribed (#1078) * Fix: new certificat * Fix: conversion warnings * Fix: new bill listing * Fix: bad method name in tests Module: ganassurances * Enable support to Groupama Banque (#1112) Module: gdcvault * Only logout when actually logged in Module: hsbc * Support international accounts * Add a retry if login fail * Add missing file (login.py) * Fix: update module to support last website update * Fix: new certificate Module: imdb * Replace latin2unicode by HTMLParser Module: ing * Add support of investments * Add type of accounts * Raise NotImplementedError if account type not supported * Add some tests * Fix: conversion warnings * Fix: history of accounts * Fix: new certificat * Fix: transfer Module: kickass * Use https * Fix: new URL Module: lcl * Add enterprise site * Fix: detection of bad passwords * Fix: support new auth system * Fix: do not need to logout if the browser hasn't been used Module: leclercmobile * Add real tests * Fix: history parsing * Fix: details parsing * Fix: compatibility with calibre 0.9 Module: marmiton * Fix: comments and accents striping Module: opensubtitles * Add attribute 'ext' Module: orange * Fix SMS sending (#1206) Module: ouifm * Fix: json describing current songs and artists changed Module: paypal * Fix: ignore extra-lines in accounts list * Fix: new certificate Module: piratebay * New URL * Support proxies Module: presseurop * Get newspaper source * Fix: body parsing * Fix: better check for daily links Module: redmine * Fix: missing id class on older versions Module: sachsen * Fix: string conversion * Fix: tests do not work with new CapGauge API Module: seeklyrics * Fix: parsing in some cases Module: societegenerale * Add entreprise support * Do not consider expired cards * support 'OPPOSITION' state for cards * Fix: conflicts in detection of balance * Fix: new hashes for login * Fix: new certificat Module: tvsubtitles * Add attribute 'ext' Module: youtube * Support for a silly protection (#1277) * Detect if video is forbidden * Use https * Change maintainer * Fix: encoding of pages * Fix: stop search queries if there is no more results * Fix: parsing details * Fix: crash with weird characters * Fix: credentials encoding Module: vimeo * Get authors Module: weather * Add a favicon Tools * boilerplate.py: fix encoding issues * boilerplate.py: add default license * pyflakes.sh: more code quality checks * pyflakes.sh: use only tracked Python files * pyflakes.sh: add lines number * pyflakes.sh: ban more stuff * local_install.py: no deps by default on local install * local_run.py: do not copy the backends file * local_run.py: do not capture stderr * local_run.py and run_tests.sh: remove stale .pyc files * local_run.py: allow complete paths for script * local_run.py: allow overriding WEBOOB_BACKENDS * weboob_bash_completion: fix bash completion Contrib: boobot * Support python-irclib >= 3 * Proper freenode server name * Irc 5.0+ compatibility * Display regular URL info * Support multiple channel * Fix weird/unicode URLs * Guess page encoding * Better newline handling * Support servers without HEAD * Faster timeout * Twitter support * Configuration of boobot storage path * Better handling of zero/invalid HTML * Proper ^C closing * Support ignored users Contrib: freemobile-munin * Fix SMS regexp Contrib: generic-munin * Allow float values * Do not start every graph to 0 * Use rigid Packaging * Add --nodeps option to ignore dependencies in setup.py * Rename Makefile -> build.mk to avoid spurious build attempts * Use Pillow, unless PIL is already there * Better README/INSTALL files * Workaround Debian compatibily layer Weboob 0.f (2013-03-26) General * New application: booblyrics * New application: cineoob * New application: cookboob * New application: suboob * New application: qcineoob * New application: qcookboob * New capabilitie: CapSubtitles * New capabilitie: CapCinema * New capabilitie: CapLyrics * New capabilitie: CapRecipe * New module: American Express (CapBank) * New module: Paypal (CapBank) * New module: Crédit du Nord (CapBank) * New module: Allocine (CapCinema) * New module: IMBD (CapCinema) * New module: Parole De Musique (CapLyrics) * New module: Parolesmania (CapLyrics) * New module: Seeklyrics (CapLyrics) * New module: 750g (CapRecipe) * New module: Cuisineaz (CapRecipe) * New module: Marmiton (CapRecipe) * New module: Attilasub (CapSubtitle) * New module: OpenSubtitles (CapSubtitle) * New module: tvsubtitles (CapSubtitle) * New module: Btmon (CapTorrent) * New module: DresdenWetter (CapWeather) * New script: Generic Munin Plugin Core * Do not cry if an object is not support to be filled by a backend * Add a CSV parser * Remove two old and useless debug messages (bcall) * Force updating modules after upgrade (#1066) Core: application * Fix crash when there is no selected fields * Allow infinite search with 0 * Add a JSON encoder * move to_dict to base.py (#1060, #1061) Core: browser * Ability to set a ENCODING constant to BasePage which overrides the Browser's one * Use recent firefox user agent (Firefox ESR) * Fall back on SSLv3 if TLSv1 fails * Cache the sucessful SSL protocol Core: tools * Create DateGuesser extracted from cragr Application: boobill * Improve documentation of download command Application: havedate * Display contact IDs in 'events' command output Application: pastoob * Add command-line parameters (#920) * Support "no expiration" Application: videoob * Add resume option for video downloading * Stop the video when closing with ESC Application: webcontentedit * Add non tty stdin support to edit command * Add "get" command to webcontentedit Application: Wetboobs * Add a default error message on SensorNotFound Module: aum * Handle GONE exception * Mimic mobile application (#1041) * Fix: date used may be UTC * Fix: display of summary * Support new 'favorite_food' field * Go on website to get stats and geographical position * Read field ratio * Don't stop on threads with a contact who left the website * Fix: send email when taken in a basket Module: axabanque * Remove mkdtemp in module * Fix: crash on a new tab Module: banquepopulaire * Fix: parsing of some card transactions * Update regexps Module: barclays * Update regexps * Support market page (but do not display any history) * Support when accounts are in javascript... Module: bnporc * Remove mkdtemp in modul * Support pro accounts Module: boursoram * Take id of the website for transactions * On page to update user information, raise BrowserIncorrectPassword * Get maximum history * Do not crash for special accounts * Remove mkdtemp in module Module: bp * Fix: New SSL certificat Module: bred * Fix: deferred debit cards parsing Module: caissedepargne * Update regexps * New SSL certificat * Support more history Module: cic * support pagination * support wher forced to go on change_password page * Fix: crash on pagination Module: cmso * Update regexps Module: cragr * Use LinearDateGuesser * New maintener * Support new website Module: creditmutuel * Transfer now accepts long and short IDs * Support pagination * Support wher forced to go on change_password page * Fix: transfer problem * Fix: crash on pagination Module: dailymotion * Fix search * Avoid redirects * Fix: extraction of video url Module: dlfp * Do not crash when revision is provided(#1067) Module: fortuneo * Update regexps * Support when website asks to renew password * Fix: card transaction pattern * Fix: accounts list on new version of website * Get more history * Support PRLV pattern Module: francetelevisions * Fix: search Module: freemobile * Add an id for calls * Fix get_bill * Add multi-account support for bills command Module: gdcvault * Retrieve now video, audio, slides and Dual Screen Video * Implement searching * Implement login * Handle pages with redirects to files * Fix a lot of bugs on some pages Module: hsbc * Support deferred cards * Do not support other kind of account histories than CPT Module: ing * Detect when website is unavailable * Fix: new website * Fix: generation of IDs * Merge LoginPage and Login2Page to prevent problems when the website fails with 500 * Prevent useless debug folders Module: isohunt * Fix: bug if leech or seed is empty corrected * Fillobj integration Module: kickass * Fix: downloading gziped torrents * Fix: download URL and support magnets * Return NotAvailable if needed * Fix: url correction * Fix: download URL and magnet support in all iter_torrents Module: mediawiki * Add support for the revision argument Module: lcl * Support when there is an error * Remove mkdtemp in modules Module: leclercmobile * Add an id for balance objects Module: nettokom * Fix: crash when no data available * Display validity date of subscriptions Module: piratebay * Fix: bug on empty result page corrected * fillobj integration * get_torrent return None if 404 * Unescape title Module: presseurop * Better id management * Increase RSS size Module: redmine * Add support for the revision argument (#1067) Module: sachsen * Fix: as0.gif keyerror * Better image analyse Module: societegenerale * Fix: login * Fix: Do not crash when an account is unavailable * Fix: Transfer regexp * Recipient of a transfer can be a name Module: trictractv * Add an icon Module: vimeo * Fix for no-embed videos Module: wordreference * Fix: Take the first word * Make result cleaner * Add favicon Module: youtube * Fix: detection of media url on youtube Capabilitie: CapBank * More robust currency guessing * Fix repr() on transactions with unicode labels * support empty dates for transactions Documentation * New git URL * Add missing "whatis entry" to man pages (#618) Contrib * Add a "id_regexp" parameter to filter results on video id * Exclude on case sensitive pattern * Add utf-8 support on title_exclude Tools * Add basic boilerplate script (replace gen_comic_reader.sh) * Remove storage_convert.py * Ignore Qt generated UI files (pyflakes) Packaging * Update setup.py for latest Gentoo installs * Use a central Makefile * Fast and silent building * Ensure proper building of man pages * Tell groff manpages are utf-8 * Update INSTALL Weboob 0.e (2013-01-25) General * New module: Axa Banque (CapBank) (#807). * New module: Barclays bank (CapBank). * New module: Carrefour Banque (CapBank). * New module: Credit Cooperatif (CapBank). * New module: Credit Mutuel Sud Ouest (CapBank). * New module: Gan Assurances (CapBank). * New module: Mangago (CapGallery). * New script: anonymiser.sh. Module: Arte * Fix: parsing video title (site changed). Module: Banque Populaire * Update transaction regexps. * Correctly encode login and password. * Fix: change hostname of Aquitaine Centre atlantique bank region. * Fix: parsing accounts for some specific versions of website (reported by Alain Spaite). * Get the full label of an account. * Fix: parsing banquepopulaire accounts with no link. * Fix: crash where there is no transactions for an account. * Fix: date parsing (happy new year). * Add Credit Maritime regions compatibles with Banque Populaire. Module: bnporc * Add a second SSL cert. * Support international withdrawal. Module: Boursorama * Fix: Correctly parse label of deferred card transactions. * Support of Livret A accounts. * Support new Virtual keyboard. Module: bp * Fix: compatibility with python2.6. * Add a SSL timeout. * Support page delestage. * Support deferred debit. Module: Bred * Correctly encode login and password. * Support payment in several times. * Fix: parsing accounts when there are interlines. * ignore special accounts. * Fix: crash when there is no ID on a transaction. Module: Caisse d'épargne * Handle login errors. * Support when website is unavailable. * Parse more card transactions. * Fix: parsing funding accounts. * New favicon. Module: cic * New favicon. * Fix: parsing amounts in special cases * Fix: parsing card pages in special cases Module: cmb * do not consider special accounts * Fix: parsing balance Module: CrAgr * update transaction regexps * Fix: do not crash if there is no history link for an account * correctly use iterators Module: dlfp * Allow anonyme browsing (#923) * Fix: deconnection error * Fix: Support new html structure Module: ebonics * Add favicon Module: ehentai * Fix: Support new website Module: Fortuneo * Add new certificate Module: France Télévisions * Fix: Support new website Module: Freemobile * Fix: Support the new website * Add a real id for all details objects (#932) * Add renew date (#931) * Support of multi-accounts (#890) Module: Gazelle * Fix: parsing torrents descriptions Module: HSBC * Raise exception on login error Module: ING * Add ICapBill to download monthly reports (#897) * Add support of proxy account * Add support of Livret Epargne Orange accounts * Fix: crash when there is no operation on an account * Support question page Module: INA * Try to handle more videos Module: LCL * Fix: login regexp * Add the new SSL certificat * Add a pattern to detect cards Module: Leclercmobile * Fix: Support new versions of website Module: Le Figaro * Fix: encoding * Remove some unused files (#819) * Fix: author selection Module: Nettokom * Add validity date Module: Opacwebaloes * Fix: Accept ID with 8 chars (#921) Module: pastalacon * Remove some test warnings Module: pastebin * Fix: visibility detection * Fix: login without API * Remove some test warnings Module: Piratebay * Fix: search (site changed) Module: Presseurop * Remove tweeter sharing * Use clean_relativ_urls * Fix: Support new version of daily titles Module: Sachsen * Add a favicon * Search is no more case sentitive * Use ICapGauge instead of ICapWaterLevel * Get alarm level Module: Societegenerale * Handle error when service is unavailable * Update transaction regexps * New SSL certificat * Check date of card debit to apply it * Fix: crash on special accounts Module: Taz * Remove some adds Module: Wordreference * Fix: parsing (#840, #1036) Module: youporn * Set domain to www.youporn.com Module: youtube * Fix: login CapBank * Support currencies CapBill * Rename iter_history to iter_bills_history * Add validity and renew date fields CapCollection * Declare fields to avoid warnings CapGauge * Change API to be more flexible * Add GaugeSensor objects CapVideo * Fix: CapVideo-related ConversionWarnings (dailymotion, ehentai, francetelevisions, ina, radiofrance, youjizz, youporn, youtube) CapWaterLevel * Rename it to ICapGauge Console Applications * Properly get to the previous path, not home * Add support for ls -d option (#858) * Add global "insecure" option to skip SSL validation (#930) * Allow to exclude some backends (#617) Application: boobill * Add the keyword "all" to download command Application: boobmsg * Improve documentation of export_thread * Allow "show" command in non-interactive mode * Datetime now human readable Application: comparoob * Remove double products with multiple backends * Sort results in comparoob (#934) Application: radioob * Support of rtmp Application: videoob * Add -e to rtmpdump options * Allow to use all videos players * Allow to give arguments to players * Add mplayer2 support Application: weboob-config * fix double message "Unable to load module" Application: wetboobs * Change to use ICapGauge instead of ICapWaterLevel * Add sensors command Tools: html2text * Set INLINE_LINKS to false Tools: Newsfeed * Take the creation date if no update available Tools: Messages/GenericBackend * Get the real thread of an article * Add clean_relativ_urls function Tools: VirtKeyboard * Add an overridable method to check pixel colos * Add a parameter "convert" Tools: IParser * Add a raw parser Tools: LxmlParser * Get all strings under this element Tools: tests * Save and submit test results * Better way of handling non-module tests * Guess the right builder name * Use the guessed nosetests for the core tests too Tools: pyflakes * Use flake8 if available instead of pyflakes Core * Check value of ValueBackendPassword during load * Add new exception BrowserForbidden and add handler in applications * Fix langage when the lastet version is already installed * Allow multiple CERTHASH values in a module * Ability to specify a parser to use on a page handler Installation * setup.py: print what executable we were looking for * setup.py: be less confusing when make is missing Contrib: boobank-munin * Fix: encoding issue Contrib: downloadboob * Correctly create symlink and support when one is removed * 'exclude' parameter is case insensitive Contrib: nettokom-munin * Print all labels in config mode and stack results Weboob 0.d (2012-10-28) General * New module: BanquePopulaire (CapBank). * New module: Bred (CapBank). * New module: CaisseDEpargne (CapBank). * New module: CIC (CapBank). * New module: Ebonics (CapTranslate). * New module: EuroParl (CapVideo). * New module: GDCVault (CapVideo). * New module: LeclercMobile (CapBill). * New module: Vimeo (CapVideo). * New module: Weather (CapWeather). * New script: contrib/downloadboob to automatically download vidéos matching some criteria (#838). * Add a basic system to check SSL certificates. Console Applications * Correctly check new version of the module. * Short commands for all applications (and suggestions). Module: AuM * Add a status field to display number of new visits. * Support new API (#871,#900,#902). Module: BNPorc * Strip trailing 'CARTE XXX' from card transactions labels. * Get last 100 transactions instead of 30. * Fix: website changes. Module: Boursorama * Correctly handle case that we don't support the history page of an account. * Fix: unlimited loop in case of bad credentials. Module: BP * Parse labels and detect type of transactions. * Get maximum of transactions in history. * Workaround to libssl 1.0.1c-4 bug in Debian (#863). Module: CappedTV * Fix: proper handling of No Results Found. Module: CMB * Support proxy. * Check SSL certificate. * Fix: a bug in the accounts listing. Module: CrAgr * Reworked the whole parsing for 'list' and 'history' operations. * Parse labels and detect type of transactions. Module: CreditMutuel * Support deferred debit. * Never store 'RELEVE CARTE' transactions (redundancy with card transactions). * Fix: login (skip page which warns about phishing). Module: Dailymotion * Add support for embed URLs. * Fix: parsing authors on anyclip videos. * Fix: search-then-play combination (#860). Module: Fortuneo * Fix: website changes. * Fix: check on login failure. Module: FreeMobile * Catch unit for data. * Support multiple accounts. * Fix: login (website change). * Fix: pdf downloading. * Fix: international calls. * Fix: parsing name with a dash. Module: Gazelle * Fix: parsing title on whatcd. Module: HDS * Fix: parsing dates. Module: HSBC * Support history and coming transactions. * Fix: duplicated IDs of accounts. * Fix: parsing of accounts and history. Module: ING * Transaction IDs are generated with a hash function. * Use static IDs of accounts. * Support tranfers. Module: LCL * Parse labels and detect type of transactions. * Display password renewal warning. * Support CB operations. * Support several contracts. Module: LeFigaro * Fix: parsing of live articles. Module: Minutes20 * Fix: RSS feed url has changed. Module: Nettokom * Fix: website change about authentication. Module: Newsfeed * Add a link on top of content. Module: NolifeTV * Fix: authentication and retrieve of video url (#880). Module: Nova * Fix: retrieving of current song. Module: PAP * Fix: parsing housings (#874). Module: PressEurop * Fetch last 140 entries instead of 50. Module: PrixCarburants * Fix: regexp of product name in page. Module: RadioFrance * Support franceinfo archives. Module: SocieteGenerale * Truncate password to 6 digits. * Handle error message when unable to login. * Support deferred debit cards. * Fix: follow next pages of history. * Fix: website changes about authentication (#891). Module: Transilien * Support when a train is removed (#894). * Fix: parsing time (#866). Module: Yahoo * Fix: search of cities (website change). Module: Youjizz * Fix: parsing of duration. * Fix: parsing video url. Module: Youporn * Fix: website changed. Module: Youtube * Add support for youtube-nookie.com. * Fix: login and play of nsfw videos (#918). * Fix: finding video url (website change). Application: boobank * Fix: QIF spec requires capitalized 'type' (#898). Application: boobill * New command 'balance'. Application: comparoob * Fix: selection of product. Application: freemobile-munin * Add parameter 'phonenumber'. * Fix: data parsing. Application: havedate * Add command 'events'. Application: pastoob * Abort if paste is empty. Application: QHaveDate * Nicknames are selectable (#869). Application: QVideoob * White background is not forced anymore (#823). Application: translaboob * Add 'nigger' language. Application: weboob-debug * Support more shell libs (ipython, bpython, standard python). Appliccation: wetboobs * Ability to display temperatures either on celsius or on fahrenheit. Core * Use module 'imp' instead of '__import__'. * Fix table formatter with python-prettytable 0.6. Weboob 0.c (2012-05-04) General * New capability: CapPriceComparison. * New capability: CapTranslate. * New module: ChampsLibres (CapBook). * New module: Fortuneo (CapBank). * New module: GoogleTranslate (CapTranslate). * New module: NettoKom (CapBill). * New module: Okc (CapDating). * New module: PrixCarburants (CapPriceComparison). * New module: Taz (CapMessages). * New module: WordReference (CapTranslate). * New application: comparoob (CapPriceComparison). * New application: translaboob (CapTranslate). * New script: boobot. * New script: nettokom-munin (Nettokom). * New script: report_accounts.sh (CapBank). * Application havedate now inherits commands of boobmsg. * Fix: use the right default value for XDG_DATA_DIRS. Console Applications: * Add completion on 'ls' and 'cd' commands. * When a module requests to application a password (for example if it is not stored in the backend config), display a real question, not only 'Password:'. * Use $EDITOR env variable to edit input. * Rewrite of the formatters system. * When a bug occurs in a module, suggest user to update it. Capability: CapBank * Rename Operation object to Transaction. * Rename iter_operations() to iter_coming(). * Add a field Transaction.rdate to store date of payment. * Add field Transaction.type (enum). * Add field Account.type (enum). * Create a class FrenchTransaction in weboob.tools.capabilities.bank to help parsing transactions of french banks. * Use decimal.Decimal instead of float to store amounts of money. Module: AuM * Fix: website has changed its charset. * Fix: don't crash when a message content is empty. * Fix: save IDs as integers in storage. Module: Arte * Fix: getting latest videos. Module: BNPorc * Support private messages (CapMessages). * Fetch the last 100 transactions (instead of 30). * Support CIF accounts. Module: Boursorama * Parse categories and dates. * Ignore spurious accounts. * Fix: credit and debit where inversed. Module: BP * Don't list spurious accounts. Module: CrAgr * Fix: parsing dates. * Fix: CA Centre website has changed (#850). Module: CreditMutuel * Capitalize each words in account labels. * Fix: parsing accounts list whene there is no history associated. * Fix: parsing history when there are extra columns. * Fix: use unique IDs for accounts. Module: Dailymotion * Fix: getting latest videos. * Fix: getting thumbnails. * Fix: parsing video IDs. Module: DLFP * Do not get pages with 0 comment. * Add a cleanup function to dlfp backend. Module: FranceTelevisions * Fix: getting latest videos. Module: FreeMobile * Fix: remove timer (not more used by website). * Fix: website has changed. * Fix: multiple accounts login. Module: ING * Use FrenchTransaction for parsing transactions. * Get more historical transactions. * Catch error on password/birthday/login. * Fix: website changes on livret A history. Module: LCL * Add a regexp on password field. * Fix: getting accounts. * Fix: crash when the balance is negative. * Fix: site specific fix. * Fix: parsing account labels. * Fix: always go on the history page. * Fix: when history is empty. * Fix: prevent infinite loop when password is wrong. Module: MangaFox * Fix: domain name change. Module: PressEurop * Fix: parsing of cartoon pages. * Fix: real author catching on article pages. Module: RadioFrance * Fix: crash when there is no author. Module: Sachsen * Catch “Go Up” value. Module: SocieteGenerale * Support display of history and coming transactions. * Fix: when there is no transaction for the account. Module: Transilien * Fix: roadmap when there is no confirm page. Module: YouJizz * Fix: getting video URLs. Module: YouPorn * Fix: changes on website. Application: boobank * Add a new pretty_qif formatter (use label and rdate if exist). Application: boobathon * Fix: close command. Application: boobill * Get all available subscriptions for 'details' command (#846). Application: boobmsg * New command 'photos' to display photos of a contact with cacaview. * Add an 'atom' formatter. Application: bonboob * Add a command 'once' to process retrieving of messages once. Application: QWebContentEdit * Possibility to edit an url (#619). * Add a 'History' tab. Application: weboob-config * Remove 'install' command. Script: freemobile-munin * Fix: condition to detect empty output. Script: hds/export.py * Fix: loading HDS module. Core: * Change way to describe fields of CapBaseObjects. * Add a lot of code documentation. * Warn when implicit conversions happen on CapBaseObjects. * Create UserError exception for errors raised by modules. * Scheduler: do not stop timer because of an exception. * Repositories: support gpgv2 (#837). Weboob 0.b (2012-03-14) General * New capability: CapBill. * New capability: CapBook. * New capability: CapHousing. * New capability: CapWaterLevel. * New module: CappedTV (CapVideo). * New module: FreeMobile (CapBill). * New module: OpacWebAloes (CapBook). * New module: Pap (CapHousing). * New module: PressEurop (CapMessages). * New module: Sachsen (CapWaterLevel). * New module: SeLoger (CapHousing). * New application: boobill (CapBill). * New application: boobooks (CapBook). * New application: flatoob (CapHousing). * New application: QFlatBoob (CapHousing). * New contrib script: freemobile-munin. * Rename application: havesex -> havedate. * Rename application: QHaveSex -> QHaveDate. * Remove module: MangaToShokan (website is dead). Console applications: * Improvements of 'ls' and 'cd' commands. * Fix: crash when trying to add twice a same module. Module: Boursorama * Fix: IDs of operations. Module: BNPorc * Parse categories. * Parse account types. * Fix: getting history of secondary accounts. Module: BP * Store transactions dates as datetime.date objects. * Fix: parsing of accounts in particular cases. Module: CanalPlus * Friendlier paths in canalplus. Module: CMB * Add type and label support to CMB backend. Module: CrAgr * Now handle history-less accounts, at least for Toulouse-like layouts. * Transaction objects are now created using datetime.date (#800). * Fix: balance retrieval for history-less accounts (#788). Module: CreditMutuel * Store Transaction.date as a datetime.date object. * Support loan history. * Parse type of transactions. * Fix: lot of bugs. Module: DLFP * Parse a comment only if needed (performances enhancement). * Get comments only in case of rss-comments changes. Module: Ecrans * Store only last 20 articles. Module: HSBC * Display only real accounts. Module: ING * The account listing is now more robust. * Parse types of transactions. Module: Kickass * Fix: search was broken. Module: Mangafox * Fix: crash on Comments page. Module: NolifeTV * Support authentication. * Fix: parsing of videos with missing description. Module: OuiFM * Fix: error when no artist/title. Module: Pastebin * Fix: getting pastebin paste visiblity. Module: Piratebay * Support magnets properly. Module: RadioFrance * Support replay. * Use a better source for FIP. * Fix: fetching FIP current. Module: SocieteGenerale * Check if login is failed. Module: Youtube * Fix: crash when there is no author. Application: boobank * Display types of transactions. * Add formatter 'transactions'. Application: chatoob * Removed because unused. Application: videoob-webserver * Moved to contrib/. Application: weboob-config * Fix: command 'list' can filter by capability (#786). Application: weboorrents * Support magnet URLs. Application: wetboobs * Now supports CapWaterLevel in addition to CapWeather. Core: * Big refactoring of CapCollection. * Use libyaml for faster loading and saving. * Sign modules.list. * Support gzipped responses in Browser. * Fix: retrieving third icons if module is local. Weboob 0.a (2012-02-05) General * New backend: Boursorama (CapBank). * New backend: CMB (CapBank). * New backend: HSBC (CapBank). * New backend: ING (CapBank). * New backend: Nolifetv (CapVideo). * New backend: RadioFrance (CapVideo, CapRadio) which merges FranceInter and LeMouv backends. * Support repositories to manage backends (#747). * Support XDG Base Directory Specification (#765). * Make CapCollection understandable and usable by humans. Console applications * Ability to add all supported modules at once. * With -a, try to separate saved responses by backend (#661). Qt applications * Verbose error messages (#775). Backend: AuM * Display the 'title' field on contact profiles. * Get events (baskets, charms and visits). * Add city in status message. * Add 'Sent with my iPhone' in mail signatures when needed. * Display at least one photo url in profile even if there isn't any visible. * Add a 'ratio' field. * Fix: getting contacts list. * Fix: support new API. * Fix: ipaddr is not available anymore. * Fix: do not crash if there are php errors before json. Backend: Batoto * Change .com to .net. Backend: BNP * Fix: website changed (#769). Backend: BP * Fix: website changed. * Fix: open images in memory instead of saving them in temp files. Backend: CanalPlus * Support https?://www\.canal-?plus\.fr/.*\?vid=(\d+) urls. Backend: CreditMutuel * Internal transfer implemented. * Fix: website changed. Backend: Dailymotion * Fix: support missing descriptions. Backend: DLFP * Limit feed entries by date instead of number of results. Backend: FranceTelevisions * Fix: website changed. Backend: Gazelle * Handle general errors when website is unavailable. Backend: INA * Fix: parsing of URLs. Backend: Kickass * Fix: website changed. Backend: LCL * Add support for various types of accounts. * List 45 days history by default. * Fix: website changed. Backend: LeFigaro * Better cleanup of generic articles (#725). * Fix: website changed. Backend: Orange * Fix: unicode errors. Backend: Pastebin * Handle user pages. Backend: Piratebay * Fix: website changed. Backend: SocieteGenerale * Fix: website changed. Backend: Youjizz * Fix: video URLs have changed. Backend: Youporn * Fix: website changed. Backend: Youtube * Fix: website changed. Application: QBoobMsg * When a thread is a discussion, display messages linearly. * Add a button 'Profile'. Application: QHaveSex * Add a 'Events' tab. * Add a 'Notes' tab for taking notes about a contact. Application: QVideoob * Fix: crash when there is no thumbnail on a video (#708). Application: weboob-config-qt * Ability to edit sources.list, update repositories and install modules. Weboob 0.9 (2011-10-10) General * New backend: Batoto (CapGallery). * New backend: Eatmanga (CapGallery). * New backend: FranceTelevisions (CapVideo). * New backend: HDS (CapMessages). * New backend: Izneo (CapGallery). * New backend: Mangafox (CapGallery). * New backend: Mangahere (CapGallery). * New backend: Mangareader (CapGallery). * New backend: Mangatoshokan (CapGallery). * New backend: Nova (CapRadio). * New backend: PhpBB (CapMessages, CapMessagesReply). * New backend: Simplyread.it (CapGallery). * New application: boobtracker (CapBugTracker). * New script: hds/export.py to export stories from histoires-de-sexe.net into a sqlite database. * License change to AGPLv3+. * Ability to not save backend passwords. Applications will prompt them when needed. Backend: Arte * Set duration on video results. Backend: AuM * Rewrite to use the mobile API. Backend: BNPorc * Get operation categories. Backend: CrAgr * Get full history. * Optimized operations retrieval for CA Centre. * Fix: parsing accounts list. Backend: CreditMutuel * Maine-Anjou, Basse-Normandie bank added. Backend: Dailymotion * Fix: parsing duration. * Fix: parsing of lives in search results. Backend: DLFP * Fix: disconnection URL needs to be called with POST instead of GET. * Fix: do tests on alpha.linuxfr.org. * Fix: relevance on news/diaries. Backend: Gazelle * Handle login errors. * Fix: parsing on several gazelle websites. Backend: INA * Support videos on www.ina.fr (in addition to boutique.ina.fr). Backend: IPInfoDB * Fix: no crash when there are no coordinates from server. Backend: Kickass * Fix: parsing of search results (#663). Backend: MediaWiki * Handle errors from API. Backend: MeteoFrance * Fix: website changes. Backend: Orange * Fix: handling of pages. Backend: PirateBay * Fix: parsing of search results. Backend: Redmine * Now implements CapBugTracker (#684). Backend: Transilien * Supports the roadmap feature (#681). Backend: Youtube * Support more URLs. * Support authentication. * Fix: website changes. Application: boobank * Add the 'qif' formatter. Application: boobank-munin * When handling an incorrect password error, disable backend to prevent your bank to disable your account (for example with BNP). Application: pastoob * Ability to pipe output. Application: traveloob * New command 'roadmap' to get the roadmap between to stations. It's also possible to give a departure or an arrival time (#681). Application: videoob * Do not run player in background (#664). * Command 'download' supports mms streams. Application: weboorrents * The given filename to command 'getfile' is optional (#683). Weboob 0.8 (2011-05-08) General * New backend: E-Hentai (CapGallery). * New backend: FranceInter (CapRadio). * New backend: LeMouv (CapRadio). * New backend: PasteALaCon (CapPaste). * New backend: PasteBin (CapPaste). * New application: Boobathon (CapContent). * New application: Galleroob (CapGallery). * New application: Pastoob (CapPaste). * Fix lot of unicode problems. * License changed to AGPLv3+. * FreeBSD support. Repl applications * The new CapCollection capability is supported by every REPL Application. At the moment, only few backends implement it. * Better messages and return codes in applications. Backend: AdopteUnMec * Fix: website changes (#508,#637,#638,#639). Backend: BNPorc * The 'rotating_password' parameter is now an hidden one (#627). * Ability to do transfers to external recipients. Backend: BP * Add 'comptes titres', 'comptes vies' and 'comptes retraites' to the list of accounts (#567). * Fix: website changes (#621). Backend: CanalPlus * Implement the new CapCollection capability. Backend: CrAgr * Support of a new history layout. * Fix: login issues with the Toulouse website (#629). Backend: Dailymotion * Fix: URL was not found in special cases. Backend: FourChan * Fix: support message "AMERICA FUCK YEAH" outside of a thread. Backend: Gazelle * Fix: work with broken gazelle websites. Backend: Inrocks * Fix: matching URLs. Backend: KickAss * Several fixes and enhancements (#651). Backend: LCL * Fix: websites changes. Backend: OuiFM * Implements the new CapCollection capability. Backend: Transilien * Add the PAA code for Gare de Lyon. Backend: Youtube * Support more URLs. Backend: Youporn * Correctly set the 'ext' attribute on videos. Application: monboob * Check configuration (#484). Application: weboob-config * New commands 'enable' and 'disable'. Core * Use lowercase http_proxy environment variable. * select() function has been moved into parser. * Support for xpath in LxmlHtmlParser.select. * Fragments are removed from URLs (#636). * Remove a hack from feedparser which fixes parsing of webpages with recent versions of this library. * Also log redirects when saving responses and debugging (#398). Weboob 0.7 (2011-04-01) General * New backend: Dailymotion (CapVideo). * New backend: Ecrans (CapMessages). * Now compatible with Windows (is it really a good news? :)). Console applications * Do not print escape characters (like bold) when using -O. Backend: AuM * Fix: parsing of smileys (#532). * Fix: new page (#535). Backend: BP * Fix: parsing of accounts is case there are missing sections. Backend: CreditMutuel * Compatibility with other agencies (#550). Backend: DLFP * Ability to plusse/moinse contents. * Ability to tag a content. * Support the board. * Support comments signatures. * Support wiki, forums, polls, tracker. * Now implements CapContent to edit wiki. Backend: FourChan * All messages are children of thread. Backend: Inrocks * Add support of 'InrocksTV' pages. Backend: IpInfoDB * Fix: crash when lat/long are empty (#585). Backend: LCL * Fix login, and implement CapBank methods. Backend: Newsfeed * Fix: correctly set the UNREAD flag. Backend: Orange * Fix: posting messages (#614). Backend: Redmine * Fix: support project names with '-' in. Backend: Transilien * Fix: don't keep previous results. * Change user-agent of browser and use HTTPS (#580,#583). Backend: Youjizz * Fix: crash when duration is not available. Backend: Youtube * Now correctly handle gdata errors. * Fix: get video from page URL. * Fix: get file URL of video. Application: boobmsg * New command 'export_all'. * New xhtml formatter. Application: masstransit * Works without conic or hildon (#562). * Add banner on load. Application: QWebContentEdit * Better errors management. Application: videoob * New command 'download'. Application: webcontentedit * Set a default editor (#557). Core * Split of ReplApplication to create ConsoleApplication. * Remove the 'refresh' feature from Browser (to prevent unwanted behaviors like freezes). * Browser has upgraded Firefox from 3.0.3 to 3.6.13. * Browser doesn't retry on 404 errors. * Script to generate Sphinx API documentation. Weboob 0.6 (2011-03-01) General * New backend: Inrocks (CapMessages). * New backend: LeFigaro (CapMessages). * New backend: MediaWiki (CapContent). * New backend: Minutes20 (CapMessages). * New application: QWebContentEdit (CapContent). Backend: AuM * Fix: website changed, mails.php renamed to mail.php. Backend: BNP * Fix: the captcha has been changed on their website (#461). Backend: CrAgr * Fix: the history operation works when every non-ASCII character on the bank website is replaced with two interrogation marks. * Fix: calculation of cents. Backend: CreditMutuel * Fix: negative value correctly considered by 'list' Backend: DLFP * Fix: website changed to a new version (#503). Backend: IsoHunt * Fix: website doesn't provide seed information on info page anymore (#529). Backend: KickAss * Fix: size span detection improved (fixes #497). Backend: OuiFM * Fix: website changed to get current song. Backend: PirateBay * Fix: parsing to find leecher and seeders (#458). Backend: Redmine * Adds support for previewing modifications on redmine's wiki pages. * Fix: when login/password is invalid. Backend: Yahoo * Fix: didn't load with python < 2.6 Application: boobmsg * New command 'export_thread'. * New parameter '-t' on the 'post' command to set a title. Application: monboob * Fix: catch errors when trying to bind the SMTP server (#485). Application: QBoobMsg * Fix: reload of backends list. Application: webcontentedit * New command 'log' to display all revisions of a page. Application: weboob-config * The 'edit' command can be used to interactively edit one backend, instead of opening the './weboob/backends' file with a text editor. * Checks on given backends for 'backends ' subcommands. Application: weboob-config-qt * Fix: crash when trying to add an already existing backend. Repl Applications * New parameter '-O' to set an output file. Core: * Fix: Browser.location() crashes when uri is empty (#488). * Fix: catch every exceptions when loading a backend (not only ImportError). Weboob 0.5 (2011-01-08) General * New backend: MeteoFrance (CapWeather). * New backend: Orange (CapMessages, CapMessagesPost). * A new tool, make_man.py, is used to generate manpages. Backend: Bouygues * Fix: logging issues (#444). Backend: CrAgr * Fix: do not keep accounts list in cache. * Fix: extraction of amounts >999€ and negative amounts. Backend: KickAss * Fix: website changes (#457). * Fix: size parsing. Backend: PirateBay * Fix: size parsing. Backend: Yahoo * The 'search' command is implemented. Backend: Youtube * Fix: website changes (#450). Application: boobmsg * The 'post' command can take message from command-line instead of stdin. * Fix: encoding problems. Application: videoob * Fix: playing videos in some cases (#446). Application: wetboobs * Rewrite of application to use formatters. Weboob 0.4 (2010-12-01) General * New backend: Bouygues (CapMessagesPost). * New backend: CanalPlus (CapVideo). * New backend: CreditMutuel (CapBank). * New backend: IPInfoDB (CapGeolocIp). * New backend: IsoHunt (CapTorrent). * New backend: KickAss (CapTorrent). * New backend: PirateBay (CapTorrent). * New backend: SFR (CapMessagesPost). * New backend: SocieteGenerale (CapBank). * New application: boobmsg (CapMessages, CapMessagesPost). Console applications * New command 'inspect' to open a graphical webkit browser with the current page (to help debugging). If it is not available, the page source is displayed on stdout. * In question prompts, display a bullet list when there are too many choices. * The --save-responses (-a) parameter stores now pages in a directory and save here a debug log and a list of visited URLs associated to the files. * Fix unicode issues (#436). Backend: AuM * Do not send baskets messages from left sluts. * Anti-spam is updated. * Raise an error when user is banned. * New optimization PRIORITY_CONNECTION to create fake godchilds to allow user access to website between 18h and 1h. (#319) * New optimization QUERIES_QUEUE to enqueue queries in case no charms are available. * New backend parameter 'baskets' to enable getting baskets messages. * In profiles, look for hidden photos. Backend: BNPorc * Fix: display of coming operations. * Fix: check if the password rotation has been succeed. Backend: BP * Backend has been rewritten to use BaseBrowser (#422). * Fix: backend initialization. * Fix: parsing of accounts. * Fix: handle of transfers errors. Backend: CrAgr * Support of history operations. * Support monay transfers. * Choose the agency website in a list instead of giving URL. Backend: DLFP * Store datetime from newsfeed in the Thread object (#419). * Now the session is closed at deinit. * Fix: when posting a comment, send right parameters to immediately display it. Backend: GeolocIP * Use the website www.geolocip.com instead of www.geolocalise-ip.com. It prevents authentication and limits. Backend: INA * Fix: parsing date, duration and title. Application: boobank-munin * Save cache in .weboob/munin/ and handle errors. * New option 'boobank_cumulate' to display graph as areas instead of plots. Application: havesex * The optimizations management has been rewritten. * New command 'query' to send a query to a contact (like a charm or a poke). * Fix: do not exist after displaying a profile in interactive mode. Application: monboob * New option to pipe mails to an external process instead of sending it to a SMTP server. * Fix: when domain in In-Reply-To contains a '@'. * Fix: parsing incoming mails when no charset is supplied. * Fix: unicode issues. Application: QHaveSex * Display URL of contacts. * Contacts list is now sorted. * Have a photos caroussel on profile page. Application: weboob-config * New command 'confirm'. It takes an email in stdin and call corresponding backend to go on the confirm address. It is useful to automatically confirm account register. Application: weboorrents * Ability to complete paths. Core * The 'repeat' scheduler has been rewritten. * Ability to cancel a scheduled task. * Fix parsing of path-like in INI config. * Conditions are now treated by BackendsCall instead by formatters (#372). * Backends name can now contain only letters and digits. * Add a tool to generate manpages. Weboob 0.3 (2010-11-01) General * New backend: LCL (CapBank) -- unfinished. * New backend: OuiFM (CapRadio). * New backend: Newsfeed (CapMessages). * New backend: Redmine (CapContent). * New application: radioob (CapRadio). * New application: webcontentedit (CapContent). * New application: boobank-munin is a plugin for munin (CapBank). * New tests. * New global parameter --logging-file to log into a file instead of stdout. * Logging is now colorized when printed on stdout (depending on level). Console Applications * Formatters management have been rewritten. Now each command can set its own default formatter, and user can redefine them. * If output exceed the height of term, ask user to press a key for each page. * Do not display columns when all of these values are NotLoaded or NotAvailable. * Add a CSV formatter (#350). * Command 'backends register' to register a new account (#294). * Can use '$full' and '$direct' selectors. Backend: Arte * Fix: fall-back when the wanted quality is not available. Backend: AuM * New anti-spam feature to detect, block and report spambots (#313). * Implements the capability CapAccount to register new accounts (#389). * In profile walker, when reloading sluts list from search page, do not keep the previous queue to prevent visiting sluts who have been added a long time before and who are now disconnected. * Contact status is now 'connected'/'not connected since ... hours'. * Fix: do not crash if contact list is empty. Backend: BNPorc * If password is expired, switch with the value of the new 'rotating_password' backend setting. * Support transfers (#416). * Fix: don't crash when accounts don't support 'coming' (#401). * Fix: when website is not available, raise BrowserUnavailable instead of BrowserIncorrectPassword. Backend: DLFP * Replace 'cite>' and 'tt>' to 'i>' in read messages. Application: boobank * Use 'table' as default formatter. * Output for the 'list' command is more readable (#410). * When a transfer is made, display all information instead of ID. * Fix: do not load backends twice (#405). Application: QBoobMsg * Support threads display and answers. * Support sending answers (plaintext or HTML). * Unread messages are in yellow in list, and set message as read when one is selected. * Fix: correctly reload when backends have been changed. Application: QHaveSex * Ability to display a profile from URL. Application: QVideoob * Fix: search on youtube was unlimited, so QVideoob freezed. Application: traveloob * Use 'table' as default formatter. Application: videoob * Output for the 'search' command is more readable. Application: weboob-config * Add a 'register' command (#294). Application: weboob-config-qt * Add a 'Register' button to register a new account on a backend (#390). Application: weboorrents * Output for the 'search' and 'info' commands is more readable. Core * Weboob.load_backends() takes a new 'errors' argument. * CapBaseObject has a new method 'add_field' to add a field property which forces a specific type and is set to NotLoaded by default. * Browser.readurl() can take same arguments than Browser.openurl(). * If a page is not recognized by the Browser instance, save response even if -a is not supplied. * Introduce 'Value' classes to replace BaseBackend.ConfigField and is used by ReplApplication.ask(). * Use several loggers for parts of weboob. * Fix: crash when HTTP server returns shit (#406). Weboob 0.2 (2010-10-01) * Backend 'yweather' renamed to 'yahoo'. * New capability: ICapGeolocIp to geolocalise IP addresses. * New backend Arte: for the arte.tv french TV website. * New backend GeolocIp: for the geolocalise-ip.com website. * New backend BP: for the “Banque Postale” french bank. * AuM backend: check when the account is blocked. * AuM backend: ability to register new account. * BNPorc backend: support history. * CrAgr backend fix: support of another version of Credit Agricole. * CrAgr backend fix: check if website is down. * DLFP backend: display comments link in signature. * DLFP backend fix: parsing of comments when templeet sucks. * Youtube backend: support infinite searches. * Youtube backend fix: parsing of URLs (#388). * New application geolooc: a console application to interact with ICapGeolocIp backends. * New application weboob-cli: a console application to interact with every backends. * boobank application: new command 'history'. * boobank application: new command 'transfer'. * QHaveSex application: know if a message has been read or not. * videoob application: new command 'play'. * Console applications: can be interactive (repl) when run without any command (#353). * Console applications: if no backends are loaded at startup, ask user to add them. * Console applications: -s '*' fills objects. * Qt applications: display configuration window if no backends are loaded at startup. * Core: ability to fill objects already fetched to complete them. * Core: ICapMessages has been rewritten to be more efficient. * Core: can enable or disable backends. * Core: a test architecture has been written. * Core: all loaded backends now need to be configured (#368). * Core: new pargument --save-responses to debug HTML pages (#274). * Core fix: handle read URL data failure. * Core fix: Client is merged into mechanize >= 0.2 (#362). Weboob 0.1 (2010-08-03) * First public release. * The core system can load/unload backends, supports configured backends, can do asynchronous calls to specific backends. * Capabilities are abstractions to be overloaded by backends: - ICapBank - ICapChat - ICapContact - ICapDating - ICapMessages - ICapMessagesReply - ICapTorrent - ICapTravel - ICapVideo - ICapWeather * Backends interact with websites: - AuM - BNPorc - CanalTP - CrAgr - DLFP - FourChan - Gazelle - INA - Transilien - YouJizz - YouPorn - YouTube - YWeather * Applications interact with backends: Console - boobank - chatoob - havesex - traveloob - videoob - weboob-config - weboorrents - wetboobs Nokia N900 Phones - masstransit Daemon - monboob Qt - QBoobMsg - QHaveSex - QVideoob - weboob-config-qt Web - videoob-web-server weboob-1.2/INSTALL000066400000000000000000000040651303450110500136570ustar00rootroot00000000000000Weboob installation =================== Using the packages provided by your distribution is recommended. See http://weboob.org/install for a list of available packages. Since there are many dependencies, when you install from sources, you have to handle them by hand, according to your distribution. The requirements are provided in ``setup.py``, except for: * gpgv (for secure updates). If not packaged alone, it should be in ``gnupg`` or ``gpg``. * PyQt5 (python-pyqt5, pyqt5-dev-tools, python-pyqt5.qtmultimedia) for graphical applications. * For more performance, ensure you have ``libyaml`` and ``simplejson`` installed. Some modules may have more dependencies. All installation procedures allow you to chose whether you want graphical applications. Add ``--no-qt --no-xdg`` to disable them; ``--qt --xdg`` to enable them. After a package or system installation, you should run ``weboob-config update`` as your login user. User installation ----------------- There is a way to install weboob locally without messing with your system. Run ``./tools/local_install.sh`` as your local user. :: $ ./tools/local_install.sh ~/bin The scripts are copied to ``~/bin``. System installation (discouraged) --------------------------------- The install mode copies files to the Python system-wide packages directory (for example ``/usr/lib/python2.5/site-packages`` for Python 2.5, or ``/usr/local/lib/python2.6/dist-packages`` for Python 2.6). :: # ./setup.py install Scripts are copied to ``/usr/bin``. Development mode ---------------- This does not actually install anything, but lets you run Weboob from the source code, while also using the modules from that source. This is only recommended if using the git source and not a release. :: $ ./tools/local_run.sh APPLICATION COMMANDS For example, instead of running ``videoob -b youtube search plop``, you would run:: $ ./tools/local_run.sh videoob -b youtube search plop Bash completion --------------- To enable bash completion, just source the ``tools/weboob_bash_completion`` file from your ``~/.bashrc`` file. weboob-1.2/MANIFEST.in000066400000000000000000000004001303450110500143510ustar00rootroot00000000000000include COPYING include INSTALL include README include AUTHORS include ChangeLog include desktop/* include icons/* include man/* recursive-include contrib * recursive-include weboob/applications Makefile recursive-include weboob/tools/application Makefile weboob-1.2/MANIFEST.in.modules000066400000000000000000000000771303450110500160320ustar00rootroot00000000000000recursive-include modules *.py recursive-include modules *.png weboob-1.2/README000066400000000000000000000022501303450110500135000ustar00rootroot00000000000000Weboob is a project which provides a core library, modules and applications. Overview -------- The core library defines capabilities: features common to various websites. For example, http://www.youtube.com/ and http://www.dailymotion.com/ both provide videos; Weboob defines the "CapVideo" capability for them. Each module interfaces with a website and implements one or many of these capabilities. Modules can be configured (becoming a "backend"), which means that the end-user can provide personal information to access the underlying website, like a login and password. Applications allow the end-user to work with many modules in parallel, in a multi-threaded way. For example, one could search a video on many websites at once. Applications are toolkit-agnostic. They can use GTK+, Qt or be text-only. The latter can be used either in an interactive way or in pipes. The core library provides base classes which help developers write modules and applications. Weboob is written in Python and is distributed under the AGPLv3+ license. For more information, please go to the official website at http://weboob.org/ Installation is described on the website or in the INSTALL file. weboob-1.2/build.mk000066400000000000000000000006411303450110500142520ustar00rootroot00000000000000core := weboob/tools/application/qt5 applications := qboobmsg qhavedate qwebcontentedit qflatboob qcineoob qcookboob qhandjoob qbooblyrics ifeq ($(WIN32),) applications += qvideoob endif directories := $(core) $(applications:%=weboob/applications/%/ui) .PHONY: clean all $(directories) all: target := all all: $(directories) clean: target := clean clean: $(directories) $(directories): $(MAKE) -C $@ $(target) weboob-1.2/contrib/000077500000000000000000000000001303450110500142615ustar00rootroot00000000000000weboob-1.2/contrib/anonymiser.sh000077500000000000000000000020461303450110500170060ustar00rootroot00000000000000#!/bin/bash # A special file with the list of words to replace. The format is one word per line, with a tabulation as separation # Example: # name offuscatedname # phonenumber 111111 anonymise_list="Anonymiser" # Take the folder to anonymise as argument, and check if it is a folder if [ $# -gt 0 ] && [ -d $1 ] then dossier=$1 else echo "Usage: $0 FOLDER" echo "For example : $0 /tmp/weboob_session_NLSIls/freemobile/" exit 1 fi if [ ! -f $anonymise_list ] then echo "Please create the $anonymise_list file (see documentation)" exit 1 fi # remove potentials old files find $dossier -name \*_anonymised -delete rm -rf $dossier/Anonyme for file_to_anonymise in `find $dossier -type f` do file=$file_to_anonymise"_anonymised" cp $file_to_anonymise $file cat $anonymise_list | tr '\t' '_' | while read line do to_replace=$(echo "$line"|cut -d_ -f1) replace_with=$(echo "$line"|cut -d_ -f2) sed -i "s%$to_replace%$replace_with%Ig" $file done done mkdir $dossier/Anonyme find $dossier -name \*_anonymised -exec mv \{\} $dossier/Anonyme \; weboob-1.2/contrib/boobank_indicator/000077500000000000000000000000001303450110500177305ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/CHANGELOG.md000066400000000000000000000002211303450110500215340ustar00rootroot00000000000000This file will only list released and supported versions, usually skipping over very minor updates. 0.0.1 ===== * Mar 26, 2015 * First release weboob-1.2/contrib/boobank_indicator/MANIFEST.in000066400000000000000000000004051303450110500214650ustar00rootroot00000000000000include boobank-indicator/data/indicator-boobank.png include boobank-indicator/data/green_light.png include boobank-indicator/data/red_light.png include boobank-indicator/data/red_light.png include boobank-indicator/data/personal-loan.png exclude screenshot.pngweboob-1.2/contrib/boobank_indicator/README.md000066400000000000000000000033531303450110500212130ustar00rootroot00000000000000Weboob ========== Weboob is a project which provides a core library, modules and applications such as boobank. Overview -------- The core library defines capabilities: features common to various websites. Each module interfaces with a website and implements one or many of these capabilities. Modules can be configured (becoming a "backend"), which means that the end-user can provide personal information to access the underlying website, like a login and password. Applications allow the end-user to work with many modules in parallel, in a multi-threaded way. The core library provides base classes which help developers write modules and applications. Weboob is written in Python and is distributed under the AGPLv3+ license. For more information, please go to the official website at http://weboob.org/ ##Installation boobank_indicator is distributed as a python package. Do the following to install: ``` sh sudo pip install boobank_indicator OR sudo easy_install boobank_indicator OR #Download Source and cd to it sudo python setup.py install ``` After that, you can run `boobank_indicator` from anywhere and it will run. You can now add it to your OS dependent session autostart method. In Ubuntu, you can access it via: 1. System > Preferences > Sessions (OR) 2. System > Preferences > Startup Applications depending on your Ubuntu Version. Or put it in `~/.config/openbox/autostart` ###Dependencies - weboob >= 1.0 - gir1.2-appindicator3 >= 0.1 - gir1.2-notify >= 0.7 ###Troubleshooting If the app indicator fails to show in Ubuntu versions, consider installing python-appindicator with `sudo apt-get install python-appindicator` weboob gir1.2-appindicator3 gir1.2-notify` ##Author Information - Bezleputh () weboob-1.2/contrib/boobank_indicator/__init__.py000066400000000000000000000000001303450110500220270ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/boobank_indicator/000077500000000000000000000000001303450110500233775ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/boobank_indicator/__init__.py000066400000000000000000000000001303450110500254760ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/boobank_indicator/boobank_indicator.py000066400000000000000000000145431303450110500274270ustar00rootroot00000000000000#!/usr/bin/env python import os import logging from threading import Thread from signal import signal, SIGINT, SIG_DFL from pkg_resources import resource_filename from gi.repository import Gtk, GObject, Notify from gi.repository import AppIndicator3 as appindicator from weboob.core import Weboob, CallErrors from weboob.capabilities.bank import CapBank, Account from weboob.capabilities import UserError from weboob.tools.application.base import MoreResultsAvailable from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword, BrowserForbidden, BrowserSSLError PING_FREQUENCY = 3600 # seconds APPINDICATOR_ID = "boobank_indicator" PATH = os.path.realpath(__file__) def create_image_menu_item(label, image): item = Gtk.ImageMenuItem() img = Gtk.Image() img.set_from_file(os.path.abspath(resource_filename('boobank_indicator.data', image))) item.set_image(img) item.set_label(label) item.set_always_show_image(True) return item class BoobankTransactionsChecker(Thread): def __init__(self, weboob, menu, account): Thread.__init__(self) self.weboob = weboob self.menu = menu self.account = account def run(self): account_history_menu = Gtk.Menu() for tr in self.weboob.do('iter_history', self.account, backends=self.account.backend): label = u'%s - %s: %s%s' % (tr.date, tr.label, tr.amount, self.account.currency_text) image = "green_light.png" if tr.amount > 0 else "red_light.png" transaction_item = create_image_menu_item(label, image) account_history_menu.append(transaction_item) transaction_item.show() self.menu.set_submenu(account_history_menu) class BoobankChecker(): def __init__(self): self.ind = appindicator.Indicator.new(APPINDICATOR_ID, os.path.abspath(resource_filename('boobank_indicator.data', 'indicator-boobank.png')), appindicator.IndicatorCategory.APPLICATION_STATUS) self.menu = Gtk.Menu() self.ind.set_menu(self.menu) logging.basicConfig() if 'weboob_path' in os.environ: self.weboob = Weboob(os.environ['weboob_path']) else: self.weboob = Weboob() self.weboob.load_backends(CapBank) def clean_menu(self, menu): for i in menu.get_children(): submenu = i.get_submenu() if submenu: self.clean_menu(i) menu.remove(i) def check_boobank(self): self.ind.set_status(appindicator.IndicatorStatus.ACTIVE) self.clean_menu(self.menu) total = 0 currency = '' threads = [] try: for account in self.weboob.do('iter_accounts'): balance = account.balance if account.coming: balance += account.coming if account.type != Account.TYPE_LOAN: total += balance image = "green_light.png" if balance > 0 else "red_light.png" else: image = "personal-loan.png" currency = account.currency_text label = "%s: %s%s" % (account.label, balance, account.currency_text) account_item = create_image_menu_item(label, image) thread = BoobankTransactionsChecker(self.weboob, account_item, account) thread.start() threads.append(thread) except CallErrors as errors: self.bcall_errors_handler(errors) for thread in threads: thread.join() for thread in threads: self.menu.append(thread.menu) thread.menu.show() if len(self.menu.get_children()) == 0: Notify.Notification.new('Boobank', 'No Bank account found\n Please configure one by running boobank', 'notification-message-im').show() sep = Gtk.SeparatorMenuItem() self.menu.append(sep) sep.show() total_item = Gtk.MenuItem("%s: %s%s" % ("Total", total, currency)) self.menu.append(total_item) total_item.show() sep = Gtk.SeparatorMenuItem() self.menu.append(sep) sep.show() btnQuit = Gtk.ImageMenuItem() image = Gtk.Image() image.set_from_stock(Gtk.STOCK_QUIT, Gtk.IconSize.BUTTON) btnQuit.set_image(image) btnQuit.set_label('Quit') btnQuit.set_always_show_image(True) btnQuit.connect("activate", self.quit) self.menu.append(btnQuit) btnQuit.show() def quit(self, widget): Gtk.main_quit() def bcall_errors_handler(self, errors): """ Handler for the CallErrors exception. """ self.ind.set_status(appindicator.IndicatorStatus.ATTENTION) for backend, error, backtrace in errors.errors: notify = True if isinstance(error, BrowserIncorrectPassword): msg = 'invalid login/password.' elif isinstance(error, BrowserSSLError): msg = '/!\ SERVER CERTIFICATE IS INVALID /!\\' elif isinstance(error, BrowserForbidden): msg = unicode(error) or 'Forbidden' elif isinstance(error, BrowserUnavailable): msg = unicode(error) if not msg: msg = 'website is unavailable.' elif isinstance(error, NotImplementedError): notify = False elif isinstance(error, UserError): msg = unicode(error) elif isinstance(error, MoreResultsAvailable): notify = False else: msg = unicode(error) if notify: Notify.Notification.new('Error Boobank: %s' % backend.name, msg, 'notification-message-im').show() def main(self): self.check_boobank() GObject.timeout_add(PING_FREQUENCY * 1000, self.check_boobank) Gtk.main() def main(): signal(SIGINT, SIG_DFL) GObject.threads_init() Notify.init('boobank_indicator') BoobankChecker().main() if __name__ == "__main__": main() weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/000077500000000000000000000000001303450110500243105ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/__init__.py000066400000000000000000000000001303450110500264070ustar00rootroot00000000000000weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/green_light.png000066400000000000000000000022151303450110500273050ustar00rootroot00000000000000PNG  IHDRw=tEXtSoftwareAdobe ImageReadyqe</IDATxڴU]hU=333?1%Ң> bJQ{P)C%Ub҅(" 4$b}Ň4)6&ٝ?~3nl6/\g{ιoVRBQtgj$uR Z PTJT]w"& 'eB39f3_" pO<^E 5(pV#a +imA~ `A -J'A@k79"0DEʌmE;UTG7_.ԕogkncuZ*?^Ӣ:)'T4UMi=[}3F/gY{}qb)t۲Jd8$~!ɥvs/- BYh#Ff܋  #`  $<0'[P JhhHaW"<-$t$mX] EERσXhŝHTjT CNQf?^w[_۱聋) H4wtƭ 6&Z[lǔpgKI&'rw{FkoxK)}YakZ \x7qri92;P3ϾÃOhkZ_;fzdK ~xvM4EC滹̘7Vb:Xtϰ慃3eYysN$wpM'/W嫶z199$۟\@=96'Kc Iv`)I@ 4x?([X{ۓ !ɜ3՘$_s!}^0~6oاgh+AB5Ɣa$\Z=G۟ w<Һ?7o<x 0wv(IENDB`weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/indicator-boobank.png000066400000000000000000000031401303450110500304010ustar00rootroot00000000000000PNG  IHDRw=tEXtSoftwareAdobe ImageReadyqe<iTXtXML:com.adobe.xmp etaFIDATxVNPnm->QV"nn]&erT' u`sɵPē{{ l6i4ZnI/ cuHa@ gHYDTI@ezxm:If䌨Erv/b(F E}> 2Q_RXnh,3 حf#Q$z1K 4tq"h?-sVQ<P ]ZX&;Oƃ`]24꽺"ǑZ$˽a'8Ai0&Rl"+v` p`&tߎ{Z #UdJ:hbͯ8]Wiˎ2B0?/ѥABr5g- lmL![8Z;<:LS-㌛[MPWϩTn6w pAz=\MTYd/rTv]v8[I`&LM.6~Gm۾f9&NZ><<!.tsIENDB`weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/personal-loan.png000077500000000000000000000031631303450110500275760ustar00rootroot00000000000000PNG  IHDRw=tEXtSoftwareAdobe ImageReadyqe<iTXtXML:com.adobe.xmp VߤYIDATxV=nAfm:'FgyL@*LEJ pOM(9o0PDʓ>͊yo 9l6C,w@x$| 5@X' <JQcyg8qazMvp8;NrH.G4Isiv ';F.rۓ†d+h^:T}!Fl6/@ 3W`%=oZOHi%o/V;6fY=P9=8b1T*<7~[dkKϿG_m#>miSdIENDB`weboob-1.2/contrib/boobank_indicator/boobank_indicator/data/red_light.png000066400000000000000000000017061303450110500267630ustar00rootroot00000000000000PNG  IHDRw=tEXtSoftwareAdobe ImageReadyqe<hIDATxڴUKWnU1-ED0.˪C PAA&@wɠUp e"u1t}Lwӝ@rtݺu=sX(ր0!0Gp0WSyoA6VNjh5"~ FۦaZEd{ ɇ@NJܳhl.r>=xզvXu lSTE]C~$֑/Oﯳ\J#&*OSӏgb~{yyyZwă- GDg@ = oѸƎR֬ӴtH\t,ɔ Gp\ZWZ@5HiFۻ@/4YS5ߟk?%0K]v +5XpWrg=AVUرNAz\TiWUh*c9B~ Z Ά=I cp'f  gy/HxIENDB`weboob-1.2/contrib/boobank_indicator/screenshot.png000066400000000000000000002206021303450110500226150ustar00rootroot00000000000000PNG  IHDR9gAMA a pHYs(JtEXtSoftwarepaint.net 4.0.5e2eIDATx^XT׾>swrNI<9Qc;8F3nTEAEXbŒDETD ѨQ{zRa֞̚gtf^kwwVCb1@ b1@ 0a࠯__? 9|rzE믿n޼b1@ hz\\\V\illL׿~!dbbܬɸ;w1cb/ NJBD '+޼y_~a'Ovp=H%_ ʕ+gd.oFt'xDs(03( 2 #;NIJNLJHLa$$'%$:{b`1 ~[[[333`+O`0`ѢE0mڴֆR'3>N=zh֧3 _@F?FY7PETCqj 1@ F`@?IO?l#yٲe`@GZr(xr9'.c,69fQP*o(0 <&6I*EXWaRǂã_ 6b1M 1|0 /J7~8a_̥M+R:hXoo}۠_=xA`GZ]TT(X===- wW)~g2}n@Tt}Ϟv52kAoݸyfOo޾|.m!Ao<"D QX~`qI}}<|)|9yd`ٳgY棏>޼q!Hq8N,@7GEGiEV~w7Kකj.;_n~L!OA?*׮_~[7nt-BݖUKǹTgɯn?BfMCŢ$Ӈ+ʁ]B Āz =~`I?Xt, R `h7z{ם̬̬Lux=EOr9rd~`W3",3]LfgWW ٓAʁS-? VABDe+?۷ޝ{Lxo(ψ?gv41xw#; e"\w(dÃ߾6S5s_6UJ;ĀJ~0n=Ppߤo><hsKn@(Tr!oedd|JzoٲE%F6nV'`S޼ܧϟ]]]wvwtI?#  8s;hwo=wω䟰yV01~ȽBBg_~?DB2'fu_NoXBҀR?3OHUя~G CcNd٢>pL>O?_O`?CW) _>~R?XAPaή]!lbq__gWGVvoOiROKOijnttYW:t}? I53'Ϝ<_t#`}H轟 A{{n rGx?Zu|}@P=(+fg/ !t\R'iAPnE Z?TBv2>^LÜ9s~7߸zzAaAAaa*kDw^)ub90{ho70kj_Gzk`+E,kkoihk6e.?K//ON\agO>?> Yee'KoQ2A?bvMS}vb]}B>zJljy M5;")H-'u?Ùt0p~}8 w޼}]L[y!*х "8&>/-+3?Lm-u@,okk,@TU׽KAp'O=8#G}K|]t!Έ?SbӞ^'ȿ01!hT4:+b@]}Ot7We?> v^ fLz f͚~7wXb uckaK`߂Suvut~[ ?pCiEqEUX+048_? voՋ?|2| л.}*#oGFDT}XlI^A{nU}U`@SkcM]Uu]5UegP񺴼J6^WR9n); -~~x[Gk׭Y@*(翭 NKvg=z SLBm{˔g1?qAc͛V-?_o㮫[u +1@ rEYba`=8o`TRcEFχ]cbb*K`Iu]/Չ-@ '?re" U)w).`nH0H9˗p/ptr [ت;&=%?uNǧSR_}˩w?kRtb1 >`??j^`M8&;(G*2\{@(/\_WWW~>wR'? >Zv-X'x մo޾F#._x%^[=~x7FD 6b1M 3:IρG5[Q9_\ձ?>ﻜ{8.uR"0.cs  W ;Ȏ SM~'J֕u+~L;b`1&'Myp/Pt%I`<9 Țk} |s!g`V_!_ZU _P$pNYgMJqY|ҏ/_x~ p{?0D'j?q| ҒӜ$P2dصSLZt)X)',X[[O:h ۷opm۶MK^x&TzDwG$(b10V򃭀b30|% hi!n v-0A /$%1@ F7`l`ߗ_~d `I KKK:|5=y[#.!#o+u(b1@ W#t!Ār@k Lr}b|Bx3tKZ+"~,A@ h'`jX#-'qC+Tg}!hTIQ ģb0]=0`mOeXYr.FsL{hniġ{hO}]L/:dvL,9+Mh7nD?0m% G௶&k3c`@4Xdn<_yKG{D?_\z1b@ ڵ8i`ZɀHE@="(GAo}/ ,H m.'mH2*]?U?R^{٦8pԊiRuD  |fguw5B8BM4Z0ucTd}mO]a 1YO/"q߆iRGh @˲%Ggv渜cGceA/'9m.%oc3٠8jrϬ!G4cL_2ТBIa縜~[$O A^bc?ucѬuyӗGX]'Leϗ44_zwY@"~Sc*l :_q hg`7RQKNu^Z. SJ^2;p&jBŁN>[׸I6+^EI+d/-E苆-+nuYeoOk )}UTHQ{E }/{zO>Ӣ{>]D-<3HZ1kʚkrg̴cL\d6X2/R֌}t/ k T?WkI$3=o8ܘŽ jq-]?G7yTw6C}UAwj͉8BuLnVVg:Gq\Pta;GT 2{8Ǐ#  @(%?iKf9vpS#1?JqZrp}32 gXeٟ%ONl7i#{|پ<`-}e?)ϗ}2aW]k ڕ2jcs^<[b˦|ŏĨw_D8o"l.JޫkնƋv2dqQt)/"4U=20_r)ޢV-ӧ[MJ!uY-{* $_K⽥RXW%?7kKKdr] )u.2֐ `}_Rm_XQ/mϊtxa׸ 7 ~ pn͛5K?V]: e`נT т_M 1_ Y-b?6O$uIT$1'L5J-MNF*fnxi1ݠ'+\ܿ x8r}oˮQnMbZZX), =YMd`i`C*}P|` 7Y/u Կbf*J ԲarYE3|Mmc8\5E'ĿqStbMXh?8;@RcTk둟kvK-K_\[zcn?IPr嶐b:J8K]uq$YD,7"e=L7xl/~*@+i?@.#WCPȈb`L2f, 'Ѣ'ω Ң&ώ 쮥1/^Ù+o)ObXL' sf/RF6O1s/0;'8/[}ZGb?=( 8 ֿޒSX7*/ _`_n-KQNVcƢB aC0ot)̟I3#&'M|i|)/>e|w*IҿoyD2a^5 "SnFrયD3< W7Hf#wI8>hp+88_]5 [ ,@b1@ rYEexIK&D}0Ǔ^}a_&ݲ4}\رM~;մΧa:ʟTmV qQIl~y)DJV,;:0gu֝(M-[.8z һkx8m5_w\$znknIG WC8`:1@ 00|&>ޗctT2߲kWzkDn/#D R%AVx>^3x?p4lS]=W _>#U\onySNjԥd/{ґ[ZnZ۾@#f@"Gb1õc&L_k\kLmǺR@k9hC)2vt7b120|_uf'HcrwR4: ^ZOm{\,\%=_<_S]b`3Zak$(tZU#,AZ|~d:ueң@ `ؗQ)l#&Uqg/KG6E܃K^xY&j7`ɿ:2NFF;qUV].w,=1P+ $)l(U00_v-=erOll7KVC|.Y:mO/n\6A)=E܍u4J7>\\J0ym)Gٚ,sqih/̥$´>9wWHXRXϹċĖ^%it[5Ģm/Ior;7"çX*Dl P Le)KYKBGɳ;}AաyU&݀eɿ:2N$߮,v\| opw|l<#f1^  Im6 y4h38$$ЙF_vDˠmyBE,~vabNOr"6Ia;//znYE[ Y<#B1^f}.B(x] @-iQ *!^ߺoP!{V!)Er$Y~sؾt'ߐ𴬴谌aIm2wƍ殮.))^ 'u7=neaR뮊w7^i(Ndt L"'/yƅ(X/}Y!oU6J-'T F#20jػΟ>JY5o嬾Uf~]UGj)11[.2bf /oij4h-G!&yn&EP0fyp~[Y+N'ix͗iy {-i[BqB&:Ɖ y>KuS2k}xڜn]a^h4Vbwȇ@]}xAe[ zL{w1+7#4gM3PSWsرa[Aߝk>A|sљ\H#P\L̂M\N4݁[^\s_ KHNKzzV a_(*)%-UYW#l˭Jh>e:97^wzV7u?"#~kfzRR ee]b>3q<vNhCc@p[IZK,ynje|6;в=E@umX5`K{-ƻ~`O>[h16 WckkmG@C.e1#Y}fn0mg:5[sdt6xu\|~!q]iF25?uPVfxi7v9 x.x1[<PД_֛5ƢY4y:4r;}WO,=4}/u4.o2X?BoX#\3p5j'?j%10H4"mƝVfOY7sML+2&L :smgٝ$J?uWgEz|߱n˫ /V|nA?w5\ti>?&d9 [JrsEHrtMP4kO=yr?ܥ1 f`ݙ9 3'gϰ$`5cEߗeb<:mMޤyMz(A?r8O7Vqo OI!_vيpo#v7Fw&\6еznE+=!\#S} 浦GPb@ a8uUtiD}Hft\ĝ7Br:x[|4mA)^`xw_%ܚv;.PxHc"\#YA`NoH$(lb`30^jI4&Ib'M5Nj8E?59,e1GM$;}ۀL[OU8^%Vm 0o|VC%tM1"(󍷇IAQxH#\#YS;;UXFRU]K.:#z6b1B@9 QtJ@7E'vܔ쒌C(2B6if<X7rA!ׄ ܞq"9 fɝh`C3͐,F^4WE>k߿撋BB X5Sf%N==yNd5yvd$< _Č+C3< v'-ے}_ܯ! ֎+i[_uB'-mRTҀOnk g߀YWƒfkgCE< gF / t*܎}m16oCh%?4 l_t~AQ@Qe te'9<.9=ؒ%Pρ2j<'WHGS7Eָ݂ (am}H("20 MM<k~oK6ϳU3.x.~\@/CS9#$?.8Xos9?SQ)sWW1L!t@,Ln$#RNrQC:'"R|@t 4Rh_Ü ťŚۛ],ܕȀ2f6}63U-ȫ}񢅏W.y%tB UdQ{R?(yO~0E7?6΁RYLn$#$VBFUqvgS+ŬB?r,äaJ8H?hG7\뿩˚0M)SUV[5a&~{jٕ![ut]LCB<+)f}#V5@~w]<b\ ,%UBffp5.G7!1!+ק|Є46w@e1*"Vn>~'y}V>[$$4^!>n u ]/iīISy"yF )(*Ш/J =|g #mQ$#46NE+ɛ&wKkwWvJ6AʿhF&#CF@-pHMx3f~4N$o)!e4-wk88=r)|͸ iT^Ѩ)f,,J'ONQU$/^|QMb1 atSܝ鞳:s} LY3V}Y'f&M:ĬO5* =+o!]oKtL|еzn^>cx]tޜԏpNM oi¸ (zEO){kPxs\xSyyfb10p?=J7qN] N]~O~4rt3EH’y6;/&4o"S^SqlJ< F-M6TG^ԯ"MCw"`C٘4+qIsb&ɳ'/'9Q"6*(.;7삓ܵ|A~{GoAw֏KF1*h*QzʪՅ`]ҿu6AFֿĀñt~nMX)q 43b7|Ҕb jr)ȿZ_vي`/ ɖ|ӓeST^T{KQA y^}#xLp_99+&D}0Ǔ^}a_rsU)_%%% 66]y:"-ţX*Hx9L s!{yo96oiEɡ~nZQ|O)1 #/X ?rpxD"㓁?0A@7BOc# jTֿgm|2?e%}!K_%4 ܁SVފguQm?VKN彍[Zk$yj?5J70E #o>/WnuW۶yG;ud-{%w/#~d`忶zw׼xs#7̠D! hJUDŽ@ hohlh:[kF 5~cוi>."b1x szbQF*; <^FxYE܃K^xY&ͅvLuʛZ!2 <~!7>GB1!1չ2[ T'@H˪˄P!BICȯ/!Td1 <)"•{: (Vn_+a" _k%L99y9 o4jc}V[ul D,= 1HAWɿlE@?r"4fJ2?o[[0A{K 2'6r%0J6E_n01Ji=StXA[~U;a,'QAilM ~:F[ I+ގzYN9 Y ɿܱJl+Xwѓu! HRFhns- uuxv x%WC9胳8qCy*Gx؄3=S9ϩj/0 -\Ge2uh!WsޔY;^AQҐ#WOU. tѯrGIAy;< 6T$0+}Cp_ e;]il .kfgn8UED2BW.y%dj|!DhW Uߛ@N{R?(yO>,LQ48'}I"/D/5 %MH+JcBu\Wvx)kűu ZN4?\@Ͽ d俩˚0M)SUV[5a&~{j?cH>](ge4̾ޥ%,FsI}mQou`13)h} ᄂuB]YtA>^~{7O{>!Φni27|aQ!7 xnDvaOKOB p;2BfFW!ܾzhl SNC t1p){ !$J!02 l?.BU6BKOq/TTǥU8} HZF~;:j]N څ6/l~{ +fO筝sV E^%pa\)Ie@,#{]&+r, im&smb*Vi ]FIoTt_dgmW]lGt2)Jk ?M)U>(Łg* ku*Xo\G?ka8uUtiD}Hft\>_a96G5cgӭHB [t8٫ޭϴ[)NQ^O"Eܬcln(&/yJpQq1l-}VNx&QwЭCeP{'!'H#}kS?6If<$I 㤩 S5!&EԒ]9p{pǘfw\qQ3榓YasHr IJHAOqq7,u{p곩U*iJ܇fy5vxL pjd~1f{)1_xN:10_?ʋPYYY!BQ~3lɹFYlf9f?!ATUGB# ʩq_1!ɯ>[k*))AV٨@_64pvLq_ ]+׷hئ_*ΛZ2/R◕X%)Ug*E(3'-S]bXzcaWŇzpy'z>6wooj~3w}y [sǞ (:J?M)+˿O6%"TJ_ 1@ ں/):m[w w4p ߄:x <;Fmb>$ЙF_vDˠmyBkɏqNOr|甆6"Lc3<^\ti>?&)uK㷼B?OY)+:7]y+] umÐ}<k~oK6ϳU3.x.(me0tӅ?̃E~&1-Mm67 cuJ$9RZYtLsmfjH<)[#vȿ:~SU4V_tcuuu ijrH\ֿhs6|g?bA3X{N9~c/vU%møRߓ@r$e] #垍 :6Fq2ݦa,$58$ȿl.-S<7Bֿ2R=^ZI9u/7:8_itZ GЇG hݎ_9M>),)>/kxWt_~ONM&}I+n|}@<3%55Q\*L"\g;HL7ַvXIM:'?PbJ+Y2pn:UE֗4N%V뿡)=S]hJ'ѷ»xw߾nxWį+[7o4Үu 0fK^y+&tYM^:i5|9L}Uk?Nqlh_Ўt%ЗbW% +.Wȷ*ɿjLvkyaFGRWJD?ձ[u_. BRsӕ5\W)3Z0#güYg=3>̠t$hf47/bݹ3- >ė*/c~$'pF[^\?0ՠа䴤WL0kB2ETGJ&~OmIVJd}z<F}kHkL) Z*:7]In-ӷMߓX;p*mv~;Bn Sr+Ö|X]jh?Mɱ`V6 }폵6 nuYLh6Lj!$ Hi'n 35w{?M?.|PUZʆP!4ՕC,φP# BpSuHT~o߃F63sQP!3# Bx|*G7!vAe@_ ?禃QVйMh957kOkԍEMߑiK'rH-^'KC&ucD$O_\0ڦV+ҝFw lVʈS/3d-%:[aûǥm -ɹL\uUM6t WT|!Uj 9#SX4 8M플C$qZ+FkkdӸ3)kfɝR0sE]Rg,D]/L𣰨WB;BmE*wZy>\N[ *HG4(x=gU~بAô>ߨ}-8e%Ee ;<9wF1q$1A[㪪h˂sqw{B32 gXeٟ%ONl7pk@8Fw\i-hګŬ0_b@3U_n.|8WW1LzMГgp{h54KM6@jl0=^<@(,po @pH#|-nS'ZeLLn6M+чifYK7)4&8٨'oxgjx🙰cPa^Hq}m ˱R,m|[l֣4b72YϛOԷHEŠ}-2w!G]<9뿾zm~&L$a48aQ􏶦;&cNm]%?"s(=)1} w1c7cآ3$In#q'u\2-0/!x%MwKi(r~dcvB lF.5zgj+9nCk$HScB{n4)Mщ27#$#<ao&ʟ*(.+'+[ 0.J5O[?n ->Db0,~c':>SmM:2['|\TIxi?@akjUO b10R g𿶳fcӬ 'Ѣ'ω Ң&ώ \Gaմ+C3p WTJ}!U=R1t'b1@ @9̍=?֏0ȫSaPmen벅z{GY?Loϼ}thIՖnSazB4ns\ q/"@~I]Q\T[/$rC /rݶ53@k/hG|#O Zh5Uj>nC Uk:wǵ(x5Xȝλ ,3?B_[;{keo<9QXW!62b h*.B!Ts (ƞ|ŧ޴BKOKSob !4e? 4;BzQ{kz ddA`0d1rC!kPL^WW_iqQT2&^WR݃?i/Y-=&x]٠ E/(X?b1NCOe%<.^=eeHP G?BE VSo'(Ā4ҨYo}7qY{8;%(D -(*dQ,f`d+:U_ǹ9^&Fvc$~%?S, StXAES~dlI嵥^vd0sw2;8s߄_%Z?Iy+Wɺ<'vncUaH30`!,Dь[0|f'Aз07Jxgg^g ; [U2O}(Trd 5r^K9 7᮷Fw$w)H5SHBAS#YN)es:JYR/டl| ?s>8K(pÙ*^G~#qhzS?Qв1* Sttd5\xr|MKOINܶ/ؖx|XHtFH)fFz$[RIU=gǮ'20 UÊ˚0M)SUV[5a&~{j>YXXɏ決.,f!t+)f}#eJ(x`F<- sK:b?T))gǮmv/l~{ +fO筝sV ?^%Dݓ@yW&gG>+ S/=WݨspQˮyaI^o% ?--n32uJI=OǮkVX>e[ zL{w1+7#4gw$(ڱ@JT+7/ݹ3- >ė*/q-_(hCaRY{jKR"C.39"K/gAJyQń B_!V7Bx<BjV ֵ@on ![ĦCe!45U,Kd>P^dh]:y,n 'Y Oҕ eiLO eRV7ϱ6.1R8}K=Izz] f#W)$A0,5"aK|?G.5Sɿ%jOfj~~ʍVr!!G/ )QRw隋TRr캦俬7kOkԍEMߑiz>+ק|C^$f+7,ooEI&#~Sx:M'4U87c7ç I ]/`RE>9wWWQmx}krjޓ~7_{\TCO?ęmn#Y1^FA9mƝVfOY7sML+2&L :smgٝ/r+QYԄ?n1hG"wddU J=)o;a@?۔ff\~av4ӃqVQVC{ȫha>2olȻHb`w)Ht?Q]/ݙ9 3'gϰ$`5cEߗeb<:mMޤyMz6Y~ z[hS?_v4?=):'"RG,}ƘB*vysR?IlL?CO'R#QG?듗NF"r#G?[a7e.Nh12}e4DRe}.ݤ6W/WKtE;O)^(OhإWM y}}%6ocH%윚 _싆-O6LWZ1t́`COsy: dDɿzw ^jI4&Ib'M5Nj8E?59|Ë&6lyz^-dh <"VOh_FsUވs^|eq7,u{p곩Iڹ3}eZ~a^B^KPvl4M֯OӍ=_7sR_pɿ;ĀB{n4)Mщ27#$#<߈< 켘L~rRTT DaYC/-d FeruqG71kOD7?vc;E 6 cwחG6 5qޥF%Љ+4o$RD h9fcӬ 'Ѣ'ω Ң&ώ \GexiܢVh9^%K=}wG#Hj4_IAw֏KF1ƺZ:q}9G0qk /iJC3EmIQbJ"W-Yҁ@ "cܚűS>>if$o)qROo$ڀ'(Bfؒs eKld/,;څ^'佱'R#Q#/_VQ60fCET8įWOzQqsȭUMS^rٹiy,pURRT&;ߛc[}qZQq~r19tr*n-&]0-:L`TmxR]z1 g_0A@7BOc# jTjZ?e%}$\q[#=I:V)4:iڻ+0'CG])ܲ]:O)Ϯn`B=Cf@St/CNz1@Qj"BS_-0Տ ݉ I\K* ɿJ/;!}|KF4y(p2juuXAֿl' iM k0I((Ā: $$I5;V}p5 ?!z (ؑf`Drp]}e\Cнlyܦ-6A)~ =E܍u4JUuR0,-$Rx;3.i;YX!q$2,W66fyN|B8._GCm` >Q2)hW Im6 y4h38$?:܎}m16oCH^g ; [U2 ja*;Xz EfsjAn]o H|\|7%<.4gALy QńPP05B k!<~nV'^,Bbt,ߢ.Bj^Z=>~ žoBv]-z8n_A~%…!\( HUwd廊GlCi۷n2ܰ"_ a %ƙ:u iD~p5.$.<lK鐽S9*3=S©j/0+9ZLe2BKBoe+½)=1*C-?Nɿ.(l %GJx=|= %2bf ?A^~{7O{>!Φnk?euQG8>|YUt0ՑJJ\-Ye^زC͢c/;'qJqʺ K=%t6Fq"U f$'\ GOZ(c dA)׸bp]mmmCCÛ7oZЅBC;vÕ\MlD5IW _%S&}̂Qwgb4 Wz4GQפ3j[;nn}6<$H\LR8T3_dddnnnuuusssWWX r=ȏplK1쎽WR:,Wf9f?Bl_X/WhG LSHD`<0Œȯ"A>/{WeUB.͂Q !z!F]o< pi"Mŷ(Ύ] A녠sxoELܶ F.\ŕ%\nӬzTfO;l,(]gdꑒ3^_-'!}B(11~~.qֿ(C?WwEg[praQAŕlBFN4݁[^\?I@~:QS`i8ওS[rq6şѥ$=$ȿ#w:e=RVQBB'4#C?'S_2q.v6b}F̪_fAjl}-M:.auSOfj~`s!Y2;omו#eq<_ݣ IxV??_ѝC` 460a_Jj?$-YZ)(I00 ť<,H#lzˡ-{˜4@t\d_sI 1@ h?h_G?*"b` (_K%1E E*!jb`P So^{תNԻSى^1x U[_ޭڧ3xt N@ֿgJ b1 v|]B{ߘ`7es{:=hGkK >քH2:1S. q[ RLIWH=vMNsA,˖ R:vG/ u iqzwI#w (fb`20t72sRoo7 : ?م^VWIߙ|?%?|hݦ4=7 /l/r0Nx oFztfِg1O1C)Lq'o7$<-+-):,EughMwdL{crBG ?+9\twerSB w_N(P%gcnLTa@cEb(c˝0[Ǝ{ȿluh 饸->]tޜԏx5§GI<lKD];LoO,_۳ᅱ_ο߸rs`F.p}O: 1@ sn$]^ہsݎLJr}yCOoq&,HcÀn-RrM=pOH,8'}5Wb6T̩ ɾv+_*sxǙ^aD} MߠցS'0"m-.]?U?/jsb@иöb˂z*w鞑 |9fz2ԝ'A-™ux:h]q7,u{p곩:JSî ٪rNiRӃ}q%;(]vlJUC jbNjlI嵥^vQcdv5%ux8ςPµr+!] !=3 B# gO!d>@Wah=NNMӣΝgAPIb87/_0pyyh䖔W[Җۖ V_^୆~)i1rjv߬==nK;·nOZp'ovPeo}1NC,G%VxOfv@"f9f?BlAG!x%Dk\.v)o4_2{ʚkrg\11`zWnԙk?zGI3lؗAc6֘R*Td-/(*(lQzNjl ']ϩ w-0z3UBL"eI _Zm+U~3*,elL!qҿsҿoD%ٟc4WG$Dְwg,Μl=.}֌_Y6y61JӣqY=+YF]%s6=nU!(Ô]nNjl0Y.[M!8@:;bwctkUn"G/UnmmMXjKMГgp{h5nw~)ok4!ܦ_ -p8Nuzw*=!߮2n2[έYTnOk.3]sߖxvpiz;|ݗgLXܐ$ 'Srd}fKkM\8pƋBy@` Әa8uUtiD}Hft\>_Nt#$] s9j#='EYhwJo__BʭmG_-Ql%xQ崒@<٬G3]iki|5nS/wiBn` ,2ܹs '굩fN3Ij$qTㄩFS?ښ_*LCi+)fВBW#fco? aXL&$L/rc m 0o|VC%tM1җ%Ǒ#W[V'ix͗iy {-i[Bg}`bܿFO0ڣtOЕ7nN씹)%·+KHvU%m$2xKE(nc'#ɀ>ؔB5(]D͏N_{&@G^ڇVbaBA ?ܾ}{mgTYfGOEO3EM=|>!OOӣ*1,taMe]q*؝)omOTU/g;xƔ֎+i[_Ig଺䀫!wr!?9!BsiȌ=m]Y[ ρAzN~b8ܮbQ ' AR'=< ah^#pj+Sm/An׭fb?rQ~QC YoCP#߽{wskrL`fᓦ}hkW)vz L)Ag ]D*ɿ ]3:bS!~$*$YPj+[,;Duy 럋\rKI]~[;Đ0 "^]t)$$" ϩq_1!ɯ>[U(wz=N`́[^\@!=*X:Z)"L;^?$+%2>[=l?KA{$R6Tz/TgTXC[xMˋaB[t*3UR>bDs[xt1իpѫt#?Y? &O/|RN0waZrHe7,eMYYr䓅zNj2a@L]vO}ӯ#j~3w}$Hǃ-Pth/b錎/o_ڇC X7oܵ{ש3._|/8A诿g/=_x1m ܼyɓ'>V}mkg~Ͱvh_oI)B<ȿkք8=SZ(!㔁 5`@R_Sp#G?k0sT?JJN֤?JYОdxC{҆R@ 2TF5E%%G5ڐx$ڐ ( u@ֿ:,{z$Z;H(3PRJ@ hgɿ - T4H({kQf 20x/ oLhz9$)PZ7?m |w@ P0lPƆ? sX}S4?Vs@aBGyph(@| BP$j4nnB&싎ױ7|jKj/Aߘ%XgX>FD;ӽuǻ'~]95h"@O.$%{j^X>e[ zL{w1+7#4g$z11|=$$ ݮu hX;K̷Nߒ7}OҢ] f#Դ~8q*{ r SS!1*Ǭs;!ff@3 22!(փLF.yCe@xTȄg^Bg! S BuNZWQa 4)eY{ \n,.oLCe'rkt2KhzNe3 /nS Y6A)=E܍u4J7>\\[b0ym)Gٚ,sqҋ$^^[oG=,fxNfl:)?D͸NMj.s^ߴ-nLkƊܿ/,yudI\ /$2幀f_ zLrzї"yN-Wc}=]pnަT0e_]10`/lE_7ݍUAʹK T˼7S[_K^еzn^%Ue3nz89}h(<,H! 2oot0]:*cet˴i"X>N3x]Iam.tWE \J7J*0CЮ MZ '='EYhxUqNƛ֓Dd%DܚvɦZf"q(^G{4vgS+ŬB?EwD]H/T,ϜWb6T3o _0c&tt7=~F#*s9maᤥzm~&L$a48aQ􏶦;䨊]UGj)α2} ֧m+)f}@˿#fӷ0yn&EPl(fyz}+SmfjH<)[y $Pɪ@i=lDշokUi:")nY&#gS k:9Eym]]m]}CZ~20̖1x WڣtOЕ7nN씹)%· Xe^زC͢/&]Uv[a4+%= $=)RK#BV`{ÚrFIvp8Ib,$5pnܸz_qzf 郐`@P\ BqA.СPX]()T,C[S` G7! lBPSpY1n+yol4^v*GX`H俿op )긍wn6crpY3PPyC4J<ÕΚN'̎D<'f2K<;z2r}CJ_W];$r/J '=AG&{uO`wg&0JJTIo}}: ijo(Ev@]H8.h^ &%XO,pڣ<>z#,Ѝ,ڒȐly..S!O_L":BFu iYr[ Xp>Bѯ!\w]1;<.g"$c#'d=!t"hp4S| hT+:U_ǹ2'6r% 5Wv:Y:6E_nJnl.StXA[~UŰ~nGfB|*W/C xn | ԥxBT;sG?܏^fC(zv?B*tҠ5zN׉*6so;7J&e :rk}xڜn]anm5HY߂xΆgW8ѿb~ztک9g/`zbE6F)/vU%m$2,xK/K[m k=- ^SqlJ<f?N/W˞l4^v*EM/f!?]ud r;3u׽xN{l?v vJOK'բ*A_yv=HTtȽ*1,nnd_l,]q*؝)UaA[_x|A_}#ltڣ<>z#]НgLdO$CT\`> *%X=ߧFK%/2򗞆٩J`743?!h% ݢ [ہ'VL貚ؽt"j"sګ~\L3ǬG~1[ez~BCЕp@_O¦2t_qB~0Ec.} xyet#K'SmD(/ה "7ÖK(kd.[PF'/KeHJETF?Qߝk>A|sљ\H#P\L*MzN4݁[^\zQPĩ{~5(4,!9-U97Z)NQ[|֣W!g]9.*}RݎLsl3ӊC9Ρ D$HՓ+ 7=qa/dv[#6u%+c =3aDVɌ Y3[{韔 -ƻ~lOb H>QY[{_Kn߻5O첛{]>o&I& ߖxvYc>3mRX[}pUnZ!GG!}9',re"]fU5^kڍ]NE7^V`, /9ti'qf۸ $lPE 9u,$(R@jS?D`L[{w$ojMD_,jcH1WP "׆\P; Hզj݈eJHG[@?r ʮA$ Bj%H2[$ 1$*X$ ݪ lA?ڲ*HJNya;?T7E _TF;9ҏ@ HGQj$@ |o,.z7b`@?BĢ`H("b`0x}/ً_/{)qjw5#{#$)PZlOwg,Ki | ^WrA!&Be΄PĄ -0GZz2(joqcp-,$ue*ypn ŐhaC.,P\R !/:+BVm/ߣS! O dWu2_k%LMF@ =^xQR9^fmHeɿlɿH?P 3^vn&9·ruپ!,= M5B_4 X5rF PG9<olqn?O/{b#g1^%`D_d> gt^6 /nS 6A)=E܍u4J7>\EC.ŕaR5!5X^p":EowaB]Bף /h G?<߄ެ檮2+n;}AT.CquqCN>!S4'~;m.7an6|ώ}ۿ}̤6φqO!/9_s ۔ff\~av4Ӄq]TLGo;aPo@)l>J7[@7p OJKh6 {G@Gj41_]?7%|<}pR&'Qˮ(40d8`-TvzNut7=~FL"ڠaoa;L8s+gլVoޣZqveV]ĸ;+ǐn}P2YxхHQ) {_w;~݋4^Ollt/ įy]-*8lD5I˯©b? R+n|}@<3%55Q\*L"~ /oqdJGy}F;%t#ϘN`<dm˕_ loߑЦCU8}/= 1SohfUҿ.~(Ö@!b9oń.K'2&< jt?sjp_t%/#at_qb *0Ec.} ^*@fY%H$I2*o$~$HՖ2c6TVVVP [r.WxST% :" ivWa?*Mȿε}iW Tys N.$v(l&N$'@HLb-/ l[xy=s_ KHNKzzV 7Xa_(*)%-UYW#l˭J{@>ؼg'sCAfH/$H&r%Jf[#63?*#OncԅmįגۤBڡ]vsbGpc$danu4>kl~MN Kz+/m?V\#~3w&V_# C͉˨bobCHȆt7߄@؋{ df0 &PBVB]}=n_ E6jYs!CP騁 A Aુ_A]rO^e25Q.ʫ5.'sX/f'@8W@?tГ=|giio*#v.[~ʐ<oQdi#G B9G1j$SА#GnmkeD $ K"׈1"h0kۛ~ZQCePCi efiKZS:#Gه@?*b` Д$Z!(9h`h%Fe G1@ $ =e ײ AA FHGC.4"b1QkN;`Yt%b10@?sazw׼xsD{塙4@膠p< Zqqj=nB F|7ob w';3%k+S3j1@? %COe%<.^=eeLU\w]q<.g"$e2ՍQQ](Aya| ߪ/؂ ~"^&Fvc$N]}e\pӡ{L0$.M[.GxK'~s7uz(pq WkK >քx`wK{, ^GH_@eC}0}e=*QA ; աhW%Pь[0|f'Aз07Jx_L:`mlȳ蘧A&!%4=#b_m[Z%^YŽB_%#l~EEE-lX)w&_>qO!/9_s ۔ff\~av4Ӄq]T(3=>52)(6[@7p OJKh^to9^s|<|\|ws.gu),$j幀f_ zLrzї"yN-Wc}=]pnަK56)Co/c1fw[3EKH.b1CTߪ BFfAHe0 dp j!CHcd@!&5¯!ĕ𵙞T~e>ESe_`EnWƿ14!M.KyJ]f du]3>pÙ*8sx+ҍzD:c!<>)^ 'u7ìE뮊w7V#JlƐJbSK}$/Y38bk?v*~Y Xe^زC̈́}?*n+p'qd1$e] ~w}I^zDjpoX3_٨nI~𚢎ۘ8~fS:f~y (Ajf)d=yS+d,bVe\.qc%-u/Ozm?鳝²ҽ/w PW];$r/J ' p7L2/Č+n|}@<3%55Q\*LvGQD!^~{Go ~\B7 Ya럂yeF,fSQN_k}KOCTb%hXlC\_4lt'VL貚ؽt"j"sګ~\L3ǬG~ &AW}?r4Ag +pr S$/=Wݨh,n~$[b${`wj7`j\ɲ2'AG!*]c$c4ckiBE=wL˿xʋWwEg[pr![@qe31U vU9vBb#oyY`sBɬpSjPhXBrZҫsn&mR ҭGQI)iBκas\nUҏLsl3ӊC9Ρ DA{Qu>TF1Һ1Q1r[ (Ϯ ȆpQ*ŭiZ: 'ChbBz!9=B~j4&VS)9 e++󱅞wD J5f@?frܾ_8lwx'5ޥzkl~ [ Kz+/.Գ~&VŹFgH5+t}l#/2Q^kڍ]N7^Odۦf8}4';0z$lLkV>10lB;ft ;$HACQjrjKһa?wSPbɿD34:bE+(2 TF;HG{#$tF@Q:Q``HrwC F$#D, 1Q"B HN^7A?^s7b10 ~郠N BA4k݇z 1$vFLkK{x7~g/~썧<9*v6O?9"B T2_%E-g`du';3%k+r:Zj$C =2B #D, 10_Uwx ŋ"r2@HPđ㼖WZ䏿2 5^Yo}7ߎeXdՖ!'ԑ 9 i:J!5@ HI̅P:ﮩpaW%t63Q;[V}Lp/n\5Wv:ǢY: 6E_nnl.StXA[~UX+P\i@&-%:[Ye=."Bz]Pī P; j:_=bIJJkų;}AACqUVdBI%e@sK:{Ե^_9<=[Ssj6n)8qPiC[9ZZCУs Y2(HȴLV(aH}y^ֽ>kk.B @av!TEBP~JTg;YN Eʄ 9WҲN,ޜܬ`쯻s#wjɿ'_kki6vIqgIN}/9gi}/9Mԍ}1aMym9lMߟH€H M" mCj21xCMK;[wr0\֏A- ѻvucP~.[0U=0݂aH2ÃK1D_+P{710$W`WS 2 a~ր͟1г~wV &dgʞ\I2~AfJIZ[hm&cf BBM- P<2/T"t#I􈥔2F-2"r|XύX~Bs?s: MB\IH#[K/MD-)zfX ~W=$ޓꨁi|ku._wAYk&x?!:ۺGیJZmF!( B8(xE Ȱ(>MlK & E"U\Ȃ!u/%t xHFŅKuhcu>9Gc6(r]h30p1u-I*9i;d_gFS0_~ K ҮPXQv~V=Osoh3w4Kɀ7VY9ҙX} S}i)9@/ӕ'P6 ~7XlV%D8Ǥt:a35FSّ7ek967.nXOѓ }:IZVǖI)" NX{He!ľ Α\_&:LH 7$r r5a&yf?5Lo%\¢" Ch&c6wÕVvؾsظ7#qߩDdk& HF@ANYh ҮW>zp?™8Hu{ןWGf>}嚗V\5[r62%tߴ#JT|ԓX7md?¸zJ 8OS"F.h_sVLTE OY0xe^\/5c!^D:ǯŠ5/~Y#0K0 J e-b2]_ j1t) +P҉R1G2Պ0(z*0Z*w0n~aWs5V &T_z3 o6.(gkΥGyg&/ekb˼Nfݐ>KOX4_z̹<AegGxKbS\}K73  /5^kĥ nkZ- ޗ ʿѝ;dw'#M&C]0abyD\rjz^?k-Y׬r3ϽȀ?@_z1h?MPAֻzwh8sb bL; =>_,0xؐM'.v~b9OTvKi1]#IX#X?Sdm23K/'B|D=iM"L茿Gڶ9ȿk X) T@'^V S+ȿ3Q2oe)W3 7e`1?  ȿ%Jtx`ffj3 oE cS U1oUi1kl)δwOZ %J W?iKާ7Dž &/.<?ca 1Ho.&'nȿykSdOOOZ}ʩϡ̀~~f $l L~En6ƹ_-mW1Y 87Cg$)Qzx7~(,I~x<䭈Nu_Sl@AAgB#?J0̼Os{Us щ>,+^a9Xɀ9^\]-?QљR>\+zsRy.t \Ttdk)pW {;ɡ0Wẇ5:Q-c֠MeO.c:Gbze(tC^{E&`1DCU 0c83e  |ù+W1<J b1ܼq~ +JNp%CiI s0ܼ wngaQR0wƿj~zrZz@N,KOpA`l#o|PoWdvBԸǟqD}Ou.tPĈL]}%- ɏU2%|f>Q%E&;wro-P`8^7S7gŜ5'r 4% g5m[ bKvYu :/dtM\ 6QVI`iXrW2/V{ɛO+WtȤ2F-2"r|XύX~Bs?s: MB\IH#[K/MD-)zfX ~W=$ޓꨁi|O{~Ue5~Rlkoh3wCK܋f,U<׋+QjZ*I̠-1x'^dVq]" .!)PD‹ #Pǰ}2ñF9ؔFAxOί#oIjdV9IY'F:0=?3P,FcAŠ#ڸv[o=7PHDm"KQƤvg1Xg759xI~7sS}Y{\2M㊝q匠KE"ĈMv%s V1*z^ s# ٿy"v^?'sZ:ZOA/ Kw?ƓqYj:oѿ[tAi&ڟ 7rа׀o9 ۯ~^>9ҙX} S_c[1Kt% 8- G9i61֥ ۄUiIG781)N،A}Uvdgڽ;͍Kxۢ=.# 3N *nulĦTxMUfvL07o) ҂W8.zW雛 Lᆤ]ߗS3?OL-c4XPZXTģ!vMdn*7{@.o>WN }%"_3Y@6Bv4z1 R6ACu&P[wr0T<[Aw(RPI'\7W`(j0݂A,ƠޕR %00$W`WS Оi-h&JS^dGG8GNS15s".D^ίlpvof+̿/lJ~Dz+GrަwR59l4f-kN=j)jx)Kc>ދef3ċt"h AA:I3O)5HQ}3^]ۻ@m̭-]WQ" wKꃗ_yeߵ9O+MlM [zWÜcPɬ!҇8\}IY"WBF׫bs)y<~κVŦnf$q!_j94֓X-~mC^WkrE0 )z; wO<F6Y u„5Kr^ʵ}3pڿ-u^g&;/ckϡy_RӪfܑyh`G7LD¿`0TΙ9o4m6@zL S͔$0  8tT4њY 3[B01.1s$mL &xh!ʱ2?V๙L-H? f$mL L?c~ 42U kig,fD(~Dr0?%4C$~k#䉍 KgK?m O3 bP 1'De@mܿ-u}vJ^kTןi . ȿ-ҷO+7]7ww娶ٟ4ni:X\ 6 6V}\+zsp(is鏃< w :X.5Hu):2嵔soPmb}ѻd@xꕎAʞ?]ƄuZ[Ĵ ;Xܞqykգ0 ;6,]%|/>;'}Xwh˨RdԓztZy OJYzK6r!B33`6us>z7\S2;y@jOXS}>ێ'̺}np:P(bW&LXᮎ}NwbfuDH_wGf1PKM?|X][KK8Mw~I>M[~{~pdon9kSOns`=(<dw!/d]ʊr5;f9F:Δ=KNARe̔.U}w2;`3, X]9ҙX} S_cM1Kt% 8- G9i61֥ ۄUiIG781)N،A}\YvnsX5lDt™k;K00"COC]u%`yڀA pU )ԯ6oO n^LKA%N){J(o[\W0TY?|8KT_k=nul*^&ȃ3nkoR4?p2_]*/'77TS I/3:Fe gHG5oztD-?7hBS6dʵ^qUoFƛUSe߽uLP ۘ7?|qJE33䣼~k/zñ&SZFm # #YvM L3z9=iNʛi02'<+M%*>Iܛ62CMhjB s˘SY9XwIjx߆e8^Y}rXjD_m1Ѻ} -H)zXԽ{'%~Y\%cvAG:s0j2 `lL'u+Vy)?_}+^6'i)[^05v2! o_RV~ĢФjX'o{\rJ_ D2#c)㥛IC\V,qi'xñ[ں.Ck 6.:'ɎB9D='dw'_1JMzαTf <"V@Kv5C B#ا9!?O89U5WH*pzW3 }L-;W0z>=5p1;WlfC6ɞ;ֺةºmVbt?;޷S::bh6^ /8+Vjy-Q*7Ɣ QA-[~(7`R32% 3X Q`<򯎰i_wȉx\=$ 6֓i)K[ɬ⒉ЧLU CJ%D)qXA>! )r ~&s$$@gRi@Zd6 `flLQCFM0.b L?H]3µHA' X> _[q\tVȿd F  dߜl`M'0 X8 @ `7 0@gX@r`R4c`Ej :ڍ "X^ Lg#]F :荾Y/Pk0 X PHD L7UHT TؿJK8^0@ɐ.2~ Csw^ MO+C~N. twc c<Ԓ uiS1^U( TãRJ" 7/'bhQYԢ [  VS6s' D;aÛGhU*$qE%@A: P5,*/H>f=X>@۹h##R,R"劄z_͙?\D+@bI#]^Ko>WثL&]oKd>d׉jm*{ /wiuo=T7xW1W9xܨ)[ $6bsHܵm9<ӧ]L'նsU֘Yy׉Eh;(}]>ʳs_6Q#>O yl;t'3=\@_0a=:JZs;݉śLJuwN~䮒@-7K4aum-.).,6~ީ%5{l4m=x%׷wrU%@atcq-VWԲRXݥLٳt8)_/L)RU8 3`3CA!o͋ѦE ~(p*b$z٤2F2"r|XύX~B0yyӥ׵{Ro;ȿֻlw#6EvBpa*=b䧪yq]0?x8x~U4lZpu`Vt!ޏpΆ+1f,G!҂"xdD&XP%"q*Kd:՗HjS2!S\TG88]>4fSr)'.uۍ6]msอr}{.=Kk}N; Y߈kTe:UHG3[ah?cU 6F&YMA2 On~]6Hژ>#f]RF3/)Fx|[bN/kBs\3.9 DQT%O5nj^;bt;TRӋk.xq'z)#= 0BoP()y;>;_A\< nۅaH ScЏg^= {b|C'Ű_1N,PaPcpù CNQ6Ea*IPx:.7hµ{Z Wj%~'5^CKkkKk۳^1(6lڌNXs']3O0ٚybʍMk?%Ot&VHT+_SB&^+ Nil(?I0n.9&*NK:pI$uf j?y+KlJWLlR07X۪ߩe_PFe`@+SXU6[[&6L7`fʠC}_#35Mt6 t5ܐˁ2ce5a&yf?5ЖUm@NciCiaQ@!4e6wÕVvؾS=6r=}AҰ[!T~o4ZcUT9[w5܏p&<7ңg"gM7>z_楁WM x 7$VMޣtR59l3< ץϢ)Ijx)KMKxK>PoT>ĵt)L*βnkma1UT/=ҙwx IȌf`_`t .LGןhp[`R&-̫aN1kdJA߾K+EI?y3Sx'Bug Mq/((?H:B, vvos/|ᮈ*L2λ4ȿmV57Z:Q`ZI k#o kW3ԛ`2ȀyZN >$th܋ #7ӡ "l>h8s_QxŘ`wA{|6X`в!dOk]Ta]6^3y_ȥ̉q۹Aڥ o*7[<=cQғXRv/Dy:o|,xMo ?>q'tZeC藯g) 6!Me2kq [\A1qH&)SR!I`HgAd:هȿ9Xf)eJ&OB0o#m5ل)Jȿ8*H)@M1t@gz A``2? 4.`\t0 0 ?30HPdy00 @9ńCtfg&N)g&= ^?`@'T09 **!J<v1ntj7*䪗 30Y-ɷ &g ڈ>vAV7+P2Ej j&553?g`HcH1Tb@ C ? ã u-d€H Gr %R / R 0 a(0\;CyӀZX`g+,oF[O;P.>)6M"՗Z6_K=?ȿ ,M @:Di>dh"^zxHR I\w|lK 6zw&6Yh;p~1 EC[\'6g8sqU#]^KO>WثF&]oKdS׉jm*{ /wiuo=S,k ^uZ{*;y;M?/oe\Sd8.9xz ΠqW:(5$䮵o3N@NϚr ݲ*/H-@N.B @av!TEBP~JTg;YN Eʄ 9WҲN,ޜܬjuwN~䮒@-7K4aum-.).,6~ީ%5{l4m=x%׷wrUs<9aTO8s֔q۟3aWawWSљg))pR*_Rҥme [eNj5o@1>(-y1ڴB#BE,B7bD)5M@FRO2 ˏTҠs?s: MB\IH#[ ܍*q%E [Pw%?Us'IniyFFbW3{rWfXݏ]jC #1VK K/|nG!҂"xdD&V%"q*Kd:՗HjS2!F.Ց=aׅd\TGlʉ rtv`do8n|\^ %chɎ x U# ! &0`ǃ`7eb(jÐ! I0ğM~n?Ɛ! Z0ZG׻ [9i;dM -/U騽ĵz9[u kV]69zaq2,&ޠYg'7D~$mLag|3.)uzSw#<}1'?՗e! ;YWJE"ĈMvՒ-sUxV1XC/%\?x*r~{2ңJ # #E&{tBTK5T{5̍X*,fV NjiuWrI7\׈{ --mz&T c`-$W75?Of<`5$ss_G'֛~JRL>ީVNMOWBP~f3`c] rMXUttIxnd .)_3՛N3K`1?)v߶>R!aAGYvns4~zVǖz޲Mx`/: =a99_KDg#@ .ᆤ]ߗS3?OLڲj`_;%\¢" ) ^ WZaFOȥlኃw;IF@'Euخ@43K>͈pqxȨDdks?CJG8GN&? )z_楁WM x 7$VMP{uR59l _|-kNa'LTE #g<WZ}ċXqz &}A'Eu K)zXԽ{'%^\/5!^EGY]]U]Hgf*\6ZN` 80{qa ?DŇ37M^`šy5i=u̲)}*ۗ~)r(4id&&ֻg%O.#8>b?^QPt<ąX|>?"炪38~n7˿iOKc==%v!R;ϯTgY\]ؤ„5Kr^ʵM$pY% VY6 E^dɀ?/;Tfz=̯޻z^3 1;WlkC6ɞ;ֺةºmfl3'>DžnOwp,뿤b8ÿ~PTUA7/CNac{&@Og0ğ_\CkK0[ď{N"XJ pGy{ƪ+FͿC+jRl%o+%m4ɇ/_#Q2fO 0KgLNÔʝ"4'0oN6t0?#?=3 fz<b5 f"Lf]\X^ 結b-`,=~`h9p R0 ȿ!00 $ &)"`:іs1Wkw`= @{X03XIi a- LQa]AgwSQ]$W.kJ[0t0z11 bG^ eWcK\JCc0-)< h1` *(*0Сctl}E/C_zeArLGTOJitBΧIXoF[O;P.>)6M"՗NCgd 31!Qce<7S0ӟ D;aÛGrTH%3N?TgV@J'̺X>@۹h##ɓX8E ,Ry.?t \D/R.EGҟNw}WsM s>zlNT40hSSx˘NC~Xw@Tv"fJ7w?*HߪͅV:.9xzgROROZ6N+iSOX8`aݠ0_'BJ0;t*Ϣ}!L(?sG?%*xҝ̬xp "F|e„+it'oNnV0\)]%Z/oɗxhZ]R\YlS_Kj/h{ {l"JR{yo ǫW zX? 񌾘<6W ,UΔ=KNARe̔.UkbQ信h"t?` 8 ݈E\PJR#K9>qBz@,F,?R!Vts?s: MB\IH#[ ܍n; !8[R̰K ]|WS</FBo171?SZ^!5c J0Dۄ*J0/!ظ3R HщC/K1`HPx0'av̉O<<*EEj{k[ ~Wԝ[$ޓ(ld ~Ct6MǤ=?iԂ)bk( B8(xE Ȱ(>MIٖL~D .C T_J" MdaqaR*v]OfxԘMu4 Gm7 Kv́%NnP26r?WDݩtg`i7BZSOί#oIjd:,ۓG!HGE.[aS G4ʛ xZgeП?8&mZ1i}G̼4֙Mf^RߍŜ<.T_^/7Lbg\sf^)D $#6{U˨kVvXw㥄T]oOVRFzZ xar2:\H+{j Wxi>9$OkĽֶ֖gz\fn%@-,N/ع\淆I̓@frzOI#;9~bJpZ(ʏrl _`KA Ӓnp"cR:IOv R?S)[4 # #a9e] isoVCّ7ek967.nj>>[Ai'~-!Ъj&'?؋G*f&}mww74eCj!ieF蔻 3S-3׎e 'j ?GC |d&c6wÕVvؾS=6r=}AҰˑ埥du\H[WތǍ7,=KDVpw[gH~#=)]Ltq:1YC+׼4r٢?/Q⣞ʽi#̩TMRyGi2 ՟ES <6mE OYT,^i/bꭂg׃vXx7R ]ŽB㽸_kxN1}/Hgf,\4њsȿO4?P|8slM [zWÜcP,e҇8\}IY"WBFFmb2,o{\rJ_ D2#qd)㥛IC\ p+".9福`5bx^!Co;܊R 14vaȮxHPC_O;_G0 Tb J0f _pãv MeZ*16ah(/QB,cP\I,p!/ ~wWTf <"V@Kv5Ca)cʨ& ϽȀ?@_z1h?v Q]=;c~;W0z>y΀K/3_a ${Xb )ΜH:Kp?A-M<=cY{e?s8};S]RZTڏ QA[~(Y( ?3`>_FKeKO'} eB =ʀ&q2?Ng0?R3~f?( !? @9Cf` $ m a1xP0` t[Eͺ*L0`g"x`30+k3 y00s $h< _xg`P1@fj5ȿE $b5g f-1oKmy>o˳œ,0+cЏC O8`S" \m |Spxܪrdm`i(a7Fv»Hz9\dӖ!ضPd ڈ>vAV7+P2Ej-s oU{aOPg a79axbs3T w"]P}R*Pmb*E'/!vfߴ'6`ڨĀ0{#T*$qE3%̀.BQc<mv2&b.W$,ƓMI3O么lWЁr9#]^Kϙ>WثL&]oKT}1}^'w)|CeL@!HLOGX-dȿiH}a6tq\rZkԓz,r+R'@hm&c׉Eh;(}]>ʳs_6Q#>O yl;t'3=\@_0a=:JZs;݉śL(]%Z/oɗxhZ]R\YlS_Kj/h{ {[Xܝw=!H 140aϹ `31''`H+{j[A- xA?e0xJ ax};>v=e/韺q  , P$6A-1x'^dVq]" A%D@Rɴȥ:?T1yhʘMu4 Gm7 qXɔXAAu(/]NvFҽރIZEUI ׋Oί#oIjdVIY'b}ّګi\;}’ű&<3w4Kɀ7VYɌkR 4?]IpN eCQǸ u)6aUqZ NsLJ'6cPcPvXU ^}+;2l?%Iܛ62רjJp8o,2Lgc$dĦZ/^xv ;3. y" O~-gju%@0wV1Ġ }~(y7ZzC"]>cзcg=pr|O/x//҉ڢ0R/Hgf7.\ ?KgOs@ygnvC/jz ;F0R/)+?R bQhȖeMLwϜ!fIDATKN $]Fp֝}Y,6՗t3 x 0RnAۼ+,ȿwWZ_ wImXObcuy_]X] ,!Gx)׮fw>` LxZ?"H}ƠtءGw0/w)3N\,gtFݐK/3_a ${Xb )ZsNݎw}?3b[L˿AJT[\vKiKk)"gMDkeO7^̟|5Lwfz0 g;m5fk%i? kLO0?9 oAgt@C0 VVV6`C``  BfK d ?80 fܔA@зersfR -0 XrZ@!Aԥ 0 7n Lb~}L*u4׿0ol5,P/OePfɸv&IO6ÌOF?} {2fCie)`VZ(0J؍Ơѩݨ.bط@[L err[&]F :荾Y/PkEc1}rR>ijTRjW)"=Q})nA @˘>oe)W3`gc{E I\wx @[XçK=[, 0c<mv2&VHqh|+㉠͙?\DM+@bHf*:2嵔ʸsm)9Tjtŷo>d>; T^>2&ꐿz$q^c_9'0 *0 " -| T`1D>>o~'@Qb _mp~wç_U/DfvЫD)P}@PŸ[F~zrZ@NJ].,d|׉Eh;(}]>ʳs_6Q#>O yl;t'3=\@_0a=:JZs;݉śUɏU2%|f>Q%E&;wro-P`8ޱ+a%??l1vgcw~3eRST~ 3klfl 3h acdLPb|P[hbi{ GXn"ɀ^(L遌8!=e s#_M@#́37] q]'!VKO;l6p7bS+Qd!gKxZge!WODO<<*`u18:*lAc c ˞hz;PZ?pPJ a P|(uHٖL~D .C T_J" MdPqaR*v]OfxrӘMu4 Gm7 Xɔx !N ձ =g3HG0[aHx3fJ ~fi7ʠ?+4^78p>Mt%ڴ#ic;3ywIi3tm9y\^n. qθrF%"hDTb&}ojqp u;f[?hah'4a0`U۠v~rI7\׈{ --mz~_/tfM?չo ϓ'l<\\щ'G:w2#P 4?]IpN eCQǸ u)6aUqZ NsLJ'6cPcPv+X([Am^`ڭ !ۭ-Xz5s?іsRb LxT}MڰկohpC.ˌђ 3S-3f_} i 'j ?GC |d&c6wÕVv>uDaG& =.=wC_S ;uJ ;w'W`h,A$@j04 0 խ}:j$zC^{0ѿOH c?% k7LGӮl#?i_NG8GN3_Ck,MM?rK+g-Z9C:oZ%*>Iܛ62רjJh8o,2Lgc$NhĦZ/^xRkMMpȿ-y߰:?՗;lˬ Yc9_ SQ'(>iR&-̫aN1kd(CzA߾K+EI?y3Sx'BugMMq/((?H:B, Ԩs`w\a_$|za^o*wvl#lf„5Kr^ʵ-pM 3ɿG2Ko40C D8j|}LOўqr WWgy=3*ҰAbL; =>_,0xِM'.v~b9&Ž`ȿ-?QaWӣ<=ccрf QA[~(YRDzog7$ɇ/_#Qpk[V̀9[N+ 60gr& %$i \3Y.*` č10?x30++7``s ȿ)0` [[{geN?h9%,X40k@:DiV`߬t1R X:0o%07 f@@g`@ׁ.xrl 'xʢO,o> ]O?\O95 [? _ƐCN` &Fr_~a['/GquKU$~L &xȲӲ[-~O(5)?znjntC!ށÂ<"~u R!]z]Hڊ1\C{ R'#RP[*P( M =ch`(~:A,(0? b@,}Ce~\|A e=Aߎ~{2A!jpE `6}gf,fzqYtG׷jGrJFg~Hpݦ͙?\|+@bnT#]^K鏔>WثF&]oKdh׉jm*{ /wiuo= K/q1? |7PUv_.(z*;y;M}jI]k=fj9}  SH6De~4y]W>烪7?|35%=4͍[ޏ?%*xҝ̬xp "F|e„+it'oNnV0)U9J_/̇յ4j$y^Ѵ ; [G:u^;:ZDgʞ\I2~AfJIU\-4$T g_`iXrW2/V{ɛO+Wth2F2"r|XύX~B5H~]u˛.ݓz+QG'e(%E k/?ȿe?ٝWE:5^ j廒2s/.ֺOw k'Z:f%k-qs/dW\/jTQG/i$Itʶ`{]$=NXu,`0ߗHjUE2폸0rqp .|'31hM9qAnR;bn",wF=򃅺ZVwmu^o/qr`߅Fi_v~4O&-] o~1eEo,㽾`?DZw2eKv J\;lhXaG& AAG;!FP=6rJlpA;ˇG8GNt7f`O/W6Wg;7S_y3WGsUsq?DG={F#Tu敏}#[5QY95rh4)jx)Kb\-l߲DKV/i/bmg׃vF>՗; F]D9QU<]RW_wmN@JӖvC/jz ;=O/)+?R bQh(o{\rJ_ D2#`)㥛IC\/9WX-k %,;Muu닮 p+".9STf <"V@Kv5CSg/Sle`!zO'?ӜLU! "l>h8sGLcݙwѯٰbl=wuSu{MyL? MorGy{b-:F9>.tճs }e7BTjʧ酄0t#j*V5{玜ՖdF0doqE fOS}}~(`, /@[O~ΰu 0 h9I<>`N<R PJFVa&m2 VY)`t@'b`2dl2[63(f$0 S 7ńCt0` [C)vfg6{` 0`?  - 4o(a7Fv»HnPV0 X QLH# LwC. 7f^@P0 X VP6IXoF[O;P.>)6M"՗8ݣz1t4cK$16`axQ!+%Ý 0d`H-ӋC]I1~݇a@F~ L1?ńCtfg߉b{E*hG"a3`AA-u!j@,N7Vv.Hm'H!cw"a1%mWث=#j\탮%o2DN6=~軌 :F?;WU=v=63R|OSl K ?I=k[N+Ŷ0oE a|׉Eh;(}]>ʳs_6Q#>O yl;t'3=\@'&pWG_I˾n;xsrIɏU2%|f>Q%eh_,6~ީ%5{l4m=xs1)g1rg=2e{HX3eRST~ 3KpY$ Ylh-4Pb|P[hbi{ GXn"ɀR@FRO2 ˏTTgUGἼR= u_Z}a^\_jS+Qd!gKxY?` =*`-y.!r=c$!ޏpΆu0oh˹0v(/~>  , P$6A-1x'^dVq]" A%D@Rɴȥ:?T1yF0c6(r]h30,c-= S)˿VQh؋nO)ˎt^of-҃3 LX;Qd?+@8x7ѕhڏI;Ɍk2h%~ʆ4Rmªⴤ瘔NR'lƠʿ!|L]z>ۭ-LAf`-t&<E#A|_6;G;gpkl225ܐˁ2ct]p穖PB3&êrpKU¢" )*96wÕVv>uHC6?G!urW=z?%,m # #YvYx  /@H'iWyi`UE+g(^BMk?DG={FUm;Z"QyGaY9X5?zeዸ4x?F; _ =0E^AÙ&/ekb˼NfU>@KOX>n=s.9%/x|"tYwβW_͌B~!.K=xc< c;| ٝ&"64Raš%[/ .,K,5H6^5F^dɀ?/;Tf1?E{ɝ+_]wοF1;WljC6ɞ;ֺةºmVbluw͐X|X(1Uv/Dy6:o|du %c- ź Oi2!H<0` [qHfeMl翍`@h9,=3ڿE`` 8 g+3b&'Y;% ~JH`ϱ+u00 Ls7A},-;H9?0 1 ϘE1oQ5[00n@Ma JHaI00s12MIENDB`weboob-1.2/contrib/boobank_indicator/setup.py000066400000000000000000000015321303450110500214430ustar00rootroot00000000000000from setuptools import setup from setuptools import find_packages setup(name='boobank_indicator', version='0.0.1', description='show your bank accounts in your System Tray', long_description='boobank_indicator will show you bank accounts and associated transactions in your system tray. Your bank accounts should be configured in boobank', keywords='weboob boobank tray icon', url='http://weboob.org/', license='GNU AGPL 3', author='Bezleputh', author_email='bezleputh@gmail.com', packages=find_packages(), package_data={ 'boobank_indicator.data': ['indicator-boobank.png', 'green_light.png', 'red_light.png', 'personal-loan.png'] }, entry_points={ 'console_scripts': ['boobank_indicator = boobank_indicator.boobank_indicator:main'], }, zip_safe=False) weboob-1.2/contrib/boobot.py000077500000000000000000000357071303450110500161360ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from datetime import datetime, timedelta from dateutil.parser import parse as parse_date import logging import re import os import sys from threading import Thread, Event from math import log import urlparse import urllib from random import randint, choice import itertools from irc.bot import SingleServerIRCBot from weboob.core import Weboob from weboob.exceptions import BrowserUnavailable, BrowserHTTPError from weboob.browser import Browser from weboob.browser.exceptions import HTTPNotFound from weboob.browser.pages import HTMLPage from weboob.tools.misc import get_backtrace from weboob.tools.misc import to_unicode from weboob.tools.storage import StandardStorage from weboob.tools.application.base import ApplicationStorage IRC_CHANNELS = os.getenv('BOOBOT_CHANNELS', '#weboob').split(',') IRC_NICKNAME = os.getenv('BOOBOT_NICKNAME', 'boobot') IRC_SERVER = os.getenv('BOOBOT_SERVER', 'dickson.freenode.net') IRC_IGNORE = [re.compile(i) for i in os.getenv('BOOBOT_IGNORE', '!~?irker@').split(',')] STORAGE_FILE = os.getenv('BOOBOT_STORAGE', 'boobot.storage') def fixurl(url): url = to_unicode(url) # remove javascript crap url = url.replace('/#!/', '/') # parse it parsed = urlparse.urlsplit(url) # divide the netloc further userpass, at, hostport = parsed.netloc.rpartition('@') user, colon1, pass_ = userpass.partition(':') host, colon2, port = hostport.partition(':') # encode each component scheme = parsed.scheme.encode('utf8') user = urllib.quote(user.encode('utf8')) colon1 = colon1.encode('utf8') pass_ = urllib.quote(pass_.encode('utf8')) at = at.encode('utf8') host = host.encode('idna') colon2 = colon2.encode('utf8') port = port.encode('utf8') path = '/'.join(pce.encode('utf8') for pce in parsed.path.split('/')) # while valid, it is most likely an error path = path.replace('//', '/') query = parsed.query.encode('utf8') fragment = parsed.fragment.encode('utf8') # put it back together netloc = ''.join((user, colon1, pass_, at, host, colon2, port)) return urlparse.urlunsplit((scheme, netloc, path, query, fragment)) class BoobotBrowser(Browser): TIMEOUT = 3.0 def urlinfo(self, url, maxback=2): if urlparse.urlsplit(url).netloc == 'mobile.twitter.com': url = url.replace('mobile.twitter.com', 'twitter.com', 1) try: r = self.open(url, method='HEAD') body = False except HTTPNotFound as e: if maxback and not url[-1].isalnum(): return self.urlinfo(url[:-1], maxback-1) raise e except BrowserHTTPError as e: if e.response.status_code in (501, 405): r = self.open(url) body = True else: raise e content_type = r.headers.get('Content-Type') try: size = int(r.headers.get('Content-Length')) hsize = self.human_size(size) except TypeError: size = None hsize = None is_html = ('html' in content_type) if content_type else re.match(r'\.x?html?$', url) title = None if is_html: if not body: r = self.open(url) # update size has we might not have it from headers size = len(r.content) hsize = self.human_size(size) page = HTMLPage(self, r) for title in page.doc.xpath('//head/title'): title = to_unicode(title.text_content()).strip() title = ' '.join(title.split()) if urlparse.urlsplit(url).netloc.endswith('twitter.com'): for title in page.doc.getroot().cssselect('.permalink-tweet .tweet-text'): title = to_unicode(title.text_content()).strip() title = ' '.join(title.splitlines()) return content_type, hsize, title def human_size(self, size): if size: units = ('B', 'KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB') exponent = int(log(size, 1024)) return "%.1f %s" % (float(size) / pow(1024, exponent), units[exponent]) return '0 B' class Task(object): def __init__(self, datetime, message, channel=None): self.datetime = datetime self.message = message self.channel = channel class MyThread(Thread): daemon = True def __init__(self, bot): Thread.__init__(self) self.weboob = Weboob(storage=StandardStorage(STORAGE_FILE)) self.weboob.load_backends() self.bot = bot self.bot.set_weboob(self.weboob) def run(self): for ev in self.bot.joined.itervalues(): ev.wait() self.weboob.repeat(5, self.check_tasks) self.weboob.repeat(300, self.check_board) self.weboob.repeat(600, self.check_dlfp) self.weboob.repeat(600, self.check_twitter) self.weboob.loop() def find_keywords(self, text): for word in [ 'weboob', 'videoob', 'havesex', 'havedate', 'monboob', 'boobmsg', 'flatboob', 'boobill', 'pastoob', 'radioob', 'translaboob', 'traveloob', 'handjoob', 'boobathon', 'boobank', 'boobtracker', 'comparoob', 'wetboobs', 'webcontentedit', 'weboorrents', 'assnet', 'budget insight', 'budget-insight', 'budgetinsight', 'budgea']: if word in text.lower(): return word return None def check_twitter(self): nb_tweets = 10 for backend in self.weboob.iter_backends(module='twitter'): for thread in list(itertools.islice(backend.iter_resources(None, ['search', 'weboob']), 0, nb_tweets)): if not backend.storage.get('lastpurge'): backend.storage.set('lastpurge', datetime.now() - timedelta(days=60)) backend.storage.save() if thread.id not in backend.storage.get('seen', default={}) and\ thread.date > backend.storage.get('lastpurge'): _item = thread.id.split('#') url = 'https://twitter.com/%s/status/%s' % (_item[0], _item[1]) for msg in self.bot.on_url(url): self.bot.send_message('%s: %s' % (_item[0], url)) self.bot.send_message(msg) backend.set_message_read(backend.fill_thread(thread, ['root']).root) def check_dlfp(self): for msg in self.weboob.do('iter_unread_messages', backends=['dlfp']): word = self.find_keywords(msg.content) if word is not None: url = msg.signature[msg.signature.find('https://linuxfr'):] self.bot.send_message('[DLFP] %s talks about %s: %s' % ( msg.sender, word, url)) self.weboob[msg.backend].set_message_read(msg) def check_board(self): def iter_messages(backend): with backend.browser: return backend.browser.iter_new_board_messages() for msg in self.weboob.do(iter_messages, backends=['dlfp']): word = self.find_keywords(msg.message) if word is not None and msg.login != 'moules': message = msg.message.replace(word, '\002%s\002' % word) self.bot.send_message('[DLFP] <%s> %s' % (msg.login, message)) def check_tasks(self): for task in list(self.bot.tasks_queue): if task.datetime < datetime.now(): self.bot.send_message(task.message, task.channel) self.bot.tasks_queue.remove(task) def stop(self): self.weboob.want_stop() self.weboob.deinit() class Boobot(SingleServerIRCBot): def __init__(self, channels, nickname, server, port=6667): SingleServerIRCBot.__init__(self, [(server, port)], nickname, nickname) # self.connection.add_global_handler('pubmsg', self.on_pubmsg) self.connection.add_global_handler('join', self.on_join) self.connection.add_global_handler('welcome', self.on_welcome) self.connection.buffer_class.errors = 'replace' self.mainchannel = channels[0] self.joined = dict() for channel in channels: self.joined[channel] = Event() self.weboob = None self.storage = None self.tasks_queue = [] def set_weboob(self, weboob): self.weboob = weboob self.storage = ApplicationStorage('boobot', weboob.storage) self.storage.load({}) def on_welcome(self, c, event): for channel in self.joined.keys(): c.join(channel) def on_join(self, c, event): # irclib 5.0 compatibility if callable(event.target): channel = event.target() else: channel = event.target self.joined[channel].set() def send_message(self, msg, channel=None): for m in msg.splitlines(): msg = to_unicode(m).encode('utf-8')[:450].decode('utf-8') self.connection.privmsg(to_unicode(channel or self.mainchannel), msg) def on_pubmsg(self, c, event): # irclib 5.0 compatibility if callable(event.arguments): text = ' '.join(event.arguments()) channel = event.target() nick = event.source() else: text = ' '.join(event.arguments) channel = event.target nick = event.source for ignore in IRC_IGNORE: if ignore.search(nick): return for m in re.findall('([\w\d_\-]+@\w+)', text): for msg in self.on_boobid(m): self.send_message(msg, channel) for m in re.findall(u'(https?://[^\s\xa0+]+)', text): for msg in self.on_url(m): self.send_message(msg, channel) m = re.match('^%(?P\w+)(?P.*)$', text) if m and hasattr(self, 'cmd_%s' % m.groupdict()['cmd']): getattr(self, 'cmd_%s' % m.groupdict()['cmd'])(nick, channel, m.groupdict()['args'].strip()) def cmd_at(self, nick, channel, text): try: datetime, message = text.split(' ', 1) except ValueError: self.send_message('Syntax: %at [YYYY-MM-DDT]HH:MM[:SS] message', channel) return try: datetime = parse_date(datetime) except ValueError: self.send_message('Unable to read date %r' % datetime) return self.tasks_queue.append(Task(datetime, message, channel)) def cmd_addquote(self, nick, channel, text): quotes = self.storage.get(channel, 'quotes', default=[]) quotes.append({'author': nick, 'timestamp': datetime.now(), 'text': text}) self.storage.set(channel, 'quotes', quotes) self.storage.save() self.send_message('Quote #%s added' % (len(quotes) - 1), channel) def cmd_delquote(self, nick, channel, text): quotes = self.storage.get(channel, 'quotes', default=[]) try: n = int(text) except ValueError: self.send_message("Quote #%s not found gros" % text, channel) return quotes.pop(n) self.storage.set(channel, 'quotes', quotes) self.storage.save() self.send_message('Quote #%s removed' % n, channel) def cmd_searchquote(self, nick, channel, text): try: pattern = re.compile(to_unicode(text), re.IGNORECASE|re.UNICODE) except Exception as e: self.send_message(str(e), channel) return quotes = [] for quote in self.storage.get(channel, 'quotes', default=[]): if pattern.search(to_unicode(quote['text'])): quotes.append(quote) try: quote = choice(quotes) except IndexError: self.send_message('No match', channel) else: self.send_message('%s' % quote['text'], channel) def cmd_getquote(self, nick, channel, text): quotes = self.storage.get(channel, 'quotes', default=[]) if len(quotes) == 0: return try: n = int(text) except ValueError: n = randint(0, len(quotes)-1) try: quote = quotes[n] except IndexError: self.send_message('Unable to find quote #%s' % n, channel) else: self.send_message('[%s] %s' % (n, quote['text']), channel) def on_boobid(self, boobid): _id, backend_name = boobid.split('@', 1) if backend_name in self.weboob.backend_instances: backend = self.weboob.backend_instances[backend_name] for cap in backend.iter_caps(): func = 'obj_info_%s' % cap.__name__[3:].lower() if hasattr(self, func): try: for msg in getattr(self, func)(backend, _id): yield msg except Exception as e: print(get_backtrace()) yield u'Oops: [%s] %s' % (type(e).__name__, e) break def on_url(self, url): url = fixurl(url) try: content_type, hsize, title = BoobotBrowser().urlinfo(url) if title: yield u'URL: %s' % title elif hsize: yield u'URL (file): %s, %s' % (content_type, hsize) else: yield u'URL (file): %s' % content_type except BrowserUnavailable as e: yield u'URL (error): %s' % e except Exception as e: print(get_backtrace()) yield u'Oops: [%s] %s' % (type(e).__name__, e) def obj_info_video(self, backend, id): v = backend.get_video(id) if v: yield u'Video: %s (%s)' % (v.title, v.duration) def obj_info_housing(self, backend, id): h = backend.get_housing(id) if h: yield u'Housing: %s (%sm² / %s%s)' % (h.title, h.area, h.cost, h.currency) def main(): logging.basicConfig(level=logging.DEBUG) bot = Boobot(IRC_CHANNELS, IRC_NICKNAME, IRC_SERVER) thread = MyThread(bot) thread.start() try: bot.start() except KeyboardInterrupt: print("Stopped.") thread.stop() if __name__ == "__main__": sys.exit(main()) weboob-1.2/contrib/downloadboob/000077500000000000000000000000001303450110500167325ustar00rootroot00000000000000weboob-1.2/contrib/downloadboob/README000066400000000000000000000020451303450110500176130ustar00rootroot00000000000000This script can be used to automatically download videos matching some criteria. To avoid to download a video twice, all videos are stored in an unique way: .files//.avi For each entry in the configuration file, the script: - checks for new video - downloads the new videos - creates a link from /.avi to .files//.avi In each section of the configuration file : - backend: the backend to use - pattern: specify the search pattern - title_exclude: a pipe separated list. If an item in this list is a substring of the title, then the video is ignored. - id_regexp: an optional regular expression that the video id must match - max_results: maximum number of result to parse - directory: the above. Usage: downloadboob [path/to/file.conf] [section1 [section2 ...]] If /etc/downloadboob.conf or ~/downloadboob.conf is found, it will be used. If sections are specifed, only those are downloaded. By default, all sections of the config file are downloaded. weboob-1.2/contrib/downloadboob/downloadboob.conf000066400000000000000000000006011303450110500222470ustar00rootroot00000000000000[main] directory=~/Téléchargements/podcasts [zapping] backend=canalplus max_results=10 pattern=zapping title_exclude=semaine directory=Le zapping [guignol] backend=canalplus max_results=10 pattern=les guignols de l'info title_exclude=La semaine directory=Les guignols de l'info [ondar] backend=francetelevisions max_results=20 pattern=demande id_regexp=on_n.*rire directory=ONDAR weboob-1.2/contrib/downloadboob/downloadboob.py000077500000000000000000000234501303450110500217640ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2012 Alexandre Flament # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import subprocess import requests import os import re import ConfigParser from weboob.core import Weboob from weboob.capabilities.video import CapVideo # hack to workaround bash redirection and encoding problem import sys import codecs import locale if sys.stdout.encoding is None: (lang, enc) = locale.getdefaultlocale() if enc is not None: (e, d, sr, sw) = codecs.lookup(enc) # sw will encode Unicode data to the locale-specific character set. sys.stdout = sw(sys.stdout) # end of hack def removeNonAscii(s): return "".join(i for i in s if ord(i) < 128) rx = re.compile(u'[ \\/\\?\\:\\>\\<\\!\\\\\\*]+', re.UNICODE) def removeSpecial(s): return rx.sub(u' ', u'%s' % s) DOWNLOAD_DIRECTORY = ".files" class Downloadboob(object): def __init__(self, backend_name, download_directory, links_directory): self.download_directory = download_directory self.links_directory = links_directory self.backend_name = backend_name self.backend = None self.weboob = Weboob() self.weboob.load_backends(modules=[self.backend_name]) self.backend = self.weboob.get_backend(self.backend_name) def purge(self): if not os.path.isdir(self.links_directory): return dirList = os.listdir(self.links_directory) for local_link_name in dirList: link_name = self.links_directory + "/" + local_link_name if not self.check_link(link_name): print(u"Remove %s" % link_name) os.remove(link_name) else: print(u"Keep %s" % link_name) def check_link(self, link_name): if os.path.islink(link_name): file_name = os.readlink(link_name) absolute_file_name = os.path.join(self.links_directory, file_name) if os.path.isfile(absolute_file_name): return True return False else: return True def download(self, pattern=None, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False, max_results=None, title_exclude=[], id_regexp=None): print("For backend %s, search for '%s'" % (backend_name, pattern)) # create directory for links if not os.path.isdir(self.links_directory): print(" create link to %s" % self.links_directory) os.makedirs(self.links_directory) # search for videos videos = [] for i, video in enumerate(self.backend.search_videos(pattern, sortby, nsfw)): if i == max_results: break self.backend.fillobj(video, ('url', 'title', 'url', 'duration', 'ext')) if not self.is_downloaded(video): if not(self.is_excluded(video.title, title_exclude)) and self.id_regexp_matched(video.id, id_regexp): print(" %s\n Id:%s\n Duration:%s" % (video.title, video.id, video.duration)) videos.append(video) else: print("Already downloaded, check %s" % video.id) linkname = self.get_linkname(video) if not os.path.exists(linkname): self.remove_download(video) # download videos print("Downloading...") for video in videos: self.do_download(video) def is_excluded(self, title, title_exclude): for exclude in title_exclude: if title.find(exclude) > -1: return True return False def id_regexp_matched(self, video_id, id_regexp): if id_regexp: return re.search(id_regexp, video_id) is not None return True def get_downloaded_ext(self, video): ext = video.ext if not ext: ext = 'avi' elif ext == u'm3u8': ext = 'mp4' return ext def get_filename(self, video, relative=False): if relative: directory = os.path.join("..", DOWNLOAD_DIRECTORY, self.backend_name) else: directory = os.path.join(self.download_directory, self.backend_name) if not os.path.exists(directory): os.makedirs(directory) ext = self.get_downloaded_ext(video) return u"%s/%s.%s" % (directory, removeNonAscii(video.id), ext) def get_linkname(self, video): if not os.path.exists(self.links_directory): os.makedirs(self.links_directory) ext = self.get_downloaded_ext(video) misc = video.date if not misc: misc = video.id return u"%s/%s (%s).%s" % (self.links_directory, removeSpecial(video.title), removeSpecial(misc), ext) def is_downloaded(self, video): # check if the file is 0 byte return os.path.isfile(self.get_filename(video)) def remove_download(self, video): path = self.get_filename(video) if os.stat(path).st_size == 0: # already empty return print('Remove video %s' % video.title) # Empty it to keep information we have already downloaded it. with open(path, 'w'): pass def set_linkname(self, video): linkname = self.get_linkname(video) idname = self.get_filename(video, relative=True) absolute_idname = self.get_filename(video, relative=False) if not os.path.islink(linkname) and os.path.isfile(absolute_idname): print("%s -> %s" % (linkname, idname)) os.symlink(idname, linkname) def do_download(self, video): if not video: print('Video not found: %s' % video, file=sys.stderr) return 3 if not video.url: print('Error: the direct URL is not available.', file=sys.stderr) return 4 def check_exec(executable): with open(os.devnull, 'w') as devnull: process = subprocess.Popen(['which', executable], stdout=devnull) if process.wait() != 0: print('Please install "%s"' % executable, file=sys.stderr) return False return True dest = self.get_filename(video) if video.url.startswith('rtmp'): if not check_exec('rtmpdump'): return 1 args = ('rtmpdump', '-e', '-r', video.url, '-o', dest) elif video.url.startswith('mms'): if not check_exec('mimms'): return 1 args = ('mimms', video.url, dest) elif u'm3u8' == video.ext: _dest, _ = os.path.splitext(dest) dest = u'%s.%s' % (_dest, 'mp4') content = tuple() baseurl = video.url.rpartition('/')[0] for line in self.read_url(video.url): if not line.startswith('#'): if not line.startswith('http'): line = u'%s/%s' % (baseurl, line) content += (line,) args = ('wget', '-nv',) + content + ('-O', dest) else: if check_exec('wget'): args = ('wget', '-c', video.url, '-O', dest) elif check_exec('curl'): args = ('curl', '-C', '-', video.url, '-o', dest) else: return 1 os.spawnlp(os.P_WAIT, args[0], *args) self.set_linkname(video) def read_url(self, url): r = requests.get(url, stream=True) return r.iter_lines() config_file = 'downloadboob.conf' sections = None if len(sys.argv) >= 3: sections = sys.argv[2:] if len(sys.argv) >= 2: config_file = sys.argv[1] config = ConfigParser.ConfigParser() config.read(['/etc/downloadboob.conf', os.path.expanduser('~/downloadboob.conf'), config_file]) if sections is None: sections = config.sections() try: links_directory = os.path.expanduser(config.get('main', 'directory', '.')) except ConfigParser.NoSectionError: print("Please create a documentation file (see the README file and the downloadboob.conf example file)") sys.exit(2) links_directory = links_directory.decode('utf-8') download_directory = os.path.join(links_directory, DOWNLOAD_DIRECTORY) print("Downloading to %s" % (links_directory)) for section in config.sections(): if section != "main" and section in sections: backend_name = config.get(section, "backend") pattern = config.get(section, "pattern") if config.has_option(section, "title_exclude"): title_exclude = config.get(section, "title_exclude").decode('utf-8').split('|') else: title_exclude = [] if config.has_option(section, "id_regexp"): id_regexp = config.get(section, "id_regexp") else: id_regexp = None max_result = config.getint(section, "max_results") section_sublinks_directory = config.get(section, "directory") section_links_directory = os.path.join(links_directory, section_sublinks_directory) downloadboob = Downloadboob(backend_name, download_directory, section_links_directory) downloadboob.purge() # FIXME sortBy, title.match downloadboob.download(pattern, CapVideo.SEARCH_DATE, False, max_result, title_exclude, id_regexp) weboob-1.2/contrib/fork.py000077500000000000000000000101701303450110500155760ustar00rootroot00000000000000#! /usr/bin/python # Licensed under WTFPL. # https://linuxfr.org/users/shamanphenix/journaux/weboob-la-consecration#comment-1583941 from __future__ import print_function import os import sys import base64 import subprocess import ConfigParser scripts_tr = [('boobank','bisoubank'), ('boobathon','bisouthon'), ('boobcoming','bisoucoming'), ('boobill','bisoubill'), ('booblyrics','bisoulyrics'), ('boobmsg', 'bisoumsg'), ('boobooks', 'bisoubooks'), ('boobsize', 'bisousize'), ('boobtracker', 'bisoutracker'), ('cineoob', 'cineisou'), ('comparoob', 'comparisou'), ('cookboob', 'cookbisou'), ('flatboob', 'flatbisou'), ('galleroob', 'gallerisou'), ('geolooc', 'geolooc'), ('handjoob', 'handjisou'), ('havedate', 'havedate'), ('masstransit', 'masstransit'), ('monboob', 'monbisou'), ('parceloob', 'parcelisou'), ('pastoob', 'pastisou'), ('qboobmsg', 'qbisoumsg'), ('qcineoob', 'qcineisou'), ('qcookboob', 'qcookbisou'), ('qflatboob', 'qflatbisou'), ('qhandjoob', 'qhandjisou'), ('qhavedate', 'qhavedate'), ('qvideoob', 'qvideisou'), ('qwebcontentedit', 'qwebpasmalintentedit'), ('radioob', 'radisou'), ('suboob', 'subisou'), ('translaboob', 'translabisou'), ('traveloob', 'travelisou'), ('videoob', 'videisou'), ('webcontentedit', 'webpasmalintentedit'), ('weboob-cli', 'webisounours-cli'), ('weboob-config', 'webisounours-config'), ('weboob-config-qt', 'webisounours-config-qt'), ('weboob-debug', 'webisounours-debug'), ('weboob-repos', 'webisounours-repos'), ('weboorrents', 'webisourrents'), ('wetboobs', 'wetbisous')] icons_tr = [os.path.join('icons',i) for i in os.listdir('icons')] modules_tr = [os.path.join('modules',i,'favicon.png') for i in os.listdir('modules')] desktop_tr = [('qboobmsg.desktop','QBisoumsg'), ('qcineoob.desktop','QCineisou'), ('qcookboob.desktop','QCookbisou'), ('qflatboob.desktop','QFlatBisou'), ('qhandjoob.desktop','QHandJisou'), ('qhavedate.desktop','QHaveDate'), ('qvideoob.desktop','QVideisou'), ('qwebcontentedit.desktop','QWebPasmalinTentEdit'), ] mask = base64.b64decode('iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3wEODzYzz5PthwAAAB1pVFh0Q29tbWVudAAAAAAAQ3JlYXRlZCB3aXRoIEdJTVBkLmUHAAABsklEQVR42u3avyvEcRzH8adEXJKSOlFXStSVlDIow5UMBoOiDAyUwWAwSDIYlIG6EoPBYFAGSQZlUDdIKWUQJQxSIiElv7ow8K1L13V33oP35/t+1vsPeCyfu3p9wbIsy7Isy7K0FvIzvg/4BLr8iB/7wXvX7Cf83C/8J/AC1PgBv5oE7901UOYqvBDYSYH37gjIdw0fBE7SwHsXcwkfBm4zwHu34gI+Arxngfcuqhnf/Qd44rVrxA8L4Wc14meE8KMa8ctC+F5t8DxgWwAeB1q14UuBQwH8A1CvDV8NXAngz4FKbfgm4FkAvwcUacN3CD12Gxpf+kEh/IJG/KQQfkIjflEIP6ANngNs+vV/fTGwLwB/Ahq14UPAhQD+EqjShm8AHgXwB0CJNnwb8CGA3wJyteH7hR67JY0/c+NC+CmN+Hkh/JBG/JoQvlMbPADsCsBfUbjrlQOnAvgboFYbvh64E8Afo3THiwngYyje7wrIbKdzcrIKkt1eF8WhwsBbBvgRHCySJr4Hh0s1YsaBFnxQsjHzHqjDR00n4M+ACnzYOt/f6ASwLMuyLMuyLMv6x30B2yNJ8I8ofLMAAAAASUVORK5CYII=') f = open('bisoumask.png','w') f.write(mask) f.close() for i in icons_tr + modules_tr: try: subprocess.call('mogrify %s -blur 0x4 2> /dev/null > /dev/null' % i, shell=True) subprocess.call('composite bisoumask.png %s bisouresult.png 2> /dev/null > /dev/null' % i, shell=True) os.rename('bisouresult.png',i) except OSError: print("No picture named %s" % i, file=sys.stderr) for s in scripts_tr: os.rename (os.path.join('scripts',s[0]),os.path.join('scripts',s[1])) for d in desktop_tr: config = ConfigParser.ConfigParser() config.readfp(open(os.path.join('desktop',d[0]))) config.set('Desktop Entry','Name',d[1]) config.write(open(os.path.join('desktop',d[0]),'w')) weboob-1.2/contrib/hds/000077500000000000000000000000001303450110500150375ustar00rootroot00000000000000weboob-1.2/contrib/hds/export.py000077500000000000000000000113611303450110500167370ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function try: import sqlite3 as sqlite except ImportError as e: from pysqlite2 import dbapi2 as sqlite from weboob.core import Weboob from weboob.exceptions import ModuleLoadError import sys import logging level = logging.DEBUG logging.basicConfig(stream=sys.stdout, level=level) def main(filename): weboob = Weboob() try: hds = weboob.build_backend('hds') except ModuleLoadError as e: print('Unable to load "hds" module: %s' % e, file=sys.stderr) return 1 try: db = sqlite.connect(database=filename, timeout=10.0) except sqlite.OperationalError as err: print('Unable to open %s database: %s' % (filename, err), file=sys.stderr) return 1 sys.stdout.write('Reading database... ') sys.stdout.flush() try: results = db.execute('SELECT id, author FROM stories') except sqlite.OperationalError as err: print('fail!\nUnable to read database: %s' % err, file=sys.stderr) return 1 stored = set() authors = set() for r in results: stored.add(r[0]) authors.add(r[1]) stored_authors = set([s[0] for s in db.execute('SELECT name FROM authors')]) sys.stdout.write('ok\n') br = hds.browser to_fetch = set() sys.stdout.write('Getting stories list from website... ') sys.stdout.flush() for story in br.iter_stories(): if int(story.id) in stored: break to_fetch.add(story.id) authors.add(story.author.name) sys.stdout.write(' ok\n') sys.stdout.write('Getting %d new storiese... ' % len(to_fetch)) sys.stdout.flush() for id in to_fetch: story = br.get_story(id) if not story: logging.warning('Story #%d unavailable' % id) continue db.execute("""INSERT INTO stories (id, title, date, category, author, body) VALUES (?, ?, ?, ?, ?, ?)""", (story.id, story.title, story.date, story.category, story.author.name, story.body)) db.commit() sys.stdout.write('ok\n') authors = authors.difference(stored_authors) sys.stdout.write('Getting %d new authors... ' % len(authors)) sys.stdout.flush() for a in authors: author = br.get_author(a) if not author: logging.warning('Author %s unavailable\n' % id) continue db.execute("INSERT INTO authors (name, sex, description) VALUES (?, ?, ?)", (a, author.sex, author.description)) db.commit() sys.stdout.write(' ok\n') return 0 if __name__ == '__main__': if len(sys.argv) < 2: print('Syntax: %s [--help] SQLITE_FILENAME' % sys.argv[0], file=sys.stderr) sys.exit(1) if sys.argv[1] in ('-h', '--help'): print('Syntax: %s SQLITE_FILENAME' % sys.argv[0]) print('') print('Before running this software, please create the database with') print('this command:') print(' $ cat scheme.sql | sqlite3 hds.sql') print('') print('You can then run export.py with:') print(' $ %s hds.sql ' % sys.argv[0]) print('') print('It fill the database with stories and authors information') print('fetched from histoires-de-sexe.net') print('') print('You can next use SQL queries to find interesting stories, for') print('example:') print('') print('- To get all stories written by women') print(' sqlite> SELECT s.id, s.title, s.category, a.name') print(' FROM stories AS s LEFT JOIN authors AS a') print(' WHERE a.name = s.author AND a.sex = 2;') print('- To get all stories where it talks about bukkake') print(' sqlite> SELECT s.id, s.title, s.category, a.name') print(' FROM stories AS s LEFT JOIN authors AS a') print(' WHERE a.name = s.author AND s.body LIKE \'%bukkake%\';') sys.exit(0) sys.exit(main(sys.argv[1])) weboob-1.2/contrib/hds/scheme.sql000066400000000000000000000005741303450110500170320ustar00rootroot00000000000000DROP TABLE authors; CREATE TABLE authors ( name TEXT PRIMARY KEY, sex INTEGER, description ); CREATE INDEX author_idx ON authors(name, sex); DROP TABLE stories; CREATE TABLE stories ( id INTEGER PRIMARY KEY, title TEXT, date TEXT, category TEXT, author TEXT REFERENCES authors, body TEXT ); CREATE INDEX stories_idx ON stories(id, category); weboob-1.2/contrib/imm-o-matic000077500000000000000000000220701303450110500163210ustar00rootroot00000000000000#! /usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2015-2016, Roland Mas # # Licensed under the GNU Affero General Public License, version 3 # # This is imm-o-matic, a script that filters the results of real estate sites # according to keywords, then generates PDF files containing the description # and photos of the relevant items found # # Requires a couple of Python modules plus wkhtmltopdf # Note that unless wkhtmltopdf is built against a patched webkit, it won't # work headless so you need an X session running # # Configuration file is YAML: # [BEGIN CONFIG FILE] # --- # set1: # queries: # - city1 # - city2 # keywords: # - garden # - terrace # maxprice: 1000000 # minprice: 10000 # backends: # - logicimmo # - pap # destdir: /home/roland/immo/set1/files/ # workdir: /home/roland/immo/set1/workdir/ # set2: # queries: # - city3 # - city4 # keywords: # - balcony # maxprice: 1000000 # minprice: 10000 # backends: # - explorimmo # destdir: /home/roland/immo/set2/files/ # workdir: /home/roland/immo/set2/workdir/ # [END CONFIG FILE] # # You'll need to prepare the flatboob saved queries (city1, city2 and so on # in the config sample above) by your own means, then run imm-o-matic import csv import sqlite3 import os import json import re import subprocess import hashlib import glob import tempfile import shutil import argparse import sys import yaml fields = [ u'id', u'title', u'area', u'cost', u'currency', u'date', u'location', u'station', u'text', u'phone', u'photos', u'details', u'url' ] # Init database connection def connectdb(): db = sqlite3.connect(dbf) tmpcursor = db.cursor() cols = ", ".join(map(lambda x: x+" TEXT", fields)) try: tmpcursor.execute('CREATE TABLE ids(id TEXT UNIQUE NOT NULL, details INTEGER DEFAULT 0 NOT NULL, fetched INTEGER DEFAULT 0 NOT NULL)') db.commit() except: db.rollback() try: cursor.execute('CREATE TABLE details('+cols+')') db.commit() except: db.rollback() try: cursor.execute('CREATE TABLE searchable(id TEXT UNIQUE NOT NULL, searchable TEXT NOT NULL)') db.commit() except: db.rollback() return db # Fetch ids for all available estates def getids(): cursor = db.cursor() for b in backs: for r in reqs: wf = workdir+r+'-'+b+'.csv' try: os.unlink(wf) except FileNotFoundError: pass print("Querying %s for %s"%(b,r)) subprocess.run(['/usr/bin/flatboob','load',r,'-n','1000','-f','csv','-b',b,'-O',wf]) cursor.execute('SELECT id FROM ids') ids = set() for row in cursor: ids.add(row[0]) for csf in glob.iglob(workdir+'*.csv'): reader = csv.reader(open(csf), delimiter=";") indices = reader.__next__() for csvrow in reader: row = dict(zip(indices,csvrow)) if row['id'] not in ids: print ("Inserting id %s"%(row['id'],)) cursor.execute('INSERT INTO ids (id) VALUES (?)', (row['id'],)) ids.add(row['id']) db.commit() # Fetch and store details for estates def getdetails(): cursor = db.cursor() cursor.execute('SELECT id FROM ids WHERE details=0 ORDER BY id') ids = set() for row in cursor: ids.add(row[0]) def serializefield(x): if x.__class__==dict: return json.dumps(x) elif x.__class__==list: return json.dumps(x) else: return x for i in ids: print ("Fetching details for %s"%(i,)) try: d = json.load(os.popen('flatboob info -f json %s' % (i,))) except json.decoder.JSONDecodeError: continue try: d = d[0] except IndexError: continue q = 'INSERT INTO DETAILS ('+','.join(fields)+') VALUES ('+ ','.join(list(map(lambda x: '?',fields)))+')' p = list(map(lambda x: serializefield(d[x]),fields)) cursor.execute(q, p) cursor.execute('UPDATE ids SET details=1 WHERE id=?',(i,)) db.commit() cursor.execute('SELECT id FROM ids WHERE details=1 ORDER BY id') ids = set() for row in cursor: ids.add(row[0]) def gensearchable(x): cursor.execute('SELECT title,text FROM details WHERE id=?',(x,)) row = cursor.__next__() searchable = row[0]+' '+row[1] searchable = searchable.lower() searchable = re.sub('\\s+',' ',searchable) cursor.execute('DELETE FROM searchable WHERE id=?',(x,)) cursor.execute('INSERT INTO searchable (id,searchable) VALUES (?,?)',(x,searchable)) for i in ids: gensearchable(i) db.commit() # Filter estates according to keywords and generate PDFs def search(): cursor = db.cursor() cursor.execute('SELECT id FROM ids WHERE fetched=1 ORDER BY id') fetched = set() for row in cursor: fetched.add('pouet') fetched.add(row[0]) cursor.execute('SELECT id,searchable FROM searchable') index = {} for row in cursor: if row[0] in fetched: continue index[row[0]] = row[1] for i in index: s = index[i] keep = False for k in kws: r = '\\b'+k+'\\b' if re.search(r,s,flags=re.I): keep = True break if not keep: continue cursor.execute('SELECT title,location,cost,area,text,phone,photos,url FROM details WHERE id=?',(i,)) row = cursor.__next__() d = { 'title': row[0], 'location': row[1], 'cost': row[2], 'area': row[3], 'text': row[4], 'phone': row[5], 'photos': row[6], 'url': row[7], 'id': i, 'site': re.sub('.*@','',i), } d['location'] = re.sub('\\d','',d['location']) d['location'] = re.sub('\\s+',' ',d['location']) d['location'] = re.sub('\(.*','',d['location']) d['location'] = d['location'].strip() if int(d['cost']) < minprice or int(d['cost']) > maxprice: continue print ("Selecting %s, downloading"%(i,)) for k in kws: r = '\\b('+k+')\\b' d['text'] = re.sub(r,'\\1',d['text'],flags=re.I) dirpath = tempfile.mkdtemp() do_checks = False try: f = open(dirpath+'/tmp.html','w') f.write("""%(title)s
  • Titre : %(title)s
  • Ville : %(location)s
  • Prix : %(cost)s €
  • Surface : %(area)s
  • %(url)s (site : %(site)s)
  • Téléphone : %(phone)s
  • Descriptif : %(text)s

""" % d) photos = json.loads(row[6]) for p in photos: u = p['url'] os.chdir(dirpath) m = hashlib.md5() m.update(u.encode('utf-8')) fn = 'photo-'+m.hexdigest() subprocess.run(['/usr/bin/wget','-q','-O',fn,'-c',u],check=do_checks) f.write(""" """%(fn,)) f.write("""""") f.close() dname = "%(location)s %(cost)s€ %(id)s.pdf" % d dname = dname.strip() subprocess.run(['/usr/bin/wkhtmltopdf','-q','tmp.html',destdir+dname],check=do_checks) cursor.execute('UPDATE ids SET fetched=1 WHERE id=?',(i,)) except subprocess.CalledProcessError: print ("Error") shutil.rmtree(dirpath) os.chdir(destdir) db.commit() if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument("--action", help="What to do", default="all", required=False) parser.add_argument("--regen", help="Regen PDF", required=False) parser.add_argument("--configfile", help="Config file", default=os.getenv('HOME')+'/.config/imm-o-matic.yaml', required=False) args = parser.parse_args() configs = yaml.load(open(args.configfile)) for i in configs: config = configs[i] destdir = config['destdir'] workdir = config['workdir'] kws = config['keywords'] reqs = config['queries'] minprice = config['minprice'] maxprice = config['maxprice'] backs = config['backends'] dbf = workdir + 'imm-o-matic.sqlite' db = connectdb() cursor = db.cursor() if args.regen: i = args.regen i = re.sub('^.*/','',i) i = re.sub('.pdf$','',i) cursor = db.cursor() cursor.execute('UPDATE ids SET details=0,fetched=0 WHERE id=?',(i,)) if args.action == 'getids': getids() elif args.action == 'getdetails': getdetails() elif args.action == 'search': search() elif args.action == 'all': getids() getdetails() search() weboob-1.2/contrib/munin/000077500000000000000000000000001303450110500154075ustar00rootroot00000000000000weboob-1.2/contrib/munin/boobank-munin000077500000000000000000000202601303450110500200740ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os import sys import locale import time import logging from weboob.core import Weboob, CallErrors from weboob.capabilities.bank import CapBank from weboob.exceptions import BrowserIncorrectPassword class BoobankMuninPlugin(object): def __init__(self): if 'weboob_path' in os.environ: self.weboob = Weboob(os.environ['weboob_path']) else: self.weboob = Weboob() self.monitored_accounts = None if 'boobank_monitored' in os.environ: self.monitored_accounts = os.environ['boobank_monitored'].split(' ') self.cache_expire = long(os.environ.get('boobank_cache_expire', 3600)) self.add_coming = int(os.environ.get('boobank_add_coming', 1)) self.cumulate = int(os.environ.get('boobank_cumulate', 1)) self.cache = None def display_help(self): print('boobank-munin is a plugin for munin') print('') print('Copyright(C) 2010-2011 Romain Bignon') print('') print('To use it, create a symlink /etc/munin/plugins/boobank to this script') print('and add this section in /etc/munin/plugin-conf.d/munin-node:') print('') print('[boobank]') print('user romain') print('group romain') print('env.HOME /home/romain') print('# The weboob directory path.') print('env.weboob_path /home/romain/.config/weboob/') print('# Monitored accounts. If this parameter is missing, all accounts') print('# will be displayed.') print('env.boobank_monitored 0125XXXXXXXXXXXX@bnporc 0125XXXXXXXXXXXX@bnporc') print('# To prevent mass connections to bank websites, results are cached.') print('# You can set here the expiration delay (in seconds).') print('env.boobank_cache_expire 7200') print('# If enabled, coming operations are added to the value of accounts\'') print('# balance.') print('env.boobank_add_coming 1') print('# Cumulate accounts values') print('env.boobank_cumulate 1') print('') print('When you change configuration, you can use this command to reset cache:') print('$ boobank-munin --reset') def clear_cache(self): for name in ('boobank-munin', 'boobank-munin-config'): try: os.unlink(self.cachepath(name)) except IOError: pass def cachepath(self, name): tmpdir = os.path.join(self.weboob.workdir, "munin") if not os.path.isdir(tmpdir): os.makedirs(tmpdir) return os.path.join(tmpdir, name) def check_cache(self, name): return self.print_cache(name, check=True) def print_cache(self, name, check=False): try: f = open(self.cachepath(name), 'r') except IOError: return False try: last = int(f.readline().strip()) except ValueError: return False if check and (last + self.cache_expire) < time.time(): return False for line in f: sys.stdout.write(line) return True def new_cache(self, name): os.umask(0o077) new_name = '%s.new' % name filename = self.cachepath(new_name) try: f = open(filename, 'w') except IOError as e: print('Unable to create the cache file %s: %s' % (filename, e), file=sys.stderr) return self.cache = f self.cache.write('%d\n' % time.time()) def flush_cache(self): old_name = self.cache.name new_name = self.cache.name[:-4] self.cache.close() os.rename(old_name, new_name) def write_output(self, line): sys.stdout.write('%s\n' % line) if self.cache: self.cache.write('%s\n' % line) def config(self): if self.check_cache('boobank-munin-config'): return self.new_cache('boobank-munin-config') self.weboob.load_backends(CapBank) self.write_output('graph_title Bank accounts') self.write_output('graph_vlabel balance') self.write_output('graph_category weboob') self.write_output('graph_args -l 0') try: accounts = [] if self.monitored_accounts is not None: d = {} for account in self.weboob.do('iter_accounts'): if self.monitored(account): d['%s@%s' % (account.id, account.backend)] = account for id in self.monitored_accounts: try: accounts.append(d[id]) except KeyError: pass else: accounts = reversed([a for a in self.weboob.do('iter_accounts')]) first = True for account in accounts: id = self.account2id(account) type = 'STACK' if first: type = 'AREA' first = False self.write_output('%s.label %s' % (id, account.label.encode('iso-8859-15'))) if self.cumulate: self.write_output('%s.draw %s' % (id, type)) except CallErrors as errors: self.print_errors(errors) self.print_cache('boobank-munin-config') else: self.flush_cache() def monitored(self, account): return not self.monitored_accounts or ('%s@%s' % (account.id, account.backend)) in self.monitored_accounts def account2id(self, account): return '%s_%s' % (account.backend, account.id) def print_errors(self, errors): for backend, err, backtrace in errors: print((u'%s(%s): %s' % (type(err).__name__, backend.name, err)).encode(sys.stdout.encoding or locale.getpreferredencoding(), 'replace'), file=sys.stderr) if isinstance(err, BrowserIncorrectPassword): self.weboob.backends_config.edit_backend(backend.name, backend.NAME, {'_enabled': 'false'}) def execute(self): if self.check_cache('boobank-munin'): return self.new_cache('boobank-munin') self.weboob.load_backends(CapBank) try: for account in self.weboob.do('iter_accounts'): if self.monitored(account): balance = account.balance if account.coming and self.add_coming: balance += account.coming self.write_output('%s.value %d' % (self.account2id(account), balance)) except CallErrors as errors: self.print_errors(errors) self.print_cache('boobank-munin') else: self.flush_cache() def run(self): cmd = (len(sys.argv) > 1 and sys.argv[1]) or "execute" if cmd == 'execute': self.execute() elif cmd == 'config': self.config() elif cmd == 'autoconf': print('no') sys.exit(1) elif cmd == 'suggest': sys.exit(1) elif cmd == 'help' or cmd == '-h' or cmd == '--help': self.display_help() elif cmd == 'reload' or cmd == '--reload' or \ cmd == 'reset' or cmd == '--reset': self.clear_cache() if self.cache: self.cache.close() sys.exit(0) if __name__ == '__main__': logging.basicConfig() BoobankMuninPlugin().run() weboob-1.2/contrib/munin/freemobile-munin000077500000000000000000000135131303450110500205750ustar00rootroot00000000000000#!/usr/bin/env perl # -*- perl -*- =head1 NAME freemobile - A plugin to monitor a freemobile subscription =head1 INSTALLATION Create a link to this script in /etc/munin/plugins/ : $ ln -s /path/to/freemobile-munin /etc/munin/plugins/freemobile =head1 CONFIGURATION You need to configure the plugin like that: [freemobile] user florent env.HOME /home/florent env.phonenumber 06ABCDEFGH env.freemonitored voice sms mms data specialvoice env.cache_expire 3600 timeout 30 C I: user with freemobile backend configured C I: path to user home C (optional): add your phone number if you have more than one subscription for this account. C (optional): default only 'voice sms' The full list of monitored options is : * voice sms mms data specialvoice voicetoint voiceint smsint mmsint dataint C (optional): cache interval in second, or time between two connection to the website. The cache interval is 3 hours by default. C (optional): Munin internal option. The plugin can be slow, 30s is recommended. =head1 LICENSE AGPLv3 =cut use strict; use warnings; use Carp; use English qw(-no_match_vars); use encoding 'iso-8859-1'; # Munin doesn't like utf-8 :-( use Encode; my @monitored = split / /, $ENV{'freemonitored'} || 'voice sms'; my $cachedir = $ENV{'HOME'} . '/.config/weboob/munin/'; my $cachefile = "$cachedir/freemobile-munin"; my $refreshtime = $ENV{'cache_expire'} || 10_800; my $phone = $ENV{'phonenumber'}; my $account = ''; if (length($phone) > 0) { $account = $phone . '@freemobile'; } my $weboob = 'boobill -f table -b freemobile details ' . $account; my $cache_fh; my %label = ( 'voice' => 'Voix en France (min)', 'voicetoint' => 'Voix vers l\'international (min)', 'specialvoice' => 'Numéros spéciaux (min)', 'sms' => 'SMS en France', 'mms' => 'MMS en France', 'data' => 'Data en France', 'voiceint' => 'Voix à l\'international (min)', 'smsint' => 'SMS à l\'international', 'mmsint' => 'MMS à l\'international', 'dataint' => 'Data à l\'international', ); my %linenum = ( 'voice' => 3, 'voicetoint' => 3, 'specialvoice' => 4, 'sms' => 5, 'mms' => 6, 'data' => 7, 'voiceint' => 8, 'smsint' => 9, 'mmsint' => 10, 'dataint' => 11, ); my %regexp = ( 'voice' => 'National : (\d+)h(\d+)min(\d+)s', 'voicetoint' => 'International : (\d+)h(\d+)min(\d+)s', 'specialvoice' => '\| (\d+)h(\d+) min (\d+)s', 'sms' => 'Conso SMS \s+ \| (\d+) \/ (.*)', 'mms' => 'Vous avez consommé (\d+) MMS', 'data' => 'Vous avez consommé ([\d\-\.]+) (Mo|Go)', 'voiceint' => 'Appels émis (\d+)h(\d+)min(\d+)s', 'smsint' => 'Conso SMS (international) \| (\d+)', 'mmsint' => 'Vous avez consommé (\d+) MMS', 'dataint' => 'Vous avez consommé ([\d\-\.]+) (Mo|Go)', ); my %post = ( 'voice' => 'postvoice', 'voicetoint' => 'postvoice', 'specialvoice' => 'postvoice', 'sms' => 'simplepost', 'mms' => 'simplepost', 'data' => 'datapost', 'voiceint' => 'postvoice', 'smsint' => 'simplepost', 'mmsint' => 'simplepost', 'dataint' => 'datapost', ); sub doubleprint { my $var = shift; print {$cache_fh} $var; print $var; return 0; } sub postvoice { my @args = @_; my $minutes = $args[0] * 60 + $args[1] + $args[2] / 60; doubleprint "$minutes \n"; return 0; } sub simplepost { my @args = @_; doubleprint "$args[0] \n"; return 0; } sub datapost { my @args = @_; my $multi = 1; my $unit = $args[1]; if ($unit eq "Go") { $multi = 1024; } $multi = $args[0] * $multi; doubleprint "$multi \n"; return 0; } sub config { binmode STDOUT, ':encoding(iso-8859-1)'; print <<'EOF'; graph_title Conso Free graph_vlabel Suivi conso du forfait Free Mobile graph_category weboob graph_args -l 0 EOF foreach (@monitored) { print "$_.label $label{$_}\n"; } return 0; } sub fetch { my @cache_data; # Check if cache exist and not older than the refresh threshold. if ( open $cache_fh, '<', $cachefile ) { @cache_data = <$cache_fh>; close $cache_fh or croak "unable to close: $ERRNO"; # File not empty? if ( @cache_data > 0 ) { # Data is still fresh. Display cached values and exit. if ( time - $cache_data[0] < $refreshtime ) { print join q{}, @cache_data[ 1 .. $#cache_data ]; exit 0; } } } # execute weboob open my $data, q(-|), $weboob or croak "Couldn't execute program: $ERRNO"; my @lines = <$data>; close $data or carp "unable to close: $ERRNO"; # If error output, print the cache (if exist) and exit if ( @lines == 0 ) { if ( @cache_data > 0 ) { print join q{}, @cache_data[ 1 .. $#cache_data ]; exit 0; } exit 1; } # Open cache for writing open $cache_fh, '>', $cachefile or croak "Failed to open file $cachefile"; print {$cache_fh} time . "\n"; foreach my $monit (@monitored) { doubleprint "$monit.value "; if ( my @results = $lines[ $linenum{$monit} ] =~ /$regexp{$monit}/ ) { my $postfunction = $post{$monit}; &{ \&$postfunction }(@results); } else { doubleprint "0 \n"; } } close $cache_fh or croak "unable to close: $ERRNO"; return 0; } # Create the munin cache dir if missing if ( !-d $cachedir ) { mkdir $cachedir; } if ( $ARGV[0] and $ARGV[0] eq 'config' ) { config; } else { fetch; } __END__ weboob-1.2/contrib/munin/nettokom-munin000077500000000000000000000117321303450110500203250ustar00rootroot00000000000000#!/usr/bin/env perl # -*- perl -*- =head1 NAME nettokom - A plugin to monitor a nettokom subscription =head1 INSTALLATION Create a link to this script in /etc/munin/plugins/ : $ ln -s /path/to/nettokom-munin /etc/munin/plugins/nettokom =head1 CONFIGURATION You need to configure the plugin like that: [nettokom] user florent env.HOME /home/florent env.monitored 0177-XXXXXXXX env.exclude 02 env.cache_expire 10800 timeout 30 C I: user with nettokom backend configured C I: path to user home C I: phone number to generated the weboob ID C (optional): default nothing We can exclude some boobill output. The filter is on the label. Some examples: * festnetz community sms smsinsausland O2 C (optional): cache interval in second, or time between two connection to the website. The cache interval is 3 hours by default. Be aware that the website is only daily updated C (optional): Munin internal option. The plugin can be slow, 30s is recommended. =head1 LICENSE AGPLv3 =cut use strict; use warnings; use Carp; use English qw(-no_match_vars); use encoding 'iso-8859-1'; # Munin doesn't like utf-8 :-( use Encode; my $id = $ENV{'monitored'} || ''; if ($id eq '') { print STDERR "Error: env.monitored not exist \n"; exit 2; } $id = $id . '@nettokom'; my @exclude = split / /, $ENV{'exclude'} || ''; my $cachedir = $ENV{'HOME'} . '/.config/weboob/munin/'; my $cachefile = "$cachedir/nettokom-munin"; my $cacheconfigfile = "$cachedir/nettokom-munin-config"; my $refreshtime = $ENV{'cache_expire'} || 10800; my $weboob = "/usr/bin/env boobill details $id -f table"; my @knowlabels = ("AbgehendeSMSimAusland", "AnkommendeGespr_cheimAusland", "Community", "D1", "D2", "Deutschland", "Festnetz", "FreeCall", "Mailbox", "O2", "SMS", "SMSinsAusland"); my $cache_fh; sub config { execute($cacheconfigfile, 1); } sub fetch { execute($cachefile); } sub doubleprint { my $var = shift; print {$cache_fh} $var; print $var; return 0; } sub printconfig { my @lines = @_; doubleprint <<'EOF'; graph_title Conso Nettokom graph_vlabel Nettokom Verbindungs graph_category weboob graph_args -l 0 EOF my $first = 1; for my $label (@knowlabels) { my $shortlabel = $label; if (!(grep {$_ == $shortlabel} @exclude)) { doubleprint "$shortlabel.label $label\n"; if ($first > 0) { doubleprint "$shortlabel.draw AREA\n"; $first = 0; } else { doubleprint "$shortlabel.draw STACK\n"; } } } for my $line (@lines) { if ($line =~ /nettokom \| (.*) \| (\d)(.*) \| (\d).(\d)/) { my $label = $1; my $shortlabel = $label; $shortlabel =~ s/\s+//g; if (!(grep {$_ == $shortlabel} @exclude)) { if (!(grep {$_ == $shortlabel} @knowlabels)) { doubleprint "$shortlabel.label $label\n"; doubleprint "$shortlabel.draw STACK\n"; } } } } } sub printvalue { my @lines = @_; for my $line (@lines) { if ($line =~ /nettokom \| (.*) \| (\d)(.*) \| (\d).(\d)/) { my $label = $1; my $value = $2; my $shortlabel = $label; $shortlabel =~ s/\s+//g; if (!(grep {$_ == $shortlabel} @exclude)) { doubleprint "$shortlabel.value $value\n"; } } } } sub execute { my @cache_data; my $cachefile = $_[0]; my $doconfig = $_[1]; # Check if cache exist and not older than the refresh threshold. if ( open $cache_fh, '<', $cachefile ) { @cache_data = <$cache_fh>; close $cache_fh or croak "unable to close: $ERRNO"; # File not empty? if ( @cache_data > 0 ) { # Data is still fresh. Display cached values and exit. if ( time - $cache_data[0] < $refreshtime ) { print join q{}, @cache_data[ 1 .. $#cache_data ]; exit 0; } } } # execute weboob open my $data, q(-|), $weboob or croak "Couldn't execute program: $ERRNO"; my @lines = <$data>; close $data or carp "unable to close: $ERRNO"; # If error output, print the cache (if exist) and exit if ( @lines == 0 ) { if ( @cache_data > 0 ) { print join q{}, @cache_data[ 1 .. $#cache_data ]; exit 0; } exit 1; } # Open cache for writing open $cache_fh, '>', $cachefile or croak "Failed to open file $cachefile"; print {$cache_fh} time . "\n"; if ($doconfig) { printconfig(@lines); } else { printvalue(@lines); } } # Create the munin cache dir if missing if ( !-d $cachedir ) { mkdir $cachedir; } if ( $ARGV[0] and $ARGV[0] eq 'config' ) { config; } else { fetch; } __END__ weboob-1.2/contrib/munin/weboob-generic000077500000000000000000000356571303450110500202440ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . ### Installation ### # 1) Create a symlink from /etc/munin/plugins/yourchoice to the script # 2) Configure the plugin in /etc/munin/plugin-conf.d/ See below for the options # 3) Restart/reload munin-node # 4) Note that cached values are stored in folder ~/.config/weboob/munin/ ### Configuration ### ## Mandatory options ## # env.capa: The Weboob capability to load # Example: env.capa CapBank # # env.do: The Weboob command to call. It can take more than one argument. # With two argument, the second is used as parameter for the command. # The third is used to restrict backends. # Example: env.do get_balance # # env.import: The import line to import the capabilities # Example: from weboob.capabilities.bank import CapBank # # env.attribvalue: The attribut name of objects returned by the do command. # For example, the "balance" member of Account objects # If the attribut is itself one object, a hierarchical call can be done # with the "/" operators. # Example: env.attribvalue balance # Example: env.attribvalue temp/value ## Optionals -- more configuration ## # env.id_monitored: Restrict the results to a list of ids (space is used as separator) # Example: env.id_monitored account1@backend1 account2@backend2 # # env.exclude: Exclude some results (space is used as separator) # Example: env.exclude 550810@sachsen # # env.cache_expire: To avoid site flooding, results are cached in folder # /.config/weboob/munin/. The default lifetime of a cache value is 3600s # Example: env.cache_expire 7200 # # env.cumulate: Display data in Area mode (default) or in line mode. # Example: env.cumulate 0 # # env.get_object_list: optional pre-call to get a list of objects, to applied # the do function on it. # Exemple: env.get_object_list="iter_subscriptions" # # env.attribid: Munin needs an id for each value. The default is to use the id of results, # but another attribute can be used. "/" can be used as separator for # hierarchical calls # Example: env.attribid id # # env.title: A title for the graph (default: nothing) # Example: env.title a wonderful graph # # env.vlabel: A vertical label for the graph # Example: env.vlabel Balance # # env.label: Each data in munin as a label. Per default, the script takes the # "label" attribute of objects. However, it does not always exist, # and a better choice can be possible # Example: env.label id # # env.category: set the graph category (default: weboob) # Example: env.category bank # For some running examples, see at the end of the script from __future__ import print_function import os import sys import locale import time import logging from weboob.capabilities.base import NotAvailable from weboob.core import Weboob, CallErrors from weboob.exceptions import BrowserIncorrectPassword class GenericMuninPlugin(object): def __init__(self): if 'weboob_path' in os.environ: self.weboob = Weboob(os.environ['weboob_path']) else: self.weboob = Weboob() self.cache_expire = long(os.environ.get('cache_expire', 3600)) self.cumulate = int(os.environ.get('cumulate', 1)) self.cache = None self.name = sys.argv[0] if "/" in self.name: self.name = self.name.split('/')[-1] # Capability to load self.capa = os.environ['capa'] # Command to pass to Weboob self.do = os.environ['do'].split(',') # Not easy to load modules automatically... self.mimport = os.environ["import"] exec(self.mimport) # We can monitore only some objects self.object_list = None if 'get_object_list' in os.environ: self.object_list = os.environ["get_object_list"] self.tomonitore = None if 'id_monitored' in os.environ: self.tomonitore = os.environ['id_monitored'].decode('utf-8').split(' ') self.exclude = None if 'exclude' in os.environ: self.exclude = os.environ['exclude'].split(' ') # Attribut of object to use as ID (default: id) self.attribid = "id" if 'attribid' in os.environ: self.attribid = os.environ['attribid'] self.attribvalue = os.environ['attribvalue'] self.title = '' if 'title' in os.environ: self.title = os.environ['title'].decode('utf-8') self.attriblabel = "label" if 'label' in os.environ: self.attriblabel = os.environ['label'] self.vlabel = self.attribvalue if 'vlabel' in os.environ: self.vlabel = os.environ['vlabel'].decode('utf-8') self.category = "weboob" if 'category' in os.environ: self.category = os.environ['category'].decode('utf-8') def display_help(self): print('generic-munin is a plugin for munin') print('') print('Copyright(C) 2013 Romain Bignon, Florent Fourcot') print('') print('To use it, create a symlink /etc/munin/plugins/nameyouwant to this script') print('and add this section in /etc/munin/plugin-conf.d/munin-node:') print('') print('[nameyouwant]') print('user romain') print('group romain') print('env.HOME /home/romain') print('# The weboob directory path.') print('env.weboob_path /home/romain/.config/weboob/') print('# Monitored objects. If this parameter is missing, all objects') print('# will be displayed.') print('env.id_monitored myid@backend1 otherid@backend2') print('# To prevent mass connections to websites, results are cached.') print('# You can set here the expiration delay (in seconds).') print('env.cache_expire 7200') print('# Cumulate values') print('env.cumulate 1') print('') def cachepath(self, name): tmpdir = os.path.join(self.weboob.workdir, "munin") if not os.path.isdir(tmpdir): os.makedirs(tmpdir) return os.path.join(tmpdir, name) def check_cache(self, name): return self.print_cache(name, check=True) def print_cache(self, name, check=False): try: f = open(self.cachepath(name), 'r') except IOError: return False try: last = int(f.readline().strip()) except ValueError: return False if check and (last + self.cache_expire) < time.time(): return False for line in f: sys.stdout.write(line) return True def new_cache(self, name): os.umask(0o077) new_name = '%s.new' % name filename = self.cachepath(new_name) try: f = open(filename, 'w') except IOError as e: print('Unable to create the cache file %s: %s' % (filename, e), file=sys.stderr) return self.cache = f self.cache.write('%d\n' % time.time()) def flush_cache(self): old_name = self.cache.name new_name = self.cache.name[:-4] self.cache.close() os.rename(old_name, new_name) def write_output(self, line): sys.stdout.write('%s\n' % line) if self.cache: self.cache.write('%s\n' % line) def build_do(self): if self.object_list: results = [] for result in self.weboob.do(self.object_list): results.append(result) for result in results: try: for i in self.weboob.do(self.do[0], result.id, backends=result.backend): yield i # Do not crash if one module does not implement the feature except CallErrors: pass elif len(self.do) == 1: for i in self.weboob.do(self.do[0]): yield i elif len(self.do) == 2: for i in self.weboob.do(self.do[0], self.do[1]): yield i elif len(self.do) == 3: for i in self.weboob.do(self.do[0], self.do[1], backends=self.do[2]): yield i def get_value(self, result): attribs = self.attribvalue.split('/') for attrib in attribs: result = getattr(result, attrib) if type(result) is list: result = result[0] return result def monitored(self, result): id = self.result2weboobid(result) if self.exclude and id in self.exclude: return False return not self.tomonitore or id in self.tomonitore def result2weboobid(self, result): attribs = self.attribid.split('/') id = '%s@%s' % (getattr(result, attribs[0]), result.backend) return id def result2id(self, result): attribs = self.attribid.split('/') id = result for attrib in attribs: id = getattr(id, attrib) return '%s_%s' % (result.backend, id) def config(self): if self.check_cache('%s-config' % self.name): return self.new_cache('%s-config' % self.name) self.weboob.load_backends(self.capa) self.write_output('graph_title %s' % self.title.encode('iso-8859-15')) self.write_output('graph_vlabel %s' % self.vlabel.encode('iso-8859-15')) self.write_output('graph_category %s' % self.category) self.write_output('graph_args --rigid') if self.cumulate: self.write_output('graph_total Total') try: objects = [] if self.tomonitore or self.exclude: d = {} for result in self.build_do(): if self.monitored(result): d[self.result2weboobid(result)] = result if self.tomonitore: for id in self.tomonitore: try: objects.append(d[id]) except KeyError: pass else: for id in d: objects.append(d[id]) else: objects = reversed([a for a in self.build_do()]) first = True for result in objects: id = self.result2id(result) type = 'STACK' if first: type = 'AREA' first = False self.write_output('%s.label %s' % (id.encode('iso-8859-15'), getattr(result, self.attriblabel).encode('iso-8859-15'))) if self.cumulate: self.write_output('%s.draw %s' % (id, type)) except CallErrors as errors: self.print_errors(errors) self.print_cache('%s-config' % self.name) else: self.flush_cache() def print_errors(self, errors): for backend, err, backtrace in errors: print((u'%s(%s): %s' % (type(err).__name__, backend.name, err)).encode(sys.stdout.encoding or locale.getpreferredencoding(), 'replace'), file=sys.stderr) if isinstance(err, BrowserIncorrectPassword): self.weboob.backends_config.edit_backend(backend.name, backend.NAME, {'_enabled': 'false'}) def execute(self): if self.check_cache(self.name): return self.new_cache(self.name) self.weboob.load_backends(self.capa) try: for result in self.build_do(): if self.monitored(result): value = self.get_value(result) if value is not NotAvailable: self.write_output('%s.value %f' % (self.result2id(result).encode('iso-8859-15'), value)) except CallErrors as errors: self.print_errors(errors) self.print_cache(self.name) else: self.flush_cache() def run(self): cmd = (len(sys.argv) > 1 and sys.argv[1]) or "execute" if cmd == 'execute': self.execute() elif cmd == 'config': self.config() elif cmd == 'autoconf': print('no') sys.exit(1) elif cmd == 'suggest': sys.exit(1) elif cmd == 'help' or cmd == '-h' or cmd == '--help': self.display_help() if self.cache: self.cache.close() sys.exit(0) if __name__ == '__main__': logging.basicConfig() GenericMuninPlugin().run() ### Examples ### ## Like boobank-munin does ## Only for the example, you should use boobank-munin instead #[bank] #user florent #group florent #env.cache_expire 7200 #env.HOME /home/flo #env.capa CapBank #env.do iter_accounts #env.import from weboob.capabilities.bank import CapBank #env.attribvalue balance #env.title Solde des comptes # # ## Balance of your leclercmobile subscription #[leclercmobile] #user florent #group florent #env.cache_expire 16800 #env.HOME /home/flo #env.capa CapBill #env.do get_balance,06XXXXXXXX,leclercmobile #env.import from weboob.capabilities.bill import CapBill #env.attribvalue price #env.title Forfait leclercmobile #env.vlabel Solde # #Result: http://fourcot.fr/weboob/leclercmobile-day.png # ## Monitor water level in Dresden #[leveldresden] #user florent #group florent #env.cache_expire 7200 #env.HOME /home/flo #env.capa CapGauge #env.do get_last_measure,501060-level #env.import from weboob.capabilities.gauge import CapGauge #env.attribvalue level #env.title Niveau de l'elbe #env.label id # # ## The level of the elbe in all Sachsen's cities #[levelelbesachsen] #user florent #env.cache_expire 800 #env.HOME /home/flo #env.cumulate 0 #env.capa CapGauge #env.do iter_gauges,Elbe,sachsen #env.import from weboob.capabilities.gauge import CapGauge #env.attribvalue sensors/lastvalue/level #env.title Niveau de l'elbe en Saxe #env.label name #env.vlabel Hauteur du fleuve (cm) #env.exclude 550810@sachsen # #Result: http://fourcot.fr/weboob/elbesachsen-day.png # ## Temperature in Rennes #[temprennes] #user florent #env.HOME /home/flo #env.cumulate 0 #env.capa CapWeather #env.do get_current,619163,yahoo #env.import from weboob.capabilities.weather import CapWeather #env.attribvalue temp/value #env.attribid temp/id #env.title Température à Rennes #env.vlabel Température #env.label id # #Result: http://fourcot.fr/weboob/temprennes-day.png weboob-1.2/contrib/plugin.video.videoobmc/000077500000000000000000000000001303450110500206325ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/addon.xml000066400000000000000000000017001303450110500224370ustar00rootroot00000000000000 video linux Plugin for Videoob [B]Videoob[/B] is an application able to handle videos on supported websites. More information available on http://videoob.org/. [B]Videoob[/B] est une application qui permet de récupérer des vidéos depuis divers sites. Pour plus d'informations à ce sujet, n'hésitez pas à visiter le site : http://videoob.org/ weboob-1.2/contrib/plugin.video.videoobmc/changelog.txt000066400000000000000000000002051303450110500233170ustar00rootroot00000000000000[B]Dependance :[B] - weboob (http://weboob.org/install) [B]Version 0.1.0[/B] -Call weboob directly [B]Version 0.0.1[/B] -First try weboob-1.2/contrib/plugin.video.videoobmc/default.py000066400000000000000000000016351303450110500226350ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys import resources.lib.base.common_xbmc as common_xbmc import resources.lib.constants as constants from resources.lib.actions import actions # Plugin constants version = "0.1.0" plugin = "videoobmc" + version addon_id = "plugin.video.videoobmc" author = "Bezleputh" mail = "carton_ben@yahoo.fr" #import lxml.html import Element #print Element.__file__ #TODO gestion du logger, gestion des modules via XBMC (activation/desactivation) #Bug encodge des categories #corriger version 1 pour que v2 et v1 donctionnent if (__name__ == "__main__"): if not (sys.argv[2]): actions[constants.DISPLAY_MENU]()._do() else: params = common_xbmc.parse_params(sys.argv[2]) action = params.get("action") if (action): actions[action]()._do(params) else: common_xbmc.display_error(" ARGV Nothing done.. verify params " + repr(params)) weboob-1.2/contrib/plugin.video.videoobmc/default_test.py000077500000000000000000000011011303450110500236630ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- from __future__ import print_function import sys import resources.lib.test.common_test as common_xbmc import resources.lib.constants as constants from resources.lib.actions import actions print(sys.argv) if len(sys.argv) < 2: actions[constants.DISPLAY_MENU]()._do() else: params = common_xbmc.parse_params(sys.argv[1]) #print params action = params.get("action") if (action): actions[action]()._do(params) else: common_xbmc.display_error(" ARGV Nothing done.. verify params " + repr(params)) weboob-1.2/contrib/plugin.video.videoobmc/icon.png000066400000000000000000000174271303450110500223030ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME V tEXtCommentCreated with GIMPWrIDATx{wWuo>sh$F,!؆ BIHy=V^Hx-d!(@v ;nXݪ۾~~q=E1!u-;}_Ը8^a_|IJ A(@IC ݩ4Y>pMwWUTl"I$b%&"ijBC9̗ wVcұLjkv3قc8k1 D=;O;:;F| @`I@h i `]^DsmOgMT*iR fMVJAkU=3t\Jxzۀ_~>]@R!/N阮R R=?)ܩlwo%6:6&յd[v!+bkm R QjuNLN:Q8fN;y\>/3/ufI jפۍ֦7kaX ! PP:8;nG|x+w;N.~=nzuͲnmAD$ff$:vlj|lfgO?:9~M^w.6|h A [(f);m' a℃_ ?;7[>Wf6gۯ}vMMMMmƅ^`fD `V BRxz<~l\S_Wg~u;Дtϛt. ? 3DE<4rԬTHHwO0[monm]׮sRLdaI `1X3Hkzյ;<e/>8] ҷ&גrٖNosϷK-60`f)IU+[Zfc,l 5>a gHfGO>ィOyuyjG4-/}xն7?鵿qԵj *ػg~n*ؿﱡ*&`=œKy;}2`&3K̬H֒fPsӓ3g~+>b_(JݫlMzz?~=zyrì։0Uܽ٫L\p̓`@H2|066|´BX XX\5:{]cqw vu3.]+5)"Je 8d۽6w$XΎ w/s7ʌ<|pFV*4v-)+qX9MBߵ.ym-1>sᔤ-m=-==:Θ` L#G/`q8C;N4X\ƏuԞ7`ϫT{G{7 Gq`̤48i.svǒ񅝯 rXR>[{.gp5wCCHVsL$ϼ<|6f~.[ey4-0k+Ws "n2df#ʱ0t"p͢^ZLP"JyzCkrڰ-Z.h5C4궟T[؆K=[B=:6P>F819Ԟvu!DܰȤܨeX+C뗊7$3KWip]BXnݰ깾Mמ Δ :)sclY`AD=k/{1uĻ-[?*QZkʤzͣGR.\y+Q ft5,䛌c:V-[㶶v[DpTˡCO]$[۴鶥8@XuO,yGŞ kz{J!QN^JNT3z{`X D0fRX.|G([RĊ@aĀA0M7 8%KV.[ȤOydm2e)˯?mf t\qu^Wnm~~Cd9}޶Ԁ[tqՓSZeS &F֭nXE۷m b S}hǷV=y㮿MǾsBtbYDRs,3+)hlGj toT.sM`MN:^~:NI {~?tj`bzw߿},clwy' 3k&"&!<$XBKG6?lQHL_||g&{O~ϵ}x~JH b%$  R8ڋ` Z;\Ye0d6߯ Vou-IP`1 rN:|9` 82EjDi]r߸~[p'+ciޏV>OK̰?9ӝIWi  +k4\7-tp#ЀajbbKYnIIp{XO Lt/ܼ}_YuKwlSUY̼DJ%  h-@V`ƫReYMM%ESe*\o.u`A?pWhZ֖i|׾{7/_r@aEiXLĬn fM,Kg*_ rMB`hf08}xjk5aȔ?ת6Hzrt>n*$%$I$YM4zeRKf !%Ha !$R##( 捗r➪gJxTgY2R& A~P7&A),Gs͖ a57+0 Y)߯'dcfv­R0h倅yMJ'aܰDe5|WsV+f$ِrtd 4M[nF yUsiT+\>q>37@HUФ ig`[ABt`aYvb@23y~@Sƙ[֏A- L¬0 "mzy6aXtRPݫ $LN^D`?21im3D 338Ł$"rq؆ t:y5( 2' "i TVLr^0_B,3){lIMӎ0T:ciB$Mθv$)=I -MeL&YeWIx|rY3S&]T`fYYkCȸwcBDTJH!F#yր.<őFƜd9׾DxvKk)z+="lTfwӳ2{8nK`Fϗff*I<geM P}J)08 |E(őBT#+.zw;Z}k妻\\-/n5+U(+2} noS og@ AY1ر]U `D 0w??߶lnqH1#e؁]Ul:,ZzoJ4B7#֫/I4 U)ENgRy# a nj@Z3 hXl2-}1믩LL ]7;{>vZs;??$kJٹl5yzzrn^W:k/ONYkZPY1'v֬3'Zk=_J38*Z*I]x=% f5T05PcY?Z+0( ȗմJGsrr$Cpݺ{5wݓyy&Kr'DeID #"+zW~O_;кyhTv7.;nh.<ֺdY+"J¨ju6eHkn'qm{}^ {U#-hХR%ke^]xp߲֐ǚlTޕϬ[d!v|a}nO#C?|dm۵W!4ɡT{[o9(Yo|cjL2 NH3s" 5$n*۶ks^z9uve3ق 0 Bdlt(Bgw.ٿޡ#m~I⦦֑~RZ3{n;ϫv L?ȹ EGa]7+3D2sy}tTIE}w57O1s@D^isRh>WTxsvm}OkVq311ܒđi09;I^g_[r3w?qó"K==MI̎9$RF$Dٙuk.ۛvfQ$?֡iKG^rF Ns" 0Jz{Wn×\z /;ٹt\JYS=yGpG߲8}MXi6̸tըwOg}EB$"҈();/ri7Wa13"f'Oj@۵z bDZj`h Q\V+c||BHH0L!` !<"Ç՞*Ey4t#4$kf!88G2lϴfWo۳eu>3QU -㥖JXNn8ooE|s]Y:v?y'ID†gycT,;E aˏl9~\Lnx J3S=v ׿feo|liAUED$qrr`_gl[loϚ(f" 5n×J&ݽW&ryg@ n/ xM'yA[Y"+{w/3h2pGn{)g_WΞ?=h$q8;}" x`xRRm|_>"No^dOܸڮv|D)7V<}RƻŒ(a;/Bh,uxκW|ys #=/" ;}˂SG֙'ܕ6Ub,0X(Vǎk. ?Kz~²\ tb/o3ДYR8R_Mٿzz%^duhƭSE~rɌrα Vm9u7ox^#.`sg\AUSnF3Qc`6k" =V;bDQybOḏV^r+>oq$ENC `zf>3zVs\r105#a`Q.`-oq'TZՓ8(5sk|y:)O'W3J'3aΰY ( XRnnDFatSA[n yV\q'󶐲vc7,WB":oӛ}7W@z{< x&{:?]'8Qr jydzKBkú+>z|reC+eC AVErm-=C=]Gq2s@ > G/*]U ~x `Yo-pI/^?8 ,fSӧgZkb),lөn~Xll*vSJ 3೦U{6܌w>t/]Phjk[ڽ&*J 63L"2HZ i@J#B$BH.pZ w ׽缏$~*>őLL I$GĈ)=O7QmJeԤ4kBH`hx $*׽o7HFG.>jK;R,aEsɇРF:@"&( (9K`XucI{*BUQݍ@4; MD%(fp " "D2FNb<3_:GUhCyISºe!L`!8(4D QF`Ylڟc;S_{޹\*׷2)˥~/.-Q@ҬTD̎'_z_'gl@eona ޵YJeSNcX%C=zUI g/!W?GοvsOn]5 ˚Kl9e-0b愕+8 g>A-H^7}9P!lA)TV鼐 Vq^5f[Gy|@?Z ȍIENDB`weboob-1.2/contrib/plugin.video.videoobmc/resources/000077500000000000000000000000001303450110500226445ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/__init__.py000066400000000000000000000000571303450110500247570ustar00rootroot00000000000000# Dummy file to make this directory a package. weboob-1.2/contrib/plugin.video.videoobmc/resources/language/000077500000000000000000000000001303450110500244275ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/language/english/000077500000000000000000000000001303450110500260605ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/language/english/strings.xml000066400000000000000000000013751303450110500303010ustar00rootroot00000000000000 Search Search: Download Information Error! Information Download started Download succeed Download folder: Number of videos per backends: Display Non Safe For Work videos: Enable debug mode: weboob-1.2/contrib/plugin.video.videoobmc/resources/language/french/000077500000000000000000000000001303450110500256745ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/language/french/strings.xml000066400000000000000000000015121303450110500301060ustar00rootroot00000000000000 Recherche Recherche: Télécharger Information Erreur! Information Lancement du téléchargement Fichier téléchargé avec succès Répertoire de Téléchargement: Nombre de vidéos par backends: Afficher les vidéos interdites aux moins de 18 ans : Enable debug mode : weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/000077500000000000000000000000001303450110500234125ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/__init__.py000066400000000000000000000000571303450110500255250ustar00rootroot00000000000000# Dummy file to make this directory a package. weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/actions.py000066400000000000000000000075341303450110500254350ustar00rootroot00000000000000# -*- coding: utf-8 -*- from __future__ import print_function import sys from . import constants from .base.actions import BaseAction from .menu import MenuItem, MenuItemVideo, MenuItemPath from threading import Thread from .videoobmc import Videoobmc if hasattr(sys.modules["__main__"], "common_xbmc"): common_xbmc = sys.modules["__main__"].common_xbmc else: import common_xbmc class VideoobBaseAction(BaseAction): def __init__(self): count = common_xbmc.get_settings('nbVideoPerBackend') numbers = ["10", "25", "50", "100"] self.videoobmc = Videoobmc(count=numbers[int(count)], nsfw=common_xbmc.get_settings('nsfw')) class DisplayMenuAction(VideoobBaseAction): def _do(self, param={}): backends = self.videoobmc.backends if backends: MenuItem(common_xbmc.get_translation('30000'), constants.SEARCH).add_to_menu() for backend in backends: icon = self.videoobmc.get_backend_icon(backend) MenuItem(backend, constants.DISPLAY_BACKENDS, backend=backend, iconimage=icon).add_to_menu() common_xbmc.end_of_directory(False) else: common_xbmc.display_error(" Please install and configure weboob") class DisplayCollectionMenuAction(VideoobBaseAction): def _do(self, param={}): path = param.get('path') if 'path' in param.keys() else '' collections, videos = self.videoobmc.ls(param.get('backend'), path=path) threads = [] for col in collections: MenuItemPath(col).add_to_menu() for video in videos: aThread = Thread(target=self.add_videos, args=(video, video.backend)) threads.append(aThread) aThread.start() for t in threads: t.join() common_xbmc.end_of_directory(False) def add_videos(self, _video, backend): print(_video) video = self.videoobmc.get_video(_video, backend) if video: MenuItemVideo(video).add_to_menu() class DownloadAction(VideoobBaseAction): def _do(self, param={}): _id = param.get('id') backend = param.get('backend') if _id: aThread = Thread(target=self.download, args=(_id, backend)) aThread.start() common_xbmc.display_info(common_xbmc.get_translation('30301')) common_xbmc.end_of_directory(False) def download(self, _id, backend): dl_dir = common_xbmc.get_settings('downloadPath') self.videoobmc.download(_id, dl_dir if dl_dir else common_xbmc.get_addon_dir(), backend) common_xbmc.display_info(common_xbmc.get_translation('30302')) class SearchAction(VideoobBaseAction): def _do(self, param={}): pattern = common_xbmc.ask_user('', common_xbmc.get_translation('30001')) if pattern: for video in self.videoobmc.search(pattern, param.get('backend')): MenuItemVideo(video).add_to_menu() common_xbmc.end_of_directory(False) class DisplayBackendsAction(VideoobBaseAction): def _do(self, param={}): backend = param.get('backend') if backend: MenuItem('Search', constants.SEARCH, backend=backend).add_to_menu() DisplayCollectionMenuAction()._do(param) else: common_xbmc.end_of_directory(False) class UpdateWeboobAction(VideoobBaseAction): def _do(self, param={}): common_xbmc.display_info(common_xbmc.get_translation('30551')) self.videoobmc.update() common_xbmc.display_info(common_xbmc.get_translation('30552')) actions = {constants.DISPLAY_MENU: DisplayMenuAction, constants.DISPLAY_COLLECTION_MENU: DisplayCollectionMenuAction, constants.SEARCH: SearchAction, constants.DOWNLOAD: DownloadAction, constants.DISPLAY_BACKENDS: DisplayBackendsAction, constants.UPDATE: UpdateWeboobAction} weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/000077500000000000000000000000001303450110500243245ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/__init__.py000066400000000000000000000000571303450110500264370ustar00rootroot00000000000000# Dummy file to make this directory a package. weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/actions.py000066400000000000000000000005121303450110500263340ustar00rootroot00000000000000# -*- coding: utf-8 -*- from abc import ABCMeta, abstractmethod class BaseAction(): __metaclass__ = ABCMeta @abstractmethod def _do(self, param=None): """ Overload this method in application type subclass if you want to associate an action to the menu """ pass actions = {} weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/common_xbmc.py000066400000000000000000000110411303450110500271740ustar00rootroot00000000000000# -*- coding: utf-8 -*- from __future__ import print_function import xbmc import xbmcgui import xbmcplugin import xbmcaddon import urllib import sys from traceback import print_exc def get_addon(): if hasattr(sys.modules["__main__"], "addon_id"): _id = sys.modules["__main__"].addon_id return xbmcaddon.Addon(id=_id) def get_translation(key): addon = get_addon() if addon: return addon.getLocalizedString(int(key)) def get_settings(key): addon = get_addon() if addon: return addon.getSetting(key) def get_addon_dir(): addon = get_addon() if addon: addonDir = addon.getAddonInfo("path") else: addonDir = xbmc.translatePath("special://profile/addon_data/") return addonDir def display_error(msg): xbmc.executebuiltin("XBMC.Notification(%s, %s)" % (get_translation('30200').decode('utf-8'), msg)) print(msg) print_exc(msg) def display_info(msg): xbmc.executebuiltin("XBMC.Notification(%s, %s, 3000, DefaultFolder.png)" % (get_translation('30300').encode('utf-8'), msg.encode('utf-8'))) #print msg print_exc() def parse_params(param_str): param_dic = {} # Parameters are on the 3rd arg passed to the script param_str = sys.argv[2] if len(param_str) > 1: param_str = param_str.replace('?', '') # Ignore last char if it is a '/' if param_str[len(param_str) - 1] == '/': param_str = param_str[0:len(param_str) - 2] # Processing each parameter splited on '&' for param in param_str.split('&'): try: # Spliting couple key/value key, value = param.split('=') except: key = param value = '' key = urllib.unquote_plus(key) value = urllib.unquote_plus(value) # Filling dictionnary param_dic[key] = value return param_dic def ask_user(content, title): keyboard = xbmc.Keyboard(content, title) keyboard.doModal() if keyboard.isConfirmed() and keyboard.getText(): return keyboard.getText() return "" def create_param_url(param_dic, quote_plus=False): """ Create an plugin URL based on the key/value passed in a dictionary """ url = sys.argv[0] sep = '?' try: for param in param_dic: if quote_plus: url = url + sep + urllib.quote_plus(param) + '=' + urllib.quote_plus(param_dic[param]) else: url = "%s%s%s=%s" % (url, sep, param, param_dic[param]) sep = '&' except Exception as msg: display_error("create_param_url %s" % msg) url = None return url def create_list_item(name, itemInfoType="Video", itemInfoLabels=None, iconimage="DefaultFolder.png", c_items=None, isPlayable=False): lstItem = xbmcgui.ListItem(label=name, iconImage=iconimage, thumbnailImage=iconimage) if c_items: lstItem.addContextMenuItems(c_items, replaceItems=True) if itemInfoLabels: iLabels = itemInfoLabels else: iLabels = {"Title": name, } lstItem.setInfo(type=itemInfoType, infoLabels=iLabels) if isPlayable: lstItem.setProperty('IsPlayable', "true") return lstItem def add_menu_item(params={}): url = create_param_url(params) if params.get('name'): if params.get('iconimage'): lstItem = create_list_item(params.get('name'), iconimage=params.get('iconimage')) else: lstItem = create_list_item(params.get('name')) xbmcplugin.addDirectoryItem(handle=int(sys.argv[1]), url=url, listitem=lstItem, isFolder=True) else: display_error('add_menu_item : Fail to add item to menu') def add_menu_link(params={}): if params.get('name') and params.get('iconimage') and params.get('url') and \ params.get('itemInfoLabels') and params.get('c_items'): url = params.get('url') lstItem = create_list_item(params.get('name'), iconimage=params.get('iconimage'), itemInfoLabels=params.get('itemInfoLabels'), c_items=params.get('c_items'), isPlayable=True) xbmcplugin.addDirectoryItem(handle=int(sys.argv[1]), url=url, listitem=lstItem) else: display_error('add_menu_link : Fail to add item to menu') def end_of_directory(update=False): xbmcplugin.endOfDirectory(handle=int(sys.argv[1]), succeeded=True, updateListing=update) # , cacheToDisc=True) weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/menu.py000066400000000000000000000020541303450110500256430ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys if hasattr(sys.modules["__main__"], "common_xbmc"): common_xbmc = sys.modules["__main__"].common_xbmc else: import common_xbmc class BaseMenuItem(): def __init__(self, name, action, iconimage="DefaultFolder.png"): self.params = {} self.params['name'] = name self.params['action'] = action self.params['iconimage'] = iconimage def get(self, element): return self.params[element] def add_to_menu(self): common_xbmc.add_menu_item(self.params) class BaseMenuLink(BaseMenuItem): def __init__(self, name, url, action, iconimage="DefaultFolder.png"): BaseMenuItem.__init__(self, name, action, iconimage) self.params["url"] = url def createVideoContextMenu(self): return "" def create_info_labels(self): return "" def add_to_menu(self): self.params["itemInfoLabels"] = self.create_info_labels() self.params["c_items"] = self.createVideoContextMenu() common_xbmc.add_menu_link(self.params) weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/weboobmc.py000066400000000000000000000045161303450110500265010ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- import sys import os import re import subprocess import simplejson as json if hasattr(sys.modules["__main__"], "common_xbmc"): common_xbmc = sys.modules["__main__"].common_xbmc else: import common_xbmc class Weboobmc(): def __init__(self, count=10): self.count = count def update(self): #weboob-config update self._call_weboob('weboob-config', 'update') def _call_weboob(self, application, command, options={}, argument=""): if '-n' not in options.keys(): options['-n'] = self.count _opt = " ".join(["%s %s " % (k, v) for k, v in options.items()]) _cmd = "%s %s %s %s" % (application, _opt, command, argument) #print _cmd.encode('utf-8') return subprocess.check_output(_cmd, shell=True) def _json_call_weboob(self, application, command, options={}, argument=""): options['-f'] = 'json' try: result = self._call_weboob(application, command, options, argument) m = re.search(r"(\[{.+\}])", result) if m: result = u'%s' % m.group(1) #print result return json.loads(result) if result else [] except subprocess.CalledProcessError as e: common_xbmc.display_error(" Error while calling weboob : %s " % e) def get_loaded_backends(self, caps): #weboob-config list ICapVideo -f json backends = self._json_call_weboob('weboob-config', 'list', argument=caps) for backend in backends: if "_enabled=0" not in backend['Configuration']: yield backend['Name'] # , self.get_backend_icon(backend['Module']) def get_backend_icon(self, module): if 'WEBOOB_DATADIR' in os.environ: datadir = os.environ['WEBOOB_DATADIR'] elif 'WEBOOB_WORKDIR' in os.environ: datadir = os.environ['WEBOOB_WORKDIR'] else: datadir = os.path.join(os.environ.get('XDG_DATA_HOME', os.path.join(os.path.expanduser('~'), '.local', 'share') ), 'weboob') icons_dir = os.path.join(datadir, 'icons') return os.path.join(icons_dir, '%s.png' % module) def is_category(self, obj): return 'split_path' in obj.keys() weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/base/weboobmc2.py000066400000000000000000000054551303450110500265660ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- from __future__ import print_function from weboob.tools.application.base import Application import os import re import subprocess class Weboobmc(Application): def __init__(self, count=10): Application.__init__(self) self.count = int(count) self._is_default_count = False def update(self): self.weboob.update() def get_backend_icon(self, module): minfo = self.weboob.repositories.get_module_info(module) return self.weboob.repositories.get_module_icon_path(minfo) def obj_to_filename(self, obj, dest=None, default=None): """ This method can be used to get a filename from an object, using a mask filled by information of this object. All patterns are braces-enclosed, and are name of available fields in the object. :param obj: object type obj: BaseObject param dest: dest given by user (default None) type dest: str param default: default file mask (if not given, this is :'{id}-{title}.{ext}') type default: str rtype: str """ if default is None: default = '{id}-{title}.{ext}' if dest is None: dest = '.' if os.path.isdir(dest): dest = os.path.join(dest, default) def repl(m): field = m.group(1) if hasattr(obj, field): return re.sub('[?:/]', '-', '%s' % getattr(obj, field)) else: return m.group(0) return re.sub(r'\{(.+?)\}', repl, dest) def download_obj(self, obj, dest): def check_exec(executable): with open('/dev/null', 'w') as devnull: process = subprocess.Popen(['which', executable], stdout=devnull) if process.wait() != 0: print('Please install "%s"' % executable) return False return True dest = self.obj_to_filename(obj, dest) if obj.url.startswith('rtmp'): if not check_exec('rtmpdump'): return 1 args = ('rtmpdump', '-e', '-r', obj.url, '-o', dest) elif obj.url.startswith('mms'): if not check_exec('mimms'): return 1 args = ('mimms', '-r', obj.url, dest) elif u'm3u8' == obj.ext: _dest, _ = os.path.splitext(dest) dest = u'%s.%s' % (_dest, 'mp4') args = ('wget',) + tuple(line for line in self.read_url(obj.url) if not line.startswith('#')) + ('-O', dest) else: if check_exec('wget'): args = ('wget', '-c', obj.url, '-O', dest) elif check_exec('curl'): args = ('curl', '-C', '-', obj.url, '-o', dest) else: return 1 os.spawnlp(os.P_WAIT, args[0], *args) weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/constants.py000066400000000000000000000003341303450110500260000ustar00rootroot00000000000000# -*- coding: utf-8 -*- DISPLAY_MENU = "display_menu" DISPLAY_COLLECTION_MENU = "display_collection_menu" DISPLAY_BACKENDS = "display_backends" SEARCH = "search" VIDEO = "video" DOWNLOAD = "download" UPDATE = "update" weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/menu.py000066400000000000000000000047551303450110500247430ustar00rootroot00000000000000# -*- coding: utf-8 -*- import sys from . import constants from datetime import datetime, timedelta from .base.menu import BaseMenuItem, BaseMenuLink if hasattr(sys.modules["__main__"], "common_xbmc"): common_xbmc = sys.modules["__main__"].common_xbmc else: import common_xbmc class MenuItem(BaseMenuItem): params = {} def __init__(self, name, action, iconimage="DefaultFolder.png", backend=''): BaseMenuItem.__init__(self, name, action, iconimage) self.params['backend'] = backend class MenuItemPath(MenuItem): def __init__(self, collection, action=constants.DISPLAY_COLLECTION_MENU, iconimage="DefaultFolder.png"): MenuItem.__init__(self, collection.title, action, iconimage, collection.fullid.split('@')[-1]) self.params["path"] = '/'.join(collection.split_path) class MenuItemVideo(BaseMenuLink): def __init__(self, video, iconimage="DefaultFolder.png"): name = '[%s] %s' % (video.backend, video.title) BaseMenuLink.__init__(self, name, video.url, constants.VIDEO, video.thumbnail.url if video.thumbnail.url else iconimage) self.video = video self.params["id"] = self.video.id def createVideoContextMenu(self): cm = [] #Information cm.append((common_xbmc.get_translation('30110'), "XBMC.Action(Info)")) #Téléchargement url = "%s?action=%s&id=%s&backend=%s" % (sys.argv[0], constants.DOWNLOAD, self.video.id, self.video.backend) cm.append((common_xbmc.get_translation('30100'), "XBMC.PlayMedia(%s)" % (url))) return cm def create_info_labels(self): date, year = self.format_date(self.video.date) duration = 0 if self.video.duration: duration = u'%s' % str(self.video.duration.total_seconds()/60) if isinstance(self.video.duration, timedelta) else self.video.duration description = u'%s' % self.video.description return {"Title": self.video.title, "Year": year, "Plot": description, "PlotOutline": description[0:30] if len(description) > 30 else description, "Director": self.video.author if self.video.author else 'Unknown', "Duration": duration, "Date": date} def format_date(self, video_date): date = datetime.now().strftime("%d/%m/%Y") if video_date: date = video_date.strftime("%d/%m/%Y") year = date.split('/')[-1] return date, year weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/test/000077500000000000000000000000001303450110500243715ustar00rootroot00000000000000weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/test/__init__.py000066400000000000000000000000571303450110500265040ustar00rootroot00000000000000# Dummy file to make this directory a package. weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/test/common_test.py000066400000000000000000000054541303450110500273020ustar00rootroot00000000000000# -*- coding: utf-8 -*- from __future__ import print_function import urllib def get_addon(): pass def get_translation(key): translation = {'30000': 'Recherche', '30001': 'Recherche :', '30100': 'Télécharger', '30110': 'Information', '30200': 'Erreur!', '30300': 'Information', '30301': 'Lancement du téléchargement', '30302': 'Fichier téléchargé avec succès', '30551': 'Debut de la mise à jour', '30552': 'Weboob est maintenant à jour'} return translation.get(key) def get_addon_dir(): return '/home/benjamin' def get_settings(key): settings = {'downloadPath': get_addon_dir(), 'nbVideoPerBackend': '0', 'nsfw': 'False'} return settings.get(key) def display_error(error): print("%s: %s" % ("ERROR", error)) def display_info(msg): print("%s: %s" % ("INFO", msg)) def parse_params(paramStr): paramDic = {} # Parameters are on the 3rd arg passed to the script if len(paramStr) > 1: paramStr = paramStr.replace('?', '') # Ignore last char if it is a '/' if paramStr[len(paramStr) - 1] == '/': paramStr = paramStr[0:len(paramStr) - 2] # Processing each parameter splited on '&' for param in paramStr.split('&'): try: # Spliting couple key/value key, value = param.split('=') except: key = param value = '' key = urllib.unquote_plus(key) value = urllib.unquote_plus(value) # Filling dictionnary paramDic[key] = value return paramDic def ask_user(content, title): return raw_input(title) def create_param_url(paramsDic, quote_plus=False): #url = sys.argv[0] url = '' sep = '?' try: for param in paramsDic: if quote_plus: url = url + sep + urllib.quote_plus(param) + '=' + urllib.quote_plus(paramsDic[param]) else: url = "%s%s%s=%s" % (url, sep, param, paramsDic[param]) sep = '&' except Exception as msg: display_error("create_param_url %s" % msg) url = None return url def add_menu_item(params={}): print('%s => "%s"' % (params.get('name'), create_param_url(params))) def add_menu_link(params={}): print('[%s] %s (%s)' % (params.get('id'), params.get('name'), params.get('url'))) #print params.get('itemInfoLabels') #print params.get('c_items') def end_of_directory(update=False): print('******************************************************') def download_video(url, name, dir='./'): print('Downlaod a video %s from %s' % (name, url)) weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/videoobmc.py000066400000000000000000000071421303450110500257370ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- import time import re from datetime import timedelta, datetime from .base.weboobmc import Weboobmc from weboob.capabilities.video import BaseVideo from weboob.capabilities.image import BaseImage from weboob.capabilities.collection import Collection class Videoobmc(Weboobmc): def __init__(self, count=10, nsfw=False): Weboobmc.__init__(self, count=count) self.backends = list(self.get_loaded_backends('CapVideo')) _nsfw = 'on' if nsfw else 'off' self._call_weboob('videoob', 'nsfw', argument=_nsfw) def search(self, pattern, backend=''): #videoob search pattern -f json options = {'--select': 'id,title,date,description,author,duration,thumbnail,url'} if backend: options['-b'] = backend _videos = self._json_call_weboob('videoob', 'search', argument=pattern, options=options) if _videos: for _video in _videos: yield self.create_video_from_json(_video) def create_video_from_json(self, _video): video = BaseVideo() video.id = u'%s' % _video['id'] video.backend = u'%s' % _video['id'].split('@')[-1] if 'url' in _video.keys(): video.url = u'%s' % _video['url'] if 'thumbnail' in _video.keys() and _video['thumbnail'] and 'url' in _video['thumbnail'].keys(): video.thumbnail = BaseImage() video.thumbnail.url = u'%s' % _video['thumbnail']['url'] else: video.thumbnail.url = u'' video.title = u'%s' % _video['title'] if _video['date']: _date = re.search('(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}).*', _video['date']) try: datetime.strptime(_date.group(1), '%Y-%m-%d %H:%M:%S') except TypeError: datetime(*(time.strptime(_date.group(1), '%Y-%m-%d %H:%M:%S')[0:6])) video.description = u'%s' % _video['description'] video.author = u'%s' % _video['author'] if _video['duration']: _duration = _video['duration'].split(':') video.duration = timedelta(hours=int(_duration[0]), minutes=int(_duration[1]), seconds=int(_duration[2])) return video def get_video(self, video, backend): #videoob info _id -f json _video = self._json_call_weboob('videoob', 'info', argument=video.id) if _video and len(_video) > 0: return self.create_video_from_json(_video[0]) def ls(self, backend, path=''): options = {'-b': backend, '-n': 50} result = self._json_call_weboob('videoob', 'ls', options=options, argument=path) return self.separate_collections_and_videos(result) def separate_collections_and_videos(self, objs): videos = [] categories = [] for obj in objs: if self.is_category(obj): categories.append(self.create_category_from_json(obj)) else: video = BaseVideo() video.id = obj['id'].split('@')[0] video.backend = obj['id'].split('@')[-1] videos.append(video) return categories, videos def create_category_from_json(self, obj): collection = Collection(obj['split_path'].split('/')) collection.title = obj['title'] collection.id = obj['id'].split('@')[0] collection.backend = obj['id'].split('@')[1] return collection def download(self, _id, path, backend): #videoob download _id path options = {'-b': backend} self._call_weboob('videoob', 'download', options=options, argument=u'%s %s' % (_id, path)) weboob-1.2/contrib/plugin.video.videoobmc/resources/lib/videoobmc2.py000066400000000000000000000040511303450110500260150ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- from __future__ import print_function from .base.weboobmc2 import Weboobmc from weboob.capabilities.video import BaseVideo, CapVideo from weboob.capabilities.collection import CapCollection, Collection class Videoobmc(Weboobmc): def __init__(self, count=10, nsfw=False): Weboobmc.__init__(self, count=count) self.backends = self.weboob.load_backends(CapVideo) self.nsfw = nsfw def search(self, pattern, backend=''): kwargs = {'pattern': pattern, 'nsfw': self.nsfw, 'backends': backend} fields = ['id', 'title', 'date', 'description', 'author', 'duration', 'thumbnail', 'url'] try: for video in self.weboob.do(self._do_complete, self.count, fields, 'search_videos', **kwargs): yield video except Exception as e: print(e) def get_video(self, video, _backend): backend = self.weboob.get_backend(_backend) fields = ['id', 'title', 'date', 'description', 'author', 'duration', 'thumbnail', 'url'] return backend.fillobj(video, fields) def ls(self, backend, path=''): kwargs = {'split_path': path.split('/') if path else [], 'caps': CapCollection, 'objs': (BaseVideo, ), 'backends': backend} fields = [] # ['id', 'title', 'date', 'description', 'author', 'duration', 'thumbnail', 'url'] result = self.weboob.do(self._do_complete, self.count, fields, 'iter_resources', **kwargs) return self.separate_collections_and_videos(result) def separate_collections_and_videos(self, objs): videos = [] categories = [] for obj in objs: if isinstance(obj, Collection): categories.append(obj) else: videos.append(obj) return categories, videos def download(self, _id, dest, backend): for _video in self.weboob.do('get_video', _id, backends=backend): self.download_obj(_video, dest) weboob-1.2/contrib/plugin.video.videoobmc/resources/settings.xml000066400000000000000000000010471303450110500252300ustar00rootroot00000000000000 weboob-1.2/contrib/report_accounts.sh000077500000000000000000000030431303450110500200320ustar00rootroot00000000000000#!/bin/bash # Rapport des comptes bancaires # Liste l'intégralité des comptes dans Boobank, leurs dernières opérations et les opérations à venir # et envoie ça par mail si --mail machin@example.tld est spécifié, sinon ça affiche # (Compatible Debian Squeze) # Public domain SUBJECT="[Bank] Accounts report" TMPSTORAGE=`mktemp -d` boobank -q -f table list > ${TMPSTORAGE}/account_list ACCOUNTS=`awk '/@/ {print $2}' ${TMPSTORAGE}/account_list` for i in $ACCOUNTS do boobank -q -f table history "$i" > ${TMPSTORAGE}/account_history_${i} boobank -q -f table coming "$i" > ${TMPSTORAGE}/account_coming_${i} done printf "Bank accounts report, generated by boobank\n" > ${TMPSTORAGE}/account_mail cat ${TMPSTORAGE}/account_list >> ${TMPSTORAGE}/account_mail printf "\n" >> ${TMPSTORAGE}/account_mail for i in $ACCOUNTS do printf "Last operations for account $i \n" >> ${TMPSTORAGE}/account_mail cat ${TMPSTORAGE}/account_history_${i} >> ${TMPSTORAGE}/account_mail printf >> ${TMPSTORAGE}/account_mail if [ -s ${TMPSTORAGE}/account_coming_${i} ] then printf "Coming operations for account $i \n" >> ${TMPSTORAGE}/account_mail cat ${TMPSTORAGE}/account_coming_${i} >> ${TMPSTORAGE}/account_mail printf >> ${TMPSTORAGE}/account_mail else printf "No coming operation for $i \n" >> ${TMPSTORAGE}/account_mail fi done if [ "$1" = "--mail" ] then cat ${TMPSTORAGE}/account_mail | mail -s "$SUBJECT" -a "Content-type: text/plain; charset=UTF-8" "$2" else cat ${TMPSTORAGE}/account_mail fi rm -rf ${TMPSTORAGE} weboob-1.2/contrib/videoob_web/000077500000000000000000000000001303450110500165455ustar00rootroot00000000000000weboob-1.2/contrib/videoob_web/videoob-webserver000077500000000000000000000016221303450110500221250ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from videoob_web import VideoobWeb if __name__ == '__main__': VideoobWeb.run() weboob-1.2/contrib/videoob_web/videoob_web/000077500000000000000000000000001303450110500210315ustar00rootroot00000000000000weboob-1.2/contrib/videoob_web/videoob_web/__init__.py000066400000000000000000000014401303450110500231410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .videoob_web import VideoobWeb __all__ = ['VideoobWeb'] weboob-1.2/contrib/videoob_web/videoob_web/public/000077500000000000000000000000001303450110500223075ustar00rootroot00000000000000weboob-1.2/contrib/videoob_web/videoob_web/public/style.css000066400000000000000000000000761303450110500241640ustar00rootroot00000000000000.video-item { margin-bottom: 5ex; margin-left: 2em; } weboob-1.2/contrib/videoob_web/videoob_web/templates/000077500000000000000000000000001303450110500230275ustar00rootroot00000000000000weboob-1.2/contrib/videoob_web/videoob_web/templates/base.mako000066400000000000000000000006221303450110500246120ustar00rootroot00000000000000## -*- coding: utf-8 -*- <%def name="title()" filter="trim"> Videoob Web ${self.title()} ${next.css()} ${next.body()} weboob-1.2/contrib/videoob_web/videoob_web/templates/index.mako000066400000000000000000000017511303450110500250130ustar00rootroot00000000000000## -*- coding: utf-8 -*- <%inherit file="base.mako"/> <%def name="css()" filter="trim"> <%def name="video_item(item)"> <%def name="body()">

Videoob Web

% if merge: % for item in results: ${video_item(item)} % endfor % else: % for backend, items in sorted(results.iteritems()):

${backend}

% for item in items: ${video_item(item)} % endfor % endfor % endif
weboob-1.2/contrib/videoob_web/videoob_web/videoob_web.py000066400000000000000000000105421303450110500236710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os from mako.lookup import TemplateLookup from mako.runtime import Context from routes import Mapper from StringIO import StringIO from webob.dec import wsgify from webob import exc from wsgiref.simple_server import make_server from weboob.capabilities.video import CapVideo from weboob.tools.application.base import Application __all__ = ['VideoobWeb'] template_lookup = TemplateLookup(directories=[os.path.join(os.path.dirname(__file__), 'templates')], output_encoding='utf-8', encoding_errors='replace') class VideoobWeb(Application): APPNAME = 'videoob-webserver' VERSION = '0.j' COPYRIGHT = 'Copyright(C) 2010-2011 Christophe Benz' DESCRIPTION = 'WSGI web server application allowing to search for videos on various websites.' CAPS = CapVideo CONFIG = dict(host='localhost', port=8080) @wsgify def make_app(self, req): map = Mapper() map.connect('index', '/', method='index') results = map.routematch(environ=req.environ) if results: match, route = results req.urlvars = ((), match) kwargs = match.copy() method = kwargs.pop('method') return getattr(self, method)(req, **kwargs) else: public_path = os.path.join(os.path.dirname(__file__), 'public') if not os.path.exists(public_path): return exc.HTTPNotFound() path = req.path if path.startswith('/'): path = path[1:] public_file_path = os.path.join(public_path, path) if os.path.exists(public_file_path): if path.endswith('.css'): req.response.content_type = 'text/css' elif path.endswith('.js'): req.response.content_type = 'text/javascript' return open(public_file_path, 'r').read().strip() else: return exc.HTTPNotFound() def main(self, argv): self.load_config() self.weboob.load_backends(CapVideo) print('Web server created. Listening on http://%s:%s' % ( self.config.get('host'), int(self.config.get('port')))) srv = make_server(self.config.get('host'), int(self.config.get('port')), self.make_app) srv.serve_forever() def index(self, req): c = {} nsfw = req.params.get('nsfw') nsfw = False if not nsfw or nsfw == '0' else True q = req.params.get('q', u'') merge = req.params.get('merge') merge = False if not merge or merge == '0' else True c['merge'] = merge c['form_data'] = dict(q=q) c['results'] = [] if merge else {} if q: for backend in self.weboob.iter_backends(): videos = [dict(title=video.title, page_url=video.page_url, url=video.url if video.url else '/download?id=%s' % video.id, thumbnail_url=video.thumbnail.url, ) for video in backend.search_videos(pattern=q, nsfw=nsfw)] if videos: if merge: c['results'].extend(videos) else: c['results'][backend.name] = videos if merge: c['results'] = sorted(c['results'], key=lambda video: video['title'].lower()) template = template_lookup.get_template('index.mako') buf = StringIO() ctx = Context(buf, **c) template.render_context(ctx) return buf.getvalue().strip() weboob-1.2/contrib/windows-install/000077500000000000000000000000001303450110500174175ustar00rootroot00000000000000weboob-1.2/contrib/windows-install/HOW_TO.txt000066400000000000000000000010361303450110500212170ustar00rootroot000000000000001. Generate and Replace weboob egg file (./setup.py install --qt bdist_egg) 2. update settings.cmd 3. Download http://www.f2ko.de/downloads/Bat_To_Exe_Converter.zip extrat 32 and 64bit version and respectivly rename to Bat_To_Exe_Converter_x32.exe and Bat_To_Exe_Converter_x64.exe 4. Download http://nebm.ist.utl.pt/~glopes/wget/wget-1.11.4-x64.zip, unzip and rename to => wget-x64.exe 5. Download http://nebm.ist.utl.pt/~glopes/wget/wget-1.11.4-x86.zip, unzip and rename to => wget-x32.exe 6. run create-exe-setup-weboob.bat weboob-1.2/contrib/windows-install/ICON/000077500000000000000000000000001303450110500201475ustar00rootroot00000000000000weboob-1.2/contrib/windows-install/ICON/weboobtxt.ico000066400000000000000000013226261303450110500226740ustar00rootroot00000000000000 hf 00 %v@@ (B; (F} ( n(  @N%N/[MɆS-7@kJC=I:(!I:(1I:'7I:(UI:(CI:(Kэ[CŃQQSJ:ٖB=Q(-"/"U"."I:([I:(I:(mI:(iI:(cI:(ÃQUҎ\ǼKLy4ߒ?Rg0C"/""/";I9'EI:(?I:(aI:(_I:(II:(Kؓ`+ˈV̉WOSBERf2="/"_"/"ŃPЌYỶWsśWCCEEB uC ÂOS!s(2( @ o& C!N!L`K)\?IOT7]^+\*![(1\;|/96;K^ ]* ڔa3v=ƄQCd1AD[D\ZIm<3̉UuKwE֑^oŃQSF+8oYo%w+ӔABUM I:(I9'%I9(I:(+I:(I:(I:(I:([I:(I:(I:(ߙf%{I}Ր^yGݖds@Y9oS;^ӎ=Pa5%!/!"/""/""."KI:(I:(I:(I:(I:(I:(kI:(SI9(I:(I:(1I:(Ɖ\/~Loؒ`ɸxFMuGePLo1<RC2{}0}SCHOҎ['|IvY~EŃP%Y?[/_1T#|0 OyFЌY3|I+S! u* 5u* ^0_1(0` %D}YBzo%9!CA5s(O)_/_1_+\N b H)^7ZID]DWX9_\X% X%W$ x,]1TOu)t)z-u)AI_]W$n;3{H9j8?[)/u@}[m;_N7KW;YcUѪRs(ŨPwEU;#"/""/""/""."I9'I9(I:(MI:'UI9(I:' I:(I9(I9(I9(I9(I:(QI:(I9(I:(I9(I:(ƒP~Ci7o~uCȆSՑ^e@YQQ9Q{.^\ѭT{.\S3;*8%'"/""/"".""/""/"I:(+I:(I:(I9(I9(I:(!I9'/H9'H:('I:'1I9(I:(+I:(mH:'I:( I:(gA1~Gp>kzf4|m{9waM3Yy-LST6Qc?AC !/!w!/!"/" "/""/" I:(KH:'H:'H:'H:'H:(?I:(/I:'I:'I:(H9'I:(mI:'I:(I:(I:'ÂO~CuC}LŃQߩm;vz>fQ~0\r(ٮUt)끪3͡K;{M?3"."w!/!".""/""."H:(mH9(I:(I:(H9(I9(_I:(/I9(H9'sH:'kI:(qH9(mH:(I:(3H9(H:'t@}=sA~`.ki7n|Gı[[>oGBz-ߕAw+ϲW2W;|/!.!w!/!"/"5!.!!/!I9'I:(I9(I:(;H9'?H9'I:(H9(yI:(I:'oI9(sI:'I9( I9'H:'I:(oI9'I:(b/p3}K}j~LёY'N{{IU[YO~0]6aᗹCY}/\7Vu('"/"!/!"/"o"/""."I:(I:(yI:(H9';H9'AI:(I:(uI:(uI:(I:(I:(I:(}I:'I:(I:(I:(I:(gI9'֐]%nUxF~ΊXߘep?֡cOWC=k=^_^J3y]3Gc$)"/"".""/""/"e!.!q>z9{I͊Wng5|?D7^;1;YV}y-C]Z-p' ~Kԏ\'{Kq?sA~Me3ƄRy~9:!K+\A6up&o%2w]=O%i"e2 m1}OʇU{vD͉W{~Os3b/p&P+^9XGTK^=_+=k9l5~EO~K};L-rAo%F%]+_-^'Er)b0DžR1f9ޗd9|I5T"bx,6~0p'm< X&X&m< (@ BFYNo&~0!88}1 y.T {.#K'^'_'_%[K7qJS`Et)WMi!_]\ϻ^{.A]M1GM "/"q"/"".""/""/""/""/""/"!I:'I:(I9(I:(-I:(I:(I:(%I:(I:(I:'I:(3I:(H:(I:(CH:(I:(cI:(9I:(vS27}5Ր^ip>~}yY'w}ю[m;qdI}0LUf^v+[WOu*_UAE7"/""/""/""/""."eI:(I:(I:(I:(I:(I:'I:(EI:(I:(I:(}I:(I:( I:(AI:( H:(I:(H9'[I:(eI:(I:'Օc7{3Ր]iwE|JόU#xɨl:ЌZuor2hO;s>^eWx,V3ӻ^u*T_QA~0'"/""/""/"["/"_"/""/""/"-I:(I:(I:(I:(I9(I:(I:(eI:(I:(I:(I:(I:( H:(I:(I:(I:(I:(I:()i6t1h_r@j9xE֑_{IٸyG~wm2λcUP]z.^4ń5ߴYٚEi ߻^7Bq]=|//R"/""/""/""/""/""/"gI:(I9'iI:(I:(I:(7I9'I:(I:(I:(I:(%I:(I:(7H9(H:(I:(cI:(I:(I:(^+h-xOi7~}KѮq?{p?vDm9V_aOw+QWbጱ;ن6gݼ_?8{^=5/<"/""/"!.! "/""/"I:(#I:(I9(wI:('H:(I:(5I9(I:(I:(I:'I:(I:(?I:(?I:(H:(H9(+I:(I:(I:(?I:(QI:(I:(yЌX%~Cp>itOf4zG̉Wm;OceI?k3^I`Z炪3û^@5{_=:+/@$"/""/""."?"/"E"/""."I:(I9(I:(I:(I:(I:(I:(I:'H:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9'I9( uB}7΋XoxFג`ˍV%e3{g5̘ZWļ^GWKv*J_WI]_97s^9;%"0!"/""/""/""/""/""."-I:(I9( I:( I:(I:(I:( I:( I:(I:( I:( I:( I:( I:( I:' I9(I:( I:( I:( I:(a/ j/{Mj8g~q`.tISAV=J[s(Q___\{u*Ac_5:!*E"/" "/" "/" "/" "/"L!~;֑^mg5oyGk9~CʠY1D3_=>kq'CZ{X{4q&WI_-4b/ i/~GOc1NҎ\uC_-ЌYq~?ؕ`)z)#X-^?@gp&o%n%o%?c^;\'v+s@y5~M͉Wyf4_.a0N~S~9N']e 4#\-^=UO=g9mKW^?_/J!n& OL)|7~K{]lkue~W~Et1h5Y 6%Z+_5^;^=^9_/[#4U# L-y5~CKK~E|7}J3N W}/L'^'_)_'W!8n&P q?+ב^5u7z7i9|I9X']2G l$}0:!8y-k$c6Y(f4+i61d2)L^09rN K ^0^0^0( 9q;wN XYW:sB} aq)v-x.y.x.w-w, u);vZu*z,'~/)9'?#A!@<5z/z.=wdy,'2-G'[#_#_#_#_!_\P=}0 u,:ucy,'9+Y%_'_)_-_-_-_+_'_#_^Q: u+:oTx+%9+]%_+_/^3^5^7^7_5_1_-_'_!^\G s,=xs(4-[%_+_3^9_=_?_A_A_?_=_7_3_+_#_^T 7q_{-+O'_+_3^;_A_E_I^K[KXMZI^C_=^5_-_%^^[>xs':+^'_1^9^A_I]MI_3ww+u)v+}0s>]UE^7^/_%_^\Q y,)R'_-^7^A_IZS5yo%n$n$n$o%o%p&t*{;YZ;_/_#_^ ZPQ T!T!S!S!W$ q;7oh 4-^'^3^=_GZS~0n$m$l#l#m$o%n$n$p&q'z.kMA_-_!^] TY&\*^*']*-\)1Z(1W%/U"'])̆L]_y_k_]L[u)mx-SS%^ZU"tA/w+~5~AKS}[kiؓ`sho|c]WO~GϋYM]+e]+Mۗa4>+_-^=^Mz-l#F_^_PbYYZ[^a{/Y^s_c^U]g=`?^K}/k#S__>Wx,_ٻ_^_߼_ӻ^û^]{.fÎ=^c_SYE{/Sz.;Y[!/!A"/""/""/""/""/""/""/""/""/""/""/"!.!".""/" I:(I:(I:(I:(I:(I:(!I:(I:(I:(I:(I:(I:'I9(I:(wI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(CI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9([@!zE+y-}=~MˇUu`/g5}zb0͈R ̉Vqcg_`+|2shCcC_I5{k#K__@W1_ۼ__YWݻ^˼^_Sh!p&Zk_W_G%v+~;~O}K`.֑_kJvDwDfjÃNg5usid-|4jQfO_Qz-n%T__~1W혺Dۼ^۲XVTj!WJ__:j!Cw^S_AJ=x,;7\"/"="/""/"".""/""/""/""/"".""/""/""/""/"I:(I:(I9(I:(/I:(I:(I:(I:(I:(I:(KI:(I:(I9'I:()I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(WI:([0m:n+~9~KȅSwa/ΊXג_|HuBn= uCyDžOX'~}|ki5q-ҹeWhObQ:ul#B__K[m$^ӻ^۠J1银@U6˻__Ki!5_S^CX5x,?1Z"/"="/""/""/""/""/""/""/""/""/""/""/"I:(I:(I:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:()I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I9(I:(I:(I:'I:(I:(I:(I:(I:(Eh5ܖb+~7~Iב_ma0Mߘf}IT#r@sB}ͅPQ|~qq>i*XajOcOMan%|/^_\n%X헹C׻^Ѽ_ռ_ўHWw+ӻ__Vl#y,]U_E_5~1={.#M"/"="/""/""/""/""/""/""/""/""/""/""/" I:(I:(I:(I:(AI:(I:(I:(I:(I:(I:(I:(I:(I:()I:(I:(I:(iI:(qI:(I:(I:(I:(I:(}I:(-I9(I:(I:(I:(I9(KI:(I:(I:(I:(b/͊W+~3~Emad3r@sO[)ԏ]rwׁLOywxFc*LmkOeO\Sx,n$R__G[_Lѻ^˻^Ɉ8Xt)Ӽ__Zr'r(ZW^E^579y-'?"/"="/""/""/"S".""/""/"!/!"/""/"sI:(I:(I:(I:( I:(I:(I:(I:'CI:(gI:(I9(H:(%I:()I:(I:(I:(iI:(I:(I:(qI:(H:(I:(I:(I9(I:(I:(I:(uI:(I:(I:(+V$ }J+}/~A|Uk9d3{}c2zGڔbӎ\yFP}{}K_+DqսiQhM^O>ol#8__]z-Y\}0݁2[Y{/˼_^\v*o%WY^E^5>5y-): "/"="/""/""/"S"/" "/""/""/"I:' H:(I:(I9(I9(+I:(=I:(;I:'I:(GI:(I:(I:'9I:()I:(I:(I:(iI:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(I:(%i; t@%t-~=O{Ha/ەcэZ{HN̉WԐ]OxE\+~L_,Doȴ_WiK_KRYs(n%W__Yl#ZYYZ\啷A__]w+n$T[_E^5D1z,)4 "/"="/""/""/"S"/"#"/""/""/"I:(I:(sI:'I:(I:(I:(}I:(eI:( I:(I9'QI:'}I:(I:(I:(I:(I:(I:(UI:(I:(I:(I:(I:(}I:(}I:(}I:(}I:(}I:(I:(I:'I:(-I:(yI:(I:(I:(I:(}I:(}I:(}I:(I9(I:(I:(I:(I:(j7ߘd+~7~Kٔakb0yG}c1xFtBrAtByFxF{I^,GiXYiIaGWO;sl#6^__Us(][\q'϶Z__\u*n$SY_C_5H/u)%Zz' "/"[!/!"/""/""/""."}"."}"/""/""/""/""/""/"YH:(GI:(I:(I:(I:(I:(I:(I:'YI9('I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(5^+ƄQ+~1~CxYk8c1trŐY'yFwDxES"qtB_-…PeǪ[Sǹ_KcCUKVUt)m$M___\E4z̉V],d2yQ}=Ш_9FEN;]7^CQYs(m#5\{___^^__yUyr'n$6s^G^9_-E'p&8qi< k8!i+~7~Gy[k9`.{I~zh7^-q>}M~=o/@;>;]/_;_GFcp&l#~0V{^y_}_}_{^w\u~1m$o%I[^C^5^)@'m%8tX&}J-|/~?Oߘemd2_-{I{|I],`.̉WqK~;{+C-,;T)_1_=^IAgp%l$p&;Q{[uYuIy-m$n$y-^I^=_1_%8%h"c6k8!ߘe-~5~E~UˈUyb1_-m;ޗe{tzH],_-h5x[~I~9~)G'x#'7-])_5^?^ICeq'm$l#n%p&p&m$m$n$r'JY^C^9_-^!1%_ W$xE-x-~;~K}YŃQb0_-_-h6r@o=b1],^-a/͊WwS~E~7}'s@)ocs'#C+_+_7_?_IL[z-o%n$n$n$n$o%r'Ac^G^=_3_)R!y-S^3e2ˈU/~1~?~M~YΊX{e3`._-^,^,^-_-a/}K}]~O~A~3q'i7)mG T x,)M)_-_5_?_E[MCe}/}r'p&s(2wLW^G_?^7_-_#?#t+?vzG m:'e-~3~AO~YnkwEe3a/`/a/h6̉W}aUI~=~/͊V-d2!d7;u]|.+P'_+^3^;^A^G^I\MXO\M^G_C_=^7_/_'Y2!j" [2V#sA/p-~7~AMW~_siٓawю[}ݖduwi~_WO~C~7x-tB1Y'Z4;wf~0+P'_)_1^7^;^?_A^C^A_?^;^5_1_)_!A!y.I _4^+xE1s/~7~AKS~Y[]][UOG~=~1Ҏ[1l9-n= 3DžR7r1~3~7~;~;~;~7}3h5wD=m;1~I]28sK l$x,#~0'9'B%G#D#<#1#z/u+X^3Pj8+s@9ÂO9ܖb5r3y3r3ەb7L=q>?l:1~I^0 ;tQ j#u+y-y.!y.y.t+h Ey\3f7 Y(i8-o=9p=?o=Ap=Ap==n<5b/%l; ^0 ]/6p:tE|W R Bz8q\2^0d4 yEU$\*![)!Ql; _0 ^1^1]1^0 _0 ^0 ^0 _2`3( 6m8n:t9u=xI S Y X X [ [SL8n8t;v;x J _h"p)t,v-v.w-y-x+v*p) j& h#.c9r;u>z N e s,w.y/z.#y/#y/#y/#z/!z0y0y0{/|/{/y. v)6l:tz p'y++7/[%_)_/^5^9_?_C_G_M^QLau)o%n$m$m$m$l$l#l#l#l#m$m$m$n$o%o%p&p&q'r({v*qBOX=^5_/_)_!_^^] [OO T T"T#U#U"T"!R !P !QR V#e1 ڎVf8rL v*!z,/C)^%_+_1^7_=^A_G^K^OO]{.o%n$m$l$l#l#l#l#l#m#p&q'p&n$m$n$n$o%p&p'q'r({u*q8YT=_1_-_'_!_^^ \WO T!Z&Z)Z(!['%[''[()['+Z&+Y&+X$+V"+T'R#U#n: X=yF;N9DžR7ƄP7~J;t@Ak9Gb0M],O\+M\*I[)EY'AW%;V$3a.!e uO]+[\*W[(OY'GX'=o=t x?vk#y,+?+^'_-^5^;^A_I_OBkn$m$l#j"5\{__^__^^________^___}_y_s_m_g_a^[May-{r(ys(qt)e{/SM3_#_^^ ]Y3_3 Sj6)k85vC5n)})~/~3~7~9~;~?~A~ACEEECA~?~=~;~7w7{II`.[\)Y[)SY(KZ(=vB!t ui,|Ax n%x,-F)_'_-^5_=_C_K^Q3}m$l#k"p&J^}^^_______^__^____^__}_y_s_m_g__^YM_x,yr(wt(kt*a2MY)_!_^] Z\3g7 d0!l91p<7ܖb-})~-~3~7~;~=~AC~EG~I~I~I~IIG~E~C~A~=~9v9MI]+a[)[[(SY'I]+9ЌXz s3q?~ s'z,-K'^'_/^7_=_E_MYWx,m$l#k"|/W___^________^^^^___^__}_w^q_i_c_]]WCcv+ws(ot)eu+[>;]#__^ ]\3^1 Ml9)l:3ĂP1{'~+~1~7~;~?~AEG~KMMOOOOM~KIEC~A~;z9zHM^-a\*][)UY'Kd13py tNk ?~ r({--M'_)_/^7_?^E^MN_u)m#l#l#;^{_______^^[O6t)l$p%z-ÎդMɥNǗC|/i!^^`bdr(S^y_q_i^a_[_S^K;[u)gu*[v+M:5[__ W^3Y'p=)xD3|'~-~5~;~CIOU|[эZsf4^,],\+[*Z)Y(Y'Y'X'X&X&`.ÂP{mgc]YSKGΊWS^-q\*k[)_e0KO+F/;9S+_)_1^9_A^IXW2n$l#k"L____^_Zm$YWVU]퐴>״Yɼ^ǻ_ż__]P3d_ace1V^u_m_e^]^U^MPM|/ct)_u+S{.CO#^^ ]\3c0p=-ăP/})~/~7~?EKQ~Whik9_.^,]+\*Z)Z(Y'X&X&W&ÎW%W%W%[*€N{o~i~c_YQKuKo=g\+s\*g]*[z=;J3?=T/`+_1^9_A^IXW2n$l#l#O_____^LcXWU[<۹]˼_ͼ^˻_ɼ^ż_ü_^ZBj"`bdh Ë:\y_q_i___W_Q]I>Wv*cu+Wv+I;/^_]]0h6q>/ԏ\+~)~1~9~AGOU|[uC`._-^+\+Z)Z(X'X&ŎW&ōW%ǍV%ɌV$ǍV$ǍV$ōV$ÒZ)Myqie_~YQ~K֐^Sb0s\+o\*ec+WL7CAV5b/_1_9_A^IXU3n$k#m$P_____^=^XVV|/[ϼ_ϼ_Ѽ_Ѽ_ͼ_˼_Ǽ_ü__^Qn%abdj"F_s_k_c_[_U_KUI}/_u*[v+O|.=P_] Z]1m:!q?1f)~+~3~;~CIQWhic1`.^-]+[*Z)]+k9yG|JtBb0V%ˋT#ˋT#ˌU#ɌU$Ŗ]+ΊXoic]WQwMr@i\+w\*m_(e>GHCV9c1`3_9^A^IZU4}n$k#n$P_____^4ZWU\K׼_Ѽ_Ӽ_ռ_ռ^Ӽ_Ѽ_˼_ż____Jo%bdfv+V{_o_e___W^O^G?Su*_v+Sx,C>)\^ ^I:(+I:(I:(I:(I:(I:(I9'I:(H9(?I:(I:(I:(I:(I:(I:(I9( I:(I:(KI9'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(kI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I9'KI:(I:([2`4 p=#tA1n)~-~5~=~C~KS}[{Ha/_.^,\*\*j8ٔav||yqˈVa/ËU#ϊT"͋T#ˌU$ǖ^,o}~o~ic~]~W~OەbU_-{\*s](mr/[HIU?e5a7^9^?^G[S6{n$k#m$O__^__\|.YVTm$]Ѽ_ռ_ټ_ټ_ۼ_ټ_ռ_Ѽ^˼_ż_^^]Hk#ceh!9\s_i_c_[_Q_ITGz.]u*Ww+I25R^]"/" "/"]"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""."U".""/" I:(QI:(I:(I:(I:(I:(I:(I:(#I:(I:({I:(I:(I:(I:(I:(I:(I:(I:(+I9(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(gI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(wI:(`4o= q=%xD/p)~-~5~=~EMUmek9a/_-]+]+o=s~po=V$ωS!ъT"͋T#ɫn=v{oic]U}Mn!]]"/"".""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"I:( I:(I:(I:(I:(I:(I:(I:(I:(I:(SI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(AI:(I:(;I:'EI:(I:(I:(I:(I:(I:(H:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EI:(II:(I:(I:(I:(I:(I:(I:(I:(!I:(?H:(EH:(I:(I:(I:(I:(I:(H:(EH:(EH:(EH:(EH:(EH:(EH:(EH:(EI:(GI:(SI:(sI:(I:(I:(I:(I:(I:(I:(I9&)O r='M-u)}1}9}A~I~Qqao>b/_.^,zH}ʇUU$׆PׇR ә_-xyoiaYٓ`ac0`'k*7oμeGkCfCbE_G^MKat)l#k"7_}_^__[z-YVXR׼^ټ_߼___^\퟿I뜽GXۻ^ϼ_Ǽ__^__Vo%fg r'Ow_e_[_S^K_CY?{/Sw+Mw,?3)W^ "/""/""/"="/"E"/""/""/""/""/""/""/"E"/"E"/"E"/"E"/"E"/"E"/"E"/"E"/"G"/"Q"/"o"/""/""/""/""/""/""/"!/!/I:(!I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(EI:(I:(I:(I:(I:(I:(I:(I:(I:(eI:(OI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(KI:(I:(I:(I:(I:(I9'g~J r='ÁM-u)}1~9AK~Slej8a/_-_-ەc{o>DŽN݆P׊T"DŽR~umg]yYm;}^(i)z0¯YQjEhEcG`G_MUYv+l#k"|/^}_^__^2YVUQ׼^׼_߼___Xu*I J X>߶[ͼ_ü_____@fgj"9\i^]_W_M_E_=ݻ^ռ^ݼ__]x,A|DHK N ]Fѻ^__^_\8g h m#F}_a_Y_Q^I_AV=2Mv+Ix,94%T]"/"y"/""/""/""/""/"".""/""/""/""/"".""/"I:(cI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(GI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(OI:(I:(I:(I:(I:(I:(=I:(I:(I:(I:(IH9(I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(I:(I9(/I:(I:(I:(I:(I:(I9'{n? r>%L-t)}1~9CKSޗdke3a/^-tBwsit~΋YON߇Q ׼|J}}ukc|]g4`(n+7}϶cSkKgKdK`M_OBmm$l#n$E_^___OcWU{.^ջ_ۼ__]i!A|DGJ N QfZ_____Ko%h j!3_c_[_S_I_A\;=Ew,Kx,;|/-I^"/"y"/""/""/""/""/""/""/""/""/""/""/""/"I:(yI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(#I:(I9(I9'I:(I:(I:(I:(I9(I:(I:(I:(I:'I:(I:(OI:(I:(I:(I:(I:(I9'I:(I:(I:(I:'I:(I:(I9(I:(sI:(I:(I:(I:'iH9'I:(I:(I:(I:(I:(I:(I:( I:(I:(I:(I:(I:(I9(I9'Sb5q=#}I/u)}1}9AKSޗcke2`/^-xFyhâg5ߏX&b0ϋY~l[*ۂMOݥi7w~yog}_wD{_(l*|2éZ[mMhMeMaM_OP_n%l#l#:^____Zl#XVh \Ӽ_ټ^_]n$B}DGK N QT?ϼ___^_Yz-h i!q'[e^[^S_K_C^;F?y-Kx+=y,1A\ Z"/"y"/""/""/""/""/"".""/""/""/""/""/""/""."eI:( I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:([I:( I:(I:(I:(I:(I:(I:'I:(I:(I:(I:(H:(I:(I:(OI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(-I9' I:(5I:(SI:('I:(I:(I:(I:(I:(I:(I:(I:(I:( I:(H9';I:'I:(I:(I:(I:(I:(I:(I9&]2oqn$l#m$F^_^__UeXVq'VӼ_׼_ݼ_Yu)K K ]CWSW큩2Ӹ\____\q'h!j!{.Zc_W_O_G_=_7??w,Cw,7z-'N]"/"y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"I:('I:(I:(I:(I:(I:(I:(aI9'I:(I:(I:(I:(I:'I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:'I:(iI:(OI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(1I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9'WI:(\3c1s?-ܖb)}-5~?~IQpcn m< OxmŋT#~JMk:~wowgq>_)k*|2ҵdYlQjOeOcO`QKct)l#k"6]_^__\2ZW[C׻^Ѽ^׼_ۻ^߮Ul#St)Sn%TVn$ݯU____^4h j!t)Nk_W_O_G^?^7M7x,Ey,9z,)@["/"y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"".""/"e"/" H9(1I:(I:(I:(I:(H:(H9'GI:(I:(I:(I:(I:(I:(I:(I:(I:(H:(I:(eI:(I9(H:(I:(I:(I:(I:(I:(OI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(1I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(3X-\)s?)ЍY+~+}3~=~G~Ot_sAb1`.o=tvC~IzGvDsAn> k; p> txōV$~JL`/~{~q|iyF_)g*u.ĬZ_lQjOgOdOaQTY}/m$k#s(Y____^EbXVq'\ͼ^Ӽ_׼_ټ_ۯUߒ?Q۷[~1TVaP_____Ai j!p&Es_W_O_I^?_7X3y.Ex,9y,+9Z"/"y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"AI:(=H:(I:(I:(I:(I:(I:(+I:(WI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(5I:(kI:(I:(I:(H:(I:(I:(OI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(1I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(H9')U1T"q?'ńP-~)~1~;E~Ox[yGb1`/i7r|J~JzGzFNp?j9l; s{ɏX&|HKY(|}u~k‚O`+d)q,NkmQjQhOdObQ\U9wo%l#k"J___^^Vr(YWXEջ^ͼ_Ѽ_ջ_׻^׻_ռ_Ӻ]υ6VWZ頿I_____Kl#j!m#={^Y_Q^I_A_9^12Aw,;z+-5R"/"y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/";I:(]I:(I:(I:(I:(I:(I:( H9'!I:(I:(I:(I:(I:(I:(I:(H:'I:(I:( I:(SI9'I:(I:(I:(I:(I:(I:(OI:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(1I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(Q n;!zG/|'~1~;~EM}YăP{d2`/b0jˈV̀K{G}IuCQh8Jx{͏W&{G~JV%uwmϋY{a-a)m+EuֻhUkQiOfObO^SJcr(l#k"3]}_____C_XWfOѻ^ͼ_Ѽ_Ѽ_Ѽ^Ѽ_Ϲ]˃4VXZ長B_^^^_Qr'j!l#5Z]_Q_I_A_9_19=y+;y,-1G "/"y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"!/!'I:({I:(I:(I:(I:(I9(I:( I9(I:(I:(I:(I:(I:(I:(I:(I:(iI:(?I:'I:(I:(I:(I9(I9'I:(OI:(I:(I:(I:(I:(H:(I:(I:(I:(I:(I:(I:(/I:(H9'#H9(I:(I:(I:(I:(I:(I:(I:(I:'I:()I:(UI:(I:(I:(I:(I:(I9(I:(I9( zH l8vC/u)~/~9~C~K~UЌYsf4a/_.Ր]ܕcdžQ{GyF|JϋYb0O}uъT"zG}JU$ryqؒ`yb/_)i+@{͵bYlQjOgOcO_OVY|.m$l#t(Q_____Zp&ZXXv*Vͼ_˼_ͼ_ͼ_ͼ_˴Yv*WXZ鑵?_____Ux+j!l#|0W]_Q_I_A^9^1A7x,=x-/|/> "/"y"/""/""/""/""/"!.!"/" !.!'"."Q"/"!/!"/""/""/""/""/""/" I:(I:(I:(I:(I:(I:(I:'_I:(I:(I:(I:(I:(I:(I:(I:(%I:()I:(I:(I:(I:(I:(I:'/I:(OI:(I:(I:(I:(I:(I:( I9(I:(I:(I:(I:(I:(!I:( I9(gI9(I:(I:(I:(I:(I:(I:(I:(I:(I:(H:(I:(I:(I:(I:(I:(I9(I:(1g: g4tA-g+~-~7~AISfij8a0_.MnÖ],|HxEf4vw|gمOzF}IU$t}~sfwc0_*e*;ű\]ڿlQkOhOdM`OYU?on$l#m#>]__^__F_ZWYu*Wɺ^Ǽ_ɼ_ɻ^ǠJ^WY[甶A__^__W}/j!l#w+T__Q_I^A^9^1H3x-=x,/z.= "/"y"/""/""/""/""/""/""/""/""/""/""/""/""."AI9'I:(I:(I:(I:(H9(I:('I:(I:(I:(I:(I:(I:(I:'I:(I:(I:(I:(I:(I:(I:(?I:(OI:(I:(I:(I:(I:(I9(kI:(I:(I:(I:(oI:(I9(OI:(I:(I:(I9(I:(qI:(I:(I:(I:(I:(I:(H:'-I:(I:(I:(I:(I:(I9(OY1`-r?)ЍY-~+~5~=G~Qv_p>a0`.m<}yq?~JyF~Jڔb~ÁO|HyF}IV$y~ukwc1^+a*}8XaջhSlMiMeMbMYSP_u*m$k"y-T_____Z2]YXWk"咶@էO˩Q˚EgWXZ[圼G_____Z3j!l#s)Ra_Q_I^A^9^1N/{/;x-1z.:"/"y"/""/""/""/""/""/"#"/""/""/""/""/""/"kI:(I:(I:(I:(I:(I:(qI:'I:(I:'I:(I:(I:(I:(I9(]I:(I9(I:(I:(I:(I:(I:(OI:(OI:(I:(I:(I:(I:(I:( I:(UI:(}I9(WI:(H:(uI9(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:'I:(I:(I:(I:(H:(I:(Y^3V% p<%ĂP/z)~3=EO~Y{Ib0`/d3p~̉VφPzGvDc1xuc2vDyF|I[)}~wnwd2^+`+u5Wa̵bWnMjMfKbKZQUY7yn$l#m$C^}_^__^Pn$[YXW\gi _XXY[aN_____[5j!l#p&Pc^Q_I^A^9^1Q/~09y,1{.8E"/"y"/""/""/""/""/""/""/""/""/""/""/""/"wI:( I:(I:(I:(I:(I:(I:(MI:(3I:'wI:(yI:(yI:(yI:(uI:'I9(I:(I:(I:(I:(I:(aI:(OI:(I:(I:(I:(I:(H9(I9(I:(I:(}I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(SrA n:{H/p)~/~9~CMUג_oc1a/a0͊Xt`-|HxExFm;o}~xwFwEvCyF|Ih7~yowd2^,`+n3ūZ_X]nMkKfKcKZOWUHet)m#k"w+[}_____^Kd[ZYXXXXYYZ\p&ѯU_____\6j"l#o%Me^O^I_A^9^1T-37y,1|,!5;"/"y"/""/""/""/""/""/""/""/""/""/""/""/"oI:()I:(I:(I:(I:(I:(I:(/I9({I:(I:(I:(I:(I:(yI9'I:(OI:(I:(I:(I:(I:(I:(}I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(}I:(I:(I:(I:(I:(I:(?^4g5vB-ߙe+}-~7~?~ISw_g5b0`.vDyɆTJzGwDvCZ(yG~Lh6uBsAvCzF|I}K~{o{d2^,_,k3̪^YT_һgQlKhIdI[MVSWU3}n$l#l"D^}___^_^@e[ZZYYYZ[[_߇7\_____[}6k"m#o%Je_O^I_A^9^1W+65x,1|,!23"/"y"/""/""/""/""/""/""/"m"/""/""/""/""/""/"SI:(EI:'I:(I:(I:(I9(I:'%I:(]I:(I:(I:(I:(I:(I:( I:(OI:(I:(I:(I:(I:(I:(}I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(H9(I9'GI:(I:(I:(I:(I:(I:(I:(I9(]0\)s?+ьY-|+~5~=~GQ}[tBb0`.i7jw_.|HyFuCsAtAsBp@r@tBwDzGLݖd~}l}c1^,_-j3өaUW]³[UlIiGeG]KTQ[QHep&m#k"{.V____^_[@d\[[[[[\\k#էO^^__^_Z}4k"m#o%Ie_O_G_A_9_1Y+83y,1{-!~11"/"y"/""/""/""/""/""/""/""/"C"/""/""/""/""/""/"".""/"#I:(I9(I:(I:'I:(I:(I:(I:(I:(I:(I:(I:(I9(I:('I:(I:(cI:(I:(I:(I:(I:(I:(I:(I:(I:(H:(I:(I:(;I9( I:(sI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(H:(I:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(]wF p=%}J/w)~1~9~C~M~Uҍ[sd3a/`.DžS}~ەc͏X&{HxEvDtBsAsAtBvCxE|Hd2v}ߘfc1^,_-k6ުiOǫ[WUYѽfKiGgE]ITOZM]S3m$l#l#;\{_^_^^^^Op&a]\\]^g נJ^______X}~0k"m$o%Jc_M_G_?_9_1[);1x,/`g(8""/""/""/"".""/""/""/""/"".""."".""."".""."".""/""/""/""/""/""/""/""/"".""/"qI:(uI9'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(H9(I:(I:(5I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(UI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(uI:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:( k< i7tA/m)~/7~AI~Skgm;b0_.m;w|ȆTՆP{GyFvDvCvCvDxEzGOʇU}~}Ր^a0_,_-l9oMЪ`QSYõ\OjEgC^ESMYK^MIcs(m$l#s(N^}__^^__^U=u*h cj"|/ǟI^_^_^^__}T}y,l#m$o&La_M_E^?^7_1[)<1w,/]=s "/!Y"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"".""/"I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(eI:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(H:'H9(!f6^,q>+эZ+~+~3~?GOz[}Jc1a/`.֑_{OلO{HyFxEwExEzF~Jp>wʇU_.^-_.p={tMکgKUUSSgEhA`CRKXG^KYU5{n$l#k"1\{_^__^^__]WQNRY^_____^__{Pr(l#n$q&O]^K^E^=_7_/\)=/x,-_\{__^___________^___^__}_wGm#l#n$u)RY_K_E^=^5_/\'=-y+-a;z "/!e!/!"/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/""/"I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I9(I:(3I:( I9(}I:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(QI:( I:'I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:'%I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(I:(H9(I:(I9(I:(sI:(5I:(sAi5tB/h+~-~7~AISz]tBb0`/b0ЌZ~~qˈVٵvDp>zGՐ^xysA]+^-`.|Jq}IrEѨaESOOOg=a?PGUC]C_I]Q=on$m$l#s)I^{_____^^____^________{_u9l#l#n$y-TW_I_C^;_5_-\'<-z++a)ʇT-{)~3~;EO~Wڔboe3a0`.n=p|z|mi7]+_-`.ʆTiIyCܨgAXGMMYCƾ`=PCU?\?^E_KYUz-n$l#l#w+R^{___________^______}_w[uv*l#m$o%2{XS_G^A_;_3_-[%;-y++`=u-J"/""/"!.!#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#"/"#!.!!"/""/""."]3uC n:!yG/o+~/~7~AKSx_uCb1`/`.~L}Ր]_-]+_-`.ޗd_I~Ao=ͧ^?OGLGY=MAT;\;_A_G^MIcu)m$l#k"|/T}^y^___^____^______}_y_uGm$l$n$o%:o\M_G^?^9_1_+Z%:+y,)_;v^4c1r?-֒^+}+~5~=~GO~WՐ^qh6a/_.b1f~~zH\+^,_.d2qYIAw;اe9S?GGLCL?S9\7^=_C_I[Q;qp&m$l#m#3U{^y_}_________^___}_{_uZu2l#m#n$r'Be^K_C_=^7_1_)X%8+y+'\:sY2O o<%zG1x)~/~9CK~Sy_{Ib1`/_-l:m~ua/]+^,`.o=xUIA}9n5ȥZ5IA=IF?S5\3^7_?_C_KZS4{o%m$l#l#}/T{^w_{_}___^__^___}_y_w]sAp&m#n$o%x,OY^I_A_=^5_/_'V#5+y-#V 7qp? f4q?/ޗc+~+~5~?~GOWkij8b0`._.uBt~~ƄR\+^,_-a.}J{|SIA7v3Ԧa/L58G9CQ1[/_3_9_?_G_MSYz-o$m$l#l#|/K]w_y_{_}_}____^}^{_y_u^sN}v*l#m$n$o%3w[O_G_A_9_3_-_'S#1+y-!M 7u^5Y&n;'L/y)~1~9~A~IS~[DžS}c1a/_.`.zH}mh7]+^,_.d2эZm}Q~I}A}7{1j+O-7A0GE1[+_1_5_=^C_I^MHcw+n$m$l#l#t)B]u^w_w_y_y_{_y_y_y^w^u^qSwz.l$m$n$o%p&Ca^I^C^?^9_1^+_%Q#}/)x.B} :sn> g4r@/ؓ`-}+~5~=~E~OUwaxFc1a/_-_-zGt~ttB]+],_-`.m;naO~G~A}7}-v'ȝW'27,C29T)_+_1_7_=_C_I^MHav*n$m$l$l#l#~0HXw^u_u_u_u^u_s^sYuCv*l$m$m$o$p%y-\M_G^A_;^5_/_)_#H#z-)u,/ٓ`+~-~3~;CKS}[ڔaqf5a/`/^-_-k9ٓay~~uN_-\+],_-`.f4ggUMG}?~5-%ښe#o2-~u&+w&/|/3R'^)_/^5^9_?_E_I\OEcw+o%n$m$l#l#l#n$q't)u*u)r(o%m#l#m$m$n$o%p&3uUQ_G^C^=^7_1_+_%^7'z.#l$:u[3U"n;'wD1w)~/~7~?~EMW~]͊W{g5a0`/_-^,`.tBٓas}~u֑_n=]+\+^,_-`.a/{H~[S~K~C=~3~+|%֑]#k3+l$!j!w*-7/W'_)_/_5^9_?_E_I^MN]{.p%n$n$m$m#m$l$l$l$l$l$m$m$m$n$n%o%p&y-}MW^I_C^?^;^5_/_)_#[0'z.a 6oqA e2o=/ЌY/}+~1~;~AIQWxaƄQe3a/`/_-^,],c1ob0\+Õ]+],^-_.a.i7hk~YQK~C~;~3~+}%džR'g2)c"Tp r%!{./@+\'_+_1^7_;_?_E^I_MP[4wq&o%n$n$m$m$m$m$m$m$m$m$n$o%o%p&t)F]]K_G_A_;_7_3_-_'_!O{-%x.Z 1e]4Qm9#vC1g+}-~5~=~C~KQ~YzaʈU}f3b0a/_-_-^,]+]+Ø_-Ø_-^-Ö],Õ\+Õ]+],^,^-_.`.f3ˈV{|_UOI~A~91~+y#wD+g6%Y!I^ Ww*'|//G)^'_+_1_7_;_?_C_G^KZQDew+q&o%o$o$o$o$o$o$o$o%o%p%p'y-}G_]K_G_A_=^9^3_/_+_%^>#z-#t+Qa4]*o<-~K1u)~/~7~=EMUY}a΋X{k9b0a/`._-^-^,],],],],],^,^-_-_.`/b0Nyc~[SME~?~7~/~)p%l:/h5#~LKP>u gx+)~11P'^'_+_1^5^;_?_C_G_K^MSW1эZ-~+~1~9A~GMUY{aݖdso=b0a/`.`._-^-^-^-^-^-_-_.`.`/c0yGwe]W~QK~C=~7~/~'Տ[)j8/e4rA U:;tI n$x,+6/T'_'_-_1_7_9_=_A_E_G_K]ORWCe6sx,s(q'q's(z-}:mH]XO_I^G_C_?^=^7_3_-_)^%_I~/%y/f :r]4R m:'s@3f-}-~3~;~AG~O~SY~_vgăPk9c1a/a/`/`.`.`.`.a/a/b0i7ȆSxg_YUM~I~A~;~5~-z'zF1j8+^-i; U3:k1N1x)~/~5~=~C~IOSW~]_{eomܕcwŃPyFtBuC{IΊXjswi~c_[WSMGC~=~5~1z)DŽQ/m;3g4#|GZ39u=x ^w*#z--<-X%_'_+_/_3^5^9^;^=_?_A^C^C_C_E_E^C_C_A_A_?^=^9^7^5_1_-_)_%_]8#z.!z/T0n^3|Gk8%o<3ɆS1y+~/~7~;~A~GM~QUY]~_a~eeeeeec~a~][~WSOK~E~?~9~3~-j+tA5l:-_,e5 Y57r?y _w)#z,-8-U%^%^)_-_1_3_7^7^9_;^=_?^?^?^?^?^=_=_=^9^7^5_3_/_-_)_%_!]E|/#z0r&;vW0c4 V$l:'r?5ҍZ1y+~1~7~;~A~G~KOSWY~]__aaa_]]~YWS~OKG~A~;~5~1}+ƄP3m;5l9)L]1 Z39r?z bx*%y,-8+S%^%_'_+_/_1^3^5^7^9^9_;_;_;_;^9^9^7^5^3_1_/_+_)^%_!^P3#z0y-] Y3g8 \*n;+s@5ҍZ/{-~1~7~=~AEIMQUWWY[[[~YYWUSOKG~C~=~9~3~-j-q>9n;1f2!c3 \29sA| cw*#z,+4-K%]#_'_)_+_/_1_1^3^5^5^5^7^5^5^5^3_1_/_-_+_'_%_!^W7#z/!z/n% Du\2c4 a,n;-p>7ӎ[1y-~1~5~9~?~CG~KMQ~SSUUUSSSOMIGC?~9~5~/r-|I9o;7m:+O^/ [29s?z ]u*!y,)|/-C'Y#^#^'_)_+_-_-_/_/_1_1_1_/_/_/_-_+_)_%_#_!_W:#z.#z0v*O]1p?c0n<-r@7ϋX1y+}/~5~9~=AEGI~M~MO~Q~QO~O~MKI~GA?~;~7~1y-ɇT7p>;o<1d1!k: ^3[47r=y Zt*z,'{.-9+P#^!_#_'_'_)_+_-_-^-_-_+_+_)_)_'^%_#_^V:#{/#y0z.\ 4jY(\3n= b/n

7ÂO5t-}/~3~7;~?~AEGG~IIK~IIG~GC~A~=~;~7~1{/ЍY5q>?o=7i8'}I_1 ^35o'O#[!_!_!_!_#_#_#_!_!__]N=#/%z.!y0z-e Dx^1^0o> ^+n;-o=7uB;΋X5x-~/~3~5~7~;~=~=??~?~?~=~;~9~7~5~1z/͊W9tAAo>;o=3b0#e5^0 ]55p9uC~ Zp(z.#z,'{,+~1+7+C%Q![_^_^]WI!:%0'{-%y/!z0z-h ?za1`0_1 Rl9)o=3o=;wE;Ր]5r/}/~1~3~5~7~7~7~7~7~7~5~3~3y1h5}I?p=Cp=;o=3b0#m< ^/ \37r;uAz `q+x-z,#z+'z,)y,+|.+1)4)5'3'0)|-'{+'z,%y/!y1z.t(Y =s\5]3`2 Le2#n<1o<9p>=|I;ЌY7j3|/~/~1~3~33~3~1~1{1i5ƄP=vBCp=Ap=;o>5`/%p> ^/^1 `57o8s=wL aq)x.y.!z.#z-#y-%{,%z,%z-%{-#z.!y/z0z/v*i! L~ 7m]6]2_2 n= X'l:+o=3o=;q>=tB?|H?ɇT9ڔa5l3s1s1l3ܔb7ɆT=zHAuACq=Cp>?p=9o<3]-#l; ^/]1 a19p8r>wI U g!s*w.w/x0x0x0y/z.w,p'a O@w]5]1 _0 sA^,k9+o=3o=7o==o=?o=Cp9p=5i6)Na3^0]0 ]/1i6p8r;u;w?z R ^ b ] W K @v9r6q2dZ8]3 ^0 a1|HZ)f5)n=1p>5p=7p=;p=;o==p==p=;p=;q>9p=5g5+Y'!r@_0^/ ]1 _2:n4j7n9q9r>8,0,0, 22,0,0,0)) #bitmap-size*4,22B-offset*4 #image BITMAPINFOHEADER(ref5) a.extend((40,0,0,0)) #size of data(contains header)*4 a.extend((size,0,0,0, size*2,0,0,0))#w*4, (h+h)*4 (!shit hack! XOR+AND) a.extend((1,0, bpp,0, 0,0,0,0, 0,0,0,0)) #1 plane*2, 24 bits*2, no compress*4, rawBMPsize(no compress so 0)*4 a.extend((0,1,0,0, 0,1,0,0)) #horizontal*4/vertical*4 resolution pixels/meter(WTF?) a.extend((0,0,0,0, 0,0,0,0)) #colors fully used, all are important #!no planes #image content(ref1&5), XOR+AND bitmaps AND = array('B') vand = 0 vcnt = 0 #remember that bitmap format is reversed in y-axis for x in range(size-1,-1,-1): for y in range(0, size): b,g,r,t_or_a = data[y*size+x] a.extend((r,g,b)) if bpp == 32: a.append(t_or_a) vcnt+=1 vand<<=1 if (bpp==24 and t_or_a) or (bpp==32 and t_or_a<128): vand |= 1 if vcnt==8: AND.append(vand) vcnt=0 vand=0 #!hack! AND bits align to 32 bits per line, !shit! MSDN says nothing about this AND.extend([0] * ((32-size%32)%32/8)) a.extend(AND) return a # x,y indicate the hotspot position # simply set the type/hotspot(x&y) after generates the icon def gencur(data, size=16, bpp=24, x=0, y=0): a = genico(data, size, bpp) a[3], a[10], a[12] = 2, x, y return a #C:\Python27\Lib\site-packages\weboob-0.g-py2.7.egg\share\icons\hicolor\64x64\apps if __name__ == "__main__": if len(sys.argv) == 2: png_file = sys.argv[1] f, e = os.path.splitext(png_file) ico_file = f + ".ico" im = Image.open(r"%s" % png_file) wh = 64 rgba = im.convert("RGBA") data = [] for i in range(wh): for j in range(wh): data.append(rgba.getpixel((i,j))) icoflow = genico(data, wh, 32) _file = open(ico_file, "wb") icoflow.tofile(_file) _file.close() weboob-1.2/contrib/windows-install/create-exe-setup-weboob.bat000066400000000000000000000005611303450110500245440ustar00rootroot00000000000000@echo off call settings.cmd "Bat_To_Exe_Converter_%LOCAL_ARCHITECTURE%.exe" -bat "setup-weboob.bat" -save "setup-weboob-%WEBOOB_VERSION%-%ARCHITECTURE%.exe" -icon "ICON\weboobtxt.ico" -include "Bat_To_Exe_Converter_%ARCHITECTURE%.exe" -include "wget-%ARCHITECTURE%.exe" -include "%WEBOOB%" -include "convertPNG2ICO.py" -include "ez_setup.py" -include "settings.cmd" weboob-1.2/contrib/windows-install/ez_setup.py000066400000000000000000000271251303450110500216360ustar00rootroot00000000000000#!python """Bootstrap setuptools installation If you want to use setuptools in your package's setup.py, just include this file in the same directory with it, and add this to the top of your setup.py:: from ez_setup import use_setuptools use_setuptools() If you want to require a specific version of setuptools, set a download mirror, or use an alternate download directory, you can do so by supplying the appropriate options to ``use_setuptools()``. This file can also be run as a script to install or upgrade setuptools. """ import os import shutil import sys import tempfile import tarfile import optparse import subprocess import platform from distutils import log try: from site import USER_SITE except ImportError: USER_SITE = None DEFAULT_VERSION = "1.1.6" DEFAULT_URL = "https://pypi.python.org/packages/source/s/setuptools/" def _python_cmd(*args): args = (sys.executable,) + args return subprocess.call(args) == 0 def _check_call_py24(cmd, *args, **kwargs): res = subprocess.call(cmd, *args, **kwargs) class CalledProcessError(Exception): pass if not res == 0: msg = "Command '%s' return non-zero exit status %d" % (cmd, res) raise CalledProcessError(msg) vars(subprocess).setdefault('check_call', _check_call_py24) def _install(tarball, install_args=()): # extracting the tarball tmpdir = tempfile.mkdtemp() log.warn('Extracting in %s', tmpdir) old_wd = os.getcwd() try: os.chdir(tmpdir) tar = tarfile.open(tarball) _extractall(tar) tar.close() # going in the directory subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) os.chdir(subdir) log.warn('Now working in %s', subdir) # installing log.warn('Installing Setuptools') if not _python_cmd('setup.py', 'install', *install_args): log.warn('Something went wrong during the installation.') log.warn('See the error message above.') # exitcode will be 2 return 2 finally: os.chdir(old_wd) shutil.rmtree(tmpdir) def _build_egg(egg, tarball, to_dir): # extracting the tarball tmpdir = tempfile.mkdtemp() log.warn('Extracting in %s', tmpdir) old_wd = os.getcwd() try: os.chdir(tmpdir) tar = tarfile.open(tarball) _extractall(tar) tar.close() # going in the directory subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0]) os.chdir(subdir) log.warn('Now working in %s', subdir) # building an egg log.warn('Building a Setuptools egg in %s', to_dir) _python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir) finally: os.chdir(old_wd) shutil.rmtree(tmpdir) # returning the result log.warn(egg) if not os.path.exists(egg): raise IOError('Could not build the egg.') def _do_download(version, download_base, to_dir, download_delay): egg = os.path.join(to_dir, 'setuptools-%s-py%d.%d.egg' % (version, sys.version_info[0], sys.version_info[1])) if not os.path.exists(egg): tarball = download_setuptools(version, download_base, to_dir, download_delay) _build_egg(egg, tarball, to_dir) sys.path.insert(0, egg) # Remove previously-imported pkg_resources if present (see # https://bitbucket.org/pypa/setuptools/pull-request/7/ for details). if 'pkg_resources' in sys.modules: del sys.modules['pkg_resources'] import setuptools setuptools.bootstrap_install_from = egg def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, download_delay=15): # making sure we use the absolute path to_dir = os.path.abspath(to_dir) was_imported = 'pkg_resources' in sys.modules or \ 'setuptools' in sys.modules try: import pkg_resources except ImportError: return _do_download(version, download_base, to_dir, download_delay) try: pkg_resources.require("setuptools>=" + version) return except pkg_resources.VersionConflict: e = sys.exc_info()[1] if was_imported: sys.stderr.write( "The required version of setuptools (>=%s) is not available,\n" "and can't be installed while this script is running. Please\n" "install a more recent version first, using\n" "'easy_install -U setuptools'." "\n\n(Currently using %r)\n" % (version, e.args[0])) sys.exit(2) else: del pkg_resources, sys.modules['pkg_resources'] # reload ok return _do_download(version, download_base, to_dir, download_delay) except pkg_resources.DistributionNotFound: # NOQA return _do_download(version, download_base, to_dir, download_delay) def download_file_powershell(url, target): """ Download the file at url to target using Powershell (which will validate trust). Raise an exception if the command cannot complete. """ target = os.path.abspath(target) cmd = [ 'powershell', '-Command', "(new-object System.Net.WebClient).DownloadFile(%(url)r, %(target)r)" % vars(), ] subprocess.check_call(cmd) def has_powershell(): if platform.system() != 'Windows': return False cmd = ['powershell', '-Command', 'echo test'] devnull = open(os.path.devnull, 'wb') try: try: subprocess.check_call(cmd, stdout=devnull, stderr=devnull) except: return False finally: devnull.close() return True download_file_powershell.viable = has_powershell def download_file_curl(url, target): cmd = ['curl', url, '--silent', '--output', target] subprocess.check_call(cmd) def has_curl(): cmd = ['curl', '--version'] devnull = open(os.path.devnull, 'wb') try: try: subprocess.check_call(cmd, stdout=devnull, stderr=devnull) except: return False finally: devnull.close() return True download_file_curl.viable = has_curl def download_file_wget(url, target): cmd = ['wget', url, '--quiet', '--output-document', target] subprocess.check_call(cmd) def has_wget(): cmd = ['wget', '--version'] devnull = open(os.path.devnull, 'wb') try: try: subprocess.check_call(cmd, stdout=devnull, stderr=devnull) except: return False finally: devnull.close() return False download_file_wget.viable = has_wget def download_file_insecure(url, target): """ Use Python to download the file, even though it cannot authenticate the connection. """ try: from urllib.request import urlopen except ImportError: from urllib2 import urlopen src = dst = None try: src = urlopen(url) # Read/write all in one block, so we don't create a corrupt file # if the download is interrupted. data = src.read() dst = open(target, "wb") dst.write(data) finally: if src: src.close() if dst: dst.close() download_file_insecure.viable = lambda: True def get_best_downloader(): downloaders = [ download_file_powershell, download_file_curl, download_file_wget, download_file_insecure, ] for dl in downloaders: if dl.viable(): return dl def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, delay=15, downloader_factory=get_best_downloader): """Download setuptools from a specified location and return its filename `version` should be a valid setuptools version number that is available as an egg for download under the `download_base` URL (which should end with a '/'). `to_dir` is the directory where the egg will be downloaded. `delay` is the number of seconds to pause before an actual download attempt. ``downloader_factory`` should be a function taking no arguments and returning a function for downloading a URL to a target. """ # making sure we use the absolute path to_dir = os.path.abspath(to_dir) tgz_name = "setuptools-%s.tar.gz" % version url = download_base + tgz_name saveto = os.path.join(to_dir, tgz_name) if not os.path.exists(saveto): # Avoid repeated downloads log.warn("Downloading %s", url) downloader = downloader_factory() downloader(url, saveto) return os.path.realpath(saveto) def _extractall(self, path=".", members=None): """Extract all members from the archive to the current working directory and set owner, modification time and permissions on directories afterwards. `path' specifies a different directory to extract to. `members' is optional and must be a subset of the list returned by getmembers(). """ import copy import operator from tarfile import ExtractError directories = [] if members is None: members = self for tarinfo in members: if tarinfo.isdir(): # Extract directories with a safe mode. directories.append(tarinfo) tarinfo = copy.copy(tarinfo) tarinfo.mode = 448 # decimal for oct 0700 self.extract(tarinfo, path) # Reverse sort directories. if sys.version_info < (2, 4): def sorter(dir1, dir2): return cmp(dir1.name, dir2.name) directories.sort(sorter) directories.reverse() else: directories.sort(key=operator.attrgetter('name'), reverse=True) # Set correct owner, mtime and filemode on directories. for tarinfo in directories: dirpath = os.path.join(path, tarinfo.name) try: self.chown(tarinfo, dirpath) self.utime(tarinfo, dirpath) self.chmod(tarinfo, dirpath) except ExtractError: e = sys.exc_info()[1] if self.errorlevel > 1: raise else: self._dbg(1, "tarfile: %s" % e) def _build_install_args(options): """ Build the arguments to 'python setup.py install' on the setuptools package """ install_args = [] if options.user_install: if sys.version_info < (2, 6): log.warn("--user requires Python 2.6 or later") raise SystemExit(1) install_args.append('--user') return install_args def _parse_args(): """ Parse the command line for options """ parser = optparse.OptionParser() parser.add_option( '--user', dest='user_install', action='store_true', default=False, help='install in user site package (requires Python 2.6 or later)') parser.add_option( '--download-base', dest='download_base', metavar="URL", default=DEFAULT_URL, help='alternative URL from where to download the setuptools package') parser.add_option( '--insecure', dest='downloader_factory', action='store_const', const=lambda: download_file_insecure, default=get_best_downloader, help='Use internal, non-validating downloader' ) options, args = parser.parse_args() # positional arguments are ignored return options def main(version=DEFAULT_VERSION): """Install or upgrade setuptools and EasyInstall""" options = _parse_args() tarball = download_setuptools(download_base=options.download_base, downloader_factory=options.downloader_factory) return _install(tarball, _build_install_args(options)) if __name__ == '__main__': sys.exit(main()) weboob-1.2/contrib/windows-install/settings.cmd000066400000000000000000000005211303450110500217420ustar00rootroot00000000000000set WEBOOB_VERSION=1.2 set WEBOOB=weboob-1.2-py2.7.egg set LIST_APPLIQUATIONS_QT=qboobmsg qcineoob qcookboob qflatboob qhandjoob qhavedate qvideoob qwebcontentedit weboob-config-qt qbooblyrics REM x32 | x64 set ARCHITECTURE=x64 set LOCAL_ARCHITECTURE=x64 REM version required for WEBOOB set PYTHON_VERSION=2.7 set PYTHON_MINOR_VERSION=6 weboob-1.2/contrib/windows-install/setup-weboob.bat000066400000000000000000000126101303450110500225220ustar00rootroot00000000000000@echo off setlocal enableextensions enabledelayedexpansion call settings.cmd echo. echo 0.Set proxy set/P HTTP_PROXY=Enter HTTP_PROXY if needed : set/P HTTPS_PROXY=Enter HTTPS_PROXY if needed : echo. echo 1.GNU/WGET Init set WGET=wget-%ARCHITECTURE%.exe echo. echo 2.Check Python Installation rem check first possible key set KEY_NAME=HKLM\Software\Python\PythonCore\2.7\InstallPath if %ARCHITECTURE% == x64 ( set KEY_NAME=HKLM\SOFTWARE\Python\PythonCore\2.7\InstallPath ) set IsPythonInstalled=0 REG QUERY !KEY_NAME! >NUL 2>NUL if %ERRORLEVEL% EQU 0 ( set IsPythonInstalled=1 ) else ( rem first key doesn't exist, test the second possible key set KEY_NAME=HKCU\Software\Python\PythonCore\2.7\InstallPath REG QUERY !KEY_NAME! >NUL 2>NUL if %ERRORLEVEL% EQU 0 ( set IsPythonInstalled=1 ) ) if %IsPythonInstalled% EQU 1 ( rem check installed python version for /F "tokens=4" %%A IN ('REG QUERY !KEY_NAME!') do ( set PythonPath=%%A ) !PythonPath!python.exe --version 2>&1 | find /i "!PYTHON_VERSION!" > tmp.txt if %ERRORLEVEL% EQU 1 ( set IsPythonInstalled=0 ) else ( FOR /F "eol=; tokens=2 delims=." %%i in (tmp.txt) do set minor_version=%%i if !minor_version! LSS !PYTHON_MINOR_VERSION! ( set IsPythonInstalled=0 ) ) del tmp.txt ) if %IsPythonInstalled% EQU 0 ( rem Python is not installed set PYTHON_MSI=python-!PYTHON_VERSION!.msi if %ARCHITECTURE% == x64 ( set PYTHON_MSI=python-!PYTHON_VERSION!.amd64.msi ) echo 2.1 Download !PYTHON_MSI! "%WGET%" -o python_donwload --no-check-certificate "http://www.python.org/ftp/python/!PYTHON_VERSION!/!PYTHON_MSI!" echo 2.2 Setup !PYTHON_MSI! !PYTHON_MSI! del !PYTHON_MSI! del python_donwload ) echo Retrieve Python path rem check first possible key set KEY_NAME=HKLM\Software\Python\PythonCore\2.7\InstallPath if %ARCHITECTURE% == x64 ( set KEY_NAME=HKLM\SOFTWARE\Python\PythonCore\2.7\InstallPath ) REG QUERY !KEY_NAME! >NUL 2>NUL if %ERRORLEVEL% EQU 1 ( rem first key doesn't exist, test the second possible key set KEY_NAME=HKCU\Software\Python\PythonCore\2.7\InstallPath ) for /F "tokens=4" %%A IN ('REG QUERY !KEY_NAME!') do ( set PythonPath=%%A ) echo. echo 3.Check EasyInstall if exist "%PythonPath%Scripts\easy_install.exe" ( goto :step4 ) else ( echo 3.1 Setup setuptools %PythonPath%python.exe ez_setup.py || goto :InstallFailed del setuptools-1.1.6.tar.gz goto :step4 ) :step4 echo. echo 4.PyQt5 Installation %PythonPath%Scripts\easy_install.exe python-qt5 || goto :InstallFailed echo. echo 5.Check Gpg4win Installation set ShouldReboot=0 set KEY_NAME=HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\GPG4Win REG QUERY %KEY_NAME% > nul 2>NUL || ( echo 5.1 Download Gpg4win "%WGET%" -o gpg4win_download http://files.gpg4win.org/gpg4win-2.2.2.exe echo 5.2 Setup Gpg4win gpg4win-2.2.2.exe set ShouldReboot=1 del gpg4win-2.2.2.exe del gpg4win_download ) echo. echo 6.Install Weboob Dependances echo. echo -- cssselect %PythonPath%Scripts\easy_install.exe cssselect || goto :InstallFailed echo. echo -- lxml %PythonPath%Scripts\easy_install.exe lxml==3.2.5 || goto :InstallFailed echo. echo -- dateutils %PythonPath%Scripts\easy_install.exe dateutils || goto :InstallFailed echo. echo -- pyyaml %PythonPath%Scripts\easy_install.exe pyyaml || goto :InstallFailed echo. echo -- html2text %PythonPath%Scripts\easy_install.exe html2text || goto :InstallFailed echo. echo -- mechanize %PythonPath%Scripts\easy_install.exe mechanize || goto :InstallFailed echo. echo -- google-api-python-client %PythonPath%Scripts\easy_install.exe google-api-python-client || goto :InstallFailed echo. echo -- feedparser %PythonPath%Scripts\easy_install.exe feedparser || goto :InstallFailed echo. echo -- pillow %PythonPath%Scripts\easy_install.exe pillow==2.3.0 || goto :InstallFailed echo. echo -- requests %PythonPath%Scripts\easy_install.exe requests==2.3.0 || goto :InstallFailed echo. echo 6.Install WeBoob %PythonPath%Scripts\easy_install.exe %WEBOOB% || goto :InstallFailed set StartupFolder=%AppData%\Microsoft\Windows\Start Menu\Programs if exist "%StartupFolder%" Goto :FoundStartup set StartupFolder=%UserProfile%\Start Menu\Programs if exist "%StartupFolder%" Goto :FoundStartup echo Cannot find Startup folder. echo do not create launchers goto :InstallSucceed :FoundStartup if exist "%StartupFolder%\Weboob" ( goto :CreateLauncher ) else ( md "%StartupFolder%\Weboob" goto :CreateLauncher ) :CreateLauncher for %%i in (%LIST_APPLIQUATIONS_QT%) do ( echo Process %%i ( echo @echo off echo start %PythonPath%pythonw.exe %PythonPath%Scripts\%%i ) > %%i.bat %PythonPath%python.exe convertPNG2ICO.py "%PythonPath%\Lib\site-packages\%WEBOOB%\share\icons\hicolor\64x64\apps\%%i.png" > nul if exist "%StartupFolder%\Weboob\%%i.exe" ( del "%StartupFolder%\Weboob\%%i.exe" ) "Bat_To_Exe_Converter_%ARCHITECTURE%.exe" -bat "%%i.bat" -save "%StartupFolder%\Weboob\%%i.exe" -icon "%PythonPath%\Lib\site-packages\%WEBOOB%\share\icons\hicolor\64x64\apps\%%i.ico" "%%i" del "%%i.bat" del "%PythonPath%\Lib\site-packages\%WEBOOB%\share\icons\hicolor\64x64\apps\%%i.ico" ) goto :InstallSucceed :InstallSucceed echo. echo INSTALLATION PROCESS SUCCEED if %ShouldReboot% EQU 1 ( echo. echo YOU SHOULD REBOOT BEFORE USING WEBOOB ) goto :Quit :InstallFailed echo. echo INSTALLATION PROCESS FAILED goto :Quit :Quit del "%WEBOOB%" del ez_setup.py del convertPNG2ICO.py del settings.cmd pause weboob-1.2/desktop/000077500000000000000000000000001303450110500142725ustar00rootroot00000000000000weboob-1.2/desktop/masstransit.desktop000066400000000000000000000003711303450110500202360ustar00rootroot00000000000000[Desktop Entry] Name=Masstransit Comment=Search for train stations and departure times Exec=masstransit Icon=masstransit Terminal=false Type=Application StartupNotify=true Categories=Network;GTK; Keywords=departures;train;stations;timetable;travel; weboob-1.2/desktop/qboobmsg.desktop000066400000000000000000000003331303450110500174750ustar00rootroot00000000000000[Desktop Entry] Name=QBoobmsg Comment=Send and receive messages from various websites Exec=qboobmsg Icon=qboobmsg Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=messaging;chat;forum; weboob-1.2/desktop/qcineoob.desktop000066400000000000000000000004361303450110500174670ustar00rootroot00000000000000[Desktop Entry] Name=QCineoob Comment=Search for movies, persons, torrents and subtitles on many websites, and get info about them Exec=qcineoob Icon=qcineoob Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=video;movie;cinema;subtitles;torrent;media; weboob-1.2/desktop/qcookboob.desktop000066400000000000000000000003551303450110500176460ustar00rootroot00000000000000[Desktop Entry] Name=QCookboob Comment=Search for recipes on many websites, and get info about them Exec=qcookboob Icon=qcookboob Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=recipes;eating;cooking; weboob-1.2/desktop/qflatboob.desktop000066400000000000000000000003071303450110500176360ustar00rootroot00000000000000[Desktop Entry] Name=QFlatBoob Comment=Search housings Exec=qflatboob Icon=qflatboob Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=housing;flat;apartment;rental; weboob-1.2/desktop/qhandjoob.desktop000066400000000000000000000002741303450110500176350ustar00rootroot00000000000000[Desktop Entry] Name=QHandJoob Comment=Search for jobs Exec=qhandjoob Icon=qhandjoob Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=job;offers;working; weboob-1.2/desktop/qhavedate.desktop000066400000000000000000000004101303450110500176220ustar00rootroot00000000000000[Desktop Entry] Name=QHaveDate Comment=Optimize your probabilities to have sex on dating websites Exec=qhavedate Icon=qhavedate Terminal=false Type=Application StartupNotify=true Categories=Network;Chat;ContactManagement;Qt; Keywords=dating;sex;girls;boys;tinder; weboob-1.2/desktop/qvideoob.desktop000066400000000000000000000003611303450110500174750ustar00rootroot00000000000000[Desktop Entry] Name=QVideoob Comment=Search for videos on many websites, and get info about them Exec=qvideoob Icon=qvideoob Terminal=false Type=Application StartupNotify=true Categories=AudioVideo;Qt; Keywords=movies;video;youtube;player; weboob-1.2/desktop/qwebcontentedit.desktop000066400000000000000000000003251303450110500210640ustar00rootroot00000000000000[Desktop Entry] Name=QWebContentEdit Comment=Edit website contents Exec=qwebcontentedit Icon=qwebcontentedit Terminal=false Type=Application StartupNotify=true Categories=Network;Qt; Keywords=wikipedia;wiki;edit; weboob-1.2/desktop/weboob-config-qt.desktop000066400000000000000000000003521303450110500210270ustar00rootroot00000000000000[Desktop Entry] Name=Weboob backends configuration Comment=Configure Weboob backends Exec=weboob-config-qt Icon=weboob-config-qt Terminal=false Type=Application StartupNotify=true Categories=Utility;Qt; Keywords=configuration;weboob; weboob-1.2/docs/000077500000000000000000000000001303450110500135515ustar00rootroot00000000000000weboob-1.2/docs/Makefile000066400000000000000000000060721303450110500152160ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Weboob.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Weboob.qhc" latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ "run these through (pdf)latex." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." weboob-1.2/docs/make.bat000066400000000000000000000060031303450110500151550ustar00rootroot00000000000000@ECHO OFF REM Command file for Sphinx documentation set SPHINXBUILD=sphinx-build set BUILDDIR=build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. changes to make an overview over all changed/added/deprecated items echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\Weboob.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\Weboob.ghc goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) :end weboob-1.2/docs/source/000077500000000000000000000000001303450110500150515ustar00rootroot00000000000000weboob-1.2/docs/source/_static/000077500000000000000000000000001303450110500164775ustar00rootroot00000000000000weboob-1.2/docs/source/_static/architecture.png000066400000000000000000002700731303450110500217000ustar00rootroot00000000000000PNG  IHDRݗXsRGBbKGDC pHYs B(xtIME 'UtEXtCommentCreated with GIMPW IDATxyս7zYw1(q"F*H/(kzEܒhDn4ƛMb47QDw{(`T *n( 3{/UmUw(33uϩ.xs,ԄcbjI +J&L`0}i̞=4o8cƌ1Ӂ>@#]u| "kƏK@DDDwjjjN7 V˲NPe҇|ljT[3H1aYjU= 缚 DDDDB4 `FUY˲^˲z@)r>?Ow\la, )^m^"""*Ti"rt&f9+Jݴi\صk!;w|0H<uuuuK,QVV֭|+"dʋȥ5 Z/Dѩp}^^"""6FDl?~ZCCHUpϦ1heegp뭷ʉ'˔IC]]M^z+KDDD!b e! [5=HĉߊRkU7=v7eY=/*"?2 ѺxϲF"n0SglP(,YgwjPU+6r36t @ӿKz[K׉Up8Q[[=J-P;2t挹w{Ϯ:AU۷ַUlٲjEP("""Jb'a h]&l4 c͸q=ٳŲ{Ty6cƏiO i`[/08 yLp˴iӮ:u>u) wbf͚3 pxY 0NC2y:"KCT'N\ LR[/j3;-0g&4ju^a/uZ555>\D×Uu\$iF?3[nFkl_U-\ 555S Øf[)A4Fы<~|g(|;H ܉k ny&eYb#-c*z7as]mmi"~G֧~[QQK$y4M+(/~F 8GUsR/H$Ҧkjjƈȥ"r2apEǖe}jisyYeơy9oKUE <wwC4͓/&bzU:CEp|8NQ7үMD֫ rYYWWK.JloKDDD1NDn|H$vN4*|۶m#o/;c3 㗪:*ZCuP(qp)&l8w @KzX ˲2EdN)WU/ 0jm." v|'"""VWW{?8b_0D7`9T*wQUCP̄l=e+d򟯽Zo- [h(B1=2NہT*~U&"; p:_U_NRoڵkɓ7dYWWV cC=4ja_UD"qI3f|t=?0Bx .,'a矿GשiYU/E떿-(:= +! `7=twשi2SXȝVLܐJ.^4gD"Wkjj7n]wu͛6m:BDƪiQU2狋XJzowlߗþA$O(g~:= 9A4EOe RU&LNT ukIED6m۵kךio~sEss{sVչ>4hжh4zA9EUl}a/?_\t%"WC Ø{L2<IJWD䯪Og6v@s+HefBېLyuv5_r$]4'8S,zUULƵSr'pu=z' Bg[5RD=?TuhgKD'h]4'yy@3N_BǤR"J澵,K2!m3pZLBkӛLNv=GꧦiaN=a/ڤ$]%C407=/p)`ӭ^">}`إ4CӿnVsτ]{O[=D8R뒯~Ne2)M,˂=:0m[Tu'|#G ]DFvkjjht)@8l _GZ^fC&7|mLm- /ЊBm7\c&lE_^W_TuarQ0~j XeT՗>NhnH &>ԻoxNWJ erðS]m[l؆9t,k5kX,V&"GDZ5 /jjjw`p۲e28I]v=݆bVMjq㏿ `'3xD>JR w#?b3kKDDDeYSUkŶe|=8B;;._=Z?/Ԏeަ^A5bUuD8| )Q78}+4y8:t mmSUEQ?Ix<~eGEd~A"rx耈FWbhTce0uzt 1nW&Dp鶙[|sͷDR?ٽmS2-))ASS߮⊞J&BG"}yύx<Y>l5i M*T7 Nm*v޴^횋eYx~`rYYbj~1с0qFU] OO>qa͚{ۃ6Aeranܛ69@:۷oT*?~|I* KDDD3s {jϚNZ]37nկ3WeYХoz|0U,"tT/0/"w~nWNA11p4P5ni_ԯP}N{r0 ر˗/O&Fcc8#seYx2gΜt:Z e۶m-jMZM5sMs*5w޴ɭnǶg1cRZDMMzu3W̙3%ŶyBUUD>W_z̲d&h['67:L9{l rn[ة~L sb{Sܺ=صk l9qܹEdH|P((/}͞=yPUT:)7' D"M_eٰpVlOS(.kNV&߮gonמ69]sa7o֬Y۰x<{{CUw?.KFDDDvY_˄"Umx<&7.e?0 5M1}nyNs{$>3x<{mH&w=RDj/y< }|Ci "" Ʉ&UvS%OU}JDzg62hG!-9ǹ hb=mrv sC=Y8xNߪ:ý(HDDDZmm@eYc!kEyy1FJϲٖe֡$|v˷Z^i[ rMŮ^tR̝;x|_~PyH8bc:HUώD"i~ : =I ~Bsϝ/͌BӖξnS1 ]s׋d2|ԔM{{"ŤIyrHˀ۽[wwsŎpEGa9a;zG:DI&0e{[˴e|;9MkNnv ڴ\Zա>:fYOmu^,t ۂIxr v-lS)eS6ۺfk65O;СC,kdNbر DDDžP]n L,˚ٍ klno\\['u@ mT᦭mj5)[jGC}EUg@選?i FןwyŲe֖iW12m̲,TUUP%%%H&B<`Yvܙ}[n{,)CCCd6DvqLihiiA</jWB=mBk.?lJD>}y敆a1 n='{x]1m0qf3BWs?/m|VUUaÆAD_aQ/ ";v,|>_a(++CII Dx<|/߿W SO=mSnPRRRew ~:f̘4-;皋+**n S3awΜ9%s?QQH@u:;U])9b{pB܏?' |ߎyoD*1k,+~<3cf}vvibРAE"~@SSy\s5xg0}tY.w_\cUD~lٲٿ^D"y E.o(ޚ={a]~ CO742!+w@efCǃu裏#JeرÆ Ð!Cl2̟?'tR ;vދ.rmqzrUUUѵk1K._;vF0{59G5WT5a[}! DDDƏo}>_^f[LӼ­!-etNa&?ptֺ_0 o&RO>5kVwUDc׮]>|ZR)R)$dxH&2;ݻweYظqGcw5wIlzqV>e%""v7np8H$,W1fɯQ29x駳C `ѢEK/Cmm-<O믿U!^z|)TFD҂d2 Auuu6˥ާ"1477ߜ>FEڮ DDDԩ͝;^(};;-y+++X{]܌ g=y8p`:>_1@@}>_ L:369@_˲`%""NID͚ۖ&C9q-a>}`;+/Rif{e?q[wNCss=zCu3Mӱg+"8q"կ~;ۣ&?:ݱchUU/4^SL1 %/|w }L*7Q t~:`ٲeH$b7 A"@${gd29s==3ukv͸kCKKK6jn<:---0M|֭/U)/G^pjB!ll嶃[41zl୯ǫ#Gfw7D"عs'Q[[ 4iZZZ`rgoS׾Y1 -7Vk^=|ollD2.\l(%D"I9c}&0jo[ة("n n}bㅆ/Mв, #GCcc#M>|ݻ7*++`ӦMs9dr: zy^,^@ 1Ԟk^Fnm=֭[3cK<OuMMA#oRQ{0=^):6:b"b@ifw0뮻н{w :t#Gğ'1:]feJ UUUٟ~c@SZZ˗o쒒m*kL딯G隫*6lؐ>GDVyzXGߤ DDDlg@,H$[Vne¯Ls4iR]bȐ!{z82C&o@ѵkWt-ս{wTTT`ƌe &ь IDAT={gϞ0`ƌw}C ߞW\Mmwk^h 4&I\2|@DUGjI9# q\U544x*zײMf*4p9/3Ǖ:':e'YPܺ 4iS1?m-xzjbL4w@ t⎼KDDDٳ}>/p6&ɸ (s{\2i̊ nerqZ@v5o jnmF&LΝ;1멪{4M"{ڥ6Wyi='!\UךT>naN-:M*fMbwr7mHAkozz&׋;vC}}}fY7 6ϖeLk{ֈ]`uN08}h]PNAIg!P']1T#֮BC:M6mSO=͛7g®GWU޲}2Qoma{|w*'x\SN)clΝ~Ư Wl䞯PmjMrT K.O?ZWO~w<v#Hi`%""v u IXl,|>]~_T!7e^b"8,t[UWBGKDDD&T@5u֬Y%pG}Wx<@EDV[t Nkq1Ůe[Lܸ[L hkk1c r/Yu+"JkG)붧MǛeXK,.Ֆ9_ŋ뭷bYê:5' p_maX,/ /e˖b?J7̩ҩWmȂSlD2WiLj`׮]x'0sL|vᄈn-zYU6̡ XWeYxL .?Oرco[^:\U$jnp+L^Q{3.7x:)~mSqڔ/L˔裏+wO>< Ybt9RUD6cxCbD`0yk=_|1BD`sZMƥ3umK}٦bcods=0OMbPsDİBd%"" ת=_o+򑖖K9=s"Z |)d)&+f2S M@2d|AˤI:nF}a\dY֣rTgpyrHuXoFӧgiii^wf'>;,7͚Nۻ VL[| MNORzGtM f~cƬx<>lUNUO>awHDDDED[UK\iX,vÑ)Sضmd2;?79y̷&m nC q u kS(孭mr0pG\ ogV*`0ؙCQ f(E^jY4o^*a2Msg?9Md%""t(me˞|G[QQPGn=2<؏߾lSWK.F~n8sy#kkka%""N0|.eYE$x@?^eީ8^_ 6֧m mmSEE ø?_D~eYg3A+L1M{8 nnllӖ^Q) MnW5i>wkoerS{ۦR-))תjK DDDtP0aBWf|3+7\:}lﶛ[|Ŭ[L쏷MnL8~u&RYY UzԨQ D$,Wi ""faP.miiVZpR-ums]1[ ;noWh-|mrۛ2TU|=&"!8hp."a,`|/ݖ5kZ[jzmDɤo'euwUuSgxh2 cP`[gשWe n=ϩL+uBgˍ6*<9#""|NNeDDz( rn)ϔ˷p2NA髭Wmr mOD~z455e~7HU/۵kWx@X(//?/juL^R&NgۺھhSgDXh ÀX6ˋƍguѺ2,1E頤N܆ tĚŬ6lPMnŹ6eBpMM x&8'-ZZ[[[f ɟw;UM6U}^xtiTq| eak+VA miS ' Md?rH=zt> a20L xv… ;@U43²#F3f|nu-{ܹCuu5郞={| n;eeYHR, uuuشiV^ 6@Ux$s*63mڴ}پ'NG̩Ht1FKa|= TtСraCŀPVV43A2UWWm۶aÆ Oi&lܸ1l!lY=TaÆ7~xݻ$ɢ?זX 4pBLP|(//Gii)|>^/<mʜ4M$ $I$IHRa Ym1 ,ş@SS4 8# ~Y/^""",9k{{=˕o@ED2=ׯ$ ա;v Ԕݴc%v;CLr0n8p bfQf^i[/`ըEsse<JH~{oyGwN,"RvU=>o`%"""rϗ%Zgg|pY=d?w,Kˎ^PՔTՇOoǻ 2D5^w-B^ a|8mڴF:ߓ9pʔ)Zh޽lj'^zef _0 eO>ҥKn:l۶mtiOR Tu^"RzP찎n+,,D"ަMVZHcZ ---ىg\UU i~;gΜCMӼ "`IIɢ1cZK=Ow˲*DѲTzrg={r_x7[l9:AU+PSS30cMhYE"_:X,6lU5Кn|w_}׋/ݻ P%"^]UuEUWbڴi D$ `l&pxr, ۡPE2eOܠ4MɌ@II JJJ chiiɮ3at =M}l=Y<]DEdz 0'{<;&}]pKDz=X,PWDdeYD"/=KDDDZMMͩ"2KDzf١KJJF{E^ehǿVUIJ1D$)"cKpWTv[ϲj|6A,g2oukkkkiYc"2\ USSS)"h @4mltPUMMMu_|~!FDDDZzM>h?EFU"=^?-Ѩ&;UND*GDp8<'}x- ]D#FX555@bM|v„ jʔ)TJRDBe\UEdG?f=DUg֎+ xIJ>"2XU/kIw#z^a- jI-"eY~pxb/}bNQ3E6pC,+"los]׫T̘Zs߲;eʔ#TȑVծpۢ6&UHDVX8zgX /eY?h 8`֬Y{iaHiz_DIJ!"V(Z555i;JJJvwy=KDDD_J5553 ~2~xveY3֭M?h(4}0 aǻ~TF Wc5`2W]ub,0tڴi-_wu7DdL0l?~[nwRSS0 CJ, KD6B!Ѩ7xՈWGPKDDDD+E$@?^""""*(@ @p8/}tе@^&""""2:js?N""""/4 lDDDDep^@+8y{hLmpq'qC8MQdDDDDޖvp\wtK'/b%"""`e%""""b%""""b%""""^"""""^"""""^"""""^"""""^"""""^"""""^""""b%""""b%""""b%""""b%""""b%""""b%""""b%"""w_hHPU(=X.@|FDbDg_xP0ЃFW" X_SS?""싓BsTzi0M༘"x/ %PhT#"8uذa3&rJz<ȅv~O`0 FU!"8tȐ!oYFҡ#L d*++?k{˅ ǩb˲p ')Sw{{aڴiyf1 c|?pΝ;%W= 3Msб?OK.H$,J$޽;=X7o,zUUU+[ZZzxc0Î:(я~${c&z聣:J^}U5M0d׮]W >i׮]۷""W`s? YO^! pTIIA||t~06'㫪\s5C(:TD{<,׾5}ْL&H$|JJJ5xTU-J愫d2ѣG#J/S 8_߲#<(0{oR:,R)CWTTt^%"""ތn^Ū*TU4Qa ]ooGSDD#"< JUQYYؖnPUi8RD"""L2Z 4---裏d,Y˗/ǖ-[:a7oƲe˰~}r]vAD駟{}?_PRRҵC?:WUqGQ,Y+O,_;wG.yan喼G[577CDrQ`7xo6'A˖-i ^Ýwމ1c`ѢEQYY*TVV0] IDATPUU*\III㪊x<z*.BCUUUUsw"푭Ϟ1@7U 쁷RN?tK̝;#Fm݆2رv{W^^^FUUU(--}|{kx<"gEN naTv#DDD% Nī W]u~a[*D۶mQGǃ;V +Dee%t~ذaCv|m݆jTUUWt۶mx#`ժUXt)D2d  @cǎŎ; "뮻 "QQQlVS2˲J*++;4!{U]t a`_d/1h m0qD 03gD Ȟs8餓PRRn)\}>}Y@]vv UŋqSNxy 8x衇ε}eji%""bjS/O?~W_}5F{'`&FH$r{xᇱxb444ફ‚ PZZ7xg'`_x衇paa„ 7G}4LĮ]0qDl߾wƏcx^XP__qaĉ}DSNʬb0j(̟?ib2e >S[g=z&MBEE8z)\z饘5kV/FMM }]L6 ӟPYY3f/ӱk.[]w<V\3<`T z*F7f;w\|ǘ>}:Z̞=^eUU5 qQ[zg?nٲSNŰaO`Æ x<ٞ /&L~. >wun݊SN9vn݊?, wuK.[x<#ݳgOuY0M^{-f͚GXQ__Sbj /7|s Vz*VXz,]SNEii)/_>|82;W^СC1y7P8Cpg# ދ޽{cԨQH&P[[ 0o²,tMs1nDuٻ(7R@$zU C E)ҥ(?AE:*5N")JR(dKAyr33|G 2dȐxQpri `ѢEl8qZ-BCCӧOc֬Y95@0l0|HII1bL6M"ÇDž _ff!;;+Vq0aj5tX`D殠P(u2qHNN~)< zK[)x t:L>AAAtܹ3,Yٌ Hؿ?.^f!22 ;b9~ȵ/q1W4mFQȐ!CbK!7m8{ ҥK6!YYYR !ؾ}DΜ9Ν;8< ==<ϣo߾Xn@7tRB|rԭ[& Vf;wŋ9LRRRRH4nX:| b z/.\@r JV\`T*t?4x:+VɓaXq߿_Ça6q}>}}wN:Zu֡aÆUV|prrˡhPL+VVEtt4V+ 2dȐk|`N&>03_{ H F#z=4h.]W^0Xp!LիuuuŴiڵkE7nL&f͚p4h8p qQ\r~!,{HII͛0 T*.] oooܻw .,@+| <<٢ۉX.bًuB,3Na L&x!ΰX,P l6JON5k&RJ P(M:5EndȐ!Cƿ9V].>|D:=C4p_κ[(EJ)Evvvjj-p\"=hDzz:X sI|qk1 "$Il/Q0LyX.1 GIr=_0fff"66׮]+@`HMM}ɬ 2dȐ! {N(pLR.m#]>jO{L 2dȐ!)."Gj.m"gы34B$O/W(//%2 2dȐqv` ,mjzbAfBx2dȐ!09% Rx/d,d^y}XdȐ!CƋ@9 ^A OaT!rǀ8"*NȐ!C 2d[ ,}xpWJVc˖-yj(SRJyׯZFRRv FСCA)Ő!C`0zj<| fa„ hذtggg$$$Ȯ 2dȐ!CGx%Na$k/T&R BlFZZZpe<kB$ xxqXō7W(QtipebժU/rN A łDi=zÇHV1p@\t ,GffjPt޹sPjUXV,[ |z !޽;RSSA+WЫW/*Jt =z@ZZqU( \~teȐ!C /'\Z{aprr˜1ca]68(-[cǢbŊ'ƌOOO,YϟǤI̙֭[#-- ;v@͚5ѯ_?;8p3gC0h۶-BCCqM4m6m;e˖Ah4`0 ;;OOݺu0[}6_z}ʄ )X 9ʦ%4JV#222"BFFqh$ BJ)t:l6l6ALjEZZJJnl˲h4`6 ,h4"==FZV pvvFFFF#A@VVXZ˲HOOB& :\Os"""PfDJaA" űSM 2dȐ-,իhҤ j֬+WbݺuXx1qE<ooo.]-Z@VVVkPJQzuq04h.\ X,ܺu (Qpo`ٸ~:Ao/ fѼyѣhժXw}UVATôi>dի}zѣG?ę3g$UGwaZtR899nE1p@YÀ\zU"Er?j8p /_*#""q |GO5PT=z4-Xźuп̜9| fΣkÃGXl3Zұumm_9y=1g}ɒ%}:)(QcSYvCǎv=RZ@ K5W$<<6޽{F y1M&O/QUV +^(P*yåmBf]9R(G> ?Vd0ΖZ[P(~ !499Y=aYfŢ5XEBbԨQ9r$fϞ=z`t`YQQQٳD{= aѩS'[/_FRajժЀ#aL0+WJBN0p@Xh^^ze˖aرXl:u 5j 0qD111ׯz B6l؀`ذart ''^xX|+?\Io<[֭s@vjz_//rz0aeY0  PJEqᛊRP(8e7RJ+q$B?A8͖۸q{ K6mڔ; @-zb+~avOz\xkIzTtQbEݻD&MpmϟǥKza[z!!!pvvFtt4v;v;ڷocW\AZPlYܸq'O;BBB`pawũSPbE=z8s PfM*U ѰZP* b?ҥK͛hٲ%9 4j111seee͛RJ|z:*Gvi@`ۥYz⋡RAf0P*ήi;99pMZD91*-dY6-πb@eXL/ E(TFYAo~nnn L&SѣG dښz>""&ל6 F"@$|^^^UVطo;PJj~%Z*TҥK?Ri8B$Xf l6[}BoZ6e,2›z,PJ p=l6ܿ_R?8w*T&gJ@$&&湗xރ`ۑׯ7@rr` 55Z$'-]a ODsx.% PAEb{ .vc''z={;v>L}~zR/}7I8b>TX geeWW s! ׁ0dl۶P-aÆ\PJ8ΙRakב'}2e`ZqeFB ((+V3Z-V ;w* C*Upuh4T*d2ƍPT BJ'pyTZ*U^ 777xyyys?J*mڴݻwQfM0 K.?@PP\]]QR%)ZӦMq  E ;wL&UիUTVEj__gFtEԩS.P(hԨQKRD6m$h=|0 6*sX,bw\}-d< 2=VecxRvhSc$wO۷+:4QF75jԿ\rT*E0238A.ҥaeYP(\m_!7u?RzQ#aATt:ҥK֪Uۿfǎ2+zTÃ<ҷ}UEس`0eY@E)D)%88y.W&qZ0r+VG۷! bH<@A)g۝O9iEh4hٲe_ڵ5 U5jHTjժAԮ]v]˲8}tUN5kJ7n,>N:I\oIqRţk׮A:700Rv K>...c8pRSS8GFFBt G˲Xt)\]]QL4hC0h߾=ѢE XVcȐ!8q3x(`o'-dPtxy J8pg ؾ}{sRJ2:99Yyst҄-Ah5Gm-BqAlr=8"Zs?TRQ///w>&33jj݇ڶm{BiEe1i*qAQfYYYҌX A… ;dǎÙ3g0f̘і^ƵZj~9kO>1g5l6cXr%F#Ν+YwSSScsw@Vu([tҥK8x xGHH)WMꗧL;… ڵkP(PJhBٳg?8qƌ g;2e <<`ɅN)%''=-r㤅QR|tx-łRJI>>8e˖}j] {FQ*-[ajDEEa̝;* Ǐ7 ^ @I& CJJ F#(0 ~7_tąs=< dѲtzr=|PM |5aUReW*U-[V뭹Qk8k@!+%4+}rp-U;/FcW*6WWWg>>>{ݛ7ovk]LOhV-ZTaf gY ETTF?uwĉ ( OawBFy؆ BaҾu`0 $$;v,xGff&:t;vgϞ}ԀGԪ(LwP((~RdvKvkd׭|QK+W BN[_|4oٕ,Xv0M>tTTHF.]:O;e˖a޽%Kh_ŋeo!<<۶mÊ+Ю];\~ժU[o]v!99Y"ͻvBÆ QR%TP:t`KФIZ ۶mO?$=dBDDƍW^XrcҤI֭~G#Gg! bOsaСCaϞ=wi׮]Ý#'28ִdggc͘3g֭[0RZ777A& +V $^Q#avlݺ]tF2fӦMHf3j֬%K t+Dt Edʕk28)PnW(eYR*BG^q=!nDܟ @V JMLjٳgPxx9 N@@Zn %ϴi0a޽È#PdINRaРAիƍeQzu1 &MMٳƉ'M6zV~(_| _V;#'NA8uRSSѨQ#DGGHIIMbI&ر#z%"hl+y!ןqwzB0f IHV';>e,VrQ6lxxx^\\950|Osͷ)fసXQctw˱d8q#Gӱk.Ft}{tY {byvΒWcJRb%1VTxk@ǘ#KWhp/z5 '&foV|7%_o^kQr傝hN/<$$IǽK'^ "Zab* jHDb{BozhBZ{6T~Vm۶a|X~=:vɓ'?CBBΞ=qơf͚ ^GϞ=1i$)2%˲HMM˲1bL>[nիgjJ*t܉aہ6B;wJ4U*ӥxJDqfUa0gʔ)_.˲>}D֭CI#/Qo.屣ƍ){VD =z<σaL2E:Q"ǟ)]ԩ8q,[yT[nk׮hٲ%4heet ƍCll,j5( E-еkW)DPPjժ Bd2ɔ7EƳAӡ|ʓB?… … s:v=y&J) Z,r$8zov\L|\* J@@H7äLuw*xիWoӿt-o'w  'B#ZXq'W&_ܖwGS(^wt?(Etdeeaԩh4(_<(pwwѣGqҥ<SLeJ2Veʔb1w\:tqqqXv-5k&mpww56:g!** ǏGLL ~ǏZG}w_Exx8V+VZja>ewyE$# U*Ҍ#dYV:߂0tqFTW+x,o|%233ggg/رcܚ>=x :dݲe ׯo>[9s&N8͆Çcǎ;w.J%6oތ-[8~~~طof3b )ʥ&N7oB (Q0x`x{{#;;Xv-v؁RJaΜ98wl6J,?k׮ٳѣGCRAb׮]XnO:u`…pssMe<_̞=!Lx`X, 8...8qnXرct2e (S(pJBDDcܸqzQ@T":: )))hҤ 4 6lWWW\t y0pU ʪUAwqp84+ I8[ܳ'Ʉ~m k*dL H4 ^p;hBC_PCF&D55P*l6t?3u| ZU{VqqqPre?R)poȮ/ cHS7'/lG>"'R>l8o[rop A8H4̴6mjҭ[TE}:H+[{ IDAT[(RJѹsgdggCףW^ݻtEhԨQXJ c gI$y7UVR=zHFWWW|ᇎu@`C&/pu崟Ej&>>>3}%0Vh e#Dݼy3RSS1c t׎hzpww!k֬v*@$a >) %Ajd.ZM&Z _f&4-Z4@jd-YÇԣBAק1RрytW_ͫIj%ɶ]zd$psQ*Rݷzg:Cu[REt.z+:~"~DmRaӶ=rϔ'Jv?,t|4 H yaARq?C Ȍ)ORplgՑv9wkI_'?i.>ɮKI: gaA~5 wwwɷnKZAAAHIINVhڵ}l`=OpG 򪖚۾}ix16p(A||<-[& _EOXX/(uĻRJ/0gddȫ՚pE T)̄cS-*9bÀK /* B`B)jyBŤܹ[ȳ%}]_*G!H׬L8;uG]{%"^m۶wAAA5T*1dWZ K&,n>>.1O``Ta8DoˣrKjia$]Ϲ8WW׀m{8ɟn9󥜪cC)fN8NFcPgϞc66Ov|ps Z,Anv.ק󧟙=O19??:ߢ/g͚vg(c(_8̎l ǰyLl _k9I[dCsՁZc;bn 7Z30w x?3%??noEPJ{s !B9GSDGpΫ !9RZ\__\J.\PׯY7莍 &pCS=Bv1~$I۶m;&ck.ض3AǍKo_#tX:r}58~eL|t?J%KwcS3[ϱWqjŊVVVN:tqc{?c)99~ $u|v4ZЭr3-YD{ AAp55yrI$ILT kPY5U;(u B ksժ&''駟Fi>7 3~74K9Fy9!$9w8S@@-|OȾm9ץKX"IR缜sMhArd pkBcIUu?jc/ٲx}zg@xRqH2Xʈc@( HP)/HLdf yaXN)O۷ǣիץ+Vg4e7Ƹ >یn#OW,'ȲLBSEQ,EAhbݥ*mll,Tߖ \E.IRfc!+ו n3՗a>MMp{:=$o{ΑPJÒgDEp_mm>/\30w xL7zBAv{ $1,੠e1P9/Eq={gϞ}\l_}iX,Y,Ji4s` SB_ @ eԨ@طo_|pԴyMUR7?wHgr6sJ_+BTAzXE] Wקa8< 51q4RRJ邭[~cZoܹsŋOڵ}N3`G g Mm VA)`0HV+Uv;U.oõL9'EQnr+cn%%%% "}+2A˔RD@ldaZY\\?.\4bĈۮfkq{:!!aJUUU3} M-Yg"beY1Ƽ%%%^-Ϟ=7y7|{뭷ccck8EQKdeY !O`G)|>߲e˂֭;jIq]ϥC.K/g';p2W${PUU)%D(r(=lVwBY aRPJA$BET9s΁K,ڵk[dR#-tDW߶t+BcZl6l6 ItӍÆ {׼aKţ>h8qz^9 dҤIu)

EUUUpɓ'0H7P4⋞7XHrssO F~^RR{&MtLfN"٪ʐĥ0^ KQVOGUFtO"#F\1"u]l.X,aHrR>^fzqө˜s}qhwzmtizW?K.kfn&L8e-⼆Ol",p9̇b؜VA?f5 }gwaÀ*7=? o0N;ցN zC%AdBSJ8 ~ '#̬UUի333H$;;;q=P=%7Sõ F*++B \hvT(m|窕uݲVC׋7oINNCcှ@ L/1HmB9@1w\,hѢ^lfa-l8 ,AW38.qٕ6ouee^<]@PWg?EN_B<;G!R,A]/AlVTWWwkcccl{/xW^=DQT4ݮu. ោ_ }B(Fڱc^z5$%%b&ǭ?/]eJ2J)'&&ʽzj;w:DOcZ{q"Opkr5 S韵BX|||$I;wn9[a&m{\agߕ9fogm.VUN{?l =I5 z 9]K.JƍX 2!dݖsXpᠮ]^t:PUXff`TR}RzmM;^X;G Bx'IHHsssRx<Z\\lWt%i/- \$k|BXllYc9Sa+ē<Ox@,xI/?9'vL|c/-u@E^2fE{v[@=?}'lY羳@;3/z/:n Tu˓8A2 ,bX:?^K^'\Ճh`B:/'l(]czol_~If$&&:CڏT4)/kM# vrxN+cf fm;CMZ8v_#05TO3z쒘rDnX(w0h*ĵPcMpp-F&ĵ&[,p8,^CN7g>30\Chgx `58sOXu9zK I?٬Ȋ f@rR v.7 )]s?/9ALD<ƽ=?O{_Te#.E";s7nL؍7g^ͧM̱PC &N)NSPUy歹W3faKN6fěW[!WO8lVD'=e|ν!`Xl6鬔d;[|OϬ9=tn5y|e2{ 9. +/-Xd Y_ Ι4)^}я 9 ^YX,X0[nbbbdƘR $rW SJQ\\lWUddd- ?7o E$ }0  5QUCb{ZZZr)Z #۫ az^qۼsYLLKUթ ,8K/fa޶ML˯;seXcCCqGFt+8WIqyl .8Rʼ(.wEy %IgܙI*;TŵqĚSye.~;_f9Aۋ[nu|8{;ֳ̭$7fgzo nۋgϞرM <$/Ձ3X3j{@,˴Ԗ '$$UUgrm+Xv\HEZo-M諠9K.ނ{uu[E(A%Z9EQ<#e~,˷l΂fa x8%;[_ܷXofxi[<:gƆXDY[w/"ٰɃac922wIu4jЋ:{~;kȻ?|:tpkKOw$#..,tpZ\;|k&)a-&l{8Yn]UffBGxR#?׃6hPJyee iii~өڦl,#H+v|OUU%;wKII ^aGzu:]nZS4G6ɲ… 1b9 a"fhq=^Qxϝ.G= cܽ^",R=:dV2L{_ϯÛɝ#=z&/r;>}ݵ{ᅠqɢqKj*I-æ:ޱښF/B /}Y[oAJ>d7JeA_^6 &p*rnr% IDATڎ  W'ۻwUQҡCPC 5l˫-b'im*Wv,itávݻP!#BY(S#Sg 3NlΝd΀faƱ5ㄌP}m|)ӳ^-C6m>o;|r(zBB*!\*.>o=^ؐ[sȯ+}XVƆl߹_X'1Uġ >ީS\X=7lԟ^_/>;<: )8"%y7P 8=a bΘ.YC32z-;vpvգUj(%:Ӥp[,ƴA42PQQa"##8F^nmex-]/18/Dr8,A}?OHJJJFڳg=999`٘~l iƁu9X_vSϙ3fa^3Np&s ,-&|w`[_ǐ}p៓! *dE"5tA] UUuY%~i'ᇝ$הΝKdk)_{ϭo_q?_w?)/uw!q|]23f"wk*feFYC$xRSSv]F=k'f5\RRbOJJ nE/ӈTxnĀ&V>^ljEE5..Nm҇GCޠoIҠ /DQ6E>|x9a&5㔋\/[oӾx̵e?T8ffyIv];ӪJ7‘ tj1)?'KMf*m}0GsJOH]`ݸҥ̣K F#F>ƇcW#/lxЊ[G! j"I$Z'm-q./7පa 7SNX***iii ZBH: ێE+];D%nyߢE>|9ۙa&5㔏^>8HNN۫,A鏃sFiaA4W"$f9˙a&5Ì&7ڗ7_m/?6_%qv.F/TUI{qr\1\@eUwҊ"7U(aN?gh-[s;-V[U՗Z>ɹ]oAv=/AAYnJ'YIO) XLS{M7=b?39VvWk{߾}R׮]hmA+L/㜓{JJJr) א.2tmuJZ39菑N⠗Q9,o`!;;۷w^kyy-##oH|kXprPJ cLԠ!KIm^30fBw55]<3emZƟ~Nq享oWmrX<=!q*Ĺy9gւ11ɴwҰ`.zM1֮+vV}zd'P+ER4Vpђo}*;o  ӧ$Ƙ6|3trtz;CmmX.afN o+ qBes{%nUP1j(V%' Tf[cZ$҃1 uuuR|||PKPv+ @' `ge 6rm!II<͖?-9sB2+ڈӶƛ;ƺ3eFcJiA9A@vCCDtpB;$i[.2ݳg=///Ss*t3~J;pRMlZ(m--LpnnNVcLAmpA޻z2MbbbҼ^_jdfaƑqJ%;Y0gⳟї_pWu[ru&TU8!NU1_/cOW$'uܟu7|O+EQMX/$5^>;pٛI+#db<CB6Q0IQ6vƈsNJKKmyyyg[=1m< =a/c,4PՍC59WUUI!) 7jzҮq :H,VR-Zkbfa x0 q7x1 ?z W{t6w6;ཏR0 *@$grq-]ɽf6Qûƛq%K,׼_ևM)w۔Ĥ:</s>YO^-.Jj(!o Kž={(\\q;] 5áZ@'02т#upXcCİłIq q<99Y.))|q{.EIZË?PJ$I]!4g.30Hd4P̛:%U?_ľϖ[-+?*c\>\s@ 9Sfa*cgt[n3zprv”~v dj @'_ Fbs_]yKi?r QC $/xI5ᩩ 3 +֊rE<#wnԓU jId!m޽{E2]Y{X&@ȮU΢L:@rhU`܏mBEsrUUQXXh1HsF: K2͹A+ym6$Izs2U3Z_ 2M6$yqj{sΜ7McwԓK1ʦHP@Q,ذ5/ۅoIpE}X{uWdelk…#v[ۓ2#]c^#_H?̬b^zV}vܓuoa[}i~]ΙfBpSNhW1aΝ;{B5_й7CZZ& e}5^ MWFn/3BDz c qqqJ}}X__/(4n5u]wU|z24/s8-Zt×i1!1o$ȲHOy~qs ц-ᵵ7q bA@<&c˝<7/w}qqg~ؗGrI%-gk~\[oMXL,6ܰYmxUBpEQ(a(]*j-p6-SmFtN$,JV##(naY6ƭo+B! -+3~1FӃ}46zz^r%׻}n٨ sc:33E qx<n*CUU9 駟 "\. -6!䘎 x0J >-/X$I@JRS ^R\G>;W}K}U}&s&$+ oa)F8{;b.Gt `l6}W4z6:{& ̐O`}}|~8{nF?Y@/1yKmP6H#(2ۭYVe4kڰhg|?S zFJJw;(,,D0RXw\Uc:ZTo'=]|9 I\.8:Ie. m/ι|J _ 1u嬏>ZFާw7b`rd!k*92\3&$;wnff7y88,@|Ber6u8z_zB. K_[nv<PQ&U6Ua3f˲LE!E5ST~^zЦygxʲL~/_!X,b0Q3VZ568NvORSS[nx/SL#<d'B8pK.c$&&ƘLQe|G;E~?'$$O'NͥO&( 6oތw}Xb!9oX,>ϧt~4v2lذ' !0dY$$I!\{ϟo]LZ^qΌ?]PO;-MpA YȤx{V!(%<6FztOztv.:wshѢ?$(rpwB?=2-!ys|׬@ɬ˹ϫsmr׮]N:A8\WGQ&HϬȷmEQ\c;[m'CMq)ݻw]ebZP- G$^ 2u:Jjjj@$*m#[[m4a!cؖԵFZƈ(DUU*TU~'EQcJEAB]]O`K."S|Ar_ι0dr-@S~hXPPPW_}n޽{pΗp?k_xc| 3ϡcǎڵ+NI[lAii)<w8{  ՜O(K6o\:gΜ=bws'B /$锻^/֭[/{tEQ>5j==zN) N8L;x8N1E%ռii_wdEl*$Izsug)+/ݾ~dz/ͦjcc4L7i  r#E!p:Rө[hS~aAAî84qq~'i&>xe7n\z/b brj]]VZ+Vpj|9RJ:Icu ,##&??bGmmmbZTTdv~q;v_ILLD^^cɜ*ER '=bwذaX*FGIA8eW! :u*V^ A222ZJKKΝkٵMw}o_7m)gtbP"0F"ڑ" RLºM"ކşסSga?+"?vz v +bm9p>0YХK5<4Jz4###RAR2h`b0,=!޽ۙՁV ;i`I3HeeUUU"IONNPJy+nFL) )(Jh$鰎d#K>ceYŶ]zE!zZTӎO"R՘^=뫪* BUUC}DKwZZZݻw?V]]}ﷄi` wYYY ~!|TL>;vǟjjj ]P *bZyll,☪2! :,]UU{܏ݮj\ǎLwjC7o /x뭷+++M{Bս_-dㅉNү.$@A {S=H;ű!zTwERh 6,Y[~ 8To0~ǝO;P_|!759o:OVj 555RZZ_a ] Z0{<ѡCo ,7&M#CeeC)bs5u+yQ 7~Zc^F B&=5_*e ~w;擿RlllAUeDCCY555I@@h I`xbb"ʜz8_*vԩSq\~4;;;r1 v Nܣi!팱8dY_֭Y^v裏lAn"(bʕxЧOeY^^^^^[o^cƴN檻mI 3k268ᐬ*VٟW(9?wʓRz!֝)5s IDAT-m]̣a"8XlwOUEo|]o zv7ɣ}-!p\ %vB۷܆P#A'< R~PRRbONNnY+``vAayu6clC[~W^ҁ޶^93u/x<.u'ڜrGQp!;BZJBJL rkB~e}(UV/#qƉqqq v=@?}!ٜxB5=9UP !X B !kYfDp$=W@UUs~|}cǎϚ5F c˪;G=6i]6=R#no7Uh׺1_m݁믲s' =ع3sGd3g yy}UAaSNA[n5U!fw;L_/Or-^ᄋrб:_V ƪFbw#ZbZZZ@w>jT[au26}4( -**r$''322|1zREbq8PUXV5??RGFFO$f-VR @xUUIǎ$%% ۓyZuP5MSjhIȓ*eN8Zhss$@/E;7` `cl(k3f(C۷fA !9cR;QO^Hs/r΋^y v2܄AEXgϞ_~'O>fmƴi`09pλB6jM N0XL@" "aKI̯E@ݺѭK \arltX+V! Itr֙ԎكTPNlK : ҳ  t+^| r~K^3MH F8~_Zр]]ۡMWih۷owի64>@lRA.OLL ۶ms4:UoFֶR Ju6>zb8Np8XCChd#9EDM񆪫iVKӁaa}AŒ}"+3f`O=T,˻3Ɗ)p-WG9W9~="_(JݬYkaŋ+Uūj  !`0h$)l5x2/DAj@$ ,{7m\tIMzo ?| ׯ_xbLLLرcOu]xP[[ߤI0yd~Xdw"ŵyѣGRO?}O9xNBRNY:uSV 9\N$YX,AX@4*8i9(e$$|S!5dP1.+Cf,wKS;w9/Ҳ3SU UD&9l1[6pfh! (BxeeIJJ ^ ._QZRRԩGKF~$"!%Y ]v9n I,k0\m$҂$1"%a/UZzg >xy3ư~|>.2bK.ek׮=%8mcV.l21馛;v֭[A)E~Xt)N', йsg7b;>salݺݻw|Pؽ{70p@BPPP7Xz5bccqgэ"81 zm'Xc1g>uu^* !H!"$]@bD v$%Iʠɱ{K`` 2 A_?NDUŊuݰFn}5XbL̇bNvڙ֧8ڣ@Llߠ"dtSH.AʣN>ՊY hYm={oC4̮`BY`ΝFM-kem?4I1I՗<''dzw^[yy=++(oM/pqmFDB$&&N:#m6 0,ovM{A)͢8a dzkH=7NnV[nwaPJ IGlժUeN`uj4iғ'wsUł!CS1~x<(++#<?}1c #Gbڴi (**·~~᫯ {EEE8q"t뮻?8ƌ ⩧u]ѣGcٳ'Nzh֝wv vmcFu5GjZRֺz.@ekW#PU'%9IN'Nѻ[R!P A`TBA$%pt 7] 44eGVYPLJMٳqq61#=ڻgK@jVAs #zE&-ٚAWd"J`5D+jl `h7Ŷuwԩj\(Ɉ>Qi|2{BnU>l .K-((pnkƐlwr|ڵ __zoj L `'\o~3]䫯zN:֜Ox6cv xu&QGUUDQ̙3߻7p I0}tw}aX1c@Yw6on>=3ӧO]wX,HKKmf51_'NDbb"y 4=bccJGn TE=4>p'Cwl t߲1߾{WX&Mo@JJފFlQo':tHC.vC'7W9T Z1 W݅6Xَ5U񡤴s 8r3. wrʜ3c(6_Έ h ԈIII+HN5swоuuu-yˀRW0@lm 6;JKK 咵cD ҁp>LJ0e!33_QQaIMM 7?@O봺znQA) ,j\ n2Ud9ƛ]s1*TUŒ%K5kॗ^5 Ќ Ҍ 5]믿bРAxGQYYRKBLk5Ϭ'^sx^\}x'pǍ7ވcǢ;vd@mm-׿";;˖-дAXj:Ķ{5'? jkkCWW|A7p]w i/K\}ո۰h"TUU&۱d)NܟYzҥ>ӽu;ٖؾ ׫R.rsW'#7KF^>P#tJPl;!y .$f]{a]:6{{O;fϺ@TcH0L~A!(FNzoH1fbQCCkĉ`7A+%F|Y\DyH> sNG7559ʂRgkIу WElṓJ}4-6qn+,w3DDH4f- hmS~#ޏf-..ƒ%KD5N|cxױm6twwcʔ)'N)S?0uT~̛7Ùg9dW^y% b܂\|ضmz)L>}xxw_3f̀fCNN,Yۍ'KÈ`x;<*,$Igf$I#x]Z;y]]54t۶CU!32&EyЅI ;"a'[ރ[ٛoQE۷].U]W6? k83u'/P)*)uvv Bɢ#d_.]Cf}}ͽfNSE"nR$q`p dYPc㋬Eu l`ۡ(< x?L*;" NcZ-=Z{ F~-_ n\y啇@^}̘13fLEi0mڴ!tI8餓`ȺLQ̛7ψ2sc LˇnڞEYgO{$I23S5Mrigڈu|^`=!tv5 X(D8a}uwb--׋w?h&+'a̘.m3Ϯ?a3GIMM#ꗶ$nn_aVUUOTHk&u!mRfނpSS-Zm8@7-p8Dgg-//OMrN P v' û6(ee `6 *K~uעsW@Uh'Ok:, y9v?va$kvBe%Ip1]`$tLSU#E"atwR:vv={XcS7oD(?!F&lGg/b&`vhNDjPE'pHi,욫eW*o7n Kr1=$HYn$,S$zp0p)kDz}ӤXZ,Y-& { _=%Ko~#o=?|`?i8c6s+~#[ @H`@8?KIe'hqp.A8tw75m߾[:a^Z?Z-m, i1D3H`&pH=OSUv6&xKodۉR=zt`۶mY'NdI)7ȴ`Pr8:LP8&,cv{s=WtWZwK,S%K̒$S̹fxIG$]'1r{OL<3V72f*0f6 p}t|tc2[ʪq:f0͇1Ib'UyiےwsB"ǺXd/ٍQ̒%K<ȅ35QDLG iҭ"'@~D!<]̱;.5e4<)maà m$t]7W˨Z1R("0IE:ie}vG.°)Icpέ%K2Hx< ˭N7 <c" #c^dYp8ս<>% ®Q!׈L R?12! ? p>$\vve+((r_H|IVۭAx:`L})=7Dv'dcomܴ Wrs2,Y:jkk(Vo&s=g cNCCϏRBڤ FcR@KI&2/3{R S X X8@t[R4>R)** ݦ2~KX; xGz2fuU;>k$٦ 'sBLW3<îj8,Y:j HJZUB1f!0?imX,Ck /#JSPi!\6} !1L-6B$r0M(KEbԭAQ*ǡ2̤D ˥wttF~c?%NBw{Mڰ U*/bW^Q]wyr]dFW崃&K,Ykɒ xL<v*Yꪪ2#-di`f( c{iX,rp`PU5" ?/ۍAH+|E| Xܵ1fU2d^K?}bٔf)Ei+Qn'Mx&s4d85$(0S,҂ B8 IDAT]GF:^#~_zZm!_!Bx" zF7w$:?w<oiKϪ`7\K/ B`L@U%0hjJ˪11iۏ:6yQg/|EUH+"}UZ;J6hI,Ν{wW("&/ɓr1"1anB%;%΁{" yYh$# ,ݙm3o.߹yO$mBO~i?CCT,sr:,;ۃ'ؤv4QŸ1Uv(}a:7AA݇Rh \873,yKzpJo&:ZkWCuZsqc[mLB@ҐP;d8<𯨨𷶶:x~~~h$9ttt8Ə{"Us8l68FKd]=U}=L k+ؕ8uZ=\N4(=p-D":as־ l޳^#h #ѡ:t]@ .'ۣPV硤ȁ2Ə1;`ST0&a Ն&g;. lq7&.ͺqķ@_* x-YJ1%| a/ !$It'E Qx4던r sjIuwc@ ElTEcX$͛ď]/fS0vg0O  [ĴAG0]B#Hc_n}Y۶-ͽ}jpRl|>E.+(*̉eQoV#@!B0Abt}2>~JOw/؄ lܩ ]VsÛ nA+?,Ykɒ'MӶ뺮L#ᶍ1]QVVlnnvX͝ls+48ɇU@ I**Fv;P=&{N6.يpN_0N>A0@XJ Vх[[ݣ1չlƌ}~{~qמ$u͟?B"zʍX1}2XeY\zo\o>3_tϟoB_Fz#e]&YU2H,d7AD`tnUĆ#ǣuuueeez2L;FDx@ 9F65Ul8N' ɭN2QWu&nսF1D!R #,ip]Hv= lfJoqƆ=o?yj]mƌ96viDiRtE8VsKس/[v86 ::{ۄ0Ic^hNZ4vp;p*PZ؇:@סwAh8bFT9ncmCÁVTsgGƌ.pάw^ޟ?/lBiA^U`06Ndڵ믿~L)shL]vr, BEl…-_M x-YtwR$=|d"#ǣ߿YQQ'NkG*BDxFmatvBhDtb VEԓ˩r2wӼ;gj0zq`˜n1D=Œ Em@sS>;c?ju eeW3(WՌ-pK62*7=CDr+ә xiӦMf=cǎwh{1q'X`OFRÁkƍcv<.pg}xy?8묳7Q!o x-Y:ʚ7o^zb^it׈ZRt"buUtp#k>Z<ɉ3֟Yď\6x7=Y܀Āaydl2hDM%Hr 4 {ݏ9`'1:%2S{CXE'L#GcM8#lpσmD$ث+~}ɟ? \'+@4DYG?˟+Kvcn7sN5.s:n[D@ C!<Z^^dgtv O$.ݵ;2>\ׇ[{F(JgcϬi'gh ĉmyOzÇu}_zD4⪫B(nzqYf{BU~,EQBcƌ [|>XCG8sk!̓$i{nnܹsK,|^K"H](u8 "/ ai0IHeux Ê&3*//677 ޼>ӾƲ2ӃqY4+tM|06֍9FK?G9k8K[h4gv l6m"G0@K_+ڱs4byyN\|Ibe狋/=3i:Bx ~b-΁5k>p?Ѕ @,%rtu**L8ԌG6w|=ˏXMB $Y`8AeRUT˵-\0|MD(..,4:.& !P]] !4MdMe`%KEծ^zoNNN'̀]3 [*KYYYz" LUM>"bt]gs(oL`7>%pk0fqHd[IDc3%sv^w人NߚmCqA|Y[1n@SuTUfH|)3N4s;yØw4b1!%wݮ}oqMhfEYl֬pD_ ]ri(_>">+k%KHH885W! OBH0.H!kW˥999j99%j'JئtsPxM$97^;-M&d+o,9N8%K1gGVye/]>۹9]6qsMhm/_(7۱f 5%vҤR̝;9g:QHdJOU>cdžu*`W 5, 3زN.&X?o~9khQYeW<$UCcR21YQ^^Fw}k~?77s|C=$U \\"*e9'!Wi~ "Z[[QSSq{pꩧ {UW]c֭x7AD8s1m4;#m۶>dY#F.oudfll6VH41|d g sN2g]# 5#97Oh[uM8!q55n2_l :}͉.G__[vdc"u[Q˽?$:ܳ"s\d͈?P0;Kw VX|Urq"Ba'A{z-c㦭`'c1'pAᣥk~^BXm1_MMM:u*}Q|߄lxgQ]]rPWWz ^{-Xv-:;;QTTN; v1Bdp;;"N\χ?W^y%?bx0g444f̘jvmbO~[Ò%K0}t|W<0w\w}8q"|w^xO>$pmAQt]wg,#uww/鴙lb%ͰiΙ]L23p |}رc0aB_&ʸhlR#$U6 ~$q{yyy0Acd;ekߖ;WmG ^]|Q%jcFشņ~af]Eىp;alp ]'.]?ڽʚv0ljys`w>^?F]o_-1FS*Σ0"w?᏿k[$cb+ MK/>(N=Tlݺ'O·-;@yyyuDII ?>qM7{AAAc,kB6hXֺg̘Iۍk&|;9sfn fœ9sW\qnܹs|زe `^z)okkiG{{;8 \}oFBvv60c ;?pJfE,dt曭Ff>/_HĄ4cn46Q(,,q99xڅOQ*u] -aL1I1.ɔ6mzKnm@xF{?)7"!+KMk'/xWWhnn^vG=FSN9wqV\… xbQ,׷\pt]Dze駟b˖-=܃iӦg?F},v18c؇5k@$ =~_^3όÔ)S~zb̙֭[܌38pYg/-[7ߌ[oäIb3/ŋ!"Dpg%KP?D~a5S$!Ϊ,XMo|eeeaKoјzzzP($9=U9x>`wH DizЌ+8r{g='1),B{{iμ^;JK؉'әgTL9E6팈bjiZ(u<2+MA:vڮ]#;{v 3:M>O(N<4ӄ@9CF"ꪦ1wA__6l؀;vcسg&LO9&r-@Q]aYYY_>WU$rhUUEy睇# A$ݻ~zӟpO?4.]oEEEA?'|---1cV\[os]pI'Au2)i%K2> nlf؍wi`Qa85d4***MMMH IDAT C&{oQJIm۶ex≽iF48 k5'UZD}`Q0ǏO@Ȃyl8uBt?E$Yaa6.8M?Νyn|EA+G.bFB$qn;>"QeKqZZOUU>ŧ+fU 9L4iA!;n?ϱxbr-nɓONc &7ll6B z\%\P%6b߉Dγ{}/YގyѢEi_|1v܉+VOFYY`Ϟ=x'  p^tMwRpaqw#SCFrrrN[DTvh#x8#)18qbݻcǎz&!N5p1v»ヒs9_ױl2xOaʕO>]]]!>9mmmTRR_|1?яpu2qDO G`0aҥ4-6X<{zZkҗG=n. n\o#X3<M#І %[gP jsΩ,xAט1c4MK IR`ߏ E;B0OZSss$}· 9yg-{"u!+ˉ,\w,yؾpΩ~I'$&D ?ý`̥R5P לit "dgg̙3c~_Ŭ~L6-v g~-[sυ{74Z,;ܿqS}W!m%r`H% x-YB}M_rV|Q@+5̅T!SW{zz-p}%>E!gvvk;wfG¥D^SrXF"H6f H0===J$YYYZ{AXjy^(_5P戴)BM}}}Mkm:3l)TxP5zqEB>MӺ{:u*N:(--UW]71ﺮ UUU>o1{llcp8? E)3f{,<\6bGt]()"rss#999ݻw{eY6)qs{ /~ט0fKo1c .[uu?ڑeڧ +dDiXE!>U 8RE{ w\uUnbɒc ZJ~e.. BVu@ 05>i4 .v듢%s3CY`Ts4԰+ lٲ%;ڐ28kh%fK90ã5$Ҝ(HsWUUuhߎTڗQSL$?ZB1܌{̺8 .d;cطoy$(07^?|_WoVPP]GRo+q!Ic͚5Pyyy"ZJ6D"a]bBЃpEQT%.3035ÖT8dD]555htsu$(`GG.KOJ?>?-o,2;i$H뼡)I5ʯiO@\tsڵu^xٮ=] p$ιͦȰ%(6YPkɰ)2^ ee٘4F=fZXtrq۝M-ix~~",AI~~ǗroF(?gǛV*b2ђ Z `ѢE?~{{{ޯ~+ٹ u\ᜣV+ICG9WUzb `6 ~v(c,٥hN45p8D䰣"d4(s!i )+>␇bu3Á@@uuu@R>D3xGXc$IN{WWRTTxvIb eSF}}#v-aβ ]=*E9ˡ8]YrQ$v;dQg^u_eی׺Xd/ֆO8auu7jn]8k|R$Id!2)"(V!z]?֤5KEծZ-712{dpÌ1v!`rVV?c>Et⫝̸rl_&QNBu*yQIIIP4y]]';;[-** E#=4#GnNUUyaaa`e 'oɃlbiJ }[燮]jh7hlk x$Ucp*,QU *ʊ1ڃeL_A3 ("v& !";XӍ*vG{",YD#,32He ";g N'O?WMJhZdʕ+ŤIu].Ifi6m*rXHο""S>&2ڈh7 !;Knooyyy Ikk#Zi$52F7C6zƍ&M]eDLDK䎐 7a:ç7=3n[7nեl߾=(^!2q>G%Lt]gJ{{"-A%6%msjHQ`QL$iŖ ]^_?]oQھ55b.Dfs(- 5^v S4"a ==RʚZ֦-5U@$#hiUyy. [׿W_}5v{ˬYz*Z[[AD:Ғ6o}X$ITQycqs?c,!600ΝVr! BPh9\N0\ pןlĉw5*a03&LH||D(ڛ-BPnnԾ>. !<Sp__200 vI&Esb)M`^W6`@R.4N?־Α({k=.ؿIٴeFVmڀdsh$';{FI%?" 16H5 x# !!>7߯#hv ]q㩧 .ˇ|n:,\0cD"X~=O]{uCdo{O>馛\.f5lb"&"&0BA1]"^!D$IͲ,w8p kYk-XVX͓e96YC0~[eRHe,D3(Wl~B5shGB0<,5///t:usڔ"E]yooaB𼼼paaa( 3@J!'VZ\nDD$IBss24fM CU6G:aR:a,i MLSco|kS}$!~ 9ͩO7짚vɮMa7555(,,ľ};g#zC/_u~~>f͚{/M ,^> 222jѳgO|ݻwi믰Z5j֬YݻwXr%vڅo~=z۶mèQ0oYf7ހJ0h48s%> Hr5Pt/ g!h4HHHdv ?׭[G BMFQZp:"EiԩBFF첲Fc/0R8*Y ںm۶;vL@u:@Z(|_Co-;V%z`08e cv̿g8Pڷw8_C L5MajGNdeA::J?еkWt<Aܹs|r0 CFJJJi+馛0j(ؽ{7C0qDt={9993g,XQ1yd[oEΝѫW/ ..nJ)40iҤ@q0Rtu\gϾ\G9dȐ)ZynVt5hС?v^>J)5_06iIIIgj[j mc*dClUNjr譩y8;bYVrYR[׶tki,o7LQ4|98gyb ] qFy711z^e{?ȗ_r5F~wwM9ފ G1wA;lqO$Mq?;y* YYY;wbȑ `oPZZ`U ^4L8C ZPܹ7x#9h4{1sL? VѣG}|7CVC@Ex*K IBn{lyy9/YHEj,*ֵcq0Lٳgu HBvx=ޯmr.cyȺv]HӇ |7n51n,.s2 Caaa<0QօwA3߫Kׯ[6!ucf[p<Sڀ  #>K:A:2nlp?uTL:f̘3fh4| IDATK t$22s߾}EI:mT U;P>["EWFYnr2h4rR*"DQ$,^} (`v3999T]}"60YwyCMZ׋QMy\ \[ M.ꠠ L.ݭ!v#ޱHm> a#ICB4D=RDMOhUe^^z!&&悟App0r!==7|EY=zD_r8qܸq͛&|(Wgj5NN>uVt5hȐ!rss׻n x+Yyu:J*p7dggkU*U 6{cvyP)WwEӫryC}^M~#nM>r\VP lb"##=_1b}{F*]wb46Hfk°,j.'/fyr _WRf;v ǎÙ3g.^nBe0N f 0 hZ)[բ]RF2e>ʚW $ȉuj:e=j/( pU|)RJNv@'"F rӧO뒒; [dA| ',ҽ;ջqud<ܞU,# ' 1z׋Fnlso;yyy8qDQzv3})0{)[DTDw޿%3y쥴Utըs΂xnÿpM_oZ_W@8rVb@x[FEEYje#&#~)F4.r+~}<kjDN3g;vz]enŸu?'ɧs8:>y_;TG@)JmW9m6" !Aj'OM~`goK ѣ;w.WfӦMuΏ?A) Q/XrW"ELeeAAnBC)% HskDMHNP(uBpI=qYnU۹]!uAe7 xC4OTN[}AXu `LMMmuuY@a=xjq{)/-`O)O?HɉYpJ&E0j|kXHF KFv sGܳyWK~x%qqq9r$,Yٌ+d:u /Ə?6mڵkV333h"[999Xd 6nɽ%;;| }}mۆ ~w|gػw/ncӦMXbNgqF>|ط[nE˖-}O{ÕF)ozdKAWuBo!`(p{N:fߗGe| <Ca?0~7+ЕTWH Bu?b8NG@ըrwTMC{XFx=|6,EwXU^`gAɃ)v@h(}l|{nw+F/1UVgϞ E޽t:xb_/@E<# BnݪLjbΜ9իDQľ}sE=t9Ç0O<JJJpc޽Xp!zͅ^Gvvhobҥٳ'ĉtS(+R"E.!CPXXPIyRxAPJIHH'88?w0 @봺`=o> *j/Upu/BSHA tRng^{UvV%uҫTl PIi z-x,]?Bʼn`6yz S߇ +Tpf4k8!<>釷t'o\ǯЮ];8N̝;}}^zO>hݺuh4F0 ֭(O1h RjO;SNlp>ܾ}{8p?0ڷo0@6mpw'?S\.رrbVtUS6"EWxZTTtFN I+e ݽQcd2yӧOZhP:"= ? I+{A~0 -))Qiڶmkg&a"o9,]u 'r{7v;oVd XF'׶wUlh4v@DH--ˮBpPƥ đc*~a DZ?'sFIscuZPgq!%%|l6 ={bx<8t`8tkdffVhoݻWbmcVRrr2t:@*'kEW"EW(,,|b8 D@KGbU8ںukӧu5t$`gjL9U*lcTTFlky[WxI@i4ՠhQQDuZ>BúTFA KA&=[ٳ&/F(FŌxcn B$+`Նkpt0*0 K iyt4,=K|n5oе~ߩonC8hhi--Zػw+SO=DFзo_`Y0`9шٳg 鮻™3g-VJUS5y{3(j"ڸq{[d0xa(˲SdY,RezDz}@kl6ky',b{,^rA ]VHy$`JzqvK$'=zԐjB]lZ׹9SVV)*v 8#pχ9'0{0zv]9ᚺsWv➽g<,PfR֭CqC Dp}`YH@)Bp vܵ/?D~ 6c8_{׏_8(u#Wq޽{"eee8vXZ8l߽{JFtt܅`W"x)RT'.((k*,'ח+ 0:D|v>H СC &c244?q>$$7Lyj+GA}.մ߭R6x `j w \wP-DKvcǎAٯӤ4w (b|=+OK8`;7)iY[3b}̒a4gM"zsi[na`+F ȶG%ص۬~?r296=AYG"x)Rt4`q۶m JLL@x,ʨ(DŬ#}v/X?k%''ۋfYjp4us];)X1/PB0`0 N'k2 5t}dӨP, jjjj='V$(}/"cՁ~tbbz*#kןw<4::] .։31$ *sg6u;w]w1}ߜãy)RW"EXIIIJ? +LZ`Gz}% s @, wС֭[۽+t[@[b9Q?-Ebϝ;ZlRRCRQT8Rît̃3Y:pُ.ֱAȔB̳[rMu s6? N,ľ/.BD1 Ra@nQ\k(H)6lXիTTk㍨HA`*DQ˲c>H/Kz4˛'aFddeeT*5LQ G:%.SW;j)uO5wQQngsrr&Ӿ}{ Lu)Nn]>)7W?+Ȟ9sT^^zxwE }җc'Oo//l++uM*z=GO-bxhBJRH^E].1bMjkÉ<խuV vWiۂ MLLt8jj1ОԳZ¯x^VmQ/pGffz^hݺ8ž%ؕol?RnQQt,!df)RԴtZS*fgg?k۩I Iـc}@OFF/$=ZiѢnr8QAQ_0ߵwaLm6gϞhժ#""9"+M_֣> 2""㭺Qp8Ģ4hUH#{5 >)q8jeϝ;5L(١1rq e à3^/F>44qMx!ϮXPU)LrfOkǞ \.zkˣ- !6BNxSOinhPҘGii)A,s!˫kf!aaj10)55+zBH$aAEɡaÆ+iRSSovIa rׯj:vX/|ae˖;%HP/,ڒ\+5MaaajeU^aJ)`ye%"W,\.SVVy<N:udnPNӒ`W KHxgf󁂂M~C 1lٲp8Pt^;v!AAAZDQ=KWnKqqqо}5\܈_R*x|jr941 -:(<ې!C[˽sLyRRҳzJEN2#JdK_yE} tNH#pARy*&\)t:٬N"##=FF}{2{1R"]G?+o*!^=vok &Mb?E˗e˖СAhpw?NjR7 -^ļSn%KwyxM~Υ(bɒ%h4"! 0KшMmSSSU8ZMÉVmIEl6K 7pCɓK<Æ 6Qc}So %9..nLrrrޔ)S$N09vڍ7 ;eYO@}S@`cViڨ`BJyh(XURRx$)&<2p,[ZVqܯ-Zxf/.y[, o׮;v,ILLlytٲeZeٝZcr-ZdlJBB&N˲O96$(Vzk&`HPɸnFB<5q'QMYQJEYj4Q Nw<*F]x'J:k4i]<g6 fiiiǚ9f̙f ՚<,WpPaHBBbccJi!/Q׺-{=e˖nȑL.]{tPJiDD irχ h4hժ (~Eq~1ڵkTA8,.FGFFҷzjPJ4!!!D}.)):j(*l2}ݗcS *'Ƽy0}tHHH(իѽ{w̝;s̩ .k֨Vg`0` jY!Qө8/⠮&5Ajh?XhZo1cBzj:/GQQW_;yRvvmhӦ }xʰpB~Ji$˲!{\)9rd޼yƺ cA+ZeY $@ 2]"s1PDeP+\Bw kL&˙N'F=UzḀjN<4wa`ZRDJ Ƚ~Offe[%Ko馒]et:JbV+VQS~Slj eee('|;&oYQomȅRc3gf͚ۍݻwc߾}@.]0rHX,, ٌ'OJ0gXp18lZ8'ODvv6t:RRR|vG@||`٠h0i$x<0 Ǐ`/Cjj*n7^yz|={6cҤIDǎ!֯_-ZO>4iϟJ)N<#11#Fe|>7o9r7oƾ}ЦM,YӦMï)S &&&M AХKEؿ?Zj >?N2O}|AвeKk4x}3f @Aqq1RRR@)Ett4t:ۇ@!BDQdBBB ᮻzy͚53FcN{m%/[v`K*WV+E5Mn;V]J7|2`K"EV]Ƿ\.<0sL9U.\C ojꫯkѢ vM6aʔ)eZZ<|駈@HHBBB> M3''?!裏Я_?|IF+k,EDDUV5k$#@ӁsĆ |rlٲ~r\l`fR ͆3f ??x7лwoDGGWYxٳNqSNEFFӱtRRjdggar!99i(9ל;wϩS>,**bQ|JT/k)OMM`0k),[9ȗַ2 S h7Ҿ+cө;wW`W"E 6DڵCϞ='.!!!0Lcb޼y:u*Z 6 ǁDl޼?L&yX,={6FN:Gzz:"""+V ##;w1sL<`hݺ5bccp+P/~Go0e?~{QPP h4|WXz5?^)B+V?ܹs1c 8Nd|͘0aJKK1l0_ʊNx zk8|0M{SN߿?~dff?+Vy̘1݋p୷޺b=ty睯:ucǎrݬzy"|6qH%&FLBIV`j(Fk,Еw@A-mL?o.)//w8qbٿġCWE]555uR(91}bp^_uh||K&鑨(˲nkW&Z^j%_)5@>SEj؀O_6rorwqd/1 #꒒fÇoUF"EmDmʺ ]%$$QQQhѢvt/T_۲^MWZz58.,$$d]-3 ^la(Us.5|6%̧qHreR)Wl]$??_(((#t(^"EMI AݺuCddd"ڦ[]ԵV{Jӈ#("=֯_H||Z@)QD |+wsK.J5 |^\9;cلvSF"EUOVV׺Hfk׮]gXcccjK9PJ}- K?Hȅ`M r ~*TEt l6.??j>l6M0nE)[@ْT_a2 f0~aa\\ర0=qRJEQ$4K%o75|[rp"OPQVliEOWF¾f 0M͊)RvnO?qUTYs-[4vut Sp4ի;y_HHQ (" ח{WF%pyRHKaN'W^^^j]4rCVUH Lyy9/^lɚ60ǁhZgD 5HVטwn1b/lܸq`xB?RܒcKoѽgzo%Gq+wcT&l6ef6dU)RoE@8)|ϛ6_hZF+O&Y :`_3f̜jR", yTv%<\ƕͰƹZH<)///aZ5jT2B)Roݡ FFFb֬Y4tU'a`pw:)j\-YD\dS+VH i2t#""T^X_W*xpr6uEi+H"%8JKKY՚]^^>b0|ʈTH 9;{; 9ŇJzR ncUoGKѣGgE1l6/%66V%荶ނ6ѯK[mS\,#G#]2.777t}*v)R1 s\] ˲nf}R!h@].~)߿,+Vhxdd &FkZQ:7m̼,\)+ V+5@J;4T8Ct:_ʾ7(M"E3p;6رc}N:5ɓy*A4RV"yK-LJ /J\$e7 2 ̫шj5Zݠy*R;v>k+ڄM7 ]Z-:H^_yԷP >z"?_Hz|Q\=Y/-Tyv/*#)Rt9ɓXf :, B"66s4 -[<#%%@+öt)47!'1c@N/x`0;AA Ur&A8{oaYӡ0QPX۲e-.lKR +x10LPqx@.Gبz25=8q~A:hYuSTAj]^& ^pCR >pZ!i @E rT6|jnmĈH"Ex !z | hsaشi0yd8pQQQp8GNN@/QQ2i#G/pl܈FѓOBݠxD^ 0 W^Q믡ba!kB7p B{](>njsf VGk%=ȵkQ8v,]jtd-X!Z,WR))55u#+WpVfY6UӅ9_Q~O9 ǪSuV2˰J򨮬X,2˕|YxJKKKK^^)Rtoii)n6l|T*ߏTcǎɓ.!lTqpmߎȵkA׬`K8Rڶ@ hPBa!ܙP 0 µs'"ӡ}KPu r%¿ E A_krrxoaᬨ)-- 3uڵRt:88Ȳ,GaYe8չ<ZA_Z(Ao:QEf++))9â [hњ5k( )Rt/PVjgR " ʰuV$$$:w<F!""B~!'?8P\۶Khnŋ+R D(:>- o ?1BE.PX퀷`Mծ"/kWؖ.g˖p` Rb0^x&7PV{`o&>88J2jp^JF jNjZ-Rx/(I J)q:jvݼ(6ñrz<2~TiH"EW1c ,, ?0믿D`ݺup8{w^G_PVAn۶0?N 2#`3PQD𐦶IDAT)8C"V_JЬ:%'Cݥ m+ nqL$&"G+>v#bJ j lBWDA ~T*jժ[M 8b(^ELcǎƍ7~VBٳU*M,{l׽93ϱvm؀k`BLh"zRJ\RHTI (KKB-B QHInK nMҴ<(i>gq~5*-K#Ynd0SՈmr]5 2gn ?Zq1Wѿt)N@ [gn g&L@W ?sދ3uuHYTS8]SՐss85f `.Ό%%  e8]U?A7Z,\tw:c1+;#> m`/":f\@vC~ܮG:7f(ZCٳ͙C02.Xl{z%DQ+ۀ', zM O>  =~y`ף]6 ̘'@QddΞѝHڕϻ͛b1\՛cv{n8[UY@<T if7d`*@Zׇ;%|w1H x'ObeYO^_˳.3f4%eeDu74i`Y|u1c]jlڴ ͐Rqw}xxb̟?b֬Yhooi1('" qg]mпbEu姗 ~c45K"H- |3?F^<~!|e20}~[!~P\y4{~t*cUxbhooGSS,Z GEcc#J#`޽'`7}D{K&Q};c!Pd ÇA >T2vJ ]B, K;Hۡяַb1f͂/9YXQw L"ٞܶ /s0T*dPܜyβA4dIɠh1c PRRk24qw\1;(%x!"غ qH))lٲLvt:͍1T*05 !(Jqe18"SuwwCy4 탔RT |>߰(!9!Dy r MӰ~躎p8LR/&1N' F8dîx*|>JKK= !\cH$%t]WMGQرa (!B8RJ7N6ci>'޽{e>(\4a `D" @D:;;`裏7D8ma@)7"0 )"(JcXuyӧwד4w\QSSP(4U"JdRJR* !DypGmm9sNkii~Hs#m8<~lSuu5WJxX`18z-tkkk}̙ϝ;'7mq7nN#MӄiTUUH$)NQeYQF kںuD"8q⚮.-Em3"{m4f*,,<ϋ8ID]EQe18^yeY֩ѣGn"˲R#] Dii)2>pX)u̶pm۩t:}\mSVVFEEEBxm#887zcXvɒ%PM4!D@Lj׿UJ)"r1""={϶}glr-z34M0 DRjI2ntBADd !RQ vZcqnV1|߯ANGMR` àD"۽;w^7oAiRJ)]Q4 uww;t萷k.U1VViavxIENDB`weboob-1.2/docs/source/_static/favicon.ico000066400000000000000000000263311303450110500206250ustar00rootroot00000000000000PNG  IHDRPQEM!sRGBbKGD pHYs B(xtIME :]htEXtCommentCreated with GIMPW IDATxُeə"~{V־f7Cq`FֱlX"d Y3a136v3r_nlលy+{[ PB8//|#~:AЂaAF//>`c0$4x/3Al@VQ|`ArW(V\um!dìTf0:)D k/$s)V}|$!#쩙Ejz/Kp% `aF&_†،tBs<l uY(s[oMS':6meۓ-1Jk{`w3544~g r_\V\חsRCc`R~Xɥqd@:ўVI [Fx5j>mȹ+Xӓ ~.({dIA2 D`&X b (Z_ k)g/W^{_l۝?/O C!Z;w vy`N㝷5P1jio'߾D(-3g|c!f-iwu}mIT%`c&1$Aѭ#|_~Oڵdp J.^|)t\W2&yP͍Uy{yf޼&}YGGo?'P71XedUBȇs6m=7mN%@Q>>xCGZdgrο0i[NyiBz%-b0lٰ!4(+8i`'@$*Fzs zQR).-\*TU"/kx5i $wst(+^J$Qs\ǧ|fIOxZL&m)]hvvO7 (UY&(i4B̛+} غt,ǧ3ӧ|H]NҖeg% - Q-$e@J&a^麮GWT㝽_K/]y9x^ƴNh?G5SMD;1ZiE^;?9 es窶W8b? 4\8rzIXDPD'viRX7^:I^q.OL~}R=DscG4n\鿬jNڅwjZͱr6O-\x" DBXvI,|A7Y7'\ZP(՗/S gx'n~\۬[< !\ldb,A!9ߒ*M(~/j%ؼ&.~^yΜU :6 =}%@2:l\YBJE$-FCP_kON,x@ 9u7`N`'JBEjJc+ Vj@a@@= nxm!>l?.NN·TW_I_* O,潕wח`u>6U~>!NncT,11^ߺw>6{(5 !jI^a+S>K]xeZXV'/\x! ;8NQ)7d~7ǵMYQ`*\P*i)cL<3{jt08?=̧[:uoz+k7vef !g>>Jy"rloBtr|qV{:X0a-/:btbj4 V+_HƵ&&bETk'2|mb__r8:!Ot93qsbl$e3fF;"oŠ|?' !nnLnﮝcf+_paBC(+fN}`LhD˵] MDŽYϵߠSzt_9ǏM4;lk[گZ=Ta<8 -QYŀEDry338 V fF7="*؎"Avۗl/D]ҭBb>'@\;WCErv%8 %,ɩY& @c{{g+^}iXwNX4 - &,v6 馛^a2-%Yv\Z|rٶk=7Z1\Tf*YdRX$ Yo#p+ )G+zkT g>^X8bGK#.쎺pU ˲RoAn F_H4s Q*+!It+k776m;|kl˱NpBYVwr~OzV)+`$H"Rh>\+eY̩o2gu"}+m(+5MN-t^yl5x4Vk~KtL!zRai$$,!Nwe\"nn@'2uB e\KV?N'ܱpDih!ZXآ\ x/GjK{{akWٞoc@%weV( m恩?\xE~U<YI[.Ib"Ha$AY{t1g;5U)ϐ,8lrFTXos{E2H*ni4!H |!gnOinc?qe]X)珿&l 7`Ѫf8Jdd2}#0d$I$Ǻ$q=7H>U$7h"MGJc;}e9(xd>_Ok+3`H&}/9OHᎏϤ`i4v&.jJu<ͲCG4m4ZYJ 󨸽(Rc+7y< 1 R `BZRXBl.0^u^]*`ΒO`ə3ODYBNicH0,yJ>6mIʕ'l6k`B%>޽^M2kZ819'Ka& 8'd+m1l 4&M - vo3)䫝b{fvIlCNO.gO#6F2Xt6,Y,~ 5X#6~#HXX:t~9[!ǀ|^v\\Jlrb^\<BA`R :[FTư }7hRdH l9x89TYfN?<=`ޯ1^0sð?{Y*q$G4-QIJDGQ1g/@W+5h9ҀTCǀ0Fvlhrr>o7;  ) sCX:Q'c`1'&# V79ARNGT*Vn, W9^KGd}!rOgvrKc$r]?:oOo zHŽ\s'hlcHV+G~ǡ xg Tû*Q"H۶q;:C 0l-BI MVY$`V>Ȕ#z$|c=ߜ_H*݈lr’!4 b BHN%-Lљ4Or,eNy(hA`A8/C(C{`OH)c1HCĚFTi3idP5,91{3-ϟy5Zml/=޸Sbf+NBBDDOu!X' dAmEyZ/*B i~бJ(²-œ㸚& &,8z:MRBܦLC6ZazI!5XrAQeXJvs,f5m=ν+۶sA)-L˥zDNBp߲4+"~j1RuL2 | <\aF.($~AddfԅV6ww-"҂:` ΡO˵|!0lfCAN$Ddmm3 XRcލ7htϪC\n^8z4:iu]2Ls~%589^QI`fZˬ6d0!t-"2BSOw+@H@ Z_J̞@$.0B1F(äNje3Ҝd1lnWY8fƖ5%ABW.;o%mKzc<z RT lǑe }Z+l HQ6[b55F[/u-i,I$MAQHsbRonm6sRiӠkS*(~0jթh46-wP^]~a{;iwؠS(Fsv<Bbvtz'*Ym9@tJS8{̀"Pz5@=XRxfQln 8T'&ZhcH&[a-DeF͝"c@KJz2ھ_#OM.5''ڣ%I$*\Plx^.yD*"c CۇMtlzBZjPn4v ;K3#sYBdL !R!W"<08,‚=[(LJBs&vÓkm#\I P6c*q"&ՙ Wq|qg\-@0͓7Xe:dfm L# 9.v/Dfkk419@et:۰J=xp$z}zZ<0zDq_ 4WEi*5PĢm䐑^E'9E=7;L=YlWkQ*ug4Q T)"(rSKoZq{Sq0(6Fρ1f3詡,8Y77x3F2(˼QUo%)K_5 xuV1Yƫ{,6"Նa7l_H0`*Fu_/6.3F+ 胉4-t}lш@ bXGY aãZS1/w{W<,0Q&wPx||4Q%Dv`wr|[T4f"InVɆLf#:NB1^2*D3D-.<;`H$*D44Ѭ0iYjD}[K{. )K6Fc'/W;))j&¢RZzx)Mwc$_k֞{*Fx'KhHM/860:^x2OOI !~ TJ j?ׯxmA4ix6rBX_0 xmv@T, m'IN,TxI5#=ov:y"R/>6Wٱ>6&& b'+!T赻{2#;X>sZ`g D)I}띅}R fbI=7ޕ0 wQ'f+-T#S21::q?\PL20'M32!HBtyM)t}ð#태 j:׫UCkS8]M̺iʓ'59{LDD Lњb""!"A"^="z N,NoO1 y=SL Dd 3w{uctz}JG3k?!""59^=N@=޼[X(U)6F=OzaҤiAFtSurTmQ"bAQD$bA"&2\>s/o !!3sX;eb)i貙 BbDe*! 4wGӭ>oOj bmGvz/!m;5H@tOg47a܏mˢ ȫ˶藾+KZn֪!+ܝ^j<+lÇw>w񅇻x#$Y?ѯ0ٙ,⾝}Qx!jjri{E!~Tsnk둜ZPt'OZ3<oUXPL w^zpeF-CZ w7;[{a;&ۀ.Рmx헯߼'gèga@Hr• C2ap`J(.+TwS#,x:/꽭ǽs?ML̶ +qȅHWߺ$3wNT{x!T/N0X7QP&9sM $5B2?|?תS_[Kn^J~0<^St"jIiu/mhu g577W3{>+1Q0[Xikwnt_v|PVn6kO-Νw;emfctd.zԷ~'#T"6UNO{6&&kkgef>w JH'<Å,_yoVwUr+IyJ|V\f\\sR꺾aF]iQsalzƘ3sw.~ !xa^7ڨ4NB!|LU<7՝v{@jyڕ/}h[N: }w˛['($Ź Jo8e4bYRuL%Y 3~س̂a3 &2huJ%WҵY?-~?S8M-quB9Nca$"{7pn| 80`NOBB̹L @)3%Ƙ?u+} L!,acl43+F cuRAwb/xkc.ye|Bs|rE>,nDa0{x꣛qҗ ; G&:I=Otz%DQ#G CdEobovݾ$ W߃BXxfp/_̧&*cT|RZ mۍ,?{o=1`k!JEUm`ui))""Db('`JMFߡ󿸻{G`u(v+I?׮9U|>?u.g_'O7{{޽q,Q;AD(Kijk sׅ"/!bF4k_Iy:5̝{+3O.}r}0 ytnJ2,Ƕf= Bod rH$<3.?^RQN;( rԭ+%>VƧ\@ xد|DQ*ÉZ$Bge RF L `w\7g~6,VVIŝ??~<4q:2aNK2e#z~c1ضJ)%A3X*֭.v}hq\K>_K0}&(K0Z3+H`(IEa/_ ]߻H%!]?oMϹXcӰ D3B[A$i$?_>.\\lʄpm3um|/~j쟯DIENDB`weboob-1.2/docs/source/_static/logo.png000066400000000000000000000303041303450110500201450ustar00rootroot00000000000000PNG  IHDR`X=bKGDNcQ pHYs  tIME3#- IDATx}y$yˬkg[Zaɒ]K( , c"p^v%m`GUѴ2ac0,{%}_3;wU}Yzfgz՝~w&{'D*JD {wSzy&w~^q7䷬@!r~&y%H   P(AeK(, f.Ql}S/ίD5ȥ1 ˆl&"Ѝ*V!Q*&ޟW"86Ar秖D#u 󮟐 @&t%ɲt$QIQI@;Pů4H fJlEDT]Bl\pX"@,l,m"%"%9bQ"UeZE5SNCPC޹31)>D@a@d9#月&5*Rx =w\XN6$3[4.cqb.1`@ LJ WTd eZ{1RD٘3OטYHؘwn>^0dD\X,UͲļ41f  u0۴?$n;̀zP=PzK^?p߽" UtyDx}H@4_&^o%JYEѻ $<6/#f> 82U͍1̼4(:-ĔF@FL ZgN#J\#?Jf 8-LDpAD@̪jefb `L[Q0:Z"%AdW}d̝ 4J5cc6Wlx :i^d+[?}U(8.^^jwH,˲ C+lXۈvл ƈ1i^ 5*X-1?~e'ǨMYHH DTIfpCy<CdJb"JD@flKG lJ'VSJ"r!@ C!)b.kW8ub oqcneynS GP"!O/,{rWQJ۰R1֮ fEܦɟjcX) hTEPQ$GSWlzu٘ާ\CUͲY"ef >VpTVE:nRk''U`ZZx˴mDT䪌3Uϑe cȇ2i$h4><@f E&9Ǭ3MۈȚ,[irSC$0sߋfSN@l 4,AT5/7CKu4Hr-⼲5OŹ V-4ujLr "lxi6h6P2d٘Y٘cqQYH7KTW\x]~>>".9 =Ṋv[UuVU4t@DU+uTF/ؚ SH㹲1EF[@d2Q&dHeÇ󩩯14t> aVN~Ka*6HK̳1;.^fwoVmQJݐǘ 3$;IYZUY5LSIg@`P9?Y r1yjbO M:/-ux ͲMv<{K"SJW4*&NmA2# ` 7| ?Ɓ*C QۼTTUTuyLt 9؉5V>HKf"f#qH"5rs1c4j(*;oQ/J *N`5H4h噊x `s@ӪͲ>|hs4˳/J?gVEoRթ&88VQZ3uj{iSlmF 6y(T}yهYHGZJ7qVQDwV:PF(ͲoS\g3hr6+pT E( Kgc!C\ Dm=M4u!?<*_TT,1F&7ߦ;"8B>15OHƄWޟFcӉ͚ 4ŭ^ ٵQѤ]XD<IPiT cHxw&UIUl%i:_ö "r]!(F#l0Ddk>w&U @.bj7DƻUPGd8!%Hd M,cј&(h>WfW8 R["h(9r[GSIbX2|(ATtCG#0oZX^ Vm! Źy|sayi3+AM[_]X^b7x&r}WzB3-?ٿ`*ض!=,^n12^miNYy%a |,AjDqe{G#T9,(8&ٮ1نh| LQ /~ 㕯\PdžH}c]ǖ> Ko?"@Nv?}g| էpaOF`,qX0j\|<_奯GsP6!uXj@杻>ƨ+e6/wVײni r@N:"dg]1ADPJX֨"rlT&f~yf f0ƃ޸4&s0*>GE0T~}ODp s{]?2NSGuq߳v ]~.J&"Bjj&WV 1ay -fVѮw~{ou"~dW~`ZÆ1TCB6( qZm `x[]Z5$I=/s! >g{{C@f^" +rv.^>߯cd2!`!7&A<*V[2rDjoPy-ο -}d*1^K|^/_ٸĤQJW߉Բ1FWwk Tkoxgg g~d̄d 7-KC? bWVģlٲ_'N[&HZs*ey٘MR1Cٖ#rf>HLcGhǗƲ(;ZJʗ;\Q h [ŹyU'o:#@pbU^WlVKUEZ6-NRI U)+;tQ{yW!6 gxjU"xԅ< ۰=,ߤyW3W?AҤUdtf?ݷ I8Ob53kFJ_ /XX^lo&9ɓ-g21w²{qb "ƘFY"/&.nt8%lBY0*h0BN*/Wkk9HLp9P82I&[i?$JL9SXZwX 9;ye x(ņ奶{?D÷ "~ʇQUnwDd7̫O\1 sdaIVխ1J7T ZBd+Ĺk'cN'l'XLsULk0ωNZ ߫Sx͙w5?#_}ay)kH3qU:E7YW-ocm®a6LW0gs۫yK0x9a m kxIxMWkpL\BNGk}.Qrh6;CUŵ[U(Bied}ٞ~ [o6Hd|o|@mM182FdL\K0У= իxicWzӮ(noh9QgϲcbP<5m1]at矪"YqKD2uOXMI}n AHhMA""MADkT6#`ѳyݘ͋. 55. 7ұ.( 0s'g>mmW|).|ݖڻkm*<򁫣WOaUc`<5|h2YRlsH}RUy%F+$R0yT~E:غHX1tjTQHD'0hޯB;h>[h0cW;zPB^38#}[9^s/ش6{q錝^XX^a|NJF`]4Sb3a`"7bkX+C:yԮŮM.~*P2WH?a4#U{em{0蔽Ev'b>Q|ʎաk$JпgE;ٿo8|Ci?%j1% [X^Ƴ24LM ߜ,D$zHljé*ㅍ9XtJĄ jeT4PJ41E^WHU+^z3׹լWFP3QQL؞.o$Q4?O_kx$+ r%^]|1`'h["y-,/=87Z=+C!Q'Mؘ5/n&= *jC!ƚ(*2 xLꈋsT.#μlڥչ+;-R%cAUzHXL[jDdS% c)R:&_[~ǩإoE{07.x=TZO.YX^ --qsG_S 5rz`W<87Rl$5@3$?.,/תex0\ y/ݎz2y"u#ߍ(K`U {̼N*Fxz e"MTU)qd(y !-"uKGjP 1)TcLHvI]gEJcnܘmq+m[(^Vo6va t3sJ 6Ѡ)uq`3m"ZͧyTgzXQEDtWEC߳1OŪCgDG1iWD Q`8ږ5WC6~k3ɲ'Z* @8!Jo $qg&]#j?3m?|_H$0>_9֨f 7!GeUQt,BwӴ@$LT59SxZ1|jAO aRI6̈Px﯈P[]$mUes#þa 1zU-XˢE& ٰ8 tVY7mpL&̣ KqW<%-eqnO[brQK]bi(pŵ5Kl&I PGCnA}@(P-Ic0REa~lpx_z6˞i^<2{+Zg6G<[xQ: yڭ{{}ӣ,J f,,/bc8j1ɒJ~]$U[˘'^DidHS990*if_ĉש6@P4[X .k_aJ4gTuBbwq gn8 2nhX^3' 9_([$ )ˡ%QtLիD׿)א6[L*xb^abbR"$غG#=*Q(G HNgUdhHө)^i,f "X^Nwkmg绷@87<2~oQP2c<xcYX^e{}pL>аj5O!87#WUiy@+M*:U~.:U&&"/V M۶xÆF1*z[W 6ϿL%M!w*c qRj,TMIlhZumO(5u8V60F.!*R.2(O@UT4N_ũW9U/;Ա7D"XhXt4oqnvE٨|ayis_m檢1-"d rGU>rV6PLewJuE`pISK"Z~j~@IHG%#kRJ8Gщs7I &ó8䰈TdéR߳ߏ4U*wϴ6L,DY`yhey*VdH"5Ej4p@OD2!}wc2յyn/o0MD]ea*S_+0U/l'"qj Cms[cDg`8OcjoNo;ivԌ*bs 7YF {N|H ˼&,_3>oGP%Nua !{;Mb? zk$oR(Ĺk%t |vω+0(.cǢAiENgdݵ[c*j}N r;ʖ-5e_4A"-GۃUMT~#<na<MZ!"/ [U!em"rej1Cl̳gRe?fiT0qm;;H3(ATc yB5g_pE.ukV\yE6@44phu }6({ L4xP4D֒;wxyW}x編t 32D!< :6ȃduQu ]v \:1GwB{;:MaOĴNTDkOcCe?EmK \"߄A˪|I uNDӾtS6pi! m"_:UAzd٣U0u> hPDT* 4}fh&-z1 JC2~)#$e{؂"819^M3lZ4RGP#a&߫٘GbȾQkb*ʫtCr ADԷyePm 'ucLf˶g "6y5毻o;oH3-EaՎc]C4uf? J:dyRၴHJ "(4ЬkY'/M*M/D`h_ Ï:^m,]UPB˫ ? )|&+YSVcA 8@ a(+QZyJw{JoP:3$De4EPdq">A_)@(6Fm?y_ɅEc=8yÄ v,YC{2Ո8m\Q\%o%p*·cbwVPMcԙ LlCa$eDy~PU˻o+SUAjt2E+#ɝm )˦R٘bJrIF ky b&'0$C[o4ĉs3,ѷkLO=+GUăYTČ;kVpW>BaZ6i0T/Tql,ZK brwZ @.66k} ^iu:/b84ޣu׿VUgZ$+n}\ETrTFA2Q#sPU*Y/W'c I$1"7& p<iTZ hP'bk.C:w^*f^3֮s~C2b*YDfYT?OM= hqV{MP[3zyU[KUw6Ͼڰ?dQ ?Ex4{#vB[쎾}{d1ѱ|kqs:+)AE JAR#c1etw߱=|o,.J#ƠPBl!iEd+7hEU$νZT) X1r8@Q!w!,;w~#FDg[ÇMf<+w5qxVu]!T֤7MFDGf>}F)LR5*u4R l7ACQPjIDC'y{坟Xͼ[aeosDMu3N[Pq0A1 0@">듔_1-Uߏ @S?΋گ0;܅Mr@Vbx5)ulm+?h$X "' ~kizA Uj1&Aea P4$gu5ֈ13P@>&֪bl@*r%rkRu>b]rd^֨Tђʻ~\H4}Ɵ Tmh3pƚoVu'^ KtWh.V/g8 &^U{DU.+z4,(sPGc\uR3VYbѐWG4ƫ  "dW舵BfuDp>ܷ|D[]Abz IxU(#}ӭGs]ꥎIW}Z,/@n2{,! 2hl0ܟBCc8ĤMo,|'B=l"A&TKilӜλTŐRBŋxq 8&)E2ffc#sýS Ҝ+M1_Ep *  4|9kBD*3s,IZ9W"""8&*+Q ! XD~ ) J@!S W]UDs:TUUHW7Ĝ6˶ d>%Ɯob?]ECnk(#jbuG.g5+1 cLTlJt$3sTUWO@TTCB"X_DӋ9M2$!1f3ݮR{!):PPsWgO2@$ c:<XiJC,(C?r< @& Ǩ0cΌ1UbW[ez^jfH b :Jr)? :y@IENDB`weboob-1.2/docs/source/_templates/000077500000000000000000000000001303450110500172065ustar00rootroot00000000000000weboob-1.2/docs/source/_templates/indexcontent.html000066400000000000000000000127261303450110500226060ustar00rootroot00000000000000{% extends "defindex.html" %} {% block tables %}

Note

This is the developer documentation. If you are not a developer and you are looking for information about the project, check the official website.

Hello world:

>>> from weboob.core import Weboob
>>> from weboob.capabilities.bank import CapBank
>>> w = Weboob()
>>> w.load_backends(CapBank)
{'societegenerale': <Backend 'societegenerale'>, 'creditmutuel': <Backend 'creditmutuel'>}
>>> pprint(list(w.iter_accounts()))
[<Account id='7418529638527412' label=u'Compte de ch\xe8ques'>,
 <Account id='9876543216549871' label=u'Livret A'>,
 <Account id='123456789123456789123EUR' label=u'C/C Eurocompte Confort M Roger Philibert'>]
>>> acc = next(iter(w.iter_accounts()))
>>> acc.balance
Decimal('87.32')

Concepts:

Guides:

Indices and tables:

{% endblock %} weboob-1.2/docs/source/_templates/indexsidebar.html000066400000000000000000000005261303450110500225400ustar00rootroot00000000000000

Other resources

weboob-1.2/docs/source/conf.py000066400000000000000000000146711303450110500163610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Weboob documentation build configuration file, created by # sphinx-quickstart on Thu Nov 25 11:56:52 2010. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import os import time os.system('./genapi.py') # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.append(os.path.abspath('.')) # -- General configuration ----------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.todo', 'sphinx.ext.coverage'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8' # The master toctree document. #master_doc = 'index' # General information about the project. project = u'Weboob' copyright = '2010-%s, The Weboob Team' % time.strftime('%Y') # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '1.2' # The full version, including alpha/beta/rc tags. release = '1.2' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of documents that shouldn't be included in the build. #unused_docs = [] # List of directories, relative to source directory, that shouldn't be searched # for source files. exclude_trees = [] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {'collapsiblesidebar': True} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = 'Weboob development' # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. html_logo = '_static/logo.png' # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. html_favicon = '_static/favicon.ico' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. html_use_smartypants = True # Custom sidebar templates, maps document names to template names. html_sidebars = { 'index': 'indexsidebar.html' } # Additional templates that should be rendered to pages, maps page names to # template names. html_additional_pages = { 'index': 'indexcontent.html' } # If false, no module index is generated. #html_use_modindex = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. html_split_index = False # If true, links to the reST sources are added to the pages. html_show_sourcelink = False # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = '' # Output file base name for HTML help builder. htmlhelp_basename = 'weboob' + release.replace('.', '') # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'Weboob.tex', u'Weboob Documentation', u'Weboob Team', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_use_modindex = True weboob-1.2/docs/source/contents.rst000066400000000000000000000002131303450110500174340ustar00rootroot00000000000000%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Weboob Documentation contents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% .. toctree:: guides/index api/index weboob-1.2/docs/source/genapi.py000077500000000000000000000027601303450110500166760ustar00rootroot00000000000000#!/usr/bin/env python import os def genapi(): os.system('rm -rf api') os.system('mkdir api') os.chdir('api') for root, dirs, files in os.walk('../../../weboob/'): root = root.split('/', 4)[-1] if root.startswith('applications'): continue if root.strip(): os.system('mkdir -p %s' % root) module = '.'.join(['weboob'] + root.split('/')) else: module = 'weboob' subs = set() for f in files: if '.' not in f: continue f, ext = f.rsplit('.', 1) if ext != 'py' or f == '__init__': continue subs.add(f) with open(os.path.join(root, '%s.rst' % f), 'w') as fp: fmod = '.'.join([module, f]) fp.write(""":mod:`%(module)s` ======%(equals)s= .. automodule:: %(module)s :show-inheritance: :members: :undoc-members:""" % {'module': fmod, 'equals': '=' * len(fmod)}) for d in dirs: if not root and d == "applications": continue subs.add('%s/index' % d) with open(os.path.join(root, 'index.rst'), 'w') as fp: if module == 'weboob': m = 'API' else: m = ':mod:`%s`' % module fp.write("""%s %s Contents: .. toctree:: :maxdepth: 3 %s""" % (m, '=' * len(m), '\n '.join(sorted(subs)))) if __name__ == '__main__': genapi() weboob-1.2/docs/source/guides/000077500000000000000000000000001303450110500163315ustar00rootroot00000000000000weboob-1.2/docs/source/guides/application.rst000066400000000000000000000000601303450110500213620ustar00rootroot00000000000000Application development ======================= weboob-1.2/docs/source/guides/capability.rst000066400000000000000000000026751303450110500212160ustar00rootroot00000000000000Create a capability =================== A method can raise only its own exceptions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When you want to return an error, you **must** raise only your own exceptions defined in the capability module. Never let Python raise his exceptions, for example ``KeyError`` if a parameter given to method isn't found in a local list. Prefer returning objects ^^^^^^^^^^^^^^^^^^^^^^^^ Python is an object-oriented language, so when your capability supports entities (for example :class:`weboob.capabilities.video.BaseVideo` with the :class:`weboob.capabilities.video.CapVideo` capability), you have to create a class derived from :class:`weboob.capabilities.base.BaseObject`, and create an unique method to get it (for example :func:`get_video() `), instead of several methods like ``get_video_url()``, ``get_video_preview()``, etc. An object has an unique ID. Filled objects ^^^^^^^^^^^^^^ When an object is fetched, all of its fields are not necessarily loaded. For example, on a video search, if the *backend* gets information from the search page, the direct URL of the video isn't available yet. A field which isn't loaded can be set to :class:`weboob.capabilities.base.NotLoaded`. By default, in the object constructor, every fields should be set to :class:`NotLoaded `, and when the backend loads them, it replaces them with the new values. weboob-1.2/docs/source/guides/contribute.rst000066400000000000000000000042721303450110500212460ustar00rootroot00000000000000How to contribute ================= By coding ********* Write a patch ------------- Help yourself with the `documentation `_. Find an opened issue on `this website `_, or write your own bugfix or feature. Then, once it is necessary, commit with:: $ git commit -a Do not forget to write a helpful commit message. Check your patch ---------------- You can run these scripts to be sure your patch doesn't break anything:: $ tools/pyflakes.sh $ tools/weboob_lint.sh $ tools/run_tests.sh yourmodulename # or without yourmodulename to test everything Perhaps you should also write or fix tests. These tests are automatically run by `Gitlab CI `_ at each commit and merge requests. Create a merge request or send a patch -------------------------------------- The easiest way to send your patch is to create a fork on `the Weboob Gitlab `_ and create a merge request from there. This way, the code review process is easier and continuous integration is run automatically (see previous section). If you prefer good old email patches, just use :: $ git format-patch -n -s origin Then, send them with this command:: $ git send-email --to=weboob@weboob.org *.patch You can also send the files by yourself if you haven't any configured MTA on your system. By hosting a Gitlab CI runner ***************************** To be sure weboob works fine on lot of architectures, OS and configurations, but also that websites haven't changed and backends still support them, it's important to have enough runners with different configurations, especially since running some tests requires a working backend. If you are interested by hosting a Gitlab-CI runner, follow these instructions: You can `install a Gitlab runner `_ and make it use a specific backend file (be it either by creating a dedicated Docker image with your credentials or running it in ``shell`` mode and making the backend file available to it). Then, you should contact us at admin@weboob.org so that we could help you register your runner with our Gitlab. weboob-1.2/docs/source/guides/index.rst000066400000000000000000000001771303450110500201770ustar00rootroot00000000000000Guides ====== Contents: .. toctree:: :maxdepth: 2 setup contribute module capability application tests weboob-1.2/docs/source/guides/module.rst000066400000000000000000000534431303450110500203610ustar00rootroot00000000000000Write a new module ================== This guide aims to learn how to write a new module for `Weboob `_. Before read it, you should :doc:`setup your development environment `. What is a module **************** A module is an interface between a website and Weboob. It represents the python code which is stored in repositories. Weboob applications need *backends* to interact with websites. A *backend* is an instance of a *module*, usually with several parameters like your username, password, or other options. You can create multiple *backends* for a single *module*. Select capabilities ******************* Each module implements one or many :doc:`capabilities ` to tell what kind of features the website provides. A capability is a class derived from :class:`weboob.capabilities.base.Capability` and with some abstract methods (which raise ``NotImplementedError``). A capability needs to be as generic as possible to allow a maximum number of modules to implement it. Anyway, if you really need to handle website specificities, you can create more specific sub-capabilities. For example, there is the :class:`CapMessages ` capability, with the associated :class:`CapMessagesPost ` capability to allow answers to messages. Pick an existing capability --------------------------- When you want to create a new module, you may have a look at existing capabilities to decide which one can be implemented. It is quite important, because each already existing capability is supported by at least one application. So if your new module implements an existing capability, it will be usable from the existing applications right now. Create a new capability ----------------------- If the website you want to manage implements some extra-features which are not implemented by any capability, you can introduce a new capability. You should read the related guide to know :doc:`how to create a capability `. The module tree *************** Create a new directory in ``modules/`` with the name of your module. In this example, we assume that we want to create a module for a bank website which URL is http://www.example.com. So we will call our module **example**, and the selected capability is :class:`CapBank `. It is recommended to use the helper tool ``tools/boilerplate.py`` to build your module tree. There are several templates available: * **base** - create only base files * **comic** - create a comic module * **cap** - create a module for a given capability For example, use this command:: $ tools/boilerplate.py cap example CapBank In a module directory, there are commonly these files: * **__init__.py** - needed in every python modules, it exports your :class:`Module ` class. * **module.py** - defines the main class of your module, which derives :class:`Module `. * **browser.py** - your browser, derived from :class:`Browser `, is called by your module to interact with the supported website. * **pages.py** - all website's pages handled by the browser are defined here * **test.py** - functional tests * **favicon.png** - a 64x64 transparent PNG icon .. note:: A module can implement multiple capabilities, even though the ``tools/boilerplate.py`` script can only generate a template for a single capability. You can freely add inheritance from other capabilities afterwards in ``module.py``. Update modules list ------------------- As you are in development mode, to see your new module in ``weboob-config``'s list, you have to update ``modules/modules.list`` with this command:: $ weboob-config update To be sure your module is correctly added, use this command:: $ weboob-config info example .------------------------------------------------------------------------------. | Module example | +-----------------.------------------------------------------------------------' | Version | 201405191420 | Maintainer | John Smith | License | AGPLv3+ | Description | Example bank website | Capabilities | CapBank, CapCollection | Installed | yes | Location | /home/me/src/weboob/modules/example '-----------------' If the last command does not work, check your :doc:`repositories setup `. Module class ************* Edit ``module.py``. It contains the main class of the module derived from :class:`Module ` class:: class ExampleModule(Module, CapBank): NAME = 'example' # The name of module DESCRIPTION = u'Example bank website' # Description of your module MAINTAINER = u'John Smith' # Name of maintainer of this module EMAIL = 'john.smith@example.com' # Email address of the maintainer LICENSE = 'AGPLv3+' # License of your module # Version of weboob VERSION = '1.2' In the code above, you can see that your ``ExampleModule`` inherits :class:`CapBank `, as we have selected it for the supported website. Configuration ------------- When a module is instanced as a backend, you probably want to ask parameters to user. It is managed by the ``CONFIG`` class attribute. It supports key/values with default values and some other parameters. The :class:`Value ` class is used to define a value. Available parameters of :class:`Value ` are: * **label** - human readable description of a value * **required** - if ``True``, the backend can't be loaded if the key isn't found in its configuration * **default** - an optional default value, used when the key is not in config. If there is no default value and the key is not found in configuration, the **required** parameter is implicitly set * **masked** - if ``True``, the value is masked. It is useful for applications to know if this key is a password * **regexp** - if specified, the specified value is checked against this regexp upon loading, and an error is raised if it doesn't match * **choices** - if this parameter is set, the value must be in the list .. note:: There is a special class, :class:`ValueBackendPassword `, which is used to manage private parameters of the config (like passwords or sensible information). .. note:: Other classes are available to store specific types of configuration options. See :mod:`weboob.tools.value ` for a full list of them. For example:: from weboob.tools.value import Value, ValueBool, ValueInt, ValueBackendPassword from weboob.tools.backend import BackendConfig # ... class ExampleModule(Module, CapBank): # ... CONFIG = BackendConfig(Value('username', label='Username', regexp='.+'), ValueBackendPassword('password', label='Password'), ValueBool('get_news', label='Get newspapers', default=True), Value('choice', label='Choices', choices={'value1': 'Label 1', 'value2': 'Label 2'}, default='1'), Value('regexp', label='Birthday', regexp='^\d+/\d+/\d+$'), ValueInt('integer', label='A number', required=True)) Implement capabilities ---------------------- You need to implement each method of all of the capabilities your module implements. For example, in our case:: # ... class ExampleModule(Module, CapBank): # ... def iter_accounts(self): raise NotImplementedError() def get_account(self, id): raise NotImplementedError() def iter_history(self, account): raise NotImplementedError() def iter_coming(self, account): raise NotImplementedError() If you ran the ``boilerplate`` script command ``cap``, every methods are already in ``module.py`` and documented. Read :class:`documentation of the capability ` to know what are types of arguments, what are expected returned objects, and what exceptions it may raise. When you are done writing your module, you should remove all the not implemented methods from your module, as the base capability code will anyway ``raise NotImplementedError()``. Browser ******* Most of modules use a class derived from :class:`PagesBrowser ` or :class:`LoginBrowser ` (for authenticated websites) to interact with a website or :class:`APIBrowser ` to interact with an API. Edit ``browser.py``:: # -*- coding: utf-8 -*- from weboob.browser import PagesBrowser __all__ = ['ExampleBrowser'] class ExampleBrowser(PagesBrowser): BASEURL = 'https://www.example.com' There are several possible class attributes: * **BASEURL** - base url of website used for absolute paths given to :class:`open() ` or :class:`location() ` * **PROFILE** - defines the behavior of your browser against the website. By default this is Firefox, but you can import other profiles * **TIMEOUT** - defines the timeout for requests (defaults to 10 seconds) * **VERIFY** - SSL verification (if the protocol used is **https**) Pages ----- For each page you want to handle, you have to create an associated class derived from one of these classes: * :class:`HTMLPage ` - a HTML page * :class:`XMLPage ` - a XML document * :class:`JsonPage ` - a Json object * :class:`CsvPage ` - a CSV table In the file ``pages.py``, you can write, for example:: # -*- coding: utf-8 -*- from weboob.browser.pages import HTMLPage __all__ = ['IndexPage', 'ListPage'] class IndexPage(HTMLPage): pass class ListPage(HTMLPage): def iter_accounts(): return iter([]) ``IndexPage`` is the class we will use to get information from the home page of the website, and ``ListPage`` will handle pages which list accounts. Then, you have to declare them in your browser, with the :class:`URL ` object:: from weboob.browser import PagesBrowser, URL from .pages import IndexPage, ListPage # ... class ExampleBrowser(PagesBrowser): # ... home = URL('/$', IndexPage) accounts = URL('/accounts$', ListPage) Easy, isn't it? The first parameters are regexps of the urls (if you give only a path, it uses the ``BASEURL`` class attribute), and the last one is the class used to handle the response. .. note:: You can handle parameters in the URL using ``(?P)``. You can then use a keyword argument `someName` to bind a value to this parameter in :func:`stay_or_go() `. Each time you will go on the home page, ``IndexPage`` will be instanced and set as the ``page`` attribute. For example, we can now implement some methods in ``ExampleBrowser``:: class ExampleBrowser(PagesBrowserr): # ... def go_home(self): self.home.go() assert self.home.is_here() def iter_accounts_list(self): self.accounts.stay_or_go() return self.page.iter_accounts() When calling the :func:`go() ` method, it reads the first regexp url of our :class:`URL ` object, and go on the page. :func:`stay_or_go() ` is used when you want to relocate on the page only if we aren't already on it. Once we are on the ``ListPage``, we can call every methods of the ``page`` object. Use it in backend ----------------- Now you have a functional browser, you can use it in your class ``ExampleModule`` by defining it with the ``BROWSER`` attribute:: from .browser import ExampleBrowser # ... class ExampleModule(Module, CapBank): # ... BROWSER = ExampleBrowser You can now access it with member ``browser``. The class is instanced at the first call to this attribute. For example, we can now implement :func:`CapBank.iter_accounts `:: def iter_accounts(self): return self.browser.iter_accounts_list() For this method, we only call immediately ``ExampleBrowser.iter_accounts_list``, as there isn't anything else to do around. Login management ---------------- When the website requires to be authenticated, you have to give credentials to the constructor of the browser. You can redefine the method :func:`create_default_browser `:: class ExampleModule(Module, CapBank): # ... def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) On the browser side, you need to inherit from :func:`LoginBrowser ` and to implement the function :func:`do_login `:: class ExampleBrowser(LoginBrowser): login = URL('/login', LoginPage) # ... def do_login(self): self.login.stay_or_go() self.page.login(self.username, self.password) if self.login_error.is_here(): raise BrowserIncorrectPassword(self.page.get_error()) Also, your ``LoginPage`` may look like:: class LoginPage(HTMLPage): def login(self, username, password): form = self.get_form(name='auth') form['username'] = username form['password'] = password form.submit() Then, each method on your browser which need your user to be authenticated may be decorated by :func:`need_login `:: class ExampleBrowser(LoginBrowser): accounts = URL('/accounts$', ListPage) @need_login def iter_accounts(self): self.accounts.stay_or_go() return self.page.get_accounts() The last thing to know is that :func:`need_login ` checks if the current page is a logged one by reading the attribute :func:`logged ` of the instance. You can either define it yourself, as a class boolean attribute or as a property, or to inherit your class from :class:`LoggedPage `. Parsing of pages **************** .. note:: Depending of the base class you use for your page, it will parse html, json, csv, etc. In this section, we will describe the case of HTML documents. When your browser locates on a page, an instance of the class related to the :class:`URL ` attribute which matches the url is created. You can declare methods on your class to allow your browser to interact with it. The first thing to know is that page parsing is done in a descriptive way. You don't have to loop on HTML elements to construct the object. Just describe how to get correct data to construct it. It is the ``Browser`` class work to actually construct the object. For example:: from weboob.browser.filters.html import Attr from weboob.browser.filters.standard import CleanDecimal, CleanText from weboob.capabilities.bank import Account class ListPage(LoggedPage, HTMLPage): @method class get_accounts(ListElement): item_xpath = '//ul[@id="list"]/li' class item(ItemElement): klass = Account() obj_id = Attr('id') obj_label = CleanText('./td[@class="name"]') obj_balance = CleanDecimal('./td[@class="balance"]') As you can see, we first set ``item_xpath`` which is the xpath string used to iterate over elements to access data. In a second time we define ``klass`` which is the real class of our object. And then we describe how to fill each object's attribute using what we call filters. To set an attribute `foobar` of the object, we should fill `obj_foobar`. It can either be a filter, a constant or a function. Some example of filters: * :class:`Attr `: extract a tag attribute * :class:`CleanText `: get a cleaned text from an element * :class:`CleanDecimal `: get a cleaned Decimal value from an element * :class:`Date `: read common date formats * :class:`DateTime `: read common datetime formats * :class:`Env `: typically useful to get a named parameter in the URL (passed as a keyword argument to :func:`stay_or_go() `) * :class:`Eval `: evaluate a lambda on the given value * :class:`Format `: a formatting filter, uses the standard Python format string notations. * :class:`Link `: get the link uri of an element * :class:`Regexp `: apply a regex * :class:`Time `: read common time formats * :class:`Type `: get a cleaned value of any type from an element text The full list of filters can be found in :doc:`weboob.browser.filters `. Filters can be combined. For example:: obj_id = Link('./a[1]') & Regexp(r'id=(\d+)') & Type(type=int) This code do several things, in order: #) extract the href attribute of our item first ``a`` tag child #) apply a regex to extract a value #) convert this value to int type When you want to access some attributes of your :class:`HTMLPage ` object to fill an attribute in a Filter, you should use the function construction for this attribute. For example:: def obj_url(self): return ( u'%s%s' % ( self.page.browser.BASEURL, Link( u'//a[1]' )(self) ) ) which will return a full URL, concatenating the ``BASEURL`` from the browser with the (relative) link uri of the first ``a`` tag child. .. note:: All objects ID must be unique, and useful to get more information later Your module is now functional and you can use this command:: $ boobank -b example list Tests ***** Every modules must have a tests suite to detect when there are changes on websites, or when a commit breaks the behavior of the module. Edit ``test.py`` and write, for example:: # -*- coding: utf-8 -*- from weboob.tools.test import BackendTest __all__ = ['ExampleTest'] class ExampleTest(BackendTest): MODULE = 'example' def test_iter_accounts(self): accounts = list(self.backend.iter_accounts()) self.assertTrue(len(accounts) > 0) To try running test of your module, launch:: $ tools/run_tests.sh example For more information, look at the :doc:`tests` guides. Advanced topics *************** Filling objects --------------- .. note:: Filling objects using ``fillobj`` should be used whenever you need to fill some fields automatically based on data fetched from the scraping. If you only want to fill some fields automatically based on some static data, you should just inherit the base object class and set these fields. An object returned by a method of a capability can be not fully completed. The class :class:`Module ` provides a method named :func:`fillobj `, which can be called by an application to fill some unloaded fields of a specific object, for example with:: backend.fillobj(video, ['url', 'author']) The ``fillobj`` method will check on the object which fields (in the ones given in the list argument) are not loaded (equal to ``NotLoaded``, which is the default value), to reduce the list to the real uncompleted fields, and call the method associated to the type of the object. To define what objects are supported to be filled, and what method to call, define the ``OBJECTS`` class attribute in your ``ExampleModule``:: class ExampleModule(Module, CapVideo): # ... OBJECTS = {Video: fill_video} The prototype of the function might be:: func(self, obj, fields) Then, the function might, for each requested fields, fetch the right data and fill the object. For example:: class ExampleModule(Module, CapVideo): # ... def fill_video(self, video, fields): if 'url' in fields: return self.backend.get_video(video.id) return video Here, when the application has got a :class:`Video ` object with :func:`search_videos `, in most cases, there are only some meta-data, but not the direct link to the video media. As our method :func:`get_video ` will get all of the missing data, we just call it with the object as parameter to complete it. Storage ------- The application can provide a storage to let your backend store data. So, you can define the structure of your storage space:: STORAGE = {'seen': {}} To store and read data in your storage space, use the ``storage`` attribute of your :class:`Module ` object. It implements the methods of :class:`BackendStorage `. weboob-1.2/docs/source/guides/setup.rst000066400000000000000000000033141303450110500202240ustar00rootroot00000000000000Setup your development environment ================================== To develop on Weboob, you have to setup a development environment. Git installation ---------------- Clone a git repository with this command:: $ git clone https://git.weboob.org/weboob/devel.git We don't want to install Weboob on the whole system, so we create local directories where we will put symbolic links to sources:: $ mkdir ~/bin/ $ export PATH=$PATH:$HOME/bin/ $ mkdir ~/python/ $ export PYTHONPATH=$PYTHONPATH:$HOME/python/ All executables in ~/bin/ will be accessible in console, and all python modules in ~/python/ will be loadable. Add symbolic links:: $ ln -s $HOME/src/weboob/weboob ~/python/ $ find $HOME/src/weboob/scripts -type f -exec ln -s \{\} ~/bin/ \; Repositories setup ------------------ As you may know, Weboob installs modules from `remote repositories `_. As you probably want to use modules in sources instead of stable ones, because you will change them, or create a new one, you have to add this line at end of ``~/.config/weboob/sources.list``:: file:///home/me/src/weboob/modules Then, run this command:: $ weboob-config update Run Weboob without installation ------------------------------- This does not actually install anything, but lets you run Weboob from the source code, while also using the modules from that source:: $ ./tools/local_run.sh APPLICATION COMMANDS For example, instead of running `videoob -b youtube search plop`, you would run:: $ ./tools/local_run.sh videoob -b youtube search plop Conclusion ---------- You can now edit sources, :doc:`create a module ` or :doc:`an application `. weboob-1.2/docs/source/guides/tests.rst000066400000000000000000000046331303450110500202330ustar00rootroot00000000000000Automated tests =============== Summary ******* Weboob is a wide project which has several backends and applications, and changes can impact a lot of subsystems. To be sure that everything works fine after an important change, it's necessary to have automated tests on each subsystems. How it works ************ You need `nose `_ installed. To run the automated tests, use this script:: $ tools/run_tests.sh It looks for every files named ``test.py``, and find classes derivated from ``TestCase`` of ``BackendTest`` (in :class:`weboob.tools.test`). Then, it run every method which name starts with ``test_``. .. note:: Some environment variables are available, to use specific backend file or send the test results. Refer to `the comments in the script `_ for more infos on this. Write a test case ***************** Normal test ----------- Use the class :class:`TestCase ` to derivate it into your new test case. Then, write methods which name starts with ``test_``. A test fails when an assertion error is raised. Also, when an other kind of exception is raised, this is an error. You can use ``assert`` to check something, or the base methods ``assertTrue``, ``assertIf``, ``failUnless``, etc. Read the `unittest documentation `_ to know more. Backend test ------------ Create a class derivated from :class:`BackendTest `, and set the ``BACKEND`` class attribute to the name of the backend to test. Then, in your test methods, the ``backend`` attribute will contain the loaded backend. When the class is instancied, it loads every configured backends of the right type, and randomly choose one. If no one is found, the tests are skipped. Example:: from weboob.tools.test import BackendTest class YoutubeTest(BackendTest): MODULE = 'youtube' def test_youtube(self): l = [v for v in self.backend.iter_search_results('lol')] self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) Note: ``BackendTest`` inherits ``TestCase``, so the checks work exactly the same, and you can use the same base methods. weboob-1.2/icons/000077500000000000000000000000001303450110500137345ustar00rootroot00000000000000weboob-1.2/icons/allomatch.png000066400000000000000000000043201303450110500164050ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDC pHYs  tIME*(atEXtCommentCreated with GIMPW+IDATxmlGsh$EĮ hrnɱT_!QCPJN BXhS‹p@s! DQus>l>ܸ]/ݻ5 ϝlDGw&Ӣ $`U;*|(l$$uUm=o^_.@ϲ 3Q"y;5".5Xݠaq9BqڀJxaIv 8!aT 5”M+ `6ubyO$, KU ȶ5 pYznHx\1}?>]Վ6a}(ircuUl"ўZ!B U 9lwpO8oX#@݅U^2yfرChll~Y ~hkkӳD"`Huuuɩm۶uVL;"`dSSSI=EQػwqX,mܸREs5>0)Ky[^Pk'`hhd (++.E {%` ˢlJHYPMdwࠛuuuJpiϓMY&] !(N\Te9g%E.Wm+.]t@ ]xў#(#7ϮjR Ø{S]v]r0@*߿cž2pȑ#L|Ga~Ba!3y4ӆabjjJ^qeqY!ZZZDee=Oͧ$E*drg92Cd%N" UWW'А!sssT*O;w.}A>pŖx[ " J93l ]$ q)QSS#hoohT 8 0D"`PAH$a"z)@-lW{#8p3d>SY @,#SQQA,X,V/#]}UILTdMt|;Ԯ7H" ϗ^~QUU~zrddd]=n9QS*W(Ї1 O!y 'YIENDB`weboob-1.2/icons/banquepopulaire.png000066400000000000000000000024201303450110500176340ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME .X1IDATx[}Hg~ƚCdkn-+hYJYt+6 v[Z(:6Ơ0velS em~Ykj/wleֻ&w{̓w!q =PLyė)OWB׶i)x|kmjW]|WvlPA5lP9ۈ"՝|m;Xy٠`݃c8oa$زA ~ m44~iBEs>Fṇ-m3snII*}, :hu]@pDžkЫBmo|2V.n&sdl0*+3AQC!46M4ՒMfٕ,4|%xQhK> ZO>DbcpL-&rlݜNú=BPxM-+W]vo&d=?hRB,qX- ʺsӢ\t3^7k%Y$`o:eO0\^NRTT"nUV}SoTV&Q/*AQΒpxcHK zu6rU/>v2*W %PLFYd*_ gF0~.*y9[k형&jJVâk߻]8s~Qkp-]D`2PD h X6ǔ^%(6  5%HMѣ0߈"3^s?1w]-@zP F}M!'3ǹ<T{dn=%|c W\o+y՝(;|@L7Л=r*Tgw>Gb `o /PO+ o}oNc_/:-@NG8lO!}R?% $ syFl0"/m]+x4N_K DeY|xȇ#RIq$[%p-p`Xܾ1Kn\r9#rXq *> Hs;IENDB`weboob-1.2/icons/boobank.png000066400000000000000000000177261303450110500160720ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME0#tEXtCommentCreated with GIMPW1IDATx{i%uw^_/,=p$F(;Ȋb92edža 2(A H'A˖e ҈㘖DYE)}z׷o{O~{8E8ոzU|us-WdĀ!E /~` hkTY_??E7,rx~Ze1@*ᰛtͤmq2L[G' t/Ǟ$aU+ l}E Ҳ a&sJU2t;7:Ý)3?Fp(ҳ}IBc8ƞN_}a//g|?B*EevBJ6FSb0va񰵷{9b kMuOA6;=S[p/teB 1Yym';W1Zp|9}rR |ffx~($ =޾ᴟwѣN] _=[*D$,WiϮ)$dM27^b%౿AF\sޞ[m`M| @~'K7>V%0 ܼU;Kv}?\TWʼm5f14d$h4lu~[ORW |)Z^Yc۲].dz _Yd'ƫӅ˿w(GBpJœ,#%0\?LlBx" &}}{{ɠmLG=QeSJZ}8xlp_gxqMr-ϏFPJaM)OYƼm#,#HObNx0^$N]?,Q,_kFTWW7i"zJ~v:h-}?LJ *q% &/^Ft嚮./1 kLiDfr>{f@>B $- b)xțRwkd^}X}}`{a8UX @8h~aekul"|背G+׎<ůb ?fT*SƄgYlgYb6dnA*TPz6b=V$_0o?"g'l|ܚrs^kVka3XLI޺}ax.P*@ǢЇ?,N.3MO>|兩K6ozt Z3CJ{nدUBX bseNW/c|c_VNṋ[33 6 lzu6a1m6 ,pu^o,Csѽ˥;@|wj܉^yF YDI:,YT&ǁ 4%$%{@VVjU"@o|uڕKz~~pjP(eٹYu:]dK2 ~p݁#z9unKU,3i޹fkŅ;BHB!()|+ċ@\0-0*:puwo޸Ս8֤e9KKgkk[4~L68po(H<汑f8&jt\#p̱\HKk49Y8F*qg% Jje^M +``57 )}P;iEQ8;$ aўeIxꥵ_l:u.v Ha +j~?;C8o5 cōRqfQ&4Z+aclVnŕfkw.rR)@؅bY6?J/`1zWs =!)$?bka=#Nw*֓g8Y#`eLLTà/vn,I2ܱ hbB wy gK-,m]r!Xq\`3BIxHڣϜ&3pc t!7r?]ai/=Tw:,=cb+*T8 PVƣn |DьEՠZe;4M[WdTrq:11Y_yy'F~3uiJ4F1# |$qzVuat Ha ) }A7*V:ÝjӘ5lg+DecfksF$U/ Z%!jk?|W3SGFub!ʵHyL]Y-/>Q2>QڹqN*By^⁝$h 1.",QDQYgUy$--׿VgݖT.0J7]/<Զ]11b%k?J]4:9\^TA8CKHܟ(}el夾WHGq48 '*Cg3u+ Ka:KDdpn؛g{.;׫<$9h+/y\];ߧ R*BBx]4 #\? jx>w^h0]eg;6HP`pX\]ϷQiF^P(e3i$R CL=wkZqyeM^ 82nd!EJJ׌WcҖ6:6cV19> ° )ipnظBp1ի/t:Czwig}5ZSu'Px^`0BEDnJx3X~hnWO,M I Ʊ 3 )-,JyVڎ+3LGbwڍ)m*gR˶V vwo,{\x,KgƎ(8yMffCٮ珞{dzpg;nldgy1EDb6²x@I2L+<'ebwJl9 p ;nryF}3maXL9Hd id_}1Z%̟@* E]'B &"WǠثKؖ>$="ShJ@lHK],֎@ v0o~SX:^hen)z=%0-HhvS!d//mܻ_- cC&afɟX-$Á%/# i i4y8]*a8eЫxEҲ}?KI lH!s$0W|?2ihyвdYk򆂓pLiy9˂_yz~uv!?Q*I2 Ҕ93v\O2GT" <5e8BϥP(%Z+"z\>qq]8`عU"μRRsR Zuq0F$ynN`Ե !U!*(~Q q#`&xeFqS\Y=(پ9#-+Tf$y496^\*:=0>O1%~BEξ/N< T r[6l(]I he,ͪy {RZ$ !$Dsl ryfﻝ$#Z @M^-W-)@(Wd-ի)"]}Acc,}c:CͩhnB4 vsF q< DZֽEQ)m""cm&R,Md亾q򑅅 ,Cd_}~Vb9[?H;3+Ov)*\< lLh4w$y5#" @LXȫİ*;{x !㵵OѨ'l%E )b$ؒO6k]⾜NT2MEVr6kZ+*g+;z8Qܳ(Tyl 'ɈS_(;ޥ>r{뚟eiq~U?ujn];91>fxsJٌp @T *s Ai\&2&{5,yѨ[3ZYOQ(geGִܞj. bTKHNs0,QXgY"H[\DatG#x<ٻ^6ZŨtn a~!B 7YIi6_R`cvX(^>@dtˮIt~<6Q22Fc0Je~$0[_9,^yJ㏢B8pb$kxwz`o_s.T2sLnjf`#Uy2}OgmowO$Wo=il,>3 @: + YX{*gǔkA%+n A(ͮ6e2Mnɽ3TVbv'Nڝx*)+ڲ`ʳTzmkgfNΌ6}"jxG.% kUhZ%Zua_֊l2߫VbotI@2i{Zk EAeou;Ӡ7pЋ,iN6#qaY6HNZuǰO нF0kT'0Zr;taP:[anV27"}l۝r%z\~`qY w߿Gh%>}\HqHy*U<=ujs61'"ar ,F5r]RyfiIi8 k[ ;6~4Rc&֝k4 1hobTH@0#O|пg*Ӷe)Ae`Ɇ1ZFz;Jg_.sywEQeb)"3ˇ~{ٷ k$޹r⠱U+- c"Y~$Fi0!d? ;K'oj@ BŒ|p%%K-VLz_Ϭ>LXY}N$o^ZKPAH(#@e9JyvsymvI񐙆MooK+KgLzB&03~kVi'A" jengaq!3̴o6nZ.d^-ʲ\ fר񠘫q/-gfURрb M'Og3ե5] y|ns<,Om2l T3qcr? 0q׿fd}[4|/dpG$xxx8o ĩR: 2%mK;</(y(YfԸtT0C1 184^@o9 <0Sr"q[vyu[~>_}=i{ Roe zͺT-eO0`BD<-1۱A~\# )#0~6xw8/KNz8YH+MԷrv988!My9a5nne-Ub8bRG}CarPHuʎ0P4ZQRy$#)(u1ҙUd_}:Tv.oAD4t庇N|6r9V¨5\;zjI,(R`(Ef 2sYK{GLh <6]!-5;3P8o8 Rɣ g2_Q@nM"<;S^Wmٶ۩ |ݮ+v ")0L/cMGl7> $dC(?o!isKR(/\y|i +L.rC8xwOAm 2jO6='ܢ.FG_xWm\c[#6`> 0FQ UpdhF RYqKwKna*0{~< 2$L+E-Mp{?辰[6y|H:`")1h} D 9BuW›'!O:hjLCEKaaMR[Ƕ ;=gYJP焢 dOaaq3}a⎷͌n5B2 lnv?J>=GH B(>GWSS7Y:No/SԨЍ amnC'(PwAu5E[wUzOOς$:%8R82cTk5*b[TK3޻H 1KwJwؗeTr,kkknIn1URv*+F]"p󱁼RĄ88?B~۟ gT(D8U)Iy[tS)p~"1#H 4 |uY(`UJ:+!-;7nҢ41sykF[{m+sTCn~"7?T2+WN]ڢX\KgʳO~C`,OcwefyJ9KLdԐzA&X#uf {v\@P`LbOCyҟݢP ̸x gDO3GUy~^&gre RΉ/bd5|oG:R8d(3?APͿԇo$,rcɧ=:5Y+Pܔ|">j ޤD;<+Ľ}E L*)}yC0ci}6h.,.rf|PNRmտj_9ܜXbPK?HW.`__kk&ߣ[ʖihVW.( nkL0:7d~L]q[`_|Fi.*A듎ِb26'WΟ.Ξ,XA}S(|%>~ŷCK]VjxO +|JZ8cpM6ߓ:Pd4;mGJR1mp @Ty~ф,NSᰅ\.K.mɅS#1(O6mV%TWF#$$B4bJFFrK_Ps 6 s}ˑLu;iە境#97?U(c%T˨%YX.=rr$/LJ{բDtd 7eM; h5#ojGK˺2ttg`dJ$"O3raB5IQ:iEuBˇz"=M7 +",|tNjÑߌ7*>]}*Ӣ^#X~XNff9F,q|r S& gUPb4.JE*j;+wuIҗE%K:xzwG8 S:*g4Zk?FY̸KKƞ8Lo*"|Q~jisevaY>lJ,x9Zh8ÝrCZАM u+jY*% ym.+e0N"'1w-#^DFF.vwTHe9LУ@N--V Q & HWO_,\ؼp/?K@DK2tWƸ)d ]>]Yb(5 L>GX@RJ}>9L8 l`e*~\  ܸ4ύO;{UPͮBYYC˟̠l %2[Gq7 {  H',!'ܒ&(~X0wvB`A x8[Xjq*TYX_%(yF|*Ƣ.p?P cDV;Y[#S@i'C7hq۟/ P$~_2Tr4OHa  0*D7X˰,(@A|W1L+cux-?"1:W1PRiR?\ET;=BA@?)TdOfkd9@7<\}<B~/4+h\q5@W#3>e0RI_yR)18/wC?U:7WSPCiϣ1-qTW@Q __ 3Tn ƾ&EhTVAY>@a_ I3  fE{Ҩa;>P:*+"SXXq6U J90.0h )W8GAX|-/5IENDB`weboob-1.2/icons/boobill.png000066400000000000000000000147341303450110500160750ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIMEujtEXtCommentCreated with GIMPW7IDATx{yՕo_{U.6@ =3 ;:˽XZ0$ )f0 YNa`fm5<o^"fB iZ Ӳ faR([DZs_jL-[`׮]W\ebb> ~<x'.p ,~@h"DJE".o]42|8-v _zႀ\V?w;JW6)f$9!۷ﲛ8o߾]gnnyg}].6Y`XZk;,RGG7*w=J]zK $ J.$;wD?L&L&188yR`Νoʺ駟yg>s\Rk-5+H- Y`t!VVЗm\WU- n۴f0H&%{?<0k43ÅL\X>HdZJ@!>}A;q0U'W<7/4* ݛf3\ ${ttJNr8rg~+Ehz|Oa3|%Uu lm~W!&V SHKDD/nƵvҽU `]0\0-Y:uщe) &L $M#:_M&5 "WkK%FǏcxgo} ?8xغu+/㜷ؽ{7vލ~ɚ{ݻ[z 0,a - S`i.X;qƏl6)"22l N4ҕ@Z+zP(J%/~il9 2~c޽%C=}]/| K$Sr'Ա1;7?^2 ouLNT("F*IK,^ЗՀe7~I+SJ6#GOM "sӆR8v;Uߝ;w> Ɵ痼ӝIT B?Jt|XJ'.Ltqd3df#kf2N Ct.U M` \-kkXSn~ $^B0 PJ/'%~\L&ڥ61Ѻ7e{b˫'bO pel)]%)^^A0-cz`۶mx']@Dtp;`H,nџ$¹li^C$ %R^f9;+LA,Vl왙TwGU6NYsT$_43~E3o,Kb"!n*`h#c`vNkkxXyiNGGAF-Y]+R٠\Yf,/\D?Rr=#KYifK/Y XM"2Lf6Y#-&kّr~Ard6S0Ța1r.S!Adfeڹ0 ޳g 8fggqAdYdY7X("`@0P$٥ԅGC0vԥ4d:V-g;nN"$d6%O^+*-Ih=gQ*60 +8wP[^&)rŊIaR!%pe*L+$^MCohf\Zn*ZK0Ž3 lfu[BrYIKn˶ plJs r~~& L[9NB-hy kvX++V-A2mݜf-<t*_gf A-niYs!aZdϩf$LQRxX3z}M &g[^!"N 3BUKe;̚AQR "]'qQAdݿ d2`DD9 )a"ҩT ?+D۷-Q8qEP y~:y0FafT)ԴDDHՌm  ]m̚Z13 V (m 2\'{|J9E/ j@b޽Kr~7"鷺ϋ]pV\\^ˤ~,#Z$y)DV;k3k-.] 8*&bx.($f;vsjwKjO^!.ӲnڡP) 阕҈(ZUM=aJ}XfD@@xإ1E1XL8qQH2E֨V`]2lu&JZOϜ4[TMe25 ny5釞},_ A &bLlۮπYϛl1($`fCw$:t>z+:U#\R&X >Z\TZiqY5aJ߬^@fb-q#it&_խV=17;&3;S̗3(uYؓ.Bd戈""-z^͕X説8ffŬBu&YGlu?C| ` *nV1g`fZn,%<ԕ%&Z bKBBoҙ3GXkN%3]ZkEDqz\) ^ }ֺj6vZC+㦪DPKV؆!"} #x?N? ˲P,裏MfNlU]! q !ÉScLd4k56;5-_7 Vܣ+EjʍRQ /Iﭗ|8|w?z(Nyc=v 0s`ڱuNLD^lSkQ>΍g iNeŚaX PVQ:g!fD/X;U(t(>(@S WҀO~ss~8aLOO<ׯS Q!5۽zLkQ(9phYNnjK=0 Zm^0C(QdG8`|0Bo|җ~ecbbŐ_J:UX`V!Prk6ݖn9Z1? RQB0; Dd082YHiB;U)`_ɥ1KJko6TUlnr4NDyz<s+6R\6FaM0;պ!, 4L$v0p^\K8kx=fDzASM UXs3)N0cŬ.WcoSGQ?܈UzUJL8?Z)p[O09~"+'N Ci,n<om8nf'gihI>}}fx 4h1sˏpZn^V7_0o 7ðҲQNx ]5h f=ݞVk"vi7ؖ5ixf~8<3k%c!hnǵ«?=|VAC*+eR3+61ŀͷ]+S+֗(y݆L:"QʣG;]`hni[>㣛ӛcⶏvunV)fJ!Yo-r';sv"&UwVg:z]2HXVDК8X9J}nM:lK^ȑXFH9X͛vSf;.. pa~H_r[5k-k-X ᇬ3gl۽gm\*vK [iEq}kꍳ.($^޻~"=ZLҬOm=5`뛓(u.KRv>_ (3.u͚YcpT̺GB϶L0V)\ֻvGq!|&{jmղumjT竆a:bvˬ5Jy`7T,Ru )XIiRPkS0Q77ANXxZM͝j%SrzNr2"&bx1`) dY@zQ p d>|؟)Kp`ն2T N֔vF1 @!B& П?W3ڻ\IA+(|Dp QmG. d8vf(=v={F+hڭfU沝> bb E-E AFAx=31χbY0/K/l )1PVܖP՟18GQcYm +% ;oHqݔJ&s$&AH-VMEQR~:ᄂ+ /[Ȥrytl$HJn[ґVڃߛ8G\/tN)7-ɬ$5XEA71-PIENDB`weboob-1.2/icons/booblyrics.png000066400000000000000000000231171303450110500166150ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME  tEXtCommentCreated with GIMPW IDATx{yU{_tόfՌd ɖ-!`Yld$(!)TA%@!f H^,ho{9㵌ec%z}=s_xzd8L p0UVbBDf{+-Oy% †Ri2\5SEYAc/x@&KmjGG#ZP<RnFIR 4P<-zݬv X_ w:'_o~k3?p PQ}I禼ЄIn斪D-)U& #Cbq{o|\9߷ۛ)yw ȵnSר5Șѳ+~zl h}-~ |5Y g:1LJF0rMJDIY @6UiWޞ;"vۗrw9loڃ2[h0J+pWͽx'^ܥ+c6cr49YPP vVZ_P<}Lj$ X;As m LA \!t-|;~L|5oTjzqDQ)U C %ȃeʖDw>: }4--X>рʈ"S6,y&ZA)a%Ή uk=owM0O7MU+G^B$ @ H E}g{^} t|o|;EDMSO~Hm2fP[E8 k BDD0 p/OoPhR׺!0`WkR8|䌝[8BI(AC@h=p?zwŃG_wW " ͟{'7AG8eb0؛iRiN9zg1?{Hy/"! F* 43l\؃3u|DF.//W|6*~H5Q#@҅@iR !lp .QNOx`g-> /WY<|Ύ/A!Ŋ" Hq rWf{ `x""ZFjkpoI4 ƈHH{9@"! "9Sf@y:oUpx\/=G&SssX\8*D @w`ة;g[4:sGVH !Ns :-TB$Bb DJR `R,Bhf.zHbYtR>c4#|}}; J……\x+e$*BgiVV.; @ @*awl{{u6H0/R%f"B/v`V9S @`6J$(2~6 `Bo$ c}&y,D"͙,nLKSh#2u=zmrhbƹ?vtsw,"+3ӰpQTԏ>h>w7|ݛ$@,z&O.<#H"BIa[o;-* !0Q(všX܆ˠX+Qu?kxW3'7Zǹ\ `'?t\'8KVDQb;5wnoLZGt/}6U1_Z &&J  QlQz?;\pޗ fgR G Y6HJc{G?Qձ1a0IaYsWF]rN" ʝK 8.+щE]gfߠ8˾^TkO0%qm]cy+ӻ;{@JrVOI/e6D-J|wߝHAје.mֱ{k k(EpEͯ\8͙.!"4 %JU>CY|0֚%"@"):ك[+̒T*MW}թNDP- 1Ʌ{+ "aHfl#+"!R8 ˀĘ^Z|wl$YRpl/Y N= r"Q\AD3fJ%CRߗR(fA@eD (.}$40VG_}tdž6AP$HHb! ( S0} i2f1@d0;էw7PR(7,Ph g~sEE)>sz22%U D8yzzi/ K6j˃)щS*.Kog,+7$LH@H($00Rw\ 8Wxw54hK U6/GN r6Aqj ApɽGv,LD"J%b"@BZ^ɮUCf& i2au}wo-5Dc$eb@q5QA "(ԜQmt5RF,@ÄFQIjf5>'ˆ ^̲g$qO~FQJ@ DDʻ뵭J\idsGq8hiTk"6kD|PזC2Kw\Ad& `X\ B0Xm`!@P#,H!שZcTtZx ux7KN*"1@tӓz[e :|潯Ka*HM8)s ֍۳zJԘp BRڰAfR=U:(QD$8S Es[$HߛOd9Hdf٘"aHzykJ({A@$W>; R켾pjm{뼀Dq+Es1D(pwW:{ l EP*wך s2!*U6hxI" Ļ3kMP (0oV8Zs,vk^˹t`UuHDDF:?ʳ'7OyWHi! ^F?|Cvf\l0V/^E(_}0Aywey'^.U= i!OChŬ4r@)vМUgTVq)Tm7֮"-`~pot١0ytىBu?x׿u6E^.k/`? `15 @$Hk7_ O@گ@g l b^v-08$b"X[hب"LU\%ژ# hEVi6zaL+>9Sqfkͨ;^q{J)Dn%JÓxxy9d .As1"+EᰳٔC@ !}{gC@ g b tdŋḥ I \w.lfZkEaq-)bE?v1@g}M33/C *3=6H+; Z8wSi|Jٞ,lP{DB6NH YM-ʨZRНra$fll*)%ffvE#`\DڂYs!R`TOk BA~ :p]:J> & ՘{#%" ȇ7*f iMVEuR"ªێ2bxɕϠ-@LkA JE`Ad`s; !-W.h} "ujyrmry‘E&w1H)(@P1vSa&$VJHc r"),(B<4VA_Y(JV:a DW* {AoW0}$ *{4am߻%Į[.>1n7\Jk)ZdW{  Y+Tv&qP > .;eiqXR~ YUȉkM2(lA9/ ՚Aߺ9 Ehͦa}BH$vHk`r(-T x?D': {Rl[fth8)Lyo0ӋRX<"`4;{sݛ1\[x01>K|0`3wbeOi39?nW%TmwvN 7t[Smf{-hmQu  WsL&e D$A"0 RniԎuȂO[!3lbrqkϭiEy ^NEAt!{ҾQCG07)P;1\A05zzyǔjD ٠02m8bav3[6"eB9%Dy{āFa:y*m4CJ#qs%(ۯxDg^Y}{ DJVJ lҰF-!P`P8^0sx"Pg0Lmx'D f ,+#E#N]^)l"#<@ N9HJ!40{#RJj^+  :ވK}Q^Hq6HARDjD!"Ә^p_G_aʍE<;LA5Rv"8ÌlF{h0Iąʉem\6DDD!{¹J!ni9PDR!z6u  D@[o0+@+Yd؞QffO l~cs`: dgD Bx4>IQ#=i,@\JirTQd V 9dJѰO=LZ T82eλbkc%L֓kVHJuUFrўDDj=`x"Q}j`TP@PT ;V'alz%)3uBIBU}x/!ڌx13^&4~O:(AA?*eғ'_d}IJAׯ]nA)Xљ!(D PQeh<}YcnǰC0K#c'.8v< 7at{;\w8p1;Y\BgoNgf///`(lmZj,xwkٯ[:x*qf+vƈĄ m چ;!*wOF"Aj Y/Bmxǯ=:*nʧUpԱ@,FkkVV.Nz RZe7=lN(X`R(O>v)]Ks~a;o#z]_Z_Cec^ 0fhmJOJW}{5)Ñ"tD-@   ?ohTnw#lv(7WʋL PUWT{(mDkH཈B!x#k#Ͻ!_@Ay}튵0( H1$6>y/#w]OUt[ ""rs,LԚ]C{Z#H.]8DaX5E"Қ Vr0+~ߴӸzm_n}e{=wmgv [pēfaR$[׫f@* D^WQYHm|Bn>! 27落| CL.a S4~Fݺz7L{յKѕKOBiJA Hʄ ) At O>|dV`w0D 0+02 [nXy 9LBg22GTjm/=JU}z5;yq޿Tx0 Z 7kjj\loۙE5E&)E@joZ3^5v|> iU]=|p +W EIH[E1m7է Ϛ h "h A1Jf8@ď.B>Z C+ հ`?v8z}ER<8,+E{c$f{[ݤrq}sדbo;/eO>i_1jE$`![aU0\T C]Z B2Tf; !@C^8lxfa( \$zz廁{/Uk? Ohqu"Y:R=pDbHIL*,z5-~_Dn(3M3}f# mq;/d/?>O_p<^J3TE11V@@I&l6=q0:)#(T,lN32ߖ_V]c_pQpcOck[_q^7!R4k@p6ל 1rQ[OB @"gxy<nG xNeGw?> 2&byJ00'8" 8/DS\ZOX/hQG CP}%hhO?-?wg yb+ A %{F+ʤ~.ǯ?0>Oh| c |nA gdЖB8d$씶2cCOx^]s @Lcz~鋻_ v Wګl+R}\,l`H!dP;L0 I'I<3j?j `WF{ pnTB = &@ y0 lW[ZuR~Yz&yߘl8`Xak޾&U*$&A;  1.H( }ۆ4t[+fd?>wo4C f}uq`lRѵtMx$Aؾ1"V{8DHBRK̦R: pkBF;kk*- k ^@<*\6[\nG$nj|锍ΎJ(ӠyrRie'oSd8}_,w/>ߋ[w'rUUO2QN\<t`y`;&ʥ4)f?Z_o\Ižh8}f +@̨Qw%o'&߶Կl}\ȭ\IC@kW_Vi?;3u y%§~:ٵbdȧ`:vRRE8 'i} ̵[AIbn[]S FyD‹ρh6C|{/ᆽc[ߛ"x"q- "ȋxD(K`ǥBG0;7bw'5HP.-2 f$v>ێnX޹S)'>'>SǗoY=V,>6/C!D06"T"565;>L8;# AG' r{{سk"+/87$3ϝO~z Z_a:;r[qU* ͚@n2df#Ȳ0U/1uRG{giC)v(Aghr|Fr۱a.ٱ=|[6\?4.Q| [ɖqܣYT.Ue1} ly_ˮ7M䲅tZi$8?[sh,F F,Mv?{ q%SqO?{jv.[oYdPSɑf6d0L f>W4cN}dY%%cWVx%,bs*Jj?=yx989j?:V{VպO˺ԉH2 ' "HC:n-t|%St" 0SL#I/Βz])mê&&xG#Ë4t~%M*eaúΫҚVɱw !d+g)AdS d ZmlE`&&F sV-!~~`j)ܴB ϿxZ3f@[%jE 毘W)gpndpJ0H@ pK@m]\HVKL=y|>o\}_t1{Wؾh -5FFvjlJ[,9B:)54^6h$LL00 CJΦMfz>}$vlm ܴnAoO7AϤ/" h-@Hp֛eYbYYͦkٶlʝ7ο55I leetxV Bry}!^@6[Ls%&M ܹ]ZkwƐ"ɊC +%=iq(YM4ڏKCX*W4k )!''; RÛgT/2Wi13 .ڪL o 7[y,A*:fB׽53\̀Aq o DtEX((&ّ&k22/!lJ$BJlH 92|@ɴ"@0 \t;bN~N'09W*v7(p[nu#kJJeglROuZ|uMk̡f9_ͷ .37~]Wk,;_:߿o,ۑBӵ-J>mSC3luͦsD" S4񉳉j$)J'NrmܺP3s,ӬLXdcxx4`rt7zUЬPJ^f/0r{Ț%rMDBjq/"MDSl;Bu;J 5kN% b87\nehD=g{>5aY,i;߷c@@DAӫat2|XS#?Rw%`@Ogy[É|IiȖAlЅP fF,EqG=xehDHLeW6l;5R'Ǎ qkPȷ@Xk֦Rmu"PZX~u7=3 #;smaJZ! ~WvQvrwh13"R[^`q^m"VA|0 3ǀ0\)}S ?h0-o8{JA @Y;N2tdb !e!\X9R,tLQf|@Nvݚifض@!4G߸WNS#$fY9},qd`mGWJjMW?LLZ9ybǵo\NAcc +JkrqByebgfY 1yngcnR6rXwתiB152SJTvdlҲB+T:<53=N/Tȡ}au]3O}k]v)|C2T{[_̬B6-VzH\2Hd)?֮~|x'םК=c W)#ǀǚT|Co~mvkve=|,rubIDegZ):F,2O`#`wr|{֏][ nzY3!3 BI!\.RYr[ +I=]T*BHZ8&5kY8a#8鿈?'[[.aJNrcXű]f]Eq*?v_#ԟ~n7@E!J tl! & @1sJZi˟o6γw Woi2K6d*'$3UD[36}wK '`iuIENDB`weboob-1.2/icons/boobooks.png000066400000000000000000000214641303450110500162660ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDC pHYs B(xtIME  ( tEXtCommentCreated with GIMPW IDATx{weWy;r9NOLOTIIZ +Cb,UmL A(B(k4yzsz龛z^ޚӷuϽ}w ~пUw 4HGS_\AB(D ˰- i%x?ςZp]{hXt-Ƕ@Tk~r<JF >#gRX,%!@@` f$QsjF! ` K i٢}/f3Fzͪ~Wvp`CX*w^q e䍥,Z~]5UbGK'sE+ͷGy_?8gO퇿>;Q}?  t +ضJ(IB ':V gOGޛ0r5rUŞnF!],VĠ{?޾Sfa\ynd+r#NVPʁɁDGjT4 -{܎JOBZ-HYsD>Ykii1/?]vڰRw@iرzժ.z|^8p:qxeW^6`xVR†ep2y,-׾>! yC7oQr6ߕ2$K=QwP.@f6aV Ԗj~;nU6]X8L7}="_vel_^dp&r `Y\݇G߃ o{6/ُޔa9^/큔`6"6`" $d<8!gYM2z?2eo]wWGn vF o;6%aYl;VN?_ tt< ƶ{!!J#3 g/i㛲٢3V:ذY @$@LÌDJ+Z-W8h5fVf+kR?ݤ<3N}`1GcR*҆xew0Xz^d1Rkxe$B6:!$JT;J=p2) B C XX ŧN)VՏ]S >(a!/>@ɡnAsOMĻޅ#X:~ L}~dVn@\@~!: N7h̀h a0f[V˙L:I@Y/x0cojO&?7k``=6a mlג`֜F[cE6ݖIʹ2 |GCMf6O|ypiez 1nq`WfɕM6GqfT:zZu'@;/[.䊙mzQ٬c"$BJu"Na$$G7rL;ZȬG6\\P$ _C}åA*+R')s ݻoo|+3 KK%zE$ xMprm abSiR;( B̫Glvۋ::z{{- ]-)Itť飽R` S놷(&@fJu֖wIG`\qv5r\۱e@.\˯Ek qܯCc@A51a]@0860`1ap*~{b+d֖I6Ib&"YKŮ=;x[H2˫3BI\#`&RV@K~RwH^FV@Rl^BTqv ¤)ϴ16XiDd(n9YXڼBHeLe`F! .߬fo3BOd9XYʕ?4;FTؕ$) j+Hc U2  c`\(4gdM!LYɡXorRcfH "(Lftd$Ύb׊f-JZ$ %[PȗR e,,̗(Hj S98NAYC _`Q2,L `, f"2Boouǵk@ <ks0lYj !H ADB)<:]݃|C+a1=;s;N:{vITCHYfc]# ,J]b !ْđ>m[*v4NI Y XA2fH|M)#*d[HXBhJ]&vZfRdz/rp'[l`;ݧB6G`?ag$`5Dd" g-@ J*sxA:pt .6;p-MZJ.+6F1kkqt' !H3K)ng h*IG@g !>mJf~^☈mِ+Ozoo] A Hk0qT3H`-٥GG xd6<<><~%9 !'^Z\1FcWΌ ՄX-Ԟψa#eщb6B);^%(\7yYf%◬زi$%=dކnȎt ^}W[X'>HRi¹(h L'z$dTt,K;_{|KXʀHHhs@ QR6m)fbmwm;]4ΣaIÐ3`$@QvUYf4 "6_Q|>ۨs9r}}:IXG1/OF=g[N#-hod$2< !(%`"fĎ SxZų)t^ݪH015@;Ӯ㫧pW߮:6J?['nKp;`%_c, 񵞎 O6f%i]riJ7UGwuDAz Mת\T$,˶ d_^{ˎb3ê6b$8@u[Tn )#d؀[_ӆ' %p.,O5b .@@0b> ffx ^>ց˜͙&+r(?=лcyF0nV]3Jן Nj x``ttWvD!N`*5׳39Kr@Bـi-hpl'/Zޫ;q:7F4s"=dL$]%E`ZMU]Ѭݠ`~po0 6ƘP\^|\j]gcP*MG08^@gKo0 ꦎ:9,"KD\m i_LJu÷7(n(>'p <4Y@՛Dd,]60±WKSq"aD_(%22 gQ Y;mG$3`0̜hMzGgf/ĺ5~ `6s|^4Zn;#(HtM! \p,J% ym{R$:(3FZSJ"ðV[ J$D[Vwbćo،cm{ "ն޵s݂ϸ@~H@Ke0FkUPX0,ײ, ֧Eof1UHVb͙k03?Z Bp bhi[/s荟w|7_?~) !!HA"Ig)$DPBl@A`bƯnَwDHAtu (5 *ĉÐW 1t[X\Ե$ze\?–7n4 $I;OB;/7qnkԀ D醩hk@ ~F6SdZp3\ 8x س  C>߉^OG|W{Ճjicw<^iVbFلEqVK $RԬ>kW{~H׀p{m޳o^q1٨V2t,)"qh:ec\ ٬6aDҁ͖Q,va~}ӧƳ[}6z1ii9Fv0Z&twWnlGOA*2i m$+m6)h-P;pӦ SptEd(8>s V3QbX0ٓqקf+m9賴{9hiqv1+_}scInp{vqS[\V# v$,Hl]+kF!)MC0ZDr0$`Yv<|Kggc剆x˯GlWB@.d3a⇵ƚ[ d3ɩ/5 cgſyѿb菱<|NN^d Sy+XOxp|(j"Wg<<=7 +Ӟ3HĉF>. yC)`@J@ 8YMo=$?c/_zKQ$V`џ&Ѫg΃ja'_}^0l5랅;z_y4\\D/;'VDKM&4*cTS`CF^{֣RΤ,gepbcSރ}`ٸb}է>+*\rY“Tr#GoaF_7݃|E6E>z{ xuS8ќAx{(oaR:,²<$У#ıeO/?7zu $:b!EDDA+i_j%#X`-tZj_=2Nڏ!۹pS^+W يUw| J?ŤY%]cerfuٛu"DAo;=DJtFX)Jyml?= U_{˥5*!!Pr_.q&tg+vLD= M$؜cln,"h,F8 ,~d~MNenW7VԪqci%^<15^=ҫ߬7s"eyph҂E\ !,8NWy]x8?~ffbSyf GԌй,wDI& Lwg{ҩ:ӳǟi^?ʤ i:]Uȉ.اÿN>;l:͆Q}Z&<5Zչ_^f.j,E5!z\; NCN^^Rpj )QK!oݰ$5?W'}3Ǚ68O"]^ŧQo-J]Cw{ve.Ň~ M&TA6W+/Ɲ܇z詰J˧fOm tPaE2 "CBhbłdӱRr37$Q@J68NI5#߯>?~&0&|vufx ?aC/D4a;TPG_[֟}!n9:ͤr&!"]wR.[\(*5 떰rS*6T&]7?7Fhs6[΃/wuϻ> ={zGrmW\IDAT?迼,I۲m+a[J9 )FA`V eEJY@v RL-/xs덼sskwD{O v˷j״PXRҲIYP*JƖұBaB"@`aa:<{z ĹڧsξTu|CEtX~\ѝ DB-䅐$@01`$WfW^H !cwyYa2BZ Ć[ <*& \z>R,{~V[DǢ٬ƺl47IR/{DVmO8?n  dиrzJ`c$:V./̆OE0A^U z{zu.W` $ 1}((Z:ܱ:_ѫCo]^ԛ"iB۱&Ə'dm/$b7)ol@D Qg/X*IJmy>¹A$FXp:LՙJaA VEX|vka\/],d+sV8JY)"±) %²FsOv%k8>.#?4}hphBu$&poLyu˗EeێZXHߡCU&+T>=<0 Liq@wPhV ftx^:YߨJZ$~ּFziCc;ijC@ @ #%l) cClz+K%FKKWT:-~iIe3@_!me2%6~d:,Iaؐ禌㦒Z(ev'J̷ @_vׄ'gY0N],WJS|M`0L6 KB'1-l7m~vYؗ.*!1>?Rv07@]<3 n + h~uFJ}ɩ&}"xIե<}uu0 }&$5>\wGwz>=^򎓮%!3TOOi%!ʅl63$ RRF_]_뫙0j,'Ȥ kbJ5MlR^&EA'7rǜ]{62%4e#s#jI{֮a6 Dhl -aTfy!{)cw1*mu^-•B:.6=TuyvX f>lQm,sB(i;*G Bڒl'}fW+BO~;4s[x"{:ȑfcbX9p{_fזs Az}5JFbe;KDfJ363m^3W]:a:/$qkj^JfJ-fӶ9`xzz_E;J;w;4tFllrx?v.jafGd">}cWuɀ pT.S77yl<.ZA#'k X`(JZFI։ldA'3bҦ `2'pr]< NcBdH 'Z'"wdt;{e;𹭯 d: {G\9_:$eKҺ1 O弒&w6텐nU˝9à }t4<1v+ۆw L.^f# '>Htl !ZlQ9M㖩melc n;T.3}O5<Dž}+DdY:r8A '1p٢ޱ IzlLiNJ2:E+Uk"tv V'4~^з{wL>I9q*։W. {v}\X6F쬬j8h2*ðJ5y(g,!./f\o&tZ*k2Xn&,I@#ZͦR)J잽p|WU+wMq%C$lMg֮f[Al}`f)AZU|qDrL2)M-SNQUʒ{UIiwg!E0lDh|. (Z-,Τ4ԒU+S*GDݹskKۘ*{vhssad6fXՕe!X H@HXJ,f@&]psb~ee?ѱ}rB3Rx$A3¥3sgO,.]$ *Wז4d],e}'~&B F)ĹL<ދ:S7ސ{8lfRy`\/ `@H[y*Cl(<וffsNݻ J½:7[x?V]uT&#o63DS'"ffj67:15Zw7FMkc;MJC8Tg#&:7ZA#F MaĽ2eʕ>(܅+Iw~c!.,_ x>uсJ3 dÆ\}? /t|O?M?wן L]qݔ9`OztlqrC⁽=onA{^&X%IҋvnH6*0y{ʲr4n4jt:`8 8F׿4 [k# DFkF͹tt^kCaX"a0GhY 5\~g 5lԻ'"c_SĀq=?oRjҲS?ͬ-P"b6`Yc%=\!XRBM_>;M+i)F;=vڪǀط'#5Dcan:NH>e 7^^[.;1Jtl-"յRc+σHt, +R7]؎*W1( gP(Rɀjԛ峅( 偃oLMVtz}Zifh41p8[?'/յI\c;\cU,72|TDQKK˳偾E-;E4$@Oi$ry"l4b;d2௯xDD|DFC' 038& a 1, _a0}rxyuWڷ랹n=Vz, IMcѶ~l[$I3}(O rvR`iۮ&@2 nQ&¯~KC_J$00ʶlۉA0IzAZO/^W~lkc"8^8>lަ!"2y^:afku䃟@a!#u*8NFRHե,3m;< `h63ZZT*Ir@(4@1 BO7讻SϞxv6S F5:Q:I"0 rf+ǟ@-@J);p;Sj4j)e;`TzDzM(e#K(N2~o{ ]26YA_#b{wӊť+ޟ^=m6k Lw\C+"ҮJ힖  Ö "(:=Sl(J ,KZ|N$JR蕄ᇯ]'$JFn6{ɯ֫KO8TD:$I$_ hszD >S$INfJepݶxbb=gZ'hJƥKgRYQܿj}wo<H $\A*zM "rڒ S:E_d!330 KӅWc/u5AuBeF̌$$u\>x{?pѬ{(@0fg.d}/^*U(YۖZENYKkʶ1tCYvo=qDz~Rj\Ƕ#Gcڲ:O$ѵ^RjcKSL% 0аmԎ=W%m*3[lI*h՝(f|odFKd[$&),sh;g|?0#LdV$@ b52Aаk5c¶1AcHtr}#әhwc@Łb6ܴX03ۖ] .Ϝ1#{ 5ct7H.\>1)oqQFBmqi4uZssbcԨ[gY~񿘌ȸ_ۿIxeC| m4| aV<T5Mc"9Sl67<)vw=u]cU04cΟ?lYHpBƴx/{. wh{RpN߲ TȶI}czkc`owWRc~m4Jt" Ԙ"Dg>Cԭm9V,4gwܹr@z^XDȶ='~~ |6Im}E!cZ({gD 1IG1lBNǿN4C7٘L:~@D&Zӏj'oCD\*_1lnB/0+f ﷏N,+8t'ko@Bґ#8۳mKc+BIH ж|BQkO(@_ IsupaIa5S~vy3Rco #G]wp{ۓ[l[r?8n7:ZXgs\`GB)0Z'f|ds!{qZD`minynldEfbnfCƗ?uy+}Q$~C3_w}!n澭ޫn榗cč}/7UDZcp+yFmź3f4ވEuW&= KT`3**Yq3։ϿVe5^IUn-UK?Ԛ=A) 1 %ʶW>[eQ׺7 Z}lJzJ @r[ڞ7 XK !>Dd15B @]h6L_\ ~MBhRRXHIBÓ~׻6E@UaiK)9赒|{r g@gZx"nϸ/Nn ݿhn*ub-Q^Tl GS*ʕt'n!*Mx^ZSuRZ ːt(hr+fc#I⸩}U? V*Io/"Qt9!NH1 -9 7۷K۵eg$ fIܬ't 0ԫR+ IENDB`weboob-1.2/icons/boobtracker.png000066400000000000000000000202251303450110500167400ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME c"tEXtCommentCreated with GIMPWIDATx{y]Wy;ojukmI-$6PI ,3$SaR *$$"0&6A`[%k$K-mw??^`dNխw_Wwmkl{#-zOy7ǿ}n@&D2\?jg.ĽV`ߗx! D@ ðLi&81o~0 p-5&_ӶL0EA<7;^<@JCF pѷ^"?o[ |/Xv&BX BH&0uDg'/>H 0$coO}'G;ld8YagBXÝVg~w+w/$mQڒ]U7m;vٜN4L|n4UB+73M4N UwO[{ =Fg*,q-C3kNTE~ '"_?2j`ׁUoEs`ەξM绻ή~ϗ`B0Ct,Ѕ:2'LUx|GF]Nԙ!LI!MtaZ$8J"BW7>'z+~ \nQڭ] A`2Q AP(bU|1s)3Gw*1}>nx>MDłzhKLeV:crl`f>&Ϛ$XNx/Sz'0}uXzKwg*L& ",,"H@hkG)a`qr2+Xk{vƁ''wt_jhpKNeۊv^ ӴLBDB !m:T.0-9 wdi`g~D%ữV ;KpѺ,& g;RWgfiR 8ZdɃ:D85O^iBdLLaWa!(ҹT{{_cpl R`RȄEˆ0DUV[[j~>vh;gQxUosx:)4i$wqiӏ?qϫxѻdкU6'.38g0MioھR5c\e<3w{fvgrs:CϲLG۶YkZ|C`MGս)tc߸;w&_gbW 3%i$g|BסJIRI(XlDDz٨QO`@BD13a} qs?rߪ2%j݅{s|czc'U*+jWz0 (Vl^ f2YΚ80"0&`8ЖrX.lyeŒ,>=;;q෷zf""ohq+.n5׷0KcddDݻ׬.,fZ}h] w[};̖IAةBJ/-Zk&\'̏<`/Gzc7J#fD7rOfY/3 hrJ%l!87uz^%Lrʡ5H2&:8RPgJ?f~[mɝ\]7PBJT"J L&&Oeښed" Ǫs5ۻ*54VhY"SO~kceJq $Xf2k`n|t[$HH\T oޛ=ss<b_cl~]2BWo@ޞ5өTFF^H"~!_"@D- ^H~Xʛl&[zR .Ƕa;v`̅A"2Vm۾ڙX!5A(V|M[7:jϝM0сG>zVI._2nf|C#|ă?l[+Ϟ Jݴ*!5vE͒SDd`,]'Ha|'p3|-w` (enaH5Z;[vDY`@ ~ؓ_ښxK3Bu0?+J?8كSOMD14vaA",LEJ ilJ2gl.Jn>&z{nnX鉓8it$y 8D>wv-1@2<[^N$R2ѩ)}CT?H-IIK \{:i? }\*̐D$F:pPfsE8`X5L>ްaBܩ{yEe~*$d{&mjLr jS|RJa||ӔaIBL ~ Z.5!P_[[ ]c3l{{`1ǡ;=}7Nƍ;'p|_Ě驱lJΡC8\kcu|  SCOs#YDd1s+%!Rn&dh#I`f Hj$xU]F״LalեJg&;Vf"K+v? |RZI"*Idh6ks' Z+jm띹\ P㓧-RhȆiL`&f""b֤ =>#^P(6M)a]x[ituW Ð sǺk12[ m[Nz1=5M"E;2]&Zҗ?L\O>!H,HADłi`[ӢU¶tG rvfېJ]`4{³mqʑ] UZ=L7 ;jJ+4UG+VIlTm岂ܼan ,0Բw 0Ibv)l'k7Br Cfx^`;nbnUwtZ]pAD_G *lr1i.ܾpr_G:lr768Jh'j&ˑ*10 +e}#" bM8/ .q0$bC$&'Ϸ1ð,[YuW/^jťY-w~]J#fr;)jGGGH OW2k,3TToE63KI̺B׬[jxvrc;V6[`f^syeqRjY7,f¬y&` &@3*VΞ9k}ήL&DH$Q?gѤ`[4C};^覨UAMxd7k͙\qkh6CEDI+FG7|vs `nz Zd5tgWWq3UA"ZV΃'l@ӐhV'6^)ܡ5fsg]3u5+[xD̉2>S:QTةXVm7_g&Iduvj043'JZz᧿[BK s}pTIdI"ŋYkM3}s+MMDIMUBDI<_ҜeK5ðb?W/@~Xkjb!aD֜ ikQ!fGq8fSbGz]x_ۊc}=k'V1ERٱGKŞV:qiDqj Dr +6M+v|4ID!0ZsnkO\s22Q=HaTv~ͪѓRHDZO A~.۾˶9l/ AV[ 3霞f(Kvxx|TJbu3Aܳb`eտ#}j셞ݘ:qe~ ƚ~Y[/yd)PqufږUDDQb51u Crk+Wl\ c?agz@:{ 1V,w_qQJ0_:b0$6YHiBP3Z[Jo2eu/kSՋ֪\ (dqiʱmIr4ND1yaD-O;VEqTWPQ |aǫ_;&?:պ!<2hUС H ÌM{ @ $Js@ mdSgmncKS. ?=r9f9\?æ:Ś\ise-D1oT&1;^0u`>Fv#n~5RUl-.hy !(VJ?V8|́zc4 kbܶS>ǟ_[xzQ J.jx^vԉcm&@}"x x`xIt裟VKZM[8nR C \rs5~eAiBf(Q@D>3N5RRF6]]SaB@)-rE74J+T⤺T~;kFw\w7mxExuʘ!'j.hmuG~kNϥvn=;A?NG+ S9XMGϬV-1&pi gl-ZZeMe\|8),Gk"8PoszpmLLclU*UFxዏ~\Z[.MعFj9>uk:I(RSnnzgm;)Bf>qۑOozi;߄ 3c[fZi3 Y (Fghܿ&x_; -Wwڃ+GTSc'Xh֖X#S=V1sÞzqv%WkY/hZ:"G$Tvbͪѓ1ي :?5/}ܗ?غ͇eJG iôRmw>?W:Ny5NpuQ*q?/A3IV3MDD@/lrmkw3G=hfCƟwL=_~"7= |p׀6V]K=RkvA2BZq'}r]Х[GK `)bA:}8'c"On~j_;C/eܾXYv6\ݗO9WFDpx 0٘=^Yh68,,Ӯ\P-+n* 3೦ïnw<>%nQ(::6F|I0&ZnRT"Ru)DKiR!rȀO2??xݝ?{ LA0=;f*1(!J)=Ӵkiץ4<}}f $(Duwu dn&gXk*;b@LDf'  1"EL!QQ0=y6|U_<5|]:|,K'Q(ڊf "ExJJBBH8M\<?82xKZ{v 5(ُX0i{ͪ,;ĬCuQ|(QDAѹs[8‘iaK s\*nYjk2^*gI.f4+$q277Ε$cwy#wϟo-to+7Ax%8 XWlq݌J 뤕&AXGϞ_׾WSqx*Hs=o3~w>m+6!rYsɕtl$nuZ%7ұVڇB >_mįjfo  `iq2 IYUƱHٲ>NZK2mIENDB`weboob-1.2/icons/chatoob.png000066400000000000000000000217621303450110500160710ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME)앭FtEXtCommentCreated with GIMPW IDATx{yGuw}}{,F Blc@`1cNJؤvc  Y`$ hF3}o?Dv~Un{v|_ C=׼{p{{!H#A0ÎX/2~8ywnL@k  @:JXH'EX8p瀣 ;? z;aB0HXPq$[]]榣(f& n66|v \ Ht\Z΅? p_HWS ǷJKR91[`Pdn~3A>n!wnt04!+#wrBH)hn?lYYG_8SAol`dO/ b^-6n*sEr $@B``l,k?>#'msaB >࣏ 3`R(Z!|R7J(69N#EẢE++ɯ+-_Gӄ>Ӎ}Rgb58~ƍ aj`iɢR5(2NnظiI3*~7me'PN6 l r+c2mR4n-]\=Cw΀O}yo- {8Q&3O2/ :]nv5nxZnuP48Jo>}a j|fs|Rn[6MGw01:fD||bSnٴ/ry7r^޷y#B"-?>O}Hx)aeEG\=5vb>ԬmIlf @q  ܜ4qu|/Vjo\{^V 61 5 2ȞZ{ju@^PoƱkBm`x`ݼqO"}P;1GDe//Аw-+N6s=W*!|zmprtjtn` GTZu9Xm# OBB(O Ni䅐c %JVF󳀺1ܑP&A'ZW&P"v}R)sP1^qᔄ74Y+cR"#9\Y/_ NɘefR nVYG l|/$ʹ/~e7~ZO?8{V߿cj6d9,ԧ*Cِ.,Lz9oڼ^M3|}! F>3Z|f󆽓i03 P6 Olv/O#G[H:۾׼|q`ʴ^1fȁH۷ɠmPH3'kN;ruv`vP "3@aɴVS3gv 9mDgb-UEqL柒M793 /$9B B@P6Ȍ)i#Q_Dƿ.s'S0%>p ƭ;r9RGag$빾 4F4nP(TV=7H׻bu(N|]ƣϜ?ßs^Y [XyW 3|g>aMX2^zo?*@@d .K ]++3OaB380XusA)Bβ73{ty`qu~P>W !2$U|Y/θ֫d9oԑ׾xN {#iL5ʸpĉ  \01LJzC ԦК2ml:yBrR7BH!d*'OzQ Ãn?S+J9=crk-Sn/7HD 4焐 -HIu~.^.:wa-۟/5K7Og%J* c`z33.:q8~\yJdZ2obA(#jr3Fqg?;:<;< [o}d(bx6c[C-"ffC.\08~\c.xxj@݇nף=X(ut :qief0_ڵkuOȹbV3#CDdF}d Z _pLOq>4ZJ_.]X.k_u嬴+1PrkӏU8CNrTsCWExsxm;>99^nKMlwz NinR9s3Zu>s=?fBr0:NDR A$HBtMMj ˥ᰵ(U*h˖]]QaW 47_]ZC=?^gG6\>7Ȉ;ޞC1]Q#W'|rƵ|mٸoZcfV23I!6bS"@V J ۵$ĺ/ગ\Tl4fI0??9js'W|Vk(ĩGvb_x%@wP +_pSc̢F^i|tǜxY|ϧbnRڍH#=':uRk4F#>ӳg7}֙__byS "9qiԈ2m. m֦}%sk!uܠS \F&I* 278z~ZZ`@_Hfsdv57,S!;|ZK[vsV3DX#$lMա|irt)IBj6Rʴ 8 NXWn !dHWgNg5'XV Jڋz>ZN,G'2ˬJZn96??95??Y 90:88rdg@Ls%cLRi^2|}cF( VJiZCY ,c[6>=pH,+> al) rw|f6)Zvf\ lɵ7N^KHuI6 cdttS|DhRJ#v[muIY"XfK}/"":_Z3 G٦F}dadhL:HDY 昁ɩ3Gð(6L{^rb?0'z^n]p*8q쐧8"B@@(%u̽985kk_qzˎ p ^Lt +XkLܜ#@g&N6dߓ`Gbfm|EQN1[:}xx-_v^[@m!U6\&4u|v\3 LjHˍr`ff{)2NqGEx1}o=U|vEHmDqv\AZ˅Cz8ǖHЍc WѱTr媫oou^E 4Ӱ53_ V{iA&'OھsuO*?"!75sf]_Xص`JXF` wy/G?XY6֬YVu֒n-(ecr&ˮ\ܱ[ÃDTW]z^'3lL"2e-J&&R]"$YЩӇ]|*)􄐡2Rr;cg?[󄔝|W=׃FyՊqUE (|^Fk%!QFɴMS㿎~ /OP \|wNK5!JEg``+H `Y156f1Iv^Oo T SHse._&K,bq{o;l̯nlIENDB`weboob-1.2/icons/cineoob.png000066400000000000000000000213501303450110500160610ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME &8qtEXtCommentCreated with GIMPW IDATx͛idqYU~}wOϱsA%4((- .R AӦ-۴eY"pHLAmYx A'DZ `}tOw2a))Dy*3I۰+hP׈Jc \l>q4+QS đ8N,\n#`O^0K2fXnϕf-TQ\GQѤ<^7nv<eYE 6SWU @WL]TcӵFsk5֡RD@ <6KGhm)EQd. Ve˟^<  PbF=1_p#OcWDtZmFP8.L)%@']5 vjFݝfsr#埇8|rٔ+J"Q[GZcScZiOVg_lt(haP&M f}bH\ľߚa +~>tzzWLy;K>Ad{5J0""ΖeWQhwG':z,\}o4;͉9==skmR "d HD@i׶ O͋K״Z$7#2|#ƆIg6ZnO!"hbO-.HSlos~뾿LpgySdcqor %^B4@ ,( W z[3_?zdz{̌}- ^8vLMM~ʋwu^WMy0 ǵR0 @& ;^vVKeu灷-dk*;rX{_kbA^b!D4D@"nSFuC峟-?u@~=@ .\JBSSS>?O >Jcꖛ>o>&beMu@y18np@сko7љi?~k& $ PQ@Q?sOM}hwcZև^w66?^zիpwe/{]/|qɧθRzǟee%$ " <\b B77=<|Lzo2s'_n&GhY B+Qoo/72ʔo-=z?|-h ~g߃__G yk{^={クp9:D~˗/*]X<%d/ !/D*'RJ( lY2аE|Ym2\6 qlmo ^xx433Cd>9k/s#/oIt?"C(-SJP=+=*R4:iMKf&Kxr|o `;3159O( RW/uΝ?u=5.='ŖϺ;pooӇ>! '/;;@ ПBcB^MoI$V8@:6FhgFJ(G_뱹pnH@ Ee3=~b4쏓Sxp4~GiWJQѝwމ[}Q|S/666Э/ФSk]?Y?t_q:F3Pc\:؍o"p's}5bfXTa8@R^'0孵v{: è  g=wd8ΐZml$NwWnYy&RF  إ箿_]E/m魟F޴_-[+sg -ȕr؊rvnCW@؀9Y fx6ac QV:}`gm/z<>y:֞'&V:56g˛7tl`|6t>rLEpm$gFQ5Fc%3#$rI?Z]6'Evlwm;htT N}3ՆAI|i2TtkG]30: %悍Y[ߎ}@nk:vHDpڢeD;Oy'}q~O-6"_5V?|uiW"boNlgl/̨GD!ڔlIhoPmOc,VS'' "P E1<^$7$\gqm!"a 18wOZoĉZ">eˎ$=`RJQ<{|+L!F=<4Zڛ2@Aח@ԉvkL@Qc=x(D73ƘP %@b*Qnm4 "[| _d>7NR"P^A׾tLo`,EpK@aoWh.w5Ճ @""!eOy!'IŒD&4l4#"fV\5ufS GP& ^-72}.L((]ֹ % D"1ƱjO6QaRPrQ$eؾRBP"E*tSl,˵wTXqщW6{ Q ((l^PG1D@ b RADRPN(Xiõj)E"ZY-z^FL/%DAr9Q"BY!bq*0 (-`/! j&T ZH",KJ(B@$~!PQ ]`q,yF:=>)(Ю/{ZUji 0H'Pʪj;xb8W(Q*;(T""P$M#. ȩ6 I/U1R/HQ@@;}qbj'02|"7AacVh'w Se(Qx@5YRBD" =a3&'i# !d,jLV 6bډb1h]P\adYX4Nt3DB&dok`0{@D ;|亭p璵9|gXC QmȫmS`^dPCn9 %`0'rO7Ċd/i !<WW/0{њՕݼ7}ɛ~ Wzl W9\Gx(vٵ 0@<巗hR#4#~0}tЉ Rή `lngюAj{Rr6LsSJ<U. bx "\0y"zijf/BpmŻD2T( <`H]E+>|>\Dzsk^&3y̎;ɓy&JCDY/)K5,&PĐb,#D+**笍8H}0Alٙf]p ;)ml (z{$Bv jJK^l9rurϳ6.@*=7ZA"#hSE"Jl{eGV>(P@D(&ZgAB-}޶ JC Q>(mx4VJ:KXPd}: !QiP$dsj*R0"SE|hv @1*͏v9BTQd{ ]č% FJ}U X8W $Z x ٽL"v: [ Wڐ0k/*Sb0s>Ud_<;֔R2Vd{y ZoAr-QX TXm\  _\<3 e.F]_7ruq )@!aEH"xxYȲw#}p/+-?G?W% (Qŋgĵ `̤A8Y׿t#Y_x !RxD.XФR D9&F!LԥĖjNܰk1T8 gf//)iR-DJH @ 01)xE4/>dRf㆗E7:a& WmR`V_s .9 Zމ.1Ife(<۸TAwm6QnsWnI:t'|ceT[;+=u|W9-e ` dQ${A M^^Kg||jR[|7=5 l@q5GP o\hs'!BHT`3諒uPȉT&ؼTg'TԍfNjA(be 5)y(7 W]^)""^El'3gjml\0D[c3 |כ XJJu P鰄ao̡8>$"pދsN[r!ɉ||ůd4~# l0Wh`SB PTWg}Qb _{#'G mO;Tq[**BI1ŚPb%b[+*  @{@emO}+AX$ )ū*߽yG)3s;3$P%KbHĠ1o\; 8i 8( {ܢj) /r MyšК%Z궗(@HDkzXF}]bΒH|` \qKpvH=7IDATUCa9arTw;Vs[/[ojlVV. B @ fH@A Dxg58ܓ o3dh;?흱3A%td$-ē@^O/=w  䖐Zp}[!K `juFH>Ħ:f= 4M퍂UBVA %(P82e>-9~K N>YgDAڲJ^ċx-+K{Ē{RxRS:/bsaLr nj/9>%)#6 QA;rJ)@P.6X"X[rv1B? /싰 _()ua_m-MMKqE"@<g=a*yF*9_~1v=#-Bj M85:-i[& ^^/]@JCF po?vXsGƦb$ `Y'IԹrWIF;ҴDo,f3Z/}FirOvdxT,U;C|Ag2ym&IB2x-ۭʹ3V|"H5G !>2ee9eC`f͉:|ڟ%?nyo^wlqpp<b!31 @'Jյktg}}I&Qrᫍ^-F}N#H!MtaZ$8J"BҾ_|ĜW KUk׮} df .AP(bU| si;׎}jb['vK䞼% RGضVYf7WswTЄܷݡeZ69 D`` @2 BRGa`}a!7WՇhfr˞>; U>RiZ& i! "!60m8 G%z$y{[pitdjddxRlc s}lt/ ;Kžpm}ɴ,X\ox7UVgvsRnlh&U &$" @E!WQWcHQq%OTq``ltr9u#f̰Ȑ4XCk?2]yѤキP)uclh6rܬdf& ܓ4@ F, ZKz7^9 jm>>>#gv+fd%"6l˂P(*^Vk1/ӹr^;""ai1L̞ 0@ 0iQꈾ油z (睟0 Gݙ{0g 0h ӧL/יDq2fGB=SԪ!cb|G`bI ZKJBK!FD!!ҕ`h,Yhf%}484gv'RH.$gumr 'EJ^_9{hdTfGDpr&"0,yRY/2+2M'gKZ&SJXe|fJ _~g>1 OX,pѝ}ʥcn`J)[BDF*[gJʼniXfvv^x|du}a8 C38`43) NQdnn]wR3Sc "wqjѣ N3DdMϮ̋O]t? T'v15! "'̩s^JA114 /.2kJ%kKZD4r٢qk=n `_plev!u- "ۅ>j R_λ8S,J`1 R2<2zyyi"}0<0ECkpAAp_t=` PBJT"J LNr42m[0+j#TIKDpa"r'f~* 79I]ϤjO p4NQ8y;{:mQ\_O 5_ `eu>e@Z hW]\'C3lA7-9G*|S=sNrۆyk.m!"srj޽ 90; PSe{$ BANk KWZȬLlLDZ KګYYա0CI\9`&o,MȻ~Rn``T8n#O?x}˽&@ìݳlNoxuvy!uencZi![\~(AN2Bוb4%(Ǎb""{-Mq 5+ZlPk~e!.@.A2Ӭmk UAB2X, il[(Kv btUD&3DE| JYf~{Օ,fj:Y (@$RtS|P^zko H4ms+S~wl2;}`Nk2iiIi؆aB˕;l# &d27&0RZǡՇ׆'i/VV׮N !᭗]'+jH؀䵥Kuf-j}v|`f&H HA$h-&y Lx5-өVui7it:M…%U˫mch'+{X DNV~'u҉ ^Y8/|}AD)a6k1Y&ja7(jb!`0)%KgS@8U$ kՈ\1M詾l=[Efm([ZkT[X5ڭii2'ZZ+rO~n[k8Z025Zs.[^\Z+"JGJih;5(Xk;^zLJk@qs„!"mRZF0;$!dtuBADe3cgbZ ʵy͌=̜(53lzĞ4s鴵iǛp>zb쎗bGkQG. iz|_0>fiUhDH9Q5'[]ۄK~pt-5Fkb"0sǫGCt|3EDq6 D bӴR #:GJ.ϟB1Ͼ0le|8<<ʄ豽%Tùע8l;w,Z%0~hW߼RGsWώ6kHYImtl]6{3BbD`} whc}|Ps}c9D~i6yZD'N>7l ϶2+gqZi$yPkNnoQŁx30naqfjzJPtˬC&\92ݭia1X2kUhMFg?|moʰebWq {;q,'nCoJ :N&wurMwt~l}-koKs@ױLH mV28`rvmLg#(ff䶙nxU&aD/_`(" !ۅBw3s@!krwu_Aݳ_n_Ph w~o>_]Og[ xʼn'̂L# CBx?q?}<&4On&Om׳/ <7ߪWwٜx􃟼±O!:'߸vl{[v-/_ygv_J=#ع} Dov'K}~ {z[?ϾӳO|wO" zpeV]Q*J)koȓ8Qi٭|E$c{ޟʅ)=U,x ?߯?c| [':~}/}v'h=TGv(@_}cu&~0CBv瞃/?5 S%R [L}W<}_|@(@?_9G_٩$?w {aG . N>M{*8pz9uު˿pɃm?wО;x]Cw Ә}'!V$=w?֦cMOSfugrrw̕Ue|O tL ~}g?4FaKukc{oDSw LMckghtpYRwW= 5Y+J%I,/_յ$۝hvvdOqݜfKd&AXGϞҾTqx*HS|T]~ࡽBf> }m:6 a$%P:ac+^ ב̗iq2- IYUƱN>G{=aIENDB`weboob-1.2/icons/cookboob.png000066400000000000000000000230351303450110500162420ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME 14HtEXtCommentCreated with GIMPW IDATx{geGyUI7};陞Q @ 5 6IeD02 ,&x < #B& #!H!48ɡs} U=HXwysOs޷ToU~ֺ Ӹo:."<k@)Y\q40woFs<p,yCn!IJJB$k@ۺ5qdaL$ jKsWw&]m=^ ?ظή,t?Uyt :`kMri.\s31#X?gnyf)uײt@*weZsAV#mB1,'$̈ruWx5`+ `n JADU^–u==Mi ! ֢X>q=+ځ&܋۵lkBJS&wE;WJkE" ːJ 5wvmG~SGr;7펥>0-LXѽ--пE{|ޅ纀R`1֨JǸi<O{w2BsyqњI7ZK+MSr}/{lR  G,[#x͗^z)^Z !8~w?'_~p` iH@ np\ )C'  "Qxq jonv9X%K^֒Z[[zdo7uyU׎xuam~ݻ607(PJG?z]~\_qggGv޽z;뮻֯_?~e_g7A\([.VoCyixM50R@%bZ"&$ c+,s/ @}}lmMdG^w+8?.n.dOW\y%?#kxWE_~9aixxzkVqS|dP+׶r=c'JHVQ$#G*IhLH*>YN18PmdA@bA zu[6o_#^ӿiwףI|;9J8 l˖-ۿJ {zzUzQ#=ضmR)|EcЗLSzYTGWG7eNu~:.^>Cn,_ lF[[ӈzw7l@===_r䭻R%x~ $"I$yt]8~F3x_ ܦ|Geh~dT~>`-+w>7yz[WWg/9BE:L~8BqݱccjjVw,,c#L}߀{0,^qg?z>ifx!Tt-vrhvM4lIrIdp vT<=ʑ8DC9H9ٜvG7:%)EGg?Ykhx.X>%$ =oFK~opWŧ~, X^){,9[kEAK3?Մv#!A93r:X+"RH^!KVL5rt_PShΝ>75]JK]|&Weu@$lu~:>=|3J"ۜs Knė]vV\>5\^׎LHL'ڌBMCRHoǤT`}& -/8 7`:BIL ")Ύk)ȶ6z]{.-+4K^NcZkTUq*4;;׽u>h ijj™3g'?n4|bO l/,g,7HЮ`KL(O;N$ع&)-An!!!lj]4:҅\+-C#^eBT^喯ѣ!Pqޟb=r&ӟ```SOQQ$)7js}H<Ϫ 4YkȗDy,g@{6n =%nm96p&kmj7Bx^;J)lݺ`ܵE&ϣ /%>3<+W9oXɋ3m\m+Z9 "E`ŀ j0$VHŀ`kӔkTZ)Wp*q=2Q'FV0=3ǵZO} +_  6@n/᪫jJ'&\;)߶ !ʫބz%X+U`V(Mt蹱"uRR2%du11$`Kp|%b.\.h53 +) KBB>CE?8x%hn)h{CuDQ)%8ߎlˍ˥txM,康Sm7/EPw ]e)bf_rk S>t$l%JS(B'.I @6| feI A>jCR~XfnUR $ت\ewҾT*=ցA0|]˄'nIsځBsuH ^qm> k2\q$8$f01 (ёVqaB+cTNqTuqnѷNT랕B6$κM{[NJR!$SS[c+ S̃Ȑr"m3[(+s r dL"Zp}Zozʍh]|k>zعrDc?GM]ץ_}P'{'[Mm> Lލcו#(MXTl!b~)r R9ڲFc+%2H3FIJ)!@d@ 4"8b0`A VoMtt2CgժM}ewr!}mE*,V3?!<(Ϩ|ˋ>tLP!$0cJF"kʳj7 3d rm'!V+AB nca&k -/۱bVo0SZ'F?W_"7ҋ7mTi9RJ)$r'J*ɀȁr`L#&;$`!NtP^](h&"ks'e+:Ld&A M~FM0 "V=߷?yT[=zԩ=W-Oz^/n7|w >V25ja|}sMxɤJClaUavB)>.l# !S7GHV4D%:~, ɡ,[c"(ZCufaWL*?]Þ*8~uւ[nc-я7UkLZr l NɹcE׌8bb6 'eӻ~2acV]+ƕw<ΊfkH2؀e ՔrB5~:& D$.,tP|C7ʗJff _\=22ZشiG?JT I`~~7x# cǀtf)ZTZ`-483)ŪF 80az jhl$%:Yv^U`S6_a#45$$۠Z [(t=7rRgymK<ϳ TʹLۺ+*g<Nqc߾}+WM7D]w?t߽=*Q.9![FZV8ij1 Ac9ja(vH⌏&N#VSPj[ B![d HJvN:ctVPr]jUϜ8XtVYomiOK_l.s;s[lŦMػo>IPKΙo|%vqך0LBX<V R@I (bctxb7)DZW|0"~0'&|"Klm86;3Y3eOI=$NBV]L3[|ؖyC$Qsx衇o~$:Z5+NkwXjEŞ<#ϒD,AxQ*,I mh`O 6Fu_iV[fⴇzŁrEP+c{$ㆿtT @&@=7?8p@0  Yg!Ph۹E ͭ;w]ԍw|۽c#xx~wގ v<쇎Εjޯ_3(H0`[LsʍaGa)Vm-I" GHuHj;0sBDW̍g6F*@:ƮM ZYrja̞IImXM@F:vvXBe-f~~Ox={D=ݲWB@ P$Pl`Usfh8`4'nώz"VQՒCK/mO:" ύ=8޴ȶ,\3hkTozm 'Rם?ֺb~dDG<'Ae}B{2D"z3Qf׭}{_R\ (Sp$C*4e͆-,`)TbV+XP֔|pJgb411UΜ)"y*TtZ|$\S&'F+W["dcw▖S(/"b!K41a`$`D=w{ 6m=>k>]s*%zr乀%eP^dA f ʵTA0C cCř,g keii j^k@昉bcǎab '-]sd,F{7$k(x9̂ИېRZD  #""ס' l%77Gǽ;$-ζ{iP&xa 1nk4$ V}IuAȂ ѹ椷'lquhb0ų#yD0WA=P&3\ q-uA, !)XH^ 0,["h?~?ql_6"37l:1n&u4dKˬ1Ȁ-sӖGQ@ Đұ噔k&J5-*dLp(.? \lQQ8|,>v]fEJGu Ķ!B%=mM$}k`ҲySCf hl`f X8DLeU:[@Kd5bέI71@9(PW(9 k7~X ˉWu4CGdzxV/OzO>cb!e[BT8<؏rGojΥM/y+ǿ}{ LIh(" ˰VZdhD3Eˬ Ql-BIbfAR#+` !+c^;(AG1T |m#${r)SKA:q="cؔf'hpԙ1I,^nymGNj^Z!۔R<@Pb`xIC9+Vؐr (QL$,e=W]QoWØmwe?G#Wɡjj'ܸN5YO[%%:QXsjyZYe[۪j߸=HN=w3nM hcl , 3k,/]BTU0JOt :[&aH5K ̈́ 5q<~2k&H" *co`GĔ#>q<Q~ƠlIDATEm*Jq 6I"Q-;Yw~a[vˋ%Uk1ᮭW;֚pnݻlQ>A.&YZ^n }SKT6_L% N!'2>sΌ4eCͧIU֡&Di2+Ӱ}3J #L'Jr50㡩 So޽dAL,%:2YEIl7kZD&tDa|oH8rfP?JKU5nUaճqX4&Tz LaT~Y/8l@JCF ;uvN>WEG=Rp.KnE|  `Y'IT?}ll˵$ B#06}~+W_6Xu  ;ҴZ:f Oi8<#k`\q04lIwwMێt6S6 $!߫ZEֽJŵjecSgHpۀ'~-F[wfr帖! IB 5'*"?cCg"GMW\y@@2aSq}0 !!AL:QZLJԉĈLxɡ:n/QPK][A0ږ]42d))i.LV$GIDQ聀7 qOlyۯgtp2oBk+_hVwܔ!HLf j0@o!_U굷C3 )tu-\ ׭:L& ",,"H@hnnG)oEY֟`@읿ʱ7M0M$! A0 A$BڦCbw, ޤXp,K&w&xp_wI0kXbc uVx6/  bxS7^AN\?_=j/ L"@0 d0 !Z2qj6 ՞ܧ5xg}^Ēek5)bN1K`.9R&&t c0{M@[,wKo)(3HME7pD&@&$$Z IkIjٖ달09/2 t㟤:v.5,]&]"r-0RKpldi&[m}/Nf,vglZ1P0;kN>Lte #&jDiXR8F{‡;_C\yѻpekWgoneZeFF[o{j=Ϯ?38p/; \ ~/>q$]jpeX\M0^M`|rE{M& !D y bӰU^6Y|  ~h_GV^^ض|F:)4i$wbrygxiϫ6HĨs}Squ^w\߂Vf ąII ¢C1F&h5ik2CϲLG۶YklzST(Ӊ5> 1p _lF{ew,5::Inh>cjߵucc=`vR8S*I tvd'ND9lzYm)"6@D0a{z|-v` {KQ3LX!m3»~NZvjy3bj^Wu.a4 ~VژmW<>Ҕ/M(VaHdSJ< Bsbۣ#pgŋW 55Re8X:ӥfә% Q7yI-.)\QDFSA)jL 8SК7DGfCiagB,HaĬ 4!Z#)d[Z58ݡS]q6tr=tZa6_{x2Su%) "j IcZ1|$ @@khL1YFgwѦ]2v zm*],e2}~ XӇ$=+zI, &H'G"Ly 5غ[Ox37/ LDD̚4A:>)w=d  t> !`M J+}ބaD!a>֎q1[x(d f^a hc 1h'}.K6׭ɢ͆c Ċqd]aIÙl9خOw ZCiNe~}¶tS rdL! [})Q{S(\o}֜AN5s,sNokWfH V bS Ak%CV!*Zs\.(Ja(`3 P IK !p ;L!]*u(0 YGMlMB̍QJW*.oהe5`Hk rM`iK[h&mt31Εs0yf )Ě py..mM ÐĊ i8 òlNg^:a{臘ˮ+d&Ȭ<$EY i#ycp\"zj;Lyj.;Ng ^jDLZk7Cplf rbb4L;qb@0 ##g솉`m.rrڮ_O-mXR۵0KPs&a6X~ݬ3 ZDz*37{A(.Ϯ ôOc!^ r f +4-HSx.(&m`vw\w%d÷:7s sq]=\ȷV580 "J\'dgZfLs課 " a445B drR13 ӂ0LDOh mppJP-6I鼳 72s`?)З>`9װ_ñiwl[n,HJmԊI|{=ҼD뤘FBIdcg L쭴vUs= 'ȴx?'h>g^ة-8y?_B8E<:zѿe႔FBD "555n3k-f88U0LhټAqK04 ̪Phtao0{^ʥ;['+):U޵o JoY%ம21+Z=XJ BRs=/*l^?pܥ8sJH/$™w3k?O^s+b9Hc +|LUˮ2G?9 `{5Nkf*D4Y)9iD'jr߳:^$SG*xf}>xb\/.ݸV҉JrNŚ5?\>,KjIK9QEˢ2♇WH!YKDf$vf!giS*_Qܬ"8 ` C_z-eƼ?@2 &ԡ9,yQjy/}Ґp.[2 +15ĕJ B5' J椧wGLLvC(NLfDBH ]h-I۬KiLR:wbØ_hs8=7psTMut|[qqߚg1A=(é @BDiZ߱cd iBiX} 69}<꣧&:Yu+ބ+i40ھؾ2jV.L ?meÚ_O?d39#Z1Q,&G[T[ JΉd?BmΕ]]0[.dnI@ 760qZÐ9ml+SZ6"ۆd6MәZ0D 3ԅOwD=ҝ]] a2@#e$YHiBg{7.m3[6#n!K҆/GAZYf9C(.@Ng Ey=S̶yK)(Vƻ_xla.FBt"*UC1jnn9dj"0/rJ+x/ I9J.H; A^[ۂ(Ŭ(`γ7C-E}ȋ캫cG BHF~k 5tԴ^~3pl<|2|rW$B9.T٢m[ A@$To̘Qylh@G3.[bA3vmgzN'{[c _Q{ꔛ9UG<9}U<A[ {HS'_Zw`Oo |hw;{D׺-RwgOײTZ bҊ=xxjm]+eqUpP,H]'=:DO(Bf Ӭ!yM s[ JUt΢u9a -剡zRB2j͕ ֑bSGquD>}|Tjw=kn=gB۽2 6@*J%R)EZ'HF(`3F_yC.0FNJ11E ARJ4릪t8)ӴR>>3|`xLum[{[owoՆ6ao b04N(0bE8#" B `hX[>nx[x{黈-! %^:B0Q[)+!PLD1#15}P'3F;c}umK.p&`| Jjc?ijj5n{,[ĬѸߛ4^O (" ztD}a6}??}wBSo_.6n[[dkh^/ɳ]MPҬTh?LJد ogsZkAw]N+n* zVf nFӅuJC IgϯjߛRqx*H^uϧ󟯌 QW1k"5;\ѱM&!LbdY't(|KkU+^= B if ҴMYJ$*ث%lI{i_78r(fgIENDB`weboob-1.2/icons/geolooc.png000066400000000000000000000206721303450110500161000ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME5q\tEXtCommentCreated with GIMPW IDATx{geGu淫߽ܯsPnJ` [$ c@c`MN.[@Wb]G^6_pBW0L8=y6 FlcM>ƟJ@J@-C`l ?c pz/z2JW)r%% b5֚c\n~/xg|QUzdONV)m Ճ<68/)}]7NV^S_^)W,rK2R$D $m&c'OO/3:?ԉks=jȆ-xK cTĩ ѝ uwkئfUJW+7]V)G b޻k&G/N1iVJ[V;:n>?># R8vdK宂 rsF ņ5GI(°.9p~aqB|? w{uVRiVڻ֭ =v7kkg.&- AlgT<`7KxqP"js2%=]#R|n &ᱳZuFQY~Mz< .`-wu䕫.2lya}~u~l,!=йzscLԭV$( `R)9{I cfN]С|!Mx=ؔՆ+koo[j^ڟ}+L'J@Z!O`Ãbgp! !Z\(Tz[喻 > 3x +ּRc`zpp9u_[ L c'&<~rs\o`G D.L D@*^xRHVq˓ \w@UhٲUd-w_G֤ U.P(kv&rTݫ}ZݹhK]ExY Dt`x5Mlm x\"@2C$(2 ºBͻWfpJmlLmC|[g\wv$n&ˤ o`@#bd(`D$ n&(K&&. KlqI! HKAL8CJa,otXg^}fV:ϗlA٬H2 }? "H%}+ wB*: $p 1Û8kж # -=Za @J)x~ eaa0Z hW(٥ɳIhfl)I3M=0 5x#O3;YJHHK˔2ŶV\u$S7?=^+4`L1`XJ@ !- B6\e T4;c-F |Mg `&"$K!;V뺊[MNu$<qP[̌ii"R+;-9BƀdhIВ`Lj%!s6/lAlpBr,P}`rgfAD !t>W AQb c n@9:`*-::{hM835uي` Ա45FXkɤXg  03T , Aĵ6""?[i!+=0>;DP$3`eA$K PP |eq]R`f 8BV[Blk2LJ)3YT4jH$l#I:`Hù B9q3Zs %+bhMsEL:3"cO I)\mQ7V[XTI I\d[$IHl-њ Ykb[:9}Aݝϔ˝uk;֮oCW%QPxd7#咇f΁4f̜NQFcEsVኈS"JDrQtzVl 3kk]J5sk-`hX؁GOo1a}Oo'efMoٖQYi33g\,Ovw ͥilHI|FIgilm{m(6[qxK٨ys'fB Ǐ|o^TEsŇo{\ ?˺t{~ɿZ 9zlo_=pN,3k!d:>qͲѹ\yjڶk<<^Ug1dW|tt !fy;?xnaJkLgp|?8?g妏×?_8ͻUs e~6Z i7yn~(gX) &G΢w59q 3)fVbYM悰Q8q,[0[KDHbyU]PX2s޿zfxÕ4I4Mg"3ۜ\&js-gBoփ+Wl9 P۞$`Inzy׾;pon{vHlqǵڂ`!Uʉn{ى!*d`뇝}[^T{{{5M9iu陙JnpXv`㞕[=ʹyqT48hnkc{^nlX `{Ϯ2/- L;E^xna­RR[|>WJaS-{nvZ 8%zRˬJ*]\jӝ:M|-X:ӔS1pb8sfhJc=+NXss5(/3Kds}13'F/`K+G]?po|鯴Nhlc֖ _LPa" DƎR!a)1bk9dr5;w¡K_uM_}o٤a  iD=D1w/p86u䗃u3XTI(h"2ſV-}%5ƶrɛ}?n4pLH #W<%2 (u|sy&fs &" "-4%1=퓓c"̭^@OPn-v $Ak_(T|9V>A@uh)]|l'.Rs' o:4stۉ|} 6$3 fg(&J&@@:8aF,E8}hHw`!P_,u\(zng&-gʴ{ꭤJ#?fZx./\8չdO=6Ό:NS$5wu=$J*jRgw_ d2 `$RPEaz}활1;׎]e Jg_>bp<S'{IsqzG*qipcÚ}*Br ٩Iyu޾PUVa== ѧǎ4Z䉧2OS A!d s gwu`z欲,ZvȊ5`^L7- 2hL[[$:5ԭwU/Swya = UT^F">ia Ilƚfu?_ҳ_xv}ɥ$2{)?W(e X !Z0[iv֚\^RK3BRyN?xp*pbvSnsݼch%F%zGzc_۞tWx뫽]6UFF6icLe~p2C+-'Jg&-6HGQíVg at w ~Y |'/.=Ggk(KKHdꁑ F=1S'Y+lmL P]fd 1Ic95} N,"!!a.|hwxj겗UJ{?1-n7|` ǁ_]&\Q]c:;ZaBڤᣏf{1^+G lJ$ -Lww kpӋgϒ|K7|%O MΫ:"tte5^ܭ fJHh\@b"|qwPRk1Iұq!iZb?^ImR{T p*5Z+1dJ !2BKW 戁|9-w\6Y+饕{!+S'\M͌y|)VJ%8Kv+LOf֭('f!-~-m4PX`re?LZc 8&P@ĄC!B̈8QGItѸ>zzuwߣ.G_lq5X*TQ̦I$*ި+b %"CLUH)$D(BFsSǒn|'1wwaɟ`,I+n'^\`j (] R%$%i?~L ǎ_ɟ;] IDAT;*X7K}\BH:%-[c=3s'uO|8xc~S8]Q^-PPn&m.ז^Fka݆ͺIL>v?b[H> CŊł53{BL )lLbpI>OtVÛ}9PN$~ 6!!l8M.?zŭ>q7~2 sN۟IENDB`weboob-1.2/icons/google.png000066400000000000000000000044561303450110500157270ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDC pHYs  tIME/tEXtCommentCreated with GIMPWIDATx{PT?X`S;4iMbƚ>IK:$6&d"5&8bi1j}ZF5 ,];z/BBWG$qx`OQgB7=RNs@S]YQ\r 2X'дr N1S68 *3x֟jU!2y汦U/?$bL_C~c m=Tk*8 —ev=VlOM@Iր@r]*(t{H%#9Z. Ccuqn4un\ʥɭS`t r!DYkw\fcؘJ"j`ûq߽sޜBK}5CDy`ןӵ?ʸ,Ilԟ]r 9`\@tVנ8_uU>rFmzMu>B\1BХ9DTn0 ˃е' %n4u+DBB< pHu@ ;4prǧ ѵ0!aIRoSiV([@0By(2fO.uٮ_O2&-S[0!yBb'FOSuT@h7g}9| KLԒQZ{~yaot`ot:qڿ'QX<2Q~ܧl6!DEUWY3{7=7\K͵g>OӾI/N5^j?]n 2͑x./},䳁 6kV*Xm®~!jd(d1td2)^>Bco(lNw)ovؼexfOD_4'e6xe ~\=n[g? YRJjKFQQ@)E/Jߗjv)0|W?2un\FRfYIΊ' %0Kjl8|P D+[{=`pG^/0Z[lqyAv['vcfT`:e&.& 1Sݟ?Y1m1#bJ &Fc1pE~L3j-dSx m0V#5^;_N[Hz~"#JJ"dK_)/^toLU++eiV*#ܴεJh7~!O5.{C'/vxʥl'[?ĕ6$ĥ)-.S}hrMn^e-.nuI7DD@Z+?ї_3}JRrBڻם2&Qf~#.`$[$1u;ѕ?8/ͻɟI݉$dM ']m=\m|ع9<"FSmVĎcR#A k|orvjNN{{h48&sv`>> ƛX7 ;xG֞ixz^"_~~4x41QFɗtP)PiYm[$he- -vpfO#5s5Zoڍga< yx8h|OQmA)*E]`DfDrJMEBu΁)OsM!X'8:1_2~dgJ~Ijob kRE&לH7W:p[׽Q{c2d;9yVSV#┒@D],ZhԾ'0ƹFbZrkԘ센 Wg;=3Q(Cfr/ms^՚|blѡB/KL9gۙ;˄q-q&Xhwڤ?{IUTw1wj9UqM1ÍTyFCHmfYuz!X<&AD0}yd TV_ E,xqE`Ӝ b ٍc#\K}VU(m)f4%HmvZu=VYP}o$+uօM1;Rc&Q=LoY/HCs>Ղ{jވ-Ԙ , = X˭>rʕ Oh ƭ"rN/2#>U<{rw A] o0 Ðw?xSVjFӀ/[VigIJRt"J\oC4p}܅W5.hNbs={S/G[jS4>"8k|X54XrutPёM4G| a'+B+Bkt$>yW#p_ݕ$" $ XblOn7Z ώ?H;dD} 5`9ƳwɾH#WRM %]uDL>~Be߂Wx#d $p r1t2sCeXuH@D:Dp|K_TݧdsMh.` 3CZA0=ߕw[MjG{I^펓 f% `O@Ci$OAȘd98RP E Ӧ>5ƪIziicuWK秽~*;G$n˿}vxgioYbǹБۿ@Pp+UEcWT\ c6Pr 1vd- p /a)(88VPoLP,]KBu\'yOI|?*D۵ 4.r0Os-δsr-O |c.$R$BHSP?a5$ k\3B:d#^`5y)Y"XA%\s2agWa]x0_FAL|6/ |p R`aTx $f)@,h 2}'d'4$S[~@ᦼ!遲L;VjE#Eob~B$ႜoTཞS Pt]c0B>mB;4&t~ ><Ҹџܯ #%R'{دo%ti{T݅OAU‘ʷ0sm`(Q?%o?eJ@S#AhGvJfq_*Ymr.Uz$X'vONPcwX_> z .7CRX:kX# YEͧQ4,|3>{ b۾+#9?{,>ڛ~̅~-u_>T1: yck 5= $`rtW*#g  XGv~{FP#.&i0}gZ>Be-ջs2y@ Kq\M)4>(=%u>ʂjB!qW1_!Y+@Jͱ{j;tlIZ:@idNPS),Kͳp u~?!%&Ap[U;C[ϼІJ2rUT$t/OCoC!ې˘ujud\Vā=O#ިOv5z҄d yv* vu>G!y=)eU}E9:+XLGs\>,} w0Ec|2b:{'$ gA-XK>G#Mݶ:\/ev 0C1㉂(Oݧ< ny JyX-vW-f}*7v[#zUhWqEq4nM}eke6Dõ pVZ]TOz?DzַJ26;8tK|hCͣ +捡GPZAo.@N׺Tg!1SGew-7w{Uí$U0\A$A4 78 ?޸/>q^b\ Th*;D-y-,v0$'';1}nspS Ɔ]%`SxG;#aW_k; `|ԘNKhօd3\v;K;A(JՅ $_<9SM{ I=oor>8$s}O\sL7%;ῖr\6z{X%>@ ijwGavU\5Y>=J9du~":a4R,Soq8 Rmm01@.TO_ϱX)9d[~BP3🝨u{.vHjy,(<2'!8ECtjOCLg6%:'=g8+= >}?t bJn%C۴exGR# lsݹT&BbBw"Mo1 |pQ)q 99B }9#p~⎚!Ck+#l}fq̳[l~ Jz.ٯP ةu@l"k+T @PT=yB2ODB㻼OGOveY=|f>W0cby/_gA` }`~52zB׌ίq`?)RVQʰ;OOdLsv9 Ix!#w zsLdD|56_1go_pEW,>T eTm166D4~^_Ǖ"2eAM a_bRdӓHPn87vŔsj_ ={kK2Ng^PvϓP,U(RV(T{~HrZcP|QT[]JF`$!+UG_RJx_[!lr8bR2/EmKTꢤP+~«CШPDvz+B2+IWRu9?\ ormVN^Z`{gyj }q>nrB:"s4PKnCo%h( -J _B-g*QT|_dJ,)N aAD)`뽧{mjW#`~x"D:lP~W_rpGέ1Oz \5Ni8@JK-ԯk>5 I)4:c4'md,]3K|h~~*+n?l "xG$g Z4622'MS܍VޒSZ5e+O7@ȋ )ⷾpYpyI<6k^Fygq b6d͉4)7/ycCL%k\rvw]Q?n6HXW Ak4 AOPwxUވԸZgz|ԈkFK6ZRѫ-̺S @%dy*ꈭxEzcvI5BEK5r{.7^mTlXJ9q3.vjYQ{:*S1KTeDZ:[bM xfs!n36f *fZI3 0CbKLYXj[QC YBr[ZIENDB`weboob-1.2/icons/havedate.png000066400000000000000000000203521303450110500162250ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME8(b|tEXtCommentCreated with GIMPW IDATx{yuV}۷~E%1608&M$9I&˱O3s/zcs[UD&Ulbwu|eqrb-?>w\~=I`净rd iXmxGKj|ff:~|trbf'O<]ņZ|_[|ekkR˷Ԟg4J2jJzsj:x%)N`C@Q:Z퇟3[n( ABR{{kV0??L$24_m:>k߾x j puuGؒX{[?7>Yh]8ONFp7>w{cp<'^mc>1C ߂uM*Hyٶ%!Ssf3-Z5՘١AO_^" kh,w>M H-oh &z_(l@W1fb ̶Xl]HkS(r[ \7=LZz۵,dn6"H2K~>}wbOL{`3&"\.lkFSh>;k&S\kGk^dmHh։@+ZS'ۭ|u0 R;z`yۨ땄d d"# a.[A`w Y&ZwrB[K/uw/J140Gr6D"1#m;oT=mX#?$w<~TL f624'c3Uwrvkyp" DmEuɋ1Fbo7k6SYlߟ i/vHR*͵UaZl`A*MKH:w/2SB-Qrqld5Kn׻dX|:,[_sg.|\# ggzN>&I0Kf`ֶn?ս~iΖ!E3f#D˩7m2[ QضdBZF< <5;*ZĒet`(Fd.Y}Eovbb(J4b[[ۼRA5xx4ogr@ @0L8??1g6Z5X2mݻPJ㧗dH?ao旿JD! O*aM={ZOLGv.2LGOLM,eo'zLЪs=\ҍ50òl_1򧭃/|{u%Ǽc!g3ٖ% @T˙#wn}7hG[䲁 t{?¯#e@hΫ)[ =BȀ@ @(HS&^hWG`m%D{{/YVؒɉS쫬\-8.Yٚre*~OGVύnkxXD( vKm`wO.4Ŭ-`x{T<zF^/dz:%d$!#k6v#)$BXa9g mGi㦫&=vÙÙ5Um9sjKr>l9sZZZsV\6c ! h~̿#3LsB6- M)2l iҹH:#6kzz<[Up[GT|f+@5+Y7=jh6|; N559=R~P[bx fjL"-x1K$ j6,J0H@ Dq(ڙLrdY ''dZյ0D<%/Dq!FK‹ekpRiSG~\.@*g-LC3AȉNS4aȔK/=ӡ fX`e7ɶ}+;I`B$=ꡇ{WؙUn/TN~ahtVeB:<C_00 Lir3U?M²xЩY`H 93=a&!dTSb1huo %>{v-4"@PP,4Sɲaitdǽ矿ݯ՗n԰;['NoY\nLKIrxo}w(rT$ǭ Xxr̙\}.w1"!wS+BoeZc?;hӀH̿t Tѝg0O^̭ɽ-Wox*K5XgdiD@$$^2}בHJ0!Ԋ};/n͌T*73ΗfL pQZpI63lg63Bّa@I;l~ĶZKi㓃9mD%w^o'>$J^rł }Ru5 !4D̊#v)aN^êTF&[j {s)^p[kh<7=}&jSĔS*v/lY^ pM&)b53K!1g̑piڙD"SefED!@jhPsdtzd+l+hiz{nmAwfb`R\hT(_oeݢ̍x CxOM&ɀiOz"kmZ=Bi%5h0fvөB}kDHCW0ebP)KLP C_}c |V_ C&@73tw,3'wTkeLmЗ6NnOɿFkv grrĨUKL?aHT$>9;J}" u"ό`9llkc֊BϯZ-F i6Zr]04ܾW܆fֺި vZC J<* AV}U?zO73o uyevZS2"`>C !~+RZDRJn>3{'t6- Y|053B`|TZ Tٶcۛnx1fZ- +O4_&闟Z*3჉Zl9YE]ۙd n\eځaXRO.{؋S3#y%G"1w!.Wdʐf=W }5kGfTs D.:GAy]y ЎСDTcI#KeebzǸ @b)7b!qNB8M7y ?>sS ZOw/Z|!?8t@ߴՖlg]+|{ӝ྇qVW 0Lw+=_8wg5].{@ki,V,XN*R36MfS-LcSנB"Jbff4z iHg l<&t9 UǤ3tk1usns\S#RC|t:y}\6Dv+RTe}k ,ZB,/ JP,Y;fQG !ܙ;[{cxpcܥk+hF? }=>9k4#%kJ`}=c>|f5Ӛ3G_rRs? )3m8 !u~SU#b $j61L3nX{A"@'!BHiDox,Ua/ƩّhVpV1W _K~b'1Uj2&'DQ }'wܹVlix1BUO" gS>R!iՃME+aKǭ7F -]˷ܸ+ϖl3Ұ av=KW/lϖM|I0y Sy0ta2_讪0S-<4ʷTC8'F0 _.Ywmqf9GOxԏi00ڷJh>w+ef.WfZ+v]vnXw5طq5o{O]:ҢG$0 S:cۊzL2#q?zxNe:/(v>k%]V̧>쓂v,}(ݮsCvG~[<Ϻ03ivz*ǭuO00M+Ap if-YH0sOˀKD.'vu5at[kߘmG|h|s͋W泇bptx^Nڿ6Lrju3gTj-?^ h,x~RPW#4Z+N& ˑ3G}0a.3"DD\"rYk]FR҉ɮes j9|>:~y}zkw;˰c'_W]o!wN:+Cm Zkno$!<)7>53_.%r 9Oi/M'4"Vlv-#Zk﬿wx xp{*EuU"*= w{fo鎗''FMf.0,b,4*Bw`)cd=h# H+Ӧ!"_!BFq~ȼ٦/֫b*H5-C) wTF^xtãBȺau!dCِҨUJ8|lGS/%[HY˗n8iYG5k }~o~ܐYu k՞%k].s,*xybjt&(B"¶B\qbTZ9 |kxHĩ\}&KՈ(tݚU.&nT$Xzl;VӬ=3\R7\ #1?`Uxk°xS8c # n7$!VosbS[;l LZCSC`@IH88RT*7zǚA]p}V ]O0 />|; |e#KD熫Rގ"9p[a*;{E2ۡ=MQd#KNz{VM1C@UyA~q2dۺR\ty "a cfv$7[hK Xfgٶ\hцRC`5=6|nwoŦL&ֶ{@X0 VRk-Ru)y0[`]" nLM9_ KQF;̰aO <ЫJI!Cu۶khjYfMh#vvǯnz IҔf{#O Zd3mA3AMBFB@ @>S%繾;16O럺X q&*\ |W.f &"ELyɑ"  /tIo|%"E%.-la.f6ݨd&L`!\(¦'<ԩzfr}?q-ƿ/{?WY#ڒ lhaDԊQwF˱{sP%If000b̏)BP54L22GI[Rwr/tk~r-#@0ԩ{y{88_<éN;xA*%1}g|o*1@45 !3bO?})'wxx*[*]K4 )HT+hlmo.^o4^g8?{I=MB߈, H!`0+]f\cG?>.= nܼ[󃃻e|QVJ=i6_ԶBJ (֖X[[r(Io޺|B BgwvH z>cldr6E $M幰_|c9Rkَ79y( i0 "$1534յkݕY޼~ʭ^'~r?A shdXi!2\Ӷ)C80D1Glnm?h0=7 9pk`=hR/WzcAXE " `i0+!).˄aPSjt?W/Z?SʩTt! Vzc0a gƺfѸُw~)] {=ԆGv󁃧0\0D2M` @& `(+ZVr>s#\w ~>h\Dr3q<*j` MLf6YJ!E<:~Ò$X]I|`䡷U8*J!5:! "C "PA.+$vn-%_e>IpX7>T(z=:?U&$"! !Ykq˕ek*ǟSW{)ա}Oû@.3\"r K|b .2RWgWHV=}4؎-C_S{ZuvJ,˱aD@dgtnK!D|_{rzތ{:LccS5gA!"i[]]DQضBPWi0k/F؎V|& }sBOomF수rp @2C$v$Lai&~2Qϧwi|]SG>{d9#$9N'?7{~nzڪjfA]˲;CC35499u_[wF&'9:D`\VFhhhJ)fE&M$"NB3CYi؉d"fVZk]qu9mm.※-#VzחjTed89 s=ק/ɢ.fW)sk^ިxJߖc  shyiADasy0댌ё} QWpȱ-l4#K+7GruL'bDL4 ĦeK4~pf<=vgrϱ]SkKg^viu:*2J%QJYl"a`7 3%IksOssK㑐e&`W\ūڝ(JIg4$I vask$[JD0 L7+aYagYzb\ە 23G(,jiZkb|9t*n^KUJH÷n]^ܽϚ%$w(Heb U[uR8̰vmwk7V &wBbLXXO$xdhJ!8ʑ9' ɫT(1!A^Q9{ ']#!CDǶ2@m;i N.cK)v=3 % i$͍g\e&FB`f@FJjU=>fm9*"Fa*RE1Haf[H6q6n|qn=Q5g(KSO>~BZ>yRJ㮲2,U iN7Z_^q(R0MkwK) 6^J]S 0L{sk:}0{4L.[F ]+ 5:~H(LbID)uJHm7i[}"'`n!_ʎJ#qgwҞ=G[Bv,fae"[kv{ I w|eUr `.3SVR/u: $Kq2 4cࡻ8nؙ3ZZ')E g'$ٶKǎ?L.w4d&"`@٤XJ-x)Ժ([.l`3puZgzC{w̬߳3;uDpum"XJ){|=B V~emfJ|6w4lS5_vZ3s+DLJ0H@ p`=BLe0Yk{ey7:b'NSp [k"vY|܀`ʵ `6oکxB 4my{8,{^1;0eyRYKf&)d`D"zE7Qzu] (M/L&h]]T:uѬյQLtCb@4,&b & |s-_cW,T3R˜Z[}MCL)`9Nexիޜ?uhuX[]ȩ4) R v5I,R)L0TZ0or/cVFƶ$A,/-_Bl@@W.i,YE4 ءl۫4r}m0Lj4bG{^!cvVJi)33 0YBjhRBCAe~:0kiէa'تUۻNN6e Z.dfBJv30(;$٥!k$JOi!V+pw7 )jraн.LE`JAjNU"4sJT)hfY(m{v=Ff팍XZ,!uֳcF¬G߰pX;uy7-`2NBg']SZ13`?h"ȗz,7R$fCJȥř2idlmocc==q  mNH$ (V,fsCi]uDbEɅr°9saDb=kCSQ*)TͥZk gfEqr $EZh҆:z/l++23$z`aYvWq&/7_R:[!!#"- A!i6%JM;]S$VtZ:XiמP*5Z4:/;[8d[nz¶lɬ0Ö "mz=b ðUKI1յ~)Pnt TML&ﳀ~Q8l-IaK)S08NJH*űd:Rٷl4N?= Nrox@ ^=2e=gSDE˙3:s;~r 8q,JYK d8I G<9vcfʍrcgZ|U~ДQI_mn3t9f&bVLؙ`:|4ZFDڑ f-Օu93slVg`p|f w5IWJ[7/f?{?1%DF_ Qy\RXLOԲ$ M&&ɱ= RMox_0p95rh=-YQ 8"=QU[J괷8 m~2m=.qkJtV@PnkoYKnTk1w14IߓDe2CGJo!xGKsӎ'{nXQ`g{SX#+MZeWZ ͚e %uR>N(!jul)큾yZ1suqXki؛E"`ZC)Swg&`V̬ ҠViX[[(:xrY)wi/ܺyr6}Ay\8/έL?pvƛ;FyQpT18&@ raZ?k\egdY+"JVk5Wʃ44|O~(5E@Wi ]U$Nis{sf:c@k'^1s:2Cng\@Agf52)ISwpCDWf])eČ@{Gx?h0S.i9B&K7 Uymg GV`}yGGVVЬPJ޽3 !N>ħDžf-JffJ|Y4?:Ͷuo6mOfzDQYع?p=\YYtR ^~8x*Z\۷ A03Q; Hi/?9>WBwL(J4̕|4 3&uE'f4 D֜LWk AQY':&!!dP(Tշ?8^3ynTpF4gıBȐ"ml-[VFJ^,lfb :)|؈rټZ^#fNl ֽ|-Mb+-i(I8yD`ώ,eZ>Ӂu=1v[U1l CB(0+!eLBęLnW=e MfQ^ZQ43ͱq`Gs04y07:+T$`3`**վ|BDZsouw RG=}WȲCecxzuU(4bm9T*񳇳/5l]ږPK '_0"襌` ={tܿI|#5]n0#@%^_S5powZMs־'BVݚnq(e̻ ?t礳s ݨo]KK50R٧E935llmBa\Ysb2:;f9d+Nt1ُLr=@ ck_ZXI!RJi(PϞs'WM_ Yw{ԭji5H+ )4֭0(8E+B5+yNyM ͙on0~.|2x|g:.]F>ʀc_7ۿw"|h/,MW-[H.9e xgKswgC[cBF-Nǫ Io\>9o7;ms2%  !\/gIDAT3kTjnm,/.l71O^Y^U #-/SXut tZAxoOlpAXY۲RHAAæ}ӧOí;ջj4pKf.3Q^狫l7*JVZiŁwm',6Xx^aqjδ?Y-9Eo/EZt᳏9l`~Ъ}4)zhdٳg ~@H M߉/rH<X J - ukn#_":ʕ6Erf.}Ufw?s[.pmgbfG_Qe#wfs`Ɯ3ͣ,RkJ=ӟSJ"u[?<3M!8qX8u胗4/5KwDZvn|rtl[n~ W>q '7 g>@ {aK_A׎w?8}4})x? ΀~sx|!+VGD0w$n_g2%tύ9~O?Ux޸;['_Y^1.= Y^Q*mo.șf!!Bbu08H^}lؽN-?=xJ~np \ph#ils^Z78u!@0B0 ßzǡl.;unVN.fُ<'!uM*XnxZut^RZs,B"0;5]5; G, |y~noئ a聜aYmj+$)! -$JSD>AKJWa_w~~'գ; }O3'~p,D>gعBŵMB`I3TYqq\~?rz?qY2+J6d+ `DI\GxcS12 /[IENDB`weboob-1.2/icons/monboob.png000066400000000000000000000165321303450110500161040ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME jz|vtEXtCommentCreated with GIMPWIDATx{wutr3@ $B(Y첽P[vVmc-[B`lIH( #0&0f o^7}7fF0 U_N~ն#\i`W/wR (p4V^<m5cH  @Аf3ckOg {x ʱLtjoI3y0MQNzcÞ5"fBׯkw\|*@`I@k S)5kuf#ۻ:](TViHID5ZA+UsvvrٓĐDNjk\)`^}y@\#t|[t̤R R\/ޯ($lrM`'#2:wܒ^Q-;eym H)B^#Nx:™O:TG61n:uSc '0, ((Fq?v_Nk?覽ڛ b[e;U6{mAD$5ffǡHY陓'F&Nױ pw<_>|nA gdЖB8d$씶2cCOx^]s @Lcz~鋻_ v Wګl+R}\,l`H!dP;L0 I'I<3j?j `WF{ pnTB = &@ y0 lW[ZuR~Yz&yߘl8`Xak޾&U*$&A;  1.H( }ۆ4t[+fd?>wo4C f}uq`lRѵtMx$Aؾ1"V{8DHBRK̦R: pkBF;kk*- k ^@<*\6[\nG$nj|锍ΎJ(ӠyrRie'oSd8}_,w/>ߋ[w'rUUO2QN\<t`y`;&ʥ4)f?Z_o\Ižh8}f +@̨Qw%o'&߶Կl}\ȭ\IC@kW_Vi?;3u y%§~:ٵbdȧ`:vRRE8 'i} ̵[AIbn[]S FyD‹ρh6C|{/ᆽc[ߛ"x"q- "ȋxD(K`ǥBG0;7bw'5HP.-2 f$v>ێnX޹S)'>'>SǗoY=V,>6/C!D06"T"565;>L8;# AG' r{{سk"+/87$3ϝO~z Z_a:;r[qU* ͚@n2df#Ȳ0U/1uRG{giC)v(Aghr|Fr۱a.ٱ=|[6\?4.Q| [ɖqܣYT.Ue1} ly_ˮ7M䲅tZi$8?[sh,F F,Mv?{ q%SqO?{jv.[oYdPSɑf6d0L f>W4cN}dY%%cWVx%,bs*Jj?=yx989j?:V{VպO˺ԉH2 ' "HC:n-t|%St" 0SL#I/Βz])mê&&xG#Ë4t~%M*eaúΫҚVɱw !d+g)AdS d ZmlE`&&F sV-!~~`j)ܴB ϿxZ3f@[%jE 毘W)gpndpJ0H@ pK@m]\HVKL=y|>o\}_t1{Wؾh -5FFvjlJ[,9B:)54^6h$LL00 CJΦMfz>}$vlm ܴnAoO7AϤ/" h-@Hp֛eYbYYͦkٶlʝ7ο55I leetxV Bry}!^@6[Ls%&M ܹ]ZkwƐ"ɊC +%=iq(YM4ڏKCX*W4k )!''; RÛgT/2Wi13 .ڪL o 7[y,A*:fB׽53\̀Aq o DtEX((&ّ&k22/!lJ$BJlH 92|@ɴ"@0 \t;bN~N'09W*v7(p[nu#kJJeglROuZ|uMk̡f9_ͷ .37~]Wk,;_:߿o,ۑBӵ-J>mSC3luͦsD" S4񉳉j$)J'NrmܺP3s,ӬLXdcxx4`rt7zUЬPJ^f/0r{Ț%rMDBjq/"MDSl;Bu;J 5kN% b87\nehD=g{>5aY,i;߷c@@DAӫat2|XS#?Rw%`@Ogy[É|IiȖAlЅP fF,EqG=xehDHLeW6l;5R'Ǎ qkPȷ@Xk֦Rmu"PZX~u7=3 #;smaJZ! ~WvQvrwh13"R[^`q^m"VA|0 3ǀ0\)}S ?h0-o8{JA @Y;N2tdb !e!\X9R,tLQf|@Nvݚifض@!4G߸WNS#$fY9},qd`mGWJjMW?LLZ9ybǵo\NAcc +JkrqByebgfY 1yngcnR6rXwתiB152SJTvdlҲB+T:<53=N/Tȡ}au]3O}k]v)|C2T{[_̬B6-VzH\2Hd)?֮~|x'םК=c W)#ǀǚT|Co~mvkve=|,rubIDegZ):F,2O`#`wr|{֏][ nzY3!3 BI!\.RYr[ +I=]T*BHZ8&5kY8a#8鿈?'[[.aJNrcXű]f]Eq*?v_#ԟ~n7@E!J tl! & @1sJZi˟o6γw Woi2K6d*'$3UD[36}wK '`iuIENDB`weboob-1.2/icons/parceloob.png000066400000000000000000000134521303450110500164150ustar00rootroot00000000000000PNG  IHDR@@iqIDATx{{u=3]kM*crHbn"5I#V+mQ FmAa5HM mS)–:JdRdSDKrwg^pe;M/0;7swXz=H2\O7! D @ ðLiF8&5fsEǴ- g'W.zR<>'~U Nڹl;kO6焐6L`0( jW^z5<᣷;ҴDs,f2{Zy}[ȍIes&mNP2щDZaQ^1eYWV*V~s+53oGcX2zw[n*cYk$Ys hx5njPߖqd8Rl!31 @GJƫ⬌pg-/MWmyѻVЛ"LH!MtaZ$8 :Y\ڿO[ _7w;֡C67a&%bAVΙxn4j3/>k٨/oNُ${ƎD:+I\oS mHp:Y,@2WW.X}厔?cdϑ`iD;seZ6 9 D`` @2 B>AHCzSS!f1U=&C#;we4g~e  R!mӡ`( +%DZ\ןn豟 o;H`#~YLI8DD"e iY1={#̞ =.{J{R\jWio0ؿO`3k,L "HfT^9nZTC$n&衑~37482_? Hs\pȑ0A00,$3Lˈ, )'/u+k1{g3|:yɬmfDd$$Xk:Im9 Raˏs?Q+3`NQ.Ek׮ttiF*T`L ␣ҙL+eE4STTűCB0(FiY va`Sa. 0bW#7gcJbϯ_7RyGhw;~4s *Y.Ous{˽Z"pʞ/s  Js1?7mN $3bqU2F6,2HԬ !2" 0Y+Jw\48;uIIBz+y0[ @(gg'{VOa @bVLqE# C{!:|'5:|0R8D2CDaXU37gf/WzY߯[hv5kM@o̶J}H3z[쉃( 3$I)$grr屹k`vR0 *" z%YPi`t@DD!3Ud^Fo=&#Grlf8B[i_K_V|/hT+KKY!ߕi(`& @K)YӴoy߿.tuw:i>9;{xVѬDd->+:98{`,/gR:uu* )!4,q,sB8>vsi.]"r0H{噃 Scae0c"FRR3BCkD™IS܏C1d"7v 0 rj{Qt3fsj6Q*ŀLsЕ5%^z}#c{Hdb MdJe0Zk "j/|`ߌ ) %TJERЬ (ԅ=T) j%N_u0%[Pz7>v{ظЊD=}?.m+ &ثAJLRA_pyrB>"Da D\(B"1WN{'+a ֖ږŁݣ/}lrxp@it"Az"\{ (\YU\қ2`;N:T6gARۆgGff.9@CPx@$>7/}'/WEDb!jy3WH-=:#zە]tT*JFQDvʏ]ҵ`ia>$Tc&1fpCJt.7$7a5;RСwH9"IyxoZ/~z.OߑJCDv/DQ)9~Uih&!)n @+"!d81~˕3lQ+D6Szt@l#7dl;3HRH+s/}o, aZƱ~0͚ J<<)+?~_?Ǵ\gĻ(lA @Q0G^:3D00Uv/*VaHdSn@&sEs^o˳RɃM;|_m >za`W Sdq@'fD@`ff$0k.BҒRӄ?:{LtG0dfkڥ/ۻ\Thp%\ˀY0Bi| /wPr߇_cQ?'Ť7la#E`ADqJ&fMH$Tc]F2K}3]K?|zFŃWH@, t3&~c k+}`ۀ|#f=)$gfA$h-"^{nW`z QH&`#B| 'eܛK[Zgmq/8vضDZtkk3o8b&ML3r4)`Wis7柸&V|/ @dH  k=RqwKHi[ضQkx~APAKh*a9aq0 NTbqӤ0T3;y^$1t͒*tBLC@3C)Ҽ 6b` ) ^҄ b.ޤT6Mh7RE"  7ja KD\o d3N }&Wa7 ߜUpEyˆHi~/4 ]5C4&d͋XZj Զzh0BrV"$-v۱Ӕ  QpEc{!fqO^AnkDOKq@u4Il|}yṯ qݫX4g3:`LfZ `nHp\ H ?ߟ0a1;k?3ڃvm,3ka@~P7(rD9x?d&fM-n!4v`?w'f D fπw4Y5Kf[n,Hꜩ3P]پD$ѴPZ&Ds n 2!\ Ԍ!R/! &5- q|lu,W:R1ʂͬf{X'Ļ сM@x ,ɦ`$1 !<ԏGǺ\Ī q֬Ols\ QFJ!t)(z}bpM” `0ͼZJڑnUXx2Q6ݺ]:q6Xn.l[ 7V4$0T_qHZ żJԮsjXq?k׫ƮuuHZI@1[g (wbDڲ0 =xN#5kZj2_h dZv}V , л۔%`fŬr9Ylu}N45V#K(XX 3-VZP\:t}kTt⏶V56܉"o`2JiC0Xkk )@DA } @+p;YhCZojjH7-j26}1tc'Bͺc&`nfFT,uǔMfpk N&M6qS?osSYCLbV'JkycCZEg^gjcA9 zT ]V:\ǒ8VN#}eeQt.'۸8)Mm[?oN-\RjdŅL:+u$VVT27To?`cR)gl _{3`훖26SmΝ .|N̓ a9Mq7?{,N>x*NʯcN CB@i0֚ϽU˕o< a';PŎB&m}v)iJ%$t~wd%H1oȻ5/~X k-MM ao<ހ'|<Nm0:|p* ? -LY9,\i ;& .}3uTfOӶaC=Zl 8N4@!ܴlqLsxkkJ 3-bZܻ)J emq*;N:"ӧ;\?{4 E $n8?Oڊ#ksrM¬7pO+\vO}~C7fD/ Dրm-|6fh٠yȌߡ0߯Yt@` n|NP5p JҖm#1\yr_=|Mbs.l"ڞM\rhEfb+^<To M F!>޷~7D'N4S>PD BonZX6ͯ`'@4mԬ?"PBDѕ˯Ro|&x2D 3}Px=ʲwWM 5Y+JEQ˓<0jݟoF ' +g HqݔJ&s$&AP~덊nWTFuEY~GȤ]N*SplӱI ʄYGtn@;υo?}ZK3m MD2+$ f Vj4ɯz -IENDB`weboob-1.2/icons/pastoob.png000066400000000000000000000214751303450110500161220ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME 8ÈiTXtCommentCreated with GIMPd.e IDATx{ieUw2~߾{v2==[fVXeDFlAVC8d @ѾHjiF6]]{zٯ|o@Ȧl,+4JjVoTI=htq};d2^ɴ;-J}ٽn;F^kn4WbڎYmϮ)-ffUJqSmTi:lD?S/ xo_\X=..Vť㼰ʖmCp`,!~#Ne΅ _*N@x8 B|)J+XbT-A!vph,v6ǻ {vYˀb~pyD\,;q?BRHΆ3CΆ@pTɦ S8`]K@X#zcV(֊2/Եl˱Ml" D1BP6ΥRzz[E9[P'iLSFsnx;tҰ&Eo>Cf7SPSq8[ q3fYI$WPr(>SҶѿ=G o4243+-dK'X ^ZR^0h]D??q\\?unS0Ý'+,aO{r* HX B.Zd)tdS+ukReafLX^Qm))o[GoXivϽ'@H8549$kw7Yv{uYj4lB _nՆ~_~^|b0"b"0a6LB@K"MtH,o˰ۭ#=a&=~20ImH{]yoL&g&p5GGτ\uyj%A77X<5)$E @t8HN4}l3!1'v!,d< 3wN;%d 53+&7/W4^z]iU""P8܉kG4.131gS >w@ypމ^^0yT{wRAp+4L*|kuub>[s5RB !pPx:$u=>xӋ#g];yBEP5J23W°4K$$D& TM\{JC"`--3oahaa%?*BU"DYL@H M|և{tZx!AP=g@2n$-qvprKcqvOYtbB! !H>{"aҰ°( t\>vPH=|KBHnV= A_3D1CocjoS.0]-`ì>84MyZ)锈HD GTJ_.ׄ`FBbccbAz𱕍r iq4f?p O@k}|~Lo{9\ވ=zT*\7y|7LjPJ'h) roהV6Jt<`&֚gB ¢[.#^w:盥c"q4 ৗ])Kc ȐhMOO :wdে?|Qb_|'VJ8^rJiq~˾;qdqgIiHdh"6ffyq!,]'VO=W\WS|WXyaP 0,S^Y^A ;/=Ը~Be'nCA1TfO$=|cj7jNڸ;}rz#c_7$ 뛗t{YHZAȰ Wjs8/D VGpЭlmI0(/>Tynu/c'x<'2~:'m<'5vǨ5!^~E4y) * Z!dyBcbwhߑBi5|_N+XB*N`TVr aDDexٳFS瓿t˲sf:9I93l9[WJeǖei1Fxk_S۶ bfc9$Ae(B>?'oK-[#eauzR3&"۹^Iҩa^[3C#$K+-m9,igաߓ BGh'&Ib<ȃ q2r Fk^Z͵a;A(giTM*zQusKBy{W`Vkn~W*gFlc.VW甶K*IWx0}YOZ܄(aU~7ȈG]w~29P`׹\%\[G93+2oĞWTsҙagn#GϨh:0C~,-{8ZJbgzXl8𕔔wC͹3R9tv`y'/=Tq]w;)%v30L0].7G%I[;W #KpbwoY+y vܑݙS̒42VYg_k Qs4 8iy:Ri(93r![]0[_G"3[d͹oJ 32 뗜!J.*8q o-H@6AUPkݫFlm?QZ+f6@m#gֈD cxl[5K}!d)o aZi{{nX)%8qh(*34j!tvpC,O7ع~ @|`)--T $A3v]?Y^<9aL"AZ]7-]Z]e1 t4nhVJn21s0Oy74z[yka5wi4ð!C53m;9׎d<4=`c4CWZqw_yfQf˲%`l44zm=i4,r8\seKJ+: JrF9\ ieʘ|$ߢEfH7֟N3'n/17!AQDȦӉy/V6f+gnn+9u(ffv?-7$iD*5ᒈnKQoE#GJ;+R8CcLs8|!dVmKȰ`~Ǚo G@21{{q\U ŲY\<&x"Ay^~{6|3̝|a6TNGl[NzTjVXٚ4}-AtBP6 ӽuݨ0cT*Tv+}Мg&3/?{Bn^( -{ygos4'.?mn\!fs00hm"}+no]ʇluQq<4Yb٫l AydYn)ƂnY?`D@w 3'ކ{t{nЛoq 1l2@eo6S`Akz0?{-YH9ub&I@ZVg~^Qd9>#Syș)cT͑z,O*Q4wVVWn$d&QN;usr-WYj|ku\nLؘ ę!}ҫ CIPv`kjvlV"&nM 3l6JU%ΓqRjѤ:M Ӆ!Pd&(#BٽVtJĶArY_9ep +-;{#`K˧SDRJ^MfF服N֊,/"˺>9NSgi1ʍl7Y5Wv{Qo^#z3)v\Z7[-w4SHib|53<22&hEbA 8WV/ɵ{mWkׂp%hM Z+9C8Cp? ڭd:>q;*؜^X8^TJi@eyLف[s-+$6/{ xo960QQdt}҆Q^+( D9{v/[ܾ#~aq !HC@$z:cm #:>""Ɔ}' 'C^s[3 (S͹"wʄVE!"@H^C췮*+X>JO(!:G>eg{D (`@(0<7~'?#haoz[6z]a("`0G>Olx}Xyca8r(րRBqRxbU޺akY!ql?N>>}wV*Gn)f*%QD1Db"p  A%@{wbb?nHZ /R%f"!B/v`V9] @`6J$(z2^6 `Bo$ =&y,D",lL7HW@?u{}6V9xP1 JĽn왇ll^[ΕE0"z@Ji4(\U'ܟ;m$I"= }ݷ+(D$H!ݵl-D>»)Rmۍϯ- xZGW`'#f}~1.WF DI[?ژ'8KVGQb;NwLZGt;_{|1usvv~7F[ K%i:G4-@]镔PHVQ_~zw V" j>%O/EǖH&x<<;32^ݍ+?xx UQ!RwEۭݍy}9BwtkfzW*U10{@mo]ϽL o^Y~@qHNB]4*}ǣKt t%#Ǥ3 vyqìKDAO>66]PXԠKlpnKZ:1@a"G?ޜ̥jaou&!@8Dzԧum0p#ع> 5"_8č!"4 %JU^1ԥFm||.m =fI*}޹49ϓ։J! Q8118q4DB^ Xߨ-dE[$D*>-w!\$ԁ[-/޼4eIu%3Yxj58yi1D"iX~aZRĚE*: D,<7FiA O߮-->̥(*qR"R"AdHB H`rr_:;spó P/>_՞'( T(DavK'\ :7kE$Ȁ( >ycޗޘ(AZ i&QAD3c/FIu\oLD3BLB1N?}1犊R&]FFF ^')A B p) " Nb:%7WMÉ 3ݞKa,gˍI, 1&b! I$ L!YҚ D4%*?Ԧʍ8QE$4իKYfo/;z2%(P  ha葑l>ѱ݊Rz"u)"M&7LE Q_H21sHB8Ś(ҠDLpXjL;`mb{)hbbaB!+I(5fE1B¥K=$K|Ӎw JDVZy˕z6;wGf@UVk+o˶"+v8́/I 20Y $^4Aj|a;;7}_RA 4էJJMWUG /cλ[/{K.>=s*cw-wW.A@=oIi$'|@(Byu @:1@16,b*4*^0# Es[$HM'2 4ʍ@ $ CWYW*zoH X$YZzf/|R~ch׾㼀Dq+EUp"R;+p T6g-BT…WMm-" :Ewg֚@Q(^-PmQTY@k w:no\Qڷp:$"pwg_vcksٓep엦nKD${Al _eᅯ|CsXh0V/ܢu];3K!xhq| gtIHÄyBD+fAJ#\lL*."6WE07s ѹY+ ĩ[d]__z3 /`uby~Ўg0Zc#|(kS}b42"mLԽ1"DA%1ZkSnge]LJ+F $b"X[hب"LU\%#QGt(hLv0&<ė.<5Ɖwo6p;J)D67'JÓxxjȞhoO#.Ac1"+EᠽѐC@ !}k gC@ c b txqSAԫ,!"8KU\lδ֊-fmJkœܱYRŊ?vsI4N̄/3-O4TD*^= <Jk䫗3F~ @ D 8"A60"f55{ TFՑ1@+(6M@bFG'SRhfkKUE(JG`lDڂYՈA)p0t} <: ;*t϶J> U{#%" 7*F iMV:EuR"ª݊2l{sɥ F ra" D퓐?U.h} "ujcyr -ry‘E&w1H)(@P1vSa&$VJH!c r"{),(B<{4V~OY(JV:a DW* {~wG&0-*TccsSe9vǖb@N%;ؘݸ\s<+]jn v2ڇxxdE"Rٙ@A), {+Ⱥ6Ko3wFWfbHk٪Jڮ@auvs 74!1vךڢTkk8f^+gqM =*g"u2<#i|=~)Q{YR60&LvڹɢD@r"HA>tZ Tm?C2]y_뉉US':Ъ{Z9hO"D"g5f0R4JX2ڃ x|l=+[!b$^{nZk`v(D%%> jmgCv^50 ZGۮ.>s۬6\! ^aMO\:rF}-ֆ,]>3Vybz]٠}7 {c [GB{6;~ E;?6n_T}"+v pmsgrˠbg!6UuMATHo |jjon,#X`1Z ]߷|~ݩ"*ݩۍd7V 2 |LA7ۗ0:?7@o[ |0 (T`94zEsvC^vPy[+1HPG 67s ~#޺%]jܠ稗N36;k}GR& (HF|1D$J8x/ a#=Լ'?wٱݽuWNܨX 2Q vbjc}"p >|땴UEg b, r r(76-ーQO[OhwGo`/1#,QY#%ioB@>o@aҤLضvA7)>x!:g?3ۓ뵟~owf[77>_8=R~ҟ>eŀ7^* {mlM*jHP5ADpRT\v5SutP~?qϞڨ~ 7'I c{j[?:^Xg.z_+{-k!Y *y" d}.">[<{^[{jdj%=vS/+erk_po]Q fiJײ3fs:"MR"Z3W㕕 Ng;f0o|AҬlg:<|p +WҳmgCeh "L}r yƮVI%7# b2CDPTQC9p5%d^liA+λcYMTo_zυ9+G"N yU KHHqԂݾ(ctRRRҺ:IDATF@Jde4_x/nۯh}Kb"Nfuζs]UXS+sMfN~ίn'+d#ťx_^7A 9<߸RK6_ m]6ԽW䴐3`k(sG.'ȳ< d{e]4CŰwfnty:|+g+dx F};\81>gj[o=]_TBxr7y<=Wu<K42 B&`8vs'`* `9W *>S_=OT[pELrGx/(]}PWWcZţnWtt)BB$TɥRkɠ`/w = tt{<㻝#ޡ^vS^ pBLCh|}ͽ>x 3yӖ:|Go~KO0B#"& ֆ؛b}iwV'Jv8,tSf0MU꽠'}{FOYd{W"[G ;)^kg1Zy_- Gv"`H6tgsy9.gA;(`?|V;4$@ 21_5|/{X?m׏ I`{/_MjnT @iBRsT[I^這tT8 W.v+C|HIENDB`weboob-1.2/icons/qboobmsg.png000066400000000000000000000157251303450110500162650ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME 4Q\tEXtCommentCreated with GIMPW0IDATx{iYymw}7־F#i4Ƙ866*IUTv\l&`04hZHzΛ-1 㜪ޯ}޵3-TBJ2bfNbӦ]n[i nήYEQNǦ&mVt鉙kUvflKkJ v+ \UA̾Rmɿt2n?doܛ;r6c߾ X"C (g.Ja +ЩsgO?;rޗh`>gk]"Qhs~?m *@ Y0g l=J\ÿk`?/C=_7lT @rl@ӂtbk! 1F*֊;"QO~x Q.\?ƎB.cŀW熁ٷ}`wïE 6Z[/ f -L:ұeP"yGb;W;x6#ўA_oBtR{cVл34IeseN6ƳBۦMTww?(O~3\"3 l##;V[{Oɐ" k?= rDQ YM4y ˶bf !%xaϵKj*_qx#3ognPWr33I\nLc\=$|]i!jՍn=8(D _F\'uc9j;L./#q-g k% jI!%Ib6 MVxR 5έ׮2|SohcL DHHnԫiIjk-J !hqq. ,[;NL1 B}V{m7=B 6Fndtë ҆it0 +H3vS 7*  Ӳ#Jœbq%[,$݌ Po$V 7BOhp@f"ּd3@ /9tʨ[7Rb듟 Ǝrc)nR꼥3 2kh$1'PEFE` %#|eZKSkSe`fBW%YkCX@ߞh N,&BHcOıA`4ko C-`GWlkM,3G8t}cŵK5{#S3ѨZb/f&P&Z!"HPʺA,0,XuTsR;y*g"^&dJ޺*+=074bfu鎎j6v>|]A|ш`9z^[4CZo| &- oX˵JK[DCӑ-3^-;Njn2Մ{2c^PPix5YkNgSma+"j3uգ׷|7F*( B%H굲]U~֝{l\r+9-r 9h0sqhoZXߞީt{ ̆mVg̍J%ў#D" U]n׉X{[_9p F`WNv6ܪmw@3s$NLѬT"Wx[羾QZ{4+R\>SE!_Wc>ЬVL o>|-' àgZ0Ms2Jd G $^ h20P'R۶M(df'᝻LH^ ؿ=fVD~KSi0B0/v>oaP>YADB5G=g Q ׬Cfd}YITAD(bܹDhǝ)]P-YZ^!i*rDqWЧg)B\7ԋ/wxb2(滯uoZ!dp"ӊU-utx<+<<a޾;1]bgfiZ[,vNwb>/!D ǟ~#صwd[+2u+R !53kӴtit|( *z*Rʠި$ggGߐ4DvfKBHC"/LZB\J1ʹlPwm`.T2&'GdgGo#LzϡGNOOMO0Ɍ[l^hk)ÐC>wrbT]M+lq}kZXJaD ֊0ikJHA,fIP`VBʀg** =1u5־i+WmF6x#Cކ (`!fF,྾-^olI$jnC`0O~b͏~ɯYUKq9ffְ Whi4o?f"@D, "Z:F~E!dYFbZ&׵m b1v#%0*Յ!Դ(, h"2@P;xk?(l"<IgS]IV*"f0Ys"CEQ ]f͹ (J텳 $EhN{땾׵V.3|7"+Z%Vb\KUDPHDBi\K}Gs|4B;fZ l(?{4O/zj\Xz׾}oܧ =O)\| gg& ݎ8@%ðv;:&?ޓg|߷@ Y;N X4:n&rAef@CJM͌}w̎ZEyf lWB^]PZ+Bȑ dMvx1ko "M|mۉm}3/eM  |c_00ӼٹL%ˆ,P.ʡS,=ֺȕ~ms_эhZgty AD34@Zkm}S3ÃɘS,t/f3,:O} d怈|{߆ۺ: `-]w<~Zz^'E<<iڬY oX[7bpIȓ,ydk'nSUXEA Z%Q);z:,Wҿ0t|pKl2a5 !|"F'6//▮뎓6xpL>}-˶ܪ},}+QWV5w<"QJG~[N=G_ֿ{nL^*VR)66p$JèC.<Jg}6O׈(U.ϥ\n - ޱK4k +{r~.o~To 4)"@ ӊ7hԗ~V՟9a\neǍb+К*M,4{}/]|b\JDh4UFHX<5Se_4 {\f&_C_ͥ~%_zbⲧ"/LJ+␉"Z@ 6x@ZZ_1M٫;V|&m߿m^|ӑ,Oೆķz{nް}\[[ -X0@YjV  !"!d /OOfgK?qml:UNp T|K>vy&JƬ8uwnn$YBBr֬f(fgy4nvCS>MKvKۑ2X`;KD":vB ah!Uopíj^Ua55.~~_.{{H^;lӱ&@R0+h=VϿn%_>J6Kx"#$3Ua|׽G'SIENDB`weboob-1.2/icons/qcineoob.png000066400000000000000000000215351303450110500162470ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME 'վ5tEXtCommentCreated with GIMPW IDATx{idGuw#⽗/ʬk[2ˀ}8Y018ǃ=eg=q(JJ97{x?s Uh6c\ 0K BJD(O@~quVrr 3w'pixC^=4G>,// Ln^g 7?=Jfѿ1BD!bZ owVz񍿫?ye\K]v>`޽2::J >Ϟ-tMx_45@+ NEB6|X@>W`-/ޚ~0xM}c]/sCC BHCB "E$p'_:GwȮ(<=|3C=Hmo~[ -PvLg o`t!R8a!πפy^SOpv17D;.vpsoaרI$P)2ZCk<e'=\/W\bрԧjZE/zn\uU;k_m\*GCc=2?q? " <\bsBVfkk-dA(VD {yv,Vهjhu}eQ@by~w~q%/{,;q]w/S{;̯;sx6JHV@"B,$^TFP@ؘ~G%DpWZ yk6G0ǢHBֈ~2{4O+ O>~ O6IW<|G&+|fh||vI *"C?| ݸ_5oz~mUZԑ"@iQXR€2X1PV DAM}]ҍ0>R=`c91#$"**xi\#ڛk>7ktzo믗~}ȭJpm_!ݧ76 ڋr_O^8$8|x _xTm_5=r/ԲÅ]wh.}{A`iM\!`AaH;~N`\qk*+E6&ZPk s  BBzW'7CqU.4 PcwL!@E{ZܳҎa,1lUڜz|@O@=[nDn308؄y݈37܈x#z_ѹFR\'?Is(o}] ?0ܳJg /W44C)ْ1\. ZE,R dzssu0;{y[) K@D@>b'w?+WUa9'HD N>n?Mo _}@gT`V/Oc yI: nUThED(PQg轺R++CZ D[^]>֚%xEiSK X@RRti93tLZ[[ "Y8>OZg}ZYЎm0I{(xbj>?pdxdgjhof5CFs %~yՄ%NcBAjDQ00d<t:OGǟLR P^AӿO:`4ǖ1j"N6+ʵ)F [BUj\̌Hotj'rgK۷_5x+"q׮>mcNi(J^|qycDd*|ܾiOjx_,-jM[R-JU1\VD)G'JZ)}5+m/s s'H)P|Hѻ]S$ D鰐 rp6`o WQ`(X,1fh}m~yyrը+D N217{b@)M^:&7ۧewH-OSͨ DК)󀄍qLO ((PoLQT,VѤϞ=lR`.ZBf_8{ƩGAV4J<_^@ hf؜Da D H1 R>ēoPj#TX 4,-W*VmBH$x#D=/} n3_֡+ .׼~ILP-fa'Ŋb\ 3BBۮ:CC-"lVm+.6 n6\ŵTd&@w[U.@P[Dy"A7ƱjL6QdRPb1m}RJ A>TUT"=Ê͝%| K.: T3x\ " %a  \>Cd$RaD*CR (y+mRY"HD3X,jpBNN]yAP*.{@t %"d43҂, Z} ŊD+J~;EQk/$DNg]6V4X MMJ??.T{Z*)i 0H;1 UF \F蔢P@60"$D$Q hcdpxW.:H>'Gn׊A$6ԇ1' uu/¢y@T[%JT! ErH)!"@MڍO]a3FF:{^¨ A*)#el F_W@1HA )l0,,Les#g:Wq"!DBRu4=@ }jquIgllg3!(Vz*l{t])Bc@)V yn+BSA*9 %`01e(TSXo)V"i !$ϲ<k^cg ΉhM^_9}?q{JS5f =M;yCH. |3J|4׈rH<0٠ t='F!"$\cY)a'žʕDҌ 'hR.K{&/KȘ1;4֖CpKH3R>O;FC>j#*rSfoF9LO[l|R0,Zi caiܶ.D9Y!cե2驃}Xnn/U{~'M0_/[8[h+@Oѥ&e8*90y 1]m@ZA:ϽTUڦX pdR4&#R HB&=pjuZXWBmr3gN_Z 4d7&`@Hs>U&Hmcŝp p"j6 VgiHً!TZmnt;#9\\/B822.>`muxe;v\ttzs 8=7{﷕j+-{\BR,@JH){"؄^Њ]|] ,DR2aⳔ@Vtzk7 IQmPyC$m*V,wx׮KgWVfY"+KUm0;eiHO<'¨AB$I&wÕ|$jwuwL *f ` 43GRb8X+_\` D}v9=q8+0:}21OŽErc^)  K #7@N[b w_qhQFDwgi2 ML;zšW>R DgV{蝃Yh1ӽl5LmD$Q,JE=De 4SIႨgBW¨.Dɑۧde|a'I+"O:$U2j-}3JNRXa 䧧+c\f,BUv_=W΅APΐTiJϪuri>*egHyV%!u OlGD9ZꇃrMD2!]Ev~69w5K%K驋D*`L)ȖKj}Q [7"`/ K2w7Isdqzr|_{g^ kC¬0 NYU FىszIΏ$ouB#a3 }q>s zd:u޳X3gouVqX{*ܓy!%^H@"$h0 A P/O Y.{5bnayGxK3(XxF\`t֊Lj D%oxūۓa(#{4LaM*W_wԺaF1\{rqo5tec=]gi#bVbnB>n(y+C+!*C('E9i9:›J$&ok./T/&Q\T(&$tf#48V,@ "^mWTضd&τ=5Ɨwuw?w]Z1ZDrWٱ9;G}fq[erkwHPV7[J߈:)E ` dQ$[A M^VkNu/~SZݾS/zӃ1 "fba AWC|LXg-R "My_8mVY7nKkʹEW hKuc7|K>vKw<`K 5"Y(M!.0 R+EDN魊4N'__^>[b.VgׯHroRLU PInk̡8 2>$"pދsN[2!|Nڨ/ nR]=qաOxY{+\Ӡ-{r}|>@ nhh2}V +sqi A6W'WrLD؋xӜXv20$=|0u"n%ȋ%=+897\)[rZou`b*̟*6&Ѥr딳6Qy)]"&*׶-Zx1==<[β "%A4^Pl8gMeφl '*[;?=4o'geokzV.{$2=Zi9β4"@ 3\{XV:ޔ<9sc\>PA +"k>z&&ھԄ'(bf/YB>jZ,`Ql#usE|1gb}F.̤-S7e8aNISa\X*T6==ǖ/H"Y*W|NC'DVUm443*:RJ_2Xf$vv4uPx;j>l@,!!BV8 |vo, #lLQŁUYd5_TdDg@OIDAT&~3_{;u5p%F.xr8}YJQ(QQH(@PZSsڏ t'PQyK:R㉛>?5̣ȷzsZ +uԴ1EVڴJmqD""*r*5ԕVPj፠HXH9WYN+:/|xX7@@@paԿ^߰ ~7+WǸ&ҀޕF=syҁz=+PH^T^@yQ5!BӀt3vCU]U]t3Q}{W~{᷹fBsZ &i <=\[n!IJJB$k@ۚ5qdaL$@T/礛Bˤ ^|q$[k¹45#11Z^h٤5~s{O1\i1R&+3rBIV-U?^x%VZ`+?)kXS0K5::$CE)8W~4B!-}eT\vZeH%[\z߰Gnʇm nXXO mAZwBb u%!`cQ1Oq5׻B1K5[IO;RƎ? A2\ccC\\-m9άkw(O_}5  um??%}v^x p=W^I˗/?~k;[> Uy(/ S2ߜz#F !VD̑֜T #e/#X_wuBSW*ۻdcuSo }{M|\s Errd2מ>?~ߡKxwGwADcǎo !;r^?̡{i/';߱2vrĩdK!ERIH)I9q|ud#Njո A$S֟MLwxI:oǽOvZMgS8} fzdGGod"Ѽ֭[ǩT ]YA.nɶ^c)}.@$Y/ac2O 8J0;qy:ZٷIrM5'[_VnB bXmBkXPhK=_tnсX'lv+" eŞ={<-X?=pl}&$]͌ybVgYL a2rj잀51B2Cç n~u, ?s7(o|nt<^/8_?o!8&.n+{JiME]cGVViI&Y$daLq`G ICD1*#͹G,:!O_2_ve~zY>$$ =cFC~{p۷^Xr0SvPks. 2*s sp:ffSC0FBsf@- VD$C*+_oB.3j3 mܸڗuDD̥.MD > #.+b =lDelhןlzѠFԷǝ5gafq-h+XZ2+J1 ^Ӎ >yAIZpT3g@i_dsNa5y 111"µ^˗]v{%ki"2zl5j-6 Jٞ R9@=&S 2לoփ1)̎0 )%D m ?. ۘFew|4W*[c 4DZR Bxߌk=H]]=O?˫?>Z'si3p}ڛ.4kk9ط|>:b`u92vjBJK,bHHzƥN]%]5R8oYw #MvڅBm݊} 1;v}/4Z mI5֘k>(?O8L$FMvjIYA&kmp@ ȓ e,lT%B6?k]$H9JI rj^>rT%N>v~xu1ߓo}^r%}wro>|x~GDCC:\f}?,6NjZ|#A56瓒*dQKgC>I1 T4Yl|aHk]$q1 HR*[WH `u!HtIy)X(F']/y:וGFF100Xj<{NSN9/cP.188Ȟ$ gU?F%)35 QiiZT !hxCV {xMC+X+) buhX+A `؅r8W_=/ )ș˕3ETWq)G H*өذ wuFGGozӛgq'|g2,[ٺZ}).w}7RXv-rsL2Gu^*QM1|fxVH:#~nz;sM\l]ၝt"b@ r5RX+$b@k7QRi\ r@1ki%tȍ?41)Vk/| }7om`S $YhooDž^"_<_fXSiNZB3<)W;x|j:`VYazEO a A,A ;#Ib W"vk傺\=rzzaPh.O rPQo[ QAJ8qw-@6WUrr!>o&lq;&|;Q(ԜB[IJX\Ɓtl1 [ L$ m feIj0/J |rEa5Zb[IJ)`FFsҤߵUI GO1`fbT*EuuuJ%"!ϟgPP5f p~nj$UWȜgAJ ( /&H9Z3#*0Z %l]!KI-o`@Zk |Z2@/NZ;dBo)I$]Hlj<ӉtluN;{R)87Wa53l0IK @QU0ʊLN@R,* Ib_J&"A²҄/=Oy%)6},[ct]H\Cz4(g((g>+ς ?X8!&?]ck `k|Z8~YDU|X0:I3V) ͒;gff jk pCHRdh jV>!ui*|!Ak %xfL(He60HpI03Ri-;)7`b P#W[)ļyL XBXro۶ qTuqʦSX7;c#*5J&$֚[M=̞GSE[!$S]Sc; 3=!D"[_cP$,X b!]A!hE1dk RwṵPKsU[q=wb29Av]^w@4;m4ՙ4f30#;W f_'  cz4xbEe !@İD3#EH і5B[`,T ?c$DgC f,$t5q,ZxY45 ݓw韼pr`[*00Zr`fF=đoR`vyn҇ !đF,T\<Țd덫BY\sI. @P.9Ddc spӇ7Ղs9ϟ^!"V|d4_Rf[z#%eo*-ɡfU|q)fYҤH7$>BRb`X,lB Ͱ8>:ek [$E TxU l^}ɨ1tceKADmK{{m6ʱ? `X+ >GgB9Z J[maEZ[(dl~v4IΑû;NTW׊iVӓ3>9;3B[.~|yM7bc׼/fZZzzz>>@ .8i36vnTp,(My&He0(APvTfR Q7]c JSA j3'&Ug}"Klmvv8Z5%OI?>4tIn2fR5+O{n<-D>~?r###x[зo]EKsrIYݪpK>u#y9,ARıJɨMYŀfB:cEoV j@lP+;PpelwbPJiVwKIimm;+cTNܹI B\(4o\ήB}PƳ|a?^`߹zTu:uS/_w@vJ ]GE+2ePn c=7"oiIdX(ɱZyb0k5RuS{ H9!*dg6F*7K*5}Jy6 1{kh0'&bûW$́ Rt\7lm]ҷ|ŦBa-j>sÏv-[n֭[@45?itrw.}%GPj$@̰ $R`cj2Bđ9xCmr2L+;7!04Yρ91p aɃ㍋l,9^dPiTgrm 'R˗moja9 &<>5>Hq8%3j]^cTd'@b ;ʺZTos鑡$\RFG]ke &"9?] shXHR*M c5 롲{[:O{-`3Ա~lOR)ё#,1," b0s\ T2ZKOMd17QY`-؀HH#][핧?3s#4~ϛN_wQ6EJ"*k{_OIJK(x9̂0_ېRZD  #""סgn$f77|r`hinlC^Z:Į*)kÁt9K:-,IUKdRuP- a@BCy5.9Rfx{}GVdjaV0+,8[<67Oj3a[S8"fO Dܦ мK a h 1'D  Aᡃ}{m&0czŪ mstYl,zHN]u #fl`]uE˒1%`C iUk5uR-ٺ նYz֠qޫa\0ZO_?xTxtP{ԉ\wt̷$/h9+l: oيB0' @!GmMx"-`;а)4@g hl`f X*8DʌdUcWV b%2e 1րm_f'3osíS2k]7~w7-{hOy>II:NRu!ȡgcL,$@2P5KYB Tf禒?o8+T*]wEo8;_yU F5-21&zh2kC[ 8"p dDlل͏= nx]íM3IyOt&B.?rpq .6yY&FDs۝)ֶ޸Xl =O XLOѱܤo)1۲6u9)1b&Љ"V`a,ŶQ33F¥l2ڗs[ϱO,@|+0M5/9gwĒʾtųq˚m >Z< }~~x0qD Gm{]~I v]T8#:fCE򈄲 (URF˪N7 F#$%VI{5ٱ{*(, IgF i!xffuE#3sg|suk)†"rT^gt(+TAۑ&z~|W5SCW*ݾ\8pK&S賫DY%6V-B(n(́~uˁMn>3/d!4vC 5䧿yn%wR4 u hY$JLA a<iRh A2Ԥo;.ZwiylѧqS{ rd9Msbܗ*|*I_.^A^-Z\^:Vxd]9(K$TpDFEqXmI Y6lC!V>뭧ܽIM9;`pp:!Oĉf Vc:#؄UMքMY|r׫KH@ H$%'(0Q"MD`X2?<vm M85j/t9m` xfzܟ>{* @}>WS Nۅ|\.w6L`0$gOϟ}$Aa8sp5d4-~X}QٞX[:7m;vٜN4L|n4UB+7k3xࢻfX]FZ,q-C3kNTE~ ˓"ӏ%o5wݘvto8R|!31 @'J9377-(.}Du~!PZ >`/>tQɐae0-[ `V%EBVʫTʍ}o`ok7,t.k۶|M df P JQs1SDbO= ;?_?fyI,D]Cm9 ^0c&O,$7W~đIm2V^_-w|chM@ D69X XD0 (Sg/#<kh&Wyni$!0BHtTarݛwG&?}75;o) \H]C֏`ڇ@ą|1z}c$ Б1p HctG0l?u}H$B&$-'@3@; }Q:muv@5w' m}cpg󖫕 MܹxӋ/yN0ܲ؞^~=O=GCM+]}knؚ%)lLǘ_c"05mWgaYhv5kM@mQuFJbjuq7_qmFoH21Mr:ryb̮R*'qJ%Tb1AB|0]'Qf۔O "ݰsי@1g BxJ7aWt^v}̤h)e2_6M j9h[?V*v:m6aFZv>xf[N=882w^Qg_AԡomcK-qrMrndwvMY;D֕D˵(38q88iSIN^RzlZ34:x�)-|jGavh!/, wIe_="Qjڏ#"rGzY~LN ֒DRϊ(xzz!_ D} uvJZ^89ë7^{τpSL޻ǫO iܺ=ӸIm뤱(rxEh⯞vetȺkrzw->Ȥsz:iɍۥSJMN6ϭ[wR .ǶaX=5uf #k\u՞X!5A /gckln۵!+m}{枝DT=Ց/)VaHdSj@.[ u^oL*Sqׯ6ёghn NH=uh䑧J~]Lx B /l Oqlw@$RYZQJ݃lC:#a2591yL&orl܉?zӵӇUDDD^R pfD C6[E^1P89\%BwDi8,W 858 A 5!P;tf8t'' Iz5p!INe{{(N(hɉ_yd;2†A 5΄82c!_+!(v۟IA$h-_]skZիalJO&{zW9 ";>rp +PʛXc#e TcMGy s ױVlM {N4wo LDD̚4Avw#=ɤ B:! idyOi!Ad3fr1u}IA"f8@3̄qv&Ď/ظe{=6S V}7F|X~6woziEX02M""$ ۶Ů~.SJs}4PFgμGr j wHPK0XNtڸy[nb "6`Hh)8W2$aۚ4Eɹ*A$1ioBڥR Cfx^`;nbnUwJy2]κ [o}%6 *M$ ІM0R}uSUZ2/_3Az:5P7- C+6Aӝ0,VtV]sרZkoBڵw,|]C2,"&8~ $mDO!<88YO ߅ԖO@YDO.rp"nj'3 ^nDLZ;⇗؎L`av8)ŀ`f 47;--[`,]Hrrznǿ"-Ҥad7q߷%Mlle~=Ww:3`"P(MS}6L0L+B(54dZcb AhViZbp)&}`zsn9-eO.{H[oϸ *fqGFAD뤒v*mNUUeAD Hihj&"!bfaZEA+P?oSjpHJ&w@fB[{;Eds89`.Րmˍ ]mMm b[)%gH`Tjw@\ ="-h^Rh⻈&RW=`j3gfV+— R 1T6k3k-:aB$. jq,(0B7}@?wWi Q Co.o3ռF@i=7)u `-27X"aB(6ҙl#QÌ/a=hϯ %8ܖ U@cM̚ Z}d ("kAVJj.l!]d^7Le',IY5(Ri8 UnӀ( HkZx#il:53='Bdh!)b&&b7[cDJ!> ?gV2k7AFqe?m<~MO "Z˙^ȥ'3):uojn UzvV x``MfO?,%|!Zf/*QƏ.8CoCvǟ K5wȭpgK _G^q4Fm(Xk^긙 W+ul@Ӑh7ֵaBC3O=v׬-+e7=(ZmN0C(QlVsMs~` ifX(fN6go![E0^Țkavn;haa'_ɨe05W A~.5v k0s{ 1@3bixv|ryK%庙TKUb]z wf׵mFG\82λYk]0tWފvpe Vn5Ld9bMtjWm c?yp2c`tBL0*dri.?I,4XcpD]!K҆oFdjeC̼)z.@.D Y0}-'XqC )(^9#[bIQLO+搩UVtH ÌГza+"G_IuR$Y] QgUԧidwۺc ~x?X%ꔛ:D<6uDLV:,8u|͈C''cT/ f^;2QĩcHjnFCE*֔Dbŭ}?o?ݴwvىOWcͮb8q03}ݼZ.ۡHpF*iG~-Q~žTSS᫶oޏ#S},&gum}NN$aT|rԥte; 츛.L^k8),Gk"~mw~Z03e6,*RJIaԻGfKCo|0;*w!ߨ0j33(J9Quj!{~֘q;:+ +ZS7Pyҥ|JZtM` 6L+_|?=yOAcwl+nu˭AKgXO;>rM4^QlCf'">l}\b[?/׷ߡ5m%J_a1sO4?<))e-$roc1۫њ/eD XlN=tzxpğ^Q#Ixۮs\n>pɂZ 8 `^c\qP gg'Ύވ1$֊ ib7>5Yn68n5#t)7W)}I7ȏdٛ=o_S뽂P5sU0Z|~&"۳f5TV*J):F"4B)@ euo~'O\kW^k5U9aLNJ11E ARJ4t8iu) ``~U/m]c_>vHFh8i!Cb 223Z1g'gD(8t5 CO1p7] ?i'k >+iwtwWJ' &:[XH@Q"(N3.Ox;he3;O0+%S]=ȕ%2Sv5|d5L @Far1]{VHp]ڵ;Lq mR 7ΥwQE-E7Quuh:TD)g~JRggr/gP)qNk*Ck%de;OJiveJitTnMq)z}vBY^+(ӏxx `.7kcŲ›=5vBnBG?{ڵR3kH!zAQ{Cdt;=h f}h4@te'fsrӮ6m jb.obTUn7Rwsg˸e=ƆeYc,8[m{.`6K*6 ZIX Z&nN%e@T/q b #T]( r1b`1΂ݧY pN[+$Jk#f DPf?Gw>3L0P3Of3>oc)+RxGs, [ N[cJ5ӄh AkbDzv81+Cuy Ol#['p"Q{^dƝO pZdS)\J#|e&^buy~ G#8YU,dxw!;.FH \{Ss?=b3$Q(9D@yH/;^<`VDjF7/;2a{hugD(7'Fw y:h`*0 K {>W\!F7 *{ jqygBu~bK^ 0&W~Bѳ i`< \ t@ ' j#v;0hM[e*Cr1VJTKIUO3taӚ"#}Qdic(4PhOvP -^ $E4+,\E"7S8-?TQOZ04xƱlE*fWڻ N1. @Kc-fNx56B*kqJAB^]V'LGE]=Y uusQޟzM-q|$ٓq09U%L:"U/M{S>0Y*jHBLf2@AS%J*$J*aY6xyء`+qJM AH9Hd.Kn[rja ٞ\ع/;AQ>V$<}+X\m\H9/۫WYg@4Nwe|#^8¡%1kc1FɅ#C UkO˒(I&HXpF6Rz)U8TxhoPdaPsOJx^=q?̀o\l74"O{U[S1e(R}Bdcy1r)ߐfnԄq j]HߧƌQV`_{1DfgcNaz)8y^Ҵ7.Ec3c$8l3dQ#Ex"ο1r+}Jr]xFWӛ0NXؖTVXG?(ffO-[Rh[ï}!Vܤɺ%+w zHcŎZ~FįQ\\H,ĘɁs%% hArSb|"*_exL^,VRBwa&g7~ =pxgpb)GbjAQmbH)l [$ i'_ ,{g$>juU||W×}-7#$R$Y: N_{Kу I(|;>{<mYhџM )_|jM48@G*XRnx*z oVاb6'ctwòQ>׭#Y] r~aQdgIiPCj? WCڏn4|JuVlűBd}-71$)t7ذ-8>}毈TkxD}%ŏ2Igy۳'B݉kKNwm2zy2|H|X\/T"#i-1,ޚlOq| [aC//x6,SC|S9$ AsLcO>h _ӿqL<]ċ z q*BطI;3ml79 Ƹ[3ퟢ+A9ONX$oU f@tXWr!@+9`9@L K8C}'.aq!}޼IDfV)n҈'K\#_ JDpV&Lx%!@4*odrgR7(Pt4't|j:'{҆8Mf1W-!t^Y3C^*SҀ=ɵ"D<^w}WBǟ:?p/`_ ƻ)}rٰ H4gޟF(iU$@ u&H+ gh<9gY ~ABjw:miܪdՆ|w֥f}88 +_(G}jy!kK;h o$AvNI,ieOANpjC ZMWE'dr^ o:Wnejr=#8L7?>1 wii|2?'&L`r2Bō{;3S8g@ ]SfO6cT:UE=jA^ <@ !dL@^ zd+nSC BV8~R$ :$jEЀvy!Kx8ŏ˵%N?׌L2dYAH; w7/a؇\ qѤҚra[mD7:"߿TZӷ^5{]$mӣ[ yݽx.9&yvטeРJmE?ڐipSQ㫐aӦٙ1"5nͅ'q;`[6?k HGpeefJH.q$D E+;;q̾$=jï(wh){dϔ\m6>s-.! 3|qK]ÜkΣݿ{n "K߲ ɐ6A9Vyp^!{nrCa.}]|B>Uzd4\|ׄ]vC%G uHLe 8` -ּZqVn>zk /ᢶk 1*ޥB x^nGFevDz2ޏ`J? 9K6i@v-YNA>ސy3NOd~cbQ>* {ޮCjxSܙ qNcmPB JIUd=N,<~vǾ 97c҄ 6YІ, ߜ .xok?r5n13BrPur.otc-wz SdԖz?P&U+bK%O7S7T>ceG9sځkThr\O2T!sj"))]"02Sm.,Rc|缝dY &F*p}UCH J'bHUlP$W.V%9>!QcxG汜dʧϐTȳFF(mWH; hVq<}Ǒ@&ZB7 SD9%ݓ:1o[6tep=eK8$FED`DžY'HcE]4ÅyyBS Fxe nV5ߑ s-[w_}m&Rb2@rT0^FMJ-tz@P}P6\ iܙpAqؚ 䪏Ma-nl{&oZ p֣Pn+mT:XIU>TxR-;zTw1ʤ({A^@-'h^GH닲\oqt)xGJC5̀BEHncC|V[Y~A{KvC L uwF=ݐ8IY#Z컁^CiA(E=sg= "~3dE(6ƻ8U9qm%`~wXp6-u}UU&G*c2f,5^QS/8e ɹ:8ɿP?'Jg.RJ?fpQv! AwڄUlz</ GW ]3ov^ و_ruf7['uUi6>P #Iד^R2&]r6ùjc{LlV;RlMlx}iӳ(q xy! VB]Hf<_",L$8qP]ֹ =-Y{f MDj\ȽLe^HeA@6$ TEEAdX:&Q ˱Ω8:0@,LUFtvot[۽np8!Қnf+#%A@!-%mzx{.wx yΤB06YrQS],Kl^ 6dzTQ0;B>?O3Hf}Q1%cSL8iz1y\/A#t{pL-SL$u@3E?X(V*߿D!:(BC\e}p:Sw#`yaFKr+$Y l'Z$<|gL |5!FA'>BƖ _|1cS_\>{ +D/ '~G ?r~DT}|,TV% (jkM.GVbJ0(Edž9l8X٘V:5ȹ !bXߴx#T[`Z"XV%ETPmKȓ2X)rպ+hvb6lUG+UaN6iP:$:[j#v$uQQFC6r *|-<_01G{al(/6X.drCm>kE ّwhH-'lx%sP(v|"]?17OIENDB`weboob-1.2/icons/qhavedate.png000066400000000000000000000175251303450110500164160ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIMEL%tEXtCommentCreated with GIMPWIDATx{wfGuV9MOOAa$YBLԒ0&xA^/g={d^ۘ` iI@Bq'gu_Ū|=҄! z>w꽪~uo=F?7nA*%1 DIJLD04*Ұf;|+ޮYEQY҉&gk{S );]O#V?AeK_kcc=pqW ˗)zԁN,CVZirfyחl @ !3 +\\M%YZ,75&_X >C$:zc+a;O??1hv^_aʟ\x{d&Hm?H:;Pı=[7uZ5#-Zklc#'}Z>2|xlڐj3Nz` [''l @vkv<ֻ@"8-9)HzC=[ >ze&lf{f{:VspONx3ϪzIDH.Ll~澿`-0RJcM"a]F$Yyw5TV蓫s{Ae-h8O^-^׾'ά/yޒi3aa V>\=rX21|O^btnؕJ']z劋B9aZ'_ RmʴDz8duv:J׬v4դd d"#`Z}QVwP "E"[sm~]Ąi(b1;t'nݱ@vX>],tf +vOx,K)F6d2;nag0mNg`[>iÀ(h"߱2É9q1_2z}QHrwry٧ZER۴2 RL:@c&cBЛ^{DLďifdw^rPl|i1+V[~]}>m:>x_)'wA̒$7dhnYe,zvWH`bF%'/^s %]o`#CHHR1x^0 BgW6]GGU~#boݩ-wFN^7݆-*5G(q&_#[x2&UL`0s @\JYxzKY]_ F\% 7_!`M$FL^Pم8P bpݎvoπԚD$Jsg' BcK/?b΁txR+ӴJJ!<œёLC>8aviqx[=ٮYS\s[ Tr @ r[_QDW꼨rTW 0xm749O< 33ca;z G}ݹϿ3w>Աr[̰,;Bx#GZ\k=L&H3!"'Rasl C^=}Y[ 6ζr}b\Jk?\QI KIljҐqA'Lul[ER@63Ç e1ڶ)"8:IDscw}vixߏ6)<DW;u55ӆ6`>uh_أ{֭y/cgޮd s+.&BtR$HJ6 ZVrX:w2<X\W֍%1B^>o-WV?u]t2D .< lZ鶶ukvMFQdf֜I${FS8YǓfӭf5G$ D`)@\ڨޝʤdY F;\fݡ(BGy/Ep!fapÏ";z%1iÏ/Hdݡ[qfvJj'<kwz,ّm6ռz Z.&XexSŶ.`aQ:r6muRi88Bc&/;%L+ oT"32ٹb27 }=k-1N}z#=KXTg#JE@Z(04ɳXbeY|QY.&t:]!dyDzIBK3Lz&̶U૎mݹkGJb)4F]:rbmsvScip[y5ĬW fM,.O,|")0 lccǻVGg_ɐ"S O=@6/:1Êk^x|k($Y LpᆴyԐ4/Uv~QdN>.eBߵk :-<@00 LiN0]X(ukְRBNtɅ|+ j->1\VEXZdqE>&@PP,4dٱ<9<~{PoϜg춎?7WURn&z `yz%HXB CsF:GT&b&X(īT@ovV]8ɴ^S%P@t_6.n Y(m'L;_|_z%'fԮ( (}<[Gx^AMbB&)\'Dؐɉ<iǓvY3m=M !ٶ쐈l|Y ;U D]Yk:𴡽`QHn8۲T*̐ B4²qbJ3$3Sӭ8){?cVg;_O~) <녜<13g~ !SK}b/l/> ? jɽc Tuz,&hH)5iّgZ91XTwkF= 4!|k0vlOވǼBJfGDܾnme'nz*3_P= $X<M7]d&i DlX ʜ,t*aF:M/ ;Y/֩ѭ}]ool܇ݳdLVBJH!uW`{CaHu>AjЊ0k Ȉ9q<6hw` TO`w4UZҰWwf,O-lڽ?P7ɑ0 ζf6g@B"h;v-cA{~ӪVKF6SҼsG{x"Tڟ=t\r-z,[/eEzB ?;My=zpd2[cfED@jd`"H$2 TtFl'l+,zQL \bX-)=it&_A75knn2F@ w-?߼,sc-4>3a33)2f;SaѷLZ|PZ f LKd!H#߰0U㚵bH)չ8lEן Yk ELnEޡy0+fVǏ+~Iۿ<3\{{ AQM+ixE+?_~b0HͩѨH"@ T38 @pı.֚өtG{VDAZ)fHY4}n4kŶb5HQ>pǺ}㏷6*s/Z ukEe~. &KiDBWE C hLJ3k]0 8~[aV:EfHNN=ѬD"]x`ֈjA^u׬r2&eoyzPu:|@^iV,ϛT ^"LlvhV(RADD2ؓ}{fnHZb>lj{K~!/ӆ4tjft{֬je b@(ҚsŶq T~$wUuN3=ǎB>;R/p[fg?zH2h+wwZ!dprdӊ V ߯VsKsAhm˙SۯP7:g?=Wz]=+rܼrZiZ~.׹X)03OK~ڊ}U)ehVssIoJCd";jhQ!!Zm1.fKD6hQJeC_MMIA f2{/i؍s7*jtasg͇ ,Bst0h9`TV\4a1W !©đc{}ktP4% 㩅Mv?gh9 =9}2־i:AFm}H3s fԑGX&^KIR a|ACc( ڛ Z+V)]'zdM&)BtN oZoɞtk:f)GC%Z:p zZzٓ'IJق+!;brj 2G3bÏjL\㨑:|՗OzPoSo9\':jD"($"B>׵peoݳPJGof3틆a6|R͏5[|4\ޯ_n@.O_;73|?%sGƷseŏk߸c3?Xx o}{Ⱦ/>qV[W>ά1#iV']3(L A"HYt7RFQ~OJ sii]Fxʩ>uٛM!/_ǢcFZ>;omJ[ί>gOe?^ hu|Ni,UkV*i& U N.ʖ'вY`x{w߱}W{ڷ gU  Ƴ41p!"^-׬Qeb߽o^:s6{t=;=5n2sIEQ $+VZ!* /h Z/rČ(da Y̚BH+ B4=gq^| L>foӱ~atmWtqߋUbZרݻV\[{]Rӓ*GfFBG_PZ3b:JS QX&:t"ϫ[|&_ R*HwcfݺuBIiP07罃O>|/Q I`ZC \rhn~lz4^osaǸl3V3\l>Q:rfm߽V8z5b [^Q9yjX=wtdE`&JjV 6F2BKWٹү0v]qj?#%u|^iWsU43? כ5(0j fO hض]Œ5rRu!dEc?lF~jnQn̥MX7Wn̾ d*Q+7 "[íGK{7W/]k~K>($"|￲~o~$n(UJAK f @1 @.]p "OJA /9Z#V(:CC[ LYq^L$, Af4+EA477 (tkf`+p𱝆+6 +؎Œ:ȄPBZHbE:tkmTEME#7}gV!)Ne 1tl! & @1sJZi{~viҴMR2 `aجE{0@daIENDB`weboob-1.2/icons/qvideoob.png000066400000000000000000000167251303450110500162650ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME3HtEXtCommentCreated with GIMPW0IDATx{yWY=[^uo[wWWwWld!d! (`PG<8*(Р#$11 ItwzߗnmwsT*i G>%HJ A(@IC_C*6$@45 !*e&(2c2Y;4Ma<;['_U!BKOf45\7iR fMVJAk՘-Ϝ>QNp!Ax_WZ;cR0JtIa_;|?̈ѱ\Wתmمt MI&WжBJ 0ڬuq >:9vr |Wۍ+7N*kaX ! PP:8;T>`v{7[,I^oiֆaD$Hbg̬$RǎsdžmVzĉ}k᫮+ R MY-#@k! VpkCgfg# 0X\l3.k0A$L f@,$gO:s4j?qHڜ馳0̖Ψued&<fRBsPmT*'{7$~{,twX6^%"l;L2 dl0֌|iר,N:tdȠl_oqSlKs^ۿ-[X l0BRsgiU*$adV$3ur_(57v lV=%`͌yuGs}|fPQJ-vZ{z1Z{<͖}{N})o)f\w\wStA0k%D̬H֓S̠^ f&WN5b[( '==@ MoЗ?y齏Q\0u8z򪱓yoB I烱a'C:~avp'V)f@d0d0$$43YřL!m)${k0~f)\zk``3X1@ZsDٱo;=J Fiთz˰Mt gf3wT @rֹ7jX9 73LHfĂŁ4,Dkf$le3M~.[gy4-0k95>1_N4dFͱ0vzeXwh*7uul&c(DpĦz*6_eZMHZfF~D~8{BZZGN&δ4(hnO !D<!IQ^AjʍW\SoZ4Rǁ{k榺G7o~'īh :xsKlY`AD=yQXS-##nVZtGiI75v:JlV&l`fI7f|,yZӴrTkkE.O[qJlGg_}kVl ӣsKkan{_c6TYk׷iH Bf<}t1>zq0y_Q @03frQ;J%F!:0ⱉSf%bm]jȤ~jC:eۮq1Ms:nk`[mT}n~)!S9}Ŧ3FT65\situ]UTȦj?Wlm b S}X;/y[mg""%3̬H0Ӗ5j5y 1}h*|559947[HTں3+U)))h̊=Ы~oxwc;u_$$d̬`mBm??ndfyEbb׾/9|<Xcl'Lϫ6IA !X P9-TjsdY dž|Q(Zzc>}偯΀wnCd ʘ/s#J ho78l%5S=}%o4,&b & \//\s/2 lgZ[m=34 Rܵ{z r03k'GJJ @t^x$`fa6< MX.tj0RB'uv_!j-uܩBbۖN-]QafBJ,$LA1YL{$b]i!Zw}s7f DZuaКUk ŋf N f-Ly>WLd]kjTFHI )!G 5iT* zmٹ"j'Ӵfd63KILA\k͕y5\uZj D i_E DzŦko቉|*ymc;=oLdӭn}_5*o|crc|%Cwm ;YkZpY1+vѬ3'Zk=W- -$/u0PI4+?f}zA>@4U<􃝓2 7nrO6Tf3Og?rɩ~s݋ FEQ∈|uW'n4궕*-+go<Z{"$f\C^ġ0fֺՌl PZCZJt:7ӨWF~{۸媑U!O:VܿU3,3'}}kot(B879ďu[懿Ԇ_WD'&J+"9Q`g<9px<}6kܺ%I9҈GN5$nv*A/:l*[f SuEQ茜9޾{{oxHxv)-ͧ=3?h Oj7r)&Ga8#3D2sy}ttIEvE~tj@4R{4+տMA֬je b@(њށrsD" 5YG$/⹫y󳎓2/tEot1|b߁iV*  COh#RȨ}}՘: !C[[+6u62iQVy^x1b0K]OX/`fiZ9kJs4;q?9%4ɔ jX'`J4# CZJ=U)ery$4e҅BZ"1VMI)ʅ|L&]~#YJe:frjl5sP`#o]MLiId~Kkl[[O4eό#74*[.]P|V޿kK.:Yr~it6==3 JHQ*޸=Tʬ""$ң':4TuE(&luxkepfXSEL0#p4 z{הI" no_'Nog^ai[tɉ^nLՌƷw.RK. IJ@HJM#[7߰KYc13"É3n^ u cBK-^T6[7-5K>jh"2@-dX<OT=U|랚#7+RD-?އΉZDcv)3+$ElN{݉ CZ+OȽK߸ȠyfM)Oh!(ckK__8y[(}) 9o DP/2=H{>if[9-L lD\/Ju*-ǿg=[4Skٟ>2fo5 S* I?|n7'x u{[k&~_eٯNѹJek@ |.'yAYt׮ ʮ۵dπbvvgíSNS~_k8?>}""?Ioҧ޼2QO}vW~w˦o]_XE&&#F2t{p0'b>}TS\=@(Ski>UK(lCqT9yPG- רž߈j8>6{ޡ'v5X/uD{.NUX p9% G'NV&5zloR`Նu7} wʓZX-JIè!㐎1H !74R/xJwU[OX+V!~q~[dj&UQC.(f$ ` Y\ϊǖA?L#RgC\9A" "?#  ~]Z, ~=xN ;Oٺf|c- W֧@xg<4)"@ JWn%'χٗ.7&RjJl${AX^'5? |YFdG8Jֽwy xx(_).#অhd-8i~ۤ[lu@LSWmIt930)(L JETf?_N[,9[LquwxttDMkJRI%9ObV?xǟ&_-02_fM{E߆amt:;vZ ah!U8koT$ G?-[e^;/BLX()fNXH+/qkAIMق4mS8nVy!!0Z25 *N$IENDB`weboob-1.2/icons/qwebcontentedit.png000066400000000000000000000216601303450110500176460ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME +2NdCtEXtCommentCreated with GIMPW IDATx{geWu97\ʩ::IVBA"I` 3ؘ౅a61Ɉ`,@,꤬ΩrzsZ-a†sWzUV{9Yw 4KG_\nB+D  ˰- i ,=8ʮC?lM!!BηIiF| t( 0l8lLN_8Qc?P>p6_?pp2Y!-[l"fh=k^ kbҎ(͙T*k,eqjT^fM>Pm`x}ۄ}GNbra]y3G|GyG?ܧ^&gۮg+$ )f60 [~#X96|w`zk+]G=:/A)!!Aoݱݷsh~86y؅CHzw(2+(Me@"O"5{z2FTKݎRwBZ-HYsMŅtk ]B޺ҖTߺrab_~>.;Xж: HA IeC_ >t/C@j([·އްEPG]=kyI)î`.@f6A™V\.VOg?]Y~a`ptzw;/]֖foAzm7'MZAJ dQY pP{~/?r}y{ׁAk曾}}fcx|hFRZ(؀0H¸>f6ʩ? *R+ yȮ#_Q Fb6,ˁmq؃wD* ȡc fi|+t:u~6-fV 330#& |zY+[߿9z]EW|j?*uf/?muwa;aqy ==؍T P[P(q5b`j&q345XJG^Z&""@Ah0DD![ XtԳ|+\UO @kEG)TR~x(cA}HO.؁Џt hܯѾ@`"03unyⶑ!  @ c(9&CL%Q0t:'0aqw?;ʼnZшG]գ+13B@,9mfH"QL-ipd3"@ca- a113KB a2g omTh20Ao!قP^E@aDX3 8pb$' sRbX}k1︶ kH$i+  dfZÀ mN[ $ *ȴ&m0'}3Sc63<O,?;:F>`Ip\ 9gpd̞]Sk׍EOIYx(J1=K@A &Axj{۬v ³Aa+Jn>a;V/"bb0P'ǟloȬYR,:Cʤ.6^^:.n0 Fʲ\mEtl߸1dZV!@AQ2qH\P 6 F Ä!G>,Ⱦ:wKQms"$BJu,a$83SGGsb2M2fu*ޡBg%c9_o ߊCi=+Q{2Q ^5ܴIB0Lv3TjakTO<F`DZ{{B!ْBXXX<:S--Y;ݳfE(ì2Yrv ]:7FGhڎ%- ċUj&+ĝIXF;cn CrdFU٬`н<<86u:NqDdž3Yb)-dzz!le 5Ƈ!Jn /&)y@d``%`%Qrl`\_8R!)9FBhlE7(5[T>Wj&v @,Kawf;H ipҨ@`|@bsE&A!8l0< fnc̀L0!Ks-cOhyl"<ˤ!(aa2#[N?w1A"\\*w;5kAP")ْ?\ %/+ g~~QIR<2U3 !vR Y}y@FPR%@H@r2Iu!* ToQzhwcZ9+nAj4V:HAHfH"B@H7e Gfsu] ]QgI +O0WAp B-XG$a(!ALu ҵ`r%p: $xyƦ8jD$tC"b"2Bt4|D A _J$־V3lclGQ̌kci`I:XI4Ɇ6&@`=Kj RXg@ #-C ==pc?kgƻ,nrHAHyaT. @ Ty^8F/zgٖ[.p{jz='@ He ,ρ` d4i*dI/Z$/߉Cp8{Sal!"Z}GSϭF6[ɩ#sJ'De9L`6V25r˜ld9^PLsE̖PǏjB~_y.e$3`V ! uה(B$%d5IhWLPHP2:ū_.vZ(m"c W:fw %)H"|H1BY.Y/@K>KJȹ3J*qfY=~6̮LVq(A! 2NBHS[}D4@1 5\4 Y;ئFR:˶6F1kkaLD$J7$٣̋@i !^mJF6ՂzcBeCC,?o45M 0@ZD.i ,8:}ԉ#cf]yAPh !'\X6F-cOUXWwa#mѱb6B);8ۈqU@lHs…Rzi$֬"15u eێv䧔H,"3L}I* ao+_Yx? -'E""$muJ"y<_؆cqؠk*61^:IbuCrtwfVt5es~p|8N9 }ddQ)*IE!aSQ칩c7@V:cf0 "o j"!N_Pc=Xy]3HU%.֏@0z/_+}}[v܉{yFayꛏnL;~A'O>SʊzB()Bfu9erg  S/XF33 ݑ^@f")&Pk($!6I@6:F?u9a}hde5"0e< læu{<09V/@v -x W{|28e;),yKT{ 8"xnnX܊*&"VfcusBHeiO@E5lH@n`Ѹ䁈MNDlyؕ|zzUP8|>΍oײַGn^8HB~o Lƻ%dOu,IbffpKe[’gOV D@E[$R i ЌT> }KXvѸWoN;n& Ciπs DZ>0 ]@@VY|dV@ ƫ~ du3K#_W10\^ؖSϤ Ui[@]lE@BP" D̆0ߴ2L_@I_]+ K01b$=@[;;ߙ 3^+Nm-5x`w'g- ׽RuoLc) <7_XX㼴$mKN%%Sd^Fráߨ@@f*-䇇 ˲m$,..dzfwmv#ȵ 6tbHF>An$Q 0k$ !{K>H.T>ysPti!id!$$ VkH x7~e v]A n>wE#?nB<\~񸰽Ð~*Ks"O< hU&7hc`{z*d&..ajUm 75SC , ". vOE&QsOpaz&nX:q=Q}}ЩyT ¶\=_ &x!nF0yڑo$34)V~aXkG5=qmLdH#auwuܴ೩&ZIhjd0J(,vUsYB3 ClF1 \j2WAfn@b+ʶg>yD[7\o]JXPP HA3RH"(/۩0DM[ X@ Dus{9*DEEp[Z_XS"YyEX; 7f1$I={?#] S?.WcsSc4%S& !A" )ɤjgQ(1({p2R) A!l;lM{[~ "?(kFh^]>zo.DR ©VԔT*QPD<|Ї7Ax̄~?x#1<?\<Ǒ0x^ȳ~L("trל7fU* " q,&O,=wa\5;Q_^LʳO\~m;9zh 9DQhs.?İ] [זa:MmEyRhȄ9\;x /R.{uM aץf]AaDOwzvܳ4N42vm0L`6B@J\X8{h?|[ɩ#7r_ΘXa$oEM*={QKeU<¡n֭Rm9? <؟&<ۙr|qklElB|SݵBܨ}ꁯ=i^Y{x=22°߯C$|f $aU&{rvNcO#{i|Tk.zdd T1 :+ͥB3jxNmX(NuX qܨo]X}9%99/Jf|v3|䡋W;t'n7rʿn՗I7WZ:]ڱ{oKv,2vu[.C0# ε'Z qaYn.<xxf:~ ne9C]x9$Aql"5JB%TݫyَT%F\/>4_WTEZ>7:|>vTT ڼ9iW5o].h&FaBXm'dx!8uf=?yfezں4+ -*r•<"$!BRID-!D}%IUȁ?ܴYKR=nTɞ@v*ϮUCݲOMĨE6z>Ͼaӕ14'&= \H t&g: &qhA{s}؂E2iAD D Ɇc{ lROwR͝뒉طnPtv>Զ?fU$. Oċ3K_w ض%|[0pgfg7OFfF$)e#u4tD~.I#-e# IDAT\;I҂myS?W8|:Hy=]?v+  5L/k8g s[W6^׼jsY$>yoaQeKV¶rlAR&$b!0,+ |iYm_(ٴR^?Kv?c6D@_]@Kkp^vw(:p߃:x wؑʕ=l+VR-LʊRP2IRRضz!0X t4?=mGǽly=yr}B*wkn$!3lٌ om d 7f&P BLAaS[o>[&hkk~5薭TfmBi1[A184GQ'y|nTǢHةfxsV^`2:.DB(Cđ 7[5jVuMC Rq=fr%ױ\hf6Ѧ -H˱ee*(f=oe7zB3wNIENDB`weboob-1.2/icons/radioob.png000066400000000000000000000227421303450110500160700ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME  lq<tEXtCommentCreated with GIMPW IDATx{wfGuVy:M=3P" وdX:r8Ż`c,H4M9|_|_ y{}몺wV/um/[- !A( fJ723Y!MKŬc@`ƅc~}+CBlx#p{904. ͭ;ElNRYm&IBW5j{UkS+ Su}uC}mt[n&gYk$Ys(z8;y|0>Av <-`^ Y6_mwt?S| Cfb@Nť9z3y5Ks}.||oFN#CB´l%HYqDY)X\|!?z(^ӓ/Z77e&% "@}c>5[YMo\ |i훈 Ѭ% Ŷ$x%:}L͞5k%H&oR=Qz0;xh - vumۮ -Ӳ Ha&" O~DX ;y޾h67ZiM]0M$! A0 A$BڦCm-4Lz%DZ\ᎎ$ 2Cd.uvcwgH߰#d<~``Of? `'(\ woᤳkw/vS &$" @E.Ӭ7*EǐGs?v\gg_X;U"v#%L!``a0XH,Ynfoui:zNc*T?:fo]$"D&@&$$Z IkIJhKV<^8n{kp}zg@i"r&"4az~d/O84rf(`f<)QGzGm^ Y\{{0@44f@(#8(8;q53ujawmہHvc]}}kZn6$HT^Xn:yf- @8Nz͖ӫWoB|ť=3gH Ʃ.01~ eĦ_f% "0G FOp'q뀹#@6x]wcuR%MRg9~lUds# Jb{~~}yyjo)4nfͥF|poV }_FQ^q>҆sgARy!emW ̶騪W!B1N|<9+/b?<Z/s4ɝ5vxϖju [VZ`XHÈ($@͟.ЯUM'L}KscCްii];tE8)K3;Ow#5?4>ת}(vrO3=]82OD~V_8,s1{WF_m-MmMGJFܧg~ڮYZ2zĦ;'{{\* x#ɣTeY+Y*-ai^z(O{KݽT~TZ(+.V̶DN$r_k:xnq@xfO@xt佩-W'rQk}Og{Zdp*_hn׮XΤ dَdj``xqj\KZ2k@'VrV.N?|a7л{g)깉ƆNMYΥ&;Y)@Y~ZMN ֒DR.(x~h'B}= o3Z+,e@ٲ0="`_:nZ` bdFkYKme Zk\QQ:{a{{Ŝ]{G^ys}1.$E$BBDPEK?>їs添m=[bð2C'6p7o~A3̼"Ȧ"f8N*/LDlOm3-@u)V*{iLfZ2; C['F-<ӣÁYXu,tb^) 'hkBR TT?t,aNA^Z) d:MtTi9t+8ηv;|"%h?@Ԉ?;\qz^U5vz<ą U@R;7 N fl\E_\Z앆qum0gVO;;hb(_1z,l1"J iV[5tB@,̞^q54KS}ֶZ+ HHgOmeN8 Mҁe'6-OgTL63H9Qx{:\|-3^{Ͻ4cvKoiɼó(b{^xA#GF摚<7=k,,|m;Ψ$Bsh3%Bhfj™'瓠5՟̜pxt:mJ)d?xgNnxT5swwA1"n/Mu>7$<[MJ˕_u/ aޞqߵ$A_\GN@T5I%sMG$}.jN"D83{ΪJmnؼ Wp{HBwONRl2oZ|r=DzKmӳg H%n/t#Q`a:d&7TW4ᰰm;ڭX`J 9?7eH-`{8 ;oy^n vTVUˎVJ% dۮݿ5o|nz}?1|x=z=suspe#:aVbqޮMĉ!$@AP7&I.eVd4BHKi!z=r>wy[ˤEj㪸?ͯIҗf}Sݸs\\P*16mv?cd;wꃗPe/̖ ƈ+&5\G3?[i;nZlH 1=}e*ΪKjujew8W>HT0f39߶ M@\>m>߾znm3OhȾ&y>'/֊(:n|~=ԖԬfx`&O4g]0KbLTr8- fBWZZ؎ yeqRk3hyi^q0Եj90(B(Ӳyz7ƵaVʩ^y}ŅYhw?G'M9DDi+Z+?)ӎ723 b/ZiiZƊji9^BP^N@()X. A\iZqRq*( Ԥţ?uzs?E<[u_58qYv/luWlg`fl>_}WL2kqDaDN*>g*.Lsܨ 0RT‚Hg2@HIe\_r\*ܤ$"5/ i-N4ܙNka=Ѥ+pϭU_7׻̭Ύ֚qK^9vb[nL ]Mjwƽo:eJ%,zɑTw`lZV24xuwL(_/\81 3jmZ0 cbr&OH)$"4x"a/ʢ2IJ.0k&"=??ެYt,HU*K6Rh[0q%i*&t#A8%VjyBS;=yU#(pr\.ʔ[Z(pNƽB]j^N !( lϫd6DWgG\zkM`&0-+ټک8H Ra!׺(PZ+`Me}aJje@$$XD@@ Qq q8Ng(BfϲQٿVJgsMŎA{7>"LSbM7m=en25'CYB=bRkeiZ$1[70N&\ݫflۭr "_H:ZRuc_ix A 4&bLlnϬT|%("`CIPR\,љcma׻.fr ȶTF_rbna|7sbm\-駺ۏׯ䶮^LGLEVb̔r2MӎnWjp慦B{IDQ"^[+Tnoj4^/qӰ(Q֚hFJ\SynaR{^5U\u :7j'6N"8-WR !?SBmOض?۪&3Do]F2}R0M[HVT"VҲ8*2afnmZ:3+x|DW\)RW$IcuҬ|u|j|97*a4pO`JgN?dH$NE{>7=;# v)0#*jo[s;OVYJK8\?9}zRKi3K~En&akB"?ĉ.֚3Ρ8%AR-R־Z^} ֺhe͔T.'m@ݐMN^O:Zۮd"6jGQ-'2P(CR RH-H(")dʒ 2sP|κWv4uwK9BF3sJ'*ʕ;k ~<`qnJIjՒѣ̜(Ejͧ=)dPZZ<ЬJ"3nLæօckxIOt%۰^&a MBG+lvlf,ԓӧ\"bfaNi=.U*!$^,N i-ðb?^Tx~M^zPkrh!aD֜57O(8DZ3GBH/uLti llOmמ3EIQ"V4ӥ 0 Sn.T A英c!uJ=H 77uvZDIagZ,˩4u׵緽<<gaX, f(%"M mb7C m w>ޱQ <^{D̼ӰQkR1 a33k˲B{Toff8^e!$ս==sJe o;!0sD$Y./\u)mk~=O<\,tNOɎD3ElvřVĖfUk-fpqTA)U/ ,Lh23XJ#V'0M[PJ)5ؿyjrt7s@,g3$bVJJ#"!BI/oXlĬ""3`[Xq9R@wƏ '== a2@#v?IifR,PT43J%gHoWN^zȪZCkff rJb3 }rIuޖ{з -`"bf)>=i~rJ=r-,N:ڲ׽vJHE 5^[Ž[ .$ZjH#3?Hͭ14L$4a wJ~Vb!=ڲ0zcNڭ~HN"np=k֜NjqI?F|"h_j{|$CDHL{FNl>J(ú9ɚ\\G@%D1X|¾_Z[GjmgvWbUf-X)hǚD1)4ztVʡ:IB Z)Q!a(W鎶Ihwa^Z,NfZa_J!uEJ-o a *R7#f騸8["Pי3c溑7f&@A8f-aO=133h0̹M.&M @0&ffжR gg Ê/e'"Қ$"{D0s¬8"n|ԙZ4 kmդm|"(q?zĥda J.y^rl&@}0<Ӥbq6y{gmUIki6*xiNHN6JkY8 ͉S-RȠ!pj:Z:Q|fNU-ligM !Q@^=O&wCa\\^NOML.0VZ)'R=tО0#ֺ^ڶƓ׽jKw:˫5* np-i;3e>ADjsѱS1^>jeeJHu$]k-.Hc7m4<|ffc_gG_W25QiyʈBǿ8),Gk"|CT|8 qS)08&A!PJ陦]sT5Η']1M*3#'@i榢nw>zHfvsp3908ZS!`"bAW $`$ 1qFD@@DAD]~_!OOElվ9_hA襓(M]f "E|'jLD/.FSgǾ1?|g*PRISSu۫e! &fER|I(QDAG̅`,?%yNk3ni2ܖVgxxk e\;E}]kT&ϒVF `ZiV*IdaaKIGx'~GgmWʰ @ƬaͨtNZ ah:uU{lj;?yw>t*pY\Xdr-m:6 aX+P:ac@GA|Wrz-_^32eH3[m T:/$ f Vq^-fǾ|qe_{~aU;RXIENDB`weboob-1.2/icons/suboob.png000066400000000000000000000244251303450110500157420ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME %EtEXtCommentCreated with GIMPW IDATxŻiGu&ܳn.!o,B B~6fr5w|Y l0, mI7Z]ݵ/g~[DZğ'3='NCMÛ`*`V &5/j^''7Խ pH-;$Il^I(db hZƤin֠wQ­@~| _ \xr')\AwWAB&ä]Ejܮ'HMX'h~jd$'N&;PU{ѩ|X-\^'+ A5ic`IתVgOtIU9 <|žccŬ"kN-97 =l^M(gܑաrɖْf  ,b ICm7dהiDV}Cjc dQTr.$l 5:A㹧|{J}9oۥה}C\rek!@`"0XۘKKg^ngsFg^ؿw:zer_RP0B )Cql6Bp~ ]{?:S.'kԟ191+a"Xd H;=uLS3\jGX 71ɋ~p.*AyYx0R@%['ԗ%Ž\8Ƈ~agSJ;ᭃ|y`.;4\ " "%\ EtDΏ˫ :koram]3p)T39ʗB:Rrd*#^3 ^n}o` zo9ϕfS L3A1 6A&"f0\G|]|?#Ο.i$V5#Ëk$.~4}#DqIG) +Sax%ZfD=ٌbG0.`@8;oR*,&@'' df"A ~Fkn]U(&Z&Q-Oeo,X /9%B ȱ HI! aH$ddMԁ)G?ϝrk*OM73@  A䳵NLFN@cJII$@ &@\ g\) K'/ [\}c~?M$=YĮdvec ` Ŧ LnF%!I=m_@SZΐ@2DڠծO 9ԙ/l=wtraRQM 331-43ڲ#k]x&"བྷ8se ~N D3<\IҵB: R(RJR2  b"r@nX]H$&x<kL쎿QTFmi!?ܱND $k͏.oqũjІ1I1AtΞ}1S xwޝGiB[j 1p]ITI:,Žoȅ5 B'QnB bXmp̚x KBnz-!|0e;FX]0a28`@DFX__7p/Lh-0ʁl]{mgn4k;.90#I(Z-{Lptn,x}j{ ^FH(d :dHϗ2[u||6~-͎ݱ*gtt<9vξf}uL/髌trlNk4VsiX3|t/7iKnD( VkCw:usM f&g:k%jcI ^2wd6+0+ aA] N/$Ԗ(#@hf]}6ߐx(Ɗ`uLHitd;оFcu-gs WN-ȭ YNL9I#9_p1s-wnd˶ѝ;t'\0 ̜s=}O '9nܳvbuҤ%7Rifk1r\.>1TzImDaVgww!5z]I_ŵ5h(x/<0Ɂ 5( B nB'4V-  !"Es9d RL=%D={Dr}G~@'%3rZs*#$yǏ=;$3Z84\?L\>qoh{?% e ! \X<=w@2^@#;nnOͧm"g3љdmu^Qs|gf3x돣8ZRXl>q{fdf%=툺>`$a)?9L>gXZbidi.OW|ByKfpd2>]yAtΕk>'oK\iy?o=|e7JpPfQv\Z6VˣuJB1qu[)sa:T` 0, :AQ7jqum>=˗ѱz&pSj!Rϗzڒ=͕f+:j1ѽ!&V9Irj+(kuf᝿_/|le}p{Ϳx"j iVK'NXq}uzmxiSNt6rp7Ͻ.o<טn",Ba<$UE5p,PW]s~'~5񗿵vˡL1IA3U%k]I1 HBFC_辥٧q?Pf >iY_Ygp kG Q `I+&bD=YF\S,#޴(" 3&͍Y!e-/#RdqŇp}5Ƃ20!) L^g]48]"A^"Gu;B M4IC$ E|dE}<% 9q->rfBq IO=@B2zKۇL}"CʉE/dkHشZ؈(*2P3߼x=8sM{[\?C>ykwfr#&&ۻ~UIѕ%f\yu?phmDwmؚgwoY[˲z^\SJ9'+0_Ÿ%wH?}}p`aP_ `-Xi|9,,Ȣ\)2T6qjY:veJ猒$R m˟om%0rh0onCd r~1*@P.`@^vtsg^J\,v[끵>wܹcmq@0@G+'kvay%=uYˈh^Ȕ ]#";|WE!S [ɒ$W-#o}]93 i 5 okPZS,-uDdmp X"_meId"pƷ_) ,4o,_uz 铯}VK$kSZ&WMLZll,ѿ6Ad3N 3Bb˅|h W޷A} ZCS2ax]rʖ"l$QWacaB4$lc !SSFHVD% !vby/]uoie._Jթc?eI(S{IzIv>>v/T+s j"130EoOM\_C Յ2Õ3թ1l &5Jg)!`(z3/sĮg>Ǿ1\& !3(dݻ> *Mc?y/&ɦ(n EL1<杘_DL}Ŏx]ܼVKU쁑N!(5=-@P}CM["֚aW)|Fh|m4 ;?tzrԍ}g\Ghf ,\~i4=nxu3 :W;ϙʁZFnUy,/~$8pA8ܽ>y>i,`%q f((1s٭`P^ϛwHMkGړpgK1"(mE]E Vn7>:^>2W~^c/_ €IP7rN=7lIhhh֩}sBe6:7__輮~/>~7'da)p#N-ӳ%V.gȊL>33QJ@ A1SOȀDO o;ِ6 Һ۔S- a=?[3p`=29\˒no+%ɇw>_ZrQ8pDRHJ _Cle!Q&/<Иv'ۇ Hẻ/$Z +#0dĆ〈`Gm@(XpC2Ll8uv+4&@[ST[8EZi/>>^;]OH[tx㷾GQ#݀H9}פ/SH9;3,-(Q+`-6 VeuJ{x)coro]Y08Nk"pt V#Sy[/=N@)%6쒩-6AФ|_nq.4B^sR3ϳM\Зv׃F}ɩScM6<kǭ5Y<0s ^u2BBtx?]ʖUv#7S|_oV i4+ى-;BK2̤1S. J:V9/Γ`i,^7׷hW@X 6 ȖK_*#b0'LuR&p;Fg6w 4eus~PB`YmFY!I)RfNt#?SR7_@gBmSk/p5ﯹg,^Jyȟ7ry`فt[oIK,$ VCSi6UFP^ގG|F@D@/| ©SFwQ/FͿ Cr=Q(RC[c" &bw h4)B :~ŠiX;ФM{K=xׯaK+jow{p+ǯA9j^X2A11V o[ m[_W#3眙+C[P=bG`D INf(Z2JUdIDATԥA[/<'3W`I,*Ց4NjQoK MD bcm|3O q+!Ýo+w}E] 9Yn/gG0ȵ =s A Ō|TB6-AV"Uu+!YA9(89 ܚ3m-TZaڍf+$ gaRBq7cBJ%,6oZ 1A٪ѣfg: M[ۮy3sOAø'BE82O?+e:OkVZվyfCY5@11`c"K;u$gB5Eid.ڻ'aKG^Y} @"FcȶR6'!HbڲZY=/..Z&d+,]|?#;bKu,~NsGwì (=:`~3DG+,ȢТݫ.բDK~% R` Y0/A$=  +N[u#]tZ]\NÇ2i&+ss SlTbatve3EYD'"zNpub,:vLq钝o]ij9/w?Ћ鎩H䲾=s.ۃURtbEBbVP %ffBBRFƚ$Y:7˳@پE5`$1:)Wv'K{[lV0 !A,XaX˜S jeȾۛOcr1WL!j6/( Kj@!."iz~/ŶЦ$ckIȄIisIOW1@*Tf-O[/66$ō¼ڱl'-4JH *&6FZktJߐ[G  _$߆ 29ɹ||=bk=|FN_4`M]w>tb 48</ Vs_l _V/WIR.eFE򈄲 (t[`hW\iF_%AUʄ86Dg RAI $3 5#BAe32=(Fkz&P[q ̈Wn _@l|jXlwӅn/P6[l.H~q*տ&/nWϜ5hwStEngt&FruoLn `N-P S@ԃo]]\8A+f5g^+JʖM&cII{`nCԩ/33 X6_.%Jc"{Q*tzW3ۦ~L]ڏGاIL&Z@+@6#|TZlXCBP'!FλN:#ۖIDM"M$QdmS?dߣ.n ߧ&GP^*%f@I+@qW~6r\/FmRCM,DQbm8 ܉o_nzqhu(![9 ='_MH(0XZ2TnJ'd5Tk6)bfD $ig|!+qx93g~ Z/ƛ"fV mj JA#!p"ccĂ"n ~,.0T%t+`lXXL|d )1M+dr̵#1EO?|6O4<xym2&=dySW2[LhX0&@ @6$!JR ݮ' T{26u߫;9]uU-~ `?4n sG;) 8bIe\6b^ZBc1H{]ƯG沅l&Y+ ,uDamsjsةNO//z:tBR |.JzrRbk(I4EaZ46[+3ӵ~v JB \Yb󸪠`ba?mli.[.eY.jE!q (β-L9[v29Gav$F.u}~TCdIXQZ56p_>$u_Op5u^^)u%IH*)mqT77/烞`y#L4@uhGJRʐ# k%U8YV,j^y{Oy>B '_[.~M\ݷ.qw5-~D>$<\ HR |!¨P֍ڇw&?P+ʽ=#`ZS'W1{;CsBXhHVr܀'gfiZv.KsIE?0w[ƞcm.OoȓƳF$HAg FAڪ-JHlh?  #5jo 2Jz+@pX[wﭟr:>)s.dzU$*8n(˱}6y=x)rEy+ OdNB"S7 L]?7;9uxpj܉Bh-=?J*c,`4m H_+_W-6+W L>3yDYܳW 3+%+1Y mټ㞣_35o9_[]v+9OLH rS_ {6M|fH;5uS ̍o=UX-P$*8㕅[6&[WUu]&VKJ8$ĩ'-7Lfǿqϳ}}Xp 6r96KN4ς`?"VCX8@LuV& 괖qX2@06: C֬ᰀJ5T\ A|ŝۼY@ {z{ʘu C#enXo3V^N(Bw= djr $²C2U)Mk~W/Vw4L)ZI ,4V:ː_]!A<+@ZԤD(#jNR' ʑJ{Zӧ1cc~uPZm[t`M@Tӳ0閘C a1#GNM;k``ͩ"k*#a Rv18H%O NsG&@d$7˥wm{_*ԋx˦Kj,cNK-LefNn5fiyn6o$D:hdUڱR5W r9Wg+|R:F5∵:Ö$w xa?;s\JHS,Jj*d:GyQ,Uu}՛2ϕZBf*'u*/ l؍ uޡցZej͞8](BH 8n앪L<˅^$ao4Kӳַ#9C9^㺙zs3@ `DQway2duW1Ep39m\aKzB*X>wI]`\uR_]!5 oZc`uU,-Ν*$Dj!:zEq9=0jYp&+kG7ό9wXюJJgUdK|g[K0 ,KGdH􉗴 Z8*&*TGf6^On&Wawxo##{wY'ZOz|y/^f4A$c뭝;>=5wC"t2.I !oϜ>7}JgG7.= xP,tADyS_nEr-B$[y\7Kp̖LtPz{>yhrXt1GǍ[kCbKikoj?b{{lo|쎥/y 3n/`r_׭VuPw SJ }okg8Om'8W_?L%HP%1VV#n-:q+Y;@fXBb)#B~ 7{nv2RE`&8*ğ;9qZ ל$|1&skǟ)[f [Z[ɛ^:I/_ ޲C=WlYƫhie+ еVy._~y?q H$ "q~+^°d%f>h򉁁BhCre:zhOpo~n`]]Z.oeryABEa]>pD/%: o8 < B*]Xsk_5(IRYZfU睵i~'( f}j痥&FQO8 !'OGq8'IG?w$aVX,;d͔C+ N5Niܭ{hphm@ot<{о6\HPYyRM@syE>o|\FuD6i/ C O>..mC8CmN۶(\~꾇O#v%=7[>5q@y o{ݣit·\8Wtm) c^\m#^U/W@(%Xktp/]7#L%#G"n]>Lo>\xѩ|k_|<81ޏ?tTJC@XkDN|oiZ`cY/뿽8mw +d+,<"f۝zYZXe-+kϯ`,3XVdPB"}ÂWakҦSw7 =ىZsnNHzFodfc9;+#0[o5J&NU츹>/I_uN߳o 7KB̙0@/~G?|Z$k?R6V߸_Fk']nЉ)`bv\*9~˽wo)eƯԳ4up0*{ʈIݥS9W=p/O>Jk?p};n}۷' i27<SOܚsrP.u%@$ynBX#6Yu5("| (LB +u-U@z !?3|20xU7L^ߝy[ٲ۶,-5kߠ1 XJED m~fdسXt°ín>&ܑr ? %ԾG*D֮=8^Œxnn*r%b0Nnsl3er:q@l 5r ŀjV:nЛzArM{naf2F;꫎Hv: idRB. )Ga0aFxW\F|>*BVk-(  iPiozIB̡ESSh[*.}u6Y $ UL)"(MU9L!" Tu:E6(jIR㪌fWRfdӲ),[̋+4l,M"e -M: R& {Zofn 6Չh7Wr/$37 ,tfi33'ٓ9)̈6L>Ejl烮qe]m`?Ȧ !XXcT>+$QnJ¸M֍n^_$`¨V3n.brvE7]l)-ѰT.[&H)&Iئ D/x5w`9`uE?3:q!FƵd Zӧ^M::UJYOeiL&"HX"NF6Pl{~faa [ ՞Rwo>v}E]( 2ƂP)7..L&kX$I~ohfdQ9n{xd|@x* ZBe i+|ct}f˻FIX:]ɲ|kA eK!#"Jx56dm7W*6aXƤl1l p,&-\n;rdO@Hh6w?Iɳ6]p􁇽mt ;~lY>~uS';|iÿtLRJ& aF7lIH`3H6M҄fTa\oudNHJYfc,W G;H9Q!',j*妽=Ka ϟ.Qne2y4b1|eaböCAo~$l.nKwn]dk}6{| ]_V se(jm(p6[8c{`#O_]ZjehT[ -uSڏ-$Y'AjaT* Z 9֊zm1[-YZk%!LPp5k60!5YHNɒW{ݽ@(yb) \T#a۝3=w߰4(MCs챲e+І)0sjщ }tp:̥ë36Ns7L8bhfbY;TGN0R 9/.OLl=OzGZZf./lyow[ o|ّfyyf a {;% gWs'1FN>XM7KiN+/dHІAJԔMUV[hTd]mWD6LV _N<7kK}h$zQܻ4>~2b Ē,-ƫNW(x:닢v|wcǞ>|-KO;=IH l;k1 nǡpifԉR}c0Տ;naau 'Z %Ms.nփMx[89`Vݻ9LzFWrjĞ[X7[-,OhX,i#t +&Ԛtp5ſsffi[+3i9n53 $`$zOOoS j.l]9Y=[^cC|o-m֍0wjo޹Ofrroا`j_}H,4=u#`_wdiNݸ\qI6گ8CTF+g5E-~˖7/SLu2Xq1BBnLw]/7QsѯolS4[L6g9Y̴m#_vf׽{l89]-KX>3e{.qwDžz1"aTȕ+u>{i׷ilI%׿k}^g|Gw"/+~t:t:~>3sM=[egV,\XiDq~\nÐ6p;~`AdM+ PHB [?'#,_,IDAT_z+.m8o4:2y_y>2oO,Bۀ3`%ֻe>~h?y|Ç+@y]hi%%k/yNFh``562[c䲕:Hƨ6='&K݅%Y?~J?;AfwqC:~>wosMufI,Lz{m@+0?+̫մ8k4(G˧_kV=nx±8N &ݡ1eQ32q@eftk^sՔ'] Ȯ_Z(& u8OmlO;{쎻9jkr,f}9XXx}ÿ[~,[>yF6Vw/}ORaX/^ם=@0$c&lGh^![yȑu''ڼ9$\7fCl<l/9!"eR^u ޟtOI%JΑ$9IH(MAw/I7h6,|<NeDOQJyz;wߑBb~P{zv9"3ݍkMМh& 'U+=|]ox{#K_0îS ]bN)8x/fEp-> 3Kou\Hw8j81yS0\dL&.ԏ(]a2b)g3tbxpbNbꆚ;axw+n?C L88e'6 I+fѩn^p99wVݑ- ֌o1,k3''̬a>3+KU  H("{z'{, S7Ρcݣ) &n8G^fcg{}|_x}i1|ق\̚M%Ck6RHY$ @`d@P""|ugdia"(W~jbf">rF_ޗk[˝\'+YhT(G0i5OfsF6Z )cq[\. .f # G54Zi]b-o;C~X)|GIh5+T,SRADҲee{c =nuuuR ]b/zEbb9JM("\qO\M]CU_/zmvgZx#Ҍ8R$p4,3/̒ LԤP,ĭDN7ޔ4Uc?Kw},]_)521 |u<>4ES? sl.WebJV̒ebʐ$ Ҧ%BwmJ<<ͧzO]=c|ֶr}f32MbynVrZCM!E AMX'I&E}x n< `qZV&%Rm|F Ֆ(FpXf1ҧL'NFZ- W_M W~9?k2Ze4 m/|!YWB] F˂bO#=X4)&&F'ibF4>㖟K1_O=︵IENDB`weboob-1.2/icons/traveloob.png000066400000000000000000000226121303450110500164420ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME:UXrtEXtCommentCreated with GIMPW IDATx{geu޷>z7oz/liP*)9ebJ,ىG\ ` @C͒%6%lK9!gr{og{>۫qqO<M@t8KO|>v!&IZt̔R R=?it\SB p13FߡTl.sY.֒LmۅJ)Da Zu3ɯFxӀ w}Zuodts4KD.v`d`֌|ii,WC1v[/v;?:wESVweǫn*Ca>_[`m2`2R )*qӪ5,I&k/5~ @}{/˽['̌)9l ]D pP7V an`}>F"{/> pP7ܓ/{k|V˃`[km` &АD̐L9fKq[akQ>8!Y-CB2sdhh H9 % !$߽X$ˬ * SN*N=㉝5x|CsRo2D[FƖX̰ d``"I Iif 3BZms=?XS{z&&i H W$lff.-͖wlBH! 4u:kk؈Vtw|:ƣ#fvp6Eu9 l !f&Ӱ4io1՗4'/twéY}Õ1sll֊9%$9N'7=uofH^̂ ew'm?0Xk(&Is^oyiw/?>^*TRbom^tnc;f5JRrR3HAƚ988LDIBLq4-gGdbnbP(pB89/=mzvg7bD$Sqf_[_&*N* ̤h0ƢiXV'1kߝ|hnI)BMM];wQ4t4-2kilˮ\v:M3&*!Am۶c͒AY[l'3l"r۝F{pkW4lp T m;IoH}aa2EJ4u(\fy>O] mak닕WOY7801?:kqxpJZoj.Wnh"@)e&%;*CѬ&'mu  0N7+fFFk8̓kF}@a}t~D!iS N125;to?0GD13}yt(T)Ddvt0 * vzwcrwk۱Rk(r3޶|9T `kBXa?rWɭJ_ԁݯN]bNHIwrL1 /42$V& Zi~J`f$rt?I"0,2]&eJ_jQ)!W rԨFJ]+DQ(n A:xNJi7mX0C1CLe &%4Q_PZTV|f$Ixlgt2*jemdЬHf,I73IcL]l NK^$[-Nq$' kKfcݕA (ku8Q f64 (,l l4kgtx` !_m4efm:pG޲ch#o%!efd+610 ;dfp70!XTk)I!%d 4M[RE`@[s{$cCFq0t`[vLD DāCd1i7UGY-D_H;٠\ 6@Vt2&όH옸my"Z8BJDk-,зY:Ng^ezؖmefdQ,,֎*y~@y3VJ^+ `pH$e9"""֚dV3\K|٪WN-KZ'64myҳJ%f!W Gv7Bvg=šC]lMn~N@6Sl2k AeHo0 +1vS x4iى7҉~ ðfT*fC?V*A*iI!CIiiC*ՖB-(ɎרJiJ0;^#SHHv "wvroDĹ\gf034kx bdfGQ 8DT:N {LsW0`) ,s l!ʴlubZmA A 4 @$CZL6h0 F0Iۆ4 !0U$42Zr+AW.FqdH!u_XSkMD8 5n`FkҊHJxfj}D(IE`u=Xc`p|-vY7$ |)ѹ~|VVXHiB7[k& bqEqZ CbF6Stk ET-ˑD^Ϧ; fΤTZux#ʅ̚Z\,kfʤR Ltf3km曆n )&noEqdѭ1'l{}]U(;~0?M2rNDFq6‡RsJ&z߹;IDJ'VVYq٬4(N*ڳ5q1>) v T$yMT*籀%6 D@݊m "ELHvRB҈#Y6ǟ\TZ7V&~0dPHJ)ubLM,WmŁ̢EZjd:D82umۍBWK~๎jsfVy~S/ĽoE&Y1c fs(1!hS.$W䅒eZti1\-[ARqsSoo@1?{ktKFJbi R=O͞O ]d -fVD&K)t:VZJ'7( m^< @(fb%LOZ+ /5g絬yXLŠt|ؓT*w]aڿГ_p1=nuV綷D7,X¨p Y(r&-V3O*fMҲ F`U{{;J)YDhrl%;FZ1s͕4BLH}`0k뻳RЃu0+fV/k0 Am駿ҿfֺ*;+J3έ qݴ;M^N<)йv/r*3'#{*:+N{#HB4 5Cߨ5׽8({xgA8DDziyʕR`wD._9-[c$43'Bx~Z^J\öSN]+0##Va)hV (Ho۶߬ !N{w}'f-JffrҝwIך_` ͵T͉it>w7._]T}s;wܱ*|ff"y)gVRoӎ 6B^][̙Ua&^xKoRWN}IkVqjA1@hJ?Du̬#B|y1I7sGG7^xZMK]sur AN͒RI(b =uůO^BF- Z|*"r[bG+_Y]Ea.r3VK>W@wLGUo$F_lо&msǜܥ\zT 5QkfVVf2AIC^&]X:~2`H VRLuN+`n4Vl&ep"|u^Z]Z&qde2yZ\j ^JDJL#Z'|5Neu& :68ma2%T z52Yq9^MVVgӫk   R=^b:o2$Xjڈ"?CzH{2aGFJd bLR|?644;[BPkkx/oqeʮ3#.;l+QWBz?'_S @eN5[mC%QDYV@hHFd*'˺rfxF| @O}ف⮉;8 Ja|viyj_顎}6?tG̚?_$ElNgdpǕݓZ+Dv&U.j`+X1'_^E\L^In$jiq g23my._̟P4ڗr<ز- W0k[JCǷcgOlDC)'54v>Կmj˥Rߊ* ^ѥ+'cvn9x >0>_gePvo.+J]tڹp[s  M6n%x=7O:5=ڍr_ҕ//A/~NeT3aP. iQix@94picf$ ݠK.0 s62gۮ@O~ց_NsdfRi6&_pϞ~6L 'π/׊R&'oI7|"7rZ.x [Hgʋ[W%mreYͥ|ϗzn~n*en(J)$C_7HF/ 87sQV஝?)-E Io!efygaf[?kd_/ۉF(Jw) :y%!Z^ڳZOϬ~//t~૏Miw#%7k28 ӴYGzGJA'zmk+L<#3>F~ޥ7~˖ 3Jk]΁F>€cvlW^7صcy L_/ߩٳ-\wm-Y5}0}q{bP$qTZmhF=9RػVW T߾vht68bJ5k ED DJ#ՆgaqHe;IDAT.=71[׼ PcFR.~gʵGڝ [HJ [\,WB":z;S_ѻ+JؖB 2 o1%!?~ntٸ\yT + a&\a Ôk;CetLpn8Y'0m"Is/8v3Z6jT坿o}1 GJ ⻮2pI^xoZM$1ĀmFI[^$ +!Oaϛ?]#\kMmw\d怙"Vsv';0G}^ soN:"rՀБ8џ}5dyrG31Yu>gfnAW&Eڹ%)ckƅOͶ߾!`Bf^O+ƞW>8~8[,Xpfv:d.)qk<99Ͼ3~Z`>%o>(:pث:_x3PIm߷}fapt~~T4T6 pΞڝa ƧOCWoJxJ~GO{;4裏[<Ǐ#<{?p_= 2v3ROOeX5+ sL,^NuH~a"uiOWDcC}МÊWyz?=ѥ ;J%Y99}֒ @9# P@L;8}kH>}*QQ_n /s`MO[s{e}A\zaCA(QfЏ?0gHή]Is+hFb#Xlˉ R%s9ǚ8haakֹ ?S;9裏ػ?#۴ͣp2bٝ5,7NcN+! -$J{~KSA]??;䇧ہG=4 ΕeZmˮm:H 9a"t?}> g P!lA)7+S鼐 Vq^+fk>w0''}ȒwIENDB`weboob-1.2/icons/videoob-web.png000066400000000000000000000177761303450110500166660ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME-tEXtCommentCreated with GIMPWYIDATx{yeUygU橻깛APDh FQBg{&b0yqJvC{k;yQt5_Z{ݺk~.56WA`]/7A&jVgnW`ݯG6_@%BcolBYX;8WX~v`-!i2-- i咙VS4MaryVLLa( gN=yLǮWxW:^ʕIF} I[&H+$ 8 FCl?oyXkZ+\o0-;|ZU麡~Nο `йZA߸R L f@$4 Nh;Myzؓ* 96}殮um9$H0gFNd=c{Eqr[Fm=r_?\hY9O)=7;j4~#;  @3ɇl~/ fe0F|13+"(fHb! *q@WJj5}mJ\ڽjKֶ8`NN{ÏOu2ib[FA-M.={[>ЄL%$ß}+|@{ uCWVNXrS̰d`Ig"@MNr\y҄ ;Y~m&z`_m&Z[ђ%]@=Q]~ 4J819phKއMݴ:֖D033MtvuzV*S?ɞ#S5]K :3@bdb"ze#BT>tmqNƦBsRcnV?}cKzb4YCG{Hcv\o}+<{oBb;WTb6 )0l0 05ضs \~{MԱ;SmvvD@¶8{uTs=`mfdf}_:/^-ejɝ;c$4=Lrsò|E11m9q2Y07VŸ>-WvIIsUV֥ɔ@0>zpyg`>9D)tѓ{pgŢ2{tTy1]Mzu #$I*p\o fFe𕝯bRj݅l:ou,YƬ<ڵ̤q:g)3i~/ a$Dq9=һVM "t fD:|V$l'׽zs c$ .<.ݚDm{t?纩|Qis?⓲0{tJf=0lbglX,`T"B,t5a(eNt)&{_[5]{YH{MﮙaaJj^y"vtEWm!X)Ry+LL 6ϏO 4&lUGxzvĴM`[K\f%+Tpض cϮ+Zf-#6]1۹D_5R~w=g5L,m;H'_NcЖ?|KTJ&(L*%d^Xiqٰ؜-N2YZ*g0r}<`8mF-+Kr[M"8FɉɺC(8QٰiɆ%k(HofqM g}FJɎȥf&MWF.z̮M\?ҭE0L t"3S]enjf4-eDcbIXhtψ>"UlYT FP/{M>* 'oVomeԜHR6x@@PJH0 3' 4kj TMJLW[hl`<п,W29_7֖ilkݗ[o&1+SYkLmhgLM2e"u{RfCcی: Mؽ'MJ s0S7WO^㺓 q fn<|ςiYB]bi&'tښFOK3<ܗ @lZa:T;]xj5f ϯ jchFS0kkbBV.{ ϫfl;٘}zf)gӊe] kuf% y> V->#̈́F1mdft$@0Us떍%^_~Ų ,fl;1j$%)]ntəXe9f !hnn*0MKvB213^@S#A?׎y~% TĬ0r"||&dHMP-% $ ӊ_D`w+*1fuWSD$338 }FFD2$غw d2y5( |ք mND*nĂ4i 2dNMJ(-vhV $iSIf"6G-/iHO/DZ>E`5eis&Vo'"ϠiDbT2?i3 U.XJB:;D"'h'BBM/A Q1{cSn:UW_D^-^:5U۩o4f a2cvݒd\Pg†nHhyjia( ZS (״Kx.\۷nһ^d~Ffq: ݄m9lP H+iA Qc];Wdx{Ͻ `E̒x (?pRiFg # B@ ndOa| mV.S6`QV?D}A`Mh7V|ITNit6](Pd2/Lrkk[àZb &RL,B!2ٚĐrݲ999 $`"dՕ7~Pq3ٖ]TOGOV0IW?=mt7hIU6S]=;gǞۗehJ{EӬzܸ|H+\i20+q(>J)8fGjmdfTi:U'ap@5SvQ)zw=O5p' ^$DWEbDew/Oz/oZ;h(KgoZ7I׎5ԷO3+IDqVQ.$tpkkZq(xמUJUݲNR)B]LffU֬psiWI)h}}k-01F=Hmec͌ Q>OtgGOQչiWmM$3D䆡,>[(d3O3{AXFNɀQm]! (P"f_x[dgV0\ir\{ 3JEiq ~r>ڬ -m=ܸlT)“f\o*l{ (f""Dn7]zhijjcfWW>tz||.B3zu mEЂ(hf$=ZX肊DT/u9_z%1y?<`m5$NNό$㤧׬x_ɖeHDajdd Xᔖ C/CZ}& |.`8nk:Bf #PD"YX9e_x5׭xs1M*R "سgUuLw=7][c怙" \Ѿ\nPZqIV=}W oPN?Pe;sV_{/}ID^G!_L@G{wnuP1lY oE# 4<$⡁jT?u4q9K7#b.=Qel|_?<(OmZd O ڻ(98D2O)4n\3z{#Qb)s{ǼBXYUhJGWqs/o?=Ȓ>p6|U>_::ٹ8w8?95X#!BvЎ]VhrUHbkYX( &gU{idd[]un;?gyyp,<0^|j`Z+cn Xo8;65GҌn9D8zVab%5=j(A88 bG?[]O|ѭQ;H2 y',aHo`yՉ y76 Ϣք6.;n? d5TrN1=T*+5-U<4˨qJBgZ$Khd5]!˯#];]Tp3'ǔZ3 *1?qW)Zxǟ3efJ 1pP2F,c ұr񿺶}{w]x֪5}G]>)177D[QgOvsa v'vT&^fC|swvX? (^)6t[$@H.]>WoO>̾ N{~-oJ5;WcRօk޵($!B`$~׹mMaٯb1P+wS4\w@>|R.Ƃxw$x7w`TL(y/[BsݚKݵXjRRZD(I%@FM8鉉+޼@_ӗ#S8o@c|, {l uʍB082SB ^,HʦiW4ͨ{o~Ոƨ#!8cZy B[y \ +Re#' AZB@@>/_1cW=,0#hq# (bestD(s[y9:\p.W/D/jM>f""ILbED187}}'1~G@x ~K)>6O_m l88px̕Q{rJ ‰O^4,](hM*a& 3scvsh$F,!؆ BIHy=V^Hx-d!(@v ;nXݪ۾~~q=E1!u-;}_Ը8^a_|IJ A(@IC ݩ4Y>pMwWUTl"I$b%&"ijBC9̗ wVcұLjkv3قc8k1 D=;O;:;F| @`I@h i `]^DsmOgMT*iR fMVJAkU=3t\Jxzۀ_~>]@R!/N阮R R=?)ܩlwo%6:6&յd[v!+bkm R QjuNLN:Q8fN;y\>/3/ufI jפۍ֦7kaX ! PP:8;nG|x+w;N.~=nzuͲnmAD$ff$:vlj|lfgO?:9~M^w.6|h A [(f);m' a℃_ ?;7[>Wf6gۯ}vMMMMmƅ^`fD `V BRxz<~l\S_Wg~u;Дtϛt. ? 3DE<4rԬTHHwO0[monm]׮sRLdaI `1X3Hkzյ;<e/>8] ҷ&גrٖNosϷK-60`f)IU+[Zfc,l 5>a gHfGO>ィOyuyjG4-/}xն7?鵿qԵj *ػg~n*ؿﱡ*&`=œKy;}2`&3K̬H֒fPsӓ3g~+>b_(JݫlMzz?~=zyrì։0Uܽ٫L\p̓`@H2|066|´BX XX\5:{]cqw vu3.]+5)"Je 8d۽6w$XΎ w/s7ʌ<|pFV*4v-)+qX9MBߵ.ym-1>sᔤ-m=-==:Θ` L#G/`q8C;N4X\ƏuԞ7`ϫT{G{7 Gq`̤48i.svǒ񅝯 rXR>[{.gp5wCCHVsL$ϼ<|6f~.[ey4-0k+Ws "n2df#ʱ0t"p͢^ZLP"JyzCkrڰ-Z.h5C4궟T[؆K=[B=:6P>F819Ԟvu!DܰȤܨeX+C뗊7$3KWip]BXnݰ깾Mמ Δ :)sclY`AD=k/{1uĻ-[?*QZkʤzͣGR.\y+Q ft5,䛌c:V-[㶶v[DpTˡCO]$[۴鶥8@XuO,yGŞ kz{J!QN^JNT3z{`X D0fRX.|G([RĊ@aĀA0M7 8%KV.[ȤOydm2e)˯?mf t\qu^Wnm~~Cd9}޶Ԁ[tqՓSZeS &F֭nXE۷m b S}hǷV=y㮿MǾsBtbYDRs,3+)hlGj toT.sM`MN:^~:NI {~?tj`bzw߿},clwy' 3k&"&!<$XBKG6?lQHL_||g&{O~ϵ}x~JH b%$  R8ڋ` Z;\Ye0d6߯ Vou-IP`1 rN:|9` 82EjDi]r߸~[p'+ciޏV>OK̰?9ӝIWi  +k4\7-tp#ЀajbbKYnIIp{XO Lt/ܼ}_YuKwlSUY̼DJ%  h-@V`ƫReYMM%ESe*\o.u`A?pWhZ֖i|׾{7/_r@aEiXLĬn fM,Kg*_ rMB`hf08}xjk5aȔ?ת6Hzrt>n*$%$I$YM4zeRKf !%Ha !$R##( 捗r➪gJxTgY2R& A~P7&A),Gs͖ a57+0 Y)߯'dcfv­R0h倅yMJ'aܰDe5|WsV+f$ِrtd 4M[nF yUsiT+\>q>37@HUФ ig`[ABt`aYvb@23y~@Sƙ[֏A- L¬0 "mzy6aXtRPݫ $LN^D`?21im3D 338Ł$"rq؆ t:y5( 2' "i TVLr^0_B,3){lIMӎ0T:ciB$Mθv$)=I -MeL&YeWIx|rY3S&]T`fYYkCȸwcBDTJH!F#yր.<őFƜd9׾DxvKk)z+="lTfwӳ2{8nK`Fϗff*I<geM P}J)08 |E(őBT#+.zw;Z}k妻\\-/n5+U(+2} noS og@ AY1ر]U `D 0w??߶lnqH1#e؁]Ul:,ZzoJ4B7#֫/I4 U)ENgRy# a nj@Z3 hXl2-}1믩LL ]7;{>vZs;??$kJٹl5yzzrn^W:k/ONYkZPY1'v֬3'Zk=_J38*Z*I]x=% f5T05PcY?Z+0( ȗմJGsrr$Cpݺ{5wݓyy&Kr'DeID #"+zW~O_;кyhTv7.;nh.<ֺdY+"J¨ju6eHkn'qm{}^ {U#-hХR%ke^]xp߲֐ǚlTޕϬ[d!v|a}nO#C?|dm۵W!4ɡT{[o9(Yo|cjL2 NH3s" 5$n*۶ks^z9uve3ق 0 Bdlt(Bgw.ٿޡ#m~I⦦֑~RZ3{n;ϫv L?ȹ EGa]7+3D2sy}tTIE}w57O1s@D^isRh>WTxsvm}OkVq311ܒđi09;I^g_[r3w?qó"K==MI̎9$RF$Dٙuk.ۛvfQ$?֡iKG^rF Ns" 0Jz{Wn×\z /;ٹt\JYS=yGpG߲8}MXi6̸tըwOg}EB$"҈();/ri7Wa13"f'Oj@۵z bDZj`h Q\V+c||BHH0L!` !<"Ç՞*Ey4t#4$kf!88G2lϴfWo۳eu>3QU -㥖JXNn8ooE|s]Y:v?y'ID†gycT,;E aˏl9~\Lnx J3S=v ׿feo|liAUED$qrr`_gl[loϚ(f" 5n×J&ݽW&ryg@ n/ xM'yA[Y"+{w/3h2pGn{)g_WΞ?=h$q8;}" x`xRRm|_>"No^dOܸڮv|D)7V<}RƻŒ(a;/Bh,uxκW|ys #=/" ;}˂SG֙'ܕ6Ub,0X(Vǎk. ?Kz~²\ tb/o3ДYR8R_Mٿzz%^duhƭSE~rɌrα Vm9u7ox^#.`sg\AUSnF3Qc`6k" =V;bDQybOḏV^r+>oq$ENC `zf>3zVs\r105#a`Q.`-oq'TZՓ8(5sk|y:)O'W3J'3aΰY ( XRnnDFatSA[n yV\q'󶐲vc7,WB":oӛ}7W@z{< x&{:?]'8Qr jydzKBkú+>z|reC+eC AVErm-=C=]Gq2s@ > G/*]U ~x `Yo-pI/^?8 ,fSӧgZkb),lөn~Xll*vSJ 3೦U{6܌w>t/]Phjk[ڽ&*J 63L"2HZ i@J#B$BH.pZ w ׽缏$~*>őLL I$GĈ)=O7QmJeԤ4kBH`hx $*׽o7HFG.>jK;R,aEsɇРF:@"&( (9K`XucI{*BUQݍ@4; MD%(fp " "D2FNb<3_:GUhCyISºe!L`!8(4D QF`Ylڟc;S_{޹\*׷2)˥~/.-Q@ҬTD̎'_z_'gl@eona ޵YJeSNcX%C=zUI g/!W?GοvsOn]5 ˚Kl9e-0b愕+8 g>A-H^7}9P!lA)TV鼐 Vq^5f[Gy|@?Z ȍIENDB`weboob-1.2/icons/webcontentedit.png000066400000000000000000000214641303450110500174670ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDC pHYs B(xtIME  ( tEXtCommentCreated with GIMPW IDATx{weWy;r9NOLOTIIZ +Cb,UmL A(B(k4yzsz龛z^ޚӷuϽ}w ~пUw 4HGS_\AB(D ˰- i%x?ςZp]{hXt-Ƕ@Tk~r<JF >#gRX,%!@@` f$QsjF! ` K i٢}/f3Fzͪ~Wvp`CX*w^q e䍥,Z~]5UbGK'sE+ͷGy_?8gO퇿>;Q}?  t +ضJ(IB ':V gOGޛ0r5rUŞnF!],VĠ{?޾Sfa\ynd+r#NVPʁɁDGjT4 -{܎JOBZ-HYsD>Ykii1/?]vڰRw@iرzժ.z|^8p:qxeW^6`xVR†ep2y,-׾>! yC7oQr6ߕ2$K=QwP.@f6aV Ԗj~;nU6]X8L7}="_vel_^dp&r `Y\݇G߃ o{6/ُޔa9^/큔`6"6`" $d<8!gYM2z?2eo]wWGn vF o;6%aYl;VN?_ tt< ƶ{!!J#3 g/i㛲٢3V:ذY @$@LÌDJ+Z-W8h5fVf+kR?ݤ<3N}`1GcR*҆xew0Xz^d1Rkxe$B6:!$JT;J=p2) B C XX ŧN)VՏ]S >(a!/>@ɡnAsOMĻޅ#X:~ L}~dVn@\@~!: N7h̀h a0f[V˙L:I@Y/x0cojO&?7k``=6a mlג`֜F[cE6ݖIʹ2 |GCMf6O|ypiez 1nq`WfɕM6GqfT:zZu'@;/[.䊙mzQ٬c"$BJu"Na$$G7rL;ZȬG6\\P$ _C}åA*+R')s ݻoo|+3 KK%zE$ xMprm abSiR;( B̫Glvۋ::z{{- ]-)Itť飽R` S놷(&@fJu֖wIG`\qv5r\۱e@.\˯Ek qܯCc@A51a]@0860`1ap*~{b+d֖I6Ib&"YKŮ=;x[H2˫3BI\#`&RV@K~RwH^FV@Rl^BTqv ¤)ϴ16XiDd(n9YXڼBHeLe`F! .߬fo3BOd9XYʕ?4;FTؕ$) j+Hc U2  c`\(4gdM!LYɡXorRcfH "(Lftd$Ύb׊f-JZ$ %[PȗR e,,̗(Hj S98NAYC _`Q2,L `, f"2Boouǵk@ <ks0lYj !H ADB)<:]݃|C+a1=;s;N:{vITCHYfc]# ,J]b !ْđ>m[*v4NI Y XA2fH|M)#*d[HXBhJ]&vZfRdz/rp'[l`;ݧB6G`?ag$`5Dd" g-@ J*sxA:pt .6;p-MZJ.+6F1kkqt' !H3K)ng h*IG@g !>mJf~^☈mِ+Ozoo] A Hk0qT3H`-٥GG xd6<<><~%9 !'^Z\1FcWΌ ՄX-Ԟψa#eщb6B);^%(\7yYf%◬زi$%=dކnȎt ^}W[X'>HRi¹(h L'z$dTt,K;_{|KXʀHHhs@ QR6m)fbmwm;]4ΣaIÐ3`$@QvUYf4 "6_Q|>ۨs9r}}:IXG1/OF=g[N#-hod$2< !(%`"fĎ SxZų)t^ݪH015@;Ӯ㫧pW߮:6J?['nKp;`%_c, 񵞎 O6f%i]riJ7UGwuDAz Mת\T$,˶ d_^{ˎb3ê6b$8@u[Tn )#d؀[_ӆ' %p.,O5b .@@0b> ffx ^>ց˜͙&+r(?=лcyF0nV]3Jן Nj x``ttWvD!N`*5׳39Kr@Bـi-hpl'/Zޫ;q:7F4s"=dL$]%E`ZMU]Ѭݠ`~po0 6ƘP\^|\j]gcP*MG08^@gKo0 ꦎ:9,"KD\m i_LJu÷7(n(>'p <4Y@՛Dd,]60±WKSq"aD_(%22 gQ Y;mG$3`0̜hMzGgf/ĺ5~ `6s|^4Zn;#(HtM! \p,J% ym{R$:(3FZSJ"ðV[ J$D[Vwbćo،cm{ "ն޵s݂ϸ@~H@Ke0FkUPX0,ײ, ֧Eof1UHVb͙k03?Z Bp bhi[/s荟w|7_?~) !!HA"Ig)$DPBl@A`bƯnَwDHAtu (5 *ĉÐW 1t[X\Ե$ze\?–7n4 $I;OB;/7qnkԀ D醩hk@ ~F6SdZp3\ 8x س  C>߉^OG|W{Ճjicw<^iVbFلEqVK $RԬ>kW{~H׀p{m޳o^q1٨V2t,)"qh:ec\ ٬6aDҁ͖Q,va~}ӧƳ[}6z1ii9Fv0Z&twWnlGOA*2i m$+m6)h-P;pӦ SptEd(8>s V3QbX0ٓqקf+m9賴{9hiqv1+_}scInp{vqS[\V# v$,Hl]+kF!)MC0ZDr0$`Yv<|Kggc剆x˯GlWB@.d3a⇵ƚ[ d3ɩ/5 cgſyѿb菱<|NN^d Sy+XOxp|(j"Wg<<=7 +Ӟ3HĉF>. yC)`@J@ 8YMo=$?c/_zKQ$V`џ&Ѫg΃ja'_}^0l5랅;z_y4\\D/;'VDKM&4*cTS`CF^{֣RΤ,gepbcSރ}`ٸb}է>+*\rY“Tr#GoaF_7݃|E6E>z{ xuS8ќAx{(oaR:,²<$У#ıeO/?7zu $:b!EDDA+i_j%#X`-tZj_=2Nڏ!۹pS^+W يUw| J?ŤY%]cerfuٛu"DAo;=DJtFX)Jyml?= U_{˥5*!!Pr_.q&tg+vLD= M$؜cln,"h,F8 ,~d~MNenW7VԪqci%^<15^=ҫ߬7s"eyph҂E\ !,8NWy]x8?~ffbSyf GԌй,wDI& Lwg{ҩ:ӳǟi^?ʤ i:]Uȉ.اÿN>;l:͆Q}Z&<5Zչ_^f.j,E5!z\; NCN^^Rpj )QK!oݰ$5?W'}3Ǚ68O"]^ŧQo-J]Cw{ve.Ň~ M&TA6W+/Ɲ܇z詰J˧fOm tPaE2 "CBhbłdӱRr37$Q@J68NI5#߯>?~&0&|vufx ?aC/D4a;TPG_[֟}!n9:ͤr&!"]wR.[\(*5 떰rS*6T&]7?7Fhs6[΃/wuϻ> ={zGrmW\IDAT?迼,I۲m+a[J9 )FA`V eEJY@v RL-/xs덼sskwD{O v˷j״PXRҲIYP*JƖұBaB"@`aa:<{z ĹڧsξTu|CEtX~\ѝapTl"I$b%&"ijBCNy { ʱL^:cyL1LdIƵڪ4~+fVq}vaf*dPͻn*+ 4$"A֪̚]V;\opS{ G|tLO)YD?~VዋpgWn-;eЛym.PJ! lʂ:{c16uMaH &@A(NpijP/E zoTl'un; 4HwY%I.\x]ZY/t?~A shrz3ZG NiI)C8 71WkW:0n=m_TK}wt@XE " `i0+b!):xy//<=zR H9C9Me a@T _`t"?o6Bhz7篼!vYw.GǶ]b0\0D2M` @& bKZuMrpʥkg82YL]/)a.W[`m2`2R )zTS$ #38T|<(nR2ur`Ե`{@򥨶* shno7' |lO1ZDd 4$$3$k-='͙L1nvj6;&Qomb~P,u8fED 08d # AF]9h]#P_ix8f^?̸0=dIA k'bfĂ(섭)3h7o<[?=}[VCCc@af[i6R.I"iڑ${LD fcQֿmz\ "ɘ4O]Зh9gQ]hf($u%ifv{ˣj1T*lUGzZ+N/$9v;{uBՋfG3 2,n M\ٶ}ߕL&ZC0ȸ?SQ.KN{0 Jd~&Yiرx!3+Md6[}ahw9M000VޱcZYfsr]37dvr` KzʬmזRRn @H6<۩gNOOK˲޲p@&ķw3ϬZmgL'NVJ1lm$/V{n2E==}&O&.yۍ^*~$?0ָ<6>Ѷ-Ȳm##ɕk߶I}cۺm";W=.8*qdE󦭀ia:_BTb@ Ar|Ͷ`DnmPxe}-~sn`{hpTji"J@o?<]J!{UJV 2,UiNR-.^FQ R0MW|g/̝_z՘AD1ӳ:FSJ) mQB]QqƚhpRwy o|n.[HmJ8S'ض6(v<}f֚ p֚T*۷'"zqвZ#i=dstM%TAoz`ae^+l+mf6d0L f>W44 Z\R!rL-)^Z\-Lf`}40L" Lr-:{Bobb l5^fT!ekw|Ssɺ5y5rV;ON_7B]OmKI@iHxՊk3#/gz$d3>wDGvi;e#"OkUu|nzt"@{ZG뫗J 5ժ0],Dald vF05gDR9 _.!dYJa )ٔ1 ({lEMmPب'*vNϧRf'Fc#wO=?2TJajj껉sbfn TKh-vIhԇe'Qgy>,,fө\y3N@$13  !X y#(SO@&-eY3L^Zv>'&vy Neihh8W\^M/ zl )LӴsg!l~+6 -Hҩ|i8H @=n4h};4 dT>Wb"4 2^8*UCL)`w2My䑮:P.7n䤑ZpftPT`fB7UYkCXR!u$#n9Ig'_]퉢L@jlƶk-U={&&&VQ fff@D`fH)a&݋v `m?_q,YA$IiX n{^L PE\JQ(R$hőxm8Yx NZ2ɋC_VIer]fJ)(^qӧOC\.I_)t7",ߐa q_n$Ibh f5TpV zhhb ̊ŋ/7[uҠVIS ̧ u;Sך}!_ZjWO(H B]$Hh6CZ駞LVj 4 ={ƚ{>tv/ bYu ivJvƺ"0hf۝&mؔ=F* qհor3 t@{fWb23'#}R![ZgWr#;J8\+ xT0tMK--E&/Je:y]kluqGGfnU* J)۶xBnO?թcO=,4kU"53()Uo?SZgaPVn*Y(#J"ʼnAt;jlvlV\^#+M{|agwz ԚUܨDP (%Zs2<2SQB:f !\4>!N̉ m؞dgfe dО d\zW"3رcxcTZ Fahswb"ĆaqT\%!̑iwǟ뫬nn-ڧN~l1D]J(86M+,*5!DBDEKfZ(E>逄=~DS"Ɂxo jgҹЖ[v oW&-ʏOazGfmuklZVVh{rfB5͚uˋ9$WO+͑#H$ O[$#X+fJ<'ΦK(A.v˶[}dk Wgna+8o`jj k WKhC->Oz}'7*I6V-7nJ'a[{ew9yCA+VѨ Td9҈z{GaqH/]8>9;w lBv)aXVkgr:A+N= O>'6O+Nu+Qb~ӱ / ۲)d m8h Q2> |8:jm{bTB]BMʣ1 IYsFgj˶( Uem - ~qhӗ?ѣG߸ҵ[_h~:i8Ry%mV24'?ee|Ezaww o>ǽ*3iE P.\ &PDBDD'f= Kp9pplKiq{12MJ;KW>8ӟ4(++SS77v_[7=lPW ) us/Vokl"4.$eVsp+>Lt#w5裏΄ah.,,H'_b7yg腅JT^[2V 1ED֑Fcmg'm؅=63cXn`u:ŎRBZHb$:;o7U$$~?On_'D6cؙ\ɵMB`I3'TXqp|2!v_; L^)7#TNHf8N3Y/_+L~IENDB`weboob-1.2/icons/weboob-config.png000066400000000000000000000171461303450110500171730ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME3!JqIDATx{kp]u޷>sx  AR|D)eڎ4ΣI:qNNz:6Iifuqt\q8G][8%Q2)FR")A/pq_ *%$-iO< 2d8xoJMdDA M ChH3GʱL^.o y|1LdYza7eV ܧaӹre~AiHID5eZA+U\߼6j#M~jP阞R R0k ?[w,1|m]eBIV˃YPҶBJ 8NS-9I4M6/_[bbSo?SZ7vNw/oaX ! PP:I4^Y~g?PV,w>Ǵa 53,Kԅ /+6+/,\}ʃy-#@kv|eg%Î Xfw˟ iw}̞sRԇ+!ȑ X"C0f4_U^Ea^ UgD}ڝrF aHRL _`t_&vefxx r8,0$"1R5inKA궻;q.cr}T;fZulkmL"2A|vE -IQ=bÿRT;)1! !@`38(%p@FDzV3r}ؐ4J<w8f^ ?ɻp @&!@H 8Ź̨mͿ|f+_ݐy;~+v>vV#̶֙鴼FceKp4D__&"HAI>^ )&b2fcCsߊ JVv ufHR_fk IN+\0pD̂˲{ccSWXk(&Nw \~]}NVV9]Iddv8^Jk-y֕P\6ʥw[0x<2o1eYK@qȹvߜy⎅W8(ɢ$>)ըl4Vrtl7d[gucǎI˲nZ G鉠 KFsy`^L'RDL, _7{LJ3GFw횥!I _|#^{5J)OUZ9J)m( vkZ#/_ʍ[$l5nt07˵L@etFD3i0lb\{nh2T yD/,̍ɣI5gܵkv]^'w[BVkp}Z`΢ůOEc{.Xˠ ԚNwY`r?]m\s~+Z׏?5 ڨ>w;à[#!ᑝ坓S= ,vc|5::z-=[J=w\!4ԤचQT["X\AS* (꺹\i[n~2D"u(J1K]{>{ltBj9"ʑSk6Fu) [e3`dX Ҷ^h,/_)$I$R0Mkshhi]wvիWyڹs- @Xꎏ=F/h pT*(>4V!ڜu}x7y[,s;g8SOj !}/d 99"`l<^$"XzqԲi}).Euҥ['v޳H}עVjF3l`fˬfX1m9効z},1${ey87Śᑉ60L" |v=;L)Z7寜ﴻeR:Ih]\6Lө5M:vs3SGVZRqPZ~4c$tDto|(j0I!,̰_~,Mґ/ˀGD4I |vgp3ǖ^%q, k;3dN~jEj#U*#Ie[rnoSsMK)eON\`ͥ!ouR$HJ6 zCʥAP8BE`[ۯ-ÙJ}ߕnoOzbɯ?c? 6RVѢ{"Օ]jrnA9O~ I<`d1f6]&H,HXeY3L^Y ^T u % 勵=AеV4#W\NMGuvLīCn+[5ir;Gy[ǰmf::m"@ZR ڕ8|ݯ&Xo`a`a%i쮮.*94!YZZ( ޭ2VT2B"28!{)vpF^v֯VzZ2"Dt\TH!1Ak n`[]ӲJe@n7svsPՁ0?ݭUfhf8vC8V <`;H2pym2d IӰ Y:@̚4X|3@PK*̦aq5T߱iH)ӧ6vlttlny×ۤ[V8.KkJeIAr/ȲD00 Li$X8*,j#uE\_[6 zͨFJIz0:~TQ}h$56nog03.'j\!"y̒2w `P IK>l VZWaA 0bƀA[MW}Q*# f$37TFζH`4옙} lGnByr$1RB.-^i+)ٲcL>??/4NIAVcúיYqh3t?C Ac-K !hkQXXU/ ® KƥW_R: :Sj t:S"ͼGZ0ϕC ҆ia_c@ m@´I@L@]7x^@N@D ;@ J.ffM#"U.AI$$)EO뎏L/h>D$lX 9Y|I eZ=}VWk=an|0؎̮HXA;:~4H:V,DV)7C{Dxx-megQfX,5::X_.sײ iw˵N6]r2M}uVm›yI/sr~)$t6mf aSp\WH!Fk]O$I`ޘ\?jvmp I0IKyۮ4a97<>455f6077"3CJ 4q!z= 6a~{Kd8OPYq-<p@$?ѢL.$ e(RLȴvzQ̬mzO< gBQXWűw7olnnN(t:PJA)Vy饗 @XĭůEr70n l8 ^:|.rƚ QX@ FL3BPoue]^J2;m:~|I5fIu 2yzlSn'O( ]'NTu$eA:Ľ6IkJ􃞴V@Xi- XKn\01,MTcSZCھ_$^m?;wg@onט9/% be{tddj(IcϵBVE C7u~UJnߖw|%_hٶjuGcˋ<11u:-^ߡY1"gϡϝ":O~y䓏ZjI92s`{KneVM8RFlD3?Qc˴SðҖK"_8gΜ[[ۘxǗ ꫆af>#֬vk b@(Ӛ% (֬Sf2,'{ӎ㷙9 8<{TlY1RF2j;2us/Y"B!>]D^ffç_@[0@.BbH)LfӴTX퍌Ln1Zo "֚c,Kz;)r̍W'KdF!R w0iٶ%#( 'ʼόaXwK`9z{d6yP`m;beJ=Sǡ_\R_dWsZƱٓfRJ,,,*'O<;7a=7[ڭ+_qϾm(qH!@Jꮬ]S>ECk5= S3{s[sEPq%"Ek YyRȈ0a? 9b3G!7bB>f(x|V}F.FsQXZ,ۍZ),q͍U:}J8f;yO~mڢk?481kgӲl~uD1P,6\7̬HmG^1EɑG}ϼnq011qs(fOGHwg^1.LfZq`aԊRY$ ǃ=4\.䫣wz0`p}|3+'ڍ Ӵw]+Hke23m%GNJD1U/U"w)9sF7 z衇 P9خDїYsi0Ҫi=O,o{c2۶'`,jWT*3[eUZ. 5iW$!d,v:ҕ+kQP(?ڵ{qqZZZpYgΜ7t>δXũkfLDi+,K8lwGw68]3 HR68rX+%[Fyqyneu4 s_Pu fɮWwC;,|brrunk׮iw9sfU1 GJW҆ie?9 ;Z+?CĐ`,@ PJS(T|$9ޱ+RJ2Mz,|%c歇z C;FLJgTm`f`[ae*u_t}k\/^|Oi! *-3%TCOR78ہt/ #F׮zzִٖ]H \H2mRJ(h~?+?{5p/M{˥Nvkn;za 욙UDȑİJ;v`|zd;?t5?60{v Ŏ õv+N8|uA̡ >oLuO|\!`W=A{hݎRT*;;@XE " `i0+b!)>&ٟ Ёē_Q+Wo{R"mNoPtYCf{;t / 0@:Q6k!4fz}&_Q=r{[;{V[pD6&X26Z5#_(GZk5)˱SCǟm/!~jMٮv7϶]a>, ddfRMW$ #۳*L8T}Lz;XV.+kU)R r8,0$|-,HA\lxÌ]ѷ|X(큾Ma{lkmL"2AX)'l){ 6bGT{YOlN]:=w}P(nte׷eKDdo=,RVB I烉a'S才5}x)]sR l-˱aD@dg2d>cK![/MvLw{-ޞmji H W$U-FGOTNwlBH! 4u:sXzt<6*۲|GgWj^59afg,PKӀe ` X803H{~݌JӸ ZYV& )!i6#]#G6,`YvwԺ;Ne9 DF\;՛uݣ?cb+zRbOCJ$2-2l&QQ̚LÎ'2ZKn۲ץ\g0?a [ǯhp;ݫ7nܭ,v 0rrNzvFG E-P͖Slinv2],Wۍ( T 'O0is3dM|3п1&j9l~x3=sمL28.$Z)@i H_[V~zRW{/7}ʹb ׯn11s4)u/D*k6AB|"^5W"˲ &kmON w~P(tITؤ0_ X!MgBf0bi+w6_gRl[{ XQSS=J'Ζ-W) GJrxYzb#;J"6`q3R"^ h-yXiYV̰@0kLБ˕r[W f+ԉ' g~Uv/P'>  4 qb֤-zert!_f"lJ cdh i 2wQr6[߬8՗+ HcBH RſN$3 ðȔ&XKmڻ5rfz0L.*G5jzjlDOVY_@/1z%cYs %"Hf,I2IiL],%HXrEi!zw}s7f TV]piO\0\JL'%Q8KZ Y ӰCfn%%DU5|O0\rS!%Ib6;U MVTF yu{U ~]>IRZ.99Ѳsa,'3sdYH MP;v]۲l̐ Bgs ˲㸊[E0̸hRzYItpoVcmd[n JA# L2 ^PA TW`N)5Z@´DALO$$8x7xe8+EW*;j ( DJb 3i^D( lX ڜ,t6[a& U4}0s@H!uW֚qJ߫e]']5M;R3 qϹ3vt#99=%Ib(3TOjO|&niZer$ʗ~T.dL$fL $H6ֆd9ฮB!^K?{cN2ٜw 8ϥÇ֊ty2A"qnyA Ie[8""b"RSӧ4|m$aϫY nR9Ygbi8 ju2QHJ3!ъ#I570.ΩYVMv#҈'sө\>V[**ϯ0>-gs$DL 0g ^;TKdKƮ޽qu9˵ LhRxj*U "(Tǚm@- C<_ݕ.# vʓZGkYxUuyv"DV:Ʈ~istaڂ`Q y}`|l"O~ "O/DG~ofz㇕wmvnaªRRBc>.DMo{)\Ĉ5sP~\?}6fmz(nEu,Gu_]?FDz|x CLVoZ"?C? taГ!G)@Lpw|;rAO?+_M%n~݋̈9q@0гq bDZjsED 2ɔU&ab&&\[wo뵇Ftm=-zgn|߽oy^PWNZyBȪ͓I sɃ?ֽ; oCËқYII չ&֪UO¹dZBЫA?+Sbjm9E$$#vǎN]>3|$a|9%K}~D&;] 0rIS{gw@낏nl6kS'_t;mOw[Nl~])uoZa. B 0KSΆ9f)= ?Q@DkxW|<ӽvO^W&wCX|XX͍ , Z)$VX72&f_M 8ٽsZX/E5Za;kѷ<]W( 9~jTvzj7՚}|f?V}GH7ۀ= 캃Kn*Zeqlif-г|ijX(ӟ%\@lVM鷿}÷޶UHՈD0;7f?Zn4l!NoZws5f4?=Ԏv< 7V m9v2L%֖f6Xk̬UJ$VG_9׬r'DMYx#y(_stH[6JT !T~tpTnQnh*_kom󔂕-+#d2Z $*Q={6\]siQ ׼*`5grw՗ Ufi $t:?64ENdFC8ʎ[.BY9~߷7N:HV+iôJ3o<]wo^Pm_ Чg˩E1n3&4@ _J˕F703Ap͐U'7<w ?eb5]/l8w izOb R1DS$Izn*;7׳a2}<&4旷%emz1ܷܾcL UN(v-t3C !"tjT~X.:mR!| @;ދ;}_{5&pV\lalO6Z i@J#B$BHh=>{3SS.x_ڻw}᏷suܸ7Ѵmẙe9 )͆8`FB@:}=G\ސ~rhTS IK~˴w7 $`$ 1qFD@@DaD7BƣS;7 &Af6Ci/g f#! Q BsUo,oY;2S̖ꁭ> aY5*sB BBaQ#?E} h6gppG2!/.[f4+$Q233sI}Om_ _^^xh+}t8 XW lv݌NcX%C=f]I ^_]>'@ws%!rYˮm:H 9a"tOˣ'q,Cق4mS8nVy!!0zv8Cq^۪IENDB`weboob-1.2/icons/wetboobs.png000066400000000000000000000210361303450110500162700ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME1?tEXtCommentCreated with GIMPW IDATx{geuwo~9M;33vD "(L)R,ɶTʖvY%v%d2 D"EX`񥛻<(z՛}'|yO 8Rd8}cp2_JLDAtL` XV{e ?.}8t/-˔esVw׀~<)$} .gXHp&=cg 3eK客f0r;ßu~3?5?>Ai3$]}1=! fRAZ³ӌ~v'KC0z5߿j[v1/J+ug>?sYP 2cml2 @$5 JID3ao|j?לTxoo徾>6m팫>N>XG2lܽ繣3 .,ɭoYnZ6lAJ3qf.W9H$ +թgY* kd7ӓߧ\n#J\6yŚ-̩>sI *`f̀Ӈ}PNE L73a}qO0L0 NU&_1y!4v~񅳼d>1)w{Fytl{4Dd1`5WS3#?j3w7AuYޕVʎuto'!Q  au,O3 So; n0]wkZݮ6lZ#ݻ0& J8;vkNխ9cp獅ZmX#vFWC"@fvT\M^r]0pmv~o|lP'_p!/nڼ73a $fhAv'e9g:B.[U=q[eK D.L6wpr``)lCq\~| ~f(8j``iy r:uͺ!=u mHC`iY;y|ϝFԡ~Ky0ngSuzhhsTb"Fg@fHA43\;]a5zz|~ޡj_zs-Z+1'$9v;?~xfs=Վ:M]H BQ qo:% g{+C )R & a 'ǡɬ4qZ*2 hyi([`ׇ-j02] fq>wnK,0kL"K;lTT޵Iuc).lMG.gfvA3Tb13 iĮiuo,zIwd+L8 <[#L?9*<9`׺kc:ɚ`x I/4t;((f[)ek̔rqɿhv/{/ %e#ax쉑\R|q"Hޞ '6 "IipZ)D+SvONb\^!MxYܹӽ/=+¢LӢ #[goغ9o0Ʊ\r{$BXތWeJ .l&"ծg:50  N045OJ$xh`J!8ʑqs.`|l0\+nٶ/R)WK/($׻~uLg2Ju?q.$%/4g1߻%v8{_nZZ): hl !mڱe9R*qXq9Z =;RWli`8l!aW`DSlP"r<̣;ZzڲmԶZIbA-̬S߾|JC5w};ZPޞEAUvrRkh\iyn>B=5}:0MЈ4;w'Bɬl..OY6aF_; yMZid8'۸/l?iV",|Ve 5/%E+l2,Cb+!"t3qVvviMޟnǚ Liث,p}tgfeilkL$ͥn!J nqx`fa>j^avZӬa05 +@!Fӛ<]H\B==g o*E-]͝J4M^yrK*ZuqɦYe"D$Hf,) l fe/o}%HXJ CfA;^RlťYQ_vaЁirBT)[7v6f/:0T] `q /2zpM.`Ɂ2 r;@X'XϬeΪ+ҩisG+Du5B|r$1RBNM-i+*\}iOg>ߖq mNZ:\mM0xKep{L@N>ƢNUS.(ͥZk YDQ`3tL̝8v"-4iC{3_Al˶r"3C2HZYY,,NU!e)ciiΪoV28":@s͋Dx rpY; ] ]`Wa4[˒ -7Ja+l0æ 0Mae2yFnJ!@m!0-;'Зaiƞk1DQ("p6, 9\ƽ wNk~oC+40Ixom۞S'2FafaQWf3C_* 8Eeλ@f&q'"Qh3RLt.W a$0-Gv,; #T&7%9 Vd酎Вh=8?ݸ JJ)ixpKw} N9RذL+^Z!#'Lӎ0ThiBB_9] DxX6JdQfX,߿?08?mϧ)@̙c5iZQ{ ՝vB\x9&m XEAYRX#Ss 3-O/NxҮGqR̝hffJR ʨ%YkCXV[R!*`sƜfsymw~dv1sW*= .,i;wa/Uuܪ(UVYI۬4a4vx$&"b"Rsg{4B)Pi7,=/ﳀ\4  &hH)S08NJH*űxbZOb[6f|38/}k6]Hv*k@YYK x Yy۸m1BdjT9}ױf!_m4d@SW >`"fĎ+Xa[ƒQ,0""j\H0k!H˳g_.[_.qx4{h3jdĩ X^9pMItM2Ο Ν?n1̬(H=wtHJg2BT:TbQ(l+8h0|u{;q@5+qzZB>37}iOR@qVL$'T%r+;vx,/zDNhj`$ օ|-ٳuBD NQۖ\Ь3ZkҘ08.k4WYk LfFY8q٪0($ (Jonn2KhN]k _8#U2M/{лΞkܴqLDRiI;h{Yk*3]CZQm4K!MRoIg@Y$촆֪L&$H$V~;_| 0K݇-νRcthx;cD\߶I2E޿6a~ tKgKJ1#9"f@|3}~дafTLM.hVi&۶hKU`~zfTЬPJ޸q׉g?RBDv}]hRTjf&PXp۟Ѻ گ?0qь.~T)ggWC}WU(q(`f&WA{ov~"7yqp5/$Qťi3|af3A~>p[Ҩ/DH5@j G6"! nG:N!iѣGLh.|`(*}*c:k^+EG^|csa* 3GQcǟ/լUlZD !Sg_!6-Y)ɗ8>Wp\.."X$DH }g?J CiZNPMs˕J$8O?R"? )bt>&K(\K|T6qod'aKC~6S2dHDVI)担lv \/Fl^MO˾ޡD3 6 r<2;;q~vv&Zrw`4e$؟ϟ+3?sMrf:C1Zk j~|fqi!!2&!b-nzs`V1i멙SeLkںA ZEf/> * `DMyjD(8Q(I#O  !Ƕ\+h9.a''wO*xH$DD"%I)i;{[ !:aF p40ZupI&ĊkC_ǟ{U"禁Ih.Z:fffsDBB" I'jk'O8c4qܖ,દN]4ƿ5ȄeJ+am=/eg}a!d3GDaӧ`L6Ti(}_l|7p@S'_tJ !;BLA yI`͌TJ4MzĉCe,^+NիDpaq>?uZˎ۲`iF47,CPJwl'D@.4Z fffRTΏn~_ۓH"/3~w c9cWm;hD^7'WE/UI"hf]dfVZ>—vuWȃ-++fZHU a57-o>Ow_}U(R|[h^F_d&aز\MD&F^;-jmfN~{~ ֖~S:H mV3wޙ^zx[J,^z]e6޴J h$DL$B)f>_زMt$܆F;b}E=?s6<.Ʊ%oao~8wՒV e]ˣ1?).Q@ҬT$/.ͤIl̿đY~rA?yQ]*놷 lnVg2ı3JC IT'~A4U/W'~_'~ t8_"3\ڦc !L0`SV*J88J}0ވwt,C4mS8nNz VI$~3`Sſo30`IENDB`weboob-1.2/man/000077500000000000000000000000001303450110500133745ustar00rootroot00000000000000weboob-1.2/man/boobank.1000066400000000000000000000240141303450110500150720ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBANK 1 "08 January 2017" "boobank 1\&.2" .SH NAME boobank \- manage bank accounts .SH SYNOPSIS .B boobank [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobank [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to list your bank accounts and get their balance, display accounts history and coming bank operations, and transfer money from an account to another (if available). .SS Supported websites: * afer (afer website) .br * alloresto (Allo Resto) .br * amazonstorecard (Amazon Store Card) .br * americanexpress (American Express) .br * amundi (amundi website) .br * apivie (Apivie) .br * axabanque (AXA Banque) .br * banqueaccord (Banque Accord) .br * banquepopulaire (Banque Populaire) .br * barclays (Barclays) .br * bforbank (BforBank) .br * bnporc (BNP Paribas) .br * boursorama (Boursorama) .br * bp (La Banque Postale) .br * bred (Bred) .br * caels (Crédit Agricole \- Epargne Longue des Salariés) .br * caissedepargne (Caisse d'Épargne) .br * carrefourbanque (Carrefour Banque) .br * cic (CIC) .br * citelis (Citélis) .br * citibank (Citibank) .br * cmb (Credit Mutuel de Bretagne) .br * cmso (Crédit Mutuel Sud\-Ouest) .br * cragr (Crédit Agricole) .br * creditcooperatif (Crédit Coopératif) .br * creditdunord (Crédit du Nord, Banque Courtois, Kolb, Nuger, Tarneaud, Société Marseillaise de Crédit) .br * creditdunordpee (Site de gestion du PEE du groupe Credit du nord) .br * creditmutuel (Crédit Mutuel) .br * delubac (Banque Delubac & Cie) .br * fortuneo (Fortuneo) .br * ganassurances (Groupama) .br * groupamaes (Groupama Épargne Salariale) .br * hsbc (HSBC France) .br * ing (ING Direct) .br * kiwibank (Kiwibank) .br * lcl (LCL) .br * n26 (Bank N26) .br * oney (Oney) .br * paypal (PayPal) .br * s2e (Épargne Salariale) .br * societegenerale (Société Générale) .br * spirica (Spirica) .br * vicseccard (Victoria's Secret Angel Card) .br * wellsfargo (Wells Fargo) .br * yomoni (Yomoni) .SH BOOBANK COMMANDS .TP \fBadvisor\fR .br Display advisors. .TP \fBbudgea\fR \fIUSERNAME\fR \fIPASSWORD\fR .br Export your bank accounts and transactions to Budgea. .br .br Budgea is an online web and mobile application to manage your bank .br accounts. To avoid giving your credentials to this service, you can use .br this command. .br .br https://www.budgea.com .TP \fBcoming\fR \fIID\fR [\fIEND_DATE\fR] .br Display future transactions. .br .br If END_DATE is supplied, show all transactions until this date. .br .br Default is limited to 10 results. .TP \fBhistory\fR \fIID\fR [\fIEND_DATE\fR] .br Display history of transactions. .br .br If END_DATE is supplied, list all transactions until this date. .br .br Default is limited to 10 results. .TP \fBinvestment\fR \fIID\fR .br Display investments of an account. .TP \fBlist\fR [\-\fIU\fR] .br List accounts. .br Use \-U to disable sorting of results. .TP \fBtransfer\fR \fIACCOUNT\fR [\fIRECIPIENT\fR \fIAMOUNT\fR [\fIREASON\fR]] .br Make a transfer beetwen two account .br \- ACCOUNT the source account .br \- RECIPIENT the recipient .br \- AMOUNT amount to transfer .br \- REASON reason of transfer .br .br If you give only the ACCOUNT parameter, it lists all the .br available recipients for this account. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (account_list, advisor_list, csv, htmltable, investment_list, json, json_line, multiline, ofx, ops_list, pretty_qif, qif, recipient_list, simple, table, transfer, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon, Christophe Benz .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobank weboob-1.2/man/boobathon.1000066400000000000000000000177011303450110500154370ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBATHON 1 "08 January 2017" "boobathon 1\&.2" .SH NAME boobathon \- participate in a Boobathon .SH SYNOPSIS .B boobathon [\-dqv] [\-b \fIbackends\fR] [\-cnfs] \fIboobathon\fR .br .B boobathon [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to participate to a Boobathon. .SS Supported websites: * dlfp (Da Linux French Page news website) .br * mediawiki (Wikis running MediaWiki, like Wikipedia) .br * redmine (The Redmine project management web application) .SH BOOBATHON COMMANDS .TP \fBaddtask\fR \fIBACKEND\fR \fICAPABILITY\fR .br Add a new task. .TP \fBcancel\fR .br Cancel the current task. .TP \fBclose\fR \fIWINNER\fR .br Close the event and set the winner. .TP \fBdone\fR .br Set the current task as done. .TP \fBedit\fR [event | me] .br Edit information about you or about event. .TP \fBinfo\fR .br Display information about this event. .TP \fBjoin\fR .br Join this event. .TP \fBleave\fR .br Leave this event. .TP \fBmembers\fR .br Display members information. .TP \fBprogress\fR .br Display progress of members. .TP \fBremtask\fR \fITASK_ID\fR .br Remove a task. .TP \fBstart\fR [\fITASK_ID\fR] .br Start a task. If you don't give a task ID, the first available .br task will be taken. .TP \fBtasks\fR .br Display all tasks of members. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2011-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobathon weboob-1.2/man/boobcoming.1000066400000000000000000000212061303450110500155750ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBCOMING 1 "08 January 2017" "boobcoming 1\&.2" .SH NAME boobcoming \- see upcoming events .SH SYNOPSIS .B boobcoming [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobcoming [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to see upcoming events. .SS Supported websites: * agendaculturel (agendaculturel website) .br * agendadulibre (agendadulibre website) .br * allocine (AlloCiné French cinema database service) .br * biplan (lebiplan.org website) .br * centquatre (centquatre website) .br * hybride (hybride website) .br * pariskiwi (ParisKiwi website) .br * razibus (site annonçant les évènements attendus par les punks a chiens) .br * residentadvisor (residentadvisor website) .br * senscritique (senscritique website) .br * sueurdemetal (SueurDeMetal French concerts list website) .SH BOOBCOMING COMMANDS .TP \fBattends\fR \fIID\fR1 [\fIID\fR2 \fIID\fR3 ...] .br Register as participant of an event. .br ID is the identifier of the event. .TP \fBexport\fR \fIFILENAME\fR [\fIID\fR1 \fIID\fR2 \fIID\fR3 ...] .br ID is the identifier of the event. If no ID every events are exported .br .br FILENAME is where to write the file. If FILENAME is '\-', the file is written to stdout. .br .br Export event in ICALENDAR format .TP \fBinfo\fR \fIID\fR .br Get information about an event. .TP \fBlist\fR [\fIPATTERN\fR]List upcoming events, pattern can be an english or french week day, 'today' or a date (dd/mm/yy[yy]) .br .br Default is limited to 10 results. .TP \fBload\fR [query name]without query name : list loadable queries .br with query name laod query .br .br Default is limited to 10 results. .TP \fBsearch\fR .br search for an event. Parameters interactively asked .br .br Default is limited to 10 results. .TP \fBunattends\fR \fIID\fR1 [\fIID\fR2 \fIID\fR3 ...] .br Unregister you participation for an event. .br ID is the identifier of the event. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, ical_formatter, json, json_line, multiline, simple, simple_upcoming, table, upcoming, upcoming_list, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Bezleputh .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobcoming weboob-1.2/man/boobill.1000066400000000000000000000217041303450110500151040ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBILL 1 "08 January 2017" "boobill 1\&.2" .SH NAME boobill \- get/download documents and bills .SH SYNOPSIS .B boobill [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobill [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to get/download documents and bills. .SS Supported websites: * amazon (Amazon) .br * ameli (Ameli website: French Health Insurance) .br * amelipro (Ameli website: French Health Insurance for Professionals) .br * bouygues (Bouygues Télécom French mobile phone provider) .br * edf (EDF) .br * freemobile (Free Mobile website) .br * gdfsuez (GDF\-Suez French energy provider) .br * ing (ING Direct) .br * ldlc (ldlc website) .br * leclercmobile (Leclerc Mobile website) .br * materielnet (Materiel.net) .br * nettokom (Nettokom website) .br * onlinenet (Online.net) .br * orange (Orange French mobile phone provider) .br * ovh (Ovh) .br * poivy (Poivy website) .br * trainline (trainline website) .SH BOOBILL COMMANDS .TP \fBbalance\fR [\fIID\fR] .br Get balance of subscriptions. .br If no ID given, display balance of all backends. .TP \fBbills\fR [\fIID\fR] .br Get the list of bills documents for subscriptions. .br If no ID given, display bills of all backends .br .br Default is limited to 10 results. .TP \fBdetails\fR [\fIID\fR] .br Get details of subscriptions. .br If no ID given, display all details of all backends. .TP \fBdocuments\fR [\fIID\fR] .br Get the list of documents for subscriptions. .br If no ID given, display documents of all backends .br .br Default is limited to 10 results. .TP \fBdownload\fR [\fIID\fR | all] [\fIFILENAME\fR] .br download ID [FILENAME] .br .br download the document .br id is the identifier of the document (hint: try documents command) .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .br .br download all [ID] .br .br You can use special word "all" and download all documents of .br subscription identified by ID. .br If Id not given, download documents of all subscriptions. .TP \fBhistory\fR [\fIID\fR] .br Get the history of subscriptions. .br If no ID given, display histories of all backends. .br .br Default is limited to 10 results. .TP \fBsubscriptions\fR .br List all subscriptions. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, subscriptions, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Florent Fourcot .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobill weboob-1.2/man/booblyrics.1000066400000000000000000000172431303450110500156340ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBLYRICS 1 "08 January 2017" "booblyrics 1\&.2" .SH NAME booblyrics \- search and display song lyrics .SH SYNOPSIS .B booblyrics [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B booblyrics [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for song lyrics on various websites. .SS Supported websites: * lyricsdotcom (Lyrics.com lyrics website) .br * lyricsmode (Lyrics.com lyrics website) .br * lyricsplanet (Lyricsplanet.com song lyrics website) .br * paroles2chansons (Paroles2chansons.com song lyrics website) .br * parolesmania (Paroles Mania lyrics website) .br * parolesmusique (paroles\-musique lyrics website) .br * parolesnet (paroles.net lyrics website) .SH BOOBLYRICS COMMANDS .TP \fBget\fR \fIID\fR .br Display lyrics of the song. .TP \fBsearch\fR [artist | song] [\fIPATTERN\fR] .br Search lyrics by artist name or by song title. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, lyrics_get, lyrics_list, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/booblyrics weboob-1.2/man/boobmsg.1000066400000000000000000000223271303450110500151140ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBMSG 1 "08 January 2017" "boobmsg 1\&.2" .SH NAME boobmsg \- send and receive message threads .SH SYNOPSIS .B boobmsg [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobmsg [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to send messages on various websites and to display message threads and contents. .SS Supported websites: * aum ("Adopte un Mec" French dating website) .br * bnporc (BNP Paribas) .br * bouygues (Bouygues Télécom French mobile phone provider) .br * dlfp (Da Linux French Page news website) .br * feedly (handle the popular RSS reading service Feedly) .br * fourchan (4chan image board) .br * guerrillamail (GuerrillaMail temp mailbox) .br * happn (Happn dating mobile application) .br * hds (Histoires de Sexe French erotic novels) .br * inrocks (Les Inrocks French news website) .br * lefigaro (Le Figaro French newspaper website) .br * liberation (Libération newspaper website) .br * mailinator (mailinator temp mailbox) .br * minutes20 (20 Minutes French newspaper website) .br * newsfeed (Loads RSS and Atom feeds from any website) .br * okc (OkCupid) .br * openedx (Discussions on OpenEDX\-powered coursewares) .br * orange (Orange French mobile phone provider) .br * ovs (OnVaSortir website. Handles private messages only) .br * phpbb (phpBB forum) .br * playme (PlayMe dating mobile application) .br * presseurop (Presseurop website) .br * sfr (SFR French mobile phone provider) .br * tapatalk (Tapatalk\-compatible sites) .br * taz (Taz newspaper website) .br * tinder (Tinder dating mobile application) .br * twitter (twitter website) .SH BOOBMSG COMMANDS .TP \fBexport_all\fR .br Export All threads .TP \fBexport_thread\fR \fIID\fR .br Export the thread identified by ID .TP \fBlist\fR .br Display all threads. .br .br Default is limited to 10 results. .TP \fBphotos\fR \fIID\fR .br Display photos of a profile .TP \fBpost\fR \fIRECEIVER\fR@\fIBACKEND\fR[,\fIRECEIVER\fR@\fIBACKEND\fR[...]] [\fITEXT\fR] .br Post a message to the specified receivers. .br Multiple receivers are separated by a comma. .br .br If no text is supplied on command line, the content of message is read on stdin. .TP \fBprofile\fR \fIID\fR .br Display a profile .TP \fBshow\fR \fIMESSAGE\fR .br Read a message .TP \fBstatus\fR .br Display status information about a backend. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH BOOBMSG OPTIONS .TP \fB\-E\fR, \fB\-\-accept\-empty\fR Send messages with an empty body. .TP \fB\-t TITLE\fR, \fB\-\-title=TITLE\fR For the "post" command, set a title to message .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (atom, csv, htmltable, json, json_line, msg, msglist, multiline, profile, simple, table, webkit, xhtml) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Christophe Benz .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobmsg weboob-1.2/man/boobooks.1000066400000000000000000000164311303450110500153000ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBOOKS 1 "08 January 2017" "boobooks 1\&.2" .SH NAME boobooks \- manage rented books .SH SYNOPSIS .B boobooks [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobooks [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to list your books rented or booked at the library, book and search new ones, get your booking history (if available). .SS Supported websites: * champslibres (Champs Libres (Rennes) Library) .br * opacwebaloes (Aloes Library software) .SH BOOBOOKS COMMANDS .TP \fBrenew\fR \fIID\fR .br Renew a book .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, rented_list, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Jeremy Monnet .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobooks weboob-1.2/man/boobsize.1000066400000000000000000000200561303450110500152750ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBSIZE 1 "08 January 2017" "boobsize 1\&.2" .SH NAME boobsize \- display sensors and gauges values .SH SYNOPSIS .B boobsize [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobsize [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to display various sensors and gauges values. .SS Supported websites: * dresdenwetter (Private wetter station Dresden) .br * jcvelaux (City bike renting availability information. Cities: Paris, Rouen, Toulouse, Luxembourg, Valence, Stockholm, Goteborg, Santander, Amiens, Lillestrom, Mulhouse, Lyon, Ljubljana, Seville, Namur, Nancy, Creteil, Bruxelles\-Capitale, Cergy\-Pontoise, Vilnius, Toyama, Kazan, Marseille, Nantes, Besancon) .br * mareeinfo (Un module qui permet d' aller a la pêche aux moules totalement informé) .br * sachsen (Level of Sachsen river) .br * vlille (Lille bike renting availability information) .SH BOOBSIZE COMMANDS .TP \fBdetails\fR \fIGAUGE_ID\fR .br Display details of all sensors of the gauge. .TP \fBhistory\fR \fISENSOR_ID\fR .br Get history of a specific sensor (use 'search' to find a gauge, and sensors GAUGE_ID to list sensors attached to the gauge). .TP \fBlast_sensor_measure\fR \fISENSOR_ID\fR .br Get last measure of a sensor. .TP \fBsearch\fR [\fIPATTERN\fR] .br Display all gauges. If PATTERN is specified, search on a pattern. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, gauge_list, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Florent Fourcot .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/Boobsize weboob-1.2/man/boobtracker.1000066400000000000000000000213511303450110500157550ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH BOOBTRACKER 1 "08 January 2017" "boobtracker 1\&.2" .SH NAME boobtracker \- manage bug tracking issues .SH SYNOPSIS .B boobtracker [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B boobtracker [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to create, edit, view bug tracking issues. .SS Supported websites: * github (GitHub issues tracking) .br * redmine (The Redmine project management web application) .SH BOOBTRACKER COMMANDS .TP \fBattach\fR \fIISSUE\fR \fIFILENAME\fR .br Attach a file to an issue (Not implemented yet). .TP \fBcomment\fR \fIISSUE\fR [\fITEXT\fR] .br Comment an issue. If no text is given, enter it in standard input. .TP \fBedit\fR \fIISSUE\fR [\fIKEY\fR [\fIVALUE\fR]] .br Edit an issue. .br If you are not in interactive mode, you can use these parameters: .br \-\-title TITLE .br \-\-assignee ASSIGNEE .br \-\-target\-version VERSION .br \-\-category CATEGORY .br \-\-status STATUS .TP \fBget\fR \fIISSUE\fR .br Get an issue and display it. .TP \fBlogtime\fR \fIISSUE\fR \fIHOURS\fR [\fITEXT\fR] .br Log spent time on an issue. .TP \fBpost\fR \fIPROJECT\fR .br Post a new issue. .br .br If you are not in interactive mode, you can use these parameters: .br \-\-title TITLE .br \-\-assignee ASSIGNEE .br \-\-target\-version VERSION .br \-\-category CATEGORY .br \-\-status STATUS .TP \fBremove\fR \fIISSUE\fR .br Remove an issue. .TP \fBsearch\fR \fIPROJECT\fR .br List issues for a project. .br .br You can use these filters from command line: .br \-\-author AUTHOR .br \-\-title TITLE_PATTERN .br \-\-assignee ASSIGNEE .br \-\-target\-version VERSION .br \-\-category CATEGORY .br \-\-status STATUS .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH BOOBTRACKER OPTIONS .TP \fB\-\-author=AUTHOR\fR .TP \fB\-\-title=TITLE\fR .TP \fB\-\-assignee=ASSIGNEE\fR .TP \fB\-\-target\-version=VERSION\fR .TP \fB\-\-tracker=TRACKER\fR .TP \fB\-\-category=CATEGORY\fR .TP \fB\-\-status=STATUS\fR .TP \fB\-\-priority=PRIORITY\fR .TP \fB\-\-start=START\fR .TP \fB\-\-due=DUE\fR .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, issue_info, issues_list, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2011-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/boobtracker weboob-1.2/man/cineoob.1000066400000000000000000000320731303450110500151010ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH CINEOOB 1 "08 January 2017" "cineoob 1\&.2" .SH NAME cineoob \- search movies and persons around cinema .SH SYNOPSIS .B cineoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B cineoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for movies and persons on various cinema databases , list persons related to a movie, list movies related to a person and list common movies of two persons. .SS Supported websites: * allocine (AlloCiné French cinema database service) .br * attilasub ("Attila's Website 2.0" French subtitles) .br * btmon (BTMon BitTorrent database) .br * cpasbien (Cpasbien Torrents BitTorrent tracker) .br * gazelle (Gazelle\-based BitTorrent trackers) .br * imdb (Internet Movie Database service) .br * kickass (Kickass Torrents BitTorrent tracker) .br * opensubtitles (Opensubtitles subtitle website) .br * piratebay (The Pirate Bay BitTorrent tracker) .br * podnapisi (Podnapisi movies and tv series subtitle website) .br * t411 (T411 BitTorrent tracker) .br * torrentz (Torrentz Search Engine.) .br * tvsubtitles (Tvsubtitles subtitle website) .SH CINEOOB COMMANDS .TP \fBbiography\fR person\fI_ID\fR .br Show the complete biography of a person. .TP \fBcasting\fR movie\fI_ID\fR [\fIROLE\fR] .br List persons related to a movie. .br If ROLE is given, filter by ROLE .TP \fBfilmography\fR person\fI_ID\fR [\fIROLE\fR] .br List movies of a person. .br If ROLE is given, filter by ROLE .TP \fBgetfile_subtitle\fR subtitle\fI_ID\fR [\fIFILENAME\fR] .br Get the subtitle or archive file. .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .TP \fBgetfile_torrent\fR \fIID\fR [\fIFILENAME\fR] .br Get the .torrent file. .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .TP \fBinfo_movie\fR movie\fI_ID\fR .br Get information about a movie. .TP \fBinfo_person\fR person\fI_ID\fR .br Get information about a person. .TP \fBinfo_subtitle\fR subtitle\fI_ID\fR .br Get information about a subtitle. .TP \fBinfo_torrent\fR \fIID\fR .br Get information about a torrent. .TP \fBmovies_in_common\fR person\fI_ID\fR person\fI_ID\fR .br Get the list of common movies between two persons. .TP \fBpersons_in_common\fR movie\fI_ID\fR movie\fI_ID\fR .br Get the list of common persons between two movies. .TP \fBreleases\fR movie\fI_ID\fR [\fICOUNTRY\fR] .br Get releases dates of a movie. .br If COUNTRY is given, show release in this country. .TP \fBsearch_movie\fR [\fIPATTERN\fR] .br Search movies. .br .br Default is limited to 10 results. .TP \fBsearch_movie_subtitle\fR language movie\fI_ID\fR .br Search subtitles of movie_ID. .br .br Language Abbreviation .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br Arabic ar Esperanto eo Irish ga Russian ru .br Afrikaans af Estonian et Italian it Serbian sr .br Albanian sq Filipino tl Japanese ja Slovak sk .br Armenian hy Finnish fi Kannada kn Slovenian sl .br Azerbaijani az French fr Korean ko Spanish es .br Basque eu Galician gl Latin la Swahili sw .br Belarusian be Georgian ka Latvian lv Swedish sv .br Bengali bn German de Lithuanian lt Tamil ta .br Bulgarian bg Greek gr Macedonian mk Telugu te .br Catalan ca Gujarati gu Malay ms Thai th .br Chinese zh Haitian ht Maltese mt Turkish tr .br Croatian hr Hebrew iw Norwegian no Ukrainian uk .br Czech cz Hindi hi Persian fa Urdu ur .br Danish da Hungaric hu Polish pl Vietnamese vi .br Dutch nl Icelandic is Portuguese pt Welsh cy .br English en Indonesian id Romanian ro Yiddish yi .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br .br Default is limited to 10 results. .TP \fBsearch_movie_torrent\fR movie\fI_ID\fR .br Search torrents of movie_ID. .br .br Default is limited to 10 results. .TP \fBsearch_person\fR [\fIPATTERN\fR] .br Search persons. .br .br Default is limited to 10 results. .TP \fBsearch_subtitle\fR language [\fIPATTERN\fR] .br Search subtitles. .br .br Language Abbreviation .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br Arabic ar Esperanto eo Irish ga Russian ru .br Afrikaans af Estonian et Italian it Serbian sr .br Albanian sq Filipino tl Japanese ja Slovak sk .br Armenian hy Finnish fi Kannada kn Slovenian sl .br Azerbaijani az French fr Korean ko Spanish es .br Basque eu Galician gl Latin la Swahili sw .br Belarusian be Georgian ka Latvian lv Swedish sv .br Bengali bn German de Lithuanian lt Tamil ta .br Bulgarian bg Greek gr Macedonian mk Telugu te .br Catalan ca Gujarati gu Malay ms Thai th .br Chinese zh Haitian ht Maltese mt Turkish tr .br Croatian hr Hebrew iw Norwegian no Ukrainian uk .br Czech cz Hindi hi Persian fa Urdu ur .br Danish da Hungaric hu Polish pl Vietnamese vi .br Dutch nl Icelandic is Portuguese pt Welsh cy .br English en Indonesian id Romanian ro Yiddish yi .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br .br Default is limited to 10 results. .TP \fBsearch_torrent\fR [\fIPATTERN\fR] .br Search torrents. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, movie_info, movie_list, movie_releases, multiline, person_bio, person_info, person_list, simple, subtitle_info, subtitle_list, table, torrent_info, torrent_list, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/cineoob weboob-1.2/man/comparoob.1000066400000000000000000000166271303450110500154530ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH COMPAROOB 1 "08 January 2017" "comparoob 1\&.2" .SH NAME comparoob \- compare products .SH SYNOPSIS .B comparoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B comparoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to compare products. .SS Supported websites: * lacentrale (Vehicule prices at LaCentrale.fr) .br * prixcarburants (French governement website to compare fuel prices) .SH COMPAROOB COMMANDS .TP \fBinfo\fR \fIID\fR .br Get information about a product. .TP \fBprices\fR [\fIPATTERN\fR] .br Display prices for a product. If a pattern is supplied, do not prompt .br what product to compare. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, price, prices, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/comparoob weboob-1.2/man/cookboob.1000066400000000000000000000172771303450110500152710ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH COOKBOOB 1 "08 January 2017" "cookboob 1\&.2" .SH NAME cookboob \- search and consult recipes .SH SYNOPSIS .B cookboob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B cookboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for recipes on various websites. .SS Supported websites: * 750g (750g French recipe website) .br * allrecipes (Allrecipes English recipe website) .br * cuisineaz (Cuisine AZ French recipe website) .br * marmiton (Marmiton French recipe website) .br * supertoinette (Super Toinette, la cuisine familiale French recipe website) .SH COOKBOOB COMMANDS .TP \fBexport\fR \fIID\fR [\fIFILENAME\fR] .br Export the recipe to a KRecipes XML file .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .TP \fBinfo\fR \fIID\fR .br Get information about a recipe. .TP \fBsearch\fR [\fIPATTERN\fR] .br Search recipes. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, recipe_info, recipe_list, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/cookboob weboob-1.2/man/flatboob.1000066400000000000000000000172211303450110500152510ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH FLATBOOB 1 "08 January 2017" "flatboob 1\&.2" .SH NAME flatboob \- search for housing .SH SYNOPSIS .B flatboob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B flatboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to search for housing. .SS Supported websites: * entreparticuliers (entreparticuliers.com website) .br * explorimmo (explorimmo website) .br * leboncoin (search house on leboncoin website) .br * logicimmo (logicimmo website) .br * pap (French housing website) .br * seloger (French housing website) .SH FLATBOOB COMMANDS .TP \fBinfo\fR \fIID\fR .br Get information about a housing. .TP \fBload\fR [query name]without query name : list loadable queries .br with query name laod query .br .br Default is limited to 10 results. .TP \fBsearch\fR .br Search for housing. Parameters are interactively asked. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, housing, housing_list, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/flatboob" .SH SEE ALSO Home page: http://weboob.org/applications/flatboob weboob-1.2/man/galleroob.1000066400000000000000000000175451303450110500154400ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH GALLEROOB 1 "08 January 2017" "galleroob 1\&.2" .SH NAME galleroob \- browse and download web image galleries .SH SYNOPSIS .B galleroob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B galleroob [\-\-help] [\-\-version] .SH DESCRIPTION .LP galleroob browses and downloads web image galleries .SS Supported websites: * batoto (Batoto manga reading website) .br * eatmanga (EatManga manga reading website) .br * ehentai (E\-Hentai galleries) .br * imgur (imgur image upload service) .br * izneo (Izneo digital comics) .br * mangafox (Manga Fox manga reading website) .br * mangago (Mangago manga reading site) .br * mangahere (Manga Here manga reading website) .br * mangareader (MangaReader manga reading website) .br * simplyreadit (SimplyReadIt manga reading website) .SH GALLEROOB COMMANDS .TP \fBdownload\fR \fIID\fR [\fIFIRST\fR [\fIFOLDER\fR]] .br Download a gallery. .br .br Begins at page FIRST (default: 0) and saves to FOLDER (default: title) .TP \fBinfo\fR \fIID\fR .br Get information about a gallery. .TP \fBsearch\fR \fIPATTERN\fR .br List galleries matching a PATTERN. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, gallery_list, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2011-2014 Noé Rubinstein .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/galleroob weboob-1.2/man/geolooc.1000066400000000000000000000162551303450110500151160ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH GEOLOOC 1 "08 January 2017" "geolooc 1\&.2" .SH NAME geolooc \- geolocalize IP addresses .SH SYNOPSIS .B geolooc [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B geolooc [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to geolocalize IP addresses. .SS Supported websites: * geolocip (GeolocIP IP addresses geolocation service) .br * ipapi (IP\-API Geolocation API) .br * ipinfodb (IPInfoDB IP addresses geolocation service) .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/geolooc weboob-1.2/man/handjoob.1000066400000000000000000000163661303450110500152560ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH HANDJOOB 1 "08 January 2017" "handjoob 1\&.2" .SH NAME handjoob \- search for a job .SH SYNOPSIS .B handjoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B handjoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to search for a job. .SS Supported websites: * adecco (adecco website) .br * apec (apec website) .br * cci (cci website) .br * indeed (indeed website) .br * linuxjobs (linuxjobs website) .br * lolix (Lolix French free software employment website) .br * manpower (manpower website) .br * monster (monster website) .br * popolemploi (Pole Emploi website) .br * regionsjob (regionsjob website) .SH HANDJOOB COMMANDS .TP \fBinfo\fR \fIID\fR .br Get information about an advert. .TP \fBsearch\fR \fIPATTERN\fR .br Search for an advert matching a PATTERN. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR .br advanced search .br .br Search for an advert matching to advanced filters. .br .br Default is limited to 10 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, job_advert, job_advert_list, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Bezleputh .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/handjoob weboob-1.2/man/havedate.1000066400000000000000000000214701303450110500152430ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH HAVEDATE 1 "08 January 2017" "havedate 1\&.2" .SH NAME havedate \- interact with dating websites .SH SYNOPSIS .B havedate [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B havedate [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to interact with various dating websites and to optimize seduction algorithmically. .SS Supported websites: * aum ("Adopte un Mec" French dating website) .br * happn (Happn dating mobile application) .br * okc (OkCupid) .br * playme (PlayMe dating mobile application) .br * tinder (Tinder dating mobile application) .SH HAVEDATE COMMANDS .TP \fBevents\fR .br Display dating events. .TP \fBexport_all\fR .br Export All threads .TP \fBexport_thread\fR \fIID\fR .br Export the thread identified by ID .TP \fBlist\fR .br Display all threads. .br .br Default is limited to 10 results. .TP \fBoptim\fR [list | start | edit | stop] \fIBACKEND\fR [\fIOPTIM\fR1 [\fIOPTIM\fR2 ...]] .br All dating backends offer optimization services. This command can be .br manage them. .br Use * us BACKEND value to apply command to all backends. .br .br Commands: .br * list list all available optimizations of a backend .br * start start optimization services on a backend .br * edit configure an optimization service for a backend .br * stop stop optimization services on a backend .TP \fBphotos\fR \fIID\fR .br Display photos of a profile .TP \fBpost\fR \fIRECEIVER\fR@\fIBACKEND\fR[,\fIRECEIVER\fR@\fIBACKEND\fR[...]] [\fITEXT\fR] .br Post a message to the specified receivers. .br Multiple receivers are separated by a comma. .br .br If no text is supplied on command line, the content of message is read on stdin. .TP \fBprofile\fR \fIID\fR .br Display a profile .TP \fBquery\fR \fIID\fR .br Send a query to someone. .TP \fBshow\fR \fIMESSAGE\fR .br Read a message .TP \fBstatus\fR .br Display status information about a backend. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH HAVEDATE OPTIONS .TP \fB\-E\fR, \fB\-\-accept\-empty\fR Send messages with an empty body. .TP \fB\-t TITLE\fR, \fB\-\-title=TITLE\fR For the "post" command, set a title to message .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (atom, csv, events, htmltable, json, json_line, msg, msglist, multiline, profile, simple, table, webkit, xhtml) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/havedate weboob-1.2/man/masstransit.1000066400000000000000000000070471303450110500160360ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH MASSTRANSIT 1 "08 January 2017" "masstransit 1\&.2" .SH NAME masstransit \- search for train stations and departures .SH SYNOPSIS .B masstransit [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B masstransit [\-\-help] [\-\-version] .SH DESCRIPTION .LP Maemo application allowing to search for train stations and get departure times. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Julien Hébert .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/masstransit weboob-1.2/man/monboob.1000066400000000000000000000135111303450110500151120ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH MONBOOB 1 "08 January 2017" "monboob 1\&.2" .SH NAME monboob \- daemon to send and check messages .SH SYNOPSIS .B monboob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B monboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Daemon allowing to regularly check for new messages on various websites, and send an email for each message, and post a reply to a message on a website. .SS Supported websites: * aum ("Adopte un Mec" French dating website) .br * bnporc (BNP Paribas) .br * bouygues (Bouygues Télécom French mobile phone provider) .br * dlfp (Da Linux French Page news website) .br * feedly (handle the popular RSS reading service Feedly) .br * fourchan (4chan image board) .br * guerrillamail (GuerrillaMail temp mailbox) .br * happn (Happn dating mobile application) .br * hds (Histoires de Sexe French erotic novels) .br * inrocks (Les Inrocks French news website) .br * lefigaro (Le Figaro French newspaper website) .br * liberation (Libération newspaper website) .br * mailinator (mailinator temp mailbox) .br * minutes20 (20 Minutes French newspaper website) .br * newsfeed (Loads RSS and Atom feeds from any website) .br * okc (OkCupid) .br * openedx (Discussions on OpenEDX\-powered coursewares) .br * orange (Orange French mobile phone provider) .br * ovs (OnVaSortir website. Handles private messages only) .br * phpbb (phpBB forum) .br * playme (PlayMe dating mobile application) .br * presseurop (Presseurop website) .br * sfr (SFR French mobile phone provider) .br * tapatalk (Tapatalk\-compatible sites) .br * taz (Taz newspaper website) .br * tinder (Tinder dating mobile application) .br * twitter (twitter website) .SH MONBOOB COMMANDS .TP \fBonce\fR .br Send mails only once, then exit. .TP \fBpost\fR .br Pipe with a mail to post message. .TP \fBrun\fR .br Run the fetching daemon. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH MONBOOB OPTIONS .TP \fB\-S SMTPD\fR, \fB\-\-smtpd=SMTPD\fR run a fake smtpd server and set the port .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/monboob" .SH SEE ALSO Home page: http://weboob.org/applications/monboob weboob-1.2/man/parceloob.1000066400000000000000000000170431303450110500154310ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH PARCELOOB 1 "08 January 2017" "parceloob 1\&.2" .SH NAME parceloob \- manage your parcels .SH SYNOPSIS .B parceloob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B parceloob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to track your parcels. .SS Supported websites: * chronopost (Chronopost website) .br * colisprive (Colisprive parcel tracking website) .br * colissimo (Colissimo parcel tracking website) .br * dhl (DHL website) .br * dpd (DPD website) .br * gls (GLS website) .br * itella (Itella website) .br * ups (UPS website) .SH PARCELOOB COMMANDS .TP \fBinfo\fR \fIID\fR .br Get information about a parcel. .TP \fBstatus\fR .br Display status for all of the tracked parcels. .TP \fBtrack\fR \fIID\fR .br Track a parcel. .TP \fBuntrack\fR \fIID\fR .br Stop tracking a parcel. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, history, htmltable, json, json_line, multiline, simple, status, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/parceloob weboob-1.2/man/pastoob.1000066400000000000000000000204001303450110500151210ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH PASTOOB 1 "08 January 2017" "pastoob 1\&.2" .SH NAME pastoob \- post and get pastes from pastebins .SH SYNOPSIS .B pastoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B pastoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to post and get pastes from pastebins. .SS Supported websites: * imgur (imgur image upload service) .br * jirafeau (Jirafeau\-based file upload website) .br * lutim (lutim website) .br * pastealacon (Paste à la con text sharing tool) .br * pastebin (Pastebin text sharing service) .br * pixtoilelibre (toile\-libre image hosting website) .br * unsee (unsee.cc expiring image hosting) .SH PASTOOB COMMANDS .TP \fBget\fR \fIID\fR .br Get a paste contents. .TP \fBget_bin\fR \fIID\fR .br Get a paste contents. .br File will be downloaded from binary services. .TP \fBinfo\fR \fIID\fR [\fIID\fR2 [...]] .br Get information about pastes. .TP \fBpost\fR [\fIFILENAME\fR] .br Submit a new paste. .br The filename can be '\-' for reading standard input (pipe). .br If 'bin' is passed, file will be uploaded to binary services. .TP \fBpost_bin\fR [\fIFILENAME\fR] .br Submit a new paste. .br The filename can be '\-' for reading standard input (pipe). .br File will be uploaded to binary services. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH PASTOOB OPTIONS .TP \fB\-p\fR, \fB\-\-public\fR Make paste public. .TP \fB\-t TITLE\fR, \fB\-\-title=TITLE\fR Paste title .TP \fB\-m MAX_AGE\fR, \fB\-\-max\-age=MAX_AGE\fR Maximum age (duration), default "1 month", "never" for infinite .TP \fB\-E ENCODING\fR, \fB\-\-encoding=ENCODING\fR Input encoding .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2011-2017 Laurent Bachelier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/pastoob weboob-1.2/man/qbooblyrics.1000066400000000000000000000076111303450110500160130ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QBOOBLYRICS 1 "08 January 2017" "qbooblyrics 1\&.2" .SH NAME qbooblyrics \- search lyrics .SH SYNOPSIS .B qbooblyrics [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qbooblyrics [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to search song lyrics. .SS Supported websites: * lyricsdotcom (Lyrics.com lyrics website) .br * lyricsmode (Lyrics.com lyrics website) .br * lyricsplanet (Lyricsplanet.com song lyrics website) .br * paroles2chansons (Paroles2chansons.com song lyrics website) .br * parolesmania (Paroles Mania lyrics website) .br * parolesmusique (paroles\-musique lyrics website) .br * parolesnet (paroles.net lyrics website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2016 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qbooblyrics" .SH SEE ALSO Home page: http://weboob.org/applications/qbooblyrics weboob-1.2/man/qboobmsg.1000066400000000000000000000113731303450110500152740ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QBOOBMSG 1 "08 January 2017" "qboobmsg 1\&.2" .SH NAME qboobmsg \- send and receive message threads .SH SYNOPSIS .B qboobmsg [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qboobmsg [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to read messages on various websites and reply to them. .SS Supported websites: * aum ("Adopte un Mec" French dating website) .br * bnporc (BNP Paribas) .br * bouygues (Bouygues Télécom French mobile phone provider) .br * dlfp (Da Linux French Page news website) .br * feedly (handle the popular RSS reading service Feedly) .br * fourchan (4chan image board) .br * guerrillamail (GuerrillaMail temp mailbox) .br * happn (Happn dating mobile application) .br * hds (Histoires de Sexe French erotic novels) .br * inrocks (Les Inrocks French news website) .br * lefigaro (Le Figaro French newspaper website) .br * liberation (Libération newspaper website) .br * mailinator (mailinator temp mailbox) .br * minutes20 (20 Minutes French newspaper website) .br * newsfeed (Loads RSS and Atom feeds from any website) .br * okc (OkCupid) .br * openedx (Discussions on OpenEDX\-powered coursewares) .br * orange (Orange French mobile phone provider) .br * ovs (OnVaSortir website. Handles private messages only) .br * phpbb (phpBB forum) .br * playme (PlayMe dating mobile application) .br * presseurop (Presseurop website) .br * sfr (SFR French mobile phone provider) .br * tapatalk (Tapatalk\-compatible sites) .br * taz (Taz newspaper website) .br * tinder (Tinder dating mobile application) .br * twitter (twitter website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/qboobmsg weboob-1.2/man/qcineoob.1000066400000000000000000000103231303450110500152540ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QCINEOOB 1 "08 January 2017" "qcineoob 1\&.2" .SH NAME qcineoob \- search movies, people, torrent and subtitles .SH SYNOPSIS .B qcineoob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qcineoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to search movies, people, torrent and subtitles. .SS Supported websites: * allocine (AlloCiné French cinema database service) .br * attilasub ("Attila's Website 2.0" French subtitles) .br * btmon (BTMon BitTorrent database) .br * cpasbien (Cpasbien Torrents BitTorrent tracker) .br * gazelle (Gazelle\-based BitTorrent trackers) .br * imdb (Internet Movie Database service) .br * kickass (Kickass Torrents BitTorrent tracker) .br * opensubtitles (Opensubtitles subtitle website) .br * piratebay (The Pirate Bay BitTorrent tracker) .br * podnapisi (Podnapisi movies and tv series subtitle website) .br * t411 (T411 BitTorrent tracker) .br * torrentz (Torrentz Search Engine.) .br * tvsubtitles (Tvsubtitles subtitle website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qcineoob" .SH SEE ALSO Home page: http://weboob.org/applications/qcineoob weboob-1.2/man/qcookboob.1000066400000000000000000000074371303450110500154470ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QCOOKBOOB 1 "08 January 2017" "qcookboob 1\&.2" .SH NAME qcookboob \- search recipes .SH SYNOPSIS .B qcookboob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qcookboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to search recipes. .SS Supported websites: * 750g (750g French recipe website) .br * allrecipes (Allrecipes English recipe website) .br * cuisineaz (Cuisine AZ French recipe website) .br * marmiton (Marmiton French recipe website) .br * supertoinette (Super Toinette, la cuisine familiale French recipe website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2014 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qcookboob" .SH SEE ALSO Home page: http://weboob.org/applications/qcookboob weboob-1.2/man/qflatboob.1000066400000000000000000000074131303450110500154340ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QFLATBOOB 1 "08 January 2017" "qflatboob 1\&.2" .SH NAME qflatboob \- search for housing .SH SYNOPSIS .B qflatboob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qflatboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application to search for housing. .SS Supported websites: * entreparticuliers (entreparticuliers.com website) .br * explorimmo (explorimmo website) .br * leboncoin (search house on leboncoin website) .br * logicimmo (logicimmo website) .br * pap (French housing website) .br * seloger (French housing website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2014 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qflatboob" .SH SEE ALSO Home page: http://weboob.org/applications/qflatboob weboob-1.2/man/qhandjoob.1000066400000000000000000000075441303450110500154350ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QHANDJOOB 1 "08 January 2017" "qhandjoob 1\&.2" .SH NAME qhandjoob \- search for job .SH SYNOPSIS .B qhandjoob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qhandjoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application to search for job. .SS Supported websites: * adecco (adecco website) .br * apec (apec website) .br * cci (cci website) .br * indeed (indeed website) .br * linuxjobs (linuxjobs website) .br * lolix (Lolix French free software employment website) .br * manpower (manpower website) .br * monster (monster website) .br * popolemploi (Pole Emploi website) .br * regionsjob (regionsjob website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2014 Sébastien Monel .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qhandjoob" .SH SEE ALSO Home page: http://weboob.org/applications/qhandjoob weboob-1.2/man/qhavedate.1000066400000000000000000000073501303450110500154250ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QHAVEDATE 1 "08 January 2017" "qhavedate 1\&.2" .SH NAME qhavedate \- interact with dating websites .SH SYNOPSIS .B qhavedate [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qhavedate [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to interact with various dating websites. .SS Supported websites: * aum ("Adopte un Mec" French dating website) .br * happn (Happn dating mobile application) .br * okc (OkCupid) .br * playme (PlayMe dating mobile application) .br * tinder (Tinder dating mobile application) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2014 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/qhavedate weboob-1.2/man/qvideoob.1000066400000000000000000000112561303450110500152730ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QVIDEOOB 1 "08 January 2017" "qvideoob 1\&.2" .SH NAME qvideoob \- search and play videos .SH SYNOPSIS .B qvideoob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qvideoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to search videos on various websites and play them. .SS Supported websites: * allocine (AlloCiné French cinema database service) .br * arretsurimages (arretsurimages website) .br * arte (Arte French and German TV) .br * canalplus (Canal Plus French TV) .br * cappedtv (Capped.tv demoscene website) .br * dailymotion (Dailymotion video streaming website) .br * europarl (Europarl parliamentary video streaming website) .br * francetelevisions (France Télévisions video website) .br * funmooc (France\-Université\-Numérique MOOC website) .br * gdcvault (Game Developers Conferences Vault video streaming website) .br * ina (INA French TV video archives) .br * jacquieetmichel (Jacquie et Michel TV) .br * nolifetv (NolifeTV French video streaming website) .br * pornhub (Pornhub pornographic video streaming website) .br * quvi (Multi\-website video helper with quvi. Handles Youtube, BBC, and a lot more) .br * rmll (Videos from RMLL) .br * trictractv (TricTrac.tv video website) .br * vimeo (Vimeo video streaming website) .br * vine (vine website) .br * youjizz (YouJizz pornographic video streaming website) .br * youporn (YouPorn pornographic video streaming website) .br * youtube (YouTube video streaming website) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2014 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" "~/.config/weboob/qvideoob" .SH SEE ALSO Home page: http://weboob.org/applications/qvideoob weboob-1.2/man/qwebcontentedit.1000066400000000000000000000073341303450110500166640ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH QWEBCONTENTEDIT 1 "08 January 2017" "qwebcontentedit 1\&.2" .SH NAME qwebcontentedit \- manage websites content .SH SYNOPSIS .B qwebcontentedit [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B qwebcontentedit [\-\-help] [\-\-version] .SH DESCRIPTION .LP Qt application allowing to manage content of various websites. .SS Supported websites: * dlfp (Da Linux French Page news website) .br * mediawiki (Wikis running MediaWiki, like Wikipedia) .br * redmine (The Redmine project management web application) .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2011-2014 Clément Schreiner .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/qwebcontentedit weboob-1.2/man/radioob.1000066400000000000000000000177121303450110500151050ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH RADIOOB 1 "08 January 2017" "radioob 1\&.2" .SH NAME radioob \- search, show or listen to radio stations .SH SYNOPSIS .B radioob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B radioob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for web radio stations, listen to them and get information like the current song. .SS Supported websites: * audioaddict (Internet radios powered by audioaddict.com services) .br * ina (INA French TV video archives) .br * nectarine (Nectarine Demoscene Radio) .br * nihonnooto (« Le son du Japon » french operated web radio, diffusing japanese music) .br * nova (Nova French radio) .br * ouifm (OÜI FM French radio) .br * radiofrance (Radios of Radio France: Inter, Info, Bleu, Culture, Musique, FIP, Le Mouv') .br * somafm (SomaFM web radio) .br * virginradio (VirginRadio french radio) .SH RADIOOB COMMANDS .TP \fBdownload\fR \fIID\fR [\fIDIRECTORY\fR] .br Download an audio file .TP \fBinfo\fR \fIID\fR .br Get information about a radio or an audio file. .TP \fBplay\fR \fIID\fR [stream\fI_\fRid] .br Play a radio or a audio file with a found player (optionnaly specify the wanted stream). .TP \fBplaylist\fR cmd [args]playlist add ID [ID2 ID3 ...] .br playlist remove ID [ID2 ID3 ...] .br playlist export [FILENAME] .br playlist display .TP \fBsearch\fR (radio|song|file|album|playlist) \fIPATTERN\fR .br List (radio|song|file|album|playlist) matching a PATTERN. .br .br If PATTERN is not given, this command will list all the (radio|song|album|playlist). .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR .br List radios .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (album_tracks_list_info, csv, htmltable, json, json_line, multiline, playlist_tracks_list_info, radio_list, simple, song_list, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon Copyright(C) 2017 Pierre Maziere .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/radioob weboob-1.2/man/shopoob.1000066400000000000000000000170301303450110500151300ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH SHOPOOB 1 "08 January 2017" "shopoob 1\&.2" .SH NAME shopoob \- Obtain details and status of e-commerce orders .SH SYNOPSIS .B shopoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B shopoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to obtain details and status of e\-commerce orders. .SS Supported websites: * amazon (Amazon) .br * ideel (Ideel) .br * myhabit (MYHABIT) .br * vicsec (Victoria's Secret) .SH SHOPOOB COMMANDS .TP \fBitems\fR [\fIID\fR] .br Get items of orders. .TP \fBorders\fR [\fIBACKEND_NAME\fR] .br Get orders of a backend. .br If no BACKEND_NAME given, display all orders of all backends. .br .br Default is limited to 10 results. .TP \fBpayments\fR [\fIID\fR] .br Get payments of orders. .br If no ID given, display payment of all backends. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, items, json, json_line, multiline, orders, payments, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2015 Christophe Lampin .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/shopoob weboob-1.2/man/suboob.1000066400000000000000000000223271303450110500147550ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH SUBOOB 1 "08 January 2017" "suboob 1\&.2" .SH NAME suboob \- search and download subtitles .SH SYNOPSIS .B suboob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B suboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for subtitles on various services and download them. .SS Supported websites: * attilasub ("Attila's Website 2.0" French subtitles) .br * opensubtitles (Opensubtitles subtitle website) .br * podnapisi (Podnapisi movies and tv series subtitle website) .br * tvsubtitles (Tvsubtitles subtitle website) .SH SUBOOB COMMANDS .TP \fBdownload\fR \fIID\fR [\fIFILENAME\fR] .br Get the subtitle or archive file. .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .TP \fBinfo\fR \fIID\fR .br Get information about a subtitle. .TP \fBsearch\fR language [\fIPATTERN\fR] .br Search subtitles. .br .br Language Abbreviation .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br Arabic ar Esperanto eo Irish ga Russian ru .br Afrikaans af Estonian et Italian it Serbian sr .br Albanian sq Filipino tl Japanese ja Slovak sk .br Armenian hy Finnish fi Kannada kn Slovenian sl .br Azerbaijani az French fr Korean ko Spanish es .br Basque eu Galician gl Latin la Swahili sw .br Belarusian be Georgian ka Latvian lv Swedish sv .br Bengali bn German de Lithuanian lt Tamil ta .br Bulgarian bg Greek gr Macedonian mk Telugu te .br Catalan ca Gujarati gu Malay ms Thai th .br Chinese zh Haitian ht Maltese mt Turkish tr .br Croatian hr Hebrew iw Norwegian no Ukrainian uk .br Czech cz Hindi hi Persian fa Urdu ur .br Danish da Hungaric hu Polish pl Vietnamese vi .br Dutch nl Icelandic is Portuguese pt Welsh cy .br English en Indonesian id Romanian ro Yiddish yi .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, subtitle_info, subtitle_list, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2013-2017 Julien Veyssier .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/suboob weboob-1.2/man/translaboob.1000066400000000000000000000220451303450110500157670ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH TRANSLABOOB 1 "08 January 2017" "translaboob 1\&.2" .SH NAME translaboob \- translate text from one language to another .SH SYNOPSIS .B translaboob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B translaboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application to translate text from one language to another .SS Supported websites: * ebonics (English to Ebonics translation service) .br * googletranslate (Google translation web service) .br * wordreference (Free online translator) .SH TRANSLABOOB COMMANDS .TP \fBtranslate\fR \fIFROM\fR \fITO\fR [\fITEXT\fR] .br Translate from one language to another. .br * FROM : source language .br * TO : destination language .br * TEXT : language to translate, standard input if \- is given .br .br Language Abbreviation .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .br Arabic ar Esperanto eo Irish ga Russian ru .br Afrikaans af Estonian et Italian it Serbian sr .br Albanian sq Filipino tl Japanese ja Slovak sk .br Armenian hy Finnish fi Kannada kn Slovenian sl .br Azerbaijani az French fr Korean ko Spanish es .br Basque eu Galician gl Latin la Swahili sw .br Belarusian be Georgian ka Latvian lv Swedish sv .br Bengali bn German de Lithuanian lt Tamil ta .br Bulgarian bg Greek gr Macedonian mk Telugu te .br Catalan ca Gujarati gu Malay ms Thai th .br Chinese zh Haitian ht Maltese mt Turkish tr .br Croatian hr Hebrew iw Norwegian no Ukrainian uk .br Czech cz Hindi hi Persian fa Urdu ur .br Danish da Hungaric hu Polish pl Vietnamese vi .br Dutch nl Icelandic is Portuguese pt Welsh cy .br English en Indonesian id Romanian ro Yiddish yi .br \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, translation, webkit, xmltrans) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Lucien Loiseau .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/translaboob weboob-1.2/man/traveloob.1000066400000000000000000000202351303450110500154550ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH TRAVELOOB 1 "08 January 2017" "traveloob 1\&.2" .SH NAME traveloob \- search for train stations and departures .SH SYNOPSIS .B traveloob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B traveloob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for train stations and get departure times. .SS Supported websites: * blablacar (blablacar website) .br * canaltp (French trains) .br * jvmalin (Multimodal public transportation for whole Région Centre, France) .br * transilien (Public transportation in the Paris area) .br * voyagessncf (Voyages SNCF) .SH TRAVELOOB COMMANDS .TP \fBdepartures\fR \fISTATION\fR [\fIARRIVAL\fR [\fIDATE\fR]]] .br List all departures for a given station. .br The format for the date is "yyyy\-mm\-dd HH:MM" or "HH:MM". .br .br Default is limited to 10 results. .TP \fBroadmap\fR \fIDEPARTURE\fR \fIARRIVAL\fR .br Display the roadmap to travel from DEPARTURE to ARRIVAL. .br .br Command\-line parameters: .br \-\-departure\-time TIME requested departure time .br \-\-arrival\-time TIME requested arrival time .br .br TIME might be in form "yyyy\-mm\-dd HH:MM" or "HH:MM". .br .br Example: .br > roadmap Puteaux Aulnay\-sous\-Bois \-\-arrival\-time 22:00 .TP \fBstations\fR \fIPATTERN\fR .br Search stations. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH TRAVELOOB OPTIONS .TP \fB\-\-departure\-time=DEPARTURE_TIME\fR .TP \fB\-\-arrival\-time=ARRIVAL_TIME\fR .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, departures, htmltable, json, json_line, multiline, simple, stations, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/traveloob weboob-1.2/man/videoob.1000066400000000000000000000222041303450110500151050ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH VIDEOOB 1 "08 January 2017" "videoob 1\&.2" .SH NAME videoob \- search and play videos .SH SYNOPSIS .B videoob [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B videoob [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for videos on various websites, play and download them and get information. .SS Supported websites: * allocine (AlloCiné French cinema database service) .br * arretsurimages (arretsurimages website) .br * arte (Arte French and German TV) .br * canalplus (Canal Plus French TV) .br * cappedtv (Capped.tv demoscene website) .br * dailymotion (Dailymotion video streaming website) .br * europarl (Europarl parliamentary video streaming website) .br * francetelevisions (France Télévisions video website) .br * funmooc (France\-Université\-Numérique MOOC website) .br * gdcvault (Game Developers Conferences Vault video streaming website) .br * ina (INA French TV video archives) .br * jacquieetmichel (Jacquie et Michel TV) .br * nolifetv (NolifeTV French video streaming website) .br * pornhub (Pornhub pornographic video streaming website) .br * quvi (Multi\-website video helper with quvi. Handles Youtube, BBC, and a lot more) .br * rmll (Videos from RMLL) .br * trictractv (TricTrac.tv video website) .br * vimeo (Vimeo video streaming website) .br * vine (vine website) .br * youjizz (YouJizz pornographic video streaming website) .br * youporn (YouPorn pornographic video streaming website) .br * youtube (YouTube video streaming website) .SH VIDEOOB COMMANDS .TP \fBdownload\fR \fIID\fR [\fIFILENAME\fR] .br Download a video .br .br Braces\-enclosed tags are replaced with data fields. Use the 'info' .br command to see what fields are available on a given video. .br .br Example: download KdRRge4XYIo@youtube '{title}.{ext}' .TP \fBinfo\fR \fIID\fR [\fIID\fR2 [...]] .br Get information about a video. .TP \fBnsfw\fR [on | off] .br If argument is given, enable or disable the non\-suitable for work behavior. .br .br If no argument is given, print the current behavior. .TP \fBplay\fR \fIID\fR .br Play a video with a found player. .TP \fBplaylist\fR cmd [args] .br playlist add ID [ID2 ID3 ...] .br playlist remove ID [ID2 ID3 ...] .br playlist export [FILENAME] .br playlist display .br playlist download [PATH] .br playlist play .TP \fBsearch\fR \fIPATTERN\fR .br Search for videos matching a PATTERN. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, video_list, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Christophe Benz, Romain Bignon, John Obbele .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/videoob weboob-1.2/man/webcontentedit.1000066400000000000000000000170171303450110500165020ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBCONTENTEDIT 1 "08 January 2017" "webcontentedit 1\&.2" .SH NAME webcontentedit \- manage websites content .SH SYNOPSIS .B webcontentedit [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B webcontentedit [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to display and edit contents on various websites. .SS Supported websites: * dlfp (Da Linux French Page news website) .br * mediawiki (Wikis running MediaWiki, like Wikipedia) .br * redmine (The Redmine project management web application) .SH WEBCONTENTEDIT COMMANDS .TP \fBedit\fR \fIID\fR [\fIID\fR...] .br Edit a content with $EDITOR, then push it on the website. .TP \fBget\fR \fIID\fR [\-r revision] .br Get page contents .TP \fBlog\fR \fIID\fR .br Display log of a page .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/webcontentedit weboob-1.2/man/weboob-cli.1000066400000000000000000000104351303450110500155030ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB-CLI 1 "08 January 2017" "weboob-cli 1\&.2" .SH NAME weboob-cli \- call a method on backends .SH SYNOPSIS .B weboob\-cli [\-dqv] [\-b \fIbackends\fR] [\-cnfs] \fIcapability\fR \fImethod\fR [\fIarguments\fR..] .br .B weboob\-cli [\-\-help] [\-\-version] .SH DESCRIPTION .LP Weboob\-Cli is a console application to call a specific method on backends which implement the given capability. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboob-cli weboob-1.2/man/weboob-config-qt.1000066400000000000000000000071421303450110500166240ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB-CONFIG-QT 1 "08 January 2017" "weboob-config-qt 1\&.2" .SH NAME weboob-config-qt \- manage backends or register new accounts .SH SYNOPSIS .B weboob\-config\-qt [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B weboob\-config\-qt [\-\-help] [\-\-version] .SH DESCRIPTION .LP weboob\-config\-qt is a graphical application to add/edit/remove backends, and to register new website accounts. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2014 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/qweboobcfg weboob-1.2/man/weboob-config.1000066400000000000000000000126231303450110500162020ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB-CONFIG 1 "08 January 2017" "weboob-config 1\&.2" .SH NAME weboob-config \- manage backends or register new accounts .SH SYNOPSIS .B weboob\-config [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B weboob\-config [\-\-help] [\-\-version] .SH DESCRIPTION .LP Weboob\-Config is a console application to add/edit/remove backends, and to register new website accounts. .SH WEBOOB\-CONFIG COMMANDS .TP \fBadd\fR \fIMODULE_NAME\fR [\fIBACKEND_NAME\fR] [\fIPARAMETERS\fR ...] .br Create a backend from a module. By default, if BACKEND_NAME is omitted, .br that's the module name which is used. .br .br You can specify parameters from command line in form "key=value". .TP \fBapplications\fR .br Show applications. .TP \fBconfirm\fR \fIBACKEND\fR .br For a backend which support CapAccount, parse a confirmation mail .br after using the 'register' command to automatically confirm the .br subscribe. .br .br It takes mail from stdin. Use it with postfix for example. .TP \fBdisable\fR \fIBACKEND\fR .br Disable a backend .TP \fBedit\fR \fIBACKEND\fR .br Edit a backend .TP \fBenable\fR \fIBACKEND\fR .br Enable a disabled backend .TP \fBinfo\fR \fINAME\fR .br Display information about a module. .TP \fBlist\fR [\fICAPS\fR ..] .br Show backends. .TP \fBmodules\fR [\fICAPS\fR ...] .br Show available modules. .TP \fBregister\fR \fIMODULE\fR .br Register a new account on a module. .TP \fBremove\fR \fINAME\fR .br Remove a backend. .TP \fBupdate\fR .br Update weboob. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, info_formatter, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Christophe Benz, Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboob-config weboob-1.2/man/weboob-debug.1000066400000000000000000000071261303450110500160250ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB-DEBUG 1 "08 January 2017" "weboob-debug 1\&.2" .SH NAME weboob-debug \- debug backends .SH SYNOPSIS .B weboob\-debug [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B weboob\-debug [\-\-help] [\-\-version] .SH DESCRIPTION .LP Weboob\-Debug is a console application to debug backends. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH WEBOOB\-DEBUG OPTIONS .TP \fB\-B\fR, \fB\-\-bpython\fR Prefer bpython over ipython .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Christophe Benz .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboobdebug weboob-1.2/man/weboob-repos.1000066400000000000000000000121621303450110500160630ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB-REPOS 1 "08 January 2017" "weboob-repos 1\&.2" .SH NAME weboob-repos \- manage a weboob repository .SH SYNOPSIS .B weboob\-repos [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B weboob\-repos [\-\-help] [\-\-version] .SH DESCRIPTION .LP Weboob\-repos is a console application to manage a Weboob Repository. .SH WEBOOB\-REPOS COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBbuild\fR \fISOURCE\fR \fIREPOSITORY\fR .br Build backends contained in SOURCE to REPOSITORY. .br .br Example: .br $ weboob\-repos build $HOME/src/weboob/modules /var/www/updates.weboob.org/0.a/ .TP \fBcreate\fR \fINAME\fR [\fIPATH\fR] .br Create a new repository. If PATH is missing, create repository .br on the current directory. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2012-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboob-repos weboob-1.2/man/weboob.1000066400000000000000000000067431303450110500147450ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOOB 1 "08 January 2017" "weboob 1\&.2" .SH NAME weboob \- launch weboob applications .SH SYNOPSIS .B weboob [\-h] [\-dqv] [\-b \fIbackends\fR] ... .br .B weboob [\-\-help] [\-\-version] .SH DESCRIPTION .LP This is a console script to launch weboob applications, .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 The Weboob Team .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboob weboob-1.2/man/weboorrents.1000066400000000000000000000174321303450110500160360ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WEBOORRENTS 1 "08 January 2017" "weboorrents 1\&.2" .SH NAME weboorrents \- search and download torrents .SH SYNOPSIS .B weboorrents [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B weboorrents [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to search for torrents on various trackers and download .torrent files. .SS Supported websites: * btmon (BTMon BitTorrent database) .br * cpasbien (Cpasbien Torrents BitTorrent tracker) .br * gazelle (Gazelle\-based BitTorrent trackers) .br * kickass (Kickass Torrents BitTorrent tracker) .br * piratebay (The Pirate Bay BitTorrent tracker) .br * t411 (T411 BitTorrent tracker) .br * torrentz (Torrentz Search Engine.) .SH WEBOORRENTS COMMANDS .TP \fBgetfile\fR \fIID\fR [\fIFILENAME\fR] .br Get the .torrent file. .br FILENAME is where to write the file. If FILENAME is '\-', .br the file is written to stdout. .TP \fBinfo\fR \fIID\fR .br Get information about a torrent. .TP \fBsearch\fR [\fIPATTERN\fR] .br Search torrents. .br .br Default is limited to 10 results. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (csv, htmltable, json, json_line, multiline, simple, table, torrent_info, torrent_list, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/weboorrents weboob-1.2/man/wetboobs.1000066400000000000000000000171711303450110500153110ustar00rootroot00000000000000.\" -*- coding: utf-8 -*- .\" This file was generated automatically by tools/make_man.sh. .TH WETBOOBS 1 "08 January 2017" "wetboobs 1\&.2" .SH NAME wetboobs \- display weather and forecasts .SH SYNOPSIS .B wetboobs [\-dqv] [\-b \fIbackends\fR] [\-cnfs] [\fIcommand\fR [\fIarguments\fR..]] .br .B wetboobs [\-\-help] [\-\-version] .SH DESCRIPTION .LP Console application allowing to display weather and forecasts in your city. .SS Supported websites: * ilmatieteenlaitos (Get forecasts from the Ilmatieteenlaitos.fi website) .br * meteofrance (Get forecasts from the MeteoFrance website) .br * weather (Get forecasts from weather.com) .br * yahoo (Yahoo! Weather.) .SH WETBOOBS COMMANDS .TP \fBcities\fR \fIPATTERN\fR .br Search cities. .br .br Default is limited to 10 results. .TP \fBcurrent\fR \fICITY_ID\fR .br Get current weather for specified city. Use the 'cities' command to find them. .TP \fBforecasts\fR \fICITY_ID\fR .br Get forecasts for specified city. Use the 'cities' command to find them. .SH WEBOOB COMMANDS .TP \fBbackends\fR [\fIACTION\fR] [\fIBACKEND_NAME\fR]... .br Select used backends. .br .br ACTION is one of the following (default: list): .br * enable enable given backends .br * disable disable given backends .br * only enable given backends and disable the others .br * list list backends .br * add add a backend .br * register register a new account on a website .br * edit edit a backend .br * remove remove a backend .br * list\-modules list modules .TP \fBcd\fR [\fIPATH\fR] .br Follow a path. .br ".." is a special case and goes up one directory. .br "" is a special case and goes home. .TP \fBcondition\fR [\fIEXPRESSION\fR | off] .br If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. .br If the "off" value is given, conditional filtering is disabled. .br .br If no argument is given, print the current condition expression. .TP \fBcount\fR [\fINUMBER\fR | off] .br If an argument is given, set the maximum number of results fetched. .br NUMBER must be at least 1. .br "off" value disables counting, and allows infinite searches. .br .br If no argument is given, print the current count value. .TP \fBformatter\fR [list | \fIFORMATTER\fR [\fICOMMAND\fR] | option \fIOPTION_NAME\fR [on | off]] .br If a FORMATTER is given, set the formatter to use. .br You can add a COMMAND to apply the formatter change only to .br a given command. .br .br If the argument is "list", print the available formatters. .br .br If the argument is "option", set the formatter options. .br Valid options are: header, keys. .br If on/off value is given, set the value of the option. .br If not, print the current value for the option. .br .br If no argument is given, print the current formatter. .TP \fBlogging\fR [\fILEVEL\fR] .br Set logging level. .br .br Availables: debug, info, warning, error. .br * quiet is an alias for error .br * default is an alias for warning .TP \fBls\fR [\-d] [\-\fIU\fR] [\fIPATH\fR] .br List objects in current path. .br If an argument is given, list the specified path. .br Use \-U option to not sort results. It allows you to use a "fast path" to .br return results as soon as possible. .br Use \-d option to display information about a collection (and to not .br display the content of it). It has the same behavior than the well .br known UNIX "ls" command. .br .br Default is limited to 40 results. .TP \fBquit\fR .br Quit the application. .TP \fBselect\fR [\fIFIELD_NAME\fR]... | "$direct" | "$full" .br If an argument is given, set the selected fields. .br $direct selects all fields loaded in one http request. .br $full selects all fields using as much http requests as necessary. .br .br If no argument is given, print the currently selected fields. .SH OPTIONS .TP \fB\-\-version\fR show program's version number and exit .TP \fB\-h\fR, \fB\-\-help\fR show this help message and exit .TP \fB\-b BACKENDS\fR, \fB\-\-backends=BACKENDS\fR what backend(s) to enable (comma separated) .TP \fB\-e EXCLUDE_BACKENDS\fR, \fB\-\-exclude\-backends=EXCLUDE_BACKENDS\fR what backend(s) to exclude (comma separated) .TP \fB\-I\fR, \fB\-\-insecure\fR do not validate SSL .SH LOGGING OPTIONS .TP \fB\-d\fR, \fB\-\-debug\fR display debug messages. Set up it twice to more verbosity .TP \fB\-q\fR, \fB\-\-quiet\fR display only error messages .TP \fB\-v\fR, \fB\-\-verbose\fR display info messages .TP \fB\-\-logging\-file=LOGGING_FILE\fR file to save logs .TP \fB\-a\fR, \fB\-\-save\-responses\fR save every response .SH RESULTS OPTIONS .TP \fB\-c CONDITION\fR, \fB\-\-condition=CONDITION\fR filter result items to display given a boolean expression. See CONDITION section for the syntax .TP \fB\-n COUNT\fR, \fB\-\-count=COUNT\fR limit number of results (from each backends) .TP \fB\-s SELECT\fR, \fB\-\-select=SELECT\fR select result item keys to display (comma separated) .SH FORMATTING OPTIONS .TP \fB\-f FORMATTER\fR, \fB\-\-formatter=FORMATTER\fR select output formatter (cities, csv, current, forecasts, htmltable, json, json_line, multiline, simple, table, webkit) .TP \fB\-\-no\-header\fR do not display header .TP \fB\-\-no\-keys\fR do not display item keys .TP \fB\-O OUTFILE\fR, \fB\-\-outfile=OUTFILE\fR file to export result .SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\fBfield operator value\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \fBgrep\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \fB" AND "\fR, \fB" OR "\fR an \fB" LIMIT "\fR. .LP The \fBLIMIT\fR keyword can be used to limit the number of items upon which running the expression. \fBLIMIT\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions .SH COPYRIGHT Copyright(C) 2010-2017 Romain Bignon .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" .SH SEE ALSO Home page: http://weboob.org/applications/wetboobs weboob-1.2/modules/000077500000000000000000000000001303450110500142715ustar00rootroot00000000000000weboob-1.2/modules/.keys/000077500000000000000000000000001303450110500153225ustar00rootroot00000000000000weboob-1.2/modules/.keys/florent.asc000066400000000000000000000054451303450110500174730ustar00rootroot00000000000000-----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.22 (GNU/Linux) mQGiBEZvDQMRBACcSc5xvGE7EnAFonjB9P2U66xk3kL7grVJTvxdiY6ErwaqfRU0 XFSzwTnRoaQQFk//l+ctVaZ/Ex39y9lmivJLF4ME2RghIXTenJqbr6NQfnYIcPbx fKo+SUt91Yf7cei1j22Qz6UdfdFgbllJ/JHcNMA0Qvjn+lLhk+YaB34rswCgqU2p EoVlGtdUpx2/5yJh2PGqAe0D+QHf4gq5ZGKT8zF6HdySHvFaBZVwaymtxlaHINkE SpLjhlEr2h3TKpBqZKS1W4njDpszMkgVxmjZPKLSkNkn1Mh8JISvv/6R5MOCZW9k k+AnUhzvYLieM4WaBFPU9JgSb5ChVzuIgdQk+9w4gHYpkzBW91mnaPcdgZ2NTysB HvFLA/98qxGdvSQGxw6wL05BUV0Qh6boeUyrzmV0rx+C0arVt3WXBf5He4HimioA Lnq5ZfH7eqh1Ix+b4GcVeG3vQkOhbLqL73OnQ5B+lCFjM0UqvhtKlDBr0oTIUT4w u/VlycwaZUKoRyd/ugA/mJjCZw4urN5Jpqn3bIO8CMIiZPpEwLQgRmxvcmVudCBG b3VyY290IDxmbG9AZm91cmNvdC5mcj6IYgQTEQIAIgUCSz/zoQIbAwYLCQgHAwIG FQgCCQoLBBYCAwECHgECF4AACgkQ5Y0WuYomQc7jyACgmI/LjLyq9OTkLiXEJznF Pi/hYF8An21akzCCIZt8JrI9btSCyZOmu7LEtB5GbG9yZW50IEZvdXJjb3QgPGZs b0ByZXNlbC5mcj6IYAQTEQIAIAUCRm/mFAIbAwYLCQgHAwIEFQIIAwQWAgMBAh4B AheAAAoJEOWNFrmKJkHO9OIAnA7InVQiLQnI0/aGN2rcj33rGjVVAKCkh261K9Ac gkrrDoR5cPtpSdGgrbQqRmxvcmVudCBGb3VyY290IDxmbG9yZW50LmZvdXJjb3RA cmVzZWwuZnI+iGAEExECACAFAkZv5jcCGwMGCwkIBwMCBBUCCAMEFgIDAQIeAQIX gAAKCRDljRa5iiZBzgRBAJ0cxm9LiVw0+POgH3VBexI84AHXNwCgkA+5/0EaxOCE Qg8vxs8ioEqwVXi0MkZsb3JlbnQgRm91cmNvdCA8ZmxvcmVudC5mb3VyY290QGVu c3QtYnJldGFnbmUuZnI+iGAEExECACAFAkZv5m4CGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRDljRa5iiZBzga5AJ9IjXbM7kFUy/BWPZw5Ssg3YakIeQCfSewl MhqkRvzkCJRFIJ6dni1Es0G0NUZsb3JlbnQgRm91cmNvdCA8ZmxvcmVudC5mb3Vy Y290QHRlbGVjb20tYnJldGFnbmUuZXU+iGAEExECACAFAkexdNECGwMGCwkIBwMC BBUCCAMEFgIDAQIeAQIXgAAKCRDljRa5iiZBzhGKAJ4sNXoThux0ljZEEETVRobq oFrdMwCeMz/U7iQY3vdHGukPssiu2n5jSM20OEZvdXJjb3QgRmxvcmVudCA8Zmxv cmVudC5mb3VyY290QHJlc2VsLmVuc3QtYnJldGFnbmUuZnI+iGAEExECACAFAkZv DQMCGwMGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRDljRa5iiZBzqlMAJ9ZjeJy Swq8H0Pgi+OdYYkhV/bGRACbB/nBrBTJq7KrNMjsdlSIo+/ov2y0H0Zsb3JlbnQg PHdlYm9vYkBmbG8uZm91cmNvdC5mcj6IYgQTEQIAIgUCUQPb4QIbAwYLCQgHAwIG FQgCCQoLBBYCAwECHgECF4AACgkQ5Y0WuYomQc4T6ACZAaTccHDc7WusxWC89xy2 5N+KjscAn1VQjSXeHSbxyx/ngz1cTfmVkgEYuQINBEZvDRMQCACNQs9XW0fUPIId l30N0k4ZdIPcRUZXl0o+iVXcLn11xmZQ/zPTzJCHtQqLC/hZueyNX6Q5o5FZnEEa nbg/6/YPOrCrQTCITIUUsPqzQWNYxfTI+qsQefvszNSvFU/ZBpVLLoHpYrgrg6V4 aWt/21wXIZeM+Hx5jwKquVaAPco25jxKFxR8feCMG2Kt+tiCg/V8qYBN0J1AA6S/ jKAkyhPRKsyky8d77JvtUPbe/LIV5URJ3+1Kjdb3Zd4C3brvY/6uYymJ7miVztbr YyTxRRWejFy61kCqxSAQyqNYWblECfWjFZpLcrbmNYDeF3qEsc+iYtfKgQQHKtnd 4u8dWCDbAAMFB/9qmlNHZb8iRzSEMVUrRtxrxFOHlTfWH+5YChmv2A/86ZPdq4XJ uzkumV7F75m0hOtoZl47y3pAJBGB+5FencPTS1/A4IgiZY9DYzJmw+MG1cPukMJU l356MN2jc15anXp8qFmbsJRuPQME5Km1awZG+jJBoLqLtJoCPORObSq583VlFKVO MHOelAo/y9IDxM4iE165Wi3J+ZQrolcF4ST/p+faeSaHW4pqOCgM5LrGnUF4SKqG ZYPrYMWgyaGZvbOH0Q2PRwdHzHR+mlYSGj/1X/Nx77lWZ29T5WVt4LGwbGFF9Wz4 zQf0ZdhPa1roVnXyUBJCCiukyKmsy02A51iniEkEGBECAAkFAkZvDRMCGwwACgkQ 5Y0WuYomQc7VogCdG/u6WJn2c5TC+VJhOS8J8qhY/a8AoJzT2FNtArRFFeH4hLqx g+aY0iqx =aGSj -----END PGP PUBLIC KEY BLOCK----- weboob-1.2/modules/.keys/laurentb.asc000066400000000000000000000151511303450110500176310ustar00rootroot00000000000000-----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.22 (GNU/Linux) mQINBFLlrhYBEAC/oKbgDYYHfH0DJ/y9cb8aoUyvswiGlgDdf13fbCDEHXnrlmVC TvAgxR0/RHiXFzjKntu0lI9aXvFEBJxPIfGzyWswLEUKU8Ewf1W1d6ERfFJ3v7wJ 3A3i0NjZrq2bv7eEd17JtlujNBIp4X4TTnVtCE32GCrn8IMS6d2Yg1nNCSZfgqKu 9Pc5PnGQVOCevFk5Q7yGHHk5AiBe2C8NxaQFOr3zB8Yb3dvnlWW30KeulxsQPHZ9 c+Mlb+sbhz2eV8CAUY07tg9H2rsRUx/mgpqzMTIk+6VF+uiV4sQtBdwnmdrhPKqu 6Ol9YwukFKNbwXaQapT+1SOO2iqYQEuth1oLUgHwgFLo9Yg+ZYX74mRpoJ0CKOaz /sErFc833j1rbpt9+hDI7YqXJ5d46lq/bDfpSNgQcU1iTjcWayUbTzv6g56flatH iizuVDJBtey4Dhol/0Gfrr6VWK3JPPtRgBKXHCe/BQMtVPZMO2mUpC8F/cbnSuCS Ha9igP7NOht4nds9Cm81kkP818SUA/1nkeSNWX4+i+vWN3eI+KXx5PphVgkPJ0wT t33S5yAcNorVDqy3rpegVVj/UC2k/IvLAn8JdjHHMJtdEdPt2jzUCaCWb9+AD257 0sIcyFH+YqWPqBTfWzfXRvY6qa7Dbb/eGlDwcYR7eWTQJgiS/GgpBhkx/QARAQAB tCpMYXVyZW50IEJhY2hlbGllciA8bGF1cmVudEBiYWNoZWxpZXIubmFtZT6JAjcE EwEKACEFAlLlrhYCGwMFCwkIBwMFFQoJCAsFFgMCAQACHgECF4AACgkQNGPqWlGK nHUWVA//ZpZ+XMticeO2SU003jeCA3tUr993ZiV8ShdXASpYsOhi65HRQdVnkhFF e63MVdlEcavu7+x0Vai2zezqaE13IYBee6DsSggjEFXwsTbyxf0qJyk2PQmeaEtz /LZswk8VYQwkzwXmivL2EwKCOWkEGGxxuQW2BK+TZrRUq9Xs0NnDhrCNXuaw+cUd 93YG0Wf5xuzhmi2XPZ7hBno7aSCJbSURjPqQWXHe5Yon11k0ZAw6b2MNYGYwAAeM +y2gRCaHKQwBtLgbAFGIWQVZhnuHywUF59dMXssCMAxzJbo3DpjhQIjSYJZ9r65c Eym7rj34QUMFKvQVGTbbouv3s4eTJ06TYyF9LBD4q7bbAy/iHGtAX0TKdGGwIvYd JKKMeAJlTqXUDMs4Ho5lXr5mK5BeyEgvSuN7OzUpy/keN8tmIDCRmMvVQIz610uE boLtiyM/gMc2yWJEgmBMYdHNywoOze/BvFM8rOgY4e/0QKzP/EV7bwSoPsMt6r8c a36UPoZGRT6LTN/y82t9rT2VCBYB6BPhRIzb4wb7YbWd8iwV1t+aXE5CAN545PUO 5XzXAglcZifqXlnzrgdgFo5JX0mT8otBQOzdnSeXGoL71RXrLBlzvkNyStyeKLi+ R3sVdJIgIYybwDMdqGDygc88z0TOQFGL2Zd5IBorzo1TOtN+uqW5Ag0EUuWuFgEQ AOnQstJTwJfYREPJp0wy5lM9oKuGyRlYAzLDJWBcHN0r5X2SX6fYuRXB6ZL1XSPc CKx61LvxJMQF4s147xK50FpGteFkXHXBYxVd4fAfELU+gw6Wm9ZAVcca8ImCNJka /84ASogPmh0yY2WnE8AHqWrtZLaoYjd+FLUPxHyE9eg/ZNoL4kvN81KPqckRcOp+ dZHI1ltFZWNmEaPxagCAkWP3jvZ5nj7PHpsF+EncMKtK6C07Toie/hkZLKvpjs1G V0hoYq+nmQVkM7jxKT4vVU3TLG6TOXeMmjXRKHcPNwI4sUvgc7WQu9rgO/EtPavk p3QetACmzE73soWAVnISF954NJaOvf0eM5y8tPtV3wONnj7p5maCXVHsm3EHRxsO sFJr7Byle6uxBC/xyuWpKY/hS6O8goq/3dkwnYgly6JkkTrCXpBvnP+LAIQ07n9u iNTTo8iDPNZY0BwZFYo+uIqab4Ij9BrkI0xrYTh9Azo3+7s/1c7lWaEL/kvfOupZ 9It5Y66bp3vOYmmaXEta6aYAoRGv1PkPhVC0Avua0fs0LMn/+qBEOskDVK70/9CD Rj03cJ5Vf7ha4OBM31H5L8efPBxIWREb93pzbPHmh+sgqenaKgAOFBTX5j0SnPHT heSKxvSaY9VR2STfvpBOjfRy+GJDK7KOuz0YrnK393pRABEBAAGJAh8EGAEKAAkF AlLlrhYCGwwACgkQNGPqWlGKnHXbmRAAn/lC2bdhAeITQi2ut0mrehSN7cLgP18a IyBDKnvGcDqWcEk+zALOKtBEkfICz6HUQN8DTWw3qK4vQ/kT8ADJIT3lzX+0pvv7 eplwVlMn8N/XJGdS9/pGvYYNmb5F9CfAicXkU5Dk3Z8I2qxN+bN+KbwIiAOuhfeZ wIQ4JrhCTm+s67+Yb0aVz6QlFYSCoOvyRezDvfxQW4lCn627t2JWyVXZ0IbQvln3 e7rgUgFDIsZTntNcdxzrb3eS6Law0mwKWd4G13XgfXogVb4CrC9pMfU59nvjN3Hq 87U+pg0XodWdyNgtBe5UEwCzMA5ZjnLYR5hqcrDZ6WanqH2gzeLZsR9+ITk5wbaH r30bgD+7VzieKTVJl084xKCecmqdON88I9yLcD8J6wcng7Gd3x5yOZeyi3vT/cnW VAcF+4LDlGgfXfhTlEBxJFohgyovI9pw2xK5NC8OEGlqO/2MjDKNMbhwokXqNiLP cB7Pf7VWNouYLogrLLZE/CufyiHPdQeOJFJNT7iUJOYDgnIBoeQUF0sBzg0LLSWO lY+0Che42ClxSLi/GvOS6quiqzsSXRd1KBcwyaGuzPABJghLsMpsWg14A09LkWCx yMv4ajT/jhhYnEqmssa3TfNrdu+iy2Gtbt9GZOaIDNeN3+cyRNDJFRL1D3MVb21L 8wPBzD0MJyG5Ag0EUuWytAEQANWGnn1hX3tARvc5KC4wz039H8NWR0hqfLJJylhz 49xLwPPMaQKLIBG3WpyR6nhEzlRiJePIs+YeWWMZ3n2T5A6aQM/00zHZRQIGHZrj j7g9OEDZgzvpzGA7eK7O5sP6Ze76/GQ3eEymOHd4Syg9fgoKq6ybWK1iqV3XFRDQ pDtEoj+ThOwuzkBEF4lnkR4g/ITuiSQdGzLuyMF1OimcRQUnW+rolnBAM4xPE+cD lftZ/MBpqgXiUxBhhmapF0YvlNueAl9Jv5A6onR8sQEG/0msqT8+7tSupsfW57G1 qoUavpT7wrkTl/4BCLD+pOgBCXbnnsi0U7nJYieS2WV3aLwKOSyvJhilo9tr1SYc gnUIkFn2KlDm6SD2+6bZywFrD5r3IHgrzVsD1/vngS4wA26z7HiYZl3VJ0wK/9TM Xys1tVjlrb4Hu6bDGB5lGkD1cqXiWMnpaWCRVQz7OtDt0HUe7oDZ9OrPte8dtzCJ lFJ0NlAjrUUe8BvbHIAgzobBELNi+DRSdOYjT0JTIa611TtQhiSg4NglBAFT+A5n rm9cc57jnVD7lCu9/gB8HC8lqNxIm/Vjl9fJMjW6mwVTQtnuuflJGPSAlr1+luE1 /4Xb+ABReVQ4O+jbeh3O3SLRjxjaPGK+r3lZ3SnLgSxZAYUjxhNOEEOY73zNj/vV KEMNABEBAAGJAh8EGAEKAAkFAlLlsrQCGwwACgkQNGPqWlGKnHW2Tg//UKioCkNR lF0ytpX86EFBwIhAkotNJ5uxQuRPNrToDdCgn+JvYI4wra0nMq+FMIgvZJXaE/Gk 9NctwQPivuz1Koib60NKtRnS4o7knMrSsuzqFG/Ts0eGhvwwS1u/isD4epEj1cNi gNI1EzbmAlW8MV7jh8UkgiO9GsO8nSzi5F8rKRQNE63akMo6u2jT3v4Aa2jRcHVC HwaPd1AGks5LqkDzG2AyHmEO76stm00XMISwg2DLc5/RlK6ErmWLjnqo/GLrms0q iexp59f64R/zOVbmkKq2tswgUhXIElfJbfTz7OTLYv9T7+vE2/7n2Z7kCAO1N76G ed2llZeLqLkETCp1GM2p9GrL3oIlwFlpTSLrymOPxdWsXWcSp8/LRlwIhZmL9Qi7 +N/4dLDWm/5yS01+UAh0Y7PleWSOe4yr2irYdw+qUIs/f+DqPNLqjcPca/Qm4+Dt 2ryslQ/pJvQiKxgNKR/klTVwRgajxtiLZthfDVheuhbs2FjBiulJlka60OqUKXsL Rnk1vclMLMTjvpVg3iFK+AXB201ki0ci6NNXNqL7Qm1+OU/WGCxPY3UPI1CZaZIM TZHNRPojNVIDgkZxlnCIO7ew3P0sbMk93iJHU3MrwhNgyShqE4D/JxdbD3dY6GSH kUN2ZTGOJFVIQDCB29LbFh35hr8V9Rm5PXi5Ag0EUuWzGwEQAM/aI/loKDbdiw2C OMT8oKeLrWZCsx/OiRCyCW3urpGOCZYN3WfFyi851iS1/px1hVPCt5pUu9qbZedo 2fhGLumaVFy/+6VikWUC5vGSOrkUKKR/OAv5DT4t75PCEq401Uuo1U87oyhtYm// mUBOPpK0lU85wKJAeLG+l9I4pK6pveijNMWVM7kJJHlUgRjf1EdgjMSKYuuF+Fg7 KsdK3QoxGVXD19wKRzZoyH6ai60uxcoFSW+SJdNWw6z6IbSkqbughxbL4JLkD03w JRfquNPZUIhyYrS1RE4g7uVDtJLTFCx4Rx6+kOD/KUdwxu6ukKDZxIOXcGI7K9pX jb68m3YCp5ieyznkMZJzj2SjRtFjDYLmsYq1V3BgRvTZhY3rEJL11Ul5QjcFaKZh icSRyIzRgcBhJEev+jJCPXC8XdWcD1dONKpI84A//YWpOCSz3Ns9ce9mAkuxFJ3V wjbOX5LjGZARSOtVqKhxABVy+nX7g+YJU4WLC/oi155NGIXECRD1IrYmdV1z5dEE 5g1pjWBLh29JLMYiPtWFbkbCDLwWjzcACEGZlliCqFdjNELt0b2C6l4De6FwkL9Q 9y1ahFg2BzGy5Y5UQJgjC/ll7sB6sYZFCFc+7zduxdyER2tsad9nWOEDbozXbutq 6744blm6vUrvnzjuRK0LkK9cyprJABEBAAGJBD4EGAEKAAkFAlLlsxsCGwICKQkQ NGPqWlGKnHXBXSAEGQEKAAYFAlLlsxsACgkQIO/wNLh3vDwsUxAAgZy1P/jjkUFZ Nil2QLgWTKDzIjBOFaTrudRj+ZShbLkk5T5PyhRWfk3aklFAIMBQjVGjM0g8J6RN MHN146j4sSY+4qYSKrbHYouJ2SpcD7/0mjWgTd1AGFQ3uCXpLcVdjsjU42K9kB5c 56vpBkn1Khff7gVGp2B7Qd25wuOjpT5V0qBn2yNi03Jz3Rjlpy/0J9tAcXbt246b Byq9h3A+eSz6Wzts6IKhdnp1sw/kvj1iPZZ1CH1RrWMLbizgbWLS6tLUmzJqdt01 /IOcjP3McjbUIPNE71UqtoHH/CKj0OfoHd6tBkD8Z1R7gbGWD6+rAMXZjWqJ4NnE +3OhTe2jzNfZgYOdA0S6YQm0Uz1SUVT1CTAvBa8Rfr3f4qwl/E0zRSS1BU5QmQU2 tIhk2Tbhd6cP4bisHY430CosuwCLURynnjTmS8JkA9aBhIadO+u6+ld9z0qoAa1h puajUhXWodTdMUfOVBpnBgraoPfvZaRYcV5FuIq7uazgFywTX8BHOxobJz9qxSRS WbdevOXnCEM/jRpJWgjK68c36u2i2Wob9ksMJIx9p/4YeUMZRBK7r446WIb/fAH1 C5z2EZooQIM9XGhSHMtg/bPsxyupMrC6dabBs0qHu8yJGQV/TLK4rpZBLcLLjfPw TeJAytqABt+kXeLKSibDd9L2DtYCfjcpehAAkIvPoK8qovXLDReg64AEJV1EXaMR n7MqQ5YEl7T9BNtghbv6mV4YHwLuWl07esp67U/KDwbOYu3LWw7ELVvhMP4IZy3M PMdyM1b5Epcr1ZXrPrW7kEMIHl2Yle4PSYKQiFbs0q7oOvOk/ImB0Vv+rjBsp6JN MUy88RbS7y26YXPgTjMPkml7+U1lL5t9iu3jESyBXJrqPCCa91vnblPludMad06M +HBCIw9Y0dcGGISOWyPDgisoHYZetGp7pFT/S8kbRgPh0mZATxgAjHhbEhtxOypU o53JveMZiP68R4eEAjnGm3lrtCjXG9PrIJ1HU282ZgOEwLSOL3UEiLPjVNiqRhL+ WiYIN8/1x5QUIGxQT2ZG8i1VVhFazZ99XNQtO3c0XEXvU+k1ExG9go0ktSrDQ4Zt Z4zHxAxyiQEuytdt5vjvOuuWcvhip9Jpq634Fo6X/MJxYqSmg2d6MLRcQb5bc1MM 8WRMTWql46OvSIqSPI0wjLrAndSQ8lDvpybEquVqzHAH8BJpXdvbL2gm3SmOmsUv 9JaJi2hyJnpt+FbzkHZnZ9NM5ggbNPCJ7rn0wgxq9aa90ttPglJWo2O3EMHgfzZK 4z/DGBVlUZ60SgJZxRYIlyqWdVTa9rmxEOImjGNlwk7eEGtjOXCHSAUbJlOLX6vP FHIy6Bn5CTgFpVA= =Rjwq -----END PGP PUBLIC KEY BLOCK----- weboob-1.2/modules/.keys/romain.asc000066400000000000000000000035621303450110500173050ustar00rootroot00000000000000-----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.17 (GNU/Linux) mQENBEoOr78BCADAsCwWXDvedstjc9L3YSlXJ6KDd12WVD+iEiVGUqToJkWwp05k uVvvibhNxjDJzZqfWxBvfhFXkf/APF8PoTvkA4l51WOfPGSZejiKMcenij+3HjKa GCPjLlY9rlTUjA5Alw533WP4Wja0EILXy0xjS9eXN2/Vbtrh7sbwU3cs9KOgDjEH dcR/ZgRGrqiLuVd83L2rlyoWZ5KM2KCkMjtpPBFUXOf5pyCV683wd+grdsHzL8Lm MAyVPPCmwrCKdkrG4/xjlBo0Ghjwq4gH6ShclcdJlwjfBY/CLtv5voM+TxKZ2wg0 6FIwyZdh0F8G4c5PhiL+TXKrAV76WNwgwE1VABEBAAG0I1JvbWFpbiBCaWdub24g PHJvbWFpbkBwZWVyZnVzZS5vcmc+iQE2BBMBAgAgBQJKDq+/AhsvBgsJCAcDAgQV AggDBBYCAwECHgECF4AACgkQ8a5sCGs03G3Djwf7B8CKV9ZOvarZlQsaqzzA5AVM /jB+oi86tjCDvTB4pgdbVUcni4MxWzTCU5OUXPbjzaFr+liB8tsGbSSFCJxMSdMu DMhCcjU48XB3aSTXfvrKRqf/7kBdIqRqNB4KvmXGQI6NQl68O/BNQ8zX++M39GXR tCzKS9VcWOdra8KN45ADVjVx3v8MCBguc9GTXM5EXxdoIUjU9ClWAklfWWJZBHGB 9NnPITfup28rjxXoITgXjF5Irk3uOEyleQ3fDM1uzLw5u/EHNh+uQJgV5Y5YHY9g b9VT97iIfimw4vJ5stCKCwf7wz9VbGdBpNRMNepGV+/wVOg37nl2nFLZV9pCtLQg Um9tYWluIEJpZ25vbiA8cm9tYWluQGJpZ25vbi5tZT6JATgEEwECACIFAk1ytV4C Gy8GCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEPGubAhrNNxtHVwH+wTw7zcC Xx+pQPR+s4gbtKwb0L95psuN5fUl2f9jEnAWwOSvkYoBmbA0+dYCtokFGIbFNmzU ajBXwFkVEdphyVYxH4DnIwmwmafMA/QjYiQB91zRPpZQKCG1ioKlM/He9wJsrskl zaqUEs3BuAx4n45vsiuERjJFwejbfqlI3PYSnS1/ncVpvn1pREn3a+QJ2hDMx2Ns Znh0HkhJ6LLB8F2Z5bFSDcpTAiNgECRc3eleiTQT+89RrbvpPUi8iIFUcWmGxYBU Atk6miGgjKpy98Ezc1LWfPOSaZjNBxhvtlicx+bDqkcMYPakZ1gLNWWyW2iNljDF KiLhXstA/eNC3qC0IVJvbWFpbiBCaWdub24gPHJvbWFpbkBzeW1saW5rLm1lPokB OAQTAQIAIgUCTXK1SgIbLwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ8a5s CGs03G2b6Qf/VjmEs0pBdpg/Ewj7HmikxIQ7lvX5um1YHjLyLNdp6W6wylmzh3u5 dsaajEWmPbUhaDre7gTtxPNJFs6FXksfquUiZySXA45R+XH8Qh/TWJAY8FjdaZwB tgOn4rka+n0rQeG1fklHw4Z3AREm/GmfeA63nkmS5SvdXskAz1iXB20Hyb6bGy5b Y7ZJNhJ8Ocz2THizc2od3Bx8Pzw3DHs+NaW5sK/NJJN9QgpdLA4lo0KeidCALEXF CReDEaT4Cs/Ds3u8QkuR+s+8qN5BOYcIWz79T/hLFw46SezEz9gjJ+NR1nIMd2um bwnyciYEGVOfG4DsU/IRwnbk+HWcLNTFnw== =mTlW -----END PGP PUBLIC KEY BLOCK----- weboob-1.2/modules/750g/000077500000000000000000000000001303450110500147535ustar00rootroot00000000000000weboob-1.2/modules/750g/__init__.py000066400000000000000000000014531303450110500170670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import SevenFiftyGramsModule __all__ = ['SevenFiftyGramsModule'] weboob-1.2/modules/750g/browser.py000066400000000000000000000034641303450110500170170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser, URL from .pages import RecipePage, ResultsPage __all__ = ['SevenFiftyGramsBrowser'] class SevenFiftyGramsBrowser(PagesBrowser): BASEURL = 'http://www.750g.com' search = URL('/recettes_(?P.*).htm', ResultsPage) recipe = URL('/(?P.*).htm', RecipePage) def iter_recipes(self, pattern): try: self.search.go(pattern=pattern.replace(' ', '_')) except BrowserHTTPNotFound: return [] if isinstance(self.page, ResultsPage): return self.page.iter_recipes() return [self.get_recipe_content()] def get_recipe(self, id, recipe=None): try: self.recipe.go(id=id) return self.get_recipe_content(recipe) except BrowserHTTPNotFound: return def get_recipe_content(self, recipe=None): recipe = self.page.get_recipe(obj=recipe) comments = list(self.page.get_comments()) if comments: recipe.comments = comments return recipe weboob-1.2/modules/750g/favicon.png000066400000000000000000000026251303450110500171130ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME: <&`MIDATxKEU=Oz(&j' ºAI؋H xp\5K0 b`]!ȒQfg{ʃ5Rhhƙ^# n$EҼۋNr:7C@&+i7LŻNuº ۷늀8%4G'MB*D q!Im4O,SG]䠀G !OdUD2S+܏.aRDHjs=y߯+qUJk~SeW=w>Y.TR)'Qϟ Î}v:RՖ}&~(Vq0"әLC`ol#a B!]xVu7 |MBk`\x4_֚7 1!xlC CfcaY"%{\ Ԛ 7p\UcF@Ek>($eﱁ|iUN(cϷV{מk Ak* y޴f!KOb#jݎSŨr`P\>7,wˋýc ILEڸul&RrK+U7hԞ Rrl.Xk2'1NY;jiy\+,1TY@k&,]wQ)~RZ8dld,)AcF]TCA^z!XҚWPڳZd1 9cg<687)hOsG*AcB.ϸ.3߽)eELR0dTҺ-vx R u͌^К0&%;LjE۞GIkpLzO4. from weboob.capabilities.recipe import CapRecipe, Recipe from weboob.tools.backend import Module from .browser import SevenFiftyGramsBrowser import unicodedata def strip_accents(s): return ''.join(c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn') __all__ = ['SevenFiftyGramsModule'] class SevenFiftyGramsModule(Module, CapRecipe): NAME = '750g' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'750g French recipe website' LICENSE = 'AGPLv3+' BROWSER = SevenFiftyGramsBrowser def get_recipe(self, id): return self.browser.get_recipe(id) def iter_recipes(self, pattern): return self.browser.iter_recipes(strip_accents(unicode(pattern)).encode('utf-8')) def fill_recipe(self, recipe, fields): if 'nb_person' in fields or 'instructions' in fields: recipe = self.browser.get_recipe(recipe.id, recipe) return recipe OBJECTS = { Recipe: fill_recipe, } weboob-1.2/modules/750g/pages.py000066400000000000000000000073601303450110500164320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.recipe import Recipe, Comment from weboob.capabilities.base import NotAvailable from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, Env, Filter, DateTime, CleanDecimal from weboob.browser.filters.html import CleanHTML from datetime import datetime, date, time class Time(Filter): def filter(self, el): _time = DateTime(CleanText(el, replace=[('PT', '')]), default=None)(self) if _time: _time_ = _time - datetime.combine(date.today(), time(0)) return _time_.seconds // 60 class ResultsPage(HTMLPage): """ Page which contains results as a list of recipies """ @pagination @method class iter_recipes(ListElement): item_xpath = '//section[has-class("c-recipe-row")]' def next_page(self): return CleanText('//li[@class="suivante"]/a/@href')(self) class item(ItemElement): klass = Recipe def condition(self): return not CleanText('./div[@class="c-recipe-row__media"]/span[@class="c-recipe-row__video"]/@class', default=None)(self) obj_id = Regexp(CleanText('./div/h2/a/@href'), '/(.*).htm') obj_title = CleanText('./div/h2/a') obj_thumbnail_url = CleanText('./div/img/@src') obj_short_description = CleanText('./div/p') class RecipePage(HTMLPage): """ Page which contains a recipe """ @method class get_comments(ListElement): item_xpath = '//div[has-class("c-comment__row")]' class item(ItemElement): klass = Comment def validate(self, obj): return obj.id obj_id = CleanText('./@data-id') obj_author = CleanText('./article/div/header/strong/span[@itemprop="author"]') obj_text = CleanText('./article/div/div/p') @method class get_recipe(ItemElement): klass = Recipe obj_id = Env('id') obj_title = CleanText('//h1[has-class("fn")]') def obj_ingredients(self): ingredients = [] for el in self.page.doc.xpath('//li[@class="ingredient"]'): ingredients.append(CleanText('.')(el)) return ingredients obj_cooking_time = Time('//time[@itemprop="cookTime"]/@datetime') obj_preparation_time = Time('//time[@itemprop="prepTime"]/@datetime') def obj_nb_person(self): return [CleanDecimal('//span[@class="yield"]', default=0)(self)] obj_instructions = CleanHTML('//div[has-class("c-recipe-steps__item")]') obj_picture_url = CleanText('(//picture[has-class("c-swiper__media")]/img/@src)[1]', default='') obj_author = Regexp(CleanText('//meta[@name="description"]/@content', default=''), '.* par (.*)', default=NotAvailable) weboob-1.2/modules/750g/test.py000066400000000000000000000026121303450110500163050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest import itertools class SevenFiftyGramsTest(BackendTest): MODULE = '750g' def test_recipe(self): recipes = list(itertools.islice(self.backend.iter_recipes('fondue'), 0, 20)) for recipe in recipes: full_recipe = self.backend.get_recipe(recipe.id) self.assertTrue(full_recipe.instructions, 'No instructions for %s' % recipe.id) self.assertTrue(full_recipe.ingredients, 'No ingredients for %s' % recipe.id) self.assertTrue(full_recipe.title, 'No title for %s' % recipe.id) self.assertTrue(full_recipe.preparation_time, 'No preparation time for %s' % recipe.id) weboob-1.2/modules/__init__.py000066400000000000000000000000001303450110500163700ustar00rootroot00000000000000weboob-1.2/modules/adecco/000077500000000000000000000000001303450110500155075ustar00rootroot00000000000000weboob-1.2/modules/adecco/__init__.py000066400000000000000000000014321303450110500176200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AdeccoModule __all__ = ['AdeccoModule'] weboob-1.2/modules/adecco/browser.py000066400000000000000000000052601303450110500175470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import SearchPage, AdvertPage, AdvertsJsonPage import urllib from datetime import date, timedelta __all__ = ['AdeccoBrowser'] class AdeccoBrowser(PagesBrowser): BASEURL = 'http://www.adecco.fr' TIMEOUT = 30 search_page = URL('/resultats-offres-emploi/\?k=(?P.*)&l=(?P.*)&pageNum=1&display=50', SearchPage) json_page = URL('/AdeccoGroup.Global/api/Job/AsynchronousJobSearch/', AdvertsJsonPage) advert_page = URL('/offres-d-emploi/\?ID=(?P<_id>.*)', '/offres-d-emploi/.*', AdvertPage) def call_json(self, params, date_min=None): self.session.headers.update({"Accept": "application/json, text/javascript, */*; q=0.01", "X-Requested-With": "XMLHttpRequest"}) return self.json_page.go(data=params).iter_job_adverts(data=params, date_min=date_min) def search_job(self, pattern=None): if pattern: return self.advanced_search_job(job=pattern) return [] def advanced_search_job(self, publication_date=0, contract_type=None, conty=None, activity_domain=None, job='', town=''): params = self.search_page.go(job=urllib.quote_plus(job.encode('utf-8')), town=urllib.quote_plus(town.encode('utf-8'))).get_post_params() if contract_type: self.page.url += '&employmenttype=%s' % contract_type if conty: self.page.url += '&countrysubdivisionfacet=%s' % conty if activity_domain: self.page.url += '&industryfacet=%s' % activity_domain date_min = date.today() - timedelta(days=publication_date) if publication_date > 0 else None params['filterUrl'] = self.page.url return self.call_json(params, date_min=date_min) def get_job_advert(self, _id, advert): return self.advert_page.go(_id=_id).get_job_advert(obj=advert) weboob-1.2/modules/adecco/favicon.png000066400000000000000000000121561303450110500176470ustar00rootroot00000000000000PNG  IHDR??W_sBIT|d pHYsodIDAThu?]ξq8!93$EVuMb;I  p/]~H]@ @"hpM)R$K%"M-&%qrf8Y{Iy9{gyF\ ܢ"oU VoU VoU VoU7d(a8SzJN޸RSD{ 2%TL#@A^X!1ey_җL \cmJƕ0Q(ډޔB^(8Qf<.*| 4//}OO{ RǗ\GTJiT KOʵL<+o$\ORO^ ''rN"3ʇePj(Q(=*XP! }oflVsisZC|Q-JoE8N.Y^^B/Rҗ9@pN7-X8PE 2peVK'$'JGtUdQ7T(}>P&$s3Qբ]JY2ܤ'Y}'`E% hڿ;='=9 \/`FKVB+;S@lNJ&+~xgXt^2l0V[^C'J֞\ԵRr 1KE=үt22"/KA+W/ Jo҇ zMIJ@74KuDNP+#eYzͨ-]#p!īlV\dB@RHI$]#ي%m#ɥDv,Q޳B"Cw)2@{ :1xt˭'D^MGUWBK\*:=~tڣD,=+''˞`-%7`F8YV`A фE,"hڌ M*LS [Vj\ ֤@ 5}BrI+fsȊAc2Oj0B@P^H($Ж"F;BDYF#cBDlİ:V ;ws뱢i e M"Ӭus< qqܱ#1RD;Z\& DU0RNDJRkE;"9"xr&FF < '62hyϖwlYK:LRAA;677R"PZCȮQʪsKh ؐFCMb`u"F^csA-yhd*NQ!JÕ+K(&3db41tàj) JF7MQUjBhY*sPuj}hG "j}$hSCMȡޏcjeT:!T£0~+1Nwo`8wJh UL N]J"b=^j^D73⤆r^d!58#VrRKEzJYU6i5ym18.HYVI֔H9orq `ӱL /H_  g".T9xLb+# ck|K Lpӭ7X.ɐԨ(((}5κi}U.&- uiע#"  .W40:JV1lc\{&z3Zsk,>KhԠGD5?Lm,Ih4O]s6'1*Ih'sOO!f`sR'I C/PN|p7~ ;9=w%U> */P7 I"tӔ8!a(C?8Ss>u0wwhu8)koR_AWۻN;[Lc t=}vpx`?as3g+CO#uh{{uv^WFtlA;vzv'p}1ě&y SNP AAFbP֚WNupNw7?>X׾Je\ƮlC~A# u  go}!&O㣿y^ o_bdd {x$ۧL!N-EwAµ fxs0tGGyŗx s_<g^cuiZkkͶ}@ĕϰ~}~coߝ~6; s:o{l ݂n$ScBrpgxywϜoFsb}sݏ Wx?~_cbbxdY.wy/I`0g?IW O8fyۯ>f-6\ԞVEO0~> :F?KK,(?yo_o(LMCQp앗Z- I8×wn:S=` sCj=ӴU.OBOz82L_Kl1Q-n:bw.X]_\$o ߽ʌv#7!:F{}}my=_ w0TazW7h_}ڝTwNs}x$T!Jz{뜾pvȻm^etvUS mUg`0..CmyN%+FQ]oSTȊ-F& M%ϻ%vI$Y`+u + wQ s_CAt%0TLh]k0/<̋?zgꒉ1G/qJ_!A"O")0#?I}I\s.`}l U :3,;Q☵ EڵULL>dhvr6[saXV:MI6Lz)c7Uz(?[?> ׏+/ӼH0m/?_sԘbq[OFϽNuu38̃>ʽS;9 O3p&_8FLm1Y>k|xgkCJ]c x,b#-ꃃtzKOqԿvReq:{~]3\~|+kT1Y& O<@Ĵ>+?gg#>w>Ɨŷvr79eO _*0˿Gd9Gw7Yxf;f#3L Mp ҷs݌f^~a=ϝcϧAOpgFL_UF=1ԦeQU2Gy0/0deX&k5^wIؾs'o0{(A43V>~nvbr9Ο s%1<3TYX19J[  15Ydlik'ާJM8g &HackϿ󯱫+eqib{,0#}E& 6k7\w};xgڛ'N30λSۘvΥSg;YSꝂHgl.C Wp jrdE589 3Ӱqqt{h:.v}UbЈI@"6W399ř` Rx\^]f6._c^C)Ef6lZJl؂j Id3ɕIEC"5\8KeJ3O$יIUu]7(VZ5&huVFC&=hW+}P(RĊv"(fƠzK%FH" }F&]QZ#}@ @ A!!r+56-znc*UYRUZ` ") lNyJU2 ɕ T"@: 0{Ԃc"|NW \ RG!"<jBWx1'K2!BzAɄ.Z5! 0Std0B"R HYq_"F!-" i4Z(8BYP"#C{pSDlѐ$J ((ʬH*QdF(ExBs-"#Gr-}@RP1n(f .4mNb=P!Ѵ KEk`!/0J9(Plg&@lȔ#sPD^ (:BiM5k-}ReL&%1<R"C.-d^ .҄pB+U%[  E93V,/qTHc xF@zG,iQ9M-QoR,f%%Q婉a*1Nl(0B$q [hysvDZa2Oq"Z U(P%)8&BA MZ#wK,8 j(9(DA[ 9uDJ Ky|,,( H8)pJ ňommm- &- umIENDB`weboob-1.2/modules/adecco/module.py000066400000000000000000000205471303450110500173560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from weboob.capabilities.job import CapJob, BaseJobAdvert from .browser import AdeccoBrowser __all__ = ['AdeccoModule'] class AdeccoModule(Module, CapJob): NAME = 'adecco' DESCRIPTION = u'adecco website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' BROWSER = AdeccoBrowser publicationDate_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '000000': u'-- Indifferent --', '2': u'Moins de 48 heures', '7': u'Moins de 1 semaine', '14': u'Moins de 2 semaines', }.iteritems())]) type_contract_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'-- Indifferent --', 'ADCFREMP005': u'CDD', 'ADCFREMP004': u'CDI', 'ADCFREMP003': u'Intérim', 'ADCFREMP009': u'Autres', 'ADCFREMP010': u'Libéral', }.iteritems())]) places_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'-- Indifferent --', 'AIN': u'Ain', 'AISNE': u'Aisne', 'ALLIER': u'Allier', 'ALPES-DE-HAUTE-PROVENCE': u'Alpes-De-Haute-Provence', 'ALPES-MARITIMES': u'Alpes-Maritimes', 'ARDECHE': u'Ardeche', 'ARDENNES': u'Ardennes', 'ARIEGE': u'Ariege', 'AUBE': u'Aube', 'AUDE': u'Aude', 'AVEYRON': u'Aveyron', 'BAS-RHIN': u'Bas-Rhin', 'BOUCHES-DU-RHONE': u'Bouches-Du-Rhone', 'CALVADOS': u'Calvados', 'CANTAL': u'Cantal', 'CHARENTE': u'Charente', 'CHARENTE-MARITIME': u'Charente-Maritime', 'CHER': u'Cher', 'CORREZE': u'Correze', 'CORSE-DU-SUD': u'Corse du Sud', 'COTE-D%27OR': u'Cote D\'Or', 'COTES-D%27ARMOR': u'Cotes D\'Armor', 'CREUSE': u'Creuse', 'DEUX-SEVRES': u'Deux-Sevres', 'DORDOGNE': u'Dordogne', 'DOUBS': u'Doubs', 'DROME': u'Drome', 'ESSONNE': u'Essonne', 'EURE': u'Eure', 'EURE-ET-LOIR': u'Eure-Et-Loir', 'FINISTERE': u'Finistere', 'GARD': u'Gard', 'GERS': u'Gers', 'GIRONDE': u'Gironde', 'GUADELOUPE': u'Guadeloupe', 'GUYANE': u'Guyane', 'HAUT-RHIN': u'Haut-Rhin', 'HAUTE-CORSE': u'Haute-Corse', 'HAUTE-GARONNE': u'Haute-Garonne', 'HAUTE-LOIRE': u'Haute-Loire', 'HAUTE-MARNE': u'Haute-Marne', 'HAUTE-SAONE': u'Haute-Saone', 'HAUTE-SAVOIE': u'Haute-Savoie', 'HAUTE-VIENNE': u'Haute-Vienne', 'HAUTES-ALPES': u'Hautes-Alpes', 'HAUTES-PYRENEES': u'Hautes-Pyrenees', 'HAUTS-DE-SEINE': u'Hauts-De-Seine', 'HERAULT': u'Herault', 'ILLE-ET-VILAINE': u'Ille-Et-Vilaine', 'INDRE': u'Indre', 'INDRE-ET-LOIRE': u'Indre-Et-Loire', 'ISERE': u'Isere', 'JURA': u'Jura', 'LA+REUNION': u'La Reunion', 'LANDES': u'Landes', 'LOIR-ET-CHER': u'Loir-Et-Cher', 'LOIRE': u'Loire', 'LOIRE-ATLANTIQUE': u'Loire-Atlantique', 'LOIRET': u'Loiret', 'LOT': u'Lot', 'LOT-ET-GARONNE': u'Lot-Et-Garonne', 'LOZERE': u'Lozere', 'MAINE-ET-LOIRE': u'Maine-Et-Loire', 'MANCHE': u'Manche', 'MARNE': u'Marne', 'MARTINIQUE': u'Martinique', 'MAYENNE': u'Mayenne', 'MAYOTTE': u'Mayotte', 'MEURTHE-ET-MOSELLE': u'Meurthe et Moselle', 'MEUSE': u'Meuse', 'MONACO': u'Monaco', 'MORBIHAN': u'Morbihan', 'MOSELLE': u'Moselle', 'NIEVRE': u'Nievre', 'NORD': u'Nord', 'OISE': u'Oise', 'ORNE': u'Orne', 'PARIS': u'Paris', 'PAS-DE-CALAIS': u'Pas-de-Calais', 'PUY-DE-DOME': u'Puy-de-Dome', 'PYRENEES-ATLANTIQUES': u'Pyrenees-Atlantiques', 'PYRENEES-ORIENTALES': u'Pyrenees-Orientales', 'RHONE': u'Rhone', 'SAONE-ET-LOIRE': u'Saone-et-Loire', 'SARTHE': u'Sarthe', 'SAVOIE': u'Savoie', 'SEINE-ET-MARNE': u'Seine-et-Marne', 'SEINE-MARITIME': u'Seine-Maritime', 'SEINE-SAINT-DENIS': u'Seine-Saint-Denis', 'SOMME': u'Somme', 'ST+PIERRE+ET+MIQUELON': u'St Pierre et Miquelon', 'SUISSE': u'Suisse', 'TARN': u'Tarn', 'TARN-ET-GARONNE': u'Tarn-et-Garonne', 'TERRITOIRE+DE+BELFORT': u'Territoire de Belfort', 'VAL-D%27OISE': u'Val-D\'Oise', 'VAL-DE-MARNE': u'Val-De-Marne', 'VAR': u'Var', 'VAUCLUSE': u'Vaucluse', 'VENDEE': u'Vendee', 'VIENNE': u'Vienne', 'VOSGES': u'Vosges', 'YONNE': u'Yonne', 'YVELINES': u'Yvelines', }.iteritems())]) activityDomain_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ 'Z': '-- Indifferent --', 'A': u'Accueil - Secrétariat - Fonctions Administratives', 'B': u'Achats - Juridique - Qualité - RH - Direction', 'C': u'Agriculture - Viticulture - Pêche - Espaces Verts', 'D': u'Automobile', 'E': u'Banque - Finance - Gestion Comptabilité - Assurance', 'F': u'Bâtiment - Travaux Publics - Architecture - Immobilier', 'G': u'Bureaux d\'Etudes - Méthodes', 'H': u'Commerce - Vente - Grande Distribution', 'I': u'Environnement - Nettoyage - Sécurité', 'J': u'Hôtellerie - Restauration - Métiers de Bouche', 'K': u'Industrie', 'L': u'Informatique - Technologie de l\'Information', 'M': u'Logistique - Manutention - Transport', 'N': u'Marketing - Communication - Imprimerie - Edition', 'O': u'Médical - Paramédical - Esthétique', 'P': u'Pharmacie (Industrie, Officine) - Recherche clinique', 'Q': u'Télémarketing - Téléservices', 'R': u'Tourisme - Loisirs - Spectacle - Audiovisuel', }.iteritems())]) CONFIG = BackendConfig(Value('job', label='Job name', masked=False, default=''), Value('town', label='Town name', masked=False, default=''), Value('place', label=u'County', choices=places_choices), Value('publication_date', label=u'Publication Date', choices=publicationDate_choices), Value('contract', labe=u'Contract type', choices=type_contract_choices), Value('activity_domain', label=u'Activity Domain', choices=activityDomain_choices, default=''), ) def search_job(self, pattern=None): for advert in self.browser.search_job(pattern): yield advert def advanced_search_job(self): activity_domain = self.config['activity_domain'].get() if self.config['activity_domain'].get() != u'Z' else None for advert in self.browser.advanced_search_job(publication_date=int(self.config['publication_date'].get()), contract_type=self.config['contract'].get(), conty=self.config['place'].get(), activity_domain=activity_domain, job=self.config['job'].get(), town=self.config['town'].get() ): yield advert def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} weboob-1.2/modules/adecco/pages.py000066400000000000000000000115331303450110500171630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import requests from weboob.browser.pages import HTMLPage, pagination, JsonPage from weboob.browser.elements import ItemElement, method, DictElement from weboob.browser.filters.standard import CleanText, Regexp, Date from weboob.browser.filters.html import CleanHTML from weboob.browser.filters.json import Dict from weboob.browser.filters.javascript import JSVar from weboob.capabilities.job import BaseJobAdvert from weboob.capabilities.base import empty class SearchPage(HTMLPage): def get_post_params(self): return {'facetSettingId': JSVar(CleanText('//script'), var='_ItemName')(self.doc), 'currentLanguage': JSVar(CleanText('//script'), var='_CurrentLanguage')(self.doc), 'clientId': JSVar(CleanText('//script'), var='_ClientId')(self.doc), 'branchId': JSVar(CleanText('//script'), var='_BranchId')(self.doc), 'clientName': JSVar(CleanText('//script'), var='_ClientName')(self.doc)} class AdvertsJsonPage(JsonPage): @pagination @method class iter_job_adverts(DictElement): item_xpath = 'Items' def next_page(self): if len(self.page.doc['Pagination']) >= 2: if self.page.doc['Pagination'][-2]['keyName'] == u'Suivant': url = self.page.doc['Pagination'][-2]['valueName'] self.env['data']['filterUrl'] = u'http://www.adecco.fr%s' % url return requests.Request("POST", self.page.url, data=self.env['data']) class item(ItemElement): klass = BaseJobAdvert def validate(self, advert): if empty(advert.publication_date) or not self.env['date_min']: return advert if advert.publication_date >= self.env['date_min']: return advert obj_id = Dict('JobId') obj_title = Dict('JobTitle') obj_place = Dict('JobLocation') obj_publication_date = Date(Dict('PostedDate')) class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert def obj_id(self): _id = Regexp(CleanText('//meta[@property="og:url"]/@content'), '.*\?ID=(.*)', default=None)(self) if _id is None: _id = JSVar(CleanText('//script'), var='_JobDetailsId')(self) return _id def obj_title(self): title = CleanText('//meta[@property="og:title"]/@content', default=None)(self) if title is None: title = JSVar(CleanText('//script'), var='_JobTitle')(self) return title def obj_place(self): place = CleanText('//span[@itemprop="jobLocation"]', default=None)(self) if not place: place = Regexp(CleanText('//meta[@property="og:title"]/@content'), u'.*\ à (.*)')(self) return place def obj_publication_date(self): date = Date(CleanText('//time[@itemprop="startDate"]'), default=None)(self) if date is None: date = Date(CleanText('//span[@id="posted-date"]'))(self) return date obj_contract_type = CleanText('//li[@class="job--meta_employment-type"]/div/div/span[@class="job-details-value"]') # obj_pay = CleanText('//div[@class="jobGreyContain"]/div/div[4]/span[@class="value"]') def obj_job_name(self): job_name = Regexp(CleanText('//meta[@property="og:title"]/@content'), '(.*)\|.*', default=None)(self) if job_name is None: job_name = JSVar(CleanText('//script'), var='_JobTitle')(self) return job_name obj_description = CleanHTML('//div[@class="VacancyDescription"]') def obj_url(self): url = CleanText('//meta[@property="og:url"]/@content', default=None)(self) if url is None: url = JSVar(CleanText('//script'), var='_JobUrl')(self) if not url.startswith('http'): url = 'www.adecco.fr%s' % url return url weboob-1.2/modules/adecco/test.py000066400000000000000000000035341303450110500170450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.value import Value import itertools class AdeccoTest(BackendTest): MODULE = 'adecco' def setUp(self): if not self.is_backend_configured(): self.backend.config['publication_date'] = Value(value='000000') self.backend.config['place'] = Value(value='') self.backend.config['job'] = Value(value='') self.backend.config['town'] = Value(value='') self.backend.config['contract'] = Value(value='ADCFREMP004') def test_adecco_search(self): l = list(itertools.islice(self.backend.search_job(u'manutentionnaire'), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_adecco_advanced_search(self): l = list(itertools.islice(self.backend.advanced_search_job(), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) weboob-1.2/modules/afer/000077500000000000000000000000001303450110500152065ustar00rootroot00000000000000weboob-1.2/modules/afer/__init__.py000066400000000000000000000014271303450110500173230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AferModule __all__ = ['AferModule'] weboob-1.2/modules/afer/browser.py000066400000000000000000000046741303450110500172560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from random import randint from weboob.browser import URL, LoginBrowser, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, IndexPage, BadLogin, AccountDetailPage, AccountHistoryPage class AferBrowser(LoginBrowser): BASEURL = 'https://adherent.gie-afer.fr' login = URL('/web/ega.nsf/listeAdhesions\?OpenForm', LoginPage) bad_login = URL('/names.nsf\?Login', BadLogin) index = URL('/web/ega.nsf/listeAdhesions\?OpenForm', IndexPage) account_detail = URL('/web/ega.nsf/soldeEpargne\?openForm', AccountDetailPage) account_history = URL('/web/ega.nsf/generationSearchModule\?OpenAgent', AccountHistoryPage) history_detail = URL('/web/ega.nsf/WOpendetailOperation\?OpenAgent', AccountHistoryPage) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.go() self.page.login(self.username, self.password) if self.bad_login.is_here(): raise BrowserIncorrectPassword() @need_login def iter_accounts(self): self.index.stay_or_go() return self.page.iter_accounts() @need_login def iter_investments(self, account): self.account_detail.go(params={'nads': account.id}) return self.page.iter_investments() @need_login def iter_history(self, account): al = randint(0, 1000) data = {'cdeAdh': account.id, 'al': al, 'page': 1, 'form': 'F'} self.account_history.go(data={'cdeAdh': account.id, 'al': al, 'page': 1, 'form': 'F'}) return self.page.iter_history(data=data) weboob-1.2/modules/afer/favicon.png000066400000000000000000000036241303450110500173460ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME  :v#iTXtCommentCreated with GIMPd.ebKGDIDATx[{L[e?@al<d7-0h duנ0Yh L1>Kn@tC9d3M?g[b/llrauOZ3>_9+C,Ԝ'8BSs^az8\㻡`5dln# ~.`T$tT7NWal@+3v^mb+δ@vFҎ.ppwgn-eK۳f'%@\KDŽ\R9yXr}lV1 "Q{0$1lFTYɻcAz/_郲;!!uQX] s9{W1m'JYQK0$[? ر!B I'..!-,zEP/p~ ]# m'Y, H''̹_ӈaGq1GRز6U  0obCR7B $@(ܔѵ۷8Ӫ|&?I҅H!TB|| O3!#IbرXmrj p~ ׬[P$|.( ēY:@=MS\gޠl V0v 624 B HAVJ=S41q1Jb´:BE]I XATL@"]tufESDAVGUGu6: X\e["xY#u͞?@EW#*(qDWCDmp (8P'.kR+xO<Lޖ!胜=%_z؁n޲4'@X Ԍ&/UevVD6v1dG(RեY03exS%!NPX p ~X 6M08#=: g!ݸbT%QPwz>j-,Ad |/,eR梋0&$*#@{J{(|zfP\%@j#tWrꑀPa}z; AXHG#D'`VAp,D4(I IM"K!u@W WDIB r AVF<,` 1!ᕹ TC/MҲX اMٚ\h_ C[U WěSvl'5:N wk[V {wnI%P+I›kJOZ V3e6E3yS8O`%z m)JH{C?}_1IENDB`weboob-1.2/modules/afer/module.py000066400000000000000000000060471303450110500170540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import find_object from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AferBrowser __all__ = ['AferModule'] class AferModule(Module, CapBank): NAME = 'afer' DESCRIPTION = u'afer website' MAINTAINER = u'James GALT' EMAIL = 'jgalt@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = AferBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Username', regexp='[A-z]\d+', masked=False), ValueBackendPassword('password', label=u"mdp", regexp='\d+')) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def get_account(self, id): """ Get an account from its ID. :param id: ID of the account :type id: :class:`str` :rtype: :class:`Account` :raises: :class:`AccountNotFound` """ return find_object(self.iter_accounts(), id=id, error=AccountNotFound) def iter_accounts(self): """ Iter accounts. :rtype: iter[:class:`Account`] """ return self.browser.iter_accounts() def iter_coming(self, account): """ Iter coming transactions on a specific account. :param account: account to get coming transactions :type account: :class:`Account` :rtype: iter[:class:`Transaction`] :raises: :class:`AccountNotFound` """ raise NotImplementedError() def iter_history(self, account): """ Iter history of transactions on a specific account. :param account: account to get history :type account: :class:`Account` :rtype: iter[:class:`Transaction`] :raises: :class:`AccountNotFound` """ return self.browser.iter_history(account) def iter_investment(self, account): """ Iter investment of a market account :param account: account to get investments :type account: :class:`Account` :rtype: iter[:class:`Investment`] :raises: :class:`AccountNotFound` """ return self.browser.iter_investments(account) weboob-1.2/modules/afer/pages.py000066400000000000000000000141771303450110500166710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from random import randint import requests from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, Date, Async, BrowserURL from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.capabilities.bank import Account, Investment, Transaction from weboob.capabilities.base import NotAvailable class LoginPage(HTMLPage): def login(self, login, passwd): form = self.get_form(name='_DominoForm') form['Username'] = login form['password'] = passwd form.submit() def is_here(self): return bool(self.doc.xpath('//form[@name="_DominoForm"]')) class IndexPage(LoggedPage, HTMLPage): def is_here(self): return bool(self.doc.xpath('//img[contains(@src, "deconnexion.jpg")]')) @method class iter_accounts(ListElement): item_xpath = '//div[@id="adhesions"]/table//tr[td//a]' class item(ItemElement): klass = Account obj_id = CleanText('.//a') obj_label = CleanText('.//td[3]') obj_currency = u'EUR' def obj_balance(self): if not '%' in CleanText('.//td[last()-2]')(self): return CleanDecimal('.//td[last()-2]', replace_dots=True)(self) elif not '%' in CleanText('.//td[last()-3]')(self): return CleanDecimal('.//td[last()-3]', replace_dots=True)(self) else: return NotAvailable obj_type = Account.TYPE_LIFE_INSURANCE class AccountDetailPage(LoggedPage, HTMLPage): def is_here(self): return bool(self.doc.xpath('//*[@id="linkadhesion"]/a')) @method class iter_investments(ListElement): item_xpath = '//div[@id="savingBalance"]/table[1]//tr' class item(ItemElement): def condition(self): return self.el.xpath('./td[contains(@class,"dateUC")]') klass = Investment obj_label = CleanText('.//td[1]') obj_code = NotAvailable obj_description = NotAvailable obj_quantity = CleanDecimal('.//td[7]', replace_dots=True, default=NotAvailable) obj_unitvalue = CleanDecimal('.//td[5]', replace_dots=True, default=NotAvailable) obj_valuation = CleanDecimal('.//td[3]', replace_dots=True) obj_vdate = Date(Regexp(CleanText('.//td[2]'), '(((0[1-9]|[12][0-9]|3[01])[- /.]' '(0[13578]|1[02])|(0[1-9]|[12][0-9]|30)[- /.](0[469]|11)|' '(0[1-9]|1\d|2[0-8])[- /.]02)[- /.]\d{4}|29[- /.]02[- /.]' '(\d{2}(0[48]|[2468][048]|[13579][26])|([02468][048]|' '[1359][26])00))$', nth=0), dayfirst=True) def obj_unitprice(self): try: return CleanDecimal(replace_dots=True, default=NotAvailable).filter( self.el.xpath('.//td[6]')[0].text) / \ CleanDecimal(replace_dots=True, default=NotAvailable).filter( self.el.xpath('.//td[7]')[0].text) except TypeError: return NotAvailable def obj_diff(self): try: return self.obj.valuation - (self.obj.unitprice * self.obj.quantity) except TypeError: return NotAvailable class AccountHistoryPage(LoggedPage, HTMLPage): @pagination @method class iter_history(ListElement): item_xpath = '//table[@class="confirmation-table"]//tr' def next_page(self): if self.page.doc.xpath("//table[@id='tabpage']//td"): array_page = self.page.doc.xpath("//table[@id='tabpage']//td")[0][3].text curr_page, max_page = array_page.split(' ')[1::2] if int(curr_page) < int(max_page): data = self.env['data'] data['page'] += 1 data['al'] = randint(1, 1000) return requests.Request("POST", self.page.url, data=data) return class item(ItemElement): def load_details(self): a = self.el.xpath(".//img[@src='../../images/commun/loupe.png']") if len(a) > 0: values = a[0].get('onclick').replace('OpenDetailOperation(', '') \ .replace(')', '').replace(' ', '').replace("'", '').split(',') keys = ["nummvt", "&numads", "dtmvt", "typmvt"] data = dict(zip(keys, values)) url = BrowserURL('history_detail')(self) r = self.page.browser.async_open(url=url, data=data) return r return None klass = Transaction obj_date = obj_rdate = obj_vdate = Date(CleanText('.//td[3]')) obj_label = CleanText('.//td[1]') def obj_amount(self): am = CleanDecimal('.//td[2]', replace_dots=True, default=NotAvailable)(self) if am is not NotAvailable: return am return (Async('details') & CleanDecimal('//div//tr[2]/td[2]', replace_dots=True, default=NotAvailable))( self) class BadLogin(HTMLPage): pass weboob-1.2/modules/agendaculturel/000077500000000000000000000000001303450110500172705ustar00rootroot00000000000000weboob-1.2/modules/agendaculturel/__init__.py000066400000000000000000000014521303450110500214030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AgendaculturelModule __all__ = ['AgendaculturelModule'] weboob-1.2/modules/agendaculturel/browser.py000066400000000000000000000041431303450110500213270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import BasePage from urlparse import urlparse import re class AgendaculturelBrowser(PagesBrowser): BASEURL = '' base = URL('https://www.agendaculturel.fr/search_bw', 'https://(?P\d{2}).agendaculturel.fr/(?P<_id>.*).html', 'https://\d{2}.agendaculturel.fr/.*', BasePage) def __init__(self, place, *args, **kwargs): self.default_place = place PagesBrowser.__init__(self, *args, **kwargs) def set_base_url(self, place): if not place: place = self.default_place self.base.go(data={'query': place}) parsed_uri = urlparse(self.page.url) self.BASEURL = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri) def list_events(self, place, date_from, date_to, categories=None): self.set_base_url(place) query_date_from = date_from.strftime('%Y%m') self.page.search_events(query_date_from) region = re.match('https://(\d{2}).agendaculturel.fr/.*', self.page.url).group(1) return self.page.list_events(region=region, date_from=date_from, date_to=date_to, categories=categories) def get_event(self, id, event=None): splitted_id = id.split('.') region = splitted_id[0] _id = splitted_id[1] return self.base.go(_id=_id, region=region).get_event(obj=event) weboob-1.2/modules/agendaculturel/calendar.py000066400000000000000000000020451303450110500214140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS class AgendaculturelEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.timezone = u'Europe/Paris' weboob-1.2/modules/agendaculturel/favicon.png000077500000000000000000000253561303450110500214410ustar00rootroot00000000000000PNG  IHDR<<:rsRGBgAMA a pHYs B4*IDAThCuwxW}sdqwƞlrƀ1&9`2H" %r4EB(,r: (>2'EwݫvڵK?}` +F3Zz.]֥N]#hyN}t[CwG0@ot]~iwm#WSK}_7zvMG_Mƻl=yw¸fȯODeo_'-Fv&=<~l\=(?'fѯHOdjCƽu肾 [OjOun巽z_APL_O/]黻UZ5i6 s/=a^@f`@&1Ɉo7I&lRBouKչ> +_ 3ZA3uP}6|WWz+3qRgLWgi2Ėh[|bi(DK\YRrэ3^Ԝ3?; N{PvчKTKqR4m%%G8HU3 2hp=I?Co)S@ǩt|П|ތ+tP;fb Q-bULHEm:MLi4t|"+LB مzJ,ɼpw>uGܩc>6h8e=_~J:+_?ꡣSY &~gD)a 1uI4ݦ# iK=N6})= IH {c:`bsdD O8'mKG"Nd%LGoR. {Q'TGšAXݦ}*ߡJy6fV%'E[OCxQ!Y^yR.-v`o!I?#!04*~.³DXp"|p;ÃdY CI?vӾ8ݤamjoSt7)̎ש{Ǻ5J!5*ZШkWBծiDGcoU,uhK:IwQ>+aQÃt'Ds93n iTvs$!?DTbr{ܽ! 'Ud+P,mp'S[nbvCDQaObYC eu闳'$#2 8@hvwqz*+Ǐ'ȷ?MڳTڞlu'1m8aE *ם%g~ڿ!(rښ 7)'_Aji;v GB@BO?R6H[g#" JE>DchH!Jrٟ#u]7F®E1"6&ZS⏺P,lLFp5GUtKGk˜3ku^M D4&Zg YLʊȹjṅ6քYmJS**JW" :6AחXq1G٧Sk  z,:_$Mi@A!z=z$(͠Уܘ^yFec,ijS`b7Y$Wh(a xG郧CxT aǓA2DgIU<%jSW<0WZZUZDz0C#i()2 J) )ȿb_0yS䑃9ΕG[",ŽO$HrnUҮ,V&xY2x6]Ģ N,ܸۙi { nʗ7fvNlRO(ݓ9Ff'*]Q`GD&9iIfkx '2ih8a֞K89k]oru.m-Gn1?BAޢhZY!ݍ՛Ŕ $3Ƒbafb|qbmɌ-Nrg1c^;7ȓz%hNs( GcA6m8EU}L…ef~ќZ vKxUɹw8΅+[97IH97B}F:%8VRL}KʉjQ Gzv?cg^&8f=ڸvN^d}|tDu# T:U`macdXXc&u1H.Ƭp3"{Kk/ήřw8k{\sc+%ۛ Μq%1mn꠪suCڛ(+\Y fd VlgF⓵GNX1bnuYvL/wGzGyail*&+!bŒ V<,cU\#+YUɢh?ěX %]oiή8˹9ʙSqrHtra+'ႍ guE~=OV=req9%[(/3(/VNx弳hT?T;Ld?Zc۝jƻk7ϵF2q7QZ1mfC`,"$EO(gA|9<,U[1KcJ!·W1|`biT(]ȑg8Y,eΜ\yS=̕ kEUBq9G˚P,gAvyŔU0qk3(!}7˄yn<^^kxsF́Vmajf =|Ɩk0ɑnYjNJt.8sUH|G4RY/b'"*#)ittıe9"g%W8TiM/y;fa۔lQDV-on"7#L hl"?2&\S3͸Yf{iB^{~]ۋxg4t /Ym"& gId&K"FW0Jf^ʲR*;i}yL:&pā'9I=~Q <ζ'8(>9̪ᶬt%GWS|r )1VȰ+ (ZVv噱nLk,a65s~7a& ?coyi^5ÿYw~-+87Y&U2%>),p+Tfz'2/nq8eadYa>vL{4q7ªX9Ɖf>|vv>n:wJPSFnndV*˨6u 3dҪ4_۩픙Axj,ۑsxv^V1t|1 1>Ƿv;7ss=iLwN䳫L˘QL,`6~)cf[4r ?݊eX6֑wc7F`۸؏1DE& Z٤qXs:)oUg=7ƞMMӤ<=qώ3L嵱3yK5p".yݙG.)||7O$1#&_;9 ?$ &j3alf>O?)Y/[(Ö)Կa;Þ'ɏ˗6Q^TJQ^>yi#,/,l1)4Umb[! n mxv )Ϛ+̱?=}OO[o'穩 yox盍\ʨHJ5 \&n|Ͱ ͧ.{|gO:]0nɚ8crb'';m&:~NF9#؍ځ-Ms` ;:RB3R3ErZՕ"x\DYU1sw.Oʤe8EZk1{%2_+_HS۷Kz8̋!tId4z?SeYNɔ;i|T &(Y炙u/Dt|")4ށ}.vNۏԃvcdu!i fgX"sjΕ :*kÜWNcM%U&J,%DEҵ1 ޞק|igxu,^WJ}y=b2eiݒnVݍ;sK#~u ԇ݌cքf^fngG0r/j# yp7qؤlc5i/vc`3E3Y9ێ˶}ۗzϦ 됛1,6K"\i 1WVr݃?bZ毵e|a _Y|i=\+9b.kfvyDZ'^1=~=G=`F✒ALe )ȭr7V,^ᣭ5<*pÛNpb9\=HIY i?чk̊ %,UX(.I#E&!EeQHŪ- ~QUNcFY.5Oe?yIKJ)TTFUbrI$Wm*,UՀsȊ/!7,9TTOɣJRkVTR}M픔 Fdd[C2bꫨ.<ƪd 3ߜK>:CcrTOh,nhJ(2B/U!7\wj7? q]o\^ h09p<7L"tF 8Ն^3OG}Y~y3E)y/k毼7:t(O? Ͻ _7>A~p+ZNKeH]'wA2Kd-&s*>x *p݆Mnye|&I(_w壸|K~"~?Bo =_,7۽0YȳO/{ }ez }+2?g_^&=Ow9_ę$ .WΣM iJΣSd}>4yO 7GQ;N<<) ئ}:rcuoه.@.%g?=Wkdb77ƛ1͗xyS>4E{ Ͽׇ_^zJ/?ڝ[xq+aUVaf9vs4 bOyJs #7񱵭*?yYBq֚J4Q-_6BG|\/|G/@޵e=[^e^{(}%|yUOظf%3Na̘1c-ru!<&\~/ːFT)T3/R0R1TS5ωTyN~7:h fJH?PCa/ʚ)NhWLmlx Z9V|2{7$VK0HW7a4KINP읜DFz*qqqʚ^7kҶVR!M*UQW^sW) I;Ue_K_R4` $dUÕ3J"L2Rwr\n#ڌHBW#l~,b'srh""VMYک[7CyxƆ**\R%̈́E0'+,RR  5o殛'ށ*èa-.Yim>K2-7d)sU3م"7SЗJO&#a80|9|Vpw4Wo>7P\ ;S$I\L,"A w]\rOB#xi*rs *z|\FC'"JggԊp,ausն63Z'Ijz]~]$_فˆ/𰛎׌aa—O'|AĚD~HQ׆9xZI) K?^絷1q󮋔d樌y D>.!4+1(ϪRID]0j/U©4L0&oӉϡ"p%E^[ޣr"9ޛfHS[?ȵJkbVA:f8a .guVIiۼa%*.C~a#K[ OK(T f]yk=Nrb\j1(Va bϹ@^*"Q: Ԅ^Ω^ h~L:B>W_,szq}c? N&m4:pDyqAȆI8@M{7?m=g5[b]s&''fOTl^XXh14ZK4טhNRKM ZNqB>2lJ)bCm8+C Dv:^BEZrZu8rsӶl uN'mK؆4"7N%pD>g9#عi5zM>PX\@VVٙWQNNKY2H*?M?O]~npkE+jNQaJ T#"i^Kߩn$8BCj6HO@Jp*|DgCoٜ8uu0f<Ḹ"^EHH9 khd2Y񄬡G"lv}fFJ6ȧŬO,Ȕb !6.HqV$hHJSA979O Īp7(ijlwC4W 4֫}Ǥ}ramHP8WӦPҥ5g}c$m*")"ܨR) ^4ZR]FRfq 20C" J8<߷"5M*7C$ˬD+++Dj ??iFf A >в~zcu6_K{UZMuB5U-Jz)^I>оq\Zf4HMƱj@TeW9,mf JVTkjTK)W5ݜluSZ0Ͳ&Zd16+Q 9HvgIENDB`weboob-1.2/modules/agendaculturel/module.py000066400000000000000000000043721303450110500211350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES from .browser import AgendaculturelBrowser from .calendar import AgendaculturelEvent __all__ = ['AgendaculturelModule'] class AgendaculturelModule(Module, CapCalendarEvent): NAME = 'agendaculturel' DESCRIPTION = u'agendaculturel website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT, CATEGORIES.THEATRE, CATEGORIES.EXPO, CATEGORIES.AUTRE, CATEGORIES.FEST] BROWSER = AgendaculturelBrowser CONFIG = BackendConfig(Value('place', label='Default place')) def create_default_browser(self): return self.create_browser(self.config['place'].get()) def get_event(self, _id): return self.browser.get_event(_id) def list_events(self, date_from, date_to=None): default_place = self.config['place'].get() return self.browser.list_events(default_place, date_from, date_to) def search_events(self, query): if self.has_matching_categories(query): return self.browser.list_events(query.city, query.start_date, query.end_date, query.categories) def fill_obj(self, event, fields): return self.browser.get_event(event.id, event) OBJECTS = {AgendaculturelEvent: fill_obj} weboob-1.2/modules/agendaculturel/pages.py000066400000000000000000000132241303450110500207430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Date, Regexp, Filter, Env, Format, Decode, Time, Type from weboob.browser.filters.html import CleanHTML, XPath from weboob.browser.filters.json import Dict from weboob.capabilities.calendar import CATEGORIES from .calendar import AgendaculturelEvent from datetime import datetime, time class AgendaculturelCategory(Filter): def filter(self, text): if text == u'MusicEvent': return CATEGORIES.CONCERT elif text == u'TheaterEvent': return CATEGORIES.THEATRE elif text == u'VisuelArtsEvent': return CATEGORIES.EXPO elif text == u'Festival': return CATEGORIES.FEST else: return CATEGORIES.AUTRE class AgendaculturelDate(Filter): def filter(self, text): return datetime.strptime(text, "%Y-%m-%d") class BasePage(HTMLPage): def search_events(self, query_date_from): form = self.get_form(nr=0) form['search_month'] = query_date_from form.submit() @method class get_event(ItemElement): klass = AgendaculturelEvent def parse(self, el): _json = CleanText('.')(XPath('//script[@type="application/ld+json"][1]')(el)[0]) try: from weboob.tools.json import json self.env['_json'] = json.loads(_json) except ValueError: self.env['_json'] = {} def validate(self, obj): return self.env['_json'] obj_id = Format('%s.%s', Env('region'), Decode(Env('_id'))) obj_summary = CleanText('//h1') def obj_description(self): desc = CleanHTML('//div[@class="description"]')(self) if not desc: desc = CleanText('//meta[@name="description"]/@content')(self) return desc def obj_start_date(self): if not self.env['_json']: return _time = Time(CleanText('//div[@class="hours"]'), default=None)(self) if not _time: _time = time.min date = AgendaculturelDate(Dict('startDate'))(self.env['_json']) return datetime.combine(date, _time) def obj_end_date(self): if not self.env['_json']: return date = AgendaculturelDate(Dict('endDate'))(self.env['_json']) return datetime.combine(date, time.max) def obj_url(self): if not self.env['_json']: return return Dict('url')(self.env['_json']) def obj_city(self): if not self.env['_json']: return return Dict('location/address/addressLocality')(self.env['_json']) def obj_category(self): if not self.env['_json']: return return AgendaculturelCategory(Dict('@type'))(self.env['_json']) def obj_location(self): if not self.env['_json']: return return Format('%s, %s', Dict('location/name'), Dict('location/address/streetAddress'))(self.env['_json']) def obj_price(self): if not self.env['_json']: return return Type(CleanText(Dict('offers/price', default="0")), type=float, default=0)(self.env['_json']) @method class list_events(ListElement): item_xpath = '//ul[has-class("list-event")]/li' class item(ItemElement): klass = AgendaculturelEvent def validate(self, obj): return self.check_date(obj) and self.check_category(obj) def check_date(self, obj): if self.env['date_from'] and obj.start_date >= self.env['date_from']: if not self.env['date_to']: return True elif obj.end_date and obj.end_date <= self.env['date_to']: return True elif self.env['date_to'] >= obj.start_date: return True return False def check_category(self, obj): return (not self.env['categories'] or obj.category in self.env['categories']) obj_id = Format('%s.%s', Env('region'), Regexp(CleanText('./div/a[@itemprop="url"]/@href'), '/(.*).html')) obj_summary = CleanText('./div/a[@itemprop="url"]') def obj_start_date(self): _date = Date(CleanText('./meta[@itemprop="startDate"]/@content'))(self) return datetime.combine(_date, time.min) obj_category = AgendaculturelCategory(Regexp(CleanText('./@itemtype'), 'http://schema.org/(.*)')) weboob-1.2/modules/agendaculturel/test.py000066400000000000000000000024271303450110500206260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.value import Value from datetime import datetime class AgendaculturelTest(BackendTest): MODULE = 'agendaculturel' def setUp(self): if not self.is_backend_configured(): self.backend.config['place'] = Value(value='paris') def test_agendaculturel(self): l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) weboob-1.2/modules/agendadulibre/000077500000000000000000000000001303450110500170575ustar00rootroot00000000000000weboob-1.2/modules/agendadulibre/__init__.py000066400000000000000000000014501303450110500211700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AgendadulibreModule __all__ = ['AgendadulibreModule'] weboob-1.2/modules/agendadulibre/browser.py000066400000000000000000000037241303450110500211220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import EventListPage, EventPage from datetime import timedelta, date class AgendadulibreBrowser(PagesBrowser): event_list_page = URL('events\?start_date=(?P.*)(?P.*)', EventListPage) event_page = URL('events/(?P<_id>.*)', EventPage) def __init__(self, website, region, *args, **kwargs): self.BASEURL = u'%s/' % website self.region = '®ion=%s' % region if region else '' PagesBrowser.__init__(self, *args, **kwargs) def list_events(self, date_from, date_to, city=None, categories=None, max_date=None): _max_date = date_from + timedelta(days=365) max_date = date(year=_max_date.year, month=_max_date.month, day=_max_date.day) return self.event_list_page.go(date_from=date_from.strftime("%Y-%m-%d"), region=self.region)\ .list_events(date_from=date_from, date_to=date_to, city=city, categories=categories, max_date=max_date) def get_event(self, event_id, event=None): _id = event_id.split('#')[-1] return self.event_page.go(_id=_id).get_event(obj=event) weboob-1.2/modules/agendadulibre/calendar.py000066400000000000000000000020701303450110500212010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES class AgendaDuLibreCalendarEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.category = CATEGORIES.CONF weboob-1.2/modules/agendadulibre/favicon.png000066400000000000000000000034311303450110500212130ustar00rootroot00000000000000PNG  IHDR@@iqgAMA a pHYsodtEXtAuthorJakub Steiner/!tEXtSourcehttp://jimmac.musichall.czif^ItEXtCopyrightPublic Domain http://creativecommons.org/licenses/publicdomain/YtEXtSoftwarepaint.net 4.0.3PIDATx^OIg!TSPKN9TApzA0HuXdo+~X_WV6=̑igO.zz8Ç>|#J HCCCyY"%Hyy:uJ̙3 !2dggKII(ƍjSSSKrE(ZDwQ]]-Sy Kggtuu%W^+WH~~ܾ}}%=zTPS .4illt"$vGAAr@nn -{tJ80++DIr1G^kî04iҞ$ZDwA0Pw F_C]Q 89tw0 999+X?|<"c11;;{ I'C. 9߽{=%I%Z;7G>cftO̹gϔ c"v~6lGVVT]ō??/ hݩ90ۙNt7 hݩ90wQ+j  hݩ9ɓ'27"Єݩ9ɝu)?J@`7}( W@$ .?3鄃 D֢hu˃?~,yyyS@kkpH0T\*@/21vMMM|Jɀ|?֯>R/`[#v$SXؕtBFMVlBM JɌmuV?`ccC%Ev"uA D'p!>Êh`fҳا"#Dg7sqJcm|h ; ΀߿/*پ "=p UMנȲrݤQ{UY^^9u'`phHەlSx*f!?Q1%?i2*@@ pP7>~,pNd5 c2u+N%3RbRׯs)pg %%%6Na?ʇݻw"o߾7oׯeiiIeaaA]/^% * rɓ'*>gffdzzZ6GÇebbB˽{N[wܑ[n͛7 nUZrP @ ,B9̹y ky9|"Tp )XU"M!655)r(녨zeZEBRU!f0{xVd@?b/bb.VRTEIzhmMܓqd!4JxFMyj' !F5GZ{ %א;QȈ,DLcƒhWY/i*;3NT|Jb$WbOVӸzxޜ͹DNEn,O8Ç>|#G~ >scZIENDB`weboob-1.2/modules/agendadulibre/module.py000066400000000000000000000150371303450110500207240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from .browser import AgendadulibreBrowser __all__ = ['AgendadulibreModule'] class AgendadulibreModule(Module, CapCalendarEvent): NAME = 'agendadulibre' DESCRIPTION = u'agendadulibre website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.CONF] BROWSER = AgendadulibreBrowser region_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ "http://www.agendadulibre.org": u'--France--', "http://www.agendadulibre.org#1": u'Alsace', "http://www.agendadulibre.org#2": u'Aquitaine', "http://www.agendadulibre.org#3": u'Auvergne', "http://www.agendadulibre.org#4": u'Basse-Normandie', "http://www.agendadulibre.org#5": u'Bourgogne', "http://www.agendadulibre.org#6": u'Bretagne', "http://www.agendadulibre.org#7": u'Centre', "http://www.agendadulibre.org#8": u'Champagne-Ardenne', "http://www.agendadulibre.org#9": u'Corse', "http://www.agendadulibre.org#10": u'Franche-Comté', "http://www.agendadulibre.org#23": u'Guadeloupe', "http://www.agendadulibre.org#24": u'Guyane', "http://www.agendadulibre.org#11": u'Haute-Normandie', "http://www.agendadulibre.org#12": u'Île-de-France', "http://www.agendadulibre.org#13": u'Languedoc-Roussillon', "http://www.agendadulibre.org#14": u'Limousin', "http://www.agendadulibre.org#15": u'Lorraine', "http://www.agendadulibre.org#25": u'Martinique', "http://www.agendadulibre.org#16": u'Midi-Pyrénées', "http://www.agendadulibre.org#17": u'Nord-Pas-de-Calais', "http://www.agendadulibre.org#18": u'Pays de la Loire', "http://www.agendadulibre.org#19": u'Picardie', "http://www.agendadulibre.org#20": u'Poitou-Charentes', "http://www.agendadulibre.org#21": u'Provence-Alpes-Côte d\'Azur', "http://www.agendadulibre.org#26": u'Réunion', "http://www.agendadulibre.org#22": u'Rhône-Alpes', "http://www.agendadulibre.be": u'--Belgique--', "http://www.agendadulibre.be#11": u'Antwerpen', "http://www.agendadulibre.be#10": u'Brabant wallon', "http://www.agendadulibre.be#9": u'Bruxelles-Capitale', "http://www.agendadulibre.be#8": u'Hainaut', "http://www.agendadulibre.be#7": u'Liege', "http://www.agendadulibre.be#6": u'Limburg', "http://www.agendadulibre.be#5": u'Luxembourg', "http://www.agendadulibre.be#4": u'Namur', "http://www.agendadulibre.be#3": u'Oost-Vlaanderen', "http://www.agendadulibre.be#2": u'Vlaams-Brabant', "http://www.agendadulibre.be#1": u'West-Vlaanderen', "http://www.agendadulibre.ch": u'--Suisse--', "http://www.agendadulibre.ch#15": u'Appenzell Rhodes-Extérieures', "http://www.agendadulibre.ch#16": u'Appenzell Rhodes-Intérieures', "http://www.agendadulibre.ch#19": u'Argovie', "http://www.agendadulibre.ch#13": u'Bâle-Campagne', "http://www.agendadulibre.ch#12": u'Bâle-Ville', "http://www.agendadulibre.ch#2": u'Berne', "http://www.agendadulibre.ch#10": u'Fribourg', "http://www.agendadulibre.ch#25": u'Genève', "http://www.agendadulibre.ch#8": u'Glaris', "http://www.agendadulibre.ch#18": u'Grisons', "http://www.agendadulibre.ch#26": u'Jura', "http://www.agendadulibre.ch#3": u'Lucerne', "http://www.agendadulibre.ch#24": u'Neuchâtel', "http://www.agendadulibre.ch#7": u'Nidwald', "http://www.agendadulibre.ch#6": u'Obwald', "http://www.agendadulibre.ch#17": u'Saint-Gall', "http://www.agendadulibre.ch#14": u'Schaffhouse', "http://www.agendadulibre.ch#5": u'Schwytz', "http://www.agendadulibre.ch#11": u'Soleure', "http://www.agendadulibre.ch#21": u'Tessin', "http://www.agendadulibre.ch#20": u'Thurgovie', "http://www.agendadulibre.ch#4": u'Uri', "http://www.agendadulibre.ch#23": u'Valais', "http://www.agendadulibre.ch#22": u'Vaud', "http://www.agendadulibre.ch#9": u'Zoug', "http://www.agendadulibre.ch#1": u'Zurich', }.iteritems())]) CONFIG = BackendConfig(Value('region', label=u'Region', choices=region_choices)) def create_default_browser(self): choice = self.config['region'].get().split('#') selected_region = '' if len(choice) < 2 else choice[-1] return self.create_browser(website=choice[0], region=selected_region) def search_events(self, query): return self.browser.list_events(query.start_date, query.end_date, query.city, query.categories) def list_events(self, date_from, date_to=None): return self.browser.list_events(date_from, date_to) def get_event(self, event_id): return self.browser.get_event(event_id) def fill_obj(self, event, fields): event = self.browser.get_event(event.id, event) choice = self.config['region'].get().split('#') selected_region = '' if len(choice) < 2 else choice[-1] if selected_region == '23': event.timezone = 'America/Guadeloupe' elif selected_region == '24': event.timezone = 'America/Guyana' elif selected_region == '26': event.timezone = 'Indian/Reunion' elif selected_region == '25': event.timezone = 'America/Martinique' else: event.timezone = 'Europe/Paris' return event OBJECTS = {AgendadulibreBrowser: fill_obj} weboob-1.2/modules/agendadulibre/pages.py000066400000000000000000000147731303450110500205440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Regexp, CleanText, DateTime, Env, Format, BrowserURL from weboob.browser.filters.html import Link, XPath, CleanHTML from .calendar import AgendaDuLibreCalendarEvent from datetime import time, datetime, date import re class EventPage(HTMLPage): @method class get_event(ItemElement): klass = AgendaDuLibreCalendarEvent obj_id = Env('_id') obj_url = BrowserURL('event_page', _id=Env('_id')) obj_summary = Format('%s %s', CleanText('//meta[@name="geo:placename"]/@content'), CleanText('//meta[@name="DC:title"]/@content')) obj_description = CleanHTML('//div[@class="description"]') obj_location = CleanText('//p[@class="full_address"]/span[1]') obj_city = CleanText('//meta[@name="geo:placename"]/@content') def obj_start_date(self): m = re.findall(u'(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', (CleanText('(//p)[1]')(self))) if m: return DateTime(Regexp(CleanText('(//p)[1]'), '\w* \w* (\d?\d \w* \d{4}) \w* (\d{2}h\d{2})', '\\1 \\2'))(self) def obj_end_date(self): m = re.findall(u'(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', (CleanText('(//p)[1]')(self))) if m: if len(m) == 1: return DateTime(Regexp(CleanText('(//p)[1]'), '\w* \w* (\d?\d \w* \d{4}) \w* \d{2}h\d{2} \w* (\d{2}h\d{2})', '\\1 \\2'))(self) else: return DateTime(Regexp(Regexp(CleanText('(//p)[1]'), '(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', nth=-1), '\w* \w* (\d?\d \w* \d{4}) \w* (\d{2}h\d{2})', '\\1 \\2'))(self) class EventListPage(HTMLPage): @pagination @method class list_events(ListElement): item_xpath = '//td[starts-with(@class, "day")]/ul/li' def next_page(self): m = re.match('.*/events\?start_date=(\d{4})-(\d{2})-(\d{2})(®ion=.*)?', self.page.url) if m: start = date(year=int(m.group(1)), month=int(m.group(2)), day=1) region = m.group(4) if m.group(4) else '' try: next_month = start.replace(month=start.month + 1) except ValueError: if start.month == 12: next_month = start.replace(year=start.year + 1, month=1) else: raise if (self.env['date_to'] is None and start < self.env['max_date']) or\ (self.env['date_to'] is not None and datetime.combine(next_month, time.min) < self.env['date_to']): return '/events?start_date=%s%s' % (next_month.strftime("%Y-%m-%d"), region) class item(ItemElement): klass = AgendaDuLibreCalendarEvent def condition(self): return len(XPath('.')(self.el)) > 0 and \ ('current-month' in XPath('./ancestor::td/@class')(self.el)[0]) obj_id = Format('%s#%s', CleanText('./ancestor::td/div[@class="day_number"]'), Regexp(Link('./a'), '/events/(.*)')) obj_city = CleanText('./a/strong[@class="city"]') obj_summary = CleanText('./a') def obj_start_date(self): m = re.findall(u'(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', (CleanText('./@title')(self))) if m: return DateTime(Regexp(CleanText('./@title'), '\w* \w* (\d?\d \w* \d{4}) \w* (\d{2}h\d{2})', '\\1 \\2'))(self) def obj_end_date(self): m = re.findall(u'(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', (CleanText('./@title')(self))) if m: if len(m) == 1: return DateTime(Regexp(CleanText('./@title'), '\w* \w* (\d?\d \w* \d{4}) \w* \d{2}h\d{2} \w* (\d{2}h\d{2})', '\\1 \\2'))(self) else: return DateTime(Regexp(Regexp(CleanText('./@title'), '(\w* \w* \d?\d \w* \d{4} \w* \d{2}h\d{2})', nth=-1), '\w* \w* (\d?\d \w* \d{4}) \w* (\d{2}h\d{2})', '\\1 \\2'))(self) def validate(self, obj): return (self.is_valid_event(obj, self.env['city'], self.env['categories']) and self.is_event_in_valid_period(obj.start_date, self.env['date_from'], self.env['date_to'])) def is_valid_event(self, event, city, categories): if city and city != '' and city.upper() != event.city.upper(): return False if categories and len(categories) > 0 and event.category not in categories: return False return True def is_event_in_valid_period(self, event_date, date_from, date_to): if event_date >= datetime.combine(date_from, time.min): if not date_to: return True else: if event_date <= date_to: return True return False weboob-1.2/modules/agendadulibre/test.py000066400000000000000000000024541303450110500204150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.value import Value from datetime import datetime class AgendadulibreTest(BackendTest): MODULE = 'agendadulibre' def setUp(self): if not self.is_backend_configured(): self.backend.config['region'] = Value(value='http://www.agendadulibre.org') def test_agendadulibre(self): l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) weboob-1.2/modules/allocine/000077500000000000000000000000001303450110500160575ustar00rootroot00000000000000weboob-1.2/modules/allocine/__init__.py000066400000000000000000000014351303450110500201730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AllocineModule __all__ = ['AllocineModule'] weboob-1.2/modules/allocine/browser.py000066400000000000000000000606441303450110500201260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES from weboob.capabilities.collection import Collection from weboob.capabilities.video import BaseVideo from weboob.capabilities.image import Thumbnail from weboob.capabilities.base import NotAvailable, NotLoaded, find_object from weboob.capabilities.cinema import Movie, Person from weboob.deprecated.browser import Browser from weboob.tools.json import json import base64 import hashlib from datetime import datetime, date, timedelta import time import urllib __all__ = ['AllocineBrowser'] class AllocineBrowser(Browser): DOMAIN = 'api.allocine.fr' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = 'Dalvik/1.6.0 (Linux; U; Android 4.2.2; Nexus 4 Build/JDQ39E)' PARTNER_KEY = '100043982026' SECRET_KEY = '29d185d98c984a359e6e6f26a0474269' def __do_request(self, method, params): params_encode = urllib.urlencode(params) sed = time.strftime('%Y%m%d', time.localtime()) sig = base64.b64encode(hashlib.sha1(self.SECRET_KEY + params_encode + '&sed=' + sed).digest()) query_url = 'http://api.allocine.fr/rest/v3/' + method + '?' + params_encode + '&sed=' + sed + '&sig=' + sig return self.readurl(query_url) def iter_movies(self, pattern): params = [('partner', self.PARTNER_KEY), ('q', pattern), ('format', 'json'), ('filter', 'movie')] res = self.__do_request('search', params) if res is None: return jres = json.loads(res) if 'movie' not in jres['feed']: return for m in jres['feed']['movie']: tdesc = u'' if 'title' in m: tdesc += '%s' % m['title'] if 'productionYear' in m: tdesc += ' ; %s' % m['productionYear'] elif 'release' in m: tdesc += ' ; %s' % m['release']['releaseDate'] if 'castingShort' in m and 'actors' in m['castingShort']: tdesc += ' ; %s' % m['castingShort']['actors'] short_description = tdesc.strip('; ') thumbnail_url = NotAvailable if 'poster' in m: thumbnail_url = unicode(m['poster']['href']) movie = Movie(m['code'], unicode(m['originalTitle'])) movie.other_titles = NotLoaded movie.release_date = NotLoaded movie.duration = NotLoaded movie.short_description = short_description movie.pitch = NotLoaded movie.country = NotLoaded movie.note = NotLoaded movie.roles = NotLoaded movie.all_release_dates = NotLoaded movie.thumbnail_url = thumbnail_url yield movie def iter_persons(self, pattern): params = [('partner', self.PARTNER_KEY), ('q', pattern), ('format', 'json'), ('filter', 'person')] res = self.__do_request('search', params) if res is None: return jres = json.loads(res) if 'person' not in jres['feed']: return for p in jres['feed']['person']: thumbnail_url = NotAvailable if 'picture' in p: thumbnail_url = unicode(p['picture']['href']) person = Person(p['code'], unicode(p['name'])) desc = u'' if 'birthDate' in p: desc += '(%s), ' % p['birthDate'] if 'activity' in p: for a in p['activity']: desc += '%s, ' % a['$'] person.real_name = NotLoaded person.birth_place = NotLoaded person.birth_date = NotLoaded person.death_date = NotLoaded person.gender = NotLoaded person.nationality = NotLoaded person.short_biography = NotLoaded person.short_description = desc.strip(', ') person.roles = NotLoaded person.thumbnail_url = thumbnail_url yield person def get_movie(self, id): params = [('partner', self.PARTNER_KEY), ('code', id), ('profile', 'large'), ('mediafmt', 'mp4-lc'), ('filter', 'movie'), ('striptags', 'synopsis,synopsisshort'), ('format', 'json')] res = self.__do_request('movie', params) if res is not None: jres = json.loads(res) if 'movie' in jres: jres = jres['movie'] else: return None else: return None title = NotAvailable duration = NotAvailable release_date = NotAvailable pitch = NotAvailable country = NotAvailable note = NotAvailable short_description = NotAvailable thumbnail_url = NotAvailable other_titles = [] genres = [] roles = {} if 'originalTitle' not in jres: return title = unicode(jres['originalTitle'].strip()) if 'poster' in jres: thumbnail_url = unicode(jres['poster']['href']) if 'genre' in jres: for g in jres['genre']: genres.append(g['$']) if 'runtime' in jres: nbsecs = jres['runtime'] duration = nbsecs / 60 if 'release' in jres: dstr = str(jres['release']['releaseDate']) tdate = dstr.split('-') day = 1 month = 1 year = 1901 if len(tdate) > 2: year = int(tdate[0]) month = int(tdate[1]) day = int(tdate[2]) release_date = datetime(year, month, day) if 'nationality' in jres: country = u'' for c in jres['nationality']: country += '%s, ' % c['$'] country = country.strip(', ') if 'synopsis' in jres: pitch = unicode(jres['synopsis']) if 'statistics' in jres and 'userRating' in jres['statistics']: note = u'%s/5 (%s votes)' % (jres['statistics']['userRating'], jres['statistics']['userReviewCount']) if 'castMember' in jres: for cast in jres['castMember']: if cast['activity']['$'] not in roles: roles[cast['activity']['$']] = [] person_to_append = (u'%s' % cast['person']['code'], cast['person']['name']) roles[cast['activity']['$']].append(person_to_append) movie = Movie(id, title) movie.other_titles = other_titles movie.release_date = release_date movie.duration = duration movie.genres = genres movie.pitch = pitch movie.country = country movie.note = note movie.roles = roles movie.short_description = short_description movie.all_release_dates = NotLoaded movie.thumbnail_url = thumbnail_url return movie def get_person(self, id): params = [('partner', self.PARTNER_KEY), ('code', id), ('profile', 'large'), ('mediafmt', 'mp4-lc'), ('filter', 'movie'), ('striptags', 'biography,biographyshort'), ('format', 'json')] res = self.__do_request('person', params) if res is not None: jres = json.loads(res) if 'person' in jres: jres = jres['person'] else: return None else: return None name = NotAvailable short_biography = NotAvailable biography = NotAvailable short_description = NotAvailable birth_place = NotAvailable birth_date = NotAvailable death_date = NotAvailable real_name = NotAvailable gender = NotAvailable thumbnail_url = NotAvailable roles = {} nationality = NotAvailable if 'name' in jres: name = u'' if 'given' in jres['name']: name += jres['name']['given'] if 'family' in jres['name']: name += ' %s' % jres['name']['family'] if 'biographyShort' in jres: short_biography = unicode(jres['biographyShort']) if 'birthPlace' in jres: birth_place = unicode(jres['birthPlace']) if 'birthDate' in jres: df = jres['birthDate'].split('-') birth_date = datetime(int(df[0]), int(df[1]), int(df[2])) if 'deathDate' in jres: df = jres['deathDate'].split('-') death_date = datetime(int(df[0]), int(df[1]), int(df[2])) if 'realName' in jres: real_name = unicode(jres['realName']) if 'gender' in jres: gcode = jres['gender'] if gcode == '1': gender = u'Male' else: gender = u'Female' if 'picture' in jres: thumbnail_url = unicode(jres['picture']['href']) if 'nationality' in jres: nationality = u'' for n in jres['nationality']: nationality += '%s, ' % n['$'] nationality = nationality.strip(', ') if 'biography' in jres: biography = unicode(jres['biography']) if 'participation' in jres: for m in jres['participation']: if m['activity']['$'] not in roles: roles[m['activity']['$']] = [] pyear = '????' if 'productionYear' in m['movie']: pyear = m['movie']['productionYear'] movie_to_append = (u'%s' % (m['movie']['code']), u'(%s) %s' % (pyear, m['movie']['originalTitle'])) roles[m['activity']['$']].append(movie_to_append) person = Person(id, name) person.real_name = real_name person.birth_date = birth_date person.death_date = death_date person.birth_place = birth_place person.gender = gender person.nationality = nationality person.short_biography = short_biography person.biography = biography person.short_description = short_description person.roles = roles person.thumbnail_url = thumbnail_url return person def iter_movie_persons(self, movie_id, role_filter): params = [('partner', self.PARTNER_KEY), ('code', movie_id), ('profile', 'large'), ('mediafmt', 'mp4-lc'), ('filter', 'movie'), ('striptags', 'synopsis,synopsisshort'), ('format', 'json')] res = self.__do_request('movie', params) if res is not None: jres = json.loads(res) if 'movie' in jres: jres = jres['movie'] else: return else: return if 'castMember' in jres: for cast in jres['castMember']: if (role_filter is None or (role_filter is not None and cast['activity']['$'].lower().strip() == role_filter.lower().strip())): id = cast['person']['code'] name = unicode(cast['person']['name']) short_description = unicode(cast['activity']['$']) if 'role' in cast: short_description += ', %s' % cast['role'] thumbnail_url = NotAvailable if 'picture' in cast: thumbnail_url = unicode(cast['picture']['href']) person = Person(id, name) person.short_description = short_description person.real_name = NotLoaded person.birth_place = NotLoaded person.birth_date = NotLoaded person.death_date = NotLoaded person.gender = NotLoaded person.nationality = NotLoaded person.short_biography = NotLoaded person.roles = NotLoaded person.thumbnail_url = thumbnail_url yield person def iter_person_movies(self, person_id, role_filter): params = [('partner', self.PARTNER_KEY), ('code', person_id), ('profile', 'medium'), ('filter', 'movie'), ('format', 'json')] res = self.__do_request('filmography', params) if res is not None: jres = json.loads(res) if 'person' in jres: jres = jres['person'] else: return else: return for m in jres['participation']: if (role_filter is None or (role_filter is not None and m['activity']['$'].lower().strip() == role_filter.lower().strip())): prod_year = '????' if 'productionYear' in m['movie']: prod_year = m['movie']['productionYear'] short_description = u'(%s) %s' % (prod_year, m['activity']['$']) if 'role' in m: short_description += ', %s' % m['role'] movie = Movie(m['movie']['code'], unicode(m['movie']['originalTitle'])) movie.other_titles = NotLoaded movie.release_date = NotLoaded movie.duration = NotLoaded movie.short_description = short_description movie.pitch = NotLoaded movie.country = NotLoaded movie.note = NotLoaded movie.roles = NotLoaded movie.all_release_dates = NotLoaded movie.thumbnail_url = NotLoaded yield movie def iter_person_movies_ids(self, person_id): params = [('partner', self.PARTNER_KEY), ('code', person_id), ('profile', 'medium'), ('filter', 'movie'), ('format', 'json')] res = self.__do_request('filmography', params) if res is not None: jres = json.loads(res) if 'person' in jres: jres = jres['person'] else: return else: return for m in jres['participation']: yield unicode(m['movie']['code']) def iter_movie_persons_ids(self, movie_id): params = [('partner', self.PARTNER_KEY), ('code', movie_id), ('profile', 'large'), ('mediafmt', 'mp4-lc'), ('filter', 'movie'), ('striptags', 'synopsis,synopsisshort'), ('format', 'json')] res = self.__do_request('movie', params) if res is not None: jres = json.loads(res) if 'movie' in jres: jres = jres['movie'] else: return else: return if 'castMember' in jres: for cast in jres['castMember']: yield unicode(cast['person']['code']) def get_movie_releases(self, id, country): return def get_person_biography(self, id): params = [('partner', self.PARTNER_KEY), ('code', id), ('profile', 'large'), ('mediafmt', 'mp4-lc'), ('filter', 'movie'), ('striptags', 'biography,biographyshort'), ('format', 'json')] res = self.__do_request('person', params) if res is not None: jres = json.loads(res) if 'person' in jres: jres = jres['person'] else: return None else: return None biography = NotAvailable if 'biography' in jres: biography = unicode(jres['biography']) return biography def get_categories_movies(self, category): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('mediafmt', 'mp4'), ('filter', category) ] res = self.__do_request('movielist', params) if res is None: return result = json.loads(res) for movie in result['feed']['movie']: if 'trailer' not in movie or 'productionYear' not in movie: continue yield self.parse_movie(movie) def get_categories_videos(self, category): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('mediafmt', 'mp4'), ('filter', category) ] res = self.__do_request('videolist', params) if res is None: return result = json.loads(res) if 'feed' in result and 'media' in result['feed']: for episode in result['feed']['media']: if 'title' in episode: yield self.parse_video(episode, category) def parse_video(self, _video, category): video = BaseVideo(u'%s#%s' % (_video['code'], category)) video.title = unicode(_video['title']) video._video_code = unicode(_video['code']) video.ext = u'mp4' if 'runtime' in _video: video.duration = timedelta(seconds=int(_video['runtime'])) if 'description' in _video: video.description = unicode(_video['description']) renditions = sorted(_video['rendition'], key=lambda x: 'bandwidth' in x and x['bandwidth']['code'], reverse=True) video.url = unicode(max(renditions, key=lambda x: 'bandwidth' in x)['href']) return video def parse_movie(self, movie): video = BaseVideo(u'%s#%s' % (movie['code'], 'movie')) video.title = unicode(movie['trailer']['name']) video._video_code = unicode(movie['trailer']['code']) video.ext = u'mp4' if 'poster' in movie: video.thumbnail = Thumbnail(movie['poster']['href']) video.thumbnail.url = unicode(movie['poster']['href']) tdate = movie['release']['releaseDate'].split('-') day = 1 month = 1 year = 1901 if len(tdate) > 2: year = int(tdate[0]) month = int(tdate[1]) day = int(tdate[2]) video.date = date(year, month, day) if 'userRating' in movie['statistics']: video.rating = movie['statistics']['userRating'] elif 'pressRating' in movie['statistics']: video.rating = movie['statistics']['pressRating'] * 2 video.rating_max = 5 if 'synopsis' in movie: video.description = unicode(movie['synopsis'].replace('

', '').replace('

', '')) elif 'synopsisShort' in movie: video.description = unicode(movie['synopsisShort'].replace('

', '').replace('

', '')) if 'castingShort' in movie: if 'directors' in movie['castingShort']: video.author = unicode(movie['castingShort']['directors']) if 'runtime' in movie: video.duration = timedelta(seconds=int(movie['runtime'])) return video def get_movie_from_id(self, _id): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('mediafmt', 'mp4'), ('filter', 'movie'), ('code', _id), ] res = self.__do_request('movie', params) if res is None: return result = json.loads(res) return self.parse_video(result['movie']) def get_video_from_id(self, _id, category): return find_object(self.get_categories_videos(category), id=u'%s#%s' % (_id, category)) def get_video_url(self, code): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('mediafmt', 'mp4'), ('code', code), ('profile', 'large'), ] res = self.__do_request('media', params) if res is None: return result = json.loads(res) renditions = sorted(result['media']['rendition'], key=lambda x: 'bandwidth' in x and x['bandwidth']['code'], reverse=True) return max(renditions, key=lambda x: 'bandwidth' in x)['href'] def get_emissions(self, basename): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('filter', 'acshow'), ] res = self.__do_request('termlist', params) if res is None: return result = json.loads(res) for emission in result['feed']['term']: yield Collection([basename, unicode(emission['nameShort'])], unicode(emission['$'])) def search_events(self, query): params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('zip', query.city) ] if query.summary: movie = self.iter_movies(query.summary).next() params.append(('movie', movie.id)) res = self.__do_request('showtimelist', params) if res is None: return result = json.loads(res) for event in self.create_event(result): if (not query.end_date or event.start_date <= query.end_date)\ and event.start_date >= query.start_date: yield event def get_event(self, _id): split_id = _id.split('#') params = [('partner', self.PARTNER_KEY), ('format', 'json'), ('theaters', split_id[0]), ('zip', split_id[1]), ('movie', split_id[2]), ] res = self.__do_request('showtimelist', params) if res is None: return result = json.loads(res) for event in self.create_event(result): if event.id.split('#')[-1] == split_id[-1]: event.description = self.get_movie(split_id[2]).pitch return event def create_event(self, data): sequence = 1 transp = TRANSP.TRANSPARENT status = STATUS.CONFIRMED category = CATEGORIES.CINE if 'theaterShowtimes' not in data['feed']: return for items in data['feed']['theaterShowtimes']: cinema = items['place']['theater'] city = unicode(cinema['city']) location = u'%s, %s' % (cinema['name'], cinema['address']) postalCode = cinema['postalCode'] cinemaCode = cinema['code'] for show in items['movieShowtimes']: summary = unicode(show['onShow']['movie']['title']) movieCode = show['onShow']['movie']['code'] for jour in show['scr']: tdate = jour['d'].split('-') year = int(tdate[0]) month = int(tdate[1]) day = int(tdate[2]) for seance in jour['t']: ttime = seance['$'].split(':') heure = int(ttime[0]) minute = int(ttime[1]) start_date = datetime(year=year, month=month, day=day, hour=heure, minute=minute) seanceCode = seance['code'] _id = u'%s#%s#%s#%s' % (cinemaCode, postalCode, movieCode, seanceCode) event = BaseCalendarEvent() event.id = _id event.sequence = sequence event.transp = transp event.status = status event.category = category event.city = city event.location = location event.start_date = start_date event.summary = summary event.timezone = u'Europe/Paris' yield event weboob-1.2/modules/allocine/favicon.png000066400000000000000000000047711303450110500202230ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME ۉtEXtCommentCreated with GIMPW aIDATxypT?{dsِ Bv;b8BaԖ:TQ[QAδcjGx`)$` "&$$&$lvew&cy3{oeU,rrH$ *奛 ['Jtx!S>8,$02Py#N5MD_?dg<p7S!B)Vg<2_G}`e'w`6C?O;7`0;`En"RB&Wp - 9BԸd$A7M_ `0MDpG1yb>&'~EOϛb:1:6a2QG `$5]؀VRӵeNka]*>/6K ޾D\vP^3Ml/baf 6n'nɾLb߀&o2#5GςʡX\܉I?! E}VTPP"vN~6W_5/S[]]5 821}lJ..RٙgUvpD p)8ѓjb$6G_z{no0Ӑp δ^`}< 8zPF.nc0۬8ݣv;}HaOh>!wg$}Nybw‡_㺸V X5XL}wD2݄3 Pn0n5NEVQ.%$D;f$%uZS,`1`YN5ffڅd2;|֋bM"S&&ls4.%33@G[8fb|<ӵ?rۢԚ]/AU(Ϥ&9[E6 6Wa]S" }Z7f.H+xi*N;fIډ6w0*EI~Hdž*'߲yP+ה-)KuDeB.E.e ٛE2. @H+Y}K}?>l*A򊦿1p jWCk7ގV5 EsP:= 骱A`3fF~,Gi5|ŸCaa.2r&_w+\,.l zV{JoƯ]oy}ҩE+TRݤ Mz^K 8>51_{Os[Oyuw=Tܨ} eGy'yOyZX3g],,`j-zoqq[ \( P`fRNu~kH6o D+!^M͵<׵y GHc|.0RNa! zo.qrH$ @ H$WLl^9 IENDB`weboob-1.2/modules/allocine/module.py000066400000000000000000000200321303450110500177130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.base import UserError from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES, BaseCalendarEvent from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.capabilities.cinema import CapCinema, Person, Movie from weboob.tools.backend import Module from .browser import AllocineBrowser __all__ = ['AllocineModule'] class AllocineModule(Module, CapCinema, CapVideo, CapCalendarEvent, CapCollection): NAME = 'allocine' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'AlloCiné French cinema database service' LICENSE = 'AGPLv3+' BROWSER = AllocineBrowser ASSOCIATED_CATEGORIES = [CATEGORIES.CINE] def get_movie(self, id): return self.browser.get_movie(id) def get_person(self, id): return self.browser.get_person(id) def iter_movies(self, pattern): return self.browser.iter_movies(pattern.encode('utf-8')) def iter_persons(self, pattern): return self.browser.iter_persons(pattern.encode('utf-8')) def iter_movie_persons(self, id, role=None): return self.browser.iter_movie_persons(id, role) def iter_person_movies(self, id, role=None): return self.browser.iter_person_movies(id, role) def iter_person_movies_ids(self, id): return self.browser.iter_person_movies_ids(id) def iter_movie_persons_ids(self, id): return self.browser.iter_movie_persons_ids(id) def get_person_biography(self, id): return self.browser.get_person_biography(id) def get_movie_releases(self, id, country=None): return self.browser.get_movie_releases(id, country) def fill_person(self, person, fields): if 'real_name' in fields or 'birth_place' in fields\ or 'death_date' in fields or 'nationality' in fields\ or 'short_biography' in fields or 'roles' in fields\ or 'birth_date' in fields or 'thumbnail_url' in fields\ or 'biography' in fields\ or 'gender' in fields or fields is None: per = self.get_person(person.id) person.real_name = per.real_name person.birth_date = per.birth_date person.death_date = per.death_date person.birth_place = per.birth_place person.gender = per.gender person.nationality = per.nationality person.short_biography = per.short_biography person.short_description = per.short_description person.roles = per.roles person.biography = per.biography person.thumbnail_url = per.thumbnail_url return person def fill_movie(self, movie, fields): if 'other_titles' in fields or 'release_date' in fields\ or 'duration' in fields or 'country' in fields\ or 'roles' in fields or 'note' in fields\ or 'thumbnail_url' in fields: mov = self.get_movie(movie.id) movie.other_titles = mov.other_titles movie.release_date = mov.release_date movie.duration = mov.duration movie.pitch = mov.pitch movie.country = mov.country movie.note = mov.note movie.roles = mov.roles movie.genres = mov.genres movie.short_description = mov.short_description movie.thumbnail_url = mov.thumbnail_url if 'all_release_dates' in fields: movie.all_release_dates = self.get_movie_releases(movie.id) return movie def fill_video(self, video, fields): if 'url' in fields: with self.browser: if not isinstance(video, BaseVideo): video = self.get_video(self, video.id) if hasattr(video, '_video_code'): video.url = unicode(self.browser.get_video_url(video._video_code)) if 'thumbnail' in fields and video and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video def get_video(self, _id): with self.browser: split_id = _id.split('#') if split_id[-1] == 'movie': return self.browser.get_movie_from_id(split_id[0]) return self.browser.get_video_from_id(split_id[0], split_id[-1]) def iter_resources(self, objs, split_path): with self.browser: if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield Collection([u'comingsoon'], u'Films prochainement au cinéma') yield Collection([u'nowshowing'], u'Films au cinéma') yield Collection([u'acshow'], u'Émissions') yield Collection([u'interview'], u'Interviews') if collection.path_level == 1: if collection.basename == u'acshow': emissions = self.browser.get_emissions(collection.basename) if emissions: for emission in emissions: yield emission elif collection.basename == u'interview': videos = self.browser.get_categories_videos(collection.basename) if videos: for video in videos: yield video else: videos = self.browser.get_categories_movies(collection.basename) if videos: for video in videos: yield video if collection.path_level == 2: videos = self.browser.get_categories_videos(':'.join(collection.split_path)) if videos: for video in videos: yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if collection.path_level == 1 and (collection.basename in [u'comingsoon', u'nowshowing', u'acshow', u'interview']): return if collection.path_level == 2 and collection.parent_path == [u'acshow']: return raise CollectionNotFound(collection.split_path) def search_events(self, query): with self.browser: if CATEGORIES.CINE in query.categories: if query.city and re.match('\d{5}', query.city): events = list(self.browser.search_events(query)) events.sort(key=lambda x: x.start_date, reverse=False) return events raise UserError('You must enter a zip code in city field') def get_event(self, id): return self.browser.get_event(id) def fill_event(self, event, fields): if 'description' in fields: movieCode = event.id.split('#')[2] movie = self.get_movie(movieCode) event.description = movie.pitch return event OBJECTS = { Person: fill_person, Movie: fill_movie, BaseVideo: fill_video, BaseCalendarEvent: fill_event } weboob-1.2/modules/allocine/test.py000066400000000000000000000107641303450110500174200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo from weboob.capabilities.calendar import Query, CATEGORIES from datetime import datetime import re class AllocineTest(BackendTest): MODULE = 'allocine' def test_search_movie(self): movies = list(self.backend.iter_movies('usual suspects')) assert len(movies) > 0 for movie in movies: assert movie.id def test_get_movie(self): movie = self.backend.get_movie('5032') assert movie assert movie.id assert movie.original_title def test_search_person(self): persons = list(self.backend.iter_persons('patrick dewaere')) assert len(persons) > 0 for person in persons: assert person.id def test_get_person(self): person = self.backend.get_person('1115') assert person assert person.id assert person.name assert person.birth_date def test_movie_persons(self): persons = list(self.backend.iter_movie_persons('5032')) assert len(persons) > 0 for person in persons: assert person.id assert person.name def test_person_movies(self): movies = list(self.backend.iter_person_movies('1115')) assert len(movies) > 0 for movie in movies: assert movie.id assert movie.original_title def test_get_person_biography(self): bio = self.backend.get_person_biography('1115') assert bio != '' assert bio is not None assert re.match(r'^Patrick Dewaere, fils.*', bio) def test_get_movie_releases(self): rel = self.backend.get_movie_releases('5032', 'fr') assert rel != '' assert rel is not None def test_iter_person_movies_ids(self): movies_ids = list(self.backend.iter_person_movies_ids('1115')) assert len(movies_ids) > 0 for movie_id in movies_ids: assert movie_id def test_iter_movie_persons_ids(self): persons_ids = list(self.backend.iter_movie_persons_ids('5032')) assert len(persons_ids) > 0 for person_id in persons_ids: assert person_id def test_emissions(self): l = list(self.backend.iter_resources([BaseVideo], [u'acshow'])) assert len(l) l1 = list(self.backend.iter_resources([BaseVideo], l[0].split_path)) assert len(l1) v = l1[0] self.backend.fillobj(v, 'url') self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_interview(self): l = list(self.backend.iter_resources([BaseVideo], [u'interview'])) assert len(l) v = l[0] self.backend.fillobj(v, 'url') self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_comingsoon(self): l = list(self.backend.iter_resources([BaseVideo], [u'comingsoon'])) assert len(l) v = l[0] self.backend.fillobj(v, 'url') self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_nowshowing(self): l = list(self.backend.iter_resources([BaseVideo], [u'nowshowing'])) assert len(l) v = l[0] self.backend.fillobj(v, 'url') self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_showtimelist(self): query = Query() query.city = u'59000' query.categories = [CATEGORIES.CINE] query.start_date = datetime.now() l = self.backend.search_events(query) assert len(l) e = l[0] self.backend.fillobj(e, 'description') self.assertTrue(e.description, 'Description of "%s" not found' % (e.id)) e = self.backend.get_event(e.id) self.assertTrue(e.description, 'Description of "%s" not found' % (e.id)) weboob-1.2/modules/alloresto/000077500000000000000000000000001303450110500162755ustar00rootroot00000000000000weboob-1.2/modules/alloresto/__init__.py000066400000000000000000000014361303450110500204120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AlloRestoModule __all__ = ['AlloRestoModule'] weboob-1.2/modules/alloresto/browser.py000066400000000000000000000037411303450110500203370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountsPage __all__ = ['AlloRestoBrowser'] class AlloRestoBrowser(LoginBrowser): BASEURL = 'https://www.alloresto.fr' login = URL('/identification-requise.*', LoginPage) accounts = URL('/chez-moi/releve-compte-miams', AccountsPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.accounts.stay_or_go() self.page.login(self.username, self.password) if self.login.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): return self.accounts.stay_or_go().iter_accounts() @need_login def get_account(self, id): assert isinstance(id, basestring) for a in self.get_accounts_list(): if a.id == id: return a return None @need_login def get_history(self, account): return self.accounts.stay_or_go().get_transactions(type='consommable') @need_login def get_coming(self, account): return self.accounts.stay_or_go().get_transactions(type='acquisition') weboob-1.2/modules/alloresto/favicon.png000066400000000000000000000044201303450110500204300ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME16tEXtCommentCreated with GIMPWxIDATxYkp~.5VA^h3u"U; -El6ժculN;jE#NG+:ZuP+Wi*TB II̍$\7˷w?on6{={Ç>|Ç>|);w-7u ;gzJ)wU_8DǑ=Ā)2gݴmN0?c4F$C5vvzx"bCW-wl>><9_|Q^7M9S|9ODB ͛/80m ~-kN'"@"czˠ,zfl ׌3of< `q 0a`S@WE wx:{cMpLO(&3a_$'t} ,vFbcҥb m;9'8 RJZqq`@1zĮ ~7hTL& ~m[u /]$]Z-՝4*2锗3f7?hlU+8!Z5>Y p5~>x"130Й  .X ?N :KPR.@$z;|Pޜ  Lآ s@[[XB˲.F3l *Rx7tt@`8#  ;;%Ix|Z|}zsꚼcrW.%%OG}̂C68F䇑HdR\")aoZ3,]JO2d @60Ǚ9R9X6NTZW]5a\'OĦMswkd2c~Hd[9Junv&N8׍سW:oxs21-ۇIrEEE0/}}b,T')@öԤcen<谼\1Vu#۷"uI3 Ι#n y{u+lׇ=ndNK45Ƕ+X7lǔxa}h@'$pi @@/JO[#VMjX4nD`%coÇ>|Ç>| UuAIENDB`weboob-1.2/modules/alloresto/module.py000066400000000000000000000037011303450110500201350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AlloRestoBrowser __all__ = ['AlloRestoModule'] class AlloRestoModule(Module, CapBank): NAME = 'alloresto' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Allo Resto' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = AlloRestoBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_coming(account) weboob-1.2/modules/alloresto/pages.py000066400000000000000000000053611303450110500177530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ItemElement, method from weboob.browser.filters.standard import CleanDecimal, CleanText, Filter, TableCell from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction as Transaction class LoginPage(HTMLPage): def login(self, username, password): form = self.get_form(xpath='//form[has-class("form_o")]') form['uname'] = username form['pass'] = password form.submit() class AccountsPage(LoggedPage, HTMLPage): @method class iter_accounts(ItemElement): def __call__(self): return self klass = Account obj_id = '0' obj_label = u'Compte miams' obj_balance = CleanDecimal('//div[@class="compteur"]//strong', replace_dots=True) obj_currency = u'MIAM' obj_coming = CleanDecimal('//table[@id="solde_acquisition_lignes"]//th[@class="col_montant"]', default=Decimal('0'), replace_dots=True) class MyDate(Filter): MONTHS = ['janv', u'févr', u'mars', u'avr', u'mai', u'juin', u'juil', u'août', u'sept', u'oct', u'nov', u'déc'] def filter(self, txt): day, month, year = txt.split(' ') day = int(day) year = int(year) month = self.MONTHS.index(month.rstrip('.')) + 1 return datetime.date(year, month, day) def get_transactions(self, type='consommable'): class get_history(Transaction.TransactionsElement): head_xpath = '//table[@id="solde_%s_lignes"]//thead//tr/th/text()' % type item_xpath = '//table[@id="solde_%s_lignes"]//tbody/tr' % type col_date = u"Date de valeur" col_raw = u"Motif" class item(Transaction.TransactionElement): obj_amount = Transaction.Amount('./td[last()]') obj_date = AccountsPage.MyDate(CleanText(TableCell('date'))) return get_history(self)() weboob-1.2/modules/alloresto/test.py000066400000000000000000000017671303450110500176410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AlloRestoTest(BackendTest): MODULE = 'alloresto' def test_alloresto(self): l = list(self.backend.iter_accounts()) a = l[0] list(self.backend.iter_history(a)) list(self.backend.iter_coming(a)) weboob-1.2/modules/allrecipes/000077500000000000000000000000001303450110500164145ustar00rootroot00000000000000weboob-1.2/modules/allrecipes/__init__.py000066400000000000000000000014411303450110500205250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AllrecipesModule __all__ = ['AllrecipesModule'] weboob-1.2/modules/allrecipes/browser.py000066400000000000000000000037101303450110500204520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import ResultsPage, RecipePage, HomePage import urllib __all__ = ['AllrecipesBrowser'] class AllrecipesBrowser(PagesBrowser): BASEURL = 'https://apps.allrecipes.com' results = URL('/v1/recipes\?(?P.*)', ResultsPage) recipe = URL('/v1/recipes/(?P<_id>.*)/', RecipePage) home = URL('http://allrecipes.com', HomePage) TOKEN = None def fill_token(self): self.home.open() self.TOKEN = 'Bearer %s' % self.session.cookies.get('ARToken') self.session.headers['X-Requested-With'] = 'XMLHttpRequest' self.session.headers['Authorization'] = self.TOKEN def iter_recipes(self, pattern): query = {'query': pattern, 'page': 1, 'pagesize': 20, 'sort': 're'} if not self.TOKEN: self.fill_token() return self.results.go(query=urllib.urlencode(query)).iter_recipes() def get_recipe(self, _id, obj=None): if not self.TOKEN: self.fill_token() recipe = self.recipe.go(_id=_id).get_recipe(obj=obj) comments = list(self.page.get_comments()) if comments: recipe.comments = comments return recipe weboob-1.2/modules/allrecipes/favicon.png000066400000000000000000000024621303450110500205530ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME0*IDATxhU??ZVQihfT0) 4(9Ċ"%+Dd߃n2aiia4ÌHj.y}ny{`9{g(-BΨ4ϔ|sib(=Fi@(D"@HS*)gPt2FJ\Yn!_z@'o@ D"e-#u`Ψ@30+#j&=h.[:# xS^/L\'zw["u!B`1[i^~>ׄeKk< xU[[[pF]< ꁯ%(=?8M(`z?劒x!/K[,_ + Dٿ pp,x,am}:W[g`^Z˖-"p>߿]`;%x4W'J j;[bj3bRt{I\6$''{@S4L?m@+V̽%uv$dr3Xj@I. l(¹"io8ҤoHӖhIlV/dXlV ΨJ~piw $W[cJZ!^򒇵?Iş_Vm?Ψ3,;,yzt8^`U@GPjw;Kђv OrE Ru`ll:{JcJVL 9W2,>}uR/TgԄ$!8 wj[#zmyI,uxF[:Ȇ'ͦC ^_Oo G^[FSDx20I~g4>q-u[ JAI{Qى\&41߳U匪qFMJ`wcQŶ>h9]\3Xjh;,66~GNA;o~SXC9fѷ(+'b>/fAB7_+&ߏBQ%{$|ʵ+uygT3jW9r0%J5@IENDB`weboob-1.2/modules/allrecipes/module.py000066400000000000000000000031661303450110500202610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.recipe import CapRecipe, Recipe from weboob.tools.backend import Module from .browser import AllrecipesBrowser from urllib import quote_plus __all__ = ['AllrecipesModule'] class AllrecipesModule(Module, CapRecipe): NAME = 'allrecipes' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'Allrecipes English recipe website' LICENSE = 'AGPLv3+' BROWSER = AllrecipesBrowser def get_recipe(self, id): return self.browser.get_recipe(id) def iter_recipes(self, pattern): return self.browser.iter_recipes(quote_plus(pattern.encode('utf-8'))) def fill_recipe(self, recipe, fields): if 'nb_person' in fields or 'instructions' in fields or 'thumbnail_url' in fields: recipe = self.browser.get_recipe(recipe.id, recipe) return recipe OBJECTS = {Recipe: fill_recipe} weboob-1.2/modules/allrecipes/pages.py000066400000000000000000000056101303450110500200670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage, JsonPage, pagination from weboob.browser.elements import ItemElement, DictElement, method from weboob.capabilities.recipe import Recipe, Comment from weboob.capabilities.base import NotAvailable from weboob.browser.filters.standard import Env, Format, Join from weboob.browser.filters.json import Dict class HomePage(HTMLPage): pass class ResultsPage(JsonPage): @pagination @method class iter_recipes(DictElement): item_xpath = 'recipes' def next_page(self): return Dict('links/next/href', default=None)(self.page.doc) class item(ItemElement): klass = Recipe obj_id = Dict('recipeID') obj_title = Dict('title') obj_short_description = Dict('description') class RecipePage(JsonPage): @method class get_recipe(ItemElement): klass = Recipe obj_id = Env('_id') obj_title = Dict('title') obj_short_description = Dict('description') obj_preparation_time = Dict('prepMinutes') obj_cooking_time = Dict('cookMinutes') def obj_nb_person(self): nb_pers = u'%s' % Dict('servings', default='')(self) return [nb_pers] if nb_pers else NotAvailable def obj_ingredients(self): ingredients = [] for el in Dict('ingredients')(self): ing = Format('%s (%s gramm)', Dict('displayValue'), Dict('grams'))(el) ingredients.append(ing) return ingredients def obj_instructions(self): ins = [Dict('displayValue')(el) for el in Dict('directions')(self)] return Join('\n * ', ins, addBefore=' * ', addAfter='\n')(self) obj_thumbnail_url = Dict('photo/photoDetailUrl') obj_picture_url = Dict('photo/photoDetailUrl') @method class get_comments(DictElement): item_xpath = 'topReviews' class item(ItemElement): klass = Comment obj_author = Dict('submitter/name') obj_rate = Dict('rating') obj_text = Dict('text') obj_id = Dict('reviewID') weboob-1.2/modules/allrecipes/test.py000066400000000000000000000022271303450110500177500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest import itertools class AllrecipesTest(BackendTest): MODULE = 'allrecipes' def test_recipe(self): recipes = list(itertools.islice(self.backend.iter_recipes('french fries'), 0, 20)) assert len(recipes) full_recipe = self.backend.get_recipe(recipes[0].id) assert full_recipe.instructions assert full_recipe.ingredients assert full_recipe.title weboob-1.2/modules/amazon/000077500000000000000000000000001303450110500155565ustar00rootroot00000000000000weboob-1.2/modules/amazon/__init__.py000066400000000000000000000014401303450110500176660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmazonModule __all__ = ['AmazonModule'] weboob-1.2/modules/amazon/browser.py000066400000000000000000000120031303450110500176070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from requests.exceptions import Timeout, ConnectionError from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.exceptions import ServerError, HTTPNotFound from weboob.capabilities.bill import Subscription, Bill from weboob.capabilities.shop import OrderNotFound from weboob.capabilities.base import NotAvailable from weboob.tools.decorators import retry from weboob.exceptions import BrowserIncorrectPassword from .pages import HomePage, LoginPage, AmazonPage, HistoryPage, \ OrderOldPage, OrderNewPage __all__ = ['Amazon'] class Amazon(LoginBrowser): BASEURL = 'https://www.amazon.com' MAX_RETRIES = 10 CURRENCY = u'$' home = URL(r'/$', r'http://www.amazon.com/$', HomePage) login = URL(r'/ap/signin.*$', LoginPage) history = URL(r'/gp/css/order-history.*$', r'/gp/your-account/order-history.*$', HistoryPage) order_old = URL(r'/gp/css/summary.*$', r'/gp/css/summary/edit.html\?orderID=%\(order_id\)s', r'/gp/digital/your-account/order-summary.html.*$', r'/gp/digital/your-account/orderPe-summary.html\?orderID=%\(order_id\)s', OrderOldPage) order_new = URL(r'/gp/css/summary.*$', r'/gp/your-account/order-details.*$', r'/gp/your-account/order-details\?orderID=%\(order_id\)s', OrderNewPage) unknown = URL(r'*', AmazonPage) def get_currency(self): return self.CURRENCY def get_order(self, id_): order = self.to_order(id_).order() if order: return order else: raise OrderNotFound() def iter_orders(self): histRoot = self.to_history() for histYear in histRoot.iter_years(): for order in histYear.iter_orders(): if order.order(): yield order.order() def iter_payments(self, order): return self.to_order(order.id).payments() def iter_items(self, order): return self.to_order(order.id).items() @need_login def to_history(self): self.history.stay_or_go() assert self.history.is_here() return self.page @need_login def to_order(self, order_id): """ Amazon updates its website in stages: they reroute a random part of their users to new pages, and the rest to old ones. """ for i in xrange(self.MAX_RETRIES): if (self.order_new.is_here() or self.order_old.is_here()) \ and self.page.order_number() == order_id: return self.page try: self.order_new.go(order_id=order_id) except HTTPNotFound: self.order_old.go(order_id=order_id) raise OrderNotFound() def do_login(self): self.session.cookies.clear() self.home.go().to_login().login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() def location(self, *args, **kwargs): """ Amazon throws 500 HTTP status code for apparently valid requests from time to time. Requests eventually succeed after retrying. """ for i in xrange(self.MAX_RETRIES): try: return super(Amazon, self).location(*args, **kwargs) except (ServerError, Timeout, ConnectionError) as e: pass raise e @need_login def get_subscription_list(self): sub = Subscription() sub.label = u'amazon' sub.id = u'amazon' yield sub @need_login def iter_documents(self, subscription): orders = self.iter_orders() for o in orders: b = Bill() b.url = unicode(o._bill['url']) b.id = '%s.%s' % (subscription.id, o.id) b.date = o.date b.price = o.total b.format = o._bill['format'] b.type = u'bill' b.currency = b.get_currency(self.get_currency()) b.label = '%s %s' % (subscription.label, o.date) b.vat = o.tax yield b @retry(HTTPNotFound) @need_login def download_document(self, url): doc = self.location(url) if not self.order_new.is_here(): return doc.content return NotAvailable weboob-1.2/modules/amazon/de/000077500000000000000000000000001303450110500161465ustar00rootroot00000000000000weboob-1.2/modules/amazon/de/__init__.py000066400000000000000000000000001303450110500202450ustar00rootroot00000000000000weboob-1.2/modules/amazon/de/browser.py000066400000000000000000000030641303450110500202060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import URL from ..browser import Amazon from weboob.exceptions import BrowserIncorrectPassword from .pages import OrderNewPageDE from ..pages import HomePage __all__ = ['AmazonDE'] class AmazonDE(Amazon): BASEURL = 'https://www.amazon.de' CURRENCY = u'€' home = URL(r'/$', r'.*/homepage\.html.*', HomePage) order_new = URL(r'/gp/css/summary.*$', r'/gp/your-account/order-details.*$', r'/gp/your-account/order-details\?orderID=%\(order_id\)s', OrderNewPageDE) def do_login(self): self.session.cookies.clear() self.home.go().to_login().login(self.username, self.password) # Switch language to english self.page.to_switchlanguage() if not self.page.logged: raise BrowserIncorrectPassword() weboob-1.2/modules/amazon/de/pages.py000066400000000000000000000020241303450110500176150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import datetime from ..pages import OrderNewPage class OrderNewPageDE(OrderNewPage): def order_date(self): return datetime.strptime( re.match('.*Ordered on ([0-9]+ [\w]+ [0-9]+) .*', self.date_num()).group(1), '%d %B %Y') weboob-1.2/modules/amazon/favicon.png000066400000000000000000000042671303450110500177220ustar00rootroot00000000000000PNG  IHDR@@iqgAMA a cHRMz&u0`:pQ<bKGD pHYs!!IDATx{leimm\F7   8AT`&B`B4`"2qh$) LFڭ9z= <>>ILb$ޱ(L}0`oNMMOv GD1 SQ"b=EF0x8G+MVqU _N8(эXk!Gáx-4:qA-a—ƿT};(H;z]&APwpXxDD[p>]s!"mڵDQO9-8D@! u(I]1-&Wq&| ')tPds{Z&WRw&Lgc\)9翄U"(DH*", CW~s E  [s\.rWo%]Q3 'D8Ԡ=^9>.8cIX |U^BACx hCy0L?cN8Ő#UI K@Oy0? =n qx5/Z)Mhh{oO97MiadEѐ-LfpMBÀ؜آL1TMFpTX%{ mh$&\͟2%tEXtdate:create2014-10-26T14:24:29-05:00p%tEXtdate:modify2014-10-26T14:24:29-05:00DutEXtSoftwareAdobe ImageReadyqe<IENDB`weboob-1.2/modules/amazon/fr/000077500000000000000000000000001303450110500161655ustar00rootroot00000000000000weboob-1.2/modules/amazon/fr/__init__.py000066400000000000000000000000001303450110500202640ustar00rootroot00000000000000weboob-1.2/modules/amazon/fr/browser.py000066400000000000000000000034001303450110500202170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import URL from ..browser import Amazon from .pages import HomePage, LoginPage, AmazonPage, HistoryPage, \ OrderOldPage, OrderNewPage __all__ = ['AmazonFR'] class AmazonFR(Amazon): BASEURL = 'https://www.amazon.fr' CURRENCY = u'€' home = URL(r'/$', r'.*/homepage\.html.*', HomePage) login = URL(r'/ap/signin.*$', LoginPage) history = URL(r'/gp/css/order-history.*$', r'/gp/your-account/order-history.*$', HistoryPage) order_old = URL(r'/gp/css/summary.*$', r'/gp/css/summary/edit.html\?orderID=%\(order_id\)s', r'/gp/digital/your-account/order-summary.html.*$', r'/gp/digital/your-account/orderPe-summary.html\?orderID=%\(order_id\)s', OrderOldPage) order_new = URL(r'/gp/css/summary.*$', r'/gp/your-account/order-details.*$', r'/gp/your-account/order-details\?orderID=%\(order_id\)s', OrderNewPage) unknown = URL(r'*', AmazonPage) weboob-1.2/modules/amazon/fr/pages.py000066400000000000000000000360561303450110500176500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.shop import Order, Payment, Item from weboob.browser.pages import HTMLPage, pagination, NextPage from weboob.capabilities.base import empty, NotAvailable from datetime import datetime from decimal import Decimal import re # Ugly array to avoid the use of french locale FRENCH_MONTHS = [u'janvier', u'février', u'mars', u'avril', u'mai', u'juin', u'juillet', u'août', u'septembre', u'octobre', u'novembre', u'décembre'] class AmazonPage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//*[contains(text(),"Déconnectez-vous")]')) class HomePage(AmazonPage): def to_login(self): url1 = self.doc.xpath('//a[@id="nav-link-yourAccount"]/@href') url2 = self.doc.xpath('//a[@id="nav-your-account"]/@href') self.browser.location((url1 or url2)[0]) return self.browser.page class LoginPage(AmazonPage): def login(self, email, password): form = self.get_form(name='signIn') form['email'] = email form['password'] = password form.submit() class HistoryPage(AmazonPage): forced_encoding = True ENCODING = 'UTF-8' def iter_years(self): for year in self.opt_years(): yield self.to_year(year) @pagination def iter_orders(self): for id_ in self.doc.xpath(u'//span[contains(text(),"N° de commande")]/../span[2]/text()'): yield self.browser.to_order(id_.strip()) for next_ in self.doc.xpath(u'//ul[@class="a-pagination"]' u'//a[contains(text(),"Suivante")]/@href'): raise NextPage(next_) def to_year(self, year): form = self.get_form('//form[contains(@class,"time-period-chooser")]') form['orderFilter'] = [year] form.submit() return self.browser.page def opt_years(self): return [x for x in self.doc.xpath( '//select[@name="orderFilter"]/option/@value' ) if x.startswith('year-')] class OrderPage(AmazonPage): def shouldSkip(self): # Reports only fully shipped and delivered orders, because they have # finalized payment amounts. # Payment for not yet shipped orders may change, and is not always # available. return bool([x for s in [u'En préparation pour expédition', u'En cours de préparation'] # TODO : Other French status applied ? for x in self.doc.xpath(u'//*[contains(text(),"%s")]' % s)]) def decimal_amount(self, amount): m = re.match(u'.*EUR ([,0-9]+).*', amount) if m: return Decimal(m.group(1).replace(",", ".")) def month_to_int(self, text): for (idx, month) in enumerate(FRENCH_MONTHS): text = text.replace(month, str(idx + 1)) return text class OrderNewPage(OrderPage): is_here = u'//*[contains(text(),"Commandé le")]' def order(self): if not self.shouldSkip(): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = self.tax() order.discount = self.discount() order.shipping = self.shipping() order.total = self.grand_total() order._bill = self.bill() return order def bill(self): pdf = self.doc.xpath(u'//a[contains(text(), "Imprimer une facture")]') htlm = self.doc.xpath(u'//a[contains(text(), "Imprimer un récapitulatif de commande")]') format = u'pdf' if pdf else u'html' url = pdf[0].attrib['href'] if pdf else htlm[0].attrib['href'] return {'url': url, 'format': format} def order_date(self): return datetime.strptime( re.match(u'.*Commandé le ([0-9]+ [0-9]+ [0-9]+) .*', self.month_to_int(self.date_num())).group(1), '%d %m %Y') def order_number(self): m = re.match(u'.*N° de commande : +([^ ]+) .*', self.date_num()) if m: return m.group(1) def payments(self): if self.gift(): pmt = Payment() pmt.date = self.order_date() pmt.method = u'GIFT CARD' pmt.amount = -self.gift() yield pmt transactions = list(self.transactions()) if transactions: for t in transactions: yield t else: for method in self.paymethods(): pmt = Payment() pmt.date = self.order_date() pmt.method = method pmt.amount = self.grand_total() yield pmt break def paymethods(self): for root in self.doc.xpath(u'//h5[contains(text(),"Méthode de paiement")]'): alt = root.xpath('../div/img/@alt')[0] span = root.xpath('../div/span/text()')[0] digits = re.match(r'[^0-9]*([0-9]+)[^0-9]*', span).group(1) yield u'%s %s' % (alt, digits) def grand_total(self): return self.decimal_amount(self.doc.xpath( '//span[contains(text(),"Montant total TTC")]/..' '/following-sibling::div[1]/span/text()')[0].strip()) def date_num(self): return u' '.join( self.doc.xpath( '//span[@class="order-date-invoice-item"]/text()' )).replace('\n', '') def tax(self): return self.amount(u' TVA') def shipping(self): return self.amount(u'Livraison :') def discount(self): # TODO : French translation return self.amount(u'Bon de réduction', u'Subscribe & Save', u'Your Coupon Savings', u'Lightning Deal') def gift(self): # TODO : French translation return self.amount(u'Gift Card Amount') def amount(self, *names): try: return Decimal(sum(self.decimal_amount(amount.strip()) for n in names for amount in self.doc.xpath( '(//span[contains(text(),"%s")]/../..//span)[2]/text()' % n))) except TypeError: return NotAvailable def transactions(self): for row in self.doc.xpath('//span[contains(text(),"Transactions")]' '/../../div/div'): text = row.text_content().strip().replace('\n', ' ') if u'Expédition' not in text: continue date, method, amount = re.match( '.* ' '([0-9]+ [^ ]+ [0-9]+)' '[ -]+' '([A-z][^:]+)' ': +' '(EUR [^ ]+)', text).groups() date = datetime.strptime(self.month_to_int(date), '%d %m %Y') method = method.replace(u'finissant par ', u'').upper() amount = self.decimal_amount(amount) pmt = Payment() pmt.date = date pmt.method = method pmt.amount = amount yield pmt def items(self): for item in self.doc.xpath('//div[contains(@class,"a-box shipment")]' '/div/div/div/div/div/div'): url = (item.xpath(u'*//a[contains(@href,"/gp/product")]/@href') + [u''])[0] label = u''.join(item.xpath( '*//a[contains(@href,"/gp/product")]/text()')).strip() price = u''.join(x.strip() for x in item.xpath( '*//span[contains(text(),"EUR")]/text()') if x.strip().startswith('EUR')) price = self.decimal_amount(price) multi = re.match(u'([0-9]+) de (.*)', label) if multi: amount, label = multi.groups() price *= Decimal(amount) if url: url = unicode(self.browser.BASEURL) + \ re.match(u'(/gp/product/.*)/ref=.*', url).group(1) if label and price: itm = Item() itm.label = label itm.url = url itm.price = price yield itm class OrderOldPage(OrderPage): forced_encoding = True ENCODING = 'ISO-8859-15' is_here = u'//*[contains(text(),"Amazon.fr numéro de commande")]' def order(self): if not self.shouldSkip(): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = Decimal(self.tax()) if not empty(self.tax()) else Decimal(0.00) order.discount = Decimal(self.discount()) if not empty(self.discount()) else Decimal(0.00) order.shipping = Decimal(self.shipping()) if not empty(self.shipping()) else Decimal(0.00) order.total = Decimal(self.grand_total()) if not empty(self.grand_total()) else Decimal(0.00) order._bill = self.bill() return order def bill(self): html = self.doc.xpath(u'//img[contains(@src, "print-invoice")]/parent::a') return {'url': html[0].attrib['href'], 'format': u'html'} def order_date(self): date_str = self.doc.xpath(u'//b[contains(text(),"Commande numérique")]')[0].text month_str = re.match(u'.*Commande numérique : [0-9]+ ([^ ]+) [0-9]+.*', date_str).group(1) return datetime.strptime( re.match(u'.*Commande numérique : ([0-9]+ [0-9]+ [0-9]+).*', date_str.replace(month_str, str(FRENCH_MONTHS.index(month_str) + 1))).group(1), '%d %m %Y') def order_number(self): num_com = u' '.join(self.doc.xpath( u'//b[contains(text(),"Amazon.fr numéro de commande")]/../text()') ).strip() return num_com def tax(self): return self.sum_amounts(u'TVA:') def discount(self): # TODO : French translation return self.sum_amounts(u'Subscribe & Save:', u'Bon de réduction:', u'Promotion Applied:', u'Your Coupon Savings:') def shipping(self): # TODO : French translation return self.sum_amounts(u'Shipping & Handling:', u'Free shipping:', u'Free Shipping:') def payments(self): for shmt in self.shipments(): gift = self.gift(shmt) if gift: pmt = Payment() pmt.date = self.order_date() pmt.method = u'CARTE CADEAU' pmt.amount = -gift yield pmt transactions = list(self.transactions()) if transactions: for t in transactions: yield t else: for method in self.paymethods(): pmt = Payment() pmt.date = self.order_date() pmt.method = method pmt.amount = self.grand_total() yield pmt break def shipments(self): # TODO : French translation for cue in (u'Shipment #', u'Subscribe and Save Shipment'): for shmt in self.doc.xpath('//b[contains(text(),"%s")]' % cue): yield shmt def items(self): for shmt in self.shipments(): root = shmt.xpath(u'../../../../../../../..' u'//b[text()="Articles commandés"]')[0] for item in root.xpath('../../../tr')[1:]: count = url = label = None for div in item.xpath('*//div'): m = re.match(u'^\s*(\d+)\s*of:(.*)$', div.text, re.MULTILINE + re.DOTALL) if not m: continue count = Decimal(m.group(1).strip()) label = unicode(m.group(2).strip()) if label: url = u'' else: a = div.xpath('*//a[contains(@href,"/gp/product")]')[0] url = unicode(a.attrib['href']) label = unicode(a.text.strip()) price1 = item.xpath('*//div')[-1].text.strip() price = count * self.decimal_amount(price1) itm = Item() itm.label = label itm.url = url itm.price = price yield itm def sum_amounts(self, *names): return sum(self.amount(shmt, x) for shmt in self.shipments() for x in names) def amount(self, shmt, name): for root in shmt.xpath(u'../../../../../../../..' u'//td[text()="Sous-total articles: "]/../..'): for node in root.xpath(u'tr/td[text()="%s"]' % name): return self.decimal_amount( node.xpath('../td')[-1].text.strip()) for node in root.xpath(u'tr/td/b[text()="%s"]' % name): return self.decimal_amount( node.xpath('../../td/b')[-1].text.strip()) return Decimal(0) def gift(self, shmt): # TODO : French translation return self.amount(shmt, u'Gift Card Amount:') def paymethods(self): # TODO : French translation root = self.doc.xpath('//b[text()="Payment Method: "]/..') if len(root) == 0: return root = root[0] text = root.text_content().strip() while text: # TODO : French translation for pattern in [ u'^.*Payment Method:', u'^([^\n]+)\n +\| Last digits: +([0-9]+)\n', u'^Gift Card\n', # Skip gift card. u'^Billing address.*$']: match = re.match(pattern, text, re.DOTALL+re.MULTILINE) if match: text = text[match.end():].strip() if match.groups(): yield u' '.join(match.groups()).upper() break else: break def transactions(self): # TODO : French translation for tr in self.doc.xpath( u'//div[contains(b,"Credit Card transactions")]' u'/following-sibling::table[1]/tr'): label, date = tr.xpath('td[1]/text()')[0].strip().split(u'\xa0') amount = tr.xpath('td[2]/text()')[0].strip() date = datetime.strptime(date, '%B %d, %Y:') method = label.replace(u'ending in ', u'')[:-1].upper() amount = self.decimal_amount(amount) pmt = Payment() pmt.date = date pmt.method = method pmt.amount = amount yield pmt def grand_total(self): return self.decimal_amount(self.doc.xpath( u'//td[contains(b,"Total pour cette commande")]/b')[0].text) weboob-1.2/modules/amazon/module.py000066400000000000000000000100351303450110500174140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bill import CapDocument, Subscription, Bill, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.shop import CapShop, Order from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.ordereddict import OrderedDict from weboob.browser.exceptions import HTTPNotFound from .browser import Amazon from .fr.browser import AmazonFR from .de.browser import AmazonDE __all__ = ['AmazonModule'] class AmazonModule(Module, CapShop, CapDocument): NAME = 'amazon' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Amazon' website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'www.amazon.com': u'Amazon.com', 'www.amazon.fr': u'Amazon France', 'www.amazon.de': u'Amazon.de', }.iteritems())]) BROWSERS = { 'www.amazon.com': Amazon, 'www.amazon.fr': AmazonFR, 'www.amazon.de': AmazonDE, } CONFIG = BackendConfig( Value('website', label=u'Website', choices=website_choices, default='www.amazon.com'), ValueBackendPassword('email', label='Username', masked=False), ValueBackendPassword('password', label='Password')) def create_default_browser(self): self.BROWSER = self.BROWSERS[self.config['website'].get()] return self.create_browser(self.config['email'].get(), self.config['password'].get()) def get_currency(self): return self.browser.get_currency() def get_order(self, id_): return self.browser.get_order(id_) def iter_orders(self): return self.browser.iter_orders() def iter_payments(self, order): if not isinstance(order, Order): order = self.get_order(order) return self.browser.iter_payments(order) def iter_items(self, order): if not isinstance(order, Order): order = self.get_order(order) return self.browser.iter_items(order) def iter_resources(self, objs, split_path): if Order in objs: self._restrict_level(split_path) return self.iter_orders() if Subscription in objs: self._restrict_level(split_path) return self.iter_subscription() def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.split('.')[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) if bill.url: try: return self.browser.download_document(bill.url) except HTTPNotFound: pass return None weboob-1.2/modules/amazon/pages.py000066400000000000000000000341161303450110500172340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.capabilities.shop import Order, Payment, Item from weboob.browser.pages import HTMLPage, pagination, NextPage from datetime import datetime from decimal import Decimal import re class AmazonPage(HTMLPage): def to_switchlanguage(self): url = self.doc.xpath('//a[contains(@href, "switchlang")]/@href') if not url: return m = re.search('language=([\w_]+)', url[0]) if m: url = url[0].replace(m.group(1), 'en_GB') self.browser.location(url) @property def logged(self): return bool(self.doc.xpath(u'//*[contains(text(),"Sign Out")]')) class HomePage(AmazonPage): def to_login(self): urls = [ self.doc.xpath('//a[@id="nav-link-yourAccount"]/@href'), self.doc.xpath('//a[@id="nav-your-account"]/@href'), self.doc.xpath('//a[@id="nav-link-accountList"]/@href'), ] url = filter(None, urls)[0][0] self.browser.location(url) return self.browser.page class LoginPage(AmazonPage): def login(self, email, password): form = self.get_form(name='signIn') form['email'] = email form['password'] = password form.submit() class HistoryPage(AmazonPage): def iter_years(self): for year in self.opt_years(): yield self.to_year(year) @pagination def iter_orders(self): for id_ in self.doc.xpath( u'//span[contains(text(),"Order #")]/../span[2]/text()'): yield self.browser.to_order(id_.strip()) for next_ in self.doc.xpath(u'//ul[@class="a-pagination"]' u'//a[contains(text(),"Next")]/@href'): raise NextPage(next_) def to_year(self, year): form = self.get_form('//form[contains(@class,"time-period-chooser")]') form['orderFilter'] = [year] form.submit() return self.browser.page def opt_years(self): return [x for x in self.doc.xpath( '//select[@name="orderFilter"]/option/@value' ) if x.startswith('year-')] class OrderPage(AmazonPage): def shouldSkip(self): # Reports only fully shipped and delivered orders, because they have # finalized payment amounts. # Payment for not yet shipped orders may change, and is not always # available. return bool([x for s in [u'Not Yet Shipped', u'Not yet shipped', u'Preparing for Shipment', u'Shipping now', u'In transit', u'On the way'] for x in self.doc.xpath(u'//*[contains(text(),"%s")]' % s)]) class OrderNewPage(OrderPage): is_here = u'//*[contains(text(),"Ordered on")]' def order(self): if not self.shouldSkip(): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = self.tax() order.discount = self.discount() order.shipping = self.shipping() order.total = self.grand_total() order._bill = self.bill() return order def bill(self): html = self.doc.xpath(u'//a[contains(text(), "Printable Order Summary")]') return {'url': html[0].attrib['href'], 'format': u'html'} def order_date(self): return datetime.strptime( re.match('.*Ordered on ([^ ]+ [0-9]+, [0-9]+) .*', self.date_num()).group(1), '%B %d, %Y') def order_number(self): m = re.match('.*Order[^#]+# +([^ ]+) .*', self.date_num()) if m: return m.group(1) def payments(self): if self.gift(): pmt = Payment() pmt.date = self.order_date() pmt.method = u'GIFT CARD' pmt.amount = -self.gift() yield pmt transactions = list(self.transactions()) if transactions: for t in transactions: yield t else: for method in self.paymethods(): pmt = Payment() pmt.date = self.order_date() pmt.method = method pmt.amount = self.grand_total() yield pmt break def paymethods(self): for root in self.doc.xpath(u'//h5[contains(text(),"Payment Method")]'): div = u''.join(root.xpath('../div/text()')).strip() if div: yield div else: alt = root.xpath('../div/img/@alt')[0] span = root.xpath('../div/span/text()')[0] digits = re.match(r'[^0-9]*([0-9]+)[^0-9]*', span).group(1) yield u'%s %s' % (alt, digits) def grand_total(self): return AmTr.decimal_amount(self.doc.xpath( u'//span[contains(text(),"Grand Total:")]/..' u'/following-sibling::div[1]/span/text()')[0].strip()) def date_num(self): return u' '.join(self.doc.xpath( '//span[@class="order-date-invoice-item"]/text()' ) or self.doc.xpath( '//*[contains(text(),"Ordered on")]/text()')).replace('\n', '') def tax(self): tax = self.amount(u'Estimated tax to be collected') if not tax: return self.amount(u' VAT') return tax def shipping(self): return self.amount(u'Free shipping', u'Free Shipping', u'Free Shipping Promo', u'Prime Pantry Delivery', u'Shipping & Handling') def discount(self): return self.amount(u'Promotion applied', u'Promotion Applied', u'Subscribe & Save', u'Your Coupon Savings', u'Lightning Deal', u'No-Rush Credit', u'Courtesy Credit') def gift(self): return self.amount(u'Gift Card Amount') def amount(self, *names): return Decimal(sum(AmTr.decimal_amount(amount.strip()) for n in names for amount in self.doc.xpath( '//span[contains(text(),"%s:")]' '/../following::div[1]/span/text()' % n))) def transactions(self): for row in self.doc.xpath('//span[contains(text(),"Transactions")]' '/../../div/div'): text = row.text_content().strip().replace('\n', ' ') if u'Items shipped:' not in text: continue date, method, amount = re.match( '.* ' '([A-z]+ [0-9]+, [0-9]+)' '[ -]+' '([A-z][^:]+)' ': +' '([^ ]+)', text).groups() date = datetime.strptime(date, '%B %d, %Y') method = method.replace(u'ending in ', u'').upper() amount = AmTr.decimal_amount(amount) pmt = Payment() pmt.date = date pmt.method = method pmt.amount = amount yield pmt def items(self): for item in self.doc.xpath('//div[contains(@class,"a-box shipment")]' '/div/div/div/div/div/div'): url = (item.xpath(u'*//a[contains(@href,"/gp/product")]/@href') + [u''])[0] label = u''.join(item.xpath( '*//a[contains(@href,"/gp/product")]/text()')).strip() price = u''.join(x.strip() for x in item.xpath( '*//span[contains(text(),"$")]/text()')[:1] if x.strip().startswith('$')) price = AmTr.decimal_amount(price) multi = re.match(u'([0-9]+) of (.*)', label) if multi: amount, label = multi.groups() price *= Decimal(amount) if url: url = unicode(self.browser.BASEURL) + \ re.match(u'(/gp/product/.*)/ref=.*', url).group(1) if label: itm = Item() itm.label = label itm.url = url itm.price = price yield itm class OrderOldPage(OrderPage): is_here = u'//*[contains(text(),"Amazon.com order number")]' def order(self): if not self.shouldSkip(): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = self.tax() order.discount = self.discount() order.shipping = self.shipping() return order def order_date(self): return datetime.strptime(u' '.join(self.doc.xpath( u'//b[contains(text(),"Order Placed")]/../text()')).strip(), '%B %d, %Y') def order_number(self): return u' '.join(self.doc.xpath( u'//td/b[contains(text(),"Amazon.com order number")]/../text()') ).strip() def tax(self): return self.sum_amounts(u'Sales Tax:') def discount(self): return self.sum_amounts(u'Subscribe & Save:', u'Promotion applied:', u'Promotion Applied:', u'Your Coupon Savings:') def shipping(self): return self.sum_amounts(u'Shipping & Handling:', u'Free shipping:', u'Free Shipping:') def payments(self): for shmt in self.shipments(): gift = self.gift(shmt) if gift: pmt = Payment() pmt.date = self.order_date() pmt.method = u'GIFT CARD' pmt.amount = -gift yield pmt transactions = list(self.transactions()) if transactions: for t in transactions: yield t else: for method in self.paymethods(): pmt = Payment() pmt.date = self.order_date() pmt.method = method pmt.amount = self.grand_total() yield pmt break def shipments(self): for cue in (u'Shipment #', u'Subscribe and Save Shipment'): for shmt in self.doc.xpath('//b[contains(text(),"%s")]' % cue): yield shmt def items(self): for shmt in self.shipments(): root = shmt.xpath(u'../../../../../../../..' u'//b[text()="Items Ordered"]')[0] for item in root.xpath('../../../tr')[1:]: count = url = label = None for div in item.xpath('*//div'): m = re.match(u'^\s*(\d+)\s*of:(.*)$', div.text, re.MULTILINE + re.DOTALL) if not m: continue count = Decimal(m.group(1).strip()) label = unicode(m.group(2).strip()) if label: url = u'' else: a = div.xpath('*//a[contains(@href,"/gp/product")]')[0] url = unicode(a.attrib['href']) label = unicode(a.text.strip()) price1 = item.xpath('*//div')[-1].text.strip() price = count * AmTr.decimal_amount(price1) itm = Item() itm.label = label itm.url = url itm.price = price yield itm def sum_amounts(self, *names): return sum(self.amount(shmt, x) for shmt in self.shipments() for x in names) def amount(self, shmt, name): for root in shmt.xpath(u'../../../../../../../..' u'//td[text()="Item(s) Subtotal: "]/../..'): for node in root.xpath(u'tr/td[text()="%s"]' % name): return AmTr.decimal_amount( node.xpath('../td')[-1].text.strip()) for node in root.xpath(u'tr/td/b[text()="%s"]' % name): return AmTr.decimal_amount( node.xpath('../../td/b')[-1].text.strip()) return Decimal(0) def gift(self, shmt): return self.amount(shmt, u'Gift Card Amount:') def paymethods(self): root = self.doc.xpath('//b[text()="Payment Method: "]/..')[0] text = root.text_content().strip() while text: for pattern in [ u'^.*Payment Method:', u'^([^\n]+)\n +\| Last digits: +([0-9]+)\n', u'^Gift Card\n', # Skip gift card. u'^Billing address.*$']: match = re.match(pattern, text, re.DOTALL+re.MULTILINE) if match: text = text[match.end():].strip() if match.groups(): yield u' '.join(match.groups()).upper() break else: break def transactions(self): for tr in self.doc.xpath( u'//div[contains(b,"Credit Card transactions")]' u'/following-sibling::table[1]/tr'): label, date = tr.xpath('td[1]/text()')[0].strip().split(u'\xa0') amount = tr.xpath('td[2]/text()')[0].strip() date = datetime.strptime(date, '%B %d, %Y:') method = label.replace(u'ending in ', u'')[:-1].upper() amount = AmTr.decimal_amount(amount) pmt = Payment() pmt.date = date pmt.method = method pmt.amount = amount yield pmt def grand_total(self): return AmTr.decimal_amount(self.doc.xpath( '//td[b="Grand Total:"]/following-sibling::td[1]/b')[0].text) weboob-1.2/modules/amazon/test.py000066400000000000000000000021461303450110500171120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AmazonTest(BackendTest): MODULE = 'amazon' def test_history(self): """ Test that at least one item was ordered in the whole history. """ b = self.backend items = (i for o in b.iter_orders() for i in b.iter_items(o)) item = next(items, None) self.assertNotEqual(item, None) weboob-1.2/modules/amazonstorecard/000077500000000000000000000000001303450110500174655ustar00rootroot00000000000000weboob-1.2/modules/amazonstorecard/__init__.py000066400000000000000000000014671303450110500216060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmazonStoreCardModule __all__ = ['AmazonStoreCardModule'] weboob-1.2/modules/amazonstorecard/browser.py000066400000000000000000000141711303450110500215260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import AccountNotFound from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword import json import os from tempfile import mkstemp from subprocess import check_output, STDOUT, CalledProcessError from urllib import unquote from .pages import SomePage, StatementsPage, StatementPage, SummaryPage, \ ActivityPage __all__ = ['AmazonStoreCard'] class AmazonStoreCard(LoginBrowser): BASEURL = 'https://www.synchronycredit.com' MAX_RETRIES = 10 TIMEOUT = 120.0 stmts = URL('/eService/EBill/eBillAction.action$', StatementsPage) statement = URL('eService/EBill/eBillViewPDFAction.action.*$', StatementPage) summary = URL('/eService/AccountSummary/initiateAccSummaryAction.action$', SummaryPage) activity = URL('/eService/BillingActivity' '/initiateBillingActivityAction.action$', ActivityPage) unknown = URL('.*', SomePage) def __init__(self, phone, code_file, *args, **kwargs): super(AmazonStoreCard, self).__init__(*args, **kwargs) self.phone = phone self.code_file = code_file def do_login(self): scrf, scrn = mkstemp('.js') cookf, cookn = mkstemp('.json') os.write(scrf, LOGIN_JS % { 'timeout': 300, 'username': self.username, 'password': self.password, 'output': cookn, 'code': self.code_file, 'phone': self.phone[-4:], 'agent': self.session.headers['User-Agent']}) os.close(scrf) os.close(cookf) for i in xrange(self.MAX_RETRIES): try: check_output(["phantomjs", scrn], stderr=STDOUT) break except CalledProcessError as error: pass else: raise error with open(cookn) as cookf: cookies = json.loads(cookf.read()) os.remove(scrn) os.remove(cookn) self.session.cookies.clear() for c in cookies: for k in ['expiry', 'expires', 'httponly']: c.pop(k, None) c['value'] = unquote(c['value']) self.session.cookies.set(**c) if not self.summary.go().logged: raise BrowserIncorrectPassword() @need_login def get_account(self, id_): a = next(self.iter_accounts()) if (a.id != id_): raise AccountNotFound() return a @need_login def iter_accounts(self): yield self.summary.go().account() @need_login def iter_history(self, account): for t in self.activity.go().iter_recent(): yield t for s in self.stmts.go().iter_statements(): for t in s.iter_transactions(): yield t LOGIN_JS = u'''\ var TIMEOUT = %(timeout)s*1000; // milliseconds var page = require('webpage').create(); page.settings.userAgent = "%(agent)s"; page.open('http://www.syncbank.com/amazon'); var waitForForm = function() { var hasForm = page.evaluate(function(){ return !!document.getElementById('secLoginBtn') }); if (hasForm) { page.evaluate(function(){ document.getElementById('loginUserID').value = '%(username)s'; document.getElementById('loginPassword').value = '%(password)s'; var href = document.getElementById('secLoginBtn').getAttribute('href'); window.location.href = href; }); } else { setTimeout(waitForForm, 1000); } } var waitForLogin = function() { var hasLogout = page.content.indexOf('Logout') != -1; if (hasLogout) { var cookies = JSON.stringify(phantom.cookies); require('fs').write('%(output)s', cookies, 'w'); phantom.exit(); } else { setTimeout(waitForLogin, 2000); } } var waitForSendCode = function() { var hasSendCode = page.evaluate(function(){ return document.getElementsByClassName('sendCodeTo').length > 0 && document.getElementById('phoneNumbers').children.length > 0; }); if (hasSendCode) { page.evaluate(function(){ var nums = document.getElementById('phoneNumbers').children; for (var inum = 0; inum < nums.length; inum++) { var num = nums[inum].children[0]; if (num.text.indexOf('%(phone)s') != -1) { selectPhone((inum+1).toString()); var methods = document.getElementById('deliveryMethods').children; for (var imtd = 0; imtd < methods.length; imtd++) { var method = methods[imtd].children[0]; if (method.text.indexOf('SMS') != -1) { selectDeliveryMenthod((imtd+1).toString()); otpGenerateAjax('otpGenerateAjax'); return; } } } } }); } else { setTimeout(waitForSendCode, 2000); } } var waitForEnterCode = function() { var hasEnterCode = page.evaluate(function(){ return !!document.getElementById('fourDigitId'); }); var code = ""; try { code = require('fs').read('%(code)s'); } catch (e) { } if (hasEnterCode && code.length >= 4) { page.evaluate(function(code){ document.getElementById('fourDigitId').value = code.substr(0,4); document.getElementById('Yes1').checked = true; otpVerifyAjax('otpVerifyAjax'); }, code); } else { setTimeout(waitForEnterCode, 2000); } } waitForForm(); waitForLogin(); waitForSendCode(); waitForEnterCode(); setTimeout(function(){phantom.exit(-1);}, TIMEOUT); ''' weboob-1.2/modules/amazonstorecard/favicon.png000066400000000000000000000115461303450110500216270ustar00rootroot00000000000000PNG  IHDR@@iqgAMA a cHRMz%u0`:o_FbKGDC pHYs  tIME WUIDATxkWuvG$ʖll)زl8l?ȡ T*M!Ʃ _EE|JR Bp FQeKD"Bλo>ܞّ]ު}{9mm6FhmZ6 p3.&T}Ofbb5eFJt!D4Y!ÐjJP@)mhra OJIeH6˲Gh?7xsUkS˱cm{Os1QB3 /8ѯA8{U9{?(Q8}4g/|Rij>ώDz,~;||>OZ0GŶl*+&&&,_СC=?,.,Rx|xɏq{,ϲg~~O|9sG=|tD lRHcea]y8/rol޺.{AJ8߿SN)#96>Ω'i|OQS׸ۙݳ Bo>woյ5R(G><ٵkra&&&!ҩt{%)%2 #EH r/Klè0Dt\#ZBB*z򋌎ODBmN ~|M=>?y9 U;T ~ϭXO</H']aXdbbkwŭ"[f܄,fgg#_(bY2mz!nm/;o2uwP.)|8t$C#tMTV*slَ_z9c;la+/FJ0SSS=?}᠔R"N#,'FXr,PC;,ؒP4t*r+PDE.8K}T5M,> ]4aƻf|IR1hE<#!c6>vl6AfiZ=gL% #anh~_Q;qS0B:(%@ E2xv7iecvKJIZ144D5*PRR*| <4/2LɉI.QY\\$޾4 lb݄0 b/@)ø@k +]ԋmcZI,X,b-,Q+zcԳ!fTBl6ٶmv |_[h4hi@6[ng?;|)%j;_?OV'ǀ$Dv/7@QTjPY,>F9u+2lU5N-7/u3"G} ZdsY0$Q( S'Or}lټr7`uu7_'fͱ{v/r쫌zo (,6BH86ZX2LDwAR[J25' BT:`0\'d}Ď!:`mG[ Q EGɋd3= E*ᮥ,4JHcmYI'bYͮ$U@wv]c25 i)z)'lҥ%1׃\p.lT[7.pյ?n o?X ͛%^{7w\Ri2B>$i7TRq F@b;RvXe!6 ڨ@ 17 )%ADډ0 i1(ظ뺱ͶZ-R v/2i ۡAg,0%w8ο܉eY,/-Ө׹if;KKxGGAP`dd$ޝE.]Įݻz*7lb" d2<_rϽfj:y!$cccuZ&(ָ̎|[0::?[TXZ\bl|5|ϟitf ۶%ܡD^kM:&Qh5['͒JI9TתץԪUY]Ya8BT*rNOOuHql&&'<Z&zU2,hk:b}AQ2E4"\.yAiyHƽZXX`CdY :!tjp^BЉ⦦jLou=yn݊mxH)q]U)/rr]bFU2 m3>9I²,FFFߧZ8?GҤlㄈ1!s(oH)H9)ܰL6ˎwD\>J*P(PTDAumbqm{vqص{zMR,l`bBnh6)J-t#6m"Ea-];P()H;dT*B'8 ); {Z$F,NE$,!I䯓^w&$Q6*XoM?A8G^&ɉ&xCǪ뺴~Vv=f~=ϓw`S)и7p0 <`P$פּҿ/ G,As8)d뭈M?8pCM3}ӛZi<}!tQu(ݜGT߬dVz]dΐ'#~[7 )$} RVJQ5)Qcԍt\NϷEbÓ {1&ݥB!D:'''i4`6Zw߮a`$UP>ooV?6l]#;%SSS,..4}MMMr9PĂ烀q= h׏H(& qgv`vm|>G*B0 )J^wxO.`\C7u џe1NŸ 5%-Pw*NdlfzAXRYV6y+RX#dCA٤VRYXKKr9Zq&p]5ҙ(ŽRP(Q*hBlRiEFFF( %K/1`s_uH:~f~kx6LTѤ\.ݟUk\vZk\ ZT@T.T b˲=zR##Ph&~bl6D#mVH yRt4НqFFc< D5\.yqBkL)~K3NW6ύ\؝a2E{m۱ċPhL\JR)|>O']/ Źs !)8s %KI'g\m1 W0b XqihxZ+ض} !9 P빺,Q/}D{l$ b^fIxOAkĥKh4HCA@:uI8XPnncbs\fI`hh!$*BLan [nfyB'|j4u24fxDB6!8kFGf R׎ꡡmm6Fh׵|9%tEXtdate:create2014-12-28T13:20:51-06:00(s%tEXtdate:modify2014-12-28T13:20:51-06:00u$IENDB`weboob-1.2/modules/amazonstorecard/module.py000066400000000000000000000042321303450110500213250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AmazonStoreCard __all__ = ['AmazonStoreCardModule'] class AmazonStoreCardModule(Module, CapBank): NAME = 'amazonstorecard' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Amazon Store Card' CONFIG = BackendConfig( ValueBackendPassword('username', label='User ID', masked=False), ValueBackendPassword('password', label='Password'), ValueBackendPassword('phone', label='Phone to send verification code to', masked=False), ValueBackendPassword('code_file', label='File to read the verification code from', masked=False)) BROWSER = AmazonStoreCard def create_default_browser(self): return self.create_browser(username = self.config['username'].get(), password = self.config['password'].get(), phone = self.config['phone'].get(), code_file = self.config['code_file'].get()) def iter_accounts(self): return self.browser.iter_accounts() def get_account(self, id_): return self.browser.get_account(id_) def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/amazonstorecard/pages.py000066400000000000000000000210001303450110500211270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import Account, Transaction from weboob.browser.exceptions import ServerError from weboob.browser.pages import HTMLPage, RawPage from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.tools.date import closest_date from weboob.tools.pdf import decompress_pdf from weboob.tools.tokenizer import ReTokenizer from datetime import datetime, timedelta import re import json class SomePage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//a[text()="Logout"]')) class SummaryPage(SomePage): def account(self): label = u' '.join(self.doc.xpath( '//div[contains(@class,"myCreditCardDetails")]')[0]\ .text_content().split()) balance = self.amount(u'Balance') cardlimit = self.doc.xpath( u'//li[text()="Available to Spend"]')[0].text_content()\ .replace(u'Available to Spend', u'').replace(u'Limit', u'').strip() paymin = self.amount(u'Payment Due') if self.doc.xpath(u'//li[@class="noPaymentDue"]'): # If payment date is not scheduled yet, set it somewhere in a # distant future, so that we always have a valid date. paydate = datetime.now() + timedelta(days=999) else: rawtext = self.doc.xpath( u'//li[contains(text(),"Due Date")]')[0].text_content() datetext = re.match('.*(\d\d/\d\d/\d\d\d\d).*', rawtext).group(1) paydate = datetime.strptime(datetext, '%m/%d/%Y') a = Account() a.id = label[-4:] a.label = label a.currency = Account.get_currency(balance) a.balance = -AmTr.decimal_amount(balance) a.type = Account.TYPE_CARD a.cardlimit = AmTr.decimal_amount(cardlimit) a.paymin = AmTr.decimal_amount(paymin) if paydate is not None: a.paydate = paydate return a def amount(self, name): return u''.join(self.doc.xpath( u'//li[text()[.="%s"]]/../li[1]'%name)[0].text_content().split())\ .replace(u'\xb7',u'.').replace(u'*',u'') class ActivityPage(SomePage): def iter_recent(self): records = json.loads(self.doc.xpath( '//div[@id="completedActivityRecords"]//input[1]/@value')[0]) recent = [x for x in records if x['PDF_LOC'] is None] for rec in sorted(recent, ActivityPage.cmp_records, reverse=True): desc = u' '.join(rec['TRANS_DESC'].split()) trans = Transaction((rec['REF_NUM'] or u'').strip()) trans.date = ActivityPage.parse_date(rec['TRANS_DATE']) trans.rdate = ActivityPage.parse_date(rec['POST_DATE']) trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = -AmTr.decimal_amount(rec['TRANS_AMOUNT']) yield trans @staticmethod def cmp_records(rec1, rec2): return cmp(ActivityPage.parse_date(rec1['TRANS_DATE']), ActivityPage.parse_date(rec2['TRANS_DATE'])) @staticmethod def parse_date(recdate): return datetime.strptime(recdate, u'%B %d, %Y') class StatementsPage(SomePage): def iter_statements(self): jss = self.doc.xpath(u'//a/@onclick[contains(.,"eBillViewPDFAction")]') for js in jss: url = re.match("window.open\('([^']*).*\)", js).group(1) for i in xrange(self.browser.MAX_RETRIES): try: self.browser.location(url) break except ServerError as e: pass else: raise e yield self.browser.page class StatementPage(RawPage): LEX = [ ('charge_amount', r'^\(\$(\d+(,\d{3})*\.\d{2})\) Tj$'), ('payment_amount', r'^\(\\\(\$(\d+(,\d{3})*\.\d{2})\\\)\) Tj$'), ('date', r'^\((\d+/\d+)\) Tj$'), ('full_date', r'^\((\d+/\d+/\d+)\) Tj$'), ('layout_td', r'^([-0-9]+ [-0-9]+) Td$'), ('ref', r'^\(([A-Z0-9]{17})\) Tj$'), ('text', r'^\((.*)\) Tj$') ] def __init__(self, *args, **kwArgs): RawPage.__init__(self, *args, **kwArgs) assert self.doc[:4] == '%PDF' self._pdf = decompress_pdf(self.doc) self._tok = ReTokenizer(self._pdf, '\n', self.LEX) def iter_transactions(self): return sorted(self.read_transactions(), cmp=lambda t1, t2: cmp(t2.date, t1.date) or cmp(t1.label, t2.label) or cmp(t1.amount, t2.amount)) def read_transactions(self): # Statement typically cover one month. # Do 60 days, just to be on a safe side. date_to = self.read_closing_date() date_from = date_to - timedelta(days=60) pos = 0 while not self._tok.tok(pos).is_eof(): pos, trans = self.read_transaction(pos, date_from, date_to) if trans: yield trans else: pos += 1 def read_transaction(self, pos, date_from, date_to): startPos = pos pos, tdate = self.read_date(pos) pos, pdate_layout = self.read_layout_td(pos) pos, pdate = self.read_date(pos) pos, ref_layout = self.read_layout_td(pos) pos, ref = self.read_ref(pos) pos, desc_layout = self.read_layout_td(pos) pos, desc = self.read_text(pos) pos, amount_layout = self.read_layout_td(pos) pos, amount = self.read_amount(pos) if tdate is None or pdate is None \ or desc is None or amount is None or amount == 0: return startPos, None else: tdate = closest_date(tdate, date_from, date_to) pdate = closest_date(pdate, date_from, date_to) desc = u' '.join(desc.split()) trans = Transaction(ref or u'') trans.date = tdate trans.rdate = pdate trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = amount return pos, trans def read_amount(self, pos): pos, ampay = self.read_payment_amount(pos) if ampay is not None: return pos, ampay return self.read_charge_amount(pos) def read_charge_amount(self, pos): return self._tok.simple_read('charge_amount', pos, lambda xs: -AmTr.decimal_amount(xs[0])) def read_payment_amount(self, pos): return self._tok.simple_read('payment_amount', pos, lambda xs: AmTr.decimal_amount(xs[0])) def read_closing_date(self): pos = 0 while not self._tok.tok(pos).is_eof(): pos, text = self.read_text(pos) if text == u'Statement Closing Date': break pos += 1 while not self._tok.tok(pos).is_eof(): pos, date = self.read_full_date(pos) if date is not None: return date pos += 1 def read_text(self, pos): t = self._tok.tok(pos) #TODO: handle PDF encodings properly. return (pos+1, unicode(t.value(), errors='ignore')) \ if t.is_text() else (pos, None) def read_full_date(self, pos): t = self._tok.tok(pos) return (pos+1, datetime.strptime(t.value(), '%m/%d/%Y')) \ if t.is_full_date() else (pos, None) def read_date(self, pos): t = self._tok.tok(pos) return (pos+1, datetime.strptime(t.value(), '%m/%d')) \ if t.is_date() else (pos, None) def read_ref(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_ref() else (pos, None) def read_layout_td(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_layout_td() else (pos, None) weboob-1.2/modules/amazonstorecard/test.py000066400000000000000000000022211303450110500210130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from itertools import chain class AmazonStoreCardTest(BackendTest): MODULE = 'amazonstorecard' def test_history(self): """ Test that there's at least one transaction in the whole history. """ b = self.backend ts = chain(*[b.iter_history(a) for a in b.iter_accounts()]) t = next(ts, None) self.assertNotEqual(t, None) weboob-1.2/modules/ameli/000077500000000000000000000000001303450110500153605ustar00rootroot00000000000000weboob-1.2/modules/ameli/__init__.py000066400000000000000000000014451303450110500174750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmeliModule __all__ = ['AmeliModule'] weboob-1.2/modules/ameli/browser.py000066400000000000000000000104571303450110500174240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, LoginValidationPage, HomePage, AccountPage, LastPaymentsPage, PaymentsPage, PaymentDetailsPage __all__ = ['AmeliBrowser'] class AmeliBrowser(LoginBrowser): BASEURL = 'https://assure.ameli.fr' loginp = URL('/PortailAS/appmanager/PortailAS/assure\?.*_pageLabel=as_login_page', LoginPage) login_validationp = URL('https://assure.ameli.fr:443/PortailAS/appmanager/PortailAS/assure;jsessionid=[a-zA-Z0-9!;-]+\?_nfpb=true&_windowLabel=connexioncompte_2&connexioncompte_2_actionOverride=%2Fportlets%2Fconnexioncompte%2Fvalidationconnexioncompte&_pageLabel=as_login_page$', LoginValidationPage) homep = URL('/PortailAS/appmanager/PortailAS/assure\?_nfpb=true&_pageLabel=as_accueil_page', HomePage) accountp = URL('/PortailAS/appmanager/PortailAS/assure\?_nfpb=true&_pageLabel=as_info_perso_page', AccountPage) paymentsp = URL('/PortailAS/appmanager/PortailAS/assure\?_nfpb=true&_pageLabel=as_paiements_page', PaymentsPage) paymentdetailsp = URL('/PortailAS/paiements.do\?actionEvt=chargerDetailPaiements.*', PaymentDetailsPage) lastpaymentsp = URL('/PortailAS/paiements.do\?actionEvt=afficherPaiements.*', LastPaymentsPage) logged = False def do_login(self): self.logger.debug('call Browser.do_login') if self.logged: return True self.loginp.stay_or_go() if self.homep.is_here(): self.logged = True return True self.page.login(self.username, self.password) error = self.page.is_error() if error: raise BrowserIncorrectPassword(error) self.homep.stay_or_go() # Redirection not interpreted by browser. Mannually redirect on homep if not self.homep.is_here(): raise BrowserIncorrectPassword() self.logged = True @need_login def iter_subscription_list(self): self.logger.debug('call Browser.iter_subscription_list') self.accountp.stay_or_go() return self.page.iter_subscription_list() @need_login def get_subscription(self, id): self.logger.debug('call Browser.get_subscription') assert isinstance(id, basestring) for sub in self.iter_subscription_list(): if id == sub._id: return sub return None @need_login def iter_history(self, sub): self.logger.debug('call Browser.iter_history') self.paymentsp.stay_or_go() payments_url = self.page.get_last_payments_url() self.location(payments_url) assert self.lastpaymentsp.is_here() urls = self.page.iter_last_payments() for url in urls: self.location(url) assert self.paymentdetailsp.is_here() for payment in self.page.iter_payment_details(sub): yield payment @need_login def iter_documents(self, sub): self.logger.debug('call Browser.iter_documents') self.paymentsp.stay_or_go() payments_url = self.page.get_last_payments_url() self.location(payments_url) assert self.lastpaymentsp.is_here() for document in self.page.iter_documents(sub): yield document @need_login def get_document(self, id): self.logger.debug('call Browser.get_document') assert isinstance(id, basestring) subs = self.iter_subscription_list() for sub in subs: for b in self.iter_documents(sub): if id == b.id: return b return False weboob-1.2/modules/ameli/favicon.png000066400000000000000000000155271303450110500175250ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYsodtIME=\IDATxyt\W?^*%ۑbgqĄI i&tw N2CtϴJ=3}0>Ðt@wX4 h8ˉCNl)be-UR;R=sf~kG?(Kw!= / 4=!K. ,ik%@ghO>"U7?8t>b# K-(Ƚdח Djoո@ xN*[1wE4!'}LN_}`I`o%4}?߁vs1 u^IU0_4SK!%E&}>Q` ))$pLQ]qZp׾b}{V|_?3W/ q0w,OY7^)->|NKz-^z͡ϼdBg>c*Cli(D^~_| #; /}s7eC2zu9*,d gg0&|7`?l/?B#qDzxv_@ KJPRtwFYussyڥ Lk DD2ObzuPV@X?wLoEm=!K>{3 0q'슱1 GgL>>~ (s:}%YΆ+e\^ \ ^fQmp=6l0w3zm4(CJ{Q,Yfi7`C{wxZl,B^c>)t xdWJ1=dAASKY]㐽%`eb;1Os;z+3!sVX!bS&sznZ=~Ջ ac860!Z7:l<,M.J[ztې1R2ѱ1VCy3d ,m.Vo=u)]B=,f{rtr2HERIfSg`k/PiӺ>OjGw ,S’\15zy 10H}bۻ1ί8[꺰BB{M>bZwB sMf|mwD0$DTK~f͐A8~ƺ(;O1=!4]`Dk;ďK5 =O(lV /8r|"M޼K.V ʦG[Wo#zxB'l FBAc!)f<޷{x!X;۶vZ=d= ijm"Jl8Sh c.pY1YinBVRq6~~/z*]RI*$kB=G1}8< pLDĜ#erqu'fʐY (TBF8Y|[ ?# ߿r޿$'Sf :0(9~'֮5Uޓ I}{` ^ߎԬ% RtibKrjhDL !Ϯә߹؟I ,DdW-2x_oĶCT=o"3یzD›j$b{9Ol ?ck7ա-,d8>4g-grIr'Y0!99qt__.> _M}*8fole|*Ο"n 5ЈfIxw:@ZC<ќ|ޥSo% ]T 0=vfLG5.J0ufYJӓBڸDZ̧vRȏ)QRI V1dYĐLYk?η!蒼Ssy ֬lr jZ :lT0ZȬdԿ%{워Ce3Qs\ySfg <~kWI _&|(ģ6)".CI&}`flC)VrhPN -ekC),&z&Ȝq\cw&q\gZҿĈ5  aXWS?辁K>ظ5h6^:NiPBz moACXV#*rYa_~5B7%+䮀cg]9zI DЇ u|7v0d2UWdaރ"Il iu 8~T% >/=<R4sIfIo0ƀ ֽ,%7l=\Z`dw_{*8 '^%1PDh+MK&R% IQ;#,p?^iUa?~^_*&Є.oKIkS +D |_p\)+/P9'*FX0^Ղ:0E7&ܚ$Xbbf z#-Jkd'_QL- FĪ*R8֯S[¦ڢλϏLəV_;( Y1\+ (@8?0{i |ߐ~ff[ȭ݀ #tl>@>CaMі)mQ4E`]rBaжC@\Q3Z}G9 )f61>G2HuO!s`ف_U>S gU غM+@@ы7&c>lǴ 1388>6NsCrqG' oȻH]) ,S K!"^E9V+C.? b֭hhe6C|t{Ԡ5GuϥXxW5Fۭ|oABppSlrb5&)29Wdvx{q#`m.HW׀)ĭ/}s#=[SL07ى7F|{p}Yn{9iJ p@(g4r) P޹͢q6@c&k d)ACHϕ ԴžxgƐ"5@Hm~EhO<$BDEcQřt~wmY|'йaL` ѸEUyj\ޒ4 {<ȡ(Sm$*@v~{C`UH~iDr_1yHӘ۝Ԧ/47cfj k9 3Ollb?SIh` l1[GՃ-LVmKdh? 5$2ݸj/o &7N~cAw\e ~yL?~W57{ŲT-f_g>F'Tߚnݷ9VuԮ[m[Zw$X1Ӡ5 S,2c} V`6 sVYD(v*5`[eQs ]knM\I;N>񕶘*Z;~e{)i"k `w|.cF :ZΚ,H +pA nA/v UM "*Uſ&E+e\LʢH٪$9-oыB9Zi *78<Ŗ$,@l 4} cu5>aye_fp 6H/O `)H)Wx_F|21L˖OǓuH.kl|N P^DYuS(.(hS NÝTU]= W&Y9˨HbT-ev<$o0hݵ׻ţ.mK%e=Dk]ʜeq,3aGVC ՜˞D hd!0ڶrǖ!VYY1ENh_T#",~(jxeVdct`~`q[n5 ]MGyox6 d=l+TB` `) Dyh +`X&l/v\3fWxMuj/9tА:@ yq*&U*Ht aKf"), ۘ|XN2hz2k)HZGG:͹ PH_JZps[Lbwtw~tށSi^BV iKRb%Gy!NS~oLz1ȴȱM$[_}^dG JU`IZ!,MOYepf''86~B(pcX,bٳ+ഢ*uJE 17814U|Yqy@`h ֶ{k"BNN`_'Fq flj|+o>of7W5K&T2 Piat?y!L3?YgJ%4BAͣD[OZ_bʛ;_zw@ O@L$d4]ϡ^a/c5>fZ~ i-TS~\Y:_SPd]YWza@z]7k/;&A~ƿ@WD8S^K}-GIj>-/kͩ!= 1yq  ϝWgbb$ WOP]etyE}ݍGP!0ElN-z\,EHkEoB`CkO㙬aW'm~0?עzz(~ Nab1_r E@L^طbtώ ARN}~"jL^cr YEAT/jݠ-Li iaPD6Mq&=@nk#: )C>%(׼_,ӿD3P C X@fƏkGl߱zVd( 315&1s.f^[ V&Иs1I3bfdȾC6]c8_Llnua'. Зg  n ̭0tK(Hֳp$v-ݿX=^V`۲e݅w)mݳ+k̅@o+ 3?+DE\IENDB`weboob-1.2/modules/ameli/module.py000066400000000000000000000057611303450110500172300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bill import CapDocument, SubscriptionNotFound, DocumentNotFound, Subscription, Bill from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AmeliBrowser __all__ = ['AmeliModule'] class AmeliModule(Module, CapDocument): NAME = 'ameli' DESCRIPTION = u'Ameli website: French Health Insurance' MAINTAINER = u'Christophe Lampin' EMAIL = 'weboob@lampin.net' VERSION = '1.2' LICENSE = 'AGPLv3+' BROWSER = AmeliBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Numero de SS', masked=False), ValueBackendPassword('password', label='Password', masked=True) ) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.iter_subscription_list() def get_subscription(self, _id): subscription = self.browser.get_subscription(_id) if not subscription: raise SubscriptionNotFound() else: return subscription def iter_documents_history(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_history(subscription) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def get_document(self, id): bill = self.browser.get_document(id) if not bill: raise DocumentNotFound() else: return bill def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) request = self.browser.open(bill.url, stream=True) assert(request.headers['content-type'] == "application/pdf") return request.content weboob-1.2/modules/ameli/pages.py000066400000000000000000000211641303450110500170350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime import re import urllib from decimal import Decimal from weboob.browser.pages import HTMLPage from weboob.capabilities.bill import Subscription, Detail, Bill from weboob.browser.filters.standard import CleanText, Regexp # Ugly array to avoid the use of french locale FRENCH_MONTHS = [u'janvier', u'février', u'mars', u'avril', u'mai', u'juin', u'juillet', u'août', u'septembre', u'octobre', u'novembre', u'décembre'] class AmeliBasePage(HTMLPage): def is_logged(self): if self.doc.xpath('//a[@id="id_lien_deco"]'): logged = True else: logged = False self.logger.debug('logged: %s' % (logged)) return logged def is_error(self): errors = self.doc.xpath(u'//*[@id="r_errors"]') if errors: return errors[0].text_content() return False class LoginPage(AmeliBasePage): def login(self, login, password): form = self.get_form('//form[@name="connexionCompteForm"]') form['connexioncompte_2numSecuriteSociale'] = login.encode('utf8') form['connexioncompte_2codeConfidentiel'] = password.encode('utf8') form.submit() class LoginValidationPage(AmeliBasePage): pass class HomePage(AmeliBasePage): pass class AccountPage(AmeliBasePage): def iter_subscription_list(self): names_list = self.doc.xpath('//span[@class="NomEtPrenomLabel"]') fullname = CleanText(newlines=True).filter(names_list[0]) number = re.sub('[^\d]+', '', CleanText('//span[@class="blocNumSecu"]', replace=[(' ','')])(self.doc)) sub = Subscription(number) sub._id = number sub.label = unicode(fullname) firstname = CleanText('//span[@class="prenom-titulaire"]')(self.doc) sub.subscriber = unicode(firstname) yield sub class PaymentsPage(AmeliBasePage): def get_last_payments_url(self): dateDebut = self.doc.xpath('//input[@id="paiements_1dateDebut"]/@value')[0]; dateFin = self.doc.xpath('//input[@id="paiements_1dateFin"]/@value')[0]; url = "/PortailAS/paiements.do?actionEvt=afficherPaiementsComplementaires&DateDebut=" + dateDebut + "&DateFin=" + dateFin + "&Beneficiaire=tout_selectionner&afficherReleves=false&afficherIJ=false&afficherInva=false&afficherRentes=false&afficherRS=false&indexPaiement=&idNotif=" return url class LastPaymentsPage(AmeliBasePage): def iter_last_payments(self): elts = self.doc.xpath('//li[@class="rowitem remboursement"]') for elt in elts: items = Regexp(CleanText('./@onclick'), r'.*ajaxCallRemoteChargerDetailPaiement \(\'(\w+)\', \'(\w+)\', \'(\d+)\', \'(\d+)\'\).*', '\\1,\\2,\\3,\\4')(elt).split(',') yield "/PortailAS/paiements.do?actionEvt=chargerDetailPaiements&idPaiement=" + items[0] + "&naturePaiement=" + items[1] + "&indexGroupe=" + items[2] + "&indexPaiement=" + items[3] def iter_documents(self, sub): elts = self.doc.xpath('//li[@class="rowdate"]') for elt in elts: try: elt.xpath('.//a[contains(@id,"lienPDFReleve")]')[0] except IndexError: continue date_str = elt.xpath('.//span[contains(@id,"moisEnCours")]')[0].text month_str = date_str.split()[0] date = datetime.strptime(re.sub(month_str, str(FRENCH_MONTHS.index(month_str) + 1), date_str), "%m %Y").date() bil = Bill() bil.id = sub._id + "." + date.strftime("%Y%m") bil.date = date bil.format = u'pdf' bil.type = u'bill' bil.label = u'' + date.strftime("%Y%m%d") bil.url = unicode('/PortailAS/PDFServletReleveMensuel.dopdf?PDF.moisRecherche='+date.strftime("%m%Y")) yield bil def get_document(self, bill): self.location(bill.url, urllib.urlencode(bill._args)) class PaymentDetailsPage(AmeliBasePage): def iter_payment_details(self, sub): id_str = self.doc.xpath('//div[@class="entete container"]/h2')[0].text.strip() m = re.match('.*le (.*) pour un montant de.*', id_str) if m: blocs_benes = self.doc.xpath('//span[contains(@id,"nomBeneficiaire")]') blocs_prestas = self.doc.xpath('//table[@id="tableauPrestation"]') i = 0 last_bloc = len(blocs_benes) for i in range(0, last_bloc): bene = blocs_benes[i].text; id_str = m.group(1) id_date = datetime.strptime(id_str, '%d/%m/%Y').date() id = sub._id + "." + datetime.strftime(id_date, "%Y%m%d") table = blocs_prestas[i].xpath('.//tr') line = 1 last_date = None for tr in table: tds = tr.xpath('.//td') if len(tds) == 0: continue det = Detail() # TO TEST : Indemnités journalières : Pas pu tester de cas de figure similaire dans la nouvelle mouture du site if len(tds) == 4: date_str = Regexp(r'.*
(\d+/\d+/\d+)\).*', '\\1')(tds[0].text) det.id = id + "." + str(line) det.label = unicode(tds[0].xpath('.//span')[0].text.strip()) jours = tds[1].text if jours is None: jours = '0' montant = tds[2].text if montant is None: montant = '0' price = tds[3].text if price is None: price = '0' if date_str is None or date_str == '': det.infos = u'' det.datetime = last_date else: det.infos = date_str + u' (' + unicode(re.sub('[^\d,-]+', '', jours)) + u'j) * ' + unicode(re.sub('[^\d,-]+', '', montant)) + u'€' det.datetime = datetime.strptime(date_str.split(' ')[3], '%d/%m/%Y').date() last_date = det.datetime det.price = Decimal(re.sub('[^\d,-]+', '', price).replace(',', '.')) if len(tds) == 5: date_str = Regexp(pattern=r'\w*(\d{2})/(\d{2})/(\d{4}).*', template='\\1/\\2/\\3', default="").filter("".join(tds[0].itertext())) det.id = id + "." + str(line) det.label = bene + u' - ' + unicode(tds[0].xpath('.//span')[0].text.strip()) paye = tds[1].text if paye is None: paye = '0' base = tds[2].text if base is None: base = '0' tdtaux = tds[3].xpath('.//span')[0].text if tdtaux is None: taux = '0' else: taux = tdtaux.strip() tdprice = tds[4].xpath('.//span')[0].text if tdprice is None: price = '0' else: price = tdprice.strip() if date_str is None or date_str == '': det.infos = u'' det.datetime = last_date else: det.infos = u' Payé ' + unicode(re.sub('[^\d,-]+', '', paye)) + u'€ / Base ' + unicode(re.sub('[^\d,-]+', '', base)) + u'€ / Taux ' + unicode(re.sub('[^\d,-]+', '', taux)) + '%' det.datetime = datetime.strptime(date_str, '%d/%m/%Y').date() last_date = det.datetime det.price = Decimal(re.sub('[^\d,-]+', '', price).replace(',', '.')) line = line + 1 yield det weboob-1.2/modules/ameli/test.py000066400000000000000000000021461303450110500167140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AmeliTest(BackendTest): MODULE = 'ameli' def test_ameli(self): for subscription in self.backend.iter_subscription(): list(self.backend.iter_documents_history(subscription.id)) for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) weboob-1.2/modules/amelipro/000077500000000000000000000000001303450110500161015ustar00rootroot00000000000000weboob-1.2/modules/amelipro/__init__.py000066400000000000000000000014531303450110500202150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmeliProModule __all__ = ['AmeliProModule'] weboob-1.2/modules/amelipro/browser.py000066400000000000000000000107631303450110500201450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bill import Detail from decimal import Decimal from .pages import LoginPage, HomePage, AccountPage, HistoryPage, BillsPage, SearchPage __all__ = ['AmeliProBrowser'] class AmeliProBrowser(LoginBrowser): BASEURL = 'https://espacepro.ameli.fr:443' loginp = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_pageLabel=vp_login_page', LoginPage) homep = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_pageLabel=vp_accueil_page', HomePage) accountp = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_pageLabel=vp_coordonnees_infos_perso_page', AccountPage) billsp = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_pageLabel=vp_releves_mensuels_page', BillsPage) searchp = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_pageLabel=vp_recherche_par_date_paiements_page', SearchPage) historyp = URL('/PortailPS/appmanager/portailps/professionnelsante\?_nfpb=true&_windowLabel=vp_recherche_paiement_tiers_payant_portlet_1&vp_recherche_paiement_tiers_payant_portlet_1_actionOverride=%2Fportlets%2Fpaiements%2Frecherche&_pageLabel=vp_recherche_par_date_paiements_page', HistoryPage) logged = False def do_login(self): self.logger.debug('call Browser.do_login') if self.logged: return True self.loginp.stay_or_go() if self.homep.is_here(): self.logged = True return True self.page.login(self.username, self.password) if not self.homep.is_here(): raise BrowserIncorrectPassword() self.logged = True @need_login def get_subscription_list(self): self.logger.debug('call Browser.get_subscription_list') self.accountp.stay_or_go() return self.page.iter_subscription_list() @need_login def get_subscription(self, id): assert isinstance(id, basestring) return self.get_subscription_list() @need_login def iter_history(self, subscription): self.searchp.stay_or_go() date_deb = self.page.doc.xpath('//input[@name="vp_recherche_paiement_tiers_payant_portlet_1dateDebutRecherche"]')[0].value date_fin = self.page.doc.xpath('//input[@name="vp_recherche_paiement_tiers_payant_portlet_1dateFinRecherche"]')[0].value data = {'vp_recherche_paiement_tiers_payant_portlet_1dateDebutRecherche': date_deb, 'vp_recherche_paiement_tiers_payant_portlet_1dateFinRecherche': date_fin, 'vp_recherche_paiement_tiers_payant_portlet_1codeOrganisme': 'null', 'vp_recherche_paiement_tiers_payant_portlet_1actionEvt': 'rechercheParDate', 'vp_recherche_paiement_tiers_payant_portlet_1codeRegime': '01', } self.session.headers.update({'Content-Type': 'application/x-www-form-urlencoded'}) self.historyp.go(data=urllib.urlencode(data)) if self.historyp.is_here(): return self.page.iter_history() @need_login def get_details(self, sub): det = Detail() det.id = sub.id det.label = sub.label det.infos = '' det.price = Decimal('0.0') return det @need_login def iter_documents(self): self.billsp.stay_or_go() return self.page.iter_documents() @need_login def get_document(self, id): assert isinstance(id, basestring) for b in self.iter_documents(): if id == b.id: return b return None @need_login def download_document(self, bill): request = self.open(bill.url, data=bill._data, stream=True) return request.content weboob-1.2/modules/amelipro/favicon.png000066400000000000000000000153301303450110500202360ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYsodtIME"IeIDATxi\W羗{e־I%X,yXؖlcn3`C73ѕ=C0`Gxnj^d㰌mY*kWRm{·U,w{YI1w\+apDHIxrעKo~+Eyλdq)6vb#l3HyyxVD}'<`ďM7X4֛B7.Ҋ>vgQy %> Dwee&a߅(Ҏ$[zO$:O~M{W1ZںzcЈh@Un Ta.yCiŲoCxsnv>0w-/"oU~J| 螇v~(G!;^Q+V-sPM}G^uŲe>j%|eTɝ^#cU X*yϯUJ_~J YZV78˟/;Hvnw9º~ |~9CTyuI5lh^n?]~=RMylx`v]5Y}sƷOwZe5(2;=yЍa;d_8EUߢ,]$YV=e8{P ^eÊ(-EwuT֎Ƚ:Ua _Wu?xnP_yf5aD4c0F0F x/d{c ^]fœ3- k#E_&/ 9f<)e6Bz}Ww fu{uu!t.L' ',D# nX?v։cC80!8l4ú,7]z*,iiOiX1E4Uq/o[L"xVg {aes7/BQI;MLR:tI"< 8rycyVIm)V]@" 8$oqNDգ1\Qo:K۰@BȻGwZa(tsc$}8_](  OsUJb=yӆ,3 =\WpeGZ<Ӈ g>m yV|c$p4!X!ؾ(XI18w3yn,#%_EZpsj`z+CZgC\P1 Q|&CAW2l'c ;R@Bo>ƀJ֟bϽbIzEӀ!JyEE} 7o1zPSݝ( wcw^0Q 9_#ȹMA| ed]0ʎ1  L Aqj^=haU58CM%L;n> ;L~,w\q ozyT?% 5rBXX @ i_?MhuSd|rBF?DE2Kze4D2_Vxs_?BMKUS,ErFay }?xwse`m)LLQ0ea%Б9T)N7[eQ:sEuk9s'w8[DWr14KPe\d * ?\@>Z~,H#pA nA9V'TM(R"@"&;5-oӋB9Zii+Ww88Ė$$@l 4#|Wa>vqau`p2TH5/O)rSВ]j,dd O'8:;os8@ه(x oZE\1D2X¦0SB\7smrI2h)`},9e1ENh_T#"ʫ,~(jJ nuGit\cx6n3  .RDF2Tp~0 Urx@ %xgI\# ~DjC[Dҭa[-Mb,H>YC>ۄ6(-XQ!HA藇I*kcr5}s %b8 >_8bZ_ĊdU_0.NFcIKMP覢uXf ;3ܷY 0-H'| 񥌎kGL~{&qxocX=u_$?)<_0WIafۚzoI7MSӑy Gȧ.a8wDfKOM|Ͼ,e[H&b/1 _‰k܀l)l~RTf񌆰gR|/hE5AӍo,2+ ,,18F! Pag!Xeз8βyq\+O-ÿBҏ?1eH.iQg0CGrb2XVLlT !O:#% )YOZh8E&]dt,iC>7LnY9<ǚDT>JZ<3ĶMUL[YO>+84V_PqP#COSy˘I^I&>fFWx,YZd[s!uP_wbmn{8xwf٫,|=KGKN}I$T-O,!ZgcϢ;RUYZ,1l)c@zMnO qGbBBMfqF0 (*}L522\Tkk?f)VMsy,+}YNBEuFA=l+/ԛneW%}GIœH7Yu1qwʼnSv .زn= qB:W:ꅾE,f1^?~X?\Wnfex,˚4UI64g i(pd-2d@g !qja+~Y|01k7zEaw2m.X̼km%g 'mTC _$zuS/>@.b\!aA26,(u0ES5gM[S^bแœθH_}.WxxTtw_=A#?3L3i+U#8 ~)dpj#b(BZ#E(zg->XQDό8:}|GA CXzvꑭ t8&GMB7e#tǥ{O{0B Jاn W )5&tHe,TB3@HZѯSHmPG~ƐOe 0Ț:ۗ2ݻJ3@/t)3q670mh?VV3K̄U13. from weboob.capabilities.bill import CapDocument, SubscriptionNotFound, DocumentNotFound, Subscription, Bill from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AmeliProBrowser __all__ = ['AmeliProModule'] class AmeliProModule(Module, CapDocument): NAME = 'amelipro' DESCRIPTION = u'Ameli website: French Health Insurance for Professionals' MAINTAINER = u'Christophe Lampin' EMAIL = 'weboob@lampin.net' VERSION = '1.2' LICENSE = 'AGPLv3+' BROWSER = AmeliProBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='numero de SS', masked=False), ValueBackendPassword('password', label='Password', masked=True) ) def create_default_browser(self): self.logger.settings['save_responses'] = False # Set to True to help debugging return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): if not _id.isdigit(): raise SubscriptionNotFound() subscription = self.browser.get_subscription(_id) if not subscription: raise SubscriptionNotFound() else: return subscription def iter_documents_history(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_history(subscription) def get_details(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.get_details(subscription) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents() def get_document(self, id): bill = self.browser.get_document(id) if not bill: raise DocumentNotFound() else: return bill def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) return self.browser.download_document(bill) weboob-1.2/modules/amelipro/pages.py000066400000000000000000000114161303450110500175550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime import re from decimal import Decimal from weboob.browser.pages import HTMLPage from weboob.capabilities.bill import Subscription, Detail, Bill # Ugly array to avoid the use of french locale FRENCH_MONTHS = [u'janvier', u'février', u'mars', u'avril', u'mai', u'juin', u'juillet', u'août', u'septembre', u'octobre', u'novembre', u'décembre'] class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form('//form[@name="connexionCompteForm"]') form['vp_connexion_portlet_1numPS'] = login.encode('utf8') form['vp_connexion_portlet_1password'] = password.encode('utf8') form.submit() class HomePage(HTMLPage): def on_loaded(self): pass class SearchPage(HTMLPage): def on_loaded(self): pass class AccountPage(HTMLPage): def iter_subscription_list(self): ident = self.doc.xpath('//div[@id="identification"]')[0] prof = self.doc.xpath('//div[@id="profession"]')[0] name = ident.xpath('//p/b')[0].text.replace(' ', ' ').strip() number = ident.xpath('//p')[1].text.replace('Cabinet', '').strip() label = prof.xpath('//div[@class="zoneTexte"]')[0].text.strip() sub = Subscription(number) sub._id = number sub.label = unicode(name) + ' ' + unicode(label) sub.subscriber = unicode(name) return sub class HistoryPage(HTMLPage): def iter_history(self): tables = self.doc.xpath('//table[contains(concat(" ", @class, " "), " cTableauTriable ")]') if len(tables) > 0: lines = tables[0].xpath('.//tr') sno = 0 for tr in lines: list_a = tr.xpath('.//a') if len(list_a) == 0: continue date = tr.xpath('.//td')[0].text.strip() lot = list_a[0].text.replace('(*)', '').strip() if lot == 'SNL': sno = sno + 1 lot = lot + str(sno) factures = tr.xpath('.//div[@class="cAlignGauche"]/a') factures_lbl = '' for a in factures: factures_lbl = factures_lbl + a.text.replace('(**)', '').strip() + ' ' montant = tr.xpath('.//div[@class="cAlignDroite"]')[0].text.strip() det = Detail() det.id = u''+lot det.label = u''+lot det.infos = u''+factures_lbl det.datetime = datetime.strptime(date, "%d/%m/%Y").date() det.price = Decimal(montant.replace(',', '.')) yield det class BillsPage(HTMLPage): def iter_documents(self): table = self.doc.xpath('//table[@id="releveCompteMensuel"]')[0].xpath('.//tr') for tr in table: list_tds = tr.xpath('.//td') if len(list_tds) == 0: continue date_str = tr.xpath('.//td[@class="cAlignGauche"]')[0].text month_str = date_str.split()[0] date = datetime.strptime(re.sub(month_str, str(FRENCH_MONTHS.index(month_str) + 1), date_str), "%m %Y").date() amount = tr.xpath('.//td[@class="cAlignDroite"]')[0].text amount = re.sub('[^\d,-]+', '', amount) for format in ('CSV', 'PDF'): bil = Bill() bil.id = date.strftime("%Y%m") + format bil.date = date clean_amount = amount.strip().replace(',','.') if clean_amount != '': bil.price = Decimal('-'+clean_amount) else: bil.price = 0 bil.label = u''+date.strftime("%Y%m%d") bil.format = u''+format bil.type = u'bill' filedate = date.strftime("%m%Y") bil.url = u'/PortailPS/fichier.do' bil._data = {'FICHIER.type': format.lower()+'.releveCompteMensuel', 'dateReleve': filedate, 'FICHIER.titre': 'Releve' + filedate } yield bil weboob-1.2/modules/amelipro/test.py000066400000000000000000000021451303450110500174340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AmeliProTest(BackendTest): MODULE = 'amelipro' def test_AmeliPro(self): for subscription in self.backend.iter_subscription(): list(self.backend.iter_history(subscription.id)) for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) weboob-1.2/modules/americanexpress/000077500000000000000000000000001303450110500174625ustar00rootroot00000000000000weboob-1.2/modules/americanexpress/__init__.py000066400000000000000000000014521303450110500215750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmericanExpressModule __all__ = ['AmericanExpressModule'] weboob-1.2/modules/americanexpress/browser.py000066400000000000000000000103511303450110500215170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from urlparse import urlsplit, parse_qsl from urllib import urlencode from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.browsers import LoginBrowser, need_login from weboob.browser.url import URL from .pages import LoginPage, AccountsPage, TransactionsPage, WrongLoginPage, AccountSuspendedPage __all__ = ['AmericanExpressBrowser'] class AmericanExpressBrowser(LoginBrowser): BASEURL = 'https://global.americanexpress.com' login = URL('/myca/logon/.*', LoginPage) wrong_login = URL('/myca/fuidfyp/emea/.*', WrongLoginPage) account_suspended = URL('/myca/onlinepayments/', AccountSuspendedPage) partial_account = URL(r'/myca/intl/isummary/emea/summary.do\?method=reloadCardSummary&Face=fr_FR&sorted_index=(?P\d+)', AccountsPage) accounts = URL('/myca/intl/isummary/.*', AccountsPage) transactions = URL('/myca/intl/estatement/.*', TransactionsPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.login.is_here(): self.location('/myca/logon/emea/action?request_type=LogonHandler&DestPage=https%3A%2F%2Fglobal.americanexpress.com%2Fmyca%2Fintl%2Facctsumm%2Femea%2FaccountSummary.do%3Frequest_type%3D%26Face%3Dfr_FR%26intlink%3Dtopnavvotrecompteneligne-HPmyca&Face=fr_FR&Info=CUExpired') self.page.login(self.username, self.password) if self.wrong_login.is_here() or self.login.is_here() or self.account_suspended.is_here(): raise BrowserIncorrectPassword() @need_login def go_on_accounts_list(self): if self.transactions.is_here(): form = self.page.get_form(name='leftnav') form.url = '/myca/intl/acctsumm/emea/accountSummary.do' form.submit() else: self.partial_account.go(idx='0') @need_login def get_accounts_list(self): if not self.accounts.is_here(): self.go_on_accounts_list() for idx, cancelled in self.page.get_idx_list(): account = self.get_account_by_idx(idx) if account._link or not cancelled: yield account @need_login def get_account_by_idx(self, idx): # xhr request fetching partial html of account info form = self.page.get_form(name='j-session-form') form.url = self.partial_account.build(idx=idx) form.submit() assert self.partial_account.is_here() return self.page.get_account() @need_login def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == id: return a return None @need_login def get_history(self, account): if not self.accounts.is_here(): self.go_on_accounts_list() url = account._link if not url: return while url is not None: if self.accounts.is_here(): self.location(url) else: form = self.page.get_form(name='leftnav') form.url = url form.submit() assert self.transactions.is_here() for tr in self.page.get_history(account.currency): yield tr if self.page.is_last(): url = None else: v = urlsplit(url) args = dict(parse_qsl(v.query)) args['BPIndex'] = int(args['BPIndex']) + 1 url = '%s?%s' % (v.path, urlencode(args)) weboob-1.2/modules/americanexpress/favicon.png000066400000000000000000000072221303450110500216200ustar00rootroot00000000000000PNG  IHDR@@iqbKGDC pHYsgRtIME 5%uIDATxڕ[[^U?gaz&@hE(PZX㛷 QPQ1A4C4J`T#%\q:0Sڙ9{^gs^o}kzef0ڿrk$89 08gfb] Z<0!B{1:U! (%uD!9 nGEdRPyU`sNŘwX?|%HG0R(PssZ)Jip'\5q?uh4p`:X4}cR5}cvYwT#[%Huh. „7;$k {.Ԯ'!^9`ѯ*A"u+< e+",c06 YwEvCkD"vL0J}XCm0mpqSSl!"OJ? ѝYYB0Q9"fjZ9 K:wreU@M(sФLjG,"aPd6i9*/F1W_lZ(uPH0׬:d䘌:F}U-ҘD[V.Tie`f}=+Pz]<ldK ɃdhTT}uA jޏNBhߗJHG#g Hf^0\HK>e`HEyC|2U@m^7C,> 9x* r%) uKC?:D ^ڜWpfcܭĂ}ej@Tj) \BWC!m _agjy,3q;\0nGAjB+!!֯dJ}nq,bKƀW0uƀƱ%? )m7Ri7L.dldO\pP(=ʞy;6~yl ưcS߾|6) lF+qQa[T褐ZWI*=8󻇇) qƫxMk7&hht|=d$h9+Ik$W2_㑗}(41m*<3=T ȻojDi~t6.rҁ>uX[f6]'>>켨u7rP rwdE$Tk(IhR#z{Gilքdb/#h5{aLGrxu|ra_WH@-eFSJrQQO%V)i e%0|.`GʬsSQ3RcDJ0͊l)OݴiQR]Nl nTFRQ I=( tt)SIwcUz 7 R(ϖ=k޺޺ 7n>zfb˺|u$Ň6,x%sZܶ}m^[#/bsӋ (`E_It:YdE6op[֍El8/SS|8TY.'N` O">xX fvXQ"br=W@t]U1v_f'gq"nfh}Sc'p🅠,`׾)|i ޸#f{wN 8tb0 w*̞qdvf"d ,u@YXNbUkfp2CmN{e-juu8pZYORS!&116/Μσfv-4k?=Ҭ]O-p f`S?e"[k+H; $ϫbh^ v RwhA9@Ew(P0uƕ-ex9_ NZIctp鿦))WWm7ĦLqx1[$I(,`X9i+;bڙrL뮎Rov4yoX?Vvw$.0v4`&B:U%c >k]:&Q{hӣ#'A#o¤yH$IYOL$"CvCNy]C實iOWY"#?GjP3"*3F6w ]z"?:^y=vzKm|D2aˣd&ʳ7鲠2_ż- @vtl [KlgrrS D'SD  plxtVkRC6E[ˤ9oe=32~z#f= l殩;qAWAaњeL_?9߳yp]\8/IEcUr4 uD^p!'J[JvE#SGAD6Ս/5Ⅷ7DUN{~+;=^9SU9D )8ʈ,gGv83L(+Hu!-fPr&ZrSQǺI*YؤsUE޽ 2;]ھҘ:v&[>' N=@ ?9WjP hL6`-@z`j[yUo_rRȟ'/4Xpd3+Df)Jn I}5OG!IENDB`weboob-1.2/modules/americanexpress/module.py000066400000000000000000000037731303450110500213330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AmericanExpressBrowser __all__ = ['AmericanExpressModule'] class AmericanExpressModule(Module, CapBank): NAME = 'americanexpress' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'American Express' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Code utilisateur', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = AmericanExpressBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): transactions = list(self.browser.get_history(account)) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions weboob-1.2/modules/americanexpress/pages.py000066400000000000000000000162071303450110500211410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime import re from weboob.browser.pages import HTMLPage, LoggedPage, PartialHTMLPage from weboob.browser.filters.standard import CleanText, CleanDecimal from weboob.capabilities.bank import Account from weboob.capabilities import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction as Transaction from weboob.tools.date import ChaoticDateGuesser class WrongLoginPage(HTMLPage): pass class AccountSuspendedPage(HTMLPage): pass class LoginPage(HTMLPage): def login(self, username, password): form = self.get_form(name='ssoform') form['UserID'] = username form['USERID'] = username form['Password'] = password form['PWD'] = password form.submit() class AccountsPage(LoggedPage, PartialHTMLPage): def get_account(self): for div in self.doc.xpath('.//div[@id="card-details"]'): a = Account() a.id = CleanText().filter(div.xpath('.//span[@class="acc-num"]')) a.label = CleanText().filter(div.xpath('.//span[@class="card-desc"]')) if "carte" in a.label.lower(): a.type = Account.TYPE_CARD balance = CleanText().filter(div.xpath('.//span[@class="balance-data"]')) if balance in (u'Indisponible', u'Indisponible Facturation en cours', ''): a.balance = NotAvailable else: a.currency = a.get_currency(balance) a.balance = - abs(CleanDecimal(replace_dots=a.currency == 'EUR').filter(balance)) # Cancel card don't have a link to watch history link = self.doc.xpath('.//div[@class="wide-bar"]/h3/a') if len(link) == 1: a._link = link[0].attrib['href'] else: a._link = None return a def get_idx_list(self): fetched = False for div in self.doc.xpath('//div[@id="card-list"]//div[has-class("card-details")]'): _id = div.attrib['id'] idx = re.match(r'card-(\d+)-detail', _id).group(1) message = CleanText('.//div[has-class("messages")]')(div).lower() cancelled = ('annul' in message or 'cancel' in message) yield idx, cancelled fetched = True if fetched: return for div in self.doc.xpath('//div[@id="card-detail"]'): idx = div.xpath('//span[@id="cardSortedIndex"]/@data')[0] message = CleanText('.//div[has-class("messages")]')(div).lower() cancelled = ('annul' in message or 'cancel' in message) yield idx, cancelled return def get_session(self): return self.doc.xpath('//form[@id="j-session-form"]//input[@name="Hidden"]/@value') class TransactionsPage(LoggedPage, HTMLPage): def is_last(self): current = False for option in self.doc.xpath('//select[@id="viewPeriod"]/option'): if 'selected' in option.attrib: current = True elif current: return False return True def get_end_debit_date(self): for option in self.doc.xpath('//select[@id="viewPeriod"]/option'): if 'selected' in option.attrib: m = re.search('(\d+) ([\w\.]+) (\d{4})$', option.text.strip(), re.UNICODE) if m: return datetime.date(int(m.group(3)), self.MONTHS.index(m.group(2).rstrip('.')) + 1, int(m.group(1))) def get_beginning_debit_date(self): for option in self.doc.xpath('//select[@id="viewPeriod"]/option'): if 'selected' in option.attrib: m = re.search('^(\d+) ([\w\.]+) (\d{4})', option.text.strip(), re.UNICODE) if m: return datetime.date(int(m.group(3)), self.MONTHS.index(m.group(2).rstrip('.')) + 1, int(m.group(1))) return datetime.date.today() COL_DATE = 0 COL_TEXT = 1 COL_CREDIT = -2 COL_DEBIT = -1 FR_MONTHS = ['janv', u'févr', u'mars', u'avr', u'mai', u'juin', u'juil', u'août', u'sept', u'oct', u'nov', u'déc'] US_MONTHS = ['Jan', u'Feb', u'Mar', u'Apr', u'May', u'Jun', u'Jul', u'Aug', u'Sep', u'Oct', u'Nov', u'Dec'] def get_history(self, currency): self.MONTHS = self.FR_MONTHS if currency == 'EUR' else self.US_MONTHS #checking if the card is still valid if self.doc.xpath('//div[@id="errorbox"]'): return #adding a time delta because amex have hard time to put the date in a good interval beginning_date = self.get_beginning_debit_date() - datetime.timedelta(days=360) end_date = self.get_end_debit_date() guesser = ChaoticDateGuesser(beginning_date, end_date) for tr in reversed(self.doc.xpath('//div[@id="txnsSection"]//tr[@class="tableStandardText"]')): cols = tr.findall('td') t = Transaction() day, month = CleanText().filter(cols[self.COL_DATE]).split(' ', 1) day = int(day) month = self.MONTHS.index(month.rstrip('.')) + 1 date = guesser.guess_date(day, month) rdate = None try: detail = cols[self.COL_TEXT].xpath('./div[has-class("hiddenROC")]')[0] except IndexError: pass else: m = re.search(r' (\d{2} \D{3,4})', (' '.join([txt.strip() for txt in detail.itertext()])).strip()) if m: rday, rmonth = m.group(1).strip().split(' ') rday = int(rday) rmonth = self.MONTHS.index(rmonth.rstrip('.')) + 1 rdate = guesser.guess_date(rday, rmonth) detail.drop_tree() raw = (' '.join([txt.strip() for txt in cols[self.COL_TEXT].itertext()])).strip() credit = CleanText().filter(cols[self.COL_CREDIT]) debit = CleanText().filter(cols[self.COL_DEBIT]) t.date = date t.rdate = rdate or date t.raw = re.sub(r'[ ]+', ' ', raw) t.label = re.sub('(.*?)( \d+)? .*', r'\1', raw).strip() t.amount = CleanDecimal(replace_dots=currency == 'EUR').filter(credit or debit) * (1 if credit else -1) if t.amount > 0: t.type = t.TYPE_ORDER else: t.type = t.TYPE_CARD yield t weboob-1.2/modules/americanexpress/test.py000066400000000000000000000020111303450110500210050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AmericanExpressTest(BackendTest): MODULE = 'americanexpress' def test_americanexpress(self): l = list(self.backend.iter_accounts()) a = l[0] list(self.backend.iter_history(a)) list(self.backend.iter_coming(a)) weboob-1.2/modules/amundi/000077500000000000000000000000001303450110500155465ustar00rootroot00000000000000weboob-1.2/modules/amundi/__init__.py000066400000000000000000000014331303450110500176600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AmundiModule __all__ = ['AmundiModule'] weboob-1.2/modules/amundi/browser.py000066400000000000000000000052131303450110500176040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .pages import LoginPage, AccountsPage, AccountHistoryPage from weboob.browser import URL, LoginBrowser, need_login from weboob.tools.json import json from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.exceptions import ClientError class AmundiBrowser(LoginBrowser): TIMEOUT = 120.0 login = URL('/psf/authenticate', LoginPage) authorize = URL('/psf/authorize', LoginPage) accounts = URL('/psf/api/individu/positionFonds\?flagUrlFicheFonds=true&inclurePositionVide=false', AccountsPage) account_history = URL('/psf/api/individu/operations\?valeurExterne=false&filtreStatutModeExclusion=false&statut=CPTA', AccountHistoryPage) def __init__(self, website, *args, **kwargs): self.BASEURL = website super(AmundiBrowser, self).__init__(*args, **kwargs) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) try: self.login.go(data=json.dumps({'username' : self.username, 'password' : self.password}), \ headers={'Content-Type': 'application/json;charset=UTF-8'}) self.token = self.authorize.go().get_token() except ClientError: raise BrowserIncorrectPassword() @need_login def iter_accounts(self): return self.accounts.go(headers={'X-noee-authorization': ('noeprd %s' % self.token)}).iter_accounts() @need_login def iter_investments(self, account): return self.accounts.go(headers={'X-noee-authorization': ('noeprd %s' % self.token)})\ .iter_investments(account_id=account.id) @need_login def iter_history(self, account): return self.account_history.go(headers={'X-noee-authorization': ('noeprd %s' % self.token)}).iter_history(account=account) weboob-1.2/modules/amundi/favicon.png000066400000000000000000000032161303450110500177030ustar00rootroot00000000000000PNG  IHDR@@ pHYs  tIME  (%hiTXtCommentCreated with GIMPd.ePLTEUff UUqUL*'$m U3 /  ʔ <.  88F" tvbYG'HS8O|60 .gJ,3$y3w6 r;o@mAhBiA%%U]fC\TdE[V"aH]U^O%\K\N%ZKKyZK "YQ!! 04VTUWUXUZ$L4SZ';I'Of=Ny*>*FMi)9GLm+,Kw@KpKq,KpBFJs/Iu.1GG~G/EG~G123>??EECEA>@58CBBC9::@9CC:;?A;<AB=ABB>?AAAA>?@AAAAAAAAAA0B"ntRNS  !"$%%((+,-159:;?@EIJLMOQSSSTVWX\]bbfloqrvyz~bKGD(0^IDATXc`#86LO_ LWi.;[{w6gmLnϩC"H5(Zzw"wt˪[j\k7lk``p;n(wLCzdٽhzMw+,?b;P  sD6 U| \vH;rՀ:!w*yL4OdxLfb*sne0h-j@;c>MW a7f ȁ.2 컓w\B512]`1YwP `Zn2Kۭ `)D,B p>|.=7Q `p v b!}JS@3ɣɵ. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import AmundiBrowser __all__ = ['AmundiModule'] class AmundiModule(Module, CapBank): NAME = 'amundi' DESCRIPTION = u'amundi website' MAINTAINER = u'James GALT' EMAIL = 'james.galt.bi@gmail.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', regexp='\d+', masked=False), ValueBackendPassword('password', label=u"Mot de passe", regexp='\d+'), Value('website', label='Type de compte', default='ee', choices={'ee': 'Amundi Epargne Entreprise', 'tc': 'Amundi Tenue de Compte'})) BROWSER = AmundiBrowser def create_default_browser(self): w = {'ee': 'https://www.amundi-ee.com', 'tc': 'https://epargnants.amundi-tc.com'} return self.create_browser(w[self.config['website'].get()], self.config['login'].get(), self.config['password'].get()) def get_account(self, id): """ Get an account from its ID. :param id: ID of the account :type id: :class:`str` :rtype: :class:`Account` :raises: :class:`AccountNotFound` """ return find_object(self.iter_accounts(), id=id, error=AccountNotFound) def iter_accounts(self): """ Iter accounts. :rtype: iter[:class:`Account`] """ return self.browser.iter_accounts() def iter_investment(self, account): """ Iter investment of a market account :param account: account to get investments :type account: :class:`Account` :rtype: iter[:class:`Investment`] :raises: :class:`AccountNotFound` """ return self.browser.iter_investments(account) def iter_history(self, account): """ Iter history of transactions on a specific account. :param account: account to get history :type account: :class:`Account` :rtype: iter[:class:`Transaction`] :raises: :class:`AccountNotFound` """ return self.browser.iter_history(account) weboob-1.2/modules/amundi/pages.py000066400000000000000000000126701303450110500172250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 James GALT # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from decimal import Decimal from weboob.browser.elements import ItemElement, method, DictElement from weboob.browser.filters.standard import CleanDecimal, Date, Field, CleanText, Env from weboob.browser.filters.json import Dict from weboob.browser.pages import LoggedPage, JsonPage from weboob.capabilities.bank import Account, Investment, Transaction from weboob.capabilities.base import NotAvailable class LoginPage(JsonPage): def get_token(self): return Dict('token')(self.doc) class AccountsPage(LoggedPage, JsonPage): ACCOUNT_TYPES = {'PEE': Account.TYPE_PEE, 'PEG': Account.TYPE_PEE, 'PERCO': Account.TYPE_PERCO, 'RSP': Account.TYPE_RSP } @method class iter_accounts(DictElement): item_xpath = "listPositionsSalarieFondsDto/*/positionsSalarieDispositifDto" class item(ItemElement): klass = Account obj_id = CleanText(Dict('codeDispositif')) obj_balance = CleanDecimal(Dict('mtBrut')) obj_number = Field('id') obj_currency = u"EUR" def obj_type(self): return self.page.ACCOUNT_TYPES.get(Dict('typeDispositif')(self), Account.TYPE_LIFE_INSURANCE) def obj_label(self): try: return Dict('libelleDispositif')(self).encode('iso-8859-2').decode('utf8') except (UnicodeEncodeError, UnicodeDecodeError): try: return Dict('libelleDispositif')(self).encode('latin1').decode('utf8') except UnicodeDecodeError: return Dict('libelleDispositif')(self) @method class iter_investments(DictElement): def find_elements(self): for psds in Dict('listPositionsSalarieFondsDto')(self): for psd in psds.get('positionsSalarieDispositifDto'): if psd.get('codeDispositif') == Env('account_id')(self): return psd.get('positionsSalarieFondsDto') return {} class item(ItemElement): klass = Investment obj_label = Dict('libelleFonds') obj_unitvalue = Dict('vl') & CleanDecimal obj_quantity = Dict('nbParts') & CleanDecimal obj_valuation = Dict('mtBrut') & CleanDecimal obj_code = Dict('codeIsin', default=NotAvailable) obj_vdate = Date(Dict('dtVl')) obj_diff = Dict('mtPMV') & CleanDecimal class AccountHistoryPage(LoggedPage, JsonPage): def belongs(self, instructions, account): for ins in instructions: if 'nomDispositif' in ins and 'codeDispositif' in ins and '%s%s' % (ins['nomDispositif'], ins['codeDispositif']) == \ '%s%s' % (account.label, account.id): return True return False def get_amount(self, instructions, account): amount = 0 for ins in instructions: if 'nomDispositif' in ins and 'montantNet' in ins and 'codeDispositif' in ins and '%s%s' % (ins['nomDispositif'], ins['codeDispositif']) == \ '%s%s' % (account.label, account.id): amount += ins['montantNet'] return Decimal(amount) def iter_history(self, account): for hist in self.doc['operationsIndividuelles']: if len(hist['instructions']) > 0: if self.belongs(hist['instructions'], account): tr = Transaction() tr.amount = self.get_amount(hist['instructions'], account) tr.rdate = datetime.strptime(hist['dateComptabilisation'].split('T')[0], '%Y-%m-%d') tr.date = tr.rdate tr.label = hist['libelleOperation'] if 'libelleOperation' in hist else hist['libelleCommunication'] tr.type = Transaction.TYPE_UNKNOWN # Bypassed because we don't have the ISIN code # tr.investments = [] # for ins in hist['instructions']: # inv = Investment() # inv.code = NotAvailable # inv.label = ins['nomFonds'] # inv.description = ' '.join([ins['type'], ins['nomDispositif']]) # inv.vdate = datetime.strptime(ins.get('dateVlReel', ins.get('dateVlExecution')).split('T')[ # 0], '%Y-%m-%d') # inv.valuation = Decimal(ins['montantNet']) # inv.quantity = Decimal(ins['nombreDeParts']) # inv.unitprice = inv.unitvalue = Decimal(ins['vlReel']) # tr.investments.append(inv) yield tr weboob-1.2/modules/apec/000077500000000000000000000000001303450110500152015ustar00rootroot00000000000000weboob-1.2/modules/apec/__init__.py000066400000000000000000000014261303450110500173150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ApecModule __all__ = ['ApecModule'] weboob-1.2/modules/apec/browser.py000066400000000000000000000130341303450110500172370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.profiles import Profile from weboob.browser import PagesBrowser, URL from .pages import IdsPage, OffrePage __all__ = ['ApecBrowser'] class JsonProfile(Profile): def setup_session(self, session): session.headers["Content-Type"] = "application/json" class ApecBrowser(PagesBrowser): BASEURL = 'https://cadres.apec.fr' PROFILE = JsonProfile() start = 0 json_count = URL('/cms/webservices/rechercheOffre/count', IdsPage) json_ids = URL('/cms/webservices/rechercheOffre/ids', IdsPage) json_offre = URL('/cms/webservices/offre/public\?numeroOffre=(?P<_id>.*)', OffrePage) def create_parameters(self, pattern='', fonctions='[]', lieux='[]', secteursActivite='[]', typesContrat='[]', typesConvention='[]', niveauxExperience='[]', salaire_min='', salaire_max='', date_publication='', start=0, range=20): if date_publication: date_publication = ',"anciennetePublication":%s' % (date_publication) if salaire_max: salaire_max = ',"salaireMaximum":%s' % (salaire_max) if salaire_min: salaire_min = ',"salaireMinimum":%s' % (salaire_min) return '{"activeFiltre":true,"motsCles":"%s","fonctions":%s,"lieux":%s,"pointGeolocDeReference":{},"secteursActivite":%s,"typesContrat":%s,"typesConvention":%s,"niveauxExperience":%s%s%s%s,"sorts":[{"type":"SCORE","direction":"DESCENDING"}],"pagination":{"startIndex":%s,"range":%s},"typeClient":"CADRE"}' % (pattern, fonctions, lieux, secteursActivite, typesContrat, typesConvention, niveauxExperience, salaire_min, salaire_max, date_publication, start, range) def search_job(self, pattern=None): data = self.create_parameters(pattern=pattern).encode('utf-8') return self.get_job_adverts(data, pattern=pattern) def get_job_adverts(self, data, pattern='', lieux='', fonctions='', secteursActivite='', salaire_min='', salaire_max='', typesContrat='', date_publication='', niveauxExperience='', typesConvention=''): count = self.json_count.go(data=data).get_adverts_number() self.start = 0 if count: ids = self.json_ids.go(data=data).iter_job_adverts(pattern=pattern, fonctions='[%s]' % fonctions, lieux='[%s]' % lieux, secteursActivite='[%s]' % secteursActivite, typesContrat='[%s]' % typesContrat, niveauxExperience='[%s]' % niveauxExperience, typesConvention='[%s]' % typesConvention, salaire_min=salaire_min, salaire_max=salaire_max, date_publication=date_publication, start=self.start, count=count, range=20) for _id in ids: yield self.json_offre.go(_id=_id.id).get_job_advert() def get_job_advert(self, _id, advert=None): return self.json_offre.go(_id=_id).get_job_advert(obj=advert) def advanced_search_job(self, region='', fonction='', secteur='', salaire='', contrat='', limit_date='', level=''): salaire_max = '' salaire_min = '' if salaire: s = salaire.split('|') salaire_max = s[1] salaire_min = s[0] data = self.create_parameters(fonctions='[%s]' % fonction, lieux='[%s]' % region, secteursActivite='[%s]' % secteur, typesContrat='[%s]' % contrat, niveauxExperience='[%s]' % level, salaire_min=salaire_min, salaire_max=salaire_max, date_publication=limit_date) return self.get_job_adverts(data, fonctions=fonction, lieux=region, secteursActivite=secteur, typesContrat=contrat, niveauxExperience=level, salaire_min=salaire_min, salaire_max=salaire_max, date_publication=limit_date) weboob-1.2/modules/apec/favicon.png000066400000000000000000000172231303450110500173410ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYsodtIME}r IDATxy|]uLwhkeWNupLz&)*'N^$:J_&PI^2pnC 1J36 ϲ5ǹW &Li>sw^komR"ƍRm*?DZ8rd|J)F\T7ŀuy6H.D۽w48nMl<3o#nnw߭߈q97?A b:KY "ry}(G\ޞ xUx#NOuIYq"Jk~˸hzwF†na7 j  g0$(P\@cG7E:2o }Ê=P$R`#1JuiS (m Ŀ?@; WyD]\;Dttm uz_VDg=9rpM cdEyI,2ƄDJyƤ#)"u@ 1& йHP_;`}\d"&W8=$3fQ8iB!`tN:AB0b:Cʠ ffE&vtɦH-ssu$sЩX_5utFLtW"q4֜ԅU Fգiuf_9:\'8u'xmm h@VDn Ft 0w!(BQT~n5DA.J ck븶?3GÎXN.LybUň(Z)hx%46\Ēl_@u,nC]@ 'f~MD\lk-MpwOe-d5hUSrg))1Ι8µ1B[T(fkTKM\vh3@l}lڣf~ MjU1P^r]! A4Ab!@$߮8 JltIb'jlųW@┭}m غ $ Dhi5U[NCi1(Q¶3VюX~ǎ(jŪEj"U]q0b[h'$&"p ]]\tcDD FK!};w,}S̔'(r [h11+Ĉ1vA[2"`:TFHY8 !ױ/":" +D&XfS+iut[cI_ǩ/湶3q_uѓ "?vz`l!tT(0S m2TāJ(gȯIsV{;f^u*TΠc%πnD$m:+7j{r1w묹%Mә?"4 -&m"9SU`枀tw: GB=ȗ#?܍rpp>qSП<ʚҗA#LݽT~1 O*d6ŊnHΘr_EMgc@y:7Y]PMf JD_bĵHEXC\tan^Z[B+d{kBZBt5"O!R)cւKY-!dMhLd_(ogL\7D "J;32OEg>IHu:taU3}A Pc0Э;&|D^H3}r$;Oq$LΌ|·z  ,~KNjôo ,5.1YA}~ 5sGaav]-\ټ4Ǐ_.♞+ Yԭ<ŕ34#|eJ-]٤:Xf7Bow̺u Xwjb6X=GH-v0yR6o2GG&! F:\ EiN0tpAFp;0#hÔa^VVr$ڄ!Bfv,sQC>vuc/HP*dR؍8鱰=:5n/O'"J =q+Rm`_/U$,&U+fT g0غ-:lD,0VqB AFN 0{0`IWC*' \\1Y=$#ǰ=T~9AD>qnYS޻1pbhbbi1IT5 Xorϱ$Fa&𮵕%}[e-NKkd՟nz)o3f{ >y`TD*!F=| H20O5\Y~)P6M!HaDL/]MuG=Wmyu`!-ޓ1ZW(eG5hɤ(Lܔ*e1m{QS'd܃|sG\!b>gR]Qh~:T|k|ėn&&@h욬h&9?sν?8h6z• N?Km=:ler1[@Kzi~:0D _&tyo}ڈÈRJ*ױg{]52̣L}E/;Nj4@ɫp l̠ץ5a@?EGa~1cf@ǎ h$z,;^ߏi>u}ꓱ b%3(fVW -lYh8-ZvI=.=j+R/_KpIs?=LEI9"~дvu GXXt5ָR_P*/R_бvf,*H@ j'Y"hGvJ '[Jí;=l0SU&k,~0aգ+\EMdkuHקv?Yߣ Nbjm͠ݚ&vuk_~mڃ!gpoW+6oFdu|rS庾wײb &O0O=R#89Ã\g'3_/%ZZX.#,ie\>_N)qC#̋K|rbn1+H׿ ZegR$49U4ܳb9q(MH7XKMou2ҕRDЩc=Juq%Y/̼I ]|( 8tL/p1?9Tʦ'rl^sZ&03ao&߿rexK K~,sߟ-sJ'/}i+1C&OC:I67P_PMo&welm*z93!^K;ǁUr Q~S9#41@\(Ui0K8W'?ʗ1` PTcpEiF4[5~[5ԧoaS?ͷm+q7Bha,6 Ъxn;Q8 dw B2E$#"y,+J81RI5azVйHz!Eܲf"JSn#?iihXCWRDJ5gMe~ucT ?|kQߟˊ>ƜngX RMT"2\ypق9;1тM먻:jE(9FY޹͏B%[ԗNsc j8p x5'40ѕ[iƑf&ZcGx u]EZ: :o){QY myy`fany 7fFi$'ofs=voHNO0xZblÁs{ rq8yVjj< 6˓HJ ʊN [Ѽ /`?Y^Q5'sRVˮV'W<ǎ^L&SCΐ=uo曷WxgB(F(10Y8v -d0\̽Z 7&9XKN,*>~q57Yȩ)&0k:~.,ʧM?ZK}ǶqfTtm(E)b6^Ï3J/O{$;~t秷q̖BccŚ))@Cy9%O >R~y*b] G&;{%H=P_"md;As;i0`kBuKxἯ蓸}(!Y8aOg ǟC{hE \T%3f cNJ9N0]7D%y}|98(IFuǒb_Dizv*TwlIҹ'bF㽷h4J MP8- 7`z]jCVAn&p:]h)|)\a² ڳ@IG2;8—XgEա5@f*ӱySQyG}^At%%TH+(h肘h |-AS"B{;f4J( ̤:B򄵫cPX- HфS \Y:OEssS&ah )Bh7aňSԇNqL0Ƭ ձ` 1GL\[T`0 (S[S=qg`؂ pV/έrmͻ4+~q"[**kbA6y[.`rQ'y7мbq8;1ݨXl~$ڼ-q}Q-$1Ϸ,U@?a뢍?镇X϶NH)*Dߴ2Πicx<fC7iQqxoaƯ=}"e๟x6@{{_]wo9#lTw'xEQ?–.쭞|7O??tPR8_;I\{IENDB`weboob-1.2/modules/apec/job.py000066400000000000000000000017561303450110500163360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . APEC_CONTRATS = { ' ': u'-- Indifférent --', '101888': u'CDI', '101887': u'CDD', '101889': u'Interim', } APEC_EXPERIENCE = { '101882': u'Tous niveaux d\'expérience', '101881': u'Débutant', '101883': u'Expérimenté', } weboob-1.2/modules/apec/module.py000066400000000000000000000364461303450110500170550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.job import BaseJobAdvert from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.job import CapJob from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from .browser import ApecBrowser from .job import APEC_CONTRATS, APEC_EXPERIENCE __all__ = ['ApecModule'] class ApecModule(Module, CapJob): NAME = 'apec' DESCRIPTION = u'apec website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' BROWSER = ApecBrowser places_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '001|99700': u'UE Hors France', '002|99126': u'..Grèce', '003|99132': u'..Royaume Uni', '004|99134': u'..Espagne', '005|99136': u'..Irlande', '006|99139': u'..Portugal', '007|99254': u'..Chypre', '008|99127': u'..Italie', '009|99131': u'..Belgique', '010|99135': u'..Pays Bas', '011|99137': u'..Luxembourg', '012|99144': u'..Malte', '013|99145': u'..Slovénie', '014|99101': u'..Danemark', '015|99104': u'..Suède', '016|99105': u'..Finlande', '017|99106': u'..Estonie', '018|99107': u'..Lettonie', '019|99108': u'..Lituanie', '020|99109': u'..Allemagne', '021|99110': u'..Autriche', '022|99111': u'..Bulgarie', '023|99112': u'..Hongrie', '024|99114': u'..Roumanie', '025|99116': u'..République Tchèque', '026|99117': u'..Slovaquie', '027|99119': u'..Croatie', '028|99122': u'..Pologne', '029|799': u'France', '030|711': u'..Ile-de-France', '031|75': u'....Paris', '032|77': u'....Seine-et-Marne', '033|78': u'....Yvelines', '034|91': u'....Essonne', '035|92': u'....Hauts-de-Seine', '036|93': u'....Seine-Saint-Denis', '037|94': u'....Val-de-Marne', '038|95': u'....Val-d\'Oise', '039|703': u'..Basse-Normandie', '040|14': u'....Calvados', '041|50': u'....Manche', '042|61': u'....Orne', '043|705': u'..Bretagne', '044|22': u'....Côtes d\'Armor', '045|29': u'....Finistère', '046|35': u'....Ille-et-Vilaine', '047|56': u'....Morbihan', '048|706': u'..Centre', '049|18': u'....Cher', '050|28': u'....Eure-et-Loir', '051|36': u'....Indre', '052|37': u'....Indre-et-Loire', '053|41': u'....Loir-et-Cher', '054|45': u'....Loiret', '055|710': u'..Haute-Normandie', '056|27': u'....Eure', '057|76': u'....Seine-Maritime', '058|717': u'..Pays de La Loire', '059|44': u'....Loire-Atlantique', '060|49': u'....Maine-et-Loire', '061|53': u'....Mayenne', '062|72': u'....Sarthe', '063|85': u'....Vendée', '064|700': u'..Alsace', '065|67': u'....Bas-Rhin', '066|68': u'....Haut-Rhin', '067|704': u'..Bourgogne', '068|21': u'....Côte d\'Or', '069|58': u'....Nièvre', '070|71': u'....Saône-et-Loire', '071|89': u'....Yonne', '072|707': u'..Champagne', '073|8': u'....Ardennes', '074|10': u'....Aube', '075|51': u'....Marne', '076|52': u'....Haute-Marne', '077|709': u'..Franche-Comté', '078|25': u'....Doubs', '079|39': u'....Jura', '080|70': u'....Haute-Saône', '081|90': u'....Territoire de Belfort', '082|714': u'..Lorraine', '083|54': u'....Meurthe-et-Moselle', '084|55': u'....Meuse', '085|57': u'....Moselle', '086|88': u'....Vosges', '087|716': u'..Nord-Pas-de-Calais', '088|59': u'....Nord', '089|62': u'....Pas-de-Calais', '090|718': u'..Picardie', '091|2': u'....Aisne', '092|60': u'....Oise', '093|80': u'....Somme', '094|20': u'..Corse', '095|750': u'....Corse du Sud', '096|751': u'....Haute-Corse', '097|702': u'..Auvergne', '098|3': u'....Allier', '099|15': u'....Cantal', '100|43': u'....Haute-Loire', '101|63': u'....Puy-de-Dôme', '102|720': u'..PACA', '103|4': u'....Alpes-de-Haute-Provence', '104|5': u'....Hautes-Alpes', '105|6': u'....Alpes-Maritimes', '106|13': u'....Bouches-du-Rhône', '107|83': u'....Var', '108|84': u'....Vaucluse', '109|721': u'..Rhône-Alpes', '110|1': u'....Ain', '111|7': u'....Ardèche', '112|26': u'....Drôme', '113|38': u'....Isère', '114|42': u'....Loire', '115|69': u'....Rhône', '116|73': u'....Savoie', '117|74': u'....Haute-Savoie', '118|701': u'..Aquitaine', '119|24': u'....Dordogne', '120|33': u'....Gironde', '121|40': u'....Landes', '122|47': u'....Lot-et-Garonne', '123|64': u'....Pyrénées-Atlantiques', '124|712': u'..Languedoc-Roussillon', '125|11': u'....Aude', '126|30': u'....Gard', '127|34': u'....Hérault', '128|48': u'....Lozère', '129|66': u'....Pyrénées-Orientales', '130|713': u'..Limousin', '131|19': u'....Corrèze', '132|23': u'....Creuse', '133|87': u'....Haute-Vienne', '134|715': u'..Midi-Pyrénées', '135|9': u'....Ariège', '136|12': u'....Aveyron', '137|31': u'....Haute-Garonne', '138|32': u'....Gers', '139|46': u'....Lot', '140|65': u'....Hautes-Pyrénées', '141|81': u'....Tarn', '142|82': u'....Tarn-et-Garonne', '143|719': u'..Poitou-Charentes', '144|16': u'....Charente', '145|17': u'....Charente-Maritime', '146|79': u'....Deux-Sèvres', '147|86': u'....Vienne', '148|99712': u'..France Outre-Mer', '149|99519': u'....Terres Australes et Antarctiques Françaises', '150|97100': u'....Guadeloupe', '151|97200': u'....Martinique', '152|97300': u'....Guyane', '153|97400': u'....La Réunion', '154|97500': u'....Saint-Pierre-et-Miquelon', '155|97600': u'....Mayotte', '156|98300': u'....Polynésie Française', '157|98600': u'....Wallis et Futuna', '158|98800': u'....Nouvelle Calédonie', '159|97800': u'....Saint-Martin', '160|97700': u'....Saint-Barthélémy', '161|102099': u'International', '162|99715': u'..Afrique', '163|99716': u'..Asie', '164|99700': u'..UE Hors France', '165|99701': u'..Europe Hors UE', '166|99702': u'..Amérique du Nord', '167|99711': u'..Océanie', '168|99714': u'..Amérique Latine', }.iteritems())]) fonction_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '00|': u'-- Indifférent --', '01|101828': u'Commercial, Marketing', '02|101782': u'.....Administration des ventes et SAV', '03|101783': u'.....Chargé d\'affaires, technico-commercial', '04|101784': u'.....Commercial', '05|101785': u'.....Commerce international', '06|101786': u'.....Direction commerciale et marketing', '07|101787': u'.....Direction régionale et d\'agence', '08|101788': u'.....Marketing', '09|101789': u'.....Ventes en magasin', '10|101829': u'Communication, Création', '11|101790': u'.....Communication', '12|101791': u'.....Création', '13|101792': u'.....Documentation, rédaction technique', '14|101793': u'.....Journalisme, édition', '15|101830': u'Direction d\'entreprise', '16|101794': u'.....Adjoint, conseil de direction', '17|101795': u'.....Direction générale', '18|101831': u'Etudes, Recherche et Développement', '19|101796': u'.....Conception, recherche', '20|101797': u'.....Direction recherche et développement', '21|101798': u'.....Etudes socio-économiques', '22|101799': u'.....Projets scientifiques et techniques', '23|101800': u'.....Test, essai, validation, expertise', '24|101832': u'Gestion, Finance, Administration', '25|101801': u'.....Administration, gestion, organisation', '26|101802': u'.....Comptabilité', '27|101803': u'.....Contrôle de gestion, audit', '28|101804': u'.....Direction gestion, finance', '29|101805': u'.....Droit, fiscalité', '30|101806': u'.....Finance, trésorerie', '31|101833': u'Informatique', '32|101807': u'.....Direction informatique', '33|101808': u'.....Exploitation, maintenance informatique', '34|101809': u'.....Informatique de gestion', '35|101810': u'.....Informatique industrielle', '36|101811': u'.....Informatique web, sites et portails Internet', '37|101812': u'.....Maîtrise d\'ouvrage et fonctionnel', '38|101813': u'.....Système, réseaux, données', '39|101834': u'Production Industrielle, Travaux, Chantiers', '40|101814': u'.....Cadres de chantier', '41|101815': u'.....Cadres de production industrielle', '42|101816': u'.....Direction d\'unité industrielle', '43|101835': u'Ressources Humaines', '44|101817': u'.....Administration des RH', '45|101818': u'.....Développement des RH', '46|101819': u'.....Direction des ressources humaines', '47|101820': u'.....Formation initiale et continue', '48|101836': u'Sanitaire, Social, Culture', '49|101821': u'.....Activités sanitaires, sociales et culturelles', '50|101837': u'Services Techniques', '51|101822': u'.....Achats', '52|101823': u'.....Direction des services techniques', '53|101824': u'.....Logistique', '54|101825': u'.....Maintenance, sécurité', '55|101826': u'.....Process, méthodes', '56|101827': u'.....Qualité', }.iteritems())]) secteur_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'-- Indifférent --', '101752': u'Activités des organisations associatives et administration publique', '101753': u'Activités informatiques', '101754': u'Activités juridiques et comptables', '101755': u'Agroalimentaire', '101756': u'Automobile, aéronautique et autres matériels de transport', '101757': u'Banque et Assurances', '101758': u'Bois - Papier - Imprimerie', '101759': u'Chimie - Caoutchouc - Plastique', '101760': u'Commerce interentreprises', '101761': u'Communication et médias', '101762': u'Conseil et gestion des entreprises', '101763': u'Construction', '101764': u'Distribution généraliste et spécialisée', '101765': u'Energies - Eau', '101766': u'Equipements électriques et électroniques', '101767': u'Formation initiale et continue', '101768': u'Gestion des déchets', '101769': u'Hôtellerie - Restauration - Loisirs', '101770': u'Immobilier', '101771': u'Industrie pharmaceutique', '101772': u'Ingénierie - R et D', '101773': u'Intermédiaires du recrutement', '101774': u'Mécanique - Métallurgie', '101775': u'Meuble, Textile et autres industries manufacturières', '101776': u'Santé - action sociale', '101777': u'Services divers aux entreprises', '101778': u'Télécommunications', '101779': u'Transports et logistique', }.iteritems())]) type_contrat_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted(APEC_CONTRATS.iteritems())]) salary_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'-- Indifférent --', '0|35': u'Moins de 35 K€', '35|50': u'Entre 35 et 49 K€', '50|70': u'Entre 50 et 69 K€', '70|90': u'Entre 70 et 90 K€', '90|1000': u'Plus de 90 K€', }.iteritems())]) date_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'-- Indifférent --', '101850': u'Aujourd\'hui', '101851': u'Les 7 derniers jours', '101852': u'Les 30 derniers jours', '101853': u'Toutes les offres', }.iteritems())]) level_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted(APEC_EXPERIENCE.iteritems())]) CONFIG = BackendConfig(Value('place', label=u'Lieu', choices=places_choices, default=''), Value('fonction', label=u'Fonction', choices=fonction_choices, default=''), Value('secteur', label=u'Secteur', choices=secteur_choices, default=''), Value('contrat', label=u'Contrat', choices=type_contrat_choices, default=''), Value('salaire', label=u'Salaire', choices=salary_choices, default=''), Value('limit_date', label=u'Date', choices=date_choices, default=''), Value('level', label=u'Expérience', choices=level_choices, default='')) def search_job(self, pattern=None): for job_advert in self.browser.search_job(pattern=pattern): yield self.fill_obj(job_advert) def decode_choice(self, choice): splitted_choice = choice.split('|') if len(splitted_choice) == 2: return splitted_choice[1] else: return '' def advanced_search_job(self): for job_advert in self.browser.advanced_search_job(region=self.decode_choice(self.config['place'].get()), fonction=self.decode_choice(self.config['fonction'].get()), secteur=self.config['secteur'].get(), salaire=self.config['salaire'].get(), contrat=self.config['contrat'].get(), limit_date=self.config['limit_date'].get(), level=self.config['level'].get()): yield self.fill_obj(job_advert) def get_job_advert(self, _id, advert=None): job_advert = self.browser.get_job_advert(_id, advert) return self.fill_obj(job_advert) def fill_obj(self, advert, fields=None): if advert.contract_type in self.type_contrat_choices: advert.contract_type = self.type_contrat_choices[advert.contract_type] if advert.experience in self.level_choices: advert.experience = self.level_choices[advert.experience] return advert OBJECTS = {BaseJobAdvert: fill_obj} weboob-1.2/modules/apec/pages.py000066400000000000000000000075621303450110500166640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import requests from .job import APEC_CONTRATS, APEC_EXPERIENCE from weboob.browser.elements import ItemElement, method, DictElement from weboob.browser.pages import JsonPage, pagination from weboob.browser.filters.standard import DateTime, Format, Regexp from weboob.browser.filters.json import Dict from weboob.browser.filters.html import CleanHTML from weboob.capabilities.job import BaseJobAdvert from weboob.capabilities.base import NotAvailable class IdsPage(JsonPage): def get_adverts_number(self): return self.doc['totalCount'] @pagination @method class iter_job_adverts(DictElement): item_xpath = 'resultats' def next_page(self): self.page.browser.start += self.env['range'] if self.page.browser.start <= self.env['count']: data = self.page.browser.create_parameters(pattern=self.env['pattern'], fonctions=self.env['fonctions'], lieux=self.env['lieux'], secteursActivite=self.env['secteursActivite'], typesContrat=self.env['typesContrat'], typesConvention=self.env['typesConvention'], niveauxExperience=self.env['niveauxExperience'], salaire_min=self.env['salaire_min'], salaire_max=self.env['salaire_max'], date_publication=self.env['date_publication'], start=self.page.browser.start, range=self.env['range']) return requests.Request("POST", self.page.url, data=data) class item(ItemElement): klass = BaseJobAdvert obj_id = Regexp(Dict('@uriOffre'), '.*=(.*)') class OffrePage(JsonPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Dict('numeroOffre') obj_title = Dict('intitule') obj_description = CleanHTML(Dict('texteHtml')) obj_job_name = Dict('intitule') obj_publication_date = DateTime(Dict('datePublication')) obj_society_name = Dict('nomCommercialEtablissement', default=NotAvailable) def obj_contract_type(self): ctr = '%s' % Dict('idNomTypeContrat')(self) return APEC_CONTRATS.get(ctr) if ctr in APEC_CONTRATS else NotAvailable obj_place = Dict('lieux/0/libelleLieu') obj_pay = Dict('salaireTexte') def obj_experience(self): exp = u'%s' % Dict('idNomNiveauExperience')(self) return APEC_EXPERIENCE.get(exp) if exp in APEC_EXPERIENCE else NotAvailable obj_url = Format('https://cadres.apec.fr/home/mes-offres/recherche-des-offres-demploi/liste-des-offres-demploi/detail-de-loffre-demploi.html?numIdOffre=%s', Dict('numeroOffre')) weboob-1.2/modules/apec/test.py000066400000000000000000000035151303450110500165360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.value import Value import itertools class ApecTest(BackendTest): MODULE = 'apec' def setUp(self): if not self.is_backend_configured(): self.backend.config['level'] = Value(value='101883') self.backend.config['salaire'] = Value(value='') self.backend.config['secteur'] = Value(value='') self.backend.config['place'] = Value(value='089|62') self.backend.config['contrat'] = Value(value='101888') def test_apec_search(self): l = list(itertools.islice(self.backend.search_job(u'informaticien'), 0, 50)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_apec_advanced_search(self): l = list(itertools.islice(self.backend.advanced_search_job(), 0, 50)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) weboob-1.2/modules/apivie/000077500000000000000000000000001303450110500155465ustar00rootroot00000000000000weboob-1.2/modules/apivie/__init__.py000066400000000000000000000014361303450110500176630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ApivieModule __all__ = ['ApivieModule'] weboob-1.2/modules/apivie/browser.py000066400000000000000000000045061303450110500176100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, AccountsPage, OperationsPage __all__ = ['ApivieBrowser'] class ApivieBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'www.apivie.fr' ENCODING = None PAGES = { 'https?://www.apivie.fr/': LoginPage, 'https?://www.apivie.fr/accueil': LoginPage, 'https?://www.apivie.fr/perte.*': LoginPage, 'https?://www.apivie.fr/accueil-connect': AccountsPage, 'https?://www.apivie.fr/historique-contrat.*': OperationsPage, } def home(self): self.location('https://www.apivie.fr/accueil-connect') def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.is_on_page(LoginPage): self.location('/accueil', no_login=True) self.page.login(self.username, self.password) if not self.is_logged(): raise BrowserIncorrectPassword() def is_logged(self): return not self.is_on_page(LoginPage) def iter_accounts(self): self.location('/accueil-connect') return self.page.iter_accounts() def get_account(self, _id): try: return next(a for a in self.iter_accounts() if a.id == _id) except StopIteration: return None def iter_history(self, account): self.location(self.buildurl('/historique-contrat', contratId=account.id)) assert self.is_on_page(OperationsPage) return self.page.iter_history() weboob-1.2/modules/apivie/favicon.png000066400000000000000000000025551303450110500177100ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs76vtIME 4-MIDATxLUe?^@!%d9FM+Kә\.N9A 0kFMRR2WFa5rG?te* uejw{˽t|u}y< 0`yEy>r_;=N"+n7_GIP2 :ۀL./>ـ?=E6z~x&* Ҟ<J@{?Q"iҲ{?C) Xw8c//YCh!ocptG }:G u'G 't>ԩ4$T/$ `H%a~&1ɚ 'ҤK;pC>Heq 4y_*e)fs'M_V@~)8 ?K+8x2Z^n2kdHDMm5}RtOYAӂi͝5F)beXmd,A5%_xhXXN|-ce{@Zó12桘b ح͡րHvK ey-m'(0z!YPo\p ^rke4'04Էwzlg[K!. from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import ApivieBrowser __all__ = ['ApivieModule'] class ApivieModule(Module, CapBank): NAME = 'apivie' DESCRIPTION = u'Apivie' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' BROWSER = ApivieBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): with self.browser: return self.browser.iter_accounts() def get_account(self, _id): with self.browser: return self.browser.get_account(_id) def iter_history(self, account): with self.browser: return self.browser.iter_history(account) weboob-1.2/modules/apivie/pages.py000066400000000000000000000054531303450110500172260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from weboob.capabilities.bank import Account from weboob.deprecated.browser import Page from weboob.tools.capabilities.bank.transactions import FrenchTransaction class LoginPage(Page): def login(self, username, password): self.browser.select_form(nr=0) self.browser['_58_login'] = username.encode('utf-8') self.browser['_58_password'] = password.encode('utf-8') self.browser.submit(nologin=True) class AccountsPage(Page): COL_LABEL = 0 COL_OWNER = 1 COL_ID = 2 COL_AMOUNT = 3 def iter_accounts(self): for line in self.document.xpath('//table[@summary="informations contrat"]/tbody/tr'): yield self._get_account(line) def _get_account(self, line): tds = line.findall('td') account = Account() account.id = self.parser.tocleanstring(tds[self.COL_ID]) account.label = self.parser.tocleanstring(tds[self.COL_LABEL]) balance_str = self.parser.tocleanstring(tds[self.COL_AMOUNT]) account.balance = Decimal(FrenchTransaction.clean_amount(balance_str)) account.currency = account.get_currency(balance_str) return account class Transaction(FrenchTransaction): pass class OperationsPage(Page): COL_DATE = 0 COL_LABEL = 1 COL_AMOUNT = 2 def iter_history(self): for line in self.document.xpath('//table[@role="treegrid"]/tbody/tr'): tds = line.findall('td') operation = Transaction() date = self.parser.tocleanstring(tds[self.COL_DATE]) label = self.parser.tocleanstring(tds[self.COL_LABEL]) amount = self.parser.tocleanstring(tds[self.COL_AMOUNT]) if len(amount) == 0: continue color = tds[self.COL_AMOUNT].find('span').attrib['class'] if color == 'black': continue operation.parse(date, label) operation.set_amount(amount) if color == 'red' and operation.amount > 0: operation.amount = - operation.amount yield operation weboob-1.2/modules/apivie/test.py000066400000000000000000000017471303450110500171100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class ApivieTest(BackendTest): MODULE = 'apivie' def test_apivie(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/arretsurimages/000077500000000000000000000000001303450110500173265ustar00rootroot00000000000000weboob-1.2/modules/arretsurimages/__init__.py000066400000000000000000000014471303450110500214450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 franek # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ArretSurImagesModule __all__ = ['ArretSurImagesModule'] weboob-1.2/modules/arretsurimages/browser.py000066400000000000000000000045151303450110500213700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 franek # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.deprecated.browser.decorators import id2url from .pages import VideoPage, IndexPage, LoginPage, LoginRedirectPage from .video import ArretSurImagesVideo __all__ = ['ArretSurImagesBrowser'] class ArretSurImagesBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'www.arretsurimages.net' ENCODING = None PAGES = { '%s://%s/contenu.php\?id=.+' % (PROTOCOL, DOMAIN): VideoPage, '%s://%s/emissions.php' % (PROTOCOL, DOMAIN): IndexPage, '%s://%s/forum/login.php' % (PROTOCOL, DOMAIN): LoginPage, '%s://%s/forum/index.php' % (PROTOCOL, DOMAIN): LoginRedirectPage, } def home(self): self.location('http://www.arretsurimages.net') def search_videos(self, pattern): self.location(self.buildurl('/emissions.php')) assert self.is_on_page(IndexPage) return self.page.iter_videos(pattern) @id2url(ArretSurImagesVideo.id2url) def get_video(self, url, video=None): self.login() self.location(url) return self.page.get_video(video) def is_logged(self): return not self.is_on_page(LoginPage) def login(self): if not self.is_on_page(LoginPage): self.location('http://www.arretsurimages.net/forum/login.php', no_login=True) self.page.login(self.username, self.password) if not self.is_logged(): raise BrowserIncorrectPassword() def latest_videos(self): self.location(self.buildurl('/emissions.php')) assert self.is_on_page(IndexPage) return self.page.iter_videos() weboob-1.2/modules/arretsurimages/favicon.png000066400000000000000000000074631303450110500214730ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME *-fP>tEXtCommentCreated with GIMPWIDATx[Yo$uVKwnfFCR<%6" yCQyϟC',<;g!^kVYKsƲ3dWW=|;ȿ?i BB߀i u'סB(Hyz_B!RPRBgB(,˅4`.@) ZA) _FPJAke܎RP !NӀA)KB`Y.p@4 Z)x.zg=Dl҆qr1x+њ n -P.&R:YR2!e}Ƹc n kZG&0j>J+QY /}| ՛ؼ k( Bt>@dnmN_8`u4f!,9p bfv[}۟hgY`|)fVoGa/B)`L`~Ǹa,AA z}픂qኸv{[?4,ƻqg.,[?ď> n }`֟ceݱRaohV=NxDJ~%B۟­MǀfkW[.C*(&]Cg$H?jC2Ҥ*g:WIh2 eۮ1DB N3.37.^k %c8=ޅQ]sa9҄'ZgPJ"C8BB?;'(9Q8A){[> w+kyKj9Gov~_'G/ GY}N_38*0P*:q17wќ[)8W6`bPК VިWϿdF^'^m^"Hf61ZŠw4Mǝn^_?pg^9ou#<Kx^]CkZ.)PB֚`܂ZI켸~A]}@.cq&,-NrLQGZkޕא26 .-n  i*J)? *:iqt k(-4w֩a; 1.R$֣"web34E>DxnmΨOiP+%Y 4\pQk&H 1#KPW !u88ES=Ϥ. !! 8nc°Qa/pq$eƨQa:+ R=tMxo#HCTTRƅ B~#XV-W*dݳN!J`_?C̵(pr N*C B]kZ;6֗‰~BhH!ZZ+زjpj`=2 My88F{:(e2Kp*Wf'm(f.鑔%H?٣_@JkN@q;yՔ=~;9z~h0^H)*R)cbLu3qXi0-=86k ?Ax%:هlwHkU eܿ #SJT%mpn?rSR{:8Sjs1$cZ| "b `]PE(*o8 ,l.]n m2z\2koFvh<StCk4tr2^5\rpR)a1B; otLL).tC9Q]ӓ]c~XZ|ԃ#Qep1{Sdb$@3$T [\skX~gj(|l ިW^VMy/~VΆ3ڡ7QXcf(e/KG x΀1xPb6p8sQ`4N4& G?Vocy6yIxyt 읟2e47v'82Z#I /Z}ͥc*7N^catbۼDzP*6YqO;/W7O` Zkx3|/d4q`/;iMO]C)ÃmlOEW}9~q,句 W+R/Dž<8Dg},laemvܲMz" 4\LqU<\,}Ɉ !;پ? .eԍFSܾGϴ֚7ڨ՚pkMsOˤF>I2q 8X#iUdCRFFódHjJ+xtO;8z)Hި 뙅%}J8 ,ӀdO>`#J+0& *)C4L<"JK8D;Ӵn#`x^Fi>Fu2yNydf;!rv%bIENDB`weboob-1.2/modules/arretsurimages/module.py000066400000000000000000000064541303450110500211760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 franek # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import ArretSurImagesBrowser from .video import ArretSurImagesVideo __all__ = ['ArretSurImagesModule'] class ArretSurImagesModule(Module, CapVideo, CapCollection): NAME = 'arretsurimages' DESCRIPTION = u'arretsurimages website' MAINTAINER = u'franek' EMAIL = 'franek@chicour.net' VERSION = '1.2' CONFIG = BackendConfig(ValueBackendPassword('login', label='email', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = ArretSurImagesBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get(), get_home=False) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): with self.browser: return self.browser.search_videos(pattern) # raise UserError('Search does not work on ASI website, use ls latest command') def get_video(self, _id): if _id.startswith('http://') and not _id.startswith('http://www.arretsurimages.net'): return None with self.browser: return self.browser.get_video(_id) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(ArretSurImagesVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest ArretSurImages videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {ArretSurImagesVideo: fill_video} weboob-1.2/modules/arretsurimages/pages.py000066400000000000000000000100301303450110500207710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 franek # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.base import UserError from weboob.capabilities.image import Thumbnail from weboob.deprecated.browser import Page, BrokenPageError from weboob.capabilities import NotAvailable from .video import ArretSurImagesVideo class IndexPage(Page): def iter_videos(self, pattern=None): videos = self.document.getroot().cssselect("div[class=bloc-contenu-8]") for div in videos: title = self.parser.select(div, 'h1', 1).text_content().replace(' ', ' ') if pattern: if pattern.upper() not in title.upper(): continue m = re.match(r'/contenu.php\?id=(.*)', div.find('a').attrib['href']) _id = '' if m: _id = m.group(1) video = ArretSurImagesVideo(_id) video.title = unicode(title) video.rating = None video.rating_max = None thumb = self.parser.select(div, 'img', 1) url = u'http://www.arretsurimages.net' + thumb.attrib['src'] video.thumbnail = Thumbnail(url) video.thumbnail.url = video.thumbnail.id yield video class ForbiddenVideo(UserError): pass class VideoPage(Page): def is_logged(self): try: self.parser.select(self.document.getroot(), '#user-info', 1) except BrokenPageError: return False else: return True def on_loaded(self): if not self.is_logged(): raise ForbiddenVideo('This video or group may contain content that is inappropriate for some users') def get_video(self, video=None): if not video: video = ArretSurImagesVideo(self.get_id()) video.title = unicode(self.get_title()) video.url = unicode(self.get_url()) video.set_empty_fields(NotAvailable) return video def get_firstUrl(self): obj = self.parser.select(self.document.getroot(), 'a.bouton-telecharger', 1) firstUrl = obj.attrib['href'] return firstUrl def get_title(self): title = self.document.getroot().cssselect('div[id=titrage-contenu] h1')[0].text return title def get_id(self): m = re.match(r'http://videos.arretsurimages.net/telecharger/(.*)', self.get_firstUrl()) if m: return m.group(1) self.logger.warning('Unable to parse ID') return 0 def get_url(self): firstUrl = self.get_firstUrl() doc = self.browser.get_document(self.browser.openurl(firstUrl)) links = doc.xpath('//a') url = None i = 1 for link in links: # we take the second link of the page if i == 2: url = link.attrib['href'] i += 1 return url class LoginPage(Page): def login(self, username, password): response = self.browser.response() response.set_data(response.get_data().replace("
", "
")) # Python mechanize is broken, fixing it. self.browser.set_response(response) self.browser.select_form(nr=0) self.browser.form.set_all_readonly(False) self.browser['redir'] = '/forum/index.php' self.browser['username'] = username self.browser['password'] = password self.browser.submit() class LoginRedirectPage(Page): pass weboob-1.2/modules/arretsurimages/test.py000066400000000000000000000024311303450110500206570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 franek # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo from weboob.tools.test import BackendTest, SkipTest class ArretSurImagesTest(BackendTest): MODULE = 'arretsurimages' def test_latest_arretsurimages(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) assert len(l) if self.backend.browser.username != u'None': v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) else: raise SkipTest("User credentials not defined") weboob-1.2/modules/arretsurimages/video.py000066400000000000000000000016541303450110500210140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo class ArretSurImagesVideo(BaseVideo): @classmethod def id2url(cls, _id): return 'http://www.arretsurimages.net/contenu.php?id=%s' % _id weboob-1.2/modules/arte/000077500000000000000000000000001303450110500152245ustar00rootroot00000000000000weboob-1.2/modules/arte/__init__.py000066400000000000000000000014301303450110500173330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ArteModule __all__ = ['ArteModule'] weboob-1.2/modules/arte/browser.py000066400000000000000000000240311303450110500172610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.collection import Collection from weboob.capabilities.base import UserError from weboob.capabilities import NotAvailable from weboob.browser import PagesBrowser, URL from .pages import VideosListPage, ArteJsonPage from .video import VERSION_VIDEO, LANG, QUALITY, SITE, ArteEmptyVideo __all__ = ['ArteBrowser'] class ArteBrowser(PagesBrowser): BASEURL = 'http://arte.tv/' webservice = URL('papi/tvguide/(?P.*)/(?P.*)/(?P.*).json', 'http://(?P<__site>.*).arte.tv/(?P<_lang>\w{2})/player/(?P<_id>.*)', 'https://api.arte.tv/api/player/v1/config/(?P<__lang>\w{2})/(?P.*)\?vector=(?P<___site>.*)', ArteJsonPage) videos_list = URL('http://(?P.*).arte.tv/(?P\w{2})/?(?P.*?)', 'http://(?P<_site>.*).arte.tv/(?P.+)', VideosListPage) def __init__(self, lang, quality, order, format, version, *args, **kwargs): self.order = order self.lang = (value for key, value in LANG.items if key == lang).next() self.version = (value for key, value in VERSION_VIDEO.items if self.lang.get('label') in value.keys() and version == key).next() self.quality = (value for key, value in QUALITY.items if key == quality).next() self.format = format if self.lang.get('label') not in self.version.keys(): raise UserError('%s is not available for %s' % (self.lang.get('label'), version)) PagesBrowser.__init__(self, *args, **kwargs) def search_videos(self, pattern): class_name = 'videos/plus7' method_name = 'search' parameters = '/'.join([self.lang.get('webservice'), 'L1', pattern, 'ALL', 'ALL', '-1', self.order, '10', '0']) return self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).iter_videos() def get_video(self, id, video=None): class_name = 'videos' method_name = 'stream/player' parameters = '/'.join([self.lang.get('webservice'), id, 'ALL', 'ALL']) video = self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).get_video(obj=video) video.ext, video.url = self.get_url() return video def get_url(self): url, found_format = self.page.get_video_url(self.quality.get('label'), self.format, self.version.get(self.lang.get('label')), self.lang.get('version')) if found_format.startswith('HLS'): ext = u'm3u8' url = self.get_m3u8_link(url) else: ext = u'mp4' url = url return ext, url def get_m3u8_link(self, url): r = self.open(url).content.split('\n') baseurl = url.rpartition('/')[0] links_by_quality = [] for line in r: if not line.startswith('#'): if baseurl not in line: link = u'%s/%s' % (baseurl, line.replace('\n', '')) else: link = unicode(line.replace('\n', '')) links_by_quality.append(link) if len(links_by_quality): try: return links_by_quality[self.quality.get('order')] except: return links_by_quality[0] return NotAvailable def get_video_from_program_id(self, _id): class_name = 'epg' method_name = 'program' parameters = '/'.join([self.lang.get('webservice'), 'L2', _id]) video = self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).get_program_video() if video: return self.get_video(video.id, video) else: video = self.webservice.go(__lang=self.lang.get('site'), vid=_id, ___site='ARTEPLUS7').get_program_video() video.ext, video.url = self.get_url() # buggy URLs video.url = video.url.replace('%255B', '%5B').replace('%255D', '%5D') return video def latest_videos(self): class_name = 'videos' method_name = 'plus7' parameters = '/'.join([self.lang.get('webservice'), 'L1', 'ALL', 'ALL', '-1', self.order, '10', '0']) return self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).iter_videos() def get_arte_programs(self): class_name = 'epg' method_name = 'clusters' parameters = '/'.join([self.lang.get('webservice'), '0', 'ALL']) return self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).iter_programs(title=self.lang.get('title')) def get_arte_program_videos(self, program): class_name = 'epg' method_name = 'cluster' parameters = '/'.join([self.lang.get('webservice'), program[-1]]) available_videos = self.webservice.go(class_name=class_name, method_name=method_name, parameters=parameters).iter_program_videos() for item in available_videos: video = self.get_video_from_program_id(item.id) if video: yield video def get_arte_concert_categories(self): return self.videos_list.go(site=SITE.CONCERT.get('id'), lang=self.lang.get('site'), cat='').iter_arte_concert_categories() def get_arte_concert_videos(self, cat): return self.videos_list.go(site=SITE.CONCERT.get('id'), lang=self.lang.get('site'), cat='').iter_arte_concert_videos(cat=cat[-1]) def get_arte_concert_video(self, id, video=None): json_url = self.videos_list.go(_site=SITE.CONCERT.get('id'), id=id).get_json_url() m = re.search('http://(?P<__site>.*).arte.tv/(?P<_lang>\w{2})/player/(?P<_id>.*)', json_url) if m: video = self.webservice.go(__site=m.group('__site'), _lang=m.group('_lang'), _id=m.group('_id')).get_arte_concert_video(obj=video) video.id = u'%s.%s' % (video._site, id) video.ext, video.url = self.get_url() return video def get_arte_cinema_categories(self, cat=[]): menu = self.videos_list.go(site=SITE.CINEMA.get('id'), lang=self.lang.get('site'), cat='').get_arte_cinema_menu() menuSplit = map(lambda x: x.split("/")[2:], menu) result = {} for record in menuSplit: here = result for item in record[:-1]: if item not in here: here[item] = {} here = here[item] if "end" not in here: here["end"] = [] here["end"].append(record[-1]) cat = cat if not cat else cat[1:] if not cat and "end" in result: del result["end"] for el in cat: result = result.get(el) if "end" in result.keys(): return self.page.iter_arte_cinema_categories(cat='/'.join(cat)) else: categories = [] for item in result.keys(): categories.append(Collection([SITE.CINEMA.get('id'), unicode(item)], unicode(item))) return categories def get_arte_cinema_videos(self, cat): return self.videos_list.go(site=SITE.CINEMA.get('id'), lang=self.lang.get('site'), cat='/%s' % '/'.join(cat[1:])).get_arte_cinema_videos() def get_arte_cinema_video(self, id, video=None): json_url = self.videos_list.go(_site=SITE.CINEMA.get('id'), id=id).get_json_url() m = re.search('https://api.arte.tv/api/player/v1/config/(\w{2})/(.*)\?(vector|platform)=(.*)\&.*', json_url) if m: video = self.webservice.go(__lang=m.group(1), vid=m.group(2), ___site=m.group(4)).get_arte_cinema_video(obj=video) video.ext, video.url = self.get_url() video.id = id return video return ArteEmptyVideo() def get_arte_creative_categories(self): return self.videos_list.go(site=SITE.CREATIVE.get('id'), lang=self.lang.get('site'), cat='').iter_arte_creative_categories() def get_arte_creative_videos(self, cat): _cat = cat[-1].replace('^', '/') if cat[-1] != u'accueil' else '' return self.videos_list.go(site=SITE.CREATIVE.get('id'), lang=self.lang.get('site'), cat='/%s' % _cat).iter_arte_creative_videos(cat=cat[-1]) def get_arte_creative_video(self, id, video=None): json_url = self.videos_list.go(_site=SITE.CREATIVE.get('id'), id=id).get_json_url() m = re.search('https://api.arte.tv/api/player/v1/config/(\w{2})/(.*)\?vector=(.*)\&.*', json_url) if m: video = self.webservice.go(__lang=m.group(1), vid=m.group(2), ___site=m.group(3)).get_arte_cinema_video(obj=video) video.ext, video.url = self.get_url() video.id = id return video return ArteEmptyVideo() weboob-1.2/modules/arte/favicon.png000066400000000000000000000014631303450110500173630ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME5p IDATxOVUϝ?̘\f J!2n>h+E-"ĕ` D0(IA X(Mj9.< afyw=_89y~<}I$D"H$1h]x9]ex Kww?q S+YQ՝.!l^ m3Fz<p:+[}.VΝI|~Pw|MZw̷ x2>J\A@|xPBG,|^lo^spL*ƱLq;vbK8>_yM38տ Üˊ߆N 5!|ٰ]i1s!jVT_e>= sfz֙XQ,0TDuB&Rg{y&dk+Bmk&@]kdyu2߁x;AKZ.B~V|2+,pF8'K!54G_wp E,WN,߸yw;B}L>;br%ɊLplq4y>o]6Eq q)r  ï<{ "~ .@̟&P"H$D"H$^qQcIENDB`weboob-1.2/modules/arte/module.py000066400000000000000000000151771303450110500170760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.exceptions import BrowserHTTPSDowngrade from .browser import ArteBrowser from .video import ArteVideo, ArteSiteVideo, VERSION_VIDEO, FORMATS, LANG, QUALITY, SITE __all__ = ['ArteModule'] class ArteModule(Module, CapVideo, CapCollection): NAME = 'arte' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' DESCRIPTION = 'Arte French and German TV' LICENSE = 'AGPLv3+' order = {'AIRDATE_DESC': 'Date', 'VIEWS': 'Views', 'ALPHA': 'Alphabetic', 'LAST_CHANCE': 'Last chance' } versions_choice = OrderedDict([(k, u'%s' % (v.get('label'))) for k, v in VERSION_VIDEO.items]) format_choice = OrderedDict([(k, u'%s' % (v)) for k, v in FORMATS.items]) lang_choice = OrderedDict([(k, u'%s' % (v.get('label'))) for k, v in LANG.items]) quality_choice = [u'%s' % (k) for k, v in QUALITY.items] CONFIG = BackendConfig(Value('lang', label='Lang of videos', choices=lang_choice, default='FRENCH'), Value('order', label='Sort order', choices=order, default='AIRDATE_DESC'), Value('quality', label='Quality of videos', choices=quality_choice, default=QUALITY.HD), Value('format', label='Format of videos', choices=format_choice, default=FORMATS.HTTP_MP4), Value('version', label='Version of videos', choices=versions_choice)) BROWSER = ArteBrowser def create_default_browser(self): return self.create_browser(lang=self.config['lang'].get(), quality=self.config['quality'].get(), order=self.config['order'].get(), format=self.config['format'].get(), version=self.config['version'].get()) def parse_id(self, _id): sites = '|'.join(k.get('id') for k in SITE.values) m = re.match('^(%s)\.(.*)' % sites, _id) if m: return m.groups() m = re.match('https?://www.arte.tv/guide/\w+/(?P.+)/(.*)', _id) if m: return SITE.PROGRAM.get('id'), m.group(1) m = re.match('https?://(%s).arte.tv/(\w+)/(.*)' % (sites), _id) if m: return m.group(1), '/%s/%s' % (m.group(2), m.group(3)) if not _id.startswith('http'): return 'videos', _id return None, None def get_video(self, _id): site, _id = self.parse_id(_id) if not (site and _id): return None if site in [value.get('id') for value in SITE.values]: _site = (value for value in SITE.values if value.get('id') == site).next() return getattr(self.browser, _site.get('video'))(_id) else: return self.browser.get_video(_id) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern) def fill_arte_video(self, video, fields): if fields != ['thumbnail']: video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def fill_site_video(self, video, fields): if fields != ['thumbnail']: for site in SITE.values: m = re.match('%s\.(.*)' % site.get('id'), video.id) if m: video = getattr(self.browser, site.get('video'))(m.group(1), video) break if 'thumbnail' in fields and video and video.thumbnail: try: video.thumbnail.data = self.browser.open(video.thumbnail.url).content except BrowserHTTPSDowngrade: pass return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield Collection([u'arte-latest'], u'Latest Arte videos') for site in SITE.values: yield Collection([site.get('id')], site.get('label')) if collection.path_level == 1: if collection.split_path == [u'arte-latest']: for video in self.browser.latest_videos(): yield video else: for site in SITE.values: if collection.split_path[0] == site.get('id') and collection.path_level in site.keys(): for item in getattr(self.browser, site.get(collection.path_level))(): yield item if collection.path_level >= 2: for site in SITE.values: if collection.split_path[0] == site.get('id') and collection.path_level in site.keys(): for item in getattr(self.browser, site.get(collection.path_level))(collection.split_path): yield item def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and (collection.split_path == [u'arte-latest'] or collection.split_path[0] in [value.get('id') for value in SITE.values]): return if BaseVideo in objs and collection.path_level >= 2 and\ collection.split_path[0] in [value.get('id') for value in SITE.values]: return raise CollectionNotFound(collection.split_path) OBJECTS = {ArteVideo: fill_arte_video, ArteSiteVideo: fill_site_video} weboob-1.2/modules/arte/pages.py000066400000000000000000000257551303450110500167130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import timedelta import urllib from weboob.capabilities.image import Thumbnail from weboob.capabilities.base import BaseObject, NotAvailable from weboob.capabilities.collection import Collection from weboob.capabilities.base import empty from weboob.browser.pages import HTMLPage, JsonPage from weboob.browser.elements import DictElement, ItemElement, ListElement, method from weboob.browser.filters.standard import Date, Format, Env, CleanText, Field, Regexp, Join from weboob.browser.filters.json import Dict from weboob.browser.filters.html import XPath from .video import ArteVideo, ArteSiteVideo, SITE class ArteItemElement(ItemElement): def condition(self): return 'VID' in self.el obj_id = Dict('VID') def obj_title(self): vti = Dict('VTI')(self) vtu = Dict('VSU', default=None)(self) if not vtu: return vti return '%s: %s' % (vti, vtu) obj_rating = Dict('VRT', default=NotAvailable) obj_rating_max = 10 obj_description = Dict('VDE', default=NotAvailable) obj_date = Date(Dict('VDA', default=NotAvailable), default=NotAvailable) def obj_duration(self): seconds = Dict('videoDurationSeconds', default=NotAvailable)(self) if empty(seconds): return seconds elif isinstance(seconds, basestring): seconds = int(seconds) return timedelta(seconds=seconds) def obj_thumbnail(self): url = Dict('VTU/IUR', default=NotAvailable)(self) if empty(url): return url thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail class VideosListPage(HTMLPage): @method class iter_arte_concert_categories(ListElement): item_xpath = '//ul[@class="filter-liste"]/li' class item(ItemElement): klass = Collection obj_title = CleanText('./a/span') obj_id = CleanText('./@data-target', replace=[('video_box_tab_', '')]) def obj_split_path(self): _id = CleanText('./@data-target', replace=[('video_box_tab_', '')])(self) return [SITE.CONCERT.get('id'), u'%s' % _id] @method class iter_arte_concert_videos(ListElement): def find_elements(self): self.item_xpath = '//div[@id="video_box_tab_%s"]/article' % Env('cat')(self) for el in self.el.xpath(self.item_xpath): yield el class item(ItemElement): klass = ArteSiteVideo obj__site = SITE.CONCERT.get('id') obj_id = Format('%s.%s', Field('_site'), CleanText('./@about')) obj_title = CleanText('div/div[@class="info-article "]/div/h3/a') def obj_thumbnail(self): url = CleanText('div/div/a/figure/span/span/@data-src')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail @method class iter_arte_creative_categories(ListElement): item_xpath = '//ul[@class="menu"]/li/a[not(@target)]' class item(ItemElement): klass = Collection obj_title = CleanText('.', default=u'Accueil') obj_id = CleanText('./@href') def obj_split_path(self): _id = Regexp(CleanText('./@href'), '/\w{2}/(.*)', default=u'accueil')(self) return [SITE.CREATIVE.get('id')] + [_id.replace('/', '^')] @method class iter_arte_creative_videos(ListElement): item_xpath = '//div[div/i]' ignore_duplicate = True class item(ItemElement): klass = ArteSiteVideo obj__site = SITE.CREATIVE.get('id') obj_id = Format('%s.%s', Field('_site'), CleanText('./div/h3/a/@href|./div/h1/a/@href')) obj_title = CleanText('./div/h3/a|./div/h1/a') def obj_thumbnail(self): url = CleanText('./div/a/img/@src')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail @method class iter_arte_cinema_categories(ListElement): item_xpath = '//li[has-class("leaf")]' class item(ItemElement): klass = Collection def condition(self): return Regexp(CleanText('./a/@href'), '^(/\w{2}/%s/.*)' % self.env['cat'], default=None)(self) obj_title = CleanText('./a') obj_id = CleanText('./a/@href') def obj_split_path(self): _id = Regexp(CleanText('./a/@href'), '/\w{2}/(.*)')(self) return [SITE.CINEMA.get('id')] + _id.split('/') def get_arte_cinema_menu(self): return self.doc.xpath('//li[has-class("leaf")]/a[starts-with(@href,"/")]/@href') @method class get_arte_cinema_videos(ListElement): item_xpath = '//div[has-class("article-list")]/article[@id]' class item(ItemElement): klass = ArteSiteVideo def condition(self): return len(XPath('.//div[@class="article-secondary "]')(self)) == 1 obj__site = SITE.CINEMA.get('id') obj_id = Format('%s.%s', Field('_site'), Regexp(CleanText('./div/div/a/@href|./div/a/@href'), '(http://.*\.arte\.tv)?/(.*)', '\\2')) obj_title = Join(u' - ', './/div[@class="article-secondary "]/div/div') def obj_thumbnail(self): url = CleanText('.//div[@class="article-primary "]/div[has-class("field-thumbnail")]/span/noscript/img/@src')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail def get_json_url(self): if self.doc.xpath('//div[@class="video-container"]'): return self.doc.xpath('//div[@class="video-container"]')[0].attrib['arte_vp_url'] elif self.doc.xpath('//iframe'): url = Regexp(CleanText('./@src'), '.*json_url=(.*)')(self.doc.xpath('//iframe')[0]) return urllib.unquote(url) return '' class ArteJsonPage(JsonPage): def get_video_url(self, quality, format, version, language_version): _urls = Dict('videoJsonPlayer/VSR')(self.doc) if _urls: urls = _urls.keys() key = '_'.join([format, quality, version]) found = self.find_url(key, urls, version, quality) if not found: # We use the default language version key = '_'.join([format, quality, language_version]) found = self.find_url(key, urls, version, quality) if not found: # We only keep the quality key = '_'.join([quality, language_version]) found = self.find_url(key, urls, version, quality) if not found: found = urls[0] streamer = Dict('videoJsonPlayer/VSR/%s/streamer' % (found), default=None)(self.doc) url = Dict('videoJsonPlayer/VSR/%s/url' % (found))(self.doc) if streamer: return '%s%s' % (streamer, url), found return url, found return NotAvailable, '' def find_url(self, key, urls, version, quality): self.logger.debug('available urls: %s' % urls) self.logger.debug('search url matching : %s' % key) # Best Case: key is mathing matching = [s for s in urls if key in s] self.logger.debug('best case matching: %s' % matching) if matching: return matching[0] # Second Case: is the version available matching = [s for s in urls if version in s] self.logger.debug('is version available: %s' % matching) if matching: # Do the quality + version match matching_quality = [s for s in matching if quality in s] self.logger.debug('does quality + version match: %s' % matching_quality) if matching_quality: return matching[0] # Only format + version mathes return matching[0] @method class iter_videos(DictElement): item_xpath = 'videoList' class item(ArteItemElement): klass = ArteVideo @method class iter_programs(DictElement): item_xpath = 'configClusterList' class item(ItemElement): klass = Collection obj_title = Dict(CleanText(Env('title'))) obj_id = Dict('clusterId') def obj_split_path(self): return [SITE.PROGRAM.get('id'), Dict('clusterId')(self)] @method class get_video(ArteItemElement): def __init__(self, *args, **kwargs): super(ArteItemElement, self).__init__(*args, **kwargs) self.el = self.el.get('videoJsonPlayer') klass = ArteVideo @method class get_arte_concert_video(ArteItemElement): def __init__(self, *args, **kwargs): super(ArteItemElement, self).__init__(*args, **kwargs) self.el = self.el.get('videoJsonPlayer') klass = ArteSiteVideo obj__site = SITE.CONCERT.get('id') @method class get_arte_cinema_video(ArteItemElement): def __init__(self, *args, **kwargs): super(ArteItemElement, self).__init__(*args, **kwargs) self.el = self.el.get('videoJsonPlayer') klass = ArteSiteVideo obj__site = SITE.CINEMA.get('id') obj_date = Date(Dict('VRA')) @method class get_program_video(ArteItemElement): def __init__(self, *args, **kwargs): super(ArteItemElement, self).__init__(*args, **kwargs) if 'videoJsonPlayer' in self.el: self.el = self.el.get('videoJsonPlayer') elif 'abstractProgram' not in self.el: return None elif 'VDO' in self.el['abstractProgram'].keys(): self.el = self.el['abstractProgram']['VDO'] klass = ArteVideo @method class iter_program_videos(DictElement): item_xpath = 'clusterWrapper/broadcasts' ignore_duplicate = True class item(ItemElement): klass = BaseObject def condition(self): return 'VDS' in self.el.keys() and len(self.el['VDS']) > 0 obj_id = Dict('programId') weboob-1.2/modules/arte/test.py000066400000000000000000000056201303450110500165600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.value import Value from weboob.capabilities.video import BaseVideo from .video import SITE class ArteTest(BackendTest): MODULE = 'arte' def setUp(self): if not self.is_backend_configured(): self.backend.config['lang'] = Value(value='FRENCH') self.backend.config['quality'] = Value(value='HD') self.backend.config['order'] = Value(value='LAST_CHANCE') self.backend.config['format'] = Value(value='HLS') self.backend.config['version'] = Value(value='VOSTF') def test_search(self): l = list(self.backend.search_videos('a')) assert len(l) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_sites(self): for site in SITE.values: if site.get('id') == SITE.PROGRAM.get('id'): continue l1 = list(self.backend.iter_resources([BaseVideo], [site.get('id')])) assert len(l1) l1 = l1[0] while not isinstance(l1, BaseVideo): l1 = list(self.backend.iter_resources([BaseVideo], l1.split_path)) assert len(l1) l1 = l1[0] self.backend.fillobj(l1, ('url',)) self.assertTrue(l1.url, 'URL for video "%s" not found' % (l1.id)) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'arte-latest'])) assert len(l) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) def test_program(self): l1 = list(self.backend.iter_resources([BaseVideo], [u'program'])) assert len(l1) # some categories may contain no available videos (during summer period for example) for l in l1: l2 = list(self.backend.iter_resources([BaseVideo], l.split_path)) if len(l2) == 0: continue break assert len(l2) v = l2[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) weboob-1.2/modules/arte/video.py000066400000000000000000000060671303450110500167150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import enum from weboob.capabilities.video import BaseVideo FORMATS = enum(HTTP_MP4=u'HBBTV', HLS=u'M3U8', RTMP=u'RTMP', HLS_MOBILE=u'MOBILE') LANG = enum(FRENCH={u'label': u'French', u'webservice': u'F', u'site': u'fr', u'version': u'1', u'title': u'titleFR'}, GERMAN={u'label': u'German', u'webservice': u'D', u'site': u'de', u'version': u'1', u'title': u'titleDE'}) SITE = enum(PROGRAM={u'id': u'program', u'label': u'Arte Programs', 1: 'get_arte_programs', 2: 'get_arte_program_videos', u'video': 'get_video_from_program_id'}, CONCERT={u'id': u'concert', u'label': u'Arte Concert videos', 1: 'get_arte_concert_categories', 2: 'get_arte_concert_videos', 'video': 'get_arte_concert_video'}, CINEMA={u'id': u'cinema', u'label': u'Arte Cinema', 1: 'get_arte_cinema_categories', 2: 'get_arte_cinema_categories', 3: 'get_arte_cinema_videos', 'video': 'get_arte_cinema_video'}, CREATIVE={u'id': u'creative', u'label': u'Arte Creative', 1: 'get_arte_creative_categories', 2: 'get_arte_creative_videos', 'video': 'get_arte_creative_video'}) QUALITY = enum(HD={'label': u'SQ', 'order': 3}, MD={'label': u'EQ', 'order': 2}, SD={'label': u'MQ', 'order': 1}, LD={'label': u'LQ', 'order': 0}, XD={'label': u'XQ', 'order': 4},) VERSION_VIDEO = enum(VOSTA={u'label': u'Original version subtitled (German)', LANG.GERMAN.get('label'): u'3'}, VOSTF={u'label': u'Original version subtitled (French)', LANG.FRENCH.get('label'): u'3'}, VASTA={u'label': u'Translated version (German)', LANG.GERMAN.get('label'): u'1', LANG.FRENCH.get('label'): u'2'}, VFSTF={u'label': u'Translated version (French)', LANG.FRENCH.get('label'): u'1', LANG.GERMAN.get('label'): u'2'}, VASTMA={u'label': u'Deaf version (German)', LANG.GERMAN.get('label'): u'8'}, VFSTMF={u'label': u'Deaf version (French)', LANG.FRENCH.get('label'): u'8'}) class ArteVideo(BaseVideo): pass class ArteSiteVideo(BaseVideo): pass class ArteEmptyVideo(BaseVideo): def __init__(self): self.description = u'There is no video on this page' weboob-1.2/modules/attilasub/000077500000000000000000000000001303450110500162615ustar00rootroot00000000000000weboob-1.2/modules/attilasub/__init__.py000066400000000000000000000014371303450110500203770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AttilasubModule __all__ = ['AttilasubModule'] weboob-1.2/modules/attilasub/browser.py000066400000000000000000000035031303450110500203170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from .pages import SubtitlesPage, SearchPage __all__ = ['AttilasubBrowser'] class AttilasubBrowser(Browser): DOMAIN = 'davidbillemont3.free.fr' PROTOCOL = 'http' ENCODING = 'windows-1252' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://search.freefind.com/find.html.*': SearchPage, 'http://davidbillemont3.free.fr/.*.htm': SubtitlesPage, } def iter_subtitles(self, language, pattern): self.location('http://search.freefind.com/find.html?id=81131980&_charset_=&bcd=%%F7&scs=1&pageid=r&query=%s&mode=Find%%20pages%%20matching%%20ALL%%20words' % pattern.encode('utf-8')) assert self.is_on_page(SearchPage) return self.page.iter_subtitles(language, pattern) def get_subtitle(self, id): url_end = id.split('|')[0] try: self.location('http://davidbillemont3.free.fr/%s' % url_end) except BrowserHTTPNotFound: return if self.is_on_page(SubtitlesPage): return self.page.get_subtitle(id) weboob-1.2/modules/attilasub/favicon.png000066400000000000000000000064061303450110500204220ustar00rootroot00000000000000PNG  IHDR@@% sRGBbKGD pHYs  tIME0;! IDAThZrGr='zRV]c×'?+ioa$x\2{0$f(hEu0j39_,5~gfGf}_ \"()23r{+#3j+; $ed޼-t!(I$Yk"UON&;$%f4tOf/\)#͝?}ζo"HE铇v}ݴ9Hl6^!"_dffnHwtW%!%o߽í;FN_~}xp85O鍣PKRׯGDkd2֖Ax= `Ӭ}5jԵ՗dfFJ"rFXjqz#??S^%unfyR4fVJ麎$7nefq J+¨SZKZ οA$嵩$")]5BJku_YFɾ-̦WGJL<:(]~'1"M;iqp3/<}ZOqv:puE:ͦl5E"Rw~ݷv$2NzYEKr>g<9LtFJ-3K biōM&wssoDf0 h% $Z0/gNϯ\}gok{+ 9dqDٽffPKز̌kׯ}?#3}o@ C Upgf^3PPD!E|:ݹ(k8a s}2^B 7toxms '  V5LL&ޭsV(D@U L&;7w]#r^&+ЬK(D0G)0_zQ$* " "jWЖ6Ft@4M{D@ffdzf22W2 3 -6! A B`r%JC3h|P[Da^)`b)PdH!F:49Z&2 XeD1:\``Ae4F]na+ 'VA9)&L$BՔeS ySHi6|>S-I M Ht?/xe!(`%G %aRHkMX7D$.$R.P̈́FCk6&G7@B$jV/)+d$#$dORJ@BA`4QFJ2XEEhM"h]̎j I PQܼ#טP$B)hI@r=̭i+P [)Ns!pPK;"Lܔb ??v&aL$HgL?y>. UiƈL:;=?;=^ͅ㳓㳫,-YP|l'훙ϚRG|{{ݮ$J8w5Y#N'{cgg:͔h Tkg =K< ~ޝk׶:Io}oڋLx;@E& #Uo?>>,V2*P0h"l_Rovn^ +/ k޽ν" KBJ41|Z[[7wֵ\&va0@M)A>y6(ϲjIkpÁ@H!82~L0Hd)LDvF#E/\IaV܊y l_a%/ӗ5{M`$5 bN)J4  Xty"R.[C+T fC&"9֯m )) ^I kG:\0>Z1s@֪>UjE(#rF2+h=ND"org4=HPd3߰d$lC\;ZZ֏mf]Wʤ0D&3 /ffE Jq%Hր!4t]fC)FBW#`oUB:NNk_342Nj@7gziseK"idbOVFe9؜ ּT/v +=hlB??;=k7kR&6y4fs&gO=^,''?=J`-4K+8b"F}Kz퓧Z$_Bhϩ|cs{i;]=\TJ5@{|z!֞!/ "f`}hB7IQ뀶x9[ջ8,0IEFFYݼ[k) @2^mݍ$eT*UQVYdž"5"k%{oݾsqht%ǠYPVϕ V6AD+d^2ԑDDJ"zY ʬ}jn?;cq>8 0\ߖ+ZCx`̨,Ç\Mձ*RA֊U J*/*&V_BiiW}EE;YC# ֍`9:Ų? ',F.ρ熂Ϥ6 bIENDB`weboob-1.2/modules/attilasub/module.py000066400000000000000000000033521303450110500201230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.subtitle import CapSubtitle, LanguageNotSupported from weboob.tools.backend import Module from .browser import AttilasubBrowser from urllib import quote_plus __all__ = ['AttilasubModule'] class AttilasubModule(Module, CapSubtitle): NAME = 'attilasub' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = '"Attila\'s Website 2.0" French subtitles' LICENSE = 'AGPLv3+' LANGUAGE_LIST = ['fr'] BROWSER = AttilasubBrowser def get_subtitle(self, id): return self.browser.get_subtitle(id) def get_subtitle_file(self, id): subtitle = self.browser.get_subtitle(id) if not subtitle: return None return self.browser.openurl(subtitle.url.encode('utf-8')).read() def iter_subtitles(self, language, pattern): if language not in self.LANGUAGE_LIST: raise LanguageNotSupported() return self.browser.iter_subtitles(language, quote_plus(pattern.encode('utf-8'))) weboob-1.2/modules/attilasub/pages.py000066400000000000000000000120141303450110500177300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.subtitle import Subtitle from weboob.capabilities.base import NotAvailable from weboob.deprecated.browser import Page class SearchPage(Page): def iter_subtitles(self, language, pattern): fontresult = self.parser.select(self.document.getroot(), 'div.search-results font.search-results') # for each result in freefind, explore the subtitle list page to iter subtitles for res in fontresult: a = self.parser.select(res, 'a', 1) url = a.attrib.get('href', '') self.browser.location(url) assert self.browser.is_on_page(SubtitlesPage) # subtitles page does the job for subtitle in self.browser.page.iter_subtitles(language, pattern): yield subtitle class SubtitlesPage(Page): def get_subtitle(self, id): href = id.split('|')[1] # we have to find the 'tr' which contains the link to this address a = self.parser.select(self.document.getroot(), 'a[href="%s"]' % href, 1) line = a.getparent().getparent().getparent().getparent().getparent() cols = self.parser.select(line, 'td') traduced_title = self.parser.select(cols[0], 'font', 1).text.lower() original_title = self.parser.select(cols[1], 'font', 1).text.lower() nb_cd = self.parser.select(cols[2], 'font', 1).text.strip() nb_cd = int(nb_cd.split()[0]) traduced_title_words = traduced_title.split() original_title_words = original_title.split() # this is to trash special spacing chars traduced_title = " ".join(traduced_title_words) original_title = " ".join(original_title_words) name = unicode('%s (%s)' % (original_title, traduced_title)) url = unicode('http://davidbillemont3.free.fr/%s' % href) subtitle = Subtitle(id, name) subtitle.url = url subtitle.ext = url.split('.')[-1] subtitle.language = unicode('fr') subtitle.nb_cd = nb_cd subtitle.description = NotAvailable return subtitle def iter_subtitles(self, language, pattern): pattern = pattern.strip().replace('+', ' ').lower() pattern_words = pattern.split() tab = self.parser.select(self.document.getroot(), 'table[bordercolor="#B8C0B2"]') if len(tab) == 0: tab = self.parser.select(self.document.getroot(), 'table[bordercolordark="#B8C0B2"]') if len(tab) == 0: return # some results of freefind point on useless pages if tab[0].attrib.get('width', '') != '100%': return for line in tab[0].getiterator('tr'): cols = self.parser.select(line, 'td') traduced_title = self.parser.select(cols[0], 'font', 1).text.lower() original_title = self.parser.select(cols[1], 'font', 1).text.lower() traduced_title_words = traduced_title.split() original_title_words = original_title.split() # if the pattern is one word and in the title OR if the # intersection between pattern and the title is at least 2 words if (len(pattern_words) == 1 and pattern in traduced_title_words) or\ (len(pattern_words) == 1 and pattern in original_title_words) or\ (len(list(set(pattern_words) & set(traduced_title_words))) > 1) or\ (len(list(set(pattern_words) & set(original_title_words))) > 1): # this is to trash special spacing chars traduced_title = " ".join(traduced_title_words) original_title = " ".join(original_title_words) nb_cd = self.parser.select(cols[2], 'font', 1).text.strip() nb_cd = int(nb_cd.strip(' CD')) name = unicode('%s (%s)' % (original_title, traduced_title)) href = self.parser.select(cols[3], 'a', 1).attrib.get('href', '') url = unicode('http://davidbillemont3.free.fr/%s' % href) id = unicode('%s|%s' % (self.browser.geturl().split('/')[-1], href)) subtitle = Subtitle(id, name) subtitle.url = url subtitle.ext = url.split('.')[-1] subtitle.language = unicode('fr') subtitle.nb_cd = nb_cd subtitle.description = NotAvailable yield subtitle weboob-1.2/modules/attilasub/test.py000066400000000000000000000024361303450110500176170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest import urllib from random import choice class AttilasubTest(BackendTest): MODULE = 'attilasub' def test_subtitle(self): subtitles = list(self.backend.iter_subtitles('fr', 'spiderman')) assert (len(subtitles) > 0) for subtitle in subtitles: path, qs = urllib.splitquery(subtitle.url) assert path.endswith('.rar') # get the file of a random sub if len(subtitles): subtitle = choice(subtitles) self.backend.get_subtitle_file(subtitle.id) weboob-1.2/modules/audioaddict/000077500000000000000000000000001303450110500165435ustar00rootroot00000000000000weboob-1.2/modules/audioaddict/__init__.py000066400000000000000000000014451303450110500206600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AudioAddictModule __all__ = ['AudioAddictModule'] weboob-1.2/modules/audioaddict/favicon.png000066400000000000000000000026761303450110500207110ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME6|tEXtCommentCreated with GIMPW&IDATxoG?bP"\*PAADbɭ[B"[phPBBhCby=$&؞7%;JIv}go~rK3^       ‹C݁`L?9[V=d&! h+< (L6@ڠ-j$`_# >Xז2=h T˙ U(4yӯ`Ww.߻ln_w1咮p.3[6oyDidSqM lM xpn꾃~u૯M&{PM_MzP5Ǽ/umMN0iq-l}Sf{6Y$\MΆ -qwUeﴝI\m(3*~876}s|ERGewD{ ԄjK% 2c~ xaiBRW"G6=  RPPPPp]-âIENDB`weboob-1.2/modules/audioaddict/module.py000066400000000000000000000277371303450110500204220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection, Collection from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.deprecated.browser import StandardBrowser import time __all__ = ['AudioAddictModule'] # # WARNING # # AudioAddict playlists do not seem to be appreciated by mplayer # VLC plays them successfully, therefore I advice to set the media_player # option to another player in the ~/.config/weboob/radioob config file: # [ROOT] # media_player = your_non_mplayer_player class AudioAddictModule(Module, CapRadio, CapCollection): NAME = 'audioaddict' MAINTAINER = u'Pierre Mazière' EMAIL = 'pierre.maziere@gmx.com' VERSION = '1.2' DESCRIPTION = u'Internet radios powered by audioaddict.com services' LICENSE = 'AGPLv3+' BROWSER = StandardBrowser # Data extracted from http://tobiass.eu/api-doc.html NETWORKS = { 'DI': { 'desc': 'Digitally Imported addictive electronic music', 'domain': 'di.fm', 'streams': {'android_low': {'rate': 40, 'fmt': 'aac'}, 'android': {'rate': 64, 'fmt': 'aac'}, 'android_high': {'rate': 96, 'fmt': 'aac'}, 'android_premium_low': {'rate': 40, 'fmt': 'aac'}, 'android_premium_medium': {'rate': 64, 'fmt': 'aac'}, 'android_premium': {'rate': 128, 'fmt': 'aac'}, 'android_premium_high': {'rate': 256, 'fmt': 'aac'}, 'public1': {'rate': 64, 'fmt': 'aac'}, 'public2': {'rate': 40, 'fmt': 'aac'}, 'public3': {'rate': 96, 'fmt': 'mp3'}, 'premium_low': {'rate': 40, 'fmt': 'aac'}, 'premium_medium': {'rate': 64, 'fmt': 'aac'}, 'premium': {'rate': 128, 'fmt': 'aac'}, 'premium_high': {'rate': 256, 'fmt': 'mp3'} } }, 'RadioTunes': { 'desc': 'Radio Tunes', 'domain': 'radiotunes.com', 'streams': {'appleapp_low': {'rate': 40, 'fmt': 'aac'}, 'appleapp': {'rate': 64, 'fmt': 'aac'}, 'appleapp_high': {'rate': 96, 'fmt': 'mp3'}, 'appleapp_premium_medium': {'rate': 64, 'fmt': 'aac'}, 'appleapp_premium': {'rate': 128, 'fmt': 'aac'}, 'appleapp_premium_high': {'rate': 256, 'fmt': 'mp3'}, 'public1': {'rate': 40, 'fmt': 'aac'}, 'public5': {'rate': 40, 'fmt': 'wma'}, 'public3': {'rate': 96, 'fmt': 'mp3'}, 'premium_low': {'rate': 40, 'fmt': 'aac'}, 'premium_medium': {'rate': 64, 'fmt': 'aac'}, 'premium': {'rate': 128, 'fmt': 'aac'}, 'premium_high': {'rate': 256, 'fmt': 'mp3'} } }, 'JazzRadio': { 'desc': 'Jazz Radio', 'domain': 'jazzradio.com', 'streams': {'appleapp_low': {'rate': 40, 'fmt': 'aac'}, 'appleapp': {'rate': 64, 'fmt': 'aac'}, 'appleapp_premium_medium': {'rate': 64, 'fmt': 'aac'}, 'appleapp_premium': {'rate': 128, 'fmt': 'aac'}, 'appleapp_premium_high': {'rate': 256, 'fmt': 'mp3'}, 'public1': {'rate': 40, 'fmt': 'aac'}, 'public3': {'rate': 64, 'fmt': 'mp3'}, 'premium_low': {'rate': 40, 'fmt': 'aac'}, 'premium_medium': {'rate': 64, 'fmt': 'aac'}, 'premium': {'rate': 128, 'fmt': 'aac'}, 'premium_high': {'rate': 256, 'fmt': 'mp3'} } }, 'RockRadio': { 'desc': 'Rock Radio', 'domain': 'rockradio.com', 'streams': {'android_low': {'rate': 40, 'fmt': 'aac'}, 'android': {'rate': 64, 'fmt': 'aac'}, 'android_premium_medium': {'rate': 64, 'fmt': 'aac'}, 'android_premium': {'rate': 128, 'fmt': 'aac'}, 'android_premium_high': {'rate': 256, 'fmt': 'mp3'}, 'public3': {'rate': 96, 'fmt': 'mp3'} } }, 'FrescaRadio': { 'desc': 'Fresca Radio', 'domain': 'frescaradio.com', 'streams': { 'public3': {'rate': 96, 'fmt': 'mp3'} } } } CONFIG = BackendConfig(Value('networks', label='Selected Networks [%s](space separated)' % ' '.join(NETWORKS.keys()), default=''), Value('quality', label='Radio streaming quality', choices={'h': 'high', 'l': 'low'}, default='h') ) def __init__(self, *a, **kw): super(AudioAddictModule, self).__init__(*a, **kw) self.RADIOS = {} self.HISTORY = {} def _get_tracks_history(self, network): self._fetch_radio_list(network) domain = self.NETWORKS[network]['domain'] url = 'http://api.audioaddict.com/v1/%s/track_history' %\ (domain[:domain.rfind('.')]) self.HISTORY[network] = self.browser.location(url) return self.HISTORY def create_default_browser(self): return self.create_browser(parser='json') def _get_stream_name(self, network, quality): streamName = 'public3' for name in self.NETWORKS[network]['streams'].keys(): if name.startswith('public') and \ self.NETWORKS[network]['streams'][name]['rate'] >= 64: if quality == 'h': streamName = name break else: if quality == 'l': streamName = name break return streamName def _fetch_radio_list(self, network=None): quality = self.config['quality'].get() for selectedNetwork in self.config['networks'].get().split(): if network is None or network == selectedNetwork: streamName = self._get_stream_name(selectedNetwork, quality) if not self.RADIOS: self.RADIOS = {} if selectedNetwork not in self.RADIOS: document = self.browser.location('http://listen.%s/%s' % (self.NETWORKS[selectedNetwork]['domain'], streamName)) self.RADIOS[selectedNetwork] = {} for info in document: radio = info['key'] self.RADIOS[selectedNetwork][radio] = {} self.RADIOS[selectedNetwork][radio]['id'] = info['id'] self.RADIOS[selectedNetwork][radio]['name'] = info['name'] self.RADIOS[selectedNetwork][radio]['playlist'] = info['playlist'] return self.RADIOS def iter_radios_search(self, pattern): self._fetch_radio_list() pattern = pattern.lower() for network in self.config['networks'].get().split(): for radio in self.RADIOS[network]: radio_dict = self.RADIOS[network][radio] if pattern in radio_dict['name'].lower() : yield self.get_radio(radio+"."+network) def iter_resources(self, objs, split_path): self._fetch_radio_list() if Radio in objs: for network in self.config['networks'].get().split(): if split_path == [network]: for radio in self.RADIOS[network]: yield self.get_radio(radio+"."+network) return for network in self.config['networks'].get().split(): yield Collection([network], self.NETWORKS[network]['desc']) def get_current(self, network, radio): channel = {} if network not in self.HISTORY: self._get_tracks_history(network) channel = self.HISTORY[network].get(str(self.RADIOS[network][radio]['id'])) else: now=time.time() channel = self.HISTORY[network].get(str(self.RADIOS[network][radio]['id'])) if channel is None: return 'Unknown', 'Unknown' if (channel.get('started')+channel.get('duration')) < now: self._get_tracks_history(network) channel=self.HISTORY[network].get(str(self.RADIOS[network][radio]['id'])) artist = u'' + (channel.get('artist', '') or 'Unknown') title = u''+(channel.get('title', '') or 'Unknown') if artist == 'Unknown': track = u'' + (channel.get('track', '') or 'Unknown') if track != 'Unknown': artist = track[:track.find(' - ')] return artist, title def get_radio(self, radio): if not isinstance(radio, Radio): radio = Radio(radio) radioName, network = radio.id.split('.', 1) self._fetch_radio_list(network) if radioName not in self.RADIOS[network]: return None radio_dict = self.RADIOS[network][radioName] radio.title = radio_dict['name'] radio.description = radio_dict['name'] artist, title = self.get_current(network, radioName) current = StreamInfo(0) current.who = artist current.what = title radio.current = current radio.streams = [] defaultname = self._get_stream_name(network, self.config['quality'].get()) stream = BaseAudioStream(0) stream.bitrate = self.NETWORKS[network]['streams'][defaultname]['rate'] stream.format = self.NETWORKS[network]['streams'][defaultname]['fmt'] stream.title = u'%s %skbps' % (stream.format, stream.bitrate) stream.url = 'http://listen.%s/%s/%s.pls' %\ (self.NETWORKS[network]['domain'], defaultname, radioName) radio.streams.append(stream) i = 1 for name in self.NETWORKS[network]['streams'].keys(): if name == defaultname: continue stream = BaseAudioStream(i) stream.bitrate = self.NETWORKS[network]['streams'][name]['rate'] stream.format = self.NETWORKS[network]['streams'][name]['fmt'] stream.title = u'%s %skbps' % (stream.format, stream.bitrate) stream.url = 'http://listen.%s/%s/%s.pls'%\ (self.NETWORKS[network]['domain'], name, radioName) radio.streams.append(stream) i = i + 1 return radio def fill_radio(self, radio, fields): if 'current' in fields: radioName, network = radio.id.split('.', 1) radio.current = StreamInfo(0) radio.current.who, radio.current.what = self.get_current(network, radioName) return radio OBJECTS = {Radio: fill_radio} weboob-1.2/modules/audioaddict/test.py000066400000000000000000000026521303450110500201010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.radio import Radio class AudioAddictTest(BackendTest): MODULE = 'audioaddict' def test_audioaddict(self): ls = list(self.backend.iter_resources((Radio, ), [])) self.assertTrue(len(ls) > 0) search = list(self.backend.iter_radios_search('classic')) self.assertTrue(len(search) > 0) radio = self.backend.get_radio('classicrock.RockRadio') self.assertTrue(radio.title) self.assertTrue(radio.description) self.assertTrue(radio.current.who) self.assertTrue(radio.current.what) self.assertTrue(radio.streams[0].url) self.assertTrue(radio.streams[0].title) weboob-1.2/modules/aum/000077500000000000000000000000001303450110500150535ustar00rootroot00000000000000weboob-1.2/modules/aum/API.txt000066400000000000000000001354441303450110500162400ustar00rootroot00000000000000 Adopte un Mec API ------------------ Constants: APIKEY = fb0123456789abcd URL = http://api.adopteunmec.com/api.php ME Commands =========== me.login --------- Parameters: - login - pass Errors: - 1.1.1 : invalid login Return value: {u'errors': [], u'result': {u'baskets': u'384', u'events': {u'lastBasket': {u'alert': u'1', u'birthday': u'1989-02-04', u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'id': u'14471939', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 13:56:36', u'list5': u'0', u'login': u'kloo', u'mod_level': u'0', u'path': u'9/3/9/1/7/4/4/', u'pseudo': u'Kloolloo', u'region': u'11', u'sex': 1, u'shard': 9, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14471939/Kloolloo', u'zip': u'75001'}, u'lastChat': {u'alert': u'1', u'birthday': u'1987-10-10', u'city': u'Paris 18e Arrondissement', u'country': u'fr', u'cover': u'0', u'id': u'13280274', u'isBan': False, u'isOnline': False, u'last_cnx': u'2010-11-20 03:37:00', u'list5': u'0', u'login': u'#c1a63bda81cc03ccdf080ca6e003919e', u'mod_level': u'0', u'path': u'4/7/2/0/8/2/3/', u'pseudo': u'Katie Lee', u'region': u'11', u'sex': 1, u'shard': 4, u'style': u'0', u'table': u'adopteun.girls_coma', u'url': u'/api.php?member/view/13280274/KatieLee', u'zip': u'75018'}, u'lastFlash': {u'alert': u'1', u'birthday': u'1983-12-20', u'city': u'Longjumeau', u'country': u'fr', u'cover': u'1', u'id': u'13883475', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 18:52:13', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'5/7/4/3/8/8/3/', u'pseudo': u'Pas Tjs Sage', u'region': u'11', u'sex': 1, u'shard': 5, u'style': u'2', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13883475/PasTjsSage', u'zip': u'91160'}, u'lastMail': {u'alert': u'1', u'birthday': u'1989-04-28', u'city': u'Paris 8e Arrondissement', u'country': u'fr', u'cover': u'5', u'id': u'13268738', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 19:24:14', u'list5': u'0', u'login': u'@13268738', u'mod_level': u'0', u'path': u'8/3/7/8/6/2/3/', u'pseudo': u'Th\xe9na', u'region': u'11', u'sex': 1, u'shard': 8, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13268738/Thna', u'zip': u'75008'}, u'lastVisit': {u'alert': u'1', u'birthday': u'1988-02-27', u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'id': u'14477637', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 18:31:00', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'7/3/6/7/7/4/4/', u'pseudo': u'Lolly', u'region': u'11', u'sex': 1, u'shard': 7, u'style': u'4', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14477637/Lolly', u'zip': u'75005'}}, u'flashs': 10, u'mails': u'731', u'me': {u'about1': u"Je n'ai pas de temps à perdre, je n'ai ni MSN ni Facebook, je considère qu'on apprécie davantage la discussion face à face autour d'un verre que dans les yeux de son écran.\r
\r
Bon et ne venez que si vous avez quelque chose à me dire, je n'envoie jamais de charmes.", u'about2': u'', u'admin': u'0', u'alert': u'0', u'alert_add': u'6', u'birthday': u'1986-08-13', u'books': u"Orwell (1984, La ferme des animaux)
Barjavel (La nuit des temps, Le voyageur imprudent, Ravage, \x85)
Boris Vian (J'irai cracher sur vos tombes, L'écume des jours\x85)
Bukowski, Desproges, San Antonio
Sartre, Le Canard", u'cat': u'1', u'checks1': u'0', u'checks2': u'16686', u'checks3': u'0', u'checks4': u'0', u'checks5': u'0', u'checks6': u'2', u'checks7': u'0', u'cinema': u'Le Grand Détournement \x97 La Classe Américaine
V pour Vendetta, Pulp Fiction, The Truman Show
Eternal Sunshine of the Spotless Mind, Match Point
Idiocracy, The Big Lebowski, La cité de la peur
Sin City, Orange Mecanique, Buffet Froid, L', u'city': u'Paris', u'country': u'fr', u'cover': u'5', u'drink': u'2', u'email': u'tesiruna@parano.me', u'eyes': u'3', u'f': u'', u'first_cnx': u'2010-05-16 09:13:53', u'first_ip': u'81.57.125.104', u'food': u'1', u'godfather': u'0', u'hair_color': u'5', u'hair_size': u'3', u'hobbies': u'', u'id': u'22450639', u'img_count': u'5', u'isBan': False, u'isOnline': True, u'job': u'Dieu', u'last_chat': u'-0001-11-29 23:09:21', u'last_cnx': u'2011-09-20 19:24:25', u'last_ip': u'88.161.27.232', u'lat': u'48.861961', u'latR': u'0.852802098431', u'list1': u'2', u'list2': u'2', u'list3': u'2', u'list4': u'3', u'list5': u'0', u'list6': u'0', u'lng': u'2.33594', u'lngR': u'0.040769844129', u'login': u'@22450639', u'mod_level': u'1', u'music': u"Pink Floyd, Scorpions, Emperor
Metallica, Iron Maiden, Accept
Slash's Snakepit, Queen, Deep Purple
Led Zeppelin, Rolling Stones
Brassens, Souchon, Brel, Vian", u'origins': u'1', u'pass': u'8f3fa83cec9a243ae53c1337d2b5e1cf', u'path': u'9/3/6/0/5/4/2/', u'phone': u'-', u'pictures': [{u'file': u'5', u'height': u'427', u'id': u'7315391', u'md5': u'859fcd2e425617c33c16d6a1bc510ad3', u'member': u'22450639', u'rank': u'1', u'valid': u'1578737', u'width': u'500'}], u'pseudo': u'Nazification', u'region': u'11', u'sex': 0, u'shape': u'1', u'shard': 9, u'size': u'175', u'smoke': u'2', u'style': u'0', u'subregion': u'76', u'table': u'adopteun.boys', u'texts1': u'', u'texts2': u'', u'texts3': u'', u'texts4': u'', u'texts5': u'', u'texts6': u'', u'title': u'', u'tvs': u'Je ne possède pas la TV



', u'url': u'/api.php?member/view/22450639/Nazification', u'validated': True, u'visites': u'0', u'w': u'', u'warn': u'0', u'weight': u'55', u'zip': u'75000'}, u'news': {u'newBaskets': 3, u'newMails': 1, u'newVisits': 113}, u'popu': u'71540', u'subMobile': False, u'subWebsite': False, u'token': u'ea7bac8837f6e5bb1e2a3267d6a08b32e388d593', u'visites': u'958'}} me.[default] ------------ Return value: {u'errors': [], u'result': {u'events': {u'lastBasket': {u'alert': u'1', u'birthday': u'1989-02-04', u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'id': u'14471939', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 13:56:36', u'list5': u'0', u'login': u'kloo', u'mod_level': u'0', u'path': u'9/3/9/1/7/4/4/', u'pseudo': u'Kloolloo', u'region': u'11', u'sex': 1, u'shard': 9, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14471939/Kloolloo', u'zip': u'75001'}, u'lastChat': {u'alert': u'1', u'birthday': u'1987-10-10', u'city': u'Paris 18e Arrondissement', u'country': u'fr', u'cover': u'0', u'id': u'13280274', u'isBan': False, u'isOnline': False, u'last_cnx': u'2010-11-20 03:37:00', u'list5': u'0', u'login': u'#c1a63bda81cc03ccdf080ca6e003919e', u'mod_level': u'0', u'path': u'4/7/2/0/8/2/3/', u'pseudo': u'Katie Lee', u'region': u'11', u'sex': 1, u'shard': 4, u'style': u'0', u'table': u'adopteun.girls_coma', u'url': u'/api.php?member/view/13280274/KatieLee', u'zip': u'75018'}, u'lastFlash': {u'alert': u'1', u'birthday': u'1983-12-20', u'city': u'Longjumeau', u'country': u'fr', u'cover': u'1', u'id': u'13883475', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 18:52:13', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'5/7/4/3/8/8/3/', u'pseudo': u'Pas Tjs Sage', u'region': u'11', u'sex': 1, u'shard': 5, u'style': u'2', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13883475/PasTjsSage', u'zip': u'91160'}, u'lastMail': {u'alert': u'1', u'birthday': u'1989-04-28', u'city': u'Paris 8e Arrondissement', u'country': u'fr', u'cover': u'5', u'id': u'13268738', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 19:47:28', u'list5': u'0', u'login': u'@13268738', u'mod_level': u'0', u'path': u'8/3/7/8/6/2/3/', u'pseudo': u'Th\xe9na', u'region': u'11', u'sex': 1, u'shard': 8, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13268738/Thna', u'zip': u'75008'}, u'lastVisit': {u'alert': u'1', u'birthday': u'1988-02-27', u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'id': u'14477637', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 19:09:00', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'7/3/6/7/7/4/4/', u'pseudo': u'Lolly', u'region': u'11', u'sex': 1, u'shard': 7, u'style': u'4', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14477637/Lolly', u'zip': u'75005'}}, u'news': {u'newBaskets': 0, u'newMails': 1, u'newVisits': 113}, u'token': u'9a97a03774c9f440e676c78f48794a7221a67285'}} me.basket ---------- Return value: {u'errors': [], u'popu': 71840, u'result': {u'basket': [{u'alert': u'1', u'birthday': u'1989-02-04', u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'date': u'2011-09-19 01:52:29', u'id': u'14471939', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 13:56:36', u'list5': u'0', u'login': u'kloo', u'mod_level': u'0', u'path': u'9/3/9/1/7/4/4/', u'pseudo': u'Kloolloo', u'region': u'11', u'sex': 1, u'shard': 9, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14471939/Kloolloo', u'zip': u'75001'}], u'inBasket': 57, u'token': u'ea7bac8837f6e5bb1e2a3267d6a08b32e388d593'}} me.flashs --------- Return value: {u'errors': [], u'result': {u'all': [{u'date': u'2011-10-19 10:50:43', u'fid': u'41311269', u'id': u'12656592', u'member': {u'alert': u'1', u'birthday': u'1986-08-08', u'city': u'Armes', u'country': u'fr', u'cover': u'3', u'id': u'12656592', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-10-19 17:28:00', u'list5': u'0', u'login': u'@12656592', u'mod_level': u'0', u'path': u'2/9/5/6/5/6/2/', u'pseudo': u'Yayanne', u'region': u'5', u'sex': 1, u'shard': 2, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/12656592/Yayanne', u'zip': u'58500'}, u'seen': u'2011-10-19 10:54:09'}], u'count': 1467, u'news': [], u'olds': [], u'popu': u'110350', u'token': u'3af90dd563431fe6b4c65930d18337c997fac34e'}} me.visits --------- Return value: {u'errors': [], u'result': {u'count': u'607', u'news': [], u'offset': 0, u'olds': [{u'alert': u'0', u'birthday': u'1985-01-29', u'city': u'Albi', u'country': u'fr', u'cover': u'11', u'date': u'2011-10-19 17:40:43', u'id': u'13461054', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-10-19 17:47:25', u'list5': u'0', u'login': u'@13461054', u'mod_level': u'0', u'path': u'4/5/0/1/6/4/3/', u'pseudo': u'Bruume', u'region': u'15', u'seen': u'2011-10-19 17:42:55', u'sex': 1, u'shard': 4, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13461054/Bruume', u'vid': u'199226074', u'zip': u'81000'}], u'popu': u'110350', u'token': u'd7380871794008c94971acec82b7b679dd800307'}} MESSAGE Commands ================ message.[default] ----------------- Arguments: - P=, Return value: {u'errors': [], u'result': {u'count': 1, u'threads': [{u'cat': u'0', u'date': u'2011-09-20 19:22:11', u'id': u'11132125', u'id_from': u'13268738', u'id_to': u'22450639', u'member': {u'alert': u'1', u'birthday': u'1989-04-28', u'city': u'Paris 8e Arrondissement', u'country': u'fr', u'cover': u'5', u'id': u'13268738', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-20 19:54:15', u'list5': u'0', u'login': u'@13268738', u'mod_level': u'0', u'path': u'8/3/7/8/6/2/3/', u'pseudo': u'Th\xe9na', u'region': u'11', u'sex': 1, u'shard': 8, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13268738/Thna', u'zip': u'75008'}, u'message': u'0', u'status': u'0', u'title': u"J'aime bien les \xe9l\xe9phants. Toi ?"}], u'token': u'0f70c31bc6d05d45bee64e6f2eab9b537640f2f8'}} message.thread -------------- Parameters: - memberId - count Return value: {u'result': {u'popu': u'15910', u'thread': {u'isNew': False, u'member': {u'alert': u'3', u'birthday': u'1987-04-22', u'city': u'Maisons-Alfort', u'country': u'fr', u'cover': u'6', u'id': u'14022243', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 18:18:00', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'3/4/2/2/2/0/4/', u'pseudo': u'Sophkipeut', u'region': u'11', u'sex': 1, u'shard': 3, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14022243/Sophkipeut', u'zip': u'94700'}, u'messages': [{u'date': u'2011-09-19 17:52:03', u'id': u'46583458', u'id_from': u'14022243', u'id_to': u'23185402', u'message': u"Lol, moi j'ai fait attention, mais bon ça n'empêche pas son bidou, ceci dit c'est planqué par ses loooooooooooong poils ^^", u'src': u'', u'title': u"Lol, moi j'ai fait attention, mais bon \xe7a n'emp\xea..."}, {u'date': u'2011-09-19 17:49:39', u'id': u'47467748', u'id_from': u'23185402', u'id_to': u'14022243', u'message': u"Ah oui, justement le vétérinaire m'avait dit après la castration de Futex qu'il fallait faire attention à son poids, du coup ça m'a tellement vexé que j'ai fais attention au point qu'il est sans doute même trop maigre.", u'src': u'', u'title': u"Ah oui, justement le v\xe9t\xe9rinaire m'avait dit apr\xe8..."}], u'remoteStatus': u'2', u'status': u'1', u'warning': 0}, u'token': u'dbeccf96256d4f11991626707881fdba28f54d73'}} message.new ----------- Parameters: - memberId - message Return value: {u'errors': [], u'result': {u'thread': {u'isNew': False, u'member': {u'alert': u'1', u'birthday': u'1986-12-08', u'city': u'Rosny-sous-Bois', u'country': u'fr', u'cover': u'6', u'id': u'11099536', u'isBan': False, u'isOnline': False, u'last_cnx': u'2011-09-20 16:26:32', u'list5': u'0', u'login': u'@11099536', u'mod_level': u'0', u'path': u'6/3/5/9/9/0/1/', u'pseudo': u'Debo', u'region': u'11', u'sex': 1, u'shard': 6, u'style': u'0', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/11099536/Debo', u'zip': u'93110'}, u'messages': [{u'date': u'2011-09-20 20:09:07', u'id': u'46588573', u'id_from': u'23185402', u'id_to': u'11099536', u'message': u'Coucou', u'src': u'iphone', u'title': u'Coucou'}, {u'date': u'2011-09-20 16:27:34', u'id': u'46638469', u'id_from': u'11099536', u'id_to': u'23185402', u'message': u"Coucou pour tout t'avouer je ne m'y etais pas connecté depuis septembre ! un peu moins de boulot alors j'y traine !!\r\n\r\nEt toi ça va? tu devais pas partir au canada? tu es deja revenu.?", u'src': u'', u'title': u"Coucou pour tout t'avouer je ne m'y etais pas co..."}], u'remoteStatus': u'0', u'status': u'2', u'warning': 0}, u'token': u'2491f8ccb6741ceee2a461dc523939796904a0fa'}} message.delete -------------- Parameters: - id_user Return Value: Unknown MEMBER Commands =============== member.view ----------- Parameters: - id Return value: {u'errors': [], u'result': {u'member': {u'about1': u"comment te dire.. j'ai autant envie te laisser boire dans ma bouteille que mettre ma langue dans ta bouche !", u'about2': u"LES BISOUS C'EST BIEN LES BIJOUX C'EST MIEUX!\r
\r
Et l'humour encore plus, mouhaha\r
\r
PS: Si vous êtes le sosie de Pharell Williams, adoptez moi ;)", u'admin': u'0', u'alert': u'1', u'alert_add': u'0', u'birthday': u'1991-05-31', u'books': u"L'Analphabète - Rendell
Etat limite - Assouline
Marie-Antoinette - Zweig

", u'cat': u'2', u'checks1': u'2', u'checks2': u'1588', u'checks3': u'0', u'checks4': u'0', u'checks5': u'64', u'checks6': u'192', u'checks7': u'0', u'cinema': u"My Blueberry Night
Shinning - L'exorciste - Ester - Gothika
How High ! - Requiem for a dream
Remember Me - trainspotting
He gots game - Buffet froid", u'city': u'Paris', u'country': u'fr', u'cover': u'1', u'drink': u'2', u'eyes': u'5', u'f': u'', u'first_cnx': u'2011-09-17 00:18:59', u'first_ip': u'82.120.134.233', u'food': u'3', u'godfather': u'0', u'hair_color': u'4', u'hair_size': u'3', u'hobbies': u'Le théâtre définitivement ! et le sport !', u'id': u'14465370', u'img_count': u'6', u'isBan': False, u'isOnline': False, u'job': u'etudiante', u'last_chat': u'0000-00-00 00:00:00', u'last_cnx': u'2011-09-24 00:09:29', u'last_ip': u'92.151.177.164', u'lat': u'48.8814', u'latR': u'0.853141372984', u'list1': u'0', u'list2': u'42', u'list3': u'0', u'list4': u'0', u'list5': u'0', u'list6': u'0', u'lng': u'2.3365', u'lngR': u'0.0407796179728', u'login': u'@', u'mailable': True, u'mod_level': u'0', u'music': u'Daft Punk - Bloody Beetrots - Kid Cudi - Jamiroquai
Red Hot - Ray Charles - BEP - Gorillaz - Birdy nam nam
Citizen Cope - Angus & Julia Stone - Portishead
50cent - Sia - Ben Harper - Busta Rhymes
Musiques de gansta ! ET Debussy', u'origins': u'1', u'path': u'0/7/3/5/6/4/4/', u'phone': u'-', u'popu': {u'bonus': u'6', u'contacts': u'1', u'flashs': u'246', u'id': u'14465370', u'invits': u'0', u'mails': u'39', u'popu': u'11345', u'visites': u'645'}, u'pseudo': u'Ruslana', u'region': u'11', u'sex': 1, u'shape': u'1', u'shard': 0, u'size': u'170', u'smoke': u'2', u'style': u'4', u'subregion': u'76', u'table': u'adopteun.girls', u'texts1': u'', u'texts2': u'', u'texts3': u'', u'texts4': u'', u'texts5': u'', u'texts6': u'MON SOURIRE HAHAHA', u'title': u'', u'tvs': u'Dexter
OC
True blood
Envoyé spécial ;) - Arte !
', u'url': u'/api.php?member/view/14465370/Ruslana', u'visites': u'0', u'w': u'', u'warn': u'0', u'weight': u'50', u'zip': u'75008'}, u'token': u'e0247704012e01bc32756b357b010e5206ac9c76'}} member.pictures --------------- Parameters: - id Return value: {u'errors': [], u'result': {u'pictures': [{u'id': u'12363004', u'rating': 4.4473684210500002, u'url': u'http://s0.adopteunmec.com/0/6/0/1/6/image7.jpg'}], u'token': u'e131f1b194f2a19337882398b10b79457a638252'}} member.addBasket ---------------- Parameters: - id Errors: - 5.1.1 : member does not exist - 5.1.5 : already sent charm to this one - 5.1.6 : no enough charms available Return value: {u'errors': u'0', u'flashs': 4, u'result': {u'token': u'55039d0557393bb7c5e4381792143d003f0e60c0'}} SEARH Commands ============== search.[default] ---------------- Return value: {u'errors': [], u'result': {u'qsearch': {u'ageMax': u'25', u'ageMin': u'18', u'dist': u'0', u'new': u'0', u'query': u'{"sex":1,"ageMin":"18","ageMax":"25","region":"fr","new":"0","dist":"0"}', u'region': u'fr', u'sex': 1}, u'search': {u'ageMax': u'27', u'ageMin': u'20', u'checks1': u'0', u'checks2': u'0', u'country': u'fr', u'dist': u'50', u'drink': u'0', u'eyes': u'0', u'food': u'0', u'hair_color': u'0', u'hair_size': u'0', u'origins': u'0', u'pseudo': u'', u'query': u'{"ageMin":"20","ageMax":"27","country":"fr","region":"11","subregion":"0","dist":"50","pseudo":"","sex":"1","sizeMin":"0","sizeMax":"0","weightMin":"0","weightMax":"75","shape":"0","hair_size":"0","hair_color":"0","eyes":"0","origins":"0","style":"0","checks1":"0","checks2":"0","smoke":"0","drink":"0","food":"0","search":"true"}', u'region': u'11', u'search': u'true', u'sex': u'1', u'shape': u'0', u'sizeMax': u'0', u'sizeMin': u'0', u'smoke': u'0', u'style': u'0', u'subregion': u'0', u'weightMax': u'75', u'weightMin': u'0'}, u'token': u'3196a3365d927f2ee8738ec8dfc4a5abd75e3ee3'}} search.quick ------------ Parameters: - sex (int[0,1]) - ageMin (int) - ageMax (int) - region (str) - new (int) - dist (int) Return Value: {u'errors': [], u'result': {u'regions': {u'be': None, u'be_24': u' - wallonie', u'be_25': u' - bruxelles capitale', u'be_26': u' - flandre', u'ca': None, u'ca_34': u' - canada', u'ca_35': u' - quebec', u'ch': None, u'ch_27': u' - r\xe9gion l\xe9manique', u'ch_28': u' - espace Mittelland', u'ch_29': u' - suisse du nord-ouest', u'ch_30': u' - zurich', u'ch_31': u' - suisse orientale', u'ch_32': u' - suisse centrale', u'ch_33': u' - tessin', u'fr': None, u'fr_1': u' - alsace', u'fr_10': u' - haute-normandie', u'fr_11': u' - ile-de-france', u'fr_12': u' - languedoc-roussillon', u'fr_13': u' - limousin', u'fr_14': u' - lorraine', u'fr_15': u' - midi-pyr\xe9n\xe9es', u'fr_16': u' - nord-pas-de-calais', u'fr_17': u' - paca', u'fr_18': u' - pays de la loire', u'fr_19': u' - picardie', u'fr_2': u' - aquitaine', u'fr_20': u' - poitou-charentes', u'fr_21': u' - rh\xf4ne-alpes', u'fr_22': u' - corse', u'fr_23': u' - dom+tom', u'fr_3': u' - auvergne', u'fr_4': u' - basse-normandie', u'fr_5': u' - bourgogne', u'fr_6': u' - bretagne', u'fr_7': u' - centre', u'fr_8': u' - champagne-ardenne', u'fr_9': u' - franche-comt\xe9'}, u'search': [{u'alert': u'2', u'birthday': u'1985-11-21', u'city': u'Boulogne-Billancourt', u'country': u'fr', u'cover': u'12', u'id': u'14252744', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-24 15:19:48', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'4/4/7/2/5/2/4/', u'pseudo': u'Birdy', u'region': u'11', u'sex': 1, u'shard': 4, u'style': u'5', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/14252744/Birdy', u'zip': u'92100'}], u'token': u'ef78ac6d812ab15f2f8efd578f4da4ef2e23aa71'}} search.advanced --------------- Parameters: - ageMin (int) - ageMax (int) - country (str) - region (int) - subregion (int) - dist (int) - pseudo (str) - sex (int[0,1]) - sizeMin (int) - sizeMax (int) - weightMin (int) - weightMax (int) - shape (int) - hair_size (int) - hair_color (int) - eyes (int) - origins (int) - style (int) - checks1 (int) - checks2 (int) - smoke (int) - drink (int) - food (int) - search (bool) Return Value: {u'errors': [], u'regions': {u'be': None, u'be_24': u' - wallonie', u'be_25': u' - bruxelles capitale', u'be_26': u' - flandre', u'ca': None, u'ca_34': u' - canada', u'ca_35': u' - quebec', u'ch': None, u'ch_27': u' - r\xe9gion l\xe9manique', u'ch_28': u' - espace Mittelland', u'ch_29': u' - suisse du nord-ouest', u'ch_30': u' - zurich', u'ch_31': u' - suisse orientale', u'ch_32': u' - suisse centrale', u'ch_33': u' - tessin', u'fr': None, u'fr_1': u' - alsace', u'fr_10': u' - haute-normandie', u'fr_11': u' - ile-de-france', u'fr_12': u' - languedoc-roussillon', u'fr_13': u' - limousin', u'fr_14': u' - lorraine', u'fr_15': u' - midi-pyr\xe9n\xe9es', u'fr_16': u' - nord-pas-de-calais', u'fr_17': u' - paca', u'fr_18': u' - pays de la loire', u'fr_19': u' - picardie', u'fr_2': u' - aquitaine', u'fr_20': u' - poitou-charentes', u'fr_21': u' - rh\xf4ne-alpes', u'fr_22': u' - corse', u'fr_23': u' - dom+tom', u'fr_3': u' - auvergne', u'fr_4': u' - basse-normandie', u'fr_5': u' - bourgogne', u'fr_6': u' - bretagne', u'fr_7': u' - centre', u'fr_8': u' - champagne-ardenne', u'fr_9': u' - franche-comt\xe9'}, u'result': {u'search': [{u'alert': u'1', u'birthday': u'1988-04-07', u'city': u'Dammartin', u'country': u'fr', u'cover': u'25', u'id': u'13579115', u'isBan': False, u'isOnline': True, u'last_cnx': u'2011-09-24 15:17:22', u'list5': u'0', u'login': u'@', u'mod_level': u'0', u'path': u'5/1/1/9/7/5/3/', u'pseudo': u"S\xe9 's\xe9", u'region': u'11', u'sex': 1, u'shard': 5, u'style': u'1', u'table': u'adopteun.girls', u'url': u'/api.php?member/view/13579115/Ss', u'zip': u'77230'}], u'token': u'f92aede46118dcdba6d484146b4627777fbe7188'}} weboob-1.2/modules/aum/__init__.py000066400000000000000000000015051303450110500171650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .browser import AuMBrowser from .module import AuMModule __all__ = ['AuMBrowser', 'AuMModule'] weboob-1.2/modules/aum/antispam.py000066400000000000000000000116611303450110500172460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re class AntiSpam(object): def check_thread(self, thread): resume = thread['title'] # Check if there is an email address in the offer. if re.match('^[\w\d\.\-_]+@[\w\d\.]+ vous offre la pos', resume): return False if thread['who']['pseudo'] == 'Ekaterina': return False return True def check_profile(self, profile): # The name of profile is in form #123456789 if profile['pseudo'] == '': return False if profile['announce'].startswith('salut! je te donne mon msn'): return False if profile['shopping_list'].startswith('cam to cam'): return False if profile['shopping_list'].startswith('je suis une femme tres tres belle et je recherche un homme qui aime le sexe'): return False if profile['shopping_list'].endswith('mmmmmmmmmmmmmmmm'): return False return True # ipaddr is not available anymore. for ipaddr in (profile['last_ip'], profile['first_ip']): if ipaddr.startswith('41.202.'): return False if ipaddr.startswith('41.250.'): return False if ipaddr.startswith('41.251.'): return False if ipaddr.startswith('41.141.'): return False if ipaddr.startswith('194.177.'): return False if ipaddr.startswith('41.85.'): return False if ipaddr.startswith('41.86.'): return False if ipaddr.startswith('196.47.'): return False if re.match('105\.13\d.*', ipaddr): return False if ipaddr in ('62.157.186.18', '198.36.222.8', '212.234.67.61', '203.193.158.210', '41.189.34.180', '41.66.12.36', '196.47.137.21', '213.136.125.122', '41.191.87.188'): return False return True def check_contact(self, contact): if contact.id == 1: return True if not self.check_profile(contact._aum_profile): return False return True # ipaddr is not available anymore. first_ip = contact.profile['info']['IPaddr'].value.split(' ')[0] last_ip = contact.profile['info']['IPaddr'].value.rstrip(')') for ipaddr in (first_ip, last_ip): if ipaddr.endswith('.afnet.net'): return False if ipaddr.endswith('.iam.net.ma'): return False if ipaddr.endswith('.amsterdam.ananoos.net'): return False if ipaddr.endswith('.tedata.net'): return False if ipaddr.endswith('kupo.fr'): return False if ipaddr.endswith('.static.virginmedia.com'): return False if ipaddr.endswith('frozenway.com'): return False if ipaddr.endswith('.rev.bgtn.net'): return False if ipaddr.endswith('real-vpn.com'): return False if ipaddr.endswith('.nl.ipodah.net'): return False if ipaddr.endswith('.wanamaroc.com'): return False if ipaddr.endswith('.ukservers.com'): return False if ipaddr.endswith('.startdedicated.com'): return False if ipaddr.endswith('.clients.your-server.de'): return False if ipaddr.endswith('.cba.embratel.net.br'): return False if ipaddr.endswith('.idstelcom.com'): return False if ipaddr.endswith('proxy.chg-support.com'): return False if ipaddr.endswith('.sprintsvc.net'): return False if ipaddr.endswith('.relakks.com'): return False return True def check_mail(self, mail): # Spambot with a long first-message. if mail['message'] is None: return True if mail['message'].find('Je veux que vous m\'ayez ecrit directement sur le mon e-mail') >= 0: return False if mail['message'].find('ilusa12010@live.fr') >= 0: return False return True weboob-1.2/modules/aum/browser.py000066400000000000000000000333401303450110500171130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2008-2011 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from base64 import b64encode from hashlib import sha256 from datetime import datetime import math import re import urllib import urllib2 from weboob.exceptions import BrowserIncorrectPassword, BrowserHTTPNotFound, BrowserUnavailable from weboob.deprecated.browser import Browser from weboob.browser.browsers import LoginBrowser from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import CleanText from weboob.tools.json import json from weboob.tools.date import local2utc from weboob.tools.misc import to_unicode from weboob.capabilities.base import UserError from weboob.capabilities.messages import CantSendMessage __all__ = ['AuMBrowser'] class AuMException(UserError): ERRORS = {"0.0.0": "Bad signature", "0.0.1": "Malformed request", "0.0.2": "Not logged", "1.1.1": "No member has this login", "1.1.2": "Password don't match", "1.1.3": "User has been banned", "1.12.1": "Invalid country or region", # "1.12.1": "Invalid region", "4.0.1": "Member not found", "4.1.1": "Thread doesn't exist", "4.1.2": "Cannot write to this member", "5.1.1": "Member tergeted doesn't exist", "5.1.2": "Sex member targeted is not the opposite of the member logged", "5.1.3": "Not possible to send a charm", "5.1.4": "Not possible to send a charm because the 5 charms has been already used", "5.1.5": "Not possible because the guy has already send a charm to this girl", "5.1.6": "No more money", "5.1.7": "Not possible to add to basket", "5.2.1": "Member doesn't exist", "5.3.1": "Member doesn't exist", } def __init__(self, code): Exception.__init__(self, self.ERRORS.get(code, code)) self.code = code class WebsiteBrowser(LoginBrowser): BASEURL = 'https://www.adopteunmec.com' VERIFY = False TIMEOUT = 3.0 def login(self): data = {'username': self.username, 'password': self.password, 'remember': 'on', } self.open('/auth/login', data=data) #self.readurl('https://www.adopteunmec.com/auth/login', urllib.urlencode(data)) def get_profile(self, id): profile = {} if datetime.now().hour >= 18 or datetime.now().hour < 1: return profile r = None try: r = self.open('https://www.adopteunmec.com/profile/%s' % id) except BrowserUnavailable: pass if r is None or not re.match('https://www.adopteunmec.com/profile/\d+', r.url): self.login() try: r = self.open('https://www.adopteunmec.com/profile/%s' % id) except BrowserUnavailable: r = None if r is None: return {} page = HTMLPage(self, r) doc = page.doc profile['popu'] = {} for tr in doc.xpath('//div[@id="popularity"]//tr'): cols = tr.findall('td') if not cols[0].text: continue key = CleanText('./th')(tr).strip().lower() value = int(re.sub(u'[^0-9]+', u'', cols[0].text).strip()) profile['popu'][key] = value for script in doc.xpath('//script'): text = script.text if text is None: continue m = re.search("'memberLat'\s*:\s*([\-\d\.]+),", text, re.IGNORECASE) if m: profile['lat'] = float(m.group(1)) m = re.search("'memberLng'\s*:\s*([\-\d\.]+),", text, re.IGNORECASE) if m: profile['lng'] = float(m.group(1)) return profile class AuMBrowser(Browser): DOMAIN = 'www.adopteunmec.com' APIKEY = 'fb0123456789abcd' APITOKEN = 'DCh7Se53v8ejS8466dQe63' APIVERSION = '2.2.5' USER_AGENT = 'Mozilla/5.0 (Linux; U; Android4.1.1; fr_FR; GT-N7100; Build/JRO03C) com.adopteunmec.androidfr/17' GIRL_PROXY = None consts = None my_sex = 0 my_id = 0 my_name = u'' my_coords = (0,0) def __init__(self, username, password, search_query, *args, **kwargs): kwargs['get_home'] = False Browser.__init__(self, username, password, *args, **kwargs) # now we do authentication ourselves #self.add_password('https://www.adopteunmec.com/api/', self.username, self.password) self.login() kwargs.pop('get_home') self.website = WebsiteBrowser(self.username, self.password, *args, **kwargs) self.website.login() self.home() self.search_query = search_query def id2url(self, id): return u'https://www.adopteunmec.com/profile/%s' % id def url2id(func): def inner(self, id, *args, **kwargs): m = re.match('^https?://.*adopteunmec.com.*/(\d+)$', str(id)) if m: id = int(m.group(1)) else: m = re.match('^https?://.*adopteunmec.com/(index.php/)?profile/(\d+).*', str(id)) if m: id = int(m.group(2)) return func(self, id, *args, **kwargs) return inner def api0_request(self, command, action, parameter='', data=None, nologin=False): if data is None: # Always do POST requests. data = '' elif isinstance(data, (list,tuple,dict)): data = urllib.urlencode(data) elif isinstance(data, unicode): data = data.encode('utf-8') url = self.buildurl('http://api.adopteunmec.com/api.php', S=self.APIKEY, C=command, A=action, P=parameter, O='json') buf = self.openurl(url, data).read() try: r = json.loads(buf[buf.find('{'):]) except ValueError: raise ValueError(buf) if 'errors' in r and r['errors'] != '0' and len(r['errors']) > 0: code = r['errors'][0] if code in (u'0.0.2', u'1.1.1', u'1.1.2'): if not nologin: self.login() return self.api0_request(command, action, parameter, data, nologin=True) else: raise BrowserIncorrectPassword(AuMException.ERRORS[code]) else: raise AuMException(code) return r def login(self): self.api_request('applications/android') # XXX old API is disabled #r = self.api0_request('me', 'login', data={'login': self.username, # 'pass': self.password, # }, nologin=True) #self.my_coords = (float(r['result']['me']['lat']), float(r['result']['me']['lng'])) #if not self.search_query: # self.search_query = 'region=%s' % r['result']['me']['region'] def api_request(self, command, **kwargs): if 'data' in kwargs: data = to_unicode(kwargs.pop('data')).encode('utf-8', 'replace') else: data = None headers = {} if not command.startswith('applications'): today = local2utc(datetime.now()).strftime('%Y-%m-%d') token = sha256(self.username + self.APITOKEN + today).hexdigest() headers['Authorization'] = 'Basic %s' % (b64encode('%s:%s' % (self.username, self.password))) headers['X-Platform'] = 'android' headers['X-Client-Version'] = self.APIVERSION headers['X-AUM-Token'] = token url = self.buildurl(self.absurl('/api/%s' % command), **kwargs) if isinstance(url, unicode): url = url.encode('utf-8') req = self.request_class(url, data, headers) buf = self.openurl(req).read() try: r = json.loads(buf) except ValueError: raise ValueError(buf) return r def get_exception(self, e): if isinstance(e, urllib2.HTTPError) and hasattr(e, 'getcode'): if e.getcode() in (410,): return BrowserHTTPNotFound return Browser.get_exception(self, e) def home(self): r = self.api_request('home/') self.my_sex = r['user']['sex'] self.my_id = int(r['user']['id']) self.my_name = r['user']['pseudo'] if self.my_coords == (0,0): profile = self.get_full_profile(self.my_id) if 'lat' in profile and 'lng' in profile: self.my_coords = [profile['lat'], profile['lng']] return r def get_consts(self): if self.consts is not None: return self.consts self.consts = [{}, {}] for key, sexes in self.api_request('values').iteritems(): for sex, values in sexes.iteritems(): if sex in ('boy', 'both'): self.consts[0][key] = values if sex in ('girl', 'both'): self.consts[1][key] = values return self.consts def score(self): r = self.home() return int(r['user']['points']) def get_my_name(self): return self.my_name def get_my_id(self): return self.my_id def nb_new_mails(self): r = self.home() return r['counters']['new_mails'] def nb_new_baskets(self): r = self.home() return r['counters']['new_baskets'] def nb_new_visites(self): r = self.home() return r['counters']['new_visits'] def nb_available_charms(self): r = self.home() return r['subscription']['flashes_stock'] def get_baskets(self): r = self.api_request('basket', count=30, offset=0) return r['results'] def get_flashs(self): r = self.api_request('charms/', count=30, offset=0) return r['results'] def get_visits(self): r = self.api_request('visits', count=30, offset=0) return r['results'] def get_threads_list(self, count=30): r = self.api_request('threads', count=count, offset=0) return r['results'] @url2id def get_thread_mails(self, id, count=30): r = self.api_request('threads/%s' % id, count=count, offset=0) return r @url2id def post_mail(self, id, content): content = content.replace('\n', '\r\n') try: self.api_request('threads/%s' % id, data=content) except BrowserHTTPNotFound: raise CantSendMessage('Unable to send message.') @url2id def delete_thread(self, id): r = self.api_request('message', 'delete', data={'id_user': id}) self.logger.debug('Thread deleted: %r' % r) @url2id def send_charm(self, id): try: self.api_request('users/%s/charms' % id, data='') except BrowserHTTPNotFound: return False else: return True @url2id def add_basket(self, id): try: self.api_request('basket/%s' % id, data='') except BrowserHTTPNotFound: return False else: return True def search_profiles(self, **kwargs): if not self.search_query: # retrieve query self.login() r = self.api_request('users?count=100&offset=0&%s' % (self.search_query % {'lat': self.my_coords[0], 'lng': self.my_coords[1]})) ids = [s['id'] for s in r['results']] return set(ids) @url2id def get_full_profile(self, id): if self.GIRL_PROXY is not None: res = self.openurl(self.GIRL_PROXY % id) profile = json.load(res) if 'lat' in profile and 'lng' in profile: profile['dist'] = self.get_dist(profile['lat'], profile['lng']) else: profile = self.get_profile(id) return profile def get_dist(self, lat, lng): coords = (float(lat), float(lng)) R = 6371 lat1 = math.radians(self.my_coords[0]) lat2 = math.radians(coords[0]) lon1 = math.radians(self.my_coords[1]) lon2 = math.radians(coords[1]) dLat = lat2 - lat1 dLong = lon2 - lon1 a= pow(math.sin(dLat/2), 2) + math.cos(lat1) * math.cos(lat2) * pow(math.sin(dLong/2), 2) c= 2 * math.atan2(math.sqrt(a), math.sqrt(1-a)) return R * c @url2id def get_profile(self, id): # XXX OLD API IS DISABLED (fucking faggots) #r = self.api0_request('member', 'view', data={'id': id}) #if not 'result' in r: # print r #profile = r['result']['member'] profile = {} profile.update(self.api_request('users/%s' % id)) profile.update(self.website.get_profile(id)) # Calculate distance in km. profile['dist'] = 0.0 if 'lat' in profile and 'lng' in profile: profile['dist'] = self.get_dist(profile['lat'], profile['lng']) return profile weboob-1.2/modules/aum/contact.py000066400000000000000000000266461303450110500170760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2008-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from html2text import unescape import socket from datetime import datetime from dateutil.parser import parse as parse_dt from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.contact import Contact as _Contact, ProfileNode from weboob.tools.html import html2text class FieldBase(object): def __init__(self, key, key2=None): self.key = key self.key2 = key2 def get_value(self, value, consts): raise NotImplementedError() class FieldStr(FieldBase): def get_value(self, profile, consts): return html2text(unicode(profile[self.key])).strip() class FieldBool(FieldBase): def get_value(self, profile, consts): return bool(int(profile[self.key])) class FieldDist(FieldBase): def get_value(self, profile, consts): return '%.2f km' % float(profile[self.key]) class FieldIP(FieldBase): def get_hostname(self, s): try: return socket.gethostbyaddr(s)[0] except (socket.gaierror, socket.herror): return s def get_value(self, profile, consts): s = self.get_hostname(profile[self.key]) if profile[self.key] != profile[self.key2]: s += ' (first %s)' % self.get_hostname(profile[self.key2]) return s class FieldProfileURL(FieldBase): def get_value(self, profile, consts): id = int(profile[self.key]) if id > 0: return 'http://www.adopteunmec.com/index.php/profile/%d' % id else: return '' class FieldPopu(FieldBase): def get_value(self, profile, consts): return unicode(profile['popu'][self.key]) class FieldPopuRatio(FieldBase): def get_value(self, profile, consts): v1 = float(profile['popu'][self.key]) v2 = float(profile['popu'][self.key2]) if v2 == 0.0: return 'NaN' else: return '%.2f' % (v1 / v2) class FieldOld(FieldBase): def get_value(self, profile, consts): birthday = parse_dt(profile[self.key]) return int((datetime.now() - birthday).days / 365.25) class FieldList(FieldBase): def get_value(self, profile, consts): return profile[self.key] class FieldBMI(FieldBase): def __init__(self, key, key2, fat=False): FieldBase.__init__(self, key, key2) self.fat = fat def get_value(self, profile, consts): height = int(profile[self.key]) weight = int(profile[self.key2]) if height == 0 or weight == 0: return '' bmi = (weight / float(pow(height / 100.0, 2))) if not self.fat: return bmi elif bmi < 15.5: return 'severely underweight' elif bmi < 18.4: return 'underweight' elif bmi < 24.9: return 'normal' elif bmi < 30: return 'overweight' else: return 'obese' class FieldConst(FieldBase): def get_value(self, profile, consts): v = profile[self.key] if isinstance(v, (basestring,int)): try: return consts[self.key][str(v)] except KeyError: return '' elif isinstance(v, (tuple,list)): labels = [] for i in v: labels.append(consts[self.key][i]) return labels class Contact(_Contact): TABLE = OrderedDict(( ('_info', OrderedDict(( ('title', FieldStr('title')), # ipaddr is not available anymore. #('IPaddr', FieldIP('last_ip', 'first_ip')), ('admin', FieldBool('admin')), ('ban', FieldBool('isBan')), ('first', FieldStr('first_cnx')), ('godfather', FieldProfileURL('godfather')), ))), ('_stats', OrderedDict(( ('mails', FieldPopu('mails')), ('charms', FieldPopu('charmes')), ('visites', FieldPopu('visites')), ('baskets', FieldPopu('panier')), ('invits', FieldPopu('invits')), ('bonus', FieldPopu('bonus')), ('score', FieldStr('points')), ('ratio', FieldPopuRatio('mails', 'charmes')), ('mailable', FieldBool('can_mail')), ))), ('details', OrderedDict(( #('old', FieldStr('age')), ('old', FieldOld('birthdate')), ('birthday', FieldStr('birthdate')), ('zipcode', FieldStr('zip')), ('location', FieldStr('city')), ('distance', FieldDist('dist')), ('country', FieldStr('country')), ('phone', FieldStr('phone')), ('eyes', FieldConst('eyes_color')), ('hair_color', FieldConst('hair_color')), ('hair_size', FieldConst('hair_size')), ('height', FieldConst('size')), ('weight', FieldConst('weight')), ('BMI', FieldBMI('size', 'weight')), ('fat', FieldBMI('size', 'weight', fat=True)), ('shape', FieldConst('shape')), ('origins', FieldConst('origins')), ('signs', FieldConst('features')), ('job', FieldStr('job')), ('style', FieldConst('styles')), ('food', FieldConst('diet')), ('favorite_food', FieldConst('favourite_food')), ('drink', FieldConst('alcohol')), ('smoke', FieldConst('tobacco')), ))), ('tastes', OrderedDict(( ('hobbies', FieldStr('hobbies')), ('music', FieldList('music')), ('cinema', FieldList('cinema')), ('books', FieldList('books')), ('tv', FieldList('tvs')), ))), ('+sex', OrderedDict(( ('underwear', FieldConst('underwear')), ('practices', FieldConst('sexgames')), ('favorite', FieldConst('arousing')), ('toys', FieldConst('sextoys')), ))), ('+personality', OrderedDict(( ('snap', FieldStr('fall_for')), ('exciting', FieldStr('turned_on_by')), ('hate', FieldStr('cant_stand')), ('vices', FieldStr('vices')), ('assets', FieldStr('assets')), ('fantasies', FieldStr('fantasies')), ('is', FieldConst('character')), ))), ('-personality', OrderedDict(( ('accessories', FieldConst('accessories')), ('skills', FieldConst('skills')), ('socios', FieldConst('socios')), ('family', FieldConst('family')), ('pets', FieldConst('pets')), ))) )) def parse_profile(self, profile, consts): if profile['online']: self.status = Contact.STATUS_ONLINE self.status_msg = u'online' self.status_msg = u'since %s' % profile['last_cnx'] else: self.status = Contact.STATUS_OFFLINE self.status_msg = u'last connection %s' % profile['last_cnx'] self.summary = unicode(unescape(profile.get('announce', '').strip())) if len(profile.get('shopping_list', '')) > 0: self.summary += u'\n\nLooking for:\n%s' % unescape(profile['shopping_list'].strip()) for photo in profile['pics']: self.set_photo(photo.split('/')[-1], url=photo + '/full', thumbnail_url=photo + '/small', hidden=False) self.profile = OrderedDict() if 'sex' in profile: for section, d in self.TABLE.iteritems(): flags = ProfileNode.SECTION if section.startswith('_'): flags |= ProfileNode.HEAD if (section.startswith('+') and int(profile['sex']) != 1) or \ (section.startswith('-') and int(profile['sex']) != 0): continue section = section.lstrip('_+-') s = ProfileNode(section, section.capitalize(), OrderedDict(), flags=flags) for key, builder in d.iteritems(): try: value = builder.get_value(profile, consts[int(profile['sex'])]) except KeyError: pass else: s.value[key] = ProfileNode(key, key.capitalize().replace('_', ' '), value) self.profile[section] = s self._aum_profile = profile weboob-1.2/modules/aum/favicon.png000066400000000000000000000042171303450110500172120ustar00rootroot00000000000000PNG  IHDR@@% sRGB pHYs  tIME 2!IDAThip]=!ᒅbFAE.P):J]AjVjGFvZ3.R U[VK$D,I{O?2%عoy{s"("(Te|D.[)EtV4I?2 +׊eL'vI?R)FP\0BB.l0AiSN~E0:5KC+O8x>[y}!jpNqx2έ>Y98=eO&n) oްoFĸ!Aq$~=~BOM_gdZnKa1Eb?{GC!QSeS|7ϩq>*3lL0Ɖx-|O^Ԓ6E[dzV>*=z/zTbxj.ng:4@߫kS:)D]&dؕ!K3IJB/h><B$`+;\M f!(-G72A= ǹ^䒶F.rOGUԣc\9]|v'ʹ"| iJuc>UR }?- (7Ҧ3eWPx!yI ![= n$H UCKe:w%HO?D>(KJ!s7,$~f}7uOC͔.чmp)%`|ZUAdqIO|W($o::hoH%mx9/{^dcʷ'Ħ^ZƓER% 80

z `w|w|2|؊6D4<[?,g;,^.zϙn;V@GwOiL@d _2}%׼ x#D_f$LêA]\]/s'蚁T*FM,PF]xB}9?f&`D6!u:ЫI$R5[KǫH*th ڷXO\utljlJF/uvIsth"d0wR FE#yJdQڟFw%sr$Njtݍag("("(_y*lIENDB`weboob-1.2/modules/aum/module.py000066400000000000000000000461241303450110500167210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time import datetime from html2text import unescape from dateutil import tz from dateutil.parser import parse as _parse_dt from weboob.capabilities.base import NotLoaded from weboob.capabilities.chat import CapChat from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message, Thread from weboob.capabilities.dating import CapDating, OptimizationNotFound, Event from weboob.capabilities.contact import CapContact, ContactPhoto, Query, QueryError from weboob.capabilities.account import CapAccount, StatusField from weboob.tools.backend import Module, BackendConfig from weboob.deprecated.browser import BrowserUnavailable, BrowserHTTPNotFound from weboob.tools.value import Value, ValueBool, ValueBackendPassword from weboob.tools.date import local2utc from weboob.tools.misc import to_unicode from .contact import Contact from .antispam import AntiSpam from .browser import AuMBrowser from .optim.profiles_walker import ProfilesWalker from .optim.visibility import Visibility from .optim.queries_queue import QueriesQueue __all__ = ['AuMModule'] def parse_dt(s): d = _parse_dt(s) return local2utc(d) class AuMModule(Module, CapMessages, CapMessagesPost, CapDating, CapChat, CapContact, CapAccount): NAME = 'aum' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'"Adopte un Mec" French dating website' CONFIG = BackendConfig(Value('username', label='Username'), ValueBackendPassword('password', label='Password'), ValueBool('antispam', label='Enable anti-spam', default=False), ValueBool('baskets', label='Get baskets with new messages', default=True), Value('search_query', label='Search query', default='')) STORAGE = {'profiles_walker': {'viewed': []}, 'queries_queue': {'queue': []}, 'sluts': {}, 'notes': {}, } BROWSER = AuMBrowser MAGIC_ID_BASKET = 1 def __init__(self, *args, **kwargs): Module.__init__(self, *args, **kwargs) if self.config['antispam'].get(): self.antispam = AntiSpam() else: self.antispam = None def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get(), self.config['search_query'].get()) def report_spam(self, id): with self.browser: pass #self.browser.delete_thread(id) # Do not report fakes to website, to let them to other guys :) #self.browser.report_fake(id) # ---- CapDating methods --------------------- def init_optimizations(self): self.add_optimization('PROFILE_WALKER', ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)) self.add_optimization('VISIBILITY', Visibility(self.weboob.scheduler, self.browser)) self.add_optimization('QUERIES_QUEUE', QueriesQueue(self.weboob.scheduler, self.storage, self.browser)) def iter_events(self): all_events = {} with self.browser: all_events[u'baskets'] = (self.browser.get_baskets, 'You were put into %s\'s basket') all_events[u'flashs'] = (self.browser.get_flashs, 'You sent a charm to %s') all_events[u'visits'] = (self.browser.get_visits, 'Visited by %s') for type, (events, message) in all_events.iteritems(): for event in events(): e = Event(event['who']['id']) e.date = parse_dt(event['date']) e.type = type if 'who' in event: e.contact = self._get_partial_contact(event['who']) else: e.contact = self._get_partial_contact(event) if not e.contact: continue e.message = message % e.contact.name yield e def iter_new_contacts(self): with self.browser: for _id in self.browser.search_profiles():#.difference(self.OPTIM_PROFILE_WALKER.visited_profiles): contact = Contact(_id, '', 0) yield contact # ---- CapMessages methods --------------------- def fill_thread(self, thread, fields): return self.get_thread(thread) def iter_threads(self): with self.browser: threads = self.browser.get_threads_list() for thread in threads: #if thread['member'].get('isBan', thread['member'].get('dead', False)): # with self.browser: # self.browser.delete_thread(thread['member']['id']) # continue if self.antispam and not self.antispam.check_thread(thread): self.logger.info('Skipped a spam-thread from %s' % thread['pseudo']) self.report_spam(thread['who']['id']) continue t = Thread(int(thread['who']['id'])) t.flags = Thread.IS_DISCUSSION t.title = u'Discussion with %s' % to_unicode(thread['who']['pseudo']) yield t def get_thread(self, id, contacts=None, get_profiles=False): """ Get a thread and its messages. The 'contacts' parameters is only used for internal calls. """ thread = None if isinstance(id, Thread): thread = id id = thread.id if not thread: thread = Thread(int(id)) thread.flags = Thread.IS_DISCUSSION full = False else: full = True with self.browser: mails = self.browser.get_thread_mails(id, 100) my_name = self.browser.get_my_name() child = None msg = None slut = self._get_slut(id) if contacts is None: contacts = {} if not thread.title: thread.title = u'Discussion with %s' % mails['who']['pseudo'] self.storage.set('sluts', int(thread.id), 'status', mails['status']) self.storage.save() for mail in mails['results']: flags = 0 if self.antispam and not self.antispam.check_mail(mail): self.logger.info('Skipped a spam-mail from %s' % mails['who']['pseudo']) self.report_spam(thread.id) break if parse_dt(mail['date']) > slut['lastmsg']: flags |= Message.IS_UNREAD if get_profiles: if mail['from'] not in contacts: try: with self.browser: contacts[mail['from']] = self.get_contact(mail['from']) except BrowserHTTPNotFound: pass if self.antispam and mail['from'] in contacts and not self.antispam.check_contact(contacts[mail['from']]): self.logger.info('Skipped a spam-mail-profile from %s' % mails['who']['pseudo']) self.report_spam(thread.id) break if int(mail['from']) == self.browser.my_id: if mails['remote_status'] == 'new' and msg is None: flags |= Message.IS_NOT_RECEIVED else: flags |= Message.IS_RECEIVED signature = u'' #if mail.get('src', None): # signature += u'Sent from my %s\n\n' % mail['src'] if mail['from'] in contacts: signature += contacts[mail['from']].get_text() msg = Message(thread=thread, id=int(time.strftime('%Y%m%d%H%M%S', parse_dt(mail['date']).timetuple())), title=thread.title, sender=to_unicode(my_name if int(mail['from']) == self.browser.my_id else mails['who']['pseudo']), receivers=[to_unicode(my_name if int(mail['from']) != self.browser.my_id else mails['who']['pseudo'])], date=parse_dt(mail['date']), content=to_unicode(unescape(mail['message'] or '').strip()), signature=signature, children=[], flags=flags) if child: msg.children.append(child) child.parent = msg child = msg if full and msg: # If we have get all the messages, replace NotLoaded with None as # parent. msg.parent = None if not full and not msg: # Perhaps there are hidden messages msg = NotLoaded thread.root = msg return thread def iter_unread_messages(self): try: contacts = {} with self.browser: threads = self.browser.get_threads_list() for thread in threads: #if thread['member'].get('isBan', thread['member'].get('dead', False)): # with self.browser: # self.browser.delete_thread(int(thread['member']['id'])) # continue if self.antispam and not self.antispam.check_thread(thread): self.logger.info('Skipped a spam-unread-thread from %s' % thread['who']['pseudo']) self.report_spam(thread['member']['id']) continue slut = self._get_slut(thread['who']['id']) if parse_dt(thread['date']) > slut['lastmsg'] or thread['status'] != slut['status']: try: t = self.get_thread(thread['who']['id'], contacts, get_profiles=True) except BrowserUnavailable: continue for m in t.iter_all_messages(): if m.flags & m.IS_UNREAD: yield m if not self.config['baskets'].get(): return # Send mail when someone added me in her basket. # XXX possibly race condition if a slut adds me in her basket # between the aum.nb_new_baskets() and aum.get_baskets(). with self.browser: slut = self._get_slut(-self.MAGIC_ID_BASKET) new_baskets = self.browser.nb_new_baskets() if new_baskets > 0: baskets = self.browser.get_baskets() my_name = self.browser.get_my_name() for basket in baskets: if parse_dt(basket['date']) <= slut['lastmsg']: continue contact = self.get_contact(basket['who']['id']) if self.antispam and not self.antispam.check_contact(contact): self.logger.info('Skipped a spam-basket from %s' % contact.name) self.report_spam(basket['who']['id']) continue thread = Thread(int(basket['who']['id'])) thread.title = 'Basket of %s' % contact.name thread.root = Message(thread=thread, id=self.MAGIC_ID_BASKET, title=thread.title, sender=contact.name, receivers=[my_name], date=parse_dt(basket['date']), content='You are taken in her basket!', signature=contact.get_text(), children=[], flags=Message.IS_UNREAD) yield thread.root except BrowserUnavailable as e: self.logger.debug('No messages, browser is unavailable: %s' % e) pass # don't care about waiting def set_message_read(self, message): if int(message.id) == self.MAGIC_ID_BASKET: # Save the last baskets checks. slut = self._get_slut(-self.MAGIC_ID_BASKET) if slut['lastmsg'] < message.date: slut['lastmsg'] = message.date self.storage.set('sluts', -self.MAGIC_ID_BASKET, slut) self.storage.save() return slut = self._get_slut(message.thread.id) if slut['lastmsg'] < message.date: slut['lastmsg'] = message.date self.storage.set('sluts', int(message.thread.id), slut) self.storage.save() def _get_slut(self, id): id = int(id) sluts = self.storage.get('sluts') if not sluts or id not in sluts: slut = {'lastmsg': datetime.datetime(1970,1,1), 'status': None} else: slut = self.storage.get('sluts', id) slut['lastmsg'] = slut.get('lastmsg', datetime.datetime(1970,1,1)).replace(tzinfo=tz.tzutc()) slut['status'] = slut.get('status', None) return slut # ---- CapMessagesPost methods --------------------- def post_message(self, message): with self.browser: self.browser.post_mail(message.thread.id, message.content) # ---- CapContact methods --------------------- def fill_contact(self, contact, fields): if 'profile' in fields: contact = self.get_contact(contact) if contact and 'photos' in fields: for name, photo in contact.photos.iteritems(): with self.browser: if photo.url and not photo.data: data = self.browser.openurl(photo.url).read() contact.set_photo(name, data=data) if photo.thumbnail_url and not photo.thumbnail_data: data = self.browser.openurl(photo.thumbnail_url).read() contact.set_photo(name, thumbnail_data=data) def fill_photo(self, photo, fields): with self.browser: if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.readurl(photo.url) if 'thumbnail_data' in fields and photo.thumbnail_url and not photo.thumbnail_data: photo.thumbnail_data = self.browser.readurl(photo.thumbnail_url) return photo def get_contact(self, contact): with self.browser: if isinstance(contact, Contact): _id = contact.id elif isinstance(contact, (int,long,basestring)): _id = contact else: raise TypeError("The parameter 'contact' isn't a contact nor a int/long/str/unicode: %s" % contact) profile = self.browser.get_full_profile(_id) if not profile: return None _id = profile['id'] if isinstance(contact, Contact): contact.id = _id contact.name = profile['pseudo'] else: contact = Contact(_id, profile['pseudo'], Contact.STATUS_ONLINE) contact.url = self.browser.id2url(_id) contact.parse_profile(profile, self.browser.get_consts()) return contact def _get_partial_contact(self, contact): s = 0 if contact.get('online', False): s = Contact.STATUS_ONLINE else: s = Contact.STATUS_OFFLINE c = Contact(contact['id'], to_unicode(contact['pseudo']), s) c.url = self.browser.id2url(contact['id']) if 'age' in contact: c.status_msg = u'%s old, %s' % (contact['age'], contact['city']) if contact['cover'] is not None: url = contact['cover'] + '/%(type)s' else: url = u'http://s.adopteunmec.com/www/img/thumb0.jpg' c.set_photo(u'image%s' % contact['cover'], url=url % {'type': 'full'}, thumbnail_url=url % {'type': 'small'}) return c def iter_contacts(self, status=Contact.STATUS_ALL, ids=None): with self.browser: threads = self.browser.get_threads_list(count=100) for thread in threads: c = self._get_partial_contact(thread['who']) if c and (c.status & status) and (not ids or c.id in ids): yield c def send_query(self, id): if isinstance(id, Contact): id = id.id queries_queue = None try: queries_queue = self.get_optimization('QUERIES_QUEUE') except OptimizationNotFound: pass if queries_queue and queries_queue.is_running(): if queries_queue.enqueue_query(id): return Query(id, 'A charm has been sent') else: return Query(id, 'Unable to send charm: it has been enqueued') else: with self.browser: if not self.browser.send_charm(id): raise QueryError('No enough charms available') return Query(id, 'A charm has been sent') def get_notes(self, id): if isinstance(id, Contact): id = id.id return self.storage.get('notes', id) def save_notes(self, id, notes): if isinstance(id, Contact): id = id.id self.storage.set('notes', id, notes) self.storage.save() # ---- CapChat methods --------------------- def iter_chat_messages(self, _id=None): with self.browser: return self.browser.iter_chat_messages(_id) def send_chat_message(self, _id, message): with self.browser: return self.browser.send_chat_message(_id, message) #def start_chat_polling(self): #self._profile_walker = ProfilesWalker(self.weboob.scheduler, self.storage, self.browser) def get_account_status(self): with self.browser: return ( StatusField(u'myname', u'My name', unicode(self.browser.get_my_name())), StatusField(u'score', u'Score', unicode(self.browser.score())), StatusField(u'avcharms', u'Available charms', unicode(self.browser.nb_available_charms())), StatusField(u'newvisits', u'New visits', unicode(self.browser.nb_new_visites())), ) OBJECTS = {Thread: fill_thread, Contact: fill_contact, ContactPhoto: fill_photo } weboob-1.2/modules/aum/optim/000077500000000000000000000000001303450110500162035ustar00rootroot00000000000000weboob-1.2/modules/aum/optim/__init__.py000066400000000000000000000000001303450110500203020ustar00rootroot00000000000000weboob-1.2/modules/aum/optim/profiles_walker.py000066400000000000000000000072271303450110500217550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from random import randint from weboob.deprecated.browser import BrowserUnavailable from weboob.capabilities.dating import Optimization from weboob.tools.log import getLogger class ProfilesWalker(Optimization): def __init__(self, sched, storage, browser): super(ProfilesWalker, self).__init__() self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('walker', browser.logger) self._walk_cron = None self._view_cron = None self._visited_profiles = set(storage.get('profiles_walker', 'viewed')) self._logger.info(u'Loaded %d already visited profiles from storage.' % len(self._visited_profiles)) self._profiles_queue = set() def save(self): self._storage.set('profiles_walker', 'viewed', list(self._visited_profiles)) self._storage.save() def start(self): self._walk_cron = self._sched.repeat(60, self.enqueue_profiles) self._view_cron = self._sched.schedule(randint(5, 10), self.view_profile) return True def stop(self): self._sched.cancel(self._walk_cron) self._sched.cancel(self._view_cron) self._walk_cron = None self._view_cron = None return True def is_running(self): return self._walk_cron is not None def enqueue_profiles(self): try: with self._browser: profiles_to_visit = self._browser.search_profiles().difference(self._visited_profiles) self._logger.info(u'Enqueuing profiles to visit: %s' % profiles_to_visit) self._profiles_queue = set(profiles_to_visit) self.save() except BrowserUnavailable: return def view_profile(self): try: try: id = self._profiles_queue.pop() except KeyError: return # empty queue try: with self._browser: profile = self._browser.get_profile(id) self._logger.info(u'Visited profile %s (%s)' % (profile['pseudo'], id)) # Get score from the aum_score module #d = self.nucentral_core.callService(context.Context.fromComponent(self), 'aum_score', 'score', profile) # d.addCallback(self.score_cb, profile.getID()) # deferredlist.append(d) # do not forget that we visited this profile, to avoid re-visiting it. self._visited_profiles.add(id) self.save() except BrowserUnavailable: # We consider this profil hasn't been [correctly] analysed self._profiles_queue.add(id) return except Exception as e: self._logger.exception(e) finally: if self._view_cron is not None: self._view_cron = self._sched.schedule(randint(5, 10), self.view_profile) weboob-1.2/modules/aum/optim/queries_queue.py000066400000000000000000000065601303450110500214450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import BrowserUnavailable from weboob.capabilities.dating import Optimization from weboob.capabilities.contact import QueryError from weboob.tools.log import getLogger class QueriesQueue(Optimization): def __init__(self, sched, storage, browser): super(QueriesQueue, self).__init__() self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('queriesqueue', browser.logger) self._queue = storage.get('queries_queue', 'queue', default=[]) self._check_cron = None def save(self): self._storage.set('queries_queue', 'queue', self._queue) self._storage.save() def start(self): self._check_cron = self._sched.repeat(3600, self.flush_queue) return True def stop(self): self._sched.cancel(self._check_cron) self._check_cron = None return True def is_running(self): return self._check_cron is not None def enqueue_query(self, id, priority=999): id_queue = [_id[1] for _id in self._queue] if int(id) in id_queue: raise QueryError('This id is already queued') self._queue.append((int(priority), int(id))) self.save() # Try to flush queue to send it now. self.flush_queue() # Check if the enqueued query has been sent for _, i in self._queue: if i == int(id): return False return True def flush_queue(self): self._queue.sort() priority = 0 id = None try: try: while len(self._queue) > 0: priority, id = self._queue.pop() if not id: continue with self._browser: if self._browser.send_charm(id): self._logger.info('Charm sent to %s', id) else: self._queue.append((priority, id)) self._logger.info("Charm can't be send to %s", id) break # As the charm has been correctly sent (no exception raised), # we don't store anymore ID, because if nbAvailableCharms() # fails, we don't want to re-queue this ID. id = None priority = 0 except BrowserUnavailable: # We consider this profil hasn't been [correctly] analysed if id is not None: self._queue.append((priority, id)) finally: self.save() weboob-1.2/modules/aum/optim/visibility.py000066400000000000000000000027141303450110500207500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import BrowserUnavailable from weboob.capabilities.dating import Optimization class Visibility(Optimization): def __init__(self, sched, browser): super(Visibility, self).__init__() self._sched = sched self._browser = browser self._cron = None def start(self): self._cron = self._sched.repeat(60*5, self.reconnect) return True def stop(self): self._sched.cancel(self._cron) self._cron = None return True def is_running(self): return self._cron is not None def reconnect(self): try: with self._browser: self._browser.login() except BrowserUnavailable: pass weboob-1.2/modules/aum/test.py000066400000000000000000000027341303450110500164120ustar00rootroot00000000000000# -*- CODing: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.deprecated.browser import BrowserUnavailable class AuMTest(BackendTest): MODULE = 'aum' def test_new_messages(self): try: for message in self.backend.iter_unread_messages(): pass except BrowserUnavailable: # enough frequent to do not care about. pass def test_contacts(self): try: contacts = list(self.backend.iter_contacts()) if len(contacts) == 0: # so bad, we can't test that... return self.backend.fillobj(contacts[0], ['photos', 'profile']) except BrowserUnavailable: # enough frequent to do not care about. pass weboob-1.2/modules/axabanque/000077500000000000000000000000001303450110500162365ustar00rootroot00000000000000weboob-1.2/modules/axabanque/__init__.py000066400000000000000000000014361303450110500203530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import AXABanqueModule __all__ = ['AXABanqueModule'] weboob-1.2/modules/axabanque/browser.py000066400000000000000000000202211303450110500202700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.exceptions import ClientError from weboob.capabilities.base import NotAvailable from weboob.exceptions import BrowserIncorrectPassword, ActionNeeded from .pages import KeyboardPage, LoginPage, PredisconnectedPage, BankAccountsPage, \ InvestmentPage, CBTransactionsPage, TransactionsPage, UnavailablePage, \ IbanPage class AXABanque(LoginBrowser): BASEURL = 'https://www.axabanque.fr/' # Login keyboard = URL('https://connect.axa.fr/keyboard/password', KeyboardPage) login = URL('https://connect.axa.fr/api/identity/auth', LoginPage) predisconnected = URL('https://www.axa.fr/axa-predisconnect.html', 'https://www.axa.fr/axa-postmaw-predisconnect.html', PredisconnectedPage) # Bank bank_accounts = URL('transactionnel/client/liste-comptes.html', 'transactionnel/client/liste-(?P.*).html', 'webapp/axabanque/jsp/visionpatrimoniale/liste_panorama_.*\.faces', r'/webapp/axabanque/page\?code=(?P\d+)', 'webapp/axabanque/client/sso/connexion\?token=(?P.*)', BankAccountsPage) iban_pdf = URL('http://www.axabanque.fr/webapp/axabanque/formulaire_AXA_Banque/.*\.pdf.*', IbanPage) cbttransactions = URL('webapp/axabanque/jsp/detailCarteBleu.*.faces', CBTransactionsPage) transactions = URL('webapp/axabanque/jsp/panorama.faces', 'webapp/axabanque/jsp/visionpatrimoniale/panorama_.*\.faces', 'webapp/axabanque/jsp/detail.*.faces', 'webapp/axabanque/jsp/.*/detail.*.faces', TransactionsPage) unavailable = URL('login_errors/indisponibilite.*', '.*page-indisponible.html.*', '.*erreur/erreurBanque.faces', UnavailablePage) # Investment investment = URL('https://espaceclient.axa.fr', 'https://connexion.adis-assurances.com', InvestmentPage) investment_transactions = URL('https://espaceclient.axa.fr/accueil/savings/savings/contract/_jcr_content/' + \ 'par/savingsmovementscard.savingscard.pid_(?P.*).aid_(?P.*).html\?skip=(?P.*)') def __init__(self, *args, **kwargs): super(AXABanque, self).__init__(*args, **kwargs) self.tokens = {} # Need to cache every pages, website is too slow self.account_list = [] self.investment_list = {} self.history_list = {} def do_login(self): # due to the website change, login changed too, this is for don't try to login with the wrong login if len(self.username) > 7: raise ActionNeeded() if self.password.isdigit(): vk_passwd = self.keyboard.go().get_password(self.password) login_data = { 'email': self.username, 'password': vk_passwd, 'rememberIdenfiant': False, 'version': 1 } self.login.go(data=login_data) if not self.password.isdigit() or self.page.check_error(): raise BrowserIncorrectPassword() @need_login def iter_accounts(self): if not self.account_list: accounts = [] ids = set() # Get accounts self.transactions.go() self.bank_accounts.go() # Ugly 3 loops : nav through all tabs and pages for tab in self.page.get_tabs(): for page, page_args in self.bank_accounts.stay_or_go(tab=tab).get_pages(tab): for a in page.get_list(): if a.id in ids: # the "-comptes" page may return the same accounts as other pages, skip them continue ids.add(a.id) args = a._args # Trying to get IBAN for checking accounts if a.type == a.TYPE_CHECKING and 'paramCodeFamille' in args: iban_params = {'action': 'RIBCC', 'numCompte': args['paramNumCompte'], 'codeFamille': args['paramCodeFamille'], 'codeProduit': args['paramCodeProduit'], 'codeSousProduit': args['paramCodeSousProduit'] } try: r = self.open('/webapp/axabanque/popupPDF', params=iban_params) a.iban = r.page.get_iban() except ClientError: a.iban = NotAvailable # Need it to get accounts from tabs a._tab, a._pargs, a._purl = tab, page_args, self.url accounts.append(a) # Get investment accounts if there has self.investment.go() if self.investment.is_here(): for a in self.page.iter_accounts(): accounts.append(a) self.account_list = accounts return iter(self.account_list) @need_login def go_account_pages(self, account, action): # Default to "comptes" tab = "comptes" if not hasattr(account, '_tab') else account._tab self.bank_accounts.go(tab=tab) args = account._args args['javax.faces.ViewState'] = self.page.get_view_state() # Nav for accounts in tab pages if tab != "comptes" and hasattr(account, '_url') \ and hasattr(account, '_purl') and hasattr(account, '_pargs'): self.location(account._purl, data=account._pargs) self.location(account._url, data=args) # Check if we are on the good tab if isinstance(self.page, TransactionsPage): self.page.go_action(action) else: self.transactions.go(data=args) @need_login def iter_history(self, account): if account.id not in self.history_list: trs = [] # Side investment's website if account._acctype is "investment": skip = 0 try: while self.investment_transactions.go(pid=account.id.zfill(16), aid=account._aid, skip=skip): for a in self.page.iter_history(): trs.append(a) skip += 10 except AssertionError: # assertion error when page is empty pass # Main website withouth investments elif account._acctype is "bank" and not account._hasinv: self.go_account_pages(account, "history") if self.page.more_history(): trs = [a for a in self.page.get_history()] self.history_list[account.id] = trs return iter(self.history_list[account.id]) @need_login def iter_investment(self, account): if account.id not in self.investment_list: invs = [] if account._acctype is "bank" and account._hasinv: self.go_account_pages(account, "investment") invs = [i for i in self.page.iter_investment()] self.investment_list[account.id] = invs return iter(self.investment_list[account.id]) weboob-1.2/modules/axabanque/favicon.png000066400000000000000000000043451303450110500203770ustar00rootroot00000000000000PNG  IHDR@@iqbKGDC pHYs B(xtIME :.drIDATxyTɢ,hT(Ĉ=jc@p?u͉ъӜxĈZڪK$5J"&F1U4* 2 >cUYqdnּ SY5κ|1Ɉ ^,Nͬs*1ewu9?[uV՜erGGMƠgc/JCI:C/۔} Sy?BaL'WU,#=@œ}}ohws6 'wʮ|k#:)%hKٔNO!)ۊ&{#`~z 3bi5w)+*d\O[ .C@H_6ZB-lNGkU"D5S= F#IYe v\u 5oVhK KM"a,OeA~l̚<lDa ܌Xz)hYN\$`qZ4s3bQ'.yG8m|Ӣ {o'sٺuP뼦y\ŷ}FE;󩾷C0?=F륮Vk0&3ζN$uIB9[o}v%@cSK6ZhSF+DU^V˰  ܵLI{N֤|{KdnkI=# ydݠ.6D,<{ Ɨ6`+.!w}$o ǯ6",p%N%6W|qg2 G; ŧS\\Cwwcga{L>е߱o&5;`kY9T\yj z7T*Nj' #\]O'_( Ur8|EoX`oHX V.K>>΢F}M;tB.cĠ>F2<%mZhZoCZ&>}]Je7df?o~)bjA V|~R#J >C֥S|MF2Rߦ֡E J-Wy:>wٺr23_|L 9Hs W?ti Kd畘3=aIJe \MwI$e؋K// ~Z\-ly'glx7 ce> A룈KfG'`bUy\LD-neUETUoclzi{7CEQ9w Mˌ)hױxSr~Y"آϣ˰Шj'0ifm`m c?̙I{!hLQn㗱yIV߸";/1gK^{q>Q(Puhcnv {C==q$d/0~+kS#+}I\BqWu&&{ÑQjMٜ:V$l{;_oKNl~S!/0#.`s-b95Pԏ _qc9cTM`k&`A_M(}2qd⪨ ?gp++9)|jֿBmjeFctHw^U ~6Wq57KJLaΜX Wn=IxzHWbz&$mj<<*s$>}y%C$-y;@!sj`ũE&A}=65RZ|b_O>{s3H"7A#v40y~:WRJ{K3!(*-7 ~Zš_meizC| 3%I,:\xe3AZ͚پ&7T/7)DM8/qQR_S63 uQ ⓌݵxF3n3oMOxIENDB`weboob-1.2/modules/axabanque/module.py000066400000000000000000000036231303450110500201010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import AXABanque __all__ = ['AXABanqueModule'] class AXABanqueModule(Module, CapBank): NAME = 'axabanque' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'AXA Banque' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Code', regexp='\d+')) BROWSER = AXABanque def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_investment(self, account): return self.browser.iter_investment(account) def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/axabanque/pages.py000066400000000000000000000523711303450110500177170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from decimal import Decimal, InvalidOperation from cStringIO import StringIO from weboob.exceptions import BrowserBanned, BrowserUnavailable from weboob.browser.pages import HTMLPage, RawPage, JsonPage, PDFPage, LoggedPage from weboob.browser.elements import ItemElement, ListElement, TableElement, method from weboob.browser.filters.standard import CleanText, Date, CleanDecimal, Field, BrowserURL, \ TableCell, Async, AsyncLoad, Regexp from weboob.browser.filters.html import Attr from weboob.browser.filters.json import Dict from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.captcha.virtkeyboard import VirtKeyboard, VirtKeyboardError from weboob.tools.ordereddict import OrderedDict def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class MyVirtKeyboard(VirtKeyboard): margin = 5, 5, 5, 5 color = (255, 255, 255) symbols = {'0': '6959163af44cc50b3863e7e306d6e571', '1': '98b32dff471e903b6fa8e3a0f1544b17', '2': '32722d5b6572f9d46350aca7fb66263a', '3': '835a9c8bf66e28f3ffa2b12994bc3f9a', '4': 'e7457342c434da4fb0fd974f7dc37002', '5': 'c8b74429a805e12a08c5ed87fd9730ce', '6': '70a84c766bc323343c0c291146f652db', '7': 'e4e7fb4f8cc90c8ad472906b5eceeb99', '8': 'ffb78dbea5a171990e14d707d4772ba2', '9': '063dcb4179beaeff60fb73c80cbd429d' } coords = {'0': (0, 0, 40, 40), '1': (40, 0, 80, 40), '2': (80, 0, 120, 40), '3': (120, 0, 160, 40), '4': (0, 40, 40, 80), '5': (40, 40, 80, 80), '6': (80, 40, 120, 80), '7': (120, 40, 160, 80), '8': (0, 80, 40, 120), '9': (40, 80, 80, 120), '10': (80, 80, 120, 120), '11': (120, 80, 160, 120), '12': (0, 120, 40, 160), '13': (40, 120, 80, 160), '14': (80, 120, 120, 160), '15': (120, 120, 160, 160) } def __init__(self, page): VirtKeyboard.__init__(self, StringIO(page.content), self.coords, self.color, convert='RGB') self.check_symbols(self.symbols, None) def get_string_code(self, string): return ','.join(self.get_position_from_md5(self.symbols[c]) for c in string) def get_position_from_md5(self, md5): for k, v in self.md5.iteritems(): if v == md5: return k def check_color(self, pixel): return pixel[0] > 0 class KeyboardPage(RawPage): def get_password(self, password): vk_passwd = None try: vk = MyVirtKeyboard(self) vk_passwd = vk.get_string_code(password) except VirtKeyboardError as e: self.logger.error(e) return vk_passwd class LoginPage(JsonPage): def check_error(self): return (not Dict('errors')(self.doc)) is False class PredisconnectedPage(HTMLPage): def on_load(self): raise BrowserBanned() class InvestmentTransaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PVersement|versement.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(u'^(?P(Arbitrage|Prélèvement(s).*))'), FrenchTransaction.TYPE_ORDER), (re.compile(u'^(?PRetrait.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(u'^(?P.*)'), FrenchTransaction.TYPE_BANK), ] class InvestmentPage(LoggedPage, HTMLPage): TYPES = {u'Assurance vie Arpèges': Account.TYPE_LIFE_INSURANCE, 'Epargne Retraite PERP PAIR': Account.TYPE_SAVINGS} @method class iter_accounts(ListElement): item_xpath = '//section[has-class("cards") and has-class("contracts")]/article[has-class("card")]' class item(ItemElement): klass = Account obj_id = Regexp(CleanText('.//h2/small'), ur'(?:#|N°\s+)(.*)') obj_balance = MyDecimal('.//span[@class="card-amount"]', default=NotAvailable) obj_iban = NotAvailable obj_valuation_diff = MyDecimal('.//p[@class="card-description"]') obj_label = CleanText('.//h2/text()') obj__acctype = "investment" def obj__aid(self): return Regexp(Attr('//a[@class="menu-item" and @href="%s"]' % Attr('.', 'data-redirect')(self), 'data-target'), \ r'aid_(.*).html')(self) def obj_type(self): return self.page.TYPES.get(Field('label')(self), Account.TYPE_UNKNOWN) def condition(self): return Field('balance')(self) is not NotAvailable @method class iter_history(ListElement): item_xpath = "//div[has-class('t-data')]" class item(ItemElement): klass = InvestmentTransaction load_details = Field('_url') & AsyncLoad obj_label = CleanText('(.//div[has-class("t-data__label")])[1]') obj_raw = InvestmentTransaction.Raw(Field('label')) obj_date = Date(CleanText('(.//div[has-class("t-data__date")])[1]'), dayfirst=True) obj_amount = MyDecimal('(.//div[has-class("t-data__amount")])[1]') obj__hasinv = True def obj__url(self): return '%s%s' % (BrowserURL('investment')(self), Attr('.', 'data-url')(self)) def obj_investments(self): inv_page = Async('details').loaded_page(self) return list(inv_page.iter_investments()) @method class iter_investments(ListElement): item_xpath = "//div[@class='white-bg']/following-sibling::div[not(@*)]" class item(ItemElement): klass = Investment obj_label = CleanText('.//div[@class="t-data__label"]') obj_portfolio_share = MyDecimal('.//div[has-class("t-data__amount_label")]') obj_valuation = MyDecimal('.//div[has-class("t-data__amount") and has-class("desktop")]') class UnavailablePage(HTMLPage): def on_load(self): raise BrowserUnavailable() class MyHTMLPage(HTMLPage): def get_view_state(self): return self.doc.xpath('//input[@name="javax.faces.ViewState"]')[0].attrib['value'] def is_password_expired(self): return len(self.doc.xpath('//div[@id="popup_client_modifier_code_confidentiel"]')) def parse_number(self, number): # For some client they randomly displayed 4,115.00 and 4 115,00. # Browser is waiting for for 4 115,00 so we format the number to match this. if '.' in number and len(number.split('.')[-1]) == 2: return number.replace(',', ' ').replace('.', ',') return number def js2args(self, s): args = {} # For example: # noDoubleClic(this);;return oamSubmitForm('idPanorama','idPanorama:tableaux-comptes-courant-titre:0:tableaux-comptes-courant-titre-cartes:0:_idJsp321',null,[['paramCodeProduit','9'],['paramNumContrat','12234'],['paramNumCompte','12345678901'],['paramNumComptePassage','1234567890123456']]); for sub in re.findall("\['([^']+)','([^']+)'\]", s): args[sub[0]] = sub[1] sub = re.search('oamSubmitForm.+?,\'([^:]+).([^\']+)', s) args['%s:_idcl' % sub.group(1)] = "%s:%s" % (sub.group(1), sub.group(2)) args['%s_SUBMIT' % sub.group(1)] = 1 return args class BankAccountsPage(LoggedPage, MyHTMLPage): ACCOUNT_TYPES = OrderedDict(( ('visa', Account.TYPE_CARD), ('courant-titre', Account.TYPE_CHECKING), ('courant', Account.TYPE_CHECKING), ('livret', Account.TYPE_SAVINGS), ('ldd', Account.TYPE_SAVINGS), ('pel', Account.TYPE_SAVINGS), ('pea', Account.TYPE_MARKET), ('titres', Account.TYPE_MARKET), )) def get_tabs(self): links = self.doc.xpath('//strong[text()="Mes Comptes"]/following-sibling::ul//a/@href') links.insert(0, "-comptes") return list(set([re.findall('-([a-z]+)', x)[0] for x in links])) def has_accounts(self): return self.doc.xpath('//table[not(@id) and contains(@class, "table-produit")]') def get_pages(self, tab): pages = [] pages_args = [] if len(self.has_accounts()) == 0: table_xpath = '//table[contains(@id, "%s")]' % tab links = self.doc.xpath('%s//td[1]/a[@onclick and contains(@onclick, "noDoubleClic")]' % table_xpath) if len(links) > 0: form_xpath = '%s/ancestor::form[1]' % table_xpath form = self.get_form(form_xpath, submit='%s//input[1]' % form_xpath) data = {k: v for k, v in dict(form).items() if v} for link in links: d = self.js2args(link.attrib['onclick']) d.update(data) pages.append(self.browser.location(form.url, data=d).page) pages_args.append(d) else: pages.append(self) pages_args.append(None) return zip(pages, pages_args) def get_list(self): for table in self.has_accounts(): tds = table.xpath('./tbody/tr')[0].findall('td') if len(tds) < 3: continue boxes = table.xpath('./tbody//tr[not(.//strong[contains(text(), "Total")])]') foot = table.xpath('./tfoot//tr') for box in boxes: account = Account() if len(box.xpath('.//a')) != 0 and 'onclick' in box.xpath('.//a')[0].attrib: args = self.js2args(box.xpath('.//a')[0].attrib['onclick']) account.label = u'{0} {1}'.format(unicode(table.xpath('./caption')[0].text.strip()), unicode(box.xpath('.//a')[0].text.strip())) elif len(foot[0].xpath('.//a')) != 0 and 'onclick' in foot[0].xpath('.//a')[0].attrib: args = self.js2args(foot[0].xpath('.//a')[0].attrib['onclick']) account.label = unicode(table.xpath('./caption')[0].text.strip()) else: continue self.logger.debug('Args: %r' % args) if 'paramNumCompte' not in args: try: label = unicode(table.xpath('./caption')[0].text.strip()) except Exception: label = 'Unable to determine' self.logger.warning('Unable to get account ID for %r' % label) continue try: account.id = args['paramNumCompte'] + args['paramNumContrat'] if 'Visa' in account.label: card_id = re.search('(\d+)', box.xpath('./td[2]')[0].text.strip()) if card_id: account.id += card_id.group(1) if 'Valorisation' in account.label or u'Liquidités' in account.label: account.id += args[next(k for k in args.keys() if "_idcl" in k)].split('Jsp')[-1] except KeyError: account.id = args['paramNumCompte'] for l in table.attrib['class'].split(' '): if 'tableaux-comptes-' in l: account_type_str = l[len('tableaux-comptes-'):].lower() break else: account_type_str = '' for pattern, type in self.ACCOUNT_TYPES.iteritems(): if pattern in account_type_str or pattern in account.label.lower(): account.type = type break else: account.type = Account.TYPE_UNKNOWN account.type = Account.TYPE_MARKET if "Valorisation" in account.label else \ Account.TYPE_CARD if "Visa" in account.label else \ account.type currency_title = table.xpath('./thead//th[@class="montant"]')[0].text.strip() m = re.match('Montant \((\w+)\)', currency_title) if not m: self.logger.warning('Unable to parse currency %r' % currency_title) else: account.currency = account.get_currency(m.group(1)) try: account.balance = Decimal(FrenchTransaction.clean_amount(self.parse_number(u''.join([txt.strip() for txt in box.cssselect("td.montant")[0].itertext()])))) except InvalidOperation: #The account doesn't have a amount pass account._args = args account._acctype = "bank" account._hasinv = True if "Valorisation" in account.label else False account._url = self.doc.xpath('//form[contains(@action, "panorama")]/@action')[0] yield account class IbanPage(PDFPage): def get_iban(self): iban = u'' for part in re.findall(r'0 -273.46 Td /F\d 10 Tf \((\d+|\w\w\d\d)\)', self.doc, flags=re.MULTILINE): iban += part return iban[:len(iban)/2] class BankTransaction(FrenchTransaction): PATTERNS = [(re.compile('^RET(RAIT) DAB (?P

\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(CARTE|CB ETRANGER) (?P
\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?PVIR(EMEN)?T? (SEPA)?(RECU|FAVEUR)?)( /FRM)?(?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^PRLV (?P.*)( \d+)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^(CHQ|CHEQUE) .*$'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(CONVENTION \d+ |F )?COTIS(ATION)? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class TransactionsPage(LoggedPage, MyHTMLPage): COL_DATE = 0 COL_TEXT = 1 COL_DEBIT = 2 COL_CREDIT = 3 def check_error(self): error = CleanText(default="").filter(self.doc.xpath('//p[@class="question"]')) return error if u"a expiré" in error else None def go_action(self, action): names = {'investment': "Portefeuille", 'history': "Mouvements"} for li in self.doc.xpath('//div[@class="onglets"]/ul/li[not(script)]'): if not Attr('.', 'class', default=None)(li) and names[action] in CleanText('.')(li): url = Attr('./ancestor::form[1]', 'action')(li) args = self.js2args(Attr('./a', 'onclick')(li)) args['javax.faces.ViewState'] = self.get_view_state() self.browser.location(url, data=args) break @method class iter_investment(TableElement): item_xpath = '//table[contains(@id, "titres") or contains(@id, "OPCVM")]/tbody/tr' head_xpath = '//table[contains(@id, "titres") or contains(@id, "OPCVM")]/thead/tr/th[not(caption)]' col_label = u'Intitulé' col_quantity = u'NB' col_unitprice = re.compile(u'Prix de revient') col_unitvalue = u'Dernier cours' col_diff = re.compile(u'\+/\- Values latentes') col_valuation = re.compile(u'Montant') class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_quantity = CleanDecimal(TableCell('quantity')) obj_unitprice = CleanDecimal(TableCell('unitprice')) obj_unitvalue = CleanDecimal(TableCell('unitvalue')) obj_valuation = CleanDecimal(TableCell('valuation')) obj_diff = CleanDecimal(TableCell('diff')) def obj_code(self): onclick = Attr(None, 'onclick').filter((TableCell('label')(self)[0]).xpath('.//a')) m = re.search(',\s+\'([^\'_]+)', onclick) return NotAvailable if not m else m.group(1) def condition(self): return CleanText(TableCell('valuation'))(self) def more_history(self): link = None for a in self.doc.xpath('.//a'): if a.text is not None and a.text.strip() == 'Sur les 6 derniers mois': link = a break form = self.doc.xpath('//form')[-1] if not form.attrib['action']: return None if link is None: # this is a check account args = {'categorieMouvementSelectionnePagination': 'afficherTout', 'nbLigneParPageSelectionneHautPagination': -1, 'nbLigneParPageSelectionneBasPagination': -1, 'nbLigneParPageSelectionneComponent': -1, 'idDetail:btnRechercherParNbLigneParPage': '', 'idDetail_SUBMIT': 1, 'javax.faces.ViewState': self.get_view_state(), } else: # something like a PEA or so value = link.attrib['id'] id = value.split(':')[0] args = {'%s:_idcl' % id: value, '%s:_link_hidden_' % id: '', '%s_SUBMIT' % id: 1, 'javax.faces.ViewState': self.get_view_state(), 'paramNumCompte': '', } self.browser.location(form.attrib['action'], data=args) return True def get_history(self): #DAT account can't have transaction if self.doc.xpath('//table[@id="table-dat"]'): return #These accounts have investments, no transactions if self.doc.xpath('//table[@id="InfosPortefeuille"]'): return tables = self.doc.xpath('//table[@id="table-detail-operation"]') if len(tables) == 0: tables = self.doc.xpath('//table[@id="table-detail"]') if len(tables) == 0: tables = self.doc.getroot().cssselect('table.table-detail') if len(tables) == 0: assert len(self.doc.xpath('//td[has-class("no-result")]')) > 0 return for tr in tables[0].xpath('.//tr'): tds = tr.findall('td') if len(tds) < 4: continue t = BankTransaction() date = u''.join([txt.strip() for txt in tds[self.COL_DATE].itertext()]) raw = u''.join([txt.strip() for txt in tds[self.COL_TEXT].itertext()]) debit = self.parse_number(u''.join([txt.strip() for txt in tds[self.COL_DEBIT].itertext()])) credit = self.parse_number(u''.join([txt.strip() for txt in tds[self.COL_CREDIT].itertext()])) t.parse(date, re.sub(r'[ ]+', ' ', raw)) t.set_amount(credit, debit) yield t class CBTransactionsPage(TransactionsPage): COL_CB_CREDIT = 2 def get_history(self): tables = self.doc.xpath('//table[@id="idDetail:dataCumulAchat"]') transactions = list() if len(tables) == 0: return transactions for tr in tables[0].xpath('.//tr'): tds = tr.findall('td') if len(tds) < 3: continue t = BankTransaction() date = u''.join([txt.strip() for txt in tds[self.COL_DATE].itertext()]) raw = self.parse_number(u''.join([txt.strip() for txt in tds[self.COL_TEXT].itertext()])) credit = self.parse_number(u''.join([txt.strip() for txt in tds[self.COL_CB_CREDIT].itertext()])) debit = "" t.parse(date, re.sub(r'[ ]+', ' ', raw)) t.set_amount(credit, debit) transactions.append(t) for histo in super(CBTransactionsPage, self).get_history(): transactions.append(histo) transactions.sort(key=lambda transaction: transaction.date, reverse=True) return iter(transactions) weboob-1.2/modules/axabanque/test.py000066400000000000000000000017531303450110500175750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class AXABanqueTest(BackendTest): MODULE = 'axabanque' def test_axabanque(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/banqueaccord/000077500000000000000000000000001303450110500167205ustar00rootroot00000000000000weboob-1.2/modules/banqueaccord/__init__.py000066400000000000000000000014521303450110500210330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BanqueAccordModule __all__ = ['BanqueAccordModule'] weboob-1.2/modules/banqueaccord/browser.py000066400000000000000000000051641303450110500207630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, need_login, URL from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, IndexPage, AccountsPage, OperationsPage __all__ = ['BanqueAccordBrowser'] class BanqueAccordBrowser(LoginBrowser): BASEURL = 'https://www.oney.fr' TIMEOUT = 30.0 login = URL('/site/s/login/login.html', LoginPage) index = URL('/site/s/detailcompte/detailcompte.html', IndexPage) accounts = URL('/site/s/detailcompte/ongletdetailcompte.html', AccountsPage) operations = URL('/site/s/detailcompte/ongletdernieresoperations.html', OperationsPage) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.go() self.page.login(self.username, self.password) if not self.index.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): self.index.stay_or_go() for a in self.page.get_list(): post = {'numeroCompte': a.id,} self.index.go(data=post) a.balance = self.page.get_loan_balance() if a.balance is not None: a.type = a.TYPE_LOAN else: self.accounts.go() a.balance = self.page.get_balance() a.type = a.TYPE_CARD if a.balance is None: continue yield a @need_login def iter_history(self, account): post = {'numeroCompte': account.id} self.index.go(data=post) if account.type == account.TYPE_LOAN: return self.page.iter_loan_transactions() self.operations.go() return sorted(self.page.iter_transactions(), key=lambda tr: tr.rdate, reverse=True) weboob-1.2/modules/banqueaccord/favicon.png000066400000000000000000000076521303450110500210650ustar00rootroot00000000000000PNG  IHDR@@iqbKGDks pHYs  tIME 903tEXtCommentCreated with GIMPWIDATxy՝?U^^C/,M52h)HhhTq$5s\A3ꈢQ#'@ŰtM76?zM749uߍ硜+pO! E)Q/Xw"?ԕEtz#*eV3 M݅-,jb#`}"y ƣ*}QlMJ˲-Є%6`$cٟsK@!~KQ(AH>D.µ!Ų'iJ˥'UG\Qє(J(_ph v5o~ L°V[:8qT%*Wc)G.@[ig{ = Y!LİHb٫g;$!p>ry_Á@%Jۓ%X" \H.XU͊ѯ BHUߞ|#U92vlXd,ۑig%}kwe@rUq.EjG% eDSO_ :ȣQH[gh#y'iJ!]BrPX;G'%I4e [`xsp˧#(Y-#vDљCzQbt@ͬ=7\Ц@6)nqMK7% ]q6[By)= c H!C½ |֭KRP"sȖ%tr{QǹҾK O= ''J.okwGճ{-4yۀ@H;ip?OBH e99)*BSRx;Ad; s?n\)m!O,C3žeD &<_΂w;K۴i"nF=wYB"O i1?̮FFi1ԥg ݍ˷9G;k7JߟLXzom}0 dJF LAye|oİLSč@>yQucPaw>5bH_%J_7!a Z~R4V(O̤@^jXOe~)o)ڥձ}op>`nbz3# sEs1ſnZ[ [:O(C&7P w@w ƾ<}L9W6fol"V7!iHeas{(v)`ILJY})k^vr ؟2G]ïNMQ9-ҝF]KG]'?/k0`w1%sdٵ7A-Om|\ a59kgw~4HE0L*[(]{/vtckA]&CsS.-0%_O_n,"tA0|oWQms Uu`5 6~qi̖UrFl"nG 1CfV~s4`ZHG-yӟ3t^޴O*3.s{!fJ63koɯ9/]$n[겪mޜ u5_,-JEN6X)= v$yC8 y=!O~1HPP)M(^ku+޼G\BCʱXfEc z=slw E”acw HM4&@4`kr?3t)y?#9өN~b ٞp[.|`iBAnr,o++`kߜ>9fuO$,ey/|% 4'aETTkYټp M4ΰr&/{XӴ33\*1al7v8x|怑`֦&LFv mR[,(sטbUö 5fbu4F\t6 d7^1SrQIIp H]Xyi٪th >'޻OM4uF2S᠓\biAs9u*R~-IOe Jr1ZSiceMӑp;l^\sl'y|4-ɝ4DGZ/)[NEMӁr[mВe_UkGR/!>;dPym%ߜ1:d7S颙Д NY9;<7IMCkj guUY%VoGQ-2:qҜx S<;D^]9W"9MS2^Snsl^r -?v̭ 0=fɿA:ILmvw ihN4,=V?#ƦZ&WE()/I[ku5)#+u'Ʒ?6=\ h@`  TA + wg} #'q]Ko)1}KXմGKF(*0*׆2 ڴw_:!-^<񲾎{)HPDJ;H+RA{&/9[[Al%^IK9ܼxCPRO~GKjץ)e=fYfYC<"'W鷵8>yuT jNۙdh۲i|9u$eK/%~rF\ؿMd:;v'T}>ߣ_䦛&)s bZ/M5n 7@S \I@M@ AlG#un567_'$]'MȣOhvO`(4ڹ6 t+iנ0>#]-'k~3UE!P< ;a 8%geWIENDB`weboob-1.2/modules/banqueaccord/module.py000066400000000000000000000036361303450110500205670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import find_object from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import BanqueAccordBrowser __all__ = ['BanqueAccordModule'] class BanqueAccordModule(Module, CapBank): NAME = 'banqueaccord' DESCRIPTION = u'Banque Accord' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', regexp='\d+', masked=False), ValueBackendPassword('password', label=u"Code d'accès", regexp='\d+')) BROWSER = BanqueAccordBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/banqueaccord/pages.py000066400000000000000000000173541303450110500204030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from dateutil.relativedelta import relativedelta from decimal import Decimal, InvalidOperation import re from cStringIO import StringIO from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, Env from weboob.browser.filters.html import Attr from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.exceptions import ParseError class Transaction(FrenchTransaction): PATTERNS = [(re.compile(ur'^(?P.*?) - traité le \d+/\d+$'), FrenchTransaction.TYPE_CARD)] class VirtKeyboard(MappedVirtKeyboard): symbols={'0':('8664b9cdfa66b4c3a1ec99c35a2bf64b','9eb80c6e99410eaac32905b2c77e65e5','37717277dc2471c8a7bf37e2068a8f01'), '1':('1f36986f9d27dde54ce5b08e8e285476','9d0aa7a0a2bbab4f2c01ef1e820cb3f1'), '2':('b560b0cce2ca74d3d499d73775152ab7',), '3':('d16e426e71fc29b1b55d0fbded99a473',), '4':('19c68066e414e08d17c86fc5c4acc949','c43354a7f7739508f76c538d5b3bce26'), '5':('4b9abf98e30a1475997ec770cbe5e702','2059b4aa95c7b3156b171255fa10bbdd'), '6':('804be4171d61f9cc10e9978c43b1d2a0','a41b091d4a11a318406a5a8bd3ed3837'), '7':('8adf951f4eea5f446f714214e101d555',), '8':('568135f3844213c30f2c7880be867d3d',), '9':('a3750995c511ea1492ac244421109e77','eeb3a8ba804f19380dfe94a91a37595b'), } color=(0,0,0) def __init__(self, page): img = page.doc.find("//img[@usemap='#cv']") res = page.browser.open(img.attrib['src']) MappedVirtKeyboard.__init__(self, StringIO(res.content), page.doc, img, self.color, 'href', convert='RGB') self.check_symbols(self.symbols, page.browser.responses_dirname) def check_color(self, pixel): for p in pixel: if p >= 0xd5: return False return True def get_symbol_coords(self, coords): # strip borders x1, y1, x2, y2 = coords return MappedVirtKeyboard.get_symbol_coords(self, (x1+10, y1+10, x2-10, y2-10)) def get_symbol_code(self, md5sum_list): for md5sum in md5sum_list: try: code = MappedVirtKeyboard.get_symbol_code(self,md5sum) except VirtKeyboardError: continue else: return ''.join(re.findall(r"'(\d+)'", code)[-2:]) raise VirtKeyboardError('Symbol not found') def get_string_code(self, string): code = '' for c in string: code += self.get_symbol_code(self.symbols[c]) return code class LoginPage(HTMLPage): def login(self, login, password): vk = VirtKeyboard(self) form = self.get_form('//form[@id="formulaire-login"]') code = vk.get_string_code(password) assert len(code)==10, ParseError("Wrong number of character.") form['accordirect.identifiant'] = login form['accordirect.code'] = code form.submit() class IndexPage(LoggedPage, HTMLPage): @method class get_list(ListElement): item_xpath = '//li[@id="menu-n2-mesproduits"]//li//a' class item(ItemElement): klass = Account obj_id = Regexp(Attr('.', 'onclick'), r"^[^']+'([^']+)'.*", r"\1") obj_label = CleanText('.') def condition(self): return self.el.get('onclick') is not None loan_init_date = Transaction.Date(CleanText('//table//td[contains(text(), "Date de sous")]/../td[2]')) loan_next_date = Transaction.Date(CleanText('//table//td[contains(text(), "Prochaine")]/../td[2]')) loan_nb = CleanText('//table//td[contains(text(), "Nombre de mensu") and contains(text(), "rembours")]/../td[2]') loan_total_amount = CleanDecimal('//table//td/strong[contains(text(), "Montant emprunt")]/../../td[2]', replace_dots=False) loan_amount = CleanDecimal('//table//td/strong[contains(text(), "Montant de la")]/../../td[2]', replace_dots=False) def get_loan_balance(self): try: total_amount = - self.loan_total_amount(self.doc) except InvalidOperation: return None nb = int(self.loan_nb(self.doc)) amount = self.loan_amount(self.doc) return total_amount + (nb*amount) def iter_loan_transactions(self): init_date = self.loan_init_date(self.doc) next_date = self.loan_next_date(self.doc) nb = int(self.loan_nb(self.doc)) total_amount = - self.loan_total_amount(self.doc) amount = self.loan_amount(self.doc) if init_date is NotAvailable and total_amount == Decimal('0.0'): return for _ in xrange(nb): next_date = next_date - relativedelta(months=1) tr = Transaction() tr.raw = tr.label = u'Mensualité' tr.date = tr.rdate = tr.vdate = next_date tr.amount = amount yield tr tr = Transaction() tr.raw = tr.label = u'Emprunt initial' tr.date = tr.rdate = init_date tr.amount = total_amount yield tr def get_card_name(self): return CleanText('//h1[1]')(self.doc) class AccountsPage(LoggedPage, HTMLPage): def get_balance(self): balance = Decimal('0.0') lines = self.doc.xpath('//div[@class="detail"]/table//tr') if len(lines) == 0: return None for line in lines: try: left = line.xpath('./td[@class="gauche"]')[0] right = line.xpath('./td[@class="droite"]')[0] except IndexError: #useless line continue if len(left.xpath('./span[@class="precision"]')) == 0 or \ (left.text is None or 'total' not in left.text.lower()): continue balance -= CleanDecimal('.', replace_dots=False)(right) return balance class OperationsPage(LoggedPage, HTMLPage): @method class iter_transactions(ListElement): item_xpath = '//div[contains(@class, "mod-listeoperations")]//table/tbody/tr' class credit(ItemElement): klass = Transaction obj_type = Transaction.TYPE_CARD obj_date = Transaction.Date('./td[1]') obj_raw = Transaction.Raw('./td[2]') obj_amount = Env('amount') def condition(self): self.env['amount'] = Transaction.Amount('./td[4]')(self.el) return self.env['amount'] > 0 class debit(ItemElement): klass = Transaction obj_type = Transaction.TYPE_CARD obj_date = Transaction.Date('./td[1]') obj_raw = Transaction.Raw('./td[2]') obj_amount = Env('amount') def condition(self): self.env['amount'] = - Transaction.Amount('./td[3]')(self.el) return self.env['amount'] != 0 weboob-1.2/modules/banqueaccord/test.py000066400000000000000000000015151303450110500202530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BanqueAccordTest(BackendTest): MODULE = 'banqueaccord' weboob-1.2/modules/banquepopulaire/000077500000000000000000000000001303450110500174655ustar00rootroot00000000000000weboob-1.2/modules/banquepopulaire/__init__.py000066400000000000000000000014521303450110500216000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BanquePopulaireModule __all__ = ['BanquePopulaireModule'] weboob-1.2/modules/banquepopulaire/browser.py000066400000000000000000000257301303450110500215310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from collections import OrderedDict import urllib from weboob.exceptions import BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from weboob.capabilities.bank import Account from weboob.capabilities.base import NotAvailable from .pages import LoginPage, IndexPage, AccountsPage, AccountsFullPage, CardsPage, TransactionsPage, \ UnavailablePage, RedirectPage, HomePage, Login2Page, ErrorPage, \ LineboursePage, NatixisPage, InvestmentNatixisPage, InvestmentLineboursePage, MessagePage, \ IbanPage, NatixisErrorPage, EtnaPage __all__ = ['BanquePopulaire'] class BrokenPageError(Exception): pass class BanquePopulaire(LoginBrowser): login_page = URL(r'https://[^/]+/auth/UI/Login.*', LoginPage) index_page = URL(r'https://[^/]+/cyber/internet/Login.do', IndexPage) accounts_page = URL(r'https://[^/]+/cyber/internet/StartTask.do\?taskInfoOID=mesComptes.*', r'https://[^/]+/cyber/internet/StartTask.do\?taskInfoOID=maSyntheseGratuite.*', r'https://[^/]+/cyber/internet/StartTask.do\?taskInfoOID=accueilSynthese.*', r'https://[^/]+/cyber/internet/StartTask.do\?taskInfoOID=equipementComplet.*', r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=VUE_COMPLETE.*', AccountsPage) iban_page = URL(r'https://[^/]+/cyber/internet/StartTask.do\?taskInfoOID=cyberIBAN.*', r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=DETAIL_IBAN_RIB.*', IbanPage) accounts_full_page = URL(r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=EQUIPEMENT_COMPLET.*', AccountsFullPage) cards_page = URL(r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=ENCOURS_COMPTE.*', CardsPage) transactions_page = URL(r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=SELECTION_ENCOURS_CARTE.*', r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=SOLDE.*', r'https://[^/]+/cyber/internet/ContinueTask.do\?.*dialogActionPerformed=CONTRAT.*', r'https://[^/]+/cyber/internet/Page.do\?.*', r'https://[^/]+/cyber/internet/Sort.do\?.*', TransactionsPage) error_page = URL(r'https://[^/]+/cyber/internet/ContinueTask.do', ErrorPage) unavailable_page = URL(r'https://[^/]+/s3f-web/.*', r'https://[^/]+/static/errors/nondispo.html', UnavailablePage) redirect_page = URL(r'https://[^/]+/portailinternet/_layouts/Ibp.Cyi.Layouts/RedirectSegment.aspx.*', RedirectPage) home_page = URL(r'https://[^/]+/portailinternet/Catalogue/Segments/.*.aspx(\?vary=(?P.*))?', r'https://[^/]+/portailinternet/Pages/.*.aspx\?vary=(?P.*)', r'https://[^/]+/portailinternet/Pages/default.aspx', r'https://[^/]+/portailinternet/Transactionnel/Pages/CyberIntegrationPage.aspx', HomePage) login2_page = URL(r'https://[^/]+/WebSSO_BP/_(?P\d+)/index.html\?transactionID=(?P.*)', Login2Page) # linebourse linebourse_page = URL(r'https://www.linebourse.fr/ReroutageSJR', LineboursePage) message_page = URL(r'https://www.linebourse.fr/DetailMessage.*', MessagePage) invest_linebourse_page = URL(r'https://www.linebourse.fr/Portefeuille', InvestmentLineboursePage) # natixis natixis_page = URL(r'https://www.assurances.natixis.fr/espaceinternet-bp/views/common.*', NatixisPage) invest_natixis_page = URL(r'https://www.assurances.natixis.fr/espaceinternet-bp/views/contrat.*', InvestmentNatixisPage) natixis_error_page = URL(r'https://www.assurances.natixis.fr/espaceinternet-bp/error-redirect.*', NatixisErrorPage) etna = URL(r'https://www.assurances.natixis.fr/etna-ihs-bp/.*', EtnaPage) def __init__(self, website, *args, **kwargs): self.BASEURL = 'https://%s' % website self.token = None super(BanquePopulaire, self).__init__(*args, **kwargs) #def home(self): # self.do_login() def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.login_page.is_here(): self.location(self.BASEURL) self.page.login(self.username, self.password) if self.login_page.is_here(): raise BrowserIncorrectPassword() ACCOUNT_URLS = ['mesComptes', 'mesComptesPRO', 'maSyntheseGratuite', 'accueilSynthese', 'equipementComplet'] @need_login def go_on_accounts_list(self): for taskInfoOID in self.ACCOUNT_URLS: data = OrderedDict([('taskInfoOID', taskInfoOID), ('token', self.token)]) self.location(self.absurl('/cyber/internet/StartTask.do?%s' % urllib.urlencode(data), base=True)) if not self.page.is_error(): if self.page.pop_up(): self.logger.debug('Popup displayed, retry') data = OrderedDict([('taskInfoOID', taskInfoOID), ('token', self.token)]) self.location('/cyber/internet/StartTask.do?%s' % urllib.urlencode(data)) self.ACCOUNT_URLS = [taskInfoOID] break else: raise BrokenPageError('Unable to go on the accounts list page') if self.page.is_short_list(): form = self.page.get_form(nr=0) form['dialogActionPerformed'] = 'EQUIPEMENT_COMPLET' form['token'] = self.page.build_token(form['token']) form.submit() @need_login def get_accounts_list(self, get_iban=True): # We have to parse account list in 2 different way depending if we want the iban number or not # thanks to stateful website self.go_on_accounts_list() next_pages = [] accounts = [] for a in self.page.iter_accounts(next_pages): if get_iban: accounts.append(a) else: yield a while len(next_pages) > 0: next_page = next_pages.pop() if not self.accounts_full_page.is_here(): self.go_on_accounts_list() # If there is an action needed to go to the "next page", do it. if 'prevAction' in next_page: params = self.page.get_params() params['dialogActionPerformed'] = next_page.pop('prevAction') params['token'] = self.page.build_token(self.token) self.location('/cyber/internet/ContinueTask.do', data=params) next_page['token'] = self.page.build_token(self.token) self.location('/cyber/internet/ContinueTask.do', data=next_page) for a in self.page.iter_accounts(next_pages): if get_iban: accounts.append(a) else: yield a if get_iban: for a in accounts: a.iban = self.get_iban_number(a) yield a @need_login def get_iban_number(self, account): url = self.absurl('/cyber/internet/StartTask.do?taskInfoOID=cyberIBAN&token=%s' % self.page.build_token(self.token), base=True) self.location(url) # Sometimes we can't choose an account if account.type in [Account.TYPE_LIFE_INSURANCE, Account.TYPE_MARKET] or (self.page.need_to_go() and not self.page.go_iban(account)): return NotAvailable return self.page.get_iban(account.id) @need_login def get_account(self, id): assert isinstance(id, basestring) for a in self.get_accounts_list(False): if a.id == id: return a return None @need_login def get_history(self, account, coming=False): account = self.get_account(account.id) if coming: params = account._coming_params else: params = account._params if params is None: return params['token'] = self.page.build_token(params['token']) self.location(self.absurl('/cyber/internet/ContinueTask.do', base=True), data=params) if not self.page or self.page.no_operations(): return # Sort by values dates (see comment in TransactionsPage.get_history) if len(self.page.doc.xpath('//a[@id="tcl4_srt"]')) > 0: form = self.page.get_form(id='myForm') form.url = self.absurl('/cyber/internet/Sort.do?property=tbl1&sortBlocId=blc2&columnName=dateValeur') params['token'] = self.page.build_token(params['token']) form.submit() while True: assert self.transactions_page.is_here() for tr in self.page.get_history(account, coming): yield tr next_params = self.page.get_next_params() if next_params is None: return self.location('/cyber/internet/Page.do?%s' % urllib.urlencode(next_params)) @need_login def get_investment(self, account): if not account._invest_params: raise NotImplementedError() account = self.get_account(account.id) params = account._invest_params params['token'] = self.page.build_token(params['token']) self.location(self.absurl('/cyber/internet/ContinueTask.do', base=True), data=params) if self.error_page.is_here(): raise NotImplementedError() url, params = self.page.get_investment_page_params() if params: self.location(url, data=params) if self.linebourse_page.is_here(): self.location('https://www.linebourse.fr/Portefeuille') while self.message_page.is_here(): self.page.skip() self.location('https://www.linebourse.fr/Portefeuille') elif self.natixis_page.is_here(): self.page.submit_form() if self.natixis_error_page.is_here(): return iter([]) return self.page.get_investments() return iter([]) weboob-1.2/modules/banquepopulaire/favicon.png000066400000000000000000000024201303450110500216160ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME .X1IDATx[}Hg~ƚCdkn-+hYJYt+6 v[Z(:6Ơ0velS em~Ykj/wleֻ&w{̓w!q =PLyė)OWB׶i)x|kmjW]|WvlPA5lP9ۈ"՝|m;Xy٠`݃c8oa$زA ~ m44~iBEs>Fṇ-m3snII*}, :hu]@pDžkЫBmo|2V.n&sdl0*+3AQC!46M4ՒMfٕ,4|%xQhK> ZO>DbcpL-&rlݜNú=BPxM-+W]vo&d=?hRB,qX- ʺsӢ\t3^7k%Y$`o:eO0\^NRTT"nUV}SoTV&Q/*AQΒpxcHK zu6rU/>v2*W %PLFYd*_ gF0~.*y9[k형&jJVâk߻]8s~Qkp-]D`2PD h X6ǔ^%(6  5%HMѣ0߈"3^s?1w]-@zP F}M!'3ǹ<T{dn=%|c W\o+y՝(;|@L7Л=r*Tgw>Gb `o /PO+ o}oNc_/:-@NG8lO!}R?% $ syFl0"/m]+x4N_K DeY|xȇ#RIq$[%p-p`Xܾ1Kn\r9#rXq *> Hs;IENDB`weboob-1.2/modules/banquepopulaire/module.py000066400000000000000000000077421303450110500213360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import ValueBackendPassword, Value from .browser import BanquePopulaire __all__ = ['BanquePopulaireModule'] class BanquePopulaireModule(Module, CapBank): NAME = 'banquepopulaire' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Banque Populaire' LICENSE = 'AGPLv3+' website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'www.ibps.alpes.banquepopulaire.fr': u'Alpes', 'www.ibps.alsace.banquepopulaire.fr': u'Alsace', 'www.ibps.bpaca.banquepopulaire.fr': u'Aquitaine Centre atlantique', 'www.ibps.atlantique.banquepopulaire.fr': u'Atlantique', 'www.ibps.banquedesavoie.banquepopulaire.fr': u'Banque de Savoie', 'www.ibps.bpbfc.banquepopulaire.fr': u'Bourgogne-Franche Comté', 'www.ibps.bretagnenormandie.cmm.groupe.banquepopulaire.fr': u'Crédit Maritime Bretagne Normandie', 'www.ibps.atlantique.creditmaritime.groupe.banquepopulaire.fr': u'Crédit Maritime Atlantique', 'www.ibps.sudouest.creditmaritime.groupe.banquepopulaire.fr': u'Crédit Maritime du Littoral du Sud-Ouest', 'www.ibps.cotedazur.banquepopulaire.fr': u'Côte d\'azur', 'www.ibps.loirelyonnais.banquepopulaire.fr': u'Loire et Lyonnais', 'www.ibps.lorrainechampagne.banquepopulaire.fr': u'Lorraine Champagne', 'www.ibps.massifcentral.banquepopulaire.fr': u'Massif central', 'www.ibps.nord.banquepopulaire.fr': u'Nord', 'www.ibps.occitane.banquepopulaire.fr': u'Occitane', 'www.ibps.ouest.banquepopulaire.fr': u'Ouest', 'www.ibps.provencecorse.banquepopulaire.fr': u'Provence et Corse', 'www.ibps.rivesparis.banquepopulaire.fr': u'Rives de Paris', 'www.ibps.sud.banquepopulaire.fr': u'Sud', 'www.ibps.valdefrance.banquepopulaire.fr': u'Val de France', }.iteritems(), key=lambda k_v: (k_v[1], k_v[0]))]) CONFIG = BackendConfig(Value('website', label=u'Région', choices=website_choices), ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passee')) BROWSER = BanquePopulaire def create_default_browser(self): repls = ('alsace', 'bpalc'), ('lorrainechampagne', 'bpalc') website = reduce(lambda a, kv: a.replace(*kv), repls, self.config['website'].get()) return self.create_browser(website, self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_history(account, coming=True) def iter_investment(self, account): return self.browser.get_investment(account) weboob-1.2/modules/banquepopulaire/pages.py000066400000000000000000001022471303450110500211440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from urlparse import urlsplit, parse_qsl from decimal import Decimal import re import urllib from weboob.browser.filters.standard import CleanText, Regexp from weboob.browser.filters.html import Attr, Link, AttributeNotFound from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword from weboob.browser.pages import HTMLPage, LoggedPage, FormNotFound from weboob.capabilities.bank import Account, Investment from weboob.capabilities import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.json import json from weboob.tools.misc import to_unicode class BrokenPageError(Exception): pass class WikipediaARC4(object): def __init__(self, key=None): self.state = range(256) self.x = self.y = 0 if key is not None: self.init(key) def init(self, key): for i in range(256): self.x = (ord(key[i % len(key)]) + self.state[i] + self.x) & 0xFF self.state[i], self.state[self.x] = self.state[self.x], self.state[i] self.x = 0 def crypt(self, input): output = [None]*len(input) for i in xrange(len(input)): self.x = (self.x + 1) & 0xFF self.y = (self.state[self.x] + self.y) & 0xFF self.state[self.x], self.state[self.y] = self.state[self.y], self.state[self.x] output[i] = chr((ord(input[i]) ^ self.state[(self.state[self.x] + self.state[self.y]) & 0xFF])) return ''.join(output) class BasePage(object): ENCODING = 'iso-8859-15' def get_token(self): token = Attr('//form//input[@name="token"]', 'value', default=NotAvailable)(self.doc) if not token: try: token = Regexp(Attr('//body', 'onload'), "saveToken\('(.*?)'")(self.doc) except AttributeNotFound: self.logger.warning('Unable to update token.') return token def on_load(self): self.browser.token = self.get_token() self.logger.debug('Update token to %s', self.browser.token) def is_error(self): for script in self.doc.xpath('//script'): if script.text is not None and \ (u"Le service est momentanément indisponible" in script.text or u"Votre abonnement ne vous permet pas d'accéder à ces services" in script.text): return True return False def build_token(self, token): """ These fucking faggots have introduced a new protection on the token. Each time there is a call to SAB (selectActionButton), the token available in the form is modified with a key available in JS: ipsff(function(){TW().ipthk([12, 25, 17, 5, 23, 26, 15, 30, 6]);}); Each value of the array is an index for the current token to append the char at this position at the end of the token. """ table = None for script in self.doc.xpath('//script'): if script.text is None: continue m = re.search(r'ipthk\(([^\)]+)\)', script.text, flags=re.MULTILINE) if m: table = json.loads(m.group(1)) if table is None: return token for i in table: token += token[i] return token def get_params(self): params = {} for field in self.doc.xpath('//input'): params[field.attrib['name']] = field.attrib.get('value', '') return params def get_button_actions(self): actions = {} for script in self.doc.xpath('//script'): if script.text is None: continue for id, action, strategy in re.findall(r'''attEvt\(window,"(?P[^"]+)","click","sab\('(?P[^']+)','(?P[^']+)'\);"''', script.text, re.MULTILINE): actions[id] = {'dialogActionPerformed': action, 'validationStrategy': strategy, } return actions class MyHTMLPage(BasePage, HTMLPage): def build_doc(self, data, *args, **kwargs): # XXX FUCKING HACK BECAUSE BANQUE POPULAIRE ARE FAGGOTS AND INCLUDE NULL # BYTES IN DOCUMENTS. data = data.replace(b'\x00', b'') return super(MyHTMLPage, self).build_doc(data, *args, **kwargs) class RedirectPage(LoggedPage, MyHTMLPage): ENCODING = None """ var i = 'lyhrnu551jo42yfzx0jm0sqk'; setCookie('i', i); var welcomeMessage = decodeURI('M MACHIN'); var lastConnectionDate = decodeURI('17 Mai 2013'); var lastConnectionTime = decodeURI('14h27'); var userId = '12345678'; var userCat = '1'; setCookie('uwm', $.rc4EncryptStr(welcomeMessage, i)); setCookie('ulcd', $.rc4EncryptStr(lastConnectionDate, i)); setCookie('ulct', $.rc4EncryptStr(lastConnectionTime, i)); setCookie('uid', $.rc4EncryptStr(userId, i)); setCookie('uc', $.rc4EncryptStr(userCat, i)); var agentCivility = 'Mlle'; var agentFirstName = decodeURI('Jeanne'); var agentLastName = decodeURI('Machin'); var agentMail = decodeURI('gary@example.org'); setCookie('ac', $.rc4EncryptStr(agentCivility, i)); setCookie('afn', $.rc4EncryptStr(agentFirstName, i)); setCookie('aln', $.rc4EncryptStr(agentLastName, i)); setCookie('am', $.rc4EncryptStr(agentMail, i)); var agencyLabel = decodeURI('DTC'); var agencyPhoneNumber = decodeURI('0123456789'); setCookie('al', $.rc4EncryptStr(agencyLabel, i)); setCookie('apn', $.rc4EncryptStr(agencyPhoneNumber, i)); Note: that cookies are useless to login on website """ def add_cookie(self, name, value): # httplib/cookielib don't seem to like unicode cookies... name = to_unicode(name).encode('utf-8') value = to_unicode(value).encode('utf-8') self.browser.logger.debug('adding cookie %r=%r', name, value) self.browser.session.cookies.set(name, value) def on_load(self): redirect_url = None args = {} RC4 = None for script in self.doc.xpath('//script'): if script.text is None: continue m = re.search('window.location=\'([^\']+)\'', script.text, flags=re.MULTILINE) if m: redirect_url = m.group(1) for line in script.text.split('\r\n'): m = re.match("^var (\w+) = [^']*'([^']*)'.*", line) if m: args[m.group(1)] = m.group(2) m = re.match("^setCookie\('([^']+)', (\w+)\);", line) if m: self.add_cookie(m.group(1), args[m.group(2)]) m = re.match("^setCookie\('([^']+)', .*rc4EncryptStr\((\w+), \w+\)", line) if m: self.add_cookie(m.group(1), RC4.crypt(args[m.group(2)]).encode('hex')) if RC4 is None and 'i' in args: RC4 = WikipediaARC4(args['i']) if redirect_url is not None: url = self.browser.absurl(redirect_url) headers = {'Referer': self.url} self.browser.logger.debug('redir...') self.browser.location(url, headers=headers) try: form = self.get_form(name="CyberIngtegrationPostForm") except FormNotFound: pass else: form.submit() class ErrorPage(LoggedPage, MyHTMLPage): def get_token(self): try: buf = self.doc.xpath('//body/@onload')[0] except IndexError: return else: m = re.search("saveToken\('([^']+)'\)", buf) if m: return m.group(1) class UnavailablePage(LoggedPage, MyHTMLPage): def on_load(self): a = Link('//a[@class="btn"][1]')(self.doc) if not a: raise BrowserUnavailable() self.browser.location(a) class LoginPage(MyHTMLPage): def on_load(self): h1 = CleanText('//h1[1]')(self.doc) if h1.startswith('Le service est moment'): text = CleanText('//h4[1]')(self.doc) or h1 raise BrowserUnavailable(text) def login(self, login, passwd): form = self.get_form(name='Login') form['IDToken1'] = login.encode(self.ENCODING) form['IDToken2'] = passwd.encode(self.ENCODING) form.submit() class Login2Page(LoginPage): @property def request_url(self): transactionID = self.params['transactionID'] assert transactionID return 'https://www.icgauth.banquepopulaire.fr/dacswebssoissuer/api/v1u0/transaction/%s' % transactionID def on_load(self): r = self.browser.open(self.request_url) doc = json.loads(r.content) self.form_id = doc['step']['validationUnits'][0]['PASSWORD_LOOKUP'][0]['id'] def login(self, login, password): payload = {'validate': {'PASSWORD_LOOKUP': [{'id': self.form_id, 'login': login.encode(self.ENCODING).upper(), 'password': password.encode(self.ENCODING), 'type': 'PASSWORD_LOOKUP' }] } } url = self.request_url + '/step' headers = {'Content-Type': 'application/json'} r = self.browser.open(url, data=json.dumps(payload), headers=headers) doc = json.loads(r.content) self.logger.debug('doc = %s', doc) if 'phase' in doc and doc['phase']['state'] == 'TERMS_OF_USE': # Got: # {u'phase': {u'state': u'TERMS_OF_USE'}, u'validationUnits': [{u'LIST_OF_TERMS': [{u'type': u'TERMS', u'id': u'b7f28f91-7aa0-48aa-8028-deec13ae341b', u'reference': u'CGU_CYBERPLUS'}]}]} if 'reference' in doc['validationUnits'][0]: del doc['validationUnits'][0]['reference'] payload = {'validate': doc['validationUnits'][0]} url = self.request_url + '/step' headers = {'Content-Type': 'application/json'} r = self.browser.open(url, data=json.dumps(payload), headers=headers) doc = json.loads(r.content) self.logger.debug('doc = %s', doc) if ('phase' in doc and doc['phase']['previousResult'] == 'FAILED_AUTHENTICATION') or \ doc['response']['status'] != 'AUTHENTICATION_SUCCESS': raise BrowserIncorrectPassword() data = {'SAMLResponse': doc['response']['saml2_post']['samlResponse']} self.browser.location(doc['response']['saml2_post']['action'], data=data) class IndexPage(LoggedPage, MyHTMLPage): def get_token(self): url = self.doc.xpath('//frame[@name="portalHeader"]')[0].attrib['src'] v = urlsplit(url) args = dict(parse_qsl(v.query)) return args['token'] class HomePage(LoggedPage, MyHTMLPage): def get_token(self): vary = None if self.params.get('vary', None) is not None: vary = self.params['vary'] else: for script in self.doc.xpath('//script'): if script.text is None: continue m = re.search("'vary', '([\d-]+)'\)", script.text) if m: vary = m.group(1) break url = self.browser.absurl('/portailinternet/Transactionnel/Pages/CyberIntegrationPage.aspx?%s' % urllib.urlencode({'vary': vary})) headers = {'Referer': self.url} r = self.browser.open(url, data='taskId=aUniversMesComptes', headers=headers) if not int(r.headers.get('Content-Length', 0)): url = self.browser.absurl('/portailinternet/Transactionnel/Pages/CyberIntegrationPage.aspx') headers = {'Referer': self.url} r = self.browser.open(url, data='taskId=aUniversMesComptes', headers=headers) doc = r.page.doc date = None for script in doc.xpath('//script'): if script.text is None: continue m = re.search('lastConnectionDate":"([^"]*)"', script.text) if m: date = m.group(1) url = self.browser.absurl('/cyber/ibp/ate/portal/integratedInternet.jsp') data = 'session%%3Aate.lastConnectionDate=%s&taskId=aUniversMesComptes' % date headers = {'Referer': r.url} r = self.browser.open(url, data=data, headers=headers) v = urlsplit(r.url) args = dict(parse_qsl(v.query)) return args['token'] class AccountsPage(LoggedPage, MyHTMLPage): ACCOUNT_TYPES = {u'Mes comptes d\'épargne': Account.TYPE_SAVINGS, u'Mon épargne': Account.TYPE_SAVINGS, u'Placements': Account.TYPE_SAVINGS, u'Liste complète de mon épargne': Account.TYPE_SAVINGS, u'Mes comptes': Account.TYPE_CHECKING, u'Comptes en euros': Account.TYPE_CHECKING, u'Liste complète de mes comptes': Account.TYPE_CHECKING, u'Mes emprunts': Account.TYPE_LOAN, u'Liste complète de mes emprunts':Account.TYPE_LOAN, u'Financements': Account.TYPE_LOAN, u'Mes services': None, # ignore this kind of accounts (no bank ones) u'Équipements': None, # ignore this kind of accounts (no bank ones) u'Synthèse': None, # ignore this title } PATTERN = [(re.compile('.*Titres Pea.*'), Account.TYPE_PEA), (re.compile('.*Plan Epargne Retraite.*'), Account.TYPE_PERP), (re.compile('.*Titres.*'), Account.TYPE_MARKET), (re.compile('.*Selection Vie.*'),Account.TYPE_LIFE_INSURANCE), (re.compile('^Fructi Pulse.*'), Account.TYPE_MARKET), ] def pop_up(self): if self.doc.xpath('//span[contains(text(), "du navigateur Internet.")]'): return True return False def is_short_list(self): return len(self.doc.xpath('//script[contains(text(), "EQUIPEMENT_COMPLET")]')) > 0 COL_NUMBER = 0 COL_TYPE = 1 COL_LABEL = 2 COL_BALANCE = 3 COL_COMING = 4 def iter_accounts(self, next_pages): account_type = Account.TYPE_UNKNOWN params = self.get_params() actions = self.get_button_actions() for div in self.doc.getroot().cssselect('div.btit'): if div.text in (None, u'Synthèse'): continue account_type = self.ACCOUNT_TYPES.get(div.text.strip(), Account.TYPE_UNKNOWN) if account_type is None: # ignore services accounts self.logger.debug('Ignore account type %s', div.text.strip()) continue # Go to the full list of this kind of account, if any. btn = div.getparent().xpath('.//button/span[text()="Suite"]') if len(btn) > 0: btn = btn[0].getparent() _params = params.copy() _params.update(actions[btn.attrib['id']]) next_pages.append(_params) continue currency = None for th in div.getnext().xpath('.//thead//th'): m = re.match('.*\((\w+)\)$', th.text) if m and currency is None: currency = Account.get_currency(m.group(1)) for tr in div.getnext().xpath('.//tbody/tr'): if 'id' not in tr.attrib: continue args = dict(parse_qsl(tr.attrib['id'])) tds = tr.findall('td') if len(tds) < 4 or 'identifiant' not in args: self.logger.warning('Unable to parse an account') continue account = Account() account.id = args['identifiant'].replace(' ', '') account.label = u' '.join([u''.join([txt.strip() for txt in tds[1].itertext()]), u''.join([txt.strip() for txt in tds[2].itertext()])]).strip() for pattern, _type in self.PATTERN: match = pattern.match(account.label) if match: account.type = _type break else: account.type = account_type balance = FrenchTransaction.clean_amount(u''.join([txt.strip() for txt in tds[3].itertext()])) account.balance = Decimal(balance or '0.0') account.currency = currency if account.type == account.TYPE_LOAN: account.balance = - abs(account.balance) account._prev_debit = None account._next_debit = None account._params = None account._coming_params = None account._invest_params = None if balance != u'' and len(tds[3].xpath('.//a')) > 0: account._params = params.copy() account._params['dialogActionPerformed'] = 'SOLDE' account._params['attribute($SEL_$%s)' % tr.attrib['id'].split('_')[0]] = tr.attrib['id'].split('_', 1)[1] if len(tds) >= 5 and len(tds[self.COL_COMING].xpath('.//a')) > 0: _params = account._params.copy() _params['dialogActionPerformed'] = 'ENCOURS_COMPTE' # If there is an action needed before going to the cards page, save it. m = re.search('dialogActionPerformed=([\w_]+)', self.url) if m and m.group(1) != 'EQUIPEMENT_COMPLET': _params['prevAction'] = m.group(1) next_pages.append(_params) if not account._params: account._invest_params = params.copy() account._invest_params['dialogActionPerformed'] = 'CONTRAT' account._invest_params['attribute($SEL_$%s)' % tr.attrib['id'].split('_')[0]] = tr.attrib['id'].split('_', 1)[1] yield account # Needed to preserve navigation. btn = self.doc.xpath('.//button/span[text()="Retour"]') if len(btn) > 0: btn = btn[0].getparent() _params = params.copy() _params.update(actions[btn.attrib['id']]) self.browser.open('/cyber/internet/ContinueTask.do', data=_params) class AccountsFullPage(AccountsPage): pass class CardsPage(LoggedPage, MyHTMLPage): COL_ID = 0 COL_TYPE = 1 COL_LABEL = 2 COL_DATE = 3 COL_AMOUNT = 4 def iter_accounts(self, next_pages): params = self.get_params() account = None currency = None for th in self.doc.xpath('//table[@id="TabCtes"]//thead//th'): m = re.match('.*\((\w+)\)$', th.text) if m and currency is None: currency = Account.get_currency(m.group(1)) for tr in self.doc.xpath('//table[@id="TabCtes"]/tbody/tr'): cols = tr.xpath('./td') id = CleanText(None).filter(cols[self.COL_ID]) if len(id) > 0: if account is not None: yield account account = Account() account.id = id.replace(' ', '') account.type = Account.TYPE_CARD account.balance = account.coming = Decimal('0') account._next_debit = datetime.date.today() account._prev_debit = datetime.date(2000,1,1) account.label = u' '.join([CleanText(None).filter(cols[self.COL_TYPE]), CleanText(None).filter(cols[self.COL_LABEL])]) account.currency = currency account._params = None account._invest_params = None account._coming_params = params.copy() account._coming_params['dialogActionPerformed'] = 'SELECTION_ENCOURS_CARTE' account._coming_params['attribute($SEL_$%s)' % tr.attrib['id'].split('_')[0]] = tr.attrib['id'].split('_', 1)[1] elif account is None: raise BrokenPageError('Unable to find accounts on cards page') else: account._params = params.copy() account._params['dialogActionPerformed'] = 'SELECTION_ENCOURS_CARTE' account._params['attribute($SEL_$%s)' % tr.attrib['id'].split('_')[0]] = tr.attrib['id'].split('_', 1)[1] date_col = CleanText(None).filter(cols[self.COL_DATE]) m = re.search('(\d+)/(\d+)/(\d+)', date_col) if not m: self.logger.warning('Unable to parse date %r' % date_col) continue date = datetime.date(*reversed(map(int, m.groups()))) if date.year < 100: date = date.replace(year=date.year+2000) amount = Decimal(FrenchTransaction.clean_amount(CleanText(None).filter(cols[self.COL_AMOUNT]))) if not date_col.endswith('(1)'): # debited account.coming += - abs(amount) account._next_debit = date elif date > account._prev_debit: account._prev_balance = - abs(amount) account._prev_debit = date if account is not None: yield account # Needed to preserve navigation. btn = self.doc.xpath('.//button/span[text()="Retour"]') if len(btn) > 0: btn = btn[0].getparent() actions = self.get_button_actions() _params = params.copy() _params.update(actions[btn.attrib['id']]) self.browser.open('/cyber/internet/ContinueTask.do', data=_params) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^RET DAB (?P.*?) RETRAIT (DU|LE) (?P
\d{2})(?P\d{2})(?P\d+).*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^RET DAB (?P.*?) CARTE ?:.*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?P.*) RETRAIT DU (?P
\d{2})(?P\d{2})(?P\d{2}) .*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(RETRAIT CARTE )?RET(RAIT)? DAB (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('((\w+) )?(?P
\d{2})(?P\d{2})(?P\d{2}) CB[:\*][^ ]+ (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(PRLV|PRELEVEMENT) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PCHEQUE .*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^(CONVENTION \d+ )?COTIS(ATION)? (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?PECHEANCE PRET .*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class TransactionsPage(LoggedPage, MyHTMLPage): def get_next_params(self): nxt = self.doc.xpath('//li[contains(@id, "_nxt")]') if len(nxt) == 0 or nxt[0].attrib.get('class', '') == 'nxt-dis': return None params = {} for field in self.doc.xpath('//input'): params[field.attrib['name']] = field.attrib.get('value', '') params['validationStrategy'] = 'NV' params['pagingDirection'] = 'NEXT' params['pagerName'] = nxt[0].attrib['id'].split('_', 1)[0] return params def get_history(self, account, coming): if len(self.doc.xpath('//table[@id="tbl1"]')) > 0: return self.get_account_history() if len(self.doc.xpath('//table[@id="TabFact"]')) > 0: return self.get_card_history(account, coming) raise NotImplementedError('Unable to find what kind of history it is.') COL_COMPTA_DATE = 0 COL_LABEL = 1 COL_REF = 2 # optional COL_OP_DATE = -4 COL_VALUE_DATE = -3 COL_DEBIT = -2 COL_CREDIT = -1 def get_account_history(self): for tr in self.doc.xpath('//table[@id="tbl1"]/tbody/tr'): tds = tr.findall('td') if len(tds) < 5: continue t = Transaction() # XXX We currently take the *value* date, but it will probably # necessary to use the *operation* one. # Default sort on website is by compta date, so in browser.py we # change the sort on value date. cleaner = CleanText(None).filter date = cleaner(tds[self.COL_OP_DATE]) vdate = cleaner(tds[self.COL_VALUE_DATE]) raw = cleaner(tds[self.COL_LABEL]) debit = cleaner(tds[self.COL_DEBIT]) credit = cleaner(tds[self.COL_CREDIT]) t.parse(date, re.sub(r'[ ]+', ' ', raw), vdate) t.set_amount(credit, debit) # Strip the balance displayed in transaction labels t.label = re.sub('solde en valeur : .*', '', t.label) t.raw = re.sub('solde en valeur : .*', '', t.raw) # XXX Fucking hack to include the check number not displayed in the full label. if re.match("^CHEQUE |^CHQ VOTRE CHEQUE", t.label): t.raw = '%s No: %s' % (t.raw, cleaner(tds[self.COL_REF])) # In rare cases, label is empty .. if not t.label: t.label = cleaner(tds[self.COL_REF]) yield t COL_CARD_DATE = 0 COL_CARD_LABEL = 1 COL_CARD_AMOUNT = 2 def get_card_history(self, account, coming): if coming: debit_date = account._next_debit elif not hasattr(account, '_prev_balance'): return else: debit_date = account._prev_debit if 'ContinueTask.do' in self.url: t = Transaction() t.parse(debit_date, 'RELEVE CARTE') t.amount = -account._prev_balance yield t currency = Account.get_currency(self.doc\ .xpath('//table[@id="TabFact"]/thead//th')[self.COL_CARD_AMOUNT]\ .text\ .replace('(', ' ')\ .replace(')', ' ')) for i, tr in enumerate(self.doc.xpath('//table[@id="TabFact"]/tbody/tr')): tds = tr.findall('td') if len(tds) < 3: continue t = Transaction() cleaner = CleanText(None).filter date = cleaner(tds[self.COL_CARD_DATE]) label = cleaner(tds[self.COL_CARD_LABEL]) amount = '-' + cleaner(tds[self.COL_CARD_AMOUNT]) t.parse(debit_date, re.sub(r'[ ]+', ' ', label)) t.set_amount(amount) t.rdate = t.parse_date(date) t.original_currency = currency yield t def no_operations(self): if len(self.doc.xpath('//table[@id="tbl1" or @id="TabFact"]//td[@colspan]')) > 0: return True if len(self.doc.xpath(u'//div[contains(text(), "Accès à LineBourse")]')) > 0: return True return False def get_investment_page_params(self): script = self.doc.xpath('//body')[0].attrib['onload'] url = None m = re.search(r"','(.+?)',\[", script, re.MULTILINE) if m: url = m.group(1) params = {} for key, value in re.findall(r"key:'(?PSJRToken)'\,value:'(?P.*?)'}", script, re.MULTILINE): params[key] = value return url, params if url and params else None class LineboursePage(LoggedPage, HTMLPage): pass class InvestmentLineboursePage(LoggedPage, HTMLPage): COL_LABEL = 0 COL_QUANTITY = 1 COL_UNITVALUE = 2 COL_VALUATION = 3 COL_UNITPRICE = 4 COL_PERF_PERCENT = 5 COL_PERF = 6 def get_investments(self): for line in self.doc.xpath('//table[contains(@summary, "Contenu")]/tbody/tr[@class="color4"]'): cols1 = line.findall('td') cols2 = line.xpath('./following-sibling::tr')[0].findall('td') cleaner = CleanText(None).filter inv = Investment() inv.label = cleaner(cols1[self.COL_LABEL].xpath('.//span')[0]) inv.code = cleaner(cols1[self.COL_LABEL].xpath('./a')[0]).split(' ')[-1] inv.quantity = self.parse_decimal(cols2[self.COL_QUANTITY]) inv.unitprice = self.parse_decimal(cols2[self.COL_UNITPRICE]) inv.unitvalue = self.parse_decimal(cols2[self.COL_UNITVALUE]) inv.valuation = self.parse_decimal(cols2[self.COL_VALUATION]) inv.diff = self.parse_decimal(cols2[self.COL_PERF]) yield inv def parse_decimal(self, string): value = CleanText(None).filter(string) if value == '': return NotAvailable return Decimal(Transaction.clean_amount(value)) class NatixisPage(LoggedPage, HTMLPage): def submit_form(self): form = self.get_form(name="formRoutage") form.submit() class NatixisErrorPage(LoggedPage, HTMLPage): pass class InvestmentNatixisPage(LoggedPage, HTMLPage): COL_LABEL = 0 COL_QUANTITY = 2 COL_UNITVALUE = 3 COL_VALUATION = 4 def get_investments(self): for line in self.doc.xpath('//div[@class="row-fluid table-contrat-supports"]/table/tbody[(@class)]/tr'): cols = line.findall('td') inv = Investment() inv.label = CleanText(None).filter(cols[self.COL_LABEL]).replace('Cas sans risque ', '') inv.quantity = self.parse_decimal(cols[self.COL_QUANTITY]) inv.unitvalue = self.parse_decimal(cols[self.COL_UNITVALUE]) inv.valuation = self.parse_decimal(cols[self.COL_VALUATION]) yield inv def parse_decimal(self, string): value = CleanText(None).filter(string).replace('Si famille fonds generaux, on affiche un tiret', '').replace('Cas sans risque', '').replace(' ', '') if value == '-': return NotAvailable return Decimal(Transaction.clean_amount(value)) class MessagePage(LoggedPage, HTMLPage): def skip(self): try: form = self.get_form(name="leForm") except FormNotFound: pass else: form.submit() class IbanPage(LoggedPage, MyHTMLPage): def need_to_go(self): return len(self.doc.xpath('//div[@class="grid"]/div/span[contains(text(), "IBAN")]')) == 0 def go_iban(self, account): for tr in self.doc.xpath('//table[@id]/tbody/tr'): if account.type not in (Account.TYPE_LOAN, Account.TYPE_MARKET) and CleanText().filter(tr.xpath('./td[1]')) in account.id: form = self.get_form(id='myForm') form['token'] = self.build_token(form['token']) form['dialogActionPerformed'] = "DETAIL_IBAN_RIB" tr_id = Attr(None, 'id').filter(tr.xpath('.')).split('_') form[u'attribute($SEL_$%s)' % tr_id[0]] = tr_id[1] form.submit() return True return False def get_iban(self, acc_id): iban_class = None for div in self.doc.xpath('//div[@class="grid"]/div'): if not iban_class and "IBAN" in CleanText().filter(div.xpath('./span')): iban_class = Attr(None, 'class').filter(div.xpath('.')) elif iban_class is not None and iban_class == Attr(None, 'class').filter(div.xpath('.')): iban = CleanText().filter(div.xpath('.')).replace(' ', '') if re.sub('\D', '', acc_id) in iban: return iban return NotAvailable class EtnaPage(LoggedPage, MyHTMLPage): def on_load(self): raise NotImplementedError() weboob-1.2/modules/banquepopulaire/test.py000066400000000000000000000017671303450110500210310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BanquePopulaireTest(BackendTest): MODULE = 'banquepopulaire' def test_banquepop(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/barclays/000077500000000000000000000000001303450110500160715ustar00rootroot00000000000000weboob-1.2/modules/barclays/__init__.py000066400000000000000000000014341303450110500202040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BarclaysModule __all__ = ['BarclaysModule'] weboob-1.2/modules/barclays/browser.py000066400000000000000000000163351303450110500201360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import NotAvailable from .pages import LoginPage, Login2Page, IndexPage, AccountsPage, IbanPage, IbanPDFPage, TransactionsPage, \ CardPage, ValuationPage, LoanPage, MarketPage, AssurancePage, LogoutPage class Barclays(LoginBrowser): BASEURL = 'https://www.barclays.fr' index = URL('https?://.*.barclays.fr/\d-index.html', IndexPage) login = URL('https://.*.barclays.fr/barclaysnetV2/logininstit.do.*', LoginPage) login2 = URL('https://.*.barclays.fr/barclaysnetV2/loginSecurite.do.*', Login2Page) logout = URL('https://.*.barclays.fr/bayexterne/barclaysnet/deconnexion/index.html', LogoutPage) accounts = URL('https://.*.barclays.fr/barclaysnetV2/tbord.do.*', AccountsPage) iban = URL('https://.*.barclays.fr/barclaysnetV2/editionRIB.do.*', IbanPage) ibanpdf = URL('https://.*.barclays.fr/barclaysnetV2/telechargerRIB.pdf', IbanPDFPage) transactions = URL('https://.*.barclays.fr/barclaysnetV2/releve.do.*', TransactionsPage) card = URL('https://.*.barclays.fr/barclaysnetV2/cartes.do.*', CardPage) valuation = URL('https://.*.barclays.fr/barclaysnetV2/valuationViewBank.do.*', ValuationPage) loan = URL('https://.*.barclays.fr/barclaysnetV2/pret.do.*', LoanPage) market = URL('https://.*.barclays.fr/barclaysnetV2/titre.do.*', MarketPage) assurance = URL('https://.*.barclays.fr/barclaysnetV2/assurance.do.*', 'https://.*.barclays.fr/barclaysnetV2/assuranceSupports.do.*', AssurancePage) SESSION_PARAM = None def __init__(self, secret, *args, **kwargs): super(Barclays, self).__init__(*args, **kwargs) self.secret = secret self.cache = {} def is_logged(self): return self.page is not None and not (self.login.is_here() or self.index.is_here() or self.login2.is_here()) def go_home(self): if self.is_logged(): link = self.page.doc.xpath('.//a[contains(@id, "tbordalllink")]')[0].attrib['href'] m = re.match('(.*?fr)', self.url) if m: absurl = m.group(1) self.location('%s%s' % (absurl, link)) else: self.do_login() def set_session_param(self): if self.is_logged(): link = self.page.doc.xpath('.//a[contains(@id, "tbordalllink")]')[0].attrib['href'] m = re.search('&(.*)', link) if m: self.SESSION_PARAM = m.group(1) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if self.is_logged(): return if not self.login.is_here(): self.location('https://b-net.barclays.fr/barclaysnetV2/logininstit.do?lang=fr&nodoctype=0') self.page.login(self.username, self.password) if not self.page.has_redirect(): raise BrowserIncorrectPassword() self.location('loginSecurite.do') if self.logout.is_here(): raise BrowserIncorrectPassword() self.page.login(self.secret) if not self.is_logged(): raise BrowserIncorrectPassword() self.set_session_param() def get_ibans_form(self): link = self.page.get_ibanlink() if link: self.location(link.split('/')[-1]) return self.page.get_list() return False @need_login def get_accounts_list(self): if 'accs' not in self.cache.keys(): if not self.accounts.is_here(): self.go_home() accounts = self.page.get_list() accounts_url = self.url ibans = self.get_ibans_form() accs = [] for a in accounts: if ibans and a.id in ibans['list']: ibans['form']['checkaccount'] = ibans['list'][a.id] ibans['form'].submit() a.iban = self.page.get_iban() else: a.iban = NotAvailable accs.append(a) self.location(accounts_url) self.cache['accs'] = accs return self.cache['accs'] def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == id: return a return None def get_related_account(self, related_accid): l = [] for a in self.get_accounts_list(): if related_accid in a.id and a.type == a.TYPE_CHECKING: l.append(a) assert len(l) == 1 return l[0] @need_login def get_history(self, account): if not self.accounts.is_here(): self.go_home() self.location(account._link) assert (self.transactions.is_here() or self.valuation.is_here() or self.loan.is_here() \ or self.market.is_here() or self.assurance.is_here() or self.card.is_here()) transactions = list() while True: for tr in self.page.get_history(): transactions.append(tr) next_page = self.page.get_next_page() if next_page: self.location(next_page) else: break if account._attached_acc is not None: for tr in self.get_history(self.get_related_account(account._attached_acc)): if (tr.raw.startswith('ACHAT CARTE -DEBIT DIFFERE') or 'ACHAT-DEBIT DIFFERE' in tr.raw) and account.id[:6] in tr.raw and account.id[:4] in tr.raw: tr.amount *= -1 transactions.append(tr) for tr in sorted(transactions, key=lambda t: t.rdate, reverse=True) : yield tr @need_login def iter_investments(self, account): if account.type not in (account.TYPE_MARKET, account.TYPE_LIFE_INSURANCE): raise NotImplementedError() if not self.accounts.is_here(): self.go_home() self.location(account._link) if account.type == account.TYPE_LIFE_INSURANCE: self.location(self.url.replace('assurance.do', 'assuranceSupports.do')) return self.page.iter_investments() weboob-1.2/modules/barclays/favicon.png000066400000000000000000000076631303450110500202400ustar00rootroot00000000000000PNG  IHDR@@iq pHYs76vtIME !7lRIDATx[{p]ys_eYecl@08p: mi-Mix40L &%30C4N MM=llYeJǷ{%oI~pTlRn_h=`%(iJ*}.=+0Ʀ5ɞǮrD&'-tւ u 8M ?Xɪx!v2!w;kY\/ߞVl_z msr!x\^} \|'L\ӫΩG7Y(W{mCEDF5M43,,p/mu=Y# g~?+| ,u\.t!D ^~:k19[O\K;Y_gJ3RsbK{ s}qΪk=cRe~.88 0Q5 9XXW5q_ǟOki&c\ʶggɠcOR&…c` Ѱ:# g Vπ?&ثe?22:UjVŰ*2:\2obW_5qMw˭ۧ,.}0!@&ˤ&2Igo: ' 8|i$ Gp.5kA& UKi={/ۤGkSJ,L;x&oe*yj pUkodոpO_0"`Cau2jG a6AGU1v١VʷV^Yƽ^aʝ~i3Ixrs[1!!< XY$M~xՋ OdIg;Iw=sDTet8yK.0}AӴdTsԿU:A2I0.Hx~#3΋)Iĵ.e/{kY/|3Oq4|FIcP)hU|I}d2(s`lq @80v;+Ƹ< O?ϸTG.hQpV΁ ĥ#u-X قspn#Y噋/{:l+Μ<QZ~.5s?Wh re4Apս>2 gS`\-[r='~̺V[Y{'cI?w'N:41+rEҔGtׂR+$ʈ:\c|o(XI8siC.>i-ȷA\!U4Kd9Z \駐XZMbj3ES"N XtxFoԊ|lV1$0I8b,- YXY9 @@#},QsNa*UY  8dPH:(‘*GpN-v X3b4q9lpd[\;wƏep1Uc6!w1&x!2s .`N w9 06Og .33T}w9U mQbp~3MX}Q k45Rۨr#‡Q} q{x2`u &LƪD0fq眱Im*Asd>!zfhlM1va<6Ө^q;iҊ92c ,Oo'Nhbs5FuMeLYG#F$*>6 v NU|gd҅wIxJaw$ܔ!ce!LHVޫXƵM&u nDpkI+N xYAA/Rx>c$T:\&:GvwJd**P~垇~f>̚>n%,*:&!~*)wVǵc!sK^&AiNp.pHN;;zYk X8obBJ2&@*}[['p=x`3][o;uoMO/MMBD}VU qpC VdU}V`*tTHF c2(B2IOv };:\ti'}~qDŽX#}*{Z.կ;"\aI0eFKu6b s tFDŽ|XLR*> o_ tl _=p"神7֖[?r:vKSOs嫦]Aij :,ֽzvT-Wվ;o'̀j/)iJ3xq .3Ծj׻3f]W&~/n{A;:}{qƼ|qWQۻڵ퉸YU6knms"ms s)m'}UE&g["(+4Zm`<4gzÎ֏%钔W5{Ij /WT~;6՞m\zoʞwK3As9V wKgl=ஷ1bq+L;&ʦoɪ*ݵQߞa\~ݍ_\vF-<.7eRLB ]+CUAG& Mf'.w U^~5; #&U/߳z)W{Y,Xv]ny\|4>d1g LRK=V`uLx;e!пm\xm dM|}x=5Ew[Of.di~8} .eTڪqUZ6 U$~ZO4I(mlUIe߃o=ON$6f9Nqʅ?+:IdYYӒ4.42:%` :ܨTm]όB^!s2ؤꬭ#n} t8U?qQpd7p2񴄫dTA\z`k(uN308knݰj5ܶ^9g@ #g5IȳG2؋M$,&rMnu%g ?ejnw5IMUC\Bؽ{[͹@0η3pzH'Ҫ&Nud:L8-VE5@YkͰ:I]+r-W@]v-K.8r!Ӏ&5Aq0I'kd#Cx &nTl{D9rw;0=MLu${$ˤ}_@|EZG·P0;^Ɇ@L_Y}ŝ& #xgskWw8rDg-sg=8*GrT&]TIENDB`weboob-1.2/modules/barclays/module.py000066400000000000000000000044231303450110500177330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.base import find_object from .browser import Barclays __all__ = ['BarclaysModule'] class BarclaysModule(Module, CapBank): NAME = 'barclays' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Barclays' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label=u"N° d'abonné", masked=False), ValueBackendPassword('password', label='Code confidentiel'), ValueBackendPassword('secret', label='Mot secret')) BROWSER = Barclays def create_default_browser(self): return self.create_browser(self.config['secret'].get(), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_account(), id=_id, error=AccountNotFound) def iter_history(self, account): for tr in self.browser.get_history(account): if not tr._coming: yield tr def iter_coming(self, account): for tr in self.browser.get_history(account): if tr._coming: yield tr def iter_investment(self, account): return self.browser.iter_investments(account) weboob-1.2/modules/barclays/pages.py000066400000000000000000000305101303450110500175410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime, re from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage, PDFPage from weboob.browser.filters.standard import CleanText, CleanDecimal from weboob.browser.filters.html import Attr, Link from weboob.capabilities.bank import Account, Investment, NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class LoginPage(HTMLPage): def login(self, login, passwd): form = self.get_form(name='frmLogin') form['username'] = login form['password'] = passwd form.submit() def has_redirect(self): return not len(self.doc.getroot().xpath('//form')) > 0 class Login2Page(HTMLPage): def login(self, secret): label = self.doc.xpath('//span[@class="PF_LABEL"]')[0].text.strip() letters = '' for n in re.findall('(\d+)', label): letters += secret[int(n) - 1] form = self.get_form(name='frmControl', submit='//*[@name="valider"]') form['word'] = letters form.submit() class IndexPage(HTMLPage): pass class AccountsPage(LoggedPage, HTMLPage): ACCOUNT_TYPES = {u'Epargne': Account.TYPE_SAVINGS, u'Liquidités': Account.TYPE_CHECKING, u'Titres': Account.TYPE_MARKET, u'Prêts': Account.TYPE_LOAN, } def get_list(self): accounts = [] for block in self.doc.xpath('//div[@class="pave"]/div'): head_type = block.xpath('./div/span[@class="accGroupLabel"]')[0].text.strip() account_type = self.ACCOUNT_TYPES.get(head_type, Account.TYPE_UNKNOWN) for tr in block.cssselect('ul li.tbord_account'): id = tr.attrib.get('id', '') if id.find('contratId') != 0: self.logger.warning('Unable to parse contract ID: %r' % id) continue id = id[id.find('contratId')+len('contratId'):] link = tr.cssselect('span.accountLabel a')[0] balance = Decimal(FrenchTransaction.clean_amount(tr.cssselect('span.accountTotal')[0].text)) account = Account() account._attached_acc = None account.id = id account.label = unicode(link.text.strip()) account.type = account_type account.balance = balance account.currency = account.get_currency(tr.cssselect('span.accountDev')[0].text) account._link = link.attrib['href'] if id.endswith('CRT'): self.populate_cards(account) accounts.append(account) if len(accounts) == 0: # Sometimes, accounts are only in javascript... for script in self.doc.xpath('//script'): text = script.text if text is None: continue if 'remotePerso' not in text: continue account = None attribs = {} account_type = Account.TYPE_UNKNOWN for line in text.split('\n'): line = line.strip() m = re.match("data.libelle = '(.*)';", line) if m: account_type = self.ACCOUNT_TYPES.get(m.group(1), Account.TYPE_UNKNOWN) elif line == 'var remotePerso = new Object;': account = Account() elif account is not None: m = re.match("remotePerso.(\w+) = '?(.*?)'?;", line) if m: attribs[m.group(1)] = m.group(2) elif line.startswith('listProduitsGroup'): account.id = attribs['refContrat'] account.label = attribs['libelle'] account.type = account_type account.balance = Decimal(FrenchTransaction.clean_amount(attribs['soldeDateOpeValeurFormatted'])) account.currency = account.get_currency(attribs['codeDevise']) account._link = 'tbord.do?id=%s&%s' % (attribs['id'], self.browser.SESSION_PARAM) account._attached_acc = None if account.id.endswith('CRT'): self.populate_cards(account) elif any([word in account.label.lower() for word in ['courant', 'joint', 'perso']]): account.type = account.TYPE_CHECKING elif account.id.endswith('TTR'): account.type = account.TYPE_MARKET elif re.match('^\d+C$', account.id): account.type = account.TYPE_LIFE_INSURANCE elif re.match('^\d+PRT$', account.id): account.type = account.TYPE_LOAN elif not account.type: account.type = account.TYPE_SAVINGS accounts.append(account) account = None return accounts def get_ibanlink(self): return Link('//a[@id="editionRibLevel2link"]', default=None)(self.doc) def populate_cards(self, account): account.type = account.TYPE_CARD account.coming = account.balance account.balance = Decimal('0.0') doc = self.browser.open(account._link).page.doc self.browser.open(self.browser.url) account._attached_acc = ''.join(re.findall('\d', doc.xpath(u'//td[contains(text(), "Carte rattachée")]')[0].text)) class IbanPage(LoggedPage, HTMLPage): def get_list(self): form = self.get_form(name='frmRIB') trs = self.doc.xpath('//tr[td[a[@checkaccount]]]') return {'form': form, 'list': \ {CleanText('./td[1]', replace=[(' ', '')])(tr): Attr('.//a[@checkaccount]', 'checkaccount')(tr) for tr in trs}} class IbanPDFPage(LoggedPage, PDFPage): def get_iban(self): ibans = re.findall(r'1001\d{3}.9\d{3}.73Tm\/F18Tf000rg\(([A-Z\d]+)\)', "".join(self.doc.split())) return NotAvailable if not len(ibans) else CleanText().filter(ibans[0]) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^RET DAB (?P.*?) RETRAIT DU (?P
\d{2})(?P\d{2})(?P\d{2}).*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^RET DAB (?P.*?) CARTE ?:.*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^RET DAB (?P
\d{2})/(?P\d{2})/(?P\d{2}) (?P.*?) CARTE .*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?P.*) RETRAIT DU (?P
\d{2})(?P\d{2})(?P\d{2}) .*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('(\w+) (?P
\d{2})(?P\d{2})(?P\d{2}) CB[:\*][^ ]+ (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?PVIR(EMEN)?T? (SEPA)?(RECU|FAVEUR)?)( /FRM)?(?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^PRLV (?P.*) (REF \w+)?$'),FrenchTransaction.TYPE_ORDER), (re.compile('^CHEQUE.*? (REF \w+)?$'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(CONVENTION \d+ )?COTIS(ATION)? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class HistoryBasePage(HTMLPage): def iter_investments(self): self.logger.warning('Do not support investments on account of type %s' % type(self).__name__) return iter([]) def get_history(self): self.logger.warning('Do not support history on account of type %s' % type(self).__name__) return iter([]) def get_next_page(self): link = self.doc.xpath('//a[span[contains(text(), "Suiv.")]]') if link: return link[0].attrib['href'] class TransactionsPage(LoggedPage, HistoryBasePage): def get_history(self): for tr in self.doc.xpath('//table[@id="operation"]/tbody/tr'): tds = tr.findall('td') if len(tds) < 5: continue t = Transaction() date = u''.join([txt.strip() for txt in tds[0].itertext()]) raw = u' '.join([txt.strip() for txt in tds[1].itertext()]) debit = u''.join([txt.strip() for txt in tds[-3].itertext()]) credit = u''.join([txt.strip() for txt in tds[-2].itertext()]) t.parse(date, re.sub(r'[ ]+', ' ', raw)) t.set_amount(credit, debit) t._coming = False if t.raw.startswith('ACHAT CARTE -DEBIT DIFFERE') or 'ACHAT-DEBIT DIFFERE' in t.raw: t.type = t.TYPE_CARD_SUMMARY yield t class CardPage(LoggedPage, HistoryBasePage): def get_history(self): debit_date = None coming = True for tr in self.doc.xpath('//table[@class="report"]/tbody/tr'): tds = tr.findall('td') if len(tds) == 2: # headers m = re.match('.* (\d+)/(\d+)/(\d+)', tds[0].text.strip()) debit_date = datetime.date(int(m.group(3)), int(m.group(2)), int(m.group(1))) if debit_date < datetime.date.today(): coming = False if len(tds) != 3: continue t = Transaction() date = u''.join([txt.strip() for txt in tds[0].itertext()]) raw = u' '.join([txt.strip() for txt in tds[1].itertext()]) amount = u''.join([txt.strip() for txt in tds[-1].itertext()]) t.parse(date, re.sub(r'[ ]+', ' ', raw)) if debit_date is not None: t.date = debit_date t.label = unicode(tds[1].find('span').text.strip()) t.type = t.TYPE_DEFERRED_CARD t._coming = coming t.set_amount(amount) yield t class ValuationPage(LoggedPage, HistoryBasePage): pass class LoanPage(LoggedPage, HistoryBasePage): pass class MarketPage(LoggedPage, HistoryBasePage): pass class AssurancePage(LoggedPage, HistoryBasePage): def iter_investments(self): for tr in self.doc.xpath('//table[@id="support"]/tbody/tr'): tds = tr.findall('td') inv = Investment() inv.label = CleanText('.')(tds[0]) inv.code = NotAvailable inv.quantity = MyDecimal('.')(tds[1]) inv.unitvalue = MyDecimal('.')(tds[2]) inv.valuation = MyDecimal('.')(tds[3]) yield inv class LogoutPage(HTMLPage): pass weboob-1.2/modules/barclays/test.py000066400000000000000000000017511303450110500174260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BarclaysTest(BackendTest): MODULE = 'barclays' def test_banquepop(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/batoto/000077500000000000000000000000001303450110500155615ustar00rootroot00000000000000weboob-1.2/modules/batoto/__init__.py000066400000000000000000000014311303450110500176710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BatotoModule __all__ = ['BatotoModule'] weboob-1.2/modules/batoto/favicon.png000066400000000000000000000221311303450110500177130ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDC pHYs  tIME 8tEXtComment̖#IDATx{yYUo_s*kZnnE6m@GBtuB 1A  gFPfޫ*+3+{/eYM837{q}=w9|Aw=0t&gΦټgYMBC)oou|`" *Dחn?-g[vl!儔6I"!&0H33ZkqY^S[_8 p| %iX4-KnF"8 $UPo̽V&#BRrv4JդX\LKpin5Nͬ7 Y%IqMw|\pucd~dJeV*u-˱DY*I0~\NoΞ̌LWb6+xxhXi3AK$Y]_lnX w;/?RG?W-9J*TFTWrDy׮=5=( li-""AV"&!P il PD"ӳ,9北sgg`` !,lm67oF:Fc-3TZ  FɔtkcH ~P*VKgޕHaz O%*쳏^4GA^inZ~:)Gzt>R |[)-!;ãZCQ 7Ֆ.O v4(˫M,^92#C 4JQ䣐n !&3X[4[[f _P wV:-LϜж5ׯ__Zr< Ҙ;Ԙ:mgutՕ ]")뗏̜\s]7: Y:óe%riu{֔VP4 rC0bLaZ)[gL š abq<2Ml˶a8+jup0``wkkWn a;BYb]_.LY`0mK/1]. W=' ( 3?svsԲlymG)fx=/ @031rmvH@ I-y&Zm !0g.mm( 񱣍b~4;ǎT>,D;F`0qёmL"]hzReKRY`vٙvqb3*49rLD=0{\f/]B2efm'mH@Xtnge;&3Dd+k!eÓ:'.3;ZkY;4SۥnU+HH&KD-T^sjuLx^ZjS 1.Gss7@@ppM/ & 0LUXS3Znl.IOf&u" vQm.МR\`07CD63[>všR*qvvV BJA`2] MP, BeKD[X/PYIɓ CXO@Dk%qN({ f2QSGY2h)۶BJrmIl"8H/-?3T픚:˖f/ 5̶'ljNo7'`Ha IRʛ$(o!mf&*}?U[_x{td8 B.ȑd]}|-f9{Mq"Y XCj "f=;bPW=sq]5H@WLΐablF4'ILb53 4S fmA/-Hػ_&lf dP-"㩧H3-'NܶHp^]ٝqb,8a@Q}n|2  !%:Fnkg(ǛJũ=UFNu-WDږeŮa4N4$ Aha /aXl`EDR7j7&0+ձN!Aȶ!jf|OkKYZRk3Zq0Id7ٱ%TL! Rbqخ=&"k׫Wv$ Nǎ.LέJi03+m^eVV@Zk3ٖ7 Mq]7my0ÔB) ѣgw 4~ond|)HkEAg.dhӲ ck491 KJm`U3ZD!텅._yXõ "LfHӴe$).dreNT:<]Ve3T[NRneZNߑ;X-ğGŘvivfPR`A*+X&k|-bYIvhu,v\ۋ'Ǐ5$NM2tHJj  i$QPunvR`0KHHP qI cJņe; AfW>uzl.tr%\9y?^wX,) 4 !qUH ~)i%!TbKiX3aO)cGhrW֞+ "w~u-3@"dh" a`Bk2L˶Uw"Y2E+.zN-F8} Nb0#aJJ#>fY $c`ezRY(H!}cL)٦ jkl"$vWvyq+ݽ39~?qō#$I8$MEqhiKðo04H˴bfn!2Xol,o ΝDQ&O~h׿tzi;x bw!'fLDl>tͲʭg7ȄIêm,VvkD֩/҆-4=J^MӱAM030p6Bp&k (]iq\ZkwJŃ¿~ 'Gp֌RH揿|~0 igg30Mc&3cꌼ,7'nؗ#?w|SO|a̙^4a?3Dwvxw*uw?]|CDJ= yy)~ܘd lLN$Ym9NW|U\aXkh!X gob{ړ#tב*Jl 6NHbsƯ|Of @qRWzDž'gOGEӱb?3}{!Dw'Q,`.DBq"a2иýǴ-bp虆eps8z?l϶zq|c:urxz }_vz~|L6 "G|LiB:s $_$B3 CJi33[gN= w33vRT/[o}|~[~mfN]|_g= kR{_Y鷉`>veym~fZ)u7+~{2dSN?;i Zٍ|g@%}_{pM",ۍ'{\"]m%  Q{|?m>t{:o & JvjsXkHDҿ'_ĆLo}<{?;S`q 7 ~7Ҏ<Ic_X~ ~?{1iʬ;Jzxeg|z8|kseS;2ySY!0-fOϿo1s;+ӣLS7?*PA֝o/L98`o$axmWk$<un2yѱR0#~=اFmt]ˇ=Y+?9%&Əfdͭc7.]coKI#h=w+́AJ+r3a:]Gq~mq)eNjǏx!?pDԽTz{gvoejbZ<^{5Vs;5=t,N(^oq]89~:2Q` )2a4ҵm3T 3"" pZmJNNjxFw} `) %H(?'!dUoW4b? ? B2BFtq  3Dz¨SF<ދ0~`O~{ş{NXo& 1H 쑸/ba=SM9!@$.$ӹCg.?Տ~f DH!c!HY7xn{t(4;FV HHH !Jyb싞l#" YxqqflnMk")~wn )%"G` PkF5?V#~ѿ00MssdtfёCĬ+ mۍ4B@^+Z3BTi\8&O,ؖ#!tʹ/쩻l@ҋ i?>E>I!f:|K1Q{K5~:b:T@`̺">C}fK7^v[+9uLĞ L( ~mXp`Cuqmcg;nkeebfD%!yߕZ1w{-^XMTh]ӰnՋGO=K$ZEq@ AJN8-=VީPkD V bG M[۫΍g4kFwrؒdb:{ُ6_7捥+56:fvT"QGuDi/ڭmu0s>E~x $rXbavGa`ӧmy֊ev]DDR*^Jϗ$4:čᡙf1H@EٷVF\߮+륩YJ7Y4m&"J}CkNuCk囎ux7;QPսnA, ~]aFoW8:{auӾRw c" !~kkzU*avJsQ{o[[|캞(^[~IGu/IwTno}և~ qඒT&3a|MZkzO%&A76]C .t0b 24Gjs˫U[kJ-GgVu^W@3;vOXVJkkCUGI+:eosm|W+ 82 W#A[rzcD}g7kߑAأ^-͚~vlm'" ڭV&O_ Xss%_ C>i!K`"L l/S_j ~1sb6[:>{[)U&j7.`c M$4hpȚbdž_NmQ>YSg^Jx>ݬoL9٩ӓsQ`1:|&ЋK4+{'JH!@$HLqmz e (_mZ߬6WX^8 AD…AUe2Xn'I$b"H L:1=uS+a9dP U 'iqF״Tmc;iVAfʅYBr=OvMw{g]aءfڕX}D8Aw@4 Ͻkz+ @7Q[;+~:[ T8n:ĤTJ˔3gZ pX1ku$Nx-R:w7jrQh2H-+ĤV`V:QJ)RU PL!"0 Ý/'xȑԜNfi()B;{D0`}DD!" S  ;hcu1׾3zMo@+2$enf1 ("ߟ`7^PalGs_> KR ;Nll;^ @iP?DCP,V^-&M7]gu5N\7#"6!u3 X1өE_N{K?+7: ?g3kٜ59v4xL0I)Y #@3똙BϽ?05IENDB`weboob-1.2/modules/batoto/module.py000066400000000000000000000025141303450110500174220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['BatotoModule'] class BatotoModule(GenericComicReaderModule): NAME = 'batoto' DESCRIPTION = 'Batoto manga reading website' DOMAIN = 'www.batoto.com' BROWSER_PARAMS = dict( img_src_xpath="//img[@id='comic_page']/@src", page_list_xpath="(//select[@id='page_select'])[1]/option/@value") ID_REGEXP = r'[^/]+/[^/]+' URL_REGEXP = r'.+batoto.(?:com|net)/read/_/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.batoto.net/read/_/%s' PAGES = {URL_REGEXP: DisplayPage} weboob-1.2/modules/batoto/test.py000066400000000000000000000017451303450110500171210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class BatotoTest(GenericComicReaderTest): MODULE = 'batoto' def test_download(self): return self._test_download('26287/yurumates_ch4_by_primitive-scans') weboob-1.2/modules/bforbank/000077500000000000000000000000001303450110500160555ustar00rootroot00000000000000weboob-1.2/modules/bforbank/__init__.py000066400000000000000000000014441303450110500201710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BforbankModule __all__ = ['BforbankModule'] weboob-1.2/modules/bforbank/browser.py000066400000000000000000000141041303450110500201120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.exceptions import BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from weboob.capabilities.bank import Account, AccountNotFound from .pages import ( LoginPage, ErrorPage, AccountsPage, HistoryPage, LoanHistoryPage, RibPage, LifeInsuranceList, LifeInsuranceIframe, LifeInsuranceRedir, BoursePage, ) from .spirica_browser import SpiricaBrowser class BforbankBrowser(LoginBrowser): BASEURL = 'https://www.bforbank.com' login = URL('/connexion-client/service/login\?urlBack=%2Fespace-client', LoginPage) error = URL('/connexion-client/service/auth', ErrorPage) home = URL('/espace-client/$', AccountsPage) rib = URL('/espace-client/rib', '/espace-client/rib/(?P\d+)', RibPage) loan_history = URL('/espace-client/livret/consultation.*', LoanHistoryPage) history = URL('/espace-client/consultation/operations/.*', HistoryPage) lifeinsurance_list = URL(r'/client/accounts/lifeInsurance/lifeInsuranceSummary.action', LifeInsuranceList) lifeinsurance_iframe = URL(r'/client/accounts/lifeInsurance/consultationDetailSpirica.action', LifeInsuranceIframe) lifeinsurance_redir = URL(r'https://assurance-vie.bforbank.com:443/sylvea/welcomeSSO.xhtml', LifeInsuranceRedir) bourse_login = URL(r'/espace-client/synthese/debranchementCaTitre/(?P\d+)') bourse = URL('https://bourse.bforbank.com/netfinca-titres/servlet/com.netfinca.frontcr.synthesis.HomeSynthesis', 'https://bourse.bforbank.com/netfinca-titres/servlet/com.netfinca.frontcr.account.*', BoursePage) def __init__(self, weboob, birthdate, username, password, *args, **kwargs): super(BforbankBrowser, self).__init__(username, password, *args, **kwargs) self.birthdate = birthdate self.accounts = None self.weboob = weboob self.spirica = SpiricaBrowser(weboob, 'https://assurance-vie.bforbank.com:443/', None, None, *args, **kwargs) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.password.isdigit() self.login.stay_or_go() assert self.login.is_here() self.page.login(self.birthdate, self.username, self.password) if self.error.is_here(): raise BrowserIncorrectPassword() @need_login def iter_accounts(self): if self.accounts is None: self.home.stay_or_go() self.accounts = list(self.page.iter_accounts()) if self.page.RIB_AVAILABLE: self.rib.go().populate_rib(self.accounts) return iter(self.accounts) @need_login def get_history(self, account): if account.type == Account.TYPE_MARKET: bourse_account = self.get_bourse_account(account) self.location(bourse_account._link_id) assert self.bourse.is_here() return self.page.iter_history() elif account.type == Account.TYPE_LIFE_INSURANCE: self.goto_spirica(account) return self.spirica.iter_history(account) self.location(account._link.replace('tableauDeBord', 'operations')) return self.page.get_operations() def goto_spirica(self, account): assert account.type == Account.TYPE_LIFE_INSURANCE self.lifeinsurance_list.go() if self.lifeinsurance_list.is_here(): self.logger.debug('multiple life insurances, searching for %r', account) # multiple life insurances: dedicated page to choose for insurance_account in self.page.iter_accounts(): self.logger.debug('testing %r', account) if insurance_account.id == account.id: self.location(insurance_account._link) assert self.lifeinsurance_iframe.is_here() break else: raise AccountNotFound('account was not found in the dedicated page') else: assert self.lifeinsurance_iframe.is_here() self.location(self.page.get_iframe()) assert self.lifeinsurance_redir.is_here() redir = self.page.get_redir() assert redir account._link = self.absurl(redir) self.spirica.session.cookies.update(self.session.cookies) self.spirica.logged = True def get_bourse_account(self, account): self.bourse_login.go(id=account.id) # "login" to bourse page self.bourse.go() assert self.bourse.is_here() self.logger.debug('searching account matching %r', account) for bourse_account in self.page.get_list(): self.logger.debug('iterating account %r', bourse_account) if bourse_account.id.startswith(account.id[3:]): return bourse_account else: raise AccountNotFound() @need_login def iter_investment(self, account): if account.type == Account.TYPE_LIFE_INSURANCE: self.goto_spirica(account) return self.spirica.iter_investment(account) elif account.type == Account.TYPE_MARKET: bourse_account = self.get_bourse_account(account) self.location(bourse_account._market_link) assert self.bourse.is_here() return self.page.iter_investment() raise NotImplementedError() weboob-1.2/modules/bforbank/favicon.png000066400000000000000000000106221303450110500202110ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME  5)g_bKGDCIDATw}>, llCH1lc̾o85 ]uozҪR{[4* $bQ(KYn=g{]y[ h)YNv]M[Z*bYӢM[I nڣ9L+v.kW6,DWjFZݱvZ=@(؊TKIhv45ceUVR`hTHt&0)TiS.FJ2D B]Z1$AvΊT SAHE-BtJ֮@khB`*$k$TILea4-R]I%dv*%mF0HR6؛ e%vf+J(J4! Ij,f&+*T"BNm۰jY-`iuhnљI!i;-h# JuΨY$N1Ej[I,ՙvmQ(%EJd]hD*)I۶!h$ 4P]T$AvmJ[$*JjJ괥ETB- 1j4++YV0Ֆ߯@!XY:)J,HZ".$PP"% iEtm:VvkV+ QBZYi$RECҡM]T RI@  m۶-T$I{FL6L$.]40(H0h5)+hQЀ(@U#+aU2XdEDVLK%hh!B T% DDV$RmIfE#.IRUFYI;-i 1eYN v;mݺح4յִjP2Mh9sMo׾ܹwkTE u8]r_>&:EXTgU $+ :ce5Ք4iK%-&in§??OvǏG7n^L$V&nYHfH%I;mJ ʪ/տ#•/[OZ/Cnß|/]*Tm$H~?V-F%#2,{?='o:ͯ_~s_tRm[b:u>~?y(:3I:ZU ȊvB("D}=_ڵ)Ԉmێ2q7O>o[$ҙ$m nY*H$- $MFWRzܹnm_gO}O̴i$Ih)Xj;hV$jUٳ]|?{ ն:*KmU۶>oWk;},AJֲ$:[g I$RmJE&"k:{…W?v;OCD.Nڶ $ PI B$44zlwλu㮻w>ׯ"让* Iuff I$D@PB/EUI@z3zN3>kRV" Jm[-D5ӶjnIID~̙cccǒ%IM*HV@U4bd['+@M'FDFJ}?o~+WԲhEDDmBPUE%dnjiPU=$.Ic βGg/{p/ei-d+I@m[ Z%LI@(`Oڭ MBE)[B+;k:-aU, A[@PZ+6팑U$RUn?IvTI(9lmcHH "߯%H, m_ի5wsԙ6¨@@HҢ BXVkJծ@Wwcǎ)mk[&I%B53 PTbttbh  cOv_'@[mK@[~Zk$TےTUmDɡJ%U"IU$~o|?tKn:lk@[\{+/{n6@mtM3S$DIQBKh fG>|==qdVZZ p=ͷρj$I$!ڶ@(#Zj(l8::+n#5w3 J?~tt Jm: ZRJ(m=~-v_~OoHA5RB'Μ=O~|˯޻_;@Ր-E%$I6UFt D-Q6ٳ?>|ݏ|ehe@4kn/j$+IVH@4t"mV Ւ*;o{ǻ>O\|///0ti;Jih03th$m *Qqk;vˇ0t- $@)@ H( -ض3O?uՙ"A4$"PR(-(O"ՈMIԥ_t-BTEDdVC*hIE[P>BU&JK!5_s"fK$+IhbZ"ڶM$ @b#+"͎.iT&~5w䓏8s3g_x"h IRUFm#I hP{6YњM̹s?oZIn$ J@CI%R@Q MS&jIb)QfM/_?g/Uw}[~:e B &ӧov8IP HLm!Z+ѶVYP׿|7-MB !h"@!gN<·_OKQmv:KiBjUPB"K$WO8o}8X+2mi o~𭯺}[s*$Yk!Zk$$I$`ffffhT $R/__;|rq-HV;8?O~D(H$ жm۶{.܊@$6q{;v׏'NtٛN:qc:"J:˭}'NwoPPJ$UTTV$Hm[\yO~~/^v GͯR[̙3{իx=m~mے@m`/h*Y T Z/psΝ8y޶M7ho\OSO6V%@$LHI0- |>VTPԨUrө>wϜ9?v. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.value import ValueBackendPassword, Value from .browser import BforbankBrowser __all__ = ['BforbankModule'] class BforbankModule(Module, CapBank): NAME = 'bforbank' DESCRIPTION = u'BforBank' MAINTAINER = u'Baptiste Delpey' EMAIL = 'b.delpey@hotmail.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Code personnel'), Value('birthdate', label='Date de naissance', regexp='^(\d{2}[/-]?\d{2}[/-]?\d{4}|)$') ) BROWSER = BforbankBrowser def create_default_browser(self): return self.create_browser(self.weboob, self.config['birthdate'].get(), self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_coming(self, account): raise NotImplementedError() def iter_history(self, account): return self.browser.get_history(account) def iter_investment(self, account): return self.browser.iter_investment(account) weboob-1.2/modules/bforbank/pages.py000066400000000000000000000210611303450110500175260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from StringIO import StringIO from PIL import Image from weboob.browser.pages import LoggedPage, HTMLPage, pagination, AbstractPage from weboob.browser.elements import method, ListElement, ItemElement from weboob.capabilities.bank import Account from weboob.browser.filters.html import Link, Attr from weboob.browser.filters.standard import CleanText, Regexp, Field, Map, \ CleanDecimal from weboob.tools.capabilities.bank.transactions import FrenchTransaction class BfBKeyboard(object): symbols = {'0': '00111111001111111111111111111111000000111000000001111111111111111111110011111100', '1': '00000000000011000000011100000001100000001111111111111111111100000000000000000000', '2': '00100000111110000111111000011111000011111000011101111111100111111100010111000001', '3': '00100001001110000111111000011111001000011101100001111111011111111111110000011110', '4': '00000011000000111100000111010001111001001111111111111111111111111111110000000100', '5': '00000001001111100111111110011110010000011001000001100110011110011111110000111110', '6': '00011111000111111110111111111111001100011000100001110011001111001111110100011110', '7': '10000000001000000000100000111110011111111011111100111110000011100000001100000000', '8': '00000011001111111111111111111110001000011000100001111111111111111111110010011110', '9': '00111000001111110011111111001110000100011000010011111111111111111111110011111100', } def __init__(self, basepage): self.basepage = basepage self.fingerprints = [] for htmlimg in self.basepage.doc.xpath('.//div[@class="m-btn-pin"]//img'): url = htmlimg.attrib.get("src") imgfile = StringIO(basepage.browser.open(url).content) img = Image.open(imgfile) matrix = img.load() s = "" # The digit is only displayed in the center of image for x in range(19, 27): for y in range(17, 27): (r, g, b, o) = matrix[x, y] # If the pixel is "red" enough if g + b < 450: s += "1" else: s += "0" self.fingerprints.append(s) def get_symbol_code(self, digit): fingerprint = self.symbols[digit] for i, string in enumerate(self.fingerprints): if string == fingerprint: return i def get_string_code(self, string): code = '' for c in string: codesymbol = self.get_symbol_code(c) code += str(codesymbol) return code class LoginPage(HTMLPage): def login(self, birthdate, username, password): vk = BfBKeyboard(self) code = vk.get_string_code(password) form = self.get_form() form['j_username'] = username form['birthDate'] = birthdate form['indexes'] = code form.submit() class ErrorPage(HTMLPage): pass class MyDecimal(CleanDecimal): # BforBank uses commas for thousands seps et and decimal seps def filter(self, text): text = super(CleanDecimal, self).filter(text) text = re.sub(r'[^\d\-\,]', '', text) text = re.sub(r',(?!(\d+$))', '', text) return super(MyDecimal, self).filter(text) class RibPage(LoggedPage, HTMLPage): def populate_rib(self, accounts): for option in self.doc.xpath('//select[@id="compte-select"]/option'): if 'selected' in option.attrib: self.get_iban(accounts) else: self.browser.rib.go(id=re.sub('[^\d]', '', Attr('.', 'value')(option))).get_iban(accounts) def get_iban(self, accounts): for account in accounts: if self.doc.xpath('//option[@selected and contains(@value, "%s")]' % account.id): account.iban = CleanText('//td[contains(text(), "IBAN")]/following-sibling::td[1]', replace=[(' ', '')])(self.doc) class AccountsPage(LoggedPage, HTMLPage): RIB_AVAILABLE = True def on_load(self): if not self.doc.xpath('//span[@class="title" and contains(text(), "RIB")]'): self.RIB_AVAILABLE = False @method class iter_accounts(ListElement): item_xpath = '//table/tbody/tr' class item(ItemElement): klass = Account TYPE = {'Livret': Account.TYPE_SAVINGS, 'Compte': Account.TYPE_CHECKING, 'PEA': Account.TYPE_MARKET, 'Compte-titres': Account.TYPE_MARKET, 'PEA-PME': Account.TYPE_MARKET, 'Assurance-vie': Account.TYPE_LIFE_INSURANCE, } obj_id = CleanText('./td//div[contains(@class, "-synthese-title") or contains(@class, "-synthese-text")]') & Regexp(pattern=r'(\d+)') obj_label = CleanText('./td//div[contains(@class, "-synthese-title")]') obj_balance = MyDecimal('./td//div[contains(@class, "-synthese-num")]', replace_dots=True) obj_currency = FrenchTransaction.Currency('./td//div[contains(@class, "-synthese-num")]') obj_type = Map(Regexp(Field('label'), r'^([^ ]*)'), TYPE, default=Account.TYPE_UNKNOWN) obj__link = CleanText('./@data-href') def condition(self): return not len(self.el.xpath('./td[@class="chart"]')) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^(?PVIREMENT)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PINTERETS)'), FrenchTransaction.TYPE_BANK), ] class LoanHistoryPage(LoggedPage, HTMLPage): @method class get_operations(ListElement): item_xpath = '//table[contains(@class, "table")]/tbody/div/tr[contains(@class, "submit")]' class item(ItemElement): klass = Transaction obj_amount = MyDecimal('./td[4]', replace_dots=True) obj_date = Transaction.Date('./td[2]') obj_vdate = Transaction.Date('./td[3]') obj_raw = Transaction.Raw('./td[1]') class HistoryPage(LoggedPage, HTMLPage): @pagination @method class get_operations(ListElement): item_xpath = '//table[has-class("style-operations")]/tbody//tr' next_page = Link('//div[@class="m-table-paginator full-width-xs"]//a[@id="next-page"]') class item(ItemElement): klass = Transaction def condition(self): if 'tr-section' in self.el.attrib['class']: return False elif 'tr-trigger' in self.el.attrib['class']: return True return False def obj_date(self): return Transaction.Date(Regexp(CleanText('./preceding::tr[has-class("tr-section")][1]/th'), r'(\d+/\d+/\d+)'))(self) obj_raw = Transaction.Raw('./td[1]') obj_amount = MyDecimal('./td[2]', replace_dots=True) class LifeInsuranceList(LoggedPage, HTMLPage): @method class iter_accounts(ListElement): item_xpath = '//table[has-class("comptes_liste")]/tbody//tr' class item(ItemElement): klass = Account obj_id = CleanText('./td/a') obj__link = Link('./td/a') class LifeInsuranceIframe(LoggedPage, HTMLPage): def get_iframe(self): return Attr(None, 'src').filter(self.doc.xpath('//iframe[@id="iframePartenaire"]')) class LifeInsuranceRedir(LoggedPage, HTMLPage): def get_redir(self): # meta http-equiv redirection... for meta in self.doc.xpath('//meta[@http-equiv="Refresh"]/@content'): match = re.search(r'URL=([^\s"\']+)', meta) if match: return match.group(1) class BoursePage(AbstractPage): PARENT = 'lcl' PARENT_URL = 'bourse' weboob-1.2/modules/bforbank/spirica/000077500000000000000000000000001303450110500175075ustar00rootroot00000000000000weboob-1.2/modules/bforbank/spirica/__init__.py000066400000000000000000000000001303450110500216060ustar00rootroot00000000000000weboob-1.2/modules/bforbank/spirica_browser.py000066400000000000000000000015211303450110500216230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.browsers import AbstractBrowser class SpiricaBrowser(AbstractBrowser): PARENT = 'spirica' weboob-1.2/modules/bforbank/test.py000066400000000000000000000015071303450110500174110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BforbankTest(BackendTest): MODULE = 'bforbank' weboob-1.2/modules/biplan/000077500000000000000000000000001303450110500155365ustar00rootroot00000000000000weboob-1.2/modules/biplan/__init__.py000066400000000000000000000014321303450110500176470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BiplanModule __all__ = ['BiplanModule'] weboob-1.2/modules/biplan/browser.py000066400000000000000000000042431303450110500175760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import ProgramPage, EventPage __all__ = ['BiplanBrowser'] class BiplanBrowser(PagesBrowser): BASEURL = 'http://www.lebiplan.org' program_page = URL('/fr/biplan-prog-(?P<_category>.*).php', ProgramPage) event_page = URL('/(?P<_id>.*).html', EventPage) def list_events_concert(self, date_from, date_to=None, city=None, categories=None): return self.program_page.go(_category='concert').list_events(date_from=date_from, date_to=date_to, city=city, categories=categories, is_concert=True) def list_events_theatre(self, date_from, date_to=None, city=None, categories=None): return self.program_page.go(_category='theatre').list_events(date_from=date_from, date_to=date_to, city=city, categories=categories, is_concert=False) def get_event(self, _id, event=None): return self.event_page.go(_id=_id).get_event(obj=event) weboob-1.2/modules/biplan/calendar.py000066400000000000000000000027071303450110500176670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES class BiplanCalendarEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.city = u'LILLE' self.location = u'19, rue Colbert' self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.timezone = u'Europe/Paris' class BiplanCalendarEventConcert(BiplanCalendarEvent): def __init__(self): BiplanCalendarEvent.__init__(self) self.category = CATEGORIES.CONCERT class BiplanCalendarEventTheatre(BiplanCalendarEvent): def __init__(self): BiplanCalendarEvent.__init__(self) self.category = CATEGORIES.THEATRE weboob-1.2/modules/biplan/favicon.png000066400000000000000000000403121303450110500176710ustar00rootroot00000000000000PNG  IHDR@@iq pHYs   cHRMz%u0`:o_F@PIDATx@@  GGG  $$$""" '''*** $$$"""  ###333000   !!! $$$          ))) """  >>> GGG%%%+++ %%%  )))###   """   ,,,"""  '''$$$ 444 999777%%%...(((######  $$$  <<<  )))***---&&& ###UUUMMM(((222  ???###<<<777...GGG??? CCCEEE!!!333  !!! 000'''!!!777%%%999BBB555111333$$$ """BBB999???FFF CCC]]]vvv  ### DDD 555===!!!CCCHHH  ###PPP++++++--- !!!\\\    PPPSSSEEE 777---HHH~~~\\\DDD    :::kkkIII###(((QQQ+++***999>>>222```ggg """### ###555---"""999...iii*** iii444 [[[<<<999)))999666 &&&mmm222--- 777TTT((([[[;;;ZZZnnneeewww%%%555888llljjj@@@777 666777000___PPP$$$$$$pppNNN===<<>>'''UUUyyyOOO{{{iiiNNNaaa]]]HHH000""" +++ZZZzzzAAAOOOxxxaaa aaa iiiyyy???666!!!### 222$$$%%%000 ###EEE ^^^ ???%%%{{{===MMM(((???AAA RRRVVV))))))$$$'''...DDDrrriii{{{888SSS{{{nnn))) QQQ [[[''']]])))<<<,,,www AAA"""000@@@qqq!!!```kkk)))iii'''qqqKKKttt111UUUmmmqqq"""''' UUUGGG ~~~ '''}}}BBBXXX}}}&&&\\\000}}}BBB}}}'''XXX'''lll}}}222 oooXXX'''BBBlll'''lllBBBBBBRRR UUU???!!!:::000:::VVVhhhWWWNNNeeeJJJ @@@XXX'''BBBXXXXXXBBB}}}rrr FFF%%% ...UUU???{{{ JJJ ZZZVVVfffllloooRRR DDDQQQ555  000 ttt ### /// BBBnnnooo\\\999... EEEMMM  ...  QQQ"""fff---333,,,888000FFF;;;BBB|||%%%+++333KKK___OOO[[[444|||nnn ######  QQQ...###    BBBGGG777 >>>222VVVxxx>>>  """:::===''''''(((   ---EEE777LLL999))) $$$  %%%!!!  ''' lll ccc*** ###ggg"""      LLL   ###    (((///            444 _aU pIENDB`weboob-1.2/modules/biplan/module.py000066400000000000000000000054611303450110500174030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES import itertools from .browser import BiplanBrowser from.calendar import BiplanCalendarEvent __all__ = ['BiplanModule'] class BiplanModule(Module, CapCalendarEvent): NAME = 'biplan' DESCRIPTION = u'lebiplan.org website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT, CATEGORIES.THEATRE] BROWSER = BiplanBrowser def search_events(self, query): if self.has_matching_categories(query): theatre_events = [] concert_events = [] if CATEGORIES.CONCERT in query.categories: concert_events = self.browser.list_events_concert(query.start_date, query.end_date, query.city, query.categories) if CATEGORIES.THEATRE in query.categories: theatre_events = self.browser.list_events_theatre(query.start_date, query.end_date, query.city, query.categories) items = list(itertools.chain(concert_events, theatre_events)) items.sort(key=lambda o: o.start_date) return items def list_events(self, date_from, date_to=None): items = list(itertools.chain(self.browser.list_events_concert(date_from, date_to), self.browser.list_events_theatre(date_from, date_to))) items.sort(key=lambda o: o.start_date) return items def get_event(self, _id): return self.browser.get_event(_id) def fill_obj(self, event, fields): return self.browser.get_event(event.id, event) OBJECTS = {BiplanCalendarEvent: fill_obj} weboob-1.2/modules/biplan/pages.py000066400000000000000000000127631303450110500172200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import datetime, time import weboob.tools.date as date_util from .calendar import BiplanCalendarEventConcert, BiplanCalendarEventTheatre from weboob.browser.elements import ItemElement, SkipItem, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Filter, CleanText, Env, Regexp, CombineDate from weboob.browser.filters.html import Link, CleanHTML class BiplanPrice(Filter): def filter(self, el): index = 1 if len(el) > 1 else 0 content = CleanText.clean(CleanText('.', ['HORAIRES'])(el[index])) a_price = content.split(' - ')[-1] parsed_price = re.findall(r"\d*\,\d+|\d+", " ".join(a_price)) if parsed_price and len(parsed_price) > 0: return float(parsed_price[0].replace(',', '.')) return float(0) class BiplanDate(Filter): def filter(self, el): content = CleanText.clean(CleanText(CleanHTML('.'), ['*'])(el[0])) a_date = content[0:content.index(' - ')] for fr, en in date_util.DATE_TRANSLATE_FR: a_date = fr.sub(en, a_date) try: _month = datetime.strptime(a_date, "%A %d %B").month if (datetime.now().month > _month): a_date += u' %i' % (datetime.now().year + 1) else: a_date += u' %i' % (datetime.now().year) except ValueError: pass return datetime.strptime(a_date, "%A %d %B %Y") class StartTime(Filter): def filter(self, el): index = 1 if len(el) > 1 else 0 content = CleanText.clean(CleanText('.', ['HORAIRES'])(el[index])) _content = content.split(' - ') a_time = _content[2] if len(_content) > 2 else _content[0] regexp = re.compile(ur'(?P\d+)h?(?P\d+)') m = regexp.search(a_time) if m: return time(int(m.groupdict()['hh'] or 0), int(m.groupdict()['mm'] or 0)) return time(0, 0) class EndTime(Filter): def filter(self, el): return time.max class ProgramPage(HTMLPage): @method class list_events(ListElement): item_xpath = '//div[@class="ligne"]' class item(ItemElement): def klass(self): return BiplanCalendarEventConcert() if self.env['is_concert'] else BiplanCalendarEventTheatre() def condition(self): return (self.el.xpath('./div') and CleanText('./div/a/img/@src')(self)[-1] != '/') def validate(self, obj): return (self.is_valid_event(obj, self.env['city'], self.env['categories']) and self.is_event_in_valid_period(obj.start_date, self.env['date_from'], self.env['date_to'])) def is_valid_event(self, event, city, categories): if city and city != '' and city.upper() != event.city.upper(): return False if categories and len(categories) > 0 and event.category not in categories: return False return True def is_event_in_valid_period(self, event_date, date_from, date_to): if event_date >= date_from: if not date_to: return True else: if event_date <= date_to: return True return False obj_id = Regexp(Link('./div/a'), '/(.*?).html') obj_start_date = CombineDate(BiplanDate('div/div/b'), StartTime('div/div/b')) obj_end_date = CombineDate(BiplanDate('div/div/b'), EndTime('.')) obj_price = BiplanPrice('div/div/b') obj_summary = CleanText("div/div/div/a/strong") class EventPage(HTMLPage): encoding = u'utf-8' @method class get_event(ItemElement): klass = BiplanCalendarEventConcert if Env('is_concert') else BiplanCalendarEventTheatre def parse(self, el): _div = "//div/div/div[@id='popup']" div = el.xpath("%s" % _div)[0] if self.obj.id: event = self.obj event.url = self.page.url event.description = CleanHTML("%s/div/div[@class='presentation-popup']" % _div)(self) raise SkipItem() self.env['is_concert'] = (div.attrib['class'] != 'theatre-popup') self.env['url'] = self.page.url obj_id = Env('_id') base = "//div[@id='popup']" obj_price = BiplanPrice("%s/div/b" % base) obj_start_date = CombineDate(BiplanDate("%s/div/b" % base), StartTime("%s/div/b" % base)) obj_end_date = CombineDate(BiplanDate("%s/div/b" % base), EndTime(".")) obj_url = Env('url') obj_summary = CleanText('%s/div/div/span' % base) obj_description = CleanHTML('%s/div/div[@class="presentation-popup"]' % base) weboob-1.2/modules/biplan/test.py000066400000000000000000000024231303450110500170700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.tools.test import BackendTest, SkipTest class BiplanTest(BackendTest): MODULE = 'biplan' def test_biplan_list(self): if datetime.now() > datetime(datetime.now().year, 7, 14) and datetime.now() < datetime(datetime.now().year, 9, 15): raise SkipTest("Fermeture estivale") l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) weboob-1.2/modules/blablacar/000077500000000000000000000000001303450110500161745ustar00rootroot00000000000000weboob-1.2/modules/blablacar/__init__.py000066400000000000000000000014401303450110500203040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BlablacarModule __all__ = ['BlablacarModule'] weboob-1.2/modules/blablacar/browser.py000066400000000000000000000030461303450110500202340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import DeparturesPage from datetime import datetime import urllib class BlablacarBrowser(PagesBrowser): BASEURL = 'https://www.blablacar.fr' departures = URL('/search_xhr\?(?P.*)', DeparturesPage) def get_roadmap(self, departure, arrival, filters): pass def get_station_departures(self, station_id, arrival_id, date): query = {'fn': station_id} if arrival_id: query['tn'] = arrival_id if date: _date = datetime.strftime(date, "%d/%m/%Y") query['db'] = _date _heure = datetime.strftime(date, "%H") query['hb'] = _heure query['he'] = '24' return self.departures.open(qry=urllib.urlencode(query)).get_station_departures() weboob-1.2/modules/blablacar/favicon.png000066400000000000000000000063741303450110500203410ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME  UTiTXtCommentCreated with GIMPd.ebKGD^ `IDATx[Nw{MXbƀ(.@4QD$PCB811bvmڟ̼-=Zx?۽7ߛy+/O5]̲<%>};!v {G!rVkIc!8ZGTE4#rٲ+pTf9xVa;4"m7hȍ>(=Ewp5!mHGkS]Cvɧ8Ɍ }}i֋F%Cc3"9<;U.poZ'rk]̓ޱ+ ,(\su1Υs.7@:= oHqM6y&)J;YB}SckgI%Rw|-}=P)vs g I=leus}k*yח4T =|B\WPg$YVX`F|ý(t"t+ g 2HxhPi&73=ŬuL(sat杩 WZPJO)H݅NG$S(ѹCth 2(0@˃9AJ^+]\ ѓlo+ ׊99 ] c ;JLɽL =\k种]m.ZY<+Q6ssJΜ7m7fݘ7 GM,2kDLI9=[M5D` w2+_9ņl%^ RX ?W{l: (QrK&6KhZ 70pX 3{BDDp4F)4ⰰϠ~T(d\I: T'=h  ыvc'YƤkv/A9Fη dz ک/8" 3JlڮS%'κW7ϳh> @kVK$4H\,2>YD;5ۅզ{HGtLi3De)T/1r|JhѳTC}tT~*k5)KZJʎy: ,I 8 >o31Mht? T@;:#Zei㸉aC5n&*Ns*YƨC(#_bI y8whDFlOvx\@W&G&F5`)g|m6BQvflWsiYܘ '-OsVqz4a=ٳ 1xQw^AֶkMfpUgk/N"{4#Hi1?`?[ 8I*㑊+3^o qG _ t!N<ʈswk+ \+8jOō|dz@Om͂زSLSJQMMY1қRXy}` 3h;Y!B:5!;3Fvq = } $ }u{Uw Tu"Ou sNߐY>e};\86ȪaXM*zc?ZFmZw}@d\䧺i*E 00];<.U[GU+8`?{K=y@ KՃ0gSHBz;5;ԅfusU)8WMKp?qykGŭ9 %=/m0pVenޚu,pNΛ"zn5x}'MS -)|7;/13̷/RuTZ:` Mnr<}z6aV5޻| Jt2_T_oq/<\ze8/f1IaF Z4K tCג'TC,Qo^yߏ縊oNUW!~pxH\`}l?&%vOD"&U"PXEIn¬=R[>X GDڹEmޫ%\4^)Oꥭ&A~g;y\஥ε|SD-jLGI~gBO[XieY]̋Kk Ri(4EF 7u5ˍY&^?xI' YT +s!@'Lrp6"o_So=u)#Re0\PXcd_x+w\!vfFh)ZfRTSl'U*kM=`7銤-ўPx/Č֔Rt&0E`(Jl߰=2|^ޝXř{a7`Pֵ7hZԴXYT=l, gpIENDB`weboob-1.2/modules/blablacar/module.py000066400000000000000000000026101303450110500200320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.travel import CapTravel from .browser import BlablacarBrowser __all__ = ['BlablacarModule'] class BlablacarModule(Module, CapTravel): NAME = 'blablacar' DESCRIPTION = u'blablacar website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = BlablacarBrowser def iter_roadmap(self, departure, arrival, filters): return self.browser.get_roadmap(departure, arrival, filters) def iter_station_departures(self, station_id, arrival_id, date): return self.browser.get_station_departures(station_id, arrival_id, date) weboob-1.2/modules/blablacar/pages.py000066400000000000000000000062161303450110500176520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.travel import Departure from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal from weboob.browser.filters.html import Link from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.pages import JsonPage from weboob.capabilities.base import Currency from io import StringIO import lxml.html as html from datetime import datetime class DeparturesPage(JsonPage): ENCODING = None def __init__(self, browser, response, *args, **kwargs): super(DeparturesPage, self).__init__(browser, response, *args, **kwargs) self.encoding = self.ENCODING or response.encoding parser = html.HTMLParser(encoding=self.encoding) if 'results' in self.doc['html']: self.doc = html.parse(StringIO(self.doc['html']['results']), parser) else: self.doc = html.Element('brinbrin') @method class get_station_departures(ListElement): item_xpath = '//ul[@class="trip-search-results"]/li/a' class item(ItemElement): klass = Departure obj_id = Regexp(Link('.'), '/(.*)') def obj_time(self): _date = CleanText('./article/div/h3[@itemprop="startDate"]/@content')(self).split('-') _time = Regexp(CleanText('./article/div/h3[@itemprop="startDate"]'), u'.* à (\d+:\d+)')(self).split(':') return datetime(int(_date[0]), int(_date[1]), int(_date[2]), int(_time[0]), 0 if len(_time) < 2 or len(_time) == 2 and not _time[1] else int(_time[1])) obj_type = CleanText('./article/div/h3[@class="fromto"]/span[@class!="u-visuallyHidden"]') obj_departure_station = CleanText('./article/div/dl[@class="geo-from"]/dd', replace=[(": voir avec le conducteur", "")]) obj_arrival_station = CleanText('./article/div/dl[@class="geo-to"]/dd', replace=[(": voir avec le conducteur", "")]) obj_price = CleanDecimal(CleanText('./article/div/div[@itemprop="location"]/strong/span[last()]')) def obj_currency(self): txt = CleanText('./article/div/div[@itemprop="location"]')(self) return Currency.get_currency(txt) obj_information = CleanText('./article/div/div[@class="availability"]') weboob-1.2/modules/blablacar/test.py000066400000000000000000000020141303450110500175220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from datetime import datetime class BlablacarTest(BackendTest): MODULE = 'blablacar' def test_blablacar(self): departures = list(self.backend.iter_station_departures('lille', None, datetime.now())) self.assertTrue(len(departures) > 0) weboob-1.2/modules/bnporc/000077500000000000000000000000001303450110500155545ustar00rootroot00000000000000weboob-1.2/modules/bnporc/__init__.py000066400000000000000000000014351303450110500176700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BNPorcModule __all__ = ['BNPorcModule'] weboob-1.2/modules/bnporc/company/000077500000000000000000000000001303450110500172225ustar00rootroot00000000000000weboob-1.2/modules/bnporc/company/__init__.py000066400000000000000000000000001303450110500213210ustar00rootroot00000000000000weboob-1.2/modules/bnporc/company/browser.py000066400000000000000000000065621303450110500212700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date, timedelta from weboob.browser import LoginBrowser, URL, need_login from weboob.capabilities.base import find_object from weboob.capabilities.bank import AccountNotFound from .pages import LoginPage, AccountsPage, HistoryPage __all__ = ['BNPCompany'] class BNPCompany(LoginBrowser): BASEURL = 'https://secure1.entreprises.bnpparibas.net' login = URL('/sommaire/jsp/identification.jsp', LoginPage) accounts = URL('/NCCPresentationWeb/e10_soldes/liste_soldes.do', AccountsPage) history = URL('/NCCPresentationWeb/e11_releve_op/listeOperations.do', HistoryPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.password.isdigit() self.login.go() self.login.go() assert self.login.is_here() self.page.login(self.username, self.password) @need_login def get_accounts_list(self): self.accounts.go() return self.page.iter_accounts() @need_login def get_account(self, _id): return find_object(self.get_accounts_list(), id=_id, error=AccountNotFound) def get_transactions(self, id_account, typeReleve, dateMin, dateMax='null'): self.open('https://secure1.entreprises.bnpparibas.net/NCCPresentationWeb/e11_releve_op/init.do?e10=true') params = {} params['identifiant'] = id_account params['typeSole'] = 'C' params['typeReleve'] = typeReleve params['typeDate'] = 'O' params['ajax'] = 'true' params['dateMin'] = dateMin params['dateMax'] = dateMax self.history.go(params=params) return self.page.iter_history() @need_login def iter_history(self, account): return self.get_transactions(account.id, 'Comptable', (date.today() - timedelta(days=90)).strftime('%Y%m%d'), date.today().strftime('%Y%m%d')) @need_login def iter_coming_operations(self, account): return self.get_transactions(account.id, 'Previsionnel', (date.today().strftime('%Y%m%d'))) @need_login def iter_investment(self, account): raise NotImplementedError() @need_login def get_transfer_accounts(self): raise NotImplementedError() @need_login def transfer(self, account, to, amount, reason): raise NotImplementedError() @need_login def iter_threads(self): raise NotImplementedError() @need_login def get_thread(self, thread): raise NotImplementedError() weboob-1.2/modules/bnporc/company/pages.py000066400000000000000000000155271303450110500207050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from StringIO import StringIO import hashlib from decimal import Decimal from datetime import datetime from weboob.capabilities.bank import Account from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.pages import HTMLPage, JsonPage, LoggedPage from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError class BNPVirtKeyboard(MappedVirtKeyboard): symbols = {'0': 'ff069462836e30a39c911034048f5bb3', '1': '7969f04e4e82eaefa2ce7a9a23c26178', '2': '1e6020f97ca1c3ce3da4f39ded15d67d', '3': 'f84284b40aea93c24814e23e14e76cc8', '4': '88bab262d4b344c0ef8f06ddd01adbcf', '5': '0a270764fc5d8334bcb55053432b26cb', '6': 'e6a4444a6c752cd3e655f2883e530080', '7': '933d4ca5df6b2b3df2dea00a21a3fed6', '8': ['f28b918777d21a5fde2bffb9899e2138', 'a97e6e27159084d50f8ef00548b70252'], '9': 'be751b77af0d998ab4c2cfd38455b2a6', } color=(0,0,0) def __init__(self, basepage): img = basepage.doc.xpath('//img[@id="gridpass_img"]')[0] imgdata = basepage.browser.open(img.attrib['src']).content MappedVirtKeyboard.__init__(self, StringIO(imgdata), basepage.doc, img, self.color, convert='RGB') self.check_symbols(self.symbols, basepage.browser.responses_dirname) def get_symbol_code(self, md5sum): code = MappedVirtKeyboard.get_symbol_code(self, md5sum) code = code.split("'")[3] assert code.isdigit() return code def check_color(self, pixel): for p in pixel: if p >= 200: return False return True def checksum(self, coords): """Copy of parent checksum(), but cropping (removes empty lines)""" x1, y1, x2, y2 = coords s = '' for y in range(y1, min(y2 + 1, self.height)): for x in range(x1, min(x2 + 1, self.width)): if self.check_color(self.pixar[x, y]): s += " " else: s += "O" s += "\n" s = '\n'.join([l for l in s.splitlines() if l.strip()]) return hashlib.md5(s).hexdigest() class LoginPage(HTMLPage): def login(self, login, password): try: vk = BNPVirtKeyboard(self) except VirtKeyboardError as err: self.logger.error("Error: %s" % err) return False form = self.get_form(name='loginPwdForm') form['txtAuthentMode'] = 'PASSWORD' form['txtPwdUserId'] = login form['gridpass_hidden_input'] = vk.get_string_code(password) form.submit() def on_load(self): if self.doc.xpath('//p[contains(text(), "Your identification is wrong.")]'): raise BrowserIncorrectPassword("Your identification is wrong.") class AccountsPage(LoggedPage, JsonPage): FAMILY_TO_TYPE = { u"Compte chèque": Account.TYPE_CHECKING, } def iter_accounts(self): for f in self.path('tableauSoldes.listeGroupes'): for g in f: for a in g.get('listeComptes'): yield Account.from_dict({ 'id': a.get('numeroCompte'), 'iban': a.get('numeroCompte'), 'type': self.FAMILY_TO_TYPE.get(a.get('libelleType')) or Account.TYPE_UNKNOWN, 'label': '%s %s' % (a.get('libelleType'), a.get('libelleTitulaire')), 'currency': a.get('deviseTenue'), 'balance': Decimal(a.get('soldeComptable')) / 100, 'coming': Decimal(a.get('soldePrevisionnel')) / 100, }) class HistoryPage(LoggedPage, JsonPage): CODE_TO_TYPE = { "AUTOP": FrenchTransaction.TYPE_UNKNOWN, # Autres opérations, "BOURS": FrenchTransaction.TYPE_BANK, # Bourse / Titres, "CARTE": FrenchTransaction.TYPE_CARD, # Cartes, "CHEQU": FrenchTransaction.TYPE_CHECK, # Chèques, "CREDD": FrenchTransaction.TYPE_UNKNOWN, # Crédits documentaires, "CREDI": FrenchTransaction.TYPE_UNKNOWN, # Crédits, "EFFET": FrenchTransaction.TYPE_UNKNOWN, # Effets, "ESPEC": FrenchTransaction.TYPE_UNKNOWN, # Espèces, "FACCB": FrenchTransaction.TYPE_UNKNOWN, # Factures / Retraits cartes, "ICHEQ": FrenchTransaction.TYPE_UNKNOWN, # Impayés chèques, "IEFFE": FrenchTransaction.TYPE_UNKNOWN, # Impayés et incidents effets, "IMPAY": FrenchTransaction.TYPE_UNKNOWN, # Impayés et rejets, "IPRLV": FrenchTransaction.TYPE_UNKNOWN, # Impayés prélèvements, TIP et télérèglements, "PRLVT": FrenchTransaction.TYPE_UNKNOWN, # Prélèvements, TIP et télérèglements, "REMCB": FrenchTransaction.TYPE_UNKNOWN, # Remises cartes, "RJVIR": FrenchTransaction.TYPE_ORDER, # Rejets de virements, "VIREM": FrenchTransaction.TYPE_ORDER, # Virements, "VIRIT": FrenchTransaction.TYPE_ORDER, # Virements internationaux, "VIRSP": FrenchTransaction.TYPE_ORDER, # Virements européens, "VIRTR": FrenchTransaction.TYPE_ORDER, # Virements de trésorerie, "VIRXX": FrenchTransaction.TYPE_ORDER, # Autres virements } def one(self, path, context=None): try: return list(self.path(path, context))[0] except IndexError: return None def iter_history(self): for op in self.get('mouvementsBDDF'): codeFamille = self.one('nature.codefamille', op) tr = FrenchTransaction.from_dict({ 'id': op.get('id'), 'type': self.CODE_TO_TYPE.get(codeFamille) or FrenchTransaction.TYPE_UNKNOWN, 'category': self.one('nature.libelle', op), 'raw': ' '.join(op.get('libelle').split()) or op.get('nature')['libelle'], 'date': datetime.fromtimestamp(op.get('dateOperation') / 1000), 'vdate': datetime.fromtimestamp(op.get('dateValeur') / 1000), 'amount': Decimal(self.one('montant.montant', op)) / 100, }) yield tr weboob-1.2/modules/bnporc/deprecated/000077500000000000000000000000001303450110500176545ustar00rootroot00000000000000weboob-1.2/modules/bnporc/deprecated/__init__.py000066400000000000000000000000001303450110500217530ustar00rootroot00000000000000weboob-1.2/modules/bnporc/deprecated/browser.py000066400000000000000000000332151303450110500217150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from datetime import datetime from logging import warning from random import randint from weboob.deprecated.browser import Browser, BrowserIncorrectPassword, BrowserPasswordExpired from weboob.capabilities.bank import TransferError, Transfer, Account from .perso.accounts_list import AccountsList, AccountPrelevement from .perso.transactions import AccountHistory, AccountComing from .perso.investment import AccountMarketInvestment, AccountLifeInsuranceInvestment from .perso.transfer import TransferPage, TransferConfirmPage, TransferCompletePage from .perso.login import LoginPage, ConfirmPage, ChangePasswordPage, InfoMessagePage from .perso.messages import MessagePage, MessagesPage from .pro import ProAccountsList, ProAccountHistory __all__ = ['BNPorc'] class BNPorc(Browser): DOMAIN = 'www.secure.bnpparibas.net' PROTOCOL = 'https' CERTHASH = ['5511f0ff19c982b6351c17b901bfa7419f075edb13f2df41e446248beb7866bb', 'fa8cb72ef2e46054469af916f7ec222b1904901fecde8511a0f769ba0385410d', '86cd4ba8cfbc53937dfc402e8c2d0a2d5ffb630a73bbeafd09c39f8b54a6a6c3', 'dfab57aecb7dd8460ba0e802c798e7aff3288b10451992a372e3e8bdc5f1f0c7'] ENCODING = None # refer to the HTML encoding PAGES = {'.*pageId=unedescomptes.*': AccountsList, '.*pageId=releveoperations.*': AccountHistory, '.*FicheA': AccountHistory, '.*SAF_DPF.*': AccountMarketInvestment, '.*identifiant=Assurance_Vie_Consultation.*': AccountLifeInsuranceInvestment, '.*Action=SAF_CHM.*': ChangePasswordPage, '.*pageId=mouvementsavenir.*': AccountComing, '.*NS_AVEDP.*': AccountPrelevement, '.*NS_VIRDF.*': TransferPage, '.*NS_VIRDC.*': TransferConfirmPage, '.*/NS_VIRDA\?stp=(?P\d+).*': TransferCompletePage, '.*type=homeconnex.*': LoginPage, '.*layout=HomeConnexion.*': ConfirmPage, '.*SAF_CHM_VALID.*': ConfirmPage, '.*Action=DSP_MSG.*': InfoMessagePage, '.*MessagesRecus.*': MessagesPage, '.*BmmFicheLireMessage.*': MessagePage, # Pro 'https?://www.secure.bnpparibas.net/banque/portail/entrepros/Fiche\?.*identifiant=PRO_Une_Comptes.*': ProAccountsList, 'https?://www.secure.bnpparibas.net/SAF_ROP.*': ProAccountHistory, 'https?://www.secure.bnpparibas.net/NS_AVEDT.*': ProAccountHistory, } def __init__(self, *args, **kwargs): self.rotating_password = kwargs.pop('rotating_password', None) self.password_changed_cb = kwargs.pop('password_changed_cb', None) Browser.__init__(self, *args, **kwargs) def home(self): self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/HomeConnexion?type=homeconnex') def is_logged(self): return not self.is_on_page(LoginPage) def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.password.isdigit() if not self.is_on_page(LoginPage): self.home() self.page.login(self.username, self.password) self.location('/NSFR?Action=DSP_VGLOBALE', no_login=True) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() #self.readurl('/SAF_SOA?Action=6') def change_password(self, new_password): assert new_password.isdigit() and len(new_password) == 6 buf = self.readurl('https://www.secure.bnpparibas.net/NSFR?Action=SAF_CHM', if_fail='raise') buf = buf[buf.find('/SAF_CHM?Action=SAF_CHM'):] buf = buf[:buf.find('"')] self.location(buf) assert self.is_on_page(ChangePasswordPage) #self.readurl('/banque/portail/particulier/bandeau') #self.readurl('/common/vide.htm') self.page.change_password(self.password, new_password) if not self.is_on_page(ConfirmPage) or self.page.get_error() is not None: self.logger.error('Oops, unable to change password (%s)' % (self.page.get_error() if self.is_on_page(ConfirmPage) else 'unknown')) return self.password, self.rotating_password = (new_password, self.password) if self.password_changed_cb: self.password_changed_cb(self.rotating_password, self.password) def check_expired_password(func): def inner(self, *args, **kwargs): try: return func(self, *args, **kwargs) except BrowserPasswordExpired: if self.rotating_password is not None: warning('[%s] Your password has expired. Switching...' % self.username) self.change_password(self.rotating_password) return func(self, *args, **kwargs) else: raise return inner @check_expired_password def get_accounts_list(self): if not self.is_on_page(AccountsList): self.location('/NSFR?Action=DSP_VGLOBALE') return self.page.get_list() def get_account(self, id): assert isinstance(id, basestring) if not self.is_on_page(AccountsList): self.location('/NSFR?Action=DSP_VGLOBALE') l = self.page.get_list() for a in l: if a.id == id: return a return None def iter_history(self, account): if account.type == Account.TYPE_LIFE_INSURANCE: raise NotImplementedError() if account._link_id is None: return iter([]) if account._stp is not None: # Pro self.location(self.buildurl('/SAF_ROP', Origine='DSP_HISTOCPT', ch4=account._link_id, stp=account._stp)) else: # Perso if not self.is_on_page(AccountsList): self.location('/NSFR?Action=DSP_VGLOBALE') execution = self.page.document.xpath('//form[@name="goToApplication"]/input[@name="execution"]')[0].attrib['value'] data = {'gt': 'homepage:basic-theme', 'externalIAId': 'IAStatements', 'cboFlowName': 'flow/iastatement', 'contractId': account._link_id, 'groupId': '-2', 'pastOrPendingOperations': 1, 'groupSelected':'-2', 'step': 'STAMENTS', 'pageId': 'releveoperations', #'operationsPerPage': 100, #'_eventId': 'changeOperationsPerPage', 'sendEUD': 'true', 'execution': execution, } self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/FicheA', urllib.urlencode(data)) e = self.page.document.xpath('//form[@name="displayStatementForm"]/input[@name="_flowExecutionKey"]') if len(e) != 1: return iter([]) execution = e[0].attrib['value'] data = {'_eventId': 'changeOperationsPerPage', 'newCategoryId': '', 'categorisationInProgress': '', 'contractId': account._link_id, '_flowExecutionKey': execution, 'groupId': '-2', 'operations.objectsPerPage': 100, 'operations.pageNumber': 1, 'pageId': 'releveoperations', } # it's not a joke, BNP guys are really crappy. for i in xrange(30): data['_operations.list[%d].checkedOff' % i] = 'on' data['_operations.list[%d].selectedForCategorization' % i] = 'on' self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/FicheA', urllib.urlencode(data)) return self.page.iter_operations() def iter_coming_operations(self, account): if account._link_id is None: return iter([]) if account._stp is not None: # Pro self.location(self.buildurl('/NS_AVEDT', Origine='DSP_DT', ch4=account._link_id, stp=account._stp)) else: # Perso if not self.is_on_page(AccountsList): self.location('/NSFR?Action=DSP_VGLOBALE') execution = self.page.document.xpath('//form[@name="goToApplication"]/input[@name="execution"]')[0].attrib['value'] data = {'gt': 'homepage:basic-theme', 'externalIAId': 'IAStatements', 'cboFlowName': 'flow/iastatement', 'contractId': account._link_id, 'groupId': '-2', 'pastOrPendingOperations': 2, 'groupSelected':'-2', 'step': 'STAMENTS', 'pageId': 'mouvementsavenir', #'operationsPerPage': 100, #'_eventId': 'changeOperationsPerPage', 'sendEUD': 'true', 'execution': execution, } self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/FicheA', urllib.urlencode(data)) return self.page.iter_coming_operations() def iter_investment(self, account): if account.type == Account.TYPE_MARKET: if not account.iban or u'espèce' in account.label.lower(): return iter([]) stp = datetime.strftime(datetime.now(), '%Y%m%d%H%M%S') self.location('https://www.secure.bnpparibas.net/SAF_DPF?Origine=DSP_DPF&ch4=' + account.iban + 'stp=' + stp) data = {'ch4': account.iban, 'Origine': 'DSP_DPF', 'chD': 'oui', 'chT': 'sans', 'chU': 'alpha', 'x': randint(0, 99), 'y': randint(0, 18) } self.location('https://www.secure.bnpparibas.net/SAF_DPF', urllib.urlencode(data)) elif account.type == Account.TYPE_LIFE_INSURANCE: if not account._link_id: return iter([]) self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/Fiche?type=folder&identifiant=Assurance_Vie_Consultation_20071002051044&contrat=' + account._link_id) else: raise NotImplementedError() return self.page.iter_investment() @check_expired_password def get_transfer_accounts(self): if not self.is_on_page(TransferPage): self.location('/NS_VIRDF') assert self.is_on_page(TransferPage) return self.page.get_accounts() @check_expired_password def transfer(self, from_id, to_id, amount, reason=None): if not self.is_on_page(TransferPage): self.location('/NS_VIRDF') accounts = self.page.get_accounts() self.page.transfer(from_id, to_id, amount, reason) if not self.is_on_page(TransferCompletePage): raise TransferError('An error occured during transfer') transfer = Transfer(self.page.get_id()) transfer.amount = amount transfer.origin = accounts[from_id].label transfer.recipient = accounts[to_id].label transfer.date = datetime.now() return transfer def messages_page(self): if not self.is_on_page(MessagesPage): if not self.is_on_page(AccountsList): self.location('/NSFR?Action=DSP_VGLOBALE') self.location(self.page.get_messages_link()) assert self.is_on_page(MessagesPage) def iter_threads(self): self.messages_page() for thread in self.page.iter_threads(): yield thread def get_thread(self, thread): self.messages_page() if not hasattr(thread, '_link_id') or not thread._link_id: for t in self.iter_threads(): if t.id == thread.id: thread = t break # mimic validerFormulaire() javascript # yes, it makes no sense page_id, unread = thread._link_id self.select_form('listerMessages') self.form.set_all_readonly(False) self['identifiant'] = page_id if len(thread.id): self['idMessage'] = thread.id # the JS does this, but it makes us unable to read unread messages #if unread: # self['newMsg'] = thread.id self.submit() assert self.is_on_page(MessagePage) thread.root.content = self.page.get_content() return thread weboob-1.2/modules/bnporc/deprecated/perso/000077500000000000000000000000001303450110500210045ustar00rootroot00000000000000weboob-1.2/modules/bnporc/deprecated/perso/__init__.py000066400000000000000000000000001303450110500231030ustar00rootroot00000000000000weboob-1.2/modules/bnporc/deprecated/perso/accounts_list.py000066400000000000000000000117641303450110500242410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from decimal import Decimal from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.capabilities.bank import Account from weboob.capabilities.base import NotAvailable from weboob.deprecated.browser import Page, BrokenPageError, BrowserPasswordExpired class AccountsList(Page): ACCOUNT_TYPES = { u'Liquidités': Account.TYPE_CHECKING, u'Epargne disponible': Account.TYPE_SAVINGS, u'Titres': Account.TYPE_MARKET, u'Assurance vie': Account.TYPE_LIFE_INSURANCE, u'Capitalisation': Account.TYPE_LIFE_INSURANCE, u'Crédit immobilier': Account.TYPE_LOAN, } def on_loaded(self): pass def _parse_iban(self, account, url): m = re.search('ch4=(\w+)', url) if m: account.iban = unicode(m.group(1)) def _parse_account_group(self, table): typename = unicode(table.attrib.get('summary', '').replace('Liste des contrats/comptes ', '')) typeid = self.ACCOUNT_TYPES.get(typename, Account.TYPE_UNKNOWN) account = None for tr in table.xpath('.//tr'): if tr.find('td') is not None and tr.find('td').attrib.get('class', '') == 'typeTitulaire': account = self._parse_account(tr) account.type = typeid yield account elif tr.get('class', '') == 'listeActionBig' and account is not None: try: url = tr.xpath('.//a')[-1].get('href', '') except IndexError: pass else: self._parse_iban(account, url) account = None def _parse_account(self, tr): account = Account() # for pro usage account._stp = None account.id = tr.xpath('.//td[@class="libelleCompte"]/input')[0].attrib['id'][len('libelleCompte'):] if len(str(account.id)) == 23: account.id = str(account.id)[5:21] a = tr.xpath('.//td[@class="libelleCompte"]/a')[0] m = re.match(r'javascript:goToStatements\(\'(\d+)\'', a.get('onclick', '')) if m: account._link_id = m.group(1) else: # Find _link_id of life insurances m = re.match(r'javascript:overviewRedirectionOperation.*contrat=(\d+)', a.get('onclick', '')) if m: account._link_id = m.group(1) else: # Can't get history for this account. account._link_id = None # To prevent multiple-IDs for CIF (for example), add an arbitrary char in ID. account.id += 'C' account.label = u''+a.text.strip() tds = tr.findall('td') account.currency = account.get_currency(tds[3].find('a').text) account.balance = self._parse_amount(tds[3].find('a')) if tds[4].find('a') is not None: account.coming = self._parse_amount(tds[4].find('a')) else: account.coming = NotAvailable return account def _parse_amount(self, elem): return Decimal(FrenchTransaction.clean_amount(elem.text)) def get_list(self): accounts = [] for table in self.document.xpath('//table[@class="tableCompte"]'): for account in self._parse_account_group(table): accounts.append(account) if len(accounts) == 0: # oops, no accounts? check if we have not exhausted the allowed use # of this password for img in self.document.getroot().cssselect('img[align="middle"]'): if img.attrib.get('alt', '') == 'Changez votre code secret': raise BrowserPasswordExpired('Your password has expired') return accounts def get_execution_id(self): return self.document.xpath('//input[@name="_flowExecutionKey"]')[0].attrib['value'] def get_messages_link(self): """ Get the link to the messages page, which seems to have an identifier in it. """ for link in self.parser.select(self.document.getroot(), 'div#pantalon div.interieur a'): if 'MessagesRecus' in link.attrib.get('href', ''): return link.attrib['href'] raise BrokenPageError('Unable to find the link to the messages page') class AccountPrelevement(AccountsList): pass weboob-1.2/modules/bnporc/deprecated/perso/investment.py000066400000000000000000000057531303450110500235640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from decimal import Decimal from xml.etree import ElementTree from weboob.deprecated.browser import Page from weboob.capabilities.bank import Investment from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.capabilities.base import NotAvailable def clean_text(el): text = ElementTree.tostring(el, 'utf-8', 'text').decode('utf-8') return re.sub(ur'[\s\xa0]+', u' ', text).strip() def clean_cells(cells): return list(map(clean_text, cells)) def clean_amount(amount): return Decimal(FrenchTransaction.clean_amount(amount)) if amount and amount != ' NS' else NotAvailable def clean_amounts(amounts): return list(map(clean_amount, amounts)) class AccountMarketInvestment(Page): def iter_investment(self): table = self.document.xpath('//table[@align="center"]')[4] rows = table.xpath('.//tr[@class="hdoc1"]') for tr in rows: cells = clean_cells(tr.findall('td')) cells[2:] = clean_amounts(cells[2:]) inv = Investment() inv.label, _, inv.quantity, inv.unitvalue, inv.valuation = cells tr2 = tr.xpath('./following-sibling::tr')[0] tr2td = tr2.findall('td')[1] inv.id = inv.code = clean_text(tr2.xpath('.//a')[0]) inv.unitprice = clean_amount(tr2td.xpath('.//td[@class="hdotc1nb"]')[0].text) inv.description = u'' if inv.unitprice: inv.diff = inv.quantity * inv.unitprice - inv.valuation yield inv class AccountLifeInsuranceInvestment(Page): def iter_investment(self): rows = self.document.xpath('//table[@id="mefav_repartition_supports_BPF"]//tr') or \ self.document.xpath('//tbody[@id="mefav_repartition_supports"]//tr') for tr in rows: cells = clean_cells(tr.findall('td')) cells[3:] = clean_amounts(cells[3:]) inv = Investment() inv.label, _, inv.code, inv.quantity, inv.unitvalue, inv.valuation, _ = cells if inv.code: inv.id = inv.code if not inv.unitvalue: # XXX Fonds eu Euros inv.code = u'XX' + re.sub(ur'[^A-Za-z0-9]', u'', inv.label).upper() inv.description = u'' yield inv weboob-1.2/modules/bnporc/deprecated/perso/login.py000066400000000000000000000125011303450110500224650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2011 Romain Bignon, Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time import re import urllib from weboob.deprecated.browser import Page, BrowserUnavailable from weboob.tools.captcha.virtkeyboard import VirtKeyboard, VirtKeyboardError class BNPVirtKeyboard(VirtKeyboard): symbols={'0': '9cc4789a2cb223e8f2d5e676e90264b5', '1': 'e10b58fc085f9683052d5a63c96fc912', '2': '04ec647e7b3414bcc069f0c54eb55a4c', '3': 'fde84fd9bac725db8463554448f1e469', '4': '2359eea8671bf112b58264bec0294f71', '5': '82b55b63480114f04fad8c5c4fa5673a', '6': 'e074864faeaeabb3be3d118192cd8879', '7': 'af5740e4ca71fadc6f4ae1412d864a1c', '8': 'cab759c574038ad89a0e35cc76ab7214', '9': '828cf0faf86ac78e7f43208907620527' } url="/NSImgGrille?timestamp=%d" color=27 def __init__(self, page): coords = {} size = 136 x, y, width, height = (0, 0, size/5, size/5) for a in page.document.xpath('//div[@id="secret-nbr-keyboard"]/a'): code = a.attrib['ondblclick'] coords[code] = (x+1, y+1, x+height-2, y+height-2) if (x + width + 1) >= size: y += height x = 0 else: x += width VirtKeyboard.__init__(self, page.browser.openurl(self.url % time.time()), coords, self.color) self.check_symbols(self.symbols, page.browser.responses_dirname) def get_symbol_code(self, md5sum): code = VirtKeyboard.get_symbol_code(self, md5sum) return re.sub(u'[^\d]', '', code) def get_string_code(self, string): code = '' for c in string: code += self.get_symbol_code(self.symbols[c]) return code class LoginPage(Page): def on_loaded(self): for td in self.document.getroot().cssselect('td.LibelleErreur'): if td.text is None: continue msg = td.text.strip() if 'indisponible' in msg: raise BrowserUnavailable(msg) def login(self, login, password): try: vk=BNPVirtKeyboard(self) except VirtKeyboardError as err: self.logger.error("Error: %s"%err) return False # Mechanize does not recognize the form.. form = self.document.xpath('//form[@name="logincanalnet"]')[0] url = form.attrib['action'] params = {} for ctrl in form.findall('input'): params[ctrl.attrib['name']] = ctrl.attrib['value'] params['ch1'] = login.encode('iso-8859-1') params['ch5'] = vk.get_string_code(password) self.browser.location(url, urllib.urlencode(params)) class ConfirmPage(Page): def get_error(self): for td in self.document.xpath('//td[@class="hdvon1"]'): if td.text: return td.text.strip() return None def get_relocate_url(self): script = self.document.xpath('//script')[0] m = re.match('document.location.replace\("(.*)"\)', script.text[script.text.find('document.location.replace'):]) if m: return m.group(1) class InfoMessagePage(Page): def on_loaded(self): pass class ChangePasswordPage(Page): def change_password(self, current, new): try: vk=BNPVirtKeyboard(self) except VirtKeyboardError as err: self.logger.error("Error: %s"%err) return False from mechanize import Cookie c = Cookie(0, 'wbo_segment_369721', 'AA%7CAB%7C%7C%7C', None, False, '.secure.bnpparibas.net', True, True, '/', False, False, None, False, None, None, {}) cookiejar = self.browser._ua_handlers["_cookies"].cookiejar cookiejar.set_cookie(c) code_current=vk.get_string_code(current) code_new=vk.get_string_code(new) data = (('ch1', code_current), ('ch2', code_new), ('radiobutton3', 'radiobutton'), ('ch3', code_new), ('x', 23), ('y', 13), ) headers = {'Referer': self.url} #headers = {'Referer': "https://www.secure.bnpparibas.net/SAF_CHM?Action=SAF_CHM&Origine=SAF_CHM&stp=%s" % (int(datetime.now().strftime('%Y%m%d%H%M%S')))} #import time #time.sleep(10) request = self.browser.request_class('https://www.secure.bnpparibas.net/SAF_CHM_VALID', urllib.urlencode(data), headers) self.browser.location(request) weboob-1.2/modules/bnporc/deprecated/perso/messages.py000066400000000000000000000063471303450110500231770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page, BrokenPageError from weboob.capabilities.messages import Message, Thread from weboob.capabilities.base import NotLoaded from weboob.tools.capabilities.messages.genericArticle import try_drop_tree import re from datetime import datetime from lxml.html import make_links_absolute class MessagesPage(Page): def iter_threads(self): table = self.parser.select(self.document.getroot(), 'table#listeMessages', 1) for tr in table.xpath('./tr'): if tr.attrib.get('class', '') not in ('msgLu', 'msgNonLu'): continue author = unicode(self.parser.select(tr, 'td.colEmetteur', 1).text) link = self.parser.select(tr, 'td.colObjet a', 1) date_raw = self.parser.select(tr, 'td.colDate1', 1).attrib['data'] jsparams = re.search('\((.+)\)', link.attrib['onclick']).groups()[0] jsparams = [i.strip('\'" ') for i in jsparams.split(',')] page_id, _id, unread = jsparams # this means unread on the website unread = False if unread == "false" else True # 2012/02/29:01h30min45sec dt_match = re.match('(\d+)/(\d+)/(\d+):(\d+)h(\d+)min(\d+)sec', date_raw).groups() dt_match = [int(d) for d in dt_match] thread = Thread(_id) thread._link_id = (page_id, unread) thread.date = datetime(*dt_match) thread.title = unicode(link.text) message = Message(thread, 0) message.set_empty_fields(None) message.flags = message.IS_HTML message.title = thread.title message.date = thread.date message.sender = author message.content = NotLoaded # This is the only thing we are missing thread.root = message yield thread class MessagePage(Page): def get_content(self): """ Get the message content. This page has a date, but it is less precise than the main list page, so we only use it for the message content. """ try: content = self.parser.select(self.document.getroot(), 'div.txtMessage div.contenu', 1) except BrokenPageError: # This happens with some old messages (2007) content = self.parser.select(self.document.getroot(), 'div.txtMessage', 1) content = make_links_absolute(content, self.url) try_drop_tree(self.parser, content, 'script') return self.parser.tostring(content) weboob-1.2/modules/bnporc/deprecated/perso/transactions.py000066400000000000000000000110001303450110500240560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.deprecated.browser import Page from weboob.tools.capabilities.bank.transactions import FrenchTransaction class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PCHEQUE)(?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(?PFACTURE CARTE) DU (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*?)( CA?R?T?E? ?\d*X*\d*)?$'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P(PRELEVEMENT|TELEREGLEMENT|TIP)) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PPRLV( EUROPEEN)? SEPA) (?P.*?)( MDT/.*?)?( ECH/\d+)?( ID .*)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PECHEANCEPRET)(?P.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(?PRETRAIT DAB) (?P
\d{2})/(?P\d{2})/(?P\d{2})( (?P\d+)H(?P\d+))?( \d+)? (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?PVIR(EMEN)?T? (RECU |FAVEUR )?(TIERS )?)\w+ \d+/\d+ \d+H\d+ \w+ (?P.*)$'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PVIR(EMEN)?T? (EUROPEEN )?(SEPA )?(RECU |FAVEUR |EMIS )?(TIERS )?)(/FRM |/DE |/MOTIF |/BEN )?(?P.*?)(/.+)?$'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PREMBOURST) CB DU (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PREMBOURST)(?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PCOMMISSIONS)(?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P(?PREMUNERATION).*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?PREMISE CHEQUES)(?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] class AccountHistory(Page): def iter_operations(self): for tr in self.document.xpath('//table[@id="tableCompte"]//tr'): if len(tr.xpath('td[@class="debit"]')) == 0: continue id = tr.find('td').find('input').attrib['id'].lstrip('_') op = Transaction(id) op.parse(date=tr.findall('td')[1].text, raw=tr.findall('td')[2].text.replace(u'\xa0', u'')) debit = tr.xpath('.//td[@class="debit"]')[0].text credit = tr.xpath('.//td[@class="credit"]')[0].text op.set_amount(credit, debit) yield op def iter_coming_operations(self): i = 0 for tr in self.document.xpath('//table[@id="tableauOperations"]//tr'): if 'typeop' in tr.attrib: tds = tr.findall('td') if len(tds) != 3: continue text = tds[1].text or u'' text = text.replace(u'\xa0', u'') for child in tds[1].getchildren(): if child.text: text += child.text if child.tail: text += child.tail i += 1 operation = Transaction(i) operation.parse(date=tr.attrib['dateop'], raw=text) operation.set_amount(tds[2].text) yield operation class AccountComing(AccountHistory): pass weboob-1.2/modules/bnporc/deprecated/perso/transfer.py000066400000000000000000000075621303450110500232140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.deprecated.browser import Page, BrowserPasswordExpired from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.bank import TransferError class Account(object): def __init__(self, id, label, send_checkbox, receive_checkbox): self.id = id self.label = label self.send_checkbox = send_checkbox self.receive_checkbox = receive_checkbox class TransferPage(Page): def on_loaded(self): for td in self.document.xpath('//td[@class="hdvon1"]'): if td.text and 'Vous avez atteint le seuil de' in td.text: raise BrowserPasswordExpired(td.text.strip()) def get_accounts(self): accounts = OrderedDict() for table in self.document.getiterator('table'): if table.attrib.get('cellspacing') == '2': for tr in table.cssselect('tr.hdoc1, tr.hdotc1'): tds = tr.findall('td') id = tds[1].text.replace(u'\xa0', u'') label = tds[0].text if label is None and tds[0].find('nobr') is not None: label = tds[0].find('nobr').text send_checkbox = tds[4].find('input').attrib['value'] if tds[4].find('input') is not None else None receive_checkbox = tds[5].find('input').attrib['value'] if tds[5].find('input') is not None else None account = Account(id, label, send_checkbox, receive_checkbox) accounts[id] = account return accounts def transfer(self, from_id, to_id, amount, reason): accounts = self.get_accounts() # Transform RIBs to short IDs if len(str(from_id)) == 23: from_id = str(from_id)[5:21] if len(str(to_id)) == 23: to_id = str(to_id)[5:21] try: sender = accounts[from_id] except KeyError: raise TransferError('Account %s not found' % from_id) try: recipient = accounts[to_id] except KeyError: raise TransferError('Recipient %s not found' % to_id) if sender.send_checkbox is None: raise TransferError('Unable to make a transfer from %s' % sender.label) if recipient.receive_checkbox is None: raise TransferError('Unable to make a transfer to %s' % recipient.label) self.browser.select_form(nr=0) self.browser['C1'] = [sender.send_checkbox] self.browser['C2'] = [recipient.receive_checkbox] self.browser['T6'] = str(amount).replace('.', ',') if reason: self.browser['T5'] = reason.encode('utf-8') self.browser.submit() class TransferConfirmPage(Page): def on_loaded(self): for td in self.document.getroot().cssselect('td#size2'): raise TransferError(td.text.strip()) for a in self.document.getiterator('a'): m = re.match('/NSFR\?Action=VIRDA&stp=(\d+)', a.attrib['href']) if m: self.browser.location('/NS_VIRDA?stp=%s' % m.group(1)) return class TransferCompletePage(Page): def get_id(self): return self.group_dict['id'] weboob-1.2/modules/bnporc/deprecated/pro.py000066400000000000000000000126761303450110500210420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from urlparse import urlparse, parse_qsl from decimal import Decimal, InvalidOperation from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account from weboob.deprecated.browser import Page from .perso.transactions import Transaction class ProAccountsList(Page): COL_LABEL = 1 COL_ID = 2 COL_BALANCE = 3 COL_COMING = 5 def on_loaded(self): for msg in self.document.xpath('//div[@id="idMessageInfoUdC"]'): raise BrowserIncorrectPassword(self.parser.tocleanstring(msg)) def get_list(self, pro=True): accounts = [] for tr in self.document.xpath('//tr[@class="comptes"]'): cols = tr.findall('td') if len(cols) < 5: continue account = Account() account.id = self.parser.tocleanstring(cols[self.COL_ID]).replace(" ", "") account.label = self.parser.tocleanstring(cols[self.COL_LABEL]) account.balance = Decimal(self.parser.tocleanstring(cols[self.COL_BALANCE])) try: account.coming = Decimal(self.parser.tocleanstring(cols[self.COL_COMING])) except InvalidOperation: if self.parser.tocleanstring(cols[self.COL_COMING]) != '-': self.logger.warning('Unable to parse coming value', exc_info=True) account.coming = NotAvailable account._link_id = None account._stp = None a = cols[self.COL_LABEL].find('a') if a is not None: url = urlparse(a.attrib['href']) p = dict(parse_qsl(url.query)) account._link_id = p.get('ch4', None) account._stp = p.get('stp', None) for input_tag in tr.xpath('.//input[starts-with(@id, "urlRib")]'): m = re.search('ch4=(\w+)', input_tag.get('value', '')) if m: account.iban = unicode(m.group(1)) break else: select = tr.xpath('.//select//@onchange')[0] m = re.search("\(this,'(\w+)", select) if m: iban = unicode(m.group(1)) if iban.startswith('FR') and len(iban) == 27: account.iban = unicode(m.group(1)) accounts.append(account) # If there are also personnal accounts linked, display the page and iter on them. if pro and len(self.document.xpath('//div[@class="onglets"]//a[contains(@href, "afficherComptesPrives")]')) > 0: self.browser.select_form(name='myForm') self.browser.set_all_readonly(False) self.browser['udcAction'] = '/afficherComptesPrives' self.browser.submit() for a in self.browser.page.get_list(False): accounts.append(a) return accounts class ProAccountHistory(Page): COL_DATE = 0 COL_LABEL = 1 COL_DEBIT = -2 COL_CREDIT = -1 def on_loaded(self): # If transactions are ordered by type, force order by date. try: checkbox = self.document.xpath('//input[@name="szTriDate"]')[0] except IndexError: return if 'checked' not in checkbox.attrib: self.browser.select_form(name='formtri') self.browser['szTriDate'] = ['date'] self.browser['szTriRub'] = [] self.browser.submit() def iter_operations(self): for i, tr in enumerate(self.document.xpath('//tr[@class="hdoc1" or @class="hdotc1"]')): cols = tr.findall('td') if len(cols) < 4: continue op = Transaction(i) date = self.parser.tocleanstring(cols[self.COL_DATE]) raw = self.parser.tocleanstring(cols[self.COL_LABEL]) raw = re.sub(r'[ \xa0]+', ' ', raw).strip() op.parse(date=date, raw=raw) debit = self.parser.tocleanstring(cols[self.COL_DEBIT]) credit = self.parser.tocleanstring(cols[self.COL_CREDIT]) op.set_amount(credit, debit) yield op def iter_coming_operations(self): for i, tr in enumerate(self.document.xpath('//tr[@class="hdoc1" or @class="hdotc1"]')): cols = tr.findall('td') if len(cols) < 4: continue op = Transaction(i) date = self.parser.tocleanstring(cols[self.COL_DATE]) raw = self.parser.tocleanstring(cols[self.COL_LABEL]) raw = re.sub(r'[ \xa0]+', ' ', raw).strip() op.parse(date=date, raw=raw) credit = self.parser.tocleanstring(cols[self.COL_CREDIT]) op.set_amount(credit) yield op weboob-1.2/modules/bnporc/enterprise/000077500000000000000000000000001303450110500177345ustar00rootroot00000000000000weboob-1.2/modules/bnporc/enterprise/__init__.py000066400000000000000000000000001303450110500220330ustar00rootroot00000000000000weboob-1.2/modules/bnporc/enterprise/browser.py000066400000000000000000000125011303450110500217700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Jean Walrave # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from dateutil.rrule import rrule, MONTHLY from dateutil.relativedelta import relativedelta from weboob.browser import LoginBrowser, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.url import URL from .pages import LoginPage, AuthPage, AccountsPage, AccountHistoryViewPage, AccountHistoryPage, ActionNeededPage __all__ = ['BNPEnterprise'] class BNPEnterprise(LoginBrowser): BASEURL = 'https://secure1.entreprises.bnpparibas.net' login = URL('/sommaire/jsp/identification.jsp', '/sommaire/generateImg', LoginPage) auth = URL('/sommaire/PseMenuServlet', AuthPage) accounts = URL('/NCCPresentationWeb/e10_soldes/liste_soldes.do', AccountsPage) account_history_view = URL('/NCCPresentationWeb/e11_releve_op/init.do\?identifiant=(?P)' + \ '&typeSolde=(?P)&typeReleve=(?P)&typeDate=(?P)' + \ '&dateMin=(?P)&dateMax=(?P)&ajax=true', '/NCCPresentationWeb/e11_releve_op/init.do', AccountHistoryViewPage) account_coming_view = URL('/NCCPresentationWeb/m04_selectionCompteGroupe/init.do\?type=compte&identifiant=(?P)', AccountHistoryViewPage) account_history = URL('/NCCPresentationWeb/e11_releve_op/listeOperations.do\?identifiant=(?P)' + \ '&dateMin=(?P)&dateMax=(?P)', '/NCCPresentationWeb/e11_releve_op/listeOperations.do', AccountHistoryPage) account_coming = URL('/NCCPresentationWeb/e12_rep_cat_op/listOperations.do\?periode=date_valeur&identifiant=(?P)', '/NCCPresentationWeb/e12_rep_cat_op/listOperations.do', AccountHistoryPage) renew_pass = URL('/sommaire/PseRedirectPasswordConnect', ActionNeededPage) def __init__(self, *args, **kwargs): super(BNPEnterprise, self).__init__(*args, **kwargs) self.cache = {} self.cache['transactions'] = {} self.cache['coming_transactions'] = {} def do_login(self): self.login.go() if self.login.is_here() is False: return data = {} data['txtAuthentMode'] = 'PASSWORD' data['BEFORE_LOGIN_REQUEST'] = None data['txtPwdUserId'] = self.username data['gridpass_hidden_input'] = self.page.get_password(self.password) self.auth.go(data=data) if self.login.is_here(): raise BrowserIncorrectPassword @need_login def get_accounts_list(self): if "accounts" not in self.cache.keys(): self.cache['accounts'] = [a for a in self.accounts.stay_or_go().iter_accounts()] return self.cache['accounts'] @need_login def get_account(self, _id): for account in self.get_accounts_list(): if account.id == _id: return account @need_login def iter_history(self, account): if account.id not in self.cache['transactions']: dformat = "%Y%m%d" self.cache['transactions'][account.id] = [] for date in rrule(MONTHLY, dtstart=(datetime.now() - relativedelta(months=3)), until=datetime.now()): self.account_history_view.go(identifiant=account.iban, type_solde='C', type_releve='Comptable', \ type_date='O', date_min=(date + relativedelta(days=1)).strftime(dformat), \ date_max=(date + relativedelta(months=1)).strftime(dformat)) self.account_history.go(identifiant=account.iban, date_min=(date + relativedelta(days=1)).strftime(dformat), \ date_max=(date + relativedelta(months=1)).strftime(dformat)) for transaction in [t for t in self.page.iter_history() if t._coming is False]: self.cache['transactions'][account.id].append(transaction) self.cache['transactions'][account.id].sort(key=lambda t: t.date, reverse=True) return self.cache['transactions'][account.id] @need_login def iter_coming_operations(self, account): if account.id not in self.cache['coming_transactions']: self.account_coming_view.go(identifiant=account.iban) self.account_coming.go(identifiant=account.iban) self.cache['coming_transactions'][account.id] = [t for t in self.page.iter_coming()] return self.cache['coming_transactions'][account.id] @need_login def iter_investment(self, account): raise NotImplementedError() weboob-1.2/modules/bnporc/enterprise/pages.py000066400000000000000000000161331303450110500214110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Jean Walrave # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import datetime from cStringIO import StringIO from weboob.browser.pages import LoggedPage, HTMLPage, JsonPage from weboob.browser.filters.json import Dict from weboob.browser.elements import DictElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Date, Format, Eval from weboob.capabilities.bank import Transaction, Account from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError from weboob.capabilities import NotAvailable from weboob.exceptions import ActionNeeded class BNPVirtKeyboard(MappedVirtKeyboard): symbols = {'0': '91d2887b619ec825bb622c7770d4c2dc', '1': '9fe87bb481bde31b01c5ea434fbb391c', '2': '80c24c1586868830b8f578e41167996a', '3': 'dd4d989b1721506884914edbc1df3b91', '4': '38dd990feb7c40573e526fb69e2f17a9', '5': '579acb65bd5e98fcc413070192477528', '6': 'e133ed4e7c4c0028a2a0a7e9126751b4', '7': 'ae012ad7e1314571aef2343f40235d3c', '8': 'a619519f61da73124a2705544c45fb42', '9': 'fa625a1d4dc8357ec8eb87929bacd197', } color = (0, 0, 0) def __init__(self, page): img = page.doc.find('//img[@usemap="#gridpass_map_name"]') res = page.browser.open(img.attrib['src']) MappedVirtKeyboard.__init__(self, StringIO(res.content), page.doc, img, self.color, convert='RGB') self.check_symbols(self.symbols, None) def check_color(self, pixel): return pixel[0] < 100 def get_symbol_coords(self, coords): x1, y1, x2, y2 = coords return MappedVirtKeyboard.get_symbol_coords(self, (x1 + 6, y1 + 1, x2 - 6, y2 - 4)) def get_symbol_code(self, md5sum): m = re.search('(\d+)', MappedVirtKeyboard.get_symbol_code(self, md5sum)) if m: return m.group(1) class LoginPage(HTMLPage): def get_password(self, password): vk_passwd = None try: vk = BNPVirtKeyboard(self) vk_passwd = vk.get_string_code(password) except VirtKeyboardError as e: self.logger.error(e) return vk_passwd class AuthPage(HTMLPage): pass class ActionNeededPage(HTMLPage): def on_load(self): raise ActionNeeded(CleanText('//p[@class="message"]')(self.doc)) class AccountsPage(LoggedPage, JsonPage): TYPES = {u'Compte chèque': Account.TYPE_CHECKING} @method class iter_accounts(DictElement): item_xpath = "tableauSoldes/listeGroupes/*/listeComptes" class item(ItemElement): klass = Account def obj_id(self): return CleanText(Dict('numeroCompte'))(self)[2:] obj_label = CleanText(Dict('libelleCompte')) obj_currency = CleanText(Dict('deviseTenue')) def obj_balance(self): return Eval(lambda x, y: x / 10**y, CleanDecimal(Dict('soldeComptable')), CleanDecimal(Dict('decSoldeComptable')))(self) def obj_coming(self): return Eval(lambda x, y: x / 10**y, CleanDecimal(Dict('soldePrevisionnel')), CleanDecimal(Dict('decSoldePrevisionnel')))(self) obj_iban = CleanText(Dict('numeroCompte', default=None), default=NotAvailable) def obj_type(self): return self.page.TYPES.get(Dict('libelleType')(self), Account.TYPE_UNKNOWN) class AccountHistoryViewPage(LoggedPage, HTMLPage): pass def fromtimestamp(page, dict): return datetime.fromtimestamp(float(dict(page) / 1000)) class AccountHistoryPage(LoggedPage, JsonPage): TYPES = {u'CARTE': Transaction.TYPE_CARD, # Cartes u'CHEQU': Transaction.TYPE_CHECK, # Chèques u'REMCB': Transaction.TYPE_CARD, # Remises cartes u'VIREM': Transaction.TYPE_TRANSFER, # Virements u'VIRIT': Transaction.TYPE_TRANSFER, # Virements internationaux u'VIRSP': Transaction.TYPE_TRANSFER, # Virements européens u'VIRTR': Transaction.TYPE_TRANSFER, # Virements de trésorerie u'VIRXX': Transaction.TYPE_TRANSFER, # Autres virements u'PRLVT': Transaction.TYPE_LOAN_PAYMENT, # Prélèvements, TIP et télérèglements u'AUTOP': Transaction.TYPE_UNKNOWN, # Autres opérations } COMING_TYPES = {u'0083': Transaction.TYPE_DEFERRED_CARD, u'0813': Transaction.TYPE_LOAN_PAYMENT, u'0568': Transaction.TYPE_TRANSFER, } @method class iter_history(DictElement): item_xpath = "mouvementsBDDF" class item(ItemElement): klass = Transaction obj_original_currency = CleanText(Dict('montant/devise')) obj__coming = Dict('aVenir') def obj_raw(self): return CleanText(Dict('libelle'))(self) or CleanText(Dict('nature/libelle'))(self) def obj_type(self): return self.page.TYPES.get(Dict('nature/codefamille')(self), Transaction.TYPE_UNKNOWN) def obj_date(self): return fromtimestamp(self, Dict('dateOperation')) def obj_rdate(self): return fromtimestamp(self, Dict('dateCreation')) def obj_vdate(self): return fromtimestamp(self, Dict('dateValeur')) def obj_amount(self): return Eval(lambda x, y: x / 10**y, CleanDecimal(Dict('montant/montant')), CleanDecimal(Dict('montant/nb_dec')))(self) @method class iter_coming(DictElement): item_xpath = "infoOperationsAvenir/operationsAvenir" class item(ItemElement): klass = Transaction obj_date = Date(Dict('dateOpeMvmt')) obj_rdate = Date(Dict('dateCreatMvmt')) obj_vdate = Date(Dict('dateValMvmt')) obj_original_currency = CleanText(Dict('montantMvmt/devise')) def obj_raw(self): if not Dict('natureLibelleMvt')(self): return CleanText(Dict('libelle'))(self) return Format('%s %s', CleanText(Dict('natureLibelleMvt')), CleanText(Dict('libelle')))(self) def obj_type(self): return self.page.COMING_TYPES.get(Dict('codeMouvement')(self), Transaction.TYPE_UNKNOWN) def obj_amount(self): return Eval(lambda x, y: x / 10**y, CleanDecimal(Dict('montantMvmt/montant')), CleanDecimal(Dict('montantMvmt/nb_dec')))(self) weboob-1.2/modules/bnporc/favicon.png000066400000000000000000000121101303450110500177020ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME  xtEXtCommentCreated with GIMPWIDATxy]uu?wιӛbˠ4&A DK2XXPBWY]UZ&H*Q  Ȥ( {;oe}{gs~OqTK ?Tʻ6 $}?=yЊKA e% *jdV@_ jP ̏6(UbC%jDHAs N Z-Zd&j\yEU#%jE{dDTvd6(c_|Cr\8]h̗KBYԵδU,b|gO*> f2cS8>K\^_{ᝁC%^d&CFj5hv 2QIPCP~!!,/evk߹q]\u& _kmVPp[\ ZCRV-*/".Vq ]Pbu wp DjEQʫjn|X1S""B )ܭ9#ܮ.5JS5TSܻwDCGmŤ:6NˢtATc_9 9T:>UNX BޡUZAOTŊ`W{WKKZUQI%/E_hBZa+e5Cf(R"1f ;j'9'1vI 7Fp"ڊFMNXتM&bx;r} 2llZkǐ?ځ` ZER OcoX_>qgu!WeKxv+*xLjF۲nq< tEz/ΗӚ֥76b9Fh_k]%'{(e4{Xs)j #,Y,ٛ9f.% {RvvI$jX<vj&$>`& z=hͭ,X{͍z]p=y\8%)3 /llwz8븫*G,+ g5?}whQx]$`B)I;MyE*W8Ήf̧O̽ D. /㣥QGIcU73ok)ʹ2blrLnK(9x /JoDž \ɭ8YeG6wd$7ˉɟd:rkk Nj9L'S]s$~E\뭡]-fVOZ5 fB&7&I6v3m̒ hm >񉽒g"'8DSo*ȼuMY$ (ۇ;Jw}Q>XKpJ{U3{͎l\A5(iiIn;%`vX||JYSO':'18_?ϲh>qnjڸTW[$KEjT*nn|z/gd\t 6E_8kS[k*nO%bb>am6u#1"ľe^g2؀`KUZT&NsNOr/@ I Ӱu8\Αst_oc|T^D4Ԑ{wc\SFQA; ?|qC\.-`9AaGjc;3EAEZiu_Nܴ7{H %SR:iLHBfl wXlgFy>'eJJR\E`}hi1J`YOiea6;IrfLN=E" : > G/_ s) Ke)ۋۥVQ1)XONu*W矣_H%SPT [}N]`SՐuùI%tk!wbSTSD퀫Q,h!4EHA ϱp\_JQu1ţi4N?c2\J"}===d'KjKZ:L πcnɯRufAJY=,]8\ce26ޜ;Bז"P֊+_SS:C\/kxyDc˹VLbqY)"Eq! 8A/xz%q|B-E`4 u%S#> B`}4G Z>h­/̏Ѡ֤~u()Sӝ(+l!CwGhJI(P:] bFсرQGknY:K,V 8Lށ3,PHb'=&jc"i:|w 4Q-!z4Z9R~+XPx^^Qu>Cކ|ːv0-| ua$f]e5SPBG4*I٫H8I꒓2(|E'tp!B8Ǟs9QQewyL.rLNQpcϝe(d +?Еn-:7=9~[#'㽛)J1%!5seMvTLƣĸ=M啐Xw*P]xyGXa8'zo[lQ:qI #y~͸|^ˑ'O> -~xɀX XĻ)љre4Ͽ\nx{q4w'AeҶ]^^`$e-3>v\RA仍oQ@Rf^Ŷ`kV0 l_?.vrtUK:Qz%㯺Lobp,:~ssIqyD;6!'39|eQs_Gx,h.@:'xcelh i6fţI,)psm;hQ!!X4 @4^3OS.|w5ZzvEӥt^0IN01DH":[UmG)(ڥH4炳p$Q\ w,2 uSMN3'Wx ɭ"B$3Z38Y\x;s_Q'N4ސ5J̉@- t8QQcSCT"Sp.?] ɜB\Mٔ!B\E!2ʰ ˰ 3"#ԥN(ԩ_?E_RȪ* Fg}OBJ%yҠp`&(_{>}Ve).4вv/ǰu6~5hQ&UYֻuPq*xx8 po^-r;$lq~`fsvAEH;EY[݋.[D|@~9u;~{O/0 *7MTPB[1doyGS G&>1 yCpl+nә:,nj2~#h!a|nܷX'RI%i_'/ urd\P9_̀ a {g9noE l~ZN18NVsLN5;H0VEFAb>>}6zV{L/v@皹\UTQM Kv$MKډSvZ| Sm܅-/Ҧqۛ4n{oqpl@ m$b~4VԔz[emKpڢqZq脵ly)-kT`b^&d_@bP\dFVmNeF I*ҷh17^^@AӍN/kˍf0 c31x%ljG.k DD ,յ}0~= qZa#W'*:iLjiW]I,Q#`gҿ*Β"d}KlhlQ1ުmIENDB`weboob-1.2/modules/bnporc/module.py000066400000000000000000000163671303450110500174300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from datetime import datetime, timedelta from weboob.capabilities.bank import CapBankTransfer, AccountNotFound, \ Account, TransferError, RecipientNotFound from weboob.capabilities.messages import CapMessages, Thread from weboob.capabilities.contact import CapContact from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .deprecated.browser import BNPorc from .enterprise.browser import BNPEnterprise from .company.browser import BNPCompany from .pp.browser import BNPPartPro, HelloBank __all__ = ['BNPorcModule'] class BNPorcModule(Module, CapBankTransfer, CapMessages, CapContact): NAME = 'bnporc' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'BNP Paribas' CONFIG = BackendConfig( ValueBackendPassword('login', label=u'Numéro client', masked=False), ValueBackendPassword('password', label=u'Code secret', regexp='^(\d{6}|)$'), #ValueBackendPassword('rotating_password', default='', # label='Password to set when the allowed uses are exhausted (6 digits)', # regexp='^(\d{6}|)$'), Value('website', label='Type de compte', default='pp', choices={'pp': 'Particuliers/Professionnels', 'hbank': 'HelloBank', 'ent': 'Entreprises', 'ppold': 'Particuliers/Professionnels (ancien site)', 'ent2': 'Entreprises et PME (nouveau site)'})) STORAGE = {'seen': []} # Store the messages *list* for this duration CACHE_THREADS = timedelta(seconds=3 * 60 * 60) def __init__(self, *args, **kwargs): Module.__init__(self, *args, **kwargs) self._threads = None self._threads_age = datetime.utcnow() def create_default_browser(self): b = {'ppold': BNPorc, 'ent': BNPEnterprise, 'ent2': BNPCompany, 'pp': BNPPartPro, 'hbank': HelloBank} self.BROWSER = b[self.config['website'].get()] if self.BROWSER is BNPPartPro: return self.create_browser(self.config) return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): return self.browser.iter_history(account) def iter_coming(self, account): return self.browser.iter_coming_operations(account) def iter_investment(self, account): return self.browser.iter_investment(account) def iter_transfer_recipients(self, origin_account): if self.config['website'].get() != 'pp': raise NotImplementedError() if isinstance(origin_account, Account): origin_account = origin_account.id return self.browser.iter_recipients(origin_account) def init_transfer(self, transfer, **params): if self.config['website'].get() != 'pp': raise NotImplementedError() if transfer.label is None: raise TransferError(u'Veuillez préciser un libellé au virement') self.logger.info('Going to do a new transfer') if transfer.account_iban: account = find_object(self.iter_accounts(), iban=transfer.account_iban, error=AccountNotFound) else: account = find_object(self.iter_accounts(), id=transfer.account_id, error=AccountNotFound) if transfer.recipient_iban: recipient = find_object(self.iter_transfer_recipients(account.id), iban=transfer.recipient_iban, error=RecipientNotFound) else: recipient = find_object(self.iter_transfer_recipients(account.id), id=transfer.recipient_id, error=RecipientNotFound) try: assert account.id.isdigit() # quantize to show 2 decimals. amount = Decimal(transfer.amount).quantize(Decimal(10) ** -2) except (AssertionError, ValueError): raise TransferError('something went wrong') return self.browser.init_transfer(account, recipient, amount, transfer.label) def execute_transfer(self, transfer, **params): return self.browser.execute_transfer(transfer) def iter_contacts(self): if not hasattr(self.browser, 'get_advisor'): raise NotImplementedError() return self.browser.get_advisor() def iter_threads(self, cache=False): """ If cache is False, always fetch the threads from the website. """ old = self._threads_age < datetime.utcnow() - self.CACHE_THREADS threads = self._threads if not cache or threads is None or old: with self.browser: threads = list(self.browser.iter_threads()) # the website is stupid and does not have the messages in the proper order threads = sorted(threads, key=lambda t: t.date, reverse=True) self._threads = threads seen = self.storage.get('seen', default=[]) for thread in threads: if thread.id not in seen: thread.root.flags |= thread.root.IS_UNREAD else: thread.root.flags &= ~thread.root.IS_UNREAD yield thread def fill_thread(self, thread, fields=None): if fields is None or 'root' in fields: return self.get_thread(thread) def get_thread(self, _id): if self.config['website'].get() != 'ppold': raise NotImplementedError() if isinstance(_id, Thread): thread = _id _id = thread.id else: thread = Thread(_id) with self.browser: thread = self.browser.get_thread(thread) return thread def iter_unread_messages(self): if self.config['website'].get() != 'ppold': raise NotImplementedError() threads = list(self.iter_threads(cache=True)) for thread in threads: if thread.root.flags & thread.root.IS_UNREAD: thread = self.fillobj(thread) or thread yield thread.root def set_message_read(self, message): self.storage.get('seen', default=[]).append(message.thread.id) self.storage.save() OBJECTS = {Thread: fill_thread} weboob-1.2/modules/bnporc/pp/000077500000000000000000000000001303450110500161735ustar00rootroot00000000000000weboob-1.2/modules/bnporc/pp/__init__.py000066400000000000000000000000001303450110500202720ustar00rootroot00000000000000weboob-1.2/modules/bnporc/pp/browser.py000066400000000000000000000270241303450110500202350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from dateutil.relativedelta import relativedelta import time from requests.exceptions import ConnectionError from weboob.browser.browsers import LoginBrowser, URL, need_login from weboob.capabilities.base import find_object from weboob.capabilities.bank import AccountNotFound, Account, TransferError from weboob.tools.decorators import retry from weboob.tools.json import json from weboob.browser.exceptions import ServerError from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountsPage, AccountsIBANPage, HistoryPage, TransferInitPage, \ ConnectionThresholdPage, LifeInsurancesPage, LifeInsurancesHistoryPage, \ LifeInsurancesDetailPage, MarketListPage, MarketPage, MarketHistoryPage, \ MarketSynPage, RecipientsPage, ValidateTransferPage, RegisterTransferPage, \ AdvisorPage __all__ = ['BNPPartPro', 'HelloBank'] class CompatMixin(object): def __enter__(self): return self def __exit__(self, type, value, tb): pass def JSON(data): return ('json', data) def isJSON(obj): return type(obj) is tuple and obj and obj[0] == 'json' class JsonBrowserMixin(object): def open(self, *args, **kwargs): if isJSON(kwargs.get('data')): kwargs['data'] = json.dumps(kwargs['data'][1]) if 'headers' not in kwargs: kwargs['headers'] = {} kwargs['headers']['Content-Type'] = 'application/json' return super(JsonBrowserMixin, self).open(*args, **kwargs) class BNPParibasBrowser(CompatMixin, JsonBrowserMixin, LoginBrowser): TIMEOUT = 30.0 login = URL(r'identification-wspl-pres/identification\?acceptRedirection=true×tamp=(?P\d+)', 'SEEA-pa01/devServer/seeaserver', 'https://mabanqueprivee.bnpparibas.net/fr/espace-prive/comptes-et-contrats\?u=%2FSEEA-pa01%2FdevServer%2Fseeaserver', LoginPage) con_threshold = URL('/fr/connexion/100-connexions', '/fr/connexion/mot-de-passe-expire', '/fr/espace-prive/100-connexions.*', '/fr/espace-pro/100-connexions-pro.*', '/fr/espace-pro/changer-son-mot-de-passe', '/fr/espace-client/100-connexions', '/fr/espace-prive/mot-de-passe-expire', '/fr/systeme/page-indisponible', ConnectionThresholdPage) accounts = URL('udc-wspl/rest/getlstcpt', AccountsPage) ibans = URL('rib-wspl/rpc/comptes', AccountsIBANPage) history = URL('rop-wspl/rest/releveOp', HistoryPage) transfer_init = URL('virement-wspl/rest/initialisationVirement', TransferInitPage) lifeinsurances = URL('mefav-wspl/rest/infosContrat', LifeInsurancesPage) lifeinsurances_history = URL('mefav-wspl/rest/listMouvements', LifeInsurancesHistoryPage) lifeinsurances_detail = URL('mefav-wspl/rest/detailMouvement', LifeInsurancesDetailPage) market_list = URL('pe-war/rpc/SAVaccountDetails/get', MarketListPage) market_syn = URL('pe-war/rpc/synthesis/get', MarketSynPage) market = URL('pe-war/rpc/portfolioDetails/get', MarketPage) market_history = URL('/pe-war/rpc/turnOverHistory/get', MarketHistoryPage) recipients = URL('/virement-wspl/rest/listerBeneficiaire', RecipientsPage) validate_transfer = URL('/virement-wspl/rest/validationVirement', ValidateTransferPage) register_transfer = URL('/virement-wspl/rest/enregistrerVirement', RegisterTransferPage) advisor = URL('/conseiller-wspl/rest/monConseiller', AdvisorPage) accounts_list = None @retry(ConnectionError, tries=3) def open(self, *args, **kwargs): return super(BNPParibasBrowser, self).open(*args, **kwargs) def do_login(self): if not (self.username.isdigit() and self.password.isdigit()): raise BrowserIncorrectPassword() timestamp = lambda: int(time.time() * 1e3) self.login.go(timestamp=timestamp()) if self.login.is_here(): self.page.login(self.username, self.password) @need_login def get_accounts_list(self): if self.accounts_list is None: self.accounts_list = [] ibans = self.ibans.go().get_ibans_dict() # This page might be unavailable. try: ibans.update(self.transfer_init.go(data=JSON({'modeBeneficiaire': '0'})).get_ibans_dict('Crediteur')) except TransferError: pass accounts = self.accounts.go().iter_accounts(ibans) self.market_syn.go(data=JSON({})) for account in accounts: for market_acc in self.page.get_list(): if account.number[-4:] == market_acc['securityAccountNumber'][-4:] and account.type == Account.TYPE_MARKET: account.valuation_diff = market_acc['profitLoss'] break self.accounts_list.append(account) return iter(self.accounts_list) @need_login def get_account(self, _id): return find_object(self.get_accounts_list(), id=_id, error=AccountNotFound) @need_login def iter_history(self, account, coming=False): if account.type == account.TYPE_LIFE_INSURANCE: return self.iter_lifeinsurance_history(account, coming) elif account.type == account.TYPE_MARKET and not coming: try: self.market_list.go(data=JSON({})) except ServerError: self.logger.warning("An Internal Server Error occured") return iter([]) for market_acc in self.page.get_list(): if account.number[-4:] == market_acc['securityAccountNumber'][-4:]: self.page = self.market_history.go(data=JSON({ "securityAccountNumber": market_acc['securityAccountNumber'], })) return self.page.iter_history() return iter([]) else: self.history.go(data=JSON({ "ibanCrypte": account.id, "pastOrPending": 1, "triAV": 0, "startDate": (datetime.now() - relativedelta(years=2)).strftime('%d%m%Y'), "endDate": datetime.now().strftime('%d%m%Y') })) return self.page.iter_coming() if coming else self.page.iter_history() @need_login def iter_lifeinsurance_history(self, account, coming=False): self.lifeinsurances_history.go(data=JSON({ "ibanCrypte": account.id, })) for tr in self.page.iter_history(coming): page = self.lifeinsurances_detail.go(data=JSON({ "ibanCrypte": account.id, "idMouvement": tr._op.get('idMouvement'), "ordreMouvement": tr._op.get('ordreMouvement'), "codeTypeMouvement": tr._op.get('codeTypeMouvement'), })) tr.investments = list(page.iter_investments()) yield tr @need_login def iter_coming_operations(self, account): return self.iter_history(account, coming=True) @need_login def iter_investment(self, account): if account.type == account.TYPE_LIFE_INSURANCE: self.lifeinsurances.go(data=JSON({ "ibanCrypte": account.id, })) return self.page.iter_investments() elif account.type == account.TYPE_MARKET: try: self.market_list.go(data=JSON({})) except ServerError: self.logger.warning("An Internal Server Error occured") return iter([]) for market_acc in self.page.get_list(): if account.number[-4:] == market_acc['securityAccountNumber'][-4:]: # Sometimes generate an Internal Server Error ... try: self.market.go(data=JSON({ "securityAccountNumber": market_acc['securityAccountNumber'], })) except ServerError: self.logger.warning("An Internal Server Error occured") break return self.page.iter_investments() return iter([]) @need_login def iter_recipients(self, origin_account): raise NotImplementedError() @need_login def transfer(self, account, recipient, amount, reason): raise NotImplementedError() @need_login def iter_threads(self): raise NotImplementedError() @need_login def get_thread(self, thread): raise NotImplementedError() class BNPPartPro(BNPParibasBrowser): BASEURL_TEMPLATE = r'https://%s.bnpparibas/' BASEURL = BASEURL_TEMPLATE % 'mabanque' def __init__(self, config=None, *args, **kwargs): self.config = config kwargs['username'] = self.config['login'].get() kwargs['password'] = self.config['password'].get() super(BNPPartPro, self).__init__(*args, **kwargs) def switch(self, subdomain): self.BASEURL = self.BASEURL_TEMPLATE % subdomain @need_login def iter_recipients(self, origin_account_id): try: if not origin_account_id in self.transfer_init.go(data=JSON({'modeBeneficiaire': '0'})).get_ibans_dict('Debiteur'): raise NotImplementedError() except TransferError: return for recipient in self.page.transferable_on(origin_account_ibancrypte=origin_account_id): yield recipient if self.page.can_transfer_to_recipients(origin_account_id): for recipient in self.recipients.go(data=JSON({'type': 'TOUS'})).iter_recipients(): yield recipient @need_login def prepare_transfer(self, account, recipient, amount, reason): data = {} data['devise'] = account.currency data['motif'] = reason data['dateExecution'] = datetime.now().strftime('%d-%m-%Y') data['compteDebiteur'] = account.id data['montant'] = str(amount) data['typeVirement'] = 'SEPA' if recipient.category == u'Externe': data['idBeneficiaire'] = recipient.id else: data['compteCrediteur'] = recipient.id return data @need_login def init_transfer(self, account, recipient, amount, reason): data = self.prepare_transfer(account, recipient, amount, reason) return self.validate_transfer.go(data=JSON(data)).handle_response(account, recipient, amount, reason) @need_login def execute_transfer(self, transfer): self.register_transfer.go(data=JSON({'referenceVirement': transfer.id})) return self.page.handle_response(transfer) @need_login def get_advisor(self): return self.advisor.stay_or_go().get_advisor() class HelloBank(BNPParibasBrowser): BASEURL = 'https://www.hellobank.fr/' weboob-1.2/modules/bnporc/pp/pages.py000066400000000000000000000516501303450110500176530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from cStringIO import StringIO from random import randint from decimal import Decimal from datetime import datetime from weboob.browser.elements import DictElement, ListElement, ItemElement, method from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import Format, Regexp, CleanText from weboob.browser.pages import JsonPage, LoggedPage, HTMLPage from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account, Investment, Recipient, Transfer, TransferError from weboob.capabilities.contact import Advisor from weboob.exceptions import BrowserIncorrectPassword, BrowserUnavailable from weboob.tools.capabilities.bank.iban import rib2iban, rebuild_rib, is_iban_valid from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.captcha.virtkeyboard import GridVirtKeyboard from weboob.tools.date import parse_french_date as Date class ConnectionThresholdPage(HTMLPage): def change_pass(self, oldpass, newpass): res = self.browser.open('/identification-wspl-pres/grille?accessible=false') url = '/identification-wspl-pres/grille/%s' % res.json()['data']['idGrille'] keyboard = self.browser.open(url) vk = BNPKeyboard(self, keyboard) data = {} data['codeAppli'] = 'PORTAIL' data['idGrille'] = res.json()['data']['idGrille'] data['typeGrille'] = res.json()['data']['typeGrille'] data['confirmNouveauPassword'] = vk.get_string_code(newpass) data['nouveauPassword'] = vk.get_string_code(newpass) data['passwordActuel'] = vk.get_string_code(oldpass) self.browser.location('/mcs-wspl/rpc/modifiercodesecret', data=data) def on_load(self): new_pass = ''.join([str((int(l) + 1) % 10) for l in self.browser.password]) self.logger.warning('Password expired. Renewing it. Temporary password is %s', new_pass) self.change_pass(self.browser.password, new_pass) self.change_pass(new_pass, self.browser.password) def cast(x, typ, default=None): try: return typ(x or default) except ValueError: return default class BNPKeyboard(GridVirtKeyboard): color = (0x1f, 0x27, 0x28) margin = 3, 3 symbols = {'0': '43b2227b92e0546d742a1f087015e487', '1': '2914e8cc694de26756096d0d0d4c6e0f', '2': 'aac54304a7bb850805d29f54557be366', '3': '0376d9f8419efee42e253d195a152547', '4': '3719595f15b1ac1c5a73d84aa290b5f6', '5': '617597f07a6530479927536671485439', '6': '4f5dce7bd0d9213fdae54b79bb8dd33a', '7': '49e07fa52b9bcee798f3a663f86e6cc1', '8': 'c60b723b3d95a46416b34c2cbefba3ed', '9': 'a13b8c3617a7bf854590833ddfb97f1f'} def __init__(self, page, image): symbols = list('%02d' % x for x in range(1, 11)) super(BNPKeyboard, self).__init__(symbols, 5, 2, StringIO(image.content), self.color, convert='RGB') self.check_symbols(self.symbols, page.browser.responses_dirname) class LoginPage(JsonPage): @staticmethod def render_template(tmpl, **values): for k, v in values.iteritems(): tmpl = tmpl.replace('{{ ' + k + ' }}', v) return tmpl @staticmethod def generate_token(length=11): chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz' return ''.join((chars[randint(0, len(chars)-1)] for _ in xrange(length))) def build_doc(self, text): try: return super(LoginPage, self).build_doc(text) except ValueError: # XXX When login is successful, server sends HTML instead of JSON, # we can ignore it. return {} def on_load(self): if self.url.startswith('https://mabanqueprivee.'): self.browser.switch('mabanqueprivee') error = cast(self.get('errorCode'), int, 0) if error: codes = [201, 21510, 203, 202] msg = self.get('message') if error in codes: raise BrowserIncorrectPassword(msg) elif error == 1001: raise BrowserUnavailable(msg) self.logger.debug('Unexpected error at login: "%s" (code=%s)' % (msg, error)) def login(self, username, password): url = '/identification-wspl-pres/grille/%s' % self.get('data.grille.idGrille') keyboard = self.browser.open(url) vk = BNPKeyboard(self, keyboard) target = self.browser.BASEURL + 'SEEA-pa01/devServer/seeaserver' user_agent = self.browser.session.headers.get('User-Agent') or '' auth = self.render_template(self.get('data.authTemplate'), idTelematique=username, password=vk.get_string_code(password), clientele=user_agent) # XXX useless ? csrf = self.generate_token() response = self.browser.location(target, data={'AUTH': auth, 'CSRF': csrf}) if response.url.startswith('https://pro.mabanque.bnpparibas'): self.browser.switch('pro.mabanque') if response.url.startswith('https://banqueprivee.mabanque.bnpparibas'): self.browser.switch('banqueprivee.mabanque') class BNPPage(LoggedPage, JsonPage): def build_doc(self, text): try: return self.response.json(parse_float=Decimal) except ValueError: raise BrowserUnavailable() def on_load(self): code = cast(self.get('codeRetour'), int, 0) if code == -30: self.logger.debug('End of session detected, try to relog...') self.browser.do_login() elif code: self.logger.debug('Unexpected error: "%s" (code=%s)' % (self.get('message'), code)) return (self.get('message'), code) class AccountsPage(BNPPage): FAMILY_TO_TYPE = { 1: Account.TYPE_CHECKING, 2: Account.TYPE_SAVINGS, 3: Account.TYPE_DEPOSIT, 4: Account.TYPE_MARKET, 5: Account.TYPE_LIFE_INSURANCE, 6: Account.TYPE_LIFE_INSURANCE, 8: Account.TYPE_LOAN, 9: Account.TYPE_LOAN, } LABEL_TO_TYPE = { u'PEA Espèces': Account.TYPE_SAVINGS, } def iter_accounts(self, ibans): for f in self.path('data.infoUdc.familleCompte.*'): for a in f.get('compte'): iban = ibans.get(a.get('key')) if iban is not None and not is_iban_valid(iban): iban = rib2iban(rebuild_rib(iban)) yield Account.from_dict({ 'id': a.get('key'), 'label': a.get('libellePersoProduit') or a.get('libelleProduit'), 'currency': a.get('devise'), 'type': self.LABEL_TO_TYPE.get(a.get('libelleProduit')) or self.FAMILY_TO_TYPE.get(f.get('idFamilleCompte')) or Account.TYPE_UNKNOWN, 'balance': a.get('soldeDispo'), 'coming': a.get('soldeAVenir'), 'iban': iban, 'number': a.get('value') }) class AccountsIBANPage(BNPPage): def get_ibans_dict(self): return dict([(a['ibanCrypte'], a['iban']) for a in self.path('data.listeRib.*.infoCompte')]) class MyRecipient(ItemElement): klass = Recipient obj_currency = Dict('devise') def obj_enabled_at(self): return datetime.now().replace(microsecond=0) def validate(self, el): assert is_iban_valid(el.iban) return True class TransferInitPage(BNPPage): def on_load(self): message_code = BNPPage.on_load(self) if message_code is not None: raise TransferError('%s, code=%s' % (message_code[0], message_code[1])) def get_ibans_dict(self, account_type): return dict([(a['ibanCrypte'], a['iban']) for a in self.path('data.infoVirement.listeComptes%s.*' % account_type)]) def can_transfer_to_recipients(self, origin_account_id): return next(a['eligibleVersBenef'] for a in self.path('data.infoVirement.listeComptesDebiteur.*') if a['ibanCrypte'] == origin_account_id) == '1' @method class transferable_on(DictElement): item_xpath = 'data/infoVirement/listeComptesCrediteur' class item(MyRecipient): condition = lambda self: Dict('ibanCrypte')(self.el) != self.env['origin_account_ibancrypte'] obj_id = Dict('ibanCrypte') obj_label = Dict('libelleCompte') obj_iban = Dict('iban') obj_category = u'Interne' def obj_bank_name(self): return u'BNP PARIBAS' class RecipientsPage(BNPPage): @method class iter_recipients(DictElement): item_xpath = 'data/infoBeneficiaire/listeBeneficiaire' class item(MyRecipient): # For the moment, only yield ready to transfer on recipients. condition = lambda self: Dict('libelleStatut')(self.el) == u'Activé' obj_id = Dict('idBeneficiaire') obj_label = Dict('nomBeneficiaire') obj_iban = Dict('ibanNumCompte') obj_category = u'Externe' def obj_bank_name(self): return Dict('nomBanque')(self) or NotAvailable class ValidateTransferPage(BNPPage): def check_errors(self): if not 'data' in self.doc: raise TransferError(self.doc['message']) def abort_if_unknown(self, transfer_data): try: assert transfer_data['typeOperation'] in ['1', '2'] assert transfer_data['repartitionFrais'] == '0' assert transfer_data['devise'] == 'EUR' assert not transfer_data['montantDeviseEtrangere'] except AssertionError as e: raise TransferError(e) def handle_response(self, account, recipient, amount, reason): self.check_errors() transfer_data = self.doc['data']['validationVirement'] self.abort_if_unknown(transfer_data) if transfer_data['idBeneficiaire'] is not None: assert transfer_data['idBeneficiaire'] == recipient.id exec_date = Date(transfer_data['dateExecution']).date() today = datetime.today().date() if transfer_data['typeOperation'] == '1': assert exec_date == today else: assert exec_date > today transfer = Transfer() transfer.currency = transfer_data['devise'] transfer.amount = Decimal(transfer_data['montantEuros']) transfer.account_iban = transfer_data['ibanCompteDebiteur'] transfer.recipient_iban = transfer_data['ibanCompteCrediteur'] or recipient.iban transfer.account_id = account.id transfer.recipient_id = recipient.id transfer.exec_date = exec_date transfer.fees = Decimal(transfer_data['montantFrais']) transfer.label = transfer_data['motifVirement'] transfer.account_label = account.label transfer.recipient_label = recipient.label transfer.id = transfer_data['reference'] # This is true if a transfer with the same metadata has already been done recently transfer._doublon = transfer_data['doublon'] transfer.account_balance = account.balance return transfer class RegisterTransferPage(ValidateTransferPage): def handle_response(self, transfer): self.check_errors() transfer_data = self.doc['data']['enregistrementVirement'] transfer.id = transfer_data['reference'] assert transfer.exec_date == Date(self.doc['data']['enregistrementVirement']['dateExecution']).date() # Timestamp at which the bank registered the transfer transfer._register_date = Date(self.doc['data']['enregistrementVirement']['dateEnregistrement']) return transfer class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PCHEQUE)(?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(?PFACTURE CARTE) DU (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*?)( CA?R?T?E? ?\d*X*\d*)?$'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P(PRELEVEMENT|TELEREGLEMENT|TIP)) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PPRLV( EUROPEEN)? SEPA) (?P.*?)( MDT/.*?)?( ECH/\d+)?( ID .*)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PECHEANCEPRET)(?P.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(?PRETRAIT DAB) (?P
\d{2})/(?P\d{2})/(?P\d{2})( (?P\d+)H(?P\d+))?( \d+)? (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?PVIR(EMEN)?T? (RECU |FAVEUR )?(TIERS )?)\w+ \d+/\d+ \d+H\d+ \w+ (?P.*)$'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PVIR(EMEN)?T? (EUROPEEN )?(SEPA )?(RECU |FAVEUR |EMIS )?(TIERS )?)(/FRM |/DE |/MOTIF |/BEN )?(?P.*?)(/.+)?$'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PREMBOURST) CB DU (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PREMBOURST)(?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PCOMMISSIONS)(?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P(?PREMUNERATION).*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?PREMISE CHEQUES)(?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] class HistoryPage(BNPPage): CODE_TO_TYPE = { 1: Transaction.TYPE_CHECK, # Chèque émis 2: Transaction.TYPE_CHECK, # Chèque reçu 3: Transaction.TYPE_CASH_DEPOSIT, # Versement espèces 4: Transaction.TYPE_ORDER, # Virements reçus 5: Transaction.TYPE_ORDER, # Virements émis 6: Transaction.TYPE_LOAN_PAYMENT, # Prélèvements / amortissements prêts 7: Transaction.TYPE_CARD, # Paiements carte, 8: Transaction.TYPE_CARD, # Carte / Formule BNP Net, 9: Transaction.TYPE_UNKNOWN, # Opérations Titres 10: Transaction.TYPE_UNKNOWN, # Effets de Commerce 11: Transaction.TYPE_WITHDRAWAL, # Retraits d'espèces carte 12: Transaction.TYPE_UNKNOWN, # Opérations avec l'étranger 13: Transaction.TYPE_CARD, # Remises Carte 14: Transaction.TYPE_WITHDRAWAL, # Retraits guichets 15: Transaction.TYPE_BANK, # Intérêts/frais et commissions 16: Transaction.TYPE_UNKNOWN, # Tercéo 30: Transaction.TYPE_UNKNOWN, # Divers } COMING_TYPE_TO_TYPE = { 2: Transaction.TYPE_ORDER, # Prélèvement 3: Transaction.TYPE_CHECK, # Chèque 4: Transaction.TYPE_CARD, # Opération carte } def one(self, path, context=None): try: return list(self.path(path, context))[0] except IndexError: return None def iter_history(self): for op in self.get('data.listerOperations.compte.operationPassee') or []: codeFamille = cast(self.one('operationType.codeFamille', op), int) tr = Transaction.from_dict({ 'id': op.get('idOperation'), 'type': self.CODE_TO_TYPE.get(codeFamille) or Transaction.TYPE_UNKNOWN, 'category': op.get('categorie'), 'amount': self.one('montant.montant', op), }) tr.parse(raw=op.get('libelleOperation'), date=Date(op.get('dateOperation')), vdate=Date(self.one('montant.valueDate', op))) yield tr def iter_coming(self): for op in self.path('data.listerOperations.compte.operationAvenir.*.operation.*'): codeOperation = cast(op.get('codeOperation'), int, 0) # Coming transactions don't have real id tr = Transaction.from_dict({ 'type': self.COMING_TYPE_TO_TYPE.get(codeOperation) or Transaction.TYPE_UNKNOWN, 'amount': op.get('montant'), 'card': op.get('numeroPorteurCarte'), }) tr.parse(date=Date(op.get('dateOperation')), vdate=Date(op.get('valueDate')), raw=op.get('libelle')) yield tr class LifeInsurancesPage(BNPPage): investments_path = 'data.infosContrat.repartition.listeSupport.*' def iter_investments(self): for support in self.path(self.investments_path): inv = Investment() if 'codeIsin' in support: inv.code = inv.id = support['codeIsin'] inv.quantity = support['nbUC'] inv.unitvalue = support['valUC'] inv.label = support['libelle'] inv.valuation = support['montant'] inv.set_empty_fields(NotAvailable) yield inv class LifeInsurancesHistoryPage(BNPPage): def iter_history(self, coming): for op in self.get('data.listerMouvements.listeMouvements') or []: #We have not date for this statut so we just skit it if op.get('statut') == u'En cours': continue tr = Transaction.from_dict({ 'type': Transaction.TYPE_BANK, '_state': op.get('statut'), 'amount': op.get('montantNet'), }) tr.parse(date=Date(op.get('dateSaisie')), vdate=Date(op.get('dateEffet')), raw='%s %s' % (op.get('libelleMouvement'), op.get('canalSaisie') or '')) tr._op = op if (op.get('statut') == u'Traité') ^ coming: yield tr class LifeInsurancesDetailPage(LifeInsurancesPage): investments_path = 'data.detailMouvement.listeSupport.*' class MarketListPage(BNPPage): def get_list(self): return self.get('securityAccountsList') or [] class MarketSynPage(BNPPage): def get_list(self): return self.get('synSecurityAccounts') or [] class MarketPage(BNPPage): investments_path = 'listofPortfolios.*' def iter_investments(self): for support in self.path(self.investments_path): inv = Investment() inv.code = inv.id = support['securityCode'] inv.quantity = support['quantityOwned'] inv.unitvalue = support['currentQuote'] inv.unitprice = support['averagePrice'] inv.label = support['securityName'] inv.valuation = support['valorizationValuation'] inv.diff = support['profitLossValorisation'] inv.set_empty_fields(NotAvailable) yield inv class MarketHistoryPage(BNPPage): def iter_history(self): for op in self.get('contentList') or []: tr = Transaction.from_dict({ 'type': Transaction.TYPE_BANK, 'amount': op.get('movementAmount'), 'date': datetime.fromtimestamp(op.get('movementDate') / 1000), 'label': op.get('operationName'), }) tr.investments = [] inv = Investment() inv.code = op.get('securityCode') inv.quantity = op.get('movementQuantity') inv.label = op.get('securityName') inv.set_empty_fields(NotAvailable) tr.investments.append(inv) yield tr class AdvisorPage(BNPPage): @method class get_advisor(ListElement): class item(ItemElement): klass = Advisor obj_name = Format('%s %s %s', Dict('data/civilite'), Dict('data/prenom'), Dict('data/nom')) obj_email = Regexp(Dict('data/mail'), '(?=\w)(.*)') obj_phone = CleanText(Dict('data/telephone'), replace=[(' ', '')]) obj_mobile = CleanText(Dict('data/mobile'), replace=[(' ', '')]) obj_fax = CleanText(Dict('data/fax'), replace=[(' ', '')]) obj_agency = Dict('data/agence') obj_address = Format('%s %s %s', Dict('data/adresseAgence'), Dict('data/codePostalAgence'), Dict('data/villeAgence')) weboob-1.2/modules/bnporc/test.py000066400000000000000000000023441303450110500171100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from random import choice class BNPorcTest(BackendTest): MODULE = 'bnporc' def test_bank(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_coming(a)) list(self.backend.iter_history(a)) def test_msgs(self): threads = list(self.backend.iter_threads()) thread = self.backend.fillobj(choice(threads), ['root']) assert len(thread.root.content) weboob-1.2/modules/boursorama/000077500000000000000000000000001303450110500164435ustar00rootroot00000000000000weboob-1.2/modules/boursorama/__init__.py000066400000000000000000000015201303450110500205520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Gabriel Kerneis # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BoursoramaModule __all__ = ['BoursoramaModule'] weboob-1.2/modules/boursorama/browser.py000066400000000000000000000172551303450110500205120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from dateutil.relativedelta import relativedelta from lxml.etree import XMLSyntaxError from weboob.browser.browsers import LoginBrowser, need_login, StatesMixin from weboob.browser.url import URL from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account from .pages import LoginPage, VirtKeyboardPage, AccountsPage, AsvPage, HistoryPage, AccbisPage, AuthenticationPage,\ MarketPage, LoanPage, SavingMarketPage, ErrorPage, IncidentPage, IbanPage, ExpertPage __all__ = ['BoursoramaBrowser'] class BrowserIncorrectAuthenticationCode(BrowserIncorrectPassword): pass class BoursoramaBrowser(LoginBrowser, StatesMixin): BASEURL = 'https://clients.boursorama.com' TIMEOUT = 60.0 keyboard = URL('/connexion/clavier-virtuel\?_hinclude=300000', VirtKeyboardPage) error = URL('/connexion/compte-verrouille', '/infos-profil', ErrorPage) login = URL('/connexion/', LoginPage) accounts = URL('/dashboard/comptes\?_hinclude=300000', AccountsPage) acc_tit = URL('/comptes/titulaire/(?P.*)\?_hinclude=1', AccbisPage) acc_rep = URL('/comptes/representative/(?P.*)\?_hinclude=1', AccbisPage) history = URL('/compte/(cav|epargne)/(?P.*)/mouvements.*', HistoryPage) card_transactions = URL('/compte/cav/(?P.*)/carte/.*', HistoryPage) budget_transactions = URL('/budget/compte/(?P.*)/mouvements.*', HistoryPage) other_transactions = URL('/compte/cav/(?P.*)/mouvements.*', HistoryPage) saving_transactions = URL('/compte/epargne/csl/(?P.*)/mouvements.*', HistoryPage) incident = URL('/compte/cav/(?P.*)/mes-incidents.*', IncidentPage) asv = URL('/compte/assurance-vie/.*', AsvPage) saving_history = URL('/compte/cefp/.*/(positions|mouvements)', '/compte/.*ord/.*/mouvements', '/compte/pea/.*/mouvements', '/compte/0%25pea/.*/mouvements', '/compte/pea-pme/.*/mouvements', SavingMarketPage) market = URL('/compte/(?!assurance|cav|epargne).*/(positions|mouvements)', '/compte/ord/.*/positions', MarketPage) loans = URL('/credit/immobilier/.*/informations', '/credit/consommation/.*/informations', '/credit/lombard/.*/caracteristiques', LoanPage) authentication = URL('/securisation', AuthenticationPage) iban = URL('/compte/(?P.*)/rib', IbanPage) expert = URL('/compte/derive/', ExpertPage) __states__ = ('auth_token',) def __init__(self, config=None, *args, **kwargs): self.config = config self.auth_token = None self.webid = None self.accounts_list = None kwargs['username'] = self.config['login'].get() kwargs['password'] = self.config['password'].get() super(BoursoramaBrowser, self).__init__(*args, **kwargs) def handle_authentication(self): if self.authentication.is_here(): if self.config['enable_twofactors'].get(): self.page.sms_first_step() self.page.sms_second_step() else: raise BrowserIncorrectAuthenticationCode( """Boursorama - activate the two factor authentication in boursorama config.""" """ You will receive SMS code but are limited in request per day (around 15)""" ) def do_login(self): assert isinstance(self.config['device'].get(), basestring) assert isinstance(self.config['enable_twofactors'].get(), bool) assert self.password.isdigit() if self.auth_token and self.config['pin_code'].get(): self.page.authenticate() else: self.login.go() self.page.login(self.username, self.password) if self.login.is_here() or self.error.is_here(): raise BrowserIncorrectPassword() # After login, we might be redirected to the two factor authentication page. self.handle_authentication() if self.authentication.is_here(): raise BrowserIncorrectAuthenticationCode('Invalid PIN code') @need_login def get_accounts_list(self): for x in range(3) : if self.accounts_list is not None: break self.accounts_list = list() for account in self.accounts.go().iter_accounts(): self.accounts_list.append(account) self.acc_tit.go(webid=self.webid).populate(self.accounts_list) try: if not all([acc._webid for acc in self.accounts_list]): self.acc_rep.go(webid=self.webid).populate(self.accounts_list) except XMLSyntaxError: self.accounts_list = None continue for account in self.accounts_list: account.iban = self.iban.go(webid=account._webid).get_iban() return iter(self.accounts_list) def get_account(self, id): assert isinstance(id, basestring) for a in self.get_accounts_list(): if a.id == id: return a return None @need_login def get_history(self, account, coming=False): if account.type is Account.TYPE_LOAN or '/compte/derive' in account._link: return if account.type in (Account.TYPE_LIFE_INSURANCE, Account.TYPE_MARKET): if coming: return transactions = [] self.location('%s/mouvements' % account._link.rstrip('/')) account._history_pages = [] for t in self.page.iter_history(account=account): transactions.append(t) for t in self.page.get_transactions_from_detail(account): transactions.append(t) for t in sorted(transactions, key=lambda tr: tr.date, reverse=True): yield t else: # We look for 1 year of history. params = {} params['movementSearch[toDate]'] = (date.today() + relativedelta(days=40)).strftime('%d/%m/%Y') params['movementSearch[fromDate]'] = (date.today() - relativedelta(years=1)).strftime('%d/%m/%Y') params['movementSearch[selectedAccounts][]'] = account._webid if account.type != Account.TYPE_CARD: self.location('%s/mouvements' % account._link.rstrip('/'), params=params) else: self.location('%s' % account._link) for t in self.page.iter_history(): yield t @need_login def get_investment(self, account): if '/compte/derive' in account._link: return iter([]) if not account.type in (Account.TYPE_LIFE_INSURANCE, Account.TYPE_MARKET): raise NotImplementedError() self.location(account._link) # We might deconnect at this point. if self.login.is_here(): return self.get_investment(account) return self.page.iter_investment() weboob-1.2/modules/boursorama/favicon.png000066400000000000000000000032271303450110500206020ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDVKV pHYs  tIME DSStEXtCommentCreated with GIMPWIDATx[lU]nCP.!AAD1^tD}1xK(a4 4*IF%SPPhׇ6Ιefɶ={|绝YȔ{ӱ ,VV`0h `8dBfcʆ V`*0^u|(t: mJcXAI& *m$ww4t) (p(Ux9,F`p{}5JA hPGo]_]Cg“?r,,$ Nr _L)!ϒkH=G^>Dz nT/pmsv w$u~Xv[[ oAA00:uvm°,Hd9T|xyKSGv!Sq @u6hh4gE@f+ c ±@vCmձ~dEA78TV_ eIF`pDުcEricRvvpBB=+[ QNb4!59ݜjfqB'@Uw _xݮ`uJ7oK0~KkR$H32;3)ۀ;(KxꕤXG]XxqzUg*T X2 ep~oh 3 _<^$0%y=a%aaє/nIO*Y`P jPK܊RJqEdȴ"*0 D@!  |~M.Y^U{]oZ4KnP"qg(SQ $B+`X1w5si[MHOE)ΧR# o9Qsi@ Bv ~X.{wH~4ciX1]7=sL;Փ(yH<&\ } hfJؗyXc'ޒ<7"E^"y@r:%1O2RXKZS"`f)=bO'vX i`nغ1o4"ZC5w,j}1K Q鬧U;-e%Fપ|y1[O1v 3(ʮj#r(l2EI1~)U-Y)o#~jv`+D-.]5V~,psREr_C΋kMQ2Jm5V1oؘ)Ws2L^cݪ-7U[+gҮJMg_(!No:v{zDz(7H+p~_mnn.#x/IENDB`weboob-1.2/modules/boursorama/module.py000066400000000000000000000050411303450110500203020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gabriel Serme # Copyright(C) 2011 Gabriel Kerneis # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, ValueBool, Value from .browser import BoursoramaBrowser __all__ = ['BoursoramaModule'] class BoursoramaModule(Module, CapBank): NAME = 'boursorama' MAINTAINER = u'Gabriel Kerneis' EMAIL = 'gabriel@kerneis.info' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Boursorama' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe'), ValueBool('enable_twofactors', label='Send validation sms', default=False), Value('device', label='Device name', regexp='\w*', default=''), Value('pin_code', label='Sms code', required=False), ) BROWSER = BoursoramaBrowser def create_default_browser(self): return self.create_browser(self.config) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): for tr in self.browser.get_history(account): if not tr._is_coming: yield tr def iter_coming(self, account): for tr in self.browser.get_history(account, coming=True): if tr._is_coming: yield tr def iter_investment(self, account): return self.browser.get_investment(account) weboob-1.2/modules/boursorama/pages.py000066400000000000000000000462161303450110500201250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from StringIO import StringIO from weboob.browser.pages import HTMLPage, LoggedPage, pagination, NextPage from weboob.browser.elements import ListElement, ItemElement, method, TableElement, SkipItem from weboob.browser.filters.standard import CleanText, CleanDecimal, Field, TableCell, Regexp, Date, AsyncLoad, Async, Eval from weboob.browser.filters.html import Attr, Link from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.value import Value from weboob.tools.date import parse_french_date from weboob.tools.captcha.virtkeyboard import VirtKeyboard from weboob.exceptions import BrowserQuestion, BrowserIncorrectPassword, BrowserHTTPNotFound class BrowserAuthenticationCodeMaxLimit(BrowserIncorrectPassword): pass class IncidentPage(HTMLPage): pass class IbanPage(LoggedPage, HTMLPage): def get_iban(self): if self.doc.xpath('//div[has-class("alert")]/p[contains(text(), "Une erreur est survenue")]') or \ self.doc.xpath('//div[has-class("alert")]/p[contains(text(), "Le compte est introuvable")]'): return NotAvailable return CleanText('//table[thead[tr[th[contains(text(), "Code I.B.A.N")]]]]/tbody/tr/td[2]', replace=[(' ', '')])(self.doc) class AuthenticationPage(HTMLPage): def authenticate(self): self.logger.info('Using the PIN Code %s to login', self.browser.config['pin_code'].get()) self.logger.info('Using the auth_token %s to login', self.browser.auth_token) form = self.get_form() form['otp_confirm[otpCode]'] = self.browser.config['pin_code'].get() form['flow_secureForm_instance'] = self.browser.auth_token form['otp_confirm[validate]'] = '' form['flow_secureForm_step'] = 2 form.submit() self.browser.auth_token = None def sms_first_step(self): """ This function simulates the registration of a device on boursorama two factor authentification web page. @param device device name to register @exception BrowserAuthenticationCodeMaxLimit when daily limit is consumed """ form = self.get_form() form.submit() def sms_second_step(self): #
  • Vous avez atteint le nombre maximal de demandes pour aujourd'hui
error = CleanText('//div[has-class("form-errors")]')(self.doc) if len(error) > 0: raise BrowserIncorrectPassword(error) form = self.get_form() self.browser.auth_token = form['flow_secureForm_instance'] form['otp_prepare[receiveCode]'] = '' form.submit() raise BrowserQuestion(Value('pin_code', label='Enter the PIN Code')) class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^CHQ\. (?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(ACHAT|PAIEMENT) CARTE (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^(PRLV SEPA |PRLV |TIP )(?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^RETRAIT DAB (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^VIR( SEPA)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^AVOIR (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^REM CHQ (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] class VirtKeyboardPage(HTMLPage): pass class BoursoramaVirtKeyboard(VirtKeyboard): symbols = {'0': (17, 7, 24, 17), '1': (18, 6, 21, 18), '2': (9, 7, 32, 34), '3': (10, 7, 31, 34), '4': (11, 6, 29, 34), '5': (14, 6, 28, 34), '6': (7, 7, 34, 34), '7': (5, 6, 36, 34), '8': (8, 7, 32, 34), '9': (4, 7, 38, 34)} color = (255,255,255) md5 = {} def __init__(self, page): for span in page.doc.xpath('//span'): c = span.attrib['data-matrix-key'] img = StringIO(span.xpath('./img/@src')[0].replace('data:image/png;base64,', '').decode('base64')) self.load_image(img, self.color, convert='RGB') self.load_symbols((0,0,42,42), c) def load_symbols(self, coords, c): coord = self.get_symbol_coords(coords) if coord == (-1, -1, -1, -1): return self.md5[coord] = c def get_code(self, password): code = '' for i, d in enumerate(password): if i > 0: code += '|' code += self.md5[self.symbols[d]] return code class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form() keyboard_page = self.browser.keyboard.open() vk = BoursoramaVirtKeyboard(keyboard_page) code = vk.get_code(password) form['form[login]'] = login form['form[fakePassword]'] = len(password) * '•' form['form[password]'] = code form['form[matrixRandomChallenge]'] = re.search('val\("(.*)"', CleanText('//script')(keyboard_page.doc)).group(1) form.submit() class AccountsPage(LoggedPage, HTMLPage): def is_here(self): return not self.doc.xpath('//div[contains(@id, "alert-random")]') ACCOUNT_TYPES = {u'Comptes courants': Account.TYPE_CHECKING, u'Comptes épargne': Account.TYPE_SAVINGS, u'Comptes bourse': Account.TYPE_MARKET, u'Assurances Vie': Account.TYPE_LIFE_INSURANCE, u'Mes crédits': Account.TYPE_LOAN, u'crédit': Account.TYPE_LOAN, u'prêt': Account.TYPE_LOAN, } @method class iter_accounts(ListElement): item_xpath = '//table[@class="table table--accounts"]/tr[has-class("table__line--account") and count(descendant::td) > 1]' class item(ItemElement): klass = Account load_details = Field('_link') & AsyncLoad obj_label = CleanText('.//a[@class="account--name"] | .//div[@class="account--name"]') obj_balance = CleanDecimal('.//a[has-class("account--balance")]', replace_dots=True) obj_currency = FrenchTransaction.Currency('.//a[has-class("account--balance")]') obj_valuation_diff = Async('details') & CleanDecimal('//li[h4[text()="Total des +/- values"]]/h3 |\ //li[span[text()="Total des +/- values latentes"]]/span[has-class("overview__value")]', replace_dots=True, default=NotAvailable) obj__card = Async('details') & Attr('//a[@data-modal-behavior="credit_card-modal-trigger"]', 'href', default=NotAvailable) obj__holder = None def obj_coming(self): # Don't duplicate coming (card balance with account coming) # TODO: fetch coming which is not card coming for account with cards. if self.obj__card(self): return NotAvailable return Async('details', CleanDecimal(u'//li[h4[text()="Mouvements à venir"]]/h3', replace_dots=True, default=NotAvailable))(self) def obj_id(self): id = Async('details', Regexp(CleanText('//h3[has-class("account-number")]'), r'(\d+)', default=NotAvailable))(self) if not id: raise SkipItem() return id def obj_type(self): return self.page.ACCOUNT_TYPES.get(CleanText('./preceding-sibling::tr[has-class("list--accounts--master")]//h4')(self), Account.TYPE_UNKNOWN) \ or next((self.page.ACCOUNT_TYPES.get(word) for word in self.obj_label(self).lower().split() if self.page.ACCOUNT_TYPES.get(word)), Account.TYPE_UNKNOWN) def obj__link(self): link = Attr('.//a[@class="account--name"] | .//a[2]', 'href', default=NotAvailable)(self) if not self.page.browser.webid: self.page.browser.webid = re.search('\/([^\/|?|$]{32})(\/|\?|$)', link).group(1) return link def obj__webid(self): m = re.search('([a-z\d]{32})', self.obj__link()) if m: return m.group(1) return None # We do not yield other banks accounts for the moment. def validate(self, obj): return not Async('details', CleanText(u'//h4[contains(text(), "Établissement bancaire")]'))(self) and not \ Async('details', CleanText(u'//h4/div[contains(text(), "Établissement bancaire")]'))(self) class HistoryPage(LoggedPage, HTMLPage): @method class iter_history(ListElement): item_xpath = '//ul[has-class("list__movement")]/li[div and not(contains(@class, "summary")) \ and not(contains(@class, "graph")) \ and not (contains(@class, "separator"))]' class item(ItemElement): klass = Transaction obj_id = Attr('.', 'data-id', default=NotAvailable) or Attr('.', 'data-custom-id') obj_raw = Transaction.Raw(CleanText('.//div[has-class("list__movement__line--label__name")]')) obj_date = Date(Attr('.//time', 'datetime')) obj_amount = CleanDecimal('.//div[has-class("list__movement__line--amount")]', replace_dots=True) obj_category = CleanText('.//div[has-class("category")]') def obj_rdate(self): s = Regexp(Field('raw'), ' (\d{2}/\d{2}/\d{2}) | (?!NUM) (\d{6}) ', default=NotAvailable)(self) if not s: return Field('date')(self) s = s.replace('/', '') return Date(dayfirst=True).filter('%s%s%s%s%s' % (s[:2], '-', s[2:4], '-', s[4:])) def obj__is_coming(self): return len(self.xpath(u'.//span[@title="Mouvement à débit différé"]')) # These are on deffered cards accounts. def condition(self): return not len(self.xpath(u'.//span[has-class("icon-carte-bancaire")]')) class Myiter_investment(TableElement): item_xpath = '//table[contains(@class, "operations")]/tbody/tr' head_xpath = '//table[contains(@class, "operations")]/thead/tr/th' col_value = u'Valeur' col_quantity = u'Quantité' col_unitprice = u'Px. Revient' col_unitvalue = u'Cours' col_valuation = u'Montant' col_diff = u'+/- latentes' class Myitem(ItemElement): klass = Investment obj_quantity = CleanDecimal(TableCell('quantity'), default=NotAvailable) obj_unitprice = CleanDecimal(TableCell('unitprice'), replace_dots=True, default=NotAvailable) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=True, default=NotAvailable) obj_valuation = CleanDecimal(TableCell('valuation'), replace_dots=True, default=NotAvailable) obj_diff = CleanDecimal(TableCell('diff'), replace_dots=True, default=NotAvailable) def obj_label(self): return CleanText().filter((TableCell('value')(self)[0]).xpath('.//a')) def obj_code(self): return CleanText().filter((TableCell('value')(self)[0]).xpath('./span')) or NotAvailable def my_pagination(func): def inner(page, *args, **kwargs): while True: try: for r in func(page, *args, **kwargs): yield r except NextPage as e: try: result = page.browser.location(e.request) page = result.page except BrowserHTTPNotFound as e: page.logger.warning(e) return else: return return inner class MarketPage(LoggedPage, HTMLPage): @my_pagination @method class iter_history(TableElement): item_xpath = '//table/tbody/tr' head_xpath = '//table/thead/tr/th' col_label = 'Nature' col_amount = 'Montant' col_date = ['Date d\'effet', 'Date'] next_page = Link('//li[@class="pagination__next"]/a') class item(ItemElement): klass = Transaction obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_raw = Transaction.Raw(CleanText(TableCell('label'))) obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True, default=NotAvailable) obj__is_coming = False def parse(self, el): if el.xpath('./td[2]/a'): m = re.search('(\d+)', el.xpath('./td[2]/a')[0].get('data-modal-alert-behavior', '')) if m: self.env['account']._history_pages.append((Field('raw')(self),\ self.page.browser.open('%s%s%s' % (self.page.url.split('mouvements')[0], 'mouvement/', m.group(1))).page)) raise SkipItem() @method class iter_investment(Myiter_investment): class item (Myitem): def obj_unitvalue(self): return CleanDecimal(replace_dots=True, default=NotAvailable).filter((TableCell('unitvalue')(self)[0]).xpath('./span[not(@class)]')) def get_transactions_from_detail(self, account): for label, page in account._history_pages: amounts = page.doc.xpath('//span[contains(text(), "Montant")]/following-sibling::span') if len(amounts) == 3: amounts.pop(0) for table in page.doc.xpath('//table'): t = Transaction() t.date = Date(CleanText(page.doc.xpath('//span[contains(text(), "Date d\'effet")]/following-sibling::span')), dayfirst=True)(page) t.label = label t.amount = CleanDecimal(replace_dots=True).filter(amounts[0]) amounts.pop(0) t._is_coming = False t.investments = [] for tr in table.xpath('./tbody/tr'): i = Investment() i.label = CleanText().filter(tr.xpath('./td[1]')) i.vdate = Date(CleanText(tr.xpath('./td[2]')), dayfirst=True)(tr) i.unitvalue = CleanDecimal(replace_dots=True).filter(tr.xpath('./td[3]')) i.quantity = CleanDecimal(replace_dots=True).filter(tr.xpath('./td[4]')) i.valuation = CleanDecimal(replace_dots=True).filter(tr.xpath('./td[5]')) t.investments.append(i) yield t class SavingMarketPage(MarketPage): @pagination @method class iter_history(TableElement): item_xpath = '//table/tbody/tr' head_xpath = '//table/thead/tr/th' col_label = u'Opération' col_amount = u'Montant' col_date = u'Date opération' col_vdate = u'Date Val' next_page = Link('//li[@class="pagination__next"]/a') class item(ItemElement): klass = Transaction obj_label = CleanText(TableCell('label')) obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True) obj__is_coming = False def obj_date(self): return parse_french_date(CleanText(TableCell('date'))(self)) def obj_vdate(self): return parse_french_date(CleanText(TableCell('vdate'))(self)) @method class iter_investment(TableElement): item_xpath = '//table/tbody/tr[count(descendant::td) > 4]' head_xpath = '//table/thead/tr[count(descendant::th) > 4]/th' col_label = u'Fonds' col_code = u'Code Isin' col_unitvalue = u'Valeur de la part' col_quantity = u'Nombre de parts' col_vdate = u'Date VL' class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_code = CleanText(TableCell('code')) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=True) obj_quantity = CleanDecimal(TableCell('quantity'), replace_dots=True) obj_valuation = Eval(lambda x, y: x * y, Field('quantity'), Field('unitvalue')) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True) class AsvPage(MarketPage): @method class iter_investment(Myiter_investment): col_vdate = u'Date de Valeur' class item(Myitem): obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True, default=NotAvailable) class AccbisPage(LoggedPage, HTMLPage): def populate(self, accounts): cards = [] for account in accounts: for li in self.doc.xpath('//li[@class="nav-category"]'): title = CleanText().filter(li.xpath('./h3')) for a in li.xpath('./ul/li//a'): label = CleanText().filter(a.xpath('.//span[@class="nav-category__name"]')) balance_el = a.xpath('.//span[@class="nav-category__value"]') balance = CleanDecimal(replace_dots=True, default=NotAvailable).filter(balance_el) if 'CARTE' in label and balance: acc = Account() acc.balance = balance acc.label = label acc.currency = FrenchTransaction.Currency().filter(balance_el) acc._link = Link().filter(a.xpath('.')) acc._history_page = acc._link acc.id = acc._webid = Regexp(pattern='carte/(.*)$').filter(Link().filter(a.xpath('.'))) acc.type = Account.TYPE_CARD if not acc in cards: cards.append(acc) elif account.label == label and account.balance == balance: if not account.type: account.type = AccountsPage.ACCOUNT_TYPES.get(title, Account.TYPE_UNKNOWN) account._webid = Attr(None, 'data-account-label').filter(a.xpath('.//span[@class="nav-category__name"]')) accounts.extend(cards) class LoanPage(LoggedPage, HTMLPage): pass class ErrorPage(HTMLPage): pass class ExpertPage(LoggedPage, HTMLPage): pass weboob-1.2/modules/boursorama/test.py000066400000000000000000000021141303450110500177720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Gabriel Kerneis # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BoursoramaTest(BackendTest): MODULE = 'boursorama' def test_boursorama(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_coming(a)) list(self.backend.iter_history(a)) weboob-1.2/modules/bouygues/000077500000000000000000000000001303450110500161335ustar00rootroot00000000000000weboob-1.2/modules/bouygues/__init__.py000066400000000000000000000001011303450110500202340ustar00rootroot00000000000000from .module import BouyguesModule __all__ = ['BouyguesModule'] weboob-1.2/modules/bouygues/browser.py000066400000000000000000000062471303450110500202010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import DocumentsPage, HomePage, LoginPage, ProfilePage, SendSMSPage, SendSMSErrorPage from weboob.capabilities.messages import CantSendMessage __all__ = ['BouyguesBrowser'] class BouyguesBrowser(LoginBrowser): BASEURL = 'https://www.mon-compte.bouyguestelecom.fr/' TIMEOUT = 20 login = URL('cas/login', LoginPage) home = URL('https://www.bouyguestelecom.fr/mon-compte', HomePage) profile = URL('https://api-mc.bouyguestelecom.fr/client/me/header.json', ProfilePage) documents = URL('https://www.bouyguestelecom.fr/parcours/mes-factures', 'https://www.bouyguestelecom.fr/parcours/mes-factures/historique\?no_reference=(?P)', DocumentsPage) sms_page = URL('http://www.mobile.service.bbox.bouyguestelecom.fr/services/SMSIHD/sendSMS.phtml', 'http://www.mobile.service.bbox.bouyguestelecom.fr/services/SMSIHD/confirmSendSMS.phtml', SendSMSPage) confirm = URL('http://www.mobile.service.bbox.bouyguestelecom.fr/services/SMSIHD/resultSendSMS.phtml') sms_error_page = URL('http://www.mobile.service.bbox.bouyguestelecom.fr/services/SMSIHD/SMS_erreur.phtml', SendSMSErrorPage) logged = False def do_login(self): if self.logged: return self.login.go() if self.home.is_here(): return self.page.login(self.username, self.password) if not self.home.is_here(): raise BrowserIncorrectPassword self.logged = True self.page.logged = True @need_login def post_message(self, message): self.sms_page.go() if self.sms_error_page.is_here(): raise CantSendMessage(self.page.get_error_message()) receivers = ";".join(list(message.receivers)) if message.receivers else self.username self.page.send_sms(message, receivers) if self.sms_error_page.is_here(): raise CantSendMessage(self.page.get_error_message()) self.confirm.open() @need_login def get_subscription_list(self): return self.profile.stay_or_go().get_list() @need_login def iter_documents(self, subscription): ref = self.documents.go().get_ref(subscription.label) if not ref: return iter([]) return self.documents.go(ref=ref).get_documents(subid=subscription.id) weboob-1.2/modules/bouygues/favicon.png000066400000000000000000000121651303450110500202730ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDe) pHYs  tIME =ztEXtCommentCreated with GIMPWIDATx[}\U^wOw&L>'aen( +_kIU,]٥늉Z"bieD* DH"$o@&_J23~sy3ವުWy龿s=>/Pߩ`E|r' eZ CXQ? [&@ Bi^6Zi}ƗS;粥PcfW^3ZkwGɲ7\2{GU$APL`Ytͺ rimy.W7_dD#^vH Lp ahE)y`mmS(-sɔ$ &RpKJaoc|잮74@ޕfxͷ|0 L=U3n;'ܱP l@)h70,l.?.C ֜)w3@Cϼy_T Pl|ptHT jwju!"`/DiN*%;~ L t4h]=cf7pU 3 ]UA0A짓XtUlL!':pIDgz6z?Ȅ-j0`y?|vZmʈpofuנ fO} 3 S_}ڂ X)( h˂(+C㹯߶񍧞(ݡnNUMs@L 43D8f@,g5`6"bs0*F1^&DpSiPtc;\&[h=Ľ%sOc:` A΀, @<(X2'A3wH2+  d N" |0\_괺˯%5\O,d@\ˁ

̑{ƺzAp` wʐyPkZ2" h(@ Rar9OEkAÉ|b{n>!d2bLZ@ !0!q0;uzfjs܄syh|hr[rmzZHM5d:Ne*}Yts-Ny&^aq]2xmdͷL&$5{>JB#; fyy;6 $sc{W6wt=#gHspƧ uum nX[ f1=iL{{ !wю`ZQR'X!F;;אx.{y"o궿@+Jk'S]9X;D<@DV,^Zh'w@`L'gheŻAvKؤJXUH7PYP^<'Ug R޽F躤 3}ws3oS"@Xwg]vSD x9{aUjUu6ZCkغ|Vkغd 阀hak(x6%VntJ ,fk #wN  ~o+|O\;3AdU h}&Ҕvx;Ezx΃O¦4B[c{ 55k[J:(ǖ~uײN~uvN4/gv6>U"lU7  <}Y?v~ =x_k_po{QgsۮgW*8)ڵc{2!qS">JVD< |U{Ǩk|p޸u5sm$={:\¦S%̢l7@:QP SHWn |Q\X syeOo弛$nV$&"Y> (GgѺ:pg6+]J WeS}*B w@;5`PJFSb~I}X { &zX9Q&`Տ@q+f5^ۺVP>d@4y3#5Z]0QVaZ k06X4ѕBi@%4^Qꌘ!9 `}(FJTL[@GO,ޑNx=Wl;) v?짝~2,7]=>inKb魦^.~Ԯjqa"eT`M3L,4N [4@RR#fY=L{,P0mbt\&m F3dAf< (_YF!ұ6e.SF"h `w`Hf@9UA]ޏ9Ue휚sApDž$CoӝĎ hA^Kb(l|Bx򢛦sKnQ:b?ū#0x~+Y:mcmFcbp5ϛS=| ^oj]ש &DDБѤX'k@&L=ڇ(:<15_JRCA$Uiwt"{r7{5wuU$K:cEeXw#Fu>\Z6k4V1fuoy4{]KýRspu@ǖ$)m"tW"_ż^ubP n8X?PI@+7z8غ7 C%&tOR /%~Q|so rIfqzƭE| vg ׿=^wu1tVy|l[@yݬ18-(_1 x`_O=!AcJʻo%ѿMp(HlSeMV2\S/9vpOx ȶ8~65?aPJݞ@hKGanZm~0K>~ץe6,?Ɍ{6b_yU4?* JG@ J'=ό3,O&:ٲLu6m|<Hxt*DgAq8 _ݏ]v#Vii`*$0rOOF]""c[k>.!H{x-k8Rs%pCh' |` WwϦL}c]h7*A:)iVbf#we)v2]"vvnIˢȞ FuDP`J|`B3x>~%Z~@]ԊZuO͑鈄9X){ Uw f6e{|L8E1 ?0ʏ$cx[1}2ڙe4o=ak+"vAN oV3ϴn?9dYjJMr2ry?H/0@ǽ?[#<4=N(>K})pbc&ECB!(*jW2n̡ͅ1".\&`jg۞2Fʉs}>;`4Аk|\Ǽ۲ 56}ݍ+&|JAb P9v4z̒&bvYC/ b )Lߴ>OBl]|?j|ߖ@xjͳ.nֵWlqU:Z~Zxώc[Y(dXS=k8՗D<'pC~  YxWX;Φ67uT$`={[++WuqM|C#nC[|<3$ۧQqd|(I!`OD $Gia  2x{݀wCSKKs11b1@N }: 7/(9ئf>эU/!ucse7Xc,]v;nJNaߣ5{ ;׆P1~/ط&ʹ'X8uTs9T̎Xi}y7݌-֥+ } |tmq!l@Bm( bX. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.messages import CantSendMessage, CapMessages, CapMessagesPost from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import BouyguesBrowser __all__ = ['BouyguesModule'] class BouyguesModule(Module, CapMessages, CapMessagesPost, CapDocument): NAME = 'bouygues' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' DESCRIPTION = u'Bouygues Télécom French mobile phone provider' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('login', label='Login/Phone number'), ValueBackendPassword('password', label='Password')) BROWSER = BouyguesBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def post_message(self, message): if not message.content.strip(): raise CantSendMessage(u'Message content is empty.') self.browser.post_message(message) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) return self.browser.open(document.url).content weboob-1.2/modules/bouygues/pages.py000066400000000000000000000106221303450110500176050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.messages import CantSendMessage from weboob.capabilities.bill import Bill, Subscription from weboob.browser.pages import HTMLPage, JsonPage, LoggedPage from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import CleanDecimal, CleanText, Env, Format, Regexp from weboob.browser.filters.html import Link from weboob.browser.elements import DictElement, ItemElement, ListElement, method from weboob.tools.date import parse_french_date class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form('//form[@id="log_data"]') form['username'] = login form['password'] = password form.submit() class HomePage(HTMLPage, LoggedPage): pass class ProfilePage(JsonPage, LoggedPage): @method class get_list(DictElement): item_xpath = 'data/lignes' class item(ItemElement): klass = Subscription obj_id = CleanText(Dict('num_ligne')) obj__type = CleanText(Dict('type')) obj_label = Env('label') obj_subscriber = Format("%s %s %s", CleanText(Dict('civilite')), CleanText(Dict('prenom')), CleanText(Dict('nom'))) def parse(self, el): # add spaces number = iter(self.obj_id(el)) self.env['label'] = ' '.join(a+b for a, b in zip(number, number)) class SendSMSPage(HTMLPage): def send_sms(self, message, receivers): sms_number = CleanDecimal(Regexp(CleanText('//span[@class="txt12-o"][1]/strong'), '(\d*) SMS.*'))(self.doc) if sms_number == 0: msg = CleanText('//span[@class="txt12-o"][1]')(self.doc) raise CantSendMessage(msg) form = self.get_form('//form[@name="formSMS"]') form["fieldMsisdn"] = receivers form["fieldMessage"] = message.content form.submit() class SendSMSErrorPage(HTMLPage): def get_error_message(self): return CleanText('//span[@class="txt12-o"][1]')(self.doc) class DocumentsPage(HTMLPage): def get_ref(self, label): options = self.doc.xpath('//select[@id="factureMois"]/option[position() > 1]/@value') for option in options: ref = self.doc.xpath('//span[contains(text(), "%s")]/ \ ancestor::div[has-class("etape-content")]//a[@id="btnAnciennesFactures"]' % label) if ref: # Get ref and return it return re.search('reference=([\d]+)', Link().filter(ref)).group(1) self.doc = self.browser.open('%s?mois=%s' % (self.browser.url, option)).page.doc return None @method class get_documents(ListElement): item_xpath = '//div[@facture-id]' class item(ItemElement): klass = Bill obj__ref = CleanText('//input[@id="noref"]/@value') obj_id = Format('%s_%s', Env('subid'), CleanText('./@facture-id')) obj_url = Format('http://www.bouyguestelecom.fr/parcours/facture/download/index?id=%s&no_reference=%s', CleanText('./@facture-id'), CleanText('./@facture-ligne')) obj_date = Env('date') obj_format = u"pdf" obj_label = CleanText('./text()') obj_type = u"bill" obj_price = CleanDecimal(CleanText('./span', replace=[(u' € ', '.')])) obj_currency = u"€" def parse(self, el): self.env['date'] = parse_french_date('01 %s' % CleanText('./text()')(self)).date() def condition(self): # XXX ugly fix to avoid duplicate bills return CleanText('./@facture-id')(self.el) != CleanText('./following-sibling::div[1]/@facture-id')(self.el) weboob-1.2/modules/bouygues/test.py000066400000000000000000000015541303450110500174710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BouyguesTest(BackendTest): MODULE = 'bouygues' def test_bouygues(self): pass weboob-1.2/modules/bp/000077500000000000000000000000001303450110500146725ustar00rootroot00000000000000weboob-1.2/modules/bp/__init__.py000066400000000000000000000014601303450110500170040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BPModule __all__ = ['BPModule'] weboob-1.2/modules/bp/browser.py000066400000000000000000000321261303450110500167330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from urllib import urlencode from urlparse import urlsplit, parse_qsl from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.browsers import StatesMixin from weboob.exceptions import BrowserIncorrectPassword, BrowserBanned from .pages import ( LoginPage, Initident, CheckPassword, repositionnerCheminCourant, BadLoginPage, AccountDesactivate, AccountList, AccountHistory, CardsList, UnavailablePage, AccountRIB, Advisor, TransferChooseAccounts, CompleteTransfer, TransferConfirm, TransferSummary, ) from .pages.accounthistory import LifeInsuranceInvest, LifeInsuranceHistory, LifeInsuranceHistoryInv, RetirementHistory, SavingAccountSummary from .pages.pro import RedirectPage, ProAccountsList, ProAccountHistory, ProAccountHistoryDownload, ProAccountHistoryCSV, DownloadRib, RibPage from weboob.capabilities.bank import TransferError, Account __all__ = ['BPBrowser', 'BProBrowser'] class BPBrowser(LoginBrowser, StatesMixin): BASEURL = 'https://voscomptesenligne.labanquepostale.fr' # FIXME beware that '.*' in start of URL() won't match all domains but only under BASEURL login_page = URL(r'.*wsost/OstBrokerWeb/loginform.*', LoginPage) repositionner_chemin_courant = URL(r'.*authentification/repositionnerCheminCourant-identif.ea', repositionnerCheminCourant) init_ident = URL(r'.*authentification/initialiser-identif.ea', Initident) check_password = URL(r'.*authentification/verifierMotDePasse-identif.ea', CheckPassword) redirect_page = URL(r'.*voscomptes/identification/identification.ea.*', r'.*voscomptes/synthese/3-synthese.ea', RedirectPage) accounts_list = URL(r'.*synthese_assurancesEtComptes/afficheSynthese-synthese\.ea', r'.*synthese_assurancesEtComptes/rechercheContratAssurance-synthese.ea', r'/voscomptes/canalXHTML/comptesCommun/synthese_assurancesEtComptes/preparerRechercheListePrets-synthese.ea', AccountList) accounts_rib = URL(r'.*voscomptes/canalXHTML/comptesCommun/imprimerRIB/init-imprimer_rib.ea.*', AccountRIB) saving_summary = URL(r'/voscomptes/canalXHTML/assurance/vie/reafficher-assuranceVie.ea(\?numContrat=(?P\w+))?', r'/voscomptes/canalXHTML/assurance/retraiteUCEuro/afficher-assuranceRetraiteUCEuros.ea(\?numContrat=(?P\w+))?', r'/voscomptes/canalXHTML/assurance/retraitePoints/reafficher-assuranceRetraitePoints.ea(\?numContrat=(?P\w+))?', r'/voscomptes/canalXHTML/assurance/prevoyance/reafficher-assurancePrevoyance.ea(\?numContrat=(?P\w+))?', SavingAccountSummary) lifeinsurance_invest = URL(r'/voscomptes/canalXHTML/assurance/vie/valorisation-assuranceVie.ea\?numContrat=(?P\w+)', LifeInsuranceInvest) lifeinsurance_history = URL(r'/voscomptes/canalXHTML/assurance/vie/historiqueVie-assuranceVie.ea\?numContrat=(?P\w+)', LifeInsuranceHistory) lifeinsurance_hist_inv = URL(r'/voscomptes/canalXHTML/assurance/vie/detailMouvement-assuranceVie.ea\?idMouvement=(?P\w+)', LifeInsuranceHistoryInv) retirement_hist = URL(r'/voscomptes/canalXHTML/assurance/retraitePoints/historiqueRetraitePoint-assuranceRetraitePoints.ea(\?numContrat=(?P\w+))?', r'/voscomptes/canalXHTML/assurance/retraiteUCEuro/historiqueMouvements-assuranceRetraiteUCEuros.ea(\?numContrat=(?P\w+))?', r'/voscomptes/canalXHTML/assurance/prevoyance/consulterHistorique-assurancePrevoyance.ea(\?numContrat=(?P\w+))?', RetirementHistory) pro_accounts_list = URL(r'.*voscomptes/synthese/synthese.ea', ProAccountsList) pro_history = URL(r'.*voscomptes/historiqueccp/historiqueccp.ea.*', ProAccountHistory) pro_history_dl = URL(r'.*voscomptes/telechargercomptes/telechargercomptes.ea.*', ProAccountHistoryDownload) pro_history_csv = URL(r'.*voscomptes/telechargercomptes/1-telechargercomptes.ea', ProAccountHistoryCSV) # HistoryParser()? account_history = URL(r'.*CCP/releves_ccp/releveCPP-releve_ccp\.ea', r'.*CNE/releveCNE/releveCNE-releve_cne\.ea', r'.*CB/releveCB/preparerRecherche-mouvementsCarteDD.ea.*', AccountHistory) cards_list = URL(r'.*CB/releveCB/init-mouvementsCarteDD.ea.*', CardsList) transfer_choose = URL(r'/voscomptes/canalXHTML/virement/mpiaiguillage/init-saisieComptes.ea', TransferChooseAccounts) transfer_complete = URL(r'/voscomptes/canalXHTML/virement/mpiaiguillage/soumissionChoixComptes-saisieComptes.ea', CompleteTransfer) transfer_confirm = URL(r'/voscomptes/canalXHTML/virement/virementSafran_pea/validerVirementPea-virementPea.ea', r'/voscomptes/canalXHTML/virement/virementSafran_sepa/valider-virementSepa.ea', r'/voscomptes/canalXHTML/virement/virementSafran_sepa/confirmerInformations-virementSepa.ea', r'/voscomptes/canalXHTML/virement/virementSafran_national/validerVirementNational-virementNational.ea', TransferConfirm) transfer_summary = URL(r'/voscomptes/canalXHTML/virement/virementSafran_national/confirmerVirementNational-virementNational.ea', r'/voscomptes/canalXHTML/virement/virementSafran_pea/confirmerInformations-virementPea.ea', r'/voscomptes/canalXHTML/virement/virementSafran_sepa/confirmerInformations-virementSepa.ea', TransferSummary) badlogin = URL(r'https://transverse.labanquepostale.fr/.*ost/messages\.CVS\.html\?param=0x132120c8.*', # still valid? r'https://transverse.labanquepostale.fr/xo_/messages/message.html\?param=0x132120c8.*', BadLoginPage) disabled_account = URL(r'.*ost/messages\.CVS\.html\?param=0x132120cb.*', r'.*/message\.html\?param=0x132120c.*', r'https://transverse.labanquepostale.fr/xo_/messages/message.html\?param=0x132120cb.*', AccountDesactivate) unavailable = URL(r'https?://.*.labanquepostale.fr/delestage.html', r'https://transverse.labanquepostale.fr/xo_/messages/message.html\?param=delestage', UnavailablePage) rib_dl = URL(r'.*/voscomptes/rib/init-rib.ea', DownloadRib) rib = URL(r'.*/voscomptes/rib/preparerRIB-rib.*', RibPage) advisor = URL(r'/ws_q45/Q45/canalXHTML/commun/authentification/init-identif.ea\?origin=particuliers&codeMedia=0004&entree=HubHome', r'/ws_q45/Q45/canalXHTML/desktop/home/init-home.ea', Advisor) login_url = 'https://voscomptesenligne.labanquepostale.fr/wsost/OstBrokerWeb/loginform?TAM_OP=login&' \ 'ERROR_CODE=0x00000000&URL=%2Fvoscomptes%2FcanalXHTML%2Fidentif.ea%3Forigin%3Dparticuliers' accounts_url = "https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/comptesCommun/synthese_assurancesEtComptes/rechercheContratAssurance-synthese.ea" accounts_and_loans_url = "https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/comptesCommun/synthese_assurancesEtComptes/preparerRechercheListePrets-synthese.ea" accounts = None def do_login(self): self.location(self.login_url) self.page.login(self.username, self.password) if self.redirect_page.is_here() and self.page.check_for_perso(): raise BrowserIncorrectPassword(u"L'identifiant utilisé est celui d'un compte de Particuliers.") if self.badlogin.is_here(): raise BrowserIncorrectPassword() if self.disabled_account.is_here(): raise BrowserBanned() @need_login def get_accounts_list(self): if self.accounts is None: self.accounts = [] ids = set() self.location(self.accounts_url) assert self.accounts_list.is_here() or self.pro_accounts_list.is_here() for account in self.page.get_accounts_list(): ids.add(account.id) self.accounts.append(account) if self.accounts_and_loans_url: self.location(self.accounts_and_loans_url) assert self.accounts_list.is_here() or self.pro_accounts_list.is_here() for account in self.page.get_accounts_list(): if account.id not in ids: ids.add(account.id) self.accounts.append(account) return iter(self.accounts) @need_login def get_history(self, account): v = urlsplit(account._link_id) args = dict(parse_qsl(v.query)) args['typeRecherche'] = 10 self.location('%s?%s' % (v.path, urlencode(args))) transactions = [] if hasattr(self.page, 'get_history'): for tr in self.page.get_history(): transactions.append(tr) for tr in self.get_coming(account): transactions.append(tr) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def get_coming(self, account): transactions = [] for card in account._card_links: self.location(card) if self.cards_list.is_here(): for link in self.page.get_cards(): self.location(link) for tr in self._iter_card_tr(): transactions.append(tr) else: for tr in self._iter_card_tr(): transactions.append(tr) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def iter_investment(self, account): if not account.type == Account.TYPE_LIFE_INSURANCE: return iter([]) self.lifeinsurance_invest.go(id=account.id) assert self.lifeinsurance_invest.is_here() if self.page.has_error(): raise NotImplementedError() return self.page.iter_investments() @need_login def _iter_card_tr(self): """ Iter all pages until there are no transactions. """ ops = self.page.get_history(deferred=True) while True: for tr in ops: yield tr link = self.page.get_next_link() if link is None: return self.location(link) ops = self.page.get_history(deferred=True) @need_login def iter_recipients(self, account_id): return self.transfer_choose.stay_or_go().iter_recipients(account_id=account_id) @need_login def init_transfer(self, account, recipient, amount, label): self.transfer_choose.stay_or_go() self.page.init_transfer(account.id, recipient._value) self.page.complete_transfer(amount, label) return self.page.handle_response(account, recipient, amount, label) @need_login def execute_transfer(self, transfer, code=None): if not self.transfer_confirm.is_here(): raise TransferError('Case not handled.') self.page.confirm() # Should only happen if double auth. if self.transfer_confirm.is_here(): self.page.double_auth(transfer) return self.page.handle_response(transfer) @need_login def get_advisor(self): return iter([self.advisor.go().get_advisor()]) class BProBrowser(BPBrowser): login_url = "https://banqueenligne.entreprises.labanquepostale.fr/wsost/OstBrokerWeb/loginform?TAM_OP=login&ERROR_CODE=0x00000000&URL=%2Fws_q47%2Fvoscomptes%2Fidentification%2Fidentification.ea%3Forigin%3Dprofessionnels" accounts_and_loans_url = None BASEURL = 'https://banqueenligne.entreprises.labanquepostale.fr' def set_variables(self): v = urlsplit(self.page.url) version = v.path.split('/')[1] self.base_url = 'https://banqueenligne.entreprises.labanquepostale.fr/%s' % version self.accounts_url = self.base_url + '/voscomptes/synthese/synthese.ea' @need_login def get_accounts_list(self): self.set_variables() accounts = BPBrowser.get_accounts_list(self) for acc in accounts: self.location('%s/voscomptes/rib/init-rib.ea' % self.base_url) value = self.page.get_rib_value(acc.id) if value: self.location('%s/voscomptes/rib/preparerRIB-rib.ea?%s' % (self.base_url, value)) if self.rib.is_here(): acc.iban = self.page.get_iban() yield acc weboob-1.2/modules/bp/favicon.png000066400000000000000000000033541303450110500170320ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDW pHYs  tIME 6X7tEXtCommentCreated with GIMPWGIDATxo?ٹձsq1$,@C !ԇ"$+@Rj*H R@.!;;]^f~}zر]f|Μ̈/J`Sx-D"@ D"дw?dḍ`Y 7 n)XlfIm,Į0q vn( -H cۓko;x`x%IP(85[(\^^7tD %]G:K߾ˬ}JLJvW~f2q|sx.^}y-m}$*xAxl;wٹ#^ӆsc[*<JRF67lc| GO;g1hHRxZّQQ!VV%/\+O?nvTleeLJ7ai/yd'jCM._4g/A[wz ~i3zx"OOl\q0v,ѸgY ]q)v:i7|*s {q*۝ ձ2GOLq.қ2o(7~d'@A*4KvXwޟX;1y@]N]S\JL_dmrT`݅vSN_Neu#[Mx4Δo߲+uQ5 Cmlu1LC+ܿ7ɶFc7rza^Y^Q[ vsZ_j=1{I?1 X܁tj7 Ϝ_1X^V*r)BqCTr6ut2wqS\I^lpBTvѧOԝBb?YT D"@ `Yh9IENDB`weboob-1.2/modules/bp/module.py000066400000000000000000000103551303450110500165350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from weboob.capabilities.bank import CapBankTransfer, Account, AccountNotFound, RecipientNotFound, TransferError from weboob.capabilities.contact import CapContact from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import BPBrowser, BProBrowser __all__ = ['BPModule'] class BPModule(Module, CapBankTransfer, CapContact): NAME = 'bp' MAINTAINER = u'Nicolas Duhamel' EMAIL = 'nicolas@jombi.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'La Banque Postale' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe', regexp='^(\d{6}|)$'), Value('website', label='Type de compte', default='par', choices={'par': 'Particuliers', 'pro': 'Professionnels'})) def create_default_browser(self): b = {'par': BPBrowser, 'pro': BProBrowser} self.BROWSER = b[self.config['website'].get()] return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): if account.type == Account.TYPE_MARKET: raise NotImplementedError() for tr in self.browser.get_history(account): if not tr._coming: yield tr def iter_coming(self, account): for tr in self.browser.get_coming(account): if tr._coming: yield tr def iter_investment(self, account): return self.browser.iter_investment(account) def iter_transfer_recipients(self, origin_account): if self.config['website'].get() != 'par': raise NotImplementedError() if isinstance(origin_account, Account): origin_account = origin_account.id return self.browser.iter_recipients(origin_account) def init_transfer(self, transfer, **params): if self.config['website'].get() != 'par': raise NotImplementedError() self.logger.info('Going to do a new transfer') if transfer.account_iban: account = find_object(self.iter_accounts(), iban=transfer.account_iban, error=AccountNotFound) else: account = find_object(self.iter_accounts(), id=transfer.account_id, error=AccountNotFound) if transfer.recipient_iban: recipient = find_object(self.iter_transfer_recipients(account.id), iban=transfer.recipient_iban, error=RecipientNotFound) else: recipient = find_object(self.iter_transfer_recipients(account.id), id=transfer.recipient_id, error=RecipientNotFound) try: # quantize to show 2 decimals. amount = Decimal(transfer.amount).quantize(Decimal(10) ** -2) except (AssertionError, ValueError): raise TransferError('something went wrong') return self.browser.init_transfer(account, recipient, amount, transfer.label) def execute_transfer(self, transfer, **params): return self.browser.execute_transfer(transfer) def iter_contacts(self): if self.config['website'].get() != 'par': raise NotImplementedError() return self.browser.get_advisor() weboob-1.2/modules/bp/pages/000077500000000000000000000000001303450110500157715ustar00rootroot00000000000000weboob-1.2/modules/bp/pages/__init__.py000066400000000000000000000025671303450110500201140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .login import LoginPage, Initident, CheckPassword,repositionnerCheminCourant, BadLoginPage, AccountDesactivate, UnavailablePage from .accountlist import AccountList, AccountRIB, Advisor from .accounthistory import AccountHistory, CardsList from .transfer import TransferChooseAccounts, CompleteTransfer, TransferConfirm, TransferSummary __all__ = ['LoginPage','Initident', 'CheckPassword', 'repositionnerCheminCourant', "AccountList", 'AccountHistory', 'BadLoginPage', 'AccountDesactivate', 'TransferChooseAccounts', 'CompleteTransfer', 'TransferConfirm', 'TransferSummary', 'UnavailablePage', 'CardsList', 'AccountRIB', 'Advisor'] weboob-1.2/modules/bp/pages/accounthistory.py000066400000000000000000000224051303450110500214240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime import re from weboob.capabilities.base import NotAvailable from weboob.capabilities.bank import Investment, Transaction as BaseTransaction from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.pages import LoggedPage from weboob.browser.elements import TableElement, ItemElement, method from weboob.browser.filters.html import Link from weboob.browser.filters.standard import CleanDecimal, CleanText, Eval, TableCell, Async, AsyncLoad, Date from .base import MyHTMLPage class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PCHEQUE)( N)? (?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(r'^(?PACHAT CB) (?P.*) (?P

\d{2})\.(?P\d{2}).(?P\d{2}).*'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P(PRELEVEMENT DE|TELEREGLEMENT|TIP)) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PECHEANCEPRET)(?P.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile(r'^CARTE \w+ (?P
\d{2})/(?P\d{2})/(?P\d{2}) A (?P\d+)H(?P\d+) (?PRETRAIT DAB) (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^(?PRETRAIT DAB) (?P
\d{2})/(?P\d{2})/(?P\d{2}) (?P\d+)H(?P\d+) (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^(?PRETRAIT) (?P.*) (?P
\d{2})\.(?P\d{2})\.(?P\d{2})'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?PVIR(EMEN)?T?) (DE |POUR )?(?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PREMBOURST)(?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PCOMMISSIONS)(?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?PFRAIS POUR)(?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P(?PREMUNERATION).*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?PREMISE DE CHEQUES?) (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(u'^(?PDEBIT CARTE BANCAIRE DIFFERE.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile('^(?PFRAIS (TRIMESTRIELS) DE TENUE DE COMPTE.*)'), FrenchTransaction.TYPE_BANK) ] class AccountHistory(LoggedPage, MyHTMLPage): def get_next_link(self): for a in self.doc.xpath('//a[@class="btn_crt"]'): txt = u''.join([txt.strip() for txt in a.itertext()]) if u'mois précédent' in txt: return a.attrib['href'] def get_history(self, deferred=False): """ deffered is True when we are on a card page. """ mvt_table = self.doc.xpath("//table[@id='mouvements']", smart_strings=False)[0] mvt_ligne = mvt_table.xpath("./tbody/tr") operations = [] if deferred: # look for the card number, debit date, and if it is already debited txt = u''.join([txt.strip() for txt in self.doc.xpath('//div[@class="infosynthese"]')[0].itertext()]) m = re.search(u'sur votre carte n°\*\*\*\*\*\*(\d+)\*', txt) card_no = u'inconnu' if m: card_no = m.group(1) m = re.search('(\d+)/(\d+)/(\d+)', txt) if m: debit_date = datetime.date(*map(int, reversed(m.groups()))) coming = 'En cours' in txt else: coming = False for mvt in mvt_ligne: op = Transaction() op.parse(date=CleanText('./td[1]/span')(mvt), raw=CleanText('./td[2]/span')(mvt)) if op.label.startswith('DEBIT CARTE BANCAIRE DIFFERE'): op.deleted = True r = re.compile(r'\d+') tmp = mvt.xpath("./td/span/strong") if not tmp: tmp = mvt.xpath("./td/span") amount = None for t in tmp: if r.search(t.text): amount = t.text op.set_amount(amount) if deferred: op._cardid = 'CARTE %s' % card_no op.type = Transaction.TYPE_DEFERRED_CARD op.rdate = op.date op.date = debit_date # on card page, amounts are without sign if op.amount > 0: op.amount = - op.amount op.rdate = datetime.datetime.combine(op.rdate, datetime.time()) op._coming = coming operations.append(op) return operations class CardsList(LoggedPage, MyHTMLPage): def get_cards(self): cards = [] for tr in self.doc.xpath('//table[@class="dataNum"]/tbody/tr'): cards.append(tr.xpath('.//a')[0].attrib['href']) return cards class SavingAccountSummary(LoggedPage, MyHTMLPage): def on_load(self): link = Link('//ul[has-class("tabs")]//a[@title="Historique des mouvements"]', default=NotAvailable)(self.doc) if link: self.browser.location(link) class InvestTable(TableElement): col_label = 'Support' col_share = [u'Poids en %', u'Répartition en %'] col_quantity = 'Nb U.C' col_valuation = re.compile('Montant') class InvestItem(ItemElement): klass = Investment obj_label = CleanText(TableCell('label', support_th=True)) obj_portfolio_share = Eval(lambda x: x / 100 if x else NotAvailable, CleanDecimal(TableCell('share', support_th=True), replace_dots=True, default=NotAvailable)) obj_quantity = CleanDecimal(TableCell('quantity', support_th=True), replace_dots=True, default=NotAvailable) obj_valuation = CleanDecimal(TableCell('valuation', support_th=True), replace_dots=True, default=NotAvailable) class LifeInsuranceInvest(LoggedPage, MyHTMLPage): def has_error(self): return 'erreur' in CleanText('//p[has-class("titlePage")]')(self.doc) @method class iter_investments(InvestTable): head_xpath = '//table[starts-with(@id, "mouvements")]/thead//th' item_xpath = '//table[starts-with(@id, "mouvements")]/tbody//tr' col_unitvalue = 'Valeur Liquidative' class item(InvestItem): obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=True, default=NotAvailable) class LifeInsuranceHistory(LoggedPage, MyHTMLPage): @method class iter_transactions(TableElement): head_xpath = '//table[@id="options"]/thead//th' item_xpath = '//table[@id="options"]/tbody//tr' col_date = 'Date de valeur' col_amount = 'Montant' col_label = u"Type d'opération" class item(ItemElement): klass = BaseTransaction obj_label = CleanText(TableCell('label')) obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True) obj_date = Date(CleanText(TableCell('date'))) obj__coming = False load_invs = Link('.//a') & AsyncLoad def obj_investments(self): page = Async('invs').loaded_page(self) return list(page.iter_investments()) class LifeInsuranceHistoryInv(LoggedPage, MyHTMLPage): @method class iter_investments(InvestTable): head_xpath = '//table/thead//th' item_xpath = '//table/tbody//tr[position() > 1 and position() < last()]' class item(InvestItem): pass class RetirementHistory(LoggedPage, MyHTMLPage): @method class get_history(TableElement): head_xpath = '//table[@id="mvt" or @id="options" or @id="mouvements"]/thead//th' item_xpath = '//table[@id="mvt" or @id="options" or @id="mouvements"]/tbody//tr' col_date = re.compile('Date') col_label = u"Type d'opération" col_amount = 'Montant' class item(ItemElement): klass = BaseTransaction obj_label = CleanText(TableCell('label')) obj_date = Date(CleanText(TableCell('date'))) obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True) obj__coming = False weboob-1.2/modules/bp/pages/accountlist.py000066400000000000000000000176031303450110500207020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from cStringIO import StringIO import re from decimal import Decimal from weboob.capabilities.base import NotAvailable from weboob.capabilities.bank import Account from weboob.capabilities.contact import Advisor from weboob.browser.elements import ItemElement, method from weboob.browser.pages import LoggedPage, RawPage from weboob.browser.filters.html import Link from weboob.browser.filters.standard import CleanText, Regexp, Env from weboob.exceptions import BrowserUnavailable, NoAccountsException from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.ordereddict import OrderedDict from .base import MyHTMLPage class AccountList(LoggedPage, MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) if self.doc.xpath(u'//h2[text()="%s"]' % u'ERREUR'): self.browser.location('https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/securite/authentification/initialiser-identif.ea') raise BrowserUnavailable() self.accounts = OrderedDict() self.parse_table('comptes', Account.TYPE_CHECKING) self.parse_table('comptesEpargne', Account.TYPE_SAVINGS) self.parse_table('comptesTitres', Account.TYPE_MARKET) self.parse_table('comptesVie', Account.TYPE_LIFE_INSURANCE) self.parse_table('encoursprets', Account.TYPE_LOAN) # FIXME for loans, the balance may be the loan total, not the loan due?? self.parse_table('comptesRetraireEuros') self.parse_table('comptesRetrairePoints') self.parse_table('comptesAssurancePrevoyance') def get_accounts_list(self): if not self.accounts: raise NoAccountsException() return self.accounts.itervalues() def parse_table(self, what, actype=Account.TYPE_UNKNOWN): tables = self.doc.xpath("//table[@id='%s']" % what, smart_strings=False) if len(tables) < 1: return lines = tables[0].xpath(".//tbody/tr") for line in lines: account = Account() account.label = CleanText('./td[1]')(line) account.type = actype account._link_id = Link('./td[1]//a', default=NotAvailable)(line) if not account._link_id: continue if 'BourseEnLigne' in account._link_id: account.type = Account.TYPE_MARKET account.id = CleanText('./td[2]')(line) tmp_balance = CleanText('./td[3]')(line) account.currency = account.get_currency(tmp_balance) if not account.currency: account.currency = u'EUR' if tmp_balance: if any(w in tmp_balance for w in ['non disponible', u'Remboursé intégralement']): continue account.balance = Decimal(FrenchTransaction.clean_amount(tmp_balance)) else: # empty balance info should only be for fully-reimbursed loans assert actype == Account.TYPE_LOAN account.balance = Decimal(0) if actype == Account.TYPE_LOAN: account.balance = -account.balance if account.id in self.accounts: a = self.accounts[account.id] a._card_links.append(account._link_id) if not a.coming: a.coming = Decimal('0.0') a.coming += account.balance else: account._card_links = [] self.accounts[account.id] = account response = self.browser.open('/voscomptes/canalXHTML/comptesCommun/imprimerRIB/init-imprimer_rib.ea?compte.numero=%s' % account.id) account.iban = response.page.get_iban() class Advisor(LoggedPage, MyHTMLPage): @method class get_advisor(ItemElement): klass = Advisor obj_name = Env('name') obj_phone = Env('phone') obj_mobile = Env('mobile', default=NotAvailable) obj_agency = Env('agency', default=NotAvailable) obj_email = NotAvailable def obj_address(self): return CleanText('//div[h3[contains(text(), "Bureau")]]/div[not(@class)][position() > 1]')(self) or NotAvailable def parse(self, el): # we have two kinds of page and sometimes we don't have any advisor agency_phone = CleanText('//span/a[contains(@href, "rendezVous")]', replace=[(' ', '')], default=NotAvailable)(self) or \ CleanText('//div[has-class("lbp-numero")]/span', replace=[(' ', '')], default=NotAvailable)(self) advisor_phone = Regexp(CleanText('//div[h3[contains(text(), "conseil")]]//span[2]', replace=[(' ', '')], default=""), '(\d+)', default="")(self) if advisor_phone.startswith(("06", "07")): self.env['phone'] = agency_phone self.env['mobile'] = advisor_phone else: self.env['phone'] = advisor_phone or agency_phone agency = CleanText('//div[h3[contains(text(), "Bureau")]]/div[not(@class)][1]')(self) or NotAvailable name = CleanText('//div[h3[contains(text(), "conseil")]]//span[1]', default=None)(self) or \ CleanText('//div[@class="lbp-font-accueil"]/div[2]/div[1]/span[1]', default=None)(self) if name: self.env['name'] = name self.env['agency'] = agency else: self.env['name'] = agency class AccountRIB(LoggedPage, RawPage): iban_regexp = r'BankIdentiferCode(\w+)PSS' def __init__(self, *args, **kwargs): super(AccountRIB, self).__init__(*args, **kwargs) self.parsed_text = '' try: try: from pdfminer.pdfdocument import PDFDocument from pdfminer.pdfpage import PDFPage newapi = True except ImportError: from pdfminer.pdfparser import PDFDocument newapi = False from pdfminer.pdfparser import PDFParser, PDFSyntaxError from pdfminer.converter import TextConverter from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter except ImportError: self.logger.warning('Please install python-pdfminer to get IBANs') else: parser = PDFParser(StringIO(self.doc)) try: if newapi: doc = PDFDocument(parser) else: doc = PDFDocument() parser.set_document(doc) doc.set_parser(parser) except PDFSyntaxError: return rsrcmgr = PDFResourceManager() out = StringIO() device = TextConverter(rsrcmgr, out) interpreter = PDFPageInterpreter(rsrcmgr, device) if newapi: pages = PDFPage.create_pages(doc) else: doc.initialize() pages = doc.get_pages() for page in pages: interpreter.process_page(page) self.parsed_text = out.getvalue() def get_iban(self): m = re.search(self.iban_regexp, self.parsed_text) if m: return unicode(m.group(1)) return None weboob-1.2/modules/bp/pages/base.py000066400000000000000000000020421303450110500172530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 budget-insight # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage class MyHTMLPage(HTMLPage): ENCODING = 'iso-8859-1' def on_load(self): deconnexion = self.doc.xpath('//iframe[contains(@id, "deconnexion")] | //p[@class="txt" and contains(text(), "Session expir")]') if deconnexion: self.browser.do_login() weboob-1.2/modules/bp/pages/login.py000066400000000000000000000115011303450110500174510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from cStringIO import StringIO import re from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword, NoAccountsException from weboob.browser.pages import LoggedPage from weboob.browser.filters.standard import CleanText from weboob.tools.captcha.virtkeyboard import VirtKeyboard from .base import MyHTMLPage class UnavailablePage(MyHTMLPage): def on_load(self): raise BrowserUnavailable() class Keyboard(VirtKeyboard): symbols={'0':'daa52d75287bea58f505823ef6c8b96c', '1':'f5da96c2592803a8cdc5a928a2e4a3b0', '2':'9ff78367d5cb89cacae475368a11e3af', '3':'908a0a42a424b95d4d885ce91bc3d920', '4':'3fc069f33b801b3d0cdce6655a65c0ac', '5':'58a2afebf1551d45ccad79fad1600fc3', '6':'7fedfd9e57007f2985c3a1f44fb38ea1', '7':'389b8ef432ae996ac0141a2fcc7b540f', '8':'bf357ff09cc29ea544991642cd97d453', '9':'b744015eb89c1b950e13a81364112cd6', } color=(0xff, 0xff, 0xff) def __init__(self, page): m = re.search(r'background:url\("(.*?)"\)', ''.join(page.doc.xpath('//script/text()'))) if m: img_url = m.group(1) size = 252 else: img_url = page.doc.xpath('//img[@id="imageCVS"]')[0].attrib['src'] size = 146 coords = {} x, y, width, height = (0, 0, size/4, size/4) for i, _ in enumerate(page.doc.xpath('//div[@id="imageclavier"]//button')): code = '%02d' % i coords[code] = (x+4, y+4, x+width-8, y+height-8) if (x + width + 1) >= size: y += height + 1 x = 0 else: x += width + 1 data = page.browser.open(img_url).content VirtKeyboard.__init__(self, StringIO(data), coords, self.color) self.check_symbols(self.symbols, page.browser.responses_dirname) def get_symbol_code(self,md5sum): code = VirtKeyboard.get_symbol_code(self,md5sum) return '%02d' % int(code.split('_')[-1]) def get_string_code(self,string): code = '' for c in string: code += self.get_symbol_code(self.symbols[c]) return code def get_symbol_coords(self, coords): # strip borders x1, y1, x2, y2 = coords return VirtKeyboard.get_symbol_coords(self, (x1+3, y1+3, x2-3, y2-3)) class LoginPage(MyHTMLPage): def login(self, login, pwd): vk = Keyboard(self) form = self.get_form(name='formAccesCompte') form['password'] = vk.get_string_code(pwd) form['username'] = login.encode(self.ENCODING) form.submit() class repositionnerCheminCourant(LoggedPage, MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) response = self.browser.open("https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/securite/authentification/initialiser-identif.ea") if isinstance(response.page, Initident): response.page.on_load() if "vous ne disposez pas" in response.content: raise BrowserIncorrectPassword("No online banking service for these ids") class Initident(LoggedPage, MyHTMLPage): def on_load(self): self.browser.open("https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/securite/authentification/verifierMotDePasse-identif.ea") if self.doc.xpath(u'//span[contains(text(), "L\'identifiant utilisé est celui d\'une Entreprise ou d\'une Association")]'): raise BrowserIncorrectPassword(u"L'identifiant utilisé est celui d'une Entreprise ou d'une Association") no_accounts = CleanText(u'//div[@class="textFCK"]')(self.doc) if no_accounts: raise NoAccountsException(no_accounts) MyHTMLPage.on_load(self) class CheckPassword(LoggedPage, MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) self.browser.open("https://voscomptesenligne.labanquepostale.fr/voscomptes/canalXHTML/comptesCommun/synthese_assurancesEtComptes/init-synthese.ea") class BadLoginPage(MyHTMLPage): pass class AccountDesactivate(LoggedPage, MyHTMLPage): pass weboob-1.2/modules/bp/pages/pro.py000066400000000000000000000107271303450110500171520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import datetime from dateutil.relativedelta import relativedelta from decimal import Decimal from weboob.browser.filters.standard import CleanText from weboob.browser.pages import LoggedPage, CsvPage from weboob.capabilities.bank import Account from .accounthistory import Transaction from .base import MyHTMLPage class RedirectPage(LoggedPage, MyHTMLPage): def check_for_perso(self): return self.doc.xpath(u'//p[contains(text(), "L\'identifiant utilisé est celui d\'un compte de Particuliers")]') class ProAccountsList(LoggedPage, MyHTMLPage): ACCOUNT_TYPES = {u'Comptes titres': Account.TYPE_MARKET, u'Comptes épargne': Account.TYPE_SAVINGS, u'Comptes courants': Account.TYPE_CHECKING, } def get_accounts_list(self): for table in self.doc.xpath('//div[@class="comptestabl"]/table'): try: account_type = self.ACCOUNT_TYPES[table.get('summary')] if not account_type: account_type = self.ACCOUNT_TYPES[table.xpath('./caption/text()')[0].strip()] except (IndexError,KeyError): account_type = Account.TYPE_UNKNOWN for tr in table.xpath('./tbody/tr'): cols = tr.findall('td') link = cols[0].find('a') if link is None: continue a = Account() a.type = account_type a.id = unicode(re.search('([A-Z\d]{4}[A-Z\d\*]{3}[A-Z\d]{4})', link.attrib['title']).group(1)) a.label = unicode(link.attrib['title'].replace('%s ' % a.id, '')) tmp_balance = CleanText(None).filter(cols[1]) a.currency = a.get_currency(tmp_balance) if not a.currency: a.currency = u'EUR' a.balance = Decimal(Transaction.clean_amount(tmp_balance)) a._card_links = [] a._link_id = link.attrib['href'] yield a class ProAccountHistory(LoggedPage, MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) link = self.doc.xpath('//a[contains(@href, "telechargercomptes.ea")]/@href')[0] self.browser.location(link) class ProAccountHistoryDownload(LoggedPage, MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) form = self.get_form(name='telechargement') form['dateDebutPeriode'] = (datetime.date.today() - relativedelta(months=11)).strftime('%d/%m/%Y') form.submit() class ProAccountHistoryCSV(LoggedPage, CsvPage): FMTPARAMS = {'delimiter': ';'} def get_next_link(self): return False def get_history(self, deferred=False): operations = [] for line in self.doc: if len(line) < 4 or line[0] == 'Date': continue t = Transaction() t.parse(raw=line[1], date=line[0]) t.set_amount(line[2]) t._coming = False operations.append(t) operations = sorted(operations, lambda a, b: cmp(a.date, b.date), reverse=True) for op in operations: yield op class DownloadRib(LoggedPage, MyHTMLPage): def get_rib_value(self, acc_id): opt = self.doc.xpath('//div[@class="rechform"]//option') for o in opt: if acc_id in o.text: return o.xpath('./@value')[0] return None class RibPage(LoggedPage, MyHTMLPage): def get_iban(self): if self.doc.xpath('//div[@class="blocbleu"][2]//table[@class="datalist"]'): return CleanText()\ .filter(self.doc.xpath('//div[@class="blocbleu"][2]//table[@class="datalist"]')[0])\ .replace(' ', '').strip() return None weboob-1.2/modules/bp/pages/transfer.py000066400000000000000000000171431303450110500201750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.capabilities.bank import TransferError, Recipient, NotAvailable, Transfer, TransferStep, AccountNotFound from weboob.capabilities.base import find_object from weboob.browser.pages import LoggedPage from weboob.browser.filters.standard import CleanText, Env, Regexp, Date, CleanDecimal from weboob.browser.filters.html import Attr from weboob.browser.elements import ListElement, ItemElement, method, SkipItem from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.capabilities.bank.iban import is_iban_valid from weboob.tools.value import Value from .base import MyHTMLPage class CheckTransferError(MyHTMLPage): def on_load(self): MyHTMLPage.on_load(self) error = CleanText(u'//span[@class="app_erreur"] | //p[@class="warning"] | //p[contains(text(), "Votre virement n\'a pas pu être enregistré")]')(self.doc) if error: raise TransferError(error) class TransferChooseAccounts(LoggedPage, MyHTMLPage): def is_inner(self, text): for option in self.doc.xpath('//select[@id="donneesSaisie.idxCompteEmetteur"]/option'): if text == CleanText('.')(option): return True return False @method class iter_recipients(ListElement): def condition(self): return any(self.env['account_id'] in CleanText('.')(option) for option in self.page.doc.xpath('//select[@id="donneesSaisie.idxCompteEmetteur"]/option')) # You're not dreaming, this is real life item_xpath = '//select[@id="caca"]/option' class Item(ItemElement): klass = Recipient def condition(self): return self.el.attrib['value'] != '-1' def validate(self, obj): # Some international external recipients show those infos: # INT - CA - 0815304220511006 - CEGEP CANADA # Skipping those for the moment. return not obj.iban or is_iban_valid(obj.iban) obj_category = Env('category') obj_label = Env('label') obj_id = Env('id') obj_currency = u'EUR' obj_iban = Env('iban') obj_bank_name = Env('bank_name') obj__value = Attr('.', 'value') def obj_enabled_at(self): return datetime.now().replace(microsecond=0) def parse(self, el): if any(s in CleanText('.')(el) for s in ['Avoir disponible', 'Solde']) or self.page.is_inner(CleanText('.')(el)): self.env['category'] = u'Interne' else: self.env['category'] = u'Externe' if self.env['category'] == u'Interne': _id = Regexp(CleanText('.'), '- (.*?) -')(el) if _id == self.env['account_id']: raise SkipItem() try: account = find_object(self.page.browser.get_accounts_list(), id=_id, error=AccountNotFound) self.env['id'] = _id self.env['label'] = account.label self.env['iban'] = account.iban except AccountNotFound: self.env['id'] = Regexp(CleanText('.'), '- (.*?) -')(el).replace(' ', '') self.env['iban'] = NotAvailable label = CleanText('.')(el).split('-') self.env['label'] = '%s %s' % (label[0].strip(), label[-1].strip()) self.env['bank_name'] = u'La Banque Postale' else: self.env['id'] = self.env['iban'] = Regexp(CleanText('.'), '- (.*?) -')(el).replace(' ', '') self.env['label'] = CleanText('.')(el).split('-')[-1].strip() first_part = CleanText('.')(el).split('-')[0].strip() self.env['bank_name'] = u'La Banque Postale' if first_part in ['CCP', 'PEL'] else NotAvailable def init_transfer(self, account_id, recipient_value): matched_values = [Attr('.', 'value')(option) for option in self.doc.xpath('//select[@id="donneesSaisie.idxCompteEmetteur"]/option') \ if account_id in CleanText('.')(option)] assert len(matched_values) == 1 form = self.get_form(xpath='//form[@class="formvirement"]') form['donneesSaisie.idxCompteReceveur'] = recipient_value form['donneesSaisie.idxCompteEmetteur'] = matched_values[0] form.submit() class CompleteTransfer(LoggedPage, CheckTransferError): def complete_transfer(self, amount, label): form = self.get_form(xpath='//form[@method]') form['montant'] = amount if 'commentaire' in form and label: form['commentaire'] = label form.submit() class TransferConfirm(LoggedPage, CheckTransferError): def is_here(self): return not CleanText('//p[contains(text(), "Vous pouvez le consulter dans le menu")]')(self.doc) def double_auth(self, transfer): code_needed = CleanText('//label[@for="code_securite"]')(self.doc) if code_needed: raise TransferStep(transfer, Value('code', label= code_needed)) def confirm(self): form = self.get_form(id='formID') form.submit() def handle_response(self, account, recipient, amount, reason): account_txt = CleanText('//form//dl/dt[span[contains(text(), "biter")]]/following::dd[1]', replace=[(' ', '')])(self.doc) recipient_txt = CleanText('//form//dl/dt[span[contains(text(), "diter")]]/following::dd[1]', replace=[(' ', '')])(self.doc) try: assert account.id in account_txt assert recipient.id in recipient_txt except AssertionError: raise TransferError('Something went wrong') r_amount = CleanDecimal('//form//dl/dt[span[contains(text(), "Montant")]]/following::dd[1]', replace_dots=True)(self.doc) exec_date = Date(CleanText('//form//dl/dt[span[contains(text(), "Date")]]/following::dd[1]'), dayfirst=True)(self.doc) currency = FrenchTransaction.Currency('//form//dl/dt[span[contains(text(), "Montant")]]/following::dd[1]')(self.doc) transfer = Transfer() transfer.currency = currency transfer.amount = r_amount transfer.account_iban = account.iban transfer.recipient_iban = recipient.iban transfer.account_id = account.id transfer.recipient_id = recipient.id transfer.exec_date = exec_date transfer.label = reason transfer.account_label = account.label transfer.recipient_label = recipient.label transfer.account_balance = account.balance return transfer class TransferSummary(LoggedPage, CheckTransferError): def handle_response(self, transfer): transfer.id = Regexp(CleanText('//div[@class="bloc Tmargin"]'), 'Votre virement N.+ (\d+) ')(self.doc) return transfer weboob-1.2/modules/bred/000077500000000000000000000000001303450110500152055ustar00rootroot00000000000000weboob-1.2/modules/bred/__init__.py000066400000000000000000000014241303450110500173170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BredModule __all__ = ['BredModule'] weboob-1.2/modules/bred/bred/000077500000000000000000000000001303450110500161215ustar00rootroot00000000000000weboob-1.2/modules/bred/bred/__init__.py000066400000000000000000000000741303450110500202330ustar00rootroot00000000000000from .browser import BredBrowser __all__ = ['BredBrowser'] weboob-1.2/modules/bred/bred/browser.py000066400000000000000000000235221303450110500201620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import json import re from datetime import date from decimal import Decimal from weboob.capabilities.base import NotAvailable from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.exceptions import BrowserIncorrectPassword, BrowserHTTPError, BrowserUnavailable, ParseError from weboob.browser import DomainBrowser __all__ = ['BredBrowser'] class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^.*Virement (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'PRELEV SEPA (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(u'.*Prélèvement.*'), FrenchTransaction.TYPE_ORDER), (re.compile(u'^(REGL|Rgt)(?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?P.*) Carte \d+\s+ LE (?P
\d{2})/(?P\d{2})/(?P\d{2})'), FrenchTransaction.TYPE_CARD), (re.compile(u'^Débit mensuel.*'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile(u"^Retrait d'espèces à un DAB (?P.*) CARTE [X\d]+ LE (?P
\d{2})/(?P\d{2})/(?P\d{2})"), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(u'^Paiement de chèque (?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(u'^(Cotisation|Intérêts) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile(u'^(Remise Chèque|Remise de chèque)\s*(?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^Versement (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] class BredBrowser(DomainBrowser): BASEURL = 'https://www.bred.fr' def __init__(self, accnum, login, password, *args, **kwargs): super(BredBrowser, self).__init__(*args, **kwargs) self.login = login # Bred only use first 8 char (even if the password is set to be bigger) # The js login form remove after 8th char. No comment. self.password = password[:8] self.accnum = accnum self.universes = None self.current_univers = None def do_login(self, login, password): r = self.open('/transactionnel/Authentication', data={'identifiant': login, 'password': password}) if 'gestion-des-erreurs/erreur-pwd' in r.url: raise BrowserIncorrectPassword('Bad login/password.') if 'gestion-des-erreurs/opposition' in r.url: raise BrowserIncorrectPassword('Your account is disabled') if '/pages-gestion-des-erreurs/erreur-technique' in r.url: raise BrowserUnavailable('A technical error occured') ACCOUNT_TYPES = {'000': Account.TYPE_CHECKING, '999': Account.TYPE_MARKET, '011': Account.TYPE_CARD, '023': Account.TYPE_SAVINGS, '078': Account.TYPE_SAVINGS, '080': Account.TYPE_SAVINGS, '027': Account.TYPE_SAVINGS, '037': Account.TYPE_SAVINGS, '730': Account.TYPE_DEPOSIT, } def api_open(self, *args, **kwargs): try: return super(BredBrowser, self).open(*args, **kwargs) except BrowserHTTPError: self.do_login(self.login, self.password) return super(BredBrowser, self).open(*args, **kwargs) def set_universes(self): universes = [] r = self.api_open('/transactionnel/services/applications/menu/getMenuUnivers') for univers in r.json()['content']['menus']: universes.append(univers['universKey']) if not universes: # There is just the default univers here. universes.append('') self.current_univers = '' else: # The following is needed to get the default univers in the list. self.move_to_univers(universes[0]) r = self.api_open('/transactionnel/services/applications/menu/getMenuUnivers') for univers in r.json()['content']['menus']: if univers['universKey'] not in universes: universes.append(univers['universKey']) self.universes = universes def move_to_univers(self, univers): x_token_bred = self.api_open('/transactionnel/services/rest/User/nonce?random=').json()['content'] data = {} data['all'] = 'true' data['univers'] = univers self.api_open('/transactionnel/services/rest/User/switch', data=json.dumps(data), headers={'x-token-bred': x_token_bred}) self.current_univers = univers def get_accounts_list(self): if not self.universes: self.set_universes() accounts = [] for univers in self.universes: if univers != self.current_univers: self.move_to_univers(univers) accounts.extend(self.get_list()) return accounts def get_list(self): r = self.api_open('/transactionnel/services/rest/Account/accounts') for content in r.json()['content']: if self.accnum != '00000000000' and content['numero'] != self.accnum: continue for poste in content['postes']: a = Account() a._number = content['numeroLong'] a._nature = poste['codeNature'] a._consultable = poste['consultable'] a._univers = self.current_univers a.id = '%s.%s' % (a._number, a._nature) a.type = self.ACCOUNT_TYPES.get(poste['codeNature'], Account.TYPE_UNKNOWN) if a.type == Account.TYPE_CHECKING: iban_response = self.api_open('/transactionnel/services/rest/Account/account/%s/iban' % a._number).json() a.iban = iban_response['content']['iban'] if 'content' in iban_response else NotAvailable else: a.iban = NotAvailable if 'numeroDossier' in poste and poste['numeroDossier']: a._file_number = poste['numeroDossier'] a.id += '.%s' % a._file_number if poste['postePortefeuille']: a.label = u'Portefeuille Titres' a.balance = Decimal(str(poste['montantTitres']['valeur'])) a.currency = poste['montantTitres']['monnaie']['code'].strip() if not a.balance and not a.currency and 'dateTitres' not in poste: continue yield a if 'libelle' not in poste: continue a.label = ' '.join([content['intitule'].strip(), poste['libelle'].strip()]) a.balance = Decimal(str(poste['solde']['valeur'])) a.currency = poste['solde']['monnaie']['code'].strip() # Some accounts may have balance currency if 'Solde en devises' in a.label and a.currency != u'EUR': a.id += str(poste['monnaie']['codeSwift']) yield a def get_history(self, account): if not account._consultable: raise NotImplementedError() if account._univers != self.current_univers: self.move_to_univers(account._univers) offset = 0 next_page = True seen = set() while next_page: r = self.api_open('/transactionnel/services/applications/operations/get/%(number)s/%(nature)s/00/%(currency)s/%(startDate)s/%(endDate)s/%(offset)s/%(limit)s' % {'number': account._number, 'nature': account._nature, 'currency': account.currency, 'startDate': '2000-01-01', 'endDate': date.today().strftime('%Y-%m-%d'), 'offset': offset, 'limit': 50 }) next_page = False offset += 50 transactions = [] for op in reversed(r.json()['content']['operations']): next_page = True t = Transaction() if op['id'] in seen: raise ParseError('There are several transactions with the same ID, probably an infinite loop') t.id = op['id'] seen.add(t.id) d = date.fromtimestamp(op.get('dateDebit', op.get('dateOperation'))/1000) raw = ' '.join([op['libelle']] + op['details']) vdate = date.fromtimestamp(op.get('dateValeur', op.get('dateDebit', op.get('dateOperation')))/1000) t.parse(d, raw, vdate=vdate) t.amount = Decimal(str(op['montant'])) t.rdate = date.fromtimestamp(op.get('dateOperation', op.get('dateDebit'))/1000) if 'categorie' in op: t.category = op['categorie'] t.label = op['libelle'] transactions.append(t) # Transactions are unsorted for t in sorted(transactions, key=lambda t: t.rdate, reverse=True): yield t weboob-1.2/modules/bred/dispobank/000077500000000000000000000000001303450110500171575ustar00rootroot00000000000000weboob-1.2/modules/bred/dispobank/__init__.py000066400000000000000000000001061303450110500212650ustar00rootroot00000000000000from .browser import DispoBankBrowser __all__ = ['DispoBankBrowser'] weboob-1.2/modules/bred/dispobank/browser.py000066400000000000000000000105131303450110500212140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, LoginResultPage, AccountsPage, EmptyPage, TransactionsPage __all__ = ['DispoBankBrowser'] class DispoBankBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'www.dispobank.fr' CERTHASH = ['9b77dab9c84e1dc9e0798de561a6541ff15f038f60b36ca74c29be1def6c19a3', '375f1fed165d34aacaaf71674ab14ca6c1b38404cf748278714fde3c58385ff0', '0853a056453b56aea6a29085ef3f3721b18db2052aa8e84220720d44e0eb22af', 'a2b017a0cecf8e8bacf1e04c10d4fa647f5ade416fe64129bfc034ef95f310f5'] ENCODING = 'iso-8859-15' PAGES = {r'https://www.\w+.fr/mylittleform.*': LoginPage, r'https://www.\w+.fr/Andromede/MainAuth.*': LoginResultPage, r'https://www.\w+.fr/Andromede/Main': AccountsPage, r'https://www.\w+.fr/Andromede/Ecriture': TransactionsPage, r'https://www.\w+.fr/Andromede/applications/index.jsp': EmptyPage, r'https://www.bred.fr/': EmptyPage, r'https://www.dispobank.fr/?': LoginPage, } URLS = {'bred': {'home': 'https://www.bred.fr/Andromede/Main', 'login': 'https://www.bred.fr/mylittleform?type=1', }, 'dispobank': {'home': 'https://www.dispobank.fr', 'login': 'https://www.dispobank.fr', } } def __init__(self, accnum, *args, **kwargs): self.accnum = accnum self.website = 'dispobank' Browser.__init__(self, *args, **kwargs) def is_logged(self): return self.page is not None and not self.is_on_page(LoginPage) def home(self): if not self.is_logged(): self.login() else: self.location(self.URLS[self.website]['home']) def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.is_on_page(LoginPage): self.location(self.URLS[self.website]['login'], no_login=True) self.page.login(self.username, self.password) assert self.is_on_page((LoginResultPage, EmptyPage)) if self.is_on_page(LoginResultPage): error = self.page.get_error() if error is not None: raise BrowserIncorrectPassword(error) self.page.confirm() def get_accounts_list(self): if not self.is_on_page(AccountsPage): self.location('https://www.%s.fr/Andromede/Main' % self.website) return self.page.get_list() def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == id: return a return None def get_history(self, account): numero_compte, numero_poste = account.id.split('.') data = {'typeDemande': 'recherche', 'motRecherche': '', 'numero_compte': numero_compte, 'numero_poste': numero_poste, 'detail': '', 'tri': 'date', 'sens': 'sort', 'monnaie': 'EUR', 'index_hist': 4 } self.location('https://www.%s.fr/Andromede/Ecriture' % self.website, urllib.urlencode(data)) assert self.is_on_page(TransactionsPage) return self.page.get_history() weboob-1.2/modules/bred/dispobank/pages.py000066400000000000000000000233401303450110500206320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from mechanize import FormNotFoundError from weboob.deprecated.mech import ClientForm ControlNotFoundError = ClientForm.ControlNotFoundError from decimal import Decimal, InvalidOperation import re from weboob.deprecated.browser import Page from weboob.tools.misc import to_unicode from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction class LoginPage(Page): def login(self, login, passwd): try: length = int(self.document.xpath('//input[@id="pass"]')[0].attrib['maxlength']) except (IndexError,KeyError): pass else: passwd = passwd[:length] self.browser.select_form(name='authen') try: self.browser['id'] = login.encode(self.browser.ENCODING) self.browser['pass'] = passwd.encode(self.browser.ENCODING) except ControlNotFoundError: self.browser.controls.append(ClientForm.TextControl('text', 'id', {'value': login.encode(self.browser.ENCODING)})) self.browser.controls.append(ClientForm.TextControl('text', 'pass', {'value': passwd.encode(self.browser.ENCODING)})) self.browser.submit(nologin=True) class LoginResultPage(Page): def on_loaded(self): for script in self.document.xpath('//script'): text = script.text if text is None: continue m = re.search("window.location.replace\('([^']+)'\);", text) if m: self.browser.location(m.group(1)) try: self.browser.select_form(name='banque') except FormNotFoundError: pass else: self.browser.set_all_readonly(False) accounts = OrderedDict() for tr in self.document.getroot().cssselect('table.compteTable > tbody > tr'): if len(tr.findall('td')) == 0: continue attr = tr.xpath('.//a')[0].attrib.get('onclick', '') m = re.search("value = '(\w+)';(checkAndSubmit\('\w+','(\w+)','(\w+)'\))?", attr) if m: typeCompte = m.group(1) tagName = m.group(3) if tagName is not None: value = self.document.xpath('//input[@name="%s"]' % m.group(3))[int(m.group(4))].attrib['value'] else: value = typeCompte accounts[value] = (typeCompte, tagName) try: typeCompte, tagName = accounts[self.browser.accnum] value = self.browser.accnum except KeyError: accnums = ', '.join(accounts.keys()) if self.browser.accnum != '00000000000': self.logger.warning(u'Unable to find account "%s". Available ones: %s' % (self.browser.accnum, accnums)) elif len(accounts) > 1: self.logger.warning('There are several accounts, please use "accnum" backend parameter to force the one to use (%s)' % accnums) value, (typeCompte, tagName) = accounts.popitem(last=False) self.browser['typeCompte'] = typeCompte if tagName is not None: self.browser[tagName] = [value] self.browser.submit() def confirm(self): self.browser.location('MainAuth?typeDemande=AC', no_login=True) def get_error(self): error = self.document.xpath('//td[@class="txt_norm2"]') if len(error) == 0: return None error = error[0] if error.find('b') is not None: error = error.find('b') return error.text.strip() class EmptyPage(Page): pass class BredBasePage(Page): def js2args(self, s): cur_arg = None args = {} # For example: # javascript:reloadApplication('nom_application', 'compte_telechargement', 'numero_poste', '000', 'numero_compte', '12345678901','monnaie','EUR'); for sub in re.findall("'([^']+)'", s): if cur_arg is None: cur_arg = sub else: args[cur_arg] = sub cur_arg = None return args class AccountsPage(BredBasePage): def get_list(self): for tr in self.document.xpath('//table[@class="compteTable"]/tr'): if not tr.attrib.get('class', '').startswith('ligne_'): continue cols = tr.findall('td') if len(cols) < 2: continue try: amount = sum([Decimal(FrenchTransaction.clean_amount(txt)) for txt in cols[-1].itertext() if len(txt.strip()) > 0]) except InvalidOperation: continue a = cols[0].find('a') if a is None: for a in cols[0].xpath('.//li/a'): args = self.js2args(a.attrib['href']) if 'numero_compte' not in args or 'numero_poste' not in args: self.logger.warning('Card link with strange args: %s' % args) continue account = Account() account.id = '%s.%s' % (args['numero_compte'], args['numero_poste']) account.label = u'Carte %s' % self.parser.tocleanstring(a) account.balance = amount account.type = account.TYPE_CARD account.currency = [account.get_currency(txt) for txt in cols[-1].itertext() if len(txt.strip()) > 0][0] yield account continue args = self.js2args(a.attrib['href']) if 'numero_compte' not in args or 'numero_poste' not in args: self.logger.warning('Account link for %r with strange args: %s' % (a.attrib.get('alt', a.text), args)) continue account = Account() account.id = u'%s.%s' % (args['numero_compte'], args['numero_poste']) account.label = to_unicode(a.attrib.get('alt', a.text.strip())) account.balance = amount account.currency = [account.get_currency(txt) for txt in cols[-1].itertext() if len(txt.strip()) > 0][0] yield account class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^RETRAIT G.A.B. \d+ (?P.*?)( CARTE .*)? LE (?P
\d{2})/(?P\d{2})/(?P\d{2}).*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^PRLV (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?P.*) TRANSACTION( CARTE .*)? LE (?P
\d{2})/(?P\d{2})/(?P\d{2}) ?(.*)$'), FrenchTransaction.TYPE_CARD), (re.compile('^CHEQUE.*'), FrenchTransaction.TYPE_CHECK), (re.compile('^(CONVENTION \d+ )?COTISATION (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^CB PAIEM. EN \d+ FOIS \d+ (?P.*?) LE .* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_CARD), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class TransactionsPage(Page): def get_history(self): for tr in self.document.xpath('//div[@class="scrollTbody"]/table//tr'): cols = tr.findall('td') if len(cols) < 4: continue col_label = cols[1] if col_label.find('a') is not None: col_label = col_label.find('a') date = self.parser.tocleanstring(cols[0]) label = self.parser.tocleanstring(col_label) t = Transaction(col_label.attrib.get('id', '')) # an optional tooltip on page contain the second part of the transaction label. tooltip = self.document.xpath('//div[@id="tooltip%s"]' % t.id) raw = label if len(tooltip) > 0: raw += u' ' + u' '.join([txt.strip() for txt in tooltip[0].itertext()]) raw = re.sub(r'[ ]+', ' ', raw) t.parse(date, raw) # as only the first part of label is important to user, if there are no subpart # taken by FrenchTransaction regexps, reset the label as first part. if t.label == t.raw: t.label = label debit = self.parser.tocleanstring(cols[-2]) credit = self.parser.tocleanstring(cols[-1]) t.set_amount(credit, debit) yield t weboob-1.2/modules/bred/favicon.png000066400000000000000000000024201303450110500173360ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME .X1IDATx[}Hg~ƚCdkn-+hYJYt+6 v[Z(:6Ơ0velS em~Ykj/wleֻ&w{̓w!q =PLyė)OWB׶i)x|kmjW]|WvlPA5lP9ۈ"՝|m;Xy٠`݃c8oa$زA ~ m44~iBEs>Fṇ-m3snII*}, :hu]@pDžkЫBmo|2V.n&sdl0*+3AQC!46M4ՒMfٕ,4|%xQhK> ZO>DbcpL-&rlݜNú=BPxM-+W]vo&d=?hRB,qX- ʺsӢ\t3^7k%Y$`o:eO0\^NRTT"nUV}SoTV&Q/*AQΒpxcHK zu6rU/>v2*W %PLFYd*_ gF0~.*y9[k형&jJVâk߻]8s~Qkp-]D`2PD h X6ǔ^%(6  5%HMѣ0߈"3^s?1w]-@zP F}M!'3ǹ<T{dn=%|c W\o+y՝(;|@L7Л=r*Tgw>Gb `o /PO+ o}oNc_/:-@NG8lO!}R?% $ syFl0"/m]+x4N_K DeY|xȇ#RIq$[%p-p`Xܾ1Kn\r9#rXq *> Hs;IENDB`weboob-1.2/modules/bred/module.py000066400000000000000000000046361303450110500170550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .bred import BredBrowser from .dispobank import DispoBankBrowser __all__ = ['BredModule'] class BredModule(Module, CapBank): NAME = 'bred' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Bred' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe'), Value('website', label=u"Site d'accès", default='bred', choices={'bred': 'BRED', 'dispobank': 'DispoBank'}), Value('accnum', label=u'Numéro du compte bancaire (optionnel)', default='', masked=False) ) BROWSERS = {'bred': BredBrowser, 'dispobank': DispoBankBrowser, } def create_default_browser(self): self.BROWSER = self.BROWSERS[self.config['website'].get()] return self.create_browser(self.config['accnum'].get().replace(' ','').zfill(11), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): return self.browser.get_history(account) weboob-1.2/modules/bred/test.py000066400000000000000000000017501303450110500165410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class BredTest(BackendTest): MODULE = 'bred' def test_bred(self): l = list(self.backend.iter_accounts()) a = l[0] list(self.backend.iter_history(a)) list(self.backend.iter_coming(a)) weboob-1.2/modules/btmon/000077500000000000000000000000001303450110500154105ustar00rootroot00000000000000weboob-1.2/modules/btmon/__init__.py000066400000000000000000000014271303450110500175250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import BtmonModule __all__ = ['BtmonModule'] weboob-1.2/modules/btmon/browser.py000066400000000000000000000031341303450110500174460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from .pages import TorrentsPage, TorrentPage __all__ = ['BtmonBrowser'] class BtmonBrowser(Browser): DOMAIN = 'www.btmon.com' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://www.btmon.com/torrent/\?f=.*': TorrentsPage, 'http://www.btmon.com/.*torrent.html': TorrentPage, } def iter_torrents(self, pattern): self.location('http://www.btmon.com/torrent/?f=%s' % pattern.encode('utf-8')) assert self.is_on_page(TorrentsPage) return self.page.iter_torrents() def get_torrent(self, id): try: self.location('http://www.btmon.com/%s.html' % id) except BrowserHTTPNotFound: return if self.is_on_page(TorrentPage): return self.page.get_torrent() weboob-1.2/modules/btmon/favicon.png000066400000000000000000000033571303450110500175530ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME(DoIDATxmlEF (TVVʛz lP17CH4j_ 5DQ 'mH9ʁDm,(PiCgw[hO.7gY#F1bG1$@ 0 *T{MV:Mwv_tjxzIYƫ߃D`Ni27uߺa ƵF@14zT7)S8FOh2ƙDJV:UUiF}N `K&\xyڜn 5ؒ"ئd2ttw%jXg /V:JXH xSl 0Xj/6Zy|JO՞NZ1bĈ#FAdj9J'msXrPD~=EV l'8fS–t>as 6c=kv+w=-` (0Z[!`9fiQ_2ZD9TT@ ,>p=:ċt=gTgɒ_c\zŚLՀۥ{?((njq~dbkZVIO ahFrde)a˝pQO1?POBZJuؗSaϔC9rZ@Oڨ3:sELTFrs M–诵ePQ/eN ^Q!(o2nU6_җ#^t`Af*/t:r}`.g;Cu2a Wm&рj9F򻁕*U7S6s%7n0Cu ea B–geYu[-*p/I[ &`V!ly8!l&6F{J(a<D7 a#–5Mq@먱 \\[.QKF;!. from weboob.capabilities.torrent import CapTorrent, Torrent from weboob.tools.backend import Module from .browser import BtmonBrowser from urllib import quote_plus __all__ = ['BtmonModule'] class BtmonModule(Module, CapTorrent): NAME = 'btmon' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'BTMon BitTorrent database' LICENSE = 'AGPLv3+' BROWSER = BtmonBrowser def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None return self.browser.openurl(torrent.url.encode('utf-8')).read() def iter_torrents(self, pattern): return self.browser.iter_torrents(quote_plus(pattern.encode('utf-8'))) def fill_torrent(self, torrent, fields): if 'description' in fields: tor = self.get_torrent(torrent.id) torrent.description = tor.description torrent.magnet = tor.magnet torrent.files = tor.files torrent.url = tor.url return torrent OBJECTS = { Torrent:fill_torrent } weboob-1.2/modules/btmon/pages.py000066400000000000000000000107451303450110500170700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import string from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.deprecated.browser import Page from weboob.tools.misc import get_bytes_size class TorrentsPage(Page): def iter_torrents(self): for div in self.parser.select(self.document.getroot(),'div.list_tor'): name = NotAvailable size = NotAvailable seeders = NotAvailable leechers = NotAvailable right_div = self.parser.select(div,'div.list_tor_right',1) try: seeders = int(self.parser.select(right_div,'b.green',1).text) except ValueError: seeders = 0 try: leechers = int(self.parser.select(right_div,'b.red',1).text) except ValueError: leechers = 0 sizep = self.parser.select(right_div,'p')[0] sizespan = self.parser.select(sizep,'span')[0] nsize = float(sizespan.text_content().split(':')[1].split()[0]) usize = sizespan.text_content().split()[-1].upper() size = get_bytes_size(nsize,usize) a = self.parser.select(div,'a.list_tor_title',1) href = a.attrib.get('href','') name = unicode(a.text_content()) id = unicode(href.strip('/').split('.html')[0]) torrent = Torrent(id,name) torrent.url = NotLoaded torrent.filename = id torrent.magnet = NotLoaded torrent.size = size torrent.seeders = seeders torrent.leechers = leechers torrent.description = NotLoaded torrent.files = NotLoaded yield torrent class TorrentPage(Page): def get_torrent(self): seed = 0 leech = 0 description = NotAvailable url = NotAvailable magnet = NotAvailable title = NotAvailable id = unicode(self.browser.geturl().split('.html')[0].split('/')[-1]) div = self.parser.select(self.document.getroot(),'div#middle_content',1) title = u'%s'%self.parser.select(self.document.getroot(),'div#middle_content > h1',1).text slblock_values = self.parser.select(div,'div.sl_block b') if len(slblock_values) >= 2: seed = slblock_values[0].text leech = slblock_values[1].text href_t = self.parser.select(div,'a.down',1).attrib.get('href','') url = u'http://%s%s'%(self.browser.DOMAIN,href_t) magnet = unicode(self.parser.select(div,'a.magnet',1).attrib.get('href','')) divtabs = self.parser.select(div,'div#tabs',1) files_div = self.parser.select(divtabs,'div.body > div.doubleblock > div.leftblock') files = [] if len(files_div) > 0: size_text = self.parser.select(files_div,'h5',1).text for b in self.parser.select(files_div,'b'): div = b.getparent() files.append(div.text_content()) else: size_text = self.parser.select(divtabs,'h5',1).text_content() size_text = size_text.split('(')[1].split(')')[0].strip() size = float(size_text.split(',')[1].strip(string.letters)) u = size_text.split(',')[1].strip().translate(None,string.digits).strip('.').strip().upper() div_desc = self.parser.select(divtabs,'div#descriptionContent') if len(div_desc) > 0: description = unicode(div_desc[0].text_content()) torrent = Torrent(id, title) torrent.url = url torrent.filename = id torrent.magnet = magnet torrent.size = get_bytes_size(size, u) torrent.seeders = int(seed) torrent.leechers = int(leech) torrent.description = description torrent.files = files return torrent weboob-1.2/modules/btmon/test.py000066400000000000000000000026041303450110500167430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded from random import choice class BtmonTest(BackendTest): MODULE = 'btmon' def test_torrent(self): torrents = list(self.backend.iter_torrents('spiderman')) for torrent in torrents: assert torrent.id assert torrent.name assert torrent.description is NotLoaded assert torrent.files is NotLoaded # get the file of a random torrent # from the list (getting them all would be too long) if len(torrents): torrent = choice(torrents) self.backend.get_torrent_file(torrent.id) weboob-1.2/modules/caels/000077500000000000000000000000001303450110500153605ustar00rootroot00000000000000weboob-1.2/modules/caels/__init__.py000066400000000000000000000014361303450110500174750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CaelsModule __all__ = ['CaelsModule'] weboob-1.2/modules/caels/browser.py000066400000000000000000000015741303450110500174240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import AbstractBrowser class CAELSBrowser(AbstractBrowser): PARENT = 'amundi' PARENT_ATTR = 'package.browser.AmundiBrowser' weboob-1.2/modules/caels/favicon.png000066400000000000000000000044211303450110500175140ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME  _iTXtCommentCreated with GIMPd.ebKGDuIDATx lUEB[h B7-KklP +c@"iX{.챂*(b@ETYEAքU e !^瑛]f֔/w33g9s <SZ0Ђ~'Кy`% Tn Q7A}o`n!\=Ax6 TĻƃwH^Ղ]}(j!\=&+>xܒq֗M8.!"-f1l4@a! W%RPu׃>^f r\ I>!EY $F_]7]eG?WB*7G=SQ(;Э]K'FJtA6&leALgeq 17XEj2vttS-n?w~l~ y OSƪ)jDG9[Ab5@0ZA;g"mTqZ,詪ExuN\xc7E x|6R|xڊ{4AS1ڃ3KeVyFoo r&o.Tod x եKx W_B_vZ"j>;X'O]bI ~!Gi4۱4MSAO4P&vE??5NhOՙ|كOo'Ԅfo8_k@6D`ALx `&Q\ ΃s3|M]]|Zc&z\u yNJaQĨ͐HRY.r1l8-ҾdaRvcٻ8kbyQ׳ o g@ЏyKz8NHS||9X V8ݥ[6L=S. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import CAELSBrowser __all__ = ['CaelsModule'] class CaelsModule(Module, CapBank): NAME = 'caels' DESCRIPTION = u'Crédit Agricole - Epargne Longue des Salariés' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig( ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = CAELSBrowser def create_default_browser(self): return self.create_browser(self.weboob, "https://www.ca-els.com/", self.config['login'].get(), self.config['password'].get()) def get_account(self, id): return find_object(self.iter_accounts(), id=id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_investment(self, account): return self.browser.iter_investments(account) def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/caels/test.py000066400000000000000000000015011303450110500167060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CaelsTest(BackendTest): MODULE = 'caels' weboob-1.2/modules/caissedepargne/000077500000000000000000000000001303450110500172465ustar00rootroot00000000000000weboob-1.2/modules/caissedepargne/__init__.py000066400000000000000000000014461303450110500213640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CaisseEpargneModule __all__ = ['CaisseEpargneModule'] weboob-1.2/modules/caissedepargne/browser.py000066400000000000000000000354461303450110500213170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import json import urlparse from datetime import datetime from dateutil.relativedelta import relativedelta from weboob.browser import LoginBrowser, need_login from weboob.browser.url import URL from weboob.capabilities.bank import Account from weboob.browser.exceptions import BrowserHTTPNotFound, ClientError from weboob.exceptions import BrowserIncorrectPassword from .pages import IndexPage, ErrorPage, MarketPage, LifeInsurance, GarbagePage, \ MessagePage, LoginPage, CenetLoginPage, CenetHomePage, \ CenetAccountsPage, CenetAccountHistoryPage, CenetCardsPage __all__ = ['CaisseEpargne'] class CaisseEpargne(LoginBrowser): BASEURL = "https://www.caisse-epargne.fr" login = URL('/authentification/manage\?step=identification&identifiant=(?P.*)', 'https://.*/login.aspx', LoginPage) account_login = URL('/authentification/manage\?step=account&identifiant=(?P.*)&account=(?P.*)', LoginPage) cenet_login = URL('https://www.cenet.caisse-epargne.fr/$', CenetLoginPage) cenet_home = URL('https://www.cenet.caisse-epargne.fr/Default.aspx$', CenetHomePage) cenet_accounts = URL('https://www.cenet.caisse-epargne.fr/Web/Api/ApiComptes.asmx/ChargerSyntheseComptes', CenetAccountsPage) cenet_account_history = URL('https://www.cenet.caisse-epargne.fr/Web/Api/ApiComptes.asmx/RechercherOperations', CenetAccountHistoryPage) cenet_account_coming = URL('https://www.cenet.caisse-epargne.fr/Web/Api/ApiCartesBanquaires.asmx/ChargerEnCoursCarte', CenetAccountHistoryPage) cenet_cards = URL('https://www.cenet.caisse-epargne.fr/Web/Api/ApiCartesBanquaires.asmx/ChargerCartes', CenetCardsPage) home = URL('https://.*/Portail.aspx.*', IndexPage) home_tache = URL('https://.*/Portail.aspx\?tache=(?P).*', IndexPage) error = URL('https://.*/login.aspx', 'https://.*/Pages/logout.aspx.*', 'https://.*/particuliers/Page_erreur_technique.aspx.*', ErrorPage) market = URL('https://.*/Pages/Bourse.*', 'https://www.caisse-epargne.offrebourse.com/ReroutageSJR', 'https://www.caisse-epargne.offrebourse.com/Portefeuille.*', MarketPage) life_insurance = URL('https://.*/Assurance/Pages/Assurance.aspx', 'https://www.extranet2.caisse-epargne.fr.*', LifeInsurance) message = URL('https://www.caisse-epargne.offrebourse.com/DetailMessage\?refresh=O', MessagePage) garbage = URL('https://www.caisse-epargne.offrebourse.com/Portefeuille', 'https://www.caisse-epargne.fr/particuliers/.*/emprunter.aspx', 'https://.*/particuliers/emprunter.*', 'https://.*/particuliers/epargner.*', GarbagePage) def __init__(self, nuser, *args, **kwargs): self.BASEURL = kwargs.pop('domain', self.BASEURL) if not self.BASEURL.startswith('https://'): self.BASEURL = 'https://%s' % self.BASEURL self.is_cenet_website = False self.multi_type = False self.typeAccount = 'WE' self.nuser = nuser super(CaisseEpargne, self).__init__(*args, **kwargs) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) # Reset domain to log on pro website if first login attempt failed on personal website. if self.multi_type: self.BASEURL = 'https://www.caisse-epargne.fr' self.typeAccount = 'WP' data = self.login.go(login=self.username).get_response() if data is None: raise BrowserIncorrectPassword() if "authMode" in data and data['authMode'] == 'redirect': self.is_cenet_website = True post_data = { 'CodeEtablissement': data['codeCaisse'], 'NumeroBad': self.username, 'NumeroUtilisateur': self.nuser } self.location(data['url'], data=post_data, headers={'Referer': 'https://www.cenet.caisse-epargne.fr/'}) return self.page.login(self.username, self.password, self.nuser, data['codeCaisse']) if len(data['account']) > 1: self.multi_type = True data = self.account_login.go(login=self.username, accountType=self.typeAccount).get_response() assert data is not None typeAccount = data['account'][0] playload = { 'auth_mode': 'ajax', 'nuusager': self.nuser.encode('utf-8'), 'codconf': self.password, 'typeAccount': typeAccount, 'step': 'authentification', 'nuabbd': self.username } response = self.location(data['url'], params=playload).page.get_response() assert response is not None if not response['action']: if not self.typeAccount == 'WP' and self.multi_type: # If we haven't test PRO espace we check before raising wrong pass self.do_login() return raise BrowserIncorrectPassword(response['error']) self.BASEURL = urlparse.urljoin(data['url'], '/') try: self.home.go() except BrowserHTTPNotFound: raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): # cenet website if self.is_cenet_website is True: headers = { 'Content-Type': 'application/json; charset=UTF-8', 'Accept': 'application/json, text/javascript, */*; q=0.01' } data = { 'contexte': '', 'dateEntree': None, 'donneesEntree': 'null', 'filtreEntree': None } try: accounts = [account for account in self.cenet_accounts.go(data=json.dumps(data), headers=headers).get_accounts()] except ClientError: # Unauthorized due to wrongpass raise BrowserIncorrectPassword() for account in accounts: account._cards = [card for card in self.cenet_cards.go(data=json.dumps(data), headers=headers).get_cards() \ if card['Compte']['Numero'] == account.id] return accounts if self.home.is_here(): self.page.check_no_accounts() self.page.go_list() else: self.home.go() accounts = list(self.page.get_list()) for account in accounts: if account.type == Account.TYPE_MARKET: if not self.home.is_here(): self.home_tache.go(tache='CPTSYNT0') self.page.go_history(account._info) if self.message.is_here(): self.page.submit() self.page.go_history(account._info) # Some users may not have access to this. if not self.market.is_here(): continue self.page.submit() if self.page.is_error(): continue self.garbage.go() if self.garbage.is_here(): continue self.page.get_valuation_diff(account) return iter(accounts) @need_login def get_loans_list(self): if self.is_cenet_website is True: return iter([]) if self.home.is_here(): if self.page.check_no_accounts(): return iter([]) self.home_tache.go(tache='CRESYNT0') loan_accounts = list() if self.home.is_here(): self.page.go_loan_list() loan_accounts = list(self.page.get_loan_list()) for _ in range(3): try: self.home_tache.go(tache='CPTSYNT0') if self.home.is_here(): self.page.go_list() except ClientError: pass else: break return (loan_accounts) @need_login def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == id: return a return None @need_login def _get_history(self, info): if isinstance(info['link'], list): info['link'] = info['link'][0] if not info['link'].startswith('HISTORIQUE'): return if self.home.is_here(): self.page.go_list() else: self.home_tache.go(tache='CPTSYNT0') self.page.go_history(info) info['link'] = [info['link']] if info['type'] == "HISTORIQUE_CB": info['link'] += self.page.get_cbtabs() while True: for i, link in enumerate(info['link'], 1): if i > 1: info['link'] = link self.page.go_history(info, True) assert self.home.is_here() for tr in self.page.get_history(): yield tr if not self.page.go_next(): return @need_login def _get_history_invests(self, account): if self.home.is_here(): self.page.go_list() else: self.home.go() self.page.go_history(account._info) try: self.page.go_life_insurance(account) if self.market.is_here() is False and self.message.is_here() is False: return iter([]) self.page.submit() self.location('https://www.extranet2.caisse-epargne.fr%s' % self.page.get_cons_histo()) except (IndexError, AttributeError) as e: self.logger.error(e) return iter([]) return self.page.iter_history() @need_login def get_history(self, account): if self.is_cenet_website is True: # cenet website headers = { 'Content-Type': 'application/json; charset=UTF-8', 'Accept': 'application/json, text/javascript, */*; q=0.01' } now = datetime.now() dformat = '%Y-%m-%d' for i in reversed(range(1, 12)): data = { 'contexte': '', 'dateEntree': None, 'donneesEntree': json.dumps({ 'TypeRecherche': 'FD', \ 'DateFin': now.strftime(dformat) + 'T23:00:00.000Z', 'DateDebut': (now - relativedelta(months=i)).strftime(dformat) + 'T23:00:00.000Z', 'compte': account._formated }), 'filtreEntree': None } history = self.cenet_account_history.go(data=json.dumps(data), headers=headers).get_history() if next(history, None) is not None: return history return iter([]) if not hasattr(account, '_info'): raise NotImplementedError if account.type is Account.TYPE_LIFE_INSURANCE: return self._get_history_invests(account) return self._get_history(account._info) @need_login def get_coming(self, account): trs = [] if self.is_cenet_website is True: # cenet website headers = { 'Content-Type': 'application/json; charset=UTF-8', 'Accept': 'application/json, text/javascript, */*; q=0.01' } for card in account._cards: data = { 'contexte': '', 'dateEntree': None, 'donneesEntree': json.dumps(card), 'filtreEntree': None } for tr in self.cenet_account_coming.go(data=json.dumps(data), headers=headers).get_history(): trs.append(tr) else: if not hasattr(account, '_info'): raise NotImplementedError() for info in account._card_links: for tr in self._get_history(info.copy()): tr.type = tr.TYPE_DEFERRED_CARD tr.nopurge = True trs.append(tr) return iter(sorted(trs, key=lambda t: t.rdate, reverse=True)) @need_login def get_investment(self, account): if self.is_cenet_website is True: # not available for the moment return iter([]) if account.type is not Account.TYPE_LIFE_INSURANCE and account.type is not Account.TYPE_MARKET: raise NotImplementedError() if self.home.is_here(): self.page.go_list() else: self.home.go() self.page.go_history(account._info) if account.type is Account.TYPE_MARKET: # Some users may not have access to this. if not self.market.is_here(): return iter([]) self.page.submit() if self.page.is_error(): return iter([]) self.location('https://www.caisse-epargne.offrebourse.com/Portefeuille') if self.message.is_here(): return iter([]) if not self.page.is_on_right_portfolio(account): self.location('https://www.caisse-epargne.offrebourse.com/Portefeuille?compte=%s' % self.page.get_compte(account)) elif account.type is Account.TYPE_LIFE_INSURANCE: try: self.page.go_life_insurance(account) if self.market.is_here() is False and self.message.is_here() is False: return iter([]) self.page.submit() self.location('https://www.extranet2.caisse-epargne.fr%s' % self.page.get_cons_repart()) except (IndexError, AttributeError) as e: self.logger.error(e) return iter([]) if self.garbage.is_here(): return iter([]) return self.page.iter_investment() @need_login def get_advisor(self): if not self.is_cenet_website: raise NotImplementedError() return iter([self.cenet_home.stay_or_go().get_advisor()]) weboob-1.2/modules/caissedepargne/favicon.png000066400000000000000000000101161303450110500214000ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME  -2GIDATxyxe_UG PG\FmqFqTzqq"U@;2(]qAP\ k $$Nҝ^4i {Tn#s0%+WH ~P@/(5hW'yGFPBnhĬ&'?+.QTDK/&qTVb,BJJMA"oKr*  ?#;f QJ)T:槟u\eXfpڄI)2@xȻ7vXIyrm>3m($~>G884+#ŗA_jmv}9gMJCn _+sb4b'iNGK'R|eHBSwmP onKvs@;ww HM&.e8K1JKN2zp~?RZ?_D͓}4v1fp=p8Bd*E{bb[TaQYNw74 ^"+FEIՓ\tc&%Մ_ښk"pWe% ]Z!ǃQRp:fRv8AC펱,i|8++== QsE_ Jeq#8JKmIT. bk']{!d*77\CQG@b~~{@_h>*\RMTN(-EIIrݤ֯d!xd sϥ;Ѽ^R멿ӞA\h~?FQ!z("$eY^*ﻃg4>g" 2|@E^(Dz˹a7-{ʮ AF{?5h *Da#XfFݰc({ °YpH>}N/0W!^`lxK4@Ds8zo&N v\X6V|xCIэk.7省gg쳅$kjwy;g$ngU8 =c7hǓs QTHŤ6r{|V:'Z<~4N'_J䛩`2.'l5I+ !ۢX(*ayQD6T(+5d z @l"hw0_^XhHb-$ ž\{9}i2 w?o/p5Dl"}%mv}kp`YdBIg}7dna=e@~=H74bY p KWA4 p7}_ooL/BOLǙc㡻IQs%ߘi3P9s56ZBsPpXe -OLPF+Ԍ u*n'kГ3NT)wYTD|<"rJGYwBjZ⋗*2կ7}Fdbg=):< G=*f3HiC p!i'@3lTIÄxF#xSρd9ATc_C]轊(|3'gOZ;cL]T?5'!Z0:=!h}Ӧ#Nq=`ـ;Ȇ@)w:So Ruu.혞ʎT eY܏ pPǔ2xC%i9+3@%%TLGMy-hӰAP~ %NX^el_}M`55h~orFc/g>X8- !{t z|,BӞjn:4^^CPR6cnܤ>}0J0M?CYKд˪\198*hcp25 N'BhȄ49QUI韮3|>+O?Dw)})`vܔB:]NJDUfIC]Ϟ{:*ᾇ1n۷7c7nh:3'CvYt p8>} Ճ|Jnϰ}B\[K<bb nf1&=h*$*r䬪IV,FÝ_&G!b۽Q>׋IլprvcoEɵW?Ҳh~aYY/k~q1uވ&bMVnth~Sqa.n3kuO C3PG=(:g(p8H֬%s>B> LӶm9Or*4M{~ tioǍA/(@=ﭺ;|*̀^享qb.[NlWhkCn@`V/%)oed4 Ac(|a\srBx+/Q^f%H#8OvkTVoL{wtwp*ge*'0.Wz~2E矃k4eIW;$)e(U@)w4{Obx4/ٯ/HIh|1GN[5c4Jh|5t*[s6cJ ­Κ4Me_> =yA<آu*2E {}]ǬWYY;J̕!me}0h}u.E{HO=آi㷧^,J]kfW9~DwTwl07##8*IZ͊kp' Y~H74fu}iJ"Gkv&Dy]OYUIŔ[7o5?4 Θ5>])QfڡM4 #F;M߾{f$|K77m;rM-L{.w@YÚ.c\72S5k2:-yklkkaEV7aQ{i"IT,Je/J 㶧X-9;b8U܃w~eaVk+s'M$p"66EKR);$S(K"c1icdtss[S׽^ʒD/' kS١e@hߧH[g/P Qt6m,T:m붜֮(oɼbc#:{OeXyʸ {ݝrUe SJW`.[AW5H}F)мn4z|h0@yڝf Gя_daNZgBq2q#wwx2w pkb6)PrIP'G;v;FvX,m(ˈЏ?;'~O.IENDB`weboob-1.2/modules/caissedepargne/module.py000066400000000000000000000057641303450110500211210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.contact import CapContact from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.ordereddict import OrderedDict from .browser import CaisseEpargne __all__ = ['CaisseEpargneModule'] class CaisseEpargneModule(Module, CapBank, CapContact): NAME = 'caissedepargne' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Caisse d\'Épargne' LICENSE = 'AGPLv3+' BROWSER = CaisseEpargne website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'www.caisse-epargne.fr': u'Caisse d\'Épargne', 'www.banquebcp.fr': u'Banque BCP', }.iteritems(), key=lambda k_v: (k_v[1], k_v[0]))]) CONFIG = BackendConfig(Value('website', label='Banque', choices=website_choices, default='www.caisse-epargne.fr'), ValueBackendPassword('login', label='Identifiant client', masked=False), ValueBackendPassword('password', label='Code personnel', regexp='\d+'), Value('nuser', label='User ID (optional)', default='')) def create_default_browser(self): return self.create_browser(nuser=self.config['nuser'].get(), username=self.config['login'].get(), password=self.config['password'].get(), domain=self.config['website'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account for account in self.browser.get_loans_list(): yield account def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_coming(account) def iter_investment(self, account): return self.browser.get_investment(account) def iter_contacts(self): return self.browser.get_advisor() weboob-1.2/modules/caissedepargne/pages.py000066400000000000000000000642441303450110500207310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import json from decimal import Decimal from weboob.browser.pages import LoggedPage, HTMLPage, JsonPage from weboob.browser.elements import DictElement, ItemElement, method from weboob.browser.filters.standard import Date, CleanDecimal, Regexp, CleanText, Eval, Format from weboob.browser.filters.html import Link from weboob.browser.filters.json import Dict from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account, Transaction, Investment from weboob.capabilities.contact import Advisor from weboob.tools.ordereddict import OrderedDict from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.capabilities.bank.iban import is_rib_valid, rib2iban from weboob.exceptions import NoAccountsException, BrowserUnavailable class LoginPage(JsonPage): def get_response(self): return self.doc class CenetLoginPage(HTMLPage): def login(self, username, password, nuser, codeCaisse): form = self.get_form(id='aspnetForm') form['__EVENTTARGET'] = "btn_authentifier" form['__EVENTARGUMENT'] = '{"CodeCaisse":"%s","NumeroBad":"%s","NumeroUsager":"%s",\ "MotDePasse":"%s"}' % (codeCaisse, username, nuser, password) form.submit() class CenetHomePage(HTMLPage): @method class get_advisor(ItemElement): klass = Advisor obj_name = CleanText('//section[contains(@id, "ChargeAffaires")]//strong') obj_email = CleanText('//li[contains(@id, "MailContact")]') obj_phone = CleanText('//li[contains(@id, "TelAgence")]', replace=[('.', '')]) obj_mobile = NotAvailable obj_agency = CleanText('//section[contains(@id, "Agence")]//strong') obj_address = CleanText('//li[contains(@id, "AdresseAgence")]') def obj_fax(self): return CleanText('//li[contains(@id, "FaxAgence")]', replace=[('.', '')])(self) or NotAvailable class CenetJsonPage(JsonPage): def __init__(self, browser, response, *args, **kwargs): super(CenetJsonPage, self).__init__(browser, response, *args, **kwargs) # Why you are so ugly.... self.doc = json.loads(Dict('d')(self.doc)) self.doc['DonneesSortie'] = json.loads(Dict('DonneesSortie')(self.doc)) class CenetAccountsPage(LoggedPage, CenetJsonPage): ACCOUNT_TYPES = {u'CCP': Account.TYPE_CHECKING} @method class get_accounts(DictElement): item_xpath = "DonneesSortie" class item(ItemElement): klass = Account obj_id = CleanText(Dict('Numero')) obj_label = CleanText(Dict('Intitule')) obj_iban = CleanText(Dict('IBAN')) def obj_balance(self): return Eval(lambda x: x / 10**2, CleanDecimal(Dict('Solde/Valeur')))(self) def obj_currency(self): return CleanText(Dict('Devise'))(self).upper() def obj_type(self): return self.page.ACCOUNT_TYPES.get(Dict('TypeCompte')(self), Account.TYPE_UNKNOWN) def obj__formated(self): return self.el class CenetCardsPage(LoggedPage, CenetJsonPage): def get_cards(self): cards = Dict('DonneesSortie')(self.doc) # Remove dates to prevent bad parsing def reword_dates(card): tmp_card = card for k, v in tmp_card.iteritems(): if isinstance(v, dict): v = reword_dates(v) if k == "Date" and v is not None and "Date" in v: card[k] = None for card in cards: reword_dates(card) return cards class CenetAccountHistoryPage(LoggedPage, CenetJsonPage): TR_TYPES = {8: Transaction.TYPE_TRANSFER, # VIR 7: Transaction.TYPE_TRANSFER, # VIR COMPTE A COMPTE 6: Transaction.TYPE_CASH_DEPOSIT, # REMISE CHECQUE(s) 4: Transaction.TYPE_ORDER # PRELV } @method class get_history(DictElement): item_xpath = "DonneesSortie" class item(ItemElement): klass = Transaction obj_raw = Format('%s %s', Dict('Libelle'), Dict('Libelle2')) obj_label = CleanText(Dict('Libelle')) obj_date = Date(Dict('DateGroupImputation')) obj_rdate = Date(Dict('DateGroupReglement')) def obj_type(self): return self.page.TR_TYPES.get(Dict('TypeMouvement')(self), Transaction.TYPE_UNKNOWN) def obj_original_currency(self): return CleanText(Dict('Montant/Devise'))(self).upper() def obj_amount(self): amount = Eval(lambda x: x / 10**2, CleanDecimal(Dict('Montant/Valeur')))(self) return -amount if Dict('Montant/CodeSens')(self) == "D" else amount class GarbagePage(LoggedPage, HTMLPage): def on_load(self): go_back_link = Link('//a[@class="btn"]', default=NotAvailable)(self.doc) if go_back_link is not NotAvailable: assert len(go_back_link) != 1 go_back_link = re.search('\(~deibaseurl\)(.*)$', go_back_link).group(1) self.browser.location('%s%s' % (self.browser.BASEURL, go_back_link)) class MessagePage(GarbagePage): def submit(self): form = self.get_form(name='leForm') form['signatur1'] = ['on'] form.submit() class _LogoutPage(HTMLPage): def on_load(self): raise BrowserUnavailable(CleanText('//*[@class="messErreur"]')(self.doc)) class ErrorPage(_LogoutPage): pass class UnavailablePage(HTMLPage): def on_load(self): raise BrowserUnavailable(CleanText('//div[@id="message_error_hs"]')(self.doc)) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^CB (?P.*?) FACT (?P
\d{2})(?P\d{2})(?P\d{2})', re.IGNORECASE), FrenchTransaction.TYPE_CARD), (re.compile('^RET(RAIT)? DAB (?P
\d+)-(?P\d+)-.*', re.IGNORECASE), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^RET(RAIT)? DAB (?P.*?) (?P
\d{2})(?P\d{2})(?P\d{2}) (?P\d{2})H(?P\d{2})', re.IGNORECASE), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^VIR(EMENT)?(\.PERIODIQUE)? (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_TRANSFER), (re.compile('^PRLV (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_ORDER), (re.compile('^CHEQUE.*', re.IGNORECASE), FrenchTransaction.TYPE_CHECK), (re.compile('^(CONVENTION \d+ )?COTIS(ATION)? (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile(r'^\* (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*', re.IGNORECASE), FrenchTransaction.TYPE_ORDER), (re.compile('^CB [\d\*]+ TOT DIF .*', re.IGNORECASE), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile('^CB [\d\*]+ (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_CARD), ] class IndexPage(LoggedPage, HTMLPage): ACCOUNT_TYPES = {u'Epargne liquide': Account.TYPE_SAVINGS, u'Compte Courant': Account.TYPE_CHECKING, u'Mes comptes': Account.TYPE_CHECKING, u'Mon épargne': Account.TYPE_SAVINGS, u'Mes autres comptes': Account.TYPE_SAVINGS, u'Compte Epargne et DAT': Account.TYPE_SAVINGS, u'Plan et Contrat d\'Epargne': Account.TYPE_SAVINGS, u'Titres': Account.TYPE_MARKET, u'Compte titres': Account.TYPE_MARKET, u'Mes crédits immobiliers': Account.TYPE_LOAN, u'Mes crédits renouvelables': Account.TYPE_LOAN, u'Mes crédits consommation': Account.TYPE_LOAN, } def on_load(self): # This page is sometimes an useless step to the market website. bourse_link = Link(u'//div[@id="MM_COMPTE_TITRE_pnlbourseoic"]//a[contains(text(), "Accédez à la consultation")]', default=None)(self.doc) if bourse_link: self.browser.location(bourse_link) def check_no_accounts(self): no_account_message = CleanText(u'//span[@id="MM_LblMessagePopinError"]/p[contains(text(), "Aucun compte disponible")]')(self.doc) if no_account_message: raise NoAccountsException(no_account_message) def _get_account_info(self, a): m = re.search("PostBack(Options)?\([\"'][^\"']+[\"'],\s*['\"]([HISTORIQUE_\w|SYNTHESE_ASSURANCE_CNP|BOURSE|COMPTE_TITRE][\d\w&]+)?['\"]", a.attrib.get('href', '')) if m is None: return None else: # it is in form CB&12345[&2]. the last part is only for new website # and is necessary for navigation. link = m.group(2) parts = link.split('&') info = {} if len(parts) > 1: info['type'] = parts[0] info['id'] = parts[1] else: id = re.search("([\d]+)", a.attrib.get('title')) info['type'] = link info['id'] = id.group(1) if info['type'] in ('SYNTHESE_ASSURANCE_CNP','SYNTHESE_EPARGNE'): info['acc_type'] = Account.TYPE_LIFE_INSURANCE if info['type'] in ('BOURSE', 'COMPTE_TITRE'): info['acc_type'] = Account.TYPE_MARKET info['link'] = link return info def _add_account(self, accounts, link, label, account_type, balance): info = self._get_account_info(link) if info is None: self.logger.warning('Unable to parse account %r: %r' % (label, link)) return account = Account() account.id = info['id'] if is_rib_valid(info['id']): account.iban = rib2iban(info['id']) account._info = info account.label = label account.type = info['acc_type'] if 'acc_type' in info else account_type account.balance = Decimal(FrenchTransaction.clean_amount(balance)) if balance else self.get_balance(account) account.currency = account.get_currency(balance) account._card_links = [] if account._info['type'] == 'HISTORIQUE_CB' and account.id in accounts: a = accounts[account.id] if not a.coming: a.coming = Decimal('0.0') a.coming += account.balance a._card_links.append(account._info) return accounts[account.id] = account def get_balance(self, account): if not account.type == Account.TYPE_LIFE_INSURANCE: return NotAvailable self.go_history(account._info) balance = self.doc.xpath('.//tr[td[contains(., ' + account.id + ')]]/td[contains(@class, "somme")]') if len(balance) > 0: balance = CleanText('.')(balance[0]) balance = Decimal(FrenchTransaction.clean_amount(balance)) if balance != u'' else NotAvailable else: balance = NotAvailable self.go_list() return balance def get_list(self): accounts = OrderedDict() # Old website for table in self.doc.xpath('//table[@cellpadding="1"]'): account_type = Account.TYPE_UNKNOWN for tr in table.xpath('./tr'): tds = tr.findall('td') if tr.attrib.get('class', '') == 'DataGridHeader': account_type = self.ACCOUNT_TYPES.get(tds[1].text.strip()) or\ self.ACCOUNT_TYPES.get(CleanText('.')(tds[2])) or\ self.ACCOUNT_TYPES.get(CleanText('.')(tds[3]), Account.TYPE_UNKNOWN) else: # On the same row, there are many accounts (for example a # check accound and a card one). if len(tds) > 4: for i, a in enumerate(tds[2].xpath('./a')): label = CleanText('.')(a) balance = CleanText('.')(tds[-2].xpath('./a')[i]) self._add_account(accounts, a, label, account_type, balance) # Only 4 tds on banque de la reunion website. elif len(tds) == 4: for i, a in enumerate(tds[1].xpath('./a')): label = CleanText('.')(a) balance = CleanText('.')(tds[-1].xpath('./a')[i]) self._add_account(accounts, a, label, account_type, balance) if len(accounts) == 0: # New website for table in self.doc.xpath('//div[@class="panel"]'): title = table.getprevious() if title is None: continue account_type = self.ACCOUNT_TYPES.get(CleanText('.')(title), Account.TYPE_UNKNOWN) for tr in table.xpath('.//tr'): tds = tr.findall('td') for i in xrange(len(tds)): a = tds[i].find('a') if a is not None: break if a is None: continue label = CleanText('.')(tds[0]) balance = CleanText('.')(tds[-1]) self._add_account(accounts, a, label, account_type, balance) return accounts.itervalues() def get_loan_list(self): accounts = OrderedDict() # New website for table in self.doc.xpath('//div[@class="panel"]'): title = table.getprevious() if title is None: continue account_type = self.ACCOUNT_TYPES.get(CleanText('.')(title), Account.TYPE_UNKNOWN) for tr in table.xpath('./table/tbody/tr[contains(@id,"MM_SYNTHESE_CREDITS") and contains(@id,"IdTrGlobal")]'): tds = tr.findall('td') if len(tds) == 0 : continue for i in tds[0].xpath('.//a/strong'): label = i.text.strip() break balance = Decimal(FrenchTransaction.clean_amount(CleanText('.')(tds[-1]))) account = Account() account.id = label.split(' ')[-1] account.label = unicode(label) account.type = account_type account.balance = -abs(balance) account.currency = account.get_currency(CleanText('.')(tds[-1])) account._card_links = [] accounts[account.id] = account return accounts.itervalues() def go_list(self): form = self.get_form(name='main') form['__EVENTARGUMENT'] = "CPTSYNT0" if "MM$m_CH$IsMsgInit" in form: # Old website form['__EVENTTARGET'] = "Menu_AJAX" form['m_ScriptManager'] = "m_ScriptManager|Menu_AJAX" else: # New website form['__EVENTTARGET'] = "MM$m_PostBack" form['m_ScriptManager'] = "MM$m_UpdatePanel|MM$m_PostBack" for name in ['MM$HISTORIQUE_COMPTE$btnCumul','Cartridge$imgbtnMessagerie','MM$m_CH$ButtonImageFondMessagerie',\ 'MM$m_CH$ButtonImageMessagerie']: try: del form[name] except KeyError: pass form.submit() def go_loan_list(self): form = self.get_form(name='main') form['__EVENTARGUMENT'] = "CRESYNT0" if "MM$m_CH$IsMsgInit" in form: # Old website pass else: # New website form['__EVENTTARGET'] = "MM$m_PostBack" form['m_ScriptManager'] = "MM$m_UpdatePanel|MM$m_PostBack" for name in ['MM$HISTORIQUE_COMPTE$btnCumul','Cartridge$imgbtnMessagerie','MM$m_CH$ButtonImageFondMessagerie',\ 'MM$m_CH$ButtonImageMessagerie']: try: del form[name] except KeyError: pass form.submit() def go_history(self, info, is_cbtab=False): form = self.get_form(name='main') form['__EVENTTARGET'] = 'MM$%s' % (info['type'] if is_cbtab else 'SYNTHESE') form['__EVENTARGUMENT'] = info['link'] if "MM$m_CH$IsMsgInit" in form and form['MM$m_CH$IsMsgInit'] == "0": form['m_ScriptManager'] = "MM$m_UpdatePanel|MM$SYNTHESE" for name in ['MM$HISTORIQUE_COMPTE$btnCumul','Cartridge$imgbtnMessagerie','MM$m_CH$ButtonImageFondMessagerie',\ 'MM$m_CH$ButtonImageMessagerie']: try: del form[name] except KeyError: pass form.submit() def get_history(self): i = 0 ignore = False for tr in self.doc.xpath('//table[@cellpadding="1"]/tr') + self.doc.xpath('//tr[@class="rowClick" or @class="rowHover"]'): tds = tr.findall('td') if len(tds) < 4: continue # if there are more than 4 columns, ignore the first one. i = min(len(tds) - 4, 1) if tr.attrib.get('class', '') == 'DataGridHeader': if tds[2].text == u'Titulaire': ignore = True else: ignore = False continue if ignore: continue # Remove useless details detail = tr.cssselect('div.detail') if len(detail) > 0: detail[0].drop_tree() t = Transaction() date = u''.join([txt.strip() for txt in tds[i+0].itertext()]) raw = u' '.join([txt.strip() for txt in tds[i+1].itertext()]) debit = u''.join([txt.strip() for txt in tds[-2].itertext()]) credit = u''.join([txt.strip() for txt in tds[-1].itertext()]) t.parse(date, re.sub(r'[ ]+', ' ', raw)) card_debit_date = self.doc.xpath(u'//span[@id="MM_HISTORIQUE_CB_m_TableTitle3_lblTitle"] | //label[contains(text(), "débiter le")]') if card_debit_date: t.rdate = Date(dayfirst=True).filter(date) m = re.search('(\d{2}\/\d{2}\/\d{4})', card_debit_date[0].text) assert m t.date = Date(dayfirst=True).filter(m.group(1)) if t.date is NotAvailable: continue if 'tot dif' in t.raw.lower(): t.deleted = True t.set_amount(credit, debit) yield t i += 1 def get_cbtabs(self): cbtabs = [] for href in self.doc.xpath('//ul[@class="onglets"]/li/a[contains(text(), "CB")]/@href'): m = re.search('(DETAIL[^"]+)', href) if m: cbtabs.append(m.group(1)) return cbtabs def go_next(self): # link = self.doc.xpath('//a[contains(@id, "lnkSuivante")]') if len(link) == 0 or 'disabled' in link[0].attrib: return False account_type = 'COMPTE' m = re.search('HISTORIQUE_(\w+)', link[0].attrib['href']) if m: account_type = m.group(1) form = self.get_form(name='main') form['__EVENTTARGET'] = "MM$HISTORIQUE_%s$lnkSuivante" % account_type form['__EVENTARGUMENT'] = '' if "MM$m_CH$IsMsgInit" in form and form['MM$m_CH$IsMsgInit'] == "N": form['m_ScriptManager'] = "MM$m_UpdatePanel|MM$HISTORIQUE_COMPTE$lnkSuivante" for name in ['MM$HISTORIQUE_COMPTE$btnCumul','Cartridge$imgbtnMessagerie','MM$m_CH$ButtonImageFondMessagerie',\ 'MM$m_CH$ButtonImageMessagerie']: try: del form[name] except KeyError: pass form.submit() return True def go_life_insurance(self, account): link = self.doc.xpath('//tr[td[contains(., ' + account.id + ') ]]//a')[0] m = re.search("PostBackOptions?\([\"']([^\"']+)[\"'],\s*['\"](REDIR_ASS_VIE[\d\w&]+)?['\"]", link.attrib.get('href', '')) if m is not None: form = self.get_form(name='main') form['__EVENTTARGET'] = m.group(1) form['__EVENTARGUMENT'] = m.group(2) if "MM$m_CH$IsMsgInit" not in form: # Not available on new website pass form['MM$m_CH$IsMsgInit'] = "0" form['m_ScriptManager'] = "MM$m_UpdatePanel|MM$SYNTHESE" for name in ['MM$HISTORIQUE_COMPTE$btnCumul','Cartridge$imgbtnMessagerie','MM$m_CH$ButtonImageFondMessagerie',\ 'MM$m_CH$ButtonImageMessagerie']: try: del form[name] except KeyError: pass form.submit() class MarketPage(LoggedPage, HTMLPage): def is_error(self): try: return self.doc.xpath('//caption')[0].text == "Erreur" except IndexError: return False except AssertionError: return True def parse_decimal(self, td): value = CleanText('.')(td) if value and value != '-': return Decimal(FrenchTransaction.clean_amount(value)) else: return NotAvailable def submit(self): form = self.get_form(nr=0) form.submit() def iter_investment(self): for tbody in self.doc.xpath(u'//table[@summary="Contenu du portefeuille valorisé"]/tbody'): inv = Investment() inv.label = CleanText('.')(tbody.xpath('./tr[1]/td[1]/a/span')[0]) inv.code = CleanText('.')(tbody.xpath('./tr[1]/td[1]/a')[0]).split(' - ')[1] inv.quantity = self.parse_decimal(tbody.xpath('./tr[2]/td[2]')[0]) inv.unitvalue = self.parse_decimal(tbody.xpath('./tr[2]/td[3]')[0]) inv.unitprice = self.parse_decimal(tbody.xpath('./tr[2]/td[5]')[0]) inv.valuation = self.parse_decimal(tbody.xpath('./tr[2]/td[4]')[0]) inv.diff = self.parse_decimal(tbody.xpath('./tr[2]/td[7]')[0]) yield inv def get_valuation_diff(self, account): val = CleanText(self.doc.xpath(u'//td[contains(text(), "values latentes")]/following-sibling::*[1]')) account.valuation_diff = CleanDecimal(Regexp(val, '([^\(\)]+)'), replace_dots=True)(self) def is_on_right_portfolio(self, account): return len(self.doc.xpath('//form[@class="choixCompte"]//option[@selected and contains(text(), "%s")]' % account._info['id'])) def get_compte(self, account): return self.doc.xpath('//option[contains(text(), "%s")]/@value' % account._info['id'])[0] class LifeInsurance(MarketPage): def get_cons_repart(self): return self.doc.xpath('//tr[@id="sousMenuConsultation3"]/td/div/a')[0].attrib['href'] def get_cons_histo(self): return self.doc.xpath('//tr[@id="sousMenuConsultation4"]/td/div/a')[0].attrib['href'] def iter_history(self): for tr in self.doc.xpath(u'//table[@class="boursedetail"]/tbody/tr[td]'): t = Transaction() t.label = CleanText('.')(tr.xpath('./td[2]')[0]) t.date = Date(dayfirst=True).filter(CleanText('.')(tr.xpath('./td[1]')[0])) t.amount = self.parse_decimal(tr.xpath('./td[3]')[0]) yield t def iter_investment(self): for tr in self.doc.xpath(u'//table[@class="boursedetail"]/tr[@class and not(@class="total")]'): inv = Investment() libelle = CleanText('.')(tr.xpath('./td[1]')[0]).split(' ') inv.label, inv.code = self.split_label_code(libelle) diff = self.parse_decimal(tr.xpath('./td[6]')[0]) inv.quantity = self.parse_decimal(tr.xpath('./td[2]')[0]) inv.unitvalue = self.parse_decimal(tr.xpath('./td[3]')[0]) date = CleanText('.')(tr.xpath('./td[4]')[0]) inv.vdate = Date(dayfirst=True).filter(date) if date and date != '-' else NotAvailable inv.unitprice = self.calc(inv.unitvalue, diff) inv.valuation = self.parse_decimal(tr.xpath('./td[5]')[0]) inv.diff = self.get_diff(inv.valuation, self.calc(inv.valuation, diff)) yield inv def calc(self, value, diff): if value is NotAvailable or diff is NotAvailable: return NotAvailable return Decimal(value) / (1 + Decimal(diff)/100) def get_diff(self, valuation, calc): if valuation is NotAvailable or calc is NotAvailable: return NotAvailable return valuation - calc def split_label_code(self, libelle): m = re.search('FR\d+', libelle[-1]) if m: return ' '.join(libelle[:-1]), libelle[-1] else: return ' '.join(libelle), NotAvailable weboob-1.2/modules/caissedepargne/test.py000066400000000000000000000017711303450110500206050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CaisseEpargneTest(BackendTest): MODULE = 'caissedepargne' def test_caisse_epargne(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/canalplus/000077500000000000000000000000001303450110500162535ustar00rootroot00000000000000weboob-1.2/modules/canalplus/__init__.py000066400000000000000000000014441303450110500203670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CanalplusModule __all__ = ['CanalplusModule'] weboob-1.2/modules/canalplus/browser.py000066400000000000000000000076531303450110500203230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Nicolas Duhamel, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import requests import urllib import lxml.etree from weboob.deprecated.browser import Browser from weboob.deprecated.browser.decorators import id2url from .pages import ChannelsPage, VideoPage from .video import CanalplusVideo from weboob.capabilities.collection import CollectionNotFound __all__ = ['CanalplusBrowser'] class XMLParser(object): def parse(self, data, encoding=None): if encoding is None: parser = None else: parser = lxml.etree.XMLParser(encoding=encoding, strip_cdata=False) return lxml.etree.XML(data.get_data(), parser) class CanalplusBrowser(Browser): DOMAIN = u'service.canal-plus.com' ENCODING = 'utf-8' PAGES = { r'http://service.canal-plus.com/video/rest/initPlayer/cplus/': ChannelsPage, r'http://service.canal-plus.com/video/rest/search/cplus/.*': VideoPage, r'http://service.canal-plus.com/video/rest/getVideosLiees/cplus/(?P.+)': VideoPage, r'http://service.canal-plus.com/video/rest/getMEAs/cplus/.*': VideoPage, } #We need lxml.etree.XMLParser to read CDATA PARSER = XMLParser() FORMATS = { 'sd': 0, 'hd': -1, } def __init__(self, quality, *args, **kwargs): Browser.__init__(self, parser=self.PARSER, *args, **kwargs) self.quality = self.FORMATS.get(quality, self.FORMATS['hd']) def home(self): self.location('http://service.canal-plus.com/video/rest/initPlayer/cplus/') def search_videos(self, pattern): self.location('http://service.canal-plus.com/video/rest/search/cplus/' + urllib.quote_plus(pattern.replace('/', '').encode('utf-8'))) return self.page.iter_results() @id2url(CanalplusVideo.id2url) def get_video(self, url, video=None): self.location(url) video = self.page.get_video(video) video.url = u'%s' % self.read_url(video.url)[self.quality] return video def read_url(self, url): r = requests.get(url, stream=True) buf = r.iter_lines() return [line for line in buf if not line.startswith('#')] def iter_resources(self, split_path): if not self.is_on_page(ChannelsPage): self.home() channels = self.page.get_channels() if len(split_path) == 0: for channel in channels: if channel.path_level == 1: yield channel elif len(split_path) == 1: for channel in channels: if channel.path_level == 2 and split_path == channel.parent_path: yield channel elif len(split_path) == 2: subchannels = self.iter_resources(split_path[0:1]) try: channel = [subchannel for subchannel in subchannels if split_path == subchannel.split_path][0] self.location("http://service.canal-plus.com/video/rest/getMEAs/cplus/%s" % channel._link_id) assert self.is_on_page(VideoPage) for video in self.page.iter_channel(): yield video except IndexError: raise CollectionNotFound(split_path) else: raise CollectionNotFound(split_path) weboob-1.2/modules/canalplus/favicon.png000066400000000000000000000012141303450110500204040ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME  ()ttEXtCommentCreated with GIMPWIDATx1@JA )lDA/g>#R B$0o&c+Mf~*DDDDDDDDDI\@,!<y0un+ IGw^/qw b1ά{I)%͇RJe8^zkm#mO nyWZ~DQp`)n}LZVaZAqsf35`<Ǐ7 ?XJ!u\.v?Fʡ:(RnZ-uj 5uhY X.6. P@5KweY0Pa²,ezN-@Gֳ\|>N&4^3͔R 9,n}G>O\\(p\A4Me@u]W c|K읾BIENDB`weboob-1.2/modules/canalplus/module.py000066400000000000000000000050731303450110500201170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.video import CapVideo, BaseVideo from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from .browser import CanalplusBrowser from .video import CanalplusVideo from weboob.capabilities.collection import CapCollection __all__ = ['CanalplusModule'] class CanalplusModule(Module, CapVideo, CapCollection): NAME = 'canalplus' MAINTAINER = u'Nicolas Duhamel' EMAIL = 'nicolas@jombi.fr' VERSION = '1.2' DESCRIPTION = 'Canal Plus French TV' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('quality', label='Quality of videos', choices=['hd', 'sd'], default='hd')) BROWSER = CanalplusBrowser def create_default_browser(self): return self.create_browser(quality=self.config['quality'].get()) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): with self.browser: return self.browser.search_videos(pattern) def get_video(self, _id): m = re.match('https?://www\.canal-?plus\.fr/.*\?vid=(\d+)', _id) if m: _id = m.group(1) with self.browser: return self.browser.get_video(_id) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(CanalplusVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video OBJECTS = {CanalplusVideo: fill_video} def iter_resources(self, objs, split_path): if BaseVideo in objs: with self.browser: return self.browser.iter_resources(split_path) weboob-1.2/modules/canalplus/pages.py000066400000000000000000000101531303450110500177240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Nicolas Duhamel, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime import re from weboob.deprecated.browser import Page from weboob.capabilities.collection import Collection from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.capabilities.image import Thumbnail from .video import CanalplusVideo class ChannelsPage(Page): def get_channels(self): """ Extract all possible channels (paths) from the page """ channels = list() for elem in self.document[2].getchildren(): for e in elem.getchildren(): if e.tag == "NOM": fid, name = self._clean_name(e.text) channels.append(Collection([fid], name)) elif e.tag == "SELECTIONS": for select in e: sub_fid, subname = self._clean_name(select[1].text) sub = Collection([fid, sub_fid], subname) sub._link_id = select[0].text channels.append(sub) return channels def _clean_name(self, name): name = unicode(name.strip()) if name == name.upper(): name = name.capitalize() friendly_id = re.sub(ur"['/_ \(\)\-\+]+", u'-', name).strip(u'-').lower() return friendly_id, name class VideoPage(Page): def parse_video(self, el, video=None): _id = el.find('ID').text if _id == '-1': # means the video is not found return None if not video: video = CanalplusVideo(_id) infos = el.find('INFOS') video.title = u'' for part in infos.find('TITRAGE'): if len(part.text.strip()) == 0: continue if len(video.title) > 0: video.title += u' — ' video.title += part.text.strip() video.description = unicode(infos.find('DESCRIPTION').text) media = el.find('MEDIA') url = media.find('IMAGES').find('PETIT').text if url: video.thumbnail = Thumbnail(url) video.thumbnail.url = video.thumbnail.id else: video.thumbnail = NotAvailable for format in media.find('VIDEOS'): if format.text is None: continue if format.tag == 'HLS': video.ext = u'm3u8' video.url = unicode(format.text) break day, month, year = map(int, infos.find('PUBLICATION').find('DATE').text.split('/')) hour, minute, second = map(int, infos.find('PUBLICATION').find('HEURE').text.split(':')) video.date = datetime(year, month, day, hour, minute, second) return video def iter_results(self): for vid in self.document.getchildren(): video = self.parse_video(vid) video.url = NotLoaded yield video def iter_channel(self): for vid in self.document.getchildren(): yield self.parse_video_channel(vid) def parse_video_channel(self, el): _id = el[0].text video = CanalplusVideo(_id) video.title = u'%s' % el[2][5][0].text video.date = datetime.now() return video def get_video(self, video): _id = self.group_dict['id'] for vid in self.document.getchildren(): if _id not in vid.find('ID').text: continue return self.parse_video(vid, video) weboob-1.2/modules/canalplus/test.py000066400000000000000000000026551303450110500176140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class CanalPlusTest(BackendTest): MODULE = 'canalplus' def test_canalplus(self): l = list(self.backend.search_videos(u'guignol')) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and (v.url.startswith('rtmp://') or v.url.startswith('http://')), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_ls(self): l = list(self.backend.iter_resources((BaseVideo, ), [])) self.assertTrue(len(l) > 0) l = list(self.backend.iter_resources((BaseVideo, ), [u'sport'])) self.assertTrue(len(l) > 0) weboob-1.2/modules/canalplus/video.py000066400000000000000000000017201303450110500177330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo class CanalplusVideo(BaseVideo): swf_player = False @classmethod def id2url(cls, _id): return 'http://service.canal-plus.com/video/rest/getVideosLiees/cplus/%s' % _id weboob-1.2/modules/canaltp/000077500000000000000000000000001303450110500157135ustar00rootroot00000000000000weboob-1.2/modules/canaltp/__init__.py000066400000000000000000000014371303450110500200310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CanalTPModule __all__ = ['CanalTPModule'] weboob-1.2/modules/canaltp/browser.py000066400000000000000000000054231303450110500177540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime, date, time from weboob.deprecated.browser import Browser from weboob.tools.misc import to_unicode from weboob.deprecated.browser import BrokenPageError __all__ = ['CanalTP'] class CanalTP(Browser): DOMAIN = 'widget.canaltp.fr' def __init__(self, **kwargs): Browser.__init__(self, '', **kwargs) def iter_station_search(self, pattern): url = u'http://widget.canaltp.fr/Prochains_departs_15122009/dev/gare.php?txtrech=%s' % unicode(pattern) result = self.openurl(url.encode('utf-8')).read() for station in result.split('&'): try: _id, name = station.split('=') except ValueError: continue else: yield _id, to_unicode(name) def iter_station_departures(self, station_id, arrival_id=None): url = u'http://widget.canaltp.fr/Prochains_departs_15122009/dev/index.php?gare=%s' % unicode(station_id) result = self.openurl(url.encode('utf-8')).read() result = result departure = '' for line in result.split('&'): if '=' not in line: raise BrokenPageError('Unable to parse result: %s' % line) key, value = line.split('=', 1) if key == 'nomgare': departure = value elif key.startswith('ligne'): _type, unknown, _time, arrival, served, late, late_reason = value.split(';', 6) yield {'type': to_unicode(_type), 'time': datetime.combine(date.today(), time(*[int(x) for x in _time.split(':')])), 'departure': to_unicode(departure), 'arrival': to_unicode(arrival).strip(), 'late': late and time(0, int(late.split()[0])) or time(), 'late_reason': to_unicode(late_reason).replace('\n', '').strip()} def home(self): pass def login(self): pass def is_logged(self): """ Do not need to be logged """ return True weboob-1.2/modules/canaltp/favicon.png000066400000000000000000000046101303450110500200470ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME %p IDATx_l[W?{'cvhײt*֍l&h݀h{ U4!&D44^[6 ֔]4N$c_Ã='{^{}Ͽ΁-ْkZDG1 v@at.nG;W_I`߀A>/oG̺_< <e/G6/* ݐ?n%A`l)n΍ [cNK̖bȰujcGǜ%m QiKUA3@rL v{oVk b= n|;n"_{.ۖt˘_OwAugXm`2vN."{%UP#^JL= `5_,eQƧ3& بl̩v Zd}M|gg"/`? gϞLRSo! sxΜ#;{$})pUysoBԄRJyuF*9!OF,jȞ;Au  S^#ƛhb "b .4Z6{tj@gTq5m] 񇧟% n4TjCdU̷*nΔh[ݭ244ѣG"Hș3gVhn\O=Ky- i\ENq5[,E'ܼ?7cn>G$8_XDSqf ]P2B$q^Z)Vk%W& w~$5 6Sur* .:u3_C{&ܹ Bk&ŪE\w"jNX?Oui)]LZ- \ϋH׬j! %"穅qdVX0kV\X*d\VUR Gm~c}_x? S pYǨzvh D"\@" 蘽*0`r== 3|0>~$7LjumlvJ%.NtXP}.[%*V'N,68q>x< qE*v';*JnbQ@%0|Y: N= |7rM: q*LaF/HRLs#VXj[y ?)d332l7|CS32loGOc+qX'JA$㯹r06-I?Ah ~'adoɖlɖb}?IENDB`weboob-1.2/modules/canaltp/module.py000066400000000000000000000033211303450110500175510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.travel import CapTravel, Station, Departure from weboob.tools.backend import Module from .browser import CanalTP __all__ = ['CanalTPModule'] class CanalTPModule(Module, CapTravel): NAME = 'canaltp' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = "French trains" BROWSER = CanalTP def iter_station_search(self, pattern): for _id, name in self.browser.iter_station_search(pattern): yield Station(_id, name) def iter_station_departures(self, station_id, arrival_id=None, date=None): for i, d in enumerate(self.browser.iter_station_departures(station_id, arrival_id)): departure = Departure(i, d['type'], d['time']) departure.departure_station = d['departure'] departure.arrival_station = d['arrival'] departure.late = d['late'] departure.information = d['late_reason'] yield departure weboob-1.2/modules/canaltp/test.py000066400000000000000000000020241303450110500172420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CanalTPTest(BackendTest): MODULE = 'canaltp' def test_canaltp(self): stations = list(self.backend.iter_station_search('defense')) self.assertTrue(len(stations) > 0) list(self.backend.iter_station_departures(stations[0].id)) weboob-1.2/modules/cappedtv/000077500000000000000000000000001303450110500160775ustar00rootroot00000000000000weboob-1.2/modules/cappedtv/__init__.py000066400000000000000000000001261303450110500202070ustar00rootroot00000000000000# -*- coding: utf-8 -*- from .module import CappedModule __all__ = ['CappedModule'] weboob-1.2/modules/cappedtv/browser.py000066400000000000000000000115071303450110500201400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lord # # This module is free software. It comes without any warranty, to # the extent permitted by applicable law. You can redistribute it # and/or modify it under the terms of the Do What The Fuck You Want # To Public License, Version 2, as published by Sam Hocevar. See # http://sam.zoy.org/wtfpl/COPYING for more details. import urllib import datetime from weboob.capabilities.base import NotAvailable from weboob.tools.misc import to_unicode from weboob.deprecated.browser import Page from weboob.deprecated.browser import BrokenPageError from weboob.deprecated.browser import Browser from weboob.deprecated.browser.decorators import id2url from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import BaseVideo from weboob.tools.ordereddict import OrderedDict __all__ = ['CappedBrowser'] class CappedVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.nsfw = False self.ext = u'mp4' @classmethod def id2url(cls, _id): return 'http://capped.tv/%s' % _id # parser for search pages class IndexPage(Page): def iter_videos(self): # When no results are found, the website returns random results sb = self.parser.select(self.document.getroot(), 'div.search form input.searchbox', 1) if sb.value == 'No Results Found': return #Extracting meta data from results page vidbackdrop_list = self.parser.select(self.document.getroot(), 'div.vidBackdrop ') for vidbackdrop in vidbackdrop_list: url = self.parser.select(vidbackdrop, 'a', 1).attrib['href'] _id = url[2:] video = CappedVideo(_id) video.set_empty_fields(NotAvailable, ('url',)) video.title = to_unicode(self.parser.select(vidbackdrop, 'div.vidTitle a', 1).text) video.author = to_unicode(self.parser.select(vidbackdrop, 'div.vidAuthor a', 1).text) thumbnail_url = 'http://cdn.capped.tv/pre/%s.png' % _id video.thumbnail = Thumbnail(thumbnail_url) video.thumbnail.url = to_unicode(video.thumbnail.id) #we get the description field duration_tmp = self.parser.select(vidbackdrop, 'div.vidInfo', 1) #we remove tabs and spaces duration_tmp2 = duration_tmp.text[7:] #we remove all fields exept time duration_tmp3 = duration_tmp2.split(' ')[0] #we transform it in datetime format parts = duration_tmp3.split(':') if len(parts) == 1: hours = minutes = 0 seconds = parts[0] elif len(parts) == 2: hours = 0 minutes, seconds = parts elif len(parts) == 3: hours, minutes, seconds = parts else: raise BrokenPageError('Unable to parse duration %r' % duration_tmp) video.duration = datetime.timedelta(hours=int(hours), minutes=int(minutes), seconds=int(seconds)) yield video # parser for the video page class VideoPage(Page): def get_video(self, video=None): _id = to_unicode(self.group_dict['id']) if video is None: video = CappedVideo(_id) video.set_empty_fields(NotAvailable) title_tmp = self.parser.select(self.document.getroot(), 'title', 1) video.title = to_unicode(title_tmp.text.strip()) # Videopages doesn't have duration information (only results pages) video.url = u'http://cdn.capped.tv/vhq/%s.mp4' % _id return video class CappedBrowser(Browser): DOMAIN = 'capped.tv' PROTOCOL = 'http' ENCODING = None PAGES = OrderedDict(( (r'http://capped\.tv/?', IndexPage), (r'http://capped\.tv/newest', IndexPage), (r'http://capped\.tv/mostviews', IndexPage), (r'http://capped\.tv/leastviews', IndexPage), (r'http://capped\.tv/monthtop', IndexPage), (r'http://capped\.tv/monthbottom', IndexPage), (r'http://capped\.tv/alpha', IndexPage), (r'http://capped\.tv/ahpla', IndexPage), (r'http://capped\.tv/search\?s\=(?P.+)', IndexPage), (r'http://capped\.tv/(?P.+)', VideoPage), )) @id2url(CappedVideo.id2url) def get_video(self, url, video=None): self.location(url) assert self.is_on_page(VideoPage), 'Should be on video page.' return self.page.get_video(video) def search_videos(self, pattern): self.location('/search?s=%s' % (urllib.quote_plus(pattern.encode('utf-8')))) assert self.is_on_page(IndexPage) return self.page.iter_videos() def latest_videos(self): self.home() assert self.is_on_page(IndexPage) return self.page.iter_videos() weboob-1.2/modules/cappedtv/favicon.png000066400000000000000000000013211303450110500202270ustar00rootroot00000000000000PNG  IHDR@@% sRGBgAMA a pHYsodfIDAThCj0D9 9O0C cBKBզB;;1Jڷ3+yi<8pfi3ϳ_qlt:-J$BzJr=2H4ջ?= 6k˲n7mGFl0 bP $XUĝ'z*'^xCG0H@X/|>3`B~ CxpGH)5\`1ŗٶ`J彳%e Dz4]Hw/z!ݽУ&<Qj$d}פ3<1@u `gpgz O'kJ5 eIx 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) self.backend.browser.openurl(v.url) l = list(self.backend.search_videos('weboob')) self.assertTrue(len(l) == 0) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) weboob-1.2/modules/carrefourbanque/000077500000000000000000000000001303450110500174555ustar00rootroot00000000000000weboob-1.2/modules/carrefourbanque/__init__.py000066400000000000000000000014521303450110500215700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CarrefourBanqueModule __all__ = ['CarrefourBanqueModule'] weboob-1.2/modules/carrefourbanque/browser.py000066400000000000000000000045301303450110500215140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account from .pages import LoginPage, HomePage, TransactionsPage __all__ = ['CarrefourBanque'] class CarrefourBanque(LoginBrowser): BASEURL = 'https://www.carrefour-banque.fr' login = URL('/espace-client/connexion', LoginPage) home = URL('/espace-client$', HomePage) transactions = URL('/espace-client/(?P.*)/.*-operations.*', TransactionsPage) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.go() self.page.enter_login(self.username) self.page.enter_password(self.password) if not self.home.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): self.home.stay_or_go() return self.page.get_list() @need_login def iter_investment(self, account): if account.type != Account.TYPE_LIFE_INSURANCE: raise NotImplementedError() self.home.stay_or_go() self.location(account._link.replace('historique-des', 'solde-dernieres')) assert self.transactions.is_here() return self.page.get_investment(account) @need_login def iter_history(self, account): self.home.stay_or_go() self.location(account._link) assert self.transactions.is_here() return self.page.get_history(account) weboob-1.2/modules/carrefourbanque/favicon.png000066400000000000000000000026511303450110500216140ustar00rootroot00000000000000PNG  IHDR@@iqbKGDC pHYs B(xtIME  56IDATx[lTE"(.!(6Rm"x!H!<`4Fd_LLH4Ƈ1QH1FQ/Xh@Qk4"Y4p9ggΙٙohYB恫ˁ߀qKC!w@u/^xʞgqq.5aq/@|80a8:i/2yOFȕYE5U3EXrȦHE`%pM_lHY%tpC_OnCgyf KS%Ou&@4yY c],M)`6 ':ajPy׾f31jɭ\!_ŷ"Bu:-!M,= Xr@+S'B.`kYpꙥ<@GݚΫmUb_lUhorh2$OZYW?+qܭ1Ϩ\~a#)b!$^xxCB^@:`B~'C:=SG"#plFA`YGd-x ,w$iЂl Cu y'G暟a0lgJK>/Eh+<.Up!5W$׀k(p[@6I "bV9"Vw/'Wm :\W4ŭ@kBi:+ј#:B"v6[5pO>VN:WVk8ZiQ\ǝ {70k4*8͘:zK#98:K۔i2 ̳$"J5??":' S:!,/gBSI3g9W+1:[Uq/=]P Vp\/e#gjPm{2Ӎع?dg ^Ņ{DX ,o]m^*w|xv_ƹ)$>g" mWh1Q:m ؞|Tu6?(|RDI.,'u `JC |x|:mQdlуjHBz& '&jLNr/1dK`r0mv4 y{ߐ 3 o]Z#oWx"RYeMc@mxx4)N; *P'yjk=u5t_BS窾4Xwmdv'@\'T|0JIt\Pe JA"Ԉ| 4pE0IENDB`weboob-1.2/modules/carrefourbanque/module.py000066400000000000000000000040001303450110500213060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import find_object from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import CarrefourBanque __all__ = ['CarrefourBanqueModule'] class CarrefourBanqueModule(Module, CapBank): NAME = 'carrefourbanque' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Carrefour Banque' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label=u'Votre Identifiant Internet', masked=False), ValueBackendPassword('password', label=u"Code d'accès", regexp=u'\d+')) BROWSER = CarrefourBanque def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): return self.browser.iter_history(account) def iter_investment(self, account): return self.browser.iter_investment(account) weboob-1.2/modules/carrefourbanque/pages.py000066400000000000000000000106641303450110500211350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, TableElement, ItemElement, method from weboob.browser.filters.standard import Regexp, Field, TableCell, CleanText, CleanDecimal, Eval from weboob.browser.filters.html import Link from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class LoginPage(HTMLPage): def enter_login(self, username): form = self.get_form(nr=1) form['name'] = username form.submit() def enter_password(self, password): form = self.get_form(nr=1) form['pass'] = password form.submit() class HomePage(LoggedPage, HTMLPage): TYPES = {'carte': Account.TYPE_CARD, 'assurance': Account.TYPE_LIFE_INSURANCE, 'epargne': Account.TYPE_SAVINGS} @method class get_list(ListElement): item_xpath = '//div[@class="three_contenu_table"]' class item(ItemElement): klass = Account def obj_balance(self): if len(self.el.xpath('.//div[@class="catre_col_one"]/h2')) > 0: balance = CleanDecimal(CleanText('.//div[@class="catre_col_one"]/h2'), replace_dots=True)(self) return -balance if Field('type')(self) is Account.TYPE_CARD else balance return Decimal('0') def obj_type(self): return self.page.TYPES.get(Regexp(Field('_link'), '\/([^-]+)')(self), Account.TYPE_UNKNOWN) obj_id = CleanText('.//div[@class="carte_col_leftcol"]/p') & Regexp(pattern=r'(\d+)') obj_label = CleanText('.//div[@class="carte_col_leftcol"]/h2') obj_currency = FrenchTransaction.Currency('.//div[@class="catre_col_one"]/h2') obj__link = Link('.//a[contains(@href, "-operations")]') class Transaction(FrenchTransaction): PATTERNS = [(re.compile(r'^(?P.*?) (?P
\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_CARD)] class TransactionsPage(LoggedPage, HTMLPage): @method class get_investment(TableElement): item_xpath = '//table[@id="assets"]/tbody/tr[position() > 1]' head_xpath = '//table[@id="assets"]/tbody/tr[1]/td' col_label = u'Fonds' col_quantity = u'Nombre de parts' col_unitvalue = u'Valeur part' col_valuation = u'Total' col_portfolio_share = u'Répartition' class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_quantity = MyDecimal(TableCell('quantity')) obj_unitvalue = MyDecimal(TableCell('unitvalue')) obj_valuation = MyDecimal(TableCell('valuation')) obj_portfolio_share = Eval(lambda x: x / 100, MyDecimal(TableCell('portfolio_share'))) @pagination @method class get_history(Transaction.TransactionsElement): head_xpath = u'//div[*[contains(text(), "opérations")]]/table//thead/tr/th' item_xpath = u'//div[*[contains(text(), "opérations")]]/table/tbody/tr' def next_page(self): next_page = Link(u'//a[contains(text(), "précédentes")]', default=None)(self) if next_page: return "/%s" % next_page class item(Transaction.TransactionElement): obj_id = None def obj_type(self): return Transaction.TYPE_CARD if len(self.el.xpath('./td')) > 3 else Transaction.TYPE_BANK def condition(self): return TableCell('raw')(self) weboob-1.2/modules/carrefourbanque/test.py000066400000000000000000000017751303450110500210200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CarrefourBanqueTest(BackendTest): MODULE = 'carrefourbanque' def test_carrefourbanque(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/cci/000077500000000000000000000000001303450110500150275ustar00rootroot00000000000000weboob-1.2/modules/cci/__init__.py000066400000000000000000000014241303450110500171410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CciModule __all__ = ['CciModule'] weboob-1.2/modules/cci/browser.py000066400000000000000000000024451303450110500170710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from weboob.capabilities.job import BaseJobAdvert from .pages import SearchPage __all__ = ['CciBrowser'] class CciBrowser(PagesBrowser): BASEURL = 'http://www.cci.fr' search_page = URL('/web/recrutement/les-offres-d-emploi', SearchPage) def search_job(self, pattern): return self.search_page.go().iter_job_adverts(pattern=pattern) def get_job_advert(self, _id, advert): if advert is None: advert = BaseJobAdvert(_id) return self.search_page.stay_or_go().get_job_advert(obj=advert) weboob-1.2/modules/cci/favicon.png000066400000000000000000000403121303450110500171620ustar00rootroot00000000000000PNG  IHDR@@iq pHYs   cHRMz%u0`:o_F@PIDATx@@   0#"9 >  S;+( ?- 褻 o1#$  &TN 5>2%k  ee 8>9(F X  \m&   ,* *  ڣL  !)J?-4(! ?/>ߋ6 S@-4+ >0 C  8, '4) aL"!=̠  1' *7,t  1Ж  #  292$j   &Ϟ, !"@5% )!_D2v' jJ@=H"MB. , 0% ʁ0aS;%3&(2#% )!;6 & A5 gL65&eV=XD/X@- %  )! & 'ʅ OE1Q?,  GP! :+  fo 4/#$h=-  .u>+3(r`E}  &! (' +NA8&s bo+ )""7+=/ ," WC_ =6%eH 邚h   A;+  /%#   'PfR;8)dždOB/ `P & $ 3%UX>K<*K7&TuQ C9(E-A8   ? mV= +әUE0B4)!!D7'   ;D19)  l &+2% Q>8*($ D5&0] %"-"Q>,  ,"- *! Q>  0(>0  ;  VH5;,!^G1    h% K  ydGg  !=.  F'ZI5?0#h  !&,$ Z <*.A6% ;-   " cP)$   !! )! O>"g$ a-!%  -," $ %%#    dF 8*"44( ,1Q@ (!0( nZ  &'H# Y# 6]K!  VM>" aJ4󜤘 U@!":NO  M))&,< 4S͙+94. 'A0" z O ,#}aif  Zk܅   mI"z#%9M  UF2 NT&6}*8,C  0(6  +$2%    9 7+  \ޝ晬Pu  '1 FF ~ x `W(!7I  J>*  x!%HL!/; #" 6(9 (>K8/ n^.#q- -(K -&   KM6+~-#)* LF G? O? QC(*  OI UEn&ge +91 C% `H3O  5'<<- T `Xl6+ ZE5) pW;2  hZw 7  99,+}Cg [I7&o<䲞 -h6 q+χ SǯA/2+<5 J/>2#   J<* .75%cN6C7( Ė#." kU(:,xa 3. SG QC YM gT*!  |}J 6&     !2(O?,YH3' 7- (! %J >= ]Ns` eT ^Q9AR  ~yeI+! /$;. !,!@:*"L8( %J  7+=0=/*@6 SE&  mV>>Ǿ  {oTG2: u5(-!F3$6(! 4卿>"NP ({~% <  ` %  K6(8) * "Mw؉$!(*0!  $   D0"xU,]sK [[4.!   B$  Q)eN8kA2B5&[ukJH  h   ]$D3$t| 7    65+     YF1` {K  ~^OC/ S( ( /%NZN7{79(0(]-#s]BdM@- OC/ڈkMqA4$|>1"*!rb   .3$2-6'5*t2'Gq2*&" #2'51$9a   8+ +"   z_DK" ) YC/  ծ$0a~zIENDB`weboob-1.2/modules/cci/module.py000066400000000000000000000032331303450110500166670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.job import CapJob, BaseJobAdvert from weboob.tools.value import Value from .browser import CciBrowser __all__ = ['CciModule'] class CciModule(Module, CapJob): NAME = 'cci' DESCRIPTION = u'cci website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = CciBrowser CONFIG = BackendConfig(Value('metier', label='Job name', masked=False, default='')) def search_job(self, pattern=None): return self.browser.search_job(pattern) def advanced_search_job(self): return self.browser.search_job(pattern=self.config['metier'].get()) def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} weboob-1.2/modules/cci/pages.py000066400000000000000000000060171303450110500165040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, TableElement, method from weboob.browser.filters.standard import Filter, CleanText, Format, Env, DateTime, TableCell, Join from weboob.browser.filters.html import Link, CleanHTML from weboob.capabilities.job import BaseJobAdvert class Child(Filter): def filter(self, el): return list(el[0].iterchildren()) class SearchPage(HTMLPage): @method class iter_job_adverts(TableElement): item_xpath = '//tr[position() > 1]' head_xpath = "//tr[1]/td[@class='titreCol2Tableau']/text()" col_place = u'Région' col_job_name = u'Filière' col_id = u'Intitulé du poste' col_society_name = u'CCI(R)' class item(ItemElement): klass = BaseJobAdvert def validate(self, advert): if advert and 'pattern' in self.env and self.env['pattern']: return self.env['pattern'].upper() in advert.title.upper() or \ self.env['pattern'].upper() in advert.job_name.upper() return True obj_id = CleanText(Link(Child(TableCell('id'))), replace=[('#', '')]) obj_title = Format('%s - %s', CleanText(TableCell('id')), CleanText(TableCell('job_name'))) obj_society_name = Format(u'CCI %s', CleanText(TableCell('society_name'))) obj_place = CleanText(TableCell('place')) obj_job_name = CleanText(TableCell('id')) @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_url = Format('%s#%s', Env('url'), Env('id')) obj_description = Join('\r\n', 'div/fieldset/*[(@class="titreParagraphe" or @class="normal")]', textCleaner=CleanHTML) obj_title = CleanText('div/span[@class="intituleposte"]') obj_job_name = CleanText('div/span[@class="intituleposte"]') obj_society_name = Format('CCI %s', CleanText('div/span[@class="crci crcititle"]')) obj_publication_date = DateTime(CleanText('div/fieldset/p[@class="dateOffre"]'), dayfirst=True) def parse(self, el): self.el = el.xpath('//div[@id="%s"]/div' % self.obj.id)[0] self.env['url'] = self.page.url self.env['id'] = self.obj.id weboob-1.2/modules/cci/test.py000066400000000000000000000020721303450110500163610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CciTest(BackendTest): MODULE = 'cci' def test_cci_search(self): l = list(self.backend.search_job()) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) weboob-1.2/modules/centquatre/000077500000000000000000000000001303450110500164445ustar00rootroot00000000000000weboob-1.2/modules/centquatre/__init__.py000066400000000000000000000014361303450110500205610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CentQuatreModule __all__ = ['CentQuatreModule'] weboob-1.2/modules/centquatre/browser.py000066400000000000000000000046201303450110500205030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import itertools from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import CentQuatrePage, LoginPage, TicketsPage, TicketsDetailsPage __all__ = ['CentQuatreBrowser'] class CentQuatreBrowser(LoginBrowser): BASEURL = 'https://billetterie.104.fr' login = URL(r'/account$', LoginPage) tickets = URL(r'/account/tickets', TicketsPage) ticketDetails = URL(r'/account/file\?(.*)?fileId=(?P)', TicketsDetailsPage) unknown = URL(r'*', CentQuatrePage) def do_login(self): self.session.cookies.clear() self.login.go().login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() @need_login def list_events(self, date_from, date_to): self.tickets.stay_or_go() tickets = self.page.list_tickets(date_from, date_to) events = iter([]) for ticket in tickets: self.ticketDetails.stay_or_go(fileId=ticket) events = itertools.chain(events, self.page.get_event_details()) return events @need_login def get_event(self, id): return self.ticketDetails.stay_or_go(fileId=id).get_event_details() @need_login def search_events(self, query): events = self.list_events(query.start_date, query.end_date) matching_events = [] for event in events: if query.city and event.city != query.city: continue if query.ticket and event.ticket != query.ticket: continue if query.summary and event.summary != query.summary: continue matching_events.append(event) weboob-1.2/modules/centquatre/calendar.py000066400000000000000000000020411303450110500205640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS class CentQuatreEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.timezone = u'Europe/Paris' weboob-1.2/modules/centquatre/favicon.png000077500000000000000000000022751303450110500206100ustar00rootroot00000000000000PNG  IHDR<<:rbKGD pHYs B4tIME  &1NJIDATh]lUviJ`> FMV|@MT# qRH#آ$%`D"CX?Bږ~˺mi&w={s ɓ'Oy$=ukMoF#qem\25 9J  U׹'ԹkMeH`Řf֏٦})b6,ܚi')V.+Z .=ehDbW_yIZPQT~mhk(! `U C4 XcB#JVމ[Ñ{ 7vNly˳Z7(USϷ:6R_/8ae!0$,k L"vփ-a=;sRiT!yGZ )%9ve"HK21cۻCe3퟊ʪmNDlzA{ZK)pg ;+ݘf`Mm1L@jE1h+2VKԊZ~E[e8wfF,/^YetnHQF] 5pSqO.3nKw[q7 ӌ[PW B49w]Aݧi8*##+\ v]軸̪ gqGLFfa&3?˝yMQw _爚Ÿ&żDy5]9v8:Fkxr oa.ѕ'8Dt|S|5Ǵq)%ȴ*|b@.3~oǗҞܫqh%9U/up>*?l=/Nfےi<9W0IENDB`weboob-1.2/modules/centquatre/module.py000066400000000000000000000035371303450110500203130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.calendar import CapCalendarEvent from .browser import CentQuatreBrowser __all__ = ['CentQuatreModule'] class CentQuatreModule(Module, CapCalendarEvent): NAME = 'centquatre' DESCRIPTION = u'centquatre website' MAINTAINER = u'Phyks' EMAIL = 'phyks@phyks.me' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = CentQuatreBrowser CONFIG = BackendConfig( Value('email', label='Username', default=''), ValueBackendPassword('password', label='Password', default='') ) def create_default_browser(self): email = self.config['email'].get() password = self.config['password'].get() return self.create_browser(email, password) def get_event(self, _id): return self.browser.get_event(_id) def list_events(self, date_from, date_to=None): return self.browser.list_events(date_from, date_to) def search_events(self, query): if self.has_matching_categories(query): return self.browser.search_events(query) weboob-1.2/modules/centquatre/pages.py000066400000000000000000000102361303450110500201170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import method, ItemElement, ListElement from weboob.browser.filters.standard import CleanDecimal, CleanText from weboob.browser.filters.standard import DateTime, Env, Eval, Format from weboob.browser.filters.html import Link from weboob.capabilities.calendar import CATEGORIES, TICKET from .calendar import CentQuatreEvent from datetime import datetime, timedelta class CentQuatrePage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//*[@id="account_logout"]')) class LoginPage(CentQuatrePage): def login(self, email, password): form = self.get_form(id='login_form') form['login'] = email form['password'] = password form.submit() class TicketsPage(CentQuatrePage, LoggedPage): def list_tickets(self, date_from=None, date_to=None): tickets_containers = self.doc.xpath(u'//*[@class="product_container"]') tickets = [] for ticket_container in tickets_containers: date = datetime.strptime( ticket_container.xpath(u'//*[@class="day"]')[0].text.strip(), u'%A, %d %B %Y - %H:%M' ) if date_from and date < date_from: continue if date_to and date > date_to: continue tickets.append( ticket_container.xpath( u'//*[@class="file_number"]/a' )[0].text.strip().split(' ')[1] ) return tickets class TicketsDetailsPage(CentQuatrePage, LoggedPage): @method class get_event_details(ListElement): item_xpath = u'//*[@class="product_container"]' class EventDetails(ItemElement): klass = CentQuatreEvent obj_id = Env('fileId') obj_start_date = DateTime( CleanText(u'//*[@class="date"]') ) obj_end_date = Eval( lambda x: x + timedelta(hours=1), obj_start_date ) obj_timezone = u'Europe/Paris' obj_summary = CleanText( u'//*[@class="content_product_info"]//*[contains(@class, "title")]' ) obj_city = u'Paris' obj_location = Format( "%s, %s", CleanText( u'(//*[@class="location"])[1]' ), CleanText( u'//*[@class="address"]' ) ) obj_category = CATEGORIES.SPECTACLE obj_price = CleanDecimal( CleanText( u'(//*[@class="unit_price with_beneficiary"])[1]' ) ) obj_description = Format( u'%s. %s. %.2f€.', CleanText( u'(//*[contains(@class, "tariff") and contains(@class, "with_beneficiary")])[1]' ), CleanText( u'(//*[contains(@class, "seat") and contains(@class, "with_beneficiary")])[1]' ), obj_price, ) obj_ticket = TICKET.AVAILABLE def obj_url(self): return ( u'%s%s' % ( self.page.browser.BASEURL, Link( u'//*[@class="alternative_button mticket"]/a' )(self) ) ) weboob-1.2/modules/centquatre/test.py000066400000000000000000000017751303450110500200070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Phyks # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CentQuatreTest(BackendTest): MODULE = 'centquatre' def test_centquatre(self): l = list(self.backend.list_events(None)) assert len(l) event = self.backend.get_event(l[0].id) assert (event.id == l[0].id) weboob-1.2/modules/champslibres/000077500000000000000000000000001303450110500167455ustar00rootroot00000000000000weboob-1.2/modules/champslibres/__init__.py000066400000000000000000000014511303450110500210570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ChampslibresModule __all__ = ['ChampslibresModule'] weboob-1.2/modules/champslibres/browser.py000066400000000000000000000047641303450110500210150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, HomePage, HistoryPage, RentedPage __all__ = ['ChampslibresBrowser'] # Browser class ChampslibresBrowser(Browser): PROTOCOL = 'http' ENCODING = 'utf-8' PAGES = { '.*login.*': LoginPage, '.*home\?lang=frf.*': HomePage, 'http://.*/index.aspx\?IdPage=429': HistoryPage, '.*patroninfo.*': RentedPage, } iduser = None def is_logged(self): return self.page \ and not self.page.document.getroot().xpath('//input[contains(@id, "pin")]') def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.is_on_page(HomePage): self.location('https://sbib.si.leschampslibres.fr/iii/cas/login?null', no_login=True) self.page.login(self.username, self.password) # Get home and get ID self.location('http://opac.si.leschampslibres.fr/iii/encore/home?lang=frf', no_login=True) self.iduser = self.page.get_id() self.logger.debug('Get ID ' + self.iduser) if not self.is_logged(): raise BrowserIncorrectPassword() def get_rented_books_list(self): if not self.is_on_page(RentedPage): self.location('https://sbib.si.leschampslibres.fr/patroninfo~S1*frf/%s/items' % self.iduser) return self.page.get_list() def renew(self, id): if not self.is_on_page(RentedPage): self.location('https://sbib.si.leschampslibres.fr/patroninfo~S1*frf/%s/items' % self.iduser) self.page.renew(id) self.page.confirm_renew() return self.page.read_renew(id) # TODO def get_booked_books_list(self): return [] weboob-1.2/modules/champslibres/favicon.png000066400000000000000000000202501303450110500210770ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME9ށJ IDATxڥy\W}?wyKUuujI,;-ɋdxc08 30gd$! !! 1Ƌdl,/$K%nRUoU%a^wU]oo_Jsc 硵F)E{Bq`ܭ{u3B J)mZ_Yziq… jJ__TZ`^}ܗ+]TR !Ja{޼yt ٳ,\JBizZq9 ]{9ᔟ5çO+V<R(7oa$Igv6˽|[ڂҐ Ymg g3.1BA988hʿ*5 Ry^gs /pQ4E+8uhsA)=:c,JYTJuU ʕKjb{#@ WJaCI@u YJy\q]< ̥iӧO?iyYcؖ\V;w~4s8g1S!T( *ZiQjRe)ų%i4# %YBi'ILV;OӴZ˳>sロ[o4M$7>>oZYVrAXJ Z{A}?<PZű$Z^vΝ; DDe$Irعsuo~={q/~'NI8yeH<裼‹l۶7y'SSS #$ °JI*Zk! +x$Z)|/@*T )_(,/%BO(ȳa9nڴ}/333 =úurmH!ط% Ry>B8Z&}h]4z g,* Ji$ILREB V+\w,jhR%(O9_8ETsRbffcǎnz1m;v|mZk"S3'#^~ O]8q͛022UkT^~ wf۶mAe˘ĉ㌎kjNji5yQ.axpAX Hp&Ѫ+=z]vlr6l"==lwRSBFiɑ9'>*Ak'IHM1w-O}S 7DL59f2W g8^ji*7݄{k9?y&ϣۋ1<˙f1qn|CAYVJIQcf;RJtw_pV0dWZքa\Ș9t ,۰U :¯na"MWo i8bhHXb=ebxFkԎ,R\<"Fw0Z1~9R*u32*KPAy3RR%RDq0 ;a)N['r T)*B J)4SqkT@50p&^. 4ӓTk/}fS{VkaPLJXE{>ǎe1_*DEP9 qqΡ)!J.h_ =qn:LRһt Q1hhb,2QΞ̖֚ZOE 8xeZQ!Vi͙k;KQ8$"9A!<#‚p(xs1e)2LۄIB5 0&g֒oguT=y(?}|Ǧ0yƊ*S޾~rcs!%v\Meu'Yc) IZgdRq9kO>kՐzFR)hZl6Α[KLNNj8HIN^{5OLbLZqvY IB0֕Z!AH' ɩi< u,e8 DZK$??I\<"Z3zzzȲVˊɒ J9bNKpsy639BBCS 9~U]2[kB ǹwh4l6"Mr-Z$I,KRtQ%JzmJ2řw+m m, M !Ic mnh6(i#w/[a@e(h4,_,4 $!ZD=$! ~?ОbjzJ}]'X__H)/JX0h BPIsK()#j Z:&''Iӌ,KI$bpYaEJG@Z+$#r8"*۟-N!Ǣ럇 ,%uZ_(86VYjQDiFDž0rwNE8AIU2_^-3&"KBcitb+4SxO6v,Y1\G9~bJ!d͊Ը !EYB8 R"쑔 kVÑ_ o %"&'' ,H$I.{V5,+LZ399EB{Rs%i^ ڹ%pkz/k 2"$ZimAv h/4#I(&|qSsEQ.:z`QEJ;I@tE[`]^y 8Gh(T*t1j8Qʰ8Cv  FTw6]Ȼ}3I(zv #W M3f38gI4M 5Xg/IysMlnu}):jsxZj3PAԕWr*KםIg!=eQD%:3l]ۤJO,LPkmF+:9Aw_*jmN+jar[D И c Ә8fxh,%"̕vp׮踲1::Z >?C,^?cyUK4q?(:SsŊ+zJd7gރ8)^,eMZ 4MXW|7{S|*Vy)%5:\fqJ(IӤ@J609PJ JKDYy5tEdX;9u8Ɉ\E܊hL&ɳlX۶+A1Ȳ :<5B/"vr,+=.3_F : Z)y15=VPy[mրbF{ńaҺrx4il~:WꔍHKJp߱,PB C:p(k_CVfҪewѠ NhێPض R̊"mDmF,?+Ғ)h "X+0W8<'u5E )s c3%.n155EZ%˲'/BP 3|DUh J$ Ah-;}Bm}zn3R$J^>h}%СLLsP /R(B僤YVj+Fier4jH&BdRjZQTr=llp]I涬kڨP"O~'֪E\ $eE~A6q_G{)3zs|oѫLJ3t|-Q45kCޕ'ټc޽;B6za$G_4ګ,:C|'$*d5,^|%8 Vz OiQ$C/~j[җDUZcA UtK# "Ve߳|ſe,InyZ|ÿ{fH)5x?B6tZ$a%#M ϯ& >9*aOkիf>|F/o~%Kh6#rc-%Bz9u iӫ8~{n=AG9MZPJ0tfwU˛h6j-V=-GdXgIaj,=VR\nqӹJXzϞel|Z^u\QRlNSYBrCŦdy)pJOk< f Cɝwl13‰ y"UGX>EOOi39`謡 (aDZ%˯h,Ёe;<~hGmwx4W_ wgz7;ٻwwQ|9A֬y$ٺ~R37nGs=WƘRz3gP x/fdd kPa :7XNNr/7ъ,]rq\}g%q14z_={lrGVp*8("VJEHAAR QQX&+ $9,Ǚ}⸅@y<# QC&4fͷ¶{jZ9:e@?+<>>L66pܹ28pѢE.V8[Jk qPNڂm& Sɩժ$IY>,yAΑga4LI Ap)iAȢEkR&#֚{wsRJi?aˬ_鉉{p&Ms9K.ڻ{\jrKe5Psezk||۷oĦMػw/MCZ뛬 JtɵsRq9A~~%/IDAT^+RNyW)w/u{phOIENDB`weboob-1.2/modules/champslibres/module.py000066400000000000000000000043351303450110500206110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.library import CapBook from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import ChampslibresBrowser __all__ = ['ChampslibresModule'] class ChampslibresModule(Module, CapBook): NAME = 'champslibres' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' DESCRIPTION = 'Champs Libres (Rennes) Library' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('login', label='Account ID', regexp='^\d{1,15}|$'), ValueBackendPassword('password', label='Password of account'), ) BROWSER = ChampslibresBrowser def create_default_browser(self): browser = self.create_browser(self.config['login'].get(), self.config['password'].get()) # we have to force the login before to lauch any actions browser.login() return browser def get_rented(self): for book in self.browser.get_rented_books_list(): yield book def get_booked(self): raise NotImplementedError() def renew_book(self, id): return self.browser.renew(id) def iter_books(self): #for book in self.get_booked(): # yield book for book in self.get_rented(): yield book def get_book(self, _id): raise NotImplementedError() def search_books(self, _string): raise NotImplementedError() weboob-1.2/modules/champslibres/pages.py000066400000000000000000000062671303450110500204310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from weboob.capabilities.library import Book, Renew from weboob.deprecated.browser import Page from weboob.deprecated.mech import ClientForm from weboob.tools.html import html2text class SkipPage(Page): pass class HomePage(Page): def on_loaded(self): link = self.document.find('//a[@id="patronRSSFeedLinkComponent"]') self.id = link.attrib['href'].split('/')[4] def get_id(self): return self.id def txt2date(s): split = s.split('-') return date(int(split[2]) + 2000, int(split[1]), int(split[0])) class RentedPage(Page): def get_list(self): for tr in self.document.getroot().xpath('//tr[@class="patFuncEntry"]'): id = tr.xpath('td/input')[0].attrib["value"] book = Book(id) bigtitle = tr.xpath('td[@class="patFuncTitle"]/label/a')[0].text book.name = bigtitle.split('/')[0] book.author = bigtitle.split('/')[1] date = tr.xpath('td[@class="patFuncStatus"]')[0].text book.date = txt2date(date.replace('RETOUR', '')) yield book def renew(self, id): # find the good box input = self.document.find('//input[@value="%s"]' % id) self.browser.select_form("checkout_form") self.browser.form.set_all_readonly(False) self.browser.controls.append(ClientForm.TextControl('text', input.attrib['name'], {'value': id})) self.browser.controls.append(ClientForm.TextControl('text', 'requestRenewSome', {'value': 'requestRenewSome'})) self.browser.submit() def confirm_renew(self): self.browser.select_form("checkout_form") self.browser.form.set_all_readonly(False) self.browser.submit(name='renewsome') def read_renew(self, id): for tr in self.document.getroot().xpath('//tr[@class="patFuncEntry"]'): if len(tr.xpath('td/input[@value="%s"]' % id)) > 0: message = self.browser.parser.tostring(tr.xpath('td[@class="patFuncStatus"]')[0]) renew = Renew(id) renew.message = html2text(message).replace('\n', '') return renew class HistoryPage(Page): pass class BookedPage(Page): # TODO: book some books... pass class LoginPage(Page): def login(self, login, passwd): self.browser.select_form(nr=0) self.browser.form.set_all_readonly(False) self.browser['code'] = login self.browser['pin'] = passwd self.browser.submit() weboob-1.2/modules/champslibres/test.py000066400000000000000000000015741303450110500203050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class ChampslibresTest(BackendTest): MODULE = 'champslibres' def test_champslibres(self): pass weboob-1.2/modules/chronopost/000077500000000000000000000000001303450110500164675ustar00rootroot00000000000000weboob-1.2/modules/chronopost/__init__.py000066400000000000000000000014421303450110500206010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ChronopostModule __all__ = ['ChronopostModule'] weboob-1.2/modules/chronopost/browser.py000066400000000000000000000025571303450110500205350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser from .pages import IndexPage, TrackPage __all__ = ['ChronopostBrowser'] class ChronopostBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'www.chronopost.fr' ENCODING = None PAGES = { 'http://www.chronopost.fr/transport-express/livraison-colis': IndexPage, 'http://www.chronopost.fr/transport-express/livraison-colis/.*accueil/suivi.*': TrackPage, } def get_tracking_info(self, _id): self.home() assert self.is_on_page(IndexPage) self.page.track_package(_id) assert self.is_on_page(TrackPage) return self.page.get_info(_id) weboob-1.2/modules/chronopost/favicon.png000066400000000000000000000040701303450110500206230ustar00rootroot00000000000000PNG  IHDR@@iqIDATxY lUE} @X0=т@"AVAYھ,"KK4D uAbQ- 1Ģ@-@~K+nr2ܹwΝfkK.K.K.R5S^F2i@ #U@0^dN|p&SR׀"HgB^dAVm΀Qʄ7':`={ #V"cɬ7R  .?Z7ݩJ߁y|(A`Wj~ " k4)m, P#v.`; BӞlO+g9 &P dQPQnYlf;43 |2oҶBD`:if|?>fXV4etQVܕ`:`.3M}n9}3ߩrI*kg0n`h ]Y߉L{r ۃ9QΨ`s$Mq0X^x3I\)0钽,`$gW43_GRF"f:_cJNYm]=Cqs MdGUtk(xiNYg)T 9_:wet鈡p+~oZj,/JrKy%^;lϓ1Xn3̰t`Bt!`g/9dnv%NV>u1bӁ U vJteĠ^DgO#XMxe::#SeØ}K>?$pvB pX{x=lx)}>o%.BA<{LgatI:03*!*#0>U))>RŠa=4DOqt)Vtf0Z]b_Wѳ2ox"?0%WNQm/4qjst(n Ё_ 'Q=?H"YGpHjL}#h$?LfEEJ\3{搰zbEsJmXQa5癍4JK@)R 3LI}jm$ h@a֧>uX ]!LX@ fDn6/zZ?tǔ0̖Үnš6y=Y 4^J!2 1sHcd`XG˨ t^()/V:0(g:ʮ?O}NM"I-!x17 I:-F?̃l,(棈Hwyi$ F V:ߟjׄltk~I-Ӟ+ZOAD 1о(k{y4}#aCPZBz&U4̭Io(,;hQTt*+;!1K>ԯ6 v HO~m%\p S*@C k[I(3̃Pԙi4F h]BސQocȥdIK.K.K.KKry_AIENDB`weboob-1.2/modules/chronopost/module.py000066400000000000000000000023371303450110500203330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.parcel import CapParcel from weboob.tools.backend import Module from .browser import ChronopostBrowser __all__ = ['ChronopostModule'] class ChronopostModule(Module, CapParcel): NAME = 'chronopost' DESCRIPTION = u'Chronopost website' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' BROWSER = ChronopostBrowser def get_parcel_tracking(self, id): with self.browser: return self.browser.get_tracking_info(id) weboob-1.2/modules/chronopost/pages.py000066400000000000000000000050411303450110500201400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from dateutil.parser import parse as parse_date from weboob.capabilities.parcel import Parcel, Event from weboob.capabilities import NotAvailable from weboob.deprecated.browser import Page class IndexPage(Page): def track_package(self, _id): self.browser.select_form(predicate=lambda form: form.attrs.get('id', '') == 'suivreEnvoi') self.browser['chronoNumbers'] = _id.encode('utf-8') self.browser.submit() class TrackPage(Page): def get_info(self, id): if len(self.document.xpath('//libelle[@nom="MSG_AUCUN_EVT"]')) > 0: return None p = Parcel(id) p.arrival = NotAvailable p.history = [] for i, tr in enumerate(self.document.xpath('//table[@class="tabListeEnvois"]//tr')): tds = tr.findall('td') if len(tds) < 3: continue ev = Event(i) ev.location = unicode(tds[1].text) if tds[1].text else None ev.activity = unicode(tds[1].find('br').tail) if tds[-1].text is not None: ev.activity += ', ' + self.parser.tocleanstring(tds[-1]) date = re.sub('[a-z]+', '', self.parser.tocleanstring(tds[0])).strip() date = re.sub('(\d+)/(\d+)/(\d+)', r'\3-\2-\1', date) ev.date = parse_date(date) p.history.append(ev) p.info = ' '.join([t.strip() for t in self.document.xpath('//div[@class="numeroColi2"]')[0].itertext()][1:]) if u'Livraison effectuée' in p.history[0].activity: p.status = p.STATUS_ARRIVED elif u"en cours d'acheminement" in p.history[0].activity or \ u"en cours de livraison" in p.history[0].activity or \ u"Envoi entré dans le pays de destination" in p.history[0].activity: p.status = p.STATUS_IN_TRANSIT return p weboob-1.2/modules/chronopost/test.py000066400000000000000000000015111303450110500200160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class ChronopostTest(BackendTest): MODULE = 'chronopost' weboob-1.2/modules/cic/000077500000000000000000000000001303450110500150275ustar00rootroot00000000000000weboob-1.2/modules/cic/__init__.py000066400000000000000000000014311303450110500171370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CICModule __all__ = ['CICModule'] weboob-1.2/modules/cic/browser.py000066400000000000000000000025761303450110500170760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .pages import LoginPage from weboob.browser.browsers import AbstractBrowser from weboob.browser.profiles import Wget from weboob.browser.url import URL __all__ = ['CICBrowser'] class CICBrowser(AbstractBrowser): PROFILE = Wget() TIMEOUT = 30 BASEURL = 'https://www.cic.fr' PARENT = 'creditmutuel' login = URL('/mabanque/fr/authentification.html', '/sb/fr/banques/particuliers/index.html', '/(?P.*)/fr/$', '/(?P.*)/fr/banques/accueil.html', '/(?P.*)/fr/banques/particuliers/index.html', LoginPage) weboob-1.2/modules/cic/favicon.png000066400000000000000000000031561303450110500171670ustar00rootroot00000000000000PNG  IHDR@@iqbKGD___4ƍ pHYs  tIME pvtEXtCommentCreated with GIMPWIDATx{lSU?]u+kvɺ! &T01Ѩ5?ƨ5 0,"21Fۭtt}{1n]I&{~{:::::::::::::?|.4`Mm%%!>? ro|+LR5*?7Z@S##xCSlr:e&E%& B(|B0Lр!DŽ1wӢiE\BVl.-("sf:'5viSS0"UV@8p6oM2~  Ÿ(TT6VS;4RW㼥ǏF}v.3sI}I۟}[i)`gc^1'g`m}Y $Pa̹N]UnW~îi*RY}F@ɩ(>$V9Z5k[\q>;(rýEGzp[XT$'eIJdP!?Haxb U%c0RQagm]elllLSTUEQUdEz441Y$D8g8ߗ4$okB"!aEF\ ĹU3XʱsK#_|{"#O}qYgL8r"b8'?uTcS⬦n&*+ضɓ'M.O΄cw4o0h`o[i<c6DcrZVV!TU3ޱTX'_*GVkf1#ɢG''h(j֌Ϫ3Ḧme3&ZhnUeNhr ̢4Ϫ=E7-KO -U8-LF[ hv44.-v {wOZCM{ܨ,$M{ݱ40D6oXɻ<^yh5Y<{~m˒Lϕk=璄G}m T rк&2wy[|wtd80]sK\loh8) -մ]A QLO$N2^Y?iQxh#{׷`*_@`Fv41<ғg{|եes # PVTĞ{V) 輅lYryKoߞȪ|GPip`m9iY(H(M懓Q;}~qn?x0ͥeoޤ[@UUE%4ct9ȱEYyom#%Oώ]?{np 67bs?Jg F֔Ъf%zxf1]n6TPZXHBQ Dh)$N srdWFttttttttttttttttto Z$gcIENDB`weboob-1.2/modules/cic/module.py000066400000000000000000000033561303450110500166750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBankTransfer from weboob.capabilities.contact import CapContact from weboob.tools.backend import AbstractModule, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import CICBrowser __all__ = ['CICModule'] class CICModule(AbstractModule, CapBankTransfer, CapContact): NAME = 'cic' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'CIC' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = CICBrowser PARENT = 'creditmutuel' def create_default_browser(self): browser = self.create_browser(self.weboob, self.config['login'].get(), self.config['password'].get()) browser.new_accounts.urls.insert(0, "/mabanque/fr/banque/comptes-et-contrats.html") return browser weboob-1.2/modules/cic/pages.py000066400000000000000000000021531303450110500165010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage class LoginPage(HTMLPage): REFRESH_MAX = 10.0 def login(self, login, passwd): form = self.get_form(xpath='//form[contains(@name, "ident")]') form['_cm_user'] = login form['_cm_pwd'] = passwd form.submit() @property def logged(self): return self.doc.xpath('//div[@id="e_identification_ok"]') weboob-1.2/modules/cic/test.py000066400000000000000000000017401303450110500163620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CICTest(BackendTest): MODULE = 'cic' def test_cic(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/citelis/000077500000000000000000000000001303450110500157255ustar00rootroot00000000000000weboob-1.2/modules/citelis/__init__.py000066400000000000000000000014441303450110500200410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CitelisModule __all__ = ['CitelisModule'] weboob-1.2/modules/citelis/browser.py000066400000000000000000000062551303450110500177720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import Account from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, SummaryPage, UselessPage, TransactionSearchPage, TransactionsPage, TransactionsCsvPage __all__ = ['CitelisBrowser'] class CitelisBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'adminpayment.citelis.fr' ENCODING = 'UTF-8' CERTHASH = '3cccf1cdd31264a864bc206fc96257a6cd1602b5766054304a6b3c3768414c84' PAGES = { '%s://%s/userManager\.do\?reqCode=prepareLogin.*' % (PROTOCOL, DOMAIN): LoginPage, '%s://%s/summarySearch\.do\?reqCode=search.*' % (PROTOCOL, DOMAIN): SummaryPage, '%s://%s/userManager\.do\?reqCode=goToHomePage.+' % (PROTOCOL, DOMAIN): UselessPage, '%s://%s/userManager\.do\?org.apache.+' % (PROTOCOL, DOMAIN): UselessPage, '%s://%s/menu\.do\?reqCode=prepareSearchTransaction.+' % (PROTOCOL, DOMAIN): TransactionSearchPage, '%s://%s/transactionSearch\.do\?reqCode=search.+' % (PROTOCOL, DOMAIN): TransactionsPage, '%s://%s/documents/transaction/l_TransactionSearchWebBooster\.jsp.+' % (PROTOCOL, DOMAIN): (TransactionsCsvPage, 'csv') } def __init__(self, merchant_id, *args, **kwargs): self.merchant_id = merchant_id Browser.__init__(self, *args, **kwargs) def login(self): if not self.is_on_page(LoginPage): self.location('%s://%s/userManager.do?reqCode=prepareLogin' % (self.PROTOCOL, self.DOMAIN)) self.page.login(self.merchant_id, self.username, self.password) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() def is_logged(self): return not self.is_on_page(LoginPage) def get_accounts_list(self): self.location('%s://%s/summarySearch.do?reqCode=search' % (self.PROTOCOL, self.DOMAIN)) account = Account() account.id = u'1' account.currency = 'EUR' account.balance = self.page.get_balance() account.label = u'Synthèse financière' return [account] def iter_history(self, account): assert account.id == '1' if not self.is_on_page(TransactionSearchPage): self.location('%s://%s/menu.do?reqCode=prepareSearchTransaction&init=true&screen=new' % (self.PROTOCOL, self.DOMAIN)) self.page.search() self.location(self.page.get_csv_url()) for transaction in self.page.iter_transactions(): yield transaction weboob-1.2/modules/citelis/favicon.png000066400000000000000000000111111303450110500200530ustar00rootroot00000000000000PNG  IHDR@@iqIDAT g]}9잳ٰ$B.(IHL&I"RFD+# URRV*;SgPaZjՎLUdԪcC% ِ^u/$lfjgGK!lN*I$I++NL$UTJVd}=C٢d( 22 a saTRx3#XR"YRYll%YV,J!B-TBL#LuPO⠙u HNFm6}穜r> $gc;VPXD_ E'h0 a$+ )2w]ޑNQr:5zm3td E,  }2" I @A I8*Vh^fc<W y*~2*[-[p| ¢ө$$!)IA ( =,:Ŧ'iߪz~qԞ 8еj7J'TXV:tN(@AAAAA ( $" ۴EיvuQ{v6r}-bqpN_'kѢC' -:tMBPJ DGHR"ULTk I))%馋Ll1Cs*jO&Yf Rr-.UlUI-af\tӢ]ӉJPKu互f0ɽE3^7ٷ}&/;SIm%CsL[{(vfk6XiG̔n*FI\j ׋RoQ}|^}1~~xs[v*v6zIrLP#i`Ia=a3]k67۸S3ڭLtUe(sZddr_E)/[TꅭA릫_0[ft v*EKruKuIFؼ?fkc=7=1OiFkkHtSҌDK" jTrI^T VTmP;jFz8EѳYzC 5co{:_^7޷n=fLnq~4a⳿ ,DWY3*Y6vFx$+JHR,kUblkNڝi(/yPdj߄[*sL#wxϧn7[}H׌|O;n͉/{P;M2C6Z9F{fmwOh6‰]w4ΰډZ+  ظ/l޿fzF}xG}w6D<ߎ?ue'2EvY% 6hLmz1]n( kSR"%_8oƵNaI1P$$vJaclֶlnzDJ䈔1OgpFDሔQU/BvU/ $H #n.HZ2>}|K-]kW}y#JNLLUbeAXIZ8*")-QQ/m*n;xыșwUɄMSGz-d)x^bvÒ !pT ){Xǯz锏||#.ɄG Cx7|=K9OAXEAⰂ(ɨh ۝/}KK\r {3l戃[${j_3E+ !+y *_JO?η}vy+񗞤WD2GIITK޵UŞno$%]x;$ {!̄0@"WE^X$'G-:U/~1}-)yDuq-xXĺ!R,-PTT72;s~6)Ȏ`U&T{T-zK竇;ȕ^z<_OJDȍ7:XUAQ)X":RE=a_ob+mItxv9K.odq)3 7S랢XŚ>9S j0_o2XwDwW]Epą]E { )ݯR50onas-r}EbۜL ׽/vXuԎo&J{7HiQD{jU<%}^!~;ց|cpmKU1{eIgνS>Q)NR^R꟥^L |BKV11љ<,$YU$zVm*=[vj\PҢVӍgxV4v2GDlxES jU$UT!Ur^PL{e{06(BAUȨUQ e˲Ɋpɹxp-M|q_ccwU;_0஻kVoݹJ]+mRY˲eɢB7`u $SI$ dd!9I$XH` P+Nҡq~ڸoMܟX8FU3S0[ݬk꟯nJDG=d/pIZB+IA8,Ȓ,dT,$NRСFc}'M=hxک/f@xYRI7^$RT*/PÒIV I PP`AFуރF{>c?6 #9U?p{29M3'-=[=&zAR/JHo*Q!!!!9upTCۄ&0[ez3903_k~K3;8._*UoQ/xn8SVnU,nVOSWTe`[$""!J8SڹҌtH7N7fUiהv4#e!b?j=Aw%lz0k6X\/IՂ\/.+m5EtR JnM1GE5(>IJΤ~".f݌fL;'QᰔH9Wk>kHIENDB`weboob-1.2/modules/citelis/module.py000066400000000000000000000037641303450110500175760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.bank import CapBank, AccountNotFound from .browser import CitelisBrowser __all__ = ['CitelisModule'] class CitelisModule(Module, CapBank): NAME = 'citelis' DESCRIPTION = u'Citélis' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = CitelisBrowser CONFIG = BackendConfig( ValueBackendPassword('merchant_id', label='Merchant ID', masked=False), ValueBackendPassword('login', label='Account ID', masked=False), ValueBackendPassword('password', label='Password')) def create_default_browser(self): return self.create_browser(self.config['merchant_id'].get(), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): for account in self.iter_accounts(): if account.id == _id: return account raise AccountNotFound() def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/citelis/pages.py000066400000000000000000000106361303450110500174040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal import datetime import re from weboob.deprecated.browser import Page, BrowserIncorrectPassword from weboob.tools.capabilities.bank.transactions import FrenchTransaction class Transaction(FrenchTransaction): @classmethod def clean_amount(cls, text): text = text.strip() # Convert "American" UUU.CC format to "French" UUU,CC format if re.search(r'\d\.\d\d$', text): text = text.replace(',', ' ').replace('.', ',') return FrenchTransaction.clean_amount(text) class LoginPage(Page): def login(self, merchant_id, login, password): self.browser.select_form(name='loginForm') self.browser['merchant'] = merchant_id.encode(self.browser.ENCODING) self.browser['login'] = login.encode(self.browser.ENCODING) self.browser['password'] = password.encode(self.browser.ENCODING) self.browser.submit() class SummaryPage(Page): def clean_amount(self, el, debit): amount = Decimal(Transaction.clean_amount(el.text_content())) if amount == Decimal('0.00'): return None if debit and amount > Decimal('0'): return -1 * amount return amount def get_balance(self): zone = self.parser.select(self.document.getroot(), '#ActionZone_Euro', 1) for tr in self.parser.select(zone, '#transactionError tr'): tds = self.parser.select(tr, 'td') if tds and tds[0].text_content().strip() == 'Total': debit, credit = self.parser.select(tr, 'td.amount', 4)[-2:] # keep the last 2 debit = self.clean_amount(debit, debit=True) credit = self.clean_amount(credit, debit=False) amount = credit or debit return amount class UselessPage(Page): def on_loaded(self): for error in self.document.xpath('//li[@class="error"]'): raise BrowserIncorrectPassword(self.parser.tocleanstring(error)) class TransactionSearchPage(Page): def search(self, accepted=True, refused=False): self.browser.select_form(name='transactionSearchForm') self.browser['selectedDateCriteria'] = ['thisweek'] # TODO ask for more self.browser['transactionAccepted'] = ['0'] if accepted else [] self.browser['transactionRefused'] = ['0'] if refused else [] # simulate the javascript nonce = self.parser.select(self.document.getroot(), '#menu li.global a')[0] \ .attrib['href'].partition('CSRF_NONCE=')[2] self.browser.form.action = '%s://%s/transactionSearch.do?reqCode=%s&org.apache.catalina.filters.CSRF_NONCE=%s&screen=new' % (self.browser.PROTOCOL, self.browser.DOMAIN, 'search', nonce) self.browser.submit() class TransactionsPage(Page): def get_csv_url(self): for a in self.parser.select(self.document.getroot(), '.exportlinks a'): if len(self.parser.select(a, 'span.csv')): return a.attrib['href'] class TransactionsCsvPage(Page): def guess_format(self, amount): if re.search(r'\d\.\d\d$', amount): date_format = "%m/%d/%Y" else: date_format = "%d/%m/%Y" time_format = "%H:%M:%S" return date_format + ' ' + time_format def iter_transactions(self): ID = 0 DATE = 2 AMOUNT = 4 CARD = 7 NUMBER = 8 for row in self.document.rows: t = Transaction(row[ID]) date = row[DATE] amount = row[AMOUNT] datetime_format = self.guess_format(amount) t.set_amount(amount) t.parse(datetime.datetime.strptime(date, datetime_format), row[CARD] + ' ' + row[NUMBER]) yield t weboob-1.2/modules/citelis/test.py000066400000000000000000000017771303450110500172720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CitelisTest(BackendTest): MODULE = 'citelis' def test_citelis(self): l = list(self.backend.iter_accounts()) assert len(l) a = l[0] l = list(self.backend.iter_history(a)) assert len(l) weboob-1.2/modules/citibank/000077500000000000000000000000001303450110500160555ustar00rootroot00000000000000weboob-1.2/modules/citibank/__init__.py000066400000000000000000000014441303450110500201710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CitibankModule __all__ = ['CitibankModule'] weboob-1.2/modules/citibank/browser.py000066400000000000000000000216751303450110500201250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014, 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.pages import HTMLPage, JsonPage, RawPage from weboob.capabilities.bank import Account, AccountNotFound, Transaction from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.tools.js import Javascript from .parser import StatementParser, clean_label import re from datetime import datetime from time import sleep __all__ = ['Citibank'] class SomePage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//a[text()="Sign Off"]') + self.doc.xpath(u'//div[@id="CardsLoadingDiv"]')) class IndexPage(SomePage): def extra(self): APPEND = r'jQuery\( "form" \).each\(function\(\) {' \ r'if\(isValidUrl\(jQuery\(this\).attr\("action"\)\)\){' \ r'jQuery\(this\).append\(([^)]+)\);}}\);' script = self.doc.xpath( '//script[contains(text(),"XXX_Extra")]/text()')[0] script = re.sub(APPEND, lambda m: 'return %s;' % m.group(1), script) script = re.sub(r'jQuery\(document\)[^\n]+\n', '', script) for x in re.findall('function ([^(]+)\(', script): script += '\nvar x = %s(); if (x) return x;' % x js = Javascript("function run(){%s}" % script) html = js.call("run") return re.findall(r'name=([^ ]+) value=([^>]+)>', html) class AccountsPage(JsonPage): logged = True def inner_ids_dict(self): return dict((prod['parsedAccountName'][-4:], prod['accountInstanceId']) for bean in self.doc['summaryViewBeanList'] for cat in bean['accountsSummaryViewObj']['categoryList'] for prod in cat['products'] if cat['categoryType'] == 'CRD') class AccDetailsPage(JsonPage): logged = True def account(self): detact = self.doc['accountDetailsAndActivity'] details = detact['accountDetails'] account = Account() account.type = Account.TYPE_CARD account.label = re.sub(r'<[^>]+>', '', detact['accountName']) account.id = account.label[-4:] for bal in details['accountBalances']: label, value = bal['label'], (bal['value'] or ['0'])[0] if label == u'Current Balance:': if value[0] == '(' and value[-1] == ')': value = value[1:-1] sign = 1 else: sign = -1 account.currency = Account.get_currency(value) account.balance = sign * AmTr.decimal_amount(value) elif label == u'Total Revolving Credit Line:': account.cardlimit = AmTr.decimal_amount(value) elif label.startswith(u'Minimum Payment Due'): d = re.match(r'.*(..-..-....):$', label).group(1) account.paydate = datetime.strptime(d, '%m-%d-%Y') account.paymin = AmTr.decimal_amount(value) return account def transactions(self): return sorted(self.unsorted_trans(), lambda a, b: cmp(a.date, b.date), reverse=True) def unsorted_trans(self): for jnl in self.doc['accountDetailsAndActivity']['accountActivity'] \ ['postedTransactionJournals']: tdate = jnl['columns'][0]['activityColumn'][0] label = jnl['columns'][1]['activityColumn'][0] amount = jnl['columns'][3]['activityColumn'][0] xdescs = dict((x['label'], x['value'][0]) for x in jnl['extendedDescriptions']) pdate = xdescs[u'Posted Date :'] ref = xdescs.get(u'Reference Number:') or u'' if amount.startswith(u'(') and amount.endswith(u')'): amount = AmTr.decimal_amount(amount[1:-1]) else: amount = -AmTr.decimal_amount(amount) label = clean_label(label) trans = Transaction(ref) trans.date = datetime.strptime(tdate, '%m-%d-%Y') trans.rdate = datetime.strptime(pdate, '%m-%d-%Y') trans.type = Transaction.TYPE_UNKNOWN trans.raw = label trans.label = label trans.amount = amount yield trans class StatementsPage(SomePage): def dates(self): return [x[:10] for x in self.doc.xpath( u'//select[@id="currentStatementsDate"]/option/@value') if re.match(u'^\d\d\d\d-\d\d-\d\d All$', x)] class StatementPage(RawPage): logged = True def __init__(self, *args, **kwArgs): RawPage.__init__(self, *args, **kwArgs) self._parser = StatementParser(self.doc) def is_sane(self): return self._parser.read_first_date_range() is not None def transactions(self): return sorted(self._parser.read_transactions(), cmp=lambda t1, t2: cmp(t2.date, t1.date)) class Citibank(LoginBrowser): """ Citibank website uses some kind of Javascript magic during login negotiation, hence a real JS interpreter is being used to log in. External dependencies: V8 JavaScript Engine (http://code.google.com/p/v8/). MuPDF (http://www.mupdf.com). Tested on Arch Linux snapshot of 2014-11-11 (official and user packages). Only a single credit card account is currently supported. Contributions are welcome! """ BASEURL = 'https://online.citi.com' MAX_RETRIES = 10 TIMEOUT = 30.0 index = URL(r'/US/JPS/portal/Index.do', IndexPage) signon = URL(r'/US/JSO/signon/ProcessUsernameSignon.do', SomePage) accdetailhtml = URL(u'/US/NCPS/accountdetailactivity/flow.action.*$', SomePage) accounts = URL(r'/US/REST/accountsPanel' r'/getCustomerAccounts.jws\?ttc=(?P.*)$', AccountsPage) accdetails = URL(r'/US/REST/accountDetailsActivity' r'/getAccountDetailsActivity.jws\?accountID=(?P.*)$', AccDetailsPage) statements = URL(r'/US/NCSC/doccenter/flow.action\?TTC=1079&' 'accountID=(?P.*)$', StatementsPage) statement = URL(r'/US/REST/doccenterresource/downloadStatementsPdf.jws\?' r'selectedIndex=0&date=(?P....-..-..)&' r'downloadFormat=pdf', StatementPage) unknown = URL('/.*$', SomePage) def get_account(self, id_): innerId = self.to_accounts().inner_ids_dict().get(id_) if innerId: return self.to_account(innerId).account() raise AccountNotFound() def iter_accounts(self): for innerId in self.to_accounts().inner_ids_dict().values(): yield self.to_account(innerId).account() def iter_history(self, account): innerId = self.to_accounts().inner_ids_dict()[account.id] for trans in self.to_account(innerId).transactions(): yield trans for date in self.to_statements(innerId).dates(): for trans in self.to_statement(date).transactions(): yield trans def to_account(self, innerId): return self.to_page(self.accdetails, accountID=innerId) def to_accounts(self): return self.to_page(self.accounts, ttc='742') def to_statements(self, innerId): return self.to_page(self.statements, accountID=innerId) def to_statement(self, date): # Sometimes the website returns non-PDF file. # It recovers if we repeat whole browsing sequence all the way # from home page up to the statement. MAX_DELAY=10 for i in xrange(self.MAX_RETRIES): if self.to_page(self.statement, date=date).is_sane(): return self.page sleep(min(MAX_DELAY, 1 << i)) self.do_login() innerId = self.to_accounts().inner_ids_dict().values()[0] self.to_account(innerId) self.to_statements(innerId) @need_login def to_page(self, url, **data): return self.page if url.is_here(**data) else url.go(data=data, **data) def do_login(self): self.session.cookies.clear() data = dict([('username', self.username), ('password', self.password)]+ self.index.go().extra()) if not self.signon.go(data=data).logged: raise BrowserIncorrectPassword() weboob-1.2/modules/citibank/favicon.png000066400000000000000000000077561303450110500202270ustar00rootroot00000000000000PNG  IHDR@@% gAMA a cHRMz&u0`:pQ<bKGDIDAThZ{Uu}|Gro䒷!<"c .-u֊v.*bki}`+.Q"+Q!!䝛3sٻ|Cp+Ǭgog!t'mhD%mꉶabp8]p&f;ht Eo @۩cwҤ}nJ` ֜yX 3AdQjbV84g l@2.aubȕUl$/lF!* %wVu>lgݾŽ,om-3KYl] z$%V R錑=s_¹oF͋-(5dhm-ͼ+3{>層Zu`!ΣORP!$DÚv=UP:AJj6t+'9# H-TL}A',^BGƀ"XXDܹ]^=/"R<>b4My[kQ[士s.l@fH}q ?nd肫Ǔ;U5^g oo;xglB}1j Dd(S]@_Wrz^>HJ*dj,PX*ClMT4d#"9<$2<{$Ƹtpї}ᡁ탍eږt^8 bug;FR]>04'gv\&;~d`O(0RIN.qÞjM恧!?M/><󅵧V ;.mj#D5(e]k4r^sƬ}Z#UحXu^4-gv]lڞ?J(Z{˻{H-^.4hռ.`v7lv_~y90}3"Å W2٢Ϲ [ÌNtq9uqMw|yw<îR62 c;G.Wh?:gvǴ-63ڼf~mÓTWt aI_,4Ĥ(LQ)Ͼ޶[ޝۆQӄfgn6l.vMi! Jj1 T**Wb >7/7/XYW}w]YzZݸ/s_~*eĥ |m[}]\j v ]C`bд Ň/<zw/!! Ή | Q(h3X9kNgk7wz7i,TBJ>}so[Vo_W"E<kplˇ\GkB >+z<|jiB 443(%͟>H7,] ~k]Z:-nAeڒ/?~M)`5dFGWݼoSL<X B_2}{^5ZwRY(<԰8RMʜδPTUiwR#kN^|˽[D wo?|A4bS;ɮ+VF`Wݏuwg4Ě%ߔCf[]Hgmç޴J[yݖz],RF{ք͡SԼ*O] 3+=Q醽|pޗOlrEM?-&*%O=w&Ed#zG޶;(Q:Vxj#Z]4w-|LogJ\s }{f-*,el`jkbQiM;T,Uٶ:h~/UZ6Zcm͋^2)4*czK#Fm8sjOHGa ʑL<LRCd"KR/@LJO0 ay%d<# +bH-qlsfֶЀpy&̳2ũP*xG@UEUx,s""!|(jUj&$y!&U Jh&RBb@2ȧPX]=tE;͆wWz>4(BqT a YSCfzF{čzE86@cA۾ܾ0~~}h&@g9AN '>  'yBzIlO*%tEXtdate:create2014-10-11T13:21:44-05:00l%tEXtdate:modify2014-10-11T13:21:19-05:004IENDB`weboob-1.2/modules/citibank/module.py000066400000000000000000000033331303450110500177160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Citibank __all__ = ['CitibankModule'] class CitibankModule(Module, CapBank): NAME = 'citibank' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Citibank' CONFIG = BackendConfig( ValueBackendPassword('username', label='Username', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = Citibank def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.iter_accounts() def get_account(self, id_): return self.browser.get_account(id_) def iter_history(self, account): return self.browser.iter_history(account) weboob-1.2/modules/citibank/parser.py000066400000000000000000000163721303450110500177340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import Transaction from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.tools.date import closest_date from weboob.tools.pdf import decompress_pdf from weboob.tools.tokenizer import ReTokenizer import datetime import re def clean_label(text): """ Web view and statements use different label formatting. User shouldn't be able to see the difference, so we need to make labels from both sources look the same. """ for pattern in [r' \d+\.\d+ +POUND STERLING', u'Subject to Foreign Fee', u'Description']: text = re.sub(pattern, u'', text, re.UNICODE) return re.sub(r' +', u' ', text.strip().upper(), re.UNICODE) def formatted(read_func): """ Reads boilerplate PDF formatting around the data of interest. """ def wrapped(self, pos): startPos = pos pos, ws = self.read_whitespace(pos) pos, bt = self.read_layout_bt(pos) pos, tf = self.read_layout_tf(pos) pos, tm = self.read_layout_tm(pos) pos, data = read_func(self, pos) pos, et = self.read_layout_et(pos) if ws is None or bt is None or tf is None \ or tm is None or data is None or et is None: return startPos, None else: return pos, data return wrapped class StatementParser(object): """ Each "read_*" method takes position as its argument, and returns next token position if read was successful, or the same position if it was not. """ LEX = [ ('date_range', r'^\((\d{2}/\d{2}/\d{2})-(\d{2}/\d{2}/\d{2})\) Tj$'), ('amount', r'^\((-?\$\d+(,\d{3})*\.\d{2})\) Tj$'), ('date', r'^\((\d{2}/\d{2})\) Tj$'), ('text', r'^\((.*)\) Tj$'), ('layout_tf', r'^.* Tf$'), ('layout_tm', r'^' + (6*r'([^ ]+) ') + r'Tm$'), ('layout_bt', r'^BT$'), ('layout_et', r'^ET$'), ('whitespace', r'^$') ] def __init__(self, pdf): self._pdf = decompress_pdf(pdf) self._tok = ReTokenizer(self._pdf, '\n', self.LEX) def read_transactions(self): # Read statement dates range. date_from, date_to = self.read_first_date_range() # Read transactions. pos = 0 while not self._tok.tok(pos).is_eof(): pos, trans = self.read_transaction(pos, date_from, date_to) if trans: yield trans else: pos += 1 def read_first_date_range(self): pos = 0 while not self._tok.tok(pos).is_eof(): pos, date_range = self.read_date_range(pos) if date_range is not None: return date_range else: pos += 1 def read_date_range(self, pos): t = self._tok.tok(pos) if t.is_date_range(): return (pos+1, [datetime.datetime.strptime(v, '%m/%d/%y') for v in t.value()]) else: return (pos, None) def read_transaction(self, pos, date_from, date_to): startPos = pos pos, tdate = self.read_date(pos) pos, pdate = self.read_date(pos) # Early check to call read_multiline_desc() only when needed. if tdate is None: return startPos, None pos, desc = self.read_multiline_desc(pos) pos, amount = self.read_amount(pos) if desc is None or amount is None: return startPos, None else: # Sometimes one date is missing. pdate = pdate or tdate tdate = closest_date(tdate, date_from, date_to) pdate = closest_date(pdate, date_from, date_to) trans = Transaction() trans.date = tdate trans.rdate = pdate trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = -amount return pos, trans def read_multiline_desc(self, pos): """ Read transaction description which can span over multiple lines. Amount must always follow the multiline description. But multiline description might be split by page break. After reading first line of the description, we skip everything which is not an amount and which has different horizontal offset than the first read line. """ startPos = pos descs = [] xofs = None while not self._tok.tok(pos).is_eof(): pos, desc_tm = self.read_text(pos) if desc_tm is None: if not descs: break prev_pos = pos pos, amount = self.read_amount(pos) if amount is not None: pos = prev_pos break pos += 1 else: desc, tm = desc_tm if xofs is None: _, _, _, _, xofs, _ = tm _, _, _, _, xofs_new, _ = tm if xofs == xofs_new: descs.append(desc) else: pos += 1 if descs: return pos, clean_label(' '.join(descs)) else: return startPos, None def __getattr__(self, name): if name.startswith('read_'): return lambda pos: self._tok.simple_read(name[5:], pos) raise AttributeError() @formatted def read_date(self, pos): def parse_date(v): for year in [1900, 1904]: # try leap and non-leap years fullstr = '%s/%i' % (v, year) try: return datetime.datetime.strptime(fullstr, '%m/%d/%Y') except ValueError as e: pass raise e return self._tok.simple_read('date', pos, parse_date) @formatted def read_amount(self, pos): return self._tok.simple_read('amount', pos, lambda xs: AmTr.decimal_amount(xs[0])) def read_text(self, pos): startPos = pos pos, ws = self.read_whitespace(pos) pos, bt = self.read_layout_bt(pos) pos, tf = self.read_layout_tf(pos) pos, tm = self.read_layout_tm(pos) pos, text = self._tok.simple_read('text', pos, lambda v: unicode(v, errors='ignore')) pos, et = self.read_layout_et(pos) if ws is None or bt is None or tf is None \ or tm is None or text is None or et is None: return startPos, None else: return pos, (text, tm) weboob-1.2/modules/citibank/test.py000066400000000000000000000021761303450110500174140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from itertools import chain class CitibankTest(BackendTest): MODULE = 'citibank' def test_history(self): """ Test that there's at least one transaction in the whole history. """ b = self.backend ts = chain(*[b.iter_history(a) for a in b.iter_accounts()]) t = next(ts, None) self.assertNotEqual(t, None) weboob-1.2/modules/cmb/000077500000000000000000000000001303450110500150325ustar00rootroot00000000000000weboob-1.2/modules/cmb/__init__.py000066400000000000000000000014231303450110500171430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Johann Broudin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CmbModule __all__ = ['CmbModule'] weboob-1.2/modules/cmb/favicon.png000066400000000000000000000121161303450110500171660ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDM*E pHYs  tIME , dF|tEXtCommentCreated with GIMPWIDATx͛yp?o9'ɞ A @(V U+PĹUwzZ;ҪwNzպ[Ve*,ͅM$+YNINHA}f~3m><`ok2= o-hMR]p ,Qq6 ?(zz ôQxu9?oi p?;<仗  m^o~]SÇP7\i}5YN}0㛷)Ͼn,fܯ1"~7k98NDM ρ  ?$D?KO'׿(*BA65!ADBZZZgPzzpkjz^zi4p"Ѳ^( ϼzRRW3OЄ׫AW%ߡCp^\[[SUkWy^e}1jf"p0nC G68+ [0:sOFnM nm-=SVvmcժQq4SF|9n}=o|渜|>zS4&ZVP1ݧifM$צNECI!@!{矫).,-E$& 3rrpF650@z}g4i0"h@ඵ pt]5~A<|Xv6ZN$ԩctSWMsWr@orcr<+W֩ L-!<tK#.}3 f|=I1=_5Ɔi7. mdDl,B86S ?\Q0D΅Ea.΁8E\Fa;sn$tH OY L}3ラ/n}=zWd2dC==[]liA3moz0Yh'"22'N߁΢(7Ǹ DBЏ;pA66ҽn!@#O1q񈸸Hs4oO,g4x{cP.sL:"%Q2?`mB'G* Fdk+}WPvQ - t-3S%e{>Zo39k1f‚DSͤ">tu!;:j̖/Ǹ2Dz:ژԤ< ˔u@hjr@8ٖhY!#  ‡^/M>{647hmEnn@`XuԒ7N PJuS8"> cNz>ٵhZﺕe9QXs~2+U/+JVjuDB4 ^ب_X.՟Af"@, с^\`oڄ!- sDt44Gb.$!b5w$&b\z)Y݋h99_HM)+zM1JKI:5vdehܳyfX}| ϠF3]]cpOGL_P6e ڤI GEz@O q>O1u*Q܃;pmn,-Pq-# pՈ | qs;#ɓx~SmԒkhPE ;:p|p=\4ٻ=<<7ވAvv#p>MͤϞ,%QY΍CG^}c;q?XPC025Uy{ #|,Kϙ76} Z2V^K gϛ˱l~ rdGRj DF>0p|?UqQXHp0 V^=^A߇ӧ3-B$$څ{;v RؼgN\#C!tDtN-[^}czY={ @/.(,D<?}|wAGw`>z^7> vi#[T<78@]w@q>tIp穧^~Y)vM)DSgf" qEKKSYSQTDʕ<,nEԩZc|0 2,epD?^}YB0 kA/ja rr^x1"%?X}Ոdz6l0-Cde64l݊efBRζm*4  =\ЀqW GopqM 7?zc|udw7") =?_55Ȧ&ٳЦLA(yTqm+0YY8}>VӐMߕt{Fd06mΎ'N RA|σ L anØ8oF9{S!SS?~<ڬYد5}$Eɓyu+uq+*7,K.TI| x ?U {F7nUξ}|7AsƏٹijbNE"* mD>_6Pi^X\ic/)/z;KJ0,9x==xns*ĤIȪ*zz qw˖lq<^Rm = =>~xה6mRCVWe7Q:s~nĸ*]2BS5n̥KѦNUyϧfEKN9x5q>!&g^d rֆ[S{7&kyU!~{6W ڔ.P^iSDb̕+1,A7N)@MQTD"Db"$xՃ#23&MBT{(-B<89 /6F}..Nu-B/*R{VVb?\޹^7=tt޽φ)g8HA jHk11\`") kf׬sRdC5_Lu*f" FE梍6qs5}؛Nxc%]NUx;v_+t.]*9 pvF_ !֪vW 0Jme/i dFDz:G9|X̙яy)maWh`~ք;N-"uv"pqW}P{?f1,)޵ j_V:g 5 лeW{jg&"І?Lf(IENDB`weboob-1.2/modules/cmb/module.py000066400000000000000000000046551303450110500167030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from .par.browser import CmsoParBrowser from .pro.browser import CmsoProBrowser __all__ = ['CmbModule'] class CmbModule(Module, CapBank): NAME = 'cmb' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' VERSION = '1.2' DESCRIPTION = u'Credit Mutuel de Bretagne' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe'), Value('website', label='Type de compte', default='pro', choices={'par': 'Particuliers', 'pro': 'Professionnels'})) BROWSER = CmsoParBrowser def create_default_browser(self): b = {'par': CmsoParBrowser, 'pro': CmsoProBrowser} self.BROWSER = b[self.config['website'].get()] return self.create_browser(self.weboob, "cmb.fr", self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_history(self, account): return self.browser.iter_history(account) def iter_coming(self, account): return self.browser.iter_coming(account) def iter_investment(self, account): return self.browser.iter_investment(account) weboob-1.2/modules/cmb/par/000077500000000000000000000000001303450110500156145ustar00rootroot00000000000000weboob-1.2/modules/cmb/par/__init__.py000066400000000000000000000000001303450110500177130ustar00rootroot00000000000000weboob-1.2/modules/cmb/par/browser.py000066400000000000000000000016001303450110500176460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import AbstractBrowser class CmsoParBrowser(AbstractBrowser): PARENT = 'cmso' PARENT_ATTR = 'package.par.browser.CmsoParBrowser' weboob-1.2/modules/cmb/pro/000077500000000000000000000000001303450110500156325ustar00rootroot00000000000000weboob-1.2/modules/cmb/pro/__init__.py000066400000000000000000000000001303450110500177310ustar00rootroot00000000000000weboob-1.2/modules/cmb/pro/browser.py000066400000000000000000000015701303450110500176720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 smurail # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import AbstractBrowser class CmsoProBrowser(AbstractBrowser): PARENT = 'cmso' PARENT_ATTR = 'package.pro.browser.CmsoProBrowser' weboob-1.2/modules/cmso/000077500000000000000000000000001303450110500152325ustar00rootroot00000000000000weboob-1.2/modules/cmso/__init__.py000066400000000000000000000014241303450110500173440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CmsoModule __all__ = ['CmsoModule'] weboob-1.2/modules/cmso/favicon.png000066400000000000000000000044401303450110500173670ustar00rootroot00000000000000PNG  IHDR@@iqbKGDC pHYs  tIME /*!tEXtCommentCreated with GIMPWIDATxݛ{lUE?[hKPEB*EdB4&+`4eCdWACf}BD|@]⢨A RE(1=Ћ}>.'s~gf|37s.!ZN rKR7cY ~P{i:!Sţ!o$8,H7"x"<@ Vo:Ղ >$A%8+T{C;a0pGw{;qO!)N/AJwq\:SAà@N{z?)g`JW+u$դgkWn*\)8kPn/efJ TGdEljlDA4jy&%%I r1c=; 7(a}g+`Y+^03p\ Iz aMxS |CJJ<ѡL0bm#éS%Il*|0ӧ6&Mj{gς#c5c:YUUP(b4~o@}7%)VHlaǎj L^+/0%NlJKE 5P~Qz:ѣh4<.ƌۢ2N1O7Z--MwSo6~$ɞ^d8YVV{ @bbPR"[/1BVSgϖ9"k~X>YmVU \3SK 7ʕG}q1l染_ֵwK8[!7Y}'Nm$]}TVQ!-3Yu*+}FlYMl|dlc/^~YVV/vlŊEs8,ӝdWy'綾d+W3%KdiiNQnbxiӲpik!} 9/) /bHNl/4C~?O~D̛a ͏*p+*"nO6^PU}xR@J3W~>^j*>`ddO<'kGAuuP[pty`(*s@ޮzꮭEEEx ԸNT\ 5С&:{ut&+ʷߖ#{]-rse;v>Ylϓ˝ g7nt7![FklLWڵ/=L³:jBY^nr+e>*fWn0"g{ָVR";p VPeNcanҬYhsFL. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from .par.browser import CmsoParBrowser from .pro.browser import CmsoProBrowser __all__ = ['CmsoModule'] class CmsoModule(Module, CapBank): NAME = 'cmso' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Crédit Mutuel Sud-Ouest' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe'), Value('website', label='Type de compte', default='par', choices={'par': 'Particuliers', 'pro': 'Professionnels'})) BROWSER = CmsoParBrowser def create_default_browser(self): b = {'par': CmsoParBrowser, 'pro': CmsoProBrowser} self.BROWSER = b[self.config['website'].get()] return self.create_browser("cmso.com", self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_history(self, account): return self.browser.iter_history(account) def iter_coming(self, account): return self.browser.iter_coming(account) def iter_investment(self, account): return self.browser.iter_investment(account) weboob-1.2/modules/cmso/par/000077500000000000000000000000001303450110500160145ustar00rootroot00000000000000weboob-1.2/modules/cmso/par/__init__.py000066400000000000000000000000001303450110500201130ustar00rootroot00000000000000weboob-1.2/modules/cmso/par/browser.py000066400000000000000000000146031303450110500200550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re, json from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account from .pages import LogoutPage, InfosPage, AccountsPage, HistoryPage, LifeinsurancePage, MarketPage class CmsoParBrowser(LoginBrowser): logout = URL('/securityapi/revoke', '/auth/errorauthn', LogoutPage) infos = URL('/comptes/', InfosPage) accounts = URL('/domiapi/oauth/json/accounts/synthese(?P.*)', AccountsPage) history = URL('/domiapi/oauth/json/accounts/(?P.*)', HistoryPage) loans = URL('/creditapi/rest/oauth/v1/synthese', AccountsPage) lifeinsurance = URL('/assuranceapi/v1/oauth/sso/suravenir/DETAIL_ASSURANCE_VIE/(?P.*)', 'https://domiweb.suravenir.fr/', LifeinsurancePage) market = URL('/domiapi/oauth/json/ssoDomifronttitre', 'https://www.(?P.*)/domifronttitre/front/sso/domiweb/01/(?P.*)Portefeuille\?csrf=', 'https://www.*/domiweb/prive/particulier', MarketPage) def __init__(self, website, *args, **kwargs): super(CmsoParBrowser, self).__init__(*args, **kwargs) self.BASEURL = "https://mon.%s" % website self.name = website.split('.')[0] self.website = website arkea = {'cmso.com': "03", 'cmb.fr': "01", 'cmmc.fr': '02'} self.arkea = arkea[website] self.logged = False def deinit(self): if self.page.logged: self.logout.go(method='DELETE') super(CmsoParBrowser, self).deinit() def do_login(self): data = { 'accessCode': self.username, 'password': self.password, 'clientId': 'com.arkea.%s.siteaccessible' % self.name, 'redirectUri': '%s/auth/checkuser' % self.BASEURL, 'errorUri': '%s/auth/errorauthn' % self.BASEURL } self.location('/securityapi/tokens', data=data) if self.logout.is_here(): raise BrowserIncorrectPassword m = re.search('access_token=([^&]+).*id_token=(.*)', self.url) self.session.headers.update({ 'Content-Type': 'application/json', 'Authentication': "Bearer %s" % m.group(2), 'Authorization': "Bearer %s" % m.group(1), 'X-ARKEA-EFS': self.arkea, 'X-Csrf-Token': m.group(1) }) @need_login def iter_accounts(self): # First get all checking accounts... data = dict(self.infos.stay_or_go().get_typelist()) self.accounts.go(data=json.dumps(data), type='comptes').check_response() for key in self.page.get_keys(): for a in self.page.iter_accounts(key=key): yield a # Next, get saving accounts numbers = self.page.get_numbers() for key in self.accounts.go(data=json.dumps({}), type='epargne').get_keys(): for a in self.page.iter_products(key=key, numbers=numbers): yield a # Then, get loans for key in self.loans.go().get_keys(): for a in self.page.iter_loans(key=key): yield a @need_login def iter_history(self, account): if account.type is Account.TYPE_LOAN: return iter([]) if account.type == Account.TYPE_LIFE_INSURANCE: url = json.loads(self.lifeinsurance.go(accid=account._index).content)['url'] url = self.location(url).page.get_link(u"opérations") return self.location(url).page.iter_history() elif account.type == Account.TYPE_MARKET: self.location(json.loads(self.market.go(data=json.dumps({'place': 'SITUATION_PORTEFEUILLE'})).content)['urlSSO']) self.session.headers['Content-Type'] = 'application/x-www-form-urlencoded' if not self.market.go(website=self.website, action='historique').get_list(account.label): return iter([]) if not self.page.get_full_list(): return iter([]) # Display code ISIN history = self.location('%s?reload=oui&convertirCode=oui' % self.url).page.iter_history() self.session.headers['Content-Type'] = 'application/json' return history # Getting a year of history nbs = ["UN", "DEUX", "TROIS", "QUATRE", "CINQ", "SIX", "SEPT", "HUIT", "NEUF", "DIX", "ONZE", "DOUZE"] self.history.go(data=json.dumps({'index': account._index}), page="detailcompte") self.trs = {'lastdate': None, 'list': []} return self.page.iter_history(index=account._index, nbs=nbs) @need_login def iter_coming(self, account): if account.type is Account.TYPE_LOAN: return iter([]) comings = [] self.history.go(data=json.dumps({"index": account._index}), page="pendingListOperations") for key in self.page.get_keys(): for a in self.page.iter_history(key=key): comings.append(a) return iter(comings) @need_login def iter_investment(self, account): if account.type == Account.TYPE_LIFE_INSURANCE: url = json.loads(self.lifeinsurance.go(accid=account._index).content)['url'] url = self.location(url).page.get_link("supports") return self.location(url).page.iter_investment() elif account.type == Account.TYPE_MARKET: self.location(json.loads(self.market.go(data=json.dumps({"place": \ "SITUATION_PORTEFEUILLE"})).content)['urlSSO']) return self.page.iter_investment() if self.market.go(website=self.website, \ action="situation").get_list(account.label) else iter([]) raise NotImplementedError() weboob-1.2/modules/cmso/par/pages.py000066400000000000000000000330541303450110500174720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re, requests, json from datetime import datetime as dt from weboob.browser.pages import HTMLPage, JsonPage, RawPage, LoggedPage, pagination from weboob.browser.elements import DictElement, ItemElement, TableElement, SkipItem, method from weboob.browser.filters.standard import CleanText, Upper, Date, Regexp, Format, CleanDecimal, Env, Slugify, TableCell, Field from weboob.browser.filters.json import Dict from weboob.browser.filters.html import Attr, Link from weboob.browser.exceptions import ServerError from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.exceptions import BrowserIncorrectPassword def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class LogoutPage(RawPage): pass class InfosPage(LoggedPage, HTMLPage): def get_typelist(self): url = Attr(None, 'src').filter(self.doc.xpath('//script[contains(@src, "comptes/scripts")]')) m = re.search('synthesecomptes[^\w]+([^:]+)[^\w]+([^"]+)', self.browser.open(url).content) return {m.group(1): m.group(2)} class AccountsPage(LoggedPage, JsonPage): TYPES = {'courant': Account.TYPE_CHECKING, 'preference': Account.TYPE_LOAN, 'vie': Account.TYPE_LIFE_INSURANCE, 'actions': Account.TYPE_MARKET, 'titres': Account.TYPE_MARKET, 'livret': Account.TYPE_SAVINGS, 'epargne logement': Account.TYPE_SAVINGS } def get_keys(self): return [k for k, v in self.doc.items() if v and isinstance(v, (dict, list)) and "exception" not in self.doc] def check_response(self): if "exception" in self.doc: raise BrowserIncorrectPassword("Vous n'avez pas de comptes sur l'espace particulier de ce site.") def get_numbers(self): keys = self.get_keys() numbers = {} for key in keys: if isinstance(self.doc[key], dict): keys = [k for k in self.doc[key] if isinstance(k, unicode)] contracts = [v for v in self.doc[key][k] for k in keys] else: contracts = [v for v in self.doc[key]] numbers.update({c['index']: c['numeroContratSouscrit'] for c in contracts}) return numbers @method class iter_accounts(DictElement): def parse(self, el): self.item_xpath = "%s/*" % Env('key')(self) def find_elements(self): selector = self.item_xpath.split('/') for el in selector: if isinstance(self.el, dict) and el == '*' and self.el.values(): self.el = self.el.values()[0] if el == '*': continue self.el = self.el[el] for el in self.el: yield el class item(ItemElement): klass = Account obj_id = Dict('numeroContratSouscrit') obj_label = Upper(Dict('lib')) obj_balance = CleanDecimal(Dict('soldeEuro', default="0")) obj_currency = Dict('deviseCompteCode') obj_coming = CleanDecimal(Dict('AVenir', default=None), default=NotAvailable) # Iban is available without last 5 numbers, or by sms obj_iban = NotAvailable obj__index = Dict('index') def obj_type(self): return self.page.TYPES.get(Dict('accountType', default=None)(self).lower(), Account.TYPE_UNKNOWN) def condition(self): return "LIVRET" not in Dict('accountType')(self) @method class iter_products(DictElement): def parse(self, el): self.item_xpath = "%s/*/savingsProducts/*/savingsAccounts" % Env('key')(self) class item(ItemElement): klass = Account obj_label = Upper(Dict('libelleContrat')) obj_balance = CleanDecimal(Dict('solde', default="0")) obj_currency = u'EUR' obj_coming = CleanDecimal(Dict('AVenir', default=None), default=NotAvailable) obj__index = Dict('index') def obj_id(self): try: return Env('numbers')(self)[Dict('index')(self)] except KeyError: # index often changes, so we can't use it... and have to do something ugly return Slugify(Format('%s-%s', Dict('libelleContrat'), Dict('nomTitulaire')))(self) def obj_type(self): for key in self.page.TYPES: if key in Dict('libelleContrat')(self).lower(): return self.page.TYPES[key] return Account.TYPE_UNKNOWN @method class iter_loans(DictElement): def parse(self, el): self.item_xpath = "%s/*" % Env('key')(self) if "Pret" in Env('key')(self): self.item_xpath = "%s/lstPret/*" % self.item_xpath class item(ItemElement): klass = Account obj_id = Dict('identifiantTechnique') obj_label = Dict('libelle') obj_currency = u'EUR' obj_type = Account.TYPE_LOAN def obj_balance(self): return CleanDecimal().filter("-%s" % \ (Dict('montantRestant', default=None)(self) or Dict('montantDisponible')(self))) class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PCARTE.*)'), FrenchTransaction.TYPE_CARD), (re.compile(u'^(?P(PRLV|PRELEVEMENTS).*)'), FrenchTransaction.TYPE_ORDER), (re.compile(u'^(?PRET DAB.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(u'^(?PECH.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile(u'^(?PVIR.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^(?PANN.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile(u'^(?P(VRST|VERSEMENT).*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(u'^(?P.*)'), FrenchTransaction.TYPE_BANK), ] class HistoryPage(LoggedPage, JsonPage): def get_keys(self): return [k for k, v in self.doc.items() if v and isinstance(v, (dict, list)) and "exception" not in self.doc] @pagination @method class iter_history(DictElement): def next_page(self): if len(Env('nbs', default=[])(self)): data = {'index': Env('index')(self), 'filtreOperationsComptabilisees': "MOIS_MOINS_%s" % Env('nbs')(self)[0] } Env('nbs')(self).pop(0) return requests.Request('POST', data=json.dumps(data)) def parse(self, el): # Key only if coming key = Env('key', default=None)(self) self.item_xpath = "%s/operationList" % key if key and "CardList" not in key \ else "%s/currentMonthCardList/*/listeOperations/*" % key \ if key else "listOperationProxy" class item(ItemElement): klass = Transaction obj_raw = Transaction.Raw(Dict('libelleCourt')) obj_vdate = Date(Dict('dateValeur'), dayfirst=True) obj_amount = CleanDecimal(Dict('montantEnEuro'), default=NotAvailable) def obj_date(self): return dt.fromtimestamp(int(Dict('dateOperation')(self)[:-3])) def parse(self, el): # Skip duplicate transactions tr = "%s%s%s" % (Dict('libelleCourt')(self), Dict('dateOperation')(self), Dict('montantEnEuro')(self)) if tr in self.page.browser.trs['list'] and self.page.browser.trs['lastdate'] < Field('date')(self): raise SkipItem() self.page.browser.trs['lastdate'] = Field('date')(self) self.page.browser.trs['list'].append(tr) class LifeinsurancePage(LoggedPage, HTMLPage): def get_link(self, page): return Link().filter(self.doc.xpath(u'//a[contains(text(), "%s")]' % page)) @pagination @method class iter_history(TableElement): item_xpath = '//table/tbody/tr[contains(@class, "results")]' head_xpath = '//table/thead/tr/th' col_date = re.compile('Date') col_label = re.compile(u'Libellé') col_amount = re.compile('Montant') next_page = Link('//a[contains(text(), "Suivant") and not(contains(@href, "javascript"))]', default=None) class item(ItemElement): klass = Transaction obj_raw = Transaction.Raw(TableCell('label')) obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_amount = MyDecimal(TableCell('amount')) @method class iter_investment(TableElement): item_xpath = '//table/tbody/tr[contains(@class, "results")]' head_xpath = '//table/thead/tr/th' col_label = re.compile(u'Libellé') col_quantity = re.compile('Nb parts') col_vdate = re.compile('Date VL') col_unitvalue = re.compile('VL') col_unitprice = re.compile('Prix de revient') col_valuation = re.compile('Solde') class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_code = Regexp(Link('./td/a'), 'Isin%253D([^%]+)') obj_quantity = MyDecimal(TableCell('quantity')) obj_unitprice = MyDecimal(TableCell('unitprice')) obj_unitvalue = MyDecimal(TableCell('unitvalue')) obj_valuation = MyDecimal(TableCell('valuation')) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True, default=NotAvailable) class MarketPage(LoggedPage, HTMLPage): def get_list(self, acclabel): # Check if history is present if CleanText(default=None).filter(self.doc.xpath('//body/p[contains(text(), "indisponible pour le moment")]')): return False for a in self.doc.xpath('//a[contains(@onclick, "indiceCompte")]'): if CleanText().filter(a.xpath('.')) == acclabel: ids = re.search('indiceCompte[^\d]+(\d+).*idRacine[^\d]+(\d+)', \ Attr(None, 'onclick').filter(a.xpath('.'))).groups() break form = self.get_form(name="formCompte") form ['indiceCompte'] = ids[0] form ['idRacine'] = ids[1] return form.submit() def get_full_list(self): form = self.get_form(name="formOperation") form['dateDebut'] = "02/01/1970" try: return form.submit() except ServerError: return False @method class iter_history(TableElement): item_xpath = '//table[has-class("domifrontTb")]/tr[not(has-class("LnTit") or has-class("LnTot"))]' head_xpath = '//table[has-class("domifrontTb")]/tr[1]/td' col_date = re.compile('Date') col_label = u'Opération' col_code = u'Code' col_quantity = u'Quantité' col_amount = re.compile('Montant') class item(ItemElement): klass = Transaction obj_label = CleanText(TableCell('label')) obj_type = Transaction.TYPE_BANK obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_amount = CleanDecimal(TableCell('amount')) obj_investments = Env('investments') def parse(self, el): i = Investment() i.label = Field('label')(self) i.code = CleanText(TableCell('code'))(self) i.quantity = MyDecimal(TableCell('quantity'))(self) i.valuation = Field('amount')(self) i.vdate = Field('date')(self) self.env['investments'] = [i] @method class iter_investment(TableElement): item_xpath = '//table/tr[not(has-class("LnTit") or has-class("LnTot"))]' head_xpath = '//table/tr[1]/td' col_label = u'Valeur' col_code = u'Code' col_quantity = u'Qté' col_vdate = u'Date cours' col_unitvalue = u'Cours' col_unitprice = re.compile('P.R.U') col_valuation = u'Valorisation' class item(ItemElement): klass = Investment obj_label = Upper(TableCell('label')) obj_code = CleanText(TableCell('code')) obj_quantity = CleanDecimal(TableCell('quantity'), default=NotAvailable) obj_unitprice = MyDecimal(TableCell('unitprice')) obj_unitvalue = MyDecimal(TableCell('unitvalue')) obj_valuation = CleanDecimal(TableCell('valuation')) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True, default=NotAvailable) def condition(self): return not CleanText('//div[has-class("errorConteneur")]', default=None)(self) weboob-1.2/modules/cmso/pro/000077500000000000000000000000001303450110500160325ustar00rootroot00000000000000weboob-1.2/modules/cmso/pro/__init__.py000066400000000000000000000000001303450110500201310ustar00rootroot00000000000000weboob-1.2/modules/cmso/pro/browser.py000066400000000000000000000117701303450110500200750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 smurail # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from dateutil.relativedelta import relativedelta from itertools import chain from weboob.exceptions import BrowserHTTPError, BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.exceptions import ServerError from weboob.tools.date import LinearDateGuesser from .pages import LoginPage, AccountsPage, HistoryPage, ChoiceLinkPage, SubscriptionPage, InvestmentPage, InvestmentAccountPage, UselessPage class CmsoProBrowser(LoginBrowser): login = URL('/banque/assurance/credit-mutuel/pro/accueil\?espace=professionnels', LoginPage) choice_link = URL('/domiweb/accueil.jsp', ChoiceLinkPage) subscription = URL('/domiweb/prive/espacesegment/selectionnerAbonnement/0-selectionnerAbonnement.act', SubscriptionPage) accounts = URL('/domiweb/prive/professionnel/situationGlobaleProfessionnel/0-situationGlobaleProfessionnel.act', AccountsPage) history = URL('/domiweb/prive/professionnel/situationGlobaleProfessionnel/1-situationGlobaleProfessionnel.act', HistoryPage) useless = URL('/domiweb/prive/particulier/modificationMotDePasse/0-expirationMotDePasse.act', UselessPage) investment = URL('/domiweb/prive/particulier/portefeuilleSituation/0-situationPortefeuille.act', InvestmentPage) invest_account = URL(r'/domiweb/prive/particulier/portefeuilleSituation/2-situationPortefeuille.act\?indiceCompte=(?P\d+)&idRacine=(?P\d+)', InvestmentAccountPage) def __init__(self, website, *args, **kwargs): super(CmsoProBrowser, self).__init__(*args, **kwargs) self.BASEURL = "https://www.%s" % website self.areas = None def do_login(self): self.login.stay_or_go() try: self.page.login(self.username, self.password) except BrowserHTTPError as e: # Yes, I know... In the Wild Wild Web, nobody respects nothing if e.response.status_code == 500: raise BrowserIncorrectPassword() else: raise self.fetch_areas() def fetch_areas(self): if self.areas is None: self.subscription.go() @need_login def iter_accounts(self): self.fetch_areas() # Manage multiple areas if not self.areas: raise BrowserIncorrectPassword("Vous n'avez pas de comptes sur l'espace professionnel de ce site.") for area in self.areas: self.subscription.stay_or_go() self.location(area) try: for a in self.accounts.go().iter_accounts(): a._area = area yield a except ServerError: self.logger.warning('Area not unavailable.') @need_login def iter_history(self, account): self.fetch_areas() if account._history_url.startswith('javascript:'): raise NotImplementedError() # Manage multiple areas self.subscription.go() self.location(account._area) self.accounts.go() # Query history for 6 last months def format_date(d): return datetime.date.strftime(d, '%d/%m/%Y') today = datetime.date.today() period = (today - relativedelta(months=6), today) query = {'date': ''.join(map(format_date, period))} # Let's go self.location(account._history_url) first_page = self.page rest_page = self.history.go(data=query) date_guesser = LinearDateGuesser() return chain(first_page.iter_history(date_guesser=date_guesser), reversed(list(rest_page.iter_history(date_guesser=date_guesser)))) @need_login def iter_coming(self, account): raise NotImplementedError() @need_login def iter_investment(self, account): self.fetch_areas() self.investment.go() assert self.investment.is_here() for page_account in self.page.iter_accounts(): if page_account.id == account.id: self.page.go_account(*page_account._formdata) break else: # not an investment account return [] assert self.invest_account.is_here() invests = list(self.page.iter_investments()) assert len(invests) < 2, 'implementation should be checked with more than 1 investment' # FIXME return invests weboob-1.2/modules/cmso/pro/pages.py000066400000000000000000000233011303450110500175020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 smurail # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import datetime from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ListElement, ItemElement, TableElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, DateGuesser, Env, Field, Filter, Regexp, TableCell from weboob.browser.filters.html import Link, Attr from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction __all__ = ['LoginPage'] class UselessPage(HTMLPage): pass class ChoiceLinkPage(HTMLPage): def on_load(self): link_line = self.doc.xpath('//script')[-1].text m = re.search(r'lien\("(.*)"', link_line) if m: self.browser.location(m.group(1)) class SubscriptionPage(HTMLPage): def on_load(self): if u"Vous ne disposez d'aucun contrat sur cet accès." in CleanText(u'.')(self.doc): raise BrowserIncorrectPassword() self.browser.areas = [] for div in self.doc.xpath('//div[@class="listeAbonnementsBox"]'): site_type = div.xpath('./div[1]')[0].text if site_type != 'Particulier': for link in div.xpath('./div/@onclick'): m = re.search(r"href='(.*)'", link) if m: self.browser.areas.append(m.group(1)) class LoginPage(HTMLPage): def login(self, username, password): form = self.get_form('//form[@id="formAuth"]') form['noPersonne'] = username form['motDePasse'] = password[:16] form.submit() class CMSOPage(HTMLPage): @property def logged(self): if len(self.doc.xpath('//b[text()="Session interrompue"]')) > 0: return False return True class CmsoListElement(ListElement): item_xpath = '//table[@class="Tb" and tr[1][@class="LnTit"]]/tr[@class="LnA" or @class="LnB"]' class AccountsPage(CMSOPage): TYPES = {u'COMPTE CHEQUES': Account.TYPE_CHECKING, } @method class iter_accounts(CmsoListElement): class item(ItemElement): klass = Account class Type(Filter): def filter(self, label): for pattern, actype in AccountsPage.TYPES.iteritems(): if label.startswith(pattern): return actype return Account.TYPE_UNKNOWN obj__history_url = Link('./td[1]/a') obj_label = CleanText('./td[1]') obj_id = Env('id', default=None) obj_balance = CleanDecimal('./td[2]', replace_dots=True) obj_type = Type(Field('label')) # Last numbers replaced with XX... or we have to send sms to get RIB. obj_iban = NotAvailable def validate(self, obj): if obj.id is None: obj.id = obj.label.replace(' ', '') return True def parse(self, el): history_url = Field('_history_url')(self) if history_url.startswith('javascript:'): # Market account page = self.page.browser.investment.go() for tr in page.doc.xpath('.//table/tr[not(has-class("LnTit")) and not(has-class("LnTot"))]'): # Try to match account with id and balance. if CleanText('./td[2]//a')(tr) == Field('label')(self) \ and CleanDecimal('./td[3]//a')(tr) == Field('balance')(self): assert 'id' not in self.env self.env['id'] = CleanText('./td[1]', replace=[(' ', '')])(tr) else: page = self.page.browser.open(history_url).page self.env['id'] = Regexp(CleanText('//span[has-class("Rappel")]'), '(\d{18}) | (\d{3}\w\d{15})')(page.doc) def on_load(self): if self.doc.xpath('//p[contains(text(), "incident technique")]'): raise BrowserIncorrectPassword("Vous n'avez aucun compte sur cet espace. " \ "Veuillez choisir un autre type de compte.") class InvestmentPage(CMSOPage): @method class iter_accounts(CmsoListElement): class item(ItemElement): klass = Account obj_id = Regexp(CleanText('./td[1]'), r'(\d+)\s*(\d+)', r'\1\2') def obj__formdata(self): js = Attr('./td/a[1]', 'onclick')(self) args = re.search(r'\((.*)\)', js).group(1).split(',') form = args[0].strip().split('.')[1] idx = args[2].strip() idroot = args[4].strip().replace("'", "") return (form, idx, idroot) def go_account(self, form, idx, idroot): form = self.get_form(name=form) form['indiceCompte'] = idx form['idRacine'] = idroot form.submit() class CmsoTableElement(TableElement): head_xpath = '//table[has-class("Tb")]/tr[has-class("LnTit")]/td' item_xpath = '//table[has-class("Tb")]/tr[has-class("LnA") or has-class("LnB")]' class InvestmentAccountPage(CMSOPage): @method class iter_investments(CmsoTableElement): col_label = 'Valeur' col_isin = 'Code' col_quantity = u'Qté' col_unitvalue = 'Cours' col_valuation = 'Valorisation' class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_code = CleanText(TableCell('isin')) obj_quantity = CleanDecimal(TableCell('quantity'), replace_dots=(',', '.')) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=('', ',')) obj_valuation = CleanDecimal(TableCell('valuation'), replace_dots=(' ', '.')) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^RET DAB (?P
\d{2})/?(?P\d{2})(/?(?P\d{2}))? (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('CARTE (?P
\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?PVIR(EMEN)?T? (SEPA)?(RECU|FAVEUR)?)( /FRM)?(?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^PRLV (?P.*)( \d+)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^(CHQ|CHEQUE) .*$'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(CONVENTION \d+ |F )?COTIS(ATION)? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class CmsoTransactionElement(ItemElement): klass = Transaction def condition(self): return len(self.el) >= 5 and not self.el.get('id', '').startswith('libelleLong') class HistoryPage(CMSOPage): def iter_history(self, *args, **kwargs): if self.doc.xpath('//a[@href="1-situationGlobaleProfessionnel.act"]'): return self.iter_history_rest_page(*args, **kwargs) return self.iter_history_first_page(*args, **kwargs) @method class iter_history_first_page(CmsoListElement): class item(CmsoTransactionElement): def validate(self, obj): return obj.date >= datetime.date.today().replace(day=1) def date(selector): return DateGuesser(CleanText(selector), Env('date_guesser')) | Transaction.Date(selector) obj_date = date('./td[1]') obj_vdate = date('./td[2]') # Each row is followed by a "long labelled" version obj_raw = Transaction.Raw('./following-sibling::tr[1][starts-with(@id, "libelleLong")]/td[3]') obj_amount = Transaction.Amount('./td[5]', './td[4]') def condition(self): return len(self.el) >= 5 and not self.el.get('id', '').startswith('libelleLong') and len(self.el.xpath('.//i')) > 0 @pagination @method class iter_history_rest_page(CmsoListElement): next_page = Link('//span[has-class("Rappel")]/following-sibling::*[1][@href]') class item(CmsoTransactionElement): obj_date = Transaction.Date('./td[2]') obj_vdate = Transaction.Date('./td[1]') obj_raw = Transaction.Raw('./td[3]') obj_amount = Transaction.Amount('./td[5]', './td[4]') weboob-1.2/modules/colisprive/000077500000000000000000000000001303450110500164505ustar00rootroot00000000000000weboob-1.2/modules/colisprive/__init__.py000066400000000000000000000014441303450110500205640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ColispriveModule __all__ = ['ColispriveModule'] weboob-1.2/modules/colisprive/browser.py000066400000000000000000000021521303450110500205050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import TrackPage, ErrorPage class ColispriveBrowser(PagesBrowser): track_page = URL('https://www.colisprive.com/moncolis/pages/detailColis.aspx\?numColis=(?P.+)', TrackPage) error_page = URL('https://www.colisprive.fr', ErrorPage) def get_tracking_info(self, _id): return self.track_page.go(id=_id).get_info(_id) weboob-1.2/modules/colisprive/favicon.png000066400000000000000000000070201303450110500206020ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME  0yiTXtCommentCreated with GIMPd.ebKGDg!}j1t tIDATx[r^ ")Jg3᣿'}vm>wghFE7]WPh@f$э|2+k?|,IFNAJReɓ q]wq%㨐QZMe'Q=G 6Igޞ€k7Bt!Je>CRgFD _-\+EA*>2 o0 s8q³Ų]DFn;.1ϓʧr rPh3| "[ H} YߝORJDs"U=O 0g4nP^Ou]= \pvx 9>I8| X*婕BRFɹ#fFr,CתjiyB !8_O}*d^RJN3\ BAyU.uT0lF...L#Y!a3k KT޺b|X4JU*sRr\]]̘v`ԅz͹V =;;3(Ʈ!)S}42ֈkib }rOap ild;ΙѴ`CtH,#m`XǖJx&[ڀl]Gv|ֱҷ4OCKf086 >fH AP%A l/)y5H)\_~GA)B_&fhk5O^ZMGh,  h oI>#J^}4c\߆pLž ?F< 1D, Opߛ0>߂6فkp@\2s\+/{9#CI(ήb<+s~VR0@V B\ƮɎ2nVeݼt.ҹ'|W.=􎹫,xƑ?J;#&D_/U#,XDe7cϑ}*t2dt\wU+jFsդNܢH ?0|sT,6x̆D# A {%U ʪp]e W5U j_QE^ހLG8Jp2nܛ3~/)IzRfV62nw*UU`F0.ecIynqh[L|Ɓs6^svW):-/96'Q1[V72e.VE~"d=hp{J^'0S9jM3~2N,z9[l.ia d ow"Xq8p!!I2v:^b#f nlRPiՉ~{yq[wu4]~i?z W4$ztmi3Hq+ι-vMx' $(uſ9D%iYL#[=tij=E3KoNG&9fN"^U޻}{ԄzYWSB`,GF31 @|/铣֓FtuE ԭ޼/Sueh,"va%Kk1bτ kȍB q9X>+x^]7Ro _ cC56~$tr+{QWI1BZnQSx*y6GB@!>VnoaCE/dkN`?3\YzQINT7)I !@$={v?L1nU^Z]]o X V8+~3ľc8osYuK9{۹CIENDB`weboob-1.2/modules/colisprive/module.py000066400000000000000000000023531303450110500203120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.parcel import CapParcel from .browser import ColispriveBrowser __all__ = ['ColispriveModule'] class ColispriveModule(Module, CapParcel): NAME = 'colisprive' DESCRIPTION = u'Colisprive parcel tracking website' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' BROWSER = ColispriveBrowser def get_parcel_tracking(self, _id): return self.browser.get_tracking_info(_id) weboob-1.2/modules/colisprive/pages.py000066400000000000000000000053441303450110500201270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from weboob.browser.pages import HTMLPage from weboob.capabilities.parcel import Parcel, Event, ParcelNotFound def update_status(p, status): if p.status < status: p.status = status class TrackPage(HTMLPage): def get_info(self, _id): p = Parcel(_id) statustr = self.doc.xpath('//tr[@class="bandeauText"]')[0] status = statustr.xpath('td')[1].text p.info = status p.status = p.STATUS_UNKNOWN p.history = [] for i, tr in enumerate(self.doc.xpath('//table[@class="tableHistoriqueColis"]//tr[@class="bandeauText"]')): tds = tr.findall('td') try: if tds[0].attrib['class'] != "tdText": continue except: continue ev = Event(i) ev.location = None ev.activity = tds[1].text if u"Votre colis a été expédié par votre webmarchand" in ev.activity: update_status(p, p.STATUS_PLANNED) elif u"Votre colis est pris en charge par Colis Privé" in ev.activity: update_status(p, p.STATUS_IN_TRANSIT) elif u"Votre colis est arrivé sur notre agence régionale" in ev.activity: update_status(p, p.STATUS_IN_TRANSIT) elif u"Votre colis est en cours de livraison" in ev.activity: update_status(p, p.STATUS_IN_TRANSIT) elif u"Votre colis a été livré" in ev.activity: update_status(p, p.STATUS_ARRIVED) ev.date = date(*reversed([int(x) for x in tds[0].text.split('/')])) p.history.append(ev) try: datelivre = self.doc.xpath('//div[@class="NoInstNoRecla"]') clean = datelivre[0].text if "Votre colis a déja été livré" in clean: p.status = p.STATUS_ARRIVED except: pass return p class ErrorPage(HTMLPage): def get_info(self, _id): raise ParcelNotFound("No such ID: %s" % _id) weboob-1.2/modules/colissimo/000077500000000000000000000000001303450110500162725ustar00rootroot00000000000000weboob-1.2/modules/colissimo/__init__.py000066400000000000000000000014421303450110500204040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ColissimoModule __all__ = ['ColissimoModule'] weboob-1.2/modules/colissimo/browser.py000066400000000000000000000040521303450110500203300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.parcel import Event, ParcelNotFound from weboob.browser import PagesBrowser, URL from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Date from weboob.browser.pages import HTMLPage from weboob.browser.profiles import Firefox __all__ = ['ColissimoBrowser'] class TrackingPage(HTMLPage): ENCODING = 'iso-8859-15' @method class iter_infos(ListElement): item_xpath = '//table[@class="dataArray"]/tbody/tr' class item(ItemElement): klass = Event obj_date = Date(CleanText('td[@headers="Date"]')) obj_activity = CleanText('td[@headers="Libelle"]') obj_location = CleanText('td[@headers="site"]') def get_error(self): return CleanText("//div[@class='error']")(self.doc) class ColissimoBrowser(PagesBrowser): BASEURL = 'http://www.colissimo.fr' PROFILE = Firefox() tracking_url = URL('/portail_colissimo/suivre.do\?colispart=(?P<_id>.*)', TrackingPage) def get_tracking_info(self, _id): self.tracking_url.stay_or_go(_id=_id) events = list(self.page.iter_infos()) if len(events) == 0: error = self.page.get_error() raise ParcelNotFound(u"Parcel not found: {}".format(error)) return events weboob-1.2/modules/colissimo/favicon.png000066400000000000000000000016571303450110500204360ustar00rootroot00000000000000PNG  IHDR@@iqbKGDC pHYs  tIME %tEXtCommentCreated with GIMPWIDATxMSUu)0c ,$ !] ĄH"ΐ@\1b Yą ʅ 'WP1*c3tE_fھyc4}==1c1c1cy2O3FF/%)_y.Э+0 |T붸PDDd8v ;:@ݻ]  AfWgwkĹ;5x'RvǍ+s|k|#qgD oE)rP~T֎(P=l "CAHm7Ԛ9fmo%;sy+\ȃ a!AS.BzX_`rwyB`D Ljeߙ|ns x?}rm:"8ɉsWBǚ[][e9[cr:6Ey /}W7.ZwFj*a[JՓ!)TFKiGJuyuft1iDҜ^ ~oYk2fz pde}2H7 n 5@.ks:komDq AЂLvL@/}&__bUeyzs8Z+[i֎,9\TMDX-ԃKTr=ȱk oΣ,r=xnZ~ec1c1c1<):ȉqIENDB`weboob-1.2/modules/colissimo/module.py000066400000000000000000000035071303450110500201360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.parcel import CapParcel, ParcelNotFound, Parcel from weboob.tools.backend import Module from .browser import ColissimoBrowser __all__ = ['ColissimoModule'] class ColissimoModule(Module, CapParcel): NAME = 'colissimo' DESCRIPTION = u'Colissimo parcel tracking website' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' BROWSER = ColissimoBrowser def get_parcel_tracking(self, _id): # 13 is the magic length of colissimo tracking ids if len(_id) != 13: raise ParcelNotFound(u"Colissimo ID's must have 13 print character") events = self.browser.get_tracking_info(_id) p = Parcel(_id) p.history = events first = events[0] p.info = first.activity if u"remis au gardien ou" in p.info or u"Votre colis est livré" in p.info: p.status = p.STATUS_ARRIVED elif u"pas encore pris en charge par La Poste" in p.info: p.status = p.STATUS_PLANNED else: p.status = p.STATUS_IN_TRANSIT return p weboob-1.2/modules/cpasbien/000077500000000000000000000000001303450110500160555ustar00rootroot00000000000000weboob-1.2/modules/cpasbien/__init__.py000066400000000000000000000001011303450110500201560ustar00rootroot00000000000000from .module import CpasbienModule __all__ = ['CpasbienModule'] weboob-1.2/modules/cpasbien/browser.py000066400000000000000000000031331303450110500201120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, TorrentPage#, HomePage __all__ = ['CpasbienBrowser'] class CpasbienBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.cpasbien.cm/' search = URL('recherche/(?P.*).html,trie-seeds-d', SearchPage) torrent = URL('dl-torrent/(?P.*)\.html', TorrentPage) def iter_torrents(self, pattern): self.search.go(pattern=pattern) return self.page.iter_torrents() def get_torrent(self, fullid): try: self.torrent.go(id=fullid) torrent = self.page.get_torrent() return torrent except BrowserHTTPNotFound: return weboob-1.2/modules/cpasbien/favicon.png000066400000000000000000000055731303450110500202220ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME 3Q}( IDATx{\U?,v[`{KbXD^ix$1ڂ?  H4!AR"5)T^WKJRG@ {,m91{f:۝ٝ{}.̭5] }Bl4;f?l mی=&>1֪1Zޏ~RLXڛH>s4d>R(lhIޘ hoX3@h,#B`6|x# H\q  G7}9=BFf"OU v|cr>졀Oȱ~BIP˹iO DZ.Qo,1F2>PV*oGx7a|T$xCL:::j8pH:oq <4ms SOv3@:kXz׹dʹ"sDs<0s8ԌRmUjxcI+d(/N(q!a!@(5E<T(zt`(T7fT06|cCuP?މR .1Y ;tT%M3OF*UC֗ɓ>E5; vS([QͤS0kz p0k= C_w& Nef B1:Qe>lցv9h6yﱞ_p1dɒXȼ=Cn`eG%kwbPJ5n[9E2V~fgN0-J@ɐi^u=;a64 ݹ3ݻsg^57\XYYx'rߘ?c$IɵO  Uڱi8x` *x8}:w(8ʋ)h%<IS`#[%Um%`~H{asҥj**׮]ݻY~}u5@'tTBleg˫  _%y{5'hX?0Pfxg+7n+m޼L_c;^9" O4ϰGys9 SEZj-֣և_?b={f͚)@13FnGu;pH3u#ZR km$desN}i]xǭnƺKQ. G<}\Yw.Ӆ,jUb|cK4CD .lڴ7|L@Jɐ"%a9cpIv6 ƬǺ8JVO)UvһzwoL9Y OxjZ40T)XkJV#><^h}=̾R(EVDb-%o-eP*zUr@9, l<^ɺ7  ĕR9tH*~[*,Sj"_~>lQx*@ Q:K, b10|1׀~`AS|$"UD p`n:=?.Gc@ P.Sm u`Yh a '\(N7٫-Zjz`\w`amTXEs[R)HEYT+c) ߘ|òoLm`L̗~W>kW|ŋ2_SB,2!Q()RՌ6Wj8)$gwrvg%0atuw}/!:u^>d?-DHe2 נ$)#TѸoL6״YUYڧ@w܉*gQޟP;~YN$iaK~E J@ׅTu mpO%0 88uC%Ń-)_1从09g-1ɪ^D7ˀzM@R:m@oL\IAvo\/߭$񜈑fGQ3;0|c}Li=`oG֏3̋1q-0O|YMůę2+|cV v KYMyQ(:ׇ^a*8 SmEN:Oͺ/ as8|}z]ϊ51W9%$kxM}[@3TuXcrjk$T)+ǣ.ߘo8ZeӌJxg<'|ijFtc8넽o hBYc*[w )ͯ5ܚ[sknͭzIIENDB`weboob-1.2/modules/cpasbien/module.py000066400000000000000000000034501303450110500177160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.torrent import CapTorrent, Torrent from weboob.tools.backend import Module from .browser import CpasbienBrowser from urllib import quote_plus __all__ = ['CpasbienModule'] class CpasbienModule(Module, CapTorrent): NAME = 'cpasbien' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Cpasbien Torrents BitTorrent tracker' LICENSE = 'AGPLv3+' BROWSER = CpasbienBrowser def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None resp = self.browser.open(torrent.url) return resp.content def iter_torrents(self, pattern): return self.browser.iter_torrents(quote_plus(pattern.encode('utf-8'))) def fill_torrent(self, torrent, fields): if 'description' in fields or 'files' in fields: torrent = self.browser.get_torrent(torrent.id) return torrent OBJECTS = { Torrent: fill_torrent } weboob-1.2/modules/cpasbien/pages.py000066400000000000000000000075311303450110500175340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.tools.misc import get_bytes_size from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText, Type, Format from weboob.browser.filters.html import CleanHTML class SearchPage(HTMLPage): @method class iter_torrents(ListElement): item_xpath = '//div[has-class("ligne0") or has-class("ligne1")]' class item(ItemElement): klass = Torrent obj_id = Regexp(CleanText('.//a[has-class("titre")]/@href'), '.*dl-torrent/(.*).html') obj_name = CleanText('.//a[has-class("titre")]', default=NotAvailable) obj_magnet = NotAvailable obj_seeders = CleanText('.//div[has-class("up")]', default=NotAvailable) & Type(type=int) obj_leechers = CleanText('.//div[has-class("down")]', default=NotAvailable) & Type(type=int) obj_description = NotLoaded obj_files = NotLoaded def obj_url(self): href = CleanText('.//a[has-class("titre")]/@href')(self) subid = href.split('/')[-1].replace('.html','.torrent') return 'http://www.cpasbien.cm/telechargement/%s'%subid def obj_size(self): rawsize = CleanText('./div[has-class("poid")]')(self) rawsize = rawsize.replace(',','.').strip() nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper().replace('O','B') size = get_bytes_size(nsize,usize) return size obj_filename = Format('%s.torrent', Regexp( CleanText('.//a[has-class("titre")]/@href'), '/([^/]*)\.html') ) class TorrentPage(HTMLPage): @method class get_torrent(ItemElement): klass = Torrent obj_name = CleanText('//h2[has-class("h2fiche")]', default=NotAvailable) obj_description = CleanHTML('//div[@id="textefiche"]', default=NotAvailable) obj_seeders = CleanText('//div[@id="infosficher"]//span[has-class("seed_ok")]') & Type(type=int) obj_leechers = CleanText('(//div[@id="infosficher"]/span)[3]') & Type(type=int) obj_magnet = NotAvailable obj_id = Regexp(CleanText('//h2[has-class("h2fiche")]/a/@href'), '.*dl-torrent/(.*).html') obj_url = Format('http://www.cpasbien.cm%s', CleanText('//a[@id="telecharger"]/@href')) def obj_size(self): rawsize = CleanText('(//div[@id="infosficher"]/span)[1]')(self) rawsize = rawsize.replace(',','.').strip() nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper().replace('O','B') size = get_bytes_size(nsize,usize) return size obj_files = NotAvailable obj_filename = CleanText(Regexp(CleanText('//a[@id="telecharger"]/@href'), '.*telechargement/(.*)'), default=NotAvailable) weboob-1.2/modules/cpasbien/test.py000066400000000000000000000035251303450110500174130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded import urllib from random import choice class CpasbienTest(BackendTest): MODULE = 'cpasbien' def test_torrent(self): torrents = list(self.backend.iter_torrents('spiderman')) for torrent in torrents: path, qs = urllib.splitquery(torrent.url) assert path.endswith('.torrent') if qs: assert torrent.filename assert torrent.id assert torrent.name assert torrent.description is NotLoaded full_torrent = self.backend.get_torrent(torrent.id) # do not assert torrent.name is full_torrent.name # (or even that one contains another), it isn't always true! assert full_torrent.name assert full_torrent.url assert full_torrent.description is not NotLoaded # get the file of a random torrent # from the list (getting them all would be too long) if len(torrents): torrent = choice(torrents) self.backend.get_torrent_file(torrent.id) weboob-1.2/modules/cragr/000077500000000000000000000000001303450110500153675ustar00rootroot00000000000000weboob-1.2/modules/cragr/__init__.py000066400000000000000000000014331303450110500175010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CragrModule __all__ = ['CragrModule'] weboob-1.2/modules/cragr/favicon.png000066400000000000000000000043411303450110500175240ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDוBM pHYs  tIME *9CaIDATxpT?/?MM7ԲZ+QM*BǢ#8eT?૩q:jb+e NձӾN鯴uXM$!?wv|M۽mNNɽs=|Ϲr!bhA'V7T@0&*:"a[mwNa#w*OD:V@u}\IQ@^G_VϺHWns|XAq' KJ<X/J3|ma9~ Xaz` T7^?~eqkj f*%*s0`c?D,Gg܎ 6`qX < 2R</ * OxxhJʳ [4r 52P_MCM#|  %.R?#v~iT<(<79 lt$UI@X脣%#`UҀkкx,5V$i!>-rCi. /@`NMCo`+ƖP)\-"˥bR .?)`ܴ y4e>/vU٧ ,x,qb/ Pi yB GC)w[=TNE[X7Cx,P:{ 3QReѷ_Spu wɱ;$N|ʅFn{1*4ۃ6Ewʙ w>`}hh xh}c=)`$ kX~*VȗF= zx4 .pJ MR/ʲRdiXZ*|#K4VlKTIi4lJrR ,þT-V$ h k\uXof@C$v8$B&$e hA,+K,S :$p羦HC ..@IF󚼄Ohhe HI9~ү59R? (i4'_Q>Z*Rq X}<I}4?R-bVة9筡rn9Պݝl1*6=oϑlUSڎi=?ln\Bv7^w@Pr$-s]ܦ 7]*P0}5 " SԉRx1K 2("V oS[|{P>;n'FL$(\NE%v&*_b!'tyZZ9r8e\JTlOH_Cy )JMþXɸQ@8Jݩ\a^Ov|#뜔GM~Cz dmuv&ľcm<T 9\+5* ǯL&X XU;qV7z'wgՔi&杞 bݟ!}dVM>s-iI$In `s<8u5UD@P%v\Y˒rMce!d8r߸VFIENDB`weboob-1.2/modules/cragr/mobile/000077500000000000000000000000001303450110500166365ustar00rootroot00000000000000weboob-1.2/modules/cragr/mobile/__init__.py000066400000000000000000000000001303450110500207350ustar00rootroot00000000000000weboob-1.2/modules/cragr/mobile/browser.py000066400000000000000000000257741303450110500207120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2013 Romain Bignon, Xavier Guerrin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.tools.date import LinearDateGuesser from weboob.capabilities.bank import Transfer, TransferError from .pages import LoginPage, AccountsList import mechanize from datetime import datetime import re __all__ = ['CragrMobile'] class CragrMobile(Browser): PROTOCOL = 'https' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] # a session id that is sometimes added, and should be ignored when matching pages SESSION_REGEXP = '(?:|%s[A-Z0-9]+)' % re.escape(r';jsessionid=') is_logging = False def __init__(self, website, *args, **kwargs): self.DOMAIN = website self.PAGES = {'https://[^/]+/': LoginPage, 'https://[^/]+/.*\.c.*': AccountsList, 'https://[^/]+/login/process%s' % self.SESSION_REGEXP: AccountsList, 'https://[^/]+/accounting/listAccounts': AccountsList, 'https://[^/]+/accounting/listOperations': AccountsList, 'https://[^/]+/accounting/showAccountDetail.+': AccountsList, 'https://[^/]+/accounting/showMoreAccountOperations.*': AccountsList, } Browser.__init__(self, *args, **kwargs) def viewing_html(self): """ As the fucking HTTP server returns a document in unknown mimetype 'application/vnd.wap.xhtml+xml' it is not recognized by mechanize. So this is a fucking hack. """ return True def is_logged(self): logged = self.page and self.page.is_logged() or self.is_logging self.logger.debug('logged: %s' % (logged and 'yes' or 'no')) return logged def login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) # Do we really need to login? if self.is_logged(): self.logger.debug('already logged in') return self.is_logging = True # Are we on the good page? if not self.is_on_page(LoginPage): self.logger.debug('going to login page') Browser.home(self) self.logger.debug('attempting to log in') self.page.login(self.username, self.password) self.is_logging = False if not self.is_logged(): raise BrowserIncorrectPassword() self.addheaders = [ ['User-agent', self.USER_AGENTS['desktop_firefox']] ] def get_accounts_list(self): self.logger.debug('accounts list required') self.home() return self.page.get_list() def home(self): """ Ensure we are both logged and on the accounts list. """ self.logger.debug('accounts list page required') if self.is_on_page(AccountsList) and self.page.is_accounts_list(): self.logger.debug('already on accounts list') return # simply go to http(s)://the.doma.in/ Browser.home(self) if self.is_on_page(LoginPage): if not self.is_logged(): # So, we are not logged on the login page -- what about logging ourselves? self.login() # we assume we are logged in # for some regions, we may stay on the login page once we're # logged in, without being redirected... if self.is_on_page(LoginPage): # ... so we have to move by ourselves self.move_to_accounts_list() def move_to_accounts_list(self): """ For regions where you can stay on http(s)://the.doma.in/ while you are logged in, move to the accounts list """ self.location('%s://%s/accounting/listAccounts' % (self.PROTOCOL, self.DOMAIN)) def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == ('%s' % id): return a return None def get_history(self, account): # some accounts may exist without a link to any history page if account._link_id is None: return history_url = account._link_id operations_count = 0 # 1st, go on the account page self.logger.debug('going on: %s' % history_url) self.location('https://%s%s' % (self.DOMAIN, history_url)) if self.page is None: return # Some regions have a "Show more" (well, actually "Voir les 25 # suivants") link we have to use to get all the operations. # However, it does not show only the 25 next results, it *adds* them # to the current view. Therefore, we have to parse each new page using # an offset, in order to ignore all already-fetched operations. # This especially occurs on CA Centre. use_expand_url = bool(self.page.expand_history_page_url()) date_guesser = LinearDateGuesser() while True: # we skip "operations_count" operations on each page if we are in the case described above operations_offset = operations_count if use_expand_url else 0 for page_operation in self.page.get_history(date_guesser, operations_count, operations_offset): operations_count += 1 yield page_operation history_url = self.page.expand_history_page_url() if use_expand_url else self.page.next_page_url() if not history_url: break self.logger.debug('going on: %s' % history_url) self.location('https://%s%s' % (self.DOMAIN, history_url)) def dict_find_value(self, dictionary, value): """ Returns the first key pointing on the given value, or None if none is found. """ for k, v in dictionary.iteritems(): if v == value: return k return None def do_transfer(self, account, to, amount, reason=None): """ Transfer the given amount of money from an account to another, tagging the transfer with the given reason. """ # access the transfer page transfer_page_unreachable_message = u'Could not reach the transfer page.' self.home() if not self.page.is_accounts_list(): raise TransferError(transfer_page_unreachable_message) operations_url = self.page.operations_page_url() self.location('https://%s%s' % (self.DOMAIN, operations_url)) transfer_url = self.page.transfer_page_url() abs_transfer_url = 'https://%s%s' % (self.DOMAIN, transfer_url) self.location(abs_transfer_url) if not self.page.is_transfer_page(): raise TransferError(transfer_page_unreachable_message) source_accounts = self.page.get_transfer_source_accounts() target_accounts = self.page.get_transfer_target_accounts() # check that the given source account can be used if account not in source_accounts.values(): raise TransferError('You cannot use account %s as a source account.' % account) # check that the given source account can be used if to not in target_accounts.values(): raise TransferError('You cannot use account %s as a target account.' % to) # separate euros from cents amount_euros = int(amount) amount_cents = int((amount * 100) - (amount_euros * 100)) # let's circumvent https://github.com/jjlee/mechanize/issues/closed#issue/17 # using http://wwwsearch.sourceforge.net/mechanize/faq.html#usage adjusted_response = self.response().get_data().replace('
', '
') response = mechanize.make_response(adjusted_response, [('Content-Type', 'text/html')], abs_transfer_url, 200, 'OK') self.set_response(response) # fill the form self.select_form(nr=0) self['numCompteEmetteur'] = ['%s' % self.dict_find_value(source_accounts, account)] self['numCompteBeneficiaire'] = ['%s' % self.dict_find_value(target_accounts, to)] self['montantPartieEntiere'] = '%s' % amount_euros self['montantPartieDecimale'] = '%02d' % amount_cents if reason is not None: self['libelle'] = reason self.submit() # look for known errors content = unicode(self.response().get_data(), 'utf-8') insufficient_amount_message = u'Montant insuffisant.' maximum_allowed_balance_message = u'Solde maximum autorisé dépassé.' if content.find(insufficient_amount_message) != -1: raise TransferError('The amount you tried to transfer is too low.') if content.find(maximum_allowed_balance_message) != -1: raise TransferError('The maximum allowed balance for the target account has been / would be reached.') # look for the known "all right" message ready_for_transfer_message = u'Vous allez effectuer un virement' if not content.find(ready_for_transfer_message): raise TransferError('The expected message "%s" was not found.' % ready_for_transfer_message) # submit the last form self.select_form(nr=0) submit_date = datetime.now() self.submit() # look for the known "everything went well" message content = unicode(self.response().get_data(), 'utf-8') transfer_ok_message = u'Vous venez d\'effectuer un virement du compte' if not content.find(transfer_ok_message): raise TransferError('The expected message "%s" was not found.' % transfer_ok_message) # We now have to return a Transfer object # the final page does not provide any transfer id, so we'll use the submit date transfer = Transfer(submit_date.strftime('%Y%m%d%H%M%S')) transfer.amount = amount transfer.origin = account transfer.recipient = to transfer.date = submit_date return transfer #def get_coming_operations(self, account): # if not self.is_on_page(AccountComing) or self.page.account.id != account.id: # self.location('/NS_AVEEC?ch4=%s' % account._link_id) # return self.page.get_operations() weboob-1.2/modules/cragr/mobile/pages/000077500000000000000000000000001303450110500177355ustar00rootroot00000000000000weboob-1.2/modules/cragr/mobile/pages/__init__.py000066400000000000000000000015161303450110500220510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .accounts_list import AccountsList from .login import LoginPage __all__ = ['AccountsList', 'LoginPage'] weboob-1.2/modules/cragr/mobile/pages/accounts_list.py000066400000000000000000000410711303450110500231640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import date from weboob.capabilities.bank import Account from .base import CragrBasePage from .tokenextractor import TokenExtractor from weboob.tools.capabilities.bank.transactions import FrenchTransaction class Transaction(FrenchTransaction): PATTERNS = [ (re.compile('^(Vp|Vt|Vrt|Virt|Vir(ement)?)\s*(?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?P(Tip|Plt|Prlv|PRELEVT|Prelevement)\s*.*)', re.IGNORECASE), FrenchTransaction.TYPE_ORDER), (re.compile('^Cheque\s*(?P(No)?.*)', re.IGNORECASE), FrenchTransaction.TYPE_CHECK), (re.compile('^(?PRem\s*Chq\s*.*)', re.IGNORECASE), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^Ret(rait)?\s*Dab\s*((?P
\d{2})(?P\d{2})(?P\d{2}))?\s*(?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^Paiement\s*Carte\s*(?P
\d{2})(?P\d{2})(?P\d{2})\s*(?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_CARD), (re.compile('^(?P.*CAPITAL.*ECHEANCE.*)', re.IGNORECASE), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(\*\*)?(?P(frais|cotis(ation)?)\s*.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^(?PInterets\s*.*)', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^(?PPrelev\.\s*(C\.r\.d\.s\.|R\.s\.a\.|C\.a\.p\.s\.|C\.s\.g|P\.s\.))', re.IGNORECASE), FrenchTransaction.TYPE_BANK), (re.compile('^(ACH.)?CARTE (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_CARD), (re.compile('^RET.CARTE (?P.*)', re.IGNORECASE), FrenchTransaction.TYPE_WITHDRAWAL), ] class AccountsList(CragrBasePage): """ Unlike most pages used with the Browser class, this class represents several pages, notably accounts list, history and transfer. This is due to the Credit Agricole not having a clear pattern to identify a page based on its URL. """ def is_accounts_list(self): """ Returns True if the current page appears to be the page dedicated to list the accounts. """ # we check for the presence of a "mes comptes titres" link_id link = self.document.xpath('/html/body//a[contains(text(), "comptes titres")]') return bool(link) def is_account_page(self): """ Returns True if the current page appears to be a page dedicated to list the history of a specific account. """ # tested on CA Lorraine, Paris, Toulouse title_spans = self.document.xpath('/html/body//div[@class="dv"]/span') for title_span in title_spans: title_text = title_span.text_content().strip().replace("\n", '') if (re.match('.*Compte.*n.*[0-9]+', title_text, flags=re.IGNORECASE)): return True return False def is_transfer_page(self): """ Returns True if the current page appears to be the page dedicated to order transfers between accounts. """ source_account_select_field = self.document.xpath('/html/body//form//select[@name="numCompteEmetteur"]') target_account_select_field = self.document.xpath('/html/body//form//select[@name="numCompteBeneficiaire"]') return bool(source_account_select_field) and bool(target_account_select_field) def get_list(self): """ Returns the list of available bank accounts """ for div in self.document.getiterator('div'): if div.attrib.get('class', '') in ('dv', 'headline') and div.getchildren()[0].tag in ('a', 'br'): self.logger.debug("Analyzing div %s" % div) # Step 1: extract text tokens tokens = [] required_tokens = {} optional_tokens = {} token_extractor = TokenExtractor() for token in token_extractor.extract_tokens(div): self.logger.debug('Extracted text token: "%s"' % token) tokens.append(token) # Step 2: analyse tokens for token in tokens: if self.look_like_account_number(token): required_tokens['account_number'] = token elif self.look_like_amount(token): required_tokens['account_amount'] = token elif self.look_like_account_name(token): required_tokens['account_name'] = token elif self.look_like_account_owner(token): if 'account_owner' in optional_tokens and 'account_name' not in required_tokens: required_tokens['account_name'] = optional_tokens['account_owner'] optional_tokens['account_owner'] = token # Step 3: create account objects if len(required_tokens) >= 3: account = Account() account.label = required_tokens['account_name'] account.id = required_tokens['account_number'] account.balance = FrenchTransaction.clean_amount(required_tokens['account_amount']) account.currency = account.get_currency(required_tokens['account_amount']) # we found almost all required information to create an account object self.logger.debug('Found account %s with number %s and balance = %.2f' % (account.label, account.id, account.balance)) # we may have found the owner name too if optional_tokens.get('account_owner') is not None: # well, we could add it to the label, but is this really required? self.logger.debug(' the owner appears to be %s' % optional_tokens['account_owner']) # we simply lack the link to the account history... which remains optional first_link = div.find('a') if first_link is not None: account._link_id = first_link.get('href') self.logger.debug(' the history link appears to be %s' % account._link_id) else: account._link_id = None yield account def get_history(self, date_guesser, start_index=0, start_offset=0): """ Returns the history of a specific account. Note that this function expects the current page to be the one dedicated to this history. start_index is the id used for the first created operation. start_offset allows ignoring the `n' first Transactions on the page. """ # tested on CA Lorraine, Paris, Toulouse # avoir parsing the page as an account-dedicated page if it is not the case if not self.is_account_page(): return # Step 1: extract text tokens tokens = [] token_extractor = TokenExtractor() for div in self.document.getiterator('div'): if div.attrib.get('class', '') in ('dv'): self.logger.debug("Analyzing div %s" % div) for token in token_extractor.extract_tokens(div): self.logger.debug('Extracted text token: "%s"' % token) tokens.append(token) # Step 2: convert tokens into operations # Notes: # * the code below expects pieces of information to be in the date-label-amount order; # could we achieve a heuristic smart enough to guess this order? # * unlike the former code, we parse every operation operations = [] current_operation = {} for token in tokens: self.logger.debug('Analyzing token: "%s"' % token) date_analysis = self.look_like_date_only(token) if date_analysis: current_operation = {} current_operation['date'] = date_analysis.groups()[0] else: date_desc_analysis = self.look_like_date_and_description(token) if date_desc_analysis: current_operation = {} current_operation['date'] = date_desc_analysis.groups()[0] current_operation['label'] = date_desc_analysis.groups()[1] elif self.look_like_amount(token): # we consider the amount is the last information we get for an operation current_operation['amount'] = FrenchTransaction.clean_amount(token) if current_operation.get('label') is not None and current_operation.get('date') is not None: self.logger.debug('Parsed operation: %s: %s: %s' % (current_operation['date'], current_operation['label'], current_operation['amount'])) operations.append(current_operation) current_operation = {} else: if current_operation.get('label') is not None: current_operation['label'] = u'%s %s' % (current_operation['label'], token) else: current_operation['label'] = token # Step 3: yield adequate transactions index = start_index for op in operations[start_offset:]: self.logger.debug('will yield the following transaction with index %d: %s: %s: %s' % (index, op['date'], op['label'], op['amount'])) transaction = Transaction(index) index += 1 transaction.amount = op['amount'] transaction.parse(self.date_from_string(op['date'], date_guesser), re.sub('\s+', ' ', op['label'])) yield transaction def get_transfer_accounts(self, select_name): """ Returns the accounts proposed for a transfer in a select field. This method assumes the current page is the one dedicated to transfers. select_name is the name of the select field to analyze """ if not self.is_transfer_page(): return False source_accounts = {} source_account_options = self.document.xpath('/html/body//form//select[@name="%s"]/option' % select_name) for option in source_account_options: source_account_value = option.get('value', -1) if (source_account_value != -1): matches = re.findall('^[A-Z0-9]+.*([0-9]{11}).*$', self.extract_text(option)) if matches: source_accounts[source_account_value] = matches[0] return source_accounts def get_transfer_source_accounts(self): return self.get_transfer_accounts('numCompteEmetteur') def get_transfer_target_accounts(self): return self.get_transfer_accounts('numCompteBeneficiaire') def expand_history_page_url(self): """ When on a page dedicated to list the history of a specific account (see is_account_page), returns the link to expand the history with 25 more results, or False if the link is not present. """ # tested on CA centre france a = self.document.xpath('/html/body//div[@class="headline"]//a[contains(text(), "Voir les 25 suivants")]') if not a: return False else: return a[0].get('href', '') def next_page_url(self): """ When on a page dedicated to list the history of a specific account (see is_account_page), returns the link to the next page, or False if the link is not present. """ # tested on CA Lorraine, Paris, Toulouse a = self.document.xpath('/html/body//div[@class="navlink"]//a[contains(text(), "Suite")]') if not a: return False else: return a[0].get('href', '') def operations_page_url(self): """ Returns the link to the "Opérations" page. This function assumes the current page is the accounts list (see is_accounts_list) """ link = self.document.xpath(u'/html/body//a[contains(text(), "Opérations")]') return link[0].get('href') def transfer_page_url(self): """ Returns the link to the "Virements" page. This function assumes the current page is the operations list (see operations_page_url) """ link = self.document.xpath('/html/body//a[@accesskey=1]/@href') return link[0] def extract_text(self, xml_elmt): """ Given an XML element, returns its inner text in a reasonably readable way """ data = u'' for text in xml_elmt.itertext(): data = data + u'%s ' % text data = re.sub(' +', ' ', data.replace("\n", ' ').strip()) return data def fallback_date(self): """ Returns a fallback, default date. """ return date(date.today().year, 1, 1) def date_from_string(self, string, date_guesser): """ Builds a date object from a 'DD/MM' string """ matches = re.search('\s*([012]?[0-9]|3[01])\s*/\s*(0?[1-9]|1[012])\s*$', string) if matches is None: return self.fallback_date() return date_guesser.guess_date(int(matches.group(1)), int(matches.group(2))) def look_like_account_owner(self, string): """ Returns a date object built from a given day/month pair. """ result = re.match('^\s*(M\.|Mr|Mme|Mlle|Mle|Monsieur|Madame|Mademoiselle)', string, re.IGNORECASE) self.logger.debug('Does "%s" look like an account owner? %s', string, ('yes' if result else 'no')) return result def look_like_account_name(self, string): """ Returns True of False depending whether string looks like an account name. """ result = (len(string) >= 3 and not self.look_like_account_owner(string)) self.logger.debug('Does "%s" look like an account name? %s', string, ('yes' if result else 'no')) return result def look_like_account_number(self, string): """ Returns either False or a SRE_Match object depending whether string looks like an account number. """ # An account is a 11 digits number (no more, no less) result = re.match('[^\d]*\d{11}[^\d]*', string) self.logger.debug('Does "%s" look like an account number? %s', string, ('yes' if result else 'no')) return result def look_like_amount(self, string): """ Returns either False or a SRE_Match object depending whether string looks like an amount. """ # It seems the Credit Agricole always mentions amounts using two decimals result = re.match('-?[\d ]+[\.,]\d{2}', string) self.logger.debug('Does "%s" look like an amount? %s', string, ('yes' if result else 'no')) return result def look_like_date_only(self, string): """ Returns either False or a SRE_Match object depending whether string looks like an isolated date. """ result = re.search('^\s*((?:[012][0-9]|3[01])/(?:0[1-9]|1[012]))\s*$', string) self.logger.debug('Does "%s" look like a date (and only a date)? %s', string, ('yes' if result else 'no')) return result def look_like_date_and_description(self, string): """ Returns either False or a SRE_Match object depending on whether string looks like a date+description pair. """ result = re.search('^\s*((?:[012][0-9]|3[01])/(?:0[1-9]|1[012]))\s+(.+)\s*$', string) self.logger.debug('Does "%s" look like a date+description pair? %s', string, ('yes' if result else 'no')) return result weboob-1.2/modules/cragr/mobile/pages/base.py000066400000000000000000000035461303450110500212310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page from weboob.deprecated.browser import BrowserUnavailable class CragrBasePage(Page): def on_loaded(self): # Check for an error for div in self.document.getiterator('div'): if div.attrib.get('class', '') == 'dv' and div.getchildren()[0].tag in ('img') and div.getchildren()[0].attrib.get('alt', '') == 'Attention': # Try to find a detailed error message if div.getchildren()[1].tag == 'span': raise BrowserUnavailable(div.find('span').find('b').text) elif div.getchildren()[1].tag == 'b': # I haven't encountered this variation in the wild, # but I wouldn't be surprised if it existed # given the similar differences between regions. raise BrowserUnavailable(div.find('b').find('span').text) raise BrowserUnavailable() def is_logged(self): return not self.document.xpath('/html/body//form//input[@name = "code"]') and \ not self.document.xpath('/html/body//form//input[@name = "userPassword"]') weboob-1.2/modules/cragr/mobile/pages/login.py000066400000000000000000000036661303450110500214320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.mech import ClientForm ControlNotFoundError = ClientForm.ControlNotFoundError from .base import CragrBasePage class LoginPage(CragrBasePage): def login(self, login, password): self.browser.select_form(nr=0) self.browser.set_all_readonly(False) try: self.browser['numero'] = login self.browser['code'] = password except ControlNotFoundError: try: self.browser['userLogin'] = login self.browser['userPassword'] = password except ControlNotFoundError: self.browser.controls.append(ClientForm.TextControl('text', 'numero', {'value': ''})) self.browser.controls.append(ClientForm.TextControl('text', 'code', {'value': ''})) self.browser.controls.append(ClientForm.TextControl('text', 'userLogin', {'value': ''})) self.browser.controls.append(ClientForm.TextControl('text', 'userPassword', {'value': ''})) self.browser['numero'] = login self.browser['code'] = password self.browser['userLogin'] = login self.browser['userPassword'] = password self.browser.submit() weboob-1.2/modules/cragr/mobile/pages/tokenextractor.py000066400000000000000000000040371303450110500233670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . class TokenExtractor(object): """ Extracts texts token from an HTML document """ def __init__(self): self.iterated_elements = [] def clear(self): """ Reset any content stored within a TokenExtractor object. Useful to start a new parsing without creating a new instance. """ self.iterated_elements = [] def element_iterated_already(self, html_element): if html_element in self.iterated_elements: return True for ancestor in html_element.iterancestors(): if ancestor in self.iterated_elements: return True return False def extract_tokens(self, html_element): if self.element_iterated_already(html_element): return self.iterated_elements.append(html_element) for text in html_element.itertext(): text = text.replace(u'\xa0', ' ') text = text.replace("\n", ' ') for token in self.split_text_into_smaller_tokens(text): if self.token_looks_relevant(token): yield token.strip() @staticmethod def split_text_into_smaller_tokens(text): for subtext1 in text.split('\t'): yield subtext1 @staticmethod def token_looks_relevant(token): return len(token.strip()) > 1 weboob-1.2/modules/cragr/module.py000066400000000000000000000113711303450110500172310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Xavier Guerrin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import AccountNotFound, CapBankTransfer from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import ValueBackendPassword, Value from .web.browser import Cragr from .mobile.browser import CragrMobile __all__ = ['CragrModule'] class CragrModule(Module, CapBankTransfer): NAME = 'cragr' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Crédit Agricole' LICENSE = 'AGPLv3+' website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'm.ca-alpesprovence.fr': u'Alpes Provence', 'm.ca-alsace-vosges.fr': u'Alsace-Vosges', 'm.ca-anjou-maine.fr': u'Anjou Maine', 'm.ca-aquitaine.fr': u'Aquitaine', 'm.ca-atlantique-vendee.fr': u'Atlantique Vendée', 'm.ca-briepicardie.fr': u'Brie Picardie', 'm.ca-cb.fr': u'Champagne Bourgogne', 'm.ca-centrefrance.fr': u'Centre France', 'm.ca-centreloire.fr': u'Centre Loire', 'm.ca-centreouest.fr': u'Centre Ouest', 'm.ca-centrest.fr': u'Centre Est', 'm.ca-charente-perigord.fr': u'Charente Périgord', 'm.ca-cmds.fr': u'Charente-Maritime Deux-Sèvres', 'm.ca-corse.fr': u'Corse', 'm.ca-cotesdarmor.fr': u'Côtes d\'Armor', 'm.ca-des-savoie.fr': u'Des Savoie', 'm.ca-finistere.fr': u'Finistere', 'm.ca-franchecomte.fr': u'Franche-Comté', 'm.ca-guadeloupe.fr': u'Guadeloupe', 'm.ca-illeetvilaine.fr': u'Ille-et-Vilaine', 'm.ca-languedoc.fr': u'Languedoc', 'm.ca-loirehauteloire.fr': u'Loire Haute Loire', 'm.ca-lorraine.fr': u'Lorraine', 'm.ca-martinique.fr': u'Martinique Guyane', 'm.ca-morbihan.fr': u'Morbihan', 'm.ca-nmp.fr': u'Nord Midi-Pyrénées', 'm.ca-nord-est.fr': u'Nord Est', 'm.ca-norddefrance.fr': u'Nord de France', 'm.ca-normandie-seine.fr': u'Normandie Seine', 'm.ca-normandie.fr': u'Normandie', 'm.ca-paris.fr': u'Ile-de-France', 'm.ca-pca.fr': u'Provence Côte d\'Azur', 'm.ca-reunion.fr': u'Réunion', 'm.ca-sudmed.fr': u'Sud Méditerranée', 'm.ca-sudrhonealpes.fr': u'Sud Rhône Alpes', 'm.ca-toulouse31.fr': u'Toulouse 31', # m.ca-toulousain.fr redirects here 'm.ca-tourainepoitou.fr': u'Tourraine Poitou', 'm.ca-valdefrance.fr': u'Val de France', 'm.lefil.com': u'Pyrénées Gascogne', }.iteritems())]) CONFIG = BackendConfig(Value('website', label=u'Région', choices=website_choices), ValueBackendPassword('login', label=u'N° de compte', masked=False), ValueBackendPassword('password', label=u'Code personnel', regexp=r'\d{6}')) BROWSER = Cragr def create_default_browser(self): try: return self.create_browser(self.config['website'].get(), self.config['login'].get(), self.config['password'].get()) except Cragr.WebsiteNotSupported: self.logger.debug('falling-back on mobile version') self.BROWSER = CragrMobile return self.create_browser(self.config['website'].get(), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): return self.browser.get_history(account) def iter_investment(self, account): with self.browser: for inv in self.browser.iter_investment(account): yield inv def transfer(self, account, to, amount, reason=None): return self.browser.do_transfer(account, to, amount, reason) weboob-1.2/modules/cragr/test.py000066400000000000000000000017441303450110500167260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CrAgrTest(BackendTest): MODULE = 'cragr' def test_cragr(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/cragr/web/000077500000000000000000000000001303450110500161445ustar00rootroot00000000000000weboob-1.2/modules/cragr/web/__init__.py000066400000000000000000000000001303450110500202430ustar00rootroot00000000000000weboob-1.2/modules/cragr/web/browser.py000066400000000000000000000445021303450110500202060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import urllib from urlparse import urlparse from mechanize import FormNotFoundError from weboob.capabilities.bank import Account from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.tools.date import LinearDateGuesser from weboob.exceptions import BrowserHTTPError from .pages import HomePage, LoginPage, LoginErrorPage, AccountsPage, \ SavingsPage, TransactionsPage, UselessPage, CardsPage, \ LifeInsurancePage, MarketPage, LoansPage, PerimeterPage, \ ChgPerimeterPage, MarketHomePage, FirstVisitPage, BGPIPage __all__ = ['Cragr'] class Cragr(Browser): PROTOCOL = 'https' ENCODING = 'ISO-8859-1' PAGES = {'https?://[^/]+/': HomePage, 'https?://[^/]+/particuliers.html': HomePage, 'https?://[^/]+/stb/entreeBam': LoginPage, 'https?://[^/]+/stb/entreeBam\?.*typeAuthentification=CLIC_ALLER.*': LoginPage, 'https?://[^/]+/stb/entreeBam\?.*pagePremVisite.*': FirstVisitPage, 'https?://[^/]+/stb/entreeBam\?.*Interstitielle.*': UselessPage, 'https?://[^/]+/stb/entreeBam\?.*act=Tdbgestion': UselessPage, 'https?://[^/]+/stb/entreeBam\?.*act=Synthcomptes': AccountsPage, 'https?://[^/]+/stb/collecteNI\?.*sessionAPP=Synthcomptes.*indicePage=.*': AccountsPage, 'https?://[^/]+/stb/entreeBam\?.*act=Synthcredits': LoansPage, 'https?://[^/]+/stb/collecteNI\?.*sessionAPP=Synthcredits.*indicePage=.*': LoansPage, 'https?://[^/]+/stb/entreeBam\?.*act=Synthepargnes': SavingsPage, 'https?://[^/]+/stb/collecteNI\?.*sessionAPP=Synthepargnes.*indicePage=.*': SavingsPage, 'https?://[^/]+/stb/.*act=Releves.*': TransactionsPage, 'https?://[^/]+/stb/collecteNI\?.*sessionAPP=Releves.*': TransactionsPage, 'https?://[^/]+/stb/.*/erreur/.*': LoginErrorPage, 'https?://[^/]+/stb/entreeBam\?.*act=Messagesprioritaires': UselessPage, 'https?://[^/]+/stb/collecteNI\?.*fwkaction=Cartes.*': CardsPage, 'https?://[^/]+/stb/collecteNI\?.*sessionAPP=Cartes.*': CardsPage, 'https?://[^/]+/stb/collecteNI\?.*fwkaction=Detail.*sessionAPP=Cartes.*': CardsPage, 'https?://www.cabourse.credit-agricole.fr/netfinca-titres/servlet/com.netfinca.frontcr.account.WalletVal\?nump=.*': MarketPage, 'https?://www.cabourse.credit-agricole.fr/netfinca-titres/servlet/com.netfinca.frontcr.synthesis.HomeSynthesis': MarketHomePage, 'https://assurance-personnes.credit-agricole.fr(:443)?/filiale/.*': LifeInsurancePage, 'https://bgpi-gestionprivee.credit-agricole.fr/bgpi/.*': BGPIPage, 'https?://[^/]+/stb/entreeBam\?.*act=Perimetre': PerimeterPage, 'https?://[^/]+/stb/entreeBam\?.*act=ChgPerim.*': ChgPerimeterPage, } new_login_domain = [] new_login = False class WebsiteNotSupported(Exception): pass def __init__(self, website, *args, **kwargs): if website in self.new_login_domain: self.DOMAIN = re.sub('^m\.', 'w2.', website) self.new_login = True else: self.DOMAIN = re.sub('^m\.', 'www.', website) self.accounts_url = None self.savings_url = None self._sag = None # updated while browsing self._old_sag = None self.code_caisse = None # constant for a given website self.perimeters = None self.current_perimeter = None self.broken_perimeters = list() Browser.__init__(self, *args, **kwargs) def home(self): self.login() def is_logged(self): return self.page is not None and not self.is_on_page(HomePage) and self.page.get_error() is None def login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) # Do we really need to login? if self.is_logged(): self.logger.debug('already logged in') return self._sag = None if not self.is_on_page(HomePage): self.location(self.absurl('/'), no_login=True) if self.new_login: self.page.go_to_auth() else: # On the homepage, we get the URL of the auth service. url = self.page.get_post_url() if url is None: raise self.WebsiteNotSupported() # First, post account number to get the password prompt. data = {'CCPTE': self.username[:11].encode(self.ENCODING), 'canal': 'WEB', 'hauteur_ecran': 768, 'largeur_ecran': 1024, 'liberror': '', 'matrice': 'true', 'origine': 'vitrine', 'situationTravail': 'BANCAIRE', 'typeAuthentification': 'CLIC_ALLER', 'urlOrigine': self.page.url, 'vitrine': 0, } self.location(url, urllib.urlencode(data), no_login=True) assert self.is_on_page(LoginPage) # Then, post the password. try: self.page.login(self.username, self.password) except FormNotFoundError: raise BrowserIncorrectPassword() if self.new_login: url = self.page.get_accounts_url() else: # The result of POST is the destination URL. url = self.page.get_result_url() if not url.startswith('http'): raise BrowserIncorrectPassword(url) self.location(url.replace('Synthese', 'Synthcomptes')) if self.is_on_page(LoginErrorPage): raise BrowserIncorrectPassword() if self.page is None: raise self.WebsiteNotSupported() if not self.is_on_page(AccountsPage): # Sometimes the home page is Releves. new_url = re.sub('act=([^&=]+)', 'act=Synthcomptes', self.page.url, 1) self.location(new_url) if not self.is_on_page(AccountsPage): raise BrowserIncorrectPassword() if self.code_caisse is None: scripts = self.page.document.xpath('//script[contains(., " codeCaisse")]') self.code_caisse = re.search('var +codeCaisse *= *"([0-9]+)"', scripts[0].text).group(1) # Store the current url to go back when requesting accounts list. self.accounts_url = re.sub('sessionSAG=[^&]+', 'sessionSAG={0}', self.page.url) # we can deduce the URL to "savings" and "loan" accounts from the regular accounts one self.savings_url = re.sub('act=([^&=]+)', 'act=Synthepargnes', self.accounts_url, 1) self.loans_url = re.sub('act=([^&=]+)', 'act=Synthcredits', self.accounts_url, 1) if self.page.check_perimeters() and not self.broken_perimeters: self.perimeter_url = re.sub('act=([^&=]+)', 'act=Perimetre', self.accounts_url, 1) self.chg_perimeter_url = '%s%s' % (re.sub('act=([^&=]+)', 'act=ChgPerim', self.accounts_url, 1), '&typeaction=ChgPerim') self.location(self.perimeter_url.format(self.sag)) self.page.check_multiple_perimeters() def go_perimeter(self, perimeter): # If this fails, there is no point in retrying with same cookies. self.location(self.perimeter_url.format(self.sag), no_login=True) if self.page.get_error() is not None: self.login() self.location(self.perimeter_url.format(self.sag)) if len(self.perimeters) > 2: perimeter_link = self.page.get_perimeter_link(perimeter) if perimeter_link: self.location(perimeter_link) self.location(self.chg_perimeter_url.format(self.sag), no_login=True) if self.page.get_error() is not None: self.broken_perimeters.append(perimeter) def get_accounts_list(self): l = list() if self.perimeters: for perimeter in [p for p in self.perimeters if p not in self.broken_perimeters]: if (self.page and not self.page.get_current()) or self.current_perimeter != perimeter: self.go_perimeter(perimeter) for account in self.get_list(): if not account in l: l.append(account) else: l = self.get_list() return l def get_cards_or_card(self, account_id=None): accounts = [] if not self.is_on_page(AccountsPage): self.location(self.accounts_url.format(self.sag)) for idelco in self.page.cards_idelco_or_link(): if not self.is_on_page(AccountsPage): self.location(self.accounts_url.format(self.sag)) self.location(self.page.cards_idelco_or_link(idelco)) assert self.is_on_page(CardsPage) for account in self.page.get_list(): if account_id and account.number == account_id: return account else: accounts.append(account) return accounts def get_list(self): accounts_list = [] # regular accounts if not self.is_on_page(AccountsPage): self.location(self.accounts_url.format(self.sag)) accounts_list.extend(self.page.get_list()) # credit cards # reseting location in case of pagination self.location(self.accounts_url.format(self.sag)) accounts_list.extend(self.get_cards_or_card()) # loan accounts self.location(self.loans_url.format(self.sag)) if self.is_on_page(LoansPage): for account in self.page.get_list(): if account not in accounts_list: accounts_list.append(account) # savings accounts self.location(self.savings_url.format(self.sag)) if self.is_on_page(SavingsPage): for account in self.page.get_list(): if account not in accounts_list: accounts_list.append(account) # update market accounts for account in accounts_list: if account.type == Account.TYPE_MARKET: try: new_location = self.moveto_market_website(account, home=True) except self.WebsiteNotSupported: account._link = None self.update_sag() else: self.location(new_location) self.page.update(accounts_list) self.quit_market_website() break return accounts_list def get_account(self, id): assert isinstance(id, basestring) l = self.get_accounts_list() for a in l: if a.id == ('%s' % id): return a return None def get_history(self, account): if account.type in (Account.TYPE_MARKET, Account.TYPE_LIFE_INSURANCE): self.logger.warning('This account is not supported') raise NotImplementedError() # some accounts may exist without a link to any history page if account._link is None or 'CATITRES' in account._link: return if account._perimeter != self.current_perimeter: self.go_perimeter(account._perimeter) # card accounts need to get an updated link if account.type == Account.TYPE_CARD: account = self.get_cards_or_card(account.number) date_guesser = LinearDateGuesser() if account.type != Account.TYPE_CARD or not self.page.is_on_right_detail(account): self.location(account._link.format(self.sag)) if self.is_on_page(CardsPage): url = self.page.url state = None notfirst = False while url: if notfirst: self.location(url) else: notfirst = True assert self.is_on_page(CardsPage) for state, tr in self.page.get_history(date_guesser, state): yield tr url = self.page.get_next_url() elif self.page: self.page.order_transactions() while True: assert self.is_on_page(TransactionsPage) for tr in self.page.get_history(date_guesser): yield tr url = self.page.get_next_url() if url is None: break self.location(url) def iter_investment(self, account): if not account._link or account.type not in (Account.TYPE_MARKET, Account.TYPE_LIFE_INSURANCE): return if account._perimeter != self.current_perimeter: self.go_perimeter(account._perimeter) if account.type == Account.TYPE_MARKET: new_location = self.moveto_market_website(account) # Detail unavailable try: self.location(new_location) except BrowserHTTPError: return elif account.type == Account.TYPE_LIFE_INSURANCE: new_location = self.moveto_insurance_website(account) self.location(new_location, urllib.urlencode({})) if self.is_on_page(BGPIPage): if not self.page.go_detail(): return if self.is_on_page(LifeInsurancePage): self.page.go_on_detail(account.id) for inv in self.page.iter_investment(): yield inv if account.type == Account.TYPE_MARKET: self.quit_market_website() elif account.type == Account.TYPE_LIFE_INSURANCE: self.quit_insurance_website() def moveto_market_website(self, account, home=False): response = self.openurl(account._link % self.sag).read() self._sag = None # https://www.cabourse.credit-agricole.fr/netfinca-titres/servlet/com.netfinca.frontcr.navigation.AccueilBridge?TOKEN_ID= m = re.search('document.location="([^"]+)"', response) if m: url = m.group(1) else: self.logger.warn('Unable to go to market website') raise self.WebsiteNotSupported() self.openurl(url) if home: return 'https://www.cabourse.credit-agricole.fr/netfinca-titres/servlet/com.netfinca.frontcr.synthesis.HomeSynthesis' parsed = urlparse(url) url = '%s://%s/netfinca-titres/servlet/com.netfinca.frontcr.account.WalletVal?nump=%s:%s' return url % (parsed.scheme, parsed.netloc, account.id, self.code_caisse) def quit_market_website(self): parsed = urlparse(self.geturl()) exit_url = '%s://%s/netfinca-titres/servlet/com.netfinca.frontcr.login.ContextTransferDisconnect' % (parsed.scheme, parsed.netloc) doc = self.get_document(self.openurl(exit_url), encoding='utf-8') form = doc.find('//form[@name="formulaire"]') # 'act' parameter allows page recognition, this parameter is ignored by # server self.location(form.attrib['action'] + '&act=Synthepargnes') self.update_sag() def moveto_insurance_website(self, account): doc = self.get_document(self.openurl(account._link % self.sag), encoding='utf-8') self._sag = None # POST to https://assurance-personnes.credit-agricole.fr/filiale/ServletReroutageCookie form = doc.find('//form[@name="formulaire"]') # bgpi-gestionprivee. if not form: return re.search('location="([^"]+)"', self.openurl(account._link % self.sag).read(), flags=re.MULTILINE).group(1) data = { 'page': form.inputs['page'].attrib['value'], 'cMaxAge': '-1', } script = doc.find('//script').text for value in ('cMaxAge', 'cName', 'cValue'): m = re.search('%s.value *= *"([^"]+)"' % value, script) if m: data[value] = m.group(1) else: raise self.WebsiteNotSupported() doc = self.get_document(self.openurl(form.attrib['action'], urllib.urlencode(data)), encoding='utf-8') # POST to https://assurance-personnes.credit-agricole.fr:443/filiale/entreeBam?identifiantBAM=xxx form = doc.find('//form[@name="formulaire"]') return form.attrib['action'] def quit_insurance_website(self): if self.is_on_page(BGPIPage): return self.page.go_back() exit_url = 'https://assurance-personnes.credit-agricole.fr/filiale/entreeBam?actCrt=Synthcomptes&sessionSAG=%s&stbpg=pagePU&act=&typeaction=reroutage_retour&site=BAMG2&stbzn=bnc' doc = self.get_document(self.openurl(exit_url % self.sag), encoding='utf-8') form = doc.find('//form[@name="formulaire"]') # 'act' parameter allows page recognition, this parameter is ignored by # server self.location(form.attrib['action'] + '&act=Synthepargnes') self.update_sag() @property def sag(self): if not self._sag: self.update_sag() return self._sag def update_sag(self): if not self.is_logged(): self.login() script = self.page.document.xpath("//script[contains(.,'idSessionSag =')]") if script: self._old_sag = self._sag = re.search('idSessionSag = "([^"]+)";', script[0].text).group(1) else: self._sag = self._old_sag weboob-1.2/modules/cragr/web/pages.py000066400000000000000000000750531303450110500176270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2015 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date as ddate from decimal import Decimal from urlparse import urlparse import re from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account, Investment from weboob.deprecated.browser import Page, BrokenPageError, BrowserIncorrectPassword from weboob.tools.capabilities.bank.transactions import FrenchTransaction as Transaction from weboob.tools.date import parse_french_date, LinearDateGuesser from weboob.browser.filters.standard import Date class BasePage(Page): def on_loaded(self): self.get_current() def get_current(self): try: current_elem = self.document.xpath('//div[@id="libPerimetre_2"]/span[@class="textePerimetre_2"]')[0] except IndexError: self.logger.debug('Can\'t update current perimeter on this page (%s).', type(self).__name__) return False self.browser.current_perimeter = re.search('(.*)$', self.parser.tocleanstring(current_elem)).group(1).lower() return True def get_error(self): try: error = self.document.xpath('//h1[@class="h1-erreur"]')[0] self.logger.error('Error detected: %s', error.text_content().strip()) return error except IndexError: return None class HomePage(BasePage): def get_post_url(self): for script in self.document.xpath('//script'): text = script.text if text is None: continue m = re.search(r'var chemin = "([^"]+)"', text, re.MULTILINE) if m: return m.group(1) return None def go_to_auth(self): self.browser.select_form('bamaccess') self.browser.submit(no_login=True) class LoginPage(BasePage): def on_loaded(self): if self.document.xpath('//font[@class="taille2"]'): raise BrowserIncorrectPassword() def login(self, username, password): password = password[:6] imgmap = {} for td in self.document.xpath('//table[@id="pave-saisie-code"]/tr/td'): a = td.find('a') num = a.text.strip() if num.isdigit(): imgmap[num] = int(a.attrib['tabindex']) - 1 self.browser.select_form(name='formulaire') self.browser.set_all_readonly(False) if self.browser.new_login: self.browser['CCPTE'] = username self.browser['CCCRYC'] = ','.join(['%02d' % imgmap[c] for c in password]) self.browser['CCCRYC2'] = '0' * len(password) self.browser.submit(nologin=True) def get_result_url(self): return self.parser.tocleanstring(self.document.getroot()) def get_accounts_url(self): for script in self.document.xpath('//script'): text = script.text if text is None: continue m = re.search(r'idSessionSag = "([^"]+)"', script.text) if m: idSessionSag = m.group(1) return '%s%s%s%s' % (self.url, '?sessionSAG=', idSessionSag, '&stbpg=pagePU&actCrt=Synthcomptes&stbzn=btn&act=Synthcomptes') class UselessPage(BasePage): pass class LoginErrorPage(BasePage): pass class FirstVisitPage(BasePage): def on_loaded(self): raise BrowserIncorrectPassword(u'Veuillez vous connecter au site du Crédit Agricole pour valider vos données personnelles, et réessayer ensuite.') class _AccountsPage(BasePage): COL_LABEL = 0 COL_ID = 2 COL_VALUE = 4 COL_CURRENCY = 5 NB_COLS = 7 TYPES = {u'CCHQ': Account.TYPE_CHECKING, # par u'CCOU': Account.TYPE_CHECKING, # pro u'DAV PEA': Account.TYPE_CHECKING, u'LIV A': Account.TYPE_SAVINGS, u'LDD': Account.TYPE_SAVINGS, u'PEL': Account.TYPE_SAVINGS, u'CEL': Account.TYPE_SAVINGS, u'CODEBIS': Account.TYPE_SAVINGS, u'LJMO': Account.TYPE_SAVINGS, u'CSL': Account.TYPE_SAVINGS, u'LEP': Account.TYPE_SAVINGS, u'TIWI': Account.TYPE_SAVINGS, u'CSL LSO': Account.TYPE_SAVINGS, u'CSL CSP': Account.TYPE_SAVINGS, u'ESPE INTEG': Account.TYPE_SAVINGS, u'DAV TIGERE': Account.TYPE_SAVINGS, u'CPTEXCPRO': Account.TYPE_SAVINGS, u'PRET PERSO': Account.TYPE_LOAN, u'P. HABITAT': Account.TYPE_LOAN, u'PRET 0%': Account.TYPE_LOAN, u'INV PRO': Account.TYPE_LOAN, u'PEA': Account.TYPE_MARKET, u'CPS': Account.TYPE_MARKET, u'TITR': Account.TYPE_MARKET, u'TITR CTD': Account.TYPE_MARKET, u'réserves de crédit': Account.TYPE_CHECKING, u'prêts personnels': Account.TYPE_LOAN, u'crédits immobiliers': Account.TYPE_LOAN, u'épargne disponible': Account.TYPE_SAVINGS, u'épargne à terme': Account.TYPE_DEPOSIT, u'épargne boursière': Account.TYPE_MARKET, u'assurance vie et capitalisation': Account.TYPE_LIFE_INSURANCE, } def get_list(self): account_type = Account.TYPE_UNKNOWN for tr in self.document.xpath('//table[@class="ca-table"]/tr'): try: title = tr.xpath('.//h3/text()')[0].lower().strip() except IndexError: pass else: account_type = self.TYPES.get(title, Account.TYPE_UNKNOWN) if not tr.attrib.get('class', '').startswith('colcelligne'): continue cols = tr.findall('td') if not cols or len(cols) < self.NB_COLS: continue account = Account() account.id = self.parser.tocleanstring(cols[self.COL_ID]) account.label = self.parser.tocleanstring(cols[self.COL_LABEL]) account.type = self.TYPES.get(account.label, Account.TYPE_UNKNOWN) or account_type balance = self.parser.tocleanstring(cols[self.COL_VALUE]) # we have to ignore those accounts, because using NotAvailable # makes boobank and probably many others crash if balance in ('indisponible', ''): continue account.balance = Decimal(Transaction.clean_amount(balance)) account.currency = account.get_currency(self.parser.tocleanstring(cols[self.COL_CURRENCY])) account._link = None self.set_link(account, cols) account._perimeter = self.browser.current_perimeter yield account # Checking pagination next_link = self.document.xpath('//a[@class="btnsuiteliste"]/@href') if next_link: self.browser.location(next_link[0]) for account in self.browser.page.get_list(): yield account def set_link(self, account, cols): raise NotImplementedError() def cards_idelco_or_link(self, account_idelco=None): # Use a set because it is possible to see several times the same link. idelcos = set() for line in self.document.xpath('//table[@class="ca-table"]/tr[@class="ligne-connexe"]'): try: link = line.xpath('.//a/@href')[0] except IndexError: pass else: if not link.startswith('javascript:'): if account_idelco and 'IDELCO=%s&' % account_idelco in link: return link m = re.search('IDELCO=(\d+)&', link) if m: idelcos.add(m.group(1)) return idelcos def check_perimeters(self): return len(self.document.xpath('//a[@title="Espace Autres Comptes"]')) class PerimeterPage(BasePage): def check_multiple_perimeters(self): self.browser.perimeters = list() self.get_current() if self.browser.current_perimeter is None: return self.browser.perimeters.append(self.browser.current_perimeter) multiple = self.document.xpath(u'//p[span/a[contains(text(), "Accès")]]') if not multiple: if not len(self.document.xpath(u'//div[contains(text(), "Périmètre en cours de chargement. Merci de patienter quelques secondes.")]')): self.logger.debug('Possible error on this page.') # We change perimeter in this case to add the second one. self.browser.location(self.browser.chg_perimeter_url.format(self.browser.sag), no_login=True) if self.browser.page.get_error() is not None: self.browser.broken_perimeters.append('the other perimeter is broken') self.browser.login() else: for table in self.document.xpath('//table[@class]'): space = ' '.join(table.find('caption').text.lower().split()) for perim in table.xpath('.//label'): self.browser.perimeters.append(u'%s : %s' % (space, ' '.join(perim.text.lower().split()))) def get_perimeter_link(self, perimeter): caption = perimeter.split(' : ')[0].title() perim = perimeter.split(' : ')[1] for table in self.document.xpath('//table[@class and caption[contains(text(), "%s")]]' % caption): for p in table.xpath(u'.//p[span/a[contains(text(), "Accès")]]'): if perim in ' '.join(p.find('label').text.lower().split()): link = p.xpath('./span/a')[0].attrib['href'] return link class ChgPerimeterPage(PerimeterPage): def on_loaded(self): if self.get_error() is not None: self.logger.debug('Error on ChgPerimeterPage') return self.get_current() if not self.browser.current_perimeter.lower() in [' '.join(p.lower().split()) for p in self.browser.perimeters]: assert len(self.browser.perimeters) == 1 self.browser.perimeters.append(self.browser.current_perimeter) class CardsPage(BasePage): def get_list(self): TABLE_XPATH = '//table[caption[@class="caption tdb-cartes-caption" or @class="ca-table caption"]]' cards_tables = self.document.xpath(TABLE_XPATH) currency = self.document.xpath('//table/caption//span/text()[starts-with(.,"Montants en ")]')[0].replace("Montants en ", "") or None if cards_tables: self.logger.debug('There are several cards') xpaths = { '_id': './caption/span[@class="tdb-cartes-num"]', 'label1': './caption/span[contains(@class, "tdb-cartes-carte")]', 'label2': './caption/span[@class="tdb-cartes-prop"]', 'balance': './/tr/td[@class="cel-num"]', 'currency': '//table/caption//span/text()[starts-with(.,"Montants en ")]', 'link': './/tr//a/@href[contains(., "fwkaction=Detail")]', } else: self.logger.debug('There is only one card') xpaths = { '_id': './/tr/td[@class="cel-texte"]', 'label1': './/tr[@class="ligne-impaire ligne-bleu"]/th', 'label2': './caption/span[@class="tdb-cartes-prop"]/b', 'balance': './/tr[last()-1]/td[@class="cel-num"] | .//tr[last()-2]/td[@class="cel-num"]', 'currency': '//table/caption//span/text()[starts-with(.,"Montants en ")]', } TABLE_XPATH = '(//table[@class="ca-table"])[1]' cards_tables = self.document.xpath(TABLE_XPATH) for table in cards_tables: get = lambda name: self.parser.tocleanstring(table.xpath(xpaths[name])[0]) account = Account() account.type = account.TYPE_CARD account.number = ''.join(get('_id').split()[1:]) # account.number might be the same for two different cards .. account.id = '%s%s' % (account.number, get('label2').replace(' ', '')) account._id = ' '.join(get('_id').split()[1:]) account.label = '%s - %s' % (get('label1'), re.sub('\s*-\s*$', '', get('label2'))) try: account.balance = Decimal(Transaction.clean_amount(table.xpath(xpaths['balance'])[-1].text)) account.currency = account.get_currency(self.document .xpath(xpaths['currency'])[0].replace("Montants en ", "")) if not account.currency and currency: account.currency = Account.get_currency(currency) except IndexError: account.balance = Decimal('0.0') if 'link' in xpaths: try: account._link = table.xpath(xpaths['link'])[-1] except IndexError: account._link = None else: account._link = re.sub('[\n\r\t]+', '', account._link) else: account._link = self.url account._idelco = re.search('IDELCO=(\d+)&', self.url).group(1) account._perimeter = self.browser.current_perimeter yield account def order_transactions(self): pass def get_next_url(self): links = self.document.xpath('//font[@class="btnsuiteliste"]') if len(links) < 1: return None a = links[-1].find('a') if a.attrib.get('class', '') == 'liennavigationcorpspage': return a.attrib['href'] return None def get_history(self, date_guesser, state=None): seen = set() lines = self.document.xpath('(//table[@class="ca-table"])[2]/tr') debit_date = None for i, line in enumerate(lines): is_balance = line.xpath('./td/@class="cel-texte cel-neg"') # It is possible to have three or four columns. cols = [self.parser.tocleanstring(td) for td in line.xpath('./td')] date = cols[0] label = cols[1] amount = cols[-1] t = Transaction() t.set_amount(amount) t.label = t.raw = label if is_balance: m = re.search('(\d+ [^ ]+ \d+)', label) if not m: raise BrokenPageError('Unable to read card balance in history: %r' % label) if state is None: debit_date = parse_french_date(m.group(1)) else: debit_date = state # Skip the first line because it is balance if i == 0: continue t.date = t.rdate = debit_date # Consider the second one as a positive amount to reset balance to 0. t.amount = -t.amount state = t.date else: day, month = map(int, date.split('/', 1)) t.rdate = date_guesser.guess_date(day, month) t.date = debit_date t.type = t.TYPE_CARD try: t.id = t.unique_id(seen) except UnicodeEncodeError: self.logger.debug(t) self.logger.debug(t.label) raise yield state, t def is_on_right_detail(self, account): return len(self.document.xpath(u'//h1[contains(text(), "Cartes - détail")]')) and\ len(self.document.xpath(u'//td[contains(text(), "%s")] | //td[contains(text(), "%s")] ' % (account.number, account._id))) class AccountsPage(_AccountsPage): def set_link(self, account, cols): a = cols[0].find('a') if a is not None: account._link = a.attrib['href'].replace(' ', '%20') page = self.browser.get_page(self.browser.openurl(account._link)) account._link = re.sub('sessionSAG=[^&]+', 'sessionSAG={0}', account._link) url = page.get_iban_url() if url: page = self.browser.get_page(self.browser.openurl(url)) account.iban = page.get_iban() class LoansPage(_AccountsPage): COL_ID = 1 NB_COLS = 6 def set_link(self, account, cols): account.balance = -abs(account.balance) class SavingsPage(_AccountsPage): COL_ID = 1 def set_link(self, account, cols): origin = urlparse(self.url) if not account._link: a = cols[0].xpath('descendant::a[contains(@href, "CATITRES")]') # Sometimes there is no link. if a or account.type == Account.TYPE_MARKET: url = 'https://%s/stb/entreeBam?sessionSAG=%%s&stbpg=pagePU&site=CATITRES&typeaction=reroutage_aller' account._link = url % origin.netloc a = cols[0].xpath("descendant::a[contains(@href, \"'PREDICA','CONTRAT'\")]") if a: account.type = Account.TYPE_LIFE_INSURANCE url = 'https://%s/stb/entreeBam?sessionSAG=%%s&stbpg=pagePU&site=PREDICA&' \ 'typeaction=reroutage_aller&sdt=CONTRAT¶mpartenaire=%s' account._link = url % (origin.netloc, account.id) a = cols[0].xpath('descendant::a[not(contains(@href, "javascript"))]') if len(a) == 1 and not account._link: account._link = a[0].attrib['href'].replace(' ', '%20') account._link = re.sub('sessionSAG=[^&]+', 'sessionSAG={0}', account._link) a = cols[0].xpath('descendant::a[(contains(@href, "javascript"))]') # This aims to handle bgpi-gestionprivee. if len(a) == 1 and not account._link: m = re.findall("'([^']*)'", a[0].attrib['href']) if len(m) == 3: url = 'https://%s/stb/entreeBam?sessionSAG=%%s&stbpg=pagePU&typeaction=reroutage_aller&site=%s&sdt=%s¶mpartenaire=%s' account._link = url % (origin.netloc, m[0], m[1], m[2]) class TransactionsPage(BasePage): def get_iban_url(self): for link in self.document.xpath('//a[contains(text(), "RIB")] | //a[contains(text(), "IBAN")]'): m = re.search("\('([^']+)'", link.get('href', '')) if m: return m.group(1) return None def get_iban(self): s = '' for font in self.document.xpath('(//td[font/b/text()="IBAN"])[1]/table//font'): s += self.parser.tocleanstring(font) return s def order_transactions(self): date = self.document.xpath('//th[@scope="col"]/a[text()="Date"]') if len(date) < 1: return if 'active' in date[0].attrib.get('class', ''): return self.browser.location(date[0].attrib['href']) def get_next_url(self): links = self.document.xpath('//span[@class="pager"]/a[@class="liennavigationcorpspage"]') if len(links) < 1: return None img = links[-1].find('img') if img.attrib.get('alt', '') == 'Page suivante': return links[-1].attrib['href'] return None COL_DATE = 0 COL_TEXT = 1 COL_DEBIT = None COL_CREDIT = -1 TYPES = {'Paiement Par Carte': Transaction.TYPE_CARD, 'Remise Carte': Transaction.TYPE_CARD, 'Retrait Au Distributeur': Transaction.TYPE_WITHDRAWAL, 'Frais': Transaction.TYPE_BANK, 'Cotisation': Transaction.TYPE_BANK, 'Virement Emis': Transaction.TYPE_TRANSFER, 'Virement': Transaction.TYPE_TRANSFER, 'Cheque Emis': Transaction.TYPE_CHECK, 'Remise De Cheque': Transaction.TYPE_DEPOSIT, 'Prelevement': Transaction.TYPE_ORDER, 'Prelevt': Transaction.TYPE_ORDER, 'Prelevmnt': Transaction.TYPE_ORDER, 'Remboursement De Pret': Transaction.TYPE_LOAN_PAYMENT, } def get_history(self, date_guesser): trs = self.document.xpath('//table[@class="ca-table" and @summary]//tr') if trs: self.COL_TEXT += 1 else: trs = self.document.xpath('//table[@class="ca-table"]//tr') for tr in trs: parent = tr.getparent() while parent is not None and parent.tag != 'table': parent = parent.getparent() if parent.attrib.get('class', '') != 'ca-table': continue if tr.attrib.get('class', '') == 'tr-thead': heads = tr.findall('th') for i, head in enumerate(heads): key = self.parser.tocleanstring(head) if key == u'Débit': self.COL_DEBIT = i - len(heads) if key == u'Crédit': self.COL_CREDIT = i - len(heads) if key == u'Libellé': self.COL_TEXT = i if not tr.attrib.get('class', '').startswith('ligne-'): continue cols = tr.findall('td') # On loan accounts, there is a ca-table with a summary. Skip it. if tr.find('th') is not None or len(cols) < 3: continue # On PEL, skip summary. if len(cols[0].findall('span')) == 6: continue t = Transaction() col_text = cols[self.COL_TEXT] if len(col_text.xpath('.//br')) == 0: col_text = cols[self.COL_TEXT+1] raw = self.parser.tocleanstring(col_text) date = self.parser.tocleanstring(cols[self.COL_DATE]) credit = self.parser.tocleanstring(cols[self.COL_CREDIT]) if self.COL_DEBIT is not None: debit = self.parser.tocleanstring(cols[self.COL_DEBIT]) else: debit = '' if date.count('/') == 1: day, month = map(int, date.split('/', 1)) t.date = date_guesser.guess_date(day, month) elif date.count('/') == 2: t.date = Date(dayfirst=True).filter(date) t.rdate = t.date t.raw = raw # On some accounts' history page, there is a tag in columns. if col_text.find('font') is not None: col_text = col_text.find('font') t.category = unicode(col_text.text.strip()) t.label = re.sub('(.*) (.*)', r'\2', t.category).strip() br = col_text.find('br') if br is not None: sub_label = br.tail if br is not None and sub_label is not None: junk_ratio = len(re.findall('[^\w\s]', sub_label)) / float(len(sub_label)) nums_ratio = len(re.findall('\d', t.label)) / float(len(t.label)) if len(t.label) < 3 or t.label == t.category or junk_ratio < nums_ratio: t.label = unicode(sub_label.strip()) # Sometimes, the category contains the label, even if there is another line with it again. t.category = re.sub('(.*) .*', r'\1', t.category).strip() t.type = self.TYPES.get(t.category, t.TYPE_UNKNOWN) # Parse operation date in label (for card transactions for example) m = re.match('(?P.*) (?P
[0-3]\d)/(?P[0-1]\d)$', t.label) if not m: m = re.match('^(?P
[0-3]\d)/(?P[0-1]\d) (?P.*)$', t.label) if m: if t.type in (t.TYPE_CARD, t.TYPE_WITHDRAWAL): t.rdate = date_guesser.guess_date(int(m.groupdict()['dd']), int(m.groupdict()['mm']), change_current_date=False) t.label = m.groupdict()['text'].strip() # Strip city or other useless information from label. t.label = re.sub('(.*) .*', r'\1', t.label).strip() t.set_amount(credit, debit) yield t class MarketPage(BasePage): COL_ID = 1 COL_QUANTITY = 2 COL_UNITVALUE = 3 COL_VALUATION = 4 COL_UNITPRICE = 5 COL_DIFF = 6 def iter_investment(self): for line in self.document.xpath('//table[contains(@class, "ca-data-table")]/descendant::tr[count(td)>=7]'): for sub in line.xpath('./td[@class="info-produit"]'): sub.drop_tree() cells = line.findall('td') if cells[self.COL_ID].find('div/a') is None: continue inv = Investment() inv.label = unicode(cells[self.COL_ID].find('div/a').text.strip()) inv.code = cells[self.COL_ID].find('div/br').tail.strip().split(' ')[0].split(u'\xa0')[0].split(u'\xc2\xa0')[0] inv.quantity = self.parse_decimal(cells[self.COL_QUANTITY].find('span').text) inv.valuation = self.parse_decimal(cells[self.COL_VALUATION].text) inv.diff = self.parse_decimal(cells[self.COL_DIFF].text_content()) if "%" in cells[self.COL_UNITPRICE].text and "%" in cells[self.COL_UNITVALUE].text: inv.unitvalue = inv.valuation / inv.quantity inv.unitprice = (inv.valuation - inv.diff) / inv.quantity else: inv.unitprice = self.parse_decimal(re.search('([^(]+)', cells[self.COL_UNITPRICE].text).group(1)) inv.unitvalue = self.parse_decimal(cells[self.COL_UNITVALUE].text) date = cells[self.COL_UNITVALUE].find('span').text if ':' in date: inv.vdate = ddate.today() else: day, month = map(int, date.split('/'))[:2] date_guesser = LinearDateGuesser() inv.vdate = date_guesser.guess_date(day, month) yield inv def parse_decimal(self, value): v = value.strip() if v == '-' or v == '' or v == '_': return NotAvailable return Decimal(Transaction.clean_amount(value)) class MarketHomePage(MarketPage): COL_ID_LABEL = 1 COL_VALUATION = 5 def update(self, accounts): for line in self.document.xpath('//table[contains(@class, "tableau_comptes_details")]/tbody/tr'): cells = line.findall('td') id = cells[self.COL_ID_LABEL].find('div[2]').text.strip() for account in accounts: if account.id == id: account.label = unicode(cells[self.COL_ID_LABEL].find('div/b').text.strip()) account.balance = self.parse_decimal(cells[self.COL_VALUATION].text) class LifeInsurancePage(MarketPage): COL_ID = 0 COL_QUANTITY = 3 COL_UNITVALUE = 1 COL_VALUATION = 4 def go_on_detail(self, account_id): # Sometimes this page is a synthesis, so we need to go on detail. if len(self.document.xpath(u'//h1[contains(text(), "Synthèse de vos contrats d\'assurance vie, de capitalisation et de prévoyance")]')) == 1: self.browser.select_form('frm_fwk') self.browser.set_all_readonly(False) self.browser['ID_CNT_CAR'] = account_id self.browser['fwkaction'] = 'Enchainer' self.browser['fwkcodeaction'] = 'Executer' self.browser['puCible'] = 'SEPPU' self.browser.submit() self.browser.location('https://assurance-personnes.credit-agricole.fr/filiale/entreeBam?sessionSAG=%s&stbpg=pagePU&act=SEPPU&stbzn=bnt&actCrt=SEPPU' % self.browser.sag) def iter_investment(self): for line in self.document.xpath('//table[@summary and count(descendant::td) > 1]/tbody/tr'): cells = line.findall('td') if len(cells) < 5: continue inv = Investment() inv.label = unicode(cells[self.COL_ID].text_content().strip()) a = cells[self.COL_ID].find('a') if a is not None: try: inv.code = a.attrib['id'].split(' ')[0].split(u'\xa0')[0].split(u'\xc2\xa0')[0] except KeyError: #For "Mandat d'arbitrage" which is a recapitulatif of more investement continue else: inv.code = NotAvailable inv.quantity = self.parse_decimal(cells[self.COL_QUANTITY].text_content()) inv.unitvalue = self.parse_decimal(cells[self.COL_UNITVALUE].text_content()) inv.valuation = self.parse_decimal(cells[self.COL_VALUATION].text_content()) inv.unitprice = NotAvailable inv.diff = NotAvailable yield inv class BGPIPage(MarketPage): COL_ID = 0 COL_QUANTITY = 1 COL_UNITPRICE = 2 COL_UNITVALUE = 3 COL_VALUATION = 4 COL_PORTFOLIO = 5 COL_DIFF = 6 def iter_investment(self): for line in self.document.xpath('//table[contains(@class, "PuTableauLarge")]/tr[contains(@class, "PuLigne")]'): cells = line.findall('td') inv = Investment() inv.label = unicode(cells[self.COL_ID].find('span').text_content().strip()) a = cells[self.COL_ID].find('a') if a is not None: inv.code = unicode(a.text_content().strip()) else: inv.code = NotAvailable inv.quantity = self.parse_decimal(cells[self.COL_QUANTITY].text_content()) inv.unitvalue = self.parse_decimal(cells[self.COL_UNITVALUE].text_content()) inv.valuation = self.parse_decimal(cells[self.COL_VALUATION].text_content()) inv.unitprice = self.parse_decimal(cells[self.COL_UNITPRICE].text_content()) inv.diff = self.parse_decimal(cells[self.COL_DIFF].text_content()) inv.portfolio_share = self.parse_decimal(cells[self.COL_PORTFOLIO].text_content())/100 yield inv def go_on(self, link): origin = urlparse(self.url) self.browser.location('https://%s%s' % (origin.netloc, link)) def go_detail(self): link = self.document.xpath(u'.//a[contains(text(), "Détail")]') return self.go_on(link[0].attrib['href']) if link else False def go_back(self): self.go_on(self.document.xpath(u'.//a[contains(text(), "Retour à mes comptes")]')[0].attrib['href']) self.browser.select_form('formulaire') self.browser.submit() weboob-1.2/modules/creditcooperatif/000077500000000000000000000000001303450110500176175ustar00rootroot00000000000000weboob-1.2/modules/creditcooperatif/__init__.py000066400000000000000000000015101303450110500217250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget, based on Romain Bignon work # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CreditCooperatifModule __all__ = ['CreditCooperatifModule'] weboob-1.2/modules/creditcooperatif/favicon.png000066400000000000000000000121541303450110500217550ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME *\ IDATxyUu?߽o}eeXV{8'qYUUJU+uU*]VmSEndwcv$6v̀0,``VfY߻w}<LbHi4.s~g9#3JKyLW 3F:Q=U9ci-D'53k)9L=G:oJ DnI4@E[y9&}D&9?ӹmlc툋M7B{ K<6kl } V!AȀ-{8$zW3=7Q)9Q`ʡ9 ,yBʖjDC9ؿNHl`.)@F!RMA¨AZT>yUC%{ _g &YЄap~f\ :hP1NPbjb<8ox848^1[oѳ'7. e "#u":#s zyY=B)I;Gzt*PHv`<ĂTzT JbX(h7X X['0Vc)kՅe5,">zV>4,c|CШǦt.Fgif`` *F%@`EZKjDpd\gO)Io~M#U6U 6&gKyN}B'= pPPbɇ 4f xb [EOF .7*_chi!BD=7r.^13#AE"3V@_!`JN>R o|>Xd6Aկ(=:>#!Jv`.a]Lf^2x8G\1`v3+*]p_Ԥß$ )†j[_?!Rf<ĀW`Eh.jMj7M9Rde+#d`; *|befuE@ X FZ`}%g#HjinRA# `'0^+ŒnxaU)mJp `P"jFKj06 9Xjp% {OAiӍrR8׀&= Q 2/4\S2d"`%Y|Ւ8b6ٴx# )P~E c,Z8ee AOŷ'%w.f'cßH0;_~e3dZ B0(aWKh29,MSt)'|A%!@wǁg,nɽS Q$xTC*V9{`SXg-g=?oye9Pd-3+z`0`̱|GO~ 5 D x˓%.XhyAiW5'TK%0-i3lvVy2)x?^yB7le+Ѥ($&5?4v;xb =tYfp15w ^g3?,8h5{ł4l+O Q6c K\vt @w8֚}Nyt8%D+@Beao;`GjN<{c]5;x d;NH q=SfB?PrT6S[̢5@%TJa ؋{ 'v|1Ef<`hylP''Lx|m\*Ӕ%w§$ed\*ĕ3t<`ԗrmC=xԕ#m= vAK_$cu귞]e&BǞvf]RX5V1). Q jbۡT@]^RIeB$,ӜVjLB>gOEy-E^z}B)]n pP#hreP?OEj@26{ MK`zd}ZS`~ 9`pn.S7c/j3@*,S0CM>qI~gs0Q sҸ= | L¯mJ?-ͅڵ0*0;]'N{յfg[͆_Zcx񾗤ߩC]jʷw=] *Ro!=p,%"3]% MfHKrpɞI4޺ƻciN:Zdg >=]|2 ǐ VCwozu|$~^2?ۀc!o3pnZʇט=YV~(13]=Ǧf4@h7uؒ!ǝ&/^:K쫇k?)Zk-fg;O;.6q~nЂ6FBk~Q^l]Ari6adhR7Px槻~Oyh IQ JDK֎vБrڢdܫ<]3Q\f x#Xև0;-(2DoRZ6 xBd|͑K3ʗҭ!kDa`1eGf ⵐ׋`75pf!) 'Ivm6>i^) .`6):X l/3;tP kE pcΈxCk ˀܱ{-CCLp))et#TB?^HtU t@*KOlxN|w {>/&vI;Ix`#uJmMfj|S0d$u@YI=4 _R+#{#ڻ_(mjsKUiXJN;/>cό]-\ l2] ["+Jdi6Rx!<Ʒ}WrXFˉipy  [% !_~n_B<%ɉ#lRa@R2f|{ _",0;?R{ROR2R.BQ#;/c "0(-lcX = kdA*4 (I-N4 zwpKf7mha8IK)^ `թaytH%n`>ΠlΡa¼҆wJ8 nۓx 1UXmprak *^qNz=aND?%Xn9BN*Kr.( dXO+=M3@ nE_‡V5Xƥ" *tPpSBDž ;RG|ު\5 NV"̒ar%7UTMs3 ͓7dnwPU}l>oƸ7VsW??$Ulmw " -Yl(GؠD0IXz$i;۵jkҿʈ700܈_0 8kFh֯Nw-0MprF|ЗɼVH?13*rYw_3t=aYcCXfDD*q';4d:\ƬIENDB`weboob-1.2/modules/creditcooperatif/module.py000066400000000000000000000053321303450110500214610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import find_object from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .perso.browser import CreditCooperatif as CreditCooperatifPerso from .pro.browser import CreditCooperatif as CreditCooperatifPro __all__ = ['CreditCooperatifModule'] class CreditCooperatifModule(Module, CapBank): NAME = 'creditcooperatif' MAINTAINER = u'Kevin Pouget' EMAIL = 'weboob@kevin.pouget.me' VERSION = '1.2' DESCRIPTION = u'Crédit Coopératif' LICENSE = 'AGPLv3+' auth_type = {'particular': "Interface Particuliers", 'weak' : "Code confidentiel (pro)", 'strong': "Sesame (pro)"} CONFIG = BackendConfig(Value('auth_type', label='Type de compte', choices=auth_type, default="particular"), ValueBackendPassword('login', label='Code utilisateur', masked=False), ValueBackendPassword('password', label='Code confidentiel ou code PIN')) def create_default_browser(self): if self.config['auth_type'].get() == 'particular': self.BROWSER = CreditCooperatifPerso return self.create_browser(self.config['login'].get(), self.config['password'].get()) else: self.BROWSER = CreditCooperatifPro return self.create_browser(self.config['login'].get(), self.config['password'].get(), strong_auth=self.config['auth_type'].get() == "strong") def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_coming(account) weboob-1.2/modules/creditcooperatif/perso/000077500000000000000000000000001303450110500207475ustar00rootroot00000000000000weboob-1.2/modules/creditcooperatif/perso/__init__.py000066400000000000000000000000001303450110500230460ustar00rootroot00000000000000weboob-1.2/modules/creditcooperatif/perso/browser.py000066400000000000000000000064031303450110500230070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword, BrowserUnavailable from .pages import LoginPage, CreditLoggedPage, AccountsPage, TransactionsPage, \ TransactionsJSONPage, ComingTransactionsPage, IbanPage __all__ = ['CreditCooperatif'] class CreditCooperatif(LoginBrowser): BASEURL = "https://www.credit-cooperatif.coop" loginpage = URL('/portail//particuliers/login.do', LoginPage) loggedpage = URL('/portail/particuliers/authentification.do', CreditLoggedPage) accountspage = URL('/portail/particuliers/mescomptes/synthese.do', AccountsPage) transactionpage = URL('/portail/particuliers/mescomptes/relevedesoperations.do', TransactionsPage) transactjsonpage = URL('/portail/particuliers/mescomptes/relevedesoperationsjson.do', TransactionsJSONPage) comingpage = URL('/portail/particuliers/mescomptes/synthese/operationsencourslien.do', ComingTransactionsPage) iban = URL('/portail/particuliers/mesoperations/ribiban/telechargementribajax.do\?accountExternalNumber=(?P.*)', IbanPage) def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.loginpage.stay_or_go() self.page.login(self.username, self.password) if self.loggedpage.is_here(): error = self.page.get_error() if error is None: return else: raise BrowserUnavailable("not on the login page") raise BrowserIncorrectPassword(error) @need_login def get_accounts_list(self): self.accountspage.stay_or_go() return self.page.get_list() @need_login def get_history(self, account): data = {'accountExternalNumber': account.id} self.transactionpage.go(data=data) data = {'iDisplayLength': 400, 'iDisplayStart': 0, 'iSortCol_0': 0, 'iSortingCols': 1, 'sColumns': '', 'sEcho': 1, 'sSortDir_0': 'asc', } self.transactjsonpage.go(data=data) return self.page.get_transactions() @need_login def get_coming(self, account): data = {'accountExternalNumber': account.id} self.comingpage.go(data=data) assert self.comingpage.is_here() return self.page.get_transactions() weboob-1.2/modules/creditcooperatif/perso/pages.py000066400000000000000000000151151303450110500224230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Kevin Pouget, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.tools.json import json from weboob.capabilities.bank import Account, NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.pages import HTMLPage, JsonPage, LoggedPage from weboob.browser.filters.standard import Filter, Format, CleanText, CleanDecimal, \ BrowserURL, Field, Async, AsyncLoad from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.exceptions import ServerError class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form(xpath='//form[@id="AuthForm"]') form['j_username'] = login.encode('iso-8859-15') form['j_password'] = password.encode('iso-8859-15') form.submit() class CreditLoggedPage(HTMLPage): def get_error(self): div = self.doc.xpath('//div[@class="errorForm-msg"]') if len(div) == 0: return None msg = u', '.join([li.text.strip() for li in div[0].xpath('.//li')]) return re.sub('[\r\n\t\xa0]+', ' ', msg) class AddType(Filter): types = {u'COMPTE NEF': Account.TYPE_CHECKING, u'CPTE A VUE': Account.TYPE_CHECKING, u'LIVRET AGIR': Account.TYPE_SAVINGS, u'LIVRET A PART': Account.TYPE_SAVINGS} def filter(self, str_type): for key, acc_type in self.types.items(): if key == str_type: return acc_type return Account.TYPE_UNKNOWN class AccountsPage(LoggedPage, HTMLPage): @method class get_list(ListElement): item_xpath = '//table[has-class("table-synthese")]' class item(ItemElement): klass = Account load_details = BrowserURL('iban', account_id=Field('id')) & AsyncLoad obj_label = Format('%s %s', CleanText('.//h2[@class="tt_compte"][1]'), CleanText('.//ul[@class="nClient"]/li[1]')) obj_id = CleanText('.//ul[@class="nClient"]/li[last()]', symbols=u'N°') obj_type = AddType(CleanText('.//h2[@class="tt_compte"][1]')) obj_balance = CleanDecimal('.//td[@class="sum_solde"]//span[last()]', replace_dots=True) obj_currency = u'EUR' def obj_iban(self): try: return Async('details', CleanText('(.//div[@class="iban"]/p)[1]', replace=[(' ', '')]))(self) except ServerError: return NotAvailable class IbanPage(LoggedPage, HTMLPage): pass class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^(?PRETRAIT DAB) (?P
\d{2})-(?P\d{2})-([\d\-]+)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^RETRAIT DAB (?P
\d{2})-(?P\d{2})-([\d\-]+) (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^CARTE (?P
\d{2})(?P\d{2}) \d+ (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^VIR COOPA (?P
\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^COOPA (?P
\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^VIR(EMENT|EMT| SEPA EMET :)? (?P.*?)(- .*)?$'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(PRLV|PRELEVEMENT) SEPA (?P.*?)(- .*)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^(PRLV|PRELEVEMENT) (?P.*?)(- .*)?$'), FrenchTransaction.TYPE_ORDER), (re.compile('^CHEQUE.*'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^ABONNEMENT (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class TransactionsPage(LoggedPage, HTMLPage): pass class TransactionsJSONPage(LoggedPage, JsonPage): ROW_DATE = 0 ROW_TEXT = 2 ROW_CREDIT = -1 ROW_DEBIT = -2 def get_transactions(self): seen = set() for tr in self.doc['exportData'][1:]: t = Transaction() t.parse(tr[self.ROW_DATE], tr[self.ROW_TEXT]) t.set_amount(tr[self.ROW_CREDIT], tr[self.ROW_DEBIT]) t.id = t.unique_id(seen) yield t class ComingTransactionsPage(LoggedPage, HTMLPage): ROW_REF = 0 ROW_TEXT = 1 ROW_DATE = 2 ROW_CREDIT = -1 ROW_DEBIT = -2 def get_transactions(self): data = [] for script in self.doc.xpath('//script'): txt = script.text if txt is None: continue pattern = 'var jsonData =' start = txt.find(pattern) if start < 0: continue txt = txt[start+len(pattern):start+txt[start:].find(';')] data = json.loads(txt) break for tr in data: t = Transaction() t.parse(tr[self.ROW_DATE], tr[self.ROW_TEXT]) t.set_amount(tr[self.ROW_CREDIT], tr[self.ROW_DEBIT]) yield t weboob-1.2/modules/creditcooperatif/pro/000077500000000000000000000000001303450110500204175ustar00rootroot00000000000000weboob-1.2/modules/creditcooperatif/pro/__init__.py000066400000000000000000000000001303450110500225160ustar00rootroot00000000000000weboob-1.2/modules/creditcooperatif/pro/browser.py000066400000000000000000000100341303450110500224520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.exceptions import BrowserUnavailable from .pages import LoginPage, AccountsPage, ITransactionsPage, TransactionsPage, ComingTransactionsPage, CardTransactionsPage, \ TechnicalErrorPage __all__ = ['CreditCooperatif'] class CreditCooperatif(Browser): PROTOCOL = 'https' ENCODING = 'iso-8859-15' DOMAIN = "www.coopanet.com" PAGES = {'https://www.coopanet.com/banque/sso/.*': LoginPage, 'https://www.coopanet.com/banque/cpt/incoopanetj2ee.do.*': AccountsPage, 'https://www.coopanet.com/banque/cpt/cpt/situationcomptes.do\?lnkReleveAction=X&numeroExterne=.*': TransactionsPage, 'https://www.coopanet.com/banque/cpt/cpt/relevecompte.do\?tri_page=.*': TransactionsPage, 'https://www.coopanet.com/banque/cpt/cpt/situationcomptes.do\?lnkOpCB=X&numeroExterne=.*': CardTransactionsPage, 'https://www.coopanet.com/banque/cpt/cpt/operationscartebancaire.do\?.*': CardTransactionsPage, 'https://www.coopanet.com/banque/cpt/cpt/situationcomptes.do\?lnkOpEC=X&numeroExterne=.*': ComingTransactionsPage, 'https://www.coopanet.com/banque/cpt/cpt/operationEnCours.do.*': ComingTransactionsPage, 'http://www.coopanet.com/PbTechniqueCoopanet.htm': TechnicalErrorPage, } def __init__(self, *args, **kwargs): self.strong_auth = kwargs.pop('strong_auth', False) Browser.__init__(self, *args, **kwargs) def home(self): self.location("/banque/sso/") if self.is_on_page(TechnicalErrorPage): raise BrowserUnavailable() assert self.is_on_page(LoginPage) def is_logged(self): return not self.is_on_page(LoginPage) def login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert isinstance(self.strong_auth, bool) if self.is_logged(): return if not self.is_on_page(LoginPage): self.home() self.page.login(self.username, self.password, self.strong_auth) if not self.is_logged(): raise BrowserIncorrectPassword() def get_accounts_list(self): self.location(self.buildurl('/banque/cpt/incoopanetj2ee.do?ssomode=ok')) return self.page.get_list() def _get_history(self, link): self.location(link) while True: assert self.is_on_page(ITransactionsPage) for tr in self.page.get_history(): yield tr next_url = self.page.get_next_url() if next_url is None: return self.location(next_url) def get_history(self, account): return self._get_history('/banque/cpt/cpt/situationcomptes.do?lnkReleveAction=X&numeroExterne='+ account.id) def get_coming(self, account): # credit cards transactions for tr in self._get_history('/banque/cpt/cpt/situationcomptes.do?lnkOpCB=X&numeroExterne='+ account.id): yield tr # coming transactions for tr in self._get_history('/banque/cpt/cpt/situationcomptes.do?lnkOpEC=X&numeroExterne='+ account.id): yield tr weboob-1.2/modules/creditcooperatif/pro/pages.py000066400000000000000000000147011303450110500220730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal import re import time from weboob.deprecated.browser import Page from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction class TechnicalErrorPage(Page): pass class LoginPage(Page): def login(self, login, pin, strong_auth): form_nb = 1 if strong_auth else 0 indentType = "RENFORCE" if strong_auth else "MDP" self.browser.select_form(name='loginCoForm', nr=form_nb) self.browser['codeUtil'] = login.encode(self.browser.ENCODING) self.browser['motPasse'] = pin[:12].encode(self.browser.ENCODING) assert self.browser['identType'] == indentType self.browser.submit(nologin=True) class AccountsPage(Page): ACCOUNT_TYPES = {u'COMPTE NEF': Account.TYPE_CHECKING} CPT_ROW_ID = 0 CPT_ROW_NAME = 1 CPT_ROW_NATURE = 2 CPT_ROW_BALANCE = 3 CPT_ROW_ENCOURS = 4 def is_error(self): for par in self.document.xpath('//p[@class=acctxtnoirlien]'): if par.text is not None and u"La page demandée ne peut pas être affichée." in par.text: return True return False def get_list(self): for trCompte in self.document.xpath('//table[@id="compte"]/tbody/tr'): tds = trCompte.findall('td') account = Account() account.id = tds[self.CPT_ROW_ID].text.strip() account.label = unicode(tds[self.CPT_ROW_NAME].text.strip()) account_type_str = "".join([td.text for td in tds[self.CPT_ROW_NATURE].xpath('.//td[@class="txt" and @width="99%"]')]).strip() account.type = self.ACCOUNT_TYPES.get(account_type_str, Account.TYPE_UNKNOWN) account.balance = Decimal(FrenchTransaction.clean_amount(self.parser.tocleanstring(tds[self.CPT_ROW_BALANCE]))) account.coming = Decimal(FrenchTransaction.clean_amount(self.parser.tocleanstring( tds[self.CPT_ROW_ENCOURS]))) account.currency = account.get_currency(tds[self.CPT_ROW_BALANCE].find("a").text) yield account return class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^RETRAIT DAB (?P.*?).*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?P.*) RETRAIT DU (?P
\d{2})(?P\d{2})(?P\d{2}) .*'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^CARTE \d+ .*'), FrenchTransaction.TYPE_CARD), (re.compile('^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(PRLV|PRELEVEMENT) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^CHEQUE.*'), FrenchTransaction.TYPE_CHECK), (re.compile('^(AGIOS /|FRAIS) (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^ABONNEMENT (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^(?P.*)( \d+)? QUITTANCE .*'), FrenchTransaction.TYPE_ORDER), (re.compile('^.* LE (?P
\d{2})/(?P\d{2})/(?P\d{2})$'), FrenchTransaction.TYPE_UNKNOWN), ] class ITransactionsPage(Page): def get_next_url(self): # can be 'Suivant' or ' Suivant' next = self.document.xpath("//a[normalize-space(text()) = 'Suivant']") if not next: return None return next[0].attrib["href"] def get_history(self): raise NotImplementedError() class TransactionsPage(ITransactionsPage): TR_DATE = 0 TR_TEXT = 2 TR_DEBIT = 3 TR_CREDIT = 4 TABLE_NAME = 'operation' def get_history(self): for tr in self.document.xpath('//table[@id="%s"]/tbody/tr' % self.TABLE_NAME): tds = tr.findall('td') def get_content(td): ret = self.parser.tocleanstring(td) return ret.replace(u"\xa0", " ").strip() date = get_content(tds[self.TR_DATE]) raw = get_content(tds[self.TR_TEXT]) debit = get_content(tds[self.TR_DEBIT]) credit = get_content(tds[self.TR_CREDIT]) t = Transaction() t.parse(date, re.sub(r'[ ]+', ' ', raw)) t.set_amount(credit, debit) yield t class ComingTransactionsPage(TransactionsPage): TR_DATE = 2 TR_TEXT = 1 TR_DEBIT = -2 TR_CREDIT = -1 TABLE_NAME = 'operationAVenir' class CardTransactionsPage(ITransactionsPage): COM_TR_COMMENT = 0 COM_TR_DATE = 1 COM_TR_TEXT = 2 COM_TR_VALUE = 3 def get_history(self): comment = None for tr in self.document.xpath('//table[@id="operation"]/tbody/tr'): tds = tr.findall('td') def get_content(td): ret = td.text return ret.replace(u"\xa0", " ").strip() raw = get_content(tds[self.COM_TR_TEXT]) if comment is None: comment = get_content(tds[self.COM_TR_COMMENT]) raw = "%s (%s) " % (raw, comment) debit = get_content(tds[self.COM_TR_VALUE]) date = get_content(tds[self.COM_TR_DATE]) if comment is not None: #date is 'JJ/MM'. add '/YYYY' date += comment[comment.rindex("/"):] else: date += "/%d" % time.localtime().tm_year t = Transaction() t.parse(date, re.sub(r'[ ]+', ' ', raw)) t.set_amount("", debit) yield t weboob-1.2/modules/creditcooperatif/test.py000066400000000000000000000020471303450110500211530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Kevin Pouget # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CreditCooperatifTest(BackendTest): MODULE = 'creditcooperatif' def test_creditcoop(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) list(self.backend.iter_coming(a)) weboob-1.2/modules/creditdunord/000077500000000000000000000000001303450110500167575ustar00rootroot00000000000000weboob-1.2/modules/creditdunord/__init__.py000066400000000000000000000014441303450110500210730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CreditDuNordModule __all__ = ['CreditDuNordModule'] weboob-1.2/modules/creditdunord/browser.py000066400000000000000000000130461303450110500210200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account from .pages import LoginPage, AccountsPage, ProAccountsPage, TransactionsPage, \ ProTransactionsPage, IbanPage, RedirectPage, AVPage class CreditDuNordBrowser(LoginBrowser): ENCODING = 'UTF-8' login = URL('$', '/.*\?.*_pageLabel=page_erreur_connexion', LoginPage) redirect = URL('/swm/redirectCDN.html', RedirectPage) av = URL('/vos-comptes/particuliers/V1_transactional_portal_page_', AVPage) accounts = URL('/vos-comptes/particuliers', '/vos-comptes/particuliers/transac_tableau_de_bord', AccountsPage) transactions = URL('/vos-comptes/.*/transac/particuliers', TransactionsPage) proaccounts = URL('/vos-comptes/(professionnels|entreprises)', ProAccountsPage) protransactions = URL('/vos-comptes/.*/transac/(professionnels|entreprises)', ProTransactionsPage) loans = URL('/vos-comptes/professionnels/credit_en_cours', ProAccountsPage) iban = URL('/vos-comptes/IPT/cdnProxyResource/transacClippe/RIB_impress.asp', IbanPage) account_type = 'particuliers' def __init__(self, website, *args, **kwargs): super(CreditDuNordBrowser, self).__init__(*args, **kwargs) self.BASEURL = "https://%s" % website def is_logged(self): return self.page is not None and not self.login.is_here() and \ not self.page.doc.xpath(u'//b[contains(text(), "vous devez modifier votre code confidentiel")]') def home(self): if self.is_logged(): self.location('/vos-comptes/%s' % self.account_type) self.location(self.page.doc.xpath(u'//a[contains(text(), "Synthèse")]')[0].attrib['href']) else: self.do_login() def do_login(self): self.login.go().login(self.username, self.password) if self.login.is_here(): raise BrowserIncorrectPassword(self.page.get_error()) if not self.is_logged(): raise BrowserIncorrectPassword() m = re.match('https://[^/]+/vos-comptes/(\w+).*', self.url) if m: self.account_type = m.group(1) @need_login def _iter_accounts(self): self.home() self.location(self.page.get_av_link()) if self.av.is_here(): for a in self.page.get_av_accounts(): self.location(a._link, data=a._args) self.location(a._link.replace("_attente", "_detail_contrat_rep"), data=a._args) self.page.fill_diff_currency(a) yield a self.home() for a in self.page.get_list(): yield a self.loans.go() for a in self.page.get_list(): yield a @need_login def get_accounts_list(self): accounts = list(self._iter_accounts()) self.page.iban_page() link = self.page.iban_go() for a in [a for a in accounts if a._acc_nb]: self.location(link + a._acc_nb) a.iban = self.page.get_iban() return iter(accounts) def get_account(self, id): assert isinstance(id, basestring) for a in self._iter_accounts(): if a.id == id: return a return None @need_login def iter_transactions(self, link, args, acc_type): if args is None: return while args is not None: self.location(link, data=args) assert self.transactions.is_here() for tr in self.page.get_history(acc_type): yield tr args = self.page.get_next_args(args) @need_login def get_history(self, account, coming=False): if coming and account.type is not Account.TYPE_CARD or account.type is Account.TYPE_LOAN: return iter([]) transactions = [] for tr in self.iter_transactions(account._link, account._args, account.type): transactions.append(tr) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def get_investment(self, account): if not account._inv: return iter([]) investments = [] if (account.type == Account.TYPE_MARKET): self.location(account._link, data=account._args) investments = [i for i in self.page.get_market_investment()] elif (account.type == Account.TYPE_LIFE_INSURANCE): self.location(account._link, data=account._args) self.location(account._link.replace("_attente", "_detail_contrat_rep"), data=account._args) investments = [i for i in self.page.get_deposit_investment()] return iter(investments) weboob-1.2/modules/creditdunord/favicon.png000066400000000000000000000040671303450110500211210ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME GlIDATx}lU?Z[%:k*@s4qJdd73MX6ML/۲,efˌ!dd2Z]$p^ oc Cp }\z{{}iWins>o9ʶmf>Tyma7۳&K@>a.O5εY@y&(Q^mk.(Ks@󝔯kqvf(=` PB!PZe֯k9`pƀs?Ӕ " O/ZybH1f I<e%f0,U`l*WĜYQ oˁo͆2p\eY z4P(UC)dk5Y x´T(dpFy&$] A5PY_[kY=r5@2W >pA$Q~:H(6vFYt$QMFbY_5$Wni;)xE`jEXHopj{vH0efJ?[WӁ`p?0KZ%K sʇ&E亣>QN umd oˀ <1X4,I+GkI/[H|dd2`[µ=:eƐfrZ.L%9YT6 '+,Cq0;ƩC<.qIٶj(4EY$<gJ;-C?$OvѮ5Dy$!!aB,S,C9?OQp7U÷L=b `v7; 8 rh/+,C_\vo?>z.NA^M<"vұMeˤ_] 薡 ?oO#f6dzkct'r]p,4'L" 'z#}|$3; vp<oW K抶^T^mX>TB /; "y@GbDܷP=i-[B+"ԋʫp*V _? n<i]Yh&tP)_&kۊ?c4ZC30YZW.aG)Z/UZ%a%GwH[>엲88<m]ڽn?Yr`i S٦_UTK[9W,]o2s eY,w+w՜ߊGREHB%7@K%l^uPʙS<㸌Ld/gxxio2(m!06f18l14Ctb0%?2R3xr\`v>?nigfIENDB`weboob-1.2/modules/creditdunord/module.py000066400000000000000000000063221303450110500206210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import ValueBackendPassword, Value from .browser import CreditDuNordBrowser __all__ = ['CreditDuNordModule'] class CreditDuNordModule(Module, CapBank): NAME = 'creditdunord' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Crédit du Nord, Banque Courtois, Kolb, Nuger, Tarneaud, Société Marseillaise de Crédit' LICENSE = 'AGPLv3+' website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'www.credit-du-nord.fr': u'Crédit du Nord', 'www.banque-courtois.fr': u'Banque Courtois', 'www.banque-kolb.fr': u'Banque Kolb', 'www.banque-nuger.fr': u'Banque Nuger', 'www.banque-rhone-alpes.fr': u'Banque Rhône-Alpes', 'www.tarneaud.fr': u'Tarneaud', 'www.smc.fr': u'Société Marseillaise de Crédit', }.iteritems(), key=lambda k_v: (k_v[1], k_v[0]))]) CONFIG = BackendConfig(Value('website', label='Banque', choices=website_choices, default='www.credit-du-nord.fr'), ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Code confidentiel')) BROWSER = CreditDuNordBrowser def create_default_browser(self): return self.create_browser(self.config['website'].get(), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): account = self.get_account(account.id) for tr in self.browser.get_history(account): if not tr._is_coming: yield tr def iter_coming(self, account): account = self.get_account(account.id) for tr in self.browser.get_history(account, coming=True): if tr._is_coming: yield tr def iter_investment(self, account): account = self.get_account(account.id) return self.browser.get_investment(account) weboob-1.2/modules/creditdunord/pages.py000077500000000000000000000527041303450110500204430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from urllib import quote from decimal import Decimal from io import BytesIO from datetime import date as da from lxml import html from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.filters.standard import CleanText, Date, CleanDecimal, Regexp from weboob.exceptions import ActionNeeded, BrowserIncorrectPassword, BrowserUnavailable from weboob.capabilities.bank import Account, Investment from weboob.capabilities import NotAvailable from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.captcha.virtkeyboard import GridVirtKeyboard from weboob.tools.json import json def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) def MyStrip(x, xpath='.'): return CleanText(xpath)(html.fromstring("

%s

" % x)) if isinstance(x, basestring) else \ CleanText(xpath)(html.fromstring(CleanText('.')(x))) class CDNVirtKeyboard(GridVirtKeyboard): symbols = {'0': '3de2346a63b658c977fce4da925ded28', '1': 'c571018d2dc267cdf72fafeeb9693037', '2': '72d7bad4beb833d85047f6912ed42b1d', '3': 'fbfce4677a8b2f31f3724143531079e3', '4': '54c723c5b0b5848a0475b4784100b9e0', '5': 'd00164307cacd4ca21b930db09403baa', '6': '101adc6f5d03df0f512c3ec2bef88de9', '7': '3b48f598209718397eb1118d81cf07ba', '8': '881f0acdaba2c44b6a5e64331f4f53d3', '9': 'a47d9a0a2ebbc65a0e625f20cb07822b', } margin = 1 color = (0xff,0xf7,0xff) nrow = 4 ncol = 4 def __init__(self, browser, crypto, grid): f = BytesIO(browser.open('/sec/vk/gen_ui?modeClavier=0&cryptogramme=%s' % crypto).content) super(CDNVirtKeyboard, self).__init__(range(16), self.ncol, self.nrow, f, self.color) self.check_symbols(self.symbols, browser.responses_dirname) self.codes = grid def check_color(self, pixel): for p in pixel: if p > 0xd0: return False return True def get_string_code(self, string): res = [] ndata = self.nrow * self.ncol for nbchar, c in enumerate(string): index = self.get_symbol_code(self.symbols[c]) res.append(self.codes[(nbchar * ndata) + index]) return ','.join(res) class RedirectPage(HTMLPage): def on_load(self): for script in self.doc.xpath('//script'): self.browser.location(re.search(r'href="([^"]+)"', script.text).group(1)) class LoginPage(HTMLPage): def login(self, username, password): login_selector = self.doc.xpath('//input[@id="codsec"]') if login_selector: if not password.isdigit() or not len(password) == 6: raise BrowserIncorrectPassword('The credentials have changed on website %s. Please update them.' % self.browser.BASEURL) self.vk_login(username, password) else: self.classic_login(username,password) def vk_login(self, username, password): res = self.browser.open('/sec/vk/gen_crypto?estSession=0').content crypto = re.search(r"'crypto': '([^']+)'", res).group(1) grid = re.search(r"'grid': \[([^\]]+)]", res).group(1).split(',') vk = CDNVirtKeyboard(self.browser, crypto, grid) data = {'user_id': username, 'codsec': vk.get_string_code(password), 'cryptocvcs': crypto, 'vk_op': 'auth', } self.browser.location('/swm/redirectCDN.html', data=data) def classic_login(self, username, password): m = re.match('www.([^\.]+).fr', self.browser.BASEURL) if not m: bank_name = 'credit-du-nord' self.logger.error('Unable to find bank name for %s' % self.browser.BASEURL) else: bank_name = m.group(1) data = {'bank': bank_name, 'pagecible': 'vos-comptes', 'password': password.encode(self.browser.ENCODING), 'pwAuth': 'Authentification+mot+de+passe', 'username': username.encode(self.browser.ENCODING), } self.browser.location('/saga/authentification', data=data) def get_error(self): return CleanText('//b[has-class("x-attentionErreurLigneHaut")]', default="")(self.doc) class CDNBasePage(HTMLPage): def get_from_js(self, pattern, end, is_list=False): """ find a pattern in any javascript text """ value = None for script in self.doc.xpath('//script'): txt = script.text if txt is None: continue start = txt.find(pattern) if start < 0: continue while True: if value is None: value = '' else: value += ',' value += txt[start+len(pattern):start+txt[start+len(pattern):].find(end)+len(pattern)] if not is_list: break txt = txt[start+len(pattern)+txt[start+len(pattern):].find(end):] start = txt.find(pattern) if start < 0: break return value def get_execution(self): return self.get_from_js("name: 'execution', value: '", "'") def iban_go(self): return '%s%s' % ('/vos-comptes/IPT/cdnProxyResource', self.get_from_js('C_PROXY.StaticResourceClientTranslation( "', '"')) class AccountsPage(LoggedPage, CDNBasePage): COL_HISTORY = 2 COL_FIRE_EVENT = 3 COL_ID = 4 COL_LABEL = 5 COL_BALANCE = -1 TYPES = {u'CARTE': Account.TYPE_CARD, u'COMPTE COURANT': Account.TYPE_CHECKING, u'COMPTE EPARGNE': Account.TYPE_SAVINGS, u'COMPTE SUR LIVRET': Account.TYPE_SAVINGS, u'LIVRET': Account.TYPE_SAVINGS, u'P.E.A.': Account.TYPE_MARKET, u'PEA': Account.TYPE_MARKET, u'TITRES': Account.TYPE_MARKET, u'ÉTOILE AVANCE': Account.TYPE_LOAN, u'PRÊT': Account.TYPE_LOAN, u'CREDIT': Account.TYPE_LOAN, u'FACILINVEST': Account.TYPE_LOAN, } def get_account_type(self, label): for pattern, actype in self.TYPES.iteritems(): if label.startswith(pattern) or label.endswith(pattern): return actype return Account.TYPE_UNKNOWN def get_history_link(self): return CleanText().filter(self.get_from_js(",url: Ext.util.Format.htmlDecode('", "'")).replace('&', '&') def get_av_link(self): return self.doc.xpath('//a[contains(text(), "Consultation")]')[0].attrib['href'] def get_list(self): accounts = [] txt = self.get_from_js('_data = new Array(', ');', is_list=True) if txt is None: raise BrowserUnavailable('Unable to find accounts list in scripts') data = json.loads('[%s]' % txt.replace("'", '"')) for line in data: a = Account() a.id = line[self.COL_ID].replace(' ', '') a._acc_nb = a.id.split('_')[0] if len(a.id.split('_')) > 1 else None a.label = MyStrip(line[self.COL_LABEL], xpath='.//div[@class="libelleCompteTDB"]') # This account can be multiple life insurance accounts if a.label == 'ASSURANCE VIE-BON CAPI-SCPI-DIVERS *': continue a.balance = Decimal(FrenchTransaction.clean_amount(line[self.COL_BALANCE])) a.currency = a.get_currency(line[self.COL_BALANCE]) a.type = self.get_account_type(a.label) if line[self.COL_HISTORY] == 'true': a._inv = False a._link = self.get_history_link() a._args = {'_eventId': 'clicDetailCompte', '_ipc_eventValue': '', '_ipc_fireEvent': '', 'deviseAffichee': 'DEVISE', 'execution': self.get_execution(), 'idCompteClique': line[self.COL_ID], } else: a._inv = True a._args = {'_ipc_eventValue': line[self.COL_ID], '_ipc_fireEvent': line[self.COL_FIRE_EVENT], } a._link = self.doc.xpath('//form[@name="changePageForm"]')[0].attrib['action'] if a.type is Account.TYPE_CARD: a.coming = a.balance a.balance = Decimal('0.0') accounts.append(a) return accounts def iban_page(self): form = self.get_form(name="changePageForm") form['_ipc_fireEvent'] = 'V1_rib' form['_ipc_eventValue'] = 'bouchon=bouchon' form.submit() class AVPage(LoggedPage, CDNBasePage): COL_LABEL = 0 COL_BALANCE = 3 ARGS = ['IndiceClassement', 'IndiceCompte', 'Banque', 'Agence', 'Classement', 'Serie', 'SScompte', 'Categorie', 'IndiceSupport', 'NumPolice', 'LinkHypertext'] def get_params(self, text): url = self.get_from_js('document.detail.action="', '";') args = {} l = [] for sub in re.findall("'([^']*)'", text): l.append(sub) for i, key in enumerate(self.ARGS): args[key] = unicode(l[self.ARGS.index(key)]).encode(self.browser.ENCODING) return url, args def get_av_accounts(self): for tr in self.doc.xpath('//table[@class="datas"]/tr[not(@class)]'): cols = tr.findall('td') if len(cols) != 4: continue a = Account() a.label = CleanText('.')(cols[self.COL_LABEL]) a.type = Account.TYPE_LIFE_INSURANCE a.balance = MyDecimal('.')(cols[self.COL_BALANCE]) a._link, a._args = self.get_params(cols[self.COL_LABEL].find('span/a').attrib['href']) a.id = a._args['IndiceSupport'] + a._args['NumPolice'] a._acc_nb = None a._inv = True yield a class ProAccountsPage(AccountsPage): COL_ID = 0 COL_BALANCE = 1 ARGS = ['Banque', 'Agence', 'Classement', 'Serie', 'SSCompte', 'Devise', 'CodeDeviseCCB', 'LibelleCompte', 'IntituleCompte', 'Indiceclassement', 'IndiceCompte', 'NomClassement'] def params_from_js(self, text): l = [] for sub in re.findall("'([^']*)'", text): l.append(sub) if len(l) <= 1: #For account that have no history return None, None url = '/vos-comptes/IPT/appmanager/transac/' + self.browser.account_type + '?_nfpb=true&_windowLabel=portletInstance_18&_pageLabel=page_synthese_v1' + '&_cdnCltUrl=' + "/transacClippe/" + quote(l.pop(0)) args = {} for input in self.doc.xpath('//form[@name="detail"]/input'): args[input.attrib['name']] = input.attrib.get('value', '') for i, key in enumerate(self.ARGS): args[key] = unicode(l[self.ARGS.index(key)]).encode(self.browser.ENCODING) args['PageDemandee'] = 1 args['PagePrecedente'] = 1 return url, args def get_list(self): no_accounts_message = self.doc.xpath(u'//span/b[contains(text(),"Votre abonnement est clôturé. Veuillez contacter votre conseiller.")]/text()') if no_accounts_message: raise ActionNeeded(no_accounts_message[0]) for tr in self.doc.xpath('//table[@class="datas"]//tr'): if tr.attrib.get('class', '') == 'entete': continue cols = tr.findall('td') a = Account() a.label = unicode(cols[self.COL_ID].xpath('.//span[@class="left-underline"] | .//span[@class="left"]/a')[0].text.strip()) a.type = self.get_account_type(a.label) balance = CleanText('.')(cols[self.COL_BALANCE]) if balance == '': continue a.balance = CleanDecimal(replace_dots=True).filter(balance) a.currency = a.get_currency(balance) if cols[self.COL_ID].find('a'): a._link, a._args = self.params_from_js(cols[self.COL_ID].find('a').attrib['href']) a._acc_nb = cols[self.COL_ID].xpath('.//span[@class="right-underline"] | .//span[@class="right"]')[0].text.replace(' ', '').strip() if hasattr(a, '_args') and a._args: a.id = '%s%s%s' % (a._acc_nb, a._args['IndiceCompte'], a._args['Indiceclassement']) else: a.id = a._acc_nb # This account can be multiple life insurance accounts if any(a.label.startswith(lab) for lab in ['ASS.VIE-BONS CAPI-SCPI-DIVERS', 'BONS CAPI-SCPI-DIVERS']) or \ u'Aucun d\\351tail correspondant pour ce compte' in tr.xpath('.//a/@href')[0]: continue if a.type is Account.TYPE_CARD: a.coming = a.balance a.balance = Decimal('0.0') a._inv = False yield a def iban_page(self): self.browser.location(self.doc.xpath('.//a[contains(text(), "Impression IBAN")]')[0].attrib['href']) class IbanPage(LoggedPage, HTMLPage): def get_iban(self): try: return unicode(self.doc.xpath('.//td[@width="315"]/font')[0].text.replace(' ', '').strip()) except AttributeError: return NotAvailable class Transaction(FrenchTransaction): PATTERNS = [(re.compile(r'^(?PRET DAB \w+ .*?) LE (?P
\d{2})(?P\d{2})$'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^VIR(EMENT)?( INTERNET)?(\.| )?(DE)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^PRLV (SEPA )?(DE )?(?P.*?)( Motif :.*)?$'), FrenchTransaction.TYPE_ORDER), (re.compile(r'^CB (?P.*) LE (?P
\d{2})\.?(?P\d{2})$'), FrenchTransaction.TYPE_CARD), (re.compile(r'^CHEQUE.*'), FrenchTransaction.TYPE_CHECK), (re.compile(r'^(CONVENTION \d+ )?COTISATION (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile(r'^REM(ISE)?\.?( CHQ\.)? .*'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(r'^(?P.*?)( \d{2}.*)? LE (?P
\d{2})\.?(?P\d{2})$'), FrenchTransaction.TYPE_CARD), (re.compile(r'^(?P.*?) LE (?P
\d{2}) (?P\d{2}) (?P\d{2})$'), FrenchTransaction.TYPE_CARD), ] class TransactionsPage(LoggedPage, CDNBasePage): COL_ID = 0 COL_DATE = -5 COL_DEBIT_DATE = -4 COL_LABEL = -3 COL_VALUE = -1 def get_next_args(self, args): if self.is_last(): return None args['_eventId'] = 'clicChangerPageSuivant' args['execution'] = self.get_execution() args.pop('idCompteClique', None) return args def is_last(self): for script in self.doc.xpath('//script'): txt = script.text if txt is None: continue if txt.find('clicChangerPageSuivant') >= 0: return False return True def condition(self, t, acc_type): if t.date is NotAvailable: return True t._is_coming = t.date > da.today() if t.raw.startswith('TOTAL DES') or t.raw.startswith('ACHATS CARTE'): t.type = t.TYPE_CARD_SUMMARY elif acc_type is Account.TYPE_CARD: t.type = t.TYPE_DEFERRED_CARD return False def get_history(self, acc_type): txt = self.get_from_js('ListeMvts_data = new Array(', ');\n') if txt is None: no_trans = self.get_from_js('js_noMvts = new Ext.Panel(', ')') if no_trans is not None: # there is no transactions for this account, this is normal. return else: # No history on this account return data = json.loads('[%s]' % txt.replace('"', '\\"').replace("'", '"')) for line in data: t = Transaction() if acc_type is Account.TYPE_CARD and MyStrip(line[self.COL_DEBIT_DATE]): date = vdate = Date(dayfirst=True).filter(MyStrip(line[self.COL_DEBIT_DATE])) else: date = Date(dayfirst=True).filter(MyStrip(line[self.COL_DATE])) vdate = MyStrip(line[self.COL_DEBIT_DATE]) if vdate != '': vdate = Date(dayfirst=True).filter(vdate) raw = MyStrip(line[self.COL_LABEL]) t.parse(date, raw, vdate=vdate) t.set_amount(line[self.COL_VALUE]) if self.condition(t, acc_type): continue yield t def get_market_investment(self): COL_LABEL = 0 COL_QUANTITY = 1 COL_UNITPRICE = 2 COL_UNITVALUE = 3 COL_VALUATION = 4 COL_PERF = 5 for table in self.doc.xpath('//table[@class="datas-large"]'): for tr in table.xpath('.//tr[not(@class="entete")]'): cols = tr.findall('td') if len(cols) < 7: continue delta = 0 if len(cols) == 9: delta = 1 inv = Investment() inv.code = CleanText('.')(cols[COL_LABEL + delta].xpath('.//span')[1]).split(' ')[0].split(u'\xa0')[0] inv.label = CleanText('.')(cols[COL_LABEL + delta].xpath('.//span')[0]) inv.quantity = MyDecimal('.')(cols[COL_QUANTITY + delta]) inv.unitprice = MyDecimal('.')(cols[COL_UNITPRICE + delta]) inv.unitvalue = MyDecimal('.')(cols[COL_UNITVALUE + delta]) inv.valuation = MyDecimal('.')(cols[COL_VALUATION + delta]) inv.diff = MyDecimal('.')(cols[COL_PERF + delta]) yield inv def get_deposit_investment(self): COL_LABEL = 0 COL_QUANTITY = 3 COL_UNITVALUE = 4 COL_VALUATION = 5 for tr in self.doc.xpath('//table[@class="datas"]/tr[not(@class="entete")]'): cols = tr.findall('td') inv = Investment() inv.label = CleanText('.')(cols[COL_LABEL].xpath('.//a')[0]) inv.code = CleanText('./text()')(cols[COL_LABEL]) inv.quantity = MyDecimal('.')(cols[COL_QUANTITY]) inv.unitvalue = MyDecimal().filter(CleanText('.')(cols[COL_UNITVALUE]).split()[0]) if inv.unitvalue is not NotAvailable: inv.vdate = Date(dayfirst=True).filter(Regexp(CleanText('.'), '(\d{2})/(\d{2})/(\d{4})', '\\3-\\2-\\1')(cols[COL_UNITVALUE])) inv.valuation = MyDecimal('.')(cols[COL_VALUATION]) yield inv def fill_diff_currency(self, account): valuation_diff = CleanText(u'//td[span[contains(text(), "dont +/- value : ")]]//b', default=None)(self.doc) if valuation_diff: account.valuation_diff = MyDecimal().filter(valuation_diff) account.currency = account.get_currency(valuation_diff) class ProTransactionsPage(TransactionsPage): def get_next_args(self, args): if len(self.doc.xpath('//a[contains(text(), "Suivant")]')) > 0: args['PageDemandee'] = int(args.get('PageDemandee', 1)) + 1 return args return None def parse_transactions(self): transactions = {} for script in self.doc.xpath('//script'): txt = script.text if txt is None: continue for i, key, value in re.findall('listeopecv\[(\d+)\]\[\'(\w+)\'\]="(.*)";', txt): i = int(i) if i not in transactions: transactions[i] = {} transactions[i][key] = value.strip() return transactions.iteritems() def get_history(self, acc_type): for i, tr in self.parse_transactions(): t = Transaction() if acc_type is Account.TYPE_CARD: date = vdate = Date(dayfirst=True, default=None).filter(tr['dateval']) else: date = Date(dayfirst=True, default=None).filter(tr['date']) vdate = Date(dayfirst=True, default=None).filter(tr['dateval']) or date raw = MyStrip(' '.join([tr['typeope'], tr['LibComp']])) t.parse(date, raw, vdate) t.set_amount(tr['mont']) if self.condition(t, acc_type): continue yield t weboob-1.2/modules/creditdunord/test.py000066400000000000000000000020001303450110500203000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CreditDuNordTest(BackendTest): MODULE = 'creditdunord' def test_creditdunord(self): l = list(self.backend.iter_accounts()) a = l[0] list(self.backend.iter_history(a)) list(self.backend.iter_coming(a)) weboob-1.2/modules/creditdunordpee/000077500000000000000000000000001303450110500174515ustar00rootroot00000000000000weboob-1.2/modules/creditdunordpee/__init__.py000066400000000000000000000014541303450110500215660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CreditdunordpeeModule __all__ = ['CreditdunordpeeModule'] weboob-1.2/modules/creditdunordpee/browser.py000066400000000000000000000043321303450110500215100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, need_login, URL from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.date import LinearDateGuesser from weboob.capabilities.bank import Account from .pages import LoginPage, HomePage, AvoirPage, HistoryPage class CreditdunordpeeBrowser(LoginBrowser): BASEURL = 'https://salaries.pee.credit-du-nord.fr' home = URL('/portal/fr/salarie-cdn/', HomePage) login = URL('/portal/login', LoginPage) avoir = URL(u'/portal/salarie-cdn/monepargne/mesavoirs', AvoirPage) history = URL(u'/portal/salarie-cdn/operations/consulteroperations\?scenario=ConsulterOperationsEffectuees', HistoryPage) def do_login(self): self.home.stay_or_go() passwd = self.page.get_coded_passwd(self.password) d = {'initialURI': "/portal/fr/salarie-cdn/", 'password': passwd, 'username': self.username} self.login.go(data=d) if not (self.home.is_here() and self.page.is_logged()): raise BrowserIncorrectPassword() @need_login def iter_accounts(self): account = Account(self.username) return self.avoir.go().get_account(obj=account) @need_login def get_history(self): transactions = list(self.history.go().get_history(date_guesser=LinearDateGuesser())) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def iter_investment(self): return self.avoir.go().iter_investment() weboob-1.2/modules/creditdunordpee/favicon.png000066400000000000000000000040671303450110500216130ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME GlIDATx}lU?Z[%:k*@s4qJdd73MX6ML/۲,efˌ!dd2Z]$p^ oc Cp }\z{{}iWins>o9ʶmf>Tyma7۳&K@>a.O5εY@y&(Q^mk.(Ks@󝔯kqvf(=` PB!PZe֯k9`pƀs?Ӕ " O/ZybH1f I<e%f0,U`l*WĜYQ oˁo͆2p\eY z4P(UC)dk5Y x´T(dpFy&$] A5PY_[kY=r5@2W >pA$Q~:H(6vFYt$QMFbY_5$Wni;)xE`jEXHopj{vH0efJ?[WӁ`p?0KZ%K sʇ&E亣>QN umd oˀ <1X4,I+GkI/[H|dd2`[µ=:eƐfrZ.L%9YT6 '+,Cq0;ƩC<.qIٶj(4EY$<gJ;-C?$OvѮ5Dy$!!aB,S,C9?OQp7U÷L=b `v7; 8 rh/+,C_\vo?>z.NA^M<"vұMeˤ_] 薡 ?oO#f6dzkct'r]p,4'L" 'z#}|$3; vp<oW K抶^T^mX>TB /; "y@GbDܷP=i-[B+"ԋʫp*V _? n<i]Yh&tP)_&kۊ?c4ZC30YZW.aG)Z/UZ%a%GwH[>엲88<m]ڽn?Yr`i S٦_UTK[9W,]o2s eY,w+w՜ߊGREHB%7@K%l^uPʙS<㸌Ld/gxxio2(m!06f18l14Ctb0%?2R3xr\`v>?nigfIENDB`weboob-1.2/modules/creditdunordpee/module.py000066400000000000000000000036701303450110500213160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.bank import CapBank from .browser import CreditdunordpeeBrowser __all__ = ['CreditdunordpeeModule'] class CreditdunordpeeModule(Module, CapBank): NAME = 'creditdunordpee' DESCRIPTION = u'Site de gestion du PEE du groupe Credit du nord' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = CreditdunordpeeBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', regexp='\d{8}', masked=False), ValueBackendPassword('password', label='Mot de passe')) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def get_account(self, id): return self.browser.iter_accounts() def iter_accounts(self): return self.browser.iter_accounts() def iter_coming(self, account): raise NotImplementedError() def iter_history(self, account): return self.browser.get_history() def iter_investment(self, account): return self.browser.iter_investment() weboob-1.2/modules/creditdunordpee/pages.py000066400000000000000000000143321303450110500211250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from cStringIO import StringIO import re from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard from weboob.browser.elements import ItemElement, TableElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Format, Regexp, Date, Env, TableCell, Field from weboob.browser.filters.html import CleanHTML from weboob.capabilities.bank import Account, Transaction, Investment class VirtKeyboard(MappedVirtKeyboard): symbols = {'0': '8adee734aaefb163fb008d26bb9b3a42', '1': 'b815d6ce999910d48619b5912b81ddf1', '2': '54255a70694787a4e1bd7dd473b50228', '3': 'ba06373d2bfba937d00bf52a31d475eb', '4': '3fa795ac70247922048c514115487b10', '5': '788963d15fa05832ee7640f7c2a21bc3', '6': 'c8bf62dfaed9feeb86934d8617182503', '7': 'f7543fdda3039bdd383531954dd4fc46', '8': '5c4210e2d8e39f7667d7a9e5534b18b7', '9': '94520ac801883fbfb700f43cd4172d41', } def __init__(self, page): self.img_id = page.doc.find("//input[@id='identifiantClavierVirtuel']").attrib['value'] img = page.doc.find("//img[@id='clavier_virtuel']") res = page.browser.open('/portal/rest/clavier_virtuel/%s' % self.img_id) MappedVirtKeyboard.__init__(self, StringIO(res.content), page.doc, img, (0, 0, 0), convert='RGB') self.check_symbols(self.symbols, page.browser.responses_dirname) def get_symbol_code(self, md5sum): code = MappedVirtKeyboard.get_symbol_code(self, md5sum) return ''.join(re.findall(r"'(\d+)'", code)[-2:]) def get_string_code(self, string): code = '' for c in string: code += self.get_symbol_code(self.symbols[c]) return code class LoginPage(HTMLPage): pass class HomePage(HTMLPage): def get_coded_passwd(self, password): vk = VirtKeyboard(self) return '%s|%s|#006#' % (vk.get_string_code(password), vk.img_id) def is_logged(self): return len(self.doc.find('//a[@class="btn-deconnexion"]')) class AvoirPage(LoggedPage, HTMLPage): @method class get_account(ItemElement): klass = Account obj_label = Format('PEE %s', CleanText('//div[@id="pbGp_df83b8bd_2dd787_2d4d10_2db608_2d69c44af91e91_j_id1:j_idt1:j_idt2:j_idt15_body"]')) obj_balance = CleanDecimal('//div[@id="pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt14:j_idt23:0:j_idt65:j_idt47_body"]', default=0, replace_dots=True) obj_currency = Regexp(CleanText('//div[@id="pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt14:j_idt23:0:j_idt65:j_idt47_body"]'), '.*(.)$', default=u'€') obj_type = Account.TYPE_PEE @method class iter_investment(TableElement): head_xpath = u'//table[@id="pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt413:j_idt447"]/thead/tr/th/@id' item_xpath = u'//table[@id="pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt413:j_idt447"]/tbody/tr[@id]' col_reference = u'pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt413:j_idt447:j_idt450' col_montant = u'pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt413:j_idt447:j_idt456' col_repartition = u'pbGp_3f1d2af7_2ddd41_2d45d2_2dbb9d_2d8f27b33a375f_j_id1:j_idt1:form:j_idt2:j_idt413:j_idt447:j_idt460' class item(ItemElement): klass = Investment obj_label = CleanText(Regexp(CleanHTML(TableCell('reference')), '(.*)\n\n')) obj_vdate = Date(Regexp(CleanHTML(TableCell('reference')), '(\d{2}/\d{2}/\d{4})')) obj_unitvalue = CleanDecimal(Regexp(CleanHTML(TableCell('reference')), '.*\n\n(.*)\n\n'), replace_dots=True) obj_description = CleanText(CleanHTML(TableCell('reference'))) obj_portfolio_share = CleanDecimal(CleanHTML(TableCell('repartition')), replace_dots=True) obj_valuation = CleanDecimal(CleanHTML(TableCell('montant')), replace_dots=True) def obj_quantity(self): return Decimal(Field('valuation')(self)/Field('unitvalue')(self)) class HistoryPage(LoggedPage, HTMLPage): @method class get_history(TableElement): head_xpath = u'//table[has-class("operation-bloc-content-tableau")]/thead/tr/th/div/div/div/div/div/div/a/text()' item_xpath = u'//table[has-class("operation-bloc-content-tableau")]/tbody/tr[has-class("rf-dt-r")]' col_date = u'Date de création' col_reference = u'Référence' col_montant = u'Montant net' col_type = u'Type d’opération' class item(ItemElement): klass = Transaction obj_date = Date(CleanText(TableCell('date')), Env('date_guesser')) obj_type = Transaction.TYPE_UNKNOWN obj_id = CleanText(TableCell('reference')) obj_label = CleanText(TableCell('type')) obj_amount = CleanDecimal(CleanHTML(TableCell('montant')), replace_dots=True) weboob-1.2/modules/creditdunordpee/test.py000066400000000000000000000015171303450110500210060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CreditdunordpeeTest(BackendTest): MODULE = 'creditdunordpee' weboob-1.2/modules/creditmutuel/000077500000000000000000000000001303450110500167775ustar00rootroot00000000000000weboob-1.2/modules/creditmutuel/__init__.py000066400000000000000000000014531303450110500211130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CreditMutuelModule __all__ = ['CreditMutuelModule'] weboob-1.2/modules/creditmutuel/browser.py000066400000000000000000000375511303450110500210470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . try: from urlparse import urlparse except ImportError: from urllib.parse import urlparse from datetime import datetime from dateutil.relativedelta import relativedelta from itertools import groupby from weboob.tools.compat import basestring from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.browsers import LoginBrowser, need_login from weboob.browser.profiles import Wget from weboob.browser.url import URL from weboob.browser.pages import FormNotFound from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account from .pages import LoginPage, LoginErrorPage, AccountsPage, UserSpacePage, \ OperationsPage, CardPage, ComingPage, NoOperationsPage, \ ChangePasswordPage, VerifCodePage, EmptyPage, PorPage, \ IbanPage, NewHomePage, AdvisorPage, RedirectPage, \ LIAccountsPage, CardsActivityPage, CardsListPage, \ CardsOpePage, NewAccountsPage, InternalTransferPage, \ ExternalTransferPage __all__ = ['CreditMutuelBrowser'] class CreditMutuelBrowser(LoginBrowser): PROFILE = Wget() TIMEOUT = 30 BASEURL = 'https://www.creditmutuel.fr' login = URL('/groupe/fr/index.html', '/(?P.*)fr/$', '/(?P.*)fr/banques/accueil.html', '/(?P.*)fr/banques/particuliers/index.html', LoginPage) login_error = URL('/(?P.*)fr/identification/default.cgi', LoginErrorPage) accounts = URL('/(?P.*)fr/banque/situation_financiere.cgi', '/(?P.*)fr/banque/situation_financiere.html', AccountsPage) user_space = URL('/(?P.*)fr/banque/espace_personnel.aspx', '/(?P.*)fr/banque/accueil.cgi', '/(?P.*)fr/banque/DELG_Gestion', '/(?P.*)fr/banque/paci_engine/static_content_manager.aspx', UserSpacePage) card = URL('/(?P.*)fr/banque/operations_carte.cgi.*', '/(?P.*)fr/banque/mouvements.html\?webid=.*cardmonth=\d+$', '/(?P.*)fr/banque/mouvements.html.*webid=.*cardmonth=\d+.*cardid=', CardPage) operations = URL('/(?P.*)fr/banque/mouvements.cgi.*', '/(?P.*)fr/banque/mouvements.html.*', '/(?P.*)fr/banque/nr/nr_devbooster.aspx.*', OperationsPage) coming = URL('/(?P.*)fr/banque/mvts_instance.cgi.*', ComingPage) noop = URL('/(?P.*)fr/banque/CR/arrivee.asp.*', NoOperationsPage) info = URL('/(?P.*)fr/banque/BAD.*', EmptyPage) change_pass = URL('/(?P.*)fr/validation/change_password.cgi', '/fr/services/change_password.html', ChangePasswordPage) verify_pass = URL('/(?P.*)fr/validation/verif_code.cgi.*', VerifCodePage) new_home = URL('/(?P.*)fr/banque/pageaccueil.html', '/(?P.*)banque/welcome_pack.html', NewHomePage) empty = URL('/(?P.*)fr/banques/index.html', '/(?P.*)fr/banque/paci_beware_of_phishing.*', '/(?P.*)fr/validation/(?!change_password|verif_code).*', EmptyPage) por = URL('/(?P.*)fr/banque/POR_ValoToute.aspx', '/(?P.*)fr/banque/POR_SyntheseLst.aspx', PorPage) li = URL('/(?P.*)fr/assurances/profilass.aspx\?domaine=epargne', '/(?P.*)fr/assurances/(consultation/)?WI_ASSAVI', LIAccountsPage) iban = URL('/(?P.*)fr/banque/rib.cgi', IbanPage) new_accounts = URL('/(?P.*)fr/banque/comptes-et-contrats.html', NewAccountsPage) new_operations = URL('/(?P.*)fr/banque/mouvements.cgi', '/fr/banque/nr/nr_devbooster.aspx.*', '/(?P.*)fr/banque/RE/aiguille.asp', '/fr/banque/mouvements.html', OperationsPage) new_por = URL('/(?P.*)fr/banque/POR_ValoToute.aspx', '/(?P.*)fr/banque/POR_SyntheseLst.aspx', PorPage) new_iban = URL('/(?P.*)fr/banque/rib.cgi', IbanPage) advisor = URL('/(?P.*)fr/banques/contact/trouver-une-agence/(?P.*)', '/(?P.*)fr/infoclient/', AdvisorPage) redirect = URL('/(?P.*)fr/banque/paci_engine/static_content_manager.aspx', RedirectPage) cards_activity = URL('/(?P.*)fr/banque/pro/ENC_liste_tiers.aspx', CardsActivityPage) cards_list = URL('/(?P.*)fr/banque/pro/ENC_liste_ctr.*', '/(?P.*)fr/banque/pro/ENC_detail_ctr', CardsListPage) cards_ope = URL('/(?P.*)fr/banque/pro/ENC_liste_oper', CardsOpePage) internal_transfer = URL('/(?P.*)fr/banque/virements/vplw_vi.html', InternalTransferPage) external_transfer = URL('/(?P.*)fr/banque/virements/vplw_vee.html', ExternalTransferPage) currentSubBank = None is_new_website = False __states__ = ['currentSubBank'] accounts_list = None def do_login(self): # Clear cookies. self.do_logout() self.login.go() if not self.page.logged: self.page.login(self.username, self.password) if not self.page.logged or self.login_error.is_here(): raise BrowserIncorrectPassword() self.getCurrentSubBank() @need_login def get_accounts_list(self): if not self.accounts_list: if self.currentSubBank is None: self.getCurrentSubBank() self.accounts_list = [] # Handle cards on tiers page self.cards_activity.go(subbank=self.currentSubBank) companies = self.page.companies_link() if self.cards_activity.is_here() else \ [self.page] if self.is_new_website else [] for company in companies: page = self.open(company).page if isinstance(company, basestring) else company self.accounts_list.extend([card for card in page.iter_cards()]) if not self.is_new_website: for a in self.accounts.stay_or_go(subbank=self.currentSubBank).iter_accounts(): self.accounts_list.append(a) self.iban.go(subbank=self.currentSubBank).fill_iban(self.accounts_list) self.por.go(subbank=self.currentSubBank).add_por_accounts(self.accounts_list) else: for a in self.new_accounts.stay_or_go(subbank=self.currentSubBank).iter_accounts(): self.accounts_list.append(a) self.new_iban.go(subbank=self.currentSubBank).fill_iban(self.accounts_list) self.new_por.go(subbank=self.currentSubBank).add_por_accounts(self.accounts_list) for acc in self.li.go(subbank=self.currentSubBank).iter_li_accounts(): self.accounts_list.append(acc) return self.accounts_list def get_account(self, id): assert isinstance(id, basestring) for a in self.get_accounts_list(): if a.id == id: return a def getCurrentSubBank(self): # the account list and history urls depend on the sub bank of the user paths = urlparse(self.url).path.lstrip('/').split('/') self.currentSubBank = paths[0] + "/" if paths[0] != "fr" else "" if paths[0] in ["fr", "mabanque"]: self.is_new_website = True def list_operations(self, page): if isinstance(page, basestring): if page.startswith('/') or page.startswith('https') or page.startswith('?'): self.location(page) else: self.location('%s/%sfr/banque/%s' % (self.BASEURL, self.currentSubBank, page)) else: self.page = page # getting about 6 months history on new website if self.is_new_website and self.page: try: for x in range(0, 2): form = self.page.get_form(id="I1:fm", submit='//input[@name="_FID_DoActivateSearch"]') if x == 1: form.update({ [k for k in form.keys() if "DateStart" in k][0]: (datetime.now() - relativedelta(months=7)).strftime('%d/%m/%Y'), [k for k in form.keys() if "DateEnd" in k][0]: datetime.now().strftime('%d/%m/%Y') }) [form.pop(k, None) for k in form.keys() if "_FID_Do" in k and "DoSearch" not in k] form.submit() except (IndexError, FormNotFound): pass while self.page: try: form = self.page.get_form('//*[@id="I1:fm"]', submit='//input[@name="_FID_DoLoadMoreTransactions"]') [form.pop(k, None) for k in form.keys() if "_FID_Do" in k and "LoadMore" not in k] form.submit() except (IndexError, FormNotFound): break if self.li.is_here(): return self.page.iter_history() if not self.operations.is_here(): return iter([]) return self.pagination(lambda: self.page.get_history()) def get_monthly_transactions(self, trs): groups = [list(g) for k, g in groupby(sorted(trs, key=lambda tr: tr.date), lambda tr: tr.date)] trs = [] for group in groups: tr = FrenchTransaction() tr.raw = tr.label = u"RELEVE CARTE %s" % group[0].date tr.amount = -sum([t.amount for t in group]) tr.date = tr.rdate = tr.vdate = group[0].date tr.type = FrenchTransaction.TYPE_CARD_SUMMARY tr._is_coming = False trs.append(tr) return trs def get_history(self, account): transactions = [] if not account._link_id: return iter([]) # need to refresh the months select if account._link_id.startswith('ENC_liste_oper'): self.location(account._pre_link) if not hasattr(account, '_card_pages'): for tr in self.list_operations(account._link_id): transactions.append(tr) coming_link = self.page.get_coming_link() if self.operations.is_here() else None if coming_link is not None: for tr in self.list_operations(coming_link): transactions.append(tr) differed_date = None cards = [page.select_card(account.id) for page in account._card_pages] if hasattr(account, '_card_pages') else \ account._card_links if hasattr(account, '_card_links') else [] for card in cards: card_trs = [] for tr in self.list_operations(card): if hasattr(tr, '_differed_date') and (not differed_date or tr._differed_date < differed_date): differed_date = tr._differed_date if tr.date >= datetime.now(): tr._is_coming = True elif hasattr(account, '_card_pages'): card_trs.append(tr) transactions.append(tr) if card_trs: transactions.extend(self.get_monthly_transactions(card_trs)) if differed_date is not None: # set deleted for card_summary for tr in transactions: tr.deleted = tr.type == FrenchTransaction.TYPE_CARD_SUMMARY and differed_date.month <= tr.date.month transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions def get_investment(self, account): if account._is_inv: if account.type == Account.TYPE_MARKET: if not self.is_new_website: self.por.go(subbank=self.currentSubBank) else: self.new_por.go(subbank=self.currentSubBank) self.page.send_form(account) elif account.type == Account.TYPE_LIFE_INSURANCE: if not account._link_inv: return iter([]) self.location(account._link_inv) return self.page.iter_investment() return iter([]) def iter_recipients(self, origin_account): # access the transfer page self.internal_transfer.go(subbank=self.currentSubBank) if self.page.can_transfer(origin_account.id): for recipient in self.page.iter_recipients(origin_account=origin_account): yield recipient self.external_transfer.go(subbank=self.currentSubBank) if self.page.can_transfer(origin_account.id): origin_account._external_recipients = set() if self.page.has_transfer_categories(): for category in self.page.iter_categories(): self.page.go_on_category(category['index']) self.page.IS_PRO_PAGE = True for recipient in self.page.iter_recipients(origin_account=origin_account, category=category['name']): yield recipient else: for recipient in self.page.iter_recipients(origin_account=origin_account): yield recipient def init_transfer(self, account, to, amount, reason=None): if to.category != u'Interne': self.external_transfer.go(subbank=self.currentSubBank) else: self.internal_transfer.go(subbank=self.currentSubBank) if self.external_transfer.is_here() and self.page.has_transfer_categories(): for category in self.page.iter_categories(): if category['name'] == to.category: self.page.go_on_category(category['index']) break self.page.IS_PRO_PAGE = True self.page.RECIPIENT_STRING = 'data_input_indiceBen' self.page.prepare_transfer(account, to, amount, reason) return self.page.handle_response(account, to, amount, reason) def execute_transfer(self, transfer, **params): form = self.page.get_form(id='P:F', submit='//input[@type="submit" and contains(@value, "Confirmer")]') # For the moment, don't ask the user if he confirms the duplicate. form['Bool:data_input_confirmationDoublon'] = 'true' form.submit() return self.page.create_transfer(transfer) @need_login def get_advisor(self): advisor = None if not self.is_new_website: self.accounts.stay_or_go(subbank=self.currentSubBank) if self.page.get_advisor_link(): advisor = self.page.get_advisor() self.location(self.page.get_advisor_link()).page.update_advisor(advisor) else: advisor = self.new_accounts.stay_or_go(subbank=self.currentSubBank).get_advisor() if self.page.has_agency(): self.advisor.go(subbank=self.currentSubBank, page="contact.html").update_advisor(advisor) return iter([advisor]) if advisor else iter([]) weboob-1.2/modules/creditmutuel/favicon.png000066400000000000000000000034331303450110500211350ustar00rootroot00000000000000PNG  IHDR@@iqIDATx՛]hU(i1Ec`KZJ 5Qd RԊ1.*J)`%$*> #OU[(y1XWmAmIMjMdw>L&9prw9{s9!&OgQ P AR+MW)&@^@1ZakB7W򭂁ww 2.K+(QO(/CBDK`B,d'"`r5ÓaYAXR Pw5Py G`}qi$0a^mzǀ*_e'(g-/0^[_"$ZJb S6Ν{}j,:s),55YTMMIS = X6&ҕ+cg.Iґ#ҍ7J0/x!V`vWGF κJYbmɅ,r Kx\*S>ΟWUq@鷆0(ɓR6[,X>/  uw5A{TOOuR)kᅨD}}V ^l+鍎|,a|ҦYA%,o"!e2łϻ LΙ3rA*j*eRmtڜ?+ǥ_o/BBRS ZXn1s08hn6Ion?L)_ (XfTy緵'.]*K`CAD#ɍAJGqWݺKEA\u/}S8SIg)wmN ? Rqܑ{E(4:}QʍΟ7A]xAN9 &&G萎=33VfYFئOpI Сv9$+7˚'?V/voǎYٝ3gygó8bB.:k)-'sZ&~b%1}AJ &mpwC"l--vmua~رZ[˿,?ryll._yV[!g8< ̸yOCIf:9  pbb۠˚e2ݖx`a)>~)9#mV/BCC0=]>~xt@YKRݩu D3&:=D_j!?jr)'Y(9''Ү]K@A_/v^}uIrke7B>-o޸%W2I]]P~X2zzN=yC_ Uz]XgsY4T8]3@Aر39~=4/+{4X Ǡm]d,Jo-, 8Ppj0hrIRkV. 5߾uD,LmA.d~=| |9,ث,/M쯔dLwgϛ7Þ=av|9?O=liIy.ǮnA[RyYy=a`=Q]]ccq5~\^Qzp?/hZ7$[Z/5 %'IENDB`weboob-1.2/modules/creditmutuel/module.py000066400000000000000000000102641303450110500206410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from weboob.capabilities.base import find_object from weboob.capabilities.bank import CapBankTransfer, AccountNotFound, RecipientNotFound, \ Account, TransferError from weboob.capabilities.contact import CapContact from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import CreditMutuelBrowser __all__ = ['CreditMutuelModule'] class CreditMutuelModule(Module, CapBankTransfer, CapContact): NAME = 'creditmutuel' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'Crédit Mutuel' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', regexp='^\d{1,13}\w$', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = CreditMutuelBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_coming(self, account): for tr in self.browser.get_history(account): if tr._is_coming: yield tr def iter_history(self, account): for tr in self.browser.get_history(account): if not tr._is_coming: yield tr def iter_investment(self, account): return self.browser.get_investment(account) def iter_transfer_recipients(self, origin_account): if not self.browser.is_new_website: raise NotImplementedError() if not isinstance(origin_account, Account): origin_account = find_object(self.iter_accounts(), id=origin_account, error=AccountNotFound) return self.browser.iter_recipients(origin_account) def init_transfer(self, transfer, **params): # There is a check on the website, transfer can't be done with too long reason. if transfer.label: transfer.label = transfer.label[:27] self.logger.info('Going to do a new transfer') if transfer.account_iban: account = find_object(self.iter_accounts(), iban=transfer.account_iban, error=AccountNotFound) else: account = find_object(self.iter_accounts(), id=transfer.account_id, error=AccountNotFound) if transfer.recipient_iban: recipient = find_object(self.iter_transfer_recipients(account.id), iban=transfer.recipient_iban, error=RecipientNotFound) else: recipient = find_object(self.iter_transfer_recipients(account.id), id=transfer.recipient_id, error=RecipientNotFound) try: assert account.id.isdigit() # quantize to show 2 decimals. amount = Decimal(transfer.amount).quantize(Decimal(10) ** -2) except (AssertionError, ValueError): raise TransferError('something went wrong') return self.browser.init_transfer(account, recipient, amount, transfer.label) def execute_transfer(self, transfer, **params): return self.browser.execute_transfer(transfer) def iter_contacts(self): return self.browser.get_advisor() weboob-1.2/modules/creditmutuel/pages.py000066400000000000000000001306061303450110500204560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re try: from urlparse import urlparse, parse_qs except ImportError: from urllib.parse import urlparse, parse_qs from decimal import Decimal, InvalidOperation from dateutil.relativedelta import relativedelta from datetime import date, datetime from random import randint from weboob.browser.pages import HTMLPage, FormNotFound, LoggedPage, pagination from weboob.browser.elements import ListElement, ItemElement, SkipItem, method, TableElement from weboob.browser.filters.standard import Filter, Env, CleanText, CleanDecimal, Field, TableCell, Regexp, Async, AsyncLoad, Date, ColumnNotFound, Format from weboob.browser.filters.html import Link, Attr from weboob.exceptions import BrowserIncorrectPassword, ParseError, NoAccountsException, ActionNeeded from weboob.capabilities import NotAvailable from weboob.capabilities.base import empty from weboob.capabilities.bank import Account, Investment, Recipient, TransferError, Transfer from weboob.capabilities.contact import Advisor from weboob.tools.capabilities.bank.iban import is_iban_valid from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.date import parse_french_date class RedirectPage(LoggedPage, HTMLPage): def on_load(self): link = self.doc.xpath('//a[@id="P:F_1.R2:link"]') if link: self.browser.location(link[0].attrib['href']) class NewHomePage(LoggedPage, HTMLPage): def on_load(self): self.browser.is_new_website = True class LoginPage(HTMLPage): REFRESH_MAX = 10.0 def login(self, login, passwd): form = self.get_form(xpath='//form[contains(@name, "ident")]') form['_cm_user'] = login form['_cm_pwd'] = passwd form.submit() @property def logged(self): return self.doc.xpath('//div[@id="e_identification_ok"]') class LoginErrorPage(HTMLPage): def on_load(self): raise BrowserIncorrectPassword(CleanText('//div[has-class("blocmsg")]')(self.doc)) class EmptyPage(LoggedPage, HTMLPage): REFRESH_MAX = 10.0 class UserSpacePage(LoggedPage, HTMLPage): def on_load(self): if self.doc.xpath('//form[@id="GoValider"]'): raise ActionNeeded(u"Le site du contrat Banque à Distance a besoin d'informations supplémentaires") class ChangePasswordPage(LoggedPage, HTMLPage): def on_load(self): raise BrowserIncorrectPassword('Please change your password') class VerifCodePage(LoggedPage, HTMLPage): def on_load(self): raise BrowserIncorrectPassword('Unable to login: website asks a code from a card') class AccountsPage(LoggedPage, HTMLPage): def on_load(self): no_account_message = CleanText(u'//td[contains(text(), "Votre contrat de banque à distance ne vous donne accès à aucun compte.")]')(self.doc) if no_account_message: raise NoAccountsException(no_account_message) TYPES = {u'C/C': Account.TYPE_CHECKING, u'Livret': Account.TYPE_SAVINGS, u'Nouveau Prêt': Account.TYPE_LOAN, u'Pret': Account.TYPE_LOAN, u'Cic Immo': Account.TYPE_LOAN, u'Passeport Credit': Account.TYPE_LOAN, u'Credit En Reserve': Account.TYPE_LOAN, u'Compte Courant': Account.TYPE_CHECKING, u'Cpte Courant': Account.TYPE_CHECKING, u'Compte Cheque': Account.TYPE_CHECKING, u'Start': Account.TYPE_CHECKING, u'Contrat Personnel': Account.TYPE_CHECKING, u'Compte Epargne': Account.TYPE_SAVINGS, u'Plan D\'Epargne': Account.TYPE_SAVINGS, u'P.E.A': Account.TYPE_SAVINGS, u'Tonic Croissance': Account.TYPE_SAVINGS, u'Ldd': Account.TYPE_SAVINGS, u'Etalis': Account.TYPE_SAVINGS, } @method class iter_accounts(ListElement): item_xpath = '//tr' flush_at_end = True class item(ItemElement): klass = Account def condition(self): if len(self.el.xpath('./td')) < 2: return False first_td = self.el.xpath('./td')[0] return (("i" in first_td.attrib.get('class', '') or "p" in first_td.attrib.get('class', '')) and (first_td.find('a') is not None or (first_td.find('.//span') is not None and "cartes" in first_td.findtext('.//span') and first_td.find('./div/a') is not None))) class Label(Filter): def filter(self, text): return text.lstrip(' 0123456789').title() class Type(Filter): def filter(self, label): for pattern, actype in AccountsPage.TYPES.iteritems(): if label.startswith(pattern): return actype return Account.TYPE_UNKNOWN obj_id = Env('id') obj_label = Label(CleanText('./td[1]/a/text() | ./td[1]/a/span[@class and not(contains(@class, "doux"))] | ./td[1]/div/a[has-class("cb")]')) obj_coming = Env('coming') obj_balance = Env('balance') obj_currency = FrenchTransaction.Currency('./td[2] | ./td[3]') obj__link_id = Link('./td[1]//a') obj__card_links = [] obj_type = Type(Field('label')) obj__is_inv = False obj__is_webid = Env('_is_webid') def parse(self, el): link = el.xpath('./td[1]//a')[0].get('href', '') if 'POR_SyntheseLst' in link: raise SkipItem() url = urlparse(link) p = parse_qs(url.query) if 'rib' not in p and 'webid' not in p: raise SkipItem() for td in el.xpath('./td[2] | ./td[3]'): try: balance = CleanDecimal('.', replace_dots=True)(td) except InvalidOperation: continue else: break else: if 'lien_inter_sites' in link: raise SkipItem() else: raise ParseError('Unable to find balance for account %s' % CleanText('./td[1]/a')(el)) self.env['_is_webid'] = False if "cartes" in CleanText('./td[1]')(el): # handle cb differed card if "cartes" in CleanText('./preceding-sibling::tr[1]/td[1]', replace=[(' ', '')])(el): # In case it's the second month of card history present, we need to ignore the first # one to get the attach accoount id_xpath = './preceding-sibling::tr[2]/td[1]/a/node()[contains(@class, "doux")]' else: # first month of history, the previous tr is the attached account id_xpath = './preceding-sibling::tr[1]/td[1]/a/node()[contains(@class, "doux")]' else: # classical account id_xpath = './td[1]/a/node()[contains(@class, "doux")]' id = CleanText(id_xpath, replace=[(' ', '')])(el) if not id: if 'rib' in p: id = p['rib'][0] else: id = p['webid'][0] self.env['_is_webid'] = True page = self.page.browser.open(link).page # Handle cards if id in self.parent.objects: # be sure that we don't have that case anymore assert not page.is_fleet() account = self.parent.objects[id] if not account.coming: account.coming = Decimal('0.0') date = parse_french_date(Regexp(Field('label'), 'Fin (.+) (\d{4})', '01 \\1 \\2')(self)) + relativedelta(day=31) if date > datetime.now() - relativedelta(day=1): account.coming += balance # on old website we want card's history in account's history if not page.browser.is_new_website: account._card_links.append(link) else: card_xpath = u'//*[contains(@id, "MonthSelector")]' for el in page.doc.xpath(u'%s/li/div/div[1] | %s//span[contains(text(), "Carte")]' % (card_xpath, card_xpath)): card_id = Regexp(CleanText('.', replace=[(' ', '')]), '(?=\d)([\dx]+)')(el) if any(a.id == card_id for a in page.browser.accounts_list): continue card = Account() card.id = card_id card.label = "%s %s %s" % (Regexp(CleanText('.'), 'Carte\s(\w+)')(el), card_id, \ (Regexp(CleanText('.'), '\d{4}\s([A-Za-z\s]+)', default=None)(el) \ or CleanText('./following-sibling::div[1]')(el)).strip()) # balance_xpath = './preceding-sibling::span[contains(@class, "montant")]' card.balance = CleanDecimal(balance_xpath, replace_dots=True, default=NotAvailable)(el) card.currency = card.get_currency(CleanText(balance_xpath)(el)) card.type = Account.TYPE_CARD card._link_id = link nextmonth = Link('./following-sibling::tr[contains(@class, "encours")][1]/td[1]//a', default=None)(self) card._card_pages = [page] if not nextmonth else [page, page.browser.open(nextmonth).page] card._is_inv = False card._is_webid = False self.page.browser.accounts_list.append(card) raise SkipItem() self.env['id'] = id # Handle real balances coming = page.find_amount(u"Opérations à venir") if page else None accounting = page.find_amount(u"Solde comptable") if page else None if accounting is not None and accounting + (coming or Decimal('0')) != balance: self.page.logger.warning('%s + %s != %s' % (accounting, coming, balance)) if accounting is not None: balance = accounting self.env['balance'] = balance self.env['coming'] = coming or NotAvailable def get_advisor_link(self): return Link('//div[@id="e_conseiller"]/a', default=None)(self.doc) @method class get_advisor(ItemElement): klass = Advisor obj_name = CleanText('//div[@id="e_conseiller"]/a') class NewAccountsPage(NewHomePage, AccountsPage): def has_agency(self): return CleanText('//script[contains(text(), "lien_caisse")]', default=None)(self.doc) @method class get_advisor(ItemElement): klass = Advisor obj_name = Regexp(CleanText('//script[contains(text(), "Espace Conseiller")]'), 'consname.+?([\w\s]+)') class AdvisorPage(LoggedPage, HTMLPage): @method class update_advisor(ItemElement): obj_email = CleanText('//table//*[@itemprop="email"]') obj_phone = CleanText('//table//*[@itemprop="telephone"]', replace=[(' ', '')]) obj_mobile = NotAvailable obj_fax = CleanText('//table//*[@itemprop="faxNumber"]', replace=[(' ', '')]) obj_agency = CleanText('//div/*[@itemprop="name"]') obj_address = Format('%s %s %s', CleanText('//table//*[@itemprop="streetAddress"]'), CleanText('//table//*[@itemprop="postalCode"]'), CleanText('//table//*[@itemprop="addressLocality"]')) class CardsActivityPage(LoggedPage, HTMLPage): def companies_link(self): companies_link = [] for tr in self.doc.xpath('//table[@summary="Liste des titulaires de contrats cartes"]//tr'): companies_link.append(Link(tr.xpath('.//a'))(self)) return companies_link class Pagination(object): def next_page(self): try: form = self.page.get_form('//form[@id="paginationForm"]') except FormNotFound: return self.next_month() text = CleanText.clean(form.el) m = re.search(u'(\d+) / (\d+)', text or '', flags=re.MULTILINE) if not m: return self.next_month() cur = int(m.group(1)) last = int(m.group(2)) if cur == last: return self.next_month() form['page'] = str(cur + 1) return form.request def next_month(self): try: form = self.page.get_form('//form[@id="frmStarcLstOpe"]') except FormNotFound: return try: form['moi'] = self.page.doc.xpath('//select[@id="moi"]/option[@selected]/following-sibling::option')[0].attrib['value'] except IndexError: return return form.request class CardsListPage(LoggedPage, HTMLPage): @pagination @method class iter_cards(TableElement): item_xpath = '//table[@class="liste"]/tbody/tr' head_xpath = '//table[@class="liste"]/thead//tr/th' col_owner = 'Porteur' col_card = 'Carte' def next_page(self): try: form = self.page.get_form('//form[contains(@id, "frmStarcLstCtrPag")]') form['imgCtrPagSui.x'] = randint(1, 29) form['imgCtrPagSui.y'] = randint(1, 17) m = re.search(u'(\d+)/(\d+)', CleanText('.')(form.el)) if m and int(m.group(1)) < int(m.group(2)): return form.request except FormNotFound: return class item(ItemElement): klass = Account load_details = Field('_link_id') & AsyncLoad obj_id = Env('id', default="") obj_number = Field('_link_id') & Regexp(pattern='ctr=(\d+)') obj_label = Format('%s %s %s', CleanText(TableCell('card')), Field('id'), CleanText(TableCell('owner'))) obj_balance = CleanDecimal('./td[small][1]', replace_dots=True, default=NotAvailable) obj_currency = FrenchTransaction.Currency(CleanText('./td[small][1]')) obj_type = Account.TYPE_CARD obj__card_pages = Env('page') obj__is_inv = False obj__is_webid = False def obj__pre_link(self): return self.page.url def obj__link_id(self): return Link(TableCell('card')(self)[0].xpath('./a'))(self) def parse(self, el): page = Async('details').loaded_page(self) self.env['page'] = [page] if len(page.doc.xpath(u'//caption[contains(text(), "débits immédiats")]')): raise SkipItem() # Handle multi cards options = page.doc.xpath('//select[@id="iso"]/option') for option in options: card = Account() for attr in self._attrs: self.handle_attr(attr, getattr(self, 'obj_%s' % attr)) setattr(card, attr, getattr(self.obj, attr)) card.id = CleanText('.', replace=[(' ', '')])(option) card.label = card.label.replace(' ', ' %s ' % card.id) card.balance = NotAvailable self.page.browser.accounts_list.append(card) # Skip multi and expired cards if len(options) or len(page.doc.xpath('//span[@id="ERREUR"]')): raise SkipItem() # 1 card : we have to check on another page to get id page = page.browser.open(Link('//form//a[text()="Contrat"]')(page.doc)).page xpath = '//table[@class="liste"]/tbody/tr' active_card = CleanText('%s[td[text()="Active"]][1]/td[2]' % xpath, replace=[(' ', '')], default=None)(page.doc) if not active_card and len(page.doc.xpath(xpath)) != 1: raise SkipItem() self.env['id'] = active_card or CleanText('%s[1]/td[2]' % xpath, replace=[(' ', '')])(page.doc) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(PRLV|Plt) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?P.*) CARTE \d+ PAIEMENT CB\s+(?P
\d{2})(?P\d{2}) ?(.*)$'), FrenchTransaction.TYPE_CARD), (re.compile('^PAIEMENT PSC\s+(?P
\d{2})(?P\d{2}) (?P.*) CARTE \d+ ?(.*)$'), FrenchTransaction.TYPE_CARD), (re.compile('^(?PRELEVE CARTE.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile('^RETRAIT DAB (?P
\d{2})(?P\d{2}) (?P.*) CARTE [\*\d]+'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^CHEQUE( (?P.*))?$'), FrenchTransaction.TYPE_CHECK), (re.compile('^(F )?COTIS\.? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(REMISE|REM CHQ) (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] _is_coming = False class OperationsPage(LoggedPage, HTMLPage): @method class get_history(Pagination, Transaction.TransactionsElement): head_xpath = '//table[@class="liste"]//thead//tr/th' item_xpath = '//table[@class="liste"]//tbody/tr' class item(Transaction.TransactionElement): condition = lambda self: len(self.el.xpath('./td')) >= 3 and len(self.el.xpath('./td[@class="i g" or @class="p g" or contains(@class, "_c1")]')) > 0 class OwnRaw(Filter): def __call__(self, item): el = TableCell('raw')(item)[0] # Remove hidden parts of labels: # hideifscript: Date de valeur XX/XX/XXXX # fd: Avis d'opéré # survey to add other regx parts = [re.sub(u'Détail|Date de valeur\s+:\s+\d{2}/\d{2}(/\d{4})?', '',txt.strip()) for txt in el.itertext() if len(txt.strip()) > 0] # To simplify categorization of CB, reverse order of parts to separate # location and institution. if parts[0].startswith('PAIEMENT CB'): parts.reverse() return u' '.join(parts) obj_raw = Transaction.Raw(OwnRaw()) def find_amount(self, title): try: td = self.doc.xpath(u'//th[contains(text(), "%s")]/../td' % title)[0] except IndexError: return None else: return Decimal(FrenchTransaction.clean_amount(td.text)) def get_coming_link(self): try: a = self.doc.xpath(u'//a[contains(text(), "Opérations à venir")]')[0] except IndexError: return None else: return a.attrib['href'] class CardsOpePage(OperationsPage): def select_card(self, card_number): if CleanText('//select[@id="iso"]', default=None)(self.doc): form = self.get_form('//p[@class="restriction"]') card_number = ' '.join([card_number[j*4:j*4+4] for j in xrange(len(card_number)/4+1)]).strip() form['iso'] = Attr('//option[text()="%s"]' % card_number, 'value')(self.doc) moi = Attr('//select[@id="moi"]/option[@selected]', 'value', default=None)(self.doc) if moi: form['moi'] = moi return self.browser.open(form.url, data=dict(form)).page return self @method class get_history(Pagination, Transaction.TransactionsElement): head_xpath = '//table[@class="liste"]//thead//tr/th' item_xpath = '//table[@class="liste"]/tr' col_city = u'Ville' col_original_amount = u'Montant d\'origine' col_amount = u'Montant' class item(Transaction.TransactionElement): condition = lambda self: len(self.el.xpath('./td')) >= 5 obj_raw = obj_label = Format('%s %s', TableCell('raw') & CleanText, TableCell('city') & CleanText) obj_original_amount = CleanDecimal(TableCell('original_amount'), default=NotAvailable, replace_dots=True) obj_original_currency = FrenchTransaction.Currency(TableCell('original_amount')) obj_type = Transaction.TYPE_DEFERRED_CARD obj_rdate = Transaction.Date(TableCell('date')) obj_date = obj_vdate = Env('date') obj__is_coming = Env('_is_coming') obj_amount = CleanDecimal(Env('amount'), replace_dots=True) obj_commission = CleanDecimal(Format('-%s', Env('commission')), replace_dots=True, default=NotAvailable) def parse(self, el): self.env['date'] = Date(Regexp(CleanText(u'//td[contains(text(), "Total prélevé")]'), ' (\d{2}/\d{2}/\d{4})', \ default=NotAvailable), default=NotAvailable)(self) if not self.env['date']: try: d = CleanText(u'//select[@id="moi"]/option[@selected]')(self) or \ re.search('pour le mois de (.*)', ''.join(w.strip() for w in self.page.doc.xpath('//div[@class="a_blocongfond"]/text()'))).group(1) except AttributeError: d = Regexp(CleanText('//p[@class="restriction"]'), 'pour le mois de ((?:\w+\s+){2})', flags=re.UNICODE)(self) self.env['date'] = (parse_french_date('%s %s' % ('1', d)) + relativedelta(day=31)).date() self.env['_is_coming'] = date.today() < self.env['date'] amount = CleanText(TableCell('amount'))(self).split('dont frais') self.env['amount'] = amount[0] self.env['commission'] = amount[1] if len(amount) > 1 else NotAvailable class ComingPage(OperationsPage, LoggedPage): @method class get_history(Pagination, Transaction.TransactionsElement): head_xpath = '//table[@class="liste"]//thead//tr/th/text()' item_xpath = '//table[@class="liste"]//tbody/tr' col_date = u"Date de l'annonce" class item(Transaction.TransactionElement): obj__is_coming = True class CardPage(OperationsPage, LoggedPage): def is_fleet(self): return len(self.doc.xpath('//table[@class="liste"]/tbody/tr/td/a')) >= 5 def select_card(self, card_number): for option in self.doc.xpath('//select[@name="Data_SelectedCardItemKey"]/option'): card_id = Regexp(CleanText('.', replace=[(' ', '')]), '(?=\d)([\dx]+)')(option) if card_id != card_number: continue if Attr('.', 'selected', default=None)(option): break form = self.get_form('//form[@id="I1:fm"]', submit='//input[@type="submit"]') [form.pop(k, None) for k in form.keys() if k.startswith('_FID_Do')] form['_FID_DoChangeCardDetails'] = "" form['Data_SelectedCardItemKey'] = Attr('.', 'value')(option) return self.browser.open(form.url, data=dict(form)).page return self @method class get_history(Pagination, ListElement): class list_cards(ListElement): item_xpath = '//table[@class="liste"]/tbody/tr/td/a' class item(ItemElement): def __iter__(self): card_link = self.el.get('href') page = self.page.browser.location(card_link).page for op in page.get_history(): yield op class list_history(Transaction.TransactionsElement): head_xpath = '//table[@class="liste"]//thead/tr/th' item_xpath = '//table[@class="liste"]/tbody/tr' col_commerce = u'Commerce' col_ville = u'Ville' def parse(self, el): label = CleanText(u'//*[contains(text(), "Achats")]')(el) if not label: return label = re.findall('(\d+ [^ ]+ \d+)', label)[-1] # use the trick of relativedelta to get the last day of month. self.env['debit_date'] = (parse_french_date(label) + relativedelta(day=31)).date() class item(Transaction.TransactionElement): condition = lambda self: len(self.el.xpath('./td')) >= 4 obj_raw = Transaction.Raw(Env('raw')) obj_type = Env('type') obj_date = Env('debit_date') obj_rdate = Transaction.Date(TableCell('date')) obj_amount = Env('amount') obj_original_amount = Env('original_amount') obj_original_currency = Env('original_currency') obj__differed_date = Env('differed_date') def parse(self, el): try: self.env['raw'] = "%s %s" % (CleanText().filter(TableCell('commerce')(self)[0].text), CleanText().filter(TableCell('ville')(self)[0].text)) except (ColumnNotFound, AttributeError): self.env['raw'] = "%s" % (CleanText().filter(TableCell('commerce')(self)[0].text)) self.env['type'] = Transaction.TYPE_DEFERRED_CARD \ if CleanText(u'//a[contains(text(), "Prélevé fin")]', default=None) else Transaction.TYPE_CARD self.env['differed_date'] = parse_french_date(Regexp(CleanText(u'//*[contains(text(), "Achats")]'), 'au[\s]+(.*)')(self)).date() amount = TableCell('credit')(self)[0] if self.page.browser.is_new_website: if not len(amount.xpath('./div')): amount = TableCell('debit')(self)[0] original_amount = amount.xpath('./div')[1].text if len(amount.xpath('./div')) > 1 else None amount = amount.xpath('./div')[0] else: try: original_amount = amount.xpath('./span')[0].text except IndexError: original_amount = None self.env['amount'] = CleanDecimal(replace_dots=True).filter(amount.text) self.env['original_amount'] = CleanDecimal(replace_dots=True).filter(original_amount) \ if original_amount is not None else NotAvailable self.env['original_currency'] = Account.get_currency(original_amount[1:-1]) \ if original_amount is not None else NotAvailable class NoOperationsPage(OperationsPage, LoggedPage): def get_history(self): return iter([]) class LIAccountsPage(LoggedPage, HTMLPage): @method class iter_li_accounts(ListElement): item_xpath = '//table[@class]/tbody/tr[count(td)>4]' class item(ItemElement): klass = Account load_details = Attr('.//a', 'href', default=NotAvailable) & AsyncLoad obj__link_id = Async('details', Link('//li/a[contains(text(), "Mouvements")]')) obj__link_inv = Link('./td[1]/a', default=NotAvailable) obj_id = CleanText('./td[2]', replace=[(' ', '')]) obj_label = CleanText('./td[1]') obj_balance = CleanDecimal('./td[3]', replace_dots=True, default=NotAvailable) obj_currency = FrenchTransaction.Currency('./td[3]') obj__card_links = [] obj_type = Account.TYPE_LIFE_INSURANCE obj__is_inv = True @method class iter_history(ListElement): item_xpath = '//table[@class="liste"]/tbody/tr' class item(ItemElement): klass = FrenchTransaction obj_date = obj_rdate = Transaction.Date(CleanText('./td[1]')) obj_raw = CleanText('./td[2]') obj_amount = CleanDecimal('./td[4]', replace_dots=True, default=Decimal('0')) obj_original_currency = FrenchTransaction.Currency('./td[4]') obj_type = Transaction.TYPE_BANK obj__is_coming = False def obj_commission(self): gross_amount = CleanDecimal('./td[3]', replace_dots=True, default=NotAvailable)(self) if gross_amount: return gross_amount - Field('amount')(self) return NotAvailable @method class iter_investment(TableElement): item_xpath = '//table[@class="liste"]/tbody/tr[count(td)>7]' head_xpath = '//table[@class="liste"]/thead/tr/th' col_label = u'Support' col_unitprice = re.compile(r'^Prix d\'achat moyen') col_vdate = re.compile(r'Date de cotation') col_unitvalue = u'Valeur de la part' col_quantity = u'Nombre de parts' col_valuation = u'Valeur atteinte' class item(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_unitprice = CleanDecimal(TableCell('unitprice', default=NotAvailable), default=NotAvailable, replace_dots=True) obj_vdate = Date(CleanText(TableCell('vdate'), replace=[('-', '')]), default=NotAvailable) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), default=NotAvailable, replace_dots=True) obj_quantity = CleanDecimal(TableCell('quantity'), default=NotAvailable, replace_dots=True) obj_valuation = CleanDecimal(TableCell('valuation'), default=Decimal(0), replace_dots=True) def obj_code(self): link = Link(TableCell('label')(self)[0].xpath('./a'), default=NotAvailable)(self) if not link: return NotAvailable return Regexp(pattern='isin=([A-Z\d]+)&?', default=NotAvailable).filter(link) class PorPage(LoggedPage, HTMLPage): def find_amount(self, title): return None def add_por_accounts(self, accounts): for ele in self.doc.xpath('//select[contains(@name, "POR_Synthese")]/option'): for a in accounts: if a.id.startswith(ele.attrib['value']): a._is_inv = True a.type = Account.TYPE_MARKET self.fill(a) break else: acc = Account() acc.id = ele.attrib['value'] if acc.id == '9999': # fake account continue acc.label = unicode(re.sub("\d", '', ele.text).strip()) acc._link_id = None acc.type = Account.TYPE_MARKET acc._is_inv = True self.fill(acc) accounts.append(acc) def fill(self, acc): self.send_form(acc) ele = self.browser.page.doc.xpath('.//table[@class="fiche bourse"]')[0] balance = CleanDecimal(ele.xpath('.//td[contains(@id, "Valorisation")]'), default=Decimal(0), replace_dots=True)(ele) acc.balance = balance + acc.balance if acc.balance else balance acc.currency = FrenchTransaction.Currency('.')(ele) acc.valuation_diff = CleanDecimal(ele.xpath('.//td[contains(@id, "Variation")]'), default=Decimal(0), replace_dots=True)(ele) def send_form(self, account): form = self.get_form(name="frmMere") form['POR_SyntheseEntete1$esdselLstPor'] = re.sub('\D', '', account.id) form.submit() @method class iter_investment(ListElement): item_xpath = '//table[@id="bwebDynamicTable"]/tbody/tr[not(@id="LigneTableVide")]' class item(ItemElement): klass = Investment obj_label = CleanText('.//td[1]/a') obj_code = CleanText('.//td[1]/a/@title') & Regexp(pattern='^([^ ]+)') obj_quantity = CleanDecimal('.//td[2]', default=Decimal(0), replace_dots=True) obj_unitprice = CleanDecimal('.//td[3]', default=Decimal(0), replace_dots=True) obj_unitvalue = CleanDecimal('.//td[4]', default=Decimal(0), replace_dots=True) obj_valuation = CleanDecimal('.//td[5]', default=Decimal(0), replace_dots=True) obj_diff = CleanDecimal('.//td[6]', default=Decimal(0), replace_dots=True) class IbanPage(LoggedPage, HTMLPage): def fill_iban(self, accounts): # Old website for ele in self.doc.xpath('//table[@class="liste"]/tr[@class]/td[1]'): for a in accounts: if a._is_webid: if a.label in CleanText('.//div[1]')(ele).title(): a.iban = CleanText('.//div[5]/em', replace=[(' ', '')])(ele) elif self.browser.is_new_website: if a.id in CleanText('.//div[5]/em', replace=[(' ','')])(ele).title(): a.iban = CleanText('.//div[5]/em', replace=[(' ', '')])(ele) else: if a.id[:-3] in CleanText('.//div[5]/em', replace=[(' ','')])(ele).title(): a.iban = CleanText('.//div[5]/em', replace=[(' ', '')])(ele) # New website for ele in self.doc.xpath('//table[@class="liste"]//tr[not(@class)]/td[1]'): for a in accounts: if a.id.split('EUR')[0] in CleanText('.//em[2]', replace=[(' ', '')])(ele): a.iban = CleanText('.//em[2]', replace=[(' ', '')])(ele) class MyRecipient(ItemElement): klass = Recipient obj_currency = u'EUR' obj_label = CleanText('.//div[@role="presentation"]/em | .//div[not(@id) and not(@role)]') def obj_enabled_at(self): return datetime.now().replace(microsecond=0) def validate(self, el): return not el.iban or is_iban_valid(el.iban) class InternalTransferPage(LoggedPage, HTMLPage): RECIPIENT_STRING = 'data_input_indiceCompteACrediter' READY_FOR_TRANSFER_MSG = u'Confirmer un virement entre vos comptes' SUMMARY_RECIPIENT_TITLE = u'Compte à créditer' IS_PRO_PAGE = False def can_transfer_pro(self, origin_account): for li in self.doc.xpath('//ul[@id="idDetailsListCptDebiterVertical:ul"]//ul/li'): if CleanText(li.xpath('.//span[@class="_c1 doux _c1"]'), replace=[(' ', '')])(self) in origin_account: return True def can_transfer(self, origin_account): if self.doc.xpath('//ul[@id="idDetailsListCptDebiterVertical:ul"]') or self.doc.xpath('//ul[@id="idDetailListCptDebiter:ul"]'): self.IS_PRO_PAGE = True return self.can_transfer_pro(origin_account) for li in self.doc.xpath('//ul[@id="idDetailsListCptDebiterHorizontal:ul"]/li'): if CleanText(li.xpath('.//span[@class="_c1 doux _c1"]'), replace=[(' ', '')])(self) in origin_account: return True @method class iter_recipients(ListElement): def parse(self, el): if self.page.IS_PRO_PAGE: self.item_xpath = '//ul[@id="idDetailsListCptCrediterVertical:ul"]//ul/li' else: self.item_xpath = '//ul[@id="idDetailsListCptCrediterHorizontal:ul"]//li[@role="radio"]' class item(MyRecipient): condition = lambda self: Field('id')(self) not in self.env['origin_account'].id obj_bank_name = u'Crédit Mutuel' obj_label = CleanText('.//div[@role="presentation"]/em | .//div[not(@id) and not(@role)]') obj_id = CleanText('.//span[@class="_c1 doux _c1"]', replace=[(' ', '')]) obj_category = u'Interne' def obj_iban(self): l = [a for a in self.page.browser.get_accounts_list() if Field('id')(self) in a.id and empty(a.valuation_diff)] assert len(l) == 1 return l[0].iban def get_account_index(self, direction, account): for div in self.doc.getroot().cssselect(".dw_dli_contents"): inp = div.cssselect("input")[0] if inp.name != direction: continue acct = div.cssselect("span.doux")[0].text.replace(" ", "") if account.endswith(acct): return inp.attrib['value'] else: raise TransferError("account %s not found" % account) def get_from_account_index(self, account): return self.get_account_index('data_input_indiceCompteADebiter', account) def get_to_account_index(self, account): return self.get_account_index(self.RECIPIENT_STRING, account) def get_unicode_content(self): return self.content.decode(self.detect_encoding()) def prepare_transfer(self, account, to, amount, reason): form = self.get_form(id='P:F', submit='//input[@type="submit" and contains(@value, "Valider")]') form['data_input_indiceCompteADebiter'] = self.get_from_account_index(account.id) form[self.RECIPIENT_STRING] = self.get_to_account_index(to.id) form['[t:dbt%3adouble;]data_input_montant_value_0_'] = str(amount).replace('.', ',') form['[t:dbt%3astring;x(27)]data_input_libelleCompteDebite'] = reason form['[t:dbt%3astring;x(31)]data_input_motifCompteCredite'] = reason form['[t:dbt%3astring;x(31)]data_input_motifCompteCredite1'] = reason form.submit() def check_errors(self): # look for known errors content = self.get_unicode_content() messages = [u'Le montant du virement doit être positif, veuillez le modifier', u'Montant maximum autorisé au débit pour ce compte', u'Dépassement du montant journalier autorisé'] for message in messages: if message in content: raise TransferError(message) # look for the known "all right" message if not self.doc.xpath(u'//span[contains(text(), "%s")]' % self.READY_FOR_TRANSFER_MSG): raise TransferError('The expected message "%s" was not found.' % self.READY_FOR_TRANSFER_MSG) def check_data_consistency(self, account_id, recipient_id, amount, reason): assert account_id in CleanText(u'//div[div[p[contains(text(), "Compte à débiter")]]]', replace=[(' ', '')])(self.doc) assert recipient_id in CleanText(u'//div[div[p[contains(text(), "%s")]]]' % self.SUMMARY_RECIPIENT_TITLE, replace=[(' ', '')])(self.doc) exec_date = Date(Regexp(CleanText('//table[@summary]/tbody/tr[th[contains(text(), "Date")]]/td'), '(\d{2}/\d{2}/\d{4})'), dayfirst=True)(self.doc) assert exec_date == datetime.today().date() r_amount = CleanDecimal('//table[@summary]/tbody/tr[th[contains(text(), "Montant")]]/td', replace_dots=True)(self.doc) assert r_amount == Decimal(amount) currency = FrenchTransaction.Currency('//table[@summary]/tbody/tr[th[contains(text(), "Montant")]]/td')(self.doc) if reason is not None: assert reason.upper().strip()[:22] in CleanText(u'//table[@summary]/tbody/tr[th[contains(text(), "Intitulé pour le compte à débiter")]]/td')(self.doc) return exec_date, r_amount, currency def handle_response(self, account, recipient, amount, reason): self.check_errors() exec_date, r_amount, currency = self.check_data_consistency(account.id, recipient.id, amount, reason) parsed = urlparse(self.url) webid = parse_qs(parsed.query)['_saguid'][0] transfer = Transfer() transfer.currency = currency transfer.amount = r_amount transfer.account_iban = account.iban transfer.recipient_iban = recipient.iban transfer.account_id = account.id transfer.recipient_id = recipient.id transfer.exec_date = exec_date transfer.label = reason transfer.account_label = account.label transfer.recipient_label = recipient.label transfer.account_balance = account.balance transfer.id = webid return transfer def create_transfer(self, transfer): # look for the known "everything went well" message content = self.get_unicode_content() transfer_ok_message = u'Votre virement a été exécuté' if transfer_ok_message not in content: raise TransferError('The expected message "%s" was not found.' % transfer_ok_message) exec_date, r_amount, currency = self.check_data_consistency(transfer.account_id, transfer.recipient_id, transfer.amount, transfer.label) assert u'Exécuté' in CleanText(u'//table[@summary]/tbody/tr[th[contains(text(), "Etat")]]/td')(self.doc) assert transfer.amount == r_amount assert transfer.exec_date == exec_date assert transfer.currency == currency return transfer class ExternalTransferPage(InternalTransferPage): RECIPIENT_STRING = 'data_input_indiceBeneficiaire' READY_FOR_TRANSFER_MSG = u'Confirmer un virement vers un bénéficiaire enregistré' SUMMARY_RECIPIENT_TITLE = u'Bénéficiaire à créditer' def can_transfer_pro(self, origin_account): for li in self.doc.xpath('//ul[@id="idDetailListCptDebiter:ul"]//ul/li'): if CleanText(li.xpath('.//span[@class="_c1 doux _c1"]'), replace=[(' ', '')])(self) in origin_account: return True def has_transfer_categories(self): select_elem = self.doc.xpath('//select[@name="data_input_indiceMarqueurListe"]') if select_elem: assert len(select_elem) == 1 return True def iter_categories(self): for option in self.doc.xpath('//select[@name="data_input_indiceMarqueurListe"]/option'): # This is the default selector if option.attrib['value'] == '9999': continue yield {'name': CleanText('.')(option), 'index': option.attrib['value']} def go_on_category(self, category_index): form = self.get_form(id='P:F', submit='//input[@type="submit" and @value="Nom"]') form['data_input_indiceMarqueurListe'] = category_index form.submit() @method class iter_recipients(ListElement): def parse(self, el): if self.page.IS_PRO_PAGE: self.item_xpath = '//ul[@id="ben.idBen:ul"]/li' else: self.item_xpath = '//ul[@id="idDetailListCptCrediterHorizontal:ul"]/li' class item(MyRecipient): condition = lambda self: Field('id')(self) not in self.env['origin_account']._external_recipients obj_bank_name = CleanText('(.//span[@class="_c1 doux _c1"])[2]', default=NotAvailable) obj_label = CleanText('./div//em') def obj_category(self): return self.env['category'] if 'category' in self.env else u'Externe' def obj_id(self): if self.page.IS_PRO_PAGE: return CleanText('(.//span[@class="_c1 doux _c1"])[1]', replace=[(' ', '')])(self.el) else: return CleanText('.//span[@class="_c1 doux _c1"]', replace=[(' ', '')])(self.el) def obj_iban(self): return Field('id')(self) def parse(self, el): self.env['origin_account']._external_recipients.add(Field('id')(self)) weboob-1.2/modules/creditmutuel/test.py000066400000000000000000000017641303450110500203400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class CreditMutuelTest(BackendTest): MODULE = 'creditmutuel' def test_crmut(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/cuisineaz/000077500000000000000000000000001303450110500162635ustar00rootroot00000000000000weboob-1.2/modules/cuisineaz/__init__.py000066400000000000000000000014371303450110500204010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import CuisineazModule __all__ = ['CuisineazModule'] weboob-1.2/modules/cuisineaz/browser.py000066400000000000000000000026061303450110500203240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import RecipePage, ResultsPage __all__ = ['CuisineazBrowser'] class CuisineazBrowser(PagesBrowser): BASEURL = 'http://www.cuisineaz.com' TIMEOUT = 20 search = URL('recettes/recherche_v2.aspx\?recherche=(?P.*)', ResultsPage) recipe = URL('recettes/(?P<_id>.*).aspx', RecipePage) def iter_recipes(self, pattern): return self.search.go(pattern=pattern.replace(' ', '-')).iter_recipes() def get_recipe(self, _id, obj=None): return self.recipe.go(_id=_id).get_recipe(obj=obj) def get_comments(self, _id): return self.recipe.stay_or_go(_id=_id).get_comments() weboob-1.2/modules/cuisineaz/favicon.png000066400000000000000000000025301303450110500204160ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME95oIDATxo[učKmRGnڦ"!I[)/F @`,`9ivX"h*heA$G$nl'H=%k;Y%iK X4sɀjCW݌FF{2F(z/h5bdO=SVQ+ufEY Vs ^n'Xqz9fwR"P b15%wc]ZkV!Sv  Ywy3^:dLS^6_W9E N:##+- ?eҖnalvӂ=g"H[ 7 @V\3爓&ܔ([apU#T (sH89=uw_q2`HNLMGi\~%h!y&k(SٯMZP!{ؘBZ7-jk%LKInR@uTϛWkw_K-[pƤE!fGt" pj;ڵ:h.wacѺ{,'mO6+UV"pƔ%yRUPQPƿ{'\";hhE݌@F%.@v6ZMV AVZrN٧SƧ}D. from weboob.capabilities.recipe import CapRecipe, Recipe from weboob.tools.backend import Module from .browser import CuisineazBrowser import unicodedata __all__ = ['CuisineazModule'] def strip_accents(s): return ''.join(c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn') class CuisineazModule(Module, CapRecipe): NAME = 'cuisineaz' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'Cuisine AZ French recipe website' LICENSE = 'AGPLv3+' BROWSER = CuisineazBrowser def get_recipe(self, id): return self.browser.get_recipe(id) def iter_recipes(self, pattern): # the search form does that so the url is clean of special chars # we go directly on search results by the url so we strip it too return self.browser.iter_recipes(strip_accents(unicode(pattern)).encode('utf-8')) def fill_recipe(self, recipe, fields): if 'nb_person' in fields or 'instructions' in fields: recipe = self.browser.get_recipe(recipe.id, recipe) if 'comments' in fields: recipe.comments = list(self.browser.get_comments(recipe.id)) return recipe OBJECTS = {Recipe: fill_recipe} weboob-1.2/modules/cuisineaz/pages.py000066400000000000000000000105201303450110500177320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.recipe import Recipe, Comment from weboob.capabilities.base import NotAvailable from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ItemElement, method, ListElement from weboob.browser.filters.standard import CleanText, Regexp, Env, Time, Join, Format from weboob.browser.filters.html import XPath import re import datetime class CuisineazDuration(Time): klass = datetime.timedelta _regexp = re.compile(r'((?P\d+) h)?((?P\d+) min)?(?P\d+)?') kwargs = {'hours': 'hh', 'minutes': 'mm', 'seconds': 'ss'} class ResultsPage(HTMLPage): """ Page which contains results as a list of recipies """ @pagination @method class iter_recipes(ListElement): item_xpath = '//div[@id="divRecette"]' def next_page(self): next = CleanText('//li[@class="next"]/span/a/@href', default=None)(self) if next: return next class item(ItemElement): klass = Recipe def condition(self): return Regexp(CleanText('./div/h2/a/@href'), '/recettes/(.*).aspx', default=None)(self.el) obj_id = Regexp(CleanText('./div/h2/a/@href'), '/recettes/(.*).aspx') obj_title = CleanText('./div/h2/a') obj_thumbnail_url = Format('http:%s', CleanText('./div[has-class("searchImg")]/span/img[@data-src!=""]/@data-src|./div[has-class("searchImg")]/div/span/img[@src!=""]/@src', default=None)) obj_short_description = CleanText('./div[has-class("show-for-medium")]') class RecipePage(HTMLPage): """ Page which contains a recipe """ @method class get_recipe(ItemElement): klass = Recipe obj_id = Env('_id') obj_title = CleanText('//h1') obj_picture_url = Format('http:%s', CleanText('//img[@id="shareimg" and @src!=""]/@src', default=None)) obj_thumbnail_url = Format('http:%s', CleanText('//img[@id="shareimg" and @src!=""]/@src', default=None)) def obj_preparation_time(self): _prep = CuisineazDuration(CleanText('//span[@id="ctl00_ContentPlaceHolder_LblRecetteTempsPrepa"]'))(self) return int(_prep.total_seconds() / 60) def obj_cooking_time(self): _cook = CuisineazDuration(CleanText('//span[@id="ctl00_ContentPlaceHolder_LblRecetteTempsCuisson"]'))(self) return int(_cook.total_seconds() / 60) def obj_nb_person(self): nb_pers = CleanText('//span[@id="ctl00_ContentPlaceHolder_LblRecetteNombre"]')(self) return [nb_pers] if nb_pers else NotAvailable def obj_ingredients(self): ingredients = [] for el in XPath('//section[has-class("recipe_ingredients")]/ul/li')(self): ingredients.append(CleanText('.')(el)) return ingredients obj_instructions = Join('\n\n - ', '//div[@id="preparation"]/span/p/text()', addBefore=' - ') @method class get_comments(ListElement): item_xpath = '//div[has-class("comment")]' class item(ItemElement): klass = Comment obj_author = CleanText('./div/div/div/div[@class="author"]') obj_text = CleanText('./div/div/p') obj_id = CleanText('./@id') def obj_rate(self): return len(XPath('.//div/div/div/div/div[@class="icon-star"]')(self)) weboob-1.2/modules/cuisineaz/test.py000066400000000000000000000022201303450110500176100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest import itertools class CuisineazTest(BackendTest): MODULE = 'cuisineaz' def test_recipe(self): recipes = list(itertools.islice(self.backend.iter_recipes(u'purée'), 0, 20)) assert len(recipes) full_recipe = self.backend.get_recipe(recipes[-1].id) assert full_recipe.instructions assert full_recipe.ingredients assert full_recipe.title weboob-1.2/modules/dailymotion/000077500000000000000000000000001303450110500166215ustar00rootroot00000000000000weboob-1.2/modules/dailymotion/__init__.py000066400000000000000000000001071303450110500207300ustar00rootroot00000000000000from .module import DailymotionModule __all__ = ['DailymotionModule'] weboob-1.2/modules/dailymotion/browser.py000066400000000000000000000063141303450110500206620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from urllib import quote_plus import re from weboob.browser import PagesBrowser, URL from .pages import IndexPage, VideoPage __all__ = ['DailymotionBrowser'] class DailymotionBrowser(PagesBrowser): BASEURL = 'http://www.dailymotion.com' video_page = URL(r'http://[w\.]*dailymotion\.com/video/(?P<_id>.*)', VideoPage) latest_page = URL(r'/1', IndexPage) index_page = URL(r'http://[w\.]*dailymotion\.com/(?P.*)', r'http://[w\.]*dailymotion\.com/1', r'http://[w\.]*dailymotion\.com/[a-z\-]{2,5}/1', r'http://[w\.]*dailymotion\.com/[a-z\-]{2,5}/(\w+/)?search/.*', IndexPage) def __init__(self, resolution, format, *args, **kwargs): self.resolution = resolution self.format = format PagesBrowser.__init__(self, *args, **kwargs) def get_video(self, _id, video=None): video = self.video_page.go(_id=_id).get_video(obj=video) if video._formats and self.format in video._formats: video.ext = self.format if self.format == u'm3u8': video.url = self.retrieve_m3u8_url(video._formats.get(self.format)) elif self.resolution in video._formats.get(self.format): video.url = video._formats.get(self.format).get(self.resolution) else: video.url = video._formats.get(self.format).values()[-1] return video def retrieve_m3u8_url(self, urls): if self.resolution in urls: return urls.get(self.resolution) return_next = False for resolution, url in urls.items(): for item in self.read_url(url): if return_next: return unicode(item.split('#')[0]) m = re.match('^#.*,NAME="%s"' % self.resolution, item) if not m: continue return_next = True return unicode(item.split('#')[0]) def read_url(self, url): r = self.open(url, stream=True) buf = r.iter_lines() return buf def search_videos(self, pattern, sortby): pattern = pattern.replace('/', '').encode('utf-8') if sortby is None: url = 'en/search/%s/1' % quote_plus(pattern) else: url = 'en/%s/search/%s/1' % (sortby, quote_plus(pattern)) return self.index_page.go(search=url).iter_videos() def latest_videos(self): return self.latest_page.go().iter_videos() weboob-1.2/modules/dailymotion/favicon.png000066400000000000000000000020451303450110500207550ustar00rootroot00000000000000PNG  IHDR@@APLTE !!"#       ""$ % '"(#)#*$*%,&.(/)4-;3 ;:<4 =5 ?7 LKMLNMOE ONRH UJ USYMYN_[dWgbk]n`ohum{lqx st}~êɰ ʰ ˱ ϴ и"$$%$&&'''''''(bfbƊ3Lb%,gtRNS>,IDATXíՉRA$ r "xx EEv+=JR_:3Y0u9p,^KJEBvY*Nת8 (]Ib!k>**%tU*P-RǪV tZ4%T4 ȇʆu+Svp4t̛Dm6]*6ο_ GO96?QK6*JLy |$Z?*R"uY02קR LJ. Wy3 9@(P|~/ C, ȀU86;~s !C`0Np\pvrhcD>Z1pElO` ! :ςއVd5:FEwWA cϧټ&ojE#t ]-#ި!$3( g gy7 ֩20Pxްyv@vr~b>cšG~9̸"}l>frIENDB`weboob-1.2/modules/dailymotion/module.py000066400000000000000000000070261303450110500204650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.tools.ordereddict import OrderedDict from .browser import DailymotionBrowser import re __all__ = ['DailymotionModule'] class DailymotionModule(Module, CapVideo, CapCollection): NAME = 'dailymotion' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'Dailymotion video streaming website' LICENSE = 'AGPLv3+' BROWSER = DailymotionBrowser resolution_choice = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ u'480': u'480p', u'240': u'240p', u'380': u'380p', u'720': u'720p', u'1080': u'1080p' }.iteritems())]) format_choice = [u'm3u8', u'mp4'] CONFIG = BackendConfig(Value('resolution', label=u'Resolution', choices=resolution_choice), Value('format', label=u'Format', choices=format_choice)) SORTBY = ['relevance', 'rated', 'visited', None] def create_default_browser(self): resolution = self.config['resolution'].get() format = self.config['format'].get() return self.create_browser(resolution=resolution, format=format) def get_video(self, _id): m = re.match('http://[w\.]*dailymotion\.com/video/(.*)', _id) if m: _id = m.group(1) if not _id.startswith('http'): return self.browser.get_video(_id) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern, self.SORTBY[sortby]) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest Dailymotion videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {BaseVideo: fill_video} weboob-1.2/modules/dailymotion/pages.py000066400000000000000000000106751303450110500203030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage, pagination from weboob.browser.filters.standard import CleanText, Regexp, Env, Duration, DateTime from weboob.browser.filters.html import Link from weboob.capabilities.base import NotAvailable from weboob.capabilities.video import BaseVideo from weboob.capabilities.image import Thumbnail from weboob.exceptions import ParseError from weboob.tools.json import json from datetime import timedelta import re def determine_ext(url, default_ext='unknown_video'): if url is None: return default_ext guess = url.partition('?')[0].rpartition('.')[2] if re.match(r'^[A-Za-z0-9]+$', guess): return guess elif guess.rstrip('/') in ('mp4', 'm3u8'): return guess.rstrip('/') else: return default_ext class IndexPage(HTMLPage): @pagination @method class iter_videos(ListElement): item_xpath = '//div[@data-video-id]' next_page = Link('//a[@title="suivant"]') class item(ItemElement): klass = BaseVideo def validate(self, obj): return obj.id obj_id = CleanText('./div/@data-playable') obj_title = CleanText('./div[@class="media-block"]/h3') obj_author = CleanText('./div[@class="media-block"]/div/span/a') obj_duration = Duration(CleanText('./div/a/div[has-class("badge--duration")]'), default=NotAvailable) def obj_thumbnail(self): url = CleanText('./div/a/img/@data-src')(self) thumbnail = Thumbnail(url) thumbnail.url = url return thumbnail class VideoPage(HTMLPage): @method class get_video(ItemElement): klass = BaseVideo obj_id = Env('_id') obj_title = CleanText('//title') obj_author = CleanText('//meta[@name="author"]/@content') obj_description = CleanText('//meta[@name="description"]/@content') def obj_duration(self): seconds = int(CleanText('//meta[@property="video:duration"]/@content', default=0)(self)) return timedelta(seconds=seconds) def obj_thumbnail(self): url = CleanText('//meta[@property="og:image"]/@content')(self) thumbnail = Thumbnail(url) thumbnail.url = url return thumbnail obj_date = DateTime(CleanText('//meta[@property="video:release_date"]/@content')) def obj__formats(self): player = Regexp(CleanText('//script'), '.*var config = ({"context".*}}});\s*buildPlayer\(config\);.*', default=None)(self) if player: info = json.loads(player) if info.get('error') is not None: raise ParseError(info['error']['title']) metadata = info.get('metadata') formats = {} for quality, media_list in metadata['qualities'].items(): for media in media_list: media_url = media.get('url') if not media_url: continue type_ = media.get('type') if type_ == 'application/vnd.lumberjack.manifest': continue ext = determine_ext(media_url) if ext in formats: if quality in formats.get(ext): formats[ext][quality] = media_url else: formats[ext] = {quality: media_url} else: formats[ext] = {quality: media_url} return formats return None weboob-1.2/modules/dailymotion/test.py000066400000000000000000000047301303450110500201560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo from random import choice import itertools class DailymotionTest(BackendTest): MODULE = 'dailymotion' # Not easy to find a kids video which will always be there # This might break in the future KIDS_VIDEO_TITLE = 'Telmo et Tula' def test_search(self): l = list(itertools.islice(self.backend.search_videos('chirac'), 0, 20)) self.assertTrue(len(l) > 0) v = choice(l) self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_latest(self): l = list(itertools.islice(self.backend.iter_resources([BaseVideo], [u'latest']), 0, 20)) assert len(l) v = choice(l) self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_kids_video(self): l = list(itertools.islice(self.backend.search_videos(DailymotionTest.KIDS_VIDEO_TITLE), 0, 20)) self.assertTrue(len(l) > 0) for elt in l[:10]: video_id = elt.id video = self.backend.get_video(video_id) self.assertIsNotNone(video.title) if DailymotionTest.KIDS_VIDEO_TITLE in video.title: self.assertTrue(video.url and video.url.startswith('http://'), 'URL for video "%s" not found: %s' % (video.id, video.url)) return self.fail("Can't find test video '%s' in kids.dailymotion.com video " "on dailymotion, maybe the test video should be changed." % DailymotionTest.KIDS_VIDEO_TITLE) weboob-1.2/modules/delubac/000077500000000000000000000000001303450110500156705ustar00rootroot00000000000000weboob-1.2/modules/delubac/__init__.py000066400000000000000000000014411303450110500200010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Noe Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import DelubacModule __all__ = ['DelubacModule'] weboob-1.2/modules/delubac/browser.py000066400000000000000000000046241303450110500177330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import AccountNotFound from weboob.capabilities.base import find_object from .pages import LoginPage, MenuPage, AccountsPage, HistoryPage, IbanPage __all__ = ['DelubacBrowser'] class DelubacBrowser(LoginBrowser): BASEURL = 'https://e.delubac.com' home = URL('/es@b/fr/esab.jsp') login = URL('/es@b/fr/codeident.jsp', '/es@b/servlet/internet0.ressourceWeb.servlet.Login', LoginPage) menu = URL('/es@b/fr/menuConnecte1.jsp\?c&deploye=false&pulseMenu=false&styleLien=false&dummyDate=(?P.*)', MenuPage) accounts = URL('/es@b/servlet/internet0.ressourceWeb.servlet.EsabServlet.*', AccountsPage) history = URL('/es@b/servlet/internet0.ressourceWeb.servlet.ListeDesMouvementsServlet.*', HistoryPage) iban = URL('/es@b/fr/rib.jsp', IbanPage) def do_login(self): self.home.go() self.login.go() self.page.login(self.username, self.password) if self.page.incorrect_auth: raise BrowserIncorrectPassword() @need_login def iter_accounts(self): self.menu.go(date=int(time.time()*1000)) self.location(self.page.accounts_url) for account in self.page.get_list(): self.location(account._link) account.iban = self.page.get_iban() yield account @need_login def get_account(self, _id): return find_object(self.iter_accounts(), id=_id, error=AccountNotFound) @need_login def iter_history(self, account): self.location(account._link) return self.page.get_transactions() weboob-1.2/modules/delubac/favicon.png000066400000000000000000000053531303450110500200310ustar00rootroot00000000000000PNG  IHDR@@% pHYs76vtIME )5_ IDAThZkTSW'7 Ix^3>7߿q-Ru'f͈]Aޮ՗!c=tgPa~"==:[:<̲um?Yx o>\kòXJ$4:-3ђ%_cA%N.NL{YqJqxzbʂlgM@74/&抅ƇE|`oOS'PT.4*~bKb1Ύޮw*&`siQƺ췠hhg/vVؼ/g/,:0p.Wm'MH\/axJLP A˛K`KCE}qJr\LnjBF@d9[gkP GO+i64ArE{%p_ R>O۩A`Q:apv3qdΞ.5T4T, w_[o; GjjH*] _'G$F(RQgO熊zb;Fu\l-Wf%RbbsMWgA&0; 9bX^8cptu0dqT5*M)؛LQ&7|xL7` [ưS_JAu_JAT#.~0Ti\0E_N픉 17?x!OFi,{zH[$Rr]cOW/W8(Kg:S^3Rݸppר5T_Ĝeo cMªsY@8Mß+.>oOJ,K/V6RioAƆeC-cRee[K"Jwi:u/\x8<`9R~4/M rgl]AШ50./ېbS s88Do2wt8nvlsÜ@`d6$j E_=02[b@h|~%x_ÇPKP_Xk ?<H393{l\=54Zl@;s̥3 ~WUq*+4>lրo2`waBK\''k@G-N4$;h.g>PP _0wL7JCT!Re r{OeEI=YjeYf n>nDfCc`{yEDTA7m  M0*v:^WVg,E@湛 *T$2p?bWO_"7!ܩ-y(1K2x0k3nXوۍFA g_ aš@l׍lݐ߾Id?뿶p]F!:MaEH9r,*8L|j]P ɓ IQ_γaY{*Mj_ S*0\!*D&==x88M>Τ뺱P3-M-p ryH #rt&-My 4~lG\D fW6RCܭs(U])3t= @DU/ b=yIߒN|tkjZoY~~PO4@:0v {BY)9c~jHw^㛙f䤆QO9qbP[VSCSuM9$zQSv,qZ'(YO߬. from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import DelubacBrowser __all__ = ['DelubacModule'] class DelubacModule(Module, CapBank): NAME = 'delubac' DESCRIPTION = u'Banque Delubac & Cie' MAINTAINER = u'Noe Rubinstein' EMAIL = 'nru@budget-insight.com' VERSION = '1.2' BROWSER = DelubacBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe', regexp='^\d+$')) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.iter_accounts() def get_account(self, _id): return self.browser.get_account(_id) def iter_history(self, account, coming=False): return self.browser.iter_history(account) weboob-1.2/modules/delubac/pages.py000066400000000000000000000251121303450110500173420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from io import BytesIO from weboob.capabilities.bank import Account from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, ItemElement, method from weboob.exceptions import ParseError from weboob.tools.captcha.virtkeyboard import GridVirtKeyboard from weboob.browser.filters.standard import CleanText, CleanDecimal, Field, Format, Date, Filter from weboob.browser.filters.html import Link from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.capabilities.base import NotAvailable class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^CARTE DU'), FrenchTransaction.TYPE_CARD), (re.compile(u'^VIR (SEPA)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^Vir (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^VIREMENT DE (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^CHQ (?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(u'^PRLV SEPA (?P.*)'), FrenchTransaction.TYPE_ORDER), ] class DelubacVirtKeyboard(GridVirtKeyboard): symbols = {'0': ('ad0ae6278e58bf2300cb7e0b87729e4d', '33e6fecddeb02ad3756570b541320914', '9f47d540a8bf89056106bc2c06fdea2e', '176c63721c61da6cbae04ed95b36a343', '6af3ed27e718a93acc8923abebef4734'), '1': ('ba6ca06344687bd2337e2bfb39e2378a', 'cef70da6fb76b8bb90c025d4e62db7f6'), '2': ('c442cd1cee0c95d8423665983e9e45b7', '38aac4c0dcc1c40bfa0173bb2fc959f6', '8be33fe7de693d8d1404c0b51ce6f893', '85bf89cc03d7d772f7b0b4aa9a7897c6', '955423582ae4cde07b0831eee879c068', '19712acd76beedef3038b131ee9b5f19', '34b0276d52b8bdc2e68c98f1bac8fecc'), '3': ('f5ebe3bdfc645210c38f1b89bde6b732', '0ff42e4481af10901213636a654d56ab', '3b6b94bc9748a881fd0692f5a8c4a791', '38809719796e812285e8da365268336d', '31ee44b350e8a062fcde5c8c6e8e1dbf', '84012a1fe9c2809fb348d3dcd362d15a', '4e076b4b60852f8e8fb40ac751a8c7a5'), '4': ('6d194520cb4d7e5ba0c24fa6dad21285', '08e40532e7c4207239c543576a2fa7b1', '4f7267f5879ba5fdda61ff6ac7897d75', '7a5d5abb743d13a036a7d8582576b284', '59767e1a982ee4212d795acf1f3bda3d', '9f796703a309453d5f4c0f3baddf2f0d', '9d52b25d92866001c6035ea3abfc04ed', 'b6fe068a99e5b93af2348947ddeddf5c', '9a021f8eb7ab9091aeba512481de3301'), '5': ('b3d5990972b82abd9e8c049457794ab1', '4c761e7b65eed3baf899960e0ba0d131', '729a2baf278fe557ee54f532b8263a98', '634e8b3ec904f219b858990a739b8283', '55138402b60e3d3027e1d26b078b73f9', '7a03617f129806672ae5c4b478535b29', 'c4bb4958271d61d098bd239e403e9ecc'), '6': ('c69e8ad49ffa359ca7f77658c090e1dd', 'e0294f0af8037d544f5b4729931453f3', '359f46ac7dc0d8dee18831981c863a73', '6752eb433a0f72312f19b24f69ea6adb', '50e2c34749b3aa4010db73d108e241ae', '3fdc463cfea86a565151059d8d114123', '9e321e0d31a1e54510b51ae5c2e91749', 'fab5e1970f78c521a94d8aa52a88075a', 'b620c174da8d74d04139b0d334d064b5', 'd83eff07ba513c875dc42ec87a441989', '67b214138a7efe3f600adaea1d1cffbc', '014dc575b0a980b0037cf1e8f5ed9987', '0994aab1f56f9427d1f5dbba27eb3266', 'bda8a78783bddfbb8a3bc6b03ba1c449', '1ddb303b52769096582095e70ef481a4', '344642e1a97750328f058e1dcb4cd6e9', '4250fbdcf8859e6b45fc065f1bbd3967'), '7': ('57b179554275be15562824284395c833', '687f5020c59bdde6edac72917cbbebc2', '6752eb433a0f72312f19b24f69ea6adb', 'ea6eec45008ddb61cd830bd690a3d55f'), '8': ('394752d62aea07c5c5f5650b3e404e88', 'dc72f59b61e18dd313e16d4ee0bb8207', '06297c98471f5a06a9c180dbb8b82305', '8349e6ae068167ee565721a34a4bcc9f', '1347687d61f0fb4c9c0601a9ff4c7d60', '5dd319ab0d0dd40730b90abdf2ad27c4', '70e543a626496f719f837e57a798c288', 'ab34d9ff8f1504e949e06d7c51049837', 'bafa9bc81270863eeaba9267412ce854', '27f6b4127a02026ce5bb42a7686a05de', '4ceccf7b8f24316465838c5f97bc06c8', 'bef000f56d1907697a6e0922b2a5184b', '55b2820aec3e9cb8af88e50068620a73', 'f6722d76cca62ebb150c1b79338c2325', 'c74c0c5231a656b95b407e995793b80a', '3398b5e93b96c90557406ca461a55da0', '80cd1440a3b2a4b8f3480ee25e3e4c5d', 'b6429ded63ab3e6d62fb9542dc76a42d'), '9': ('d040ced0ae3a55365fe46b498316d734', '0f518491d0a16f91bd6e4df1f9db47b2', '59a08f18d72d52eed250cc6a88aebd40', '0e474a38e1c330613e406bf738723f01', '4bfe94b095e8c4b391abf42c0a4a8dc6', '1810d4ce38929acaa8a788a35b9b5e5d', 'e19ea6338bcbbcaf3e40017bea21c770', '2f2b034747480bdc583335a43b5cdcb7', '15497fc6422cd626d4d3639dbb01aa35', '1bf0815fee3796e0b745207fbcc4e511',)} margin = 1 color = (0xff,0xf7,0xff) nrow = 4 ncol = 4 def __init__(self, browser, session, codes): f = BytesIO(browser.open('%s.jpg' % session).content) super(DelubacVirtKeyboard, self).__init__(range(16), self.ncol, self.nrow, f, self.color) self.check_symbols(self.symbols, browser.responses_dirname) self.codes = codes def check_color(self, pixel): for p in pixel: if p < 0xd0: return False return True def get_string_code(self, string): res = [] ndata = self.nrow * self.ncol for nbchar, c in enumerate(string): index = self.get_symbol_code(self.symbols[c]) res.append(self.codes[(nbchar * ndata) + index]) return ','.join(res) class LoginPage(HTMLPage): def login(self, username, password): for script in self.doc.xpath('//script'): m = re.search("session='([^']+)'", script.text or '') if m: session = m.group(1) m = re.search('codes = "([^"]+)"', script.text or '') if m: codes = m.group(1).split(',') vk = DelubacVirtKeyboard(self.browser, session, codes) form = self.get_form(name='codeident') form['identifiant'] = username form['motpasse'] = vk.get_string_code(password) form['CodSec'] = vk.get_string_code(password) form['modeClavier'] = '1' form['identifiantDlg'] = '' form.submit() @property def incorrect_auth(self): return len(self.doc.xpath('//td[contains(text(), "Authentification incorrecte")]')) class MenuPage(LoggedPage, HTMLPage): def get_link(self, name): for script in self.doc.xpath('//script'): m = re.search(r"""\["%s",'([^']+)'""" % name, script.text or '', flags=re.MULTILINE) if m: return m.group(1) raise ParseError('Link %r not found' % name) @property def accounts_url(self): return self.get_link(u'Comptes') class AccountsPage(LoggedPage, HTMLPage): TYPES = {'COMPTE COURANT': Account.TYPE_CHECKING, } is_here = u'//title[text() = "Solde de chacun de vos comptes"]' @method class get_list(ListElement): item_xpath = '//tr' class item(ItemElement): klass = Account def condition(self): if len(self.el.xpath('./td')) < 6: return False return True class Type(Filter): def filter(self, label): for pattern, actype in AccountsPage.TYPES.iteritems(): if pattern in label: return actype return Account.TYPE_UNKNOWN obj__title = CleanText('td[@id="idCompteIntitule"]') obj__nature = CleanText('td[@id="idCompteNature"]') obj_label = Format('%s %s', Field('_title'), Field('_nature')) obj_currency = FrenchTransaction.Currency('./td[@id="idCompteSoldeUM"]') obj_id = CleanText('td[@id="idCompteLibelle"]/a') obj_balance = CleanDecimal('td[@id="idCompteSolde"]', replace_dots=True) obj__link = Link('td[@id="idCompteLibelle"]/a') obj_type = Type(Field('label')) class HistoryPage(LoggedPage, HTMLPage): is_here = u'//title[text() = "Liste des opérations sur un compte"]' @pagination @method class get_transactions(ListElement): item_xpath = '//tr' def next_page(self): for script in self.page.doc.xpath('//script'): m = re.search(r"""getCodePagination\('(?P\d+)','(?P\d+)','(?P[^']+)'""", script.text or '', flags=re.MULTILINE) if m: next_page = int(m.group('current_page')) + 1 max_page = int(m.group('max_page')) if next_page <= max_page: return m.group('link') + '&numeroPage=' + str(next_page) class item(ItemElement): klass = Transaction def condition(self): if len(self.el.xpath('./td')) < 5 or len(self.el.xpath('./td[@class="TitreTableau"]')) > 0: return False return True obj_raw = Transaction.Raw('.//td[@class="ColonneLibelle"]') obj_amount = CleanDecimal('.//td[4] | .//td[5]', replace_dots=True) obj_date = Date(CleanText('.//td[1]'), dayfirst=True) obj_vdate = Date(CleanText('.//td[3]'), dayfirst=True) def get_iban(self): iban_link = Link('//a[contains(@href, "rib")]', default=NotAvailable)(self.doc) if not iban_link: return NotAvailable self.browser.location(iban_link) return self.browser.page.get_iban() class IbanPage(LoggedPage, HTMLPage): def get_iban(self): return CleanText('//td[contains(text(), "IBAN") and @class="ColonneCode"]', replace=[('IBAN', ''), (' ', '')])(self.doc) weboob-1.2/modules/delubac/test.py000066400000000000000000000017531303450110500172270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Noe Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class DelubacTest(BackendTest): MODULE = 'delubac' def test_delubac(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/dhl/000077500000000000000000000000001303450110500150405ustar00rootroot00000000000000weboob-1.2/modules/dhl/__init__.py000066400000000000000000000014311303450110500171500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import DHLModule __all__ = ['DHLModule'] weboob-1.2/modules/dhl/browser.py000066400000000000000000000025661303450110500171060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import DHLExpressSearchPage, DeutschePostDHLSearchPage class DHLExpressBrowser(PagesBrowser): BASEURL = 'http://www.dhl.com' search_page = URL('/shipmentTracking\?AWB=(?P.+)', DHLExpressSearchPage) def get_tracking_info(self, _id): return self.search_page.go(id=_id).get_info(_id) class DeutschePostDHLBrowser(PagesBrowser): BASEURL = 'https://nolp.dhl.de' search_page = URL('/nextt-online-public/set_identcodes.do\?lang=en&idc=(?P.+)', DeutschePostDHLSearchPage) def get_tracking_info(self, _id): return self.search_page.go(id=_id).get_info(_id) weboob-1.2/modules/dhl/favicon.png000066400000000000000000000006431303450110500171760ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME! (tEXtCommentCreated with GIMPWIDAThرn@E7&D&UK 'f;rc˳zkV1뇟W);KtCkno^c+g[К/@g@䵼3`Nj"f 5 M=aۿׁ2ߥzX\Ţ[nhsw%n|ɜV1[`ҏebF V!IߝH~. from weboob.tools.backend import Module from weboob.capabilities.parcel import CapParcel, ParcelNotFound from .browser import DHLExpressBrowser, DeutschePostDHLBrowser __all__ = ['DHLModule'] class DHLModule(Module, CapParcel): NAME = 'dhl' DESCRIPTION = u'DHL website' MAINTAINER = u'Matthieu Weber' EMAIL = 'mweber+weboob@free.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' def get_parcel_tracking(self, id): """ Get information abouut a parcel. :param id: ID of the parcel :type id: :class:`str` :rtype: :class:`Parcel` :raises: :class:`ParcelNotFound` """ self._browser = None if len(id) == 10 or len(id) == 20: self.BROWSER = DHLExpressBrowser elif len(id) == 12 or len(id) == 16: self.BROWSER = DeutschePostDHLBrowser else: ParcelNotFound("Wrong length for ID: %s" % id) return self.browser.get_tracking_info(id) weboob-1.2/modules/dhl/pages.py000066400000000000000000000072071303450110500165170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from dateutil.parser import parse as parse_date from weboob.browser.pages import JsonPage, HTMLPage from weboob.capabilities.parcel import Parcel, Event, ParcelNotFound class DHLExpressSearchPage(JsonPage): # Based on http://www.dhl.com/etc/designs/dhl/docroot/tracking/less/tracking.css STATUSES = { u'105': Parcel.STATUS_PLANNED, u'104': Parcel.STATUS_PLANNED, u'102': Parcel.STATUS_IN_TRANSIT, u'101': Parcel.STATUS_ARRIVED, } def get_info(self, _id): if u'errors' in self.doc: raise ParcelNotFound("No such ID: %s" % _id) elif u'results' in self.doc: result = self.doc[u'results'][0] p = Parcel(_id) p.history = [self.build_event(e) for e in result[u'checkpoints']] p.status = self.STATUSES.get(result[u'delivery'][u'code'], Parcel.STATUS_UNKNOWN) p.info = p.history[0].activity return p else: raise ParcelNotFound("Unexpected reply from server") def build_event(self, e): index = e[u'counter'] event = Event(index) event.date = parse_date(e[u'date'] + " " + e.get(u'time', ''), dayfirst=True, fuzzy=True) event.location = e.get(u'location', '') event.activity = e[u'description'] return event class DeutschePostDHLSearchPage(HTMLPage): # Based on http://www.parcelok.com/delivery-status-dhl.html STATUSES = { "Order data sent to DHL electronically": Parcel.STATUS_PLANNED, "International shipment": Parcel.STATUS_IN_TRANSIT, "Processing in parcel center": Parcel.STATUS_IN_TRANSIT, "Delivery": Parcel.STATUS_IN_TRANSIT, "Shipment has been successfully delivered": Parcel.STATUS_ARRIVED, } def get_info(self, _id): result_id = self.doc.xpath('//th[@class="mm_sendungsnummer"]') if not result_id: raise ParcelNotFound("No such ID: %s" % _id) result_id = result_id[0].text if result_id != _id: raise ParcelNotFound("ID mismatch: expecting %s, got %s" % (_id, result_id)) p = Parcel(_id) events = self.doc.xpath('//div[@class="accordion-inner"]/table/tbody/tr') p.history = [self.build_event(i, tr) for i, tr in enumerate(events)] status_msgs = self.doc.xpath('//tr[@class="mm_mailing_process "]//img[contains(@src, "ACTIVE")]/@alt') if len(status_msgs) > 0: p.status = self.STATUSES.get(status_msgs[-1], Parcel.STATUS_UNKNOWN) else: p.status = Parcel.STATUS_UNKNOWN p.info = p.history[-1].activity return p def build_event(self, index, tr): event = Event(index) event.date = parse_date(tr.xpath('./td[1]')[0].text.strip(), dayfirst=True, fuzzy=True) event.location = unicode(tr.xpath('./td[2]')[0].text_content().strip()) event.activity = unicode(tr.xpath('./td[3]')[0].text_content().strip()) return event weboob-1.2/modules/dhl/test.py000066400000000000000000000017361303450110500164000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.parcel import ParcelNotFound class DHLTest(BackendTest): MODULE = 'dhl' def test_dhl(self): self.assertRaises(ParcelNotFound, self.backend.get_parcel_tracking, "foo") weboob-1.2/modules/dlfp/000077500000000000000000000000001303450110500152165ustar00rootroot00000000000000weboob-1.2/modules/dlfp/__init__.py000066400000000000000000000014731303450110500173340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .browser import DLFP from .module import DLFPModule __all__ = ['DLFP', 'DLFPModule'] weboob-1.2/modules/dlfp/browser.py000066400000000000000000000217001303450110500172530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib import re import hashlib import lxml from weboob.deprecated.browser import Browser, BrowserHTTPNotFound, BrowserHTTPError, BrowserIncorrectPassword, BrokenPageError from weboob.capabilities.messages import CantSendMessage from .pages.index import IndexPage, LoginPage from .pages.news import ContentPage, NewCommentPage, NodePage, CommentPage, NewTagPage, RSSComment from .pages.board import BoardIndexPage from .pages.wiki import WikiEditPage from .tools import id2url, url2id # Browser class DLFP(Browser): DOMAIN = 'linuxfr.org' PROTOCOL = 'https' PAGES = {'https?://[^/]*linuxfr\.org/?': IndexPage, 'https?://[^/]*linuxfr\.org/compte/connexion': LoginPage, 'https?://[^/]*linuxfr\.org/news/[^\.]+': ContentPage, 'https?://[^/]*linuxfr\.org/wiki/(?!nouveau)[^/]+': ContentPage, 'https?://[^/]*linuxfr\.org/wiki': WikiEditPage, 'https?://[^/]*linuxfr\.org/wiki/nouveau': WikiEditPage, 'https?://[^/]*linuxfr\.org/wiki/[^\.]+/modifier': WikiEditPage, 'https?://[^/]*linuxfr\.org/suivi/[^\.]+': ContentPage, 'https?://[^/]*linuxfr\.org/sondages/[^\.]+': ContentPage, 'https?://[^/]*linuxfr\.org/users/[^\./]+/journaux/[^\.]+': ContentPage, 'https?://[^/]*linuxfr\.org/forums/[^\./]+/posts/[^\.]+': ContentPage, 'https?://[^/]*linuxfr\.org/nodes/(\d+)/comments/(\d+)': CommentPage, 'https?://[^/]*linuxfr\.org/nodes/(\d+)/comments/nouveau': NewCommentPage, 'https?://[^/]*linuxfr\.org/nodes/(\d+)/comments': NodePage, 'https?://[^/]*linuxfr\.org/nodes/(\d+)/tags/nouveau': NewTagPage, 'https?://[^/]*linuxfr\.org/board/index.xml': BoardIndexPage, 'https?://[^/]*linuxfr\.org/nodes/(\d+)/comments.atom': RSSComment, } last_board_msg_id = None def parse_id(self, _id): if re.match('^https?://.*linuxfr.org/nodes/\d+/comments/\d+$', _id): return _id, None url = id2url(_id) if url is None: if url2id(_id) is not None: url = _id _id = url2id(url) else: return None, None return url, _id def get_wiki_content(self, _id): url, _id = self.parse_id('W.%s' % _id) if url is None: return None try: self.location('%s/modifier' % url) except BrowserHTTPNotFound: return '' assert self.is_on_page(WikiEditPage) return self.page.get_body() def _go_on_wiki_edit_page(self, name): """ Go on the wiki page named 'name'. Return True if this is a new page, or False if the page already exist. Return None if it isn't a right wiki page name. """ url, _id = self.parse_id('W.%s' % name) if url is None: return None try: self.location('%s/modifier' % url) except BrowserHTTPNotFound: self.location('/wiki/nouveau') new = True else: new = False assert self.is_on_page(WikiEditPage) return new def set_wiki_content(self, name, content, message): new = self._go_on_wiki_edit_page(name) if new is None: return None if new: title = name.replace('-', ' ') else: title = None self.page.post_content(title, content, message) def get_wiki_preview(self, name, content): if self._go_on_wiki_edit_page(name) is None: return None self.page.post_preview(content) if self.is_on_page(WikiEditPage): return self.page.get_preview_html() elif self.is_on_page(ContentPage): return self.page.get_article().body def get_hash(self, url): self.location(url) if self.page.document.xpath('//entry'): myhash = hashlib.md5(lxml.etree.tostring(self.page.document)).hexdigest() return myhash else: return None def get_content(self, _id): url, _id = self.parse_id(_id) if url is None: return None self.location(url) self.page.url = self.absurl(url) if self.is_on_page(CommentPage): content = self.page.get_comment() elif self.is_on_page(ContentPage): m = re.match('.*#comment-(\d+)$', url) if m: content = self.page.get_comment(int(m.group(1))) else: content = self.page.get_article() else: raise BrokenPageError('Not on a content or comment page (%r)' % self.page) if _id is not None: content.id = _id return content def _is_comment_submit_form(self, form): return 'comment_new' in form.action def post_comment(self, thread, reply_id, title, message): url = id2url(thread) if url is None: raise CantSendMessage('%s is not a right ID' % thread) self.location(url) assert self.is_on_page(ContentPage) self.location(self.page.get_post_comment_url()) assert self.is_on_page(NewCommentPage) self.select_form(predicate=self._is_comment_submit_form) self.set_all_readonly(False) if title is not None: self['comment[title]'] = title.encode('utf-8') self['comment[wiki_body]'] = message.encode('utf-8') if int(reply_id) > 0: self['comment[parent_id]'] = str(reply_id) self['commit'] = 'Poster le commentaire' try: self.submit() except BrowserHTTPError as e: raise CantSendMessage('Unable to send message to %s.%s: %s' % (thread, reply_id, e)) if self.is_on_page(NodePage): errors = self.page.get_errors() if len(errors) > 0: raise CantSendMessage('Unable to send message: %s' % ', '.join(errors)) return None def login(self): if self.username is None: return # not usefull for the moment #self.location('/', no_login=True) data = {'account[login]': self.username, 'account[password]': self.password, 'account[remember_me]': 1, #'authenticity_token': self.page.get_login_token(), } self.location('/compte/connexion', urllib.urlencode(data), no_login=True) if not self.is_logged(): raise BrowserIncorrectPassword() self._token = self.page.document.xpath('//input[@name="authenticity_token"]') def is_logged(self): return (self.username is None or (self.page and self.page.is_logged())) def close_session(self): if self._token: self.openurl('/compte/deconnexion', urllib.urlencode({'authenticity_token': self._token[0].attrib['value']})) def plusse(self, url): return self.relevance(url, 'for') def moinse(self, url): return self.relevance(url, 'against') def relevance(self, url, what): comment = self.get_content(url) if comment is None: raise ValueError('The given URL isn\'t a comment.') if comment.relevance_token is None: return False res = self.readurl('%s%s' % (comment.relevance_url, what), urllib.urlencode({'authenticity_token': comment.relevance_token})) return res def iter_new_board_messages(self): self.location('/board/index.xml') assert self.is_on_page(BoardIndexPage) msgs = self.page.get_messages(self.last_board_msg_id) for msg in reversed(msgs): self.last_board_msg_id = msg.id yield msg def board_post(self, msg): request = self.request_class(self.absurl('/board/'), urllib.urlencode({'board[message]': msg}), {'Referer': self.absurl('/')}) self.readurl(request) def add_tag(self, _id, tag): url, _id = self.parse_id(_id) if url is None: return None self.location(url) assert self.is_on_page(ContentPage) self.location(self.page.get_tag_url()) assert self.is_on_page(NewTagPage) self.page.tag(tag) weboob-1.2/modules/dlfp/favicon.png000066400000000000000000000117641303450110500173620ustar00rootroot00000000000000PNG  IHDR@@% sRGB pHYs  tIME ,0aIDAThi]W}{fެ{I;5-EiPJ*TʇVHtIP@D Nxyom~{8&F)]{wYq~2*;]wޡC)%tloeiW@D@HBCoF)%5=ED!yK(Ap+gOT6}F w qJ9e c ʖJ-!76ŕ_^9VN|]@W2@3B2 eK𶏰[ֵ*S i(lF!,4 !7˷"w#@G2 MP9Pz H)>BnN$NW!$-$ CC[l^Xs0er JޒrS9p cw$UCۄit 8{h806K6@AWCX>tcwt}ox? lL}r@|GukI 0 ݾ_,J$RB2:]pSFi3ɣ?]/LKUJǦ+p7T-Eoh6![`3h΀1 *o߆O5ie]RY^[%y)H[BeS jso#2B[Qa93+Mhf&cUvNl&V]ؽ'|/).峾~rd?C>|[ķO7!K߁/~`B6>ޣ WNRcc{ta~85}^saZ}q\"#f#%|G٧JS^pG+Ώ{=cgcjw]P#\mw>nJ/u07o=T\󯟩.#K;eB%NjG۱F up{4{EDLkwU O KO&W3}e|vCyaOCeeV'aJ kH  Ʀqhvx#2H8?󕧞jL;6/_TulGlni/$L.W*iإr>kj*dT*UL|#Ϗf-i$H4+ޑwda~Y^H,g ?~{v;Lo]1Il E"dm:{e`/~;z{JzԹK=陹d93<|Vav{uEUhЃR-8yύw"Pa';hϯm8ٻt4Je"WF;s/L<]u0@v(|!T,7Nu@#IA.,.Mb0!1h678<A?^vkfz2D໎U!(p9K텞{_ƨp,bcmcU6=X{c3Fp@N]BLM5v-ݬι"S)o\1] {3h] ͹V-O7zw.=Z[|J_OWyx-XVwWAf(΍֮.wR2L(\׋ kfۘ^Sf8i @@JA8X]7 ]JzU؁S9c^ , #yćyExJ3?9r`﫯](umbx}8kb#e@H Rp=ǁ\0e.tw)0#H/^+n B,[ɖsrezZJv .\vCB>̳å򐢀:sZRKs3 Ł}{FED0[h,c*Gy:pFn '7w \&ԛO:;`lFNkYSa~٨xn(W4r Sזىzfuc.\u0 +݅t )A"(\Mc=@˅'EKmCZqTb"vck:RJs fR[1 X{BG:D"CdH8NCD5ͤd Ixm:;2ުܩ}@d1 ̏ ̊z$j ?V#SWS 5FrlޥɕJʨ!!ه!EJY--DD5gIcי$Bf6kVHP$cЙ%n]]s+LȔPU.!`-WZRW&\E-~($t y>&4C3 2S45IŲb眛8i+`hPHQl%;f@(CntYWa|zA"2R )( %HS ;7n(J7"vRRdU%p:>s$U[Ԉi(NR 1H@RN5bhiL Vܠ^G"Bȭ DGc)! f;` HR`2;Wʭ_iTڈh6!`jThsEz3D*"PB$P6C|&ԍtHf6ҙ*d46w @Ptzf޲6@ٳJ h**zxRfS.1J+f9፱/A;J e))RJ8iQC$ P PS@x$u0w-̌[ +b-XIENDB`weboob-1.2/modules/dlfp/module.py000066400000000000000000000243741303450110500170670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime, timedelta import time from weboob.tools.backend import Module, BackendConfig from weboob.deprecated.browser import BrowserForbidden from weboob.tools.newsfeed import Newsfeed from weboob.tools.value import Value, ValueBool, ValueBackendPassword from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message, Thread, CantSendMessage from weboob.capabilities.content import CapContent, Content from .browser import DLFP from .tools import rssid, id2url __all__ = ['DLFPModule'] class DLFPModule(Module, CapMessages, CapMessagesPost, CapContent): NAME = 'dlfp' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = "Da Linux French Page news website" CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), ValueBool('get_news', label='Get newspapers', default=True), ValueBool('get_diaries', label='Get diaries', default=False), ValueBool('get_polls', label='Get polls', default=False), ValueBool('get_board', label='Get board', default=False), ValueBool('get_wiki', label='Get wiki', default=False), ValueBool('get_tracker', label='Get tracker', default=False)) STORAGE = {'seen': {}} BROWSER = DLFP FEEDS = {'get_news': "https://linuxfr.org/news.atom", 'get_diaries': "https://linuxfr.org/journaux.atom", 'get_polls': "https://linuxfr.org/sondages.atom", 'get_board': "https://linuxfr.org/forums.atom", 'get_wiki': "https://linuxfr.org/wiki.atom", 'get_tracker': "https://linuxfr.org/suivi.atom", } def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(username, password) def deinit(self): # don't need to logout if the browser hasn't been used. if not self._browser: return with self.browser: self.browser.close_session() #### CapMessages ############################################## def iter_threads(self): whats = set() for param, url in self.FEEDS.iteritems(): if self.config[param].get(): whats.add(url) for what in whats: for article in Newsfeed(what, rssid).iter_entries(): if article.datetime and (datetime.now() - article.datetime) > timedelta(days=60): continue thread = Thread(article.id, article.link) thread.title = article.title thread._rsscomment = article.rsscomment if article.datetime: thread.date = article.datetime yield thread def get_thread(self, id, getseen=True): if not isinstance(id, Thread): thread = None else: thread = id id = thread.id if thread.date: self.storage.set('date', id, thread.date) self.storage.save() with self.browser: content = self.browser.get_content(id) if not content: return None if not thread: thread = Thread(content.id) flags = Message.IS_HTML if thread.id not in self.storage.get('seen', default={}): flags |= Message.IS_UNREAD thread.title = content.title if not thread.date: thread.date = content.date thread.root = Message(thread=thread, id='0', # root message url=self.browser.absurl(id2url(content.id)), title=content.title, sender=content.author or u'', receivers=None, date=thread.date, parent=None, content=content.body, signature='URL: %s' % self.browser.absurl(id2url(content.id)), children=[], flags=flags) for com in content.comments: self._insert_comment(com, thread.root, getseen) return thread def _insert_comment(self, com, parent, getseen=True): """" Insert 'com' comment and its children in the parent message. """ flags = Message.IS_HTML if com.id not in self.storage.get('seen', parent.thread.id, 'comments', default=[]): flags |= Message.IS_UNREAD if getseen or flags & Message.IS_UNREAD: com.parse() message = Message(thread=parent.thread, id=com.id, url=com.url, title=com.title, sender=com.author or u'', receivers=None, date=com.date, parent=parent, content=com.body, signature=com.signature + '
'.join(['Score: %d' % com.score, 'URL: %s' % com.url]), children=[], flags=flags) else: message = Message(thread=parent.thread, id=com.id, children=[], parent=parent, flags=flags) parent.children.append(message) for sub in com.comments: self._insert_comment(sub, message, getseen) def iter_unread_messages(self): for thread in self.iter_threads(): # Check if we have seen all comments of this thread. with self.browser: oldhash = self.storage.get('hash', thread.id, default="") newhash = self.browser.get_hash(thread._rsscomment) if oldhash != newhash: self.storage.set('hash', thread.id, newhash) self.storage.save() self.fill_thread(thread, 'root', getseen=False) for m in thread.iter_all_messages(): if m.flags & m.IS_UNREAD: yield m def set_message_read(self, message): self.storage.set('seen', message.thread.id, 'comments', self.storage.get('seen', message.thread.id, 'comments', default=[]) + [message.id]) self.storage.save() lastpurge = self.storage.get('lastpurge', default=0) # 86400 = one day if time.time() - lastpurge > 86400: self.storage.set('lastpurge', time.time()) self.storage.save() # we can't directly delete without a "RuntimeError: dictionary changed size during iteration" todelete = [] for id in self.storage.get('seen', default={}): date = self.storage.get('date', id, default=0) # if no date available, create a new one (compatibility with "old" storage) if date == 0: self.storage.set('date', id, datetime.now()) elif datetime.now() - date > timedelta(days=60): todelete.append(id) for id in todelete: self.storage.delete('hash', id) self.storage.delete('date', id) self.storage.delete('seen', id) self.storage.save() def fill_thread(self, thread, fields, getseen=True): return self.get_thread(thread, getseen) #### CapMessagesReply ######################################### def post_message(self, message): if not self.browser.username: raise BrowserForbidden() if not message.parent: raise CantSendMessage('Posting news and diaries on DLFP is not supported yet') assert message.thread with self.browser: return self.browser.post_comment(message.thread.id, message.parent.id, message.title, message.content) #### CapContent ############################################### def get_content(self, _id, revision=None): if isinstance(_id, basestring): content = Content(_id) else: content = _id _id = content.id if revision: raise NotImplementedError('Website does not provide access to older revisions sources.') with self.browser: data = self.browser.get_wiki_content(_id) if data is None: return None content.content = data return content def push_content(self, content, message=None, minor=False): if not self.browser.username: raise BrowserForbidden() with self.browser: return self.browser.set_wiki_content(content.id, content.content, message) def get_content_preview(self, content): with self.browser: return self.browser.get_wiki_preview(content.id, content.content) OBJECTS = {Thread: fill_thread} weboob-1.2/modules/dlfp/pages/000077500000000000000000000000001303450110500163155ustar00rootroot00000000000000weboob-1.2/modules/dlfp/pages/__init__.py000066400000000000000000000000001303450110500204140ustar00rootroot00000000000000weboob-1.2/modules/dlfp/pages/board.py000066400000000000000000000040731303450110500177620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from logging import warning from weboob.deprecated.browser import Page class Message(object): TIMESTAMP_REGEXP = re.compile(r'(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})') def __init__(self, id, timestamp, login, message, is_me): self.id = id self.timestamp = timestamp self.login = login self.message = message self.is_me = is_me self.norloge = timestamp m = self.TIMESTAMP_REGEXP.match(timestamp) if m: self.norloge = '%02d:%02d:%02d' % (int(m.group(4)), int(m.group(5)), int(m.group(6))) else: warning('Unable to parse timestamp "%s"' % timestamp) class BoardIndexPage(Page): def is_logged(self): return True def get_messages(self, last=None): msgs = [] for post in self.parser.select(self.document.getroot(), 'post'): m = Message(int(post.attrib['id']), post.attrib['time'], post.find('login').text, post.find('message').text, post.find('login').text.lower() == self.browser.username.lower()) if last is not None and last == m.id: break msgs.append(m) return msgs weboob-1.2/modules/dlfp/pages/index.py000066400000000000000000000025011303450110500177740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page class DLFPPage(Page): def is_logged(self): for form in self.document.getiterator('form'): if form.attrib.get('id', None) == 'new_account_sidebar': return False return True class IndexPage(DLFPPage): def get_login_token(self): form = self.parser.select(self.document.getroot(), 'form#new_account_sidebar', 1) for i in form.find('div').getiterator('input'): if i.attrib['name'] == 'authenticity_token': return i.attrib['value'] class LoginPage(DLFPPage): pass weboob-1.2/modules/dlfp/pages/news.py000066400000000000000000000170061303450110500176470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.deprecated.browser import BrokenPageError from weboob.tools.date import local2utc from ..tools import url2id from .index import DLFPPage class RSSComment(DLFPPage): def on_loaded(self): pass class Content(object): TAGGABLE = False def __init__(self, browser): self.browser = browser self.url = u'' self.id = u'' self.title = u'' self.author = u'' self.username = u'' self.body = u'' self.date = None self.score = 0 self.comments = [] self.relevance_url = None self.relevance_token = None def is_taggable(self): return False class Comment(Content): def __init__(self, article, div, reply_id): Content.__init__(self, article.browser) self.reply_id = reply_id self.signature = u'' self.preurl = article.url self.div = div self.id = div.attrib['id'].split('-')[1] subs = div.find('ul') if subs is not None: for sub in subs.findall('li'): comment = Comment(article, sub, self.id) self.comments.append(comment) def parse(self): self.url = '%s#%s' % (self.preurl, self.div.attrib['id']) self.title = unicode(self.browser.parser.select(self.div.find('h2'), 'a.title', 1).text) try: a = self.browser.parser.select(self.div.find('p'), 'a[rel=author]', 1) except BrokenPageError: self.author = 'Anonyme' self.username = None else: self.author = unicode(a.text) self.username = unicode(a.attrib['href'].split('/')[2]) self.date = datetime.strptime(self.browser.parser.select(self.div.find('p'), 'time', 1).attrib['datetime'].split('+')[0], '%Y-%m-%dT%H:%M:%S') self.date = local2utc(self.date) content = self.div.find('div') try: signature = self.browser.parser.select(content, 'p.signature', 1) except BrokenPageError: # No signature. pass else: content.remove(signature) self.signature = self.browser.parser.tostring(signature) self.body = self.browser.parser.tostring(content) self.score = int(self.browser.parser.select(self.div.find('p'), 'span.score', 1).text) forms = self.browser.parser.select(self.div.find('footer'), 'form.button_to') if len(forms) > 0: self.relevance_url = forms[0].attrib['action'].rstrip('for').rstrip('against') self.relevance_token = self.browser.parser.select(forms[0], 'input[name=authenticity_token]', 1).attrib['value'] def iter_all_comments(self): for comment in self.comments: yield comment for c in comment.iter_all_comments(): yield c def __repr__(self): return "" % (self.id, self.author, self.title) class Article(Content): TAGGABLE = True def __init__(self, browser, url, tree): Content.__init__(self, browser) self.url = url self.id = url2id(self.url) if tree is None: return header = tree.find('header') self.title = u' — '.join([a.text for a in header.find('h1').xpath('.//a')]) try: a = self.browser.parser.select(header, 'a[rel=author]', 1) except BrokenPageError: self.author = 'Anonyme' self.username = None else: self.author = unicode(a.text) self.username = unicode(a.attrib['href'].split('/')[2]) self.body = self.browser.parser.tostring(self.browser.parser.select(tree, 'div.content', 1)) try: self.date = datetime.strptime(self.browser.parser.select(header, 'time', 1).attrib['datetime'].split('+')[0], '%Y-%m-%dT%H:%M:%S') self.date = local2utc(self.date) except BrokenPageError: pass for form in self.browser.parser.select(tree.find('footer'), 'form.button_to'): if form.attrib['action'].endswith('/for'): self.relevance_url = form.attrib['action'].rstrip('for').rstrip('against') self.relevance_token = self.browser.parser.select(form, 'input[name=authenticity_token]', 1).attrib['value'] self.score = int(self.browser.parser.select(tree, 'div.figures figure.score', 1).text) def append_comment(self, comment): self.comments.append(comment) def iter_all_comments(self): for comment in self.comments: yield comment for c in comment.iter_all_comments(): yield c class CommentPage(DLFPPage): def get_comment(self): article = Article(self.browser, self.url, None) return Comment(article, self.parser.select(self.document.getroot(), 'li.comment', 1), 0) class ContentPage(DLFPPage): def on_loaded(self): self.article = None def is_taggable(self): return True def get_comment(self, id): article = Article(self.browser, self.url, None) try: li = self.parser.select(self.document.getroot(), 'li#comment-%s' % id, 1) except BrokenPageError: return None else: return Comment(article, li, 0) def get_article(self): if not self.article: self.article = Article(self.browser, self.url, self.parser.select(self.document.getroot(), 'main#contents article', 1)) try: threads = self.parser.select(self.document.getroot(), 'ul.threads', 1) except BrokenPageError: pass # no comments else: for comment in threads.findall('li'): self.article.append_comment(Comment(self.article, comment, 0)) return self.article def get_post_comment_url(self): return self.parser.select(self.document.getroot(), 'p#send-comment', 1).find('a').attrib['href'] def get_tag_url(self): return self.parser.select(self.document.getroot(), 'div.tag_in_place', 1).find('a').attrib['href'] class NewCommentPage(DLFPPage): pass class NewTagPage(DLFPPage): def _is_tag_form(self, form): return form.action.endswith('/tags') def tag(self, tag): self.browser.select_form(predicate=self._is_tag_form) self.browser['tags'] = tag self.browser.submit() class NodePage(DLFPPage): def get_errors(self): try: div = self.parser.select(self.document.getroot(), 'div.errors', 1) except BrokenPageError: return [] l = [] for li in div.find('ul').findall('li'): l.append(li.text) return l weboob-1.2/modules/dlfp/pages/wiki.py000066400000000000000000000040511303450110500176320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import BrokenPageError from .index import DLFPPage class WikiEditPage(DLFPPage): def get_body(self): try: return self.parser.select(self.document.getroot(), 'textarea#wiki_page_wiki_body', 1).text except BrokenPageError: return '' def _is_wiki_form(self, form): return form.attrs.get('class', '') in ('new_wiki_page', 'edit_wiki_page') def post_content(self, title, body, message): self.browser.select_form(predicate=self._is_wiki_form) self.browser.set_all_readonly(False) if title is not None: self.browser['wiki_page[title]'] = title.encode('utf-8') self.browser['commit'] = 'Créer' else: self.browser['commit'] = 'Mettre à jour' self.browser['wiki_page[wiki_body]'] = body.encode('utf-8') if message is not None: self.browser['wiki_page[message]'] = message.encode('utf-8') self.browser.submit() def post_preview(self, body): self.browser.select_form(predicate=self._is_wiki_form) self.browser['wiki_page[wiki_body]'] = body self.browser.submit() def get_preview_html(self): body = self.parser.select(self.document.getroot(), 'article.wikipage div.content', 1) return self.parser.tostring(body) weboob-1.2/modules/dlfp/test.py000066400000000000000000000036101303450110500165470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.tools.test import BackendTest, skip_without_config from .browser import DLFP class DLFPTest(BackendTest): MODULE = 'dlfp' def __init__(self, *args, **kwargs): DLFP.DOMAIN = 'alpha.linuxfr.org' BackendTest.__init__(self, *args, **kwargs) def test_new_messages(self): feeds = {} for name, feed in self.backend.FEEDS.iteritems(): feeds[name] = feed.replace('//linuxfr.org', '//alpha.linuxfr.org') self.backend.FEEDS = feeds for message in self.backend.iter_unread_messages(): pass @skip_without_config("username") def test_get_content(self): self.backend.get_content(u"Ceci-est-un-test") @skip_without_config("username") def test_push_content(self): content = self.backend.get_content(u"Ceci-est-un-test") content.content = "test " + str(datetime.now()) self.backend.push_content(content, message="test weboob", minor=True) @skip_without_config("username") def test_content_preview(self): content = self.backend.get_content(u"Ceci-est-un-test") self.backend.get_content_preview(content) weboob-1.2/modules/dlfp/tools.py000066400000000000000000000043271303450110500167360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re RSSID_RE = re.compile('tag:.*:(\w+)/(\d+)') ID2URL_RE = re.compile('^(\w)(.*)\.([^ \.]+)$') REGEXPS = {'/users/%s/journaux/%s': 'D%s.%s', '/news/%s': 'N.%s', '/wiki/%s': 'W.%s', '/suivi/%s': 'T.%s', '/sondages/%s': 'P.%s', '/forums/%s/posts/%s': 'B%s.%s', } def f2re(f): return '.*' + f.replace('%s', '([^ /]+)') def rssid(entry): m = RSSID_RE.match(entry.id) if not m: return None ind = m.group(1).replace('Post', 'Board')[0] for url_re, id_re in REGEXPS.iteritems(): if id_re[0] != ind: continue if id_re.count('%s') == 2: mm = re.match(f2re(url_re), entry.link) if not mm: return return '%s%s.%s' % (ind, mm.group(1), m.group(2)) else: return '%s.%s' % (ind, m.group(2)) def id2url(id): m = ID2URL_RE.match(id) if not m: return None for url_re, id_re in REGEXPS.iteritems(): if id_re[0] != m.group(1): continue if id_re.count('%s') == 2: return url_re % (m.group(2), m.group(3)) else: return url_re % m.group(3) def url2id(url): for url_re, id_re in REGEXPS.iteritems(): m = re.match(f2re(url_re), url) if not m: continue return id_re % m.groups() def id2threadid(id): m = ID2URL_RE.match(id) if m: return m.group(3) weboob-1.2/modules/dpd/000077500000000000000000000000001303450110500150405ustar00rootroot00000000000000weboob-1.2/modules/dpd/__init__.py000066400000000000000000000014311303450110500171500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import DPDModule __all__ = ['DPDModule'] weboob-1.2/modules/dpd/browser.py000066400000000000000000000021031303450110500170710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import SearchPage class DPDBrowser(PagesBrowser): BASEURL = 'https://tracking.dpd.de/' search_page = URL('/cgi-bin/simpleTracking.cgi\?parcelNr=(?P.+)&locale=en_D2&type=1', SearchPage) def get_tracking_info(self, _id): return self.search_page.go(id=_id).get_info(_id) weboob-1.2/modules/dpd/favicon.png000066400000000000000000000007741303450110500172030ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME"?@tEXtCommentCreated with GIMPWvIDATh F|C,pά&Yx#*4@G'k:M] naFxn7izk5z&@4$ce묱'9#B;Sr \ȏF{xB4Q5䁞k'2FҰGQ ib:,6`ZN;<\[ځW#~N~ dU\]\ 6ghO=78^h1(CQFΎ{68x% Nz *ipdICpzz{< r H{RgUIENDB`weboob-1.2/modules/dpd/module.py000066400000000000000000000026021303450110500166770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.parcel import CapParcel from .browser import DPDBrowser __all__ = ['DPDModule'] class DPDModule(Module, CapParcel): NAME = 'dpd' DESCRIPTION = u'DPD website' MAINTAINER = u'Matthieu Weber' EMAIL = 'mweber+weboob@free.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = DPDBrowser def get_parcel_tracking(self, id): """ Get information abouut a parcel. :param id: ID of the parcel :type id: :class:`str` :rtype: :class:`Parcel` :raises: :class:`ParcelNotFound` """ return self.browser.get_tracking_info(id) weboob-1.2/modules/dpd/pages.py000066400000000000000000000045461303450110500165220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from dateutil.parser import parse as parse_date from weboob.browser.pages import JsonPage from weboob.capabilities.parcel import Parcel, Event, ParcelNotFound STATUSES = { 1: Parcel.STATUS_PLANNED, 2: Parcel.STATUS_IN_TRANSIT, 3: Parcel.STATUS_IN_TRANSIT, 4: Parcel.STATUS_IN_TRANSIT, 5: Parcel.STATUS_ARRIVED, } class SearchPage(JsonPage): def build_doc(self, text): from weboob.tools.json import json return json.loads(text[1:-1]) def get_info(self, _id): result_id = self.doc.get("TrackingStatusJSON", {}).get("shipmentInfo", {}).get("parcelNumber", None) if not result_id: raise ParcelNotFound("No such ID: %s" % _id) if not _id.startswith(result_id): raise ParcelNotFound("ID mismatch: expecting %s, got %s" % (_id, result_id)) p = Parcel(_id) events = self.doc.get("TrackingStatusJSON", {}).get("statusInfos", []) p.history = [self.build_event(i, data) for i, data in enumerate(events)] p.status = self.guess_status( self.doc.get("TrackingStatusJSON", {}). get("shipmentInfo", {}). get("deliveryStatus")) p.info = p.history[-1].activity return p def guess_status(self, status_code): return STATUSES.get(status_code, Parcel.STATUS_UNKNOWN) def build_event(self, index, data): event = Event(index) date = "%s %s" % (data["date"], data["time"]) event.date = parse_date(date, dayfirst=False) event.location = unicode(data["city"]) event.activity = unicode(", ".join([_["label"] for _ in data["contents"]])) return event weboob-1.2/modules/dpd/test.py000066400000000000000000000017361303450110500164000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.parcel import ParcelNotFound class DPDTest(BackendTest): MODULE = 'dpd' def test_dpd(self): self.assertRaises(ParcelNotFound, self.backend.get_parcel_tracking, "foo") weboob-1.2/modules/dresdenwetter/000077500000000000000000000000001303450110500171505ustar00rootroot00000000000000weboob-1.2/modules/dresdenwetter/__init__.py000066400000000000000000000014731303450110500212660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import DresdenWetterModule __all__ = ['DresdenWetterModule'] weboob-1.2/modules/dresdenwetter/browser.py000066400000000000000000000021121303450110500212010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import StartPage __all__ = ['DresdenWetterBrowser'] class DresdenWetterBrowser(PagesBrowser): BASEURL = 'http://www.dresden-wetter.de' home = URL('/Current_Vantage_Pro.htm', StartPage) def get_sensors_list(self): return self.home.stay_or_go().get_sensors_list() weboob-1.2/modules/dresdenwetter/favicon.png000066400000000000000000000016721303450110500213110ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIMEtEXtCommentCreated with GIMPW"IDATx[K0GȞP!oHJ{T,{*.lpJ0ƎǞGv-1}#R{vCODQH O~x976=/MǵN8%yYA&%PxNn>?.Sp%@B7V,CĹ?gɞ*:jY &@dڥSO#P@prz\ΡO9\<L}kӜ@ 5Ҹ0 Q-KU~kVXPJ5H[zߟqze܀b$ZdL$A]orr-WVY$ ZW 'aq[$HOYY`q8y_ +PK1ӯ`pʹ#[ɧtד({mZ''eq`qWʯɂKWyAu Xr E\:]:znD`ւxyf/㽺҂Z㿺+6Aw/Xz (Ɩ7oTp. from .browser import DresdenWetterBrowser from weboob.capabilities.gauge import CapGauge, GaugeSensor, Gauge,\ SensorNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module __all__ = ['DresdenWetterModule'] class DresdenWetterModule(Module, CapGauge): NAME = 'dresdenwetter' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"Private wetter station Dresden" BROWSER = DresdenWetterBrowser def iter_gauges(self, pattern=None): if pattern is None or pattern.lower() in u"dresden"\ or pattern.lower() in "weather": gauge = Gauge("wetter") gauge.name = u"Private Wetterstation Dresden" gauge.city = u"Dresden" gauge.object = u"Weather" gauge.sensors = list(self.browser.get_sensors_list()) yield gauge def _get_sensor_by_id(self, id): for gauge in self.iter_gauges(): for sensor in gauge.sensors: if id == sensor.id: return sensor raise SensorNotFound() def iter_sensors(self, gauge, pattern=None): if not isinstance(gauge, Gauge): gauge = find_object(self.iter_gauges(), id=gauge, error=SensorNotFound) if pattern is None: for sensor in gauge.sensors: yield sensor else: lowpattern = pattern.lower() for sensor in gauge.sensors: if lowpattern in sensor.name.lower(): yield sensor # Not in the website def iter_gauge_history(self, sensor): raise NotImplementedError() def get_last_measure(self, sensor): if not isinstance(sensor, GaugeSensor): sensor = self._get_sensor_by_id(sensor) return sensor.lastvalue weboob-1.2/modules/dresdenwetter/pages.py000066400000000000000000000045001303450110500206200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Regexp, Field, Filter, debug from weboob.capabilities.gauge import GaugeMeasure, GaugeSensor from weboob.capabilities.base import NotAvailable class Split(Filter): def __init__(self, selector, mode): super(Split, self).__init__(selector) self.mode = mode @debug() def filter(self, txt): if u"Temperatur" in txt: value = txt.split(': ')[1].split(u'°')[0] unit = u'°C' else: value = txt.split(':')[-1].split()[0] unit = txt.split(':')[-1].split()[1] if unit == u"W/m": unit = u"W/m²" try: value = float(value) except ValueError: value = NotAvailable return [value, unit][self.mode] class StartPage(HTMLPage): @method class get_sensors_list(ListElement): item_xpath = '//p[@align="center"]' class item(ItemElement): klass = GaugeSensor obj_name = Regexp(CleanText('.'), '(.*?) {0,}: .*', "\\1") obj_id = CleanText(Regexp(Field('name'), '(.*)', "dd-\\1"), " .():") obj_gaugeid = u"wetter" obj_forecast = NotAvailable obj_unit = Split(CleanText('.'), 1) def obj_lastvalue(self): lastvalue = GaugeMeasure() lastvalue.level = Split(CleanText('.'), 0)(self) lastvalue.alarm = NotAvailable return lastvalue weboob-1.2/modules/dresdenwetter/test.py000066400000000000000000000047371303450110500205140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest, skip_without_config class DresdenWetterTest(BackendTest): MODULE = 'dresdenwetter' @skip_without_config() def test_gauges_sensors(self): """ test if the gauge listing works. Only one gauge on the website, but we can test sensors after that """ l = list(self.backend.iter_gauges()) self.assertTrue(len(l) == 1, msg="Gauge not found") self.assertTrue(len(l[0].sensors) > 5, msg="Not enough sensors") @skip_without_config() def test_sensors_value(self): temperature = self.backend.get_last_measure("dd-Temperatur2m").level self.assertTrue(temperature > -50., msg="To cold") self.assertTrue(temperature < 50., msg="Temperature to high") self.assertTrue(self.backend.get_last_measure(u"dd-Wind10minØ").level >= 0) self.assertTrue(self.backend.get_last_measure("dd-RelLuftdruck").level > 800.) self.assertTrue(self.backend.get_last_measure("dd-RelLuftfeuchtigkeit").level >= 0.) self.assertTrue(self.backend.get_last_measure("dd-Niederschlagseit001Uhr").level >= 0.) self.assertTrue(self.backend.get_last_measure("dd-Globalstrahlung").level >= 0.) @skip_without_config() def test_temperature(self): """ test the first sensor return by module" """ temperature = list(self.backend.iter_sensors("wetter", "Temperatur")) assert temperature[0].name == u"Temperatur 2m" assert temperature[0].unit == u"°C" @skip_without_config() def test_globalstrahlung(self): """ Test the last sensor return by module" """ sensor = list(self.backend.iter_sensors("wetter", "Globalstrahlung")) assert sensor[0].unit == u"W/m²" weboob-1.2/modules/eatmanga/000077500000000000000000000000001303450110500160465ustar00rootroot00000000000000weboob-1.2/modules/eatmanga/__init__.py000066400000000000000000000014351303450110500201620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import EatmangaModule __all__ = ['EatmangaModule'] weboob-1.2/modules/eatmanga/favicon.png000066400000000000000000000174221303450110500202070ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B4tIME$UеiTXtCommentCreated with GIMPd.eiIDATx{ieq^Urufޛ I$JlY ;( 1d38Al #?āرHdk,7qyozsNwWU~77CR iq9v_WUWʯ|/#ԅ7d}No,C" ̩jk6TcOVVm Aeom鷖~ԸlVNGdhAEUTD$X͍W͟y)8)+:DѺSj*w9'S'c:_BP bt:jon^_-!* "Xgdt`C]{հgw绋 k+˃x;W:`Ѩ4o̻\)5 _0>݃V3xo\s`-@sT\ګtf:tnZ|M<Bt|vsv1X\ ?mvai2-ӏUT!;`TŨ*|*BWk~q_xG`yGSZ*@I7-?L*0 }kn^zx(ZIU#ZEa;hgz `l V3;X*# u3-۽l-;Q|PP̻{ OKKK30' "%._~kg.#u=몪HH*¤BDW_~@8-V!5ZEAXW *Z't\3"$v{[ۯ8&έ u>.|ٛx2 (2(NgQG^o_XOgBl!aE/|篾 eىN\Z'h|3Al6n\}<|)AU+\ *UyҶz'N7R ʌn&WN8S18@)6qlYq㗠UYmo_s߈jSY3@+W_ C3[ cɵ_9Ye9]@FÉ /?SMz >kYՒA<`~ܗ֘mjnW}iSG걟Mgkr†,[TYW ju*$p=PD{vKdyg;㧲Νh{Ȗ*C~p\x=g] bbl\3vp ZjJkǮәz^}UZ]\EZ nl]yhpɃn1hz&Y*}UAaTxrGUTC!YЉ'~hpykBڗ^~׮^@C3v8DUےB 2*UإHrjpRNAZF&vVPҖ;G)c~CNU@JTn&PDݬ ID&qM]en @goYicR2-NU$ HJE-sW}/[+/_ybow7};xѻ46|g#8~ 9sRBS㔒c >*sݩز=ȼHl`UO fR?}/tU@>1|}FBh1Rl'z#OM~w+Yd؞_/maoV-'n'X\Mv.x_3o#C4O̚&.[0y?_~kو{>v""L>!k t ~"Fo_889dcءmL~im< UuN3B~SsZ.0 UӶjrE^ NH]SwW~h[@ &f JhYu>Ȫr`~<#“< ӚXs6!yuTcf Ha*zGDMS'2?_S'M`S=  }ppxbZ$1/ }; fK\M,1Y0D $W-\A9)%*<*^Ϳ=G*ε &`QP@$1) &9"U[0*3$F-Pؼh9NE)39/LoN-Io|K2HQ3@P(,"$8a;+)+ZX: hmp(34V dsoVUXc Ј%VX0S]X qʮ䔌 d 9J[򄻦Nqoy%G& vt`Y%%H$;A0˻.xM2LƤci ؈ߐM~Uҭ){-N 7Pv3h>j0VP(UY|ht?ܤǤY9\wAA@eL2s p H^lֆnq?w#a4hYJ+vo>K..Xm@H $WoY5OdcD(ɳDsTT@UEgG[vou?C4./LEߝYk tMp ϭv#BJpmj+DK -R"bտ>)~{vUU $J@pAPZSPU*NoD"ŭ˜T1Mޥ߈xFμ6s"7Η[H(?^zi䗞:w|lՇo>Ov@ B) iȤyY#uh*̠KK._~)ƈ|}}meu4Ew*HK%GO97"`#u'ZiJKb0Q?~~dU޽($ I eSYz XS5զ$aVIU۩fh3ﶺy6̭YNFM?NFn 0 duDN/|2)"#.BjFpUEP!7nf5KO)PnI%IJ@L 16l0 ; 8roO^t)ec'㽣{9oe|0YmC@%"8_̍_UUd^GMmBq3L21LzK20Q 4dժVݙAS1K'*:QTM:fv{=_F[$sk2{3Nq@.\t5u~q* " 9!i %ٌN*h85a(!Nk-V(T%j!saVIt(*qo8zI8r{(%"M;Wd' &iwlpQU`I⬬B,"v;o7cfDRpӌ9R=G;w.c}ڨJ"i%:xNF979qr4sMdlY3p>sdZ/ÉEP;26r 2K2M*ǘPa#Y29­U1Ntu3k a h^ 0Ww@25E+q\4;8ޙ{ݳ"-f&tjS+ *+m٩sl#:VQg (Zа-5NY-!UdD8foWt,BbSo`ttyswR`;vy7#JSd,|Nʧh!EO"* (0* X&Qj@u*!4drs:S$?žKH/=#Gh )4Q,vݬ߫lH6h|_.,<~ C)<i Wk-jRn&u886GV.f`mSERhv'M YLLX %c(!STaZ쏶{u \r/{db#e&`ekN*.qG0$,p7px"M~ͣIΖ-P4ȴ|tH5!i0PAfgX99#|Ώ@˛seCi3ϲwCi498À`0*Z!$ERQ$ vV#*T{ 7ˌA }{.nD"IPU怕4BՀ*QY*$:Ng͍pO^o>Zﳾ =uw&R 486FC&1 4 چ,5hLCIbo]R_ߤ;?o^ЂLECACV5S- Dֲ&q^]&o6o74g_ og@,(J*Ivo#NMBHHElm]6.x_ϦgҟuKl!t u{C1D]yC-Uga+ oIENDB`weboob-1.2/modules/eatmanga/module.py000066400000000000000000000025671303450110500177170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['EatmangaModule'] class EatmangaModule(GenericComicReaderModule): NAME = 'eatmanga' DESCRIPTION = 'EatManga manga reading website' DOMAIN = 'www.eatmanga.com' BROWSER_PARAMS = dict( img_src_xpath="//img[@class='eatmanga_bigimage']/@src", page_list_xpath="(//select[@id='pages'])[1]/option/@value") ID_REGEXP = r'[^/]+/[^/]+' URL_REGEXP = r'.+eatmanga.com/(?:index.php/)?Manga-Scan/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.eatmanga.com/index.php/Manga-Scan/%s' PAGES = {URL_REGEXP: DisplayPage} weboob-1.2/modules/eatmanga/test.py000066400000000000000000000017401303450110500174010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class EatmangaTest(GenericComicReaderTest): MODULE = 'eatmanga' def test_download(self): return self._test_download('Glass-Mask/Glass-Mask-Vol-031') weboob-1.2/modules/ebonics/000077500000000000000000000000001303450110500157135ustar00rootroot00000000000000weboob-1.2/modules/ebonics/__init__.py000066400000000000000000000014331303450110500200250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import EbonicsModule __all__ = ['EbonicsModule'] weboob-1.2/modules/ebonics/favicon.png000066400000000000000000000145001303450110500200460ustar00rootroot00000000000000PNG  IHDR@@iqbKGD___4ƍ pHYs  tIME  &CtEXtCommentCreated with GIMPWIDATx͛{eYU?k0 %8`DIDT0E$*c$$!FI"&)IYQ1!Vd"R@AyȠC$#3 3 L?~{9gV8i;MT}{z}.U^ٜ 'T@Hy |_Y)v)EDp. JKuasp#h<98D6[Lbx'Zܜ[B*pcc܏䩒q,V rwrWo[5jh#\.~VkKk-TړUO»މ Gp\PO4."',ƛ}>:,x1gou/]\W$CRCu'xšW>~gfo01p[ -۩Rƾg>Y/r>{>@C(J5VTO%|&S-a8oj㛟XYi*/Bc%/c灵="?o2u D(y@YY\[%(;޳Awi- e`ͺ;& }˹ kV2JxŭʛйlO `?BJhԩ!6koId]}߳îDsk|ӷ [) bb3E DA/R2m݁CAqr9WNT̐TOrj1MKR|K@J(b*PEpw# &8D>[U(A҂OgbZ"aUrWv: Qq(aMA*YAr&2yфBm!a;>z3jg AIyC,k3q0M;7jFFS`0VLQbKDXOLc({~&-.b)PK6-*FuCO|b qpȔ ZJFpN@[VeðS$Jĉ2f <<97BlYږ\F ڀ0^Ɔ`+kW3^q/u&h"eXҊ1nlAK #"tbgMB<#T ׉Q'Ak 0BFڠ1?DrY;#GKbD'`8tl&]p/DQi)e5^{ D $:qfy)! H MOo4-L8[ )&(iP)%c*{H'8BXƝ8/hGHHB "zxXO0a4&hB'وUָf,S=2]kRIdIj&}61ٳMSoPIlAtd+R hN%邺~f d  Zىo=;3Nu04!0;`.n< G{eLH4Xi7'P ,Sq\ E`2PN2mh3kZSb f#I# bM8l9֏/(L# 1TZpYX2MKD$Ub!)SpxX/WePfv*e,m6, e`` BL<R2qF{r]ւu5T0"FBFK] 2VP7Q"DfI-`> Хa qBbDIxaaGaI#W h;@ Bh c[Hf'i@[BEbL}ǙiT5ʓ2 sej"e 9x|Hn&#QSD"Q45!LN*ϩUmư5^7<%`h64,:%;}v HCPT=0F#}w>ml ,`4L'$绿$GÔ&0V\d1Ḏ"B]ci}5lk &F1%. gtNyύ\&Nc`[d9DFtxOo|}!E!@+"kٌKCe ݇tĨd:lQ:B| 1/ٷ.9,pDϩ#+GX,ϱZL-:jky)DjX j/{7pd"9C>{:<%D`z] \cW $ď9#|=sHhJZ=\q"a $js g {żHc>qd_NR)j9vj\A2y"/E9f98Q Xufй4'C(.6Eh#̌{NQ\=Gjh;nWT.owT 18L-CP<+mf¼JL@šT!PS̸sNmIcS:M|$J T:RcĽ#4 =!r9A1Yu3oq҄-ԝ>`@#Tqi'&1Uq/fA gPY6iUfe_M*=}B*|g mGqXd8'GYn9 V vH} Gc%Z3֔ځk.e".ZI!3O?m(iy ݲB`f'4Fzr@YF<0jGti7}h执"zz:B値 D%pYJSc}MbI6`Z)=#ӐqO ,pzXd!\Q,CVTpsvÖ&B YbVv׾DƁMQ(Z`Lxh.#' qM5G44Tl&7&&r,{AD+4cgEFGq.zR_>Nl[ qpqrq[* %$Dn(&@RhS t2'VT`S&JZZ:B nJf3]!\q"1ږBKnU0Zs?fp,kM$_ 8̬S+)^}0 2RpnC،h0p,BױEDC~qpj ,J^6Ea#K'ljTu|r48t,@nV3l3!KY$9CLXYق9 A89N%x?xM 'z2:hBDžݑ:}>IXo!5fw2 PWVbO-?jP9=*[Rz*BM\o N A+. H= G>aD;PUUؐp6"YJ1:w)ac&T%x;ttdYfuV(= Z( 'ɚxۛ =i|vTQ~r1qMHʯ8wa"B]'ORgolT VHbJ ^&ʦB1t%'lDi0G$tg.Eǃb~52|O {!v}Ρ@O3-l21_oJ&6 E1xIMY:OwgMx.\u|8.1ղ W<6t%ҽr$bowny㓹wnM~}f3-2͆@y军v_{J!87xys^kuw_,,1C- GCI{[ZO aD뫈O8VOtqGG'BO>O}S=|3/13n뮋hwvv0 'KUm[?uꔿ}c~ww'=okf!$֑ƵWGq$ΫO_4RuaiTwu{??ɉ'R{Nz衋cv]0 *}oö1RkEU9{,vGx_LbӒf62ǎww_UJ@r gŠTEm>{ߧLjTBIENDB`weboob-1.2/modules/ebonics/module.py000066400000000000000000000037661303450110500175660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from weboob.capabilities.translate import CapTranslate, Translation, TranslationFail, LanguageNotSupported from weboob.tools.backend import Module from weboob.deprecated.browser import StandardBrowser __all__ = ['EbonicsModule'] class EbonicsModule(Module, CapTranslate): NAME = 'ebonics' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'English to Ebonics translation service' BROWSER = StandardBrowser def translate(self, lan_from, lan_to, text): if lan_from != 'English' or lan_to != 'Nigger!': raise LanguageNotSupported() with self.browser: data = {'English': text.encode('utf-8')} doc = self.browser.location('http://joel.net/EBONICS/Translator', urllib.urlencode(data)) try: text = doc.getroot().cssselect('div.translateform div.bubble1 div.bubblemid')[0].text except IndexError: raise TranslationFail() if text is None: raise TranslationFail() translation = Translation(0) translation.lang_src = unicode(lan_from) translation.lang_dst = unicode(lan_to) translation.text = unicode(text).strip() return translation weboob-1.2/modules/ebonics/test.py000066400000000000000000000016441303450110500172510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class EbonicsTest(BackendTest): MODULE = 'ebonics' def test_translate(self): self.backend.translate('English', 'Nigger!', 'I like penis.') weboob-1.2/modules/edf/000077500000000000000000000000001303450110500150275ustar00rootroot00000000000000weboob-1.2/modules/edf/__init__.py000066400000000000000000000014321303450110500171400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import EdfModule __all__ = ['EdfModule'] weboob-1.2/modules/edf/favicon.png000066400000000000000000000040021303450110500171560ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME & x㭵iTXtCommentCreated with GIMPd.ebKGDCfIDATxݛmlELJR`x+*4TA4Z#S\MMBR|KC*V-I#XM?ИBĢ~(JPPю4w{gyYB)%|b#@mokIHt;`4f4#@? xa 0C՜Q^w844huP??D2B_Eh` %.[B }~ф hp؞{:"9I }Y,Зz4`LaN tL5U>p`X\8GhB].q=*XNO>80Jî_ (@#< "p=qD06}# x!fJ,uF\Qz%*b8(BB*iWt|}?4Us&X6P"W{ĖVe6 Z\_jP7eXּy?1@>T+Q}JDCQ5Fau$ <<<0!GήdqaoG?G>V`1[LrDHcA[xkw<NXe&byf9"p I'P-@ I͕XLPlVWf|"~%DуEy'`x^BWL5SПڀv"p8`0MN0(c:B3!S@=p8OQaV1ntB+g^ W]vK`xwKc]vn+ 1糴(_`œ,3CYO}6!^J Kiq}&b 8=<}v = ]=#c0۔)_=d٦o. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .par.browser import EdfBrowser from .pro.browser import EdfproBrowser __all__ = ['EdfModule'] class EdfModule(Module, CapDocument): NAME = 'edf' DESCRIPTION = u'EDF' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='E-mail ou Identifiant'), \ ValueBackendPassword('password', label='Mot de passe'), \ Value('website', label='Type de compte', default='par', choices={'par': 'Particulier', 'pro': 'Entreprise'})) def create_default_browser(self): browsers = {'pro': EdfproBrowser, 'par': EdfBrowser} self.BROWSER = browsers[self.config['website'].get()] return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) return self.browser.download_document(document) weboob-1.2/modules/edf/par/000077500000000000000000000000001303450110500156115ustar00rootroot00000000000000weboob-1.2/modules/edf/par/__init__.py000066400000000000000000000000001303450110500177100ustar00rootroot00000000000000weboob-1.2/modules/edf/par/browser.py000066400000000000000000000036511303450110500176530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, ProfilPage, DocumentsPage class EdfBrowser(LoginBrowser): BASEURL = 'https://particulier.edf.fr' login = URL('/bin/edf_rc/servlets/authentication', LoginPage) profil = URL('/services/rest/authenticate/getListContracts', ProfilPage) documents = URL('https://monagencepart.edf.fr/ASPFront/appmanager/ASPFront/front\?service=page_mes_factures&privee=true&accord=(?P\d+)', 'https://monagencepart.edf.fr', DocumentsPage) def do_login(self): self.location(self.BASEURL) data = {'login': self.username, 'password': self.password} self.login.go(data=data) if not self.page.is_logged(): raise BrowserIncorrectPassword @need_login def get_subscription_list(self): return self.profil.stay_or_go().get_list() @need_login def iter_documents(self, subscription): return self.documents.stay_or_go(subid=subscription.id).get_documents(subid=subscription.id) @need_login def download_document(self, document): return self.open(document.url).content weboob-1.2/modules/edf/par/pages.py000066400000000000000000000057651303450110500172770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.browser.pages import HTMLPage, LoggedPage, JsonPage, pagination from weboob.browser.filters.standard import CleanText, CleanDecimal, Env, Format, TableCell, Regexp, Date from weboob.browser.elements import ItemElement, DictElement, TableElement, method from weboob.browser.filters.html import Attr from weboob.browser.filters.json import Dict from weboob.capabilities.bill import Bill, Subscription from weboob.capabilities.base import NotAvailable class LoginPage(JsonPage): def is_logged(self): if "200" in Dict().filter(self.doc)['errorCode']: return True return False class ProfilPage(LoggedPage, JsonPage): @method class get_list(DictElement): item_xpath = 'customerAccordContracts' class item(ItemElement): klass = Subscription obj_subscriber = Format('%s %s', Dict('bp/identity/firstName'), Dict('bp/identity/lastName')) obj_id = Dict('number') obj_label = obj_id class DocumentsPage(LoggedPage, HTMLPage): @pagination @method class get_documents(TableElement): ignore_duplicate = True item_xpath = '//div[@class="factures"]//table/tbody/tr' head_xpath = '//div[@class="factures"]//table/thead/tr/th' col_date = u'Consulter ma facture détaillée' col_price = u'Montant (TTC)' def next_page(self): next_page = Attr('//ul[@class="years"]//span/../following-sibling::li/a', 'href')(self) if next_page: return next_page class item(ItemElement): klass = Bill obj_id = Format('%s_%s', Env('subid'), Env('docid')) obj_url = Env('url') obj_date = Date(Regexp(CleanText(TableCell('date')), ' ([\d\/]+)')) obj_format = u"pdf" obj_label = Format('Facture %s', Env('docid')) obj_type = u"bill" obj_price = CleanDecimal(TableCell('price'), replace_dots=True, default=NotAvailable) obj_currency = u"€" obj_vat = NotAvailable def parse(self, el): self.env['docid'] = Regexp(Attr('./td/a[@class="pdf"]', 'title'), '([\d]+)')(self) self.env['url'] = re.sub('[\t\r\n]', '', Attr('./td/a[@class="pdf"]', 'href')(self)) weboob-1.2/modules/edf/par/test.py000066400000000000000000000014751303450110500171510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class EdfTest(BackendTest): MODULE = 'edf' weboob-1.2/modules/edf/pro/000077500000000000000000000000001303450110500156275ustar00rootroot00000000000000weboob-1.2/modules/edf/pro/__init__.py000066400000000000000000000000001303450110500177260ustar00rootroot00000000000000weboob-1.2/modules/edf/pro/browser.py000066400000000000000000000061401303450110500176650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Jean Walrave # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import json from datetime import datetime from weboob.browser import LoginBrowser, URL, need_login from weboob.capabilities.base import NotAvailable from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.exceptions import ServerError from .pages import LoginPage, AuthPage, SubscriptionsPage, BillsPage, DocumentsPage class EdfproBrowser(LoginBrowser): BASEURL = 'https://www.edfentreprises.fr' login = URL('https://www.edf.fr/entreprises', LoginPage) auth = URL('/openam/UI/Login', '/ice/rest/aiguillagemp/redirect', AuthPage) contracts = URL('/rest/contratmp/detaillercontrat', SubscriptionsPage) bills = URL('/rest/facturemp/getnomtelechargerfacture', BillsPage) documents = URL('/rest/facturemp/recherchefacture', DocumentsPage) def do_login(self): self.login.go().login(self.username, self.password) self.location(self.absurl('/ice/rest/aiguillagemp/redirect'), allow_redirects=False) if self.auth.is_here() and self.page.response.status_code != 303: raise BrowserIncorrectPassword self.session.headers['Content-Type'] = 'application/json;charset=UTF-8' self.session.headers['X-XSRF-TOKEN'] = self.session.cookies['XSRF-TOKEN'] @need_login def get_subscription_list(self): return self.contracts.go(data=json.dumps({'listeContrat': [{'refDevis': ''}]})) \ .get_subscriptions() @need_login def iter_documents(self, subscription): return self.documents.go(data=json.dumps({'dateDebut': '01/01/2013', \ 'dateFin': datetime.now().strftime('%d/%m/%Y'), \ 'element': subscription.id, \ 'typeElementListe': 'CONTRAT'})) \ .get_documents(subscription.id) @need_login def download_document(self, document): if document.url is not NotAvailable: try: fname = self.bills.go(data=json.dumps({'date': int(document.date.strftime('%s')), \ 'iDFelix': document._account_billing, 'numFacture': document._bill_number})).doc return self.open('%s/rest/facturemp/telechargerfichier?fname=%s' % (self.BASEURL, fname)).content except ServerError: return NotAvailable weboob-1.2/modules/edf/pro/pages.py000066400000000000000000000046531303450110500173100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Jean Walrave # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from weboob.browser.pages import HTMLPage, JsonPage, RawPage, LoggedPage from weboob.browser.filters.standard import CleanDecimal from weboob.capabilities.bill import Subscription, Bill class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form(id='form-authenticate-entreprise') form['IDToken1'] = login form['IDToken2'] = password form.submit(allow_redirects=False) class AuthPage(RawPage): pass class SubscriptionsPage(LoggedPage, JsonPage): def get_subscriptions(self): subscriptions = [] for contract in self.doc['listeContrat']: sub = Subscription() sub.id = contract['refDevis'] sub.label = contract['nomOffreModele'] sub.subscriber = ('%s %s' % (contract['prenomIntPrinc'].lower(), contract['nomIntPrinc'].lower())).title() subscriptions.append(sub) return subscriptions class BillsPage(LoggedPage, RawPage): pass class DocumentsPage(LoggedPage, JsonPage): def get_documents(self, subid): documents = [] for document in self.doc: doc = Bill() doc.id = u'%s_%s' % (subid, document['numFactureLabel']) doc.date = date.fromtimestamp(int(document['dateEmission'] / 1000)) doc.format = u'PDF' doc.label = 'Facture %s' % document['numFactureLabel'] doc.type = u'bill' doc.price = CleanDecimal().filter(document['montantTTC']) doc.currency = u'€' doc._account_billing = document['compteFacturation'] doc._bill_number = document['numFacture'] documents.append(doc) return documents weboob-1.2/modules/ehentai/000077500000000000000000000000001303450110500157065ustar00rootroot00000000000000weboob-1.2/modules/ehentai/__init__.py000066400000000000000000000014401303450110500200160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import EHentaiModule __all__ = ['EHentaiModule'] weboob-1.2/modules/ehentai/browser.py000066400000000000000000000070271303450110500177510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from urllib import urlencode from .pages import IndexPage, GalleryPage, ImagePage, HomePage, LoginPage from .gallery import EHentaiImage __all__ = ['EHentaiBrowser'] class EHentaiBrowser(Browser): ENCODING = None PAGES = { r'http://[^/]+/': IndexPage, r'http://[^/]+/\?.*': IndexPage, r'http://[^/]+/g/.+': GalleryPage, r'http://[^/]+/s/.*': ImagePage, r'http://[^/]+/home\.php': HomePage, r'http://e-hentai\.org/bounce_login\.php': LoginPage, } def __init__(self, domain, username, password, *args, **kwargs): self.DOMAIN = domain self.logged = False Browser.__init__(self, parser=('lxmlsoup',), *args, **kwargs) if password: self.login(username, password) def _gallery_url(self, gallery): return 'http://%s/g/%s/' % (self.DOMAIN, gallery.id) def _gallery_page(self, gallery, n): return gallery.url + ('?p='+str(n)) def search_galleries(self, pattern): self.location(self.buildurl('/', f_search=pattern.encode('utf-8'))) assert self.is_on_page(IndexPage) return self.page.iter_galleries() def latest_gallery(self): self.location('/') assert self.is_on_page(IndexPage) return self.page.iter_galleries() def iter_gallery_images(self, gallery): self.location(gallery.url) assert self.is_on_page(GalleryPage) for n in self.page._page_numbers(): self.location(self._gallery_page(gallery, n)) assert self.is_on_page(GalleryPage) for img in self.page.image_pages(): yield EHentaiImage(img) def get_image_url(self, image): self.location(image.id) assert self.is_on_page(ImagePage) return self.page.get_url() def gallery_exists(self, gallery): gallery.url = self._gallery_url(gallery) self.location(gallery.url) assert self.is_on_page(GalleryPage) return self.page.gallery_exists(gallery) def fill_gallery(self, gallery, fields): gallery.url = self._gallery_url(gallery) self.location(gallery.url) assert self.is_on_page(GalleryPage) self.page.fill_gallery(gallery) def login(self, username, password): assert isinstance(username, basestring) assert isinstance(password, basestring) data = {'ipb_login_username': username, 'ipb_login_password': password} self.location('http://e-hentai.org/bounce_login.php', urlencode(data), no_login=True) assert self.is_on_page(LoginPage) if not self.page.is_logged(): raise BrowserIncorrectPassword() # necessary in order to reach the fjords self.home() weboob-1.2/modules/ehentai/favicon.png000066400000000000000000000016251303450110500200450ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME,!IȀtEXtCommentCreated with GIMPWIDATxMEߎ1FM#QEP0b.hD EPB=x3ы.AK,n!$ E\ F-Ԡqm:μ? M{]MիWUAAAA0nL)<+߶fy'[y}ށIhAB֥t =HP4;q`_K4>ƣWeu&jB!۔^ɶ+(j5A0 sك}$wmQV{ɋ)^# mvܚ.8S8SZIL4T78_F-H6Olc(EXAgw'VciwO_S :ŀiͣ0|qSrvڭ *{w.Ϙ;^i¡%twC oZ^`c^K_لuV \fG@QVg06|e!Fg/P{ C)Ni$:!R.~9|wF};-:X<>qpRn0Ā8 cM;d/'N6?;%]TONִs-^6]5ߵo^\#[~Skx=O1~8\u'}jv<AAAcߊIENDB`weboob-1.2/modules/ehentai/gallery.py000066400000000000000000000021131303450110500177140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.gallery import BaseGallery, BaseImage __all_ = ['EHentaiGallery', 'EHentaiImage'] class EHentaiGallery(BaseGallery): def __init__(self, *args, **kwargs): BaseGallery.__init__(self, *args, **kwargs) class EHentaiImage(BaseImage): def __init__(self, *args, **kwargs): BaseImage.__init__(self, *args, **kwargs) weboob-1.2/modules/ehentai/module.py000066400000000000000000000100241303450110500175420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.gallery import CapGallery, BaseGallery from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.misc import ratelimit from weboob.tools.value import Value, ValueBackendPassword from .browser import EHentaiBrowser from .gallery import EHentaiGallery, EHentaiImage __all__ = ['EHentaiModule'] class EHentaiModule(Module, CapGallery, CapCollection): NAME = 'ehentai' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = 'E-Hentai galleries' LICENSE = 'AGPLv3+' BROWSER = EHentaiBrowser CONFIG = BackendConfig( Value('domain', label='Domain', default='g.e-hentai.org'), Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password')) def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(self.config['domain'].get(), username, password) def search_galleries(self, pattern, sortby=None): with self.browser: return self.browser.search_galleries(pattern) def iter_gallery_images(self, gallery): self.fillobj(gallery, ('url',)) with self.browser: return self.browser.iter_gallery_images(gallery) ID_REGEXP = r'/?\d+/[\dabcdef]+/?' URL_REGEXP = r'.+/g/(%s)' % ID_REGEXP def get_gallery(self, _id): match = re.match(r'^%s$' % self.URL_REGEXP, _id) if match: _id = match.group(1) else: match = re.match(r'^%s$' % self.ID_REGEXP, _id) if match: _id = match.group(0) else: return None gallery = EHentaiGallery(_id) with self.browser: if self.browser.gallery_exists(gallery): return gallery else: return None def fill_gallery(self, gallery, fields): if not gallery.__iscomplete__(): with self.browser: self.browser.fill_gallery(gallery, fields) def fill_image(self, image, fields): with self.browser: image.url = self.browser.get_image_url(image) if 'data' in fields: ratelimit("ehentai_get", 2) image.data = self.browser.readurl(image.url) def iter_resources(self, objs, split_path): if BaseGallery in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest_nsfw']) if collection.split_path == [u'latest_nsfw']: for gallery in self.browser.latest_gallery(): yield gallery def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseGallery in objs and collection.split_path == [u'latest_nsfw']: collection.title = u'Latest E-Hentai galleries (NSFW)' return raise CollectionNotFound(collection.split_path) OBJECTS = { EHentaiGallery: fill_gallery, EHentaiImage: fill_image} weboob-1.2/modules/ehentai/pages.py000066400000000000000000000064751303450110500173730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page from weboob.capabilities.image import Thumbnail from datetime import datetime import re from .gallery import EHentaiGallery class LoginPage(Page): def is_logged(self): success_p = self.document.xpath( '//p[text() = "Login Successful. You will be returned momentarily."]') if len(success_p): return True else: return False class HomePage(Page): pass class IndexPage(Page): def iter_galleries(self): lines = self.document.xpath('//table[@class="itg"]//tr[@class="gtr0" or @class="gtr1"]') for line in lines: a = line.xpath('.//div[@class="it3"]/a')[-1] url = a.attrib["href"] title = a.text.strip() yield EHentaiGallery(re.search('(?<=/g/)\d+/[\dabcdef]+', url).group(0), title=title) class GalleryPage(Page): def image_pages(self): return self.document.xpath('//div[@class="gdtm"]//a/attribute::href') def _page_numbers(self): return [n for n in self.document.xpath("(//table[@class='ptt'])[1]//td/text()") if re.match(r"\d+", n)] def gallery_exists(self, gallery): if self.document.xpath("//h1"): return True else: return False def fill_gallery(self, gallery): gallery.title = self.document.xpath("//h1[@id='gn']/text()")[0] cardinality_string = self.document.xpath("//div[@id='gdd']//tr[td[@class='gdt1']/text()='Length:']/td[@class='gdt2']/text()")[0] gallery.cardinality = int(re.match(r"\d+", cardinality_string).group(0)) date_string = self.document.xpath("//div[@id='gdd']//tr[td[@class='gdt1']/text()='Posted:']/td[@class='gdt2']/text()")[0] gallery.date = datetime.strptime(date_string, "%Y-%m-%d %H:%M") rating_string = self.document.xpath("//td[@id='rating_label']/text()")[0] rating_match = re.search(r"\d+\.\d+", rating_string) if rating_match is None: gallery.rating = None else: gallery.rating = float(rating_match.group(0)) gallery.rating_max = 5 try: thumbnail_url = self.document.xpath("//div[@class='gdtm']/a/img/attribute::src")[0] except IndexError: thumbnail_style = self.document.xpath("//div[@class='gdtm']/div/attribute::style")[0] thumbnail_url = re.search(r"background:[^;]+url\((.+?)\)", thumbnail_style).group(1) gallery.thumbnail = Thumbnail(thumbnail_url) class ImagePage(Page): def get_url(self): return self.document.xpath('//div[@class="sni"]/a/img/attribute::src')[0] weboob-1.2/modules/ehentai/test.py000066400000000000000000000031531303450110500172410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.gallery import BaseGallery class EHentaiTest(BackendTest): MODULE = 'ehentai' def test_search(self): l = list(self.backend.search_galleries('lol')) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for gallery "%s" not found: %s' % (v.id, v.url)) self.backend.browser.openurl(v.url) img = self.backend.iter_gallery_images(v).next() self.backend.fillobj(img, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for first image in gallery "%s" not found: %s' % (v.id, img.url)) self.backend.browser.openurl(img.url) def test_latest(self): l = list(self.backend.iter_resources([BaseGallery], [u'latest_nsfw'])) assert len(l) > 0 weboob-1.2/modules/entreparticuliers/000077500000000000000000000000001303450110500200355ustar00rootroot00000000000000weboob-1.2/modules/entreparticuliers/__init__.py000066400000000000000000000014601303450110500221470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import EntreparticuliersModule __all__ = ['EntreparticuliersModule'] weboob-1.2/modules/entreparticuliers/browser.py000066400000000000000000000126311303450110500220750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.json import json from weboob.capabilities.housing import Query, TypeNotSupported from weboob.browser import PagesBrowser, URL from .pages import CitiesPage, SearchPage, HousingPage class EntreparticuliersBrowser(PagesBrowser): BASEURL = 'http://www.entreparticuliers.com' cities = URL('/HTTPHandlers/LocalisationsAutocompleteHandler.ashx\?q=(?P.*)', CitiesPage) form_item = URL('/Default.aspx/GetElementsMoteur') search = URL('/default.aspx/CreateSearchParams') search_result = URL('/default.aspx/GetAnnonces', SearchPage) housing = URL('/default.aspx/GetAnnonceDetail', HousingPage) def search_city(self, pattern): return self.cities.open(pattern=pattern).iter_cities() TYPES = {Query.TYPE_RENT: "1", Query.TYPE_SALE: "4" } RET = {Query.TYPE_RENT: {Query.HOUSE_TYPES.HOUSE: '2', Query.HOUSE_TYPES.APART: '1', Query.HOUSE_TYPES.LAND: '', Query.HOUSE_TYPES.PARKING: '4', Query.HOUSE_TYPES.OTHER: '6'}, Query.TYPE_SALE: {Query.HOUSE_TYPES.HOUSE: '2', Query.HOUSE_TYPES.APART: '1', Query.HOUSE_TYPES.LAND: '5', Query.HOUSE_TYPES.PARKING: '6', Query.HOUSE_TYPES.OTHER: '9'} } def search_housings(self, type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, house_types): if type not in self.TYPES: raise TypeNotSupported self.update_header() result = self.form_item.open(data="{'rubrique': '%s'}" % self.TYPES.get(type)) biens = json.loads(json.loads(result.content)['d']) for house_type in house_types: id_type = self.RET[type].get(house_type, '1') data = {} data['rubrique'] = self.TYPES.get(type) data['ach_id'] = None data['FromMoteur'] = "true" for bien in biens: if bien['Idchoix'] == int(id_type): data['lstSSTbien'] = bien['SsTypebien'] data['lstTbien'] = bien['TypeBien'] data['Caracteristique'] = bien['Idchoix'] data['OrigineAlerte'] = "SaveSearchMoteurHome" data['pays'] = "fra" data['prix_min'] = cost_min if cost_min and cost_min > 0 else None data['prix_max'] = cost_max if cost_max and cost_max > 0 else None data['lstThemes'] = "" min_rooms = nb_rooms if nb_rooms else None if not min_rooms: data['lstNbPieces'] = 0 else: data['lstNbPieces'] = ','.join('%s' % n for n in range(min_rooms, 6)) data['lstNbChambres'] = None data['surface_min'] = area_min if area_min else None # var localisationType = { "all": -1, "ville": 5, "region": 2, "departement": 4, "pays": 1, "regionUsuelle": 3 }; data['localisationType'] = 5 data['reference'] = '' """ Avec un rayon a 0, on remonte en priorité les resultats les plus proches, puis de plus en plus eloignes sans limite aucune. On choisit donc arbitrairement de limiter a 100km autour de la ville choisie """ data['rayon'] = 100 data['localisation_id_rayon'] = None data['lstLocalisationId'] = ','.join(cities) data['photos'] = 0 data['colocation'] = '' data['meuble'] = '' data['pageNumber'] = 1 data['order_by'] = 1 data['sort_order'] = 1 data['top'] = 25 data['SaveSearch'] = "false" data['EmailUser'] = "" data['GSMUser'] = "" self.search.go(data="{'p_SearchParams':'%s', 'forcealerte':'0'}" % json.dumps(data)) data = '{pageIndex: 1,source:"undefined",latmin:"0",latmax:"0",lngmin:"0",lngmax:"0", frommoteur:"true"}' for item in self.search_result.go(data=data).iter_housings(): yield item def get_housing(self, _id, obj=None): self.update_header() splitted_id = _id.split('#') data = '{idannonce: %s,source:"%s",rubrique:%s, fromcache:0}' % (splitted_id[0], splitted_id[2], splitted_id[1]) obj = self.housing.go(data=data).get_housing(obj=obj) obj.id = _id return obj def update_header(self): self.session.headers.update({"X-Requested-With": "XMLHttpRequest", "Content-Type": "application/json; charset=utf-8", "Accept": "application/json, text/javascript, */*; q=0.01"}) weboob-1.2/modules/entreparticuliers/favicon.png000066400000000000000000000107331303450110500221740ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME )&OiTXtCommentCreated with GIMPd.ebKGD0LXDR?IDATx tǿFjEAvdAPR AV !(.,E DMܪ).-6@V1 HB"{'/}|Z99I2ͻw}{ߋe?wp "d0E5A@g |!8xNe!䗄gBHR<0*;D ^.(CN_$DP_ "DtG ~ =jH4A J2u@GLp&>b]A=Aq_,xCzJp¨y$ƻ0$@Yvw칹<( `!焫YVeW #)F|'dp dX* wfvuIy].m/_&qSA)Y1qn#՞gKckAmLƭ؍ІEY}%C@3w!`#`SMK_ZIwY >2eqc,Opex_QY~+8*W ʻ6 S*#(81dq+oFL5l>$$ =iW]rfoCr uqna^ =G3{CI|Ts#l^qMEƽϕԞ^c@b[r@~=Nvԛ5;CA ծLsGQ&SF B]EUJ[YciV!ӎ+ *?Rm_@%wAH&&7˜T _JwQރKۤ\{6hOq;Ldcd{7ڂRAfn|ݐ6! $h[3߂sߝ6dt6ѐPY5Lk6\?eX/MVlNY]?^ymOߒ|HE& }ѣ{6J]1)ύ!?BٲQ>m;Z'8[hЈO3v^ {)=e[H!񔻪ېt^@@۠]P9`{M8`KFrVZ 3y_;./ơ ͏MpڥQM>.].ު{Y=B8GJ0Vg~w2!^R`vyiN4@K2<DwC+_4M?hjx3dC( $g0+k;MS2dt5e{+V{u@ ThH+eV!w,0ȹK( ^]GmV|3"gzA]Vy5W2vXhLbEl7keѸ$ƪmP&H_j~ KgرUMBi'\:Ie;~D@V [l: +81Z}V㡓֢RDV#T@U?-&춅Putof5KN/c-'$T+sj;$uU){Gmsv[~[V3I^gb _#bbQr|+u`%.$>M;%G$umO\jVӛ 6'r; 1 {-߲zGJƮO k R⿓r$[w%c/YçH(l{a:seoj'h'aמ\6Շ#XgϷR#]0Eq :os:SI@v-AGݙ'7G  *WMӬGznoo`Hx=EC-in340\ztC'#L5 9U'(T}e̫QIQc`ixO}f)3kly- p{7byH_U۷CB(;Xb>4~q(g/V%VoVe3t0u{BV+CT=-?w4;J_#EOP,EitPLF!6kbqx$]]IЕ*oqU% BYNqaOJ䜑ͪ*͊kPO G j0־+^Pd+i+QU;12.c8}d"ghPyb9{p7pv><>T*!۠ OBb,.P.t1.[Y,[v1XlܙVB.G{";PO)OH R|ZPA@-YiY6YKٟ,o t  )FÈ\c 㤷7's8 vQ@rYnwy5J;b#>->X~}a\4g g ֟G} Ҵ#n7kp[9N]s|}mSh͍itu\\ 9%si?g *O|zTЎV[>k:b"Z1JS@4 \F!eV'D^޴w=V+PJrpz 7#)rJ2Aҙ}7dHy޴5 @]<Y4#%|oѦ'GM:︑^Bu= cQ>HWf}K|}+=H.0 EN/\u/F-}B\& H?{:ڳZ?P탌0v /?E^68Mꅔ┰5>̆R\=I,a}~$Ոp%]bE,W9ƃ||ʹN;HxRѶEИLn_os@y9glWHKy;^_J^9}Qń~O\"4<#VZD"7˓#`Mf_v#CXSo_R"/̄P8'?{%O`@0 OGGɵ5 7mH  -WM`_? +ڏk;gZIENDB`weboob-1.2/modules/entreparticuliers/module.py000066400000000000000000000042331303450110500216760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from .browser import EntreparticuliersBrowser __all__ = ['EntreparticuliersModule'] class EntreparticuliersModule(Module, CapHousing): NAME = 'entreparticuliers' DESCRIPTION = u'entreparticuliers.com website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = EntreparticuliersBrowser def search_city(self, pattern): return self.browser.search_city(pattern) def search_housings(self, query): cities = [c.id for c in query.cities if c.backend == self.name] if len(cities) == 0: return list([]) return self.browser.search_housings(query.type, cities, query.nb_rooms, query.area_min, query.area_max, query.cost_min, query.cost_max, query.house_types) def get_housing(self, _id): return self.browser.get_housing(_id) def fill_housing(self, housing, fields): return self.browser.get_housing(housing.id, housing) def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo} weboob-1.2/modules/entreparticuliers/pages.py000066400000000000000000000114701303450110500215110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from datetime import datetime from weboob.browser.pages import JsonPage, XMLPage from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import CleanText, CleanDecimal, Format, Regexp from weboob.browser.filters.html import CleanHTML from weboob.capabilities.housing import Housing, HousingPhoto, City from weboob.tools.capabilities.housing.housing import PricePerMeterFilter from weboob.capabilities.base import NotAvailable class CitiesPage(JsonPage): @method class iter_cities(DictElement): class item(ItemElement): klass = City def condition(self): return Dict('id', default=None)(self) and\ Dict('localisationType')(self) == u'ville' obj_id = Dict('id') obj_name = Dict('libelle') class EntreParticuliersXMLPage(XMLPage): ENCODING = 'utf-8' def build_doc(self, content): from weboob.tools.json import json json_content = json.loads(content).get('d') or u'' return super(EntreParticuliersXMLPage, self).build_doc(json_content.encode(self.ENCODING)) class SearchPage(EntreParticuliersXMLPage): @method class iter_housings(ListElement): item_xpath = '//AnnoncePresentation' class item(ItemElement): klass = Housing obj_id = Format('%s#%s#%s', CleanText('./Idannonce'), CleanText('./Rubrique'), CleanText('./Source')) obj_title = CleanText('./MiniINfos') obj_cost = CleanDecimal('./Prix', default=Decimal(0)) obj_currency = u'€' obj_text = Format('%s / %s', CleanText('Localisation'), CleanText('./MiniINfos')) obj_date = datetime.now def obj_url(self): url = CleanText('./LienDetail')(self) if not url.startswith('http'): url = u'http://www.entreparticuliers.com%s' % url return url class HousingPage(EntreParticuliersXMLPage): @method class get_housing(ItemElement): klass = Housing obj_title = CleanText('//Titre') def obj_cost(self): cost = CleanDecimal(Regexp(CleanText('//Prix'), u'(.*)\€.*', default=None), default=None)(self) return cost if cost else CleanDecimal(Regexp(CleanText('//Prix'), u'(.*)€'))(self) obj_currency = u'€' obj_text = CleanText('//Description') obj_location = CleanHTML(CleanText('//Localisation')) obj_area = CleanDecimal('//SurfaceBien', replace_dots=True, default=NotAvailable) obj_price_per_meter = PricePerMeterFilter() obj_phone = CleanText('//Telephone') obj_date = datetime.now def obj_details(self): details = {} details[u'Type de bien'] = CleanText('//Tbien')(self) details[u'Reference'] = CleanText('(//Reference)[1]')(self) details[u'Nb pièces'] = CleanText('//Nbpieces')(self) _ener = CleanText('//Energie')(self) if _ener: details[u'Energie'] = _ener _lat = CleanText('//Latitude')(self) if _lat: details[u'Latitude'] = _lat _long = CleanText('//Longitude')(self) if _long: details[u'Longitude'] = _long return details def obj_photos(self): photos = [] for i in range(1, CleanDecimal('//NbPhotos')(self) + 1): img = CleanText('//LienImage%s' % i, replace=[(u'w=69&h=52', u'w=786&h=481')])(self) url = img if img.startswith('http') else u'http://www.entreparticuliers.com%s' % img photos.append(HousingPhoto(url)) return photos weboob-1.2/modules/entreparticuliers/test.py000066400000000000000000000027111303450110500213670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import itertools from weboob.tools.test import BackendTest from weboob.capabilities.housing import Query class EntreparticuliersTest(BackendTest): MODULE = 'entreparticuliers' def test_entreparticuliers(self): query = Query() query.cities = [] for city in self.backend.search_city('lille'): city.backend = self.backend.name query.cities.append(city) query.type = Query.TYPE_SALE query.house_types = [Query.HOUSE_TYPES.HOUSE] results = list(itertools.islice(self.backend.search_housings(query), 0, 20)) self.assertTrue(len(results) > 0) obj = self.backend.fillobj(results[0]) self.assertTrue(obj.area is not None, 'Area for "%s"' % (obj.id)) weboob-1.2/modules/europarl/000077500000000000000000000000001303450110500161225ustar00rootroot00000000000000weboob-1.2/modules/europarl/__init__.py000066400000000000000000000001011303450110500202230ustar00rootroot00000000000000from .module import EuroparlModule __all__ = ['EuroparlModule'] weboob-1.2/modules/europarl/browser.py000066400000000000000000000042071303450110500201620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser from weboob.deprecated.browser.decorators import id2url #from .pages.index import IndexPage from .pages import VideoPage from .video import EuroparlVideo __all__ = ['EuroparlBrowser'] class EuroparlBrowser(Browser): DOMAIN = 'europarl.europa.eu' ENCODING = None PAGES = {r'http://[w\.]*europarl\.europa\.eu/ep-live/(?P\w+)/committees/video\?.*event=(?P[^&]+).*': VideoPage, r'http://[w\.]*europarl\.europa\.eu/ep-live/(?P\w+)/other-events/video\?.*event=(?P[^&]+).*': VideoPage #TODO:plenaries # r'http://[w\.]*europarl\.europa\.eu/ep-live/(?P\w+)/plenary/video\?.*date=(?P[^&]+).*': VideoPage # r'http://[w\.]*europarl\.europa\.eu/ep-live/(?P\w+)/plenary/video\?.*debate=(?P[^&]+).*': VideoPage } @id2url(EuroparlVideo.id2url) def get_video(self, url, video=None): self.location(url) return self.page.get_video(video) # def search_videos(self, pattern, sortby): # return None # self.location(self.buildurl('http://europarltv.europa.eu/en/search%s' % sortby, query=pattern.encode('utf-8'))) # assert self.is_on_page(IndexPage) # return self.page.iter_videos() # def latest_videos(self): # self.home() # assert self.is_on_page(IndexPage) # return self.page.iter_videos() weboob-1.2/modules/europarl/favicon.png000066400000000000000000000010161303450110500202530ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME 0RLtEXtCommentCreated with GIMPWvIDATx EۆjblOKNP(\\輮x,S jԽ ={-!?+[u= s7p$¿i DOhg ([Y&p9tXo#pڧ9?+ -byuѓOO":] JW#T7q3$I0ޑW) vwcvHg\<,ە"#oF2P$<"$Gk; x.0/F0z0oO= jtZs^a.FFu~#ZRklny 7Q@K-14D bZv_IENDB`weboob-1.2/modules/europarl/favicon_europarl.xcf000066400000000000000000000044441303450110500221700ustar00rootroot00000000000000gimp xcf file@@BBm gimp-commentCreated with GIMPgimp-image-grid(style solid) (fgcolor (color-rgba 0.000000 0.000000 0.000000 1.000000)) (bgcolor (color-rgba 1.000000 1.000000 1.000000 1.000000)) (xspacing 10.000000) (yspacing 10.000000) (spacing-unit inches) (xoffset 0.000000) (yoffset 0.000000) (offset-unit inches) gamma0.45454999804496765t@@ favicon.png     @@@@77=77077 777%77777$777"777 777"777777"77777777777777<7$777!777"77 77!77777 7:7:7777N77=77<7/777,77 7-7 7+77,77777+7:7:777777=7777$777"77#77 777777777777777777777777777#7777%777%777%777777777.7:7:7777777=77077 777%77777$777"777 777"777777"77777777777777<7$777!777"77 77!77777 7:7:7777N77=77<7/777,77 7-7 7+77,77777+7:7:777777=7777$777"77#77 777777777777777777777777777#7777%777%777%777777777.7:7:777777=707 77%777$777"777 777"7777"77777777<7$77!77"77 77!777 7:7:77N7=7<7/77,7 7-7 7+77,777+7:7:777=77$77"77#77 777777777777777777#777%777%777%77777.7:7:77@@Masque de sélection @@ @@ weboob-1.2/modules/europarl/module.py000066400000000000000000000055301303450110500177640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo, BaseVideo from weboob.tools.backend import Module from weboob.capabilities.collection import CapCollection, CollectionNotFound from .browser import EuroparlBrowser from .video import EuroparlVideo __all__ = ['EuroparlModule'] class EuroparlModule(Module, CapVideo, CapCollection): NAME = 'europarl' MAINTAINER = u'François Revol' EMAIL = 'revol@free.fr' VERSION = '1.2' DESCRIPTION = 'Europarl parliamentary video streaming website' LICENSE = 'AGPLv3+' BROWSER = EuroparlBrowser def get_video(self, _id): with self.browser: return self.browser.get_video(_id) SORTBY = ['relevance', 'rating', 'views', 'time'] # def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): # with self.browser: # return self.browser.search_videos(pattern, self.SORTBY[sortby]) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(EuroparlVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest Europarl videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {EuroparlVideo: fill_video} weboob-1.2/modules/europarl/pages.py000066400000000000000000000104131303450110500175720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.mech import ClientForm ControlNotFoundError = ClientForm.ControlNotFoundError from weboob.deprecated.browser import Page import re import datetime from weboob.capabilities.base import NotAvailable from weboob.deprecated.browser import BrokenPageError from .video import EuroparlVideo class VideoPage(Page): def get_video(self, video=None): if video is None: video = EuroparlVideo(self.group_dict['id']) video.title = unicode(self.get_title()) video.url = unicode(self.get_url()) self.set_details(video) video.set_empty_fields(NotAvailable) return video def get_url(self): # search for # TODO: plenaries can be downloaded as mp4... obj = self.parser.select(self.document.getroot(), 'input#codeUrl', 1) if obj is None: return None return obj.attrib['value'] def get_title(self): obj = self.parser.select(self.document.getroot(), 'h1#player_subjectTitle') if len(obj) < 1: obj = self.parser.select(self.document.getroot(), 'title') if len(obj) < 1: return None title = obj[0].text.strip() obj = self.parser.select(self.document.getroot(), 'span.ep_subtitle') if len(obj) < 1: return title for span in self.parser.select(obj[0], 'span.ep_acronym, span.ep_theme'): if span.text_content(): title += ' ' + span.text_content().strip() return title def set_details(self, v): v.author = u'European Parliament' obj = self.parser.select(self.document.getroot(), 'meta[name=available]', 1) if obj is not None: value = obj.attrib['content'] m = re.match('(\d\d)-(\d\d)-(\d\d\d\d)\s*(\d\d):(\d\d)', value) if not m: raise BrokenPageError('Unable to parse datetime: %r' % value) day = m.group(1) month = m.group(2) year = m.group(3) hour = m.group(4) minute = m.group(5) v.date = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(hour), minute=int(minute)) obj = self.parser.select(self.document.getroot(), 'span.ep_subtitle', 1) if obj is not None: span = self.parser.select(obj, 'span.ep_date', 1) value = span.text m = re.match('(\d\d):(\d\d)\s*\/\s*(\d\d):(\d\d)\s*-\s*(\d\d)-(\d\d)-(\d\d\d\d)', value) if not m: raise BrokenPageError('Unable to parse datetime: %r' % value) bhour = m.group(1) bminute = m.group(2) ehour = m.group(3) eminute = m.group(4) day = m.group(5) month = m.group(6) year = m.group(7) start = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(bhour), minute=int(bminute)) end = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(ehour), minute=int(eminute)) v.duration = end - start weboob-1.2/modules/europarl/test.py000066400000000000000000000031011303450110500174460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest #from weboob.capabilities.video import BaseVideo class EuroparlTest(BackendTest): MODULE = 'europarl' # def test_search(self): # l = list(self.backend.search_videos('neelie kroes')) # self.assertTrue(len(l) > 0) # v = l[0] # self.backend.fillobj(v, ('url',)) # self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) # self.backend.browser.openurl(v.url) # def test_latest(self): # l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) # self.assertTrue(len(l) > 0) # v = l[0] # self.backend.fillobj(v, ('url',)) # self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) weboob-1.2/modules/europarl/video.py000066400000000000000000000032451303450110500176060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo import re class EuroparlVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.ext = u'wmv' @classmethod def id2url(cls, _id): m = re.match('.*-COMMITTEE-.*', _id) if m: return u'http://www.europarl.europa.eu/ep-live/en/committees/video?event=%s&format=wmv' % _id m = re.match('.*-SPECIAL-.*', _id) if m: return u'http://www.europarl.europa.eu/ep-live/en/other-events/video?event=%s&format=wmv' % _id # XXX: not yet supported m = re.match('\d\d-\d\d-\d\d\d\d', _id) if m: return u'http://www.europarl.europa.eu/ep-live/en/plenary/video?date=%s' % _id # XXX: not yet supported m = re.match('\d+', _id) if m: return u'http://www.europarl.europa.eu/ep-live/en/plenary/video?debate=%s' % _id return None weboob-1.2/modules/explorimmo/000077500000000000000000000000001303450110500164645ustar00rootroot00000000000000weboob-1.2/modules/explorimmo/__init__.py000066400000000000000000000014421303450110500205760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ExplorimmoModule __all__ = ['ExplorimmoModule'] weboob-1.2/modules/explorimmo/browser.py000066400000000000000000000057761303450110500205400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from weboob.browser import PagesBrowser, URL from weboob.capabilities.housing import Query, TypeNotSupported from .pages import CitiesPage, SearchPage, HousingPage, HousingPage2, PhonePage class ExplorimmoBrowser(PagesBrowser): BASEURL = 'http://www.explorimmo.com/' cities = URL('rest/locations\?q=(?P.*)', CitiesPage) search = URL('resultat/annonces.html\?(?P.*)', SearchPage) housing_html = URL('annonce-(?P<_id>.*).html', HousingPage) phone = URL('rest/classifieds/(?P<_id>.*)/phone', PhonePage) housing = URL('rest/classifieds/(?P<_id>.*)', 'rest/classifieds/\?(?P.*)', HousingPage2) TYPES = {Query.TYPE_RENT: 'location', Query.TYPE_SALE: 'vente'} RET = {Query.HOUSE_TYPES.HOUSE: 'Maison', Query.HOUSE_TYPES.APART: 'Appartement', Query.HOUSE_TYPES.LAND: 'Terrain', Query.HOUSE_TYPES.PARKING: 'Parking', Query.HOUSE_TYPES.OTHER: 'Divers'} def get_cities(self, pattern): return self.cities.open(city=pattern).get_cities() def search_housings(self, type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, house_types): if type not in self.TYPES: raise TypeNotSupported() ret = [] for house_type in house_types: if house_type in self.RET: ret.append(self.RET.get(house_type)) data = {'location': ','.join(cities).encode('iso 8859-1'), 'areaMin': area_min or '', 'areaMax': area_max or '', 'priceMin': cost_min or '', 'priceMax': cost_max or '', 'transaction': self.TYPES.get(type, 'location'), 'recherche': '', 'mode': '', 'proximity': '0', 'roomMin': nb_rooms or '', 'page': '1'} query = u'%s%s%s' % (urllib.urlencode(data), '&type=', '&type='.join(ret)) return self.search.go(query=query).iter_housings() def get_housing(self, _id, housing=None): return self.housing.go(_id=_id).get_housing(obj=housing) def get_phone(self, _id): return self.phone.go(_id=_id).get_phone() def get_total_page(self, js_datas): return self.housing.open(js_datas=js_datas).get_total_page() weboob-1.2/modules/explorimmo/favicon.png000066400000000000000000000044231303450110500206220ustar00rootroot00000000000000PNG  IHDR@@iqsRGBgAMA a pHYs+tEXtAuthorxrq tEXtCreationTime2014:01:24 11:19:28r!tEXtCreation Time2014:01:24 11:19:28xz;IDATx^ZiM__=d̑"#O $Ŕ/$ A^f Q"12뷲v{;]sZ{sSfCnEx9PR U\pڸqի}j֬镄 &m̘1tԩ[ԯ_/yds۷oO݋j*Yfي/^<~ئ|R4hdڴit~~>ґj 6HK@˖-IдiS)IEǥMw7o:CP~-Anݬ CjՒի'sAAA ˗/PM[.=zUIg}6A޿o|qXH +4q-C/N1|޾}kׯ_.%`ƌ`Zn#FHϟ[Eԩc;C`n }ڜ8qر#2ӭ͍7 9w9x)dɴjJ< Ov߰u0Ϟ=y:/Q8zUel@ 0ݻw]>5f͚5`Ͱa2F@c=JEȕ`@Y rJW.Q ?fܹ{eg̦ƱsMB^62Y&Tgg[plF:N8$h\G(!+V@]e x9b_lĕ ;>59~ByU53[l 8;ܲ:6x]U)mhҷQ02pb-o>B Vfg8D^ @,.xg0p7]|M 4Wty9?\W^qVi"bpVj`W'= }=Z#WЖG)Tu.);v%eѢEO /lZXW/ !aէv?,8^c}ա{$'EtI:qnx(:jm3S/):ٽ{7۷K]uwf&IB{ z z|_^n 6zl?.uE B_>|x6zSZQ\#.:;Aӻ>(et `'QdyÇON͛7iĉ;UADq_dm 2viǏ6 LP`Qd`>/Z@jOw`!)8IX!-Bʕ2N*+D% o/:@.~2ĂJW;xPxу]  [æNwe%BztZlhvZV%LI+Bz<0`^X0vu(kyc UBdz} r"+BpQ"q =À.su=O[2k]/{+3 cǎ%Xdzh…iwf@qf_+ 4> nj璆,7ޭ\&F9`oK%XOSIENDB`weboob-1.2/modules/explorimmo/module.py000066400000000000000000000047371303450110500203360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from .browser import ExplorimmoBrowser __all__ = ['ExplorimmoModule'] class ExplorimmoModule(Module, CapHousing): NAME = 'explorimmo' DESCRIPTION = u'explorimmo website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ExplorimmoBrowser def get_housing(self, housing): if isinstance(housing, Housing): id = housing.id else: id = housing housing = None housing = self.browser.get_housing(id, housing) housing.phone = self.browser.get_phone(id) return housing def search_city(self, pattern): return self.browser.get_cities(pattern) def search_housings(self, query): cities = ['%s' % c.id for c in query.cities if c.backend == self.name] if len(cities) == 0: return list() return self.browser.search_housings(query.type, cities, query.nb_rooms, query.area_min, query.area_max, query.cost_min, query.cost_max, query.house_types) def fill_housing(self, housing, fields): self.browser.get_housing(housing.id, housing) if 'phone' in fields: housing.phone = self.browser.get_phone(housing.id) return housing def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo, } weboob-1.2/modules/explorimmo/pages.py000066400000000000000000000165561303450110500201520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from decimal import Decimal from datetime import datetime from weboob.browser.filters.json import Dict from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.pages import JsonPage, HTMLPage, pagination from weboob.browser.filters.standard import CleanText, CleanDecimal, Regexp, Env, BrowserURL, Filter, Format from weboob.browser.filters.html import CleanHTML, XPath from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.capabilities.housing import Housing, HousingPhoto, City from weboob.tools.capabilities.housing.housing import PricePerMeterFilter class CitiesPage(JsonPage): ENCODING = u'UTF-8' @method class get_cities(DictElement): item_xpath = '0/locations' class item(ItemElement): klass = City obj_id = Dict('label') obj_name = Dict('label') class SearchPage(HTMLPage): @pagination @method class iter_housings(ListElement): item_xpath = '//div[starts-with(@id, "bloc-vue-")]' def next_page(self): js_datas = CleanText('//div[@id="js-data"]/@data-rest-search-request')(self) total_page = self.page.browser.get_total_page(js_datas.split('?')[-1]) m = re.match(".*page=(\d?)(?:&.*)?", self.page.url) if m: current_page = int(m.group(1)) next_page = current_page + 1 if next_page <= total_page: return self.page.url.replace('page=%d' % current_page, 'page=%d' % next_page) class item(ItemElement): klass = Housing obj_id = CleanText('./@data-classified-id') obj_title = CleanText('./div/h2[@itemprop="name"]/a') obj_location = CleanText('./div/h2[@itemprop="name"]/span[class="item-localisation"]') obj_cost = CleanDecimal('./div/div/span[@class="price-label"]') obj_currency = Regexp(CleanText('./div/div/span[@class="price-label"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_text = CleanText('./div/div/div[@itemprop="description"]') obj_area = CleanDecimal(Regexp(CleanText('./div/h2[@itemprop="name"]/a'), '(.*?)(\d*) m2(.*?)', '\\2', default=None), default=NotAvailable) obj_price_per_meter = PricePerMeterFilter() def obj_phone(self): phone = CleanText('./div/div/ul/li/span[@class="js-clickphone"]', replace=[(u'Téléphoner : ', u'')], default=NotAvailable)(self) if '...' in phone: return NotLoaded return phone def obj_photos(self): url = CleanText('./div/div/a/img[@itemprop="image"]/@src')(self) return [HousingPhoto(url)] class TypeDecimal(Filter): def filter(self, el): return Decimal(el) class FromTimestamp(Filter): def filter(self, el): return datetime.fromtimestamp(el / 1000.0) class PhonePage(JsonPage): def get_phone(self): return self.doc.get('phoneNumber') class HousingPage2(JsonPage): @method class get_housing(ItemElement): klass = Housing obj_id = Env('_id') obj_title = Dict('characteristics/titleWithTransaction') obj_location = Format('%s %s %s', Dict('location/address'), Dict('location/postalCode'), Dict('location/cityLabel')) obj_cost = TypeDecimal(Dict('characteristics/price')) obj_currency = u'€' obj_text = CleanHTML(Dict('characteristics/description')) obj_url = BrowserURL('housing_html', _id=Env('_id')) obj_area = TypeDecimal(Dict('characteristics/area')) obj_date = FromTimestamp(Dict('characteristics/date')) obj_price_per_meter = PricePerMeterFilter() def obj_photos(self): photos = [] for img in Dict('characteristics/images')(self): m = re.search('http://thbr\.figarocms\.net.*(http://.*)', img.get('xl')) if m: photos.append(HousingPhoto(m.group(1))) else: photos.append(HousingPhoto(img.get('xl'))) return photos def obj_details(self): details = {} details['fees'] = Dict('characteristics/fees')(self) details['bedrooms'] = Dict('characteristics/bedroomCount')(self) details['energy'] = Dict('characteristics/energyConsumptionCategory')(self) rooms = Dict('characteristics/roomCount')(self) if len(rooms): details['rooms'] = rooms[0] details['available'] = Dict('characteristics/available', default=NotAvailable)(self) return details def get_total_page(self): return self.doc.get('pagination').get('total') if 'pagination' in self.doc else 0 class HousingPage(HTMLPage): @method class get_housing(ItemElement): klass = Housing obj_id = Env('_id') obj_title = CleanText('//h1[@itemprop="name"]') obj_location = CleanText('//span[@class="informations-localisation"]') obj_cost = CleanDecimal('//span[@itemprop="price"]') obj_currency = Regexp(CleanText('//span[@itemprop="price"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_text = CleanHTML('//div[@itemprop="description"]') obj_url = BrowserURL('housing', _id=Env('_id')) obj_area = CleanDecimal(Regexp(CleanText('//h1[@itemprop="name"]'), '(.*?)(\d*) m2(.*?)', '\\2'), default=NotAvailable) obj_price_per_meter = PricePerMeterFilter() def obj_photos(self): photos = [] for img in XPath('//a[@class="thumbnail-link"]/img[@itemprop="image"]')(self): url = Regexp(CleanText('./@src'), 'http://thbr\.figarocms\.net.*(http://.*)')(img) photos.append(HousingPhoto(url)) return photos def obj_details(self): details = dict() for item in XPath('//div[@class="features clearfix"]/ul/li')(self): key = CleanText('./span[@class="name"]')(item) value = CleanText('./span[@class="value"]')(item) if value and key: details[key] = value key = CleanText('//div[@class="title-dpe clearfix"]')(self) value = CleanText('//div[@class="energy-consumption"]')(self) if value and key: details[key] = value return details weboob-1.2/modules/explorimmo/test.py000066400000000000000000000025601303450110500200200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import itertools from weboob.capabilities.housing import Query from weboob.tools.test import BackendTest class ExplorimmoTest(BackendTest): MODULE = 'explorimmo' def test_explorimmo(self): query = Query() query.area_min = 20 query.cost_max = 900 query.type = Query.TYPE_RENT query.cities = [] for city in self.backend.search_city('paris'): city.backend = self.backend.name query.cities.append(city) results = list(itertools.islice(self.backend.search_housings(query), 0, 20)) self.assertTrue(len(results) > 0) self.backend.fillobj(results[0], 'phone') weboob-1.2/modules/feedly/000077500000000000000000000000001303450110500155415ustar00rootroot00000000000000weboob-1.2/modules/feedly/__init__.py000066400000000000000000000014321303450110500176520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import FeedlyModule __all__ = ['FeedlyModule'] weboob-1.2/modules/feedly/browser.py000066400000000000000000000101231303450110500175730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.json import json from weboob.capabilities.base import UserError from weboob.capabilities.collection import Collection from weboob.browser import LoginBrowser, URL, need_login from .pages import EssentialsPage, TokenPage, ContentsPage, PreferencesPage, MarkerPage __all__ = ['FeedlyBrowser'] class FeedlyBrowser(LoginBrowser): BASEURL = 'https://www.feedly.com/' essentials = URL('https://s3.feedly.com/essentials/essentials_fr.json', EssentialsPage) token = URL('v3/auth/token', TokenPage) contents = URL('v3/streams/contents', ContentsPage) preferences = URL('v3/preferences', PreferencesPage) marker = URL('v3/markers', MarkerPage) def __init__(self, username, password, login_browser, *args, **kwargs): super(FeedlyBrowser, self).__init__(username, password, *args, **kwargs) self.user_id = None self.login_browser = login_browser def do_login(self): if self.login_browser: if self.login_browser.code is None or self.user_id is None: self.login_browser.do_login() params = {'code': self.login_browser.code, 'client_id': 'feedly', 'client_secret': '0XP4XQ07VVMDWBKUHTJM4WUQ', 'redirect_uri': 'http://dev.feedly.com/feedly.html', 'grant_type': 'authorization_code'} token, self.user_id = self.token.go(data=params).get_token() self.session.headers['X-Feedly-Access-Token'] = token else: raise UserError(r'You need to fill your username and password to access this page') @need_login def iter_threads(self): params = {'streamId': 'user/%s/category/global.all' % self.user_id, 'unreadOnly': 'true', 'ranked': 'newest', 'count': '100'} return self.contents.go(params=params).get_articles() def get_unread_feed(self, url): params = {'streamId': url, 'backfill': 'true', 'boostMustRead': 'true', 'unreadOnly': 'true'} return self.contents.go(params=params).get_articles() def get_categories(self): if self.username is not None and self.password is not None: return self.get_logged_categories() return self.essentials.go().get_categories() @need_login def get_logged_categories(self): user_categories = list(self.preferences.go().get_categories()) user_categories.append(Collection([u'global.saved'], u'Saved')) return user_categories def get_feeds(self, category): if self.username is not None and self.password is not None: return self.get_logged_feeds(category) return self.essentials.go().get_feeds(category) @need_login def get_logged_feeds(self, category): if category == 'global.saved': type = 'tag' else: type = 'category' url = 'user/%s/%s/%s' % (self.user_id, type, category) return self.get_unread_feed(url) def get_feed_url(self, category, feed): return self.essentials.go().get_feed_url(category, feed) @need_login def set_message_read(self, _id): datas = {'action': 'markAsRead', 'type': 'entries', 'entryIds': [_id]} self.marker.open(data=json.dumps(datas)) weboob-1.2/modules/feedly/favicon.png000066400000000000000000000034041303450110500176750ustar00rootroot00000000000000PNG  IHDR@@PLTEij㥽姱鬴桪ʖߖ蚜ߒÎܘɑًۡІًԌۍފ׈ۋ܊ߌp|qАwm│upuzwxvmvq`ϊ`v`s`vZX_sVd\tSVZ`Y\YUT^HL  " [UZTTNHIEH "+(8Nbfox~׎뚖ݑٞ&KtRNS "#$%(*./0235889 \>~ LY'l$ϖ07$Hyv$r-,>D gM8,pՔp9u@i]sq_R;r9Wve  IQLzl` _(;^JoRȃJ:5`Gw)8?/B<{r}m6x)jhT7OQb4`A= x30& M!njH#.ƆcQlwA݀l6@XH"Lx1=Z OY'D0VP1#Л.J|Pt1!?Y$c߰$uz͖X<ZcmfwE&>YwgkwQӕfwIw$+fkx-: : wlJ-)nɤ+^ |Lzˬh̉l5i!-Y@fgoU$gYTzٹ $`2NϘw7LýGzSݤ~StmsՐ  W|fa\raOwl+b& 'eqpی6aLx{~Gk]x*L|\;H|e$k o9IENDB`weboob-1.2/modules/feedly/google.py000066400000000000000000000043721303450110500173750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from urlparse import urlparse, parse_qs from weboob.browser import LoginBrowser, URL from weboob.browser.pages import HTMLPage, LoggedPage from weboob.exceptions import BrowserIncorrectPassword class GoogleLoginPage(LoggedPage, HTMLPage): def login(self, login, passwd): form = self.get_form('//form[@id="gaia_loginform"]', submit='//input[@id="signIn"]') form['Email'] = login form['Passwd'] = passwd form.submit() class GoogleBrowser(LoginBrowser): BASEURL = 'https://accounts.google.com/' code = None google_login = URL('https://accounts.google.com/(?P.+)', 'AccountLoginInfo', GoogleLoginPage) def __init__(self, username, password, redirect_uri, *args, **kwargs): super(GoogleBrowser, self).__init__(username, password, *args, **kwargs) self.redirect_uri = redirect_uri def do_login(self): params = {'response_type': 'code', 'client_id': '534890559860-r6gn7e3agcpiriehe63dkeus0tpl5i4i.apps.googleusercontent.com', 'redirect_uri': self.redirect_uri} queryString = "&".join([key+'='+value for key, value in params.items()]) self.google_login.go(auth='o/oauth2/auth', params=queryString).login(self.username, self.password) if self.google_login.is_here(): self.page.login(self.username, self.password) try: self.code = parse_qs(urlparse(self.url).query).get('code')[0] except: raise BrowserIncorrectPassword() weboob-1.2/modules/feedly/module.py000066400000000000000000000103051303450110500173770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.collection import CapCollection from weboob.capabilities.messages import CapMessages, Message, Thread from weboob.tools.value import Value, ValueBackendPassword from .browser import FeedlyBrowser from .google import GoogleBrowser __all__ = ['FeedlyModule'] class FeedlyModule(Module, CapMessages, CapCollection): NAME = 'feedly' DESCRIPTION = u'handle the popular RSS reading service Feedly' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' STORAGE = {'seen': []} CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default='')) BROWSER = FeedlyBrowser def iter_resources(self, objs, split_path): collection = self.get_collection(objs, split_path) if collection.path_level == 0: return self.browser.get_categories() if collection.path_level == 1: return self.browser.get_feeds(split_path[0]) if collection.path_level == 2: url = self.browser.get_feed_url(split_path[0], split_path[1]) threads = [] for article in self.browser.get_unread_feed(url): thread = self.get_thread(article.id, article) threads.append(thread) return threads def validate_collection(self, objs, collection): if collection.path_level in [0, 1, 2]: return def get_thread(self, id, entry=None): if isinstance(id, Thread): thread = id id = thread.id else: thread = Thread(id) if entry is None: url = id.split('#')[0] for article in self.browser.get_unread_feed(url): if article.id == id: entry = article if entry is None: return None if thread.id not in self.storage.get('seen', default=[]): entry.flags = Message.IS_UNREAD entry.thread = thread thread.title = entry.title thread.root = entry return thread def iter_unread_messages(self): for thread in self.iter_threads(): for m in thread.iter_all_messages(): if m.flags & m.IS_UNREAD: yield m def iter_threads(self): for article in self.browser.iter_threads(): yield self.get_thread(article.id, article) def set_message_read(self, message): self.browser.set_message_read(message.thread.id.split('#')[-1]) self.storage.get('seen', default=[]).append(message.thread.id) self.storage.save() def fill_thread(self, thread, fields): return self.get_thread(thread) def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() login_browser = GoogleBrowser(username, password, 'https://feedly.com/v3/auth/callback&scope=profile+email&state=A8duE2XpzvtgcHt-q29qyBBK2fkpTefgqfzy7SY4GWUOPl3BgrSt4DRS-qKm9MRi_mXJRem8QW7RmNjpc_BIlkWc0JJvpay3UyzIErNvtaZLcsrUy94Ays3gTyispb8R0doguiky8gGxuCFNvJ9iXIB_SlwNhWABm7ut3nIgoMg3wodRgYOPFothhkErchrv076tBwXQA4Z8OIRyrQ') else: password = None login_browser = None return self.create_browser(username, password, login_browser) OBJECTS = {Thread: fill_thread} weboob-1.2/modules/feedly/pages.py000066400000000000000000000066121303450110500172170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.capabilities.messages import Message from weboob.capabilities.collection import Collection from weboob.browser.pages import JsonPage, LoggedPage from weboob.browser.elements import ItemElement, DictElement, method from weboob.browser.filters.standard import CleanText, Format from weboob.browser.filters.json import Dict from weboob.browser.filters.html import CleanHTML class ContentsPage(LoggedPage, JsonPage): @method class get_articles(DictElement): item_xpath = 'items' class item(ItemElement): klass = Message obj_id = Format(u'%s#%s', CleanText(Dict('origin/streamId')), CleanText(Dict('id'))) obj_sender = CleanText(Dict('author', default=u'')) obj_title = Format(u'%s - %s', CleanText(Dict('origin/title', default=u'')), CleanText(Dict('title', default=u''))) def obj_date(self): return datetime.fromtimestamp(Dict('published')(self.el) / 1e3) def obj_content(self): if 'content' in self.el.keys(): return Format(u'%s%s\r\n', CleanHTML(Dict('content/content')), CleanText(Dict('origin/htmlUrl')))(self.el) elif 'summary' in self.el.keys(): return Format(u'%s%s\r\n', CleanHTML(Dict('summary/content')), CleanText(Dict('origin/htmlUrl')))(self.el) else: return '' class TokenPage(JsonPage): def get_token(self): return self.doc['access_token'], self.doc['id'] class EssentialsPage(JsonPage): def get_categories(self): for category in self.doc: name = u'%s' % category.get('label') yield Collection([name], name) def get_feeds(self, label): for category in self.doc: if category.get('label') == label: feeds = category.get('subscriptions') for feed in feeds: yield Collection([label, feed.get('title')]) def get_feed_url(self, _category, _feed): for category in self.doc: if category.get('label') == _category: feeds = category.get('subscriptions') for feed in feeds: if feed.get('title') == _feed: return feed.get('id') class PreferencesPage(LoggedPage, JsonPage): def get_categories(self): for category, value in self.doc.items(): if value in [u"shown", u"hidden"]: yield Collection([u'%s' % category], u'%s' % category.replace('global.', '')) class MarkerPage(LoggedPage): pass weboob-1.2/modules/feedly/test.py000066400000000000000000000026201303450110500170720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest, SkipTest class FeedlyTest(BackendTest): MODULE = 'feedly' def test_login(self): if self.backend.browser.username: l1 = list(self.backend.iter_threads()) assert len(l1) thread = self.backend.get_thread(l1[0].id) assert len(thread.root.content) else: raise SkipTest("User credentials not defined") def test_feedly(self): self.backend.browser.username = None l1 = list(self.backend.iter_resources(None, ['Technologie', 'Korben'])) assert len(l1) thread = self.backend.get_thread(l1[0].id) assert len(thread.root.content) weboob-1.2/modules/fortuneo/000077500000000000000000000000001303450110500161325ustar00rootroot00000000000000weboob-1.2/modules/fortuneo/__init__.py000066400000000000000000000014671303450110500202530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import FortuneoModule __all__ = ['FortuneoModule'] # vim:ts=4:sw=4 weboob-1.2/modules/fortuneo/browser.py000066400000000000000000000100041303450110500201620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages.login import LoginPage from .pages.accounts_list import GlobalAccountsList, AccountsList, AccountHistoryPage, CardHistoryPage, \ InvestmentHistoryPage, PeaHistoryPage __all__ = ['Fortuneo'] class Fortuneo(LoginBrowser): BASEURL = 'https://mabanque.fortuneo.fr' #CERTHASH = ['4ff0301115f80f18c4e81a136ca28829b46d416d404174945b1ae48abd0634e2', '608d63d9ef394c13a64b71ed55e4564491873498dd62540a6b7f7b88f251be30'] login_page = URL(r'.*identification\.jsp.*', LoginPage) accounts_page = URL(r'.*prive/default\.jsp.*', r'.*/prive/mes-comptes/synthese-mes-comptes\.jsp', AccountsList) global_accounts = URL(r'.*/prive/mes-comptes/synthese-globale/synthese-mes-comptes\.jsp', GlobalAccountsList) account_history = URL(r'.*/prive/mes-comptes/livret/consulter-situation/consulter-solde\.jsp.*', r'.*/prive/mes-comptes/compte-courant/consulter-situation/consulter-solde\.jsp.*', r'.*/prive/mes-comptes/compte-especes.*', AccountHistoryPage) card_history = URL(r'.*/prive/mes-comptes/compte-courant/carte-bancaire/encours-debit-differe\.jsp.*', CardHistoryPage) pea_history = URL(r'.*/prive/mes-comptes/compte-titres-.*', r'.*/prive/mes-comptes/pea.*', PeaHistoryPage) invest_history = URL(r'.*/prive/mes-comptes/assurance-vie.*', InvestmentHistoryPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.login_page.is_here(): self.location('/fr/identification.jsp') self.page.login(self.username, self.password) if self.login_page.is_here(): raise BrowserIncorrectPassword() self.location('/fr/prive/mes-comptes/synthese-mes-comptes.jsp') if self.accounts_page.is_here() and self.page.need_reload(): self.location('/ReloadContext?action=1&') elif self.accounts_page.is_here() and self.page.need_sms(): raise BrowserIncorrectPassword('Authentification with sms is not supported') @need_login def get_investments(self, account): self.location(account._link_id) return self.page.get_investments() @need_login def get_history(self, account): self.location(account._link_id) if self.page.select_period(): return self.page.get_operations(account) return iter([]) @need_login def get_coming(self, account): for cb_link in account._card_links: self.location(cb_link) for tr in self.page.get_operations(account): yield tr @need_login def get_accounts_list(self): """accounts list""" if not self.accounts_page.is_here(): self.location('/fr/prive/mes-comptes/synthese-mes-comptes.jsp') return self.page.get_list() @need_login def get_account(self, id): """Get an account from its ID""" assert isinstance(id, basestring) for a in list(self.get_accounts_list()): if a.id == id: return a return None # vim:ts=4:sw=4 weboob-1.2/modules/fortuneo/favicon.png000066400000000000000000000036171303450110500202740ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME .)jsD/tEXtCommentCreated with GIMPWIDATxkLSgZ)¡AE[o V`ls`f`Mݖ}pYL4.Qc,̰6cpNQf8墴@ ZZPZFyO/}=}~s?EoOS.QQQQQQQO>1v7lV~#> h!RT* -zX N4TvVkBPɐWХd̰iwGa;`;Хwft\L-Q]7Jlu'pR?|Ɍ U"i! FC UfNO`T)F6e؉WPxl(`30h4湞ѥrpt.~ sF/<K?`y>:^K?vz~݊FtDt휑!jmCv Ȯιhc +9wɞ02y2[a}|^PbZ9bZ9M$fCO Y#B8ܬz|]j2h58Zx*)ht2N Ixoo;"0qy}QIۭF\Bt}{+.9%xldJ&9uۇY & $G 5z ",.~fzh]0è;i-7DD+@^YFX@L 'Ms6O}W0<^>16y}+AayerNQXvnȌ/\L ܨ@_*SKCKmJ"W(nS5 \H@ kvxpP J Y+ibbZRBXϪfJ 1".nzoW dRpئFJ1V @!)C̞H|@˥^Tv}Cs$yyo{17+:̷L- ZS-P.fX 95~dhBI),Q-Җ'zq{{22)* 6|v> @M`Z 9 [+`/nP>$6Tz I(:m"SXgMs![ ! 3=Ƚfev[dۭ(!!!/jptUOn^ӗ%V;۳4Pp;L*28rD(st]xYWcn|y"rJg@J.&88Vrm;ЌwY1ꭊLcD".񭵽h`am{@S- :7rP(/Fm8o1&,q (ٿyeDݳ\)ko^TAaþX./Kċ,^eJƖcZO ]Bw"6NS$:DY3}B5wI:$=|*QitX3z]"A[۳9SHxs ;TuRCL-Lm~ɰof7fݮ) ϿQYT\<9q((((((((V!&IENDB`weboob-1.2/modules/fortuneo/module.py000066400000000000000000000043351303450110500177760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Fortuneo __all__ = ['FortuneoModule'] class FortuneoModule(Module, CapBank): NAME = 'fortuneo' MAINTAINER = u'Gilles-Alexandre Quenot' EMAIL = 'gilles.quenot@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Fortuneo' CONFIG = BackendConfig( ValueBackendPassword('login', label='Identifiant', masked=False, required=True), ValueBackendPassword('password', label='Mot de passe', required=True)) BROWSER = Fortuneo def create_default_browser(self): return self.create_browser( self.config['login'].get(), self.config['password'].get() ) def iter_accounts(self): """Iter accounts""" return self.browser.get_accounts_list() def get_account(self, _id): with self.browser: account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): """Iter history of transactions on a specific account""" return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_coming(account) def iter_investment(self, account): return self.browser.get_investments(account) # vim:ts=4:sw=4 weboob-1.2/modules/fortuneo/pages/000077500000000000000000000000001303450110500172315ustar00rootroot00000000000000weboob-1.2/modules/fortuneo/pages/__init__.py000066400000000000000000000000001303450110500213300ustar00rootroot00000000000000weboob-1.2/modules/fortuneo/pages/accounts_list.py000066400000000000000000000340321303450110500224570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from urllib import urlencode import re from time import sleep from datetime import date from dateutil.relativedelta import relativedelta from lxml.html import etree from weboob.browser.filters.standard import CleanText, CleanDecimal from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account, Investment from weboob.browser.pages import HTMLPage, LoggedPage, FormNotFound from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.json import json from weboob.exceptions import ActionNeeded, BrowserUnavailable class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^(?PCHEQUE)(?P.*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(?PFACTURE CARTE) DU (?P
\d{2})(?P\d{2})(?P\d{2}) (?P.*?)( CA?R?T?E? ?\d*X*\d*)?$'), FrenchTransaction.TYPE_CARD), (re.compile('^(?PCARTE)( DU)? (?P
\d{2})/(?P\d{2}) (?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P(PRELEVEMENT|TELEREGLEMENT|TIP|PRLV)) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?PECHEANCEPRET)(?P.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(?PRET(RAIT)? DAB) (?P
\d{2})/(?P\d{2})/(?P\d{2})( (?P\d+)H(?P\d+))? (?P.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?PVIR(EMEN)?T? ((RECU|FAVEUR) TIERS|SEPA RECU)?)( /FRM)?(?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?PREMBOURST)(?P.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?PCOMMISSIONS)(?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P(?PREMUNERATION).*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?PREMISE CHEQUES)(?P.*)'), FrenchTransaction.TYPE_DEPOSIT), ] class PeaHistoryPage(LoggedPage, HTMLPage): COL_LABEL = 0 COL_UNITVALUE = 1 COL_QUANTITY = 3 COL_UNITPRICE = 4 COL_VALUATION = 5 COL_PERF = 6 COL_WEIGHT = 7 def get_investments(self): for line in self.doc.xpath('//table[@id="t_intraday"]/tbody/tr'): if line.find_class('categorie') or line.find_class('detail') or line.find_class('detail02'): continue cols = line.findall('td') inv = Investment() inv.label = CleanText(None).filter(cols[self.COL_LABEL]) link = cols[self.COL_LABEL].xpath('./a[contains(@href, "cdReferentiel")]')[0] inv.id = unicode(re.search('cdReferentiel=(.*)', link.attrib['href']).group(1)) inv.code = re.match('^[A-Z]+[0-9]+(.*)$', inv.id).group(1) inv.quantity = self.parse_decimal(cols[self.COL_QUANTITY], True) inv.unitprice = self.parse_decimal(cols[self.COL_UNITPRICE], True) inv.unitvalue = self.parse_decimal(cols[self.COL_UNITVALUE], False) inv.valuation = self.parse_decimal(cols[self.COL_VALUATION], True) diff = cols[self.COL_PERF].text.strip() if diff == "-": inv.diff = NotAvailable else: inv.diff = CleanDecimal(None, replace_dots=True).filter(diff) yield inv def parse_decimal(self, string, replace_dots): string = CleanText(None).filter(string) if string == '-': return NotAvailable return CleanDecimal(None, replace_dots=replace_dots, default=NotAvailable).filter(string) def select_period(self): return True def get_operations(self, account): return iter([]) class InvestmentHistoryPage(LoggedPage, HTMLPage): COL_LABEL = 0 COL_QUANTITY = 1 COL_UNITVALUE = 2 COL_DATE = 3 COL_VALUATION = 4 COL_WEIGHT = 5 COL_UNITPRICE = 6 COL_PERF = 7 COL_PERF_PERCENT = 8 def get_investments(self): for line in self.doc.xpath('//table[@id="tableau_support"]/tbody/tr'): cols = line.findall('td') inv = Investment() inv.id = unicode(re.search('cdReferentiel=(.*)', cols[self.COL_LABEL].find('a').attrib['href']).group(1)) inv.code = re.match('^[A-Z]+[0-9]+(.*)$', inv.id).group(1) inv.label = CleanText(None).filter(cols[self.COL_LABEL]) inv.quantity = self.parse_decimal(cols[self.COL_QUANTITY]) inv.unitprice = self.parse_decimal(cols[self.COL_UNITPRICE]) inv.unitvalue = self.parse_decimal(cols[self.COL_UNITVALUE]) inv.valuation = self.parse_decimal(cols[self.COL_VALUATION]) inv.diff = self.parse_decimal(cols[self.COL_PERF]) yield inv def parse_decimal(self, string): string = CleanText(None).filter(string) if string == '-': return NotAvailable return CleanDecimal(None, replace_dots=True).filter(string) def select_period(self): self.browser.location(self.url.replace('portefeuille-assurance-vie.jsp', 'operations/assurance-vie-operations.jsp')) assert isinstance(self.browser.page, type(self)) try: form = self.browser.page.get_form(name='OperationsForm') except FormNotFound: return False form['dateDebut'] = (date.today() - relativedelta(years=1)).strftime('%d/%m/%Y') form['nbrEltsParPage'] = '100' form.submit() return True def get_operations(self, account): for tr in self.doc.xpath('//table[@id="tableau_histo_opes"]/tbody/tr'): tds = tr.findall('td') t = Transaction() t.parse(date=CleanText(None).filter(tds[1]), raw=CleanText(None).filter(tds[2])) t.amount = CleanDecimal(None, replace_dots=True, default=0).filter(tds[-1]) yield t class AccountHistoryPage(LoggedPage, HTMLPage): def get_investments(self): return iter([]) def select_period(self): form = self.get_form(name='ConsultationHistoriqueOperationsForm') form['dateRechercheDebut'] = (date.today() - relativedelta(years=1)).strftime('%d/%m/%Y') form['nbrEltsParPage'] = '100' form.submit() return True def get_operations(self, account): """history, see http://docs.weboob.org/api/capabilities/bank.html?highlight=transaction#weboob.capabilities.bank.Transaction""" # TODO need to rewrite that with FrenchTransaction class http://tinyurl.com/6lq4r9t tables = self.doc.findall(".//*[@id='tabHistoriqueOperations']/tbody/tr") if len(tables) == 0: return for i in range(len(tables)): operation = Transaction() operation.type = 0 operation.category = NotAvailable date_oper = tables[i].xpath("./td[2]/text()")[0] date_val = tables[i].xpath("./td[3]/text()")[0] label = tables[i].xpath("./td[4]/text()")[0] label = re.sub(r'[ \xa0]+', ' ', label).strip() amount = tables[i].xpath("./td[5]/text() | ./td[6]/text()") operation.parse(date=date_oper, raw=label, vdate=date_val) if amount[1] == u'\xa0': amount = amount[0] else: amount = amount[1] operation.amount = CleanDecimal(None).filter(amount) yield operation class CardHistoryPage(LoggedPage, HTMLPage): def get_investments(self): return iter([]) def select_period(self): return True def get_operations(self, account): cleaner = CleanText(None).filter for op in self.doc.xpath('//table[@id="tableauEncours"]/tbody/tr'): rdate = cleaner(op.xpath('./td[1]')[0]) date = cleaner(op.xpath('./td[2]')[0]) raw = cleaner(op.xpath('./td[3]')[0]) credit = cleaner(op.xpath('./td[4]')[0]) debit = cleaner(op.xpath('./td[5]')[0]) tr = Transaction() tr.parse(date=date, raw=raw) tr.rdate = tr.parse_date(rdate) tr.type = tr.TYPE_CARD if credit: tr.amount = CleanDecimal(None, replace_dots=True).filter(credit) elif debit: tr.amount = -abs(CleanDecimal(None, replace_dots=True).filter(debit)) yield tr class AccountsList(LoggedPage, HTMLPage): def on_load(self): warn = self.doc.xpath('//div[@id="message_renouvellement_mot_passe"] | \ //span[contains(text(), "Votre identifiant change")] | \ //span[contains(text(), "Nouveau mot de passe")] | \ //span[contains(text(), "Renouvellement de votre mot de passe")]') if len(warn) > 0: raise ActionNeeded(warn[0].text) self.load_async(0) def load_async(self, time): total = 0 restart = True while restart: restart = False # load content of loading divs. lst = self.doc.xpath('//input[@type="hidden" and starts-with(@id, "asynch")]') if len(lst) > 0: params = {} for i, input in enumerate(lst): params['key%s' % i] = input.attrib['name'] params['div%s' % i] = input.attrib['value'] params['time'] = time r = self.browser.open('/AsynchAjax?%s' % urlencode(params)) data = json.loads(r.content) for i, d in enumerate(data['data']): div = self.doc.xpath('//div[@id="%s"]' % d['key'])[0] html = d['flux'] div.clear() div.attrib['id'] = d['key'] # needed because clear removes also all attributes div.insert(0, etree.fromstring(html, parser=etree.HTMLParser())) if 'time' in data: wait = float(data['time'])/1000.0 self.logger.debug('should wait %f more seconds', wait) total += wait if total > 120: raise BrowserUnavailable('too long time to wait') sleep(wait) restart = True def need_reload(self): form = self.doc.xpath('//form[@name="InformationsPersonnellesForm"]') return len(form) > 0 def need_sms(self): return len(self.doc.xpath('//div[@id="aidesecuforte"]')) ACCOUNT_TYPES = {'mes-comptes/compte-courant/consulter-situation': Account.TYPE_CHECKING, 'mes-comptes/compte-courant/carte-bancaire': Account.TYPE_CARD, 'mes-comptes/assurance-vie': Account.TYPE_LIFE_INSURANCE, 'mes-comptes/livret': Account.TYPE_SAVINGS, 'mes-comptes/pea': Account.TYPE_MARKET, 'mes-comptes/compte-titres': Account.TYPE_MARKET, } def get_list(self): accounts = [] account = None for cpt in self.doc.xpath('//a[@class="synthese_id_compte" or @class="synthese_carte_differe"]'): url_to_parse = cpt.xpath('@href')[0].replace("\n", "") # link # account._link_id = lien vers historique d'un compte (courant ou livret) if '/mes-comptes/livret/' in url_to_parse: compte_id_re = re.compile(r'.*\?(.*)$') link_id = '/fr/prive/mes-comptes/livret/consulter-situation/consulter-solde.jsp?%s' % \ (compte_id_re.search(url_to_parse).groups()[0]) else: link_id = url_to_parse number = cpt.xpath('./span[@class="synthese_numero_compte"]') if len(number) == 0: account._card_links.append(link_id) continue account = Account() account.id = CleanText(None).filter(number[0]).replace(u'N°', '') try: balance = CleanText(None).filter(cpt.xpath('./span[contains(@class, "synthese_solde")]')[0]) except IndexError: continue account.balance = CleanDecimal(None, replace_dots=True).filter(balance) account.currency = account.get_currency(balance) account._link_id = link_id account._card_links = [] account.label = (' '.join([CleanText.clean(part) for part in cpt.xpath('./text()')])).strip(' - ').strip() for pattern, type in self.ACCOUNT_TYPES.iteritems(): if pattern in account._link_id: account.type = type if account.type in (Account.TYPE_CHECKING, Account.TYPE_SAVINGS): # Need a token sent by SMS to customers account.iban = NotAvailable if (account.label, account.id, account.balance) not in [(a.label, a.id, a.balance) for a in accounts]: accounts.append(account) return iter(accounts) class GlobalAccountsList(LoggedPage, HTMLPage): pass weboob-1.2/modules/fortuneo/pages/login.py000066400000000000000000000024471303450110500207220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . #from logging import error from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import CleanText from weboob.exceptions import BrowserUnavailable class LoginPage(HTMLPage): def login(self, login, passwd): msg = CleanText(".//*[@id='message_client']/text()")(self.doc) if "maintenance" in msg: raise BrowserUnavailable(msg) form = self.get_form(name="acces_identification") form['login'] = login.encode('utf-8') form['passwd'] = passwd.encode('utf-8') form.submit() # vim:ts=4:sw=4 weboob-1.2/modules/fortuneo/test.py000066400000000000000000000020101303450110500174540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Gilles-Alexandre Quenot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class FortuneoTest(BackendTest): MODULE = 'fortuneo' def test_fortuneo(self): l = list(self.backend.iter_accounts()) self.assertTrue(len(l) > 0) a = l[0] list(self.backend.iter_history(a)) # vim:ts=4:sw=4 weboob-1.2/modules/fourchan/000077500000000000000000000000001303450110500160765ustar00rootroot00000000000000weboob-1.2/modules/fourchan/__init__.py000066400000000000000000000001531303450110500202060ustar00rootroot00000000000000from .module import FourChanModule from .browser import FourChan __all__ = ['FourChanModule', 'FourChan'] weboob-1.2/modules/fourchan/browser.py000066400000000000000000000025501303450110500201350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser from .pages.board import BoardPage class FourChan(Browser): DOMAIN = 'boards.4chan.org' PAGES = { 'http://boards.4chan.org/\w+/': BoardPage, 'http://boards.4chan.org/\w+/res/\d+': BoardPage, } def is_logged(self): return True def get_threads(self, board): self.location('http://boards.4chan.org/%s/' % board) return self.page.articles def get_thread(self, board, id): self.location('http://boards.4chan.org/%s/res/%d' % (board, long(id))) assert len(self.page.articles) == 1 return self.page.articles[0] weboob-1.2/modules/fourchan/favicon.png000066400000000000000000000106641303450110500202400ustar00rootroot00000000000000PNG  IHDR@@iqsRGB pHYs  tIME 3)FIDATx[iU~mwttN IhQ"3PcLC*T1UԀ82Ha R(#LXDIL ӡww͏,KMU]>{87,2yPb8F[A1x H_Y\VTUUuF|_uʬRz:~$Kү92tَsd[C?ޫʅ{F" `[WxjL:gu(mBO 1T>ta>O=zEĐ]V-M[a R7PzȺ5wr֥@Qf_ŏO#_Y)xΆ#-tEh(UY=#72_uԟsvId60P55*3o;wu<4dzG!N@芺⏛" ue,NFh^0h9"i 'IݔԬoIsCN1! b(_mio`mƘAGޖun]|Fx5*;u]f݌s6xeI ZYiBqXXKi܄rc`I~Š5V*LJȝ#ĐxDdc[i(d<(C}4H zMXuʼh UI x ¢~,i¬V@]Z}<etFtܘ#{F>zHDM@ NɋZFb(\dM] q6su,mt]'C) E(_N;gzLoQ/;]P@$!Ԙ(yX䯇qIVΪoY NF4K2Pq~:0q&.b(K2p_A=Z3S `&`1$Ht*wߗ7_^ " mV;=.ˆ=h_C Fww"CX 㕇"Ԟ6~pLpugcFnC#w t|*;=rH4- O[0ʠT@`G x=G\huýHNt !8nb(($a2`a96aw 6Md K&2оOlZ`*ĥdmrH`gШT'i PYD@Fy\OkEYWZs\ej ^ w(hJK*Vsg0SY*` Ş{<^){qC {d oĖ UbdqH}(lI\Z9 _^(1s]yrʂآJ/3.Ho,=OW'>|Ru H,qAvsvG`py` b-Q(4?40ji˕;[iS1{4ވ,mq?b:оZ^<F2q,c Էդx%:jدY7HΒ½jzM=E8aс8=.*XlͼCv@ ۰6F|*1p½TY}y:f_/cw~'kPY:wzwl֡p\K쭱M(`_C(k" <=6ʪkӵkuOл~_Z\cP%W\Xtg aju,E#\nq!҃5e쇉e8*wLU`>z׋sͅ{6F퓕pl~CsiimΘn{#بtlQ" &~F%y2fדOu!;;E89+&jw[ڬo9f-}n5&Q\$Wힵ;{=o@*Ab͉, H0]W`m|P F%5%6M8nq}-p}c^JG'Hw|*!y4~N4F<u(TET%TT%\?cX/v]Wġi$ zWk2į64s͂5F0hXљs)x241y˽CfSf]ӺU. 1,ߟs<,rsj:_q=Ӿq/L^ oY &pOxhCX!vvb2(4NpW@Zߛϳ>0 S͛jMVggyd"Ԩ xx&> `Y3afMB J)҇lġ M N[FjIPsg.Mt'uMg -94и9zӁL[upL+LU2n׿TbOЦT}0eSQ g+WyZ6xbU&%@ =++w[wݎaqFE{1pOoi^KĊf]Rwg|}7%1q} 1j67VgI㸙BnmΌpvk2J~CT[ `qB *CѠ*˧L`6;^;$AUlؑ0Yd-{wȲ>;6gHd w_4] ,\ߗ(?Z6j* ڸVFA, huac/?Z?YxĜ06sY 倥Gu쳮B]y8IxY=_E2W 4-4 M5XA46sM&qd^mno"psxxh@55ޔRcBȼ g"L/i_dϟ=uZ1$8F/ OqNC Bʡ*<|@}S&23Ѧ|qi^Ɍ,JCьfCA'pvg۾%/DP`ZC4 0 ? q(*I4xB?TC$&%,UuUVpYP5@ t h) d5q mgz`>gsk)8_:(>iσ5j6Ö́pdu0AC'pz\ΊwA,2F. from weboob.capabilities.messages import CapMessages, Message, Thread from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from .browser import FourChan __all__ = ['FourChanModule'] class FourChanModule(Module, CapMessages): NAME = 'fourchan' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = '4chan image board' CONFIG = BackendConfig(Value('boards', label='Boards to fetch')) STORAGE = {'boards': {}} BROWSER = FourChan def _splitid(self, id): return id.split('.', 1) def get_thread(self, id): thread = None if isinstance(id, Thread): thread = id id = thread.id if '.' not in id: self.logger.warning('Malformated ID (%s)' % id) return board, thread_id = self._splitid(id) with self.browser: _thread = self.browser.get_thread(board, thread_id) flags = 0 if _thread.id not in self.storage.get('boards', board, default={}): flags |= Message.IS_UNREAD if not thread: thread = Thread(id) thread.title = _thread.filename thread.root = Message(thread=thread, id=0, # root message title=_thread.filename, sender=_thread.author, receivers=None, date=_thread.datetime, parent=None, content=_thread.text, signature=None, children=[], flags=flags|Message.IS_HTML) for comment in _thread.comments: flags = 0 if comment.id not in self.storage.get('boards', board, _thread.id, default=[]): flags |= Message.IS_UNREAD m = Message(thread=thread, id=comment.id, title=_thread.filename, sender=comment.author, receivers=None, date=comment.datetime, parent=thread.root, content=comment.text, signature=None, children=None, flags=flags|Message.IS_HTML) thread.root.children.append(m) return thread def iter_threads(self): for board in self.config['boards'].get().split(' '): with self.browser: threads = self.browser.get_threads(board) for thread in threads: t = Thread('%s.%s' % (board, thread.id)) t.title = thread.filename yield t def iter_unread_messages(self): for thread in self.iter_threads(): self.fill_thread(thread, 'root') for m in thread.iter_all_messages(): if m.flags & Message.IS_UNREAD: yield m def set_message_read(self, message): board, thread_id = self._splitid(message.thread.id) self.storage.set('boards', board, thread_id, self.storage.get('boards', board, thread_id, default=[]) + [message.id]) self.storage.save() def fill_thread(self, thread, fields): return self.get_thread(thread) OBJECTS = {Thread: fill_thread} weboob-1.2/modules/fourchan/pages/000077500000000000000000000000001303450110500171755ustar00rootroot00000000000000weboob-1.2/modules/fourchan/pages/__init__.py000066400000000000000000000000001303450110500212740ustar00rootroot00000000000000weboob-1.2/modules/fourchan/pages/board.py000066400000000000000000000061751303450110500206470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import datetime from weboob.deprecated.browser import Page class Message(object): def __init__(self, browser, board, id, filename=u'', url=u''): self.id = id self.browser = browser self.board = board self.filename = filename self.datetime = datetime.now() self.url = url self.author = u'' self.text = u'' self.comments = [] def add_comment(self, div): comment = Message(self.browser, self.board, int(div.attrib.get('id', ''))) comment.author = div.cssselect('span.commentpostername')[0].text comment.text = self.browser.parser.tostring(div.find('blockquote')) self.comments.append(comment) def __repr__(self): return '' % (self.id, self.filename, self.url, len(self.comments)) class BoardPage(Page): URL_REGEXP = re.compile('http://boards.4chan.org/(\w+)/') def on_loaded(self): self.articles = [] m = self.URL_REGEXP.match(self.url) if m: self.board = m.group(1) else: self.logger.warning('Unable to find board') self.board = 'unknown' forms = self.document.getroot().cssselect('form') form = None for f in forms: if f.attrib.get('name', '') == 'delform': form = f break if form is None: self.logger.warning('No delform :(') article = None for div in form.getchildren(): if div.tag == 'span' and div.attrib.get('class', '') == 'filesize': url = div.find('a').get('href', '') filename = 'unknown.jpg' span = div.find('span') if span is not None: filename = span.text article = Message(self.browser, self.board, 0, filename, url) self.articles.append(article) if article is None: continue if div.tag == 'input' and div.attrib.get('type', 'checkbox') and div.attrib.get('value', 'delete'): article.id = int(div.attrib.get('name', '0')) if div.tag == 'blockquote': article.text = self.parser.tostring(div) if div.tag == 'table': tags = div.cssselect('td.reply') if tags: article.add_comment(tags[0]) weboob-1.2/modules/fourchan/test.py000066400000000000000000000025371303450110500174360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from logging import debug from weboob.tools.test import BackendTest class FourChanTest(BackendTest): MODULE = 'fourchan' def test_new_messages(self): tot = 0 for thread in self.backend.iter_threads(): thread = self.backend.fillobj(thread, 'root') count = 0 for m in thread.iter_all_messages(): count += 1 debug('Count: %s' % count) tot += count debug('Total messages: %s' % tot) count = 0 for message in self.backend.iter_unread_messages(): count += 1 debug('Unread messages: %s' % count) weboob-1.2/modules/francetelevisions/000077500000000000000000000000001303450110500200145ustar00rootroot00000000000000weboob-1.2/modules/francetelevisions/__init__.py000066400000000000000000000000731303450110500221250ustar00rootroot00000000000000from .module import PluzzModule __all__ = ['PluzzModule'] weboob-1.2/modules/francetelevisions/browser.py000066400000000000000000000060031303450110500220500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import IndexPage, VideoPage, Programs, VideoListPage, LatestPage, FrancetvinfoPage __all__ = ['PluzzBrowser'] class PluzzBrowser(PagesBrowser): ENCODING = 'utf-8' BASEURL = 'http://pluzz.francetv.fr' PROGRAMS = None francetvinfo = URL(r'http://www.francetvinfo.fr/(?P.*)', FrancetvinfoPage) latest = URL(r'http://pluzz.webservices.francetelevisions.fr/pluzz/liste/type/replay', LatestPage) programs_page = URL(r'http://pluzz.webservices.francetelevisions.fr/pluzz/programme', Programs) index_page = URL(r'recherche\?recherche=(?P.*)', IndexPage) video_page = URL(r'http://webservices.francetelevisions.fr/tools/getInfosOeuvre/v2/\?idDiffusion=(?P.*)&catalogue=Pluzz', VideoPage) videos_list_page = URL(r'(?Pvideos/.*)', VideoListPage) def get_video_id_from_francetvinfo(self, url): return self.francetvinfo.go(url=url).get_video_id_from_francetvinfo() def get_video_from_url(self, url): video = self.videos_list_page.go(program=url).get_last_video() if video: return self.get_video(video.id, video) def search_videos(self, pattern): if not self.PROGRAMS: self.PROGRAMS = list(self.get_program_list()) videos = [] for program in self.PROGRAMS: if pattern.upper() in program._title.upper(): video = self.videos_list_page.go(program=program.id).get_last_video() if video: videos.append(video) videos += list(self.page.iter_videos()) return videos if len(videos) > 0 else self.index_page.go(pattern=pattern).iter_videos() def get_program_list(self): return list(self.programs_page.go().iter_programs()) @video_page.id2url def get_video(self, url, video=None): self.location(url) video = self.page.get_video(obj=video) if not video: return for item in self.read_url(video.url): video.url = u'%s' % item return video def read_url(self, url): r = self.open(url, stream=True) buf = r.iter_lines() return buf def latest_videos(self): return self.latest.go().iter_videos() weboob-1.2/modules/francetelevisions/favicon.png000066400000000000000000000026151303450110500221530ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME :'?tEXtCommentCreated with GIMPWIDATx=LGఝDrPΆ(Q\@&EH YNL" q.(d)"m8"A)M$ H e,H&nۜ9/ R.;{Ozڛ737yޛ]8C)N`Z`I@f $,g9\jF h>NMN]4cϵ8-_ ,5r/9𞣁u%6/O t 9eZPn60|m{TmlҀ[hDCc0ŬTWƺ/FoR:~ xÙaoF!rcWDZ[%+gtk?f"Q n #qR)/׮yqX^+W~.8wfgCOƦCO` .:3"ٿGFDD._Y\R)vG켅;wdi% Ғٳ"{dpHcudRd|8"x.۷E,+[hP9~&ЪV  A/Ar xy8RN󸔰 F U`j{$~ m#GJ aNؘ+o m1Z ڀ[##;ܼYwPP\:q‹],3ٲ)Gz40\R~Oaj*[vtp)XYQ=fNPb?[vuU$ɕD2Ok7H44+74-bn d012MLeb5@#[޲D\cޞQLݻ{zżt,f4?ssBJjWbe:oAO}e[h`{&03ō?(}^q P5Dsc>/=r߲' vTJ&I%r`۞''mTWL.f tb%pqOxnx>ٗJnLy6 Kh(. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module from .browser import PluzzBrowser import re __all__ = ['PluzzModule'] class PluzzModule(Module, CapVideo, CapCollection): NAME = 'francetelevisions' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'France Télévisions video website' LICENSE = 'AGPLv3+' BROWSER = PluzzBrowser def get_video(self, _id): m = re.match('http://pluzz.francetv.fr/(videos/.*)', _id) if m: return self.browser.get_video_from_url(m.group(1)) m2 = re.match('http://www.francetvinfo.fr/(.*)', _id) if m2: _id = self.browser.get_video_id_from_francetvinfo(m2.group(1)) if not _id: return return self.browser.get_video(_id) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest France Télévisions videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {BaseVideo: fill_video} weboob-1.2/modules/francetelevisions/pages.py000066400000000000000000000134511303450110500214710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import BaseVideo from weboob.capabilities.base import BaseObject from datetime import timedelta from weboob.browser.pages import HTMLPage, JsonPage from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.filters.standard import Filter, CleanText, Regexp, Format, DateTime, Env, Duration from weboob.browser.filters.html import Link, Attr from weboob.browser.filters.json import Dict class DurationPluzz(Filter): def filter(self, el): duration = Regexp(CleanText('.'), r'.+\|(.+)')(el[0]) if duration[-1:] == "'": t = [0, int(duration[:-1])] else: t = map(int, duration.split(':')) return timedelta(hours=t[0], minutes=t[1]) class FrancetvinfoPage(HTMLPage): def get_video_id_from_francetvinfo(self): return Regexp(CleanText('//a[@id="catchup"]/@href'), 'http://info.francetelevisions.fr/\?id-video=(.*)@Info-web', default=None)(self.doc) class VideoListPage(HTMLPage): @method class get_last_video(ItemElement): klass = BaseVideo obj_id = CleanText('//div[@id="diffusion-info"]/@data-diffusion') obj_title = CleanText('//div[@id="diffusion-info"]/h1/div[@id="diffusion-titre"]') obj_date = DateTime(Regexp(CleanText('//div[@id="diffusion-info"]/h1|//div[@id="diffusion-info"]/div/div/*[1]', replace=[(u'à', u''), (u' ', u' ')]), '.+(\d{2}-\d{2}-\d{2}.+\d{1,2}h\d{1,2}).+'), dayfirst=True) @method class iter_videos(ListElement): item_xpath = '//div[@id="player-memeProgramme"]/a' class item(ItemElement): klass = BaseVideo def condition(self): return CleanText('div[@class="autre-emission-c3"]')(self) == "En replay" obj_id = Regexp(Link('.'), '^/videos/.+,(.+).html$') obj_title = CleanText('//meta[@name="programme_titre"]/@content') obj_date = DateTime(Regexp(CleanText('./div[@class="autre-emission-c2"]|./div[@class="autre-emission-c4"]', replace=[(u'à', u''), (u' ', u' ')]), '(\d{2}-\d{2}.+\d{1,2}:\d{1,2})'), dayfirst=True) class IndexPage(HTMLPage): @method class iter_videos(ListElement): item_xpath = '//div[@class="panel-resultat panel-separateur"]' ignore_duplicate = True class item(ItemElement): klass = BaseVideo obj_title = Format('%s du %s', CleanText('div/div[@class="resultat-titre-diff"]/a'), Regexp(CleanText('div/div[@class="resultat-soustitre-diff"]'), '.+(\d{2}-\d{2}-\d{2}).+')) obj_id = Regexp(Link('div/div[@class="resultat-titre-diff"]/a'), '^/videos/.+,(.+).html$') obj_date = DateTime(Regexp(CleanText('div/div[@class="resultat-soustitre-diff"]', replace=[(u'à', u''), (u' ', u' ')]), '.+(\d{2}-\d{2}-\d{2}.+\d{1,2}h\d{1,2}).+')) obj_duration = DurationPluzz('div/div[3]') def obj_thumbnail(self): url = Attr('a/img[@class="resultat-vignette"]', 'data-src')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail class VideoPage(JsonPage): @method class get_video(ItemElement): klass = BaseVideo def validate(self, obj): return obj.url def parse(self, el): for video in el['videos']: if video['format'] != 'm3u8-download': continue self.env['url'] = video['url'] obj_id = Env('id') obj_title = Format(u'%s - %s', Dict['titre'], Dict['sous_titre']) obj_url = Env('url') obj_date = Dict['diffusion']['date_debut'] & DateTime obj_duration = Dict['duree'] & Duration obj_description = Dict['synopsis'] obj_ext = u'm3u8' def obj_thumbnail(self): url = Format('http://www.francetv.fr%s', Dict['image'])(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail class Programs(JsonPage): @method class iter_programs(DictElement): item_xpath = 'reponse/programme' class item(ItemElement): klass = BaseObject obj_id = CleanText(Dict('url')) obj__title = CleanText(Dict('titre_programme')) class LatestPage(JsonPage): @method class iter_videos(DictElement): item_xpath = 'reponse/emissions' class Item(ItemElement): klass = BaseVideo obj_id = Dict('id_diffusion') obj_title = Dict('titre_programme') obj_date = DateTime(Dict('date_diffusion')) weboob-1.2/modules/francetelevisions/test.py000066400000000000000000000032261303450110500213500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo from weboob.tools.test import BackendTest class PluzzTest(BackendTest): MODULE = 'francetelevisions' def test_search(self): # If the test fails, it might be good news! l = list(self.backend.search_videos("pblv")) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_video_from_url(self): v = self.backend.get_video('http://pluzz.francetv.fr/videos/plus_belle_la_vie.html') self.assertTrue(v.url, 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) assert len(l) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url, 'URL for video "%s" not found' % (v.id)) weboob-1.2/modules/freemobile/000077500000000000000000000000001303450110500164025ustar00rootroot00000000000000weboob-1.2/modules/freemobile/__init__.py000066400000000000000000000014471303450110500205210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import FreeMobileModule __all__ = ['FreeMobileModule'] weboob-1.2/modules/freemobile/browser.py000066400000000000000000000047141303450110500204450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import HomePage, LoginPage, HistoryPage, DetailsPage __all__ = ['Freemobile'] class Freemobile(LoginBrowser): BASEURL = 'https://mobile.free.fr/moncompte/' homepage = URL('index.php\?page=home', HomePage) detailspage = URL('index.php\?page=suiviconso', DetailsPage) loginpage = URL('index.php', LoginPage) historypage = URL('ajax.php\?page=consotel_current_month', HistoryPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.username.isdigit() self.loginpage.stay_or_go().login(self.username, self.password) self.homepage.go() if self.loginpage.is_here(): raise BrowserIncorrectPassword() @need_login def get_subscription_list(self): subscriptions = self.homepage.stay_or_go().get_list() self.detailspage.go() for subscription in subscriptions: subscription._login = self.page.get_login(subscription.id) subscription._virtual = self.page.load_virtual(subscription.id) subscription.renewdate = self.page.get_renew_date(subscription) yield subscription def get_history(self, subscription): self.historypage.go(data={'login': subscription._login}) return sorted([x for x in self.page.get_calls()], key=lambda self: self.datetime, reverse=True) def get_details(self, subscription): return self.detailspage.stay_or_go().get_details(subscription) def iter_documents(self, subscription): return self.detailspage.stay_or_go().date_bills(subid=subscription.id) weboob-1.2/modules/freemobile/favicon.png000066400000000000000000000027011303450110500205350ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs B(xtIME Em^tEXtCommentCreated with GIMPWIDATx[lTUU jI+h0AK& -T5bB"F5`L4M1(ER-&*RXPE/ѠTxX/\Wf̙)̜u֙9k]g """"""""""TD ;@%p=𐂿z=lrFD@,w2=?)mԛ!PC_ GjKt@X<:^ U԰F*U /USuӁ<^}@ X\p.;},gW<*[m+a.bBi q_ ~roYp1p2 xΔ.^ŀkE}xmeGo"Qh8L[ 6{aD~6z:p5LJ>HCVY;**8 pHq$[ 9ne~0?~MU1`k1~*13`3@R>a1~+p p_@4lc!A^Ӂ-xy `ZXLUS\n?BP 3I)&<.+f`2p/Qp4>rIDnߙm 3yEwYf{7ŀU]#ƻ@#<a&wC"Lv h;F9Gd|ߔ:Ra 9<{{x wUVgxd@2L fɊ;4=.V= ]eV1zN#, TfK#;wPS`(0Ww1Կ1]Z|.4kܒ;@XBl 䍃cG7fJ%w0-P:6oS G֦,_@)*kl>հK;;SJ#6r\WaN%.0!vVJuB))n[_#kQ!&̏KF#Ogi?&$`g<_S@/ |OK^)Ȝ,Ƅ. from weboob.capabilities.bill import CapDocument, Subscription, Bill, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Freemobile __all__ = ['FreeMobileModule'] class FreeMobileModule(Module, CapDocument): NAME = 'freemobile' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Free Mobile website' CONFIG = BackendConfig(ValueBackendPassword('login', label='Account ID', masked=False, regexp='^(\d{8}|)$'), ValueBackendPassword('password', label='Password') ) BROWSER = Freemobile def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def iter_documents_history(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.get_history(subscription) def get_document(self, _id): subid = _id.split('.')[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def get_details(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.get_details(subscription) def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) return self.browser.open(bill.url).content weboob-1.2/modules/freemobile/pages/000077500000000000000000000000001303450110500175015ustar00rootroot00000000000000weboob-1.2/modules/freemobile/pages/__init__.py000066400000000000000000000016131303450110500216130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .homepage import HomePage from .history import HistoryPage, DetailsPage from .login import LoginPage __all__ = ['LoginPage', 'HomePage', 'HistoryPage', 'DetailsPage'] weboob-1.2/modules/freemobile/pages/history.py000066400000000000000000000143011303450110500215530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import calendar from datetime import datetime from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Date, CleanText, Filter,\ CleanDecimal, Regexp, Field, DateTime, Format, Env from weboob.browser.filters.html import Attr from weboob.capabilities.bill import Detail, Bill class FormatDate(Filter): def filter(self, txt): return datetime.strptime(txt, "%Y%m%d").date() class BadUTF8Page(HTMLPage): ENCODING = 'UTF-8' class DetailsPage(LoggedPage, BadUTF8Page): def load_virtual(self, phonenumber): for div in self.doc.xpath('//div[@class="infosLigne pointer"]'): if CleanText('.')(div).split("-")[-1].strip() == phonenumber: return Attr('.', 'onclick')(div).split('(')[1][1] def on_load(self): self.details = {} for div in self.doc.xpath('//div[@class="infosConso"]'): num = div.attrib['id'].split('_')[1][0] self.details[num] = [] # National parsing divnat = div.xpath('div[@class="national"]')[0] self._parse_div(divnat, "National : %s | International : %s", num, False) # International parsing divint = div.xpath('div[@class="international hide"]')[0] if divint.xpath('div[@class="detail"]'): self._parse_div(divint, u"Appels émis : %s | Appels reçus : %s", num, True) def _parse_div(self, divglobal, string, num, inter=False): divs = divglobal.xpath('div[@class="detail"]') # Two pieces of information in one div... div = divs.pop(0) voice = self._parse_voice(div, string, num, inter) self.details[num].append(voice) self._iter_divs(divs, num, inter) def _iter_divs(self, divs, num, inter=False): for div in divs: detail = Detail() detail.label = CleanText('div[@class="titre"]/p')(div) detail.id = "-" + detail.label.split(' ')[1].lower() if inter: detail.label = detail.label + u" (international)" detail.id = detail.id + "-inter" detail.infos = CleanText('div[@class="conso"]/p')(div) detail.price = CleanDecimal('div[@class="horsForfait"]/p/span', default=Decimal(0), replace_dots=True)(div) self.details[num].append(detail) def _parse_voice(self, div, string, num, inter=False): voicediv = div.xpath('div[@class="conso"]')[0] voice = Detail() voice.id = "-voice" voice.label = CleanText('div[@class="titre"]/p')(div) if inter: voice.label = voice.label + " (international)" voice.id = voice.id + "-inter" voice.price = CleanDecimal('div[@class="horsForfait"]/p/span', default=Decimal(0), replace_dots=True)(div) voice1 = CleanText('.//span[@class="actif"][1]')(voicediv) voice2 = CleanText('.//span[@class="actif"][2]')(voicediv) voice.infos = unicode(string) % (voice1, voice2) return voice # XXX def get_details(self, subscription): for detail in self.details[subscription._virtual]: detail.id = subscription.id + detail.id yield detail @method class date_bills(ListElement): item_xpath = '//div[has-class("factLigne")]' class item(ItemElement): klass = Bill def condition(self): num = Attr('.', 'data-fact_ligne', default='')(self) return self.env['subid'] == num obj_url = Attr('.//div[@class="pdf"]/a', 'href') obj__localid = Regexp(Field('url'), '&id=(.*)&date', u'\\1') obj_label = Regexp(Field('url'), '&date=(\d*)', u'\\1') obj_id = Format('%s.%s', Env('subid'), Field('_localid')) obj_date = FormatDate(Field('label')) obj_format = u"pdf" obj_type = u"bill" obj_price = CleanDecimal('div[@class="montant"]', default=Decimal(0), replace_dots=False) def get_renew_date(self, subscription): div = self.doc.xpath('//div[@login="%s"]' % subscription._login)[0] mydate = Date(CleanText('.//div[@class="resumeConso"]/span[@class="actif"][1]'), dayfirst=True)(div) if mydate.month == 12: mydate = mydate.replace(month=1) mydate = mydate.replace(year=mydate.year + 1) else: try: mydate = mydate.replace(month=mydate.month + 1) except ValueError: lastday = calendar.monthrange(mydate.year, mydate.month + 1)[1] mydate = mydate.replace(month=mydate.month + 1, day=lastday) return mydate def get_login(self, phonenumber): return Attr('.', 'login')(self.doc.xpath('//div[div[contains(text(), "%s")]]' % phonenumber)[0]) class HistoryPage(LoggedPage, BadUTF8Page): @method class get_calls(ListElement): item_xpath = '//tr' class item(ItemElement): klass = Detail def condition(self): txt = self.el.xpath('td[1]')[0].text return (txt is not None) and (txt != "Date") obj_id = None obj_datetime = DateTime(CleanText('td[1]', symbols=u'à'), dayfirst=True) obj_label = Format(u'%s %s %s', CleanText('td[2]'), CleanText('td[3]'), CleanText('td[4]')) obj_price = CleanDecimal('td[5]', default=Decimal(0), replace_dots=True) weboob-1.2/modules/freemobile/pages/homepage.py000066400000000000000000000030641303450110500216430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .history import BadUTF8Page from weboob.capabilities.bill import Subscription from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Field, Format class HomePage(BadUTF8Page): def is_here(self): if len(self.doc.xpath('//form[@id="form_connect"]')) > 0: return False return True @method class get_list(ListElement): item_xpath = '//div[@class="abonne"]' class item(ItemElement): klass = Subscription obj_subscriber = CleanText('div[@class="idAbonne pointer"]/p[1]', symbols='-', children=False) obj_id = CleanText('div[@class="idAbonne pointer"]/p/span') obj_label = Format(u'%s - %s', Field('id'), CleanText('(.//div[@class="section-contenu"])[1]', children=False)) weboob-1.2/modules/freemobile/pages/login.py000066400000000000000000000112131303450110500211610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time from StringIO import StringIO from PIL import Image from weboob.browser.pages import HTMLPage class FreeKeyboard(object): symbols = {'0': '001111111111110011111111111111111111111111111110000000000011110000000000011111111111111111011111111111111001111111111110', '1': '001110000000000001110000000000001110000000000011111111111111111111111111111111111111111111000000000000000000000000000000', '2': '011110000001111011110000111111111000001111111110000011110011110000111100011111111111000011011111110000011001111000000011', '3': '011100000011110111100000011111111000110000111110000110000011110001110000011111111111111111011111111111110001110001111100', '4': '000000011111000000001111111000000111110011000011110000011000111111111111111111111111111111111111111111111000000000011000', '5': '111111110011110111111110011111111001110000111111001100000011111001100000011111001111111111111001111111111010000111111110', '6': '001111111111110011111111111111111111111111111110001100000011110001100000011111001111111111111101111111111011100111111110', '7': '111000000000000111000000000000111000000011111111000011111111111011111111111111111111000000111111000000000111100000000000', '8': '001110001111110011111111111111111111111111111110000110000011110000110000011111111111111111011111111111111001111001111110', '9': '001111111000110011111111100111111111111100111110000001100011110000001100011111111111111111011111111111111001111111111110' } def __init__(self, basepage): self.basepage = basepage self.fingerprints = [] for htmlimg in self.basepage.doc.xpath('//img[@class="ident_chiffre_img pointer"]'): url = htmlimg.attrib.get("src") imgfile = StringIO(basepage.browser.open(url).content) img = Image.open(imgfile) matrix = img.load() s = "" # The digit is only displayed in the center of image for x in range(15, 23): for y in range(12, 27): (r, g, b) = matrix[x, y] # If the pixel is "red" enough if g + b < 450: s += "1" else: s += "0" self.fingerprints.append(s) def get_symbol_code(self, digit): fingerprint = self.symbols[digit] for i, string in enumerate(self.fingerprints): if string == fingerprint: return i # Image contains some noise, and the match is not always perfect # (this is why we can't use md5 hashs) # But if we can't find the perfect one, we can take the best one best = 0 result = None for i, string in enumerate(self.fingerprints): match = 0 for j, bit in enumerate(string): if bit == fingerprint[j]: match += 1 if match > best: best = match result = i self.basepage.browser.logger.debug(self.fingerprints[result] + " match " + digit) return result def get_string_code(self, string): code = '' for c in string: codesymbol = self.get_symbol_code(c) code += str(codesymbol) return code def get_small(self, string): for c in string: time.sleep(0.5) url = 'https://mobile.free.fr/moncompte/chiffre.php?pos=' + c + '&small=1' self.basepage.browser.open(url) class LoginPage(HTMLPage): def login(self, login, password): vk = FreeKeyboard(self) code = vk.get_string_code(login) vk.get_small(code) # If img are not downloaded, the server do not accept the login form = self.get_form(xpath='//form[@id="form_connect"]') form['login_abo'] = code form['pwd_abo'] = password form.submit() weboob-1.2/modules/freemobile/test.py000066400000000000000000000033661303450110500177430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class FreeMobileTest(BackendTest): MODULE = 'freemobile' def test_details(self): for subscription in self.backend.iter_subscription(): details = list(self.backend.get_details(subscription)) self.assertTrue(len(details) > 4, msg="Not enough details") def test_history(self): for subscription in self.backend.iter_subscription(): self.assertTrue(len(list(self.backend.iter_documents_history(subscription))) > 0) def test_downloadbills(self): """ Iter all bills and try to download it. """ for subscription in self.backend.iter_subscription(): for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) def test_list(self): """ Test listing of subscriptions. """ subscriptions = list(self.backend.iter_subscription()) self.assertTrue(len(subscriptions) > 0, msg="Account listing failed") weboob-1.2/modules/funmooc/000077500000000000000000000000001303450110500157375ustar00rootroot00000000000000weboob-1.2/modules/funmooc/__init__.py000066400000000000000000000014341303450110500200520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import FunmoocModule __all__ = ['FunmoocModule'] weboob-1.2/modules/funmooc/browser.py000066400000000000000000000070301303450110500177740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import unicode_literals from weboob.browser import LoginBrowser, URL, need_login from weboob.capabilities.image import Thumbnail from .pages import PageLogin, PageDashboard, PageChapter, PageSection from .video import MoocVideo import re class FunmoocBrowser(LoginBrowser): BASEURL = 'https://www.fun-mooc.fr' login = URL('/login', PageLogin) dashboard = URL('/dashboard', PageDashboard) course = URL('/courses/(?P[^/]+/[^/]+/[^/]+)/courseware/?$', '/courses/(?P[^/]+/[^/]+/[^/]+)/info/?$', PageChapter) chapter = URL('/courses/(?P[^/]+/[^/]+/[^/]+)/courseware' '/(?P[0-9a-f]+)/$', PageChapter) section = URL('/courses/(?P[^/]+/[^/]+/[^/]+)/courseware/' '(?P[0-9a-f]+)/(?P
[0-9a-f]+)/$', PageSection) file = URL(r'https://d3gzh2mxagd143\.cloudfront\.net/videos/(?P[^/]+)/' r'(?P\w+)\.mp4') def __init__(self, username, password, quality='hd', *args, **kwargs): super(FunmoocBrowser, self).__init__(username, password, *args, **kwargs) self.quality = quality def do_login(self): self.login.stay_or_go() csrf = self.session.cookies.get('csrftoken') self.page.login(self.username, self.password, csrf) self.dashboard.stay_or_go() def get_video(self, _id): if re.search('[^a-zA-Z0-9_-]', _id): match = self.file.match(_id) if not match: return None _id = match.group('id') v = MoocVideo(_id) v.url = self.file.build(id=_id, quality=self.quality) v.ext = 'mp4' v.title = _id return v @need_login def iter_videos(self, course, chapter, section): course = course.replace('-', '/') assert self.section.stay_or_go(course=course, chapter=chapter, section=section) for n, d in enumerate(self.page.iter_videos()): video = self.get_video(d['id']) if d.get('thumbnail'): video.thumbnail = Thumbnail(d['thumbnail']) if d.get('title'): video.title = d['title'] yield video @need_login def iter_sections(self, courseid, chapter): course = courseid.replace('-', '/') assert self.chapter.stay_or_go(course=course, chapter=chapter) for coll in self.page.iter_sections(): if coll.split_path[:2] == [courseid, chapter]: yield coll @need_login def iter_chapters(self, courseid): course = courseid.replace('-', '/') assert self.course.stay_or_go(course=course) return self.page.iter_chapters() @need_login def iter_courses(self): assert self.dashboard.stay_or_go() return self.page.iter_courses() weboob-1.2/modules/funmooc/favicon.png000066400000000000000000000040431303450110500200730ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME KutEXtCommentCreated with GIMPWIDATxlTE?Gz@-G9R@~A rP9&hA$-vH-(#䐻3**bOP-Q+-Pls-Ǽ*d2f3;|Y 2A fg"`$pXix0t)92 Ҁ@_  @59p( DfuϺ&H/Uܭ[Sr~`w0 J\STZ?mK2|'~-3tl@J,A^A  Ӛ SO'lEJ4á߿'XZ< HR.@%P9U1j, lv:< c*Vϙ`fmNž6*xU7?6uf#e%yHz躷}@ݸ8`3@rą8AݸOY]ɯ8f$ڍUkgR9g4t~2x>Zt[2QN|LƒU|:V]bDa,1Q] dhiP)vQsBRsT ʹ :ɤpUΟJ[Ih:wL=鮂E=esB}rJש,Ih|. }T`xY}F?"􊗺ӆGLkUXߧ *)د5--R mëpƀ \B&%,wn]W5rɌ$^Mj}n0`4D*3 Hb?NLo55u?ugs3TaR.+_ֆ>1)L.ޚC=NI8*|\g p "yt5P3hJVM@ݢ7totmfـ؄<`2<7#jYvs77ayW7#vm8C^DP< q8<{À0FKxLxZ7Y뢴 1.~aDFpǞfǝF 4\_N#^@L,E?Fp x֭1KUͩ z~Ҍpb59vju5? ԩJc , "h#zqdX;1|hc.IENDB`weboob-1.2/modules/funmooc/module.py000066400000000000000000000064561303450110500176110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.collection import CapCollection from weboob.capabilities.video import CapVideo, BaseVideo from .browser import FunmoocBrowser __all__ = ['FunmoocModule'] class FunmoocModule(Module, CapVideo, CapCollection): NAME = 'funmooc' DESCRIPTION = u'France-Université-Numérique MOOC website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('email', label='Email', default=''), ValueBackendPassword('password', label='Password', default=''), Value('quality', label='Quality', default='HD', choices=['HD', 'SD', 'LD'])) BROWSER = FunmoocBrowser def create_default_browser(self): quality = self.config['quality'].get().upper() return self.create_browser(self.config['email'].get(), self.config['password'].get(), quality=quality) def get_video(self, _id): return self.browser.get_video(_id) def iter_resources(self, objs, split_path): if len(split_path) == 0: return self.browser.iter_courses() elif len(split_path) == 1: return self.browser.iter_chapters(*split_path) elif len(split_path) == 2: return self.browser.iter_sections(*split_path) elif len(split_path) == 3: return self.browser.iter_videos(*split_path) def _matches(self, title, pattern): title = title.lower() words = pattern.lower().split() return all(word in title for word in words) def search_videos(self, pattern, sortby=0, nsfw=False): queue = [[]] while len(queue): path = queue.pop() for item in self.iter_resources(BaseVideo, path): if isinstance(item, BaseVideo): if self._matches(item.title, pattern): yield item else: # collection newpath = item.split_path if self._matches(item.title, pattern): self.logger.debug('%s matches, returning content', item.title) for item in self.iter_resources_flat(BaseVideo, newpath): yield item return queue.append(newpath) weboob-1.2/modules/funmooc/pages.py000066400000000000000000000107461303450110500174200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import method, ListElement, ItemElement from weboob.capabilities.collection import Collection from weboob.browser.filters.standard import CleanText import re class PageLogin(HTMLPage): def login(self, email, password, csrf): data = {'email': email, 'password': password, 'remember': 'true'} headers = {'X-CSRFToken': csrf} form = self.get_form(xpath='//form[contains(@class,"login-form")]') req = self.browser.build_request(form.url, data=data, headers=headers) self.browser.open(req) class PageDashboard(LoggedPage, HTMLPage): def iter_courses(self): for c in self.doc.xpath('//article[@class="course"]'): title = c.xpath('.//h3[@class="course-title"]/a')[0].text.strip() link = c.xpath('.//a[contains(@class,"enter-course")]')[0] url = self.browser.absurl(link.get('href')) match = self.browser.course.match(url) courseid = match.group('course').replace('/', '-') yield Collection([courseid], title) class PageChapter(LoggedPage, HTMLPage): @method class iter_chapters(ListElement): item_xpath = '//a[has-class("button-chapter")]' class item(ItemElement): klass = Collection obj_title = CleanText('.') def obj_split_path(self): # parse first section link url = self.xpath('./following-sibling::div[has-class("chapter-content-container")]//a')[0].get('href') url = self.page.browser.absurl(url) match = self.page.browser.section.match(url) courseid = self.env['course'].replace('/', '-') chapter = match.group('chapter') return [courseid, chapter] def obj_id(self): return '-'.join(self.obj_split_path()) @method class iter_sections(ListElement): item_xpath = '//div[has-class("chapter-content-container")]//div[has-class("menu-item")]//a' class item(ItemElement): klass = Collection obj_title = CleanText('.') def obj_split_path(self): url = self.xpath('.')[0].get('href') url = self.page.browser.absurl(url) match = self.page.browser.section.match(url) courseid = self.env['course'].replace('/', '-') chapter = match.group('chapter') section = match.group('section') return [courseid, chapter, section] def obj_id(self): return '-'.join(self.obj_split_path()) class PageSection(LoggedPage, HTMLPage): video_url = re.compile(r'[^\s;]+/HD\.mp4', re.I) video_thumb = re.compile(r'reposter="(.*?)"') video_title = re.compile(r'<h2>(.*?)</h2>') def iter_videos(self): urls = set() # this part of the site contains escaped HTML... for n, page_match in enumerate(self.video_url.finditer(self.text)): url = page_match.group(0) match = self.browser.file.match(url) _id = match.group('id') if _id in urls: # prevent duplicate urls continue urls.add(_id) beforetext = self.text[:page_match.end(0)] try: thumb = list(self.video_thumb.finditer(beforetext))[-1].group(1) except IndexError: thumb = None try: title = list(self.video_title.finditer(beforetext))[-1].group(1) except IndexError: title = u'%s - %s' % (_id, n) yield { 'id': _id, 'title': title, 'thumbnail': thumb, } weboob-1.2/modules/funmooc/test.py000066400000000000000000000046121303450110500172730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.collection import Collection from weboob.capabilities.video import BaseVideo from weboob.tools.test import BackendTest, skip_without_config class FunmoocTest(BackendTest): MODULE = 'funmooc' @skip_without_config('email', 'password') def test_basic(self): basic_id = 'FUN-00101-Trimestre_3_2014' courses = list(self.backend.iter_resources([BaseVideo], [])) for course in courses: self.assertIsInstance(course, Collection) if course.split_path == [basic_id]: break else: assert False, 'The default course was not found' videos = list(self.backend.iter_resources_flat([BaseVideo], [basic_id])) for video in videos: self.assertTrue(video) self.assertIsInstance(video, BaseVideo) self.assertTrue(video.id) self.assertTrue(video.url) self.assertTrue(video.title) self.assertTrue(self.backend.browser.open(video.url, method='HEAD')) @skip_without_config('email', 'password') def test_search(self): video = next(self.backend.search_videos('Tester le lecteur HTML 5')) self.assertTrue(video) self.assertIsInstance(video, BaseVideo) self.assertTrue(video.id) self.assertTrue(video.url) self.assertTrue(video.title) videos = list(self.backend.search_videos('Bienvenue sur FUN')) self.assertTrue(videos) for video in videos: self.assertTrue(video) self.assertIsInstance(video, BaseVideo) self.assertTrue(video.id) self.assertTrue(video.url) self.assertTrue(video.title) weboob-1.2/modules/funmooc/video.py000066400000000000000000000016411303450110500174210ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import BaseVideo class MoocVideo(BaseVideo): @classmethod def id2url(cls, _id): return 'https://d3gzh2mxagd143.cloudfront.net/videos/%s/HD.mp4' % _id weboob-1.2/modules/ganassurances/000077500000000000000000000000001303450110500171265ustar00rootroot00000000000000weboob-1.2/modules/ganassurances/__init__.py000066400000000000000000000014461303450110500212440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GanAssurancesModule __all__ = ['GanAssurancesModule'] weboob-1.2/modules/ganassurances/browser.py000066400000000000000000000060011303450110500211600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import ssl from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountsPage, TransactionsPage __all__ = ['GanAssurances'] class GanAssurances(LoginBrowser): login = URL('/wps/portal/login.*', '/wps/portal/inscription.*', LoginPage) accounts = URL('/wps/myportal/TableauDeBord', AccountsPage) transactions = URL('/wps/myportal/!ut.*', TransactionsPage) def __init__(self, website, *args, **kwargs): self.BASEURL = 'https://%s' % website super(GanAssurances, self).__init__(*args, **kwargs) def prepare_request(self, req): """ Gan Assurances does not support SSL anymore. """ preq = super(GanAssurances, self).prepare_request(req) conn = self.session.adapters['https://'].get_connection(preq.url) conn.ssl_version = ssl.PROTOCOL_TLSv1 return preq def do_login(self): """ Attempt to log in. Note: this method does nothing if we are already logged in. """ assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.stay_or_go() self.page.login(self.username, self.password) if self.login.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): self.accounts.stay_or_go() return self.page.get_list() def get_history(self, account): accounts = self.get_accounts_list() for a in accounts: if a.id == account.id: self.location(a._link) assert self.transactions.is_here() return self.page.get_history(accid=account.id) return iter([]) def get_coming(self, account): accounts = self.get_accounts_list() for a in accounts: if a.id == account.id: self.location(a._link) assert self.transactions.is_here() link = self.page.get_coming_link() if link is not None: self.location(self.page.get_coming_link()) assert self.transactions.is_here() return self.page.get_history(accid=account.id) return iter([]) weboob-1.2/modules/ganassurances/favicon.png000066400000000000000000000064001303450110500212610ustar00rootroot00000000000000PNG  IHDR@@iqbKGD{{TS pHYs  tIME  ,$tEXtCommentCreated with GIMPW hIDATxݛmTUFvi^eTfZ2h\t#3] BV$YƄFc@8/tĦJwmh꾜쇮[{gNrR:ys5H FG? ?Mpڀ^- 0 1\* 4$+|Rƿ5 \MG6/YՈ$Qf+@+pp|;f7 \ůUf}_ԕD?`G?lEר&iJ yH$!__$@_}P.}߿3՗6QD x6n Lb`R {CA!!7c #0 R?K1 xPt-y5iȰWR£}  8f DB|q}?FyBA, Z2#5trgL}TpJňU{G<*{a5l |m+݋ל7F l\ΑևJbY$r~ȱb ihh_dkѣGdɒ%466H$hiiaʕٳrL}ǜ9sH&\lڴXc9fyw4M);ƍKZQ5M>U=p 200 뺒%K6L&#CCC288(?R)9sرc駟ʘ>}Z5kHOO8#===z[^cǎI&\.'===uVI&Ⱦ}|rdppPR,tvv妖`Ek]vOk-U Vw%< x衇b۾M7$IP\.7nƴ੧O>9acj*ˊ<3WXw}7)3&r-+]J&a۶mtttB"(Y8w\ /@ϛ7|>?KՒf;wnJ;ﯞw+XuI]0NR+]x'oɓdRR LU- .䫯ѣ̟?obŵ^;ǯ8cJ@`lo;ݶm[ŶBj4uO hhh]^<ϣkWlxb֬YáCf``?e˖OzR ˲D)U櫻b$Y1+PJq]WO>{a̙׳h"vŋ/YF K./˲3g+Wd޽ W>.vwwA[[p+EEcNL0I˗/g޽>}uaݺuXţ>:dxD'R? yIhhhHl2fΜɝwYFs=GSS]w}xj&஻bݜ8qB9p to޼{wg,6m3gΰd6o\vnjj￧ .`KL&C2D)Ō3rGYJ*`OK97Wc?LļUҬX@,];vdx饗jz/L:ftttGV[+ClN|c>Z[[I׏j=~WC+Ďt5I"0=QcשJΝ;?GއE ql9j(UM*a&7x#oƸ|wƑ]NS-jT=8q4g7%8i&% 3Q\"h&HAtv!Z ^hGᎌg~U 1 d&aJC)uN|t~9}Oi4'.zDtff濱cZq}6pv࣌7~W\`^|zt?`&?k*%w}SEZ+$JF?Bl>^UW088ηH@@|Ԡ R`UÖ5) f#>v4AFl /0^>@lu6F onN_^.Ν;7߿7WJ-,뗶mزV!ϗ Q[1()|Dn [ <[qfKt2TVϟO9\.E$ )Y50dcQ0*!}E2Õ?3 o0t>u?yJ=c?+ܐIENDB`weboob-1.2/modules/ganassurances/module.py000066400000000000000000000047111303450110500207700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import ValueBackendPassword, Value from .browser import GanAssurances __all__ = ['GanAssurancesModule'] class GanAssurancesModule(Module, CapBank): NAME = 'ganassurances' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Groupama' LICENSE = 'AGPLv3+' website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'espaceclient.groupama.fr': u'Groupama Banque', 'espaceclient.ganassurances.fr': u'Gan Assurances', }.iteritems(), key=lambda k_v: (k_v[1], k_v[0]))]) CONFIG = BackendConfig(Value('website', label='Banque', choices=website_choices, default='espaceclient.ganassurances.fr'), ValueBackendPassword('login', label=u'Numéro client', masked=False), ValueBackendPassword('password', label=u"Code d'accès")) BROWSER = GanAssurances def create_default_browser(self): return self.create_browser(self.config['website'].get(), self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): return self.browser.get_history(account) def iter_coming(self, account): return self.browser.get_coming(account) weboob-1.2/modules/ganassurances/pages.py000066400000000000000000000120721303450110500206010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal import re, requests from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import method from weboob.browser.filters.standard import Env from weboob.browser.filters.html import Attr from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction class LoginPage(HTMLPage): def login(self, login, passwd): form = self.get_form(name='loginForm') form['LoginPortletFormID'] = login form['LoginPortletFormPassword1'] = passwd form.submit() class AccountsPage(HTMLPage): logged = True ACCOUNT_TYPES = {u'Solde des comptes bancaires - Groupama Banque': Account.TYPE_CHECKING, u'Epargne bancaire constituée - Groupama Banque': Account.TYPE_SAVINGS, } def get_list(self): account_type = Account.TYPE_UNKNOWN accounts = [] for tr in self.doc.xpath('//div[@class="finance"]/form/table[@class="ecli"]/tr'): if tr.attrib.get('class', '') == 'entete': account_type = self.ACCOUNT_TYPES.get(tr.find('th').text.strip(), Account.TYPE_UNKNOWN) continue tds = tr.findall('td') balance = tds[-1].text.strip() if balance == '': continue account = Account() account.label = u' '.join([txt.strip() for txt in tds[0].itertext()]) account.label = re.sub(u'[ \xa0\u2022\r\n\t]+', u' ', account.label).strip() account.id = re.findall('(\d+)', account.label)[0] account.balance = Decimal(FrenchTransaction.clean_amount(balance)) account.currency = account.get_currency(balance) account.type = account_type m = re.search(r"javascript:submitForm\(([\w_]+),'([^']+)'\);", tds[0].find('a').attrib['onclick']) if not m: self.logger.warning('Unable to find link for %r' % account.label) account._link = None else: account._link = m.group(2) accounts.append(account) return accounts class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^Facture (?P
\d{2})/(?P\d{2})-(?P.*) carte .*'), FrenchTransaction.TYPE_CARD), (re.compile(u'^(Prlv( de)?|Ech(éance|\.)) (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(Vir|VIR)( de)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^CHEQUE.*? (N° \w+)?$'), FrenchTransaction.TYPE_CHECK), (re.compile('^Cotis(ation)? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile('(?PInt .*)'), FrenchTransaction.TYPE_BANK), ] class TransactionsPage(HTMLPage): logged = True @pagination @method class get_history(Transaction.TransactionsElement): head_xpath = '//table[@id="releve_operation"]//tr/th' item_xpath = '//table[@id="releve_operation"]//tr' col_date = [u'Date opé', 'Date', u'Date d\'opé', u'Date opération'] col_vdate = [u'Date valeur'] col_credit = [u'Crédit', u'Montant', u'Valeur'] col_debit = [u'Débit'] def next_page(self): url = Attr('//a[contains(text(), "Page suivante")]', 'onclick', default=None)(self) if url: m = re.search('\'([^\']+).*([\d]+)', url) return requests.Request("POST", m.group(1), data={'numCompte': Env('accid')(self), \ 'vue': "ReleveOperations", 'tri': "DateOperation", 'sens': \ "DESC", 'page': m.group(2), 'nb_element': "25"}) class item(Transaction.TransactionElement): def condition(self): return len(self.el.xpath('./td')) > 3 def get_coming_link(self): try: a = self.doc.getroot().cssselect('div#sous_nav ul li a.bt_sans_off')[0] except IndexError: return None return re.sub('[ \t\r\n]+', '', a.attrib['href']) weboob-1.2/modules/ganassurances/test.py000066400000000000000000000017631303450110500204660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GanAssurancesTest(BackendTest): MODULE = 'ganassurances' def test_banquepop(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/gazelle/000077500000000000000000000000001303450110500157145ustar00rootroot00000000000000weboob-1.2/modules/gazelle/__init__.py000066400000000000000000000000771303450110500200310ustar00rootroot00000000000000from .module import GazelleModule __all__ = ['GazelleModule'] weboob-1.2/modules/gazelle/browser.py000066400000000000000000000050631303450110500177550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages.index import IndexPage, LoginPage from .pages.torrents import TorrentsPage __all__ = ['GazelleBrowser'] class GazelleBrowser(Browser): PAGES = {'https?://[^/]+/?(index.php)?': IndexPage, 'https?://[^/]+/login.php.*': LoginPage, 'https?://[^/]+/torrents.php.*': TorrentsPage, } def __init__(self, protocol, domain, *args, **kwargs): self.DOMAIN = domain self.PROTOCOL = protocol Browser.__init__(self, *args, **kwargs) def login(self): if not self.is_on_page(LoginPage): self.location('/login.php', no_login=True) self.page.login(self.username, self.password) # If we are not logged, the on_loaded event on LoginPage has probably # raised the exception, but to be sure, check here to prevent an # unfinite loop if we can't find the error message. if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() def is_logged(self): if not self.page or self.is_on_page(LoginPage): return False if self.is_on_page(IndexPage): return self.page.is_logged() return True def home(self): return self.location('%s://%s/' % (self.PROTOCOL, self.DOMAIN)) def iter_torrents(self, pattern): self.location(self.buildurl('/torrents.php', searchstr=pattern.encode('utf-8'))) assert self.is_on_page(TorrentsPage) return self.page.iter_torrents() def get_torrent(self, fullid): if '.' not in fullid: return None id, torrentid = fullid.split('.', 1) self.location(self.buildurl('/torrents.php', id=id, torrentid=torrentid)) assert self.is_on_page(TorrentsPage) return self.page.get_torrent(fullid) weboob-1.2/modules/gazelle/favicon.png000066400000000000000000000025051303450110500200510ustar00rootroot00000000000000PNG  IHDR@@iqsRGB pHYs  tIME 75 GtIDATxMh\UIblc|ZiH.Zt16B-ӍG҂ q!LR_Up@.tCf_үOb@X+ qyahe9OZ$d;|F1D&@G׉Pd4@ѭObvV'tӤ< 7pm <dz@hNlUѰM]%@[{1‰ʗ .'aRݏھ@Aꕐ! p}E(Jv\B'|KM)J1ַBKdͅ ,:߳~n$|C 8h~kp2 pHވasz#=-N'jݝYQJQN^D&wMm2`x,z衇儿^7AIENDB`weboob-1.2/modules/gazelle/module.py000066400000000000000000000041511303450110500175540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.torrent import CapTorrent from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import GazelleBrowser __all__ = ['GazelleModule'] class GazelleModule(Module, CapTorrent): NAME = 'gazelle' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'Gazelle-based BitTorrent trackers' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('domain', label='Domain (example "ssl.what.cd")'), Value('protocol', label='Protocol to use', choices=('http', 'https')), Value('username', label='Username'), ValueBackendPassword('password', label='Password')) BROWSER = GazelleBrowser def create_default_browser(self): return self.create_browser(self.config['protocol'].get(), self.config['domain'].get(), self.config['username'].get(), self.config['password'].get()) def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None return self.browser.openurl(torrent.url.encode('utf-8')).read() def iter_torrents(self, pattern): return self.browser.iter_torrents(pattern) weboob-1.2/modules/gazelle/pages/000077500000000000000000000000001303450110500170135ustar00rootroot00000000000000weboob-1.2/modules/gazelle/pages/__init__.py000066400000000000000000000000001303450110500211120ustar00rootroot00000000000000weboob-1.2/modules/gazelle/pages/base.py000066400000000000000000000021231303450110500202750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import BrowserUnavailable, Page as _BasePage class BasePage(_BasePage): def on_loaded(self): errors = [] for div in self.parser.select(self.document.getroot(), 'div.poetry'): errors.append(self.parser.tocleanstring(div)) if len(errors) > 0: raise BrowserUnavailable(', '.join(errors)) weboob-1.2/modules/gazelle/pages/index.py000066400000000000000000000030561303450110500205000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import BrowserIncorrectPassword, BrowserBanned from .base import BasePage class IndexPage(BasePage): def is_logged(self): return 'id' in self.document.find('body').attrib class LoginPage(BasePage): def on_loaded(self): BasePage.on_loaded(self) warns = self.parser.select(self.document.getroot(), '.warning') for warn in warns: text = self.parser.tocleanstring(warn) if text.startswith('Your '): raise BrowserIncorrectPassword(text) if text.startswith('You are banned'): raise BrowserBanned(text) def login(self, login, password): self.browser.select_form(nr=0) self.browser['username'] = login self.browser['password'] = password self.browser.submit(no_login=True) weboob-1.2/modules/gazelle/pages/torrents.py000066400000000000000000000204321303450110500212460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import urlparse from logging import warning, debug from urlparse import parse_qs from weboob.tools.misc import get_bytes_size from weboob.tools.html import html2text from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotLoaded from .base import BasePage class TorrentsPage(BasePage): TORRENTID_REGEXP = re.compile('torrents\.php\?action=download&id=(\d+)') def format_url(self, url): return '%s://%s/%s' % (self.browser.PROTOCOL, self.browser.DOMAIN, url) def iter_torrents(self): table = self.document.getroot().cssselect('table.torrent_table') if not table: table = self.document.getroot().cssselect('table#browse_torrent_table') if table: table = table[0] current_group = None for tr in table.findall('tr'): if tr.attrib.get('class', '') == 'colhead': # ignore continue if tr.attrib.get('class', '') == 'group': tds = tr.findall('td') current_group = u'' div = tds[-6] if div.getchildren()[0].tag == 'div': div = div.getchildren()[0] for a in div.findall('a'): if not a.text: continue if current_group: current_group += ' - ' current_group += a.text elif tr.attrib.get('class', '').startswith('group_torrent') or \ tr.attrib.get('class', '').startswith('torrent'): tds = tr.findall('td') title = current_group if len(tds) == 7: # Under a group i = 0 elif len(tds) in (8, 9): # An alone torrent i = len(tds) - 1 while i >= 0 and tds[i].find('a') is None: i -= 1 else: # Useless title continue if title: title += u' (%s)' % tds[i].find('a').text else: title = ' - '.join([a.text for a in tds[i].findall('a')]) url = urlparse.urlparse(tds[i].find('a').attrib['href']) params = parse_qs(url.query) if 'torrentid' in params: id = '%s.%s' % (params['id'][0], params['torrentid'][0]) else: url = tds[i].find('span').find('a').attrib['href'] m = self.TORRENTID_REGEXP.match(url) if not m: continue id = '%s.%s' % (params['id'][0], m.group(1)) try: size, unit = tds[i + 3].text.split() except ValueError: size, unit = tds[i + 2].text.split() size = get_bytes_size(float(size.replace(',', '')), unit) seeders = int(tds[-2].text) leechers = int(tds[-1].text) torrent = Torrent(id, title) torrent.url = self.format_url(url) torrent.size = size torrent.seeders = seeders torrent.leechers = leechers yield torrent else: debug('unknown attrib: %s' % tr.attrib) def get_torrent(self, id): table = self.browser.parser.select(self.document.getroot(), 'div.thin', 1) h2 = table.xpath('.//h2') if len(h2) > 0: title = u''.join([txt.strip() for txt in h2[0].itertext()]) else: title = self.browser.parser.select(table, 'div.title_text', 1).text torrent = Torrent(id, title) if '.' in id: torrentid = id.split('.', 1)[1] else: torrentid = id table = self.browser.parser.select(self.document.getroot(), 'table.torrent_table') if len(table) == 0: table = self.browser.parser.select(self.document.getroot(), 'div.main_column', 1) is_table = False else: table = table[0] is_table = True for tr in table.findall('tr' if is_table else 'div'): if is_table and 'group_torrent' in tr.attrib.get('class', ''): tds = tr.findall('td') if not len(tds) == 5: continue url = tds[0].find('span').find('a').attrib['href'] m = self.TORRENTID_REGEXP.match(url) if not m: warning('ID not found') continue if m.group(1) != torrentid: continue torrent.url = self.format_url(url) size, unit = tds[1].text.split() torrent.size = get_bytes_size(float(size.replace(',', '')), unit) torrent.seeders = int(tds[3].text) torrent.leechers = int(tds[4].text) break elif not is_table and tr.attrib.get('class', '').startswith('torrent_widget') \ and tr.attrib.get('class', '').endswith('pad'): url = tr.cssselect('a[title=Download]')[0].attrib['href'] m = self.TORRENTID_REGEXP.match(url) if not m: warning('ID not found') continue if m.group(1) != torrentid: continue torrent.url = self.format_url(url) size, unit = tr.cssselect('div.details_title strong')[-1].text.strip('()').split() torrent.size = get_bytes_size(float(size.replace(',', '')), unit) torrent.seeders = int(tr.cssselect('img[title=Seeders]')[0].tail) torrent.leechers = int(tr.cssselect('img[title=Leechers]')[0].tail) break if not torrent.url: warning('Torrent %s not found in list' % torrentid) return None div = self.parser.select(self.document.getroot(), 'div.main_column', 1) for box in div.cssselect('div.box'): title = None body = None title_t = box.cssselect('div.head') if len(title_t) > 0: title_t = title_t[0] if title_t.find('strong') is not None: title_t = title_t.find('strong') if title_t.text is not None: title = title_t.text.strip() body_t = box.cssselect('div.body,div.desc') if body_t: body = html2text(self.parser.tostring(body_t[-1])).strip() if title and body: if torrent.description is NotLoaded: torrent.description = u'' torrent.description += u'%s\n\n%s\n' % (title, body) divs = self.document.getroot().cssselect('div#files_%s,div#filelist_%s,tr#torrent_%s td' % (torrentid, torrentid, torrentid)) if divs: torrent.files = [] for div in divs: table = div.find('table') if table is None: continue for tr in table: if tr.attrib.get('class', None) != 'colhead_dark': torrent.files.append(tr.find('td').text) return torrent weboob-1.2/modules/gazelle/test.py000066400000000000000000000017351303450110500172530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GazelleTest(BackendTest): MODULE = 'gazelle' def test_torrent(self): l = list(self.backend.iter_torrents('sex')) if len(l) > 0: self.backend.get_torrent_file(l[0].id) weboob-1.2/modules/gdcvault/000077500000000000000000000000001303450110500161025ustar00rootroot00000000000000weboob-1.2/modules/gdcvault/__init__.py000066400000000000000000000001011303450110500202030ustar00rootroot00000000000000from .module import GDCVaultModule __all__ = ['GDCVaultModule'] weboob-1.2/modules/gdcvault/browser.py000066400000000000000000000117131303450110500201420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from weboob.deprecated.browser import Browser, BrowserIncorrectPassword, BrowserUnavailable,\ BrowserBanned from weboob.deprecated.browser.decorators import id2url #from .pages.index import IndexPage from .pages import VideoPage, IndexPage, SearchPage from .video import GDCVaultVideo #HACK from urllib2 import HTTPError import re from weboob.capabilities.base import NotAvailable __all__ = ['GDCVaultBrowser'] class GDCVaultBrowser(Browser): DOMAIN = 'gdcvault.com' ENCODING = 'utf-8' PAGES = {r'http://[w\.]*gdcvault.com/play/(?P[\d]+)/?.*': VideoPage, r'http://[w\.]*gdcvault.com/search\.php.*': (SearchPage, "json"), r'http://[w\.]*gdcvault.com/.*': IndexPage, } def is_logged(self): if self.password is None: return True if not self.page: return False obj = self.parser.select(self.page.document.getroot(), 'h3[id=welcome_user_name]', 1) if obj is None: return False return obj.attrib.get('class','') != "hidden" def login(self): if self.password is None: return params = {'remember_me': 0, 'email': self.username, 'password': self.password, } data = self.readurl('http://gdcvault.com/api/login.php', urllib.urlencode(params)) # some data returned as JSON, not sure yet if it's useful if data is None: self.openurl('/logout', '') raise BrowserBanned('Too many open sessions?') self.location('/', no_login=True) if not self.is_logged(): raise BrowserIncorrectPassword() def close_session(self): if self.password is None or not self.is_logged(): return self.openurl('/logout', '') @id2url(GDCVaultVideo.id2url) def get_video(self, url, video=None): requires_account = False redir_url = None # FIXME: this is quite ugly # but is required to handle cases like 1013422@gdcvault self.set_handle_redirect(False) try: self.open_novisit(url) #headers = req.info() except HTTPError as e: if e.getcode() == 302 and hasattr(e, 'hdrs'): if e.hdrs['Location'] in ['/', '/login']: requires_account = True else: # 1015865 redirects to a file with an eacute in the name redir_url = unicode(e.hdrs['Location'], encoding='utf-8') self.set_handle_redirect(True) if requires_account: raise BrowserUnavailable('Requires account') if redir_url: if video is None: m = re.match('http://[w\.]*gdcvault.com/play/(?P[\d]+)/?.*', url) if m: video = GDCVaultVideo(int(m.group(1))) else: raise BrowserUnavailable('Cannot find ID on page with redirection') video.url = redir_url video.set_empty_fields(NotAvailable) # best effort for now return video self.location(url) # redirects to /login means the video is not public if not self.is_on_page(VideoPage): raise BrowserUnavailable('Requires account') return self.page.get_video(video) def search_videos(self, pattern, sortby): post_data = {"firstfocus" : "", "category" : "free", "keyword" : pattern.encode('utf-8'), "conference_id" : "", } post_data = urllib.urlencode(post_data) # probably not required self.addheaders = [('Referer', 'http://gdcvault.com/'), ("Content-Type" , 'application/x-www-form-urlencoded') ] # is_logged assumes html page self.location('http://gdcvault.com/search.php', data=post_data, no_login=True) assert self.is_on_page(SearchPage) return self.page.iter_videos() def latest_videos(self): #self.home() self.location('/free') assert self.is_on_page(IndexPage) return self.page.iter_videos() weboob-1.2/modules/gdcvault/favicon.png000066400000000000000000000066321303450110500202440ustar00rootroot00000000000000PNG  IHDR@@iqbKGDg} pHYs  tIME 50] 0tEXtCommentCreated with GIMPW IDATxɏ\uwzjv7 KQ,@6""+ 0dU,?M @ h# eɲ,S)jDT{N'W0`9s>PۤR h34VC΂OBș2k(j'i]oVv~V2$$|1%4ݩgpʥ}6+M$HcK95,rR,Ĕ9V|GԚ|1N0ĸlj~qZ)Zʂ>ϭὭ}^TpJbStuʌ :d蠱JsO#Xg "$0\(GKh!, .ƅ]ڤ޽JZJgY[,,:ӫLC񄐰J/x)tl6 o|1 ̠t,١j$HIYYkv5\ݯB +K\[]v B[XjaGc!f<( Vyje1sD1M'8(%iMՔ,vV)!)IB$3Q21 !e1B, # tf3o[L@/Buè4!"Oޠ`TEItӚ \9$e#k͌[:W rn 0g($Y(Fu`wb9c _P((n<>e&_{V 5Ŋ 6y{;jՒ!DqWiM"LC cCn&fJ{VKDQ0%:UZiD,ZU#C cӚG⌤Z XEnEaC0(Fw~ J+| ё$-Sasg(}mS0FaB)")ø|s3CzN{V؞LI"LOJ`i?3 Q3bL̙h{XI9/ʙc,/yokwwIv7@keӦ}%_Ro^<;;N1X%(">0#ᅵI8lj1W{Iέ9|ȰSP(CB#{71F+,NAXicMScb$<$ ^a1G$MAbN)Ea9mFȴ2 cM|5 s6YrAFQఉ6f"~l8hJvi+JyYv.nO>uܔyRcnjf0P8#S}Yjݫ<#;w5PX}S mA9g閚Æ2sMd|X.pj8 wý%Wo|7=jCL08nL0Aa O#?]|JR|x}D,wK|z~Xn~&ITnJ "p]݂>g]՟?bʼ>sN˱.k=ƍ>Xrju-h7pJ`B#$!%|4ӄ8c#n5lڢ!JVY劢[Z cYtJGߥg@|;b[q GLho7g3=1FҙU^p ) MJL__g{T3 ;%,;>?|q.Sh44|?!*̬R gN%?}(b{:F.mS՚V i/#&D1-['eDiϯ-곧8ԣ,:%N~I 4sR:7M.R"@ fd1Sqc5Rʀúќ;:d9b|r0!Pha=?Edꐳ+[{Uø6FE!e5YhCmtk)Aie-0o\`o^lMQ.5#d`$qlG]-'WqUyRbYBQ8j 3 V=:2AF ) VvYpNR~}c-hQFcHqP,3ۣ)>/| jk/^f)98BmCد<r0Ai9{d ,c5Nkin-qʤ,6hZQ0)C,^ A֚O툘f,ĜIۣ1\XUx|r8ĤVx*ҒP̘ZCڛ Vi3B+՚dQ)fiޮ@*!IF>g֞ѭJG^Ypne 鸒GYY, oo1 }ǟ,FC._:s LJ ) ژ(Fh9 *.A֔FS:MBH)?P?&f.G11(BbfT+*oNiD| V1SsZ$3CbRdQXX^8/ϝXiݚ$1B= Y"}0!9 xhZ ZcUBLkq9Ka VlsbzWSVQsdfbVq"O:KtQfQP:ˠ0Zs ($#S#!<ཇ=D "д>NgZn4IK>FoSnyjmO7]Sy 0`9s0`9_Ke ޫ7IENDB`weboob-1.2/modules/gdcvault/favicon.xcf000066400000000000000000000116401303450110500202330ustar00rootroot00000000000000gimp xcf file@@BBm gimp-commentCreated with GIMPgimp-image-grid(style solid) (fgcolor (color-rgba 0.000000 0.000000 0.000000 1.000000)) (bgcolor (color-rgba 1.000000 1.000000 1.000000 1.000000)) (xspacing 10.000000) (yspacing 10.000000) (spacing-unit inches) (xoffset 0.000000) (yoffset 0.000000) (offset-unit inches) gamma0.45454999804496765@@ favicon.png     @@@@ghg6ghiknstxvpkhhpv{{|}z}~|unggijlmkliggӼ}qlzxjilsŜmgg伅tк|lptgghqp迄|㮛wp՟vhgglmggn{onsԘzpmmyԍuompxvohgg}oggnālgpГrjhm{jgghiihggxggՅlhpΒsggoٖsjggmރggykgoΓrggoԒsjgghtԉkgg܀kgnϑqggl٘ukgglyʏoggpwgglՐqggjliggm|ǘ{rwmggi֏oggtё|qnptqkggn}ͯogg׏oggjzФzggm{qgg~׍nggm~л~jggkuƹniggsāmggivŴwjgghnxulggku{rjgghnzwmgghknpolkihg gjigghjklmlljgghiig9gmz}skhjljjg gioutnhg gv־mlt~xmg gmogggg|ҍnmsg gnzhggkjggՐsqxg gjngghyjlswz|ytnggmtxrjgginllh֓|skg gvwggmrmζujj{sggnpj֖mggnӃlk~quˏrnggwxjזpggi~uuـmw~rۃig}zkז~ꚃwng gs{lvɆr؇kj}kדsu܍mg gm۟ܖvrևq׉nlkُooؑng gw‡r~ބnڋprnۋlmؒphg gnߩ|q~kz~݈p݉mmܙwmjg gҎsm򯋏ygt򿕠׊rۈkk民ujg gqljwsgkҊtԆig|̳mg gjԏphglwsgglըӽrо}kgq˷lg glqw{wqgghloqkgghhgghnqtpry{rlusjgjq}sig+ghihjklljgghklkkhggǥ񦧩ڮħطղ뭥޳۵½ƶծȰ溮ܳ侪˱緩⽪볥취⽪轪淥괨佪ܦ༨஥彫ɱ槱¯è漨侰⨥鼨Ƚͭ۫缩װ񦬸ռܳ® : !ڴ ƪ弪櫥 Ⱕʥ濬 ᄅ逸翳 ʮӯનԹ謥 ɪ洧߼ ʲǬ粨Գ궥ö ̫ڶӰ෬縨򳧷迭껩 ̨縫躩𴧷꽪轩 ܷͯ뷩麪꺨龩 ˩˱Կ򲧲𾯴귩븦 ˳㻫ι滬귥н ˪ղ幬絥ҳ 弩ѫ˹سٱӴ  橬,A!       0weboob-1.2/modules/gdcvault/module.py000066400000000000000000000070721303450110500177470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo, BaseVideo from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.value import Value, ValueBackendPassword from .browser import GDCVaultBrowser from .video import GDCVaultVideo __all__ = ['GDCVaultModule'] class GDCVaultModule(Module, CapVideo, CapCollection): NAME = 'gdcvault' MAINTAINER = u'François Revol' EMAIL = 'revol@free.fr' VERSION = '1.2' DESCRIPTION = 'Game Developers Conferences Vault video streaming website' LICENSE = 'AGPLv3+' BROWSER = GDCVaultBrowser CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default='')) def create_default_browser(self): username = self.config['username'].get() if len(username) > 0: password = self.config['password'].get() else: password = None return self.create_browser(username, password) def deinit(self): # don't need to logout if the browser hasn't been used. if not self._browser: return with self.browser: self.browser.close_session() def get_video(self, _id): with self.browser: return self.browser.get_video(_id) SORTBY = ['relevance', 'rating', 'views', 'time'] def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): with self.browser: return self.browser.search_videos(pattern, self.SORTBY[sortby]) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(GDCVaultVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest GDCVault videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {GDCVaultVideo: fill_video} weboob-1.2/modules/gdcvault/pages.py000066400000000000000000000337651303450110500175710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page import urllib import re import datetime from dateutil.parser import parse as parse_dt from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from weboob.deprecated.browser import BrokenPageError #HACK from urllib2 import HTTPError from .video import GDCVaultVideo #import lxml.etree # TODO: check title on 1439 class IndexPage(Page): def iter_videos(self): for a in self.parser.select(self.document.getroot(), 'section.conference ul.media_items li.featured a.session_item'): href = a.attrib.get('href', '') # print href m = re.match('/play/(\d+)/.*', href) if not m: continue # print m.group(1) video = GDCVaultVideo(m.group(1)) # get title try: video.title = unicode(self.parser.select(a, 'div.conference_info p strong', 1).text) except IndexError: video.title = NotAvailable # get description try: video.description = unicode(self.parser.select(a, 'div.conference_info p', 1).text) except IndexError: video.description = NotAvailable # get thumbnail img = self.parser.select(a, 'div.featured_image img', 1) if img is not None: video.thumbnail = Thumbnail(img.attrib['src']) video.thumbnail.url = video.thumbnail.id else: video.thumbnail = NotAvailable #m = re.match('id-(\d+)', a.attrib.get('class', '')) #if not m: # continue # FIXME yield video # the search page class uses a JSON parser, # since it's what search.php returns when POSTed (from Ajax) class SearchPage(Page): def iter_videos(self): if self.document is None or self.document['data'] is None: raise BrokenPageError('Unable to find JSON data') for data in self.document['data']: video = GDCVaultVideo.get_video_from_json(data) # TODO: split type 4 videos into id and id#slides if video is None: continue yield video class VideoPage(Page): def get_video(self, video=None): # check for slides id variant want_slides = False m = re.match('.*#slides', self.url) if m: want_slides = True # not sure it's safe self.group_dict['id'] += '#slides' if video is None: video = GDCVaultVideo(self.group_dict['id']) # the config file has it too, but in CDATA and only for type 4 obj = self.parser.select(self.document.getroot(), 'title') title = None if len(obj) > 0: try: title = unicode(obj[0].text) except UnicodeDecodeError as e: title = None if title is None: obj = self.parser.select(self.document.getroot(), 'meta[name=title]') if len(obj) > 0: if 'content' in obj[0].attrib: try: # FIXME: 1013483 has buggus title (latin1) # for now we just pass it as-is title = obj[0].attrib['content'] except UnicodeDecodeError as e: # XXX: this doesn't even works!? title = obj[0].attrib['content'].decode('iso-5589-15') if title is not None: title = title.strip() m = re.match('GDC Vault\s+-\s+(.*)', title) if m: title = m.group(1) video.title = title #TODO: POST back the title to /search.php and filter == id to get # cleaner (JSON) data... (though it'd be much slower) # try to find an iframe (type 3 and 4) obj = self.parser.select(self.document.getroot(), 'iframe') if len(obj) == 0: # type 1 or 2 (swf+js) # find which script element contains the swf args for script in self.parser.select(self.document.getroot(), 'script'): m = re.match(".*new SWFObject.*addVariable\('type', '(.*)'\).*", unicode(script.text), re.DOTALL) if m: video.ext = m.group(1) m = re.match(".*new SWFObject.*addVariable\(\"file\", encodeURIComponent\(\"(.*)\"\)\).*", unicode(script.text), re.DOTALL) if m: video.url = "http://gdcvault.com%s" % (m.group(1)) # TODO: for non-free (like 769), # must be logged to use /mediaProxy.php # FIXME: doesn't seem to work yet, we get 2 bytes as html # 769 should give: # http://twvideo01.ubm-us.net/o1/gdcradio-net/2007/gdc/GDC07-4889.mp3 # HACK: we use mechanize directly here for now... FIXME #print "asking for redirect on '%s'" % (video.url) #self.browser.addheaders += [['Referer', 'http://gdcvault.com/play/%s' % self.group_dict['id']]] #print self.browser.addheaders self.browser.set_handle_redirect(False) try: self.browser.open_novisit(video.url) # headers = req.info() # if headers.get('Content-Type', '') == 'text/html' and headers.get('Content-Length', '') == '2': # print 'BUG' #print req.code except HTTPError as e: #print e.getcode() if e.getcode() == 302 and hasattr(e, 'hdrs'): #print e.hdrs['Location'] video.url = unicode(e.hdrs['Location']) self.browser.set_handle_redirect(True) video.set_empty_fields(NotAvailable) return video #XXX: raise error? return None obj = obj[0] if obj is None: return None # type 3 or 4 (iframe) # get the config file for the rest iframe_url = obj.attrib['src'] # 1015020 has a boggus url m = re.match('http:/event(.+)', iframe_url) if m: iframe_url = 'http://event' + m.group(1) # print iframe_url # 1013798 has player169.html # 1012186 has player16x9.html # some other have /somethingplayer.html... # 1441 has a space in the xml filename, which we must not strip m = re.match('(http:.*/)[^/]*player[0-9a-z]*\.html\?.*xmlURL=([^&]+\.xml).*\&token=([^& ]+)', iframe_url) if not m: m = re.match('/play/mediaProxy\.php\?sid=(\d+)', iframe_url) if m is None: return None # TODO: must be logged to use /mediaProxy.php # type 3 (pdf slides) video.ext = u'pdf' video.url = "http://gdcvault.com%s" % (unicode(iframe_url)) # HACK: we use mechanize directly here for now... FIXME # print "asking for redirect on '%s'" % (video.url) self.browser.set_handle_redirect(False) try: self.browser.open_novisit(video.url) except HTTPError as e: if e.getcode() == 302 and hasattr(e, 'hdrs'): video.url = unicode(e.hdrs['Location']) self.browser.set_handle_redirect(True) video.set_empty_fields(NotAvailable) return video # type 4 (dual screen video) # token doesn't actually seem required # 1441 has a space in the xml filename xml_filename = urllib.quote(m.group(2)) config_url = m.group(1) + xml_filename + '?token=' + m.group(3) # self.browser.addheaders += [['Referer', 'http://gdcvault.com/play/%s' % self.group_dict['id']]] # print self.browser.addheaders # TODO: fix for 1015021 & others (forbidden) #config = self.browser.openurl(config_url).read() config = self.browser.get_document(self.browser.openurl(config_url)) obj = self.parser.select(config.getroot(), 'akamaihost', 1) host = obj.text if host is None: raise BrokenPageError('Missing tag in xml config file') if host == "smil": # the rtmp URL is described in a smil file, # with several available bitrates obj = self.parser.select(config.getroot(), 'speakervideo', 1) smil = self.browser.get_document(self.browser.openurl(obj.text)) obj = self.parser.select(smil.getroot(), 'meta', 1) # TODO: error checking base = obj.attrib.get('base', '') best_bitrate = 0 path = None obj = self.parser.select(smil.getroot(), 'video') # choose the best bitrate for o in obj: rate = int(o.attrib.get('system-bitrate', 0)) if rate > best_bitrate: path = o.attrib.get('src', '') video.url = unicode(base + '/' + path) else: # not smil, the rtmp url is directly here as host + path # for id 1373 host is missing '/ondemand' # only add it when only a domain is specified without path m = re.match('^[^\/]+$', host) if m: host += "/ondemand" videos = {} obj = self.parser.select(config.getroot(), 'speakervideo', 1) if obj.text is not None: videos['speaker'] = 'rtmp://' + host + '/' + urllib.quote(obj.text) obj = self.parser.select(config.getroot(), 'slidevideo', 1) if obj.text is not None: videos['slides'] = 'rtmp://' + host + '/' + urllib.quote(obj.text) # print videos # XXX if 'speaker' in videos: video.url = unicode(videos['speaker']) elif 'slides' in videos: # 1016627 only has slides, so fallback to them video.url = unicode(videos['slides']) if want_slides: if 'slides' in videos: video.url = unicode(videos['slides']) # if video.url is none: raise ? XXX obj = self.parser.select(config.getroot(), 'date', 1) if obj.text is not None: # 1016634 has "Invalid Date" try: video.date = parse_dt(obj.text) except ValueError as e: video.date = NotAvailable obj = self.parser.select(config.getroot(), 'duration', 1) m = re.match('(\d\d):(\d\d):(\d\d)', obj.text) if m: video.duration = datetime.timedelta(hours = int(m.group(1)), minutes = int(m.group(2)), seconds = int(m.group(3))) obj = self.parser.select(config.getroot(), 'speaker', 1) #print obj.text_content() #self.set_details(video) video.set_empty_fields(NotAvailable) return video obj = self.parser.select(self.document.getroot(), 'title') if len(obj) < 1: return None title = obj[0].text.strip() m = re.match('GDC Vault\s+-\s+(.*)', title) if m: title = m.group(1) def set_details(self, v): obj = self.parser.select(self.document.getroot(), 'meta[name=available]', 1) if obj is not None: value = obj.attrib['content'] m = re.match('(\d\d)-(\d\d)-(\d\d\d\d)\s*(\d\d):(\d\d)', value) if not m: raise BrokenPageError('Unable to parse datetime: %r' % value) day = m.group(1) month = m.group(2) year = m.group(3) hour = m.group(4) minute = m.group(5) v.date = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(hour), minute=int(minute)) obj = self.parser.select(self.document.getroot(), 'span.ep_subtitle', 1) if obj is not None: span = self.parser.select(obj, 'span.ep_date', 1) value = span.text m = re.match('(\d\d):(\d\d)\s*\/\s*(\d\d):(\d\d)\s*-\s*(\d\d)-(\d\d)-(\d\d\d\d)', value) if not m: raise BrokenPageError('Unable to parse datetime: %r' % value) bhour = m.group(1) bminute = m.group(2) ehour = m.group(3) eminute = m.group(4) day = m.group(5) month = m.group(6) year = m.group(7) start = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(bhour), minute=int(bminute)) end = datetime.datetime(year=int(year), month=int(month), day=int(day), hour=int(ehour), minute=int(eminute)) v.duration = end - start weboob-1.2/modules/gdcvault/test.py000066400000000000000000000030721303450110500174350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest #from weboob.capabilities.video import BaseVideo class GDCVaultTest(BackendTest): MODULE = 'gdcvault' # def test_search(self): # l = list(self.backend.search_videos('linux')) # self.assertTrue(len(l) > 0) # v = l[0] # self.backend.fillobj(v, ('url',)) # self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) # self.backend.browser.openurl(v.url) # def test_latest(self): # l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) # self.assertTrue(len(l) > 0) # v = l[0] # self.backend.fillobj(v, ('url',)) # self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) weboob-1.2/modules/gdcvault/video.py000066400000000000000000000062031303450110500175630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import BaseVideo from weboob.capabilities.base import NotAvailable import re from dateutil.parser import parse as parse_dt class GDCVaultVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) # not always flv... self.ext = NotAvailable @classmethod def id2url(cls, _id): # attempt to enlarge the id namespace to differentiate # videos from the same page m = re.match('\d+#speaker', _id) if m: return u'http://www.gdcvault.com/play/%s#speaker' % _id m = re.match('\d+#slides', _id) if m: return u'http://www.gdcvault.com/play/%s#slides' % _id return u'http://www.gdcvault.com/play/%s' % _id @classmethod def get_video_from_json(self, data): # session_id is unique per talk # vault_media_id is unique per page # (but can refer to 2 video files for dual screen) # solr_id is "${vault_media_id}.${conference_id}.${session_id}.$vault_media_type_id{}" # XXX: do we filter them or let people know about them? #if 'anchor' in data: # if data['anchor']['href'] == '#': # # file will not be accessible (not free and not logged in) # return None if 'vault_media_id' not in data: return None media_id = int(data['vault_media_id']) video = GDCVaultVideo(media_id) # 1013679 has \n in title... video.title = unicode(data.get('session_name', '').replace('\n', '')) # TODO: strip out

,
and other html... # XXX: 1013422 has all 3 and != if 'overview' in data: video.description = unicode(data['overview']) elif 'spell' in data: video.description = unicode(data['spell']) else: video.description = unicode(data.get('description', '')) if 'image' in data: video.thumbnail = Thumbnail(data['image']) video.thumbnail.url = video.thumbnail.id if 'speakers_name' in data: video.author = unicode(", ".join(data['speakers_name'])) if 'start_date' in data: video.date = parse_dt(data['start_date']) if 'score' in data: video.rating = data['score'] video.set_empty_fields(NotAvailable) return video weboob-1.2/modules/gdfsuez/000077500000000000000000000000001303450110500157405ustar00rootroot00000000000000weboob-1.2/modules/gdfsuez/__init__.py000066400000000000000000000000771303450110500200550ustar00rootroot00000000000000from .module import GdfSuezModule __all__ = ['GdfSuezModule'] weboob-1.2/modules/gdfsuez/browser.py000066400000000000000000000072431303450110500200030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import StringIO from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, HomePage, AccountPage, TimeoutPage, HistoryPage, PdfPage __all__ = ['GdfSuez'] class GdfSuez(Browser): PROTOCOL = 'https' DOMAIN = 'www.gdfsuez-dolcevita.fr' PAGES = {'.*portail/clients.*?_nfpb=true&_pageLabel=page_identification': LoginPage, '.*portail/clients.*?_nfpb=true&_pageLabel=page_accueil_compte_en_ligne': HomePage, '.*p/visualiser_mes_contrats.*?_nfpb=true': AccountPage, '.*p/page_historique_de_mes_factures': HistoryPage, '.*clients.*?_nfpb=true&_nfls=false&_pageLabel=page_erreur_timeout_session': TimeoutPage } loginp = '/portailClients/appmanager/portail/clients' homep = '/portailClients/appmanager/portail/clients?_nfpb=true&_pageLabel=page_accueil_compte_en_ligne' accountp = '/portailClients/client/p/visualiser_mes_contrats?_nfpb=true' historyp = '/portailClients/client/p/page_historique_de_mes_factures' def __init__(self, *args, **kwargs): Browser.__init__(self, *args, **kwargs) def home(self): self.location(self.homep) def is_logged(self): if self.is_on_page(LoginPage) or self.is_on_page(TimeoutPage): return False return True def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) #assert isemail(self.username) if not self.is_on_page(LoginPage): self.location(self.loginp) self.page.login(self.username, self.password) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() def get_subscription_list(self): if not self.is_on_page(AccountPage): self.location(self.accountp) return self.page.get_subscription_list() def get_subscription(self, id): assert isinstance(id, basestring) for sub in self.get_subscription_list(): if sub.id == id: return sub def get_history(self, subscription): if not self.is_on_page(HistoryPage): self.location(self.historyp) return self.page.get_history() def get_details(self, subscription): bills = self.iter_documents() id = bills[0].id if not self.is_on_page(HistoryPage): self.location(self.historyp) url = 'https://www.gdfsuez-dolcevita.fr/' + self.get_document(id).url response = self.openurl(url) pdf = PdfPage(StringIO.StringIO(response.read())) for detail in pdf.get_details(subscription.label): yield detail def iter_documents(self): if not self.is_on_page(HistoryPage): self.location(self.historyp) return self.page.get_documents() def get_document(self, id): assert isinstance(id, basestring) for b in self.iter_documents(): if b.id == id: return b weboob-1.2/modules/gdfsuez/favicon.png000066400000000000000000000054601303450110500201000ustar00rootroot00000000000000PNG  IHDR@@iq pHYs  tIME ."WtEXtCommentCreated with GIMPW IDATxk\e{fv6@[J 1b Qb(?_4ƘXт bAni v[j3}?3ۓagw۽x3s9x3_7޴P EY(`M$ޑkuk}ݛ6[2:]\m v-֜5 A> aa]wO8) 9t>@^p1"+ҍ+.!x\H𡊗+8n7ѐ%''ݛ6X{Yu.a0f-A Y0 %`؁5;=xكFj<1@L ܿ W+{ឿ~׬ؔ;n 2bqH`Dz * NU_$~g9j[B ֬Ycv^o:W $ d< d2AB`ƝȀ5BGFHnIl3lŸf@atgXL͂Ț]:q\c$ PH5V>Bkwc-xAV=Tz`X"p=p>pp[TV$ KI*C=C33$nI<Si@Rx} .tl|z!MWuZ3W 8>ʀW,x!`Dk/ "M}s1ЅH.{M = ЋV F~q3 ñH0dF6$cY^E&^ A9$'jE.`[۫f!&vB FDJ?;W?a)_&դ0'tJ`CTݗ6e^8{Ib$aN3#( `gV,j\P.U B`5ݨܿ>*O(@RKǤ^fm!g!xABgȘ4$$\15RX7$d I$YN`Bc# RMd$+G~a0jL`2x 뀿R ZZ#QLn-/z&0,@0Smk5ZĚoj%]r DV`fq?EӻWgD4/J:0jK}G .Tj16,ӥWRKye@ K[|Nny`n6$YK <𥉉lcB"kp>K0-ol IRZXTp|loR9Ûkz%t&rngRߣ$mluە5Փ"`&a=i_<8WS#&^P#c/+raԶhP)0e_ rjepvU53]-z!mӴ^oJ9V,< Bse[ݼAECο >=[( |xY';I[Y8ΧE\ְ66 Ru\U{ @AHOV1IVyNayVӼ ৪MN&~NT|ia-;lw&9r:͚,z૕CZ %&"єxLhnG{4Ql ;5"Ey-g;l/9i.LHے%٭ʨcPڨa՜ݣ ]vv: 4[G/ϜBP;c෾ZyDM`՟j_jej0F7pD~ZuZ.s ~nvZ`6 ss穅F'YZzSsmbЀ5ivh3HiyKvx*V||1DCs?5Q ӍskE9rtxw?H='IENDB`weboob-1.2/modules/gdfsuez/module.py000066400000000000000000000067411303450110500176070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bill import CapDocument, SubscriptionNotFound,\ DocumentNotFound, Subscription, Bill from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import GdfSuez __all__ = ['GdfSuezModule'] class GdfSuezModule(Module, CapDocument): NAME = 'gdfsuez' MAINTAINER = u'Mathieu Jourdan' EMAIL = 'mathieu.jourdan@gresille.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'GDF-Suez French energy provider' CONFIG = BackendConfig(ValueBackendPassword('login', label='Account ID (e-mail)', masked=False), ValueBackendPassword('password', label='Password', masked=True) ) BROWSER = GdfSuez def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): for subscription in self.browser.get_subscription_list(): yield subscription def get_subscription(self, _id): if not _id.isdigit(): raise SubscriptionNotFound() with self.browser: subscription = self.browser.get_subscription(_id) if not subscription: raise SubscriptionNotFound() else: return subscription def iter_documents_history(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) with self.browser: for history in self.browser.get_history(subscription): yield history def get_details(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) with self.browser: for detail in self.browser.get_details(subscription): yield detail def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) with self.browser: for bill in self.browser.iter_documents(): yield bill def get_document(self, id): with self.browser: bill = self.browser.get_document(id) if not bill: raise DocumentNotFound() else: return bill def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) with self.browser: return self.browser.readurl(bill.url) weboob-1.2/modules/gdfsuez/pages/000077500000000000000000000000001303450110500170375ustar00rootroot00000000000000weboob-1.2/modules/gdfsuez/pages/__init__.py000066400000000000000000000016471303450110500211600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .history import HistoryPage, PdfPage from .homepage import LoginPage, HomePage, AccountPage, TimeoutPage __all__ = ['LoginPage', 'HomePage', 'AccountPage', 'HistoryPage', 'PdfPage', 'TimeoutPage'] weboob-1.2/modules/gdfsuez/pages/history.py000066400000000000000000000217641303450110500211240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import os import subprocess import tempfile import shutil from datetime import date from decimal import Decimal from weboob.deprecated.browser import Page from weboob.capabilities.base import NotAvailable from weboob.capabilities.bill import Detail, Bill class HistoryPage(Page): def on_loaded(self): self.details = [] self.bills = [] # Latest bill div = self.document.xpath('//div[@class="consulter_dernierefacture"]')[0] bdate = div.xpath('p[@class="date"]/span[@class="textetertiaire"]')[0].text bprice = div.xpath('p[@class="montant"]/span[@class="textetertiaire"]')[0].text link = div.xpath('a[@id="display_popin"]')[0].attrib['href'] mydate = date(*reversed([int(x) for x in bdate.split("/")])) price = Decimal(bprice.strip(u' € TTC').replace(',', '.')) self.bills.append(self._create_bill(mydate, price, link)) # Previous bills table = self.document.xpath('//table[@class="afficher_factures"]')[0] for tr in table[0].xpath('//tbody/tr'): cells = tr.xpath('td') bdate = unicode(cells[0].text.strip()) mydate = date(*reversed([int(x) for x in bdate.split("/")])) bprice = unicode(cells[1].text) price = Decimal(bprice.strip(u' €').replace(',', '.')) link = cells[3].xpath('a')[0].attrib['href'] self.bills.append(self._create_bill(mydate, price, link)) def _create_bill(self, date, price, link): bill = Bill() bill.id = date.__str__().replace('-', '') bill.date = date bill._price = price bill.url = unicode(link) bill.format = u'pdf' bill.type = u'bill' bill.label = unicode(price) return bill def get_details(self): return self.details def get_documents(self): return self.bills class PdfPage(): def __init__(self, file): self.pdf = file def _parse_pdf(self): pdffile = tempfile.NamedTemporaryFile(bufsize=100000, mode='w', suffix='.pdf') temptxt = pdffile.name.replace('.pdf', '.txt') cmd = "ebook-convert" stdout = open("/dev/null", "w") shutil.copyfileobj(self.pdf, pdffile) pdffile.flush() subprocess.call([cmd, pdffile.name, temptxt], stdout=stdout) pdffile.close() txtfile = open(temptxt, 'r') txt = txtfile.read() txtfile.close() os.remove(temptxt) return txt def _parse_page(self, page): # Regexp footnote = re.compile(r'\([0-9]\) ') # (f) ht = re.compile('HT par mois') base = re.compile('la base de') enddate = re.compile('\d\d\/\d\d\/\d\d') # YY/MM/DD endwithdigit = re.compile('\d+$') # blah blah 42 textwithcoma = re.compile('([a-z]|\d{4})\,') # blah 2012, blah blah # Parsing details = [] for title in ['Abonnement', 'Consommation', 'Contributions et taxes liées à l\'énergie']: section = page.split(title, 1)[1].split('Total ')[0] # When a line holds '(0)', a newline is missing. section = re.sub(footnote, '\n', section) lines = section.split('\n') lines = [x for x in lines if len(x) > 0] # Remove empty lines detail = None for line in lines: if re.match('[A-Za-z]', line[0]): # Things we want to merge with the one just before if 'facturées' in line: # Long lines are sometimes split, so we try to join them # That is the case for: # 'Déduction du montant des consommations # estimées facturées du 00/00/00 au 00/00/00' detail.label = detail.label + u' ' + unicode(line, encoding='utf-8') # Things for which we want a new detail else: # Entering here, we will instantiate a new detail. # We hadn't so before because of fragmented lines. if detail is not None and detail.label is not NotAvailable: # We have a new element, return the other one details.append(detail) detail = Detail() detail.price = Decimal(0) # If the coma is not a decimal separator, then # this is is probably a loooong sentence. # When it comes to jokes, keep it short and sweet. line = re.split(textwithcoma, line)[0] # Things we want for sure if re.findall(enddate, line): # When a line has been badly split after a date, # We want the label to end after the date, and maybe # the second part to be the info mydate = re.search(enddate, line).group(0) mylist = line.rpartition(mydate) label = mylist[0] + mylist[1] detail.label = unicode(label, encoding='utf-8') elif re.findall(endwithdigit, line): # What is this stupid number at the end of the line? # Line should have been split before the number detail.label = unicode(re.split(endwithdigit, line)[0], encoding='utf-8') # Things we don't want for sure elif ')' in line and '(' not in line: # First part of the parenthesis should have been drop before # Avoid to create a new empty detail detail.label = NotAvailable elif re.match(base, line): # This string should come always after a date, # usually, it will match one of the cases above. # Sometimes, it appears on a new line we don't need. detail.label = NotAvailable elif re.match(ht, line): # '00,00 € HT par mois' may have been split after HT # We don't need of the second line detail.label = NotAvailable # Things we probably want to keep else: # Well, maybe our line is correct, after all. # Not much to do. detail.label = unicode(line, encoding='utf-8') detail.infos = NotAvailable elif ' %' in line: if isinstance(detail, Detail): # Sometimes the vat is not on a new line: # '00,00 00,0 %' instead of '00,0 %' vat = line.split()[line.count(' ')-1].replace(',', '.') detail.infos = unicode('TVA: ' + vat) elif ' €' in line: price = line.replace(',', '.') if isinstance(detail, Detail): detail.price = Decimal(price.strip(' €')) elif re.match(enddate, line): # Line holding dates may have been mixed up label = detail.label.split(' au ')[0] + u' au ' + unicode(line, encoding='utf-8') detail.label = label if detail.label is not NotAvailable: # Do not append empty details to the list # It seemed easier to create details anyway than dealing # with None objects details.append(detail) return details def get_details(self, label): txt = self._parse_pdf() page = None if label == u'Gaz naturel': page = txt.split('GAZ NATUREL')[1].split('TOTAL GAZ NATUREL TTC')[0] elif label == u'Electricité': page = txt.split('ELECTRICITE')[1].split('TOTAL ELECTRICITE TTC')[0] else: pass return self._parse_page(page) weboob-1.2/modules/gdfsuez/pages/homepage.py000066400000000000000000000047141303450110500212040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from weboob.deprecated.browser import Page from weboob.capabilities.bill import Subscription class LoginPage(Page): def login(self, login, password): self.browser.select_form('symConnexionForm') self.browser["portlet_login_plein_page_3{pageFlow.mForm.login}"] = unicode(login) self.browser["portlet_login_plein_page_3{pageFlow.mForm.password}"] = unicode(password) self.browser.submit() class HomePage(Page): def on_loaded(self): pass class AccountPage(Page): def get_subscription_list(self): table = self.document.xpath('//table[@id="ensemble_contrat_N0"]')[0] if len(table) > 0: # some clients may have subscriptions to gas and electricity, # but they receive a single bill # to avoid "boobill details" and "boobill bills" returning the same # table twice, we could return only one subscription for both. # We do not, and "boobill details" will take care of parsing only the # relevant section in the bill files. for line in table[0].xpath('//tbody/tr'): cells = line.xpath('td') snumber = cells[2].attrib['id'].replace('Contrat_', '') slabel = cells[0].xpath('a')[0].text.replace('offre', '').strip() d = unicode(cells[3].xpath('strong')[0].text.strip()) sdate = date(*reversed([int(x) for x in d.split("/")])) sub = Subscription(snumber) sub._id = snumber sub.label = slabel sub.subscriber = unicode(cells[1]) sub.renewdate = sdate yield sub class TimeoutPage(Page): def on_loaded(self): pass weboob-1.2/modules/gdfsuez/test.py000066400000000000000000000022251303450110500172720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Mathieu Jourdan # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . # This is a clone of freemobile/test.py for the gdfsuez module from weboob.tools.test import BackendTest class GdfSuezTest(BackendTest): MODULE = 'gdfsuez' def test_gdfsuez(self): for subscription in self.backend.iter_subscription(): list(self.backend.iter_history(subscription.id)) for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) weboob-1.2/modules/genericnewspaper/000077500000000000000000000000001303450110500176325ustar00rootroot00000000000000weboob-1.2/modules/genericnewspaper/__init__.py000066400000000000000000000001211303450110500217350ustar00rootroot00000000000000from .module import GenericNewspaperModule __all__ = ['GenericNewspaperModule'] weboob-1.2/modules/genericnewspaper/browser.py000066400000000000000000000024121303450110500216660ustar00rootroot00000000000000"browser for inrocks.fr website" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.url import URL from .pages import GenericNewsPage from weboob.browser import PagesBrowser class GenericPageBrowser(PagesBrowser): generic_news_page = URL(GenericNewsPage) weboob = None def is_logged(self): return False def login(self): pass def fillobj(self, obj, fields): pass def get_content(self, _id): "return page article content" self.location(_id) if self.page is not None: return self.page.get_article(_id) weboob-1.2/modules/genericnewspaper/favicon.png000066400000000000000000000022261303450110500217670ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME $ S2iTXtCommentCreated with GIMPd.eIDATxZ1$E}W1;h'&pفF+Ff]-jbdp^⮠p8sz7LwWo=׎=}vo?`jjUtСC>m}$(hYp[5`UU }jzg0s)AQ]{"rsi|SDNU߈rAw\Q xMҙ a6d,J"D=" ,=0 od\Y(&er }t Ǻ8.Tf٩WJR+%#x\M&R523Zx\i=Z[$D H#Ox1憈89Wk'"r$VUܲZ1"BVuʄ~@8M+ȍB[!JD<[@^ ]<!M )瞹TJa4gRa~1 @[|LUFkrZD_ȹ6_pp eKPEbE&j^p.g"$03OngZ$1Ck-sp]Խqk#`uuh29MW.E|QEru [[[H|}oo oUWY^.M1^٥U=)o3q{U9Eѫ)o=q_=> FD!]. import time from weboob.capabilities.messages import Message, Thread from weboob.capabilities.base import find_object from weboob.tools.backend import Module from weboob.tools.newsfeed import Newsfeed from .browser import GenericPageBrowser class GenericNewspaperModule(Module): """ GenericNewspaperModule class """ NAME = u'genericnewspaper' MAINTAINER = u'Julien Hebert' DESCRIPTION = u'Generic module that helps to handle newspapers modules' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} RSS_FEED = None RSSID = None URL2ID = None RSSSIZE = 0 BROWSER = GenericPageBrowser def create_default_browser(self): return self.create_browser(self.weboob, self.weboob) def get_thread(self, _id): if isinstance(_id, Thread): thread = _id id = thread.id else: thread = find_object(self.iter_threads(), id=_id) id = _id content = self.browser.get_content(id) if content is None: return None if not thread: thread = Thread(id) flags = Message.IS_HTML if thread.id not in self.storage.get('seen', default={}): flags |= Message.IS_UNREAD thread.title = content.title if not thread.date: thread.date = content.date thread.root = Message( thread=thread, id=0, url=content.url, title=content.title, sender=content.author, receivers=None, date=thread.date, parent=None, content=content.body, signature=u'URL \n' % content.url, flags=flags, children=[]) return thread def iter_threads(self): for article in Newsfeed(self.RSS_FEED, GenericNewspaperModule.RSSID).iter_entries(): thread = Thread(article.id) thread.title = article.title thread.date = article.datetime yield(thread) def fill_thread(self, thread, fields): "fill the thread" t = self.get_thread(thread) return t or thread def iter_unread_messages(self): for thread in self.iter_threads(): if thread.id in self.storage.get('seen', default={}): continue self.fill_thread(thread, 'root') for msg in thread.iter_all_messages(): yield msg def set_message_read(self, message): self.storage.set( 'seen', message.thread.id, 'comments', self.storage.get( 'seen', message.thread.id, 'comments', default=[]) + [message.id]) if self.URL2ID and self.RSSSIZE != 0: url2id = self.URL2ID lastpurge = self.storage.get('lastpurge', default=0) l = [] if time.time() - lastpurge > 7200: self.storage.set('lastpurge', time.time()) for id in self.storage.get('seen', default={}): l.append((int(url2id(id)), id)) l.sort() l.reverse() tosave = [v[1] for v in l[0:self.RSSSIZE + 10]] toremove = set([v for v in self.storage.get('seen', default={})]).difference(tosave) for id in toremove: self.storage.delete('seen', id) self.storage.save() OBJECTS = {Thread: fill_thread} weboob-1.2/modules/genericnewspaper/pages.py000066400000000000000000000113441303450110500213060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage from weboob.browser.filters.html import XPath, XPathNotFound from weboob.browser.filters.standard import CleanText from lxml.etree import Comment class Article(object): author = u'' title = u'' def __init__(self, browser, _id): self.browser = browser self.id = _id self.body = u'' self.url = u'' self.date = None class GenericNewsPage(HTMLPage): __element_body = NotImplementedError __article = Article element_title_selector = NotImplementedError main_div = NotImplementedError element_body_selector = NotImplementedError element_author_selector = NotImplementedError _selector = XPath def on_load(self): self.handle_refresh() self.on_loaded() def on_loaded(self): pass def get_body(self): try: return CleanText('.')(self.get_element_body()) except (AttributeError): return self.__article.body def get_author(self): try: return CleanText('.')(self.get_element_author()) except (AttributeError): return self.__article.author def get_title(self): try: return CleanText(self._selector(self.element_title_selector))(self.main_div) except AttributeError: if self.main_div is None: raise XPathNotFound("main_div is none on %s" % (self.browser)) elif self.element_title_selector != 'h1': self.element_title_selector = 'h1' return self.get_title() else: raise AttributeError("no title on %s" % (self.browser)) def get_element_body(self): try: return self._selector(self.element_body_selector)(self.main_div)[0] except (AttributeError, IndexError): if self.main_div is None: raise XPathNotFound("main_div is none on %s" % (self.browser)) else: raise AttributeError("no body on %s" % (self.browser)) def get_element_author(self): try: return self._selector(self.element_author_selector)(self.main_div)[0] except IndexError: if self.main_div is None: raise XPathNotFound("main_div is none on %s" % (self.browser)) else: raise AttributeError("no author on %s" % (self.browser)) def get_article(self, _id): __article = Article(self.browser, _id) __article.author = self.get_author() __article.title = self.get_title() __article.url = self.url __article.body = self.get_body() return __article def drop_comments(self, base_element): for comment in base_element.getiterator(Comment): comment.drop_tree() def try_remove(self, base_element, selector): for el in self._selector(selector)(base_element): try: el.getparent().remove(el) except (AttributeError, ValueError): continue def remove_from_selector_list(self, base_element, selector_list): for selector in selector_list: base_element.remove(self._selector(selector)(base_element)) def try_remove_from_selector_list(self, base_element, selector_list): for selector in selector_list: self.try_remove(base_element, selector) def try_drop_tree(self, base_element, selector): for el in self._selector(selector)(base_element): el.drop_tree() @staticmethod def clean_relativ_urls(base_element, domain): for a in base_element.findall('.//a'): if "href" in a.attrib: if a.attrib["href"] and a.attrib["href"][0:7] != "http://" and a.attrib["href"][0:7] != "https://": a.attrib["href"] = domain + a.attrib["href"] for img in base_element.findall('.//img'): if img.attrib["src"][0:7] != "http://" and img.attrib["src"][0:7] != "https://": img.attrib["src"] = domain + img.attrib["src"] weboob-1.2/modules/geolocip/000077500000000000000000000000001303450110500160725ustar00rootroot00000000000000weboob-1.2/modules/geolocip/__init__.py000066400000000000000000000001011303450110500201730ustar00rootroot00000000000000from .module import GeolocIpModule __all__ = ['GeolocIpModule'] weboob-1.2/modules/geolocip/favicon.png000066400000000000000000000074121303450110500202310ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGD pHYs  tIME  65\CIDATx[}y3;{c[@#0v(PLi * IZi5mP"DU#*&p0'@m\q;;_cgsgu|O#kvgs"sqC_)2JJq.֞miƼ~Uk= lZ+/44[Ve,1\L)- uRPZC)c̄oZq!h`WjRRzp a-5R)RBk c RB fW'or JV[S(7uoPmMZD:z,%1F)EBǬN;Z cҋ.ܽ;RE0R Z%0LjjA }Z uQ)bB~-,|+sŸŰց1B m)ɣk#5A)y=1RJ+c+vw?] k{RU#"!D,e[@Z ڒF h%c!.<σcyiR޷kf=0JAGQxc,%1 UZt_1!ǀ` P1z9\ׅ8aBDAp熛o@Cp.DH)Ɓ(/B\[` GD0  H!` )0s(Bͣ従 =ϥ̇#< 8¼[CԽPmK8LMV´"a> 80|2+}wG>vy ofEEH Q!5BVBr|cO#Qp] a* =:5V000m]ZGW_[3$Zm_=V)(:rp,DtnI (TBXM~фCLfiq7a`oZVZ<"DQ,xau4+=[`>Xs5?j/_;lΌXoV> O־R " uBeEPԟ]Wa8t)¡.q{-oI ZBcL;I:)%Sf]s_:߃Z8)C=ϻy7Iuc.IR zP?{̹GݶL$ ,qY˱;rW  aMi&X١{;c̢pI T SUǂ৛_bF}b2I l11Jymy]]^2,=<ďݍsD~BN]J;奵ZBWvsY5R\1ZkO2uqpj@9vrSgǻlwbYk_Mk.>JVmշ!Bx!FB@&#I06ޝZ#V*( ZAdG|cCQr8' AGބe~d/H82y0k`G$.sq6ϢBs.VsϹ1 C Fg^?߾@+zǶnu74[AtGd )Z h3*HLb>cp!mLL~&twU 1ǙZAn=:9:ѽa&8Fzߎ:nORzkEB8GFIX 1>{aJ3Zظd&@& yqhdfL5 |B8s_(8ic ,$ߑMU ݸt}ׅJ҄(  "# j1 ?qk-hFH<84ow&ٌw=Zk.>-x\c,[i/MiɬRGGGOe %2EFC;^{n^9fժ!o/qNXaFAbD?}[we籔F@9x @#`^h6{/7np@k} r6fi:gC?80]&x<;0\euRJaxhl6:6w2 o{ccD "4&& R6_~1$gKOIENDB`weboob-1.2/modules/geolocip/module.py000066400000000000000000000046471303450110500177440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.geolocip import CapGeolocIp, IpLocation from weboob.tools.backend import Module from weboob.deprecated.browser import Browser, BrowserUnavailable __all__ = ['GeolocIpModule'] class GeolocIpModule(Module, CapGeolocIp): NAME = 'geolocip' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"GeolocIP IP addresses geolocation service" BROWSER = Browser def get_location(self, ipaddr): with self.browser: content = self.browser.readurl('http://www.geolocip.com/?s[ip]=%s&commit=locate+IP!' % str(ipaddr)) if content is None: raise BrowserUnavailable() tab = {} last_line = '' line = '' for line in content.split('\n'): if len(line.split('

')) > 1: key = last_line.split('
')[1].split('
')[0][0:-2] value = line.split('
')[1].split('
')[0] tab[key] = value last_line = line iploc = IpLocation(ipaddr) iploc.city = u'%s'%tab['City'] iploc.region = u'%s'%tab['Region'] iploc.zipcode = u'%s'%tab['Postal code'] iploc.country = u'%s'%tab['Country name'] if tab['Latitude'] != '': iploc.lt = float(tab['Latitude']) else: iploc.lt = 0.0 if tab['Longitude'] != '': iploc.lg = float(tab['Longitude']) else: iploc.lg = 0.0 #iploc.host = 'NA' #iploc.tld = 'NA' #iploc.isp = 'NA' return iploc weboob-1.2/modules/geolocip/test.py000066400000000000000000000016301303450110500174230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GeolocIPTest(BackendTest): MODULE = 'geolocip' def test_geolocip(self): self.backend.get_location('88.198.11.130') weboob-1.2/modules/github/000077500000000000000000000000001303450110500155535ustar00rootroot00000000000000weboob-1.2/modules/github/__init__.py000066400000000000000000000014321303450110500176640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GithubModule __all__ = ['GithubModule'] weboob-1.2/modules/github/browser.py000066400000000000000000000203571303450110500176170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime import re import os from urllib import quote_plus from weboob.browser.browsers import APIBrowser from weboob.browser.cache import CacheMixin from weboob.browser.exceptions import ClientError __all__ = ['GithubBrowser'] class GithubBrowser(CacheMixin, APIBrowser): BASEURL = 'https://api.github.com' def __init__(self, username, password, *a, **kw): super(GithubBrowser, self).__init__(*a, **kw) self.username = username self.password = password self.fewer_requests = not bool(self.username) def get_project(self, project_id): json = self.request('https://api.github.com/repos/%s' % project_id) return { 'name': json['name'], 'id': project_id } def get_issue(self, project_id, issue_number): json = self.request('https://api.github.com/repos/%s/issues/%s' % (project_id, issue_number)) return self._make_issue(project_id, issue_number, json) def iter_project_issues(self, project_id): base_url = 'https://api.github.com/repos/%s/issues' % project_id for json in self._paginated(base_url): for jissue in json: issue_number = jissue['number'] yield self._make_issue(project_id, issue_number, jissue) if len(json) < 100: break def iter_issues(self, query): qsparts = ['repo:%s' % query.project] if query.assignee: qsparts.append('assignee:%s' % query.assignee) if query.author: qsparts.append('author:%s' % query.author) if query.status: qsparts.append('state:%s' % query.status) if query.title: qsparts.append('%s in:title' % query.title) qs = quote_plus(' '.join(qsparts)) base_url = 'https://api.github.com/search/issues?q=%s' % qs for json in self._paginated(base_url): for jissue in json['items']: issue_number = jissue['number'] yield self._make_issue(query.project, issue_number, jissue) if not len(json['items']): break def post_issue(self, issue): base_data = self._issue_post_data(issue) url = 'https://api.github.com/repos/%s/issues' % issue.project.id json = self.request(url, data=base_data) issue_number = json['number'] return self._make_issue(issue.project.id, issue_number, json) def edit_issue(self, issue, issue_number): base_data = self._issue_post_data(issue) url = 'https://api.github.com/repos/%s/issues/%s' % (issue.project.id, issue_number) self.open(url, data=base_data, method='PATCH') return issue def _issue_post_data(self, issue): data = { 'title': issue.title, 'body': issue.body } if issue.assignee: data['assignee'] = issue.assignee.id if issue.version: data['milestone'] = issue.version.id if issue.status: data['state'] = issue.status.name # TODO improve if more statuses are implemented return data def post_comment(self, issue_id, comment): project_id, issue_number = issue_id.rsplit('/', 1) url = 'https://api.github.com/repos/%s/issues/%s/comments' % (project_id, issue_number) data = {'body': comment} self.request(url, data=data) # helpers def _make_issue(self, project_id, issue_number, json): d = {} d['number'] = issue_number d['title'] = json['title'] d['body'] = json['body'].strip() d['creation'] = parse_date(json['created_at']) d['updated'] = parse_date(json['updated_at']) d['author'] = json['user']['login'] d['status'] = json['state'] d['url'] = 'https://github.com/%s/issues/%s' % (project_id, issue_number) if json['assignee']: d['assignee'] = json['assignee']['login'] else: d['assignee'] = None if json['milestone']: d['version'] = json['milestone'] else: d['version'] = None d['has_comments'] = (json['comments'] > 0) d['attachments'] = list(self._extract_attachments(d['body'])) # TODO fetch other updates? return d def iter_milestones(self, project_id): url = 'https://api.github.com/repos/%s/milestones' % project_id for jmilestone in self.request(url): yield { 'id': jmilestone['number'], 'name': jmilestone['title'] } def iter_comments(self, project_id, issue_number): url = 'https://api.github.com/repos/%s/issues/%s/comments' % (project_id, issue_number) for jcomment in self.request(url): d = {} d['id'] = jcomment['id'] d['message'] = jcomment['body'] d['author'] = jcomment['user']['login'] d['date'] = parse_date(jcomment['created_at']) d['attachments'] = list(self._extract_attachments(d['message'])) yield d def _extract_attachments(self, message): for attach_url in re.findall(r'https://f.cloud.github.com/assets/[\w/.-]+', message): yield { 'url': attach_url, 'filename': os.path.basename(attach_url) } def _paginated(self, url, start_at=1): while True: if '?' in url: page_url = '%s&per_page=100&page=%s' % (url, start_at) else: page_url = '%s?per_page=100&page=%s' % (url, start_at) yield self.request(page_url) start_at += 1 def get_user(self, _id): json = self.request('https://api.github.com/users/%s' % _id) if 'name' in json: name = json['name'] else: name = _id # wasted one request... return { 'id': _id, 'name': name } def iter_members(self, project_id): url = 'https://api.github.com/repos/%s/assignees' % project_id for json in self._paginated(url): for jmember in json: yield { 'id': jmember['login'], 'name': jmember['login'] } if len(json) < 100: break def get_rate_limit(self): return self.request('/rate_limit') def _extract_rate_info(self, headers): left = headers.get('X-RateLimit-Remaining') total = headers.get('X-RateLimit-Limit') end = headers.get('X-RateLimit-Reset') return left, total, end def open(self, *args, **kwargs): if 'headers' not in kwargs: kwargs['headers'] = {} kwargs['headers']['Accept'] = 'application/vnd.github.v3+json' kwargs.update(**self.auth_headers()) left = total = end = None try: ret = super(GithubBrowser, self).open_with_cache(*args, **kwargs) except ClientError as err: left, total, end = self._extract_rate_info(err.response.headers) raise else: left, total, end = self._extract_rate_info(ret.headers) finally: self.logger.debug('github API request quota: %s/%s (end at %s)', left, total, end) return ret def auth_headers(self): if self.username: return {'auth': (self.username, self.password)} else: return {} # TODO use a cache for objects and/or pages? def parse_date(s): if s.endswith('Z'): s = s[:-1] return datetime.datetime.strptime(s, '%Y-%m-%dT%H:%M:%S') weboob-1.2/modules/github/favicon.png000066400000000000000000000056371303450110500177210ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME1tEXtCommentCreated with GIMPW IDATxY]U>sGRE(PE!Q,F4 q$1<фD$jbL4 &&*)Rӝˇ_bsNoe';tk_ kBG!~  L ` 8 8R$4]gփ)R `BX̙ٽ 4`8=pvC i 8̞y2A`uB=o@'7fv 'pȾT>X4&}2O 7ٳ -9 49|3,3Յd;[lӁfÁk3u K`^X$ )0|̞: irۗWc'074t0|Jy=yqXw-n)~k_ KJc&1YS6Jpg8,@\.2KdNLeD$`_ؓ@jh^ߖDˁƿ`f3fmF_|Pn53p}oI$XfP~0\# }Ao˔}jt$o ;뎶>PfmH/~p3S"֔.YeT2<(k>p5h'̩1׶ɭݚ*G뤏i. '  Rƴ]目#!dB!y8c1e&X\-ӬF]r-~Bq">zs0\( ѽv}w2T;v\&XeӎςВQIk3J/¡ݪYjؘScE쥱D{w/d/,f*-EEրgԡ}8iz̪pl *3+} p2mE\dUͽ^vKjgH~'?_sJN8X`# %Šm9N,JGZJ|"8=: c*v(S#ҭs]V*XPÃ}%T%w3E.72!V nPC VUgFIہfSRqzsP l|H:")bDgmpRejԽӀqeq敚b1J@gsxsemU]ѐeμ$Z)o^B+d*07MC>r+0@>NmD-ͅՓ ѧot3> -]<1%8J/r~| x0N*5$%KT5>YxH PyI5!3J|x6'6CW|4~lzD2@<]ͻ{;~:{ww EkVvnQ]+JOSBU$wָX 0Y'3r>tJwGB#r;a !,,kV{啪x 什L rƏl}gJbATxx6ь7+Q.D'Yt"?@W!A3 NOuYhu?|4epőP8WKîL`UIp>tѣ^k<ݐ>uSw gw[ } <Ta;; nΕR{Vgr/>Z l1a(yt7{v4%qZWh'|~ \j3n]y6KڇNQZ|ŕS=y>|? +iLhgEn1=rcN-D%9L/@\[*Q3q1OY܇Q?9t\8LZѯL՞i@jiq9ȜX |Yl; i6kuX*=XR[_V.) !k=3cjVG,$_nnGҎUF\-14W2WGx)ɮ uXNK)-M |FީT$ cW*r2-!ڤkX:uY٣2W >{侉J=]vmO6v [kP'Eutq6E.nк.)pM>j~cktdR U9ETJIaer nA91>\߱^:c,Γ7X;_'--1 YyЄfgGUI]qX/>|*R㬕¯SI|Z=SƩ"ۢ%8Qiz76[4lɵ]`@6z_k'[B=v (>\AW!~ "vQtAIENDB`weboob-1.2/modules/github/module.py000066400000000000000000000140761303450110500174220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.bugtracker import CapBugTracker, Issue, Project, User, Version, Status, Update, Attachment from .browser import GithubBrowser __all__ = ['GithubModule'] STATUSES = {'open': Status('open', u'open', Status.VALUE_NEW), 'closed': Status('closed', u'closed', Status.VALUE_RESOLVED)} # TODO tentatively parse github "labels"? class GithubModule(Module, CapBugTracker): NAME = 'github' DESCRIPTION = u'GitHub issues tracking' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default='')) BROWSER = GithubBrowser def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(username, password) def get_project(self, _id): d = self.browser.get_project(_id) project = Project(_id, d['name']) project.members = list(self._iter_members(project.id)) project.statuses = list(STATUSES.values()) project.categories = [] project.versions = list(self._iter_versions(project.id)) return project def get_issue(self, _id): project_id, issue_number = self._extract_issue_id(_id) project = self.get_project(project_id) d = self.browser.get_issue(project_id, issue_number) issue = self._make_issue(d, project) if d['has_comments']: self._fetch_comments(issue) return issue def iter_issues(self, query): if ((query.assignee, query.author, query.status, query.title) == (None, None, None, None)): it = self.browser.iter_project_issues(query.project) else: it = self.browser.iter_issues(query) project = self.get_project(query.project) for d in it: issue = self._make_issue(d, project) yield issue def create_issue(self, project_id): issue = Issue(0) issue.project = self.get_project(project_id) return issue def post_issue(self, issue): assert not issue.attachments if issue.id and issue.id != '0': _, issue_number = self._extract_issue_id(issue.id) self.browser.edit_issue(issue, issue_number) else: d = self.browser.post_issue(issue) issue.id = self._build_issue_id(issue.project.id, d['number']) return issue def update_issue(self, issue_id, update): try: issue_id = issue_id.id except AttributeError: pass assert not update.attachments self.browser.post_comment(issue_id, update.message) # iter_projects, remove_issue are impossible def _iter_members(self, project_id): for d in self.browser.iter_members(project_id): yield User(d['id'], d['name']) def _iter_versions(self, project_id): for d in self.browser.iter_milestones(project_id): yield Version(d['id'], d['name']) def _make_issue(self, d, project): _id = self._build_issue_id(project.id, d['number']) issue = Issue(_id) issue.url = d['url'] issue.project = project issue.title = d['title'] issue.body = d['body'] issue.creation = d['creation'] issue.updated = d['updated'] issue.author = project.find_user(d['author'], None) if not issue.author: # may duplicate users issue.author = User(d['author'], d['author']) issue.status = STATUSES[d['status']] if d['assignee']: issue.assignee = project.find_user(d['assignee'], None) else: issue.assignee = None if d['version']: issue.version = project.find_version(d['version'], None) else: issue.version = None issue.category = None issue.attachments = [self._make_attachment(dattach) for dattach in d['attachments']] return issue def _fetch_comments(self, issue): project_id, issue_number = self._extract_issue_id(issue.id) if not issue.history: issue.history = [] issue.history += [self._make_comment(dcomment, issue.project) for dcomment in self.browser.iter_comments(project_id, issue_number)] def _make_attachment(self, d): a = Attachment(d['url']) a.url = d['url'] a.filename = d['filename'] return a def _make_comment(self, d, project): u = Update(d['id']) u.message = d['message'] u.author = project.find_user(d['author'], None) if not u.author: # may duplicate users u.author = User(d['author'], d['author']) u.date = d['date'] u.changes = [] u.attachments = [self._make_attachment(dattach) for dattach in d['attachments']] return u @staticmethod def _extract_issue_id(_id): return _id.rsplit('/', 1) @staticmethod def _build_issue_id(project_id, issue_number): return '%s/%s' % (project_id, issue_number) weboob-1.2/modules/github/test.py000066400000000000000000000076241303450110500171150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import unicode_literals from time import time from weboob.tools.test import BackendTest, skip_without_config from weboob.capabilities.bugtracker import Query, Version, User, Status, Update class GithubTest(BackendTest): MODULE = 'github' def test_project(self): project = self.backend.get_project('weboobie/testing') assert project self.assertEqual(project.name, 'testing') self.assertEqual(project.id, 'weboobie/testing') assert all(isinstance(user, User) for user in project.members) assert any(user.name == 'weboobie' for user in project.members) assert all(isinstance(version, Version) for version in project.versions) assert any(version.name == u'1.0' for version in project.versions) assert project.find_status('open').value == Status.VALUE_NEW assert project.find_status('closed').value == Status.VALUE_RESOLVED def test_get_issue(self): issue = self.backend.get_issue('weboobie/testing/1') assert issue self.assertEqual(issue.id, 'weboobie/testing/1') self.assertEqual(issue.title, 'an open issue') assert 'Hello' in issue.body assert issue.creation assert issue.history def test_search(self): query = Query() query.project = 'weboobie/testing' query.status = 'closed' query.title = 'fix' issues = self.backend.iter_issues(query) issue = issues.next() assert issue.status.name == 'closed' assert 'fix' in issue.title @skip_without_config('username', 'password') def test_post_issue(self): project = self.backend.get_project('weboobie/testing') assert project issue = self.backend.create_issue(project.id) issue.title = 'posting an issue' issue.body = 'body of the issue' issue.version = project.versions[0] self.backend.post_issue(issue) assert issue.id fetched = self.backend.get_issue(issue.id) self.assertEqual(issue.title, fetched.title) self.assertEqual(issue.body, fetched.body) self.assertEqual(fetched.status.name, 'open') @skip_without_config('username', 'password') def test_post_comment(self): issue = self.backend.get_issue('weboobie/testing/26') assert issue ts = str(int(time())) update = Update(0) update.message = "Yes! It's now %s" % ts self.backend.update_issue(issue, update) new = self.backend.get_issue('weboobie/testing/26') assert any(ts in upd.message for upd in new.history) @skip_without_config('username', 'password') def test_change_status(self): issue = self.backend.get_issue('weboobie/testing/30') assert issue closing = (issue.status.name != 'closed') if closing: issue.status = issue.project.find_status('closed') else: issue.status = issue.project.find_status('open') self.backend.post_issue(issue) new = self.backend.get_issue('weboobie/testing/30') if closing: self.assertEqual(new.status.name, 'closed') else: self.assertEqual(new.status.name, 'open') weboob-1.2/modules/gls/000077500000000000000000000000001303450110500150565ustar00rootroot00000000000000weboob-1.2/modules/gls/__init__.py000066400000000000000000000014311303450110500171660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GLSModule __all__ = ['GLSModule'] weboob-1.2/modules/gls/browser.py000066400000000000000000000024011303450110500171100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from weboob.browser.exceptions import HTTPNotFound from weboob.capabilities.parcel import ParcelNotFound from .pages import SearchPage class GLSBrowser(PagesBrowser): BASEURL = 'https://gls-group.eu' search_page = URL('/app/service/open/rest/EU/en/rstt001\?match=(?P.+)', SearchPage) def get_tracking_info(self, _id): try: return self.search_page.go(id=_id).get_info(_id) except HTTPNotFound: raise ParcelNotFound("No such ID: %s" % _id) weboob-1.2/modules/gls/favicon.png000066400000000000000000000006501303450110500172120ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME882bvtEXtCommentCreated with GIMPW"IDATh0 D[QPUn!LIw˶̼h|_ynm ZqR9$/@:#>doX z-Bz>?%%2 Hif0E+t4F֥sG_Yl Bڲ? @҃@cS]wF]% l*QbJwY"9Hjc4"<*f\!*#cN{Ȭ7tsg%IENDB`weboob-1.2/modules/gls/module.py000066400000000000000000000026021303450110500167150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.parcel import CapParcel from .browser import GLSBrowser __all__ = ['GLSModule'] class GLSModule(Module, CapParcel): NAME = 'gls' DESCRIPTION = u'GLS website' MAINTAINER = u'Matthieu Weber' EMAIL = 'mweber+weboob@free.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = GLSBrowser def get_parcel_tracking(self, id): """ Get information abouut a parcel. :param id: ID of the parcel :type id: :class:`str` :rtype: :class:`Parcel` :raises: :class:`ParcelNotFound` """ return self.browser.get_tracking_info(id) weboob-1.2/modules/gls/pages.py000066400000000000000000000034771303450110500165420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from dateutil.parser import parse as parse_date from weboob.browser.pages import JsonPage from weboob.capabilities.parcel import Parcel, Event STATUSES = { "DELIVEREDPS": Parcel.STATUS_ARRIVED, "DELIVERED": Parcel.STATUS_ARRIVED, } class SearchPage(JsonPage): def get_info(self, _id): p = Parcel(_id) events = self.doc["tuStatus"][0]["history"] p.history = [self.build_event(i, tr) for i, tr in enumerate(events)] p.status = self.guess_status(self.doc["tuStatus"][0]["progressBar"]["statusInfo"]) p.info = self.doc["tuStatus"][0]["progressBar"]["statusText"] return p def guess_status(self, code): return STATUSES.get(code, Parcel.STATUS_UNKNOWN) def build_event(self, index, data): event = Event(index) date = "%s %s" % (data["date"], data["time"]) event.date = parse_date(date, dayfirst=False) event.location = ", ".join( [unicode(data["address"][field]) for field in ["city", "countryName"] if data["address"][field]]) event.activity = unicode(data["evtDscr"]) return event weboob-1.2/modules/gls/test.py000066400000000000000000000017361303450110500164160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.capabilities.parcel import ParcelNotFound class GLSTest(BackendTest): MODULE = 'gls' def test_gls(self): self.assertRaises(ParcelNotFound, self.backend.get_parcel_tracking, "foo") weboob-1.2/modules/googletranslate/000077500000000000000000000000001303450110500174635ustar00rootroot00000000000000weboob-1.2/modules/googletranslate/__init__.py000066400000000000000000000015111303450110500215720ustar00rootroot00000000000000"GoogleTranslateModule init" # -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GoogleTranslateModule __all__ = ['GoogleTranslateModule'] weboob-1.2/modules/googletranslate/browser.py000066400000000000000000000031031303450110500215150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import TranslatePage from .gtts_token import Token __all__ = ['GoogleTranslateBrowser'] class GoogleTranslateBrowser(PagesBrowser): BASEURL = 'https://translate.google.fr' translate_page = URL('/translate_a/single\?client=t&sl=(?P.*)&tl=(?P.*)&dt=t&tk=(?P.*)&q=(?P.*)&ie=UTF-8&oe=UTF-8', TranslatePage) def translate(self, source, to, text): """ translate 'text' from 'source' language to 'to' language """ t = text.encode('utf-8') tk = Token().calculate_token(t) return self.translate_page.go(source=source.encode('utf-8'), to=to.encode('utf-8'), text=t, token=tk).get_translation() weboob-1.2/modules/googletranslate/favicon.png000066400000000000000000000175221303450110500216250ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDf` pHYs  tIME(1mIDATx{wﯪtOټZ*ȀDȟ1`l 1ZK€{6 2"a![#K(vV]l3ө16H={}N鮮&8`zN ND`fD_LrzBd4LKvXHD |i !I.5!0}Zu'3ܲ"RY<6lnz:>e {{t`?1Et̿>A^ `ÛOyu*1B2&FLnjh6O􃂰W.Xr& (,0&0(wX`"؆$\ܝXD0kQU8[_v;vO.pC$ڣ*T4It%788A/Y,M=xT16| 7a+RQZ @gn"K'bv/Yp(]yLG/LZx'm%Mfpo rTyV#<ߺT26+tpkkᎺښܥVOa{v]$ W}J,{ hZsدU$jn>.Wq^Q7N|餷D˨z|3CW9cn{m6y|-ɓo9m#,Jq;"dyZ>wiʚi}s63S]3Hu1TygLMa& L 2$x|yܡecLjk~z>s-KQ\PG V^R64`}83ZFO-є `"eLڪ ?>|lft7ނ{/QpyTZ7X>ⱏ* ?9~)KV㬪P `bSx Q@fl!8+VV?gfK啮?Z1u8׺V-EEP:;Ujg_k־|GCʎ̨(\[67WdWߣA ߼@E`)H  \j}c*' Ͻ{8@ka@&82EmʪyISl;mw~ׁq;v|2v>dC*bWsPP"{`95zY.8-u@pHq{# Z2q[ڄUpBr`T@v{_sG-s..<28C))GC*\?!;Z$u 4oIJJO\y54iMX-ڀo'NHj]o9C0!A#Y >}ݯ1tܡ_OxDH'mDD8f8w:0{lZ7߬])!70v$C RXP%:}Ë!L|)9熗37켏=⫋v_w[EԄ#z|Vq2 DȈoy>UQJc6C6x|{.\ `}oE'YWl^b0ުU\[߼yG#{o햤5v)mkϜX_r]mmm mS2W-Y}Q3YܢL0IvG~zw~f6xut}noSKu:Z(*nWŎsno)$;xuu}:wlN +(J`&{ÂQ2 I dd¬C DPYQ{k㽅ZAPf/qe3/n񋗷}oΠibls tzlI{ZI%Pjq0-ʲb4X4\WQ*r1YU6sZ?Y}o8ŝ$1?c?`;S5xW~p{wު&(fqd8(%[_/HW]Z`vnai !w׷ɧ088+]2LYqʮuʎ - )m3N0)-HUu}JDsp2K۶8$6Fl]c)c';yZC"OwPQ:QQO1*%NGr -C%r-2R=%dI'(K.Pm3{DLkR )r hzJ"ˑQW' $|\ӓB`?v=n{x}Kiܜ ] /zf%),cqS*>1)liۆ淃BՅߞvw3%(8TALJ١ZC]N95XBflYV4Iv+lJenn5™Jp_WGIlxKq$_? Ge>Ǐ6f2)^SݸG˼-faJĢnt;3pPQ%Š"&!Jӯxd[-I rsV=er%&\K'%J eN.mߟd՝3kGeY56} G" *{FE ")BV3Hy.8]>aIzu#kctWM3C3 R $M$]m Ù|%mN ڢN./-{y9o}&sOK$kޒLD)81NsiN"茁d9\@Ꙥ'1ʸM|P)+Jxܭzcl}neU DQ0B ]1˝]dZIѾ[iv{U=% 9 PX ?/tflK8E 8!WĚ>sScOw UT%QK9q` "۶ՋPdzJ~eǣ2/ׯxcbWx;]ȗj Ts{]3][]sڌI cUQK1SlLzr㱽:yoGLAqyk',~ĺe2(MdV.,٨T6:--:%SObZ+MLu|zvU KJ OmQYZ|8&YO^Pퟵ!d.ޕgfhC|(_*a3w I5㬻ɗ47}]BLUUþntG3_v)<7|Cc43 $IVlMA˲j-3 ]t*S|mSlb˝  "@-'U ʒCMBho/hjKJ2_nlyWաDiB-SR[Iʂ1[.u^)GHGމźTzMeyYZz@+;+տ>}mM'~̻Z-T̿Օ EةT\;(R7{*BedW3tև6iVN(kߟ6w]a!)R=XX0oYd"aܱol[O:;Z2FK9P*RG^Cۻw+1(!}~Ot6V Ų^XQ2M/w~\>NխdTJwTD) x%9m.J 4{AɎfi)CthA~SGo0_H0.iY_+xN:NWe۪88IX/Y$ 4 wBE !Vo|OY#9cmf(D˶(cRqX{ OF-ir|+9S/fm68 ̼1MgB-S f#~"| 6' 0bQJ Eq+[?uyDp0C{MOL- ;vM:ƖO~;Z뇔;M4qlX"4 PhB6~ܔK^?!0b[ @e`[R>zPfW2;-@퐔S[Y1^o{R59!bҦtZ沑T] "88,"ɡ`mM1N  2/oU/v3etp rm'm}xkmusm0DZpE!^b\_[" `!*o%,[7Ƴα &3-q޳f۶ٌE՚1DXO0a,U򱂓f*\7-T,9TFh:,ǝ0y(߯$wG/T(ɁO̾}ۓz&3n-mٲ#VS!phj鴦tկ߰aKTMSgWvj)+ +;8tםRlXT̈;5, m4ɩ@a,ˣqAҿbpoNhjbw\= _P͋R{0kK^q/V6W9v_Վ82pf걻"q#RK^C 't \2zfs=vN"6m[?6΅DW{_R\K)~@p z9fwfEmĕ:S\ߚ? $"j"ٯ`̨&d)*JG'k2 |h 2Xj)t8P"T_6h a=&?(Twbb~OżYhʺfIR Mlø_s ۜ[۶97H5/~%W@F.bTӑV$X B.JC_nྂr_@ xN妿תC ".XE* 5[ԅ," &! A'}SF0Ti&V46l/. C`,g'o95ze(…-s(M i")Jh4)b-ksQ+pͪѪNKÎ89|;?(39p( -d'||Pj.z'~LżCSB%RIs(77_) "ʑOJvs7::\B幑η$;4bZfF'̭T8żeE \rt)DymMgRn`̐[~45/&Ou#-w?{> o B.lHU.{kƔ`w|K>Vr sʀ+Ǐ_).F##B.'h⏞D<f6ڿ7LGRHpjS#= GkV) ( !!7爺d*ܪЌY_7/uu++&_qߡas`[ hemk'[zҏCEA1uWdi,,gFj(~A9L='|^YpF )ª$ R=KD\'+3nMwgź?9f;\ASdςoUt7NYq\m=%`:i]V|́~v{bN\WvUV\? @A13Ot*H"hi6%۲-ݰ w6X:ߟ#rPG+<$ٜ#=IENDB`weboob-1.2/modules/googletranslate/gtts_token.py000066400000000000000000000050301303450110500222140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # https://github.com/Boudewijn26/gTTS-token import calendar import math import time import requests import re class Token(object): """ Token (Google Translate Token) Generate the current token key and allows generation of tokens (tk) with it Python version of `token-script.js` itself from translate.google.com """ SALT_1 = "+-a^+6" SALT_2 = "+-3^+b+-f" def __init__(self): self.token_key = None def calculate_token(self, text, seed=None): """ Calculate the request token (`tk`) of a string :param text: str The text to calculate a token for :param seed: str The seed to use. By default this is the number of hours since epoch """ if seed is None: seed = self._get_token_key() [first_seed, second_seed] = seed.split(".") try: d = bytearray(text.encode('UTF-8')) except UnicodeDecodeError: # This will probably only occur when d is actually a str containing UTF-8 chars, which means we don't need # to encode. d = bytearray(text) a = int(first_seed) for value in d: a += value a = self._work_token(a, self.SALT_1) a = self._work_token(a, self.SALT_2) a ^= int(second_seed) if 0 > a: a = (a & 2147483647) + 2147483648 a %= 1E6 a = int(a) return str(a) + "." + str(a ^ int(first_seed)) def _get_token_key(self): if self.token_key is not None: return self.token_key timestamp = calendar.timegm(time.gmtime()) hours = int(math.floor(timestamp / 3600)) response = requests.get("https://translate.google.com/") line = response.text.split('\n')[-1] tkk_expr = re.search(".*?(TKK=.*?;)W.*?", line).group(1) a = re.search("a\\\\x3d(-?\d+);", tkk_expr).group(1) b = re.search("b\\\\x3d(-?\d+);", tkk_expr).group(1) result = str(hours) + "." + str(int(a) + int(b)) self.token_key = result return result """ Functions used by the token calculation algorithm """ def _rshift(self, val, n): return val >> n if val >= 0 else (val + 0x100000000) >> n def _work_token(self, a, seed): for i in range(0, len(seed) - 2, 3): char = seed[i + 2] d = ord(char[0]) - 87 if char >= "a" else int(char) d = self._rshift(a, d) if seed[i + 1] == "+" else a << d a = a + d & 4294967295 if seed[i] == "+" else a ^ d return a weboob-1.2/modules/googletranslate/module.py000066400000000000000000000070431303450110500213260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . "backend for http://translate.google.com" from weboob.capabilities.translate import CapTranslate, Translation, TranslationFail, LanguageNotSupported from weboob.capabilities.base import empty from weboob.tools.backend import Module from .browser import GoogleTranslateBrowser __all__ = ['GoogleTranslateModule'] class GoogleTranslateModule(Module, CapTranslate): MAINTAINER = u'Lucien Loiseau' EMAIL = 'loiseau.lucien@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' NAME = 'googletranslate' DESCRIPTION = u'Google translation web service' BROWSER = GoogleTranslateBrowser GOOGLELANGUAGE = { 'Arabic': 'ar', 'Afrikaans': 'af', 'Albanian': 'sq', 'Armenian': 'hy', 'Azerbaijani': 'az', 'Basque': 'eu', 'Belarusian': 'be', 'Bengali': 'bn', 'Bulgarian': 'bg', 'Catalan': 'ca', 'Chinese': 'zh-CN', 'Croatian': 'hr', 'Czech': 'cs', 'Danish': 'da', 'Dutch': 'nl', 'English': 'en', 'Esperanto': 'eo', 'Estonian': 'et', 'Filipino': 'tl', 'Finnish': 'fi', 'French': 'fr', 'Galician': 'gl', 'Georgian': 'ka', 'German': 'de', 'Greek': 'el', 'Gujarati': 'gu', 'Haitian': 'ht', 'Hebrew': 'iw', 'Hindi': 'hi', 'Hungaric': 'hu', 'Icelandic': 'is', 'Indonesian': 'id', 'Irish': 'ga', 'Italian': 'it', 'Japanese': 'ja', 'Kannada': 'kn', 'Korean': 'ko', 'Latin': 'la', 'Latvian': 'lv', 'Lithuanian': 'lt', 'Macedonian': 'mk', 'Malay': 'ms', 'Maltese': 'mt', 'Norwegian': 'no', 'Persian': 'fa', 'Polish': 'pl', 'Portuguese': 'pt', 'Romanian': 'ro', 'Russian': 'ru', 'Serbian': 'sr', 'Slovak': 'sk', 'Slovenian': 'sl', 'Spanish': 'es', 'Swahili': 'sw', 'Swedish': 'sv', 'Tamil': 'ta', 'Telugu': 'te', 'Thai': 'th', 'Turkish': 'tr', 'Ukrainian': 'uk', 'Urdu': 'ur', 'Vietnamese': 'vi', 'Welsh': 'cy', 'Yiddish': 'yi', } def translate(self, lan_from, lan_to, text): if lan_from not in self.GOOGLELANGUAGE.keys(): raise LanguageNotSupported() if lan_to not in self.GOOGLELANGUAGE.keys(): raise LanguageNotSupported() translation = Translation(0) translation.lang_src = unicode(self.GOOGLELANGUAGE[lan_from]) translation.lang_dst = unicode(self.GOOGLELANGUAGE[lan_to]) translation.text = self.browser.translate(self.GOOGLELANGUAGE[lan_from], self.GOOGLELANGUAGE[lan_to], text) if empty(translation.text): raise TranslationFail() return translation weboob-1.2/modules/googletranslate/pages.py000066400000000000000000000017161303450110500211410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import RawPage import re class TranslatePage(RawPage): def get_translation(self): m = re.search('\[\[\[\"(.*)\",\".*\",,,\d\]\],,".*"\]', self.doc) if m: return unicode(m.group(1)) weboob-1.2/modules/googletranslate/test.py000066400000000000000000000017771303450110500210300ustar00rootroot00000000000000# -*- CODing: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GoogleTranslateTest(BackendTest): MODULE = 'googletranslate' def test_translate(self): tr = self.backend.translate('French', 'English', 'je mange du chocolat') self.assertTrue(tr.text == u"I'm eating chocolate") weboob-1.2/modules/groupamaes/000077500000000000000000000000001303450110500164345ustar00rootroot00000000000000weboob-1.2/modules/groupamaes/__init__.py000066400000000000000000000014421303450110500205460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GroupamaesModule __all__ = ['GroupamaesModule'] weboob-1.2/modules/groupamaes/browser.py000066400000000000000000000046151303450110500204770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.date import LinearDateGuesser from .pages import LoginPage, LoginErrorPage, GroupamaesPage __all__ = ['GroupamaesBrowser'] class GroupamaesBrowser(LoginBrowser): BASEURL = 'https://www.gestion-epargne-salariale.fr' login = URL('/groupama-es/fr/index.html', LoginPage) login_error = URL('/groupama-es/fr/identification/default.cgi', LoginErrorPage) groupamaes_page = URL('/groupama-es/fr/espace/devbavoirs.aspx\?mode=net&menu=cpte(?P.*)', GroupamaesPage) def do_login(self): self.login.stay_or_go() self.page.login(self.username, self.password) if not self.page.logged or self.login_error.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): return self.groupamaes_page.stay_or_go(page='&page=situglob').iter_accounts() @need_login def get_history(self): transactions = list(self.groupamaes_page.go(page='&_pid=MenuOperations&_fid=GoOperationsTraitees').get_history(date_guesser=LinearDateGuesser())) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def get_coming(self): transactions = list(self.groupamaes_page.go(page='&_pid=OperationsTraitees&_fid=GoWaitingOperations').get_history(date_guesser=LinearDateGuesser())) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions @need_login def iter_investment(self): return self.groupamaes_page.go(page='&_fid=GoPositionsParFond&_pid=SituationGlobale').iter_investment() weboob-1.2/modules/groupamaes/favicon.png000066400000000000000000000221721303450110500205730ustar00rootroot00000000000000PNG  IHDR@@iqsRGBgAMA a pHYs  $IDATx^{w|UU%@hHCBBzrSnnMO ""}TD(MQf0eF}}<> {ؿrkdtpL\X9z{ :䚌{6\Y|,u~d;'÷7WSB5j.DV9wr-޷.O΁{2:fsj}ܹ]n㶟\T&vpυj>GG'\n^Ϗ bd| kpMBT5 ]*'[U&z2˞$tLD/G2z;г*=+8Sɖ`{yOY66fl,}~j}k2ԍP-bD:~ S4Z:HB7{"Q1։-q(v%<asŠ6 6DXPPDW9r۞0Bϒ5\dyQ}xM8Eɍ8!_N# >+Sѹx"p!*1LmI©X55MM3i 4#9:; ff[Xl:J!Ǥ3UQBe31 9%늙y\'au iHE.^&PJ?x34z HB!bx8]X}k>Y(*kE߲DSѱ4]'+x7N|-mR)xb7q4 \uno>50_AU6y*\1NddDŽ(@QItlCG.DvGOi %vi,Asw=UY(لɄʌ26qHC__$P8Wo|%_)z),? 4D_u&(S r~~={ڻm ,quS55]l)T0 Nfץ0ơ;y}L$DLKEpGY2??]_~ $ފOÊ۰c bݹØ{h r!qE bzx!GƑQ] ":hELrah/%V7\D3 WF1܌(:ԍDƑjBHu,׿GO_fZuxeAt,UD_"c%"GόH!!t-mY.5_Z6>+=@{YCQ.]EE~B_9&N(8c1a"BJG}Vݗ$6&7g72C7򢸈dto"J S!#KPVG’ARй&cHF4f ۆ+ af@$0Z B{ >39!w Ln+ТéڌJEfN@ފh-CPzr:yD|y\1 E6]Z hMA?*E](k9K )w"edd,M;yd>i"᮱4;e!T42_M2S:jGo#K ~rS ɇDfXhnSYuf8/]i__pZ v]篬Ĝ-vܝE^I Dld*fF qy&]e4;&-Cӕpj vG]xlM}?(@ O8H#iۃ dk?uD$3AP#Wm9ӭI)gįuco\Xri]}NjR\Y a(7Ӷ vQOz]<O31Aj`˂[|^^C 7v[XP3 tqY&,"PVRXVѤ+t@YB=~%a_SDx͊T]Z|18Ղq}~v_.%Y<M @Dۯ.( )Ù~сƠt : WĒ=)Qv;kRxF+'$KU82msRDgf`[I}H0oW0E"ׅ`8ő-Cht0Q5e=qj޸] !vVOjJ%w OcP+߮AAÂoi$U1fܯ1H+6x0S:ڧ\Mg3&up:TFa<8yi8޸ Z1uAލgfnFƝ!5TYC'yRz@s u_l+۳CJ] N@ r+aXfx&,ĦOnǞ ~'ׂGߚa!LJ@rTmIGĜo~(2ʙWK8f2Ox[XlJhk1+-fZl_Y1't :|bEʄ\dT;A&.8Oe"{i2i$!i*v.] X#XGw1ֹe*#` *d;d  BėJAI"]MBf} L[C*3n=2uh&p(ObH3QnEyx]eVjQTkjFb&B9e Rc@Bef%H ^mK,1NY@}KCG2BtBXtlJ8 zܥT*c.SY駲@uz WARҕ) Jb4YrgCRtU:YDDK+ùT /. J/eI %qJ몰)(^o6^w*|;>BrF/BYShiZknRZ1xEL2b+"jTIO֝=KOhD {Ѹg3\A+:)[1ks3Qt=Ǔo02(]?mJ2 x6oҠJ&`hfW2cb]N6.I?t֟;˨]vLY֊onDlpO+*\JIO "Pf: x)mmOOIJf݊:cѪcOWF{OƃM^+Ϝv1Ps6KrW,ªo`spӛVD\pT5=gv (Ix LZvkk-xvިym<*a4e熳,N@ByX V6/"I|ꄳe/{|#<&\w1n =8ogu8qjE78hZڅ5ًdGK4z0%M\T0'R,l\P^ͫʮ)1۶]2@fәipdC]"GȄ?H?? 4bu1@C9b{y}`}k kJ10 ` 4 MyE>>.7]-WqE" غWJ 䧖Zz*L݂lŏ"xr$79&/OIfm""Z=-z2qo#AťJ-8&Gq5[gHȼKg\F.`BeԨ^Ңf y#.()K*!Н[ ML5] Z#a ^@9*6HLГ>vViT<MSĹx`J&4W@Ȉm<kRT@2EUKYHCX5`+@&1<!#AvWr"b:=J% 6/3ji\£{,nv)xڗrq-LL(NvpVS.կ?De6MbE cZvKi eiQzN(Svӊ,4Ƶe*oT5AVh؅*4x/cO5{z:EI-p5g^zc-ȩ£-_H)QE#F]Xn^;>b6~9CD0G95~VD.ܺf%nB%"K+lEpb+=z)8Ng"7SPH3Cwl +QbH=Ӊ_]D YLz8ga5X~DNofEhz[0{&, &Zpc.~ ܱ} 7.]DnƓ[0d8+ܺ LZ;]|^[.2{l*Ϧk{G3 W]suuydF#0rmD K߈,@9$UgP0q <~bmO‚mV-_|88Q1>re18?q2O8v wcSd!֞9N:ϴ/iiуokv j)IR=:J+>[^؍2>@Y;ooDɒ9ԛm3F[NHhܺܶw+V{em@Rֿq >q׎a<3Ϭ`=We H_zcJq_ފU^eL N/50~HsjM}҄ʈa aU/ب~^iYq ͬ38's3kνI90ĺ^Ttyd^~8}}oa'iOg5^?gw5e;K't ܏)aו{EaT=uuO NTgObgW穧Ã~|ߏm+Tz -uNOS/+OJWTt/>哨%yF5)lU^|$_ Y|Dr`(KL'\sחunIENDB`weboob-1.2/modules/groupamaes/module.py000066400000000000000000000040051303450110500202720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.base import find_object from .browser import GroupamaesBrowser __all__ = ['GroupamaesModule'] class GroupamaesModule(Module, CapBank): NAME = 'groupamaes' DESCRIPTION = u"Groupama Épargne Salariale" MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = GroupamaesBrowser CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', regexp='\d{8}', masked=False), ValueBackendPassword('password', label='Mot de passe')) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def iter_coming(self, account): return self.browser.get_coming() def iter_history(self, account): return self.browser.get_history() def iter_investment(self, account): return self.browser.iter_investment() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) weboob-1.2/modules/groupamaes/pages.py000066400000000000000000000127511303450110500201130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal from datetime import date import re from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import TableElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, TableCell, Date, Env, Regexp, Field from weboob.capabilities.bank import Account, Transaction, Investment class LoginPage(HTMLPage): def login(self, login, passwd): form = self.get_form(nr=0) form['_cm_user'] = login form['_cm_pwd'] = passwd form.submit() class LoginErrorPage(HTMLPage): pass class GroupamaesPage(LoggedPage, HTMLPage): NEGATIVE_AMOUNT_LABELS = [u'Retrait', u'Transfert sortant'] @method class iter_accounts(TableElement): item_xpath = u'//table[@summary="Liste des échéances"]/tbody/tr' head_xpath = u'//table[@summary="Liste des échéances"]/thead/tr/th/text()' col_name = u'Plan' col_value = u'Evaluation en' class item(ItemElement): klass = Account def condition(self): return u'Vous n\'avez pas d\'avoirs.' not in CleanText(TableCell('name'))(self) obj_id = CleanText(TableCell('name')) obj_label = CleanText(TableCell('name')) obj_balance = CleanDecimal(TableCell('value'), replace_dots=True, default=Decimal(0)) obj_currency = CleanText(u'//table[@summary="Liste des échéances"]/thead/tr/th/small/text()') obj_type = Account.TYPE_PEE def iter_investment(self): item = self.doc.xpath(u'//table[@summary="Liste des échéances"]/tfoot/tr/td[@class="tot _c1 d _c1"]')[0] total = CleanDecimal(Regexp(CleanText('.'), '(.*) .*'), default=1, replace_dots=True)(item) item_xpath = u'((//table[@summary="Liste des échéances"])[1]/tbody/tr)[position() < last() and not(contains(./td[1]/@class, "tittot"))]' obj = None for tr in self.doc.xpath(item_xpath): tds = tr.xpath('./td') if len(tds) > 3: if obj is not None: obj.portfolio_share = (obj.valuation / total).quantize(Decimal('.0001')) yield obj obj = Investment() obj.label = CleanText('.')(tds[0]) obj.vdate = date.today() # * En réalité derniere date de valorisation connue obj.unitvalue = CleanDecimal('.', replace_dots=True)(tds[2]) obj.valuation = CleanDecimal('.', replace_dots=True)(tds[5]) obj.quantity = CleanDecimal('.', replace_dots=True)(tds[4]) elif obj is not None: obj.quantity += CleanDecimal('.', replace_dots=True)(tds[1]) obj.valuation += CleanDecimal('.', replace_dots=True)(tds[2]) if obj is not None: obj.portfolio_share = (obj.valuation / total).quantize(Decimal('.0001')) yield obj @method class get_history(TableElement): head_xpath = u'//table[@summary="Liste des opérations"]/thead/tr/th/text()' item_xpath = u'//table[@summary="Liste des opérations"]/tbody/tr' col_date = u'Date' col_operation = u'Opération' col_montant = u'Montant net en EUR' class item(ItemElement): klass = Transaction def condition(self): return u'Aucune opération' not in CleanText(TableCell('date'))(self) obj_date = Date(CleanText(TableCell('date')), Env('date_guesser')) obj_type = Transaction.TYPE_UNKNOWN obj_label = CleanText(TableCell('operation')) def obj_amount(self): amount = CleanDecimal(TableCell('montant'), replace_dots=True)(self) for pattern in GroupamaesPage.NEGATIVE_AMOUNT_LABELS: if Field('label')(self).startswith(pattern): amount = -amount return amount @method class get_list(TableElement): head_xpath = u'//table[@summary="Liste des opérations en attente"]/thead/tr/th/text()' item_xpath = u'//table[@summary="Liste des opérations en attente"]/tbody/tr' col_date = u'Date' col_operation = u'Opération' col_etat = u'Etat' col_montant = [u'Montant net en', re.compile(u'Montant en')] col_action = u'Action' class item(ItemElement): klass = Transaction def condition(self): return u'Aucune opération en attente' not in CleanText(TableCell('date'))(self) obj_date = Date(CleanText(TableCell('date')), Env('date_guesser')) obj_type = Transaction.TYPE_UNKNOWN obj_label = CleanText(TableCell('operation')) obj_amount = CleanDecimal(TableCell('montant'), replace_dots=True) weboob-1.2/modules/groupamaes/test.py000066400000000000000000000021541303450110500177670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GroupamaesTest(BackendTest): MODULE = 'groupamaes' def test_groupamaes(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] self.assertTrue(self.backend.get_account(l[0].id) is not None) list(self.backend.iter_history(a)) list(self.backend.iter_investment(a)) weboob-1.2/modules/guerrillamail/000077500000000000000000000000001303450110500171225ustar00rootroot00000000000000weboob-1.2/modules/guerrillamail/__init__.py000066400000000000000000000014501303450110500212330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import GuerrillamailModule __all__ = ['GuerrillamailModule'] weboob-1.2/modules/guerrillamail/browser.py000066400000000000000000000052071303450110500211630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Browser from weboob.tools.date import datetime from weboob.deprecated.browser.parsers.jsonparser import json from urllib import urlencode #from .pages import Page1, Page2 __all__ = ['GuerrillamailBrowser'] class GuerrillamailBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'www.guerrillamail.com' ENCODING = 'utf-8' def __init__(self, *args, **kw): kw['parser'] = 'raw' Browser.__init__(self, *args, **kw) def _get_unicode(self, url, *a): return self.get_document(self.openurl(url, *a)).decode(self.ENCODING, 'replace') def _get_json(self, url, *a): j = json.loads(self._get_unicode(url, *a)) return j def get_mails(self, boxid): params = {'email_user': boxid, 'lang': 'en', 'domain': 'guerrillamail.com'} d = self._get_json('https://www.guerrillamail.com/ajax.php?f=set_email_user', urlencode(params)) d = self._get_json('https://www.guerrillamail.com/ajax.php?f=get_email_list&offset=0&domain=guerrillamail.com') for m in d['list']: info = {} info['id'] = m['mail_id'] info['from'] = m['mail_from'] # info['to'] = m['mail_recipient'] info['to'] = '%s@guerrillamail.com' % boxid info['subject'] = m['mail_subject'] info['datetime'] = datetime.fromtimestamp(int(m['mail_timestamp'])) info['read'] = bool(int(m['mail_read'])) yield info def get_mail_content(self, mailid): d = self._get_json('https://www.guerrillamail.com/ajax.php?f=fetch_email&email_id=mr_%s&domain=guerrillamail.com' % mailid) return d['mail_body'] def send_mail(self, from_, to, subject, body): params = {'from': from_, 'to': to, 'subject': subject, 'body': body, 'attach': '', 'domain': 'guerrillamail.com'} self._get_json('https://www.guerrillamail.com/ajax.php?f=send_email', urlencode(params)) weboob-1.2/modules/guerrillamail/favicon.png000066400000000000000000000020711303450110500212550ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME %($tEXtCommentCreated with GIMPWIDATxkA_*"BAezj^RY֫H^kA` R"%-Z"Р(lfgΛ/%NH&JlT6oŘ_/yPߒ܀H!Q\MٞಡӦEI(dbt-[M ٭ #CP{A[*î GTyىm-F~s=vvOj0@Ƽ j4.ܘvC0e4 .}@ #14=:!m ƟAXKu0*0:Ab$Z];|Dd}LiJ37)PEd^ި`ji. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.messages import CapMessages, CapMessagesPost, Thread, Message from weboob.tools.value import Value from .browser import GuerrillamailBrowser __all__ = ['GuerrillamailModule'] class GuerrillamailModule(Module, CapMessages, CapMessagesPost): NAME = 'guerrillamail' DESCRIPTION = u'GuerrillaMail temp mailbox' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = GuerrillamailBrowser CONFIG = BackendConfig(Value('inbox', label='Inbox', default='')) def iter_threads(self): inbox = self.config['inbox'].get() if not inbox: raise NotImplementedError() else: return [self.get_thread(inbox)] def get_thread(self, _id): t = Thread(_id) t.title = 'Mail for %s' % _id t.flags = t.IS_DISCUSSION first = True for d in self.browser.get_mails(_id): m = self.make_message(d, t) if not m.content: m.content = self.browser.get_mail_content(m.id) if first: first = False t.root = m else: m.parent = t.root m.parent.children.append(m) return t def post_message(self, m): raise NotImplementedError() for receiver in m.receivers: self.browser.send_mail(m.sender, receiver, m.title, m.content) def make_message(self, d, thread): m = Message(thread, d['id']) m.children = [] m.sender = d['from'] m.flags = 0 if not d.get('read', True): m.flags = m.IS_UNREAD m.title = d['subject'] m.date = d['datetime'] m.receivers = [d['to']] return m weboob-1.2/modules/guerrillamail/test.py000066400000000000000000000023101303450110500204470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from uuid import uuid4 class GuerrillamailTest(BackendTest): MODULE = 'guerrillamail' def test_guerrillamail(self): box = uuid4() thread = self.backend.get_thread(box) self.assertTrue(thread) message = thread.root self.assertTrue(message) self.assertTrue(message.sender) self.assertTrue(message.title) self.assertTrue(message.date) self.assertTrue(message.receivers) weboob-1.2/modules/happn/000077500000000000000000000000001303450110500153775ustar00rootroot00000000000000weboob-1.2/modules/happn/__init__.py000066400000000000000000000014361303450110500175140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import HappnModule __all__ = ['HappnModule'] weboob-1.2/modules/happn/browser.py000066400000000000000000000130171303450110500174360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.browser.browsers import DomainBrowser from weboob.browser.profiles import IPhone from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import CleanText from weboob.exceptions import BrowserIncorrectPassword, ParseError from weboob.tools.json import json __all__ = ['HappnBrowser', 'FacebookBrowser'] class FacebookBrowser(DomainBrowser): BASEURL = 'https://graph.facebook.com' CLIENT_ID = "247294518656661" PROFILE = IPhone('Happn/3.0.2') access_token = None info = None def login(self, username, password): self.location('https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=fbconnect://success&scope=email,user_birthday,user_friends,public_profile,user_photos,user_likes&response_type=token' % self.CLIENT_ID) page = HTMLPage(self, self.response) form = page.get_form('//form[@id="login_form"]') form['email'] = username form['pass'] = password form.submit(allow_redirects=False) if 'Location' not in self.response.headers: raise BrowserIncorrectPassword() self.location(self.response.headers['Location']) page = HTMLPage(self, self.response) if len(page.doc.xpath('//td/div[has-class("s")]')) > 0: raise BrowserIncorrectPassword(CleanText('//td/div[has-class("s")]')(page.doc)) form = page.get_form(nr=0, submit='//input[@name="__CONFIRM__"]') form.submit() m = re.search('access_token=([^&]+)&', self.response.text) if m: self.access_token = m.group(1) else: raise ParseError('Unable to find access_token') self.info = self.request('/me') def request(self, url, *args, **kwargs): url += '?access_token=' + self.access_token r = self.location(self.absurl(url, base=True), *args, **kwargs) return json.loads(r.content) class HappnBrowser(DomainBrowser): BASEURL = 'https://api.happn.fr/' PROFILE = IPhone('Happn/18.3.1') ALLOW_REFERRER = False recs = [] def __init__(self, facebook, *args, **kwargs): super(HappnBrowser, self).__init__(*args, **kwargs) self.facebook = facebook self.session.headers['User-Agent'] = 'Happn/18.3.1 AndroidSDK/11' r = self.request('/connect/oauth/token', data={ 'client_id': 'FUE-idSEP-f7AqCyuMcPr2K-1iCIU_YlvK-M-im3c', 'client_secret': 'brGoHSwZsPjJ-lBk0HqEXVtb3UFu-y5l_JcOjD-Ekv', 'grant_type': 'assertion', 'assertion_type': 'facebook_access_token', 'assertion': facebook.access_token, 'scope': 'mobile_app', }) self.session.headers['Authorization'] = 'OAuth="%s"' % r['access_token'] self.my_id = r['user_id'] self.refresh_token = r['refresh_token'] me = self.request('/api/users/me') self.my_name = me['data']['name'] def request(self, *args, **kwargs): r = self.location(*args, **kwargs) return r.json() def get_contact(self, contact_id): return self.request('/api/users/%s?fields=birth_date,first_name,last_name,display_name,login,credits,referal,matching_preferences,notification_settings,unread_conversations,about,is_accepted,age,job,workplace,school,modification_date,profiles.mode(0).width(1000).height(1000).fields(url,width,height,mode),last_meet_position,my_relation,is_charmed,distance,gender' % contact_id)['data'] def get_threads(self): return self.request('/api/users/me/conversations')['data'] def get_thread(self, id): r = self.request('/api/users/%s/conversations/%s?fields=id,messages.limit(100).fields(id,message,creation_date,sender.fields(id)),participants.fields(user.fields(birth_date,first_name,last_name,display_name,credits,referal,matching_preferences,notification_settings,unread_conversations,about,is_accepted,age,job,workplace,school,modification_date,profiles.mode(0).width(1000).height(1000).fields(url,width,height,mode),last_meet_position,my_relation,is_charmed,distance,gender))' % (self.my_id, id))['data'] return r def post_message(self, thread_id, content): self.request('/api/conversations/%s/messages/' % thread_id, data={'message': content}) def find_users(self): return self.request('/api/users/me/notifications?types=468&fields=id,is_pushed,lon,actions,creation_date,is_notified,lat,modification_date,notification_type,nb_times,notifier.fields(id,job,is_accepted,workplace,my_relation,distance,gender,my_conversation,is_charmed,nb_photos,last_name,first_name,age),notified.fields(is_accepted,is_charmed)')['data'] def accept(self, id): self.request('/api/users/me/accepted/%s' % id, method='POST') def set_position(self, lat, lng): self.request('/api/users/me/position', data={'latitude': lat, 'longitude': lng, 'altitude': 0.0}) weboob-1.2/modules/happn/favicon.png000066400000000000000000000070211303450110500175320ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME(L؋tEXtCommentCreated with GIMPW IDAThZ[l] lMDDM QJIi!1H"!}hEMQVm(AHH `9}93k/H`=397l $'1__C#M-f D==Qv)oUxXw?6s )oGք^X1D0,\)Jd'`l%`DDDlHafy1bŢ`${gԁrr3;O&9>Q+C/@$13膞dNL Xb2ڵ11VLFEXl]wͩW[6T=WXJ1gF:hkfƆa3o=&({jyF}-E~pPByxo\ּ#1qm˚".FaL*bQye JU)~f]g,sщHftbj zBFRhwڠ0+;fPjAwWo`$s쥣/tWYذ.'J^, K*e|+/JvZjO<\W: ȖJdAScGƴSxG]ֵ]On.y(=x쐕??N%5]x"me c[; &$O,ў9y񁑌K&l~>uipt\3SG[:#KYHMrͩ" ={i~zGO}s(=NOZT[(XH"/T%ɰa@16ڣ/G M83]ta& /JԍKsN’vaV={fn#,fBFV"wg,ybB ֕=sk?/ETѵ֔-}мHfIcK'}`dԍk hu.AD8# 9bNg!zg7@crv*+beɒnj`{X܀^'%Y(/ nƧ;v D-%_y$5|~+rӹT$V5\eHRB2q왯-ܠloP?lozh^X?k(hJV?+QVʳ_W^k^^P3?jW-N.jO[eR&$Sg)M?mR;x2I<4KºPNfw8p|\ehcӔ5mC$,΀ZB[a/n;X;4lfdHiNf`>L$6*gm)&,=`\8܃\1[!\}>Ě<4?/rݡ`g|%;QD7a&r3<Ĝ\`&&.B_X1D5 uYh->>@+7Pd}aLe".T *T˿R ȗT Qhʔ|}aˇd /B-1IENDB`weboob-1.2/modules/happn/module.py000066400000000000000000000301441303450110500172400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from dateutil.parser import parse as parse_date from dateutil.relativedelta import relativedelta from dateutil.tz import tzlocal from random import randint from weboob.capabilities.base import NotAvailable from weboob.capabilities.messages import CapMessages, CapMessagesPost, Thread, Message from weboob.capabilities.dating import CapDating, Optimization from weboob.capabilities.contact import CapContact, Contact, ProfileNode from weboob.exceptions import BrowserHTTPError from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.ordereddict import OrderedDict from weboob.tools.log import getLogger from .browser import HappnBrowser, FacebookBrowser __all__ = ['HappnModule'] class ProfilesWalker(Optimization): def __init__(self, sched, storage, browser): super(ProfilesWalker, self).__init__() self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('walker', browser.logger) self._last_position_update = None self._view_cron = None def start(self): self._view_cron = self._sched.schedule(1, self.view_profile) return True def stop(self): self._sched.cancel(self._view_cron) self._view_cron = None return True def set_config(self, params): pass def is_running(self): return self._view_cron is not None INTERVALS = [(48896403, 2303976), (48820992, 2414698)] def view_profile(self): try: n = 0 for user in self._browser.find_users(): if user['notifier']['my_relation'] > 0: continue self._browser.accept(user['notifier']['id']) self._logger.info('Liked %s %s (%s at %s): https://www.facebook.com/profile.php?id=%s&fref=ufi&pnref=story', user['notifier']['first_name'], user['notifier']['last_name'], user['notifier']['job'], user['notifier']['workplace'], user['notifier']['fb_id']) n += 1 if n > 10: break if n == 0 and (self._last_position_update is None or self._last_position_update + datetime.timedelta(minutes=20) < datetime.datetime.now()): self._logger.info('No more new profiles, updating position...') lat = randint(self.INTERVALS[1][0], self.INTERVALS[0][0])/1000000.0 lng = randint(self.INTERVALS[0][1], self.INTERVALS[1][1])/1000000.0 try: self._browser.set_position(lat, lng) except BrowserHTTPError: self._logger.warning('Unable to update position for now, it will be retried later.') self._logger.warning('NB: don\'t be afraid, happn only allows to update position every 20 minutes.') else: self._logger.info('You are now here: https://www.google.com/maps/place//@%s,%s,17z', lat, lng) self._last_position_update = datetime.datetime.now() for thread in self._browser.get_threads(): other_name = '' for user in thread['participants']: if user['user']['id'] != self._browser.my_id: other_name = user['user']['display_name'] if len(thread['messages']) == 0 and parse_date(thread['creation_date']) < (datetime.datetime.now(tzlocal()) - relativedelta(hours=1)): self._browser.post_message(thread['id'], u'Coucou %s :)' % other_name) self._logger.info(u'Welcome message sent to %s' % other_name) finally: if self._view_cron is not None: self._view_cron = self._sched.schedule(60, self.view_profile) class HappnContact(Contact): def set_profile(self, *args): section = self.profile for arg in args[:-2]: try: s = section[arg] except KeyError: s = section[arg] = ProfileNode(arg, arg.capitalize().replace('_', ' '), OrderedDict(), flags=ProfileNode.SECTION) section = s.value key = args[-2] value = args[-1] section[key] = ProfileNode(key, key.capitalize().replace('_', ' '), value) def __init__(self, info): status = Contact.STATUS_OFFLINE last_seen = parse_date(info['modification_date']) if last_seen >= datetime.datetime.now(tzlocal()) - datetime.timedelta(minutes=30): status = Contact.STATUS_ONLINE super(HappnContact, self).__init__(info['id'], info['display_name'], status) self.summary = info['about'] for photo in info['profiles']: self.set_photo(photo['id'], url=photo['url']) self.status_msg = u'Last seen at %s' % last_seen.strftime('%Y-%m-%d %H:%M:%S') self.url = NotAvailable self.profile = OrderedDict() self.set_profile('info', 'id', info['id']) self.set_profile('info', 'full_name', ' '.join([info['first_name'] or '', info['last_name'] or '']).strip()) if info['fb_id'] is not None: self.set_profile('info', 'facebook', 'https://www.facebook.com/profile.php?id=%s&fref=ufi&pnref=story' % info['fb_id']) if info['twitter_id'] is not None: self.set_profile('info', 'twitter', info['twitter_id']) self.set_profile('stats', 'accepted', info['is_accepted']) self.set_profile('stats', 'charmed', info['is_charmed']) self.set_profile('stats', 'unread_conversations', info['unread_conversations']) self.set_profile('stats', 'credits', info['credits']) if info['last_meet_position'] is not None: self.set_profile('geoloc', 'last_meet', 'https://www.google.com/maps/place//@%s,%s,17z' % (info['last_meet_position']['lat'], info['last_meet_position']['lon'])) if info['distance'] is not None: self.set_profile('geoloc', 'distance', '%.2f km' % (info['distance']/1000.0)) self.set_profile('details', 'gender', info['gender']) self.set_profile('details', 'age', '%s yo' % info['age']) self.set_profile('details', 'birthday', info['birth_date']) self.set_profile('details', 'job', info['job']) self.set_profile('details', 'company', info['workplace']) self.set_profile('details', 'school', info['school']) if info['matching_preferences'] is not None: self.set_profile('settings', 'age_min', '%s yo' % info['matching_preferences']['age_min']) self.set_profile('settings', 'age_max', '%s yo' % info['matching_preferences']['age_max']) self.set_profile('settings', 'distance', '%s m' % info['matching_preferences']['distance']) self.set_profile('settings', 'female', info['matching_preferences']['female']) self.set_profile('settings', 'male', info['matching_preferences']['male']) class HappnModule(Module, CapMessages, CapMessagesPost, CapDating, CapContact): NAME = 'happn' DESCRIPTION = u'Happn dating mobile application' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Facebook email'), ValueBackendPassword('password', label='Facebook password')) BROWSER = HappnBrowser STORAGE = {'contacts': {}, } def create_default_browser(self): facebook = self.create_browser(klass=FacebookBrowser) facebook.login(self.config['username'].get(), self.config['password'].get()) return self.create_browser(facebook) # ---- CapDating methods ----------------------- def init_optimizations(self): self.add_optimization('PROFILE_WALKER', ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)) # ---- CapMessages methods --------------------- def fill_thread(self, thread, fields): return self.get_thread(thread) def iter_threads(self): for thread in self.browser.get_threads(): t = Thread(thread['id']) t.flags = Thread.IS_DISCUSSION for user in thread['participants']: if user['user']['id'] != self.browser.my_id: t.title = u'Discussion with %s' % user['user']['display_name'] t.date = parse_date(thread['modification_date']) yield t def get_thread(self, thread): if not isinstance(thread, Thread): thread = Thread(thread) thread.flags = Thread.IS_DISCUSSION info = self.browser.get_thread(thread.id) for user in info['participants']: if user['user']['id'] == self.browser.my_id: me = HappnContact(user['user']) else: other = HappnContact(user['user']) thread.title = u'Discussion with %s' % other.name contact = self.storage.get('contacts', thread.id, default={'lastmsg_date': '1970-01-01T01:01:01+00:00'}) child = None for msg in info['messages']: flags = 0 if parse_date(contact['lastmsg_date']) < parse_date(msg['creation_date']): flags = Message.IS_UNREAD if msg['sender']['id'] == me.id: sender = me receiver = other else: sender = other receiver = me msg = Message(thread=thread, id=msg['id'], title=thread.title, sender=sender.name, receivers=[receiver.name], date=parse_date(msg['creation_date']), content=msg['message'], children=[], parent=None, signature=sender.get_text(), flags=flags) if child: msg.children.append(child) child.parent = msg child = msg thread.root = child return thread def iter_unread_messages(self): for thread in self.iter_threads(): thread = self.get_thread(thread) for message in thread.iter_all_messages(): if message.flags & message.IS_UNREAD: yield message def set_message_read(self, message): contact = self.storage.get('contacts', message.thread.id, default={'lastmsg_date': '1970-01-01T01:01:01+00:00'}) if parse_date(contact['lastmsg_date']) < message.date: contact['lastmsg_date'] = str(message.date) self.storage.set('contacts', message.thread.id, contact) self.storage.save() # ---- CapMessagesPost methods --------------------- def post_message(self, message): self.browser.post_message(message.thread.id, message.content) # ---- CapContact methods --------------------- def get_contact(self, contact_id): if isinstance(contact_id, Contact): contact_id = contact_id.id info = self.browser.get_contact(contact_id) return HappnContact(info) OBJECTS = {Thread: fill_thread, } weboob-1.2/modules/happn/test.py000066400000000000000000000016431303450110500167340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class HappnTest(BackendTest): MODULE = 'happn' def test_happn(self): for m in self.backend.iter_unread_messages(): pass weboob-1.2/modules/hds/000077500000000000000000000000001303450110500150475ustar00rootroot00000000000000weboob-1.2/modules/hds/__init__.py000066400000000000000000000000671303450110500171630ustar00rootroot00000000000000from .module import HDSModule __all__ = ['HDSModule'] weboob-1.2/modules/hds/browser.py000066400000000000000000000033711303450110500171100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import ValidationPage, HomePage, HistoryPage, StoryPage, AuthorPage # Browser class HDSBrowser(PagesBrowser): BASEURL = 'http://histoires-de-sexe.net' validation_page = URL('^/$', ValidationPage) home = URL(r'/menu.php', HomePage) history = URL(r'/sexe/histoires-par-date.php\?p=(?P\d+)', HistoryPage) story = URL(r'/sexe.php\?histoire=(?P.+)', StoryPage) author = URL(r'/fiche.php\?auteur=(?P.+)', AuthorPage) def iter_stories(self): n = 1 self.history.go(pagenum=n) while self.page.get_numerous() == n: for story in self.page.iter_stories(): yield story n += 1 self.history.go(pagenum=n) def get_story(self, id): self.story.go(id=id) assert self.story.is_here() return self.page.get_story() def get_author(self, name): self.author.go(name=name) assert self.author.is_here() return self.page.get_author() weboob-1.2/modules/hds/favicon.png000066400000000000000000000024671303450110500172130ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDfp pHYs  tIME4OtEXtCommentCreated with GIMPWIDATx[lUEPЦE[+RUT"xw f*16UTh b' jiC,%*}ьJB4!>h/!*XBP"UPp|pd2g=͚:3=fC$D"H$D"H$Gbs5ܨ?'5o@B`-X\4ORb Va/'@ 0h>%N˝vkΓ6ɪ8 |%3T9i3-0*ퟑs<8zN[w Qڀt;a%[LWt>kpByYxvMڳGb촘,fL+qqŶ5~ '*4.`>AL7sq3y֝ݔ1G S0ٗnuHg(tB/^uuI^GV3Y^̓еy4$Y+Gto0Ν Wɳ(nܨ~ԄXSR.L%Y[V榝 g`ןkӞRqV{Y8gzy pgW|x tvlc1 8h1BjEE4rM_}qϞ$w~bNt!/%,NThI_~ "]]_`1s3}~"b$.|#n -ww^}cKo *ʄ6P=%*BeЃ %l{3ouFT/;Fv .9{ϒc|F#1 q >qSm)bz6X( L> Iux3D"H$D"H$Dg *ǻIENDB`weboob-1.2/modules/hds/module.py000066400000000000000000000061351303450110500167130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.messages import CapMessages, Message, Thread from .browser import HDSBrowser __all__ = ['HDSModule'] class HDSModule(Module, CapMessages): NAME = 'hds' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"Histoires de Sexe French erotic novels" STORAGE = {'seen': []} BROWSER = HDSBrowser #### CapMessages ############################################## def iter_threads(self): for story in self.browser.iter_stories(): thread = Thread(story.id) thread.title = story.title thread.date = story.date yield thread GENDERS = ['', 'boy', 'girl', 'transexual'] def get_thread(self, id): if isinstance(id, Thread): thread = id id = thread.id else: thread = None story = self.browser.get_story(id) if not story: return None if not thread: thread = Thread(story.id) flags = 0 if thread.id not in self.storage.get('seen', default=[]): flags |= Message.IS_UNREAD thread.title = story.title thread.date = story.date thread.root = Message(thread=thread, id=0, title=story.title, sender=story.author.name, receivers=None, date=thread.date, parent=None, content=story.body, children=[], signature=u'Written by a %s in category %s' % (self.GENDERS[story.author.sex], story.category), flags=flags) return thread def iter_unread_messages(self): for thread in self.iter_threads(): if thread.id in self.storage.get('seen', default=[]): continue self.fill_thread(thread, 'root') yield thread.root def set_message_read(self, message): self.storage.set('seen', self.storage.get('seen', default=[]) + [message.thread.id]) self.storage.save() def fill_thread(self, thread, fields): return self.get_thread(thread) OBJECTS = {Thread: fill_thread} weboob-1.2/modules/hds/pages.py000066400000000000000000000107161303450110500165250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.pages import HTMLPage from weboob.browser.elements import method, ListElement, ItemElement from weboob.browser.filters.standard import CleanText, Regexp, Date, Env, Filter from weboob.browser.filters.html import XPath, Link class ValidationPage(HTMLPage): pass class HomePage(HTMLPage): pass class Author(object): (UNKNOWN, MALE, FEMALE, TRANSEXUAL) = xrange(4) def __init__(self, name=None): self.name = name self.sex = self.UNKNOWN self.description = None class Sex2Enum(Filter): def filter(self, text): if text == 'homme': return Author.MALE if text == 'femme': return Author.FEMALE return Author.TRANSEXUAL class Story(object): def __init__(self, id=None): self.id = id self.title = u'' self.date = None self.category = None self.author = None self.body = None class HistoryPage(HTMLPage): ENCODING = 'iso-8859-1' def get_numerous(self): return int(CleanText('//div[@align="justify"]/table[1]//td[has-class("t0")]/font/u/strong[1]')(self.doc)) @method class iter_stories(ListElement): item_xpath = '//div[@align="justify"]/span[has-class("t4")]' class item(ItemElement): klass = Story def parse(self, el): self.env['header'] = el.getprevious().xpath('.//span')[0] self.env['body'] = el.getnext().xpath('.//a') obj_id = XPath(Env('body')) & Link & Regexp(pattern=r'.*histoire=(\d+)') obj_title = CleanText('.') obj_date = XPath(Env('header')) & CleanText & Regexp(pattern=r'le (\d+)-(\d+)-(\d+)', template=r'\3-\2-\1') & Date obj_category = XPath(Env('header')) & CleanText & Regexp(pattern=u'Catégorie :\s*(.*)\s*Auteur') def obj_author(self): return Author(self.env['header'].xpath('.//a/text()')[0]) class StoryPage(HTMLPage): ENCODING = 'iso-8859-1' @method class get_story(ItemElement): klass = Story obj_id = Env('id') obj_title = CleanText('//h1') obj_date = CleanText('//span[has-class("t4")]') & Regexp(pattern=r'le (\d+)-(\d+)-(\d+)', template=r'\3-\2-\1') & Date obj_category = CleanText('//a[starts-with(@href, "histoires-cat")]') def obj_body(self): div = self.el.xpath('//div[@align="justify"]')[0] body = '' for para in div.findall('br'): if para.text is not None: body += para.text.strip() body += '\n' if para.tail is not None: body += para.tail.strip() return body.replace(u'\x92', "'").strip() class obj_author(ItemElement): klass = Author obj_name = CleanText('//a[starts-with(@href, "fiche.php")][2]') obj_sex = CleanText('//td[has-class("t0")]') & Regexp(pattern=r"Auteur (\w+)") & Author.Sex2Enum class AuthorPage(HTMLPage): @method class get_author(ItemElement): klass = Author obj_name = CleanText('//span[has-class("t3")]') obj_sex = CleanText('//td[has-class("t0")]') & Regexp(pattern=r"Auteur (\w+)") & Author.Sex2Enum def obj_description(self): description = u'' for para in self.el.xpath('//td[has-class("t0")]')[0].getchildren(): if para.tag not in ('b', 'br'): continue if para.text is not None: description += '\n\n%s' % para.text.strip() if para.tail is not None: description += '\n%s' % para.tail.strip() return description.replace(u'\x92', "'").strip() weboob-1.2/modules/hds/test.py000066400000000000000000000017321303450110500164030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest from weboob.tools.misc import limit class HDSTest(BackendTest): MODULE = 'hds' def test_new_messages(self): for message in limit(self.backend.iter_unread_messages(), 10): pass weboob-1.2/modules/hsbc/000077500000000000000000000000001303450110500152105ustar00rootroot00000000000000weboob-1.2/modules/hsbc/__init__.py000066400000000000000000000000711303450110500173170ustar00rootroot00000000000000from .module import HSBCModule __all__ = ['HSBCModule'] weboob-1.2/modules/hsbc/browser.py000066400000000000000000000113741303450110500172530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import ssl from datetime import timedelta, date from weboob.tools.date import LinearDateGuesser from weboob.capabilities.bank import Account from weboob.exceptions import BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from .pages import AccountsPage, CBOperationPage, CPTOperationPage, LoginPage, AppGonePage, RibPage __all__ = ['HSBC'] class HSBC(LoginBrowser): BASEURL = 'https://client.hsbc.fr' app_gone = False connection = URL(r'https://www.hsbc.fr/1/2/hsbc-france/particuliers/connexion', LoginPage) login = URL(r'https://www.hsbc.fr/1/*', LoginPage) cptPage = URL(r'/cgi-bin/emcgi.*\&Cpt=.*', r'/cgi-bin/emcgi.*\&Epa=.*', r'/cgi-bin/emcgi.*\&CPT_IdPrestation.*', r'/cgi-bin/emcgi.*\&Ass_IdPrestation.*', CPTOperationPage) cbPage = URL(r'/cgi-bin/emcgi.*\&Cb=.*', r'/cgi-bin/emcgi.*\&CB_IdPrestation.*', CBOperationPage) appGone = URL(r'/.*_absente.html', r'/pm_absent_inter.html', AppGonePage) rib = URL(r'/cgi-bin/emcgi', RibPage) accounts = URL(r'/cgi-bin/emcgi', AccountsPage) def __init__(self, username, password, secret, *args, **kwargs): self.accounts_list = dict() self.secret = secret LoginBrowser.__init__(self, username, password, *args, **kwargs) def load_state(self, state): return def prepare_request(self, req): preq = super(HSBC, self).prepare_request(req) conn = self.session.adapters['https://'].get_connection(preq.url) conn.ssl_version = ssl.PROTOCOL_TLSv1 return preq def do_login(self): self.connection.go() self.page.login(self.username) no_secure_key_link = self.page.get_no_secure_key() if not no_secure_key_link: raise BrowserIncorrectPassword() self.location(no_secure_key_link) self.page.login_w_secure(self.password, self.secret) for _ in range(2): if self.login.is_here(): self.page.useless_form() home_url = self.page.get_frame() if not home_url or not self.page.logged: raise BrowserIncorrectPassword() self.location(home_url) @need_login def get_accounts_list(self): if not self.accounts_list: self.update_accounts_list() for i, a in self.accounts_list.items(): yield a @need_login def update_accounts_list(self): for a in list(self.accounts.stay_or_go().iter_accounts()): try: self.accounts_list[a.id]._link_id = a._link_id except KeyError: self.accounts_list[a.id] = a self.location('%s%s' % (self.page.url, '&debr=COMPTES_RIB')) self.page.get_rib(self.accounts_list) @need_login def get_history(self, account, coming=False): if account._link_id is None: return if account._link_id.startswith('javascript') or '&Crd=' in account._link_id or \ account.type == Account.TYPE_LIFE_INSURANCE: raise NotImplementedError() self.location(self.accounts_list[account.id]._link_id) #If we relogin on hsbc, all link have change if self.app_gone: self.app_gone = False self.update_accounts_list() self.location(self.accounts_list[account.id]._link_id) if self.page is None: return if self.cbPage.is_here(): guesser = LinearDateGuesser(date_max_bump=timedelta(45)) return [tr for tr in self.page.get_history(date_guesser=guesser) if (coming and tr.date > date.today()) or (not coming and tr.date <= date.today())] elif not coming: return self._get_history() else: raise NotImplementedError() def _get_history(self): for tr in self.page.get_history(): yield tr weboob-1.2/modules/hsbc/favicon.png000066400000000000000000000040721303450110500173460ustar00rootroot00000000000000PNG  IHDR@@iqsRGBbKGDVKV pHYs  tIME%}=`+tEXtCommentCreated with GIMPWIDATxݛmlTe7mjɮMXjH4~Un"Y%&!!h1Dw_  ta] DLYJKK[СNyw;'땟_tjqlcZN'f͜OFC-[$եKb#^Bf<@tl,X(B`*I(~5JЧs-3!1PUD`j67MoLَCH!!/<z/(۾]X/P,(`ZH4) ގ YBS<ޏuٴ10l>8lxFF,#iݥ(t>w@ڦhFgQ]]FE  ?JFx`Q+xᙹ:AYFS3/ @"V{̾>uW4>No ML $ׇ¶jӧ ࡇȯ\V`"HDHՊ#6Jtd L9B~M b>YUWc_a"}}-Y&p ιy 3ƃ`q۪U~j9"CC~/\d(jNEcxZ[~nVDU a۝Ɲ&hp~ 28,IRΑ,IK*֭7c"Y`LʼudLL&QT]e+9b뭭P a4$+A7r@PѪ9/Ozkk(e9cS)cĪ$n{{+UUKgiK dӴ6x!GFG?`|՛!v|/> 6'O|[~xb<{kr:x@;:u?tr[i`#С. "B8믿Rb0}@PD8*DOD4w~tUUFi Y,S VbFo5u~]_暌seB]ީ͜???W[bgz<.]OB{9v|&8{vJCCFH&5-L~*W7ot{M4YҸ|.LK|Xwu#59%.7l(35?9]듢&}Z5=&JcRlp,;<uv*ر O?}d]Kϩ(S͕3tՁ_ֶv\>#p!":6xᅒ+/߲̑Om g9r8쭭'iܐM@<5xLJm2V7 TO<ȵwyCƴB0 p~C ?n6FN.*C==?\"57gbN~,KL;gN\lx62vmyIbl׮GD+gxowX Gc IENDB`weboob-1.2/modules/hsbc/module.py000066400000000000000000000042431303450110500170520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import HSBC __all__ = ['HSBCModule'] class HSBCModule(Module, CapBank): NAME = 'hsbc' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'HSBC France' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe'), Value( 'secret', label=u'Réponse secrète')) BROWSER = HSBC def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get(), self.config['secret'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): for tr in self.browser.get_history(account): yield tr def iter_coming(self, account): for tr in self.browser.get_history(account, coming=True): yield tr weboob-1.2/modules/hsbc/pages.py000066400000000000000000000241361303450110500166670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.filters.standard import Filter, Env, CleanText, CleanDecimal, Field, DateGuesser, TableCell, Regexp from weboob.browser.filters.html import Link class Transaction(FrenchTransaction): PATTERNS = [(re.compile(r'^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^PRLV (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(r'^CB (?P.*)\s+(?P
\d+)/(?P\d+)\s*(?P.*)'), FrenchTransaction.TYPE_CARD), (re.compile(r'^DAB (?P
\d{2})/(?P\d{2}) ((?P\d{2})H(?P\d{2}) )?(?P.*?)( CB N°.*)?$'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^CHEQUE$'), FrenchTransaction.TYPE_CHECK), (re.compile(r'^COTIS\.? (?P.*)'), FrenchTransaction.TYPE_BANK), (re.compile(r'^REMISE (?P.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(r'^FACTURES CB (?P.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), ] class AccountsPage(LoggedPage, HTMLPage): def get_frame(self): try: a = self.doc.xpath(u'//frame["@name=FrameWork"]')[0] except IndexError: return None else: return a.attrib['src'] @method class iter_accounts(ListElement): item_xpath = '//tr' flush_at_end = True class item(ItemElement): klass = Account def condition(self): return len(self.el.xpath('./td')) > 2 class Label(Filter): def filter(self, text): return text.lstrip(' 0123456789').title() class Type(Filter): def filter(self, label): invest = ['invest'] saving = ['ldd', 'livret'] account = ['compte', 'account'] loan = ['pret', 'account'] life = ['vie', 'strategie patr.', 'essentiel', 'elysees', 'abondance'] for inv in invest: if inv in label.lower(): return Account.TYPE_MARKET for acc in saving: if acc in label.lower(): return Account.TYPE_SAVINGS for acc in account: if acc in label.lower(): return Account.TYPE_CHECKING for l in loan: if l in label.lower(): return Account.TYPE_LOAN for l in life: if l in label.lower(): return Account.TYPE_LIFE_INSURANCE return Account.TYPE_UNKNOWN obj_label = Label(CleanText('./td[1]/a')) obj_coming = Env('coming') obj_currency = FrenchTransaction.Currency('./td[2]') obj__link_id = Link('./td[1]/a') obj_type = Type(Field('label')) obj_coming = NotAvailable @property def obj_balance(self): if self.el.xpath('./parent::*/tr/th') and self.el.xpath('./parent::*/tr/th')[0].text == 'Credits': balance = CleanDecimal(replace_dots=True).filter(self.el.xpath('./td[3]')) if balance < 0: return balance else: return -balance return CleanDecimal(replace_dots=True).filter(self.el.xpath('./td[3]')) @property def obj_id(self): # Investment account and main account can have the same id # so we had account type in case of Investment to prevent conflict if Field('type')(self) == Account.TYPE_MARKET: return CleanText(replace=[('.', ''), (' ', '')]).filter(self.el.xpath('./td[2]')) + ".INVEST" return CleanText(replace=[('.', ''), (' ', '')]).filter(self.el.xpath('./td[2]')) class RibPage(LoggedPage, HTMLPage): def is_here(self): return bool(self.doc.xpath('//h1[contains(text(), "RIB/IBAN")]')) def link_rib(self, accounts): for id, acc in accounts.items(): if acc.iban or not acc.type is Account.TYPE_CHECKING: continue digit_id = ''.join(re.findall('\d', id)) if digit_id in CleanText('//div[strong[contains(text(), "Account number")]]')(self.doc): acc.iban = re.search('(FR\d{25})', CleanText('//div[strong[contains(text(), "IBAN")]]', replace=[(' ', '')])(self.doc)).group(1) def get_rib(self, accounts): self.link_rib(accounts) for nb in range(len(self.doc.xpath('//select/option')) -1): form = self.get_form(name="FORM_RIB") form['index_rib'] = str(nb+1) form.submit() self.browser.page.link_rib(accounts) class Pagination(object): def next_page(self): links = self.page.doc.xpath('//a[@class="fleche"]') if len(links) == 0: return current_page_found = False for link in links: l = link.attrib.get('href') if current_page_found and "#op" not in l: # Adding CB_IdPrestation so browser2 use CBOperationPage return l + "&CB_IdPrestation" elif "#op" in l: current_page_found = True return class CBOperationPage(LoggedPage, HTMLPage): @pagination @method class get_history(Pagination, Transaction.TransactionsElement): head_xpath = '//table//tr/th' item_xpath = '//table//tr' class item(Transaction.TransactionElement): condition = lambda self: len(self.el.xpath('./td')) >= 4 obj_rdate = Transaction.Date(TableCell('date')) def obj_date(self): return DateGuesser(Regexp(CleanText(self.page.doc.xpath('//table/tr[2]/td[1]')), r'(\d{2}/\d{2})'), Env("date_guesser"))(self) class CPTOperationPage(LoggedPage, HTMLPage): def get_history(self): if self.doc.xpath('//form[@name="FORM_SUITE"]'): m = re.search('suite[\s]+=[\s]+([\w]+)', CleanText().filter(self.doc.xpath('//script[contains(text(), "var suite")]'))) if m and m.group(1) == "true": form = self.get_form(name="FORM_SUITE") self.doc = self.browser.location("%s" % form.url, params=dict(form)).page.doc for script in self.doc.xpath('//script'): if script.text is None or script.text.find('\nCL(0') < 0: continue prev_date = None for m in re.finditer(r"CL\((\d+),'(.+)','(.+)','(.+)','([\d -\.,]+)',('([\d -\.,]+)',)?'\d+','\d+','[\w\s]+'\);", script.text, flags=re.MULTILINE): op = Transaction() op.parse(date=m.group(3), raw=re.sub(u'[ ]+', u' ', m.group(4).replace(u'\n', u' '))) op.set_amount(m.group(5)) op._coming = (re.match(r'\d+/\d+/\d+', m.group(2)) is None) op.deleted = True if op.type == Transaction.TYPE_CARD_SUMMARY else False if prev_date is not None and prev_date < op.date: break prev_date = op.date yield op class AppGonePage(HTMLPage): def on_load(self): self.browser.app_gone = True self.logger.info('Application has gone. Relogging...') self.browser.do_logout() self.browser.do_login() class LoginPage(HTMLPage): @property def logged(self): if self.doc.xpath(u'//p[contains(text(), "You are now being redirected to your Personal Internet Banking.")]'): return True return False def on_load(self): for message in self.doc.getroot().cssselect('div.csPanelErrors'): raise BrowserIncorrectPassword(CleanText('.')(message)) def login(self, login): form = self.get_form(nr=2) form['userid'] = login form.submit() def get_no_secure_key(self): try: a = self.doc.xpath(u'//a[contains(text(), "Without HSBC Secure Key")]')[0] except IndexError: return None else: return a.attrib['href'] def login_w_secure(self, password, secret): form = self.get_form(nr=0) form['memorableAnswer'] = secret inputs = self.doc.xpath(u'//input[starts-with(@id, "keyrcc_password_first")]') split_pass = u'' if len(password) != len(inputs): # HSBC only use 6 first and last two from the password password = password[:6] + password[-2:] for i, inpu in enumerate(inputs): # The good field are 1,2,3 and the bad one are 11,12,21,23,24,31 and so one if int(inpu.attrib['id'].split('first')[1]) < 10: split_pass += password[i] form['password'] = split_pass form.submit() def useless_form(self): form = self.get_form(nr=0) form.submit() weboob-1.2/modules/hsbc/test.py000066400000000000000000000017341303450110500165460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class HSBCTest(BackendTest): MODULE = 'hsbc' def test_hsbc(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) weboob-1.2/modules/hybride/000077500000000000000000000000001303450110500157175ustar00rootroot00000000000000weboob-1.2/modules/hybride/__init__.py000066400000000000000000000014341303450110500200320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import HybrideModule __all__ = ['HybrideModule'] weboob-1.2/modules/hybride/browser.py000066400000000000000000000030651303450110500177600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .pages import ProgramPage, EventPage from weboob.browser import PagesBrowser, URL from weboob.browser.profiles import Firefox __all__ = ['HybrideBrowser'] class HybrideBrowser(PagesBrowser): PROFILE = Firefox() BASEURL = 'http://www.lhybride.org/' program_page = URL('programmation/a-venir.html', ProgramPage) event_page = URL('programmation/item/(?P<_id>.*)', EventPage) def list_events(self, date_from, date_to=None, city=None, categories=None): return self.program_page.go().list_events(date_from=date_from, date_to=date_to, city=city, categories=categories) def get_event(self, _id, event=None): return self.event_page.go(_id=_id).get_event(obj=event) weboob-1.2/modules/hybride/calendar.py000066400000000000000000000022421303450110500200420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES class HybrideCalendarEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.city = u'Lille' self.location = u'18 rue Gosselet' self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.category = CATEGORIES.CINE self.timezone = u'Europe/Paris' weboob-1.2/modules/hybride/favicon.png000066400000000000000000000013021303450110500200460ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIMEOIDATx۱A/nQZXوh'( Z6z" ?xՉV wX`Qz%,ȰFf=dw'ďE)EU K̔;_h'Ap(KyuM[2 :{([<z.Dg-E@R;4hfIb~-P*X2Iϒf,R8:`A9a ?1 H20ퟕtf&3{bfl\o@B I:o- Ѯ]TSo+|n]ަ|T)k%뉉A.W5CTȀ lO$Pc&;%lKjI-iFcI3Y I.ǁŐTJZ P8}j9J.`v?<^EN@,5Jͨ(?ߠ=Y~.0ofl(0znvGfXI 6IGքNHRTG~_>IENDB`weboob-1.2/modules/hybride/module.py000066400000000000000000000035501303450110500175610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.backend import Module from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES from .browser import HybrideBrowser from .calendar import HybrideCalendarEvent __all__ = ['HybrideModule'] class HybrideModule(Module, CapCalendarEvent): NAME = 'hybride' DESCRIPTION = u'hybride website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.CINE] BROWSER = HybrideBrowser def search_events(self, query): if self.has_matching_categories(query): return self.browser.list_events(query.start_date, query.end_date, query.city, query.categories) def list_events(self, date_from, date_to=None): return self.browser.list_events(date_from, date_to) def get_event(self, _id): return self.browser.get_event(_id) def fill_obj(self, event, fields): return self.browser.get_event(event.id, event) OBJECTS = {HybrideCalendarEvent: fill_obj} weboob-1.2/modules/hybride/pages.py000066400000000000000000000062351303450110500173760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .calendar import HybrideCalendarEvent import weboob.tools.date as date_util from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Filter, CleanText, Env, Format, BrowserURL, Regexp, Decode from weboob.browser.filters.html import CleanHTML from weboob.browser.filters.html import Link class Date(Filter): def filter(self, text): return date_util.parse_french_date(text) class ProgramPage(HTMLPage): @method class list_events(ListElement): item_xpath = '//div[@class="itemContainer itemContainerLast"]' class item(ItemElement): klass = HybrideCalendarEvent def validate(self, obj): return self.check_date(obj) and self.check_city(obj) and self.check_category(obj) def check_date(self, obj): if self.env['date_from'] and obj.start_date >= self.env['date_from']: if not self.env['date_to']: return True elif obj.end_date and obj.end_date <= self.env['date_to']: return True elif self.env['date_to'] >= obj.start_date: return True return False def check_city(self, obj): return (not self.env['city'] or self.env['city'].upper() == obj.city.upper()) def check_category(self, obj): return (not self.env['categories'] or obj.category in self.env['categories']) obj_id = Regexp(Link('div/div[@class="catItemHeader"]/h3[@class="catItemTitle"]/a'), '/programmation/item/(\d*?)-.*.html') obj_start_date = Date(CleanText('div/div[@class="catItemHeader"]/span[@class="catItemDateCreated"]')) obj_summary = CleanText('div/div[@class="catItemHeader"]/h3[@class="catItemTitle"]/a') class EventPage(HTMLPage): @method class get_event(ItemElement): klass = HybrideCalendarEvent obj_id = Decode(Env('_id')) obj_start_date = Date(CleanText('//span[@class="itemDateCreated"]')) obj_summary = CleanText('//h2[@class="itemTitle"]') obj_description = Format('%s\n%s', CleanHTML('//div[@class="itemIntroText"]'), CleanHTML('//div[@class="itemFullText"]')) obj_url = BrowserURL('event_page', _id=Env('_id')) weboob-1.2/modules/hybride/test.py000066400000000000000000000024421303450110500172520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime from weboob.tools.test import BackendTest, SkipTest class HybrideTest(BackendTest): MODULE = 'hybride' def test_hybride_list(self): if datetime.now() > datetime(datetime.now().year, 6, 30) and\ datetime.now() < datetime(datetime.now().year, 9, 15): raise SkipTest("Fermeture estivale") l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) weboob-1.2/modules/ideel/000077500000000000000000000000001303450110500153535ustar00rootroot00000000000000weboob-1.2/modules/ideel/__init__.py000066400000000000000000000014361303450110500174700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import IdeelModule __all__ = ['IdeelModule'] weboob-1.2/modules/ideel/browser.py000066400000000000000000000132001303450110500174040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.pages import HTMLPage from weboob.capabilities.base import Currency from weboob.capabilities.shop import Order, Item, Payment, OrderNotFound from weboob.exceptions import BrowserIncorrectPassword import re from decimal import Decimal from datetime import datetime from itertools import takewhile, count __all__ = ['Ideel'] class IdeelPage(HTMLPage): @property def logged(self): return bool(self.doc.xpath('//a[@href="/logout"]')) class LoginPage(IdeelPage): def login(self, username, password): form = self.get_form(xpath='//form[@id="iform"]') form['login'] = username form['password'] = password form.submit() class HistoryPage(IdeelPage): def exists(self): return bool(self.doc.xpath('//table[@id="order_history"]')) def iter_orders(self): return (tr.xpath('td[1]/a/text()')[0][1:] for tr in self.doc.xpath('//table[@id="order_history"]/tbody/tr')) class OrderPage(IdeelPage): def exists(self): return bool(self.order_number()) def order(self): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = self.tax() order.shipping = self.shipping() order.discount = self.discount() order.total = self.total() return order def items(self): for tr in self.doc.xpath('//table[contains(@class,"items_table")]' '//tr[td[@class="items_desc"]]'): label = tr.xpath('*//div[@class="item_desc"]//span/text()')[0] url = tr.xpath('*//div[@class="item_img"]//@src')[0] onclk = tr.xpath('*//div[@class="item_img"]//@onclick') if onclk: url=re.match(r'window.open\(\'([^\']*)\'.*', onclk[0]).group(1) if url.startswith('/'): url = self.browser.BASEURL + url price = tr.xpath('td[@class="items_price"]/span/text()')[0] qty = tr.xpath('td[@class="items_qty"]//span/text()')[0] price = AmTr.decimal_amount(price) * Decimal(qty) item = Item() item.label = unicode(label) item.url = unicode(url) item.price = price yield item def payments(self): # There's no payment information on Ideel, so we'll make one up. p = Payment() p.date = self.order_date() p.method = u'DEFAULT PAYMENT' p.amount = self.total() yield p def order_number(self): return next(iter(self.doc.xpath( u'//b[text()="Order Number:"]/../strong/text()')), None) def order_date(self): txt = self.doc.xpath('//div[@id="purchase-notice"]/text()')[0] date = re.match(r'.* (\w+ \d+, \d+)$', txt).group(1) return datetime.strptime(date, '%b %d, %Y') def tax(self): return AmTr.decimal_amount(self.doc.xpath( '//span[@id="taxes"]/text()')[0]) def shipping(self): if self.doc.xpath('//tr[@id="shipping_fee_row"]' '//span[@class="free_shipping"]'): amount = '0' else: amount = self.doc.xpath('//span[@id="shipping_fee"]/text()')[0] return AmTr.decimal_amount(amount) def discount(self): TAGS = ['coupon_discount_amount', 'promo_discount_amount', 'total_rewards', 'applied_credit'] return -sum(AmTr.decimal_amount(x[1:][:-1]) for tag in TAGS for x in self.doc.xpath('//span[@id="%s"]/text()' % tag)) def total(self): return AmTr.decimal_amount(self.doc.xpath( '//span[@id="total"]/text()')[0]) class Ideel(LoginBrowser): BASEURL = 'http://www.ideel.com' login = URL(r'https://www.ideel.com/login$', LoginPage) history = URL(r'/my_account/orders\?page=(?P\d+)$', HistoryPage) order = URL(r'/my_account/orders/(?P\d+)$', OrderPage) unknown = URL(r'/.*$', IdeelPage) def get_currency(self): # Ideel uses only U.S. dollars. return Currency.get_currency(u'$') @need_login def get_order(self, id_): if self.order.go(order=id_).exists(): return self.page.order() raise OrderNotFound() @need_login def iter_orders(self): exists = HistoryPage.exists hists = takewhile(exists, (self.history.go(page=i) for i in count(1))) return (self.get_order(x) for h in hists for x in h.iter_orders()) @need_login def iter_payments(self, order): return self.order.stay_or_go(order=order.id).payments() @need_login def iter_items(self, order): return self.order.stay_or_go(order=order.id).items() def do_login(self): self.login.stay_or_go().login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() weboob-1.2/modules/ideel/favicon.png000066400000000000000000000050131303450110500175050ustar00rootroot00000000000000PNG  IHDR@@PegAMA a cHRMz&u0`:pQ<bKGD1tIME ^ IDATx{pTE=W`+E*Q4l`Er!]1U0"(YY/V]DG0 J@@$$$̷f LtVWu>_sH_4iN#"FD 8Np&4"8MiDpӈ4 }4 )E^>i7HR$bwUkU. ;J$Ӵmw@EMoJY#K(Mo9rp C< 󑬫rTDixפ҇ <{j.hZT'A.KuC9jDU0Q)#4q"kqUA0I=ziB9cLn i>cK7;vc%Yg6C8عnb|8<_8iN.,-ۃo} iV I 1r!jC`,_ǘfH:sP)­ T8gĕ3^Dcn^2Gүw811@:i̟-[&[vC0oiG0Z D5PWD`hNTUuIZ%LJ4iwJ+/(1\Oʬr%Ҭ)3^bBׅhx>rKO\!=#KkGJ?^jS6wR'5Ho%@%U0iq x>dpC]P \&3\'m 9/Y;}< /~aV%e @+ߞ{? ΀ CO!m4n1lb­;(L .@*U3FImvKʤ/RvK U] Is6JRѵRQ[ w KI|h䍂鳖퇥҃}ʽ~J=sCU-XӋwHqK$i蘛/*M IYRRn:IZ9l,}<pX|1i?m-+ )-a LN2Ɯũ?o4\c2As@֍G2 `SZzĘ@P ٛ ? ~kLo\󍹾Z ??_%<F;k7̓v IjS3!~7OS{mUB{pu9$mۓn h?l:#`}z%tY|f EU <^^x b y~(B!})L7K2v"-[a1tZMU,K@Ns˔Ԯ``(t#T} AIȘ /'iW#q7 `T+ׇ̊ W3᪖IÉ0!a=ץBv ':~r1ͦTx_"u&*XNVLjބ$<@rh \t9kgO͚|(z6LO$o7g˭r3\`e\ߎ*oCYe~cAv"o:YYAF$kҌ^

b>BLvTѳ0t_gp0kLHIL -q_BMZ ID(ԷJJIFz,%|eY&efdY`0)vTaROzz\\.yһ 㥞=CC\']T[۩ƲoQx)'{@ڲV&]*=(LJMJΓ<oNAK%>,}|j46M(Ue~znM4rR)#QJZ u/oR}SJ,~rW++U "b)͜!?G-ŧJ]wKl^T~(+~5(n z&K7H˥Ҩ7KnevJ LRKnR2)>K=Hx4˾WK ERMq#x=x)Z/ jxp&\3 >^.mΔ +o+u'muIwJCUwP{t@Rye3mDtӈ4iN#"FD 8Np&4"8MiDp7*@%tEXtdate:create2015-02-23T19:11:18-06:00'%tEXtdate:modify2015-02-23T19:11:18-06:00L/IENDB`weboob-1.2/modules/ideel/module.py000066400000000000000000000035261303450110500172200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.shop import CapShop from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Ideel __all__ = ['IdeelModule'] class IdeelModule(Module, CapShop): NAME = 'ideel' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Ideel' CONFIG = BackendConfig( ValueBackendPassword('username', label='User name', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = Ideel def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) def get_currency(self): return self.browser.get_currency() def get_order(self, id_): return self.browser.get_order(id_) def iter_orders(self): return self.browser.iter_orders() def iter_payments(self, order): return self.browser.iter_payments(order) def iter_items(self, order): return self.browser.iter_items(order) weboob-1.2/modules/ideel/test.py000066400000000000000000000021441303450110500167050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class IdeelTest(BackendTest): MODULE = 'ideel' def test_history(self): """ Test that at least one item was ordered in the whole history. """ b = self.backend items = (i for o in b.iter_orders() for i in b.iter_items(o)) item = next(items, None) self.assertNotEqual(item, None) weboob-1.2/modules/ilmatieteenlaitos/000077500000000000000000000000001303450110500200055ustar00rootroot00000000000000weboob-1.2/modules/ilmatieteenlaitos/__init__.py000066400000000000000000000014571303450110500221250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import IlmatieteenlaitosModule __all__ = ['IlmatieteenlaitosModule'] weboob-1.2/modules/ilmatieteenlaitos/browser.py000066400000000000000000000040651303450110500220470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.browsers import PagesBrowser from weboob.browser.url import URL from .pages import WeatherPage, SearchCitiesPage, ObservationsPage __all__ = ['IlmatieteenlaitosBrowser'] class IlmatieteenlaitosBrowser(PagesBrowser): BASEURL = 'http://ilmatieteenlaitos.fi' cities = URL('/etusivu\?p_p_id=locationmenuportlet_WAR_fmiwwwweatherportlets&p_p_lifecycle=2&p_p_state=normal&' 'p_p_mode=view&p_p_cacheability=cacheLevelFull&term=(?P.*)', SearchCitiesPage) weather_query = URL('/paikallissaa\?p_p_id=locationmenuportlet_WAR_fmiwwwweatherportlets&p_p_lifecycle=1&' 'p_p_state=normal&p_p_mode=view&_locationmenuportlet_WAR_fmiwwwweatherportlets_action=' 'changelocation') weather = URL('/saa/(?P.*)', WeatherPage) observations = URL('/observation-data\?station=(?P.*)', ObservationsPage) def iter_city_search(self, pattern): return self.cities.go(pattern=pattern).iter_cities() def iter_forecast(self, city): return self.weather_query.go(data={"place": city.name, "forecast": "short"}).iter_forecast() def get_current(self, city): station_id = self.weather_query.go(data={"place": city.name, "forecast": "short"}).get_station_id() return self.observations.go(station_id=station_id).get_current() weboob-1.2/modules/ilmatieteenlaitos/favicon.png000066400000000000000000000042451303450110500221450ustar00rootroot00000000000000PNG  IHDR@@% pHYs  tIME,stEXtCommentCreated with GIMPWIDAThZlSpLb0 {u"b`/$+ Y7nlUDH i5 J4)m YԩTdk)[a0B4f5L֗ 4Nj~/6Zs}s>IXF>rAsN  d2Rh4FTJ$,df@bT%07$IfJ]ZHRK5kdeezCCCN/ ƘL͖V__G===333IArt_7ů_53ȟٳgu:S. &s|´޲o7f'Zc̃ׯK---/^`4y<^2"088V#Hz?q;|vw:ʸNߞ֐`xccrCP,sUVV]fޏ'#KG尖R##vZp^bTTT0"pĉ:7>Gts"ޑ]Y鐭*r_hoo?x`;kkkk]]`;C)n2 3BW*SB`×.]uuutfU7yt عp~BEEݻwC|T K ^D_O(c}zm`pΡVm+}4 G6ӧTVV.M jj~2!3&'\F/is( ZK0LHdwTWm'̄޵DL&S5000hr;pB28& b?^ a 83@`?2rzѣ uݥqlBYK:v.L!$7l~k]Z@sɻ$uL,߻w:?΀niA~P 577|'O%s#1ƬVkz1ϑ˳Zcccl6ȱȔٳ' ԩS&}9!d';, lv4ZRFW< Z Vk4e|>)#kȬiZ>r;3Lb f#voi5J Y!p Rh,_nˌ^R)BXrb=>B?rm>DlL g>MOdlR @șf>!e2T*@N^`*QoLhf pF&Vg8N,*0d@l I$#_2LD"as8^Ue3ؐeu9 zPSS `<;c. ȫ7rO3g`B^?츪38~Df>Bp.DU}'?=RLfe@ggg 0:E sks;&$eܛ7K&.P3 yZ.ߺu+F`bf3W~> 7Ti=kպFfySaahv\3qp*Zc4gh5D"T[sƉZh\.7 -,lx<O}s͵mˈ.%;p8JX bpڭٴ >\˂MX,Z}rpRR`0޾l%ZСCW4y XYyeodFTUUcǎ54,2J@M!gH&-'<%gQlH䌝L8 E(q}ɭf㛘~G=U ;+Ҟ[* %>`pJ5w0jfovN;;;)\.7 Kز_~k8 __wcО0+IENDB`weboob-1.2/modules/ilmatieteenlaitos/module.py000066400000000000000000000032411303450110500216440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.weather import CapWeather, CityNotFound from weboob.tools.backend import Module from weboob.capabilities.base import find_object from .browser import IlmatieteenlaitosBrowser __all__ = ['IlmatieteenlaitosModule'] class IlmatieteenlaitosModule(Module, CapWeather): NAME = 'ilmatieteenlaitos' MAINTAINER = u'Matthieu Weber' EMAIL = 'mweber+weboob@free.fr' VERSION = '1.2' DESCRIPTION = 'Get forecasts from the Ilmatieteenlaitos.fi website' LICENSE = 'AGPLv3+' BROWSER = IlmatieteenlaitosBrowser def get_current(self, city_id): return self.browser.get_current(self.get_city(city_id)) def iter_forecast(self, city_id): return self.browser.iter_forecast(self.get_city(city_id)) def iter_city_search(self, pattern): return self.browser.iter_city_search(pattern) def get_city(self, _id): return find_object(self.iter_city_search(_id), id=_id, error=CityNotFound) weboob-1.2/modules/ilmatieteenlaitos/pages.py000066400000000000000000000130361303450110500214610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date from itertools import imap, ifilter from weboob.browser.pages import JsonPage, HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.capabilities.weather import Forecast, Current, City, Temperature from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import Filter, CleanText class Id(Filter): def filter(self, txt): return txt.split(", ")[0] class SearchCitiesPage(JsonPage): @method class iter_cities(ListElement): ignore_duplicate = True def find_elements(self): for el in self.el: yield el class item(ItemElement): klass = City obj_id = Id(Dict('id')) obj_name = Dict('value') class WeatherPage(HTMLPage): @method class iter_forecast(ListElement): item_xpath = ('//div[contains(@class, "mid") and contains(@class, "local-weather-forecast")]//' 'tr[@class="meteogram-dates"]/td') class item(ItemElement): klass = Forecast obj_id = CleanText('.//span/@title') def obj_date(self): months = [u'tammikuuta', u'helmikuuta', u'maaliskuuta', u'huhtikuuta', u'toukokuuta', u'kesäkuuta', u'heinäkuuta', u'elokuuta', u'syyskuuta', u'lokakuuta', u'marraskuuta', u'joulukuuta'] d = CleanText('.//span/@title')(self).split() return date(int(d[2]), months.index(d[1])+1, int(d[0].strip("."))) def temperatures(self): offset = int(CleanText('string(sum(./preceding-sibling::td/@colspan))')(self)) length = int(CleanText('@colspan')(self)) temps = CleanText('../../../tbody/tr[@class="meteogram-temperatures"]/td[position() > %d ' 'and position() <= %d]/div' % (offset, offset+length))(self) return [float(_.strip(u'\xb0')) for _ in temps.split()] def obj_low(self): return Temperature(min(self.temperatures()), u'C') def obj_high(self): return Temperature(max(self.temperatures()), u'C') def obj_text(self): offset = int(CleanText('string(sum(./preceding-sibling::td/@colspan))')(self)) length = int(CleanText('@colspan')(self)) hour_test = ('../../tr[@class="meteogram-times"]/td[position() > %d and position() <= %d ' 'and .//text() = "%%s"]' % (offset, offset+length)) hour_offset = 'string(count(%s/preceding-sibling::td)+1)' % (hour_test) values = [ '../../../tbody/tr[@class="meteogram-weather-symbols"]/td[position() = %d]/div/@title', '../../../tbody/tr[@class="meteogram-apparent-temperatures"]/td[position() = %d]/div/@title', '../../../tbody/tr[@class="meteogram-wind-symbols"]/td[position() = %d]/div/@title', '../../../tbody/tr[@class="meteogram-probabilities-of-precipitation"]/td[position() = %d]' + '/div/@title', '../../../tbody/tr[@class="meteogram-hourly-precipitation-values"]/td[position() = %d]/span/@title', ] def descriptive_text_for_hour(hour): hour_exists = CleanText(hour_test % hour)(self) == hour if hour_exists: offset = int(CleanText(hour_offset % hour)(self)) def info_for_value(value): return CleanText(value % offset)(self).replace(u'edeltävän tunnin ', u'') return ("klo %s: " % hour) + ", ".join(ifilter(bool, imap(info_for_value, values))) return u'\n' + u'\n'.join(ifilter(bool, imap(descriptive_text_for_hour, ["02", "03", "14", "15"]))) def get_station_id(self): return CleanText(u'//select[@id="observation-station-menu"]/option[@selected="selected"]/@value')(self.doc) class ObservationsPage(JsonPage): WINDS = ['N', 'NNE', 'NE', 'ENE', 'E', 'ESE', 'SE', 'SSE', 'S', 'SSW', 'SW', 'WSW', 'W', 'WNW', 'NW', 'NNW'] def get_current(self): obj = Current() obj.id = date.today() obj.date = date.fromtimestamp(self.doc['latestObservationTime']/1000) obj.temp = Temperature(max(self.doc['t2m'])[1], u'C') last_hour_precipitations = int(max(self.doc['Precipitation1h'])[1]) nebulosity = int(max(self.doc['TotalCloudCover'])[1]) wind_speed = int(max(self.doc['WindSpeedMS'])[1]) wind_direction = self.WINDS[int(max(self.doc['WindDirection'])[1] / 22.5)] obj.text = u'1h precipitations %d mm, wind %d m/s (%s), nebulosity %d/8' % ( last_hour_precipitations, wind_speed, wind_direction, nebulosity) return obj weboob-1.2/modules/ilmatieteenlaitos/test.py000066400000000000000000000023341303450110500213400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class IlmatieteenlaitosTest(BackendTest): MODULE = 'ilmatieteenlaitos' def test_ilmatieteenlaitos(self): l = list(self.backend.iter_city_search('helsinki')) self.assertTrue(len(l) > 0) city = l[0] current = self.backend.get_current(city.id) self.assertTrue(current.temp.value > -50 and current.temp.value < 50) forecasts = list(self.backend.iter_forecast(city.id)) self.assertTrue(len(forecasts) > 0) weboob-1.2/modules/imdb/000077500000000000000000000000001303450110500152045ustar00rootroot00000000000000weboob-1.2/modules/imdb/__init__.py000066400000000000000000000014251303450110500173170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ImdbModule __all__ = ['ImdbModule'] weboob-1.2/modules/imdb/browser.py000066400000000000000000000204431303450110500172440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from HTMLParser import HTMLParser from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.capabilities.cinema import Movie, Person from weboob.tools.json import json from .pages import PersonPage, MovieCrewPage, BiographyPage, ReleasePage from datetime import datetime __all__ = ['ImdbBrowser'] class ImdbBrowser(Browser): DOMAIN = 'www.imdb.com' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://www.imdb.com/title/tt[0-9]*/fullcredits.*': MovieCrewPage, 'http://www.imdb.com/title/tt[0-9]*/releaseinfo.*': ReleasePage, 'http://www.imdb.com/name/nm[0-9]*/*': PersonPage, 'http://www.imdb.com/name/nm[0-9]*/bio.*': BiographyPage, } def iter_movies(self, pattern): res = self.readurl('http://www.imdb.com/xml/find?json=1&nr=1&tt=on&q=%s' % pattern.encode('utf-8')) jres = json.loads(res) htmlparser = HTMLParser() for cat in ['title_popular', 'title_exact', 'title_approx']: if cat in jres: for m in jres[cat]: tdesc = unicode(m['title_description']) if '' in tdesc: short_description = u'%s %s' % (tdesc.split('<')[ 0].strip(', '), tdesc.split('>')[1].split('<')[0]) else: short_description = tdesc.strip(', ') movie = Movie(m['id'], htmlparser.unescape(m['title'])) movie.other_titles = NotLoaded movie.release_date = NotLoaded movie.duration = NotLoaded movie.short_description = htmlparser.unescape(short_description) movie.pitch = NotLoaded movie.country = NotLoaded movie.note = NotLoaded movie.roles = NotLoaded movie.all_release_dates = NotLoaded movie.thumbnail_url = NotLoaded yield movie def iter_persons(self, pattern): res = self.readurl('http://www.imdb.com/xml/find?json=1&nr=1&nm=on&q=%s' % pattern.encode('utf-8')) jres = json.loads(res) htmlparser = HTMLParser() for cat in ['name_popular', 'name_exact', 'name_approx']: if cat in jres: for p in jres[cat]: person = Person(p['id'], htmlparser.unescape(unicode(p['name']))) person.real_name = NotLoaded person.birth_place = NotLoaded person.birth_date = NotLoaded person.death_date = NotLoaded person.gender = NotLoaded person.nationality = NotLoaded person.short_biography = NotLoaded person.short_description = htmlparser.unescape(p['description']) person.roles = NotLoaded person.thumbnail_url = NotLoaded yield person def get_movie(self, id): res = self.readurl('http://www.omdbapi.com/?i=%s&plot=full' % id) if res is not None: jres = json.loads(res) else: return None htmlparser = HTMLParser() title = NotAvailable duration = NotAvailable release_date = NotAvailable pitch = NotAvailable country = NotAvailable note = NotAvailable short_description = NotAvailable thumbnail_url = NotAvailable other_titles = [] genres = [] roles = {} if 'Title' not in jres: return title = htmlparser.unescape(unicode(jres['Title'].strip())) if 'Poster' in jres: thumbnail_url = unicode(jres['Poster']) if 'Director' in jres: short_description = unicode(jres['Director']) if 'Genre' in jres: for g in jres['Genre'].split(', '): genres.append(g) if 'Runtime' in jres: m = re.search('(\d+?) min', jres['Runtime']) if m: duration = int(m.group(1)) if 'Released' in jres: released_string = str(jres['Released']) if released_string == 'N/A': release_date = NotAvailable else: months = { 'Jan':'01', 'Feb':'02', 'Mar':'03', 'Apr':'04', 'May':'05', 'Jun':'06', 'Jul':'07', 'Aug':'08', 'Sep':'09', 'Oct':'10', 'Nov':'11', 'Dec':'12', } for st in months: released_string = released_string.replace(st,months[st]) release_date = datetime.strptime(released_string, '%d %m %Y') if 'Country' in jres: country = u'' for c in jres['Country'].split(', '): country += '%s, ' % c country = country[:-2] if 'Plot' in jres: pitch = unicode(jres['Plot']) if 'imdbRating' in jres and 'imdbVotes' in jres: note = u'%s/10 (%s votes)' % (jres['imdbRating'], jres['imdbVotes']) for r in ['Actors', 'Director', 'Writer']: if '%s' % r in jres.keys(): roles['%s' % r] = [('N/A',e) for e in jres['%s' % r].split(', ')] movie = Movie(id, title) movie.other_titles = other_titles movie.release_date = release_date movie.duration = duration movie.genres = genres movie.pitch = pitch movie.country = country movie.note = note movie.roles = roles movie.short_description = short_description movie.all_release_dates = NotLoaded movie.thumbnail_url = thumbnail_url return movie def get_person(self, id): try: self.location('http://www.imdb.com/name/%s' % id) except BrowserHTTPNotFound: return assert self.is_on_page(PersonPage) return self.page.get_person(id) def get_person_biography(self, id): self.location('http://www.imdb.com/name/%s/bio' % id) assert self.is_on_page(BiographyPage) return self.page.get_biography() def iter_movie_persons(self, movie_id, role): self.location('http://www.imdb.com/title/%s/fullcredits' % movie_id) assert self.is_on_page(MovieCrewPage) for p in self.page.iter_persons(role): yield p def iter_person_movies(self, person_id, role): self.location('http://www.imdb.com/name/%s' % person_id) assert self.is_on_page(PersonPage) return self.page.iter_movies(role) def iter_person_movies_ids(self, person_id): self.location('http://www.imdb.com/name/%s' % person_id) assert self.is_on_page(PersonPage) for movie in self.page.iter_movies_ids(): yield movie def iter_movie_persons_ids(self, movie_id): self.location('http://www.imdb.com/title/%s/fullcredits' % movie_id) assert self.is_on_page(MovieCrewPage) for person in self.page.iter_persons_ids(): yield person def get_movie_releases(self, id, country): self.location('http://www.imdb.com/title/%s/releaseinfo' % id) assert self.is_on_page(ReleasePage) return self.page.get_movie_releases(country) weboob-1.2/modules/imdb/favicon.png000066400000000000000000000037651303450110500173520ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME ӊIDATxmlSIHHHM[UJ!iF5\B+Ok2I, i:A`t!%UCH$`;`KW&ϧks}y}QC`@P( `]:l@60R$ І>nׁ+@=n9w7jmfӎ_Kw'm;A0:`kRQC`ni0"`7,v얃wd  kmvf9U;_^-{ɚ (+]f4 0E!)v p!&k HCN"+[lfu7P<␏&`hNz951t}WdО͎{8\DUa4ڈ|F-scY65{QPH130 .{:>#_aN?,^”i /KØw{p1T_x{ al>'%YWۏ'Ѩ¿Gf{ ^^|FmM Y7>\p󦏣.6ld΃Wxy/0\@`*; pB|ϿЎ%*'7'.a琳mjWE\XY^*Nd?&T:HύC"#M}8nl^~):Çwrոq:jQQ;LR hl~"j<nrf$oiWNua-Pw˕~&NxtZ _}=a*5'*$j~g߁;,Zʵ#pT*ڻ_y" zRRrQUsYJ˚ȝ<Tnxͮ8v@u$7oq6(mopb#"Q璖ok"@ 3y+W6ˋ.w ˖VU+yxXtjkTUٺ='l._8G%Q7f0.^pv0M̜Bjd5[6i*O@^Vk L!k֫TU9uƅ:ҝFNʖ'\̝###yЃ"Ex_-r"9Y=2H/~UCex>r+i̘Gcs'Gz[?+3zP)mu꣏sϞǣ$)=:^(ޘ!u9:VL(@ ԒŢ:;(ZSG`1h Yi+O:oHXҝpjF/M}* 7@Ws5n{SDW;FA^>;;}p9T䗛nܗ'; |KL?4Y?WT!1-"iqO@VsEJIgdxIr9 Tg_053^DډI{XK,G(Xm`G=d~KQ9ZS&ǴDo$P!ɩa5~ց)0=W 46{ )I -Uͬ<&Ƹl̞c] cF>>̖nh#! ơlv^fl#|!R C="$cVf|C;oC ]ˀ!"}>qP7;;o @Po+gb(IENDB`weboob-1.2/modules/imdb/module.py000066400000000000000000000100701303450110500170410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.cinema import CapCinema, Person, Movie from weboob.tools.backend import Module from .browser import ImdbBrowser from urllib import quote_plus __all__ = ['ImdbModule'] class ImdbModule(Module, CapCinema): NAME = 'imdb' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Internet Movie Database service' LICENSE = 'AGPLv3+' BROWSER = ImdbBrowser def get_movie(self, id): return self.browser.get_movie(id) def get_person(self, id): return self.browser.get_person(id) def iter_movies(self, pattern): return self.browser.iter_movies(quote_plus(pattern.encode('utf-8'))) def iter_persons(self, pattern): return self.browser.iter_persons(quote_plus(pattern.encode('utf-8'))) def iter_movie_persons(self, id, role=None): return self.browser.iter_movie_persons(id, role) def iter_person_movies(self, id, role=None): return self.browser.iter_person_movies(id, role) def iter_person_movies_ids(self, id): return self.browser.iter_person_movies_ids(id) def iter_movie_persons_ids(self, id): return self.browser.iter_movie_persons_ids(id) def get_person_biography(self, id): return self.browser.get_person_biography(id) def get_movie_releases(self, id, country=None): return self.browser.get_movie_releases(id, country) def fill_person(self, person, fields): if 'real_name' in fields or 'birth_place' in fields\ or 'death_date' in fields or 'nationality' in fields\ or 'short_biography' in fields or 'roles' in fields\ or 'birth_date' in fields or 'thumbnail_url' in fields\ or 'gender' in fields or fields is None: per = self.get_person(person.id) person.real_name = per.real_name person.birth_date = per.birth_date person.death_date = per.death_date person.birth_place = per.birth_place person.gender = per.gender person.nationality = per.nationality person.short_biography = per.short_biography person.short_description = per.short_description person.roles = per.roles person.thumbnail_url = per.thumbnail_url if 'biography' in fields: person.biography = self.get_person_biography(person.id) return person def fill_movie(self, movie, fields): if 'other_titles' in fields or 'release_date' in fields\ or 'duration' in fields or 'country' in fields\ or 'roles' in fields or 'note' in fields\ or 'thumbnail_url' in fields: mov = self.get_movie(movie.id) movie.other_titles = mov.other_titles movie.release_date = mov.release_date movie.duration = mov.duration movie.pitch = mov.pitch movie.country = mov.country movie.note = mov.note movie.roles = mov.roles movie.genres = mov.genres movie.short_description = mov.short_description movie.thumbnail_url = mov.thumbnail_url if 'all_release_dates' in fields: movie.all_release_dates = self.get_movie_releases(movie.id) return movie OBJECTS = { Person: fill_person, Movie: fill_movie } weboob-1.2/modules/imdb/pages.py000066400000000000000000000224731303450110500166650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.cinema import Person, Movie from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.deprecated.browser import Page from weboob.tools.html import html2text from datetime import datetime import re class ReleasePage(Page): ''' Page containing releases of a movie ''' def get_movie_releases(self, country_filter): result = unicode() links = self.parser.select(self.document.getroot(), 'table#release_dates a') for a in links: href = a.attrib.get('href', '') # XXX: search() could raise an exception if href.strip('/').split('/')[0] == 'calendar' and\ (country_filter is None or re.search('region=([a-zA-Z]+)&', href).group(1).lower() == country_filter): country = a.text td_date = self.parser.select(a.getparent().getparent().getparent(), 'td')[1] date_links = self.parser.select(td_date, 'a') if len(date_links) > 1: date = date_links[1].attrib.get('href', '').strip('/').split('/')[-1] date += '-'+date_links[0].attrib.get('href', '').strip('/').split('/')[-1] else: date = unicode(self.parser.select(a.getparent().getparent().getparent(), 'td')[1].text_content()) result += '%s : %s\n' % (country, date) if result == u'': result = NotAvailable else: result = result.strip() return result class BiographyPage(Page): ''' Page containing biography of a person ''' def get_biography(self): bio = unicode() start = False tn = self.parser.select(self.document.getroot(), 'div#bio_content', 1) for el in tn.getchildren(): if el.attrib.get('name') == 'mini_bio': start = True if start: bio += html2text(self.parser.tostring(el)) return bio class MovieCrewPage(Page): ''' Page listing all the persons related to a movie ''' def iter_persons(self, role_filter=None): if (role_filter is None or (role_filter is not None and role_filter == 'actor')): tables = self.parser.select(self.document.getroot(), 'table.cast_list') if len(tables) > 0: table = tables[0] tds = self.parser.select(table, 'td.itemprop') for td in tds: id = td.find('a').attrib.get('href', '').strip('/').split('/')[1] name = unicode(td.find('a').text) char_name = unicode(self.parser.select(td.getparent(), 'td.character', 1).text_content()) person = Person(id, name) person.short_description = char_name person.real_name = NotLoaded person.birth_place = NotLoaded person.birth_date = NotLoaded person.death_date = NotLoaded person.gender = NotLoaded person.nationality = NotLoaded person.short_biography = NotLoaded person.roles = NotLoaded person.thumbnail_url = NotLoaded yield person for gloss_link in self.parser.select(self.document.getroot(), 'table[cellspacing="1"] h5 a'): role = gloss_link.attrib.get('name', '').rstrip('s') if (role_filter is None or (role_filter is not None and role == role_filter)): tbody = gloss_link.getparent().getparent().getparent().getparent() for line in self.parser.select(tbody, 'tr')[1:]: for a in self.parser.select(line, 'a'): role_detail = NotAvailable href = a.attrib.get('href', '') if '/name/nm' in href: id = href.strip('/').split('/')[-1] name = unicode(a.text) if 'glossary' in href: role_detail = unicode(a.text) person = Person(id, name) person.short_description = role_detail yield person # yield self.browser.get_person(id) def iter_persons_ids(self): tables = self.parser.select(self.document.getroot(), 'table.cast_list') if len(tables) > 0: table = tables[0] tds = self.parser.select(table, 'td.itemprop') for td in tds: id = td.find('a').attrib.get('href', '').strip('/').split('/')[1] yield id class PersonPage(Page): ''' Page informing about a person It is used to build a Person instance and to get the movie list related to a person ''' def get_person(self, id): name = NotAvailable short_biography = NotAvailable short_description = NotAvailable birth_place = NotAvailable birth_date = NotAvailable death_date = NotAvailable real_name = NotAvailable gender = NotAvailable thumbnail_url = NotAvailable roles = {} nationality = NotAvailable td_overview = self.parser.select(self.document.getroot(), 'td#overview-top', 1) descs = self.parser.select(td_overview, 'span[itemprop=description]') if len(descs) > 0: short_biography = unicode(descs[0].text) rname_block = self.parser.select(td_overview, 'div.txt-block h4.inline') if len(rname_block) > 0 and "born" in rname_block[0].text.lower(): links = self.parser.select(rname_block[0].getparent(), 'a') for a in links: href = a.attrib.get('href', '').strip() if href == 'bio': real_name = unicode(a.text.strip()) elif 'birth_place' in href: birth_place = unicode(a.text.lower().strip()) names = self.parser.select(td_overview, 'h1 span[itemprop=name]') if len(names) > 0: name = unicode(names[0].text.strip()) times = self.parser.select(td_overview, 'time[itemprop=birthDate]') if len(times) > 0: time = times[0].attrib.get('datetime', '').split('-') if len(time) == 3 and int(time[0]) >= 1900: birth_date = datetime(int(time[0]), int(time[1]), int(time[2])) dtimes = self.parser.select(td_overview, 'time[itemprop=deathDate]') if len(dtimes) > 0: dtime = dtimes[0].attrib.get('datetime', '').split('-') if len(dtime) == 3 and int(dtime[0]) >= 1900: death_date = datetime(int(dtime[0]), int(dtime[1]), int(dtime[2])) img_thumbnail = self.parser.select(self.document.getroot(), 'td#img_primary img') if len(img_thumbnail) > 0: thumbnail_url = unicode(img_thumbnail[0].attrib.get('src', '')) roles = self.get_roles() person = Person(id, name) person.real_name = real_name person.birth_date = birth_date person.death_date = death_date person.birth_place = birth_place person.gender = gender person.nationality = nationality person.short_biography = short_biography person.short_description = short_description person.roles = roles person.thumbnail_url = thumbnail_url return person def iter_movies_ids(self): for role_div in self.parser.select(self.document.getroot(), 'div#filmography div.filmo-category-section > div'): for a in self.parser.select(role_div, 'a'): m = re.search('/title/(tt.*)/\?.*', a.attrib.get('href')) if m: yield m.group(1) def get_roles(self): roles = {} for role_div in self.parser.select(self.document.getroot(), 'div#filmography > div.head'): role = self.parser.select(role_div, 'a')[-1].text roles[role] = [] category = role_div.attrib.get('data-category') for infos in self.parser.select(self.document.getroot(), 'div#filmography > div.filmo-category-section > div'): if category in infos.attrib.get('id'): roles[role].append(('N/A',infos.text_content().replace('\n', ' ').strip())) return roles def iter_movies(self, role_filter=None): for role_div in self.parser.select(self.document.getroot(), 'div#filmography > div.filmo-category-section > div'): for a in self.parser.select(role_div, 'a'): m = re.search('/title/(tt.*)/\?.*', a.attrib.get('href')) if m: yield Movie(m.group(1), a.text) weboob-1.2/modules/imdb/test.py000066400000000000000000000054611303450110500165430ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class ImdbTest(BackendTest): MODULE = 'imdb' def test_search_movie(self): movies = list(self.backend.iter_movies('spiderman')) assert len(movies) > 0 for movie in movies: assert movie.id def test_get_movie(self): movie = self.backend.get_movie('tt0079980') assert movie assert movie.id assert movie.original_title def test_search_person(self): persons = list(self.backend.iter_persons('dewaere')) assert len(persons) > 0 for person in persons: assert person.id def test_get_person(self): person = self.backend.get_person('nm0223033') assert person assert person.id assert person.name assert person.birth_date def test_movie_persons(self): persons = list(self.backend.iter_movie_persons('tt0079980')) assert len(persons) > 0 for person in persons: assert person.id assert person.name assert person.short_description def test_person_movies(self): movies = list(self.backend.iter_person_movies('nm0223033')) assert len(movies) > 0 for movie in movies: assert movie.id assert movie.original_title def test_get_person_biography(self): bio = self.backend.get_person_biography('nm0223033') assert bio != '' assert bio is not None def test_get_movie_releases(self): rel = self.backend.get_movie_releases('tt0079980', 'fr') assert rel != '' assert rel is not None assert rel == 'France : 25 April 1979' def test_iter_person_movies_ids(self): movies_ids = list(self.backend.iter_person_movies_ids('nm0223033')) assert len(movies_ids) > 0 for movie_id in movies_ids: assert movie_id def test_iter_movie_persons_ids(self): persons_ids = list(self.backend.iter_movie_persons_ids('tt0079980')) assert len(persons_ids) > 0 for person_id in persons_ids: assert person_id weboob-1.2/modules/imgur/000077500000000000000000000000001303450110500154145ustar00rootroot00000000000000weboob-1.2/modules/imgur/__init__.py000066400000000000000000000014301303450110500175230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import ImgurModule __all__ = ['ImgurModule'] weboob-1.2/modules/imgur/browser.py000066400000000000000000000053101303450110500174500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import URL from weboob.browser.browsers import APIBrowser from weboob.capabilities.gallery import CapGallery class ImgurBrowser(APIBrowser): BASEURL = 'https://api.imgur.com' CLIENT_ID = '87a8e692cb09382' SORT_TYPE = { CapGallery.SEARCH_DATE: 'time', CapGallery.SEARCH_VIEWS: 'viral', CapGallery.SEARCH_RATING: 'top', CapGallery.SEARCH_RELEVANCE: 'top', } search_url = URL(r'/3/gallery/search/(?P\w+)/(?P\d+)/\?q=(?P.*)') get_gallery_url = URL(r'/3/album/(?P\w+)') get_image_url = URL(r'/3/image/(?P\w+)') def open_raw(self, *args, **kwargs): return super(ImgurBrowser, self).open(*args, **kwargs) def open(self, *args, **kwargs): kwargs.setdefault('headers', {}) kwargs['headers']['Authorization'] = 'Client-ID %s' % self.CLIENT_ID return super(ImgurBrowser, self).open(*args, **kwargs) def request(self, *args, **kwargs): reply = super(ImgurBrowser, self).request(*args, **kwargs) if reply['success']: return reply['data'] def post_image(self, b64, title=''): res = {} params = {'image': b64, 'title': title or '', 'type': 'base64'} info = self.request('https://api.imgur.com/3/image', data=params) if info is not None: res['id'] = info['id'] res['delete_url'] = 'https://api.imgur.com/3/image/%s' % info['deletehash'] return res def get_image(self, id): url = self.get_image_url.build(browser=self, id=id) return self.request(url) def get_gallery(self, id): url = self.get_gallery_url.build(browser=self, id=id) return self.request(url) def search_items(self, pattern, sortby): sortby = self.SORT_TYPE[sortby] url = self.search_url.build(browser=self, sort_type=sortby, page=1, pattern=pattern) info = self.request(url) if info is None: return [] return info weboob-1.2/modules/imgur/favicon.png000066400000000000000000000007051303450110500175510ustar00rootroot00000000000000PNG  IHDR@@iqbKGD pHYs  tIME;ϝHtEXtCommentCreated with GIMPW-IDATx1DMJ* i It(o`h)EQV,q}>G #Bm M`9`]apR`v # @]𙘘 T``=ʨ `9Guذn%+qT&pSg^ TC v)O2M DӋ}H#d5L`cz"e2H }ǍԴ[!?u!;QH@4d hf^ @b pjvz!?E]j

. import re from weboob.capabilities.base import StringField from weboob.capabilities.gallery import BaseGallery, BaseImage, CapGallery from weboob.capabilities.image import CapImage, Thumbnail from weboob.capabilities.paste import BasePaste, CapPaste from weboob.tools.backend import Module from weboob.tools.capabilities.paste import image_mime from weboob.tools.date import datetime from .browser import ImgurBrowser __all__ = ['ImgurModule'] class ImgPaste(BasePaste): delete_url = StringField('URL to delete the image') @classmethod def id2url(cls, id): return 'https://imgur.com/%s' % id @property def raw_url(self): # TODO get the right extension return 'https://i.imgur.com/%s.png' % self.id class Img(BaseImage): @property def thumbnail_url(self): return ImgPaste(self.id + 't').raw_url @property def raw_url(self): return 'https://i.imgur.com/%s.%s' % (self.id, self.ext) class ImgGallery(BaseGallery): def __init__(self, *args, **kwargs): super(ImgGallery, self).__init__(*args, **kwargs) self._imgs = [] @classmethod def id2url(cls, id): return 'https://imgur.com/gallery/%s' % id def iter_image(self): return self._imgs class ImgurModule(Module, CapPaste, CapGallery, CapImage): NAME = 'imgur' DESCRIPTION = u'imgur image upload service' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ImgurBrowser IMGURL = re.compile(r'https?://(?:[a-z]+\.)?imgur.com/([a-zA-Z0-9]+)(?:\.[a-z]+)?$') GALLURL = re.compile(r'https?://(?:[a-z]+\.)?imgur.com/a/([a-zA-Z0-9]+)/?$') ID = re.compile(r'[0-9a-zA-Z]+$') # CapPaste def new_paste(self, *a, **kw): return ImgPaste(*a, **kw) def can_post(self, contents, title=None, public=None, max_age=None): if public is False: return 0 elif re.search(r'[^a-zA-Z0-9=+/\s]', contents): return 0 elif max_age: return 0 else: mime = image_mime(contents, ('gif', 'jpeg', 'png', 'tiff', 'xcf', 'pdf')) return 20 * int(mime is not None) def get_paste(self, id): mtc = self.IMGURL.match(id) if mtc: id = mtc.group(1) elif not self.ID.match(id): return None paste = ImgPaste(id) bin = self.browser.open_raw(paste.raw_url).content paste.contents = bin.encode('base64') return paste def post_paste(self, paste, max_age=None): res = self.browser.post_image(b64=paste.contents, title=paste.title) paste.id = res['id'] paste.delete_url = res['delete_url'] return paste # CapGallery def _build_img(self, d, n, gallery): img = Img(d['id'], gallery=gallery, url=d['link'], index=n) img.title = d['title'] or u'' img.description = d['description'] or u'' img.ext = img.url.rsplit('.', 1)[-1] img.date = datetime.fromtimestamp(d['datetime']) img.thumbnail = Thumbnail(img.thumbnail_url) img.thumbnail.date = img.date img.nsfw = bool(d['nsfw']) img.size = d['size'] return img def _build_gallery(self, d): gallery = ImgGallery(d['id'], url=d['link']) gallery.title = d['title'] or u'' gallery.description = d['description'] or u'' gallery.date = datetime.fromtimestamp(d['datetime']) gallery.thumbnail = Thumbnail(Img(d['cover']).thumbnail_url) if 'images' in d: for n, d in enumerate(d['images']): img = self._build_img(d, n, gallery) gallery._imgs.append(img) gallery.cardinality = len(gallery._imgs) return gallery def get_gallery(self, id): mtc = self.GALLURL.match(id) if mtc: id = mtc.group(1) elif not self.ID.match(id): return None d = self.browser.get_gallery(id) if d is None: return None return self._build_gallery(d) def iter_gallery_images(self, gallery): if not len(gallery._imgs): new = self.get_gallery(gallery.id) gallery._imgs = new._imgs return gallery._imgs def search_galleries(self, pattern, sortby=CapGallery.SEARCH_RELEVANCE): d = self.browser.search_items(pattern, sortby) for sub in d: if sub['is_album']: yield self._build_gallery(sub) # CapImage def get_file(self, id): mtc = self.IMGURL.match(id) if mtc: id = mtc.group(1) elif not self.ID.match(id): return None d = self.browser.get_image(id) if d is None: return None return self._build_img(d, 0, None) def search_file(self, pattern, sortby=CapGallery.SEARCH_RELEVANCE): d = self.browser.search_items(pattern, sortby) for sub in d: if not sub['is_album']: yield self._build_img(sub, 0, None) def search_image(self, pattern, sortby=CapGallery.SEARCH_RELEVANCE, nsfw=False): for img in self.search_file(pattern, sortby): if nsfw or not img.nsfw: yield img def fill_img(self, img, fields): if 'data' in fields: img.data = self.browser.open_raw(img.url).content return img OBJECTS = {Img: fill_img, Thumbnail: fill_img} weboob-1.2/modules/imgur/test.py000066400000000000000000000030041303450110500167420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class ImgurTest(BackendTest): MODULE = 'imgur' # small gif file DATA = 'R0lGODdhAQABAPAAAP///wAAACwAAAAAAQABAAACAkQBADs=\n' def test_post(self): assert self.backend.can_post(self.DATA, max_age=0) post = self.backend.new_paste(None) post.contents = self.DATA post.public = True self.backend.post_paste(post, max_age=0) assert post.id got = self.backend.get_paste(post.id) assert got assert got.contents.decode('base64') == self.DATA.decode('base64') def test_search(self): it = iter(self.backend.search_image('lol')) img = next(it) assert img it = iter(self.backend.search_galleries('lol')) gall = next(it) assert gall weboob-1.2/modules/ina/000077500000000000000000000000001303450110500150405ustar00rootroot00000000000000weboob-1.2/modules/ina/__init__.py000066400000000000000000000000671303450110500171540ustar00rootroot00000000000000from .module import InaModule __all__ = ['InaModule'] weboob-1.2/modules/ina/browser.py000066400000000000000000000043531303450110500171020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import SearchPage, MediaPage, RssPage __all__ = ['InaBrowser'] class InaBrowser(PagesBrowser): BASEURL = 'http://www.ina.fr/' search_page = URL('layout/set/ajax/recherche/result\?q=(?P.*)&autopromote=&b=(?P.*)&type=(?P(Audio|Video))&r=', SearchPage) video_page = URL('/video/(?P.*)', MediaPage) audio_page = URL('/audio/(?P.*)', MediaPage) rss_page = URL('https://player.ina.fr/notices/(?P<_id>.*).mrss', RssPage) @video_page.id2url def get_video(self, url, video=None): if not video: self.location(url) assert self.video_page.is_here() video = self.page.get_video(obj=video) video.url = self.get_media_url(video.id) return video def get_media_url(self, _id): return self.rss_page.go(_id=_id).get_media_url() def search_videos(self, pattern): return self.search_page.go(pattern=pattern.encode('utf-8'), type='Video', first_item='0').iter_videos() def get_audio(self, id, audio=None): if not audio: audio = self.audio_page.go(id=id).get_audio(obj=audio) audio.url = self.get_media_url(id) return audio def search_audio(self, pattern): return self.search_page.go(pattern=pattern.encode('utf-8'), type='Audio', first_item='0').iter_audios() weboob-1.2/modules/ina/favicon.png000066400000000000000000000024211303450110500171720ustar00rootroot00000000000000PNG  IHDR@@iqsRGB pHYs  tIME :8XIDATx]lU}ʺnc1APSQM4zc o71ީc$LJA#&76}X:溭]۵^t-`ўssoyW;r$\   @9J:ro|M3;7`214h E%G6+88C4b:dwg?tր= P^ж<6ZNABy"7_JTd;32_&Z\yPS|(Dfo\WAKcuf*ZdD$bt,ĉӓ<=-ydZ%QƄFoլdZ0^AGP4W{&Xc7MܽKʳM4֯x8{2Qi`ew\/ѱ0387Z 0DlD6kUȲPhN[ ŤةL$X_Zp;㭢XI8MNUG%~vclz o\jdvJ2_A͞}"sլ-AMD`"GK!/,wQXa!Z/.4bhX[A"a"C$ y_Bʄ͢QF+ƥXw:xdﺇO{'.^geK:Y,Kg>8|kvܥGFB|()@@@@@G(r>IENDB`weboob-1.2/modules/ina/module.py000066400000000000000000000047071303450110500167070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.audio import CapAudio, BaseAudio, decode_id from weboob.tools.backend import Module from .browser import InaBrowser __all__ = ['InaModule'] class InaModule(Module, CapVideo, CapAudio): NAME = 'ina' MAINTAINER = u'Christophe Benz' EMAIL = 'christophe.benz@gmail.com' VERSION = '1.2' DESCRIPTION = 'INA French TV video archives' LICENSE = 'AGPLv3+' BROWSER = InaBrowser def get_video(self, _id): return self.browser.get_video(_id) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern) def fill_media(self, media, fields): if fields != ['thumbnail'] and fields != ['url']: # if we don't want only the thumbnail, we probably want also every fields if isinstance(media, BaseVideo): media = self.browser.get_video(media.id, media) else: _id = BaseAudio.decode_id(media.id) media = self.browser.get_audio(_id, media) if 'url' in fields and not media.url: _id = BaseAudio.decode_id(media.id) if isinstance(media, BaseAudio) else media.id media.url = self.browser.get_media_url(_id) if 'thumbnail' in fields and media.thumbnail: media.thumbnail.data = self.browser.open(media.thumbnail.url).content return media def search_audio(self, pattern, sortby=CapAudio.SEARCH_RELEVANCE): return self.browser.search_audio(pattern) @decode_id(BaseAudio.decode_id) def get_audio(self, _id): return self.browser.get_audio(_id) OBJECTS = {BaseVideo: fill_media, BaseAudio: fill_media} weboob-1.2/modules/ina/pages.py000066400000000000000000000131211303450110500165070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from io import StringIO import lxml.html as html from weboob.browser.pages import JsonPage, HTMLPage, XMLPage, pagination from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, Duration, Date, BrowserURL, Env from weboob.capabilities.audio import BaseAudio from weboob.capabilities.video import BaseVideo from weboob.capabilities.image import Thumbnail from weboob.capabilities.base import NotAvailable from weboob.tools.date import DATE_TRANSLATE_FR from weboob.tools.capabilities.audio.audio import BaseAudioIdFilter class InaDuration(Duration): _regexp = re.compile(r'(?P\d+)h (?P\d+)m (?P\d+)s') class InaDuration2(Duration): _regexp = re.compile(r'(?P\d+)min (?P\d+)s') kwargs = {'minutes': 'mm', 'seconds': 'ss'} class InaJsonHTMLPage(JsonPage): ENCODING = None has_next = None scroll_cursor = None def __init__(self, browser, response, *args, **kwargs): super(InaJsonHTMLPage, self).__init__(browser, response, *args, **kwargs) self.encoding = self.ENCODING or response.encoding parser = html.HTMLParser(encoding=self.encoding) self.doc = html.parse(StringIO(self.doc['content']), parser) class InaListElement(ListElement): item_xpath = '//div[@class="media"]' def next_page(self): first_item = Regexp(CleanText('//li[@class="suiv"]/a/@data-searchparams', default=None), 'b=(.*)&q=.*', default=None) if first_item(self): return BrowserURL('search_page', pattern=Env('pattern'), type=Env('type'), first_item=first_item)(self) class InaItemElement(ItemElement): obj_title = CleanText('./div[@class="media-body"]/h3/a') obj_description = CleanText('./div[@class="media-body"]/div/p[@class="media-body__summary"]') def obj_duration(self): duration = InaDuration(CleanText('./div[@class="media-body"]/div/span[@class="duration"]'), default=None)(self) if duration is None: duration = InaDuration2(CleanText('./div[@class="media-body"]/div/span[@class="duration"]'), default=NotAvailable)(self) return duration obj_author = u'Institut National de l’Audiovisuel' obj_date = Date(CleanText('./div[@class="media-body"]/div/span[@class="broadcast"]')) def obj_thumbnail(self): url = CleanText('./a/img/@src')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail class InaMediaElement(ItemElement): obj_title = CleanText('//meta[@property="og:title"]/@content') obj_description = CleanText('//div[@class="notice__description"]') def obj_duration(self): duration = InaDuration(CleanText('(//div[@class="block-infos"])[1]/span[@class="duration"]'), default=None)(self) if duration is None: duration = InaDuration2(CleanText('(//div[@class="block-infos"])[1]/span[@class="duration"]'), default=NotAvailable)(self) return duration obj_date = Date(CleanText('(//div[@class="block-infos"])[1]/span[@class="broadcast"]'), translations=DATE_TRANSLATE_FR) obj_author = u'Institut National de l’Audiovisuel' def obj_thumbnail(self): url = CleanText('//meta[@property="og:image"]/@content')(self) thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail class SearchPage(InaJsonHTMLPage): @pagination @method class iter_audios(InaListElement): class item(InaItemElement): klass = BaseAudio def condition(self): return Regexp(CleanText('./a/@href'), '/audio/(.*)/.*.html', default=None)(self) obj_id = BaseAudioIdFilter(Regexp(CleanText('./a/@href'), '/audio/(.*)/.*.html')) @pagination @method class iter_videos(InaListElement): class item(InaItemElement): klass = BaseVideo def condition(self): return Regexp(CleanText('./a/@href'), '/video/(.*)/.*.html', default=None)(self) obj_id = Regexp(CleanText('./a/@href'), '/video/(.*)/.*.html') class MediaPage(HTMLPage): @method class get_video(InaMediaElement): klass = BaseVideo obj_ext = u'mp4' obj_id = Env('id') @method class get_audio(InaMediaElement): klass = BaseAudio obj_ext = u'mp3' obj_id = BaseAudioIdFilter(Env('id')) class RssPage(XMLPage): def get_media_url(self): url = self.doc.xpath('//media:content', namespaces={'media': 'http://search.yahoo.com/mrss/'}) return CleanText('./@url')(url[0]) weboob-1.2/modules/ina/test.py000066400000000000000000000027261303450110500164000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import itertools from weboob.tools.test import BackendTest class INATest(BackendTest): MODULE = 'ina' def test_video_ina(self): l = list(itertools.islice(self.backend.search_videos('chirac'), 0, 20)) self.assertTrue(len(l) > 0) v_id = l[0].id v = self.backend.get_video(v_id) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_audio_ina(self): l = list(itertools.islice(self.backend.search_audio('chirac'), 0, 20)) self.assertTrue(len(l) > 0) a_id = l[0].id a = self.backend.get_audio(a_id) self.assertTrue(a.url and a.url.startswith('http://'), 'URL for video "%s" not found: %s' % (a.id, a.url)) weboob-1.2/modules/indeed/000077500000000000000000000000001303450110500155215ustar00rootroot00000000000000weboob-1.2/modules/indeed/__init__.py000066400000000000000000000014321303450110500176320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .module import IndeedModule __all__ = ['IndeedModule'] weboob-1.2/modules/indeed/browser.py000066400000000000000000000034471303450110500175660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser import PagesBrowser, URL from .pages import SearchPage, AdvertPage __all__ = ['IndeedBrowser'] class IndeedBrowser(PagesBrowser): BASEURL = 'https://www.indeed.fr' search_page = URL('/emplois(?P.*)', SearchPage) advert_page = URL('/cmp/(?P.*)/jobs/(?P.*)-(?P<nb>.*)', AdvertPage) def search_job(self, metier='', contrat='', limit_date='', radius='', place=''): params = '?q=%s&limit=10&sort=date&st=employer&sr=directhire&jt=%s&fromage=%s&radius=%s'\ % (metier.replace(' ', '+'), contrat, limit_date, radius) if place: params = '%s&l=%s' % (params, place) self.search_page.go(parameters=params) assert self.search_page.is_here(parameters=params) return self.page.iter_job_adverts() def get_job_advert(self, _id, advert): splitted_id = _id.split('#') return self.advert_page.go(nb=splitted_id[0], title=splitted_id[1], company=splitted_id[2]).get_job_advert(obj=advert) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/indeed/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000007204�13034501105�0017657�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs����+���tIME  5Qi��IDATx[k\U9gfz/B(-jx(  4%H B0 FUc|$5H(E V0V(ҒҴxP^ޙ9>{"ā9ko< �Dn>R|('Rރ B$ Vא6՟}vQE=r,e n�DU]^,ˈ).?t7֣,9k~w/ݬzlnJjѦ[ɦnҹiji_Up;}:1Hfũ#(1۔UEB}T]<&~#i_}2ijĕE4Cm5mlsZ�+1j�qջd9T@8UW'HW"t,7M#Xx96OL4z]`E c,9&ƙ'uE@5PKR+Ő l-|`"f kXfn@M% .=$M6RJUVUNn5})͊�?co"\`~+!4xf1/$zM#hNc%d"k?/;s;m/qSb!1>S{0Ĕ6:@"A KaXWL`R:0?0$dfO@!oF$X ?#8eqK"s n\6ŇFo4{%yfO4I9"% D'W9]I=|xQP;"zsWFib}<k(ڲdReЭ՛ON p?{gŁӻf{g8ȸe&G8.�ḬT+\CTy,Ề|i) F[�%hJ9PRNq`i} O`5QzCP6o22#Ш8 e],hrrJfhDQT,5ScQ}ccrHZ8R*weud,oc%3O3dOkh/"<<A[wlxٝ]cc=mw3"iH<ξcEw8ls`<1U_ 諓�|uDX}okxykS\%fa� Ql"'�y]QR&p>~,ǟ=Cpݚ!m^G,Z r$-=EHOH"{~ ́OAlF�in=N#PQA,7ͰG3mnGZNS/筜A#$~Wzl @wp!"w6d@ɒr"klJ j.9n"M:5OxL6+$V>kʬ"ˁKf⍲`Y~4%n}$O\Sf%XA@eq2tb:XdH'm)e*~mP +8M鲎JL)X<uG0g�'yʴU}\6�M%;�(KS܉j&Dr>1 (m+z%ľ�lqҗ]|hax3ҷ/'LGE%xշG?LlU"g ؀ӆ>$A 4$)q Cd\bpqe/Q̳#c;ֆϭG0#V\ߗߔ% fM_] ,լX<oX-LPio'5v1L cL G?+Q9 ~V akc`H|ǿ+.4'&{/.# 4GȲ n|-#A ~oL`ބGOI֍XQ%yaNڱ�WE?U#xb1DOp;#,}o<W 4K?9^&MpʢǾ#F?%ppŕȎwV1e+J 99,*B-g9"kDOp 'a= B" bu&34n!P?"g.&K$)N[3~`S|gϋFfQkv7M YK~�`]M zkԳf@fE:qQ.c)15vI`"t-ơ9( 1:I c"Tˡ|3K[,5ƄHzڄI=(.J%P).f܍s@[#P Ў>qP[a o|ንv'geձ#@{NEd9a`M'(/<*bF<OG\)j/3?͊ˊ-L.BWobጥS؊f]Y dSRYh-t&1KX ](::65v:@g#DٓCē-_m[㲕d z)wMڄJc؟Jѱe-޺⨓TRcb.a25Am*O z_ԸY*Hc�(e%Z<;=`Nw,<V[HZc=!rpP�<Bd}vK0-V9ޤ/nJLhLP,qjQ+3xxU)[bÚyRTZ8G/5‘ۇ 3 ; SByp&Eg�!r-ٲ#`g8amKP[hPQP_7}GZ�D ي 9A`7,S*p-r s3zk3UzdθO'i G`Rӫ]b< H|rH!2b4c%N"E` Cĉd9pґuPnJhGNt'KoĞgd(1 )c"8CQ 1\ u!+Q[aMJEkp 1 p-<DaN>0f] <fS~S[Yq$5!J87l>=Ŷ4G�'NS촇EX&ioiM[%<\+=8cHZ <cj:,fmp@slgVaqH&~DED 7�z#k[t&@9L8lKMeh3^Pbv4N^qaQj+^jETJ-54 H#D,e)OR8K}-΀n$%V]~ 6՝:SYb JP [)-~@����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/indeed/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000006751�13034501105�0017371�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.job import CapJob, BaseJobAdvert from weboob.tools.value import Value from .browser import IndeedBrowser __all__ = ['IndeedModule'] class IndeedModule(Module, CapJob): NAME = 'indeed' DESCRIPTION = u'indeed website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = IndeedBrowser type_contrat_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ 'all': u'Tous les emplois', 'fulltime': u'Temps plein', 'parttime': u'Temps partiel', 'contract': u'Durée indéterminée', 'internship': u'Stage / Apprentissage', 'temporary': u'Durée déterminée', }.iteritems())]) limit_date_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ 'any': u'à tout moment', '15': u'depuis 15 jours', '7': u'depuis 7 jours', '3': u'depuis 3 jours', '1': u'depuis hier', 'last': u'depuis ma dernière visite', }.iteritems())]) radius_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '0': u'uniquement à cet endroit', '5': u'dans un rayon de 5 kilomètres', '10': u'dans un rayon de 10 kilomètres', '15': u'dans un rayon de 15 kilomètres', '25': u'dans un rayon de 25 kilomètres', '50': u'dans un rayon de 50 kilomètres', '100': u'dans un rayon de 100 kilomètres', }.iteritems())]) CONFIG = BackendConfig(Value('metier', label=u'Job name', masked=False, default=''), Value('limit_date', label=u'Date limite', choices=limit_date_choices, default=''), Value('contrat', label=u'Contract', choices=type_contrat_choices, default=''), Value('place', label=u'Place', masked=False, default=''), Value('radius', label=u'Radius', choices=radius_choices, default='')) def search_job(self, pattern=None): return self.browser.search_job(metier=pattern) def advanced_search_job(self): return self.browser.search_job(metier=self.config['metier'].get(), limit_date=self.config['limit_date'].get(), contrat=self.config['contrat'].get(), place=self.config['place'].get(), radius=self.config['radius'].get()) def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} �����������������������weboob-1.2/modules/indeed/pages.py������������������������������������������������������������������0000664�0000000�0000000�00000006633�13034501105�0017202�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import timedelta, datetime import re from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import Filter, CleanText, Regexp, Format, Env from weboob.browser.filters.html import CleanHTML, Attr from weboob.capabilities.job import BaseJobAdvert class IndeedDate(Filter): def filter(self, date): now = datetime.now() number = re.search("\d+", date) if number: if 'heures' in date: return now - timedelta(hours=int(number.group(0))) elif 'jour' in date: return now - timedelta(days=int(number.group(0))) return now class SearchPage(HTMLPage): @pagination @method class iter_job_adverts(ListElement): item_xpath = '//div[@itemtype="http://schema.org/JobPosting"]' def next_page(self): for a in self.page.doc.xpath('//a'): if a.xpath('span[@class="pn"]/span[@class="np"]') and "Suivant" in a.xpath('span[@class="pn"]/span[@class="np"]')[0].text: return a.attrib['href'] class Item(ItemElement): klass = BaseJobAdvert obj_id = CleanText(Format('%s#%s#%s', Regexp(Attr('.', 'id'), '^..(.*)'), Attr('h2/a', 'title'), CleanText('span[@class="company"]')), replace=[(" ", "-"), ("/", "-")]) obj_title = Attr('h2/a', 'title') obj_society_name = CleanText('span[@class="company"]') obj_place = CleanText('span/span[@class="location"]') obj_publication_date = IndeedDate(CleanText('table/tr/td/span[@class="date"]')) class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert def parse(self, el): self.env['url'] = self.page.url self.env['num_id'] = self.page.url.split('-')[-1] obj_id = Format('%s#%s#%s', Env('num_id'), CleanText('//div[@id="job_header"]/b[@class="jobtitle"]'), CleanText('//div[@id="job_header"]/span[@class="company"]'), ) obj_title = CleanText('//div[@id="job_header"]/b[@class="jobtitle"]') obj_place = CleanText('//div[@id="job_header"]/span[@class="location"]') obj_description = CleanHTML('//span[@class="summary"]') obj_job_name = CleanText('//div[@id="job_header"]/b[@class="jobtitle"]') obj_url = Env('url') obj_publication_date = IndeedDate(CleanText('//span[@class="date"]')) �����������������������������������������������������������������������������������������������������weboob-1.2/modules/indeed/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000003177�13034501105�0017062�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class IndeedTest(BackendTest): MODULE = 'indeed' def test_indeed_search(self): l = list(self.backend.search_job('informaticien')) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_indeed_advanced_search(self): l = list(self.backend.advanced_search_job()) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_indeep_info_from_id(self): l = list(self.backend.advanced_search_job()) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015046�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001451�13034501105�0017160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import INGModule __all__ = ['INGModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000034066�13034501105�0017114�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re import hashlib import time from requests.exceptions import SSLError from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword, ParseError from weboob.browser.exceptions import ServerError from weboob.capabilities.bank import Account, TransferError, AccountNotFound from weboob.capabilities.base import find_object from .pages import AccountsList, LoginPage, NetissimaPage, TitrePage, TitreHistory,\ TransferPage, TransferConfirmPage, BillsPage, StopPage, TitreDetails, TitreValuePage, ASVHistory,\ ASVInvest, DetailFondsPage, IbanPage, ActionNeededPage, ReturnPage __all__ = ['IngBrowser'] def start_with_main_site(f): def wrapper(*args, **kwargs): browser = args[0] if browser.url.startswith('https://bourse.ingdirect.fr/'): for i in xrange(3): try: browser.location('https://bourse.ingdirect.fr/priv/redirectIng.php?pageIng=COMPTE') except ServerError: pass else: break browser.where = 'start' elif browser.url.startswith('https://ingdirectvie.ingdirect.fr/'): browser.lifeback.go() browser.where = 'start' return f(*args, **kwargs) return wrapper class IngBrowser(LoginBrowser): BASEURL = 'https://secure.ingdirect.fr' VERIFY = 'certificate.pem' TIMEOUT = 60.0 # avoid relogin every time lifeback = URL(r'https://ingdirectvie.ingdirect.fr/b2b2c/entreesite/EntAccExit', ReturnPage) # Login and error loginpage = URL('/public/displayLogin.jsf.*', LoginPage) errorpage = URL('.*displayCoordonneesCommand.*', StopPage) actioneeded = URL('/general\?command=displayTRAlertMessage', ActionNeededPage) # CapBank accountspage = URL('/protected/pages/index.jsf', '/protected/pages/asv/contract/(?P<asvpage>.*).jsf', AccountsList) transferpage = URL('/protected/pages/cc/transfer/transferManagement.jsf', TransferPage) dotransferpage = URL('/general\?command=DisplayDoTransferCommand', TransferPage) valtransferpage = URL('/protected/pages/cc/transfer/create/transferCreateValidation.jsf', TransferConfirmPage) titredetails = URL('/general\?command=display.*', TitreDetails) ibanpage = URL('/protected/pages/common/rib/initialRib.jsf', IbanPage) # CapBank-Market netissima = URL('/data/asv/fiches-fonds/fonds-netissima.html', NetissimaPage) starttitre = URL('/general\?command=goToAccount&zone=COMPTE', TitrePage) titrepage = URL('https://bourse.ingdirect.fr/priv/portefeuille-TR.php', TitrePage) titrehistory = URL('https://bourse.ingdirect.fr/priv/compte.php\?ong=3', TitreHistory) titrerealtime = URL('https://bourse.ingdirect.fr/streaming/compteTempsReelCK.php', TitrePage) titrevalue = URL('https://bourse.ingdirect.fr/priv/fiche-valeur.php\?val=(?P<val>.*)&pl=(?P<pl>.*)&popup=1', TitreValuePage) asv_history = URL('https://ingdirectvie.ingdirect.fr/b2b2c/epargne/CoeLisMvt', 'https://ingdirectvie.ingdirect.fr/b2b2c/epargne/CoeDetMvt', ASVHistory) asv_invest = URL('https://ingdirectvie.ingdirect.fr/b2b2c/epargne/CoeDetCon', ASVInvest) detailfonds = URL('https://ingdirectvie.ingdirect.fr/b2b2c/fonds/PerDesFac\?codeFonds=(.*)', DetailFondsPage) # CapDocument billpage = URL('/protected/pages/common/estatement/eStatement.jsf', BillsPage) __states__ = ['where'] def __init__(self, *args, **kwargs): self.birthday = re.sub(r'[^\d]', '', kwargs.pop('birthday')) self.where = None LoginBrowser.__init__(self, *args, **kwargs) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert isinstance(self.birthday, basestring) assert self.password.isdigit() assert self.birthday.isdigit() self.do_logout() self.loginpage.go() self.page.prelogin(self.username, self.birthday) self.page.login(self.password) if self.page.error(): raise BrowserIncorrectPassword() if self.errorpage.is_here(): raise BrowserIncorrectPassword('Please login on website to fill the form and retry') self.page.check_for_action_needed() @need_login @start_with_main_site def get_accounts_list(self, get_iban=True): self.accountspage.go() self.where = "start" for acc in self.page.get_list(): if get_iban and acc.type in [Account.TYPE_CHECKING, Account.TYPE_SAVINGS]: self.go_account_page(acc) acc.iban = self.ibanpage.go().get_iban() yield acc def get_account(self, _id): return find_object(self.get_accounts_list(get_iban=False), id=_id, error=AccountNotFound) def go_account_page(self, account): data = {"AJAX:EVENTS_COUNT": 1, "AJAXREQUEST": "_viewRoot", "ajaxSingle": "index:setAccount", "autoScroll": "", "index": "index", "index:setAccount": "index:setAccount", "javax.faces.ViewState": account._jid, "cptnbr": account._id } self.accountspage.go(data=data) self.where = "history" @need_login @start_with_main_site def get_coming(self, account): if account.type != Account.TYPE_CHECKING and\ account.type != Account.TYPE_SAVINGS: raise NotImplementedError() account = self.get_account(account.id) self.go_account_page(account) jid = self.page.get_history_jid() if jid is None: self.logger.info('There is no history for this account') return return self.page.get_coming() @need_login @start_with_main_site def get_history(self, account): if account.type == Account.TYPE_MARKET or account.type == Account.TYPE_LIFE_INSURANCE: for result in self.get_history_titre(account): yield result return elif account.type != Account.TYPE_CHECKING and\ account.type != Account.TYPE_SAVINGS: raise NotImplementedError() account = self.get_account(account.id) self.go_account_page(account) jid = self.page.get_history_jid() if jid is None: self.logger.info('There is no history for this account') return if account.type == Account.TYPE_CHECKING: history_function = AccountsList.get_transactions_cc index = -1 # disable the index. It works without it on CC else: history_function = AccountsList.get_transactions_others index = 0 hashlist = [] while True: i = index for transaction in history_function(self.page, index=index): transaction.id = hashlib.md5(transaction._hash).hexdigest() while transaction.id in hashlist: transaction.id = hashlib.md5(transaction.id + "1").hexdigest() hashlist.append(transaction.id) i += 1 yield transaction # if there is no more transactions, it is useless to continue if self.page.islast() or i == index: return if index >= 0: index = i data = {"AJAX:EVENTS_COUNT": 1, "AJAXREQUEST": "_viewRoot", "autoScroll": "", "index": "index", "index:%s:moreTransactions" % jid: "index:%s:moreTransactions" % jid, "javax.faces.ViewState": account._jid } self.accountspage.go(data=data) @need_login @start_with_main_site def get_recipients(self, account): self.transferpage.stay_or_go() if self.page.ischecked(account.id): return self.page.get_recipients() else: # It is hard to check the box and to get the real list. # We try an alternative way like normal users self.get_history(account).next() self.transferpage.stay_or_go() return self.page.get_recipients() @start_with_main_site def transfer(self, account, recipient, amount, reason): found = False # Automatically get the good transfer page self.logger.debug('Search %s' % recipient) for destination in self.get_recipients(account): self.logger.debug('Found %s ' % destination.id) if destination.id == recipient: found = True recipient = destination break if found: self.transferpage.open(data=self.page.buildonclick(recipient, account)) self.page.transfer(recipient, amount, reason) self.valtransferpage.go() if not self.valtransferpage.is_here(): raise TransferError("Invalid transfer (no confirmation page)") else: self.page.confirm(self.password) self.valtransferpage.go() recap = self.page.recap() if len(list(recap)) == 0: raise ParseError('Unable to find confirmation') return self.page.recap() else: raise TransferError('Recipient not found') def go_on_asv_detail(self, account, link): try: if self.page.asv_is_other: jid = self.page.get_asv_jid() data = {'index': "index", 'javax.faces.ViewState': jid, 'index:j_idcl': "index:asvInclude:goToAsvPartner"} self.accountspage.go(data=data) else: self.accountspage.go(asvpage="manageASVContract") self.page.submit() self.page.submit() self.location(link) return True except SSLError: return False def go_investments(self, account): account = self.get_account(account.id) data = {"AJAX:EVENTS_COUNT": 1, "AJAXREQUEST": "_viewRoot", "ajaxSingle": "index:setAccount", "autoScroll": "", "index": "index", "index:setAccount": "index:setAccount", "javax.faces.ViewState": account._jid, "cptnbr": account._id } # On ASV pages, data maybe not available. for i in range(5): if i > 0: self.logger.debug('Investments list empty, retrying in %s seconds...', (2**i)) time.sleep(2**i) if i > 1: self.do_logout() self.do_login() account = self.get_account(account.id) data['cptnbr'] = account._id data['javax.faces.ViewState'] = account._jid self.accountspage.go(data=data) if not self.page.has_error(): break else: self.logger.warning("Unable to get investments list...") if self.page.is_asv: return self.starttitre.go() self.where = u"titre" self.titrepage.go() @need_login @start_with_main_site def get_investments(self, account): if account.type not in (Account.TYPE_MARKET, Account.TYPE_LIFE_INSURANCE): raise NotImplementedError() self.go_investments(account) if self.where == u'titre': self.titrerealtime.go() elif self.page.asv_has_detail or account._jid: if self.go_on_asv_detail(account, '/b2b2c/epargne/CoeDetCon') is False: return iter([]) self.where = u"asv" return self.page.iter_investments() def get_history_titre(self, account): self.go_investments(account) if self.where == u'titre': self.titrehistory.go() elif self.page.asv_has_detail or account._jid: if self.go_on_asv_detail(account, '/b2b2c/epargne/CoeLisMvt') is False: return iter([]) else: return iter([]) transactions = list() for tr in self.page.iter_history(): transactions.append(tr) if self.asv_history.is_here(): for tr in transactions: page = tr._detail.result().page if tr._detail else None tr.investments = list(page.get_investments()) if page and 'numMvt' in page.url else [] self.lifeback.go() return iter(transactions) ############# CapDocument ############# @start_with_main_site @need_login def get_subscriptions(self): self.billpage.go() if self.loginpage.is_here(): self.do_login() return self.billpage.go().iter_account() else: return self.page.iter_account() @need_login def get_documents(self, subscription): self.billpage.go() data = {"AJAXREQUEST": "_viewRoot", "accountsel_form": "accountsel_form", subscription._formid: subscription._formid, "autoScroll": "", "javax.faces.ViewState": subscription._javax, "transfer_issuer_radio": subscription.id } self.billpage.go(data=data) return self.page.iter_documents(subid=subscription.id) def predownload(self, bill): self.page.postpredown(bill._localid) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/certificate.pem��������������������������������������������������������������0000664�0000000�0000000�00000015475�13034501105�0020047�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������-----BEGIN CERTIFICATE----- MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0 aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjEL MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2ln biwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJp U2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y aXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1 nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbex t0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIz SdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQG BO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+ rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/ NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8E BAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAH BgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVy aXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKv MzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzE p6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y 5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlK WE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ 4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8N hnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIEPjCCAyagAwIBAgIESlOMKDANBgkqhkiG9w0BAQsFADCBvjELMAkGA1UEBhMC VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50 cnVzdC5uZXQvbGVnYWwtdGVybXMxOTA3BgNVBAsTMChjKSAyMDA5IEVudHJ1c3Qs IEluYy4gLSBmb3IgYXV0aG9yaXplZCB1c2Ugb25seTEyMDAGA1UEAxMpRW50cnVz dCBSb290IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzIwHhcNMDkwNzA3MTcy NTU0WhcNMzAxMjA3MTc1NTU0WjCBvjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUVu dHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50cnVzdC5uZXQvbGVnYWwt dGVybXMxOTA3BgNVBAsTMChjKSAyMDA5IEVudHJ1c3QsIEluYy4gLSBmb3IgYXV0 aG9yaXplZCB1c2Ugb25seTEyMDAGA1UEAxMpRW50cnVzdCBSb290IENlcnRpZmlj YXRpb24gQXV0aG9yaXR5IC0gRzIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQC6hLZy254Ma+KZ6TABp3bqMriVQRrJ2mFOWHLP/vaCeb9zYQYKpSfYs1/T RU4cctZOMvJyig/3gxnQaoCAAEUesMfnmr8SVycco2gvCoe9amsOXmXzHHfV1IWN cCG0szLni6LVhjkCsbjSR87kyUnEO6fe+1R9V77w6G7CebI6C1XiUJgWMhNcL3hW wcKUs/Ja5CeanyTXxuzQmyWC48zCxEXFjJd6BmsqEZ+pCm5IO2/b1BEZQvePB7/1 U1+cPvQXLOZprE4yTGJ36rfo5bs0vBmLrpxR57d+tVOxMyLlbc9wPBr64ptntoP0 jaWvYkxN4FisZDQSA/i2jZRjJKRxAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAP BgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRqciZ60B7vfec7aVHUbI2fkBJmqzAN BgkqhkiG9w0BAQsFAAOCAQEAeZ8dlsa2eT8ijYfThwMEYGprmi5ZiXMRrEPR9RP/ jTkrwPK9T3CMqS/qF8QLVJ7UG5aYMzyorWKiAHarWWluBh1+xLlEjZivEtRh2woZ Rkfz6/djwUAFQKXSt/S1mja/qYh2iARVBCuch38aNzx+LaUa2NSJXsq9rD1s2G2v 1fN2D807iDginWyTmsQ9v4IbZT+mD12q/OWyFcq1rca8PdCE6OoGcrBNOTJ4vz4R nAuknZoh8/CbCzB428Hch0P+vGOaysXCHMnHjf87ElgI5rY97HosTvuDls4MPGmH VHOkc8KT/1EQrBVUAdj8BbGJoX90g5pJ19xOe4pIb4tF9g== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIFLTCCBBWgAwIBAgIMYaHn0gAAAABR02amMA0GCSqGSIb3DQEBCwUAMIG+MQsw CQYDVQQGEwJVUzEWMBQGA1UEChMNRW50cnVzdCwgSW5jLjEoMCYGA1UECxMfU2Vl IHd3dy5lbnRydXN0Lm5ldC9sZWdhbC10ZXJtczE5MDcGA1UECxMwKGMpIDIwMDkg RW50cnVzdCwgSW5jLiAtIGZvciBhdXRob3JpemVkIHVzZSBvbmx5MTIwMAYDVQQD EylFbnRydXN0IFJvb3QgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHMjAeFw0x NDEyMTUxNTI1MDNaFw0zMDEwMTUxNTU1MDNaMIG6MQswCQYDVQQGEwJVUzEWMBQG A1UEChMNRW50cnVzdCwgSW5jLjEoMCYGA1UECxMfU2VlIHd3dy5lbnRydXN0Lm5l dC9sZWdhbC10ZXJtczE5MDcGA1UECxMwKGMpIDIwMTQgRW50cnVzdCwgSW5jLiAt IGZvciBhdXRob3JpemVkIHVzZSBvbmx5MS4wLAYDVQQDEyVFbnRydXN0IENlcnRp ZmljYXRpb24gQXV0aG9yaXR5IC0gTDFNMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A MIIBCgKCAQEA0IHBOSPCsdHs91fdVSQ2kSAiSPf8ylIKsKs/M7WwhAf23056sPuY Ij0BrFb7cW2y7rmgD1J3q5iTvjOK64dex6qwymmPQwhqPyK/MzlG1ZTy4kwFItln gJHxBEoOm3yiydJs/TwJhL39axSagR3nioPvYRZ1R5gTOw2QFpi/iuInMlOZmcP7 lhw192LtjL1JcdJDQ6Gh4yEqI3CodT2ybEYGYW8YZ+QpfrI8wcVfCR5uRE7sIZlY FUj0VUgqtzS0BeN8SYwAWN46lsw53GEzVc4qLj/RmWLoquY0djGqr3kplnjLgRSv adr7BLlZg0SqCU+01CwBnZuUMWstoc/B5QIDAQABo4IBKzCCAScwDgYDVR0PAQH/ BAQDAgEGMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATASBgNVHRMBAf8E CDAGAQH/AgEAMDMGCCsGAQUFBwEBBCcwJTAjBggrBgEFBQcwAYYXaHR0cDovL29j c3AuZW50cnVzdC5uZXQwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL2NybC5lbnRy dXN0Lm5ldC9nMmNhLmNybDA7BgNVHSAENDAyMDAGBFUdIAAwKDAmBggrBgEFBQcC ARYaaHR0cDovL3d3dy5lbnRydXN0Lm5ldC9ycGEwHQYDVR0OBBYEFMP30LUqMK2v DZEhcDlU3byJcMc6MB8GA1UdIwQYMBaAFGpyJnrQHu995ztpUdRsjZ+QEmarMA0G CSqGSIb3DQEBCwUAA4IBAQC0h8eEIhopwKR47PVPG7SEl2937tTPWa+oQ5YvHVje pvMVWy7ZQ5xMQrkXFxGttLFBx2YMIoYFp7Qi+8VoaIqIMthx1hGOjlJ+Qgld2dnA DizvRGsf2yS89byxqsGK5Wbb0CTz34mmi/5e0FC6m3UAyQhKS3Q/WFOv9rihbISY Jnz8/DVRZZgeO2x28JkPxLkJ1YXYJKd/KsLak0tkuHB8VCnTglTVz6WUwzOeTTRn 4Dh2ZgCN0C/GqwmqcvrOLzWJ/MDtBgO334wlV/H77yiI2YIowAQPlIFpI+CRKMVe 1QzX1CA778n4wI+nQc1XRG5sZ2L+hN/nYNjvv9QiHg3n -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIIFKzCCBBOgAwIBAgIQfuFKb2/v8tN/P61lTTratDANBgkqhkiG9w0BAQsFADCB yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0 aG9yaXR5IC0gRzUwHhcNMTMxMDMxMDAwMDAwWhcNMjMxMDMwMjM1OTU5WjB3MQsw CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNV BAsTFlN5bWFudGVjIFRydXN0IE5ldHdvcmsxKDAmBgNVBAMTH1N5bWFudGVjIENs YXNzIDMgRVYgU1NMIENBIC0gRzMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK AoIBAQDYoWV0I+grZOIy1zM3PY71NBZI3U9/hxz4RCMTjvsR2ERaGHGOYBYmkpv9 FwvhcXBC/r/6HMCqo6e1cej/GIP23xAKE2LIPZyn3i4/DNkd5y77Ks7Imn+Hv9hM BBUyydHMlXGgTihPhNk1++OGb5RT5nKKY2cuvmn2926OnGAE6yn6xEdC0niY4+wL pZLct5q9gGQrOHw4CVtm9i2VeoayNC6FnpAOX7ddpFFyRnATv2fytqdNFB5suVPu IxpOjUhVQ0GxiXVqQCjFfd3SbtICGS97JJRL6/EaqZvjI5rq+jOrCiy39GAI3Z8c zd0tAWaAr7MvKR0juIrhoXAHDDQPAgMBAAGjggFdMIIBWTAvBggrBgEFBQcBAQQj MCEwHwYIKwYBBQUHMAGGE2h0dHA6Ly9zMi5zeW1jYi5jb20wEgYDVR0TAQH/BAgw BgEB/wIBADBlBgNVHSAEXjBcMFoGBFUdIAAwUjAmBggrBgEFBQcCARYaaHR0cDov L3d3dy5zeW1hdXRoLmNvbS9jcHMwKAYIKwYBBQUHAgIwHBoaaHR0cDovL3d3dy5z eW1hdXRoLmNvbS9ycGEwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL3MxLnN5bWNi LmNvbS9wY2EzLWc1LmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwx GjAYBgNVBAMTEVN5bWFudGVjUEtJLTEtNTMzMB0GA1UdDgQWBBQBWavn3ToLWaZk Y9bPIAdX1ZHnajAfBgNVHSMEGDAWgBR/02Wnwt3su/AwCfNDOfoCrzMxMzANBgkq hkiG9w0BAQsFAAOCAQEAQgFVe9AWGl1Y6LubqE3X89frE5SG1n8hC0e8V5uSXU8F nzikEHzPg74GQ0aNCLxq1xCm+quvL2GoY/Jl339MiBKIT7Np2f8nwAqXkY9W+4nE qLuSLRtzsMarNvSWbCAI7woeZiRFT2cAQMgHVHQzO6atuyOfZu2iRHA0+w7qAf3P eHTfp61Vt19N9tY/4IbOJMdCqRMURDVLtt/JYKwMf9mTIUvunORJApjTYHtcvNUw LwfORELEC5n+5p/8sHiGUW3RLJ3GlvuFgrsEL/digO9i2n/2DqyQuFa9eT/ygG6j 2bkPXToHHZGThkspTOHcteHgM52zyzaRS/6htO7w+Q== -----END CERTIFICATE----- ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000005527�13034501105�0017212�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD��f��}a��� pHYs�� �� ����tIME 8% ���tEXtComment�Created with GIMPW�� IDATxytT?o&l$1bdU  GB]V=j9s X<.X[j[VJ](VRbH"ǝy! s-y.{|L:$:8&ً_ gN Ĥ8׳3@R)} N2xjK> U P%惜dtnOlpS13@M2޲J {6䎁Ϡc3&98x`êΟ^�zS-H�yœ烒 >s0gF@|m/`ꎙ<pWOӧ!s  )ϼvR�gL�_b0|D <*YTp]8A[ '?,#^3س5q.sUQ{VΆegú3U"_r/ ! CV>>Yrab*.=>eRoVd:&*} ݠ7NBbי=r\WItkY| 5^Ua|J/ Uc N>˩;q@u/Eq fQj|$�u%y0Z[JYyG/�}cљ=d"Ѡ> H :gڤ@Bjyݸq'цŹcqS=TpiG`8,=fjXƭMUףHk`h[ 4Ն%-޲&̕qXka8W8Nmo UѡtZzoX?7֊wJ51 mg}fFx-!8=ֿr O6:5.p :E_s V82�XlS*APzHT;્gV;% 7vUZ<SbsW 3hX'xx ;.2BNȑݤI<U'�SC̀ƚ cS i|F(p$ V0,ȉ=1%B0 ;? h,YX 3ni=]&;3 H6`laa{4TǠ�f.I@Z?=6hi�Cfu$鍗71D_=AdH#YTIKjmhB[*3(`(T{}%Cr|fݥc ljUfR�bLn+w9CeM-xjAקX+YD18<ѱF09 a3A햟ye[(eޏ'\ַ@_|6n0*\.�ڗA3PfI@<\%m  %Q3>P=0q7iTvUQ} d|4Lןc5dvȴN0('wetU8mSr\ k'Kj u2+pÓ*ax]`x$x9 pXYÀpo߲)\`5+b wkZ0SqOfbWGUN`K]aHGz ,䘦C�tXjq_6,{?#{,:K'KĢ N?'+ Ϛ.ҝC5'5._.TehHrns 6[b77IE?Lz z<{tsy<+6#5PsP Dr{=1>AZ \W`֢?RId^[kJI޳6 fEt_JH6+եpKp8B5DY;+do?Òj64|'BNˆRԝa!#U~gFNwwYylXשk 7"X ^S=l}^VT͓V[;VTVXil1fP./&5K{bBn=Ҭ&�a*IS}PAnĤ,NFMg%WBdZ8[beVERsڛ1BLÀdhWZ+r eooxBB4!gz":$=u2X6P/,G/U(1 8ݪ3qGVz[[j ́{KԪxv@ Jl;٤YR +wbk.ǁ�}:4*Mubt|7BmJ_8L og]ӔY,=νNFW)1!]>zsZ٧%ǫ7E 6O / xWw&~.]Soiw:\n-`=PeZ8:Y=IǷzW8_O`TWܭ ;GvoW'Ʋ[(h՜Jp*�0\/7}0By>ghrEKc>'WܯLS=lg˟=d2tom|V #n-23I8RyH4q{?/sꓗ t۝)o_e5ar/ 7bpI P ����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000012103�13034501105�0016702�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank, Account, Recipient from weboob.capabilities.bill import CapDocument, Bill, Subscription,\ SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import UserError, find_object, NotAvailable from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.browser.exceptions import ServerError from .browser import IngBrowser __all__ = ['INGModule'] class INGModule(Module, CapBank, CapDocument): NAME = 'ing' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'ING Direct' CONFIG = BackendConfig(ValueBackendPassword('login', label=u'Numéro client', masked=False), ValueBackendPassword('password', label='Code secret', regexp='^(\d{6}|)$'), ValueBackendPassword('birthday', label='Date de naissance', regexp='^(\d{2}[/-]?\d{2}[/-]?\d{4}|)$', masked=False) ) BROWSER = IngBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get(), birthday=self.config['birthday'].get()) def iter_resources(self, objs, split_path): if Account in objs: self._restrict_level(split_path) return self.iter_accounts() if Subscription in objs: self._restrict_level(split_path) return self.iter_subscription() def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return self.browser.get_account(_id) def iter_history(self, account): if not isinstance(account, Account): account = self.get_account(account) return self.browser.get_history(account) def iter_transfer_recipients(self, account): if not isinstance(account, Account): account = self.get_account(account) return self.browser.get_recipients(account) def transfer(self, account, recipient, amount, reason=None): if reason is None: raise UserError('Reason is mandatory to do a transfer on ING website') if not isinstance(account, Account): account = self.get_account(account) if not isinstance(recipient, Recipient): # Remove weboob identifier prefix (LA-, CC-...) if "-" in recipient: recipient = recipient.split('-')[1] return self.browser.transfer(account, recipient, amount, reason) def iter_investment(self, account): if not isinstance(account, Account): account = self.get_account(account) return self.browser.get_investments(account) def iter_coming(self, account): if not isinstance(account, Account): account = self.get_account(account) return self.browser.get_coming(account) def iter_subscription(self): return self.browser.get_subscriptions() def get_subscription(self, _id): return find_object(self.browser.get_subscriptions(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subscription = self.get_subscription(_id.split('-')[0]) return find_object(self.browser.get_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.get_documents(subscription) def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) self.get_document(bill.id) try: self.browser.predownload(bill) except ServerError: return NotAvailable assert(self.browser.response.headers['content-type'] in ["application/pdf", "application/download"]) return self.browser.response.content �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/pages/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016145�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/pages/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000002743�13034501105�0020264�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2011 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .accounts_list import AccountsList, TitreDetails, ASVInvest, DetailFondsPage, \ IbanPage from .login import LoginPage, StopPage, ActionNeededPage, ReturnPage from .transfer import TransferPage, TransferConfirmPage from .bills import BillsPage from .titre import NetissimaPage, TitrePage, TitreHistory, TitreValuePage, ASVHistory class AccountPrelevement(AccountsList): pass __all__ = ['AccountsList', 'LoginPage', 'NetissimaPage','TitreDetails', 'AccountPrelevement', 'TransferPage', 'TransferConfirmPage', 'BillsPage', 'StopPage', 'TitrePage', 'TitreHistory', 'IbanPage', 'TitreValuePage', 'ASVHistory', 'ASVInvest','DetailFondsPage', 'ActionNeededPage', 'ReturnPage'] �����������������������������weboob-1.2/modules/ing/pages/accounts_list.py�������������������������������������������������������0000664�0000000�0000000�00000024111�13034501105�0021370�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2014 Florent Fourcot, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import date, timedelta import datetime import re from weboob.capabilities.bank import Account, Investment from weboob.capabilities.base import NotAvailable from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ListElement, TableElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Filter, Field, MultiFilter, Date, \ Lower, Async, AsyncLoad, Format, TableCell, Eval from weboob.browser.filters.html import Attr, Link from weboob.tools.capabilities.bank.transactions import FrenchTransaction class Transaction(FrenchTransaction): PATTERNS = [(re.compile(u'^retrait dab (?P<dd>\d{2})/(?P<mm>\d{2})/(?P<yy>\d{4}) (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(u'^carte (?P<dd>\d{2})/(?P<mm>\d{2})/(?P<yy>\d{4}) (?P<text>.*)'), FrenchTransaction.TYPE_CARD), (re.compile(u'^virement (sepa )?(emis vers|recu|emis)? (?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(u'^cheque (?P<text>.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(u'^prelevement (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(u'^prlv sepa (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(u'^prélèvement sepa en faveur de (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), ] class AddPref(MultiFilter): prefixes = {u'Courant': u'CC-', u'Livret A': 'LA-', u'Orange': 'LEO-', u'Durable': u'LDD-', u"Titres": 'TITRE-', u'PEA': u'PEA-'} def filter(self, values): el, label = values for key, pref in self.prefixes.items(): if key in label: return pref + el return el class AddType(Filter): types = {u'Courant': Account.TYPE_CHECKING, u'Livret A': Account.TYPE_SAVINGS, u'Orange': Account.TYPE_SAVINGS, u'Durable': Account.TYPE_SAVINGS, u'Titres': Account.TYPE_MARKET, u'PEA': Account.TYPE_MARKET, u'Direct Vie': Account.TYPE_LIFE_INSURANCE, u'Assurance Vie': Account.TYPE_LIFE_INSURANCE } def filter(self, label): for key, acc_type in self.types.items(): if key in label: return acc_type return Account.TYPE_UNKNOWN class PreHashmd5(MultiFilter): def filter(self, values): concat = '' for value in values: if type(value) is datetime.date: concat += value.strftime('%d/%m/%Y') else: concat += u'%s' % value return concat.encode('utf-8') class INGDate(Date): monthvalue = {u'janv.': '01', u'févr.': '02', u'mars': '03', u'avr.': '04', u'mai': '05', u'juin': '06', u'juil.': '07', u'août': '08', u'sept.': '09', u'oct.': '10', u'nov.': '11', u'déc.': '12'} def filter(self, txt): if txt == 'hier': return date.today() - timedelta(days=1) elif txt == "aujourd'hui": return date.today() elif txt == 'demain': return date.today() + timedelta(days=1) frenchmonth = txt.split(' ')[1] month = self.monthvalue[frenchmonth] txt = txt.replace(' ', '') txt = txt.replace(frenchmonth, '/%s/' % month) return super(INGDate, self).filter(txt) class INGCategory(Filter): catvalue = {u'virt': u"Virement", u'autre': u"Autre", u'plvt': u'Prélèvement', u'cb_ret': u"Carte retrait", u'cb_ach': u'Carte achat', u'chq': u'Chèque', u'frais': u'Frais bancaire', u'sepaplvt': u'Prélèvement'} def filter(self, txt): txt = txt.split('-')[0].lower() try: return self.catvalue[txt] except: return txt class AccountsList(LoggedPage, HTMLPage): i = 0 def has_error(self): return len(self.doc.xpath('//div[has-class("alert-warning")]')) > 0 @method class get_list(ListElement): item_xpath = '//a[@class="mainclic"]' class item(ItemElement): klass = Account obj_currency = u'EUR' obj_label = CleanText('span[@class="title"]') obj_id = AddPref(Field('_id'), Field('label')) obj_type = AddType(Field('label')) obj_balance = CleanDecimal('span[@class="solde"]/label', replace_dots=True) obj_coming = NotAvailable obj__jid = Attr('//input[@name="javax.faces.ViewState"]', 'value') def obj__id(self): return CleanText('span[@class="account-number"]')(self) or CleanText('span[@class="life-insurance-application"]')(self) class generic_transactions(ListElement): class item(ItemElement): klass = Transaction obj_id = None # will be overwrited by the browser # we use lower for compatibility with the old website obj_amount = CleanDecimal('.//td[starts-with(@class, "amount")]', replace_dots=True) obj_date = INGDate(CleanText('.//td[@class="date"]'), dayfirst=True) obj_rdate = Field('date') obj__hash = PreHashmd5(Field('date'), Field('raw'), Field('amount')) obj_category = INGCategory(Attr('.//td[@class="picto"]/span', 'class')) def obj_raw(self): return Transaction.Raw(Lower('.//td[@class="lbl"]'))(self) or Format('%s %s', Field('date'), Field('amount'))(self) def condition(self): if self.el.find('.//td[@class="date"]') is None: return False if 'index' in self.env and self.env['index'] > 0 and self.page.i < self.env['index']: self.page.i += 1 return False return True @method class get_coming(generic_transactions): item_xpath = '//div[@class="transactions cc future"]//table' @method class get_transactions_cc(generic_transactions): item_xpath = '//div[@class="temporaryTransactionList"]//table' @method class get_transactions_others(generic_transactions): item_xpath = '//table' def get_history_jid(self): span = Attr('//*[starts-with(@id, "index:j_id")]', 'id')(self.doc) jid = span.split(':')[1] return jid def get_asv_jid(self): return self.doc.xpath('//input[@id="javax.faces.ViewState"]/@value')[0] def islast(self): havemore = self.doc.getroot().cssselect('.show-more-transactions') if len(havemore) == 0: return True nomore = self.doc.getroot().cssselect('.no-more-transactions') return len(nomore) > 0 @property def is_asv(self): span = self.doc.xpath('//span[@id="index:panelASV"]') return len(span) > 0 @property def asv_has_detail(self): ap = self.doc.xpath('//a[@id="index:asvInclude:goToAsvPartner"] | //p[contains(text(), "Gestion Libre")]') return len(ap) > 0 @property def asv_is_other(self): a = self.doc.xpath('//a[@id="index:asvInclude:goToAsvPartner"]') return len(a) > 0 def submit(self): form = self.get_form(name="follow_link") form['follow_link:j_idcl'] = "follow_link:goToAsvPartner" form.submit() class IbanPage(LoggedPage, HTMLPage): def get_iban(self): iban = CleanText('//tr[td[1]//text()="IBAN"]/td[2]')(self.doc).strip().replace(' ', '') if not iban or 'null' in iban: return NotAvailable return iban class TitreDetails(LoggedPage, HTMLPage): def submit(self): form = self.get_form() form.submit() class ASVInvest(LoggedPage, HTMLPage): @method class iter_investments(TableElement): item_xpath = '//table[@class="Tableau"]//tr[position()>2]' head_xpath = '//table[@class="Tableau"]//tr[position()=2]/td' col_label = u'Support(s)' col_vdate = re.compile('Date') col_unitvalue = u'Valeur de part' col_quantity = u'Nombre de parts' col_valuation = u'Contre-valeur' col_unitprice = u'Prix revient' col_diff = u'Montant' col_portfolio_share = u'%' class item(ItemElement): klass = Investment load_details = Link('.//td[1]//a') & AsyncLoad def obj_code(self): val = Async('details', CleanText('//td[@class="libelle-normal" and contains(.,"CodeISIN")]', default=NotAvailable))(self) return val.split('CodeISIN : ')[1] if val else val def obj_portfolio_share(self): p = CleanDecimal(TableCell('portfolio_share'), replace_dots=True, default=NotAvailable)(self) return Eval(lambda x: x / 100, p)(self) if p else p obj_label = CleanText(TableCell('label')) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=True, default=NotAvailable) obj_quantity = CleanDecimal(TableCell('quantity'), replace_dots=True, default=NotAvailable) obj_valuation = CleanDecimal(TableCell('valuation'), replace_dots=True) obj_unitprice = CleanDecimal(TableCell('unitprice'), replace_dots=True, default=NotAvailable) obj_diff = CleanDecimal(TableCell('diff'), replace_dots=True, default=NotAvailable) class DetailFondsPage(LoggedPage,HTMLPage): pass �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/pages/bills.py���������������������������������������������������������������0000664�0000000�0000000�00000006606�13034501105�0017634�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import Bill, Subscription from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.filters.standard import Filter, CleanText, Format, Field, Env, Date from weboob.browser.filters.html import Attr from weboob.browser.elements import ListElement, ItemElement, method from weboob.tools.date import parse_french_date class FormId(Filter): def filter(self, txt): formid = txt.split("parameters")[1] formid = formid.split("'")[2] return formid class BillsPage(LoggedPage, HTMLPage): @method class iter_account(ListElement): item_xpath = '//ul[@class="unstyled striped"]/li' class item(ItemElement): klass = Subscription obj__javax = Attr("//form[@id='accountsel_form']/input[@name='javax.faces.ViewState']", 'value') obj_id = Attr('input', "value") obj_label = CleanText('label') obj__formid = FormId(Attr('input', 'onclick')) def postpredown(self, _id): _id = _id.split("'")[3] form = self.get_form(name="downpdf_form") form['statements_form'] = 'statements_form' form['statements_form:j_idcl'] = _id form.submit() @pagination @method class iter_documents(ListElement): item_xpath = '//ul[@id="statements_form:statementsel"]/li' def next_page(self): lis = self.page.doc.xpath('//form[@name="years_form"]//li') selected = False ref = None for li in lis: if "rich-list-item selected" in li.attrib['class']: selected = True else: if selected: ref = li.find('a').attrib['id'] break if ref is None: return form = self.page.get_form(name="years_form") form.pop('years_form:j_idcl') form.pop('years_form:_link_hidden_') form['AJAXREQUEST'] = "years_form:year_region" form[ref] = ref return form.request class item(ItemElement): klass = Bill condition = lambda self: not (u"tous les relev" in CleanText('a[1]')(self.el)) and not (u'annuel' in CleanText('a[1]')(self.el)) obj_label = CleanText('a[1]', replace=[(' ', '-')]) obj_id = Format(u"%s-%s", Env('subid'), Field('label')) # Force first day of month as label is in form "janvier 2016" obj_date = Format("1 %s", Field('label')) & Date(parse_func=parse_french_date) obj_format = u"pdf" obj_type = u"bill" obj__localid = Attr('a[2]', 'onclick') ��������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/pages/login.py���������������������������������������������������������������0000664�0000000�0000000�00000012064�13034501105�0017632�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2014 Florent Fourcot, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from StringIO import StringIO from weboob.exceptions import BrowserIncorrectPassword, ActionNeeded from weboob.tools.captcha.virtkeyboard import VirtKeyboard from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.filters.html import Attr from weboob.browser.filters.standard import CleanText class INGVirtKeyboard(VirtKeyboard): symbols = {'0': '327208d491507341908cf6920f26b586', '1': '615ff37b15645da106cebc4605b399de', '2': 'fb04e648c93620f8b187981f9742b57e', '3': 'b786d471a70de83657d57bdedb6a2f38', '4': '41b5501219e8d8f6d3b0baef3352ce88', '5': 'c72b372fb035160f2ff8dae59cd7e174', '6': '392fa79e9a1749f5c8c0170f6a8ec68b', '7': 'fb495b5cf7f46201af0b4977899b56d4', '8': 'e8fea1e1aa86f8fca7f771db9a1dca4d', '9': '82e63914f2e52ec04c11cfc6fecf7e08' } color = 64 coords = {"11": (5, 5, 33, 33), "21": (45, 5, 73, 33), "31": (85, 5, 113, 33), "41": (125, 5, 153, 33), "51": (165, 5, 193, 33), "12": (5, 45, 33, 73), "22": (45, 45, 73, 73), "32": (85, 45, 113, 73), "42": (125, 45, 153, 73), "52": (165, 45, 193, 73) } def __init__(self, page): self.page = page img = page.doc.xpath("//div[has-class('clavier')]/img") if len(img) == 0: raise BrowserIncorrectPassword() url = Attr('.', "src")(img[1]) VirtKeyboard.__init__(self, StringIO(self.page.browser.open(url).content), self.coords, self.color) self.check_symbols(self.symbols, self.page.browser.responses_dirname) def get_string_code(self, string): code = '' first = True for c in string: if not first: code += "," else: first = False codesymbol = self.get_symbol_code(self.symbols[c]) x = (self.coords[codesymbol][0] + self.coords[codesymbol][2]) / 2 y = (self.coords[codesymbol][1] + self.coords[codesymbol][3]) / 2 code += "%d,%d" % (x, y) return code def get_coordinates(self, password): temppasswd = "" elems = self.page.doc.xpath('//div[@class="digitpad"]/span/font') for i, font in enumerate(elems): if Attr('.', 'class')(font) == "vide": temppasswd += password[i] self.page.browser.logger.debug('We are looking for : ' + temppasswd) coordinates = self.get_string_code(temppasswd) self.page.browser.logger.debug("Coordonates: " + coordinates) return coordinates class LoginPage(HTMLPage): def prelogin(self, login, birthday): # First step : login and birthday form = self.get_form(name='zone1Form') form['zone1Form:numClient'] = login form['zone1Form:dateDay'] = birthday[0:2] form['zone1Form:dateMonth'] = birthday[2:4] form['zone1Form:dateYear'] = birthday[4:9] form['zone1Form:idRememberMyCifCheck'] = False form.submit() def error(self): err = self.doc.find('//span[@class="error"]') return err is not None def login(self, password): # 2) And now, the virtual Keyboard vk = INGVirtKeyboard(self) form = self.get_form(name='mrc') form['mrc:mrg'] = 'mrc:mrg' form['AJAXREQUEST'] = '_viewRoot' form['mrc:mrldisplayLogin'] = vk.get_coordinates(password) form.submit() def check_for_action_needed(self): link = Attr('//meta[@content="/general?command=displayTRAlertMessage"]', 'content', default=None)(self.doc) if link: self.browser.location(link) class ActionNeededPage(HTMLPage): def on_load(self): if self.doc.xpath(u'//form//h1[contains(text(), "Accusé de reception du chéquier")]'): form = self.get_form(name='Alert') form['command'] = 'validateAlertMessage' form['radioValide_1_2_40003039944'] = 'Non' form.submit() else: raise ActionNeeded(CleanText(u'//form//h1[1]')(self.doc)) class StopPage(HTMLPage): pass class ReturnPage(LoggedPage, HTMLPage): def on_load(self): self.get_form(name='retoursso').submit() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/pages/titre.py���������������������������������������������������������������0000664�0000000�0000000�00000016752�13034501105�0017661�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from decimal import Decimal from weboob.capabilities.base import NotAvailable from weboob.capabilities.bank import Investment from weboob.browser.pages import RawPage, HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, TableElement, ItemElement, method from weboob.browser.filters.standard import CleanDecimal, CleanText, Date, TableCell, Regexp, Env, Async, AsyncLoad from weboob.browser.filters.html import Link, Attr from weboob.tools.capabilities.bank.transactions import FrenchTransaction class NetissimaPage(HTMLPage): pass class Transaction(FrenchTransaction): pass class TitreValuePage(LoggedPage, HTMLPage): def get_isin(self): return unicode(self.doc.xpath('//div[@id="headFiche"]//span[@id="test3"]/text()')[0].split(' - ')[0].strip()) class TitrePage(LoggedPage, RawPage): def iter_investments(self): # We did not get some html, but something like that (XX is a quantity, YY a price): # message='[...] # popup=2{6{E:ALO{PAR{{reel{695{380{ALSTOM REGROUPT#XX#YY,YY €#YY,YY €#1 YYY,YY €#-YYY,YY €#-42,42%#-0,98 %#42,42 %#|1|AXA#cotationValeur.php?val=E:CS&pl=6&nc=1& # popup=2{6{E:CS{PAR{{reel{695{380{AXA#XX#YY,YY €#YY,YYY €#YYY,YY €#YY,YY €#3,70%#42,42 %#42,42 %#|1|blablablab #cotationValeur.php?val=P:CODE&pl=6&nc=1& # [...] lines = self.doc.split("|1|") if len(lines) > 1: lines[0] = lines[0].split("|")[1] else: lines.pop(0) invests = [] for line in lines: _id, _pl = None, None columns = line.split('#') if columns[1] != '': _pl = columns[1].split('{')[1] _id = columns[1].split('{')[2] invest = Investment() invest.label = unicode(columns[0].split('{')[-1]) invest.code = unicode(_id) if _id is not None else NotAvailable if invest.code and ':' in invest.code: invest.code = self.browser.titrevalue.open(val=invest.code,pl=_pl).get_isin() # The code we got is not a real ISIN code. if invest.code and not re.match('^[A-Z]{2}[\d]{10}$|^[A-Z]{2}[\d]{5}[A-Z]{1}[\d]{4}$', invest.code): m = re.search('\{([A-Z]{2}[\d]{10})\{|\{([A-Z]{2}[\d]{5}[A-Z]{1}[\d]{4})\{', line) if m: invest.code = unicode(m.group(1) or m.group(2)) quantity = FrenchTransaction.clean_amount(columns[2]) invest.quantity = CleanDecimal(default=NotAvailable).filter(quantity) unitprice = FrenchTransaction.clean_amount(columns[3]) invest.unitprice = CleanDecimal(default=NotAvailable).filter(unitprice) unitvalue = FrenchTransaction.clean_amount(columns[4]) invest.unitvalue = CleanDecimal(default=NotAvailable).filter(unitvalue) valuation = FrenchTransaction.clean_amount(columns[5]) # valuation is not nullable, use 0 as default value invest.valuation = CleanDecimal(default=Decimal('0')).filter(valuation) diff = FrenchTransaction.clean_amount(columns[6]) invest.diff = CleanDecimal(default=NotAvailable).filter(diff) # On some case we have a multine investment with a total column # for now we have only see this on 2 lines, we will need to adapt it when o if columns[0] == u'|Total' and _id == 'fichevaleur': prev_inv = invest invest = invests.pop(-1) if prev_inv.quantity: invest.quantity = invest.quantity + prev_inv.quantity if prev_inv.valuation: invest.valuation = invest.valuation + prev_inv.valuation if prev_inv.diff: invest.diff = invest.diff + prev_inv.diff invests.append(invest) for invest in invests: yield invest class TitreHistory(LoggedPage, HTMLPage): @method class iter_history(ListElement): item_xpath = '//table[@class="datas retour"]/tr' class item(ItemElement): klass = Transaction condition = lambda self: len(self.el.xpath('td[@class="impaire"]')) > 0 obj_raw = Transaction.Raw('td[4] | td[3]') obj_date = Date(CleanText('td[2]'), dayfirst=True) obj_amount = CleanDecimal('td[7]', replace_dots=True) class ASVHistory(LoggedPage, HTMLPage): @method class get_investments(TableElement): item_xpath = '//table[@class="Tableau"]/tr[td[not(has-class("enteteTableau"))]]' head_xpath = '//table[@class="Tableau"]/tr[td[has-class("enteteTableau")]]/td' col_label = u'Support(s)' col_vdate = u'Date de valeur' col_unitvalue = u'Valeur de part' col_quantity = [u'(*) Nb de parts', u'Nb de parts'] col_valuation = [u'Montant', u'Montant versé'] class item(ItemElement): klass = Investment load_details = Regexp(Attr('./td/a', 'onclick', default=""), 'PageExterne\(\'([^\']+)', default=None) & AsyncLoad obj_label = CleanText(TableCell('label')) obj_code = Async('details') & CleanText('//td[contains(text(), "CodeISIN")]/b', default=NotAvailable) obj_quantity = CleanDecimal(TableCell('quantity'), replace_dots=True, default=NotAvailable) obj_unitvalue = CleanDecimal(TableCell('unitvalue'), replace_dots=True, default=NotAvailable) obj_valuation = CleanDecimal(TableCell('valuation'), replace_dots=True) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True) @pagination @method class iter_history(TableElement): item_xpath = '//table[@class="Tableau"]/tr[td[not(has-class("enteteTableau"))]]' head_xpath = '//table[@class="Tableau"]/tr[td[has-class("enteteTableau")]]/td' col_date = u'Date d\'effet' col_raw = u'Nature du mouvement' col_amount = u'Montant brut' next_page = Link('//a[contains(@href, "PageSuivante")]', default=None) class item(ItemElement): klass = Transaction obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_raw = Transaction.Raw(TableCell('raw')) obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True) obj__detail = Env('detail') def obj_id(self): try: return Regexp(Link('./td/a', default=None), 'numMvt=(\d+)', default=None)(self) except TypeError: return NotAvailable def parse(self, el): link = Link('./td/a', default=None)(self) page = self.page.browser.async_open(link) if link else None self.env['detail'] = page ����������������������weboob-1.2/modules/ing/pages/transfer.py������������������������������������������������������������0000664�0000000�0000000�00000014500�13034501105�0020343�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2014 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import Recipient, AccountNotFound, Transfer from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Format from weboob.browser.filters.html import Attr from .login import INGVirtKeyboard class TransferPage(LoggedPage, HTMLPage): @method class get_recipients(ListElement): class ExternalRecipients(ListElement): item_xpath = '//select[@id="transfer_form:externalAccounts"]/option' class item(ItemElement): klass = Recipient condition = lambda self: Attr('.', 'value')(self.el) != "na" obj_id = Attr('.', 'value') obj_label = CleanText('.') obj__type = 'ext' class InternalRecipients(ListElement): item_xpath = '//table[@id="transfer_form:receiptAccount"]/tbody/tr' class item(ItemElement): klass = Recipient obj_id = Attr('td[1]/input', 'value') obj_label = Format(u"%s %s %s", CleanText('td[1]/label'), CleanText('td[2]/label'), CleanText('td[3]/label')) obj__type = "int" def ischecked(self, _id): # remove prefix (CC-, LA-, ...) if "-" in _id: _id = _id.split('-')[1] try: option = self.doc.xpath('//input[@value="%s"]' % _id)[0] except: raise AccountNotFound() return option.attrib.get("checked") == "checked" def transfer(self, recipient, amount, reason): form = self.get_form(name="transfer_form") form.pop('transfer_form:_link_hidden_') form.pop('transfer_form:j_idcl') form['AJAXREQUEST'] = "_viewRoot" form['AJAX:EVENTS_COUNT'] = "1" form['transfer_form:transferMotive'] = reason form["transfer_form:valide"] = "transfer_form:valide" form["transfer_form:validateDoTransfer"] = "needed" form["transfer_form:transferAmount"] = str(amount) if recipient._type == "int": form['transfer_recipient_radio'] = recipient.id else: form['transfer_form:externalAccounts'] = recipient.id form.submit() def buildonclick(self, recipient, account): javax = self.doc.xpath('//input[@id="javax.faces.ViewState"]')[0].attrib['value'] if recipient._type == "ext": select = self.doc.xpath('//select[@id="transfer_form:externalAccounts"]')[0] onclick = select.attrib['onchange'] params = onclick.split(',')[3].split('{')[1] idparam = params.split("'")[1] param = params.split("'")[3] request = {"AJAXREQUEST": "transfer_form:transfer_radios_form", "transfer_form:generalMessages": "", "transfer_issuer_radio": account.id[3:], "transfer_form:externalAccounts": recipient.id, "transfer_date": "0", "transfer_form:transferAmount": "", "transfer_form:transferMotive": "", "transfer_form:validateDoTransfer": "needed", "transfer_form": "transfer_form", "autoScrol": "", "javax.faces.ViewState": javax, idparam: param} return request elif recipient._type == "int": for input in self.doc.xpath('//input[@value=%s]' % recipient.id): if input.attrib['name'] == "transfer_recipient_radio": onclick = input.attrib['onclick'] break # Get something like transfer_form:issueAccount:0:click params = onclick.split(',')[3].split('{')[1] idparam = params.split("'")[1] param = params.split("'")[3] request = {"AJAXREQUEST": "transfer_form:transfer_radios_form", 'transfer_issuer_radio': account.id[3:], "transfer_recipient_radio": recipient.id, "transfer_form:externalAccounts": "na", "transfer_date": 0, "transfer_form:transferAmount": "", "transfer_form:transferMotive": "", "transfer_form:validateDoTransfer": "needed", "transfer_form": "transfer_form", "autoScroll": "", "javax.faces.ViewState": javax, idparam: param} return request class TransferConfirmPage(LoggedPage, HTMLPage): def confirm(self, password): vk = INGVirtKeyboard(self) form = self.get_form(xpath='//div[@id="transfer_panel"]//form') for elem in form: if "_link_hidden_" in elem or "j_idcl" in elem: form.pop(elem) form['AJAXREQUEST'] = '_viewRoot' form['%s:mrgtransfer' % form.name] = '%s:mrgtransfer' % form.name form['%s:mrltransfer' % form.name] = vk.get_coordinates(password) form.submit() @method class recap(ListElement): item_xpath = '//div[@class="encadre transfert-validation"]' class item(ItemElement): klass = Transfer obj_amount = CleanDecimal('.//label[@id="confirmtransferAmount"]', replace_dots=True) obj_origin = CleanText('.//span[@id="confirmfromAccount"]') obj_recipient = CleanText('.//span[@id="confirmtoAccount"]') obj_reason = CleanText('.//span[@id="confirmtransferMotive"]') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ing/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000005072�13034501105�0016403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.bank import Account from datetime import timedelta import random class INGTest(BackendTest): MODULE = 'ing' def test_accounts(self): l = list(self.backend.iter_accounts()) for account in l: # Test if get_account works _id = self.backend.get_account(account.id) self.assertTrue(_id.id == account.id) # Methods can use Account objects or id. Try one of them id_or_account = random.choice([account, account.id]) if account.type == Account.TYPE_CHECKING or account.type == Account.TYPE_SAVINGS: history = list(self.backend.iter_history(id_or_account)) self.assertTrue(len(history) > 0) date = history.pop(0).date for elem in history: self.assertTrue(date + timedelta(days=2) >= elem.date) date = elem.date # recipients = list(self.backend.iter_transfer_recipients(id_or_account)) # elf.assertTrue(len(recipients) > 0) elif account.type == Account.TYPE_MARKET: invest = list(self.backend.iter_investment(id_or_account)) self.backend.iter_history(id_or_account) # can be empty. Only try to call it self.assertTrue(len(invest) > 0) def test_subscriptions(self): l = list(self.backend.iter_subscription()) for sub in l: _id = self.backend.get_subscription(sub.id) self.assertTrue(_id.id == sub.id) bills = list(self.backend.iter_documents(sub)) self.assertTrue(len(bills) > 0) _id = self.backend.get_document(bills[0].id) self.assertTrue(_id.id == bills[0].id) self.backend.download_document(bills[0].id) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015741�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001515�13034501105�0020054�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"init of NewspaperInrocksModule" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewspaperInrocksModule __all__ = ['NewspaperInrocksModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000002536�13034501105�0020004�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"browser for inrocks.fr website" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import ArticlePage from weboob.browser.browsers import AbstractBrowser from weboob.browser.url import URL class NewspaperInrocksBrowser(AbstractBrowser): "NewspaperInrocksBrowser class" PARENT = 'genericnewspaper' BASEURL = 'http://www.lesinrocks.com' article = URL('/\?p=.+', '/\d{4}/\d{2}/\d{2}/actualite/.*', 'http://blogs.lesinrocks.com/.*', '/.*', ArticlePage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) ������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000002275�13034501105�0020102�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD���@V)��� pHYs�� �� ����tIME&���tEXtComment�Created with GIMPW��IDATx[MOA~Zh)mZGE h4$`yA!ꅓw"Q$F=Q`bB\i%@[ xXnݙs.<~<`:P0a``P(N?@PZ �W=0o@ay '!p a:%h�_PXfWVǀ@T<!=s (dwXX̽�<~>N y@K?PY"v7Ss9Nَo�i`qTKO]|^e"r{XS� z>fS@N> st@I9~~>2y)$?uVOe)(hz`鍺=^Wʢ"[Ko5-QFiFXĥ<Q& !1Ged,QOz9"nq`q6CVE1tb-h">=dU7-G@TU*PIZ(QRz@iUGlBrI1I+xiu@qI]]HlW}vE>Cr))OL@i5{ #27e@gT- ޖ|'@R?Mmfx8|ƶmg68s*c HQ!CZ\t1C K+V6(ziU枣~DE}ۚէÑ>ckV>s)?Cس,�򗙀440GZ<e&&[Q{Z!t�׸z NՂZQ [nmX]7Fx "'5zbNZ7FDt>.e0j^K c.SݴgkCZJ89[dI<qS!0"MbvVw%'׶v$l3S?C��������BxFeye����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/module.py����������������������������������������������������������������0000664�0000000�0000000�00000002525�13034501105�0017604�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://www.lesinrocks.com" from weboob.tools.backend import AbstractModule from weboob.capabilities.messages import CapMessages from .browser import NewspaperInrocksBrowser from .tools import rssid class NewspaperInrocksModule(AbstractModule, CapMessages): MAINTAINER = u'Julien Hebert' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'inrocks' DESCRIPTION = u'Les Inrocks French news website' BROWSER = NewspaperInrocksBrowser RSS_FEED = 'http://www.lesinrocks.com/fileadmin/rss/actus.xml' RSSID = rssid PARENT = 'genericnewspaper' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000005060�13034501105�0017413�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"ArticlePage object for inrocks" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.html import XPathNotFound, CSS, CleanHTML class ArticlePage(AbstractPage): "ArticlePage object for inrocks" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_loaded(self): main = self._selector("div#block-article")(self.doc.getroot()) self.main_div = main[0] if len(main) else None self.element_title_selector = "div.header>h1" self.element_author_selector = "div.name" self.element_body_selector = "div.maincol" def get_title(self): try: return super(self.__class__, self).get_title() except(XPathNotFound): if self.main_div is None: return u"" else: raise def get_body(self): try: element_body = self.get_element_body() except XPathNotFound: return u'Ceci est un article payant' else: self.drop_comments(element_body) div_header_element = self._selector("div.header")(element_body)[0] div_content_element = self._selector("div#the-content")(element_body)[0] self.try_remove_from_selector_list(div_header_element, ["h1", "div.picture", "div.date", "div.news-single-img", "div.article-top", "div.metas_img", "strong"]) self.try_remove_from_selector_list(div_content_element, ["div.tw_button", "div.wpfblike", "blockquote", "p.wp-caption-text", "img"]) return '%s\n%s' % (CleanHTML('.')(div_header_element), CleanHTML('.')(div_content_element)) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/inrocks/test.py������������������������������������������������������������������0000664�0000000�0000000�00000001664�13034501105�0017301�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������ # -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class InrocksTest(BackendTest): MODULE = 'inrocks' def test_new_messages(self): for message in self.backend.iter_unread_messages(): pass ����������������������������������������������������������������������������weboob-1.2/modules/inrocks/tools.py�����������������������������������������������������������������0000664�0000000�0000000�00000001556�13034501105�0017462�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"common tools for inrocks backend" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. def url2id(url): "return an id from an url" return url def rssid(entry): return url2id(entry.id) ��������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipapi/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015373�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipapi/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000000073�13034501105�0017504�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import IpapiModule __all__ = ['IpapiModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipapi/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000001376�13034501105�0017535�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME[r��IDATxk`ol]J]Zۿ _(:ԋxx_!9CAOz/ZeEWetuuZ$@dPH<o7ϛ MD7� �@�� �@�� �@if=*I�X$ǃTq6Ȳ }L?zuq添k6mnϮB<{ҋ竈 i[SFpNN G@Y8޼ډ#�ņsֲyi�Dx"q=dx9Uc+TzK7n}>ZE�b\,#gŢw (^g~łٷ{wCAAoz =x4dss-,IA�A�rQjBc :]\ 2t`8_'@_*WiF%]؉631YrnII}l�(CUKY#4 zjn8'-q~z� XWk|30@�� �@�� �@���/3k#����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipapi/module.py������������������������������������������������������������������0000664�0000000�0000000�00000004015�13034501105�0017232�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.geolocip import CapGeolocIp, IpLocation from weboob.tools.backend import Module from weboob.browser.browsers import Browser from weboob.tools.json import json __all__ = ['IpapiModule'] class IpapiModule(Module, CapGeolocIp): NAME = 'ipapi' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"IP-API Geolocation API" BROWSER = Browser def get_location(self, ipaddr): res = self.browser.location('http://ip-api.com/json/%s' % ipaddr.encode('utf-8')) jres = json.loads(res.text) if "status" in jres and jres["status"] == "fail": raise Exception("IPAPI failure : %s" % jres["message"]) iploc = IpLocation(ipaddr) iploc.city = u'%s'%jres['city'] iploc.region = u'%s'%jres['regionName'] iploc.zipcode = u'%s'%jres['zip'] iploc.country = u'%s'%jres['country'] if jres['lat'] != '': iploc.lt = float(jres['lat']) else: iploc.lt = 0.0 if jres['lon'] != '': iploc.lg = float(jres['lon']) else: iploc.lg = 0.0 #iploc.host = 'NA' #iploc.tld = 'NA' if 'isp' in jres: iploc.isp = u'%s'%jres['isp'] return iploc �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipapi/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000001612�13034501105�0016724�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class IpapiTest(BackendTest): MODULE = 'ipapi' def test_ipapi(self): self.backend.get_location('88.198.11.130') ����������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016063�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000000101�13034501105�0020164�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import IpinfodbModule __all__ = ['IpinfodbModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000002645�13034501105�0020127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import HomePage, LocationPage __all__ = ['IpinfodbBrowser'] class IpinfodbBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'https://ipinfodb.com/' home = URL('$', HomePage) search = URL('ip_locator.php', LocationPage) def get_location(self, ipaddr): try: self.home.go() self.page.search(ipaddr) iploc = self.page.get_location() return iploc except BrowserHTTPNotFound: return �������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000010413�13034501105�0020215�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME  :'��IDATx[k$UνU=ݙ޷ccbC  H!+r �"`$ ~ DlE8!x{^{y{QUÏ{g5b Jꪺ~9qy\Aԋ!] Yy9}=g�=z�A w#]қ)6,)@X J'HG3<_Rp:> ¢P, ~$ֿ�8q#XHZoֳ= ]#ߔg¯=;R7Sz}Qԣ�rn q6 �+cT*55]r؃�!R�g$צr$#~<q_@C72_�N, � Gط۾{O>+:05<3x " bZ#פ<{7M^*� j_e/ɜr(W[ӷcbA"N)hn+ #NOPV[gP[C% xrX|<[:� 2QgGH�'-@m|5l`�#$&%iQ)@#y멁x8T+޳GnsNyU.Цyߧ]"굿}uõPI $ &uBq?1 5`vԡ m" `hvw޷N}̲I+S@C�XЗ)$n,(-gn�e hJs:^sD4HebG�&xe>lh 6_ZqM XT4^S;ѡM}@S{I�D豶eI-+KWL8BDBP֥@Cl O`XL 쀰Mf-rM! x+RY|FB$X dfTQȇtjvV3|v'j�i�Vʱ|'b]3 R9%ſҩ-JREm8f ؤt}I*ؽ%߷v]aԶ�m皾sja[X\Ͻ8No4Z<B�'`�lMibY9(-Gaޘ/Ѓ`W᚝+3(DCq51  WiL601J%* i.CJ,LZ$$K�P$ nFZl09B©ä:cP@AiZ+rCG?]S:JxȜ5UE:݌=8w|s�6'&zm5-;>0u_*/mI+%NA3)LRG‑=AM} (#ތRriB$(Pޕ�1�o8u¼L.f GFbhtkKo:`)�: !PЅa AD6uQ!&Z"l(WdI4@f)B(m=2o.X {/dBllqޓd!!ۈ<.FӀ�ef+%A 3PB[Yކ}D"M !!hp�la+" rR+u+[$:2O<` E X^%�AāRW{ ܆e?֌Ye_`yq2YW>UlP]<ى1ejYAb!-@1f?F#wD|+`@Q+=,!u)�P| c*CL2ͤc[@_,&Aj${/=L}[݉]?D3 9g53G$<@e\,�@5@� ߆e,,~ [vk9K0+sK'JLQr[PfՄ'� =∏�1>~ ;N<nU}v ,U#|?9lTVRY ^$@A ]�Ȩw j!PgS>ܿl#]<0НqxQ/_ {BL5=;ioq jcdb.,nw]w-Ls#]ש|J&v%�Á .^baEI$eBS] 1 I t�]oD.˜9{ob Q}~aqq]kM$̪,KЀ  h6@NPm԰ й zyћ@%x虏uX;/7)EPa?XR'@jRtuy*XB'L@K&Jc9\.@Zc9:R&�D4zBH<>zcg])"$@Le� ֐Y$w6cVM{E͕ k?o )غ�dmP ]�ָU*m%4pK/#/�BU]݁;LR/qҹM5>&UJl-W_U}%5-4B*p3�$ JGA !AxaFᥱxbbi3\ }n+ cWtܺJg�mg!WiIM8yQPQx>�>x*\cV|M*QčW[}Jm#v�pc|ohf^o� kjKh1qy+ W轀% MgS1_RFA,-Ld9u24@MT@idoۻM&N3@�_9Ǩ,4~A 8}iXRrcvVlZ 0 ի!k \(zhyiʇ dbő h)'PBFbQ,T+Ј J9lϟnA$�A#86vdhuR@t}V']|׀gd),^ ^Im70sН7`tQ[1r �`a$zO哈z@icS$ W!~߹:loXP\[8 R&&e0 IFKC;wQ^i`paW d@uYDeDMܪ0 {q֏4a:V&`^̡B,ִ /<Z�0!k�TFQ:HJ3zZDiu(&VE@l Bv[SuG,Umgؓv+`g ]Ra=[G:Qk�F8+|d!q}x"6&)2Ҥ."|Y�ѕcŕ$o}/YM I=s �\6Ò:Fo0rؘT}HlSK o b i}헞 觅?kmZ1rQB޵ϰ(a "|O6Uذ+, խs"N}s/)Lg'__` T~s8��!Pa[G@Q,BAT IjJ纠\Ӫ{@jQlꋣP.-pZ@d +g&?��'?wz'u4QU:BdUABBA&X�X1Ƥ$*K&`] Lm6iKcn_>Ρohl] VN,&6tT %xiMD}CAR'x'~~+�Vq6ܦ),2 4zP @:"I\KS83feRH}јAμ/ID%^ BS@Pdґr9@@@l .D6S]lxʍH}z�QRa&HP"FC8NјFuSR:r]%\qy\8![wl����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/module.py���������������������������������������������������������������0000664�0000000�0000000�00000002356�13034501105�0017730�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.geolocip import CapGeolocIp from weboob.tools.backend import Module from browser import IpinfodbBrowser __all__ = ['IpinfodbModule'] class IpinfodbModule(Module, CapGeolocIp): NAME = 'ipinfodb' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"IPInfoDB IP addresses geolocation service" BROWSER = IpinfodbBrowser def get_location(self, ipaddr): return self.browser.get_location(ipaddr) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000004664�13034501105�0017546�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, method from weboob.capabilities.geolocip import IpLocation from weboob.browser.filters.standard import Regexp, CleanText, Type from weboob.capabilities.base import NotAvailable class HomePage(HTMLPage): def search(self, ipaddr): form = self.get_form(xpath='//form[contains(@id, "search_form")]') form['ip'] = ipaddr form.submit() class LocationPage(HTMLPage): @method class get_location(ItemElement): klass = IpLocation obj_id = CleanText('//ul/li[starts-with(.,"IP address :")]/strong') obj_city = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"City :")]/text()'), 'City : (.*)'), default=NotAvailable) obj_country = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"Country :")]/text()'), 'Country : (.*)'), default=NotAvailable) obj_region = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"State/Province :")]/text()'), 'State/Province : (.*)'), default=NotAvailable) obj_lt = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"Latitude :")]/text()'), 'Latitude : (.*)'), default=NotAvailable) & Type(type=float) obj_lg = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"Longitude :")]/text()'), 'Longitude : (.*)'), default=NotAvailable) & Type(type=float) obj_zipcode = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"Zip or postal code :")]/text()'), 'Zip or postal code : (.*)'), default=NotAvailable) obj_host = CleanText(Regexp(CleanText('//ul/li[starts-with(.,"Hostname :")]/text()'), 'Hostname : (.*)'), default=NotAvailable) ����������������������������������������������������������������������������weboob-1.2/modules/ipinfodb/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000001722�13034501105�0017416�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class IpinfodbTest(BackendTest): MODULE = 'ipinfodb' def test_ipinfobd(self): self.backend.get_location('88.198.11.130') self.backend.get_location('2a01:4f8:130:3062::1') ����������������������������������������������weboob-1.2/modules/itella/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015543�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0017661�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import ItellaModule __all__ = ['ItellaModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000002073�13034501105�0017602�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.browsers import APIBrowser from .pages import SearchPage class ItellaBrowser(APIBrowser): def get_tracking_info(self, _id): r = self.open('https://www.posti.fi/henkiloasiakkaat/seuranta/api/shipments', data={"trackingCodes": ["%s" % _id]}) return SearchPage(self, r).get_info(_id) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000001230�13034501105�0017672�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���% ��� pHYs�� �� ����tIME xS���tEXtComment�Created with GIMPW��IDAThؿOSQ `D 8!0X&`cbHtPX\IĄտ8�]qG#?NDF!ՔBۈa0ay+yy{9^$<�(`#IB~ V #zDz%�W+�CÏYDORrm!zlg H@S�O30.bDxW^H1(pi:3rBva Y)TT(jAAߢ~\꓆sLr 0ff_C\p@ҟKm[w&R'{]za}\cnBN+@/4@:5G6E>q HY|`-$/+5ه|]VV4ịۥ�:] @sBDQӨC\{i*@6L6?b� ^fU՛zvÌ= <sX*kk<JfY-�(@ P]LQW@ P`zP����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000002624�13034501105�0017406�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.parcel import CapParcel from .browser import ItellaBrowser __all__ = ['ItellaModule'] class ItellaModule(Module, CapParcel): NAME = 'itella' DESCRIPTION = u'Itella website' MAINTAINER = u'Matthieu Weber' EMAIL = 'mweber+weboob@free.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ItellaBrowser def get_parcel_tracking(self, id): """ Get information abouut a parcel. :param id: ID of the parcel :type id: :class:`str` :rtype: :class:`Parcel` :raises: :class:`ParcelNotFound` """ return self.browser.get_tracking_info(id) ������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/pages.py������������������������������������������������������������������0000664�0000000�0000000�00000004235�13034501105�0017220�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from dateutil.parser import parse as parse_date from weboob.browser.pages import JsonPage from weboob.capabilities.parcel import Parcel, Event, ParcelNotFound class SearchPage(JsonPage): STATUSES = { "WAITING": Parcel.STATUS_PLANNED, "IN_TRANSPORT": Parcel.STATUS_IN_TRANSIT, "READY_FOR_PICKUP": Parcel.STATUS_ARRIVED, "DELIVERED": Parcel.STATUS_ARRIVED, } def get_info(self, _id): shipments = self.doc["shipments"] if not shipments: raise ParcelNotFound("No such ID: %s" % _id) shipment = shipments[0] result_id = shipment["trackingCode"] if result_id != _id: raise ParcelNotFound("ID mismatch: expecting %s, got %s" % (_id, result_id)) p = Parcel(_id) if shipment["estimatedDeliveryTime"]: p.arrival = parse_date(shipment["estimatedDeliveryTime"], ignoretz=True) events = shipment["events"] p.history = [self.build_event(i, data) for i, data in enumerate(events)] p.status = self.STATUSES.get(shipment["phase"], Parcel.STATUS_UNKNOWN) most_recent = p.history[0] p.info = most_recent.activity return p def build_event(self, index, data): event = Event(index) event.activity = data["description"]["en"] event.date = parse_date(data["timestamp"], ignoretz=True) event.location = data["locationName"] return event �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/itella/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000001747�13034501105�0017105�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.parcel import ParcelNotFound class ItellaTest(BackendTest): MODULE = 'itella' def test_itella(self): self.assertRaises(ParcelNotFound, self.backend.get_parcel_tracking, "foo") �������������������������weboob-1.2/modules/izneo/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015415�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/izneo/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0017530�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import IzneoModule __all__ = ['IzneoModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/izneo/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000002730�13034501105�0017552�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 8({��XIDATxoHu?6Vc3BH6Ŕ6Jk\G/(A4tAXYzP S%CGH3fl>9ϽK s~|w~w~BEIQJ(nk2�ׁI,d" �Mbhz' Lv`Ou]>[V{:3f)ַ�qy_l[{%C'@69ͮ/9s'Ϸ5[q/syR4[Hpn?| |ܐ[S>($p x( OxJ,|q6~4%&8wX3p!['#|sRdm�9/xx',pM[X(29AN8 AuPRFb4d@l{\l~3[ =3smP]{#x[_�/ a(sjSl1ov=F㏈�%k![v^h|`&[pB#6=E)x60`@Lk `XtРB6Ҟ7*աIE055WaxYoMiS~?KĶPWN|  ? -hP!V)lB�Pj0 ҄r\jk1oKgV#pM9ZC@RݘBV!P}iP!`RMŘ?@Mݑ "PLDp3ִ2Koh5 JaK 3qjuYQT^eACR~*&i!,"ıf&J &nc}FT{2�iKѦV_ ˍ":nrɅGBH/ޝҮD0ۣȐ1 Y 0CK: F/h.Э+8x� Iů mm "QA qGĆ~3gTn..ig="^95]Hs/c.PUϲn+:D@>WZdy IjWb|` WP0)/01�. 㯊OY?ܕejޑ#$Krl"`Z0 |k;D-:?Dau\oqJ[%]^O.~-1:r(42 $<#$mpf% gsBB({X"ʮ (-I<����IENDB`����������������������������������������weboob-1.2/modules/izneo/module.py������������������������������������������������������������������0000664�0000000�0000000�00000006030�13034501105�0017253�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.gallery import CapGallery, BaseGallery, BaseImage from weboob.tools.json import json from weboob.tools.backend import Module from weboob.deprecated.browser import Browser, Page __all__ = ['IzneoModule'] class ReaderV2(Page): def get_ean(self): return self.document.xpath("//div[@id='viewer']/attribute::rel")[0] def iter_gallery_images(self, gallery): ean = self.get_ean() pages = json.load(self.browser.openurl( 'http://www.izneo.com/playerv2/ajax.php?ean=%s&action=get_list_jpg' % ean)) for page in pages['list']: width = 1200 # maximum width yield BaseImage(page['page'], gallery=gallery, url=("http://www.izneo.com/playerv2/%s/%s/%s/%d/%s" % (page['expires'], page['token'], ean, width, page['page']))) class IzneoBrowser(Browser): PAGES = {r'http://.+\.izneo.\w+/readv2-.+': ReaderV2} def iter_gallery_images(self, gallery): self.location(gallery.url) assert self.is_on_page(ReaderV2) return self.page.iter_gallery_images(gallery) def fill_image(self, image, fields): if 'data' in fields: image.data = self.readurl(self.request_class( image.url, None, {'Referer': image.gallery.url})) class IzneoModule(Module, CapGallery): NAME = 'izneo' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = 'Izneo digital comics' LICENSE = 'AGPLv3+' BROWSER = IzneoBrowser def iter_gallery_images(self, gallery): with self.browser: return self.browser.iter_gallery_images(gallery) def get_gallery(self, _id): match = re.match(r'(?:(?:.+izneo.com/)?readv2-)?(\d+-\d+)/?$', _id) if match is None: return None _id = match.group(1) gallery = BaseGallery(_id, url=('http://www.izneo.com/readv2-%s' % _id)) with self.browser: return gallery def fill_gallery(self, gallery, fields): gallery.title = gallery.id def fill_image(self, image, fields): with self.browser: self.browser.fill_image(image, fields) OBJECTS = { BaseGallery: fill_gallery, BaseImage: fill_image} ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/�����������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017425�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/__init__.py������������������������������������������������������0000664�0000000�0000000�00000001451�13034501105�0021537�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import JacquieEtMichelModule __all__ = ['JacquieEtMichelModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/browser.py�������������������������������������������������������0000664�0000000�0000000�00000003543�13034501105�0021467�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.deprecated.browser import Browser from weboob.deprecated.browser.decorators import id2url from .video import JacquieEtMichelVideo from .pages import VideoPage, ResultsPage __all__ = ['JacquieEtMichelBrowser'] class JacquieEtMichelBrowser(Browser): DOMAIN = u'jacquieetmicheltv.net' ENCODING = None PAGES = {r'https?://.*jacquieetmicheltv.net/': ResultsPage, r'https?://.*jacquieetmicheltv.net/videolist/.*': ResultsPage, r'https?://.*jacquieetmicheltv.net/showvideo/(?P<id>\d+)/.*': VideoPage, } @id2url(JacquieEtMichelVideo.id2url) def get_video(self, url, video=None): self.location(url) assert self.is_on_page(VideoPage), 'Should be on video page.' return self.page.get_video(video) def search_videos(self, pattern): self.location('/videolist/searchmodevideo/query%s/' % (urllib.quote(pattern.encode('utf-8')))) assert self.is_on_page(ResultsPage) return self.page.iter_videos() def latest_videos(self): self.home() assert self.is_on_page(ResultsPage) return self.page.iter_videos() �������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/favicon.png������������������������������������������������������0000664�0000000�0000000�00000002130�13034501105�0021554�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME  I|���tEXtComment�Created with GIMPW��IDATxOhu?-HjRЃ˶z==x֣R/OzTA`=x( H KUlGƸdIt&eؙeMj2~~o~((((((tG@pl׀+=ƼpH&٬/af? 4As1KAĈ|> q8O&! <6z!ଵm 0 EDT* *p^."" xwݘA"�F7]Ӏ5qlwV4= (&mDQt9cnuVqEQn->M"-a:#Bb(~KcX,JP88XR1z]߁;Gw صi  It~-JlV>6MiZ|n}j-vhN�;;;""ennj>!m9}b~Ϧvhw0g!j5ن諴JE[Vb~~kF!5.|3`Z+I[ZZT+8NGeYXX(H-OrNr9�DEH=.eޣ\ wNu{,)]Ϗ?xyyE+ClC aV(g=ϋxޏ{)InfI~;& 1Ƥ[ZQ?%dԢ48;;+w^4U*_F[ܕLfvǯ@զB�\> fTcu P1)ZCs%m `0 }YYY[8LPrKEQEQEQEQ<?X����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/module.py��������������������������������������������������������0000664�0000000�0000000�00000005556�13034501105�0021277�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module from .browser import JacquieEtMichelBrowser from .video import JacquieEtMichelVideo __all__ = ['JacquieEtMichelModule'] class JacquieEtMichelModule(Module, CapVideo, CapCollection): NAME = 'jacquieetmichel' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = 'Jacquie et Michel TV' LICENSE = 'AGPLv3+' BROWSER = JacquieEtMichelBrowser def get_video(self, _id): with self.browser: video = self.browser.get_video(_id) return video def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): if not nsfw: return iter([]) with self.browser: return self.browser.search_videos(pattern) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(JacquieEtMichelVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest_nsfw']) if collection.split_path == [u'latest_nsfw']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest_nsfw']: collection.title = u'Latest Jacquie & Michel videos (NSFW)' return raise CollectionNotFound(collection.split_path) OBJECTS = {JacquieEtMichelVideo: fill_video} ��������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/pages.py���������������������������������������������������������0000664�0000000�0000000�00000005040�13034501105�0021075�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from weboob.deprecated.browser import Page, BrokenPageError from weboob.tools.misc import to_unicode from .video import JacquieEtMichelVideo class ResultsPage(Page): def iter_videos(self): for span in self.document.xpath('//ul[@id="list"]/li'): a = self.parser.select(span, 'a', 1) url = a.attrib['href'] _id = re.sub(r'/showvideo/(\d+)/.*', r'\1', url) video = JacquieEtMichelVideo(_id) url = span.find('.//img').attrib['src'] video.thumbnail = Thumbnail(url) video.thumbnail.url = video.thumbnail.id title_el = self.parser.select(span, 'h2', 1) video.title = to_unicode(title_el.text.strip()) video.description = self.parser.tocleanstring(span.xpath('.//div[@class="desc"]')[0]) video.set_empty_fields(NotAvailable, ('url,')) yield video class VideoPage(Page): def get_video(self, video=None): _id = to_unicode(self.group_dict['id']) if video is None: video = JacquieEtMichelVideo(_id) title_el = self.parser.select(self.document.getroot(), 'h1', 1) video.title = to_unicode(title_el.text.strip()) video.description = self.document.xpath('//meta[@name="description"]')[0].attrib['content'] for script in self.document.xpath('.//script'): if script.text is None: continue m = re.search('"(http://[^"]+.mp4)"', script.text, re.MULTILINE) if m: video.url = to_unicode(m.group(1)) break if not video.url: raise BrokenPageError('Unable to find URL') video.set_empty_fields(NotAvailable) return video ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/test.py����������������������������������������������������������0000664�0000000�0000000�00000003156�13034501105�0020763�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class JacquieEtMichelTest(BackendTest): MODULE = 'jacquieetmichel' def test_search(self): self.assertTrue(len(list(self.backend.search_videos('anus', nsfw=False))) == 0) l = list(self.backend.search_videos('anus', nsfw=True)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) self.backend.browser.openurl(v.url) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest_nsfw'])) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jacquieetmichel/video.py���������������������������������������������������������0000664�0000000�0000000�00000002064�13034501105�0021107�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class JacquieEtMichelVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.ext = u'mp4' self.nsfw = True @classmethod def id2url(cls, _id): return 'http://jacquieetmicheltv.net/showvideo/%s/t/' % _id ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016112�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001430�13034501105�0020221�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 dud # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import jcvelauxModule __all__ = ['jcvelauxModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000004474�13034501105�0020160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 dud # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import unicode_literals import datetime from urllib import urlencode from weboob.browser.browsers import APIBrowser __all__ = ['VelibBrowser'] class VelibBrowser(APIBrowser): ENCODING = 'utf-8' API_KEY = '2282a34b49cf45d8129cdf93d88762914cece88b' BASEURL = 'https://api.jcdecaux.com/vls/v1/' def __init__(self, api_key, *a, **kw): super(VelibBrowser, self).__init__(*a, **kw) self.api_key = api_key or VelibBrowser.API_KEY def do_get(self, path, **query): query['apiKey'] = self.api_key qs = urlencode(query.items()) url = '%s?%s' % (path, qs) return self.request(url) def get_contracts_list(self): return self.do_get('contracts') def get_station_list(self, contract=None): if contract: doc = self.do_get('stations', contract=contract) else: doc = self.do_get('stations') for jgauge in doc: self._transform(jgauge) return doc def get_station_infos(self, gauge): station_id, contract = gauge.split('.', 1) doc = self.do_get('stations/%s' % station_id, contract=contract) return self._transform(doc) def _transform(self, jgauge): jgauge['id'] = '%s.%s' % (jgauge['number'], jgauge['contract_name']) jgauge['city'] = jgauge['contract_name'] jgauge['last_update'] = datetime.datetime.fromtimestamp(jgauge['last_update'] / 1000) jgauge['latitude'] = '%s' % jgauge['position']['lat'] jgauge['longitude'] = '%s' % jgauge['position']['lng'] del jgauge['position'] return jgauge ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000004152�13034501105�0020247�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���tEXtSoftware�Adobe ImageReadyqe<�� IDATx[[lE @(B@A jbŠH$JĘJA%R|,D6h,-"5RQ- 86[ʐNlW#`ns_O?*豀n8ӣzVO�h|7R*\� L#A`@\3~CKmsp  �NO{'@Ff: p(9@x`_9"@(tHcWD=܂!D', }u':^#1q-3;E AY� � ϑ'7!y3p"_ӭkQy ǜڧ28ٌk RI8R%%E̹s@m#xl+8) N8]/�3ֻ}roj׹eI*6Te85xjnh0`jFk|vL՘( X�PyW7g8`2@f^"�Y+ �XҀpuhj04 '7;k"ܡSܚ8-= nsw�,|x}VX,rY_ިlʧ }[U@`h/ xH@ iT$� HIMFjV|>D!67fh6W ˬ4�Xɦ�FG nȡl_ 3;u N7XLƇ1L+ @,+oc<?: ?v%{/j- dE2'|Rf-p%Df`\j*6=~6md;yiGȾyGXVow[ zNoYdS FZ ɀFcҌb^A)*Q�w{]+7zcd{ܫJi43\r}Jf%P҄5`x0TR̎{#%E4&]z4-,xNqB0vb\WF=PFU5jekڔCgrktSc~9X852u>eb�so?i ^+Sz55{"<y5du�5iP`´[=4 .�YĂj(3´WxST\r< q>X%;6txqe) 픶~`@s&K`غnO UAtS>3B�(}xFy\25As}_Kbe`@> * oS^pq&Kj ".�<_izaS_S` fH)R@cH�/+IQ�lcKO-f55i0dE\> aPf _}~.wBިtGǛbCӤpVkd!JׅL^C{mۘM#:=` ͢Tio+ 68\ h0mlcT\Uh,đz"�`۵WbFSob.nLWG%j֦qň ~9Ҳ<gK {妇 qrhw>QKeruBJ+�W~aYozI)_AuW`^� }P")V}w)pIZ To1)q(X!嶂hz"I (wv|M/VĽ!HILf0pIR�?@nd\i*V6= . i'+M/V$ �m=KjSBOP(+\t�9 )E�R9O5MQ44AZדeaE�m y߳Ԓp{|Ņo/�𽺤5ֵ5@y9j`A@ �t ~x֫^/�mY9 ����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/module.py���������������������������������������������������������������0000664�0000000�0000000�00000014361�13034501105�0017756�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 dud # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import unicode_literals from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.base import StringField, UserError from weboob.capabilities.gauge import CapGauge, GaugeSensor, Gauge, GaugeMeasure, SensorNotFound from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.ordereddict import OrderedDict from .browser import VelibBrowser __all__ = ['jcvelauxModule'] SENSOR_TYPES = OrderedDict([('available_bikes', 'Available bikes'), ('available_bike_stands', 'Free stands'), ('bike_stands', 'Total stands')]) CITIES = ("Paris", "Rouen", "Toulouse", "Luxembourg", "Valence", "Stockholm", "Goteborg", "Santander", "Amiens", "Lillestrom", "Mulhouse", "Lyon", "Ljubljana", "Seville", "Namur", "Nancy", "Creteil", "Bruxelles-Capitale", "Cergy-Pontoise", "Vilnius", "Toyama", "Kazan", "Marseille", "Nantes", "Besancon") class BikeMeasure(GaugeMeasure): def __repr__(self): return '<GaugeMeasure level=%d>' % self.level class BikeSensor(GaugeSensor): longitude = StringField('Longitude of the sensor') latitude = StringField('Latitude of the sensor') class jcvelauxModule(Module, CapGauge): NAME = 'jcvelaux' DESCRIPTION = ('City bike renting availability information.\nCities: %s' % ', '.join(CITIES)) MAINTAINER = 'Herve Werner' EMAIL = 'dud225@hotmail.com' VERSION = '1.2' LICENSE = 'AGPLv3' BROWSER = VelibBrowser CONFIG = BackendConfig(Value('city', label='City', default='Paris', choices=CITIES + ("ALL",)), ValueBackendPassword('api_key', label='Optional API key', default='', noprompt=True)) def __init__(self, *a, **kw): super(jcvelauxModule, self).__init__(*a, **kw) self.cities = None def create_default_browser(self): api_key = self.config['api_key'].get() return self.create_browser(api_key) def _make_gauge(self, info): gauge = Gauge(info['id']) gauge.name = unicode(info['name']) gauge.city = unicode(info['city']) gauge.object = 'bikes' return gauge def _make_sensor(self, sensor_type, info, gauge): id = '%s.%s' % (sensor_type, gauge.id) sensor = BikeSensor(id) sensor.gaugeid = gauge.id sensor.name = SENSOR_TYPES[sensor_type] sensor.address = '%s' % info['address'] sensor.longitude = info['longitude'] sensor.latitude = info['latitude'] sensor.history = [] return sensor def _make_measure(self, sensor_type, info, gauge): id = '%s.%s' % (sensor_type, gauge.id) measure = BikeMeasure(id) measure.date = info['last_update'] measure.level = float(info[sensor_type]) return measure def _parse_gauge(self, info): gauge = self._make_gauge(info) gauge.sensors = [] for type in SENSOR_TYPES: sensor = self._make_sensor(type, info, gauge) measure = self._make_measure(type, info, gauge) sensor.lastvalue = measure gauge.sensors.append(sensor) return gauge def _contract(self): contract = self.config.get('city').get() if contract.lower() == 'all': contract = None return contract def iter_gauges(self, pattern=None): if pattern is None: for jgauge in self.browser.get_station_list(contract=self._contract()): yield self._parse_gauge(jgauge) else: lowpattern = pattern.lower() for jgauge in self.browser.get_station_list(contract=self._contract()): gauge = self._parse_gauge(jgauge) if lowpattern in gauge.name.lower() or lowpattern in gauge.city.lower(): yield gauge def iter_sensors(self, gauge, pattern=None): if not isinstance(gauge, Gauge): gauge = self._get_gauge_by_id(gauge) if gauge is None: raise SensorNotFound() if pattern is None: for sensor in gauge.sensors: yield sensor else: lowpattern = pattern.lower() for sensor in gauge.sensors: if lowpattern in sensor.name.lower(): yield sensor def get_last_measure(self, sensor): if not isinstance(sensor, GaugeSensor): sensor = self._get_sensor_by_id(sensor) if sensor is None: raise SensorNotFound() return sensor.lastvalue def _fetch_cities(self): if self.cities: return self.cities = {} jcontract = self.browser.get_contracts_list() for jcontract in jcontract: for city in jcontract['cities']: self.cities[city.lower()] = jcontract['name'] def _get_gauge_by_id(self, id): jgauge = self.browser.get_station_infos(id) if jgauge: return self._parse_gauge(jgauge) else: return None def _get_sensor_by_id(self, id): try: sensor_name, gauge_name, contract = id.split('.') except ValueError: raise UserError('Expected format NAME.ID.CITY for sensor: %r' % id) gauge = self._get_gauge_by_id('%s.%s' % (gauge_name, contract)) if not gauge: raise SensorNotFound() for sensor in gauge.sensors: if sensor.id.lower() == id.lower(): return sensor �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jcvelaux/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002236�13034501105�0017446�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 dud # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import unicode_literals from weboob.tools.test import BackendTest class VelibTest(BackendTest): MODULE = 'jcvelaux' def test_velib(self): l = list(self.backend.iter_gauges()) self.assertTrue(len(l) > 0) gauge = l[0] s = list(self.backend.iter_sensors(gauge)) self.assertTrue(len(s) > 0) sensor = s[0] self.assertTrue(self.backend.get_last_measure(sensor.id) is not None) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016057�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001436�13034501105�0020174�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import JirafeauModule __all__ = ['JirafeauModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000012310�13034501105�0020111�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import division from __future__ import unicode_literals import math import re from weboob.browser import PagesBrowser, URL from .pages import PageUpload, PageFile class JirafeauBrowser(PagesBrowser): BASEURL = 'https://jirafeau.net/' dl_page = URL(r'f.php\?h=(?P<id>[^&%]+)&d=1$') file_page = URL(r'f.php\?h=(?P<id>[^&%]+)$', PageFile) del_page = URL(r'f.php\?h=(?P<id>[^&%]+)&d=(?P<edit_key>[^&%]{2,})$') upload_page = URL('$', PageUpload) max_size = 0 age_keyword = { 60: 'minute', 3600: 'hour', 86400: 'day', 7 * 86400: 'week', 28 * 86400: 'month', } def __init__(self, base_url, *args, **kwargs): self.BASEURL = base_url super(JirafeauBrowser, self).__init__(*args, **kwargs) def recognize(self, url): match = self.dl_page.match(url) or self.file_page.match(url) or self.del_page.match(url) if match: _id = match.group('id') return { 'id': _id, 'url': self.file_page.build(id=_id) } elif re.match('[a-zA-Z0-9_-]+$', url): return { 'id': url, 'url': self.file_page.build(id=url), } def exists(self, _id): self.file_page.stay_or_go(id=_id) assert self.file_page.is_here() return self.page.has_error() def download(self, _id): if not _id.startswith('http'): _id = self.dl_page.build(id=_id) return self.open(_id).content def check_error(self, response): if response.text.startswith('Error'): raise Exception('Site returned an error: %s' % response.text) def post(self, contents, title=None, max_age=None, one_time=False): max_size, async_size = self.get_max_sizes() assert not max_size or len(contents) <= max_size if len(contents) < async_size: return self._basic_post(contents, title, max_age, one_time) else: return self._chunked_post(contents, title, max_age, one_time, async_size // 2) def _basic_post(self, contents, title=None, max_age=None, one_time=False): params = {} if one_time: params['one_time_download'] = 1 params['time'] = self.age_keyword[max_age] files = { 'file': (title or '', contents) } response = self.open('script.php', data=params, files=files) self.check_error(response) _id, edit_key = response.text.split() return { 'id': _id, 'edit_key': edit_key, 'page_url': self.file_page.build(id=_id), 'download_url': self.dl_page.build(id=_id), 'delete_url': self.del_page.build(id=_id, edit_key=edit_key), } def _chunked_post(self, contents, title=None, max_age=None, one_time=False, chunk_size=None): title = title or '' params = {} if one_time: params['one_time_download'] = 1 params['time'] = self.age_keyword[max_age] params['filename'] = title response = self.open('script.php?init_async', data=params) self.check_error(response) _id, edit_key = response.text.split('\n') chunk_size = chunk_size or (16 << 20) chunks = int(math.ceil(len(contents) / chunk_size)) i = 1 while contents: data, contents = contents[:chunk_size], contents[chunk_size:] params = { 'ref': _id, 'code': edit_key, } files = { 'data': (title, data), } self.logger.debug('uploading part %d/%d', i, chunks) response = self.open('script.php?push_async', data=params, files=files) self.check_error(response) edit_key = response.text.split('\n')[0] i += 1 params = { 'ref': _id, 'code': edit_key, } response = self.open('script.php?end_async', data=params) self.check_error(response) _id, edit_key, password = response.text.split('\n') return { 'id': _id, 'edit_key': edit_key, 'page_url': self.file_page.build(id=_id), 'download_url': self.dl_page.build(id=_id), 'delete_url': self.del_page.build(id=_id, edit_key=edit_key), } def get_max_sizes(self): self.upload_page.stay_or_go() assert self.upload_page.is_here() return self.page.get_max_sizes() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000006661�13034501105�0020223�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���������>a���sBIT|d��� pHYs����=2���tEXtSoftware�www.inkscape.org<�� .IDATxwXgǿ3 .,tH"U QTDE 1Qi!w<^iI.OE/<Xh4b/` h];?g_;-h4FzZ{ܸZ ˇi]rMFtuvb]<GepEZXHR�_`lߴ�ƽ;_lߴG} I™c0'M]VZDv:׮!BV!B�"@8h�#@8h�#@8h�#@8RHT9"CCxwI-:;I A?Ͻ l��hܳ6#;( '2%"2�q q@j!Xk=n%X Yk�%? ʴouc%d2h+J"X\n>ͭ�R8ű� KM'9VB@Ջ+!` kW8VBAO2I�A:Y*! 󜓛JkPA(xb& Ard$> l0 LZRsLæ^oBc'( z~dw�Z*P4{#u1uȬMUh�d:gWFiT\Z"ERn2~FŴtxg�X.(2S 1pr[ˏw Kw)ޠR1aK~lPhcZ:2\G=i5I#5Ed#$*ѠFidxe,ȕF_7jʟ1@yZ")28^@h�" fuR RX4y<r Lbi�R ɏ|FM]|VKro rI/N VԚ\@]Ӵ(Z]?Et`Ř+8*kٚ H>1iq$1.XGWOx:16HIZ�6Ws`o?P7JJj*1@ם;%h%u5[*1@M.* 1/ɢi &/~Ǔ81@IK MK0٥(Z› -G(B)xnp'-xceDPdo NnE$q~:"w\OHRƥ"-o,45Oݮ.Ҳ7w�g敒a4*'<^p'Rc40�EQ(rrAܗQ9Ag>  Qи42!yhL_bL�>(OZHE͊ P6Ei-?UKVA"wQ$f"<aI դe腘v=m!9#LjTN{�dr<ߡU"EZXU7�MKHKy @K(MS"3P0q&iYT~\W Nλy@r@~ .EaBZJ7h\A׆T&CU[pDZ � 7Jj^Z;8�cjxL >] 81@bvC6Ey DXZ@ӽkI1b,bnptdfh4 1Hݬ9cK TJk` rI9D*EU[Z2d o!(P8r5oQsjl Č_"W@?TĊ &[MAż :m0;,BL|:5�MKP&C ZdQ$s#.Ft`Vb3f�`H4‰<�MK0XY7ɘrn"bVrLY1W3c11KVS>6'8*O,+#}f iH >�ß|8f%*~b +ef0�CG.ɳ(Qo< @ ?EA"~əM6\i5)I (֯ۅqI� )K˵%h7: 6"Z#?oppݪ!1IOGl08scQ9| 59mg{g\n>ő};iIFbN!-2}+bThE&`럚$ ߅7?͎%W"9w2,vM,,{,ktża™chۭ7y7!W(2+k DzF}hV^:t~b..=j=.Hq^z sF&t¨;@9 _~ {zZq'џ@4=}!k ^Ӎ34 [ܧ5|֩V훱fXm<D Թ1D$f[`dWBf-7[[Nmٺ{nT&C`x' BxB|C‰1GMKVY XmXh.;O}oaՋQ?4'$np ʚu- [<r2 uަv9$-C'[qd·$(7}O_{h3�MK]d-%FF˅&\h>vb3fx] ;{{ > bCm%-m\WAV3S~=ϛ&ݼm^4w:vK^#$*11L@}[7o3ڕ Ļrz YT8Kn؁X\_|l#RDt@ag9E&W6vN�oX,'bElh^ScY_)J٣qZ isd#^<r-$;0ސY!FWtz%\n>cqwh?l#-S[o-8gG@DbF=yzZ ` ɝN^ǁ {g7D<NHa?+?дCS~{ƸE CEY^>N&-W:{ih\|^AfQ1IHmaZ[?�;!%}xD0z(lUIBZW/ΑaѤ3�#%(KT;,i;!82Em(_#����IENDB`�������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/module.py���������������������������������������������������������������0000664�0000000�0000000�00000005322�13034501105�0017720�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.paste import CapPaste, BasePaste from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from .browser import JirafeauBrowser __all__ = ['JirafeauModule'] class JirafeauPaste(BasePaste): @property def page_url(self): return self.url class JirafeauModule(Module, CapPaste): NAME = 'jirafeau' DESCRIPTION = u'Jirafeau-based file upload website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('base_url', label='Base Jirafeau URL', description='URL of the Jirafeau-based site to use', regexp=r'https?://.*', default='http://jirafeau.net/')) BROWSER = JirafeauBrowser def create_default_browser(self): return self.create_browser(self.config['base_url'].get()) def can_post(self, contents, title=None, public=None, max_age=None): if public or max_age not in self.browser.age_keyword: return 0 max_size, _ = self.browser.get_max_sizes() if max_size and len(contents.decode('base64')) > max_size: return 0 return 1 def get_paste(self, url): d = self.browser.recognize(url) if not d: return if self.browser.exists(d['id']): return ret = JirafeauPaste(d['id']) ret.url = d['url'] return ret def new_paste(self, *args, **kwargs): return JirafeauPaste(*args, **kwargs) def post_paste(self, paste, max_age=None): d = self.browser.post(paste.contents.decode('base64'), paste.title, max_age) paste.id = d['id'] paste.url = d['page_url'] return paste def fill_paste(self, obj, fields): if 'contents' in fields: data = self.browser.download(obj.id) obj.contents = data.encode('base64') OBJECTS = {JirafeauPaste: fill_paste} ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000003364�13034501105�0017536�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage from weboob.tools.misc import get_bytes_size class PageUpload(HTMLPage): def get_max_sizes(self): max_size = None for item in self.doc.getroot().cssselect('.config'): if not item.text: continue match = re.search(r'File size is limited to (\d+) ([A-Za-z]+)', item.text) if match: max_size = int(get_bytes_size(int(match.group(1)), match.group(2))) break async_size = 16 * 1024 * 1024 for item in self.doc.xpath('//script'): if not item.text: continue match = re.search(r'upload \(.*, (\d+)\)', item.text) if match: async_size = int(match.group(1)) break self.logger.debug('max size = %s, max part size = %s', max_size, async_size) return max_size, async_size class PageFile(HTMLPage): def has_error(self): return bool(self.doc.getroot().cssselect('.error')) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jirafeau/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000003637�13034501105�0017421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import unicode_literals import os from weboob.tools.test import BackendTest class JirafeauTest(BackendTest): MODULE = 'jirafeau' def test_jirafeau(self): data = os.urandom(1 << 10) assert self.backend.can_post(data.encode('base64'), title='yeah.random', max_age=60) assert self.backend.can_post(data.encode('base64'), title='yeah.random', max_age=60, public=False) assert not self.backend.can_post(data.encode('base64'), title='yeah.random', max_age=60, public=True) assert not self.backend.can_post(data.encode('base64'), title='yeah.random', max_age=False) paste = self.backend.new_paste(None, contents=data.encode('base64'), title='yeah.random') self.backend.post_paste(paste, max_age=60) self.assertTrue(paste.id) self.assertTrue(paste.page_url) fetched = self.backend.get_paste(paste.url) self.assertTrue(fetched) self.assertEqual(fetched.id, paste.id) fetched = self.backend.get_paste(paste.id) self.assertTrue(fetched) self.backend.fillobj(fetched, 'contents') self.assertTrue(fetched.contents) assert fetched.contents == data.encode('base64') �������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015731�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0020044�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Alexandre Lissy # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import JVMalinModule __all__ = ['JVMalinModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003764�13034501105�0020000�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Alexandre Lissy # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from .pages import RoadmapSearchPage, RoadmapResultsPage, RoadmapPage, RoadmapAmbiguity __all__ = ['JVMalin'] class JVMalin(Browser): DOMAIN = 'www.jvmalin.fr' PAGES = { 'http://www\.jvmalin\.fr/Itineraires/Recherche.*': RoadmapSearchPage, 'http://www\.jvmalin\.fr/Itineraires/Precision.*': RoadmapResultsPage, 'http://www\.jvmalin\.fr/route/vuesearch/result.*': RoadmapPage } def __init__(self, **kwargs): Browser.__init__(self, '', **kwargs) def get_roadmap(self, departure, arrival, filters): self.location('/Itineraires/Recherche') assert self.is_on_page(RoadmapSearchPage) self.page.search(departure, arrival, filters.departure_time, filters.arrival_time) assert self.is_on_page(RoadmapResultsPage) dest = '' try: dest = self.page.find_best() except RoadmapAmbiguity: self.page.resubmit_best_form() assert self.is_on_page(RoadmapResultsPage) dest = self.page.find_best() self.location(dest) roadmap = {} roadmap['steps'] = list(self.page.get_steps()) return roadmap def is_logged(self): """ Do not need to be logged """ return True ������������weboob-1.2/modules/jvmalin/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000021343�13034501105�0020067�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME 4#���iTXtComment�����Created with GIMPd.e��"GIDATx{idWuw}[*}f-Hh0RЀ`=^�vxclcB $%ګ2+{Gf7XLoċ*2==w;?Olld~}m#7/{DP5@wFQy ؙDp쓌Mv zlw':\kѸe9`0~C7E<8l}'^|Wx' w�{z[uZn7ҰJF>7;Y\,B0#xs ز 4.ًI@s705oߝ\"|={�\3=k#VoͺmX2Db'ڴaX5T* jˇ>})x�?_xk0Xt;&]NJP{畊܅yLM}�?`]s_^`qSOȆ~Ӵ\mDf &-kUUUJ<yG`7+y'A$vS. vm0їI%;lH  �hbVsSgf@UEYMw� l˔W޸ahh].+6z{RsJ4a 3swP# =YoTd2c./͇ )L^f5yfoBܞ9G{F\ʍl+L%suB'T{ζsx%^�~io]}Gn GiB+�Opy�t2Ce. 1)/g"_9Gm&Hv ]3v$ݖkᮎ!x<E튘@*b)R 'E?(sx8�6ޑ�o|֞ / ڲ."o�[z]Z�`C;�≶@Џ z}s= -^^oVO$24L� 6M  6�HӰC@zqM$V39^!�'`Ս&Wx\Gwdž;d"i`m&�.�c;$b)?�gӈw G/ۦmфl"�pH37x[Vf;zcC?l|\! ڸ{N?HZZ#`K/ޙ�BNJ�%(ѭRmلo.w#7drCm]0п^i\njP 6m&�[ ku@p{G ㆵ#�'~H? oްn+=aj %›ϝJɞmo6IJiLMlVk2@3nܛ90 h]�l�FvY= zq�$̆aXE+aĺZYv4sW8DUppѶ=GWLhMKkzk90pEq%CkRqef(�Fč .KbxƦk׽Fe3] & )\OE&bZѬ RɎZoeEZѐ RH"Z<M XUo\~G6L% B^F`FJAy%j71t5ShE H��EE3s4.蝿_i``L�x=@1�v;szLިF Y#Kb7F"1T2b�L$t\h /aD'>2qgQפ3VˑݻUJňʌ'R<:ٻʛk_soFVmЖ T 8~6oBkl6'4�Y�/:�kfL{K86 SJ,O~see)J$$�Y/ ē&66ہ׮^ym*vw(`1uba1X~{}ȘEf`2VwO_00vvV-bޙp&6z�"CZ2;2]Z$4/.Nf4kD&HƳ: CJKV0gK!U8}@ǖM I`Ӳ#=l;H?8 j鎾ի7ԴEK'^6;;W*4.ʆ6YfJAe2|7Pk>}(!Z͸fɎqI�m]>vbYiˋZkt޺ѝ ,̲՝mRdD[@5qz_ڔ\u#1pJ+=wl_V A <H@ܶf wJqPJTd&;̶_{gkWq�&M7ng$Ƒ 8qqra!� .;GJ�3y"zJ�Sx <`rumC#V-O02 GV� B�G. 3(!u zi3�*aQTb2D#Đ_ǃF(�`>?~l[[M,ږ+}|hn L;Hr>Efqe%�0 dX_-ؽoDR]N,�,)aLN K) a|g4 IEB`0P(B AxZɐ/ @Y/0B3L^nji۳k.;)HWyBlG$Hk%b�&,?uۯL7 �{3Htw ) mM]8B24A11qWPѻڰ_JF E VJ *H& 4 $zVOo'�X?zGB�V%")h&�T3B^jQIũ󧁏@cbd|H=x�-omޕ;I`0 H"0={kԵ_+/P)pQEuYY,hPRzvs*�AkF} |)#E9+ c;ȇ5c75C3�5D80XE*{i a@߷yo;iިeZ<ЗRP $PuZ -4+AKR9/t_^!2'#/-W &� l]L$�ffqbR)Jf%�%bBcoF[w{tL>F´L P*4/D${ZAA, ? d:Ů]꾿Czl;917;)ЊRaX,`kҔTHoͤ5)gDy"YL0'<--Zy� Ӵ0D q_CTFdvt$~L>f]d"N)%97\7 �$$}VQt["BL;Z��7-Ld�eiMe)U cGs5� ՖC }:uɬ)lWaAi% iaq1Aa�<bIi#K}K+ȰD4Oip-d& HĀ<97w\s Ci*̚@*45KM,B(|a&FDӳ=<�`o)Ѓİe LSO-DRT$'H !Қlp=;EB'tzWy'y{uZHÌEh sQ-D�*YiXo?a� !֎7о׀P[qS[C H Mud+ˮ$D2AZ�tQ[lxIuBk%{M" RgW5XϱY"�0 x<Z_tnIGcm)׍�AR08QM<AxJxI-ineFPMD}iE KqT[L)Ʌ G 7糙0kjBJX+8qT@ ":u{l3�ҶT;镹(�D2EAʊa&̗Hk0 ىDyæ˛�m!H(�:H >riεL&ۈ(uA_qsiZl<=/<T+BkMTbf"".W ;x.gdFr>-_Iŋ<| 4Y#n�'[Ѹr1E Cuï+ٮ@*~`jδwBBgڶ+Tq)?y}.#BSG_|FTɣfMT*ryY?eS3zeg>��Vl)aێ彺 Ua̚MV&fnML۽`֫[o{{^%j2P"f�!TZ_Xwcn0XsQ$cG�^{ `І=j--ϖ4TTgo& 5 G3)"/oR^AwVd%7櫘�aZX��ÀzozGkn.k糾!hذa\ ,V)k&(\!Ed̼ 5�rNpxǛuF<ڦZ\T/�8[%GoŘ6˴m׮U}sͯb�ApFH�e, COZtЖJa9AQOiմ0_4ahuTB򵒡 O fn\1 `HwX�@)dGsChnUK#ryLqeYv>k$>XX\Cm׉3AUl24?�%) 3)$"x�0 W(j&<P�lSf:]� �:pnn²bHJZd2k'ڪ!jGd"c>|~zl-߽ 9py_#r%k 'eȣzr7Ȗ4Bi$oW$CG�ʢ/H"`/yfp&驳N\$)Ьh aX=Y.D+6=E~7Lf "A7'߯}=uNM(� ^h[X&+s瞚9dc~x`\:�̼ɐr{WRaǍa {<Ό2( CzvCmⲴL$PK>?m tt��8?j:V=r+0 aI0 !0fF# W[R~bk<@ !㧞{,K?#Ji"aKk* ̤:ByZEϾcq~oNGQH&;m++lT0}j]Ӊd㋋gK+ �Haؐ04Uv2V5vA$( } @ȶ#LR(^<]W8 (c'K_�AX\hTW kղR~jx7 2?#֎_ aV+5e9GG;ڝW;}pBȢ8~Oۀe65N?R6xn(*4V^RKRHDdV+8?qdcs<sېҘcWJ ;zىZm}y PW Mc]IFO7~gmGUx syyκ@T {FFI|"D`8;oXeٲ!u40K Zt3%Yk(u)=g) M�hDoΞ?ty\oH63Ȕ+ŕی4rź@�x//تzm񏾩rzx[ץqw9o3 �c VםSehARLh58Qn�:s 瓡A7;~; 'l;ri'h �pɈ Z79urfmgiՔ{^z�0=R*<ԩ?s@вK\ָX2paUNW<񭩇xl#57i DYVlj4ܑsC̩54" ,:k+? o?s>z:J6UDq|+Cۍ[g͹ch|Dh2MR5y{0 Y~K^yzbe_zۖ V(WLo<%hkJiǦgN܋䬳}_sP aᇅe9]/nB"*Dݶr VB3qgϯzgf 'y�5j-'3C>عsl)EytSxӏ>  3]7 :dR9o*6[5R mT +_}m;?_w;wIi֩S7hhroת%U=Dx?uS7��©Y*IibU~Xf@2CZ-|ǒ3g-"d{Fm;X%<=/%6凍ͦa+׍{JîVWdZXZR-TY3Iw1do\�#sGnڠiXM QZ�*ˣہ3g#Ŏ9B*< Ei:5}<|offTJU#n 'o9l,|Qխj&p  |X w1F�]q-}T+V4T B2(qb4qgOvFm3O:WnS P8c>u( 0" z43>ͬ:Vc|nHB@3tK%:BVԭ.�NcUH=ċ'\{kHRkl2Qb:f>jz <%෮p#pĭALDq%>wH}_feFgjOTTi_a|6~u pJi+H, umBą$>ɵk2� bhj*"EJ3GgǕ _OB{;@h'ʷû/�ʩ,t`v�A"P0`jx~ÓUٸw9煿J-m$Ғ:atlolkcɀp#q]XY'YfWi'PBsgמ8ut9k]bߺy)ej[n/ޱ8![e`] o4%HI 0fC-;R =? P/^Μz~^g};F~.eFd-f'O)` RHRjCZ3C98 !mo[&P2 {MOWvB%`k?Dax~ĶL\kz0AM_!<�Fϴ/,MݝÁbe!r&/LȨ00�h:R@�i" m14a-^buꚨ^2ޟyw?T]mt&nXLdžW aH! XOEgN'~40 DmɎRwH4:kUg`E3ԥc7vA?~ӿfw|gjp5}ck{s#: "CdR뵊5Dڱ]/3k.T.3#_OM>jwf&wu_l0k0J& r|1XVL&ڃd<̑@ @k��A\aBrtqx޻>{u�4|"Z/`ox�|0<�k~ 7gTAhMũg}ȿɏ˓&86{c+ ap<[9@"v@!A+ �jFF:3yj'/L7?(��1 '=l4_^]iOJS3n�Bf/LS'f?xW?7ӿ;�,6鬒dJ$HY#90 $ !Q@ @Ij(6/U!?sZ+<pҺuſw:77Dy%XO,*֎mhKH$Y-rJ|o/zӿ'q*FظrfDd'JJiX.0`ప}gTp}$�{qw>9_l LчX4%l7 $b( _+%8xG'~3A8A teP>ob(fe?=V&����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/module.py����������������������������������������������������������������0000664�0000000�0000000�00000003216�13034501105�0017572�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Alexandre Lissy # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.travel import CapTravel, RoadStep from weboob.tools.backend import Module from .browser import JVMalin __all__ = ['JVMalinModule'] class JVMalinModule(Module, CapTravel): NAME = 'jvmalin' MAINTAINER = u'Alexandre Lissy' EMAIL = 'github@lissy.me' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"Multimodal public transportation for whole Région Centre, France" BROWSER = JVMalin def iter_roadmap(self, departure, arrival, filters): with self.browser: roadmap = self.browser.get_roadmap(departure, arrival, filters) for s in roadmap['steps']: step = RoadStep(s['id']) step.line = s['line'] step.start_time = s['start_time'] step.end_time = s['end_time'] step.departure = s['departure'] step.arrival = s['arrival'] step.duration = s['duration'] yield step ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000020204�13034501105�0017400�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- import re import datetime from weboob.capabilities.travel import RoadmapError from weboob.tools.misc import to_unicode from weboob.deprecated.mech import ClientForm from weboob.deprecated.browser import Page class RoadmapAmbiguity(RoadmapError): def __init__(self, error): RoadmapError.__init__(self, error) class RoadmapSearchPage(Page): def search(self, departure, arrival, departure_time, arrival_time): match = -1 i = 0 for form in self.browser.forms(): try: if form.attrs['id'] == 'rech-iti': match = i except KeyError: pass i += 1 self.browser.select_form(nr=match) self.browser['Departure'] = departure self.browser['Destination'] = arrival time = None if departure_time: self.browser['sens'] = ['1'] time = departure_time elif arrival_time: self.browser['sens'] = ['-1'] time = arrival_time if time: try: self.browser['dateFull'] = '%02d/%02d/%d' % (time.day, time.month, time.year) self.browser['hour'] = ['%02d' % time.hour] self.browser['minute'] = ['%02d' % (time.minute - (time.minute % 5))] except ClientForm.ItemNotFoundError: raise RoadmapError('Unable to establish a roadmap with %s time at "%s"' % ('departure' if departure_time else 'arrival', time)) self.browser.submit() class RoadmapResultsPage(Page): def html_br_strip(self, text): return "".join([l.strip() for l in text.split("\n")]).strip().replace(' ', '%20') def find_best(self): if len(self.parser.select(self.document.getroot(), 'img.img-error')) > 0: if len(self.parser.select(self.document.getroot(), 'form#iti-ambi')) > 0: raise RoadmapAmbiguity('Ambigious stop name') else: raise RoadmapError('Error when submitting form') best = self.parser.select(self.document.getroot(), 'div.alerte-bloc-important div.bloc-iti') if len(best) == 0: best = self.parser.select(self.document.getroot(), 'div.bloc-iti') if len(best) == 0: raise RoadmapError('Unable to get the best roadmap') link = self.parser.select(best[0], 'a.btn-submit') if len(link) == 0: raise RoadmapError('Unable to get a link to best roadmap') return self.html_br_strip(link[0].attrib['href']) def resubmit_best_form(self): if len(self.parser.select(self.document.getroot(), 'img.img-error')) == 0: raise RoadmapError('No error reported!') ambi = None i = 0 for form in self.parser.select(self.document.getroot(), 'form'): if 'id' in form.attrib and form.attrib['id'] == 'iti-ambi': ambi = form break i += 1 if ambi is None: raise RoadmapError('No ambigous form!') props = self.parser.select(ambi, 'span.precision-arret input') if len(props) == 0: props = self.parser.select(ambi, 'span.precision-adresse input') if len(props) == 0: raise RoadmapError('Nothing to select to get a roadmap') self.browser.select_form(nr=i) propname = props[0].attrib['name'] propvalue = props[0].attrib['value'].encode('utf-8') self.browser[propname] = [ propvalue ] self.browser.submit() class RoadmapPage(Page): def get_steps(self): errors = [] # for p in self.parser.select(self.document.getroot(), 'p.errors'): # if p.text: # errors.append(p.text.strip()) if len(errors) > 0: raise RoadmapError('Unable to establish a roadmap: %s' % ', '.join(errors)) current_step = None i = 0 for tr in self.parser.select(self.document.getroot(), 'table.itineraire-detail tr'): if current_step is None: current_step = { 'id': i, 'start_time': datetime.datetime.now(), 'end_time': datetime.datetime.now(), 'line': '', 'departure': '', 'arrival': '', 'duration': datetime.timedelta() } if 'class' in tr.attrib: if 'bg-ligne' in tr.attrib['class']: continue if 'iti-map' in tr.attrib['class']: continue for td in self.parser.select(tr, 'td'): if 'class' not in td.attrib: continue if 'iti-inner' in td.attrib['class']: continue if 'cell-infos' in td.attrib['class']: if 'id' in td.attrib: if td.attrib['id'].find('MapOpenLink') >= 0: hasA = self.parser.select(td, 'a') if len(hasA) == 0: if len(current_step['line']) > 0 and \ len(current_step['departure']) > 0 and \ len(current_step['arrival']) > 0: current_step['line'] = to_unicode("%s : %s" % (current_step['mode'], current_step['line'])) del current_step['mode'] yield current_step i += 1 current_step = None continue if 'cell-horaires' in td.attrib['class']: # real start for heure in self.parser.select(td, 'span.heure'): if heure.attrib['id'].find('FromTime') >= 0: current_step['start_time'] = self.parse_time(heure.text) if heure.attrib['id'].find('ToTime') >= 0: current_step['end_time'] = self.parse_time(heure.text) for mode in self.parser.select(td, 'span.mode-locomotion img'): current_step['mode'] = mode.attrib['title'] if 'cell-details' in td.attrib['class']: # If we get a span, it's a line indication, # otherwise check for id containing LibDeparture or # LibDestination spans = self.parser.select(td, 'span.itineraire-ligne') if len(spans) == 1: line = self.html_br_strip(spans[0].text, " ").replace('Ligne ', '') if line.index('- ') == 0: line = re.sub(r'^- ', '', line) current_step['line'] = line elif 'id' in td.attrib: stops = self.parser.select(td, 'strong') stop = self.html_br_strip(stops[0].text, " ") if td.attrib['id'].find('LibDeparture') >= 0: current_step['departure'] = to_unicode(stop) if td.attrib['id'].find('LibDestination') >= 0: current_step['arrival'] = to_unicode(stop) duree = self.parser.select(td, 'span.duree strong') if len(duree) == 1: current_step['duration'] = self.parse_duration(duree[0].text) def html_br_strip(self, text, joining=""): return joining.join([l.strip() for l in text.split("\n")]).strip() def parse_time(self, time): time = self.html_br_strip(time) h, m = time.split('h') return datetime.time(int(h), int(m)) def parse_duration(self, dur): dur = self.html_br_strip(dur) m = re.match('(\d+)min', dur) if m: return datetime.timedelta(minutes=int(m.group(1))) m = re.match('(\d+)h(\d+)', dur) if m: return datetime.timedelta(hours=int(m.group(1)), minutes=int(m.group(2))) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/jvmalin/test.py������������������������������������������������������������������0000664�0000000�0000000�00000005245�13034501105�0017270�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Alexandre Lissy # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime from weboob.capabilities.travel import RoadmapFilters from weboob.tools.test import BackendTest class JVMalinTest(BackendTest): MODULE = 'jvmalin' def test_roadmap_cities(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('Tours', 'Orléans', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_stop_intercity(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('Tours Jean-Jaurès', 'Orléans', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_stop_intracity(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('Tours Jean-Jaurès', 'Polytech Tours', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_stop_intracity2(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('J.P.Rameau', 'Polytech Tours', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_names(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('Artannes Mairie', 'Château de Blois', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_long(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('Chartres', 'Ballan-Miré', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_departure(self): filters = RoadmapFilters() filters.departure_time = datetime.datetime.now() + datetime.timedelta(days=1) roadmap = list(self.backend.iter_roadmap('Chartres', 'Ballan-Miré', filters)) self.assertTrue(len(roadmap) > 0) def test_roadmap_arrival(self): filters = RoadmapFilters() filters.arrival_time = datetime.datetime.now() + datetime.timedelta(days=1) roadmap = list(self.backend.iter_roadmap('Chartres', 'Ballan-Miré', filters)) self.assertTrue(len(roadmap) > 0) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015721�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000000077�13034501105�0020036�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import KickassModule __all__ = ['KickassModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003231�13034501105�0017755�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, TorrentPage __all__ = ['KickassBrowser'] class KickassBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'https://kat.cr/' search = URL('usearch/(?P<pattern>.*)/\?field=seeders&sorder=desc', SearchPage) torrent = URL('torrent-t(?P<id>.*).html', '.*-t[0-9]*\.html', TorrentPage) def iter_torrents(self, pattern): self.search.go(pattern=pattern) #print( self.page.content) return self.page.iter_torrents() def get_torrent(self, fullid): try: self.torrent.go(id=fullid) torrent = self.page.get_torrent() return torrent except BrowserHTTPNotFound: return �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000004535�13034501105�0020063�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME ��IDATxkl{}kq&(` * "RQR&HIūQʣ*X4%<Jxpy8`ď뻻wwÝ.q\dT7ݿ4ڝ9s $H A $H ;<X@LyB\R3b:4XCN B 3抳^2x&'1KI-c7 !- xq(n+R7/J5߂9G�<�88 | 8GJ"&va 1Vp jʮJ�.V4e8NZ_4GdSVY(/=-7ذ4Cne[l<|950f)-#n*V"G7 ̕xR ;Lv42Q;Ǧ)avHWLI#s?z `//19!b#[.lJ*FEJ$dûXeCt)~sp+wX[r*0`T%Y:4(K1"�2up6O_ꯙϩdM<YImaB{D!%A/\%wh9+q_W@SBe,!zKF7^e]= d&6o݌{=WSMY: |9MK,AD�d2kjjVZy0r"Wp\C㱂2̣H9ilhh`ժUY}ʹsR(.k H):1;u7-�C.}u? g!0Ky@xpppDf477g?Kjkkimmw2ܯ9Rq>P?'cn(-% u` *,^&3- M56H\n!p7qn:o ?dJeX_%N@X фh;DGѹuii3e`$sr)PcZ~g, Dc-^۷ˋ9+2͸i"D{~Qj`2�536`+:ﲌ[)u.'X T[U R|i>,R%=fr*C<t:=J5kt.ZS8c" 6ȓ2?k죇}bL#.YFzg %svR#+fכzvmyZ+DwAvȵl`yWcH?0'7` pDb@Lb�C R @#XH"Ot p#8XlI .?*54;mk cilWz]{(;.xZfg||\hlٲ3co>PFG\#(RGy4m>8hxrԄmWW�J h2դAb )5ᦾbBdKn?2uJ  ;Sk MMM7o>~&:Ϙ$#Ìe?:?Fw d^"t9A֜QIq2K-JaWAJ)U@-TR*cP½t L7 c-`KHacER�"AS]y(}2� <fL.7ދL*7+GWJMMB@)TVSB| n7Jϳ~ hi%Pd{ 4&-*Usggfږ2e6uh K zWޟH�}H *&gJ,ge6.h2w0*aA)ĥ@Dk%$ns>}RPPQ 򁖖>7 ;�i�m9  :ɻf: ?)ȈPS7ൽJFd<<n |<C9!C8ΏuO.:4@UXpr9&-hT!j[ާ-_@<i_!lm̱^zyQvlf̓&mEV/XO'L_?Kuys-6-ݱEhGv*<5Zkc|#G $H A $H /Sf����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/module.py����������������������������������������������������������������0000664�0000000�0000000�00000004146�13034501105�0017565�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.torrent import CapTorrent, Torrent from weboob.tools.backend import Module from .browser import KickassBrowser from urllib import quote_plus #from contextlib import closing #from gzip import GzipFile __all__ = ['KickassModule'] class KickassModule(Module, CapTorrent): NAME = 'kickass' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Kickass Torrents BitTorrent tracker' LICENSE = 'AGPLv3+' BROWSER = KickassBrowser def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None resp = self.browser.open(torrent.url) #headers = response.info() #if headers.get('Content-Encoding', '') == 'gzip': # with closing(GzipFile(fileobj=response, mode='rb')) as gz: # data = gz.read() #else: # data = response.read() return resp.content def iter_torrents(self, pattern): return self.browser.iter_torrents(quote_plus(pattern.encode('utf-8'))) def fill_torrent(self, torrent, fields): if 'description' in fields or 'files' in fields: torrent = self.browser.get_torrent(torrent.id) return torrent OBJECTS = { Torrent: fill_torrent } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000010431�13034501105�0017371�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Julien Veyssier, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.tools.misc import get_bytes_size from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText, Type class SearchPage(HTMLPage): @method class iter_torrents(ListElement): item_xpath = '//table[has-class("data")]//tr[@class="odd" or @class="even"]' class item(ItemElement): klass = Torrent obj_id = Regexp(CleanText('.//div[@class="torrentname"]//a[@class="cellMainLink"]/@href'), '.*-t([0-9]*).html') obj_name = CleanText('.//a[@class="cellMainLink"]', default=NotAvailable) obj_magnet = CleanText('.//div[has-class("iaconbox")]//a[starts-with(@href,"magnet")]/@href', default=NotAvailable) obj_seeders = CleanText('.//td[has-class("green") and has-class("center")]', default=NotAvailable) & Type(type=int) obj_leechers = CleanText('.//td[has-class("red") and has-class("center")]', default=NotAvailable) & Type(type=int) obj_description = NotLoaded obj_files = NotLoaded def obj_url(self): href = CleanText('.//div[has-class("iaconbox")]//a[starts-with(@href,"//")]/@href')(self) return 'https:%s'%href def obj_size(self): rawsize = CleanText('./td[2]')(self) rawsize = rawsize.replace(',','.') nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper() size = get_bytes_size(nsize,usize) return size obj_filename = CleanText(Regexp(CleanText('.//div[has-class("iaconbox")]//a[starts-with(@href,"//")]/@href'), '.*title=(.*)'), default=NotAvailable) class TorrentPage(HTMLPage): @method class get_torrent(ItemElement): klass = Torrent obj_description = CleanText('//div[@id="desc"]', default=NotAvailable) obj_seeders = CleanText('(//div[has-class("seedBlock")]/strong)[1]') & Type(type=int) obj_leechers = CleanText('(//div[has-class("leechBlock")]/strong)[1]') & Type(type=int) obj_name = CleanText('//h1[has-class("novertmarg")]//span', default=NotAvailable) obj_magnet = CleanText('//div[has-class("downloadButtonGroup")]//a[starts-with(@href,"magnet")]/@href', default=NotAvailable) obj_id = Regexp(CleanText('//h1[has-class("novertmarg")]/a/@href'), '.*-t([0-9]*)\.html') def obj_url(self): href = CleanText('//div[has-class("downloadButtonGroup")]//a[starts-with(@href,"//")]/@href')(self) return u'https:%s'%href def obj_size(self): rawsize = CleanText('//span[has-class("folder") or has-class("folderopen")]')(self) rawsize = rawsize.split(': ')[-1].split(')')[0].strip() rawsize = rawsize.replace(',','.') nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper() size = get_bytes_size(nsize,usize) return size def obj_files(self): res = [] for f in Type('//td[has-class("torFileName")]', type=list)(self): res.append(CleanText(f)(self)) return res obj_filename = CleanText(Regexp(CleanText('//div[has-class("downloadButtonGroup")]//a[starts-with(@href,"//")]/@href'), '.*title=(.*)'), default=NotAvailable) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kickass/test.py������������������������������������������������������������������0000664�0000000�0000000�00000003550�13034501105�0017255�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded import urllib from random import choice class KickassTest(BackendTest): MODULE = 'kickass' def test_torrent(self): torrents = list(self.backend.iter_torrents('debian')) for torrent in torrents: path, qs = urllib.splitquery(torrent.url) assert path.endswith('.torrent') if qs: assert torrent.filename assert torrent.id assert torrent.name assert torrent.description is NotLoaded full_torrent = self.backend.get_torrent(torrent.id) # do not assert torrent.name is full_torrent.name # (or even that one contains another), it isn't always true! assert full_torrent.name assert full_torrent.url assert full_torrent.description is not NotLoaded # get the file of a random torrent # from the list (getting them all would be too long) if len(torrents): torrent = choice(torrents) self.backend.get_torrent_file(torrent.id) ��������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016070�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/README.md���������������������������������������������������������������0000664�0000000�0000000�00000000236�13034501105�0017350�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# [weboob-kiwibank](https://github.com/infertux/weboob-kiwibank) [Kiwibank](http://www.kiwibank.co.nz/) module for the [Weboob](http://weboob.org/) project ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020200�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Cédric Félizard # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import KiwibankModule __all__ = ['KiwibankModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000003427�13034501105�0020133�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Cédric Félizard # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountPage, HistoryPage __all__ = ['Kiwibank'] class HistoryUnavailable(Exception): pass class Kiwibank(LoginBrowser): BASEURL = 'https://www.ib.kiwibank.co.nz/mobile/' TIMEOUT = 30 login = URL('login/', LoginPage) login_error = URL('login-error/', LoginPage) accounts = URL('accounts/$', AccountPage) account = URL('/accounts/view/[0-9A-F]+$', HistoryPage) def do_login(self): self.login.stay_or_go() self.page.login(self.username, self.password) if self.login.is_here() or self.login_error.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts(self): self.accounts.stay_or_go() return self.page.get_accounts() @need_login def get_history(self, account): if account._link is None: raise HistoryUnavailable() self.location(account._link) return self.page.get_history() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000003142�13034501105�0020223�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD�h��i_H��� pHYs�� �� ����tIME 0ś`���iTXtComment�����Created with GIMPd.e��IDATx{LSWǿ>n^˫"%B'2$Aӱe4ӱmfmn&{gq˒mL]dqqht KT[@zy}GF[.%={ιs~ReG4Lc1͍� ��� ��� ��� ��� �3)fm(lD)&@J5hYz!ǃI(UC򑞐?E[@)(f^lC1L@- Ű @CYл$"ft 4*`=/L]^hG$nvU=!9ݘ�J@�jVV&k,{l6ǘs>_}[f<^* =FB x~Z|M�x6YSQI��W.BCcUbPRH"VcjbxtyE xR -8p$PUZbfj82 xhDh؄1BTHGiA+2_D ]=XVՖ5e}`ׁ饈S,z[C4l"\=V'KT(4oGQHh9<>'*?i  +w+=&!ƄKBm xz>gVm?_FC[hhcjgq۟Ըi1j`E`*3MK՘&Z60$EmT.z1l> @΢_=^&p霭8X(,2m_6AN?ly<FuN ˸S}#Yӷ9^pV_ K#tJ=( h#&wg@)*{ט;;ta b31(܎ pJ#8L4߯Au󑩽zzT(+HUXcfp–l8V el q^'jM\^Gt�xWAS9<JY1Vys:>pJ#\i:2{;;aݼ�CKE) <|pӽhKwʱ<,dZ$hjV*UqXQ6~[^'^D=f6SBT)c ܘlfPIl|x=#'qT7Q{(8NO׭&l59xPg=�ia+5쟭ˍxd6:|!5Yt6b}}Y"¢Ǡqc^bD/@ ⷄHqvg5;ZZk8}c?N z~*4ńT{ppkVV4vE,nGk Xi tdN!(wIRԃ Q^wl@Ns#��� ��� ��� ��� ��`zߦPs'����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/module.py���������������������������������������������������������������0000664�0000000�0000000�00000003517�13034501105�0017735�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Cédric Félizard # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Kiwibank __all__ = ['KiwibankModule'] class KiwibankModule(Module, CapBank): NAME = 'kiwibank' MAINTAINER = u'Cédric Félizard' EMAIL = 'cedric@felizard.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Kiwibank' CONFIG = BackendConfig( ValueBackendPassword('login', label='Access number', masked=False), ValueBackendPassword('password', label='Password'), ) BROWSER = Kiwibank def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts() def get_account(self, _id): return find_object(self.browser.get_accounts(), id=_id, error=AccountNotFound) def iter_history(self, account): for transaction in self.browser.get_history(account): yield transaction ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000006277�13034501105�0017555�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Cédric Félizard # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime import re from decimal import Decimal from weboob.browser.pages import HTMLPage, LoggedPage from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import AmericanTransaction as EnglishTransaction __all__ = ['LoginPage', 'AccountPage', 'HistoryPage'] class LoginPage(HTMLPage): def login(self, username, password): form = self.get_form(name='aspnetForm') form['ctl00$chi$txtUserName'] = username form['ctl00$chi$txtPassword'] = password form.submit() class AccountPage(LoggedPage, HTMLPage): def get_accounts(self): for el in self.doc.getroot().cssselect('div#content tr.row'): account = Account() balance = el.cssselect('td.Balance')[0].text account.balance = Decimal(Transaction.clean_amount(balance)) account.id = el.cssselect('span')[0].text.strip() account.currency = u'NZD' # TODO: handle other currencies if el.cssselect('td.AccountName > a'): label_el = el.cssselect('td.AccountName > a')[0] account._link = label_el.get('href') else: label_el = el.cssselect('td.AccountName')[0] account._link = None account.label = unicode(label_el.text.strip()) yield account class HistoryPage(LoggedPage, HTMLPage): def get_history(self): # TODO: get more results from "next" page, only 15 transactions per page for el in self.doc.getroot().cssselect('div#content tr.row'): transaction = Transaction() label = unicode(el.cssselect('td.tranDesc')[0].text) transaction.label = label for pattern, _type in Transaction.PATTERNS: match = pattern.match(label) if match: transaction.type = _type break date = el.cssselect('td.tranDate')[0].text transaction.date = datetime.datetime.strptime(date, '%d %b \'%y') amount = el.cssselect('td.tranAmnt')[0].text transaction.amount = Decimal(Transaction.clean_amount(amount)) yield transaction class Transaction(EnglishTransaction): PATTERNS = [ (re.compile(r'^POS W/D (?P<text>.*)'), EnglishTransaction.TYPE_CARD), (re.compile(r'^ATM W/D (?P<text>.*)'), EnglishTransaction.TYPE_WITHDRAWAL), (re.compile(r'^(PAY|FROM) (?P<text>.*)'), EnglishTransaction.TYPE_TRANSFER), ] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/kiwibank/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002011�13034501105�0017413�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Cédric Félizard # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest __all__ = ['KiwibankTest'] class KiwibankTest(BackendTest): MODULE = 'kiwibank' def test_kiwibank(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016403�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000000105�13034501105�0020510�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import LaCentraleModule __all__ = ['LaCentraleModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/browser.py������������������������������������������������������������0000664�0000000�0000000�00000002577�13034501105�0020453�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vicnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import ListingAutoPage, AdvertPage __all__ = ['LaCentraleBrowser'] class LaCentraleBrowser(PagesBrowser): BASEURL = 'http://www.lacentrale.fr' list_page = URL('/listing_auto.php\?(?P<_request>.*)', ListingAutoPage) advert_page = URL('/auto-occasion-annonce-(?P<_id>.*).html', AdvertPage) def iter_prices(self, product): _request = '&'.join(['%s=%s' % (key, item) for key, item in product._criteria.iteritems()]) return self.list_page.go(_request=_request).iter_prices() def get_price(self, _id, obj): return self.advert_page.go(_id=_id).get_price(obj=obj) ���������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000040373�13034501105�0020545�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME .֖O/���tEXtComment�Created with GIMPW�� �IDATx@@��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������^^����������^^5���>���������� ������������������������������������������������������������������������������������������������������������������^^���^���<������������������������������������������������� �������������������� �����������������������������������������������������������������������������������������������������������������������������������������������������^^U���u�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������}��������������������������������������������^^s���_�������������������������������������������������������� �����������������������������(1""������������������������������������������������������^^����������   ����  �������������������������������������������������������������������������������������������������������������������� V;;����������>������������������������������������^^i���i���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� P77����������F�����������������������������������i�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������0����������>����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������8��������������������������������^^�������� �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ������������������������������������������������������������������������������������������������������������������������������������������������� �������(������������������������������������������������� ������������ �%�.�������������������������������������������������������������������������������������������������������������������������� ���������������������������������������������������������������������������B �-� �������������������������������������������������������������������������������������������������������������������������������� ^^��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� �����������%����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������&�������������������������������/������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������=����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ����������������������� ��������������������������������� �������������������������������������������������������������������������������������������������������������������������������^^������������������������������������������������������������� �������������������������������������������������������������������������������������������� � � � �����������������������������������������������������������������������������������%�+��������������������� �%���������������������������������������������������������� �<<� JJ�))�� �����������֟������ �����������������������������������������������������������������3�������������������"���������������������������������������������������������� �$SS�,ee� �������������������������������������FF���������������������������������������������*�������������������������������������������������� ����������������������������������������������������"OO�$SS� ������������������������������������������������������������������ ��������������������������������������������������������������������������������������������������������������������������������������������$$�/kk�EE� ����������������������������������������������������������������������������G���� ������������������������������������������������������������������������������������������������������??�CC� �������������������������������������������������������������������������������� ��������������������������������������������������������������������������������������������������>>� ����������侾�Ύ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������77�--�����������٨������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� �"LL�����������٧���������������������������������������������������������������� ������������������������������������������������������������������������������������������� ���������������������������������������������!!�..��������֟���������������������������������������������������������������� ����������������������������������������������� ����������������������������������������������������������������������������������66� ������������������������������������������������������������������������lt�� �IDAT�������������������� ^^5����������������������������^^���������������������������������������� ����������������������������������������%%�����������������������������������������������������������������������������������������������N�������������������������������%����������������������������������������������� � ������������������������������������������ ���������������������������������������������������������������������������������� ������������������������������������������&������������������������������������������������������ ��������������������������������������������������������������������������������������������������������������������������������� �������������������������������������������*�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ������������������������������������������ ����������������������������������������������� ������������������������������������������������������������ ��������������������������������������������������������������������������,�����������������������������������������������������������������������������������������"���������������������������������������������������������� ��������������������������������������������������������������������������������������������������������������������������������������������������������������4� �������������������������������������������������������������99� ��������������������������������������������������������������������������������������������������������������������������������������������������������� ����������������������������������������������������������==�!MM���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������+cc�%%��������������������������������������������������������������������������������������������������������������� ������������������������������������������������������������������������������������Ҙ��������&&�+aa���������������������������������������������������������������������������������������������������������� ������������������������������������������������������������������������������������������������ܮ����������� JJ�$RR����������������������������������������������������������� ���$^^��������������������������������������������������������������������������������������������������������إ������������4ww�G�7}}�@@� ��������������������������������������������������>������������������������������������������� ���������������������������� �����������������������՝� � �������33�//�"OO�������������������� ������)�����������������������������������  �������� �������������������������������������������������������������> �+� ���٧�-ff� ����������� � �22��������൵�⻻������������������������������������������������������������������������������������������������������������������������������������������ �*������֠��#OO� ������������������������� � ���������������������*����������������������������������������������� �8 ��������������������������������������������������������������������������������������В�ϑ������������������������� ������� ������ ���������������������������������������������������(�X�������������������������������������������������������������������������������ԛ�ܮ� �%%� � � � ������������� ��������������������������������������������������������������0 �!������������������������������������������ �> �!�������������������������������''�11  ������������������#������� �����������������������������������������������������������������G �@ ������������������������%��������������������������������������������������������������������!�������������������������������������������������(����������������������������������&���������������������������������������������������������������������������������������������������������������������������������������� ���������������������������������������������������!�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������( ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������;((������������������������������������������������������>++ ������������������������������������������������������������������������������������������������������������������������������������������������������� ���������������������������������������������������������� 3##����������������������������������������������������������������������������������������������������������������������������������������������������������������������-D.. �������������������������������������������������^^�������������V����������������������������������������������������������������������������������������������������������������������������������������  &.��������������}������������������������������������������������^^���O������������ ����� ����� '!����������������������������������������������������������������������������������^^5����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������^^5���^���/����������������� ������ ������ ���������������������������������������� ������������������������������������������ �����������������������������������������������������������������������������������������������������������������������������������������������������������^^5���4������(������������������������^^���%������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������Bx���KIDAT�����������������������������������������������������������������������EG����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/module.py�������������������������������������������������������������0000664�0000000�0000000�00000006000�13034501105�0020236�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vicnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.pricecomparison import CapPriceComparison, Price from weboob.tools.backend import Module from .product import LaCentraleProduct from .browser import LaCentraleBrowser __all__ = ['LaCentraleModule'] class LaCentraleModule(Module, CapPriceComparison): NAME = 'lacentrale' MAINTAINER = u'Vicnet' EMAIL = 'vo.publique@gmail.com' VERSION = '1.2' DESCRIPTION = 'Vehicule prices at LaCentrale.fr' LICENSE = 'AGPLv3+' BROWSER = LaCentraleBrowser def search_products(self, patternString=None): criteria = {} patterns = [] if patternString: patterns = patternString.split(',') for pattern in patterns: pattern = pattern.lower() if u'€' in pattern: criteria['prix_maxi'] = pattern[:pattern.find(u'€')].strip() if u'km' in pattern: criteria['km_maxi'] = pattern[:pattern.find(u'km')].strip() if u'p' in pattern[-1]: # last char = p criteria['nbdoors'] = pattern[:pattern.find(u'p')].strip() if u'cit' in pattern: criteria['Citadine'] = 'citadine&SS_CATEGORIE=40' if u'dep' in pattern: criteria['dptCp'] = pattern.replace('dep', '') if u'pro' in pattern: criteria['witchSearch'] = 1 if u'part' in pattern: criteria['witchSearch'] = 0 if u'diesel' in pattern: criteria['energie'] = 2 if u'essence' in pattern: criteria['energie'] = 1 if u'electrique' in pattern: criteria['energie'] = 4 if u'hybride' in pattern: criteria['energie'] = '8,9' if criteria: product = LaCentraleProduct() product._criteria = criteria yield product def iter_prices(self, products): product = [product for product in products if product.backend == self.name] if product: return self.browser.iter_prices(product[0]) def get_price(self, id, price=None): return self.browser.get_price(id, None) def fill_price(self, price, fields): if fields: price = self.get_price(price.id, price) return price OBJECTS = {Price: fill_price, } weboob-1.2/modules/lacentrale/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000006567�13034501105�0020072�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vicnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage, pagination from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, Format, Env, BrowserURL from weboob.browser.filters.javascript import JSVar from weboob.browser.filters.html import Link from weboob.capabilities.pricecomparison import Price, Shop from .product import LaCentraleProduct class ListingAutoPage(HTMLPage): @pagination @method class iter_prices(ListElement): item_xpath = '//div[@class="adContainer "]' next_page = Link('//section[@class="pagination"]/ul/li[@class="last"]/a') class item(ItemElement): klass = Price obj_id = CleanText('./p/a/@data-annid') obj_cost = CleanDecimal('./a/div/div/div/div[@class="fieldPrice"]') obj_currency = Regexp(CleanText('./a/div/div/div/div[@class="fieldPrice"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_message = Format('%s / %s / %s', CleanText('./a/div/div/h3'), CleanText('./a/div/div/div/div[@class="fieldYear"]'), CleanText('./a/div/div/div/div[@class="fieldMileage"]')) obj_url = Format('http://www.lacentrale.fr%s', CleanText('./a/@href')) obj_product = LaCentraleProduct() def obj_shop(self): shop = Shop(CleanText('./p/a/@data-annid')(self)) return shop class AdvertPage(HTMLPage): @method class get_price(ItemElement): klass = Price obj_id = Env('_id') obj_cost = CleanDecimal('//div[@class="mainInfos"]/div/p[@class="gpfzj"]') obj_currency = Regexp(CleanText('//div[@class="mainInfos"]/div/p[@class="gpfzj"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_message = Format('%s %s', CleanText('//div[@class="mainInfos"]/div/div/h1'), CleanText('//div[@class="mainInfos"]/div/div/p')) obj_url = BrowserURL('advert_page', _id=Env('_id')) def obj_shop(self): shop = Shop(Env('_id')(self)) shop.name = Regexp(CleanText('(//div[@xtcz="contacter_le_vendeur"]/div/ul/li)[1]'), 'Nom : (.*)')(self) shop.location = JSVar(CleanText('//script'), var='tooltip')(self) shop.info = CleanText('//div[@xtcz="contacter_le_vendeur"]/div/ul/li[has-class("printPhone")]')(self) return shop obj_product = LaCentraleProduct() �����������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/product.py������������������������������������������������������������0000664�0000000�0000000�00000001606�13034501105�0020440�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.pricecomparison import Product class LaCentraleProduct(Product): def __init__(self): self.id = 1 self.name = unicode('Voiture') ��������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lacentrale/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002300�13034501105�0017727�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vicnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest import itertools class LaCentraleTest(BackendTest): MODULE = 'lacentrale' def test_lacentrale(self): products = list(itertools.islice(self.backend.search_products(u'10000€,pro'), 0, 20)) self.assertTrue(len(products) > 0) product = products[0] product.backend = self.backend.name prices = list(itertools.islice(self.backend.iter_prices([product]), 0, 20)) self.assertTrue(len(prices) > 0) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015043�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001427�13034501105�0017160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LCLModule __all__ = ['LCLModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000024247�13034501105�0017111�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from urlparse import urlsplit, parse_qsl from weboob.exceptions import BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.exceptions import ServerError from weboob.capabilities.base import NotAvailable from weboob.capabilities.bank import Account from .pages import LoginPage, AccountsPage, AccountHistoryPage, \ CBListPage, CBHistoryPage, ContractsPage, BoursePage, \ AVPage, AVDetailPage, DiscPage, NoPermissionPage, RibPage, \ HomePage, LoansPage, TransferPage __all__ = ['LCLBrowser','LCLProBrowser'] # Browser class LCLBrowser(LoginBrowser): BASEURL = 'https://particuliers.secure.lcl.fr' login = URL('/outil/UAUT/Authentication/authenticate', '/outil/UAUT\?from=.*', '/outil/UWER/Accueil/majicER', '/outil/UWER/Enregistrement/forwardAcc', LoginPage) contracts = URL('/outil/UAUT/Contrat/choixContrat.*', '/outil/UAUT/Contract/getContract.*', '/outil/UAUT/Contract/selectContracts.*', '/outil/UAUT/Accueil/preRoutageLogin', '.*outil/UAUT/Contract/routing', ContractsPage) home = URL('/outil/UWHO/Accueil/', HomePage) accounts = URL('/outil/UWSP/Synthese', AccountsPage) history = URL('/outil/UWLM/ListeMouvements.*/accesListeMouvements.*', '/outil/UWLM/DetailMouvement.*/accesDetailMouvement.*', '/outil/UWLM/Rebond', AccountHistoryPage) rib = URL('/outil/UWRI/Accueil/detailRib', '/outil/UWRI/Accueil/listeRib', RibPage) cb_list = URL('/outil/UWCB/UWCBEncours.*/listeCBCompte.*', CBListPage) cb_history = URL('/outil/UWCB/UWCBEncours.*/listeOperations.*', CBHistoryPage) skip = URL('/outil/UAUT/Contrat/selectionnerContrat.*', '/index.html') no_perm = URL('/outil/UAUT/SansDroit/affichePageSansDroit.*', NoPermissionPage) bourse = URL('https://bourse.secure.lcl.fr/netfinca-titres/servlet/com.netfinca.frontcr.synthesis.HomeSynthesis', 'https://bourse.secure.lcl.fr/netfinca-titres/servlet/com.netfinca.frontcr.account.*', '/outil/UWBO.*', BoursePage) disc = URL('https://bourse.secure.lcl.fr/netfinca-titres/servlet/com.netfinca.frontcr.login.ContextTransferDisconnect', '/outil/UAUT/RetourPartenaire/retourCar', DiscPage) assurancevie = URL('/outil/UWVI/AssuranceVie/accesSynthese', AVPage) avdetail = URL('https://ASSURANCE-VIE-et-prevoyance.secure.lcl.fr.*', 'https://assurance-vie-et-prevoyance.secure.lcl.fr.*', '/outil/UWVI/Routage', AVDetailPage) loans = URL('/outil/UWCR/SynthesePar/', LoansPage) transfer_page = URL('/outil/UWVS/', TransferPage) confirm_transfer = URL('/outil/UWVS/Accueil/redirectView', TransferPage) #recipients = URL('/outil/UWBE/Consultation/list', RecipientPage) accounts_list = None def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.password.isdigit(): raise BrowserIncorrectPassword() # we force the browser to go to login page so it's work even # if the session expire self.login.go() if not self.page.login(self.username, self.password) or \ (self.login.is_here() and self.page.is_error()) : raise BrowserIncorrectPassword("invalid login/password.\nIf you did not change anything, be sure to check for password renewal request\non the original web site.\nAutomatic renewal will be implemented later.") self.accounts_list = None self.accounts.stay_or_go() @need_login def connexion_bourse(self): self.location('/outil/UWBO/AccesBourse/temporisationCar?codeTicker=TICKERBOURSECLI') if self.no_perm.is_here(): return False self.location(self.page.get_next()) self.bourse.stay_or_go() return True def deconnexion_bourse(self): self.disc.stay_or_go() self.page.come_back() self.page.come_back() @need_login def get_accounts_list(self): if self.accounts_list is None: self.accounts_list = [] self.assurancevie.stay_or_go() if self.no_perm.is_here(): self.logger.warning('Life insurances are unavailable.') else: for a in self.page.get_list(): self.accounts_list.append(a) self.accounts.stay_or_go() for a in self.page.get_list(): self.location('/outil/UWRI/Accueil/') self.rib.go(data={'compte': '%s/%s/%s' % (a.id[0:5], a.id[5:11], a.id[11:])}) if self.rib.is_here(): iban = self.page.get_iban() a.iban = iban if iban and a.id[11:] in iban else NotAvailable self.accounts_list.append(a) self.loans.stay_or_go() if self.no_perm.is_here(): self.logger.warning('Loans are unavailable.') else: for a in self.page.get_list(): self.accounts_list.append(a) if self.connexion_bourse(): for a in self.page.get_list(): self.accounts_list.append(a) self.deconnexion_bourse() # Disconnecting from bourse portal before returning account list # to be sure that we are on the banque portal return iter(self.accounts_list) @need_login def get_history(self, account): if hasattr(account, '_market_link') and account._market_link: self.connexion_bourse() self.location(account._market_link) self.location(account._link_id).page.get_fullhistory() for tr in self.page.iter_history(): yield tr self.deconnexion_bourse() elif hasattr(account, '_link_id') and account._link_id: try: self.location(account._link_id) except ServerError: return for tr in self.page.get_operations(): yield tr for tr in self.get_cb_operations(account, 1): yield tr elif account.type == Account.TYPE_LIFE_INSURANCE and account._form: self.assurancevie.stay_or_go() account._form.submit() self.page.sub().page.sub().page.get_details(account, "OHIPU") for tr in self.page.iter_history(): yield tr self.page.come_back().page.sub().page.come_back() @need_login def get_cb_operations(self, account, month=0): """ Get CB operations. * month=0 : current operations (non debited) * month=1 : previous month operations (debited) """ if not hasattr(account, '_coming_links'): return for link in account._coming_links: v = urlsplit(self.absurl(link)) args = dict(parse_qsl(v.query)) args['MOIS'] = month self.location('%s?%s' % (v.path, urllib.urlencode(args))) for tr in self.page.get_operations(): yield tr for card_link in self.page.get_cards(): self.location(card_link) for tr in self.page.get_operations(): yield tr @need_login def get_investment(self, account): if account.type == Account.TYPE_LIFE_INSURANCE and account._form: self.assurancevie.stay_or_go() account._form.submit() self.page.sub().page.sub() for inv in self.page.iter_investment(): yield inv self.page.come_back().page.sub().page.come_back() elif hasattr(account, '_market_link') and account._market_link: self.connexion_bourse() for inv in self.location(account._market_link).page.iter_investment(): yield inv self.deconnexion_bourse() @need_login def iter_recipients(self, origin_account): if origin_account._transfer_id is None: return self.transfer_page.go() if not self.page.can_transfer(origin_account._transfer_id): return self.page.choose_origin(origin_account._transfer_id) for recipient in self.page.iter_recipients(account_transfer_id=origin_account._transfer_id): yield recipient @need_login def init_transfer(self, account, recipient, amount, reason=None): self.transfer_page.go() self.page.choose_origin(account._transfer_id) self.page.choose_recip(recipient) self.page.transfer(amount, reason) self.page.check_data_consistency(account, recipient, amount, reason) return self.page.create_transfer(account, recipient, amount, reason) @need_login def execute_transfer(self, transfer): self.page.confirm() return self.page.fill_transfer_id(transfer) @need_login def get_advisor(self): return iter([self.accounts.stay_or_go().get_advisor()]) class LCLProBrowser(LCLBrowser): BASEURL = 'https://professionnels.secure.lcl.fr' #We need to add this on the login form IDENTIFIANT_ROUTING = 'CLA' def __init__(self, *args, **kwargs): super(LCLProBrowser, self).__init__(*args, **kwargs) self.session.cookies.set("lclgen","professionnels") ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/enterprise/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017223�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/enterprise/__init__.py�������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021322�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/enterprise/browser.py��������������������������������������������������������0000664�0000000�0000000�00000005070�13034501105�0021262�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, MovementsPage, PassExpiredPage class LCLEnterpriseBrowser(LoginBrowser): BASEURL = 'https://entreprises.secure.lcl.fr' pass_expired = URL('/outil/IQEN/Authentication/forcerChangePassword', PassExpiredPage) login = URL('/outil/IQEN/Authentication/indexRedirect', '/outil/IQEN/Authentication/(?P<page>.*)', LoginPage) movements = URL('/outil/IQMT/mvt.Synthese/syntheseMouvementPerso', '/outil/IQMT/mvt.Synthese', MovementsPage) def __init__(self, *args, **kwargs): super(LCLEnterpriseBrowser, self).__init__(*args, **kwargs) self.accounts = None def deinit(self): if self.page.logged: self.login.go(page="logout") self.login.go(page="logoutOk") assert self.login.is_here(page="logoutOk") super(LCLEnterpriseBrowser, self).deinit() def do_login(self): self.login.go().login(self.username, self.password) error = self.page.get_error() if self.login.is_here() else False if error: raise BrowserIncorrectPassword(error) @need_login def get_accounts_list(self): if not self.accounts: self.accounts = list(self.movements.go().iter_accounts()) return self.accounts @need_login def get_history(self, account): if account._data: return self.open(account._url, data=account._data).page.iter_history() return self.movements.go().iter_history() def get_cb_operations(self, account): raise NotImplementedError() def get_investment(self, account): raise NotImplementedError() class LCLEspaceProBrowser(LCLEnterpriseBrowser): BASEURL = 'https://espacepro.secure.lcl.fr' ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/enterprise/pages.py����������������������������������������������������������0000664�0000000�0000000�00000011273�13034501105�0020700�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re, requests from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, ItemElement, TableElement, method from weboob.browser.filters.standard import CleanText, Date, CleanDecimal, Env, TableCell from weboob.browser.filters.html import Link from weboob.capabilities.bank import Account from weboob.capabilities.base import NotAvailable from weboob.exceptions import BrowserPasswordExpired from ..pages import Transaction def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class LoginPage(HTMLPage): def get_error(self): return CleanText(default=False).filter(self.doc.xpath('//li[contains(@class, "erreur")]')) def login(self, login, password): form = self.get_form(id="myForm") form['login'] = login form['pwd'] = password form['mode'] = '1' form.submit() class MovementsPage(LoggedPage, HTMLPage): def get_changecompte(self, link): form = self.get_form('//form[contains(@action, "changeCompte")]') m = re.search('\'(\d+).*\'(\d+)', link) form['perimetreMandatParentData'] = m.group(1) form['perimetreMandatEnfantData'] = m.group(2) # Can't do multi with async because of inconsistency... return self.browser.open(form.url, data=dict(form)).page, form.url, dict(form) @method class iter_accounts(ListElement): def parse(self, el): multi_xpath = '//form[contains(@action, "changeCompte")]//ul[@class="listeEnfants"]/li/a' self.page.multi = True if self.page.doc.xpath(multi_xpath) else False self.item_xpath = multi_xpath if self.page.multi else '//*[@id="perimetreMandatEnfantLib"]' class item(ItemElement): klass = Account obj_id = Env('accid') obj_label = Env('label') obj_type = Account.TYPE_CHECKING obj_balance = Env('balance') obj_currency = Env('currency') obj__url = Env('url') obj__data = Env('data') def parse(self, el): page, url, data = self.page.get_changecompte(Link('.')(self)) if self.page.multi else (self.page, None, None) self.env['accid'] = CleanText('.')(self).strip().replace(' ', '').split('-')[0] self.env['label'] = CleanText('.')(self).split("-")[-1].strip() balance_xpath = '//div[contains(text(),"Solde")]/strong' self.env['balance'] = MyDecimal().filter(page.doc.xpath(balance_xpath)) self.env['currency'] = Account.get_currency(CleanText().filter(page.doc.xpath(balance_xpath))) self.env['url'] = url self.env['data'] = data @pagination @method class iter_history(TableElement): item_xpath = '//table[@id="listeOperations" or @id="listeEffets"]/tbody/tr[td[5]]' head_xpath = '//table[@id="listeOperations"]/thead/tr/th' col_raw = re.compile(u'Libellé') col_debit = re.compile(u'Débit') col_credit = re.compile(u'Crédit') def next_page(self): url = Link('//a[contains(text(), "Page suivante")]', default=None)(self) if url: m = re.search('\s+\'([^\']+).*\'(\d+)', url) return requests.Request("POST", m.group(1), data={'numPage': m.group(2)}) class item(ItemElement): klass = Transaction obj_raw = Transaction.Raw(TableCell('raw')) obj_date = Date(CleanText('./td[1]'), dayfirst=True) obj_vdate = Date(CleanText('./td[2]'), dayfirst=True) def obj_amount(self): credit = MyDecimal(TableCell('credit'))(self) debit = MyDecimal(TableCell('debit'))(self) return credit if credit else -debit class PassExpiredPage(HTMLPage): def on_load(self): raise BrowserPasswordExpired("Renouvellement de mot de passe requis.") �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000004541�13034501105�0017202�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 08u���tEXtComment�Created with GIMPW��IDATx{pT眽&l$DB6$ HKLZ2ZPzbձNGءDڢlKk)X$ؐYe{.0,źKesyl}B#M$&�7 M� (6!DO-6#+#ȚQh3꟧M~W8#wwj_$=4ax„{{ⰻ:Mt8L� g Rp it(su8 T_@?/BXԹ=:3a`T^u`œR\>?3 QCg`wSBJg~ JBzQskq~I|C },.}Ȋ@Y3G,9]%mO}�2FDAA FW|w:1FanB5<siW5?9nx̒DۃNÛM -?q_b =<<Dr( 'o!Nol+w[wf#=,B{ Qcfז&�(Ra5e ڌI‘udQcܹh"[;whmeLGt#@U3c.z=iob_v/waoK^Ɋg}0-!'tq3V$$&ŗؿZtZyd{9Z|3`닉d7% as$ aT0AYA2Uȇygr<ygypuy-d].iKhZ7N5goG]0$mJC-z\` &�QU9Ŭk2:sjAR�hn͌;sǵso(vo6s$�zvЃŸz}ъR�k㳹zұ'4\Tܖ~ת{;R9Z&'Ҷl+B {͗V=x>tr~{8�f,wbBt2n@8z<7Ň5/3 "3*c@GobÁEq0avյ;X]{㠌k*홟h&u -|VD~h:քGDCL2?3Ѵ84 T-7KP =ިH_0ٳk ~; "x<K<Xo.aך{r%^rdYig\zm줈p.P`=[ljok.ӚmW2dlz}?3{ Rv_Ux!Q՝C65cP J"`ѢX<7:1)?)@l}Pعs5Q2:}_Qʁ/A=˛o$`@ !Er{`ιkXX-:f��ir5k?D(b8 I*/1['ADW>i#jH |Mxԇet+DMRT0~�4Ecqd8SDq q}>?6m?oY7ªd0&nrTUt,YVFA`A00dEndF�0jdCvVw|7/ǿEKz<QLq:Y@Eyvht(aA]$\`jC틠x7\hzϤɫB;:Ow\-i KJ09llD> _19R UX,FBʝ#QzonZDE )(!՞Lib^vgQ6#kth7ad7aȳ fd Y%s~'^�T* y5̟7 7Rݭgx݁yL ʠh8wT^.LL_y�v G>-?ošaL< AT?0,|9?Ngr})Y ͧX@'NtpהX rzޅ?|~Px (*o-�Qކ? %C](B7 R�6l~}DZ!JŨJ#$R̂ 0%mfg&�O:A{(c1f/F u�WK$Q*F2Q[Xa0;B7 @We,i����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000012174�13034501105�0016707�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from decimal import Decimal from weboob.capabilities.bank import CapBankTransfer, AccountNotFound, \ RecipientNotFound, TransferError, Account from weboob.capabilities.contact import CapContact from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from weboob.capabilities.base import find_object from .browser import LCLBrowser, LCLProBrowser from .enterprise.browser import LCLEnterpriseBrowser, LCLEspaceProBrowser __all__ = ['LCLModule'] class LCLModule(Module, CapBankTransfer, CapContact): NAME = 'lcl' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'LCL' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Code personnel'), Value('website', label='Type de compte', default='par', choices={'par': 'Particuliers', 'pro': 'Professionnels', 'ent': 'Entreprises', 'esp': 'Espace Pro'})) BROWSER = LCLBrowser def create_default_browser(self): # assume all `website` option choices are defined here browsers = {'par': LCLBrowser, 'pro': LCLProBrowser, 'ent': LCLEnterpriseBrowser, 'esp': LCLEspaceProBrowser} website_value = self.config['website'] self.BROWSER = browsers.get(website_value.get(), browsers[website_value.default]) return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts_list() def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_coming(self, account): transactions = list(self.browser.get_cb_operations(account)) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions def iter_history(self, account): transactions = list(self.browser.get_history(account)) transactions.sort(key=lambda tr: tr.rdate, reverse=True) return transactions def iter_investment(self, account): return self.browser.get_investment(account) def iter_transfer_recipients(self, origin_account): if self.config['website'].get() not in ['par', 'pro']: raise NotImplementedError() if not isinstance(origin_account, Account): origin_account = find_object(self.iter_accounts(), id=origin_account, error=AccountNotFound) return self.browser.iter_recipients(origin_account) def init_transfer(self, transfer, **params): if self.config['website'].get() not in ['par', 'pro']: raise NotImplementedError() # There is a check on the website, transfer can't be done with too long reason. if transfer.label: transfer.label = transfer.label[:30] self.logger.info('Going to do a new transfer') if transfer.account_iban: account = find_object(self.iter_accounts(), iban=transfer.account_iban, error=AccountNotFound) else: account = find_object(self.iter_accounts(), id=transfer.account_id, error=AccountNotFound) if transfer.recipient_iban: recipient = find_object(self.iter_transfer_recipients(account.id), iban=transfer.recipient_iban, error=RecipientNotFound) else: recipient = find_object(self.iter_transfer_recipients(account.id), id=transfer.recipient_id, error=RecipientNotFound) try: # quantize to show 2 decimals. amount = Decimal(transfer.amount).quantize(Decimal(10) ** -2) except (AssertionError, ValueError): raise TransferError('something went wrong') return self.browser.init_transfer(account, recipient, amount, transfer.label) def execute_transfer(self, transfer, **params): return self.browser.execute_transfer(transfer) def iter_contacts(self): if self.config['website'].get() != "par": raise NotImplementedError() return self.browser.get_advisor() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lcl/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000077776�13034501105�0016544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re, requests, base64, math, random from decimal import Decimal from cStringIO import StringIO from urllib import urlencode from datetime import datetime, timedelta, date from weboob.capabilities import NotAvailable from weboob.capabilities.base import find_object from weboob.capabilities.bank import Account, Investment, Recipient, TransferError, Transfer from weboob.capabilities.contact import Advisor from weboob.browser.elements import method, ListElement, TableElement, ItemElement, SkipItem from weboob.exceptions import ParseError from weboob.browser.pages import LoggedPage, HTMLPage, FormNotFound, pagination from weboob.browser.filters.html import Attr, Link from weboob.browser.filters.standard import CleanText, Field, Regexp, Format, Date, \ CleanDecimal, Map, AsyncLoad, Async, Env, \ TableCell, Eval from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=Decimal(0)) return CleanDecimal(*args, **kwargs) def myXOR(value,seed): s = '' for i in xrange(len(value)): s += chr(seed^ord(value[i])) return s class LCLBasePage(HTMLPage): def get_from_js(self, pattern, end, is_list=False): """ find a pattern in any javascript text """ value = None for script in self.doc.xpath('//script'): txt = script.text if txt is None: continue start = txt.find(pattern) if start < 0: continue while True: if value is None: value = '' else: value += ',' value += txt[start+len(pattern):start+txt[start+len(pattern):].find(end)+len(pattern)] if not is_list: break txt = txt[start+len(pattern)+txt[start+len(pattern):].find(end):] start = txt.find(pattern) if start < 0: break return value class LCLVirtKeyboard(MappedVirtKeyboard): symbols={'0': '9da2724133f2221482013151735f033c', '1': '873ab0087447610841ae1332221be37b', '2': '93ce6c330393ff5980949d7b6c800f77', '3': 'b2d70c69693784e1bf1f0973d81223c0', '4': '498c8f5d885611938f94f1c746c32978', '5': '359bcd60a9b8565917a7bf34522052c3', '6': 'aba912172f21f78cd6da437cfc4cdbd0', '7': 'f710190d6b947869879ec02d8e851dfa', '8': 'b42cc25e1539a15f767aa7a641f3bfec', '9': 'cc60e5894a9d8e12ee0c2c104c1d5490' } url="/outil/UAUT/Clavier/creationClavier?random=" color=(255,255,255,255) def __init__(self, basepage): img=basepage.doc.find("//img[@id='idImageClavier']") random.seed() self.url += "%s"%str(long(math.floor(long(random.random()*1000000000000000000000)))) super(LCLVirtKeyboard, self).__init__(StringIO(basepage.browser.open(self.url).content), basepage.doc,img,self.color, "id") self.check_symbols(self.symbols,basepage.browser.responses_dirname) def get_symbol_code(self, md5sum): code=MappedVirtKeyboard.get_symbol_code(self, md5sum) return code[-2:] def get_string_code(self, string): code='' for c in string: code += self.get_symbol_code(self.symbols[c]) return code class LoginPage(HTMLPage): def on_load(self): try: form = self.get_form(xpath='//form[@id="setInfosCGS" or @name="form"]') except FormNotFound: return form.submit() def login(self, login, passwd): try: vk = LCLVirtKeyboard(self) except VirtKeyboardError as err: self.logger.exception(err) return False password = vk.get_string_code(passwd) seed = -1 s = "var aleatoire = " for script in self.doc.findall("//script"): if script.text is None or len(script.text) == 0: continue offset = script.text.find(s) if offset != -1: seed = int(script.text[offset+len(s)+1:offset+len(s)+2]) break if seed==-1: raise ParseError("Variable 'aleatoire' not found") form = self.get_form('//form[@id="formAuthenticate"]') form['identifiant'] = login form['postClavierXor'] = base64.b64encode(myXOR(password,seed)) try: form['identifiantRouting'] = self.browser.IDENTIFIANT_ROUTING except AttributeError: pass try: form.submit() except BrowserUnavailable: # Login is not valid return False return True def is_error(self): errors = self.doc.xpath(u'//*[@class="erreur" or @class="messError"]') return len(errors) > 0 and not self.doc.xpath('//a[@href="/outil/UWHO/Accueil/"]') class ContractsPage(LoginPage): def on_load(self): if self.is_error(): raise BrowserIncorrectPassword() self.select_contract() def select_contract(self): # XXX We select automatically the default contract in list. We should let user # ask what contract he wants to see, or display accounts for all contracts. link = self.doc.xpath('//a[contains(text(), "Votre situation globale")]') if len(link): self.browser.location(link[0].attrib['href']) else: form = self.get_form(nr=0) form.submit() class AccountsPage(LoggedPage, HTMLPage): def on_load(self): warn = self.doc.xpath('//div[@id="attTxt"]') if len(warn) > 0: raise BrowserIncorrectPassword(warn[0].text) @method class get_list(ListElement): # XXX Ugly Hack to replace account by second occurrence. # LCL pro website sometimes display the same account twice and only second link is valid to fetch transactions. def store(self, obj): assert obj.id if obj.id in self.objects: self.logger.warning('There are two objects with the same ID! %s' % obj.id) self.objects[obj.id] = obj return obj item_xpath = '//tr[contains(@onclick, "redirect")]' flush_at_end = True class account(ItemElement): klass = Account def condition(self): return '/outil/UWLM/ListeMouvement' in self.el.attrib['onclick'] NATURE2TYPE = {'001': Account.TYPE_SAVINGS, '005': Account.TYPE_CHECKING, '006': Account.TYPE_CHECKING, '007': Account.TYPE_SAVINGS, '012': Account.TYPE_SAVINGS, '023': Account.TYPE_CHECKING, '046': Account.TYPE_SAVINGS, '047': Account.TYPE_SAVINGS, '049': Account.TYPE_SAVINGS, '068': Account.TYPE_MARKET, '069': Account.TYPE_SAVINGS, } obj__link_id = Format('%s&mode=190', Regexp(CleanText('./@onclick'), "'(.*)'")) obj__agence = Regexp(Field('_link_id'), r'.*agence=(\w+)') obj__compte = Regexp(Field('_link_id'), r'compte=(\w+)') obj_id = Format('%s%s', Field('_agence'), Field('_compte')) obj__transfer_id = Format('%s0000%s', Field('_agence'), Field('_compte')) obj__coming_links = [] obj_label = CleanText('.//div[@class="libelleCompte"]') obj_balance = MyDecimal('.//td[has-class("right")]', replace_dots=True) obj_currency = FrenchTransaction.Currency('.//td[has-class("right")]') obj_type = Map(Regexp(Field('_link_id'), r'.*nature=(\w+)'), NATURE2TYPE, default=Account.TYPE_UNKNOWN) obj__market_link = None class card(ItemElement): def condition(self): return '/outil/UWCB/UWCBEncours' in self.el.attrib['onclick'] def parse(self, el): link = Regexp(CleanText('./@onclick'), "'(.*)'")(el) id = Regexp(CleanText('./@onclick'), r'.*AGENCE=(\w+).*COMPTE=(\w+).*CLE=(\w+)', r'\1\2\3')(el) account = self.parent.objects[id] if not account.coming: account.coming = Decimal('0') account.coming += CleanDecimal('.//td[has-class("right")]', replace_dots=True)(el) account._coming_links.append(link) raise SkipItem() @method class get_advisor(ItemElement): klass = Advisor obj_name = CleanText('//div[@id="contacterMaBqMenu"]//p[@id="itemNomContactMaBq"]/span') obj_email = obj_mobile = obj_fax = NotAvailable obj_phone = Regexp(CleanText('//div[@id="sousContentAgence"]//p[contains(text(), "Tel")]', replace=[(' ', '')]), '([\s\d]+)') obj_agency = CleanText('//div[@id="sousContentAgence"]//p[@class="itemSousTitreMenuMaBq"][1]') def obj_address(self): address = CleanText('//div[@id="sousContentAgence"]//p[@class="itemSousTitreMenuMaBq"][2]', default=None)(self) city = CleanText('//div[@id="sousContentAgence"]//p[@class="itemSousTitreMenuMaBq"][3]', default=None)(self) return "%s %s" % (address, city) if address and city else NotAvailable class LoansPage(LoggedPage, HTMLPage): @method class get_list(TableElement): item_xpath = '//table[.//th[contains(text(), "Emprunteur")]]/tbody/tr[td[3]]' head_xpath = '//table[.//th[contains(text(), "Emprunteur")]]/thead/tr/th' flush_at_end = True col_id = re.compile('Emprunteur') col_balance = [u'Capital restant dû', re.compile('Sommes totales restant dues'), re.compile('Montant disponible')] class account(ItemElement): klass = Account obj_balance = CleanDecimal(TableCell('balance'), replace_dots=True, sign=lambda x: -1) obj_currency = FrenchTransaction.Currency(TableCell('balance')) obj_type = Account.TYPE_LOAN obj_id = Env('id') obj__transfer_id = None def obj_label(self): has_type = CleanText('./ancestor::table[.//th[contains(text(), "Type")]]', default=None)(self) return CleanText('./td[2]')(self) if has_type else CleanText('./ancestor::table/preceding-sibling::div[1]')(self).split(' - ')[0] def parse(self, el): label = Field('label')(self) trs = self.xpath('//td[contains(text(), "%s")]/ancestor::tr[1] | ./ancestor::table[1]/tbody/tr' % label) i = [i for i in range(len(trs)) if el == trs[i]] i = i[0] if i else 0 label = label.replace(' ', '') self.env['id'] = "%s%s%s" % (Regexp(CleanText(TableCell('id')), r'(\w+)\s-\s(\w+)', r'\1\2')(self), label.replace(' ', ''), i) class Transaction(FrenchTransaction): PATTERNS = [(re.compile('^(?P<category>CB) (?P<text>RETRAIT) DU (?P<dd>\d+)/(?P<mm>\d+)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile('^(?P<category>(PRLV|PE)( SEPA)?) (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?P<category>CHQ\.) (?P<text>.*)'), FrenchTransaction.TYPE_CHECK), (re.compile('^(?P<category>RELEVE CB) AU (\d+)/(\d+)/(\d+)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P<category>CB) (?P<text>.*) (?P<dd>\d+)/(?P<mm>\d+)/(?P<yy>\d+)'), FrenchTransaction.TYPE_CARD), (re.compile('^(?P<category>(PRELEVEMENT|TELEREGLEMENT|TIP)) (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile('^(?P<category>(ECHEANCE\s*)?PRET)(?P<text>.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile('^(?P<category>(EVI|VIR(EM(EN)?)?T?)(.PERMANENT)? ((RECU|FAVEUR) TIERS|SEPA RECU)?)( /FRM)?(?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile('^(?P<category>REMBOURST)(?P<text>.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile('^(?P<category>COM(MISSIONS?)?)(?P<text>.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P<text>(?P<category>REMUNERATION).*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P<text>(?P<category>ABON.*?)\s*.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P<text>(?P<category>RESULTAT .*?)\s*.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P<text>(?P<category>TRAIT\..*?)\s*.*)'), FrenchTransaction.TYPE_BANK), (re.compile('^(?P<category>REM CHQ) (?P<text>.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('^VIREMENT.*'),FrenchTransaction.TYPE_TRANSFER), (re.compile('.*(PRELEVEMENTS|PRELVT|TIP).*'),FrenchTransaction.TYPE_ORDER), (re.compile('.*CHEQUE.*'), FrenchTransaction.TYPE_CHECK), (re.compile('.*ESPECES.*'), FrenchTransaction.TYPE_DEPOSIT), (re.compile('.*(CARTE|CB).*'), FrenchTransaction.TYPE_CARD), (re.compile('.*(AGIOS|ANNULATIONS|IMPAYES|CREDIT).*'), FrenchTransaction.TYPE_BANK), (re.compile('.*(FRAIS DE TENUE DE COMPTE).*'), FrenchTransaction.TYPE_BANK), ] class Pagination(object): def next_page(self): links = self.page.doc.xpath('//div[@class="pagination"] /a') if len(links) == 0: return for link in links: if link.xpath('./span')[0].text == 'Page suivante': return link.attrib.get('href') return class AccountHistoryPage(LoggedPage, HTMLPage): class _get_operations(Pagination, Transaction.TransactionsElement): item_xpath = '//table[has-class("tagTab") and (not(@style) or @style="")]/tr' head_xpath = '//table[has-class("tagTab") and (not(@style) or @style="")]/tr/th' col_raw = [u'Vos opérations', u'Libellé'] class item(Transaction.TransactionElement): load_details = Attr('.', 'href', default=None) & AsyncLoad def obj_type(self): type = Async('details', CleanText(u'//td[contains(text(), "Nature de l\'opération")]/following-sibling::*[1]'))(self) if not type: return Transaction.TYPE_UNKNOWN for pattern, _type in Transaction.PATTERNS: match = pattern.match(type) if match: return _type break return Transaction.TYPE_UNKNOWN def condition(self): return self.parent.get_colnum('date') is not None and \ len(self.el.findall('td')) >= 3 and \ self.el.get('class') and \ 'tableTr' not in self.el.get('class') def validate(self, obj): if obj.category == 'RELEVE CB': obj.type = Transaction.TYPE_CARD_SUMMARY obj.deleted = True raw = Async('details', CleanText(u'//td[contains(text(), "Libellé")]/following-sibling::*[1]|//td[contains(text(), "Nom du donneur")]/following-sibling::*[1]', default=obj.raw))(self) if raw: if obj.raw in raw or raw in obj.raw or ' ' not in obj.raw: obj.raw = raw obj.label = raw else: obj.label = '%s %s' % (obj.raw, raw) obj.raw = '%s %s' % (obj.raw, raw) if not obj.date: obj.date = Async('details', Date(CleanText(u'//td[contains(text(), "Date de l\'opération")]/following-sibling::*[1]', default=u''), default=NotAvailable))(self) obj.rdate = obj.date obj.vdate = Async('details', Date(CleanText(u'//td[contains(text(), "Date de valeur")]/following-sibling::*[1]', default=u''), default=NotAvailable))(self) obj.amount = Async('details', CleanDecimal(u'//td[contains(text(), "Montant")]/following-sibling::*[1]', replace_dots=True, default=NotAvailable))(self) # ugly hack to fix broken html if not obj.amount: obj.amount = Async('details', CleanDecimal(u'//td[contains(text(), "Montant")]/following-sibling::*[1]', replace_dots=True, default=NotAvailable))(self) return True @pagination def get_operations(self): return self._get_operations(self)() class CBHistoryPage(AccountHistoryPage): def get_operations(self): for tr in self._get_operations(self)(): tr.type = tr.TYPE_CARD yield tr class CBListPage(CBHistoryPage): def get_cards(self): cards = [] for tr in self.doc.getiterator('tr'): link = Regexp(CleanText('./@onclick'), "'(.*)'", default=None)(tr) if link is not None and link.startswith('/outil/UWCB/UWCBEncours') and 'listeOperations' in link: cards.append(link) return cards class BoursePage(LoggedPage, HTMLPage): ENCODING='latin-1' def get_next(self): return re.search('"(.*?)"', self.doc.xpath('.//body')[0].attrib['onload']).group(1) def get_fullhistory(self): form = self.get_form(id="historyFilter") form['cashFilter'] = "ALL" # We can't go above 2 years form['beginDayfilter'] = (datetime.strptime(form['endDayfilter'], '%d/%m/%Y') - timedelta(days=730)).strftime('%d/%m/%Y') form.submit() @method class get_list(TableElement): item_xpath = '//table[has-class("tableau_comptes_details")]//tr[td and not(parent::tfoot)]' head_xpath = '//table[has-class("tableau_comptes_details")]/thead/tr/th' col_label = u'Comptes' col_balance = re.compile('Valorisation') class item(ItemElement): klass = Account load_details = Field('_market_link') & AsyncLoad obj_type = Account.TYPE_MARKET obj_balance = CleanDecimal(TableCell('balance'), replace_dots=True) obj_valuation_diff = Async('details') & CleanDecimal('//td[contains(text(), "value latente")]/ \ following-sibling::td[1]', replace_dots=True) obj__market_link = Regexp(Attr(TableCell('label'), 'onclick'), "'(.*?)'") obj__link_id = Async('details') & Link(u'//a[text()="Historique"]') obj__transfer_id = None def obj_id(self): return "%sbourse" % "".join(CleanText().filter((TableCell('label')(self)[0]).xpath('./div[not(b)]')).split(' - ')) def obj_label(self): return "%s Bourse" % CleanText().filter((TableCell('label')(self)[0]).xpath('./div[b]')) @method class iter_investment(ListElement): item_xpath = '//table[@id="tableValeurs"]/tbody/tr[not(@class) and count(descendant::td) > 1]' class item(ItemElement): klass = Investment obj_label = CleanText('.//td[2]/div/a') obj_code= CleanText('.//td[2]/div/br/following-sibling::text()') & Regexp(pattern='^([^ ]+).*', default=NotAvailable) obj_quantity = MyDecimal('.//td[3]/span') obj_diff = MyDecimal('.//td[7]/span') obj_valuation = MyDecimal('.//td[5]') def obj_unitvalue(self): if "%" in CleanText('.//td[4]')(self) and "%" in CleanText('.//td[6]')(self): return NotAvailable return MyDecimal('.//td[4]/text()')(self) def obj_unitprice(self): if "%" in CleanText('.//td[4]')(self) and "%" in CleanText('.//td[6]')(self): return NotAvailable return MyDecimal('.//td[6]')(self) @pagination @method class iter_history(TableElement): item_xpath = '//table[@id="historyTable" and thead]/tbody/tr' head_xpath = '//table[@id="historyTable" and thead]/thead/tr/th' col_date = 'Date' col_label = u'Opération' col_quantity = u'Qté' col_code = u'Libellé' col_amount = 'Montant' def next_page(self): form = self.page.get_form(id="historyFilter") form['PAGE'] = int(form['PAGE']) + 1 return requests.Request("POST", form.url, data=dict(form)) \ if self.page.doc.xpath('//*[@data-page = "%s"]' % form['PAGE']) else None class item(ItemElement): klass = Transaction obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_type = Transaction.TYPE_BANK obj_amount = CleanDecimal(TableCell('amount'), replace_dots=True) obj_investments = Env('investments') def obj_label(self): return TableCell('label')(self)[0].xpath('./text()')[0].strip() def parse(self, el): i = None if CleanText(TableCell('code'))(self): i = Investment() i.label = Field('label')(self) i.code = unicode(TableCell('code')(self)[0].xpath('./text()[last()]')[0]).strip() i.quantity = MyDecimal(TableCell('quantity'))(self) i.valuation = Field('amount')(self) i.vdate = Field('date')(self) self.env['investments'] = [i] if i else [] class DiscPage(LoggedPage, HTMLPage): def come_back(self): form = self.get_form() form.submit() class NoPermissionPage(LoggedPage, HTMLPage): pass class AVPage(LoggedPage, HTMLPage): @method class get_list(ListElement): item_xpath = '//table[@class]/tbody/tr' class account(ItemElement): klass = Account obj__owner = CleanText('.//td[1]') & Regexp(pattern=r' ([^ ]+)$') obj_label = Format(u'%s %s', CleanText('.//td/a'), obj__owner) obj_balance = CleanDecimal('.//td[has-class("right")]', replace_dots=True) obj_type = Account.TYPE_LIFE_INSURANCE obj__link_id = None obj__market_link = None obj__coming_links = [] obj__transfer_id = None def obj_id(self): _id = CleanText('.//td/a/@id')(self) if not _id: _id = Regexp(CleanText('.//td/a/@href'), r'ID_CONTRAT=(\d+)')(self) return Format(u'%s%s', CleanText(Field('label'), replace=[(' ', '')]), _id)(self) def obj__form(self): form_id = Attr('.//td/a', 'id', default=None)(self) if not form_id: return form = self.page.get_form('//form[@id="formRoutage"]') form['ID_CONTRAT'] = re.search(r'^(.*?)-', form_id).group(1) form['PRODUCTEUR'] = re.search(r'-(.*?)$', form_id).group(1) return form class AVDetailPage(LoggedPage, LCLBasePage): def sub(self): form = self.get_form(name="formulaire") cName = self.get_from_js('.cName.value = "', '";') if cName: form['cName'] = cName form['cValue'] = self.get_from_js('.cValue.value = "', '";') form['cMaxAge'] = '-1' return form.submit() def come_back(self): session = self.get_from_js('idSessionSag = "', '"') params = {} params['sessionSAG'] = session params['stbpg'] = 'pagePU' params['act'] = '' params['typeaction'] = 'reroutage_retour' params['site'] = 'LCLI' params['stbzn'] = 'bnc' return self.browser.location('https://assurance-vie-et-prevoyance.secure.lcl.fr/filiale/entreeBam?%s' % urlencode(params)) def get_details(self, account, act=None): form = self.get_form(id="frm_fwk") form.submit() if act is not None: self.browser.location("entreeBam?sessionSAG=%s&act=%s" % (form['sessionSAG'], act)) @method class iter_investment(ListElement): item_xpath = '//table[@class="table"]/tbody/tr[td[6]]' class item(ItemElement): klass = Investment obj_label = CleanText('.//td[1]/a | .//td[1]/span ') obj_code = CleanText('.//td[1]/a/@id') & Regexp(pattern='^([^ ]+).*', default=NotAvailable) obj_quantity = MyDecimal('.//td[4]/span') obj_unitvalue = MyDecimal('.//td[2]/span') obj_valuation = MyDecimal('.//td[5]/span') obj_portfolio_share = Eval(lambda x: x / 100, CleanDecimal('.//td[6]/span', replace_dots=True)) @pagination @method class iter_history(TableElement): item_xpath = '//table[@class="table"]/tbody/tr' head_xpath = '//table[@class="table"]/thead/tr/th' col_date = 'Date d\'effet' col_label = u'Opération(s)' col_amount = 'Montant' def next_page(self): if Link('//a[@class="pictoSuivant"]', default=None)(self): form = self.page.get_form(id="frm_fwk") form['fwkaction'] = "precSuivDet" form['fwkcodeaction'] = "Executer" form['ACTION_CHOISIE'] = "suivant" return requests.Request("POST", form.url, data=dict(form)) class item(ItemElement): klass = Transaction obj_label = CleanText(TableCell('label')) obj_type = Transaction.TYPE_BANK obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_amount = MyDecimal(TableCell('amount')) class RibPage(LoggedPage, LCLBasePage): def get_iban(self): if (self.doc.xpath('//div[contains(@class, "rib_cadre")]//div[contains(@class, "rib_internat")]')): return CleanText().filter(self.doc.xpath('//div[contains(@class, "rib_cadre")]//div[contains(@class, "rib_internat")]//p//strong')[0].text).replace(' ','') class HomePage(LoggedPage, HTMLPage): pass class TransferPage(LoggedPage, HTMLPage): def on_load(self): # This aims to track input errors. script_error = CleanText(u"//script[contains(text(), 'if (\"true\"===\"true\")')]")(self.doc) if script_error: raise TransferError(re.search(u'\.html\("(.*?)"\)', script_error).group(1)) def can_transfer(self, account_transfer_id): for div in self.doc.xpath('//div[input[@id="indexCompteEmetteur"]]//div[@class="infoCompte" and not(@title)]'): if account_transfer_id in CleanText('.', replace=[(' ', '')])(div): return True return False def get_account_index(self, xpath, account_id): for option in self.doc.xpath('//select[@id="%s"]/option' % xpath): if account_id in CleanText('.', replace=[(' ', '')])(option): return option.attrib['value'] else: raise TransferError("account %s not found" % account_id) def choose_recip(self, recipient): form = self.get_form(id='formulaire') form['indexCompteDestinataire'] = self.get_value(recipient.id, 'recipient') form.submit() def transfer(self, amount, reason): form = self.get_form(id='formulaire') form['libMontant'] = amount form['motifVirement'] = reason form.submit() def confirm(self): form = self.get_form(id='formulaire') form.submit() def get_value(self, _id, value_type): for div in self.doc.xpath('//div[@onclick]'): if _id in CleanText('.//div[not(@title)]', replace=[(' ', '')])(div): return Regexp(Attr('.', 'onclick'), '(\d+)')(div) raise TransferError('Could not find %s account.' % value_type) def choose_origin(self, account_transfer_id): form = self.get_form() form['indexCompteEmetteur'] = self.get_value(account_transfer_id, 'origin') form.submit() @method class iter_recipients(ListElement): item_xpath = '//div[@id="listeDestinataires"]//div[@class="pointeur cardCompte"]' class Item(ItemElement): klass = Recipient validate = lambda self, obj: self.obj_id(self) != self.env['account_transfer_id'] obj_id = CleanText('./div[@class="infoCompte" and not(@title)]', replace=[(' ', '')]) obj_label = CleanText('./div[1]') obj_bank_name = Env('bank_name') obj_category = Env('category') obj_iban = Env('iban') def obj_enabled_at(self): return datetime.now().replace(microsecond=0) def condition(self): return len(self.el.xpath('./div')) > 1 def parse(self, el): if bool(CleanText('./div[@id="soldeEurosCompte"]')(self)): self.env['category'] = u'Interne' account = find_object(self.page.browser.get_accounts_list(), _transfer_id=self.obj_id(self)) self.env['iban'] = account.iban if account else NotAvailable self.env['bank_name'] = u'LCL' else: self.env['category'] = u'Externe' self.env['iban'] = self.obj_id(self) self.env['bank_name'] = NotAvailable def check_data_consistency(self, account, recipient, amount, reason): try: assert CleanDecimal('//div[@class="topBox"]/div[@class="montant"]', replace_dots=True)(self.doc) == amount assert reason in CleanText('//div[@class="motif"]')(self.doc) assert account._transfer_id in CleanText(u'//div[div[@class="libelleChoix" and contains(text(), "Compte émetteur")]] \ //div[@class="infoCompte" and not(@title)]', replace=[(' ', '')])(self.doc) assert recipient.id in CleanText(u'//div[div[@class="libelleChoix" and contains(text(), "Compte destinataire")]] \ //div[@class="infoCompte" and not(@title)]', replace=[(' ', '')])(self.doc) except AssertionError: raise TransferError('data consistency failed.') def create_transfer(self, account, recipient, amount, reason): transfer = Transfer() transfer.currency = FrenchTransaction.Currency('//div[@class="topBox"]/div[@class="montant"]')(self.doc) transfer.amount = CleanDecimal('//div[@class="topBox"]/div[@class="montant"]', replace_dots=True)(self.doc) transfer.account_iban = account.iban transfer.recipient_iban = recipient.iban transfer.account_id = account.id transfer.recipient_id = recipient.id transfer.exec_date = date.today() transfer.label = reason transfer.account_label = account.label transfer.recipient_label = recipient.label transfer._account = account transfer._recipient = recipient transfer.account_balance = account.balance return transfer def fill_transfer_id(self, transfer): transfer.id = Regexp(CleanText(u'//div[@class="alertConfirmationVirement"]//p[contains(text(), "référence")]'), u'référence (\d+)')(self.doc) return transfer ��weboob-1.2/modules/lcl/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001600�13034501105�0016371�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LCLtTest(BackendTest): MODULE = 'lcl' def test_lcl(self): list(self.backend.iter_accounts()) ��������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015207�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0017322�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LdlcModule __all__ = ['LdlcModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/browser.py������������������������������������������������������������������0000664�0000000�0000000�00000004355�13034501105�0017253�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import HomePage, BillsPage class LdlcBrowser(LoginBrowser): home = URL('/default.aspx', HomePage) bills = URL('/Account/CommandListingPage.aspx', BillsPage) def __init__(self, website, *args, **kwargs): self.website = website if website == 'pro': self.BASEURL = 'https://secure.ldlc-pro.com/' else: self.BASEURL = 'https://secure.ldlc.com/' super(LdlcBrowser, self).__init__(*args, **kwargs) def do_login(self): self.location('/Account/LoginPage.aspx', data={'log' : self.username, 'pass': self.password}) self.home.stay_or_go() if not self.home.is_here(): raise BrowserIncorrectPassword @need_login def get_subscription_list(self): return self.home.stay_or_go().get_list() @need_login def iter_documents(self, subscription): self.bills.stay_or_go() bills = list() for value in self.page.get_range(): if self.website == 'pro': event = 'ctl00$cphMainContent$ddlDate' else: event = 'ctl00$ctl00$cphMainContent$cphMainContent$ddlDate' self.bills.go(data={event: value, '__EVENTTARGET': 'ctl00$cphMainContent$ddlDate'}) for i in self.page.get_documents(subid=subscription.id): bills.append(i) return bills �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000001535�13034501105�0017346�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@������ pHYs�� �� ����tIME ;->���PLTE����{|}~TUWXXYZKLLMNN?AB ? @ @ ADEEEE*Y*Z+Z+[1_2`2`EoFoLtMtMuMuTzڻMw���bKGDPnL��IDATXm[0 Zh;]amDv2o_pk<Il+l^KsNs'h)((@AW[ yf?'c@ZA $]+\]dI'Iݚ^A]xX19q ,0΀loi˸�/JfA`]=dQzK [2%dDGwIw<kn`S-c\C2t:<ð/)37!N.ux{0!![:O}Nd�dk/ ׏2lW� ZȖ,GW{+8뿂80U+s.f'CB$S -TC T><enNoVXlй" 7y7 ;Cy_�YfevHD>(HQ?\6z<C\N܉rVy\J0ɤg&P�Pw<f7N"T ����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000005265�13034501105�0017056�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Bill, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import LdlcBrowser __all__ = ['LdlcModule'] class LdlcModule(Module, CapDocument): NAME = 'ldlc' DESCRIPTION = u'ldlc website' MAINTAINER = u'Vincent Paredes' EMAIL = 'vparedes@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='Email'), ValueBackendPassword('password', label='Password'), Value('website', label='Site web', default='part', choices={'pro': 'Professionnels', 'part': 'Particuliers'})) BROWSER = LdlcBrowser def create_default_browser(self): return self.create_browser(self.config['website'].get(), self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) return self.browser.open(bill.url).content �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/pages.py��������������������������������������������������������������������0000664�0000000�0000000�00000005036�13034501105�0016664�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.filters.standard import CleanDecimal, CleanText, Env, Format from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.html import Attr from weboob.capabilities.bill import Bill, Subscription from weboob.tools.date import parse_french_date class HomePage(HTMLPage, LoggedPage): @method class get_list(ListElement): item_xpath = '//div[@id="divAccueilInformationClient"]//div[@id="divInformationClient"]' class item(ItemElement): klass = Subscription obj_subscriber = CleanText('.//div[@id="divlblTitleFirstNameLastName"]/span') obj_id = CleanText('.//span[2]') obj_label = CleanText('.//div[@id="divlblTitleFirstNameLastName"]/span') class BillsPage(HTMLPage, LoggedPage): def get_range(self): for value in self.doc.xpath('//div[@class="commandListing content clearfix"]//select/option/@value'): yield value @method class get_documents(ListElement): item_xpath = '//table[@id="TopListing"]//tr' class item(ItemElement): klass = Bill obj_id = Format('%s_%s', Env('subid'), CleanText('./td[3]')) obj_url = Attr('./td[@class="center" or @class="center pdf"]/a', 'href') obj_date = Env('date') obj_format = u"pdf" obj_type = u"bill" obj_price = CleanDecimal('./td[@class="center montant"]/span', replace_dots=True) def parse(self, el): self.env['date'] = parse_french_date(el.xpath('./td[2]')[0].text).date() def condition(self): return CleanText().filter(self.el.xpath('.//td')[-1]) != "" and len(self.el.xpath('./td[@class="center" or @class="center pdf"]/a/@href')) == 1 ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ldlc/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000002513�13034501105�0016541�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LdlcTest(BackendTest): MODULE = 'ldlc' def test_downloadbills(self): """ Iter all bills and try to download it. """ for subscription in self.backend.iter_subscription(): for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) def test_list(self): """ Test listing of subscriptions. """ subscriptions = list(self.backend.iter_subscription()) self.assertTrue(len(subscriptions) > 0, msg="Account listing failed") �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016241�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020351�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LeboncoinModule __all__ = ['LeboncoinModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000011031�13034501105�0020272�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.capabilities.housing import Query, TypeNotSupported from .pages import CityListPage, HousingListPage, HousingPage, PhonePage class LeboncoinBrowser(PagesBrowser): BASEURL = 'https://www.leboncoin.fr/' city = URL('ajax/location_list.html\?city=(?P<city>.*)&zipcode=(?P<zip>.*)', CityListPage) search = URL('(?P<type>.*)/offres/(?P<region>.*)/occasions/\?(?P<_ps>ps|mrs)=(?P<ps>.*)&(?P<_pe>pe|mre)=(?P<pe>.*)&ros=(?P<ros>.*)&location=(?P<location>.*)&sqs=(?P<sqs>.*)&sqe=(?P<sqe>.*)&ret=(?P<ret>.*)&f=(?P<advert_type>.*)', '(?P<_type>.*)/offres/(?P<_region>.*)/occasions.*?', HousingListPage) housing = URL('ventes_immobilieres/(?P<_id>.*).htm', HousingPage) phone = URL('https://api.leboncoin.fr/api/utils/phonenumber.json', PhonePage) TYPES = {Query.TYPE_RENT: 'locations', Query.TYPE_SALE: 'ventes_immobilieres', Query.TYPE_SHARING: 'colocations', } RET = {Query.HOUSE_TYPES.HOUSE: '1', Query.HOUSE_TYPES.APART: '2', Query.HOUSE_TYPES.LAND: '3', Query.HOUSE_TYPES.PARKING: '4', Query.HOUSE_TYPES.OTHER: '5'} def __init__(self, region, *args, **kwargs): super(LeboncoinBrowser, self).__init__(*args, **kwargs) self.region = region def get_cities(self, pattern): city = '' zip_code = '' if pattern.isdigit(): zip_code = pattern else: city = pattern return self.city.go(city=city, zip=zip_code).get_cities() def search_housings(self, query, advert_type, module_name): if query.type not in self.TYPES: return TypeNotSupported() type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, ret = self.decode_query(query, module_name) if len(cities) == 0 or len(ret) == 0: return list() return self.search.go(region=self.region, location=cities, ros=nb_rooms, sqs=area_min, sqe=area_max, _ps="mrs" if query.type == Query.TYPE_RENT else "ps", ps=cost_min, _pe="mre" if query.type == Query.TYPE_RENT else "pe", pe=cost_max, type=type, advert_type=advert_type, ret=ret).get_housing_list() def get_housing(self, _id, obj=None): housing = self.housing.go(_id=_id).get_housing(obj=obj) housing.phone = self.get_phone(_id) return housing def get_phone(self, _id): api_key = self.housing.stay_or_go(_id=_id).get_api_key() data = {'list_id': _id, 'app_id': 'leboncoin_web_utils', 'key': api_key, 'text': 1, } return self.phone.go(data=data).get_phone() def decode_query(self, query, module_name): cities = [c.name for c in query.cities if c.backend == module_name] ret = [self.RET.get(g) for g in query.house_types if g in self.RET] _type = self.TYPES.get(query.type) self.search.go(_type=_type, _region=self.region) nb_rooms = '' if not query.nb_rooms else self.page.get_rooms_min(query.nb_rooms) area_min = '' if not query.area_min else self.page.get_area_min(query.area_min) area_max = '' if not query.area_max else self.page.get_area_max(query.area_max) cost_min = '' if not query.cost_min else self.page.get_cost_min(query.cost_min, query.type) cost_max = '' if not query.cost_max else self.page.get_cost_max(query.cost_max, query.type) return _type, ','.join(cities), nb_rooms, area_min, area_max, cost_min, cost_max, '&ret='.join(ret) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000001376�13034501105�0020403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���% ���sRGB����gAMA�� a��� pHYs����od��IDAThCZK0 @p. F!cǎ;vb'v4)q~ЙnPqbV _�|Z 7RSK \OJ]XHIq.۱_E*�Vؑ(0�XyڗP�rET@uz�|uM@"B&OϮθlYF�Gw}W*I-�ު��$ �te`!7ǜwĶUX69=�C<_ � N�IbwyLI1Y7`Lբ#�$=I SeX .$6|ak�@"Hɼs|ʟswFS /UT}hV@Uhn 4#F֒pC/eh<5=-]VzY0dd ;i[eتX*)@46 q.;A.>X6"�# 'E�,�t9O1]Q^R1bV�dh^5F&o�$9-w3m9)S=3 &EQnQR96Y w|cTHX!Qp™?0^q^ۘMG@m+ k�*@S [ɚ6N}����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/module.py��������������������������������������������������������������0000664�0000000�0000000�00000007235�13034501105�0020107�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from weboob.tools.value import Value from .browser import LeboncoinBrowser __all__ = ['LeboncoinModule'] class LeboncoinModule(Module, CapHousing): NAME = 'leboncoin' DESCRIPTION = u'search house on leboncoin website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = LeboncoinBrowser CONFIG = BackendConfig(Value('advert_type', label='Advert type', choices={'c': 'Agency', 'p': 'Owner', 'a': 'All'}, default='a'), Value('region', label='Region', choices=['alsace', 'aquitaine', 'auvergne', 'basse_normandie', 'bourgogne', 'bretagne', 'centre', 'champagne_ardenne', 'corse', 'franche_comte', 'haute_normandie', 'ile_de_france', 'languedoc_roussillon', 'limousin', 'lorraine', 'midi_pyrenees', 'nord_pas_de_calais', 'pays_de_la_loire', 'picardie', 'poitou_charentes', 'provence_alpes_cote_d_azur', 'rhone_alpes', 'guadeloupe', 'martinique', 'guyane', 'reunion'])) def create_default_browser(self): region = self.config['region'].get() return self.create_browser(region) def get_housing(self, _id): return self.browser.get_housing(_id) def fill_housing(self, housing, fields): return self.browser.get_housing(housing.id, housing) def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo def search_city(self, pattern): return self.browser.get_cities(pattern) def search_housings(self, query): return self.browser.search_housings(query, self.config['advert_type'].get(), self.name) OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo} �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000020016�13034501105�0017711�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, pagination, JsonPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, Env, DateTime, BrowserURL, Format, Join from weboob.browser.filters.javascript import JSVar from weboob.browser.filters.html import Attr, Link from weboob.browser.filters.json import Dict from weboob.capabilities.housing import City, Housing, HousingPhoto, Query from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.housing.housing import PricePerMeterFilter from weboob.tools.date import DATE_TRANSLATE_FR, LinearDateGuesser from decimal import Decimal from datetime import date, timedelta import re class CityListPage(HTMLPage): @method class get_cities(ListElement): item_xpath = '//li' class item(ItemElement): klass = City obj_id = Format('%s %s', CleanText('./span[has-class("city")]'), CleanText('./span[@class="zipcode"]')) obj_name = Format('%s %s', CleanText('./span[has-class("city")]'), CleanText('./span[@class="zipcode"]')) class HousingListPage(HTMLPage): ENCODING = 'iso-8859-1' def get_area_min(self, asked_area): return self.find_select_value(asked_area, '//select[@id="sqs"]/option') def get_area_max(self, asked_area): return self.find_select_value(asked_area, '//select[@id="sqe"]/option') def get_rooms_min(self, asked_rooms): return self.find_select_value(asked_rooms, '//select[@id="rooms_ros"]/option') # def get_rooms_max(self, asked_rooms): # return self.find_select_value(asked_rooms, '//select[@id="roe"]/option') def get_cost_min(self, asked_cost, _type): _id = "ps" if _type == Query.TYPE_SALE else "mrs" return self.find_select_value(asked_cost, '//select[@id="%s"]/option' % _id) def get_cost_max(self, asked_cost, _type): _id = "pe" if _type == Query.TYPE_SALE else "mre" return self.find_select_value(asked_cost, '//select[@id="%s"]/option' % _id) def find_select_value(self, ref_value, selector): select = {} for item in self.doc.xpath(selector): if item.attrib['value']: select[CleanDecimal('.')(item)] = CleanDecimal('./@value')(item) select_keys = select.keys() select_keys.sort() for select_value in select_keys: if select_value >= ref_value: return select[select_value] return select[select_keys[-1]] if select else 0 @pagination @method class get_housing_list(ListElement): item_xpath = '//a[has-class("list_item")]' next_page = Format(u'http:%s', Link('//a[@id="next"]')) class item(ItemElement): klass = Housing def validate(self, obj): return obj.id is not None obj_url = Format(u'http:%s', Link('.')) obj_id = Regexp(Link('.'), '//www.leboncoin.fr/(ventes_immobilieres|locations|colocations)/(.*).htm.*', '\\2', default=None) obj_title = CleanText('./@title|./section/p[@class="item_title"]') obj_cost = CleanDecimal('./section[@class="item_infos"]/*[@class="item_price"]/text()', replace_dots=(',', '.'), default=Decimal(0)) obj_currency = Regexp(CleanText('./section[@class="item_infos"]/*[@class="item_price"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_text = Join(' - ', './/p[@class="item_supp"]') def obj_date(self): _date = CleanText('./section[@class="item_infos"]/aside/p[@class="item_supp"]/text()', replace=[('Aujourd\'hui', str(date.today())), ('Hier', str((date.today() - timedelta(1))))])(self) if not _date: return NotAvailable for fr, en in DATE_TRANSLATE_FR: _date = fr.sub(en, _date) self.env['tmp'] = _date return DateTime(Env('tmp'), LinearDateGuesser())(self) def obj_photos(self): photos = [] url = Attr('./div[@class="item_image"]/span/span/img', 'src', default=None)(self) if url: photos.append(HousingPhoto(url)) return photos class HousingPage(HTMLPage): ENCODING = 'iso-8859-1' def get_api_key(self): return JSVar(CleanText('//script'), var='apiKey', default=None)(self.doc) @method class get_housing(ItemElement): klass = Housing def parse(self, el): details = dict() self.env['area'] = NotAvailable for item in el.xpath('//div[@class="line"]/h2'): if 'Surface' in CleanText('./span[@class="property"]')(item): self.env['area'] = CleanDecimal(Regexp(CleanText('./span[@class="value"]'), '(.*)m.*'), replace_dots=(',', '.'))(item) else: key = u'%s' % CleanText('./span[@class="property"]')(item) if 'GES' in key or 'Classe' in key: details[key] = CleanText('./span[@class="value"]/noscript/a')(item) else: details[key] = CleanText('./span[@class="value"]')(item) self.env['details'] = details obj_id = Env('_id') obj_title = CleanText('//title') obj_cost = CleanDecimal('//h2[@itemprop="price"]/@content', default=Decimal(0)) obj_currency = Regexp(CleanText('//h2[@itemprop="price"]/span[@class="value"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_text = CleanText('//p[@itemprop="description"]') obj_location = CleanText('//span[@itemprop="address"]') obj_details = Env('details') obj_area = Env('area') obj_price_per_meter = PricePerMeterFilter() obj_url = BrowserURL('housing', _id=Env('_id')) def obj_date(self): _date = Regexp(CleanText('//p[has-class("line")]', replace=[(u'à', '')]), '.*Mise en ligne le (.*)')(self) for fr, en in DATE_TRANSLATE_FR: _date = fr.sub(en, _date) self.env['tmp'] = _date return DateTime(Env('tmp'), LinearDateGuesser())(self) def obj_photos(self): items = re.findall(r'images\[\d\]\s*=\s*"([\w/\.]*\.jpg)";', CleanText('//script')(self)) photos = [HousingPhoto(u'http:%s' % item) for item in items] if not photos: img = CleanText('//meta[@itemprop="image"]/@content', default=None)(self) if img: photos.append(HousingPhoto(img)) return photos class PhonePage(JsonPage): def get_phone(self): if Dict('utils/status')(self.doc) == u'OK': return Dict('utils/phonenumber')(self.doc) return NotAvailable ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leboncoin/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002600�13034501105�0017570�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import itertools from weboob.tools.test import BackendTest from weboob.capabilities.housing import Query class LeboncoinTest(BackendTest): MODULE = 'leboncoin' def test_leboncoin(self): query = Query() query.cities = [] query.type = Query.TYPE_SALE for city in self.backend.search_city('lille'): city.backend = self.backend.name query.cities.append(city) results = list(itertools.islice(self.backend.search_housings(query), 0, 20)) self.assertTrue(len(results) > 0) obj = self.backend.fillobj(results[0]) self.assertTrue(obj.url is not None, 'Missing url for "%s"' % (obj.id)) ��������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017072�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000001450�13034501105�0021203�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LeclercMobileModule __all__ = ['LeclercMobileModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/browser.py���������������������������������������������������������0000664�0000000�0000000�00000011716�13034501105�0021135�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import time import StringIO from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import HomePage, LoginPage, HistoryPage, PdfPage from weboob.capabilities.bill import Detail from weboob.capabilities.base import NotAvailable __all__ = ['Leclercmobile'] class Leclercmobile(Browser): DOMAIN = 'www.securelmobile.fr' PROTOCOL = 'https' ENCODING = 'utf-8' PAGES = {'.*pgeWERL008_Login.aspx.*': LoginPage, '.*EspaceClient/pgeWERL013_Accueil.aspx': HomePage, '.*pgeWERL009_ReleveConso.aspx.*': HistoryPage, '.*ReleveConso.ashx.*': PdfPage } accueil = "/EspaceClient/pgeWERL013_Accueil.aspx" login = "/EspaceClient/pgeWERL008_Login.aspx" conso = "/EspaceClient/pgeWERL009_ReleveConso.aspx" bills = '/EspaceClient/pgeWERL015_RecupReleveConso.aspx?m=-' def __init__(self, *args, **kwargs): Browser.__init__(self, *args, **kwargs) def home(self): self.location(self.accueil) def is_logged(self): return not self.is_on_page(LoginPage) def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.username.isdigit() if not self.is_on_page(LoginPage): self.location(self.login) form = self.page.login(self.username, self.password) # Site display a javascript popup to wait while self.page.iswait(): # In this popup can be an error displayed if self.page.iserror(): raise BrowserIncorrectPassword() time.sleep(1) self.page.next(self.username, form) # The last document contain a redirect url in the javascript self.location(self.page.getredirect()) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() def viewing_html(self): # To prevent unknown mimetypes sent by server, we assume we # are always on a HTML document. return True def get_subscription_list(self): if not self.is_on_page(HomePage): self.location(self.acceuil) return self.page.get_list() def get_subscription(self, id): assert isinstance(id, basestring) if not self.is_on_page(HomePage): self.location(self.accueil) l = self.page.get_list() for a in l: if a.id == id: return a return None def get_history(self): if not self.is_on_page(HistoryPage): self.location(self.conso) maxid = self.page.getmaxid() for i in range(maxid + 1): response = self.openurl(self.bills + str(i)) mimetype = response.info().get('Content-Type', '').split(';')[0] if mimetype == "application/pdf": pdf = PdfPage(StringIO.StringIO(response.read())) for call in pdf.get_calls(): call.label = call.label.strip() yield call def get_details(self): if not self.is_on_page(HistoryPage): self.location(self.conso) response = self.openurl(self.bills + "0") mimetype = response.info().get('Content-Type', '').split(';')[0] if mimetype == "application/pdf": pdf = PdfPage(StringIO.StringIO(response.read())) for detail in pdf.get_details(): yield detail def iter_documents(self, parentid): if not self.is_on_page(HistoryPage): self.location(self.conso) return self.page.date_bills(parentid) def get_document(self, id): assert isinstance(id, basestring) if not self.is_on_page(HistoryPage): self.location(self.conso) parentid = id[0:10] l = self.page.date_bills(parentid) for a in l: if a.id == id: return a def get_balance(self): if not self.is_on_page(HistoryPage): self.location(self.conso) detail = Detail() detail.label = u"Balance" for calls in self.get_history(): if "Votre solde" in calls.label: detail.price = calls.price return detail detail.price = NotAvailable return detail ��������������������������������������������������weboob-1.2/modules/leclercmobile/favicon.png��������������������������������������������������������0000664�0000000�0000000�00000001534�13034501105�0021230�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME 3c���tEXtComment�Created with GIMPW��IDATx[m0 LLA(}_VP_ dIH@Pĩmxd{sλ[]xy.[)@x!X�$B-[1Lϯw3"XA!"@<mxG6Z[�(C>) ,c F@&@[1c$H<D�^sSݏ@gPhOKd~wH9kO^`*@M4<6Ш^i`EJ*-yASPӠ+y>'F^o@ F$p;DvF_ISt>?� hKeeSP" Tc�VX#)JS(c=tpwP@/=襂89U% ,QGʠqgt"'#`Y &]kWi}K`=;MB`aEHjGHQ#<]9BD+CK-҈| D ſz}#i iSW$枣M0ٵ  IQn ݷ TVQf & =%%08ayJ@4R*{&' hQPy1*8Skgg`4%PǛ,N z';Eggs֘"\78onADw5 l�oЛ ����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/module.py����������������������������������������������������������0000664�0000000�0000000�00000007425�13034501105�0020741�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, SubscriptionNotFound,\ DocumentNotFound, Subscription, Bill from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Leclercmobile __all__ = ['LeclercMobileModule'] class LeclercMobileModule(Module, CapDocument): NAME = 'leclercmobile' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Leclerc Mobile website' CONFIG = BackendConfig(ValueBackendPassword('login', label='Account ID', masked=False, regexp='^(\d{10}|)$'), ValueBackendPassword('password', label='Password') ) BROWSER = Leclercmobile def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): for subscription in self.browser.get_subscription_list(): yield subscription def get_subscription(self, _id): if not _id.isdigit(): raise SubscriptionNotFound() with self.browser: subscription = self.browser.get_subscription(_id) if subscription: return subscription else: raise SubscriptionNotFound() def iter_documents_history(self, subscription): with self.browser: for history in self.browser.get_history(): if history.label != "Votre solde": yield history def get_document(self, id): with self.browser: bill = self.browser.get_document(id) if bill: return bill else: raise DocumentNotFound() def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) with self.browser: for bill in self.browser.iter_documents(subscription.id): yield bill # The subscription is actually useless, but maybe for the futur... def get_details(self, subscription): with self.browser: for detail in self.browser.get_details(): yield detail def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) with self.browser: return self.browser.readurl(bill.url) def get_balance(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) with self.browser: balance = self.browser.get_balance() balance.label = u"Balance %s" % subscription.id balance.id = "%s-balance" % subscription.id balance.currency = u'EUR' return balance �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/pages/�������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020171�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/pages/__init__.py��������������������������������������������������0000664�0000000�0000000�00000001603�13034501105�0022302�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .homepage import HomePage from .history import HistoryPage, PdfPage from .login import LoginPage __all__ = ['LoginPage', 'HomePage', 'HistoryPage', 'PdfPage'] �����������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/pages/history.py���������������������������������������������������0000664�0000000�0000000�00000015733�13034501105�0022255�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re import os import subprocess import tempfile import shutil from datetime import datetime, date, time from decimal import Decimal from weboob.deprecated.browser import Page from weboob.capabilities.bill import Detail, Bill def _get_date(detail): return detail.datetime class PdfPage(): def __init__(self, file): self.pdf = file def _parse_pdf(self): pdffile = tempfile.NamedTemporaryFile(bufsize=100000, mode='w', suffix='.pdf') temptxt = pdffile.name.replace('.pdf', '.txt') cmd = "ebook-convert" stdout = open("/dev/null", "w") shutil.copyfileobj(self.pdf, pdffile) pdffile.flush() subprocess.call([cmd, pdffile.name, temptxt], stdout=stdout) pdffile.close() txtfile = open(temptxt, 'r') txt = txtfile.read() txtfile.close() os.remove(temptxt) return txt def get_details(self): txt = self._parse_pdf() page = txt.split('CONSOMMATION')[2].split('ACTIVITE DETAILLEE')[0] lines = page.split('\n') lines = [x for x in lines if len(x) > 0] # Remove empty lines details = [] detail = None lines.pop(-1) # Line to describes pictures twolines = False for line in lines: if "Votre consommation" in line: line = line.split(": ", 1)[1] if twolines: twolines = False detail.infos = unicode(line, encoding='utf-8') elif re.match('[A-Za-z]', line[0]): # We have a new element, return the other one if detail is not None: details.append(detail) detail = Detail() split = re.split("(\d)", line, maxsplit=1) detail.price = Decimal(0) if len(split) > 2: detail.infos = unicode(split[1] + split[2], encoding='utf-8') else: twolines = True if '€' in line: specialprice = split[1] + split[2] detail.price = Decimal(specialprice.replace('€', '')) detail.label = unicode(split[0], encoding='utf-8') elif '€' in line: detail.price = Decimal(line.replace('€', '')) else: detail.infos = unicode(line, encoding='utf-8') details.append(detail) return details # Standard pdf text extractor take text line by line # But the position in the file is not always the "real" position to display... # It produce some unsorted and unparsable data # Example of bad software: pdfminer and others python tools # This is why we have to use "ebook-convert" from calibre software, # it is the only one to 'reflow" text and give some relevant results # The bad new is that ebook-convert doesn't support simple use with stdin/stdout def get_calls(self): txt = self._parse_pdf() pages = txt.split("DEBIT") pages.pop(0) # remove headers details = [] for page in pages: page = page.split('RÉGLO MOBILE')[0].split('N.B. Prévoir')[0] # remove footers lines = page.split('\n') lines = [x for x in lines if len(x) > 0] # Remove empty lines numitems = (len(lines) + 1) / 4 # Each line has five columns lines.pop(0) # remove the extra € symbol modif = 0 i = 0 while i < numitems: if modif != 0: numitems = ((len(lines) + 1 + modif) / 4) base = i * 4 - modif dateop = base corres = base + 1 duree = base + 2 price = base + 3 if "Changement vers le Forfait" in lines[base]: modif += 1 i += 1 continue # Special case with 5 columns, the operation date is not in the first one if len(re.split("(\d+\/\d+\/\d+)", lines[dateop])) < 2: lines[base + 1] = lines[base] + " " + lines[base + 1] dateop = base + 1 corres = base + 2 duree = base + 3 price = base + 4 modif -= 1 detail = Detail() splits = re.split("(\d+\/\d+\/\d+)", lines[dateop]) mydate = date(*reversed([int(x) for x in splits[1].split("/")])) mytime = time(*[int(x) for x in splits[2].split(":")]) detail.datetime = datetime.combine(mydate, mytime) if lines[corres] == '-': lines[corres] = "" if lines[duree] == '-': lines[duree] = '' detail.label = unicode(splits[0], encoding='utf-8', errors='replace') + u" " + lines[corres] + u" " + lines[duree] # Special case with only 3 columns, we insert a price if "Activation de votre ligne" in detail.label or u"Résiliation" in detail.label: lines.insert(price, '0') try: detail.price = Decimal(lines[price].replace(',', '.')) except: # In some special cases, there are no price column. Try to detect it if "Inclus" not in lines[price]: modif += 1 detail.price = Decimal(0) details.append(detail) i += 1 return sorted(details, key=_get_date, reverse=True) class HistoryPage(Page): def on_loaded(self): pass def getmaxid(self): max = 1 while len(self.document.xpath('//li[@id="liMois%s"]' % max)) > 0: max += 1 return max - 1 def date_bills(self, parentid): max = 1 while len(self.document.xpath('//li[@id="liMois%s"]' % max)) > 0: li = self.document.xpath('//li[@id="liMois%s"]' % max)[0] max += 1 link = li.xpath('a')[0] bill = Bill() bill.url = unicode(link.attrib['href']) bill.label = unicode(link.text) bill.format = u"pdf" bill.type = u"bill" bill.id = parentid + bill.label.replace(' ', '') yield bill �������������������������������������weboob-1.2/modules/leclercmobile/pages/homepage.py��������������������������������������������������0000664�0000000�0000000�00000002737�13034501105�0022341�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import Subscription from weboob.deprecated.browser import Page class HomePage(Page): def on_loaded(self): pass def get_list(self): l = [] phone = unicode(self.document.xpath('//span[@id="ctl00_ctl00_cMain_cEspCli_lblMsIsdn"]')[0].text.replace(' ', '')) self.browser.logger.debug('Found ' + phone + ' has phone number') phoneplan = unicode(self.document.xpath('//span[@id="ctl00_ctl00_cMain_cEspCli_aoaOffreActuelle_aooOffreEtOptions"]/dl/dd/span')[0].text) self.browser.logger.debug('Found ' + phoneplan + ' has subscription type') subscription = Subscription(phone) subscription.label = phone + ' - ' + phoneplan l.append(subscription) return l ���������������������������������weboob-1.2/modules/leclercmobile/pages/login.py�����������������������������������������������������0000664�0000000�0000000�00000005425�13034501105�0021661�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import StringIO from weboob.deprecated.browser import Page from weboob.deprecated.mech import ClientForm class LoginPage(Page): def on_loaded(self): pass def login(self, login, password): form = list(self.browser.forms())[0] self.browser.select_form("aspnetForm") self.browser.set_all_readonly(False) self.browser.controls.append(ClientForm.TextControl('text', '__ASYNCPOST', {'value': "true"})) self.browser['__EVENTTARGET'] = "ctl00$cMain$lnkValider" self.browser['ctl00$cMain$ascSaisieMsIsdn$txtMsIsdn'] = login.encode('iso-8859-1') self.browser['ctl00$cMain$txtMdp'] = password.encode('iso-8859-1') self.browser.submit(nologin=True) return form def iswait(self): spanwait = self.document.xpath('//span[@id="ctl00_ascAttente_timerAttente"]') return len(spanwait) > 0 def iserror(self): error = self.document.xpath('//span[@id="ctl00_cMain_ascLibErreur_lblErreur"]') return len(error) > 0 def getredirect(self): string = StringIO.StringIO() self.document.write(string) try: redirect = string.getvalue().split('pageRedirect')[1].split('|')[2] except: redirect = '' return redirect def next(self, login, form): self.browser.form = form string = StringIO.StringIO() self.document.write(string) controlvalue = string.getvalue().split('__EVENTVALIDATION')[1].split('|')[1] state = string.getvalue().split('__VIEWSTATE')[1].split('|')[1] self.browser.controls.append(ClientForm.TextControl('text', 'ctl00$objScriptManager', {'value': "ctl00$ascAttente$panelAttente|ctl00$ascAttente$timerAttente"})) self.browser['__VIEWSTATE'] = state self.browser['__EVENTTARGET'] = "ctl00$ascAttente$timerAttente" self.browser['__EVENTVALIDATION'] = controlvalue self.browser['ctl00$cMain$ascSaisieMsIsdn$txtMsIsdn'] = login.encode('iso-8859-1') self.browser['ctl00$cMain$txtMdp'] = "" self.browser.submit(nologin=True) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/leclercmobile/test.py������������������������������������������������������������0000664�0000000�0000000�00000004264�13034501105�0020431�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LeclercMobileTest(BackendTest): MODULE = 'leclercmobile' def test_list(self): """ Test listing of subscriptions . No support of multi-account on the website, we could assume to have only one subscription. Check the balance if the subscription is ok. """ subscriptions = list(self.backend.iter_subscription()) self.assertTrue(len(subscriptions) == 1, msg="Account listing failed") self.assertTrue(self.backend.get_balance(subscriptions[0]) > 0, msg="Get balance failed") def test_downloadbills(self): """ Iter all bills and try to download it. """ for subscription in self.backend.iter_subscription(): for bill in self.backend.iter_documents(subscription.id): self.backend.download_document(bill.id) def test_history(self): for subscription in self.backend.iter_subscription(): self.assertTrue(len(list(self.backend.iter_documents_history(subscription))) > 0) def test_details(self): for subscription in self.backend.iter_subscription(): details = list(self.backend.get_details(subscription)) self.assertTrue(len(details) > 5, msg="Not enough details") total = 0 for d in details: total += d.price self.assertTrue(total > 0, msg="Parsing of price failed") ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016061�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001510�13034501105�0020167�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"NewspaperFigaroModule init" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewspaperFigaroModule __all__ = ['NewspaperFigaroModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000002513�13034501105�0020117�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"browser for lefigaro website" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import ArticlePage from weboob.browser.browsers import AbstractBrowser from weboob.browser.url import URL class NewspaperFigaroBrowser(AbstractBrowser): "NewspaperFigaroBrowser class" PARENT = 'genericnewspaper' BASEURL = '' article_page = URL('http://lefigaro.fr/(.*)/(\d{4})/(\d{2})/(\d{2})/(.*$)', 'http://\w+.lefigaro.fr/(.*)/(\d{4})/(\d{2})/(\d{2})/(.*$)', ArticlePage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000001610�13034501105�0020212�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 5T,���iTXtComment�����Created with GIMPd.e��IDATx_HSQ?:MM&e!9IRS{(% HB(S􏐢'!zE4"+(P,*0B313v{/sݛcw9_ܝ9&QEA`\ H� b:Ó}ס;\π +�)l(XM&LWlsAj@lHOi{5B o];j[߭(\�) ]ns@svۓ_<O<: `t"n+aOmGNNÕ:淫.@�#5ҬP=c/㰫.= KCT Ԍvۉ��Ūc jX( ꀯproM TR ǿ:̽ 8 |ƇGm⃛nA! ,P\VN dm`j ѢㅭMAꂳc Ԯ}>*y$&H� [qLxq شJ0UgN_s@׀ �FCpp m=kF �c:C #!. KDɋ@]3) Ԕh.fl)Zacv .xQ 'tR|p'BBl AnwՕ D@;@�$� @�$� @�$� B?V.����IENDB`������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/module.py���������������������������������������������������������������0000664�0000000�0000000�00000010546�13034501105�0017726�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://www.lefigaro.fr" from weboob.tools.newsfeed import Newsfeed from weboob.tools.backend import AbstractModule from weboob.tools.backend import BackendConfig from weboob.tools.value import Value from weboob.capabilities.messages import CapMessages, Thread from .browser import NewspaperFigaroBrowser from .tools import rssid class NewspaperFigaroModule(AbstractModule, CapMessages): MAINTAINER = u'Julien Hebert' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'lefigaro' DESCRIPTION = u'Le Figaro French newspaper website' BROWSER = NewspaperFigaroBrowser RSS_FEED = 'http://rss.lefigaro.fr/lefigaro/laune?format=xml' RSSID = staticmethod(rssid) RSSSIZE = 30 PARENT = 'genericnewspaper' CONFIG = BackendConfig(Value('feed', label='RSS feed', choices={'actualites': u'actualites', 'flash-actu': u'flash-actu', 'politique': u'politique', 'international': u'international', 'actualite-france': u'actualite-france', 'hightech': u'hightech', 'sciences': u'sciences', 'sante': u'sante', 'lefigaromagazine': u'lefigaromagazine', 'photos': u'photos', 'economie': u'economie', 'societes': u'societes', 'medias': u'medias', 'immobilier': u'immobilier', 'assurance': u'assurance', 'retraite': u'retraite', 'placement': u'placement', 'impots': u'impots', 'conso': u'conso', 'emploi': u'emploi', 'culture': u'culture', 'cinema': u'cinema', 'musique': u'musique', 'livres': u'livres', 'theatre': u'theatre', 'lifestyle': u'lifestyle', 'automobile': u'automobile', 'gastronomie': u'gastronomie', 'horlogerie': u'horlogerie', 'mode-homme': u'mode-homme', 'sortir-paris': u'sortir-paris', 'vins': u'vins', 'voyages': u'voyages', 'sport': u'sport', 'football': u'football', 'rugby': u'rugby', 'tennis': u'tennis', 'cyclisme': u'cyclisme', 'sport-business': u'sport-business'})) def __init__(self, *args, **kwargs): super(self.__class__, self).__init__(*args, **kwargs) self.RSS_FEED = "http://www.lefigaro.fr/rss/figaro_%s.xml" % self.config['feed'].get() def iter_threads(self): for article in Newsfeed(self.RSS_FEED, self.RSSID).iter_entries(): thread = Thread(article.id) thread.title = article.title thread.date = article.datetime yield(thread) ����������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000006071�13034501105�0017536�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"ArticlePage object for lefigaro" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.html import CSS from weboob.browser.filters.standard import CleanText class ArticlePage(AbstractPage): "ArticlePage object for lefigaro" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_loaded(self): self.main_div = self.doc.getroot() self.element_title_selector = "h1" self.element_author_selector = 'span[itemprop="author"], span.auteur_long>div' self.element_body_selector = "article div.fig-article-body" def get_body(self): element_body = self.get_element_body() self.drop_comments(element_body) self.try_drop_tree(element_body, "script") self.try_drop_tree(element_body, "liste") self.try_remove_from_selector_list(element_body, ["div#article-comments", "div.infos", "div.photo", "div.art_bandeau_bottom", "div.view", "span.auteur_long", "#toolsbar", 'link', 'figure']) for image in self._selector('img')(element_body): if image.attrib['src'].endswith('coeur-.gif'): image.drop_tree() for div in self._selector('div')(element_body): if div.text == ' Player Figaro BFM ': obj = div.getnext() a = obj.getnext() if obj.tag == 'object': obj.drop_tree() if a.tag == 'a' and 'BFM' in a.text: a.drop_tree() div.drop_tree() # This part of the article seems manually generated. check_next = False for crappy_content in self._selector('b, a')(element_body): if check_next is True: # Remove if it has only links if crappy_content.tag == 'a': element_body.remove(crappy_content) check_next = False if crappy_content.text == 'LIRE AUSSI :' or crappy_content.text == 'LIRE AUSSI:': element_body.remove(crappy_content) check_next = True txts = element_body.find_class("texte") if len(txts) > 0: txts[0].drop_tag() element_body.tag = "div" return CleanText('.')(element_body) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002135�13034501105�0017413�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.tools.html import html2text class LeFigaroTest(BackendTest): MODULE = 'lefigaro' def test_lefigaro(self): l = list(self.backend.iter_threads()) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) assert len(html2text(thread.root.content)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lefigaro/tools.py����������������������������������������������������������������0000664�0000000�0000000�00000001550�13034501105�0017574�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"tools for lefigaro backend" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. def url2id(url): "return an id from an url" return url def rssid(entry): return url2id(entry.id) ��������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016421�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001504�13034501105�0020532�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"NewspaperLibeModule init" # -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewspaperLibeModule __all__ = ['NewspaperLibeModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/browser.py������������������������������������������������������������0000664�0000000�0000000�00000002235�13034501105�0020460�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import ArticlePage from weboob.browser.browsers import AbstractBrowser from weboob.browser.url import URL class NewspaperLibeBrowser(AbstractBrowser): "NewspaperLibeBrowser class" PARENT = 'genericnewspaper' BASEURL = '' article = URL('http://.*liberation.fr/.*', ArticlePage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000002622�13034501105�0020556�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME ' v���tEXtComment�Created with GIMPW��IDATxoUǿgvȯ6�Kl4En Q !$ȃ11]m֤J؅Re۝n0eg;]4'3g9{gν3`0 `0 `xv8­B/h%xׅ$rd O�hjx\` & �@/@bnV�LK':HC"$E1�ҭ3D&cc%a4m !FHPE‗$bL NAk E@�Ķ J']K �6 gn~�ȣ!�AydK@R`!aJMRj%]OsG.RwKTlR@jIJ4h LtsivG^r;_|M5aoҹ4Heh捖ơ~ȅ%.zzr9\Y �u:}t?s-:]=^Ug=wG;w| IRem`~<wRT_/s}*U:P�0n9^9xR*)Q j.Khf؎kU_x$Y~LX_Oe[f<}.糹~ Kj�–x��aQl$ mƝ3^+ʶgm{CYB˅?BcU`M.n615p�{�zg ?<pi'f'|{МH!:No?I2}^WOVix[Sv�]gj{g!h._�y-;zl59ֶP,2Q*$I~A ^Z w^u>'?KRQ+_%ʋ]NoBBnc@k}JkU?=g}E�z{1?w };,_~;>}ntsֶ:;E�=-yoaˆIv+LPwDAǯV_/jee*+ز.9>b4zvُ+=P#귡&waږ:gsտ[ K+䂗4Gi j@ZѤWO@h76xFu}i# +48^LXpaf{ ~2 XP!>w@g1+`0 `0 `0 O)Xuݧު����IENDB`��������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/module.py�������������������������������������������������������������0000664�0000000�0000000�00000005313�13034501105�0020262�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://liberation.fr" from weboob.tools.newsfeed import Newsfeed from weboob.capabilities.messages import CapMessages, Thread from weboob.tools.backend import AbstractModule from weboob.tools.backend import BackendConfig from weboob.tools.value import Value from .browser import NewspaperLibeBrowser from .tools import rssid, url2id class NewspaperLibeModule(AbstractModule, CapMessages): MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'liberation' DESCRIPTION = u'Libération newspaper website' BROWSER = NewspaperLibeBrowser RSSID = staticmethod(rssid) URL2ID = staticmethod(url2id) RSSSIZE = 30 PARENT = 'genericnewspaper' CONFIG = BackendConfig(Value('feed', label='RSS feed', choices={'9': u'A la une sur Libération', '10': u'Monde', '11': u'Politiques', '12': u'Société', '13': u'Économie', '14': u'Sports', '17': u'Labo: audio, vidéo, diapos, podcasts', '18': u'Rebonds', '44': u'Les chroniques de Libération', '53': u'Écrans', '54': u'Next', '58': u'Cinéma' })) def __init__(self, *args, **kwargs): super(self.__class__, self).__init__(*args, **kwargs) self.RSS_FEED = "http://www.liberation.fr/rss/%s" % self.config['feed'].get() def iter_threads(self): for article in Newsfeed(self.RSS_FEED, self.RSSID).iter_entries(): thread = Thread(article.id) thread.title = article.title thread.date = article.datetime yield(thread) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000004330�13034501105�0020072�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.html import XPathNotFound, CSS from weboob.browser.filters.standard import CleanText class ArticlePage(AbstractPage): "ArticlePage object for Libe" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_loaded(self): self.main_div = self.doc.getroot() self.element_title_selector = "head>title" self.element_author_selector = "span.author" self.element_body_selector = "div.article-body, div[itemprop=articleBody]" def get_body(self): if '.blogs.liberation.fr/' in self.url: self.element_body_selector = "div.entry-content" try: element_body = self.get_element_body() self.try_remove(element_body, "script") return CleanText('.')(element_body) except XPathNotFound: meta = self.doc.xpath('//meta[@name="description"]')[0] txt = meta.attrib['content'] return txt except AttributeError: return "No content found" def get_title(self): title = super(self.__class__, self).get_title() return title.replace(u' - Libération', '') def get_author(self): try: author = CleanText('.')(self.get_element_author()) if author.startswith('Par '): return author.split('Par ', 1)[1] else: return author except AttributeError: return '' ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/liberation/test.py���������������������������������������������������������������0000664�0000000�0000000�00000001665�13034501105�0017762�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LiberationTest(BackendTest): MODULE = 'liberation' def test_new_messages(self): for message in self.backend.iter_unread_messages(): pass ���������������������������������������������������������������������������weboob-1.2/modules/liberation/tools.py��������������������������������������������������������������0000664�0000000�0000000�00000001553�13034501105�0020137�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re def url2id(url): return re.sub(u'[^\d]', '', url.split('0Dxtor')[0].split('0I')[-1]) def rssid(entry): return entry.link �����������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016306�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0020424�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LinuxJobsModule __all__ = ['LinuxJobsModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000002665�13034501105�0020354�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import SearchPage, AdvertPage import urllib class LinuxJobsBrowser(PagesBrowser): BASEURL = 'https://www.linuxjobs.fr' advert_page = URL('/jobs/(?P<id>.+)', AdvertPage) search_page = URL('/search/(?P<job>)', SearchPage) def get_job_advert(self, _id, advert): self.advert_page.go(id=_id) assert self.advert_page.is_here() return self.page.get_job_advert(obj=advert) def search_job(self, pattern=None): if pattern is None: return [] self.search_page.go(job=urllib.quote_plus(pattern.encode('utf-8'))) assert self.search_page.is_here() return self.page.iter_job_adverts() ���������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000007563�13034501105�0020454�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME  ?Q���iTXtComment�����Created with GIMPd.e��IDATxݛsu/3IkwvũRT9}^x+k^$E6ݧHV"i9"%ssZx93޻9ּęxs38;q}Wox?2{yFH p g˟l& QOzB%[awu6>+~ImwDow 5lkˊݨy'CD83C*;v#znD "q;Û}7Dc4qI!9g1滺}۽GXXyFp;`T>q}ȨzA?C1b5x'("3o BFr&wD "T <ǙCaWT?Q'<^2^sq1pojXI*h6[4D ]`~8s3Xy;d_2/V/"=\`M›` M-ɑrf_T˰ZgV!sG!]0ӳOpD CH!_g+`DP@sL^6r#4X |$h JyTQuİ:d3`/\U)> k;ޑ�g1Tv?E::eBREE hloÚ "C<ag/V<_2T) p^{ !"k޷#c'Xm Ń&&4G XA:83xsH?`Xh{ u**-_^Lqƛ1E@"h&':9>8ϩ!=w/b"͆ZX|US[O|Ɋ~kаy(%GDH&1YB(\[)o&fd90V%>F૚GG3XcH$M O8u䜙7K/O٘e[È}zXaBn kv#O"|ʍv>;~/W8c )Ҧ@Ο'+~US ?#8[\Z\u؃]j75}o�Ozn0<j xu=;<Ie=MlYN0{}MHWgĔ ޺뵬H}u1e] q}s*o-Obl0* }S~%W` &Bo}*1#k.] ʭ%C:9X{7hN)CHi6fʺ{/ -X:ޖq =\~E5q8ʞU|ЦHjmiB'!%?M~#=s5u6v|(oQ ͑ ?؋`6ڿXS4ڶe6 {;tqɏӷ'5ΎX\S*vO�{1JFx;Xkm鏭52YƀY 1҄.oD«x6r5S~8{_oxuvrU m LWBn{sP}[#S^Onj/N/Xcƾf\GhVa|윷3&sWg̚%Ih~C�-J=↫ -cOWdr1%wyϚ(rҊY\1k70qGh LFgxvSO'&ohc1ڇReHNH&oR??$Uf͂N߰!}ż&ؿ]B_Aw%~9߄bƋԲL+ -X&8p$uVsJ�Bao* =}.E7^ۤPy:7׵'bp@3|UfloHL@E è? ~{ MU}^%:"qz0Ts+~@Qެ6_ݔsժ-\Xi@<>=gG]}߯9s>dWRᵍdmGS5uCuU =?FUԮ;O`Q5_Y|5b /Y�|O?IY>Vv/?)>=-W*?5K/E�VJ/?ki~,bY2<k- mtg Juhѭv$M5 v?e ll'e!f!pX{O|4oÝY) ǚk.ugb"ߣ?2_ #˳z8 �b lavā6BgƧi  wW]?ƈ|6Hn17Ko7F(w? lDnG䭅KGkn"Fb-K9: ';qoXSĆ߭1T]e}Me=֬'_wۏ-A5tAu{bf3 O0bvXҎ6Ek,:j_ Vk 4\MVexGn@lfDFDƛlKQS6&!rV.Xs.q^Fd. V GP~7xhg<NngAU/L.Gn\a^M2:c.fM޼n-+Ah;Vq<hgeHjOI d Trq3&gY.&2!0>?~`01,9߽3)m ʒ?սs6] a"'b'd4(RK?`m{Xp4: OF^ D%3^_Фp vT@Ԇe|2S-vLOMF=M y,xS=iSS"/|lAȕ!IF[C^98] \ÄU<c4%r"t c #ऀX[: 4&i, 55I.YTwv}~F/XW&N:!锐(jv@sDZr1޴ bקd6t^ARl;uA6/f+6]Ѥ+Vy8a10c QHLBX`jخ",YrID"lww}C]AZWfd-7Gn#9Cy;eN1v2 1yLcTs+4 '6fm9lи؍>%f u*ܦVGa|>2S4!S4)FҌ6L'PeCV:SՍH١l18fn-UK/f?9檝tf?! m:!)QϹjtT 5,$D"52Fr6!m:fm#d@Mz*LY1-eǤ<&)A/h⌘wnPQr8JDP(J4b0oNfv#Ds"e|"2cfӄ>_6pɲꊁh2gf4wݐtife3a6Oy)WͻNyxE3g{s9y@>} +;{5vȄ?gXADҢ+vamRRN)r&9kw~owu׹~hH-f0j����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/module.py��������������������������������������������������������������0000664�0000000�0000000�00000003715�13034501105�0020153�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.job import CapJob from .browser import LinuxJobsBrowser __all__ = ['LinuxJobsModule'] class LinuxJobsModule(Module, CapJob): NAME = 'linuxjobs' DESCRIPTION = u'linuxjobs website' MAINTAINER = u'François Revol' EMAIL = 'revol@free.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = LinuxJobsBrowser def advanced_search_job(self): """ Iter results of an advanced search :rtype: iter[:class:`BaseJobAdvert`] """ raise NotImplementedError() def get_job_advert(self, _id, advert=None): """ Get an announce from an ID. :param _id: id of the advert :type _id: str :param advert: the advert :type advert: BaseJobAdvert :rtype: :class:`BaseJobAdvert` or None if not found. """ return self.browser.get_job_advert(_id, advert) def search_job(self, pattern=None): """ Iter results of a search on a pattern. :param pattern: pattern to search on :type pattern: str :rtype: iter[:class:`BaseJobAdvert`] """ for job_advert in self.browser.search_job(pattern): yield job_advert ���������������������������������������������������weboob-1.2/modules/linuxjobs/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000004357�13034501105�0017770�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.job import BaseJobAdvert from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Regexp, CleanText, Date, Env, BrowserURL from weboob.browser.filters.html import Link, CleanHTML from weboob.tools.date import parse_french_date class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Env('id') obj_url = BrowserURL('advert_page', id=Env('id')) obj_title = CleanText('//title') obj_job_name = CleanText('//title') obj_society_name = CleanText('//div[2]/div[@class="col-md-9"]/h4[1]') obj_publication_date = Date(CleanText('//div[2]/div[@class="col-md-9"]/small', replace=[(u'Ajoutée le', '')]), parse_func=parse_french_date) obj_place = Regexp(CleanText('//div[2]/div[@class="col-md-9"]/h4[2]'), '(.*) \(.*\)') obj_description = CleanHTML('//div[4]/div[@class="col-md-9"]') class SearchPage(HTMLPage): @method class iter_job_adverts(ListElement): item_xpath = '//a[@class="list-group-item "]' class item(ItemElement): klass = BaseJobAdvert obj_id = Regexp(Link('.'), '.*fr/jobs/(\d+)/.*') obj_title = CleanText('h4/span[@class="job-title"]') obj_society_name = CleanText('h4/span[@class="job-company"]') obj_publication_date = Date(CleanText('h4/span[@class="badge pull-right"]'), parse_func=parse_french_date) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/linuxjobs/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002131�13034501105�0017634�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LinuxJobsTest(BackendTest): MODULE = 'linuxjobs' def test_linuxjobs_search(self): l = list(self.backend.search_job('linux')) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/logicimmo/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016250�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/logicimmo/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020360�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LogicimmoModule __all__ = ['LogicimmoModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/logicimmo/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000006675�13034501105�0020323�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.capabilities.housing import Query, TypeNotSupported from .pages import CitiesPage, SearchPage, HousingPage, PhonePage class LogicimmoBrowser(PagesBrowser): BASEURL = 'http://www.logic-immo.com/' city = URL('asset/t9/getLocalityT9.php\?site=fr&lang=fr&json=%22(?P<pattern>.*)%22', CitiesPage) search = URL('(?P<type>location-immobilier|vente-immobilier|recherche-colocation)-(?P<cities>.*)/options/(?P<options>.*)', SearchPage) housing = URL('detail-(?P<_id>.*).htm', HousingPage) phone = URL('(?P<urlcontact>.*)', PhonePage) TYPES = {Query.TYPE_RENT: 'location-immobilier', Query.TYPE_SALE: 'vente-immobilier', Query.TYPE_SHARING: 'recherche-colocation'} RET = {Query.HOUSE_TYPES.HOUSE: '2', Query.HOUSE_TYPES.APART: '1', Query.HOUSE_TYPES.LAND: '3', Query.HOUSE_TYPES.PARKING: '10', Query.HOUSE_TYPES.OTHER: '14'} def get_cities(self, pattern): if pattern: return self.city.go(pattern=pattern).get_cities() def search_housings(self, type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, house_types): if type not in self.TYPES: raise TypeNotSupported() options = [] ret = [] for house_type in house_types: if house_type in self.RET: ret.append(self.RET.get(house_type)) if len(ret): options.append('groupprptypesids=%s' % ','.join(ret)) options.append('pricemin=%s' % (cost_min if cost_min else '0')) if cost_max: options.append('pricemax=%s' % cost_max) options.append('areamin=%s' % (area_min if area_min else '0')) if area_max: options.append('areamax=%s' % area_max) if nb_rooms: if type == Query.TYPE_SHARING: options.append('nbbedrooms=%s' % ','.join([str(i) for i in range(nb_rooms, 7)])) else: options.append('nbrooms=%s' % ','.join([str(i) for i in range(nb_rooms, 7)])) self.search.go(type=self.TYPES.get(type, 'location-immobilier'), cities=cities, options='/'.join(options)) if type == Query.TYPE_SHARING: return self.page.iter_sharing() return self.page.iter_housings() def get_housing(self, _id, housing=None): return self.housing.go(_id=_id).get_housing(obj=housing) def get_phone(self, _id): if _id.startswith('location') or _id.startswith('vente'): urlcontact, params = self.housing.stay_or_go(_id=_id).get_phone_url_datas() return self.phone.go(urlcontact=urlcontact, params=params).get_phone() �������������������������������������������������������������������weboob-1.2/modules/logicimmo/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000006767�13034501105�0020423�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����gAMA�� a��� pHYs����+�� IDATx^Y lkK wE@Ql]eI:3FI&@2n@. db@=A6Q  u q "nI˥;__l><~uǡQ@Bໝr@]P)wu>5!ӑ:Ϸϣ<)ߑzuno1w~Ҙd:Uʀʭ,@BtX܉C*dUgVq9u] y@'ϣjbBd[AlPǪ @*v"D`@<jy-T� 5cЛ�S:*S 8{�3<wP�%A�܄`r+Ljl{HrISYP1.DOQwI0Qw7T*@Tv3X s@(jzWFHʊ .Ä29I쩢Jl#!!z%A F68HļzJu:?֣1X#%Jo�T2PtT*EY={3[=U- �em,*~ su}]68:_9A k[ dE 6}lxqIX?&wxafF`@iiie>' 5jpBӧy[\ڲe ۷y4h(a{XC9sݻin:ݻ�i;7ޠ ΍7y Ǐoքݻ\`; � qF*++#&L`2BfffRII+--;wݻtQV21ƍr;wX¹I&y۶m֭[2Ge+V(,&@�oҤI&a E?cW�֭[(YYYb0+�:G<x/ƨa-[4:(\xZnmdBj͛7'1 U6mڴ{9y/ضmc/]$ݎ;dիWiʔ)|ѢE y>vXy cfEƮ]۷ @e.6m2Ԝ<yr˗/7 <x, E?3ez/~i#3##.$a(WPP`tRڵÇӑ#GT2!+PTTdisSwI ^d ]V^Mo6[޾}yVDٳgEB:k+Udt,}1E2={NPk׮b ͛m5t2�2Ν;gÓ7(TW@k{g9�wX_l(x`?72DaذadѣG ciW(!h`њ5kD/ +V v{H5k&k-''t=?PPPTϛڥK |W) ֆqƙ5S~a| � =ëy@=΅G �kڴ kEs { <;TR}@(D[Ep^6m}FY)^ӡ� :TA1�X~ Lb 6B_֑W�<A:tbq %+:nvM#{C 2Ұ?YSgQ4cL6m͘1ON8ٳ'!l>CZjo>IRrxG4N7K<6=؊c߅xP:4i4}zytʨI]{ͅgp;NL乎y0E׉�M<�@�b"1F0JPXY Iv4U') @0a# kV #깛i t_i}}t"l;>\AaVqX vQ.Q8tP6:hZٳgLaPJ#СC4k+b< oG^e􇍔՚ON\޻8-̏{!ad#VzCF뻨3zN,D/8Plcitckr|L1ؙ  O4Nݺ;"6ݠ*ke%Ը҂%`_`4S]A(0lt&rZ#'3 ϊ\+YJ=FxB_z%^|EQX33SFGf+D'X*QrBId[.qF^?p.Мٜ4`.PH烈IO"c._Sfd׷o_i^4?T�e+S:'a?6FßIL+ފQ<p pf'}:'eQyk({_葂3Զ,u7jwtLr -IOtH xݸI7K uڵkrq9\twޢͳ4Jy{_fY0# +%Bd귢vyoR)cʚ:L~lwoӏ2 R*^LSN{uZڏ=Zի+$#i\B\'wlArGQ�~} <ۍN|My ko*ag?эti`4*�(>fh��x$g}V"cْΊ $~--]G=@~_@| 8Dx]tdG/{WF1]|ouEǻr`NןJ7oCN>M(ux~ :uꔀ�+FI'OWF19tiktGΏr%b?ԅ/@^]LNs Jo:HnJQ&G%e8@-O/vwwX'#Mδ :ƥU^e /@kdES . @*E>+P#n@hrWaj3@!]@v0=řzm2?.9LR%>=cܔD:J= 0cΖ!52a+uim�C.^'r.xf0a\{!^zAx",S*cNޮKtFh'l@ qp2DCǬ gYi??IL;*Q㥔%Pg# F8ġ�`+#<e{b&<[hx+?q$O0 -4<&:㬲 k^ T h^0;Š{Hn)H8KU0T}~;dP[=WSz1rP[=WSz1rP[=WSz1rP[=W#zye����IENDB`���������weboob-1.2/modules/logicimmo/module.py��������������������������������������������������������������0000664�0000000�0000000�00000005732�13034501105�0020116�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from weboob.capabilities.base import UserError from .browser import LogicimmoBrowser __all__ = ['LogicimmoModule'] class LogicImmoCitiesError(UserError): """ Raised when more than 3 cities are selected """ def __init__(self, msg='You cannot select more than three cities'): UserError.__init__(self, msg) class LogicimmoModule(Module, CapHousing): NAME = 'logicimmo' DESCRIPTION = u'logicimmo website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = LogicimmoBrowser def get_housing(self, housing): if isinstance(housing, Housing): id = housing.id else: id = housing housing = None housing = self.browser.get_housing(id, housing) housing.phone = self.browser.get_phone(id) return housing def search_city(self, pattern): return self.browser.get_cities(pattern) def search_housings(self, query): cities_names = ['%s' % c.name.replace(' ', '-') for c in query.cities if c.backend == self.name] cities_ids = ['%s' % c.id for c in query.cities if c.backend == self.name] if len(cities_names) == 0: return list() if len(cities_names) > 3: raise LogicImmoCitiesError() cities = ','.join(cities_names + cities_ids) return self.browser.search_housings(query.type, cities.lower(), query.nb_rooms, query.area_min, query.area_max, query.cost_min, query.cost_max, query.house_types) def fill_housing(self, housing, fields): self.browser.get_housing(housing.id, housing) if 'phone' in fields: housing.phone = self.browser.get_phone(housing.id) return housing def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo, } ��������������������������������������weboob-1.2/modules/logicimmo/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000020626�13034501105�0017727�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, JsonPage from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import Format, CleanText, Regexp, CleanDecimal, Date, Env, BrowserURL from weboob.browser.filters.html import XPath, CleanHTML from weboob.capabilities.housing import Housing, HousingPhoto, City from weboob.capabilities.base import NotAvailable from weboob.tools.capabilities.housing.housing import PricePerMeterFilter class CitiesPage(JsonPage): @method class get_cities(DictElement): item_xpath = '*/children' class item(ItemElement): klass = City def condition(self): return Dict('lct_parent_id')(self) != '0' obj_id = Format('%s_%s', Dict('lct_id'), Dict('lct_level')) obj_name = Format('%s %s', Dict('lct_name'), Dict('lct_post_code')) class PhonePage(HTMLPage): def get_phone(self): return CleanText('//div[has-class("phone")]', children=False)(self.doc) class HousingPage(HTMLPage): @method class get_housing(ItemElement): klass = Housing obj_id = Env('_id') obj_title = CleanText(CleanHTML('//meta[@itemprop="name"]/@content')) obj_area = CleanDecimal(Regexp(CleanText(CleanHTML('//meta[@itemprop="name"]/@content')), '(.*?)(\d*) m\xb2(.*?)', '\\2', default=NotAvailable), default=0) obj_cost = CleanDecimal('//*[@itemprop="price"]', default=0) obj_currency = Regexp(CleanText('//*[@itemprop="price"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_price_per_meter = PricePerMeterFilter() obj_date = Date(Regexp(CleanText('//p[@class="offer-description-notes"]|//p[has-class("darkergrey")]'), u'.* Mis à jour : (\d{2}/\d{2}/\d{4}).*')) obj_text = CleanHTML('//div[@class="offer-description-text"]|//div[has-class("offer-description")]') obj_location = CleanText('//*[@itemprop="address"]') obj_url = BrowserURL('housing', _id=Env('_id')) def obj_photos(self): photos = [] for img in XPath('//div[@class="carousel-content"]/ul/li/a/img/@src|//div[@class="carousel"]/ul/li/a/img/@src')(self): photos.append(HousingPhoto(u'%s' % img.replace('75x75', '800x600'))) return photos def obj_details(self): details = {} energy = CleanText('//div[has-class("energy-summary")]/span[@class="section-label"]|//div[has-class("energy-summary")]/div/span[@class="section-label"]', default='')(self) energy_value = CleanText('//div[has-class("energy-summary")]/span[@class="energy-msg"]', default='')(self) if energy and energy_value: details[energy] = energy_value greenhouse = CleanText('//div[has-class("greenhouse-summary")]/span[@class="section-label"]|//div[has-class("greenhouse-summary")]/div/span[@class="section-label"]', default='')(self) greenhouse_value = CleanText('//div[has-class("greenhouse-summary")]/span[@class="energy-msg"]', default='')(self) if greenhouse and greenhouse_value: details[greenhouse] = greenhouse_value for li in XPath('//ul[@itemprop="description"]/li')(self): label = CleanText('./div[has-class("criteria-label")]')(li) value = CleanText('./div[has-class("criteria-value")]')(li) details[label] = value return details def get_phone_url_datas(self): a = XPath('//button[has-class("offer-contact-vertical-phone")]')(self.doc)[0] urlcontact = 'http://www.logic-immo.com/modalMail' params = {} params['universe'] = CleanText('./@data-univers')(a) params['source'] = CleanText('./@data-source')(a) params['pushcontact'] = CleanText('./@data-pushcontact')(a) params['mapper'] = CleanText('./@data-mapper')(a) params['offerid'] = CleanText('./@data-offerid')(a) params['offerflag'] = CleanText('./@data-offerflag')(a) params['campaign'] = CleanText('./@data-campaign')(a) params['xtpage'] = CleanText('./@data-xtpage')(a) params['offertransactiontype'] = CleanText('./@data-offertransactiontype')(a) params['aeisource'] = CleanText('./@data-aeisource')(a) params['shownumber'] = CleanText('./@data-shownumber')(a) params['corail'] = 1 return urlcontact, params class SearchPage(HTMLPage): @method class iter_sharing(ListElement): item_xpath = '//article[has-class("offer-block")]' class item(ItemElement): klass = Housing obj_id = Format('colocation-%s', CleanText('./div/header/@id', replace=[('header-offer-', '')])) obj_title = CleanText(CleanHTML('./div/header/section/p[@class="property-type"]/span/@title')) obj_area = CleanDecimal('./div/header/section/p[@class="offer-attributes"]/a/span[@class="offer-area-number"]', default=0) obj_cost = CleanDecimal('./div/header/section/p[@class="price"]', default=0) obj_currency = Regexp(CleanText('./div/header/section/p[@class="price"]', default=NotAvailable), '.* ([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_text = CleanText('./div/div[@class="content-offer"]/section[has-class("content-desc")]/p/span[has-class("offer-text")]/@title') obj_date = Date(Regexp(CleanText('./div/header/section/p[has-class("update-date")]'), ".*(\d{2}/\d{2}/\d{4}).*")) obj_location = CleanText('(./div/div[@class="content-offer"]/section[has-class("content-desc")]/p)[1]/span/@title') @method class iter_housings(ListElement): item_xpath = '//div[@class="offer-block "]' class item(ItemElement): klass = Housing obj_id = Format('%s-%s', Regexp(Env('type'), '(.*)-.*'), CleanText('./@id', replace=[('header-offer-', '')])) obj_title = CleanText('./div/div/div[@class="offer-details-wrapper"]/div/div/p[@class="offer-type"]/span/@title') obj_area = CleanDecimal('./div/div/div[@class="offer-details-wrapper"]/div/div/div/div/h3/a/span[@class="offer-area-number"]', default=0) obj_cost = CleanDecimal(Regexp(CleanText('./div/div/div[@class="offer-details-wrapper"]/div/div/p[@class="offer-price"]/span', default=NotAvailable), '(.*) [%s%s%s]' % (u'€', u'$', u'£'), default=NotAvailable), default=0) obj_currency = Regexp(CleanText('./div/div/div[@class="offer-details-wrapper"]/div/div/p[@class="offer-price"]/span', default=NotAvailable), '.* ([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_date = Date(Regexp(CleanText('./div/div/div[has-class("offer-picture-more")]/div/p[@class="offer-update"]'), ".*(\d{2}/\d{2}/\d{4}).*")) obj_text = CleanText('./div/div/div[@class="offer-details-wrapper"]/div/div/div/p[has-class("offer-description")]/span') obj_location = CleanText('./div/div/div[@class="offer-details-wrapper"]/div/div/div/div/h2') ����������������������������������������������������������������������������������������������������������weboob-1.2/modules/logicimmo/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002653�13034501105�0017607�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import itertools from weboob.capabilities.housing import Query from weboob.tools.test import BackendTest class LogicimmoTest(BackendTest): MODULE = 'logicimmo' def test_logicimmo(self): query = Query() query.area_min = 20 query.cost_max = 900 query.cities = [] query.type = Query.TYPE_RENT for city in self.backend.search_city('paris'): if len(query.cities) >= 3: break city.backend = self.backend.name query.cities.append(city) results = list(itertools.islice(self.backend.search_housings(query), 0, 20)) self.assertTrue(len(results) > 0) self.backend.fillobj(results[0], 'phone') �������������������������������������������������������������������������������������weboob-1.2/modules/lolix/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015420�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001430�13034501105�0017527�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LolixModule __all__ = ['LolixModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000003616�13034501105�0017463�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser.decorators import id2url from weboob.deprecated.browser import Browser from .job import LolixJobAdvert from .pages import SearchPage, AdvertPage import urllib __all__ = ['LolixBrowser'] class LolixBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'fr.lolix.org/search/offre' ENCODING = 'iso-8859-1' PAGES = { '%s://%s/search.php' % (PROTOCOL, DOMAIN): SearchPage, '%s://%s/offre.php\?id=(?P<id>.+)' % (PROTOCOL, DOMAIN): AdvertPage, } def advanced_search_job(self, region=0, poste=0, contrat=0, limit_date=0, pattern=None): data = { 'mode': 'find', 'page': '0', 'posteid': poste, 'contratid': contrat, 'regionid': region, 'limitjour': limit_date } self.location('%s://%s/search.php' % (self.PROTOCOL, self.DOMAIN), urllib.urlencode(data)) assert self.is_on_page(SearchPage) return self.page.iter_job_adverts(pattern) @id2url(LolixJobAdvert.id2url) def get_job_advert(self, url, advert): self.location(url) assert self.is_on_page(AdvertPage) return self.page.get_job_advert(url, advert) ������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000022616�13034501105�0017562�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs����+���tIME)֤�� �IDATxyx]WuksYd<v;ؙ@ L!@h Ҥ !LJ PJ %IPr ibHۉCAI<ʒ%˚;s8W|ys=Hwz}7o?"3V׋imEEPUUmЊ"K3>MrB ..=3Ļ=ai:MsҧҞ ("+:e{e=[˟h%DG$ (Vd`*+"D@+WTU`(sZW:UY0"(("5u`YNU#"Ccxo:ٍ͜/OiI),!bDa@xL?qtppLmla.ÂF"S'+@N`&B㥁il7P�4<�@a%nY5rՅqω2<+h$X8�#"B "!ʢ_aø_vU `zEI<Rq !Bl6ZK&kwJiP5kp&x.Zb9Vi`>nṾPSFS2A%""DCHlH*ДF1hD!02W|JϾ\1mp&jek_G>6S(p]W66D#*b)v}t{6#/|Kh1COgI%> .*2ăƷ[<ABD<ͯиq'~r)}vhqDZͭJzeNƄ�!�B u˻ɰ*qnj&y{82G*H(P5T@0))8%j0,dCp;?N~wԿ0p@q:mj�A#�],אkuS# @)j(x8P"1(XTq` D,md#O$ym( &Kt#dmR²Iϒ^^{Lo9.ͯo ϥ>SE@.r8jc/DgKs8LvO (pA"![ #,H=Os_9 x/=Ŭwn럲C 4H EHmk}l-Dl_F1mx64_Qo=J tҥ;Fu{GR[$4ڐ/HOt| 3L[hYCwgbw`.¾mD)kzNϺ+ $tMYQgmҪ1JO<?cpt-$, +XY`�NlDD%& ,h!EF<lb1-b VH| FgG ?EO|5~yز,nwXQx5 )K^8 ; 0j⩊VuҪ͒ټoYFjTCG58.^_NJ@U,`EQH ,%zcWMݔ]B&)]Tnȯ|{ӫ__A"\-\K»mHEгj)~*:] vr1fbRa (b(y: b g! I3?$OID)u^-~Upt6S5:B+k~JA>{[>}#>9HN uQ\Ƌ8 ^'7=vI|fiܽ4IΗ➳CcZ"VAg^w?C#L93˔e;}V,H,IrQΌ1wȧ e>}?ɯ$4&9;L^̃?\G(ځ/5D푺~z5W1-i q*)`"QĚ>i�?6pŐ\@u 1g$TT<IvN4._߬#8>ʯvH,/y64M<ë8k9zz59Qx4e|52\t_HP<kQTC÷qGn0�E%  (vx~/f$OGoc-> lð恹AKMe{ (b$H,dˈS'-"E%�-ۼov)=s'YQ |Usćs+a(A0IVEUt]PQ°deP"%(x۽B�lij?t=fnjoO3zOC)2vb2ɘ=o':Ur{u^<rG[b5Wn̲݌ɑ?hH%pma[W-($+QbFQHzʁ4!Zʵ!n2v\P'8[wSg*vW4qѿA!qǛ+g:duQ>.A~0ɹ؞0.dq>{ћofl`vC")LbrB?]C~B&R}Ǘ>ʁ f?( IW+00cltS(b# O"_#|-a$E2c9yRz}*f+wr) JjɕD2^f#\$k)#̥9p)Q�)[۴|-bYT/> VQ Nxu_f"L 0"5`ુf鯪"#b(J:BkGo ZxhA7P1f|p~j/9̹!#P)l;{6XŸ*K0Q>m;oy'[6\ G`2$�kڴMt |}%<ZpGNKwҧ%$v%-d( ѵy'OQaXiN7X2i,KD2ȓ�G)oX8L(;X=2v*"[~rH#(Nٌ!] V%!OPEqx4l\PY/N3`U"0;Ωmۿd)~_QO <s92\IޗR6c`3Xe-*ј@ϰ}/qz콠sPHTTl$l*/'(^Quo[EL:&a(TC!;Erx=;YkHc<* OWOo[ȬZ*꾑{Mh~=n{j<UN&vr1 TCzbl!4.Ȏ$ PS{t;^Fm/P5f ~9e>H=,oߟb@o 0;ͷ)G^ed(qx#\wBkTw+_>S❖x{*&�sS)im(7!?n;{1<(Ð�I8Ul`xbab%*=Nc[15;sXyn`.'v\\5RǠC\Ô  ?|]DЋ<X:khHGhc'_}r"XSCr)Xٯ7hm>ְGp*CH"}udO,"J 0OΦj.XC9G9Yo@ x0}6vk+ɺr Ng OgOchx%tA,mR[^F "r}tm?Lhmc.*1z'TVhC4%`7$A)<<1bA2xUqaū0x*gHf3{hڒ7 Og"őOX,HYh _2@TR�(X*SIs!grl!? )8]dÚ䮟P?+b}gV ځNJuъ u9 "Q/Fx piH{xBwDaNT!=΋4DY&�ՅiKd쾾O\y s HySH&f^i3DpdKe8Y2$BBE6Bn ;7I mJFr#ܲr,'.NYxSĚDFb =ۗf{ rʰT!GTcH6JYۙ簬`g%Tj\Ei*(yRXKu-̒YEEC{۞d|VV˚V4&7gx~ukEndӌU ;+N0L=XL8ބ_c<A""5Dciǹ]}+SЯ@JsSW#()"9x,Y#FᜁP8g7ITh$F<N6K~`w%zYE l(˳]!GoWXjp5@\FE-H* T4:"Bi5BTT?x#Y^KfѬܔ5Lγ(dE~M>`D !HZ2U}_u Cع|*=%h},[t&J^y:C 'aэG:ބ0 !~E)+ǯ-#Y=NLa"G(ܪRI67^Jr2K{!yoئ|+xb P.Onߵ>zJVA#ItUqU^dwi 83t~zG&+$G аct-Ԗ8xLjrDd}7ɪnG$dD`#@%|!_8Cɖ}(.r:ulS"Q7#܇(8�CHPjJf_wߞ95pxۿ?ƌe?g_ ;Xktv&q$Mu9Nd:IUC~s|~<"{ #YŖ H$LY<3,#)hqi8Щf2~Nv{1ջI/tc#4tNZ,KuO)Xx KNVݺw1Í^uK~Ǜ0Qʖ:.rK~(|Y1c!սT042xWچ88RLOSz_eGy̹0g HdgTѱ:Yu <<Q*?tzbƧUscc=_"FSq$/W7,lm("g7Pdpp#CՆ,Ufn&+T]8fjE'vXv ypQ|x c;F lb sZ�X& u'Nn_>OIM^̙E״*9+0ux p &g yvMSp<*: @aTMJk!_G*?4/s l?|ӻ<s#|J~sht+n<D$q0H"d즅So:`>`Tj;<<y}Oav."S3TtՑ;ee)*_&Yut!=+X6tCoνTro_PMJi*G !s5hqXysn-OɻỸ3~o0kyz]GpH%sݳ*XZUB\\s=g<S^ߗ3ְlx.k}"jS>5֐$SkF&cIZCX``12~aZ  /髷|S4A >D$ !b/R m~K9o¼߼ã6<g'/֡C1R3\5ʚ 3`.f-L"̮O.cֿRN?’;84Rxw} dԳ1)0V0"7oۿN ُTx\z4Ȓ:=ݮ yIfW*G G[Li_Y[v-\xLFo|[Io_n(| kM$|2e >x41bH+VK㉁MTsjdj}Q#FdP%yI$9rё(g'br"d<G3�ұ}:x[FI£+dmoΊoZFu=r-_r~E3V [<C50j)YSgTSS R 0 >Jb8 Ƕ0{F3ւ %͸O [ot<:(zF_<CfQ?r]U;x4[-<U2dE&Ic"#%qUbD>t8S"LoPc0'IfQ "�%%IDNc x'_<EW[jLpT`=NU ";_,Ĥ}~iNeXP8:s[y<%-!I sl)qB 7!it&\F.j/a}[R92__WI` U#C w\]}tGe1JgעlãejXN6=MNYS/%xQQ?7m?BTx`}K"i)OX 1BJr9[ Jz"xXkHZd2!)IS̟11PVR /*FyU~_Ey6 G ?[ @P(=!003?~5%7ߞ´tPي Rb2)R-ͼ!Q,o2Oz$NxtPCLх"$ E *rd=DmBղN6]fvdM: r,~ (/ī5buT$Q{%`$`Ē8K% c9X'jQ6#(aMi͕1Hxn >W$VWCIiW,WIS*KBAI�y8WKٱhi$!T%_}=变bpϺvd Nqa\Y4P<+DDIj mXbc}`}9_AՈߠN;KEI dC#0G0Ι9~RyF$e%x9K!š`&"kJ2̄SEP 2Ue8D2XFcct >a@+mP eR'rLDU'q*Oq+i~QGou"$BGev%-p.f## #ISPV:|>'m Еc_O<̏dELLt{7) ec{Kg{Y)`ε{ru:Y[5a XXBE\DDjY2grUx;ZgK9b$_d#UӺ:!x",6DO^1΢{/׿V ;TdkHQ!UmKE5˾9+,O r" gF+_.wGdvr.TL ecͶ*x2 m,*&";Fgx\=eÆC 0z=<sE @.D!P*I0V yKX'(dR$m?KXuU/Tɩ86_D7MSNtBo��IDAT<RRn 7ʶk?kt7;F֝3FM1euIc,jq" " DЦAxeqQs+>FrDy )eLAz F(g9g � _qIJ-jXcYV湗P5o .Z( e]{_L ې$Q z12-)+CG0(""P'/CL&iT.09(4K oj% q) t.N|wǹnYNdRN<8Z7^M'Xh1 3$_l8i<n\gWMϪcg6d9s$Фk b|8p3B@7ݲw^Hbv `&D"D9ōNǹݬ_iQNgJnm|ybM=4[qQ4Hј!j9B LY~}Q~b8r/ levEe K:>XłIG@ի:}[>ܾ%WQ=/-XƛYX~F ~nIu)>D ?laBbJdȶ?nRӫj-)})"Uс!KuCtn|8Nsxtc#"GTRWDPJ"q]VW0/Z69Dc+_`y 8(p<~~ ?k֬nƹxq5Q%-T#=|$iȾ1/cgwzaV]w® 8=S35ᆚ2ԓ>o.'`(SE\0D쐉Q85O0{h Bv3~=,ԯ8y׆W?+ޡ'J;lre,COL"cDi3Ҷ~<l4<6w5 q;n<EBE>AXo7 w8dR4Ti,uA)^Kw鉚}2ۋ^HU%z~GYS*@^mOR1GY_HQ5Uߤr?Rgq[UMk7ez!?lRU[)S^U2y*&@ mj&& |4-=TގEwC>d 6qJ m' t |Ozj����IENDB`������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/job.py���������������������������������������������������������������������0000664�0000000�0000000�00000001642�13034501105�0016547�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.job import BaseJobAdvert class LolixJobAdvert(BaseJobAdvert): @classmethod def id2url(cls, _id): return 'http://fr.lolix.org/search/offre/offre.php?id=%s' % _id ����������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/module.py������������������������������������������������������������������0000664�0000000�0000000�00000014167�13034501105�0017270�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from weboob.capabilities.job import CapJob from .browser import LolixBrowser from .job import LolixJobAdvert __all__ = ['LolixModule'] class LolixModule(Module, CapJob): NAME = 'lolix' DESCRIPTION = u'Lolix French free software employment website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' BROWSER = LolixBrowser region_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '0': u'-- Indifférent --', '100000000': u'-- France entière', '100100000': u'-- France métropolitaine', '100100001': u'-- Alsace', '100100002': u'-- Auvergne', '100100003': u'-- Aquitaine', '100100004': u'-- Bourgogne', '100100005': u'-- Bretagne', '100100025': u'-- Centre', '100100027': u'-- Champagne-Ardenne', '100100030': u'-- Corse', '100100037': u'-- Franche-Comté', '100100040': u'-- Ile de France', '100100044': u'-- Languedoc-Roussillon', '100100048': u'-- Limousin', '100100051': u'-- Lorraine', '100100055': u'-- Midi-Pyrénées', '100100060': u'-- Nord-Pas-de-Calais', '100100073': u'-- Normandie', '100100076': u'-- Pays-de-Loire', '100100079': u'-- Picardie', '100100082': u'-- Poitou-Charentes', '100100085': u'-- Provence Alpes Cote d\'azur', '100100090': u'-- Rhône Alpes', '100200000': u'-- DOM et TOM', '100200001': u'-- Guadeloupe', '100200002': u'-- Guyane', '100200003': u'-- Martinique', '100200004': u'-- Réunion', '100200005': u'-- Saint-Pierre et Miquelon', '200000000': u'-- Etranger', }.iteritems())]) poste_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '0': u'-- Indifférent --', '100000000': u'-- Service Technique', '100005000': u'-- Administrateur base de données', '100004000': u'-- Admin. Système/Réseaux', '100004004': u'-- Administrateur système', '100004002': u'-- Administrateur réseaux', '100007000': u'-- Analyste', '100002000': u'-- Chef de projet', '100002001': u'-- Chef de projet junior', '100002002': u'-- Chef de projet senior', '100021000': u'-- Consultant', '100003000': u'-- Développeur', '100003001': u'-- Développeur junior', '100003002': u'-- Développeur senior', '100009000': u'-- Directeur technique', '100006000': u'-- Ingénieur d\'étude', '100011000': u'-- Ingénieur support', '100012000': u'-- Responsable R & D', '100010000': u'-- Technicien', '100010002': u'-- Technicien hotline', '100010003': u'-- Technicien maintenance', '100020000': u'-- Webmaster', '200000000': u'-- Service Commercial', '200300000': u'-- Commercial', '200200000': u'-- Directeur commercial', '200100000': u'-- Technico commercial', '400000000': u'-- Service Marketing', '400100000': u'-- Responsable Marketing', '300000000': u'-- Service qualité', '300100000': u'-- Assistant qualité', '300200000': u'-- Responsable qualité', '2000000': u'-- Fondateur', '7000000': u'-- Formateur', '6000000': u'-- Journaliste', '500100000': u'-- Assistant(e) de direction', '4000000': u'-- Stagiaire', '5000000': u'-- Traducteur', }.iteritems())]) ''' '000000' in order to display description in console question the rule is : len(key) > 5 or ' ' in key: ''' contrat_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '000000': u'-- Indifférent --', '6': u'Alternance', '5': u'Apprentissage', '2': u'CDD', '1': u'CDI', '4': u'Freelance', '3': u'Stage', }.iteritems())]) limit_date_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '2592000': u'30 jours', '5184000': u'60 jours', '7776000': u'90 jours', '0': u'Illimitée', }.iteritems())]) CONFIG = BackendConfig(Value('region', label=u'Région', choices=region_choices), Value('poste', label=u'Poste', choices=poste_choices), Value('contrat', label=u'Contrat', choices=contrat_choices), Value('limit_date', label=u'Date limite', choices=limit_date_choices)) def search_job(self, pattern=None): with self.browser: for job_advert in self.browser.advanced_search_job(pattern=pattern): yield job_advert def advanced_search_job(self): for advert in self.browser.advanced_search_job(region=self.config['region'].get(), poste=self.config['poste'].get(), contrat=int(self.config['contrat'].get()), limit_date=self.config['limit_date'].get()): yield advert def get_job_advert(self, _id, advert=None): with self.browser: return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): self.get_job_advert(advert.id, advert) OBJECTS = {LolixJobAdvert: fill_obj} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lolix/pages.py�������������������������������������������������������������������0000664�0000000�0000000�00000006647�13034501105�0017106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page import dateutil.parser import re from .job import LolixJobAdvert class AdvertPage(Page): def get_job_advert(self, url, advert): tables = self.document.getroot().xpath('//td[@class="Contenu"]/table') rows = self.parser.select(tables[2], 'tr') if not advert: advert = LolixJobAdvert(self.group_dict['id']) advert.url = url advert.society_name = u'%s' % self.parser.select(tables[3], 'tr/td/a', 1, method='xpath').text return self.fill_job_advert(rows, advert) def fill_job_advert(self, rows, advert): advert.title = u'%s' % self.parser.select(rows[0], 'td', 1).text_content() isDescription = False for row in rows: cols = self.parser.select(row, 'td') if isDescription: advert.description = u'%s' % cols[0].text_content() isDescription = False elif cols[0].text == u'Poste :': advert.job_name = u'%s' % cols[1].text_content() elif cols[0].text == u'Contrat :': advert.contract_type = u'%s' % cols[1].text_content() elif cols[0].text and cols[0].text.find(u'Rémunération :') != -1: advert.pay = u'%s' % cols[1].text_content() elif cols[0].text and cols[0].text.find(u'Région :') != -1: advert.place = u'%s' % cols[1].text_content() elif cols[0].text == u'Détails :': isDescription = True #else: # print cols[0].text return advert class SearchPage(Page): def iter_job_adverts(self, pattern): rows = self.document.getroot().xpath('//td[@class="Contenu"]/table/tr') for row in rows: cols = self.is_row_advert(row) if cols is not None: advert = self.create_job_advert(cols) if pattern: if pattern in advert.title: yield advert else: yield advert def is_row_advert(self, row): cols = self.parser.select(row, 'td') if len(cols) > 1: d = dict(cols[1].attrib) if 'class' in d.keys(): if 'ListeDark' == d['class'] or 'ListeLight' == d['class']: return cols def create_job_advert(self, cols): a = self.parser.select(cols[2], 'a')[0] advert = LolixJobAdvert(re.match(r'offre.php\?id=(.*)', a.attrib['href']).group(1)) advert.publication_date = dateutil.parser.parse(cols[0].text).date() advert.society_name = u'%s' % self.parser.select(cols[1], 'a')[0].text advert.title = u'%s' % a.text return advert �����������������������������������������������������������������������������������������weboob-1.2/modules/lolix/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000002122�13034501105�0016746�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LolixTest(BackendTest): MODULE = 'lolix' def test_lolix_advanced_search(self): l = list(self.backend.advanced_search_job()) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lutim/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015423�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lutim/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001430�13034501105�0017532�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LutimModule __all__ = ['LutimModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lutim/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000003647�13034501105�0017472�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import math from urlparse import urljoin from StringIO import StringIO from weboob.browser import PagesBrowser, URL from .pages import ImagePage, UploadPage class LutimBrowser(PagesBrowser): BASEURL = 'https://lut.im' VERIFY = False # XXX SNI is not supported image_page = URL('/(?P<id>.+)', ImagePage) upload_page = URL('/', UploadPage) def __init__(self, base_url, *args, **kw): PagesBrowser.__init__(self, *args, **kw) self.base_url = self.BASEURL = base_url def fetch(self, paste): self.location(paste.id) assert self.image_page.is_here() paste.contents = unicode(self.page.contents.encode('base64')) paste.title = self.page.filename def post(self, paste, max_age=0): bin = paste.contents.decode('base64') name = paste.title or 'file' # filename is mandatory filefield = {'file': (name, StringIO(bin))} params = {'format': 'json'} if max_age: params['delete-day'] = math.ceil(max_age / 86400.) self.location('/', data=params, files=filefield) assert self.upload_page.is_here() info = self.page.fetch_info() paste.id = urljoin(self.base_url, info['short']) �����������������������������������������������������������������������������������������weboob-1.2/modules/lutim/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000001125�13034501105�0017555�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD���`>��� pHYs�� �� ����tIME�!&���tEXtComment�Created with GIMPW��IDATxQ0D!hmo$V6j"UMh^f j Q-B!;�F�ǽ F#¨�H.qZȫؼfXQ�oQ'b�@ �o m:ruA qSK.h؊ ,8`aJ�]` R{}yZ_ LZN@�ZV`N_Vg&@i7!H<O^z%NI5B[ pB8VZY[L8W|*TBR|">,ÛgDC#1 eqp˩x}C(G}�3 !ܘK�$n@^CCsA<(LkJ�{wuM�S0 @s渢3y:~ �@d7ꛗȀT[/,Z�0YA̓j綩GEE٫?يB =����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lutim/module.py������������������������������������������������������������������0000664�0000000�0000000�00000004745�13034501105�0017274�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from urlparse import urljoin from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.paste import CapPaste, BasePaste from weboob.tools.capabilities.paste import image_mime from weboob.tools.value import Value from .browser import LutimBrowser __all__ = ['LutimModule'] class LutimModule(Module, CapPaste): NAME = 'lutim' DESCRIPTION = u'lutim website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = LutimBrowser CONFIG = BackendConfig(Value('base_url', label='Hoster base URL', default='https://lut.im/')) @property def base_url(self): url = self.config['base_url'].get() if not url.endswith('/'): url = url + '/' return url def create_default_browser(self): return self.create_browser(self.base_url) def can_post(self, contents, title=None, public=None, max_age=None): if public: return 0 elif max_age and max_age < 86400: return 0 # it cannot be shorter than one day elif re.search(r'[^a-zA-Z0-9=+/\s]', contents): return 0 # not base64, thus not binary else: mime = image_mime(contents, ('gif', 'jpeg', 'png')) return 20 * int(mime is not None) def get_paste(self, url): if not url.startswith('http'): url = urljoin(self.base_url, url) paste = self.new_paste(url) self.browser.fetch(paste) return paste def new_paste(self, _id): paste = LutimPaste(_id) return paste def post_paste(self, paste, max_age): return self.browser.post(paste, max_age) class LutimPaste(BasePaste): @classmethod def id2url(cls, id): return id ���������������������������weboob-1.2/modules/lutim/pages.py�������������������������������������������������������������������0000664�0000000�0000000�00000002421�13034501105�0017073�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import JsonPage, RawPage from weboob.capabilities.base import UserError class ImagePage(RawPage): @property def contents(self): return self.doc @property def filename(self): header = self.response.headers['content-disposition'] m = re.match('inline;filename="(.*)"', header) return unicode(m.group(1)) class UploadPage(JsonPage): def fetch_info(self): if not self.doc['success']: raise UserError(self.doc['msg']['msg']) return self.doc['msg'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lutim/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000003301�13034501105�0016751�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class LutimTest(BackendTest): MODULE = 'lutim' # small gif file DATA = u'R0lGODlhAQABAPAAAP///wAAACH5BAAAAAAALAAAAAABAAEAAAICRAEAOw==\n' TITLE = u'foo.gif' def test_lutim(self): post = self.backend.new_paste(None) post.contents = self.DATA post.title = self.TITLE assert self.backend.can_post(post.contents, post.title) self.backend.post_paste(post, max_age=86400) self.assertTrue(post.id) got = self.backend.get_paste(post.id) self.assertTrue(got) self.assertEqual(got.title, self.TITLE) self.assertEqual(got.contents, self.DATA) # test with an empty name post.title = u'' self.backend.post_paste(post, max_age=86400) def test_invalid(self): post = self.backend.new_paste(None) post.contents = u'FAIL' post.title = self.TITLE assert not self.backend.can_post(post.contents, post.title) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017004�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0021122�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LyricsdotcomModule __all__ = ['LyricsdotcomModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/browser.py����������������������������������������������������������0000664�0000000�0000000�00000003311�13034501105�0021037�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, LyricsPage __all__ = ['LyricsdotcomBrowser'] class LyricsdotcomBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.lyrics.com/' search = URL('search\.php\?keyword=(?P<pattern>[^&]*)&what=all&search_btn=Search', SearchPage) songLyrics = URL('(?P<id>[^/]*-lyrics-[^/]*)\.html$', LyricsPage) def iter_lyrics(self, criteria, pattern): self.search.go(pattern=pattern) assert self.search.is_here() return self.page.iter_lyrics() def get_lyrics(self, id): real_id = id.split('|')[0] try: self.songLyrics.go(id=real_id) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000004000�13034501105�0021131�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME ��IDATx{P~²r) eutщ1NcZo chiN&u46:hjLxiJT QEYn+ce-rqYMg_;^}s߷ckNb@ �D�"�@ x8M>'(X."җq-@kPGXǥ}90GK%{+-eHc�@"C;LLY*"f\t6Ui~dt6a-ٝt\;3Hv95AQiDepY/JZ#P1EypRrθQ?TtS| .4q؏T&s !1!1Dd�V.-inSM� e ٻ~uH@g e8 ""}1ٯx@"gCEb3_ܼR@OLθhԱDp{LFەcT}kk2l"3_`1Q}t=r?ħoG'P}d=cm--T �~e,'(2k{=g7gxv=V.Je}�Y*pⰶz12ۻCmV!$~&zC.+ Ϧ0׋<cW7bQ� ]ܗ|'H?`3_Ҿ\q @{U}*uIwb8`�`֦2_f=LLy$᳀Gofbr2Vpnso60ן:T@k YMǴڈ~l c|hڔdt\w�[ݿ;)}x)g2{WqB̵!,ͤ,bf#PBM\_B`/~;%He RqsA?+8} ?JKK~j>ۀLϔETo+=@ptw� ~Jk_nv7c 觯(d77#Leb" #A"G"GM@g!}u;&"3GM@"C2t+KܬкyWHOEk얖Ub\SRrJ߷utЪq>j>]5'h.OP3eѻnܖS\WBÎmc EsvK̟ \r+8z9zݺAsy>%͢ӻiLD�"�@ xXM@cTjd*^&쑧B&jsSmzj3}=j XA5Ig^^R NfDD %KB<jۜ]!�spt[(}Pj%�s hl,w'LI>y'{Sqmc6_uM@沏H_3V?}%ʰn7CY [0}{mXZ. K@o^g4F*R"Sc,C6|B%XnK KZ8lSXMХ-u *g_T[;�(Jmk5Frb;h*݃c#0ʀJM_A4R+ce۠Ե#җV|Qjz:CP#n<HluOZ'Ou>[(K=3KQj,yE c񇾯Qj[*;wHD19z} GO7Jm㹝t}JHhg#UC" 8nN&biƹkkjN-ӂJmƅ I}_E*]@ٮ [Zm/Ѧ,F{s(X7(VWoSluV.] z@ �D�"�@ <z!,����IENDB`weboob-1.2/modules/lyricsdotcom/module.py�����������������������������������������������������������0000664�0000000�0000000�00000003207�13034501105�0020645�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import LyricsdotcomBrowser from urllib import quote_plus __all__ = ['LyricsdotcomModule'] class LyricsdotcomModule(Module, CapLyrics): NAME = 'lyricsdotcom' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Lyrics.com lyrics website' LICENSE = 'AGPLv3+' BROWSER = LyricsdotcomBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, quote_plus(pattern.encode('utf-8'))) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/pages.py������������������������������������������������������������0000664�0000000�0000000�00000005306�13034501105�0020461�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class SearchPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[has-class("row")]' class item(ItemElement): klass = SongLyrics def obj_id(self): real_id = Regexp(CleanText('.//a[has-class("lyrics_preview")]/@href', default=NotAvailable), '/(.*)\.html')(self) # to avoid several times the same ID (damn website) salt_id = Regexp(CleanText('.//a[has-class("lyrics_preview")]/@t_id', default=NotAvailable), 'T (.*)')(self) return '%s|%s' % (real_id, salt_id) obj_title = CleanText('.//a[has-class("lyrics_preview")]', default=NotAvailable) obj_artist = CleanText('.//a[has-class("artist_link")]', default=NotAvailable) obj_content = NotLoaded class LyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): subid = self.page.url.replace('.html','').split('/')[-1].replace('/','') # sorry for the potential id comparison mistakes in application level. you know what i mean ? id = '%s|000' % (subid) return id obj_content = CleanText(CleanHTML('//div[@id="lyrics"]', default=NotAvailable), newlines=False) def obj_title(self): artist = CleanText('//h1[@id="profile_name"]//a', default=NotAvailable)(self) fullhead = CleanText('//h1[@id="profile_name"]', default=NotAvailable)(self) return fullhead.replace('by %s' % artist, '') obj_artist = CleanText('//h1[@id="profile_name"]//a', default=NotAvailable) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsdotcom/test.py�������������������������������������������������������������0000664�0000000�0000000�00000003243�13034501105�0020337�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class LyricsdotcomTest(BackendTest): MODULE = 'lyricsdotcom' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016443�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020555�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import LyricsmodeModule __all__ = ['LyricsmodeModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/browser.py������������������������������������������������������������0000664�0000000�0000000�00000003304�13034501105�0020500�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, LyricsPage __all__ = ['LyricsmodeBrowser'] class LyricsmodeBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.lyricsmode.com/' search = URL('search\.php\?search=(?P<pattern>[^&/]*)$', SearchPage) songLyrics = URL('lyrics/(?P<letterid>[^/]*)/(?P<artistid>[^/]*)/(?P<songid>[^/]*)\.html$', LyricsPage) def iter_lyrics(self, criteria, pattern): return self.search.go(pattern=pattern).iter_lyrics() def get_lyrics(self, id): subid = id.split('|') try: self.songLyrics.go(letterid=subid[0], artistid=subid[1], songid=subid[2]) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000004012�13034501105�0020573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME3��IDATx}pT%dCR!!P/J,VJh)U[aU1dCam8 ڑHAbBE8 m"a[!(J @M?Icnv?}7os޹{ (T}?K갻ו`~iop('6AvP*�3=L"P�mjv씏3T~LG(Fil02Ea{"6gl3$*"6_ef"9#C )B7crS>^S9&BF D+BNPPaBLu3-`@*,.F*ëBڄO+�L?u?eTit;) Uv&Pe\1p<rLUW;V  v*nm4Jyܓr ThDnq>l&"e 5' Ya*<R`@*A~e e,j@0>�+>.0m SJ?a*!>̈\$_g1gڅvPL..8vdx!;i%C۹VrT\/XdƚzYDl {DxЀOOQagVqvȦ*!GِE(p`ra RkT S(4 v@F(^෱2fd'�rWf5[pwr̻�D ;Uy-W;)Kɭ2TJѵaфP6xx'ij0" +}'R&lÜ ϰ1#BE�' f3ySy:)гꀩ&=^Y+hNWEpO="x~0%KY| rT 葜K2 C(T9ТJB!IU~V>YB*q&Uv0S.(^ʨʺjQ7[%/vovaJיimUc^.+*: 2@e=OMaЋ�w;Z7- ٯY% [@LC#ED*FX'g"Ɩk3W}ۯ;F?[aO50Q-×Ĵ%ϱkA{85-6$w@Ƙ7;,]OʲO91P0 'A| ܄j}4r!{`ؓ{ ,"<C!}b @CUd0Q^8g^ zXVғƷQ>lP]Q#z_GHeouӢ]qC$e끰]Fu]f}JXbmGxj3gZ0XR٫nHڔjx=D0oxITwHK@0whu%�-:4w5)\Ο;s/C& Nꎀ-7Zw&q^d‰C[(<p9N9J�9/%g^d6 {}:'л v5HHMZ_S]Sd{Y5~");SbM٫�.mKD6F ^?*vu¸p}J\ A{v(B&~w/4iȠ&PdBs?"3 >CO{/m sV7:p@?Ț_oNUѹ|NlOȚ_g%ڦШ>UgU{8k? ?V����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/module.py�������������������������������������������������������������0000664�0000000�0000000�00000003175�13034501105�0020310�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import LyricsmodeBrowser from urllib import quote_plus __all__ = ['LyricsmodeModule'] class LyricsmodeModule(Module, CapLyrics): NAME = 'lyricsmode' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Lyrics.com lyrics website' LICENSE = 'AGPLv3+' BROWSER = LyricsmodeBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, quote_plus(pattern.encode('utf-8'))) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsmode/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000004716�13034501105�0020124�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class SearchPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//table[has-class("songs_list")]//tr[count(td) = 2]' class item(ItemElement): klass = SongLyrics obj_id = CleanText('./@href', default=NotAvailable) def obj_id(self): href = CleanText('./td[2]/a/@href', default=NotAvailable)(self) spl = href.replace('.html', '').split('/') lid = spl[2] aid = spl[3] sid = spl[4] return '%s|%s|%s' % (lid, aid, sid) obj_title = Regexp(CleanText('./td[2]', default=NotAvailable), '(.*) lyrics$') obj_artist = CleanText('./td[1]/a', default=NotAvailable) obj_content = NotLoaded class LyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): spl = self.page.url.replace('http://', '').replace('.html', '').split('/') lid = spl[2] aid = spl[3] sid = spl[4] return '%s|%s|%s' % (lid, aid, sid) obj_content = CleanText(CleanHTML('//p[@id="lyrics_text"]', default=NotAvailable), newlines=False) obj_artist = CleanText('//a[has-class("header-band-name")]', default=NotAvailable) obj_title = Regexp(CleanText('//h1[has-class("header-song-name")]', default=NotAvailable), '(.*) lyrics$') ��������������������������������������������������weboob-1.2/modules/lyricsmode/test.py���������������������������������������������������������������0000664�0000000�0000000�00000003237�13034501105�0020001�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class LyricsmodeTest(BackendTest): MODULE = 'lyricsmode' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017002�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000000111�13034501105�0021104�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import LyricsplanetModule __all__ = ['LyricsplanetModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/browser.py����������������������������������������������������������0000664�0000000�0000000�00000004360�13034501105�0021042�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, LyricsPage, HomePage, ArtistPage import itertools __all__ = ['LyricsplanetBrowser'] class LyricsplanetBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.lyricsplanet.com/' home = URL('$', HomePage) search = URL('search\.php$', SearchPage) artist = URL('search\.php\?field=artisttitle&value=(?P<artistid>[^/]*)$', ArtistPage) lyrics = URL('lyrics\.php\?id=(?P<songid>[^/]*)$', LyricsPage) def iter_lyrics(self, criteria, pattern): self.home.stay_or_go() assert self.home.is_here() self.page.search_lyrics(criteria, pattern) assert self.search.is_here() if criteria == 'song': return self.page.iter_song_lyrics() elif criteria == 'artist': artist_ids = self.page.get_artist_ids() it = [] # we just take the 3 first artists to avoid too many page loadings for aid in artist_ids[:3]: it = itertools.chain(it, self.artist.go(artistid=aid).iter_lyrics()) return it def get_lyrics(self, id): try: self.lyrics.go(songid=id) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000003506�13034501105�0021141�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME!X��IDATx{p}{%m fVƩH%bH[ SAh->u3SW%SEZPDA^)i wgdžJru2=39{sΞsVlq�x�<��M $&M�]  tuﱭ(b=]5%/`TK$ubӐRخޏ<pR뱍X;W!eWrT2A":އu`+v0w  �(ZyuKfh"C:,? K-*E8+zrb2Jz6Jz6ڝe lEܻ9) "M(C -; !5U`[SNac tiԂ'A�rPL0trI G| 9sRb#C!ta>1nXf>Ɖg#]!_[U[EsZl!QboCS;~0ռckU?x8hE61aiġw�O#$sIӛ/nn^z!VU?Y9B ygbR/;JEoP3FT F^6u~_�Np<G-X'Պun(*E7z\ )}$ï!�ˌ?#r -0+㛾ɟwl4`F򊑓RyR|QP2F#l&h{oTyFm{cC;Prg:Pu-AB߉2ṉ#[K¹it%rTD}1`nr'"Nu =wLԵ_cz چP \�.x{ĮB+&ęđcxBWAXLpѦǿts!}@pnWAX~ޭ.`اa^N&8|溥u{Cu}gkbr,ԉR!}뵨lx!y(+=k0wEvЎ(NH={.g6#_�Or4'Z1N.hfꟉS薁(}+"�YQcT\$>aPh8} ~Op-J'[_pޕv #by?x%](rNaO=SHLKr*Hceҭܴ� )=mJ*jFٯo:s۟|KC/.`]3! ?>TϜEmDq4ߟo߇2nz!)ZFjی'F;'ˆli[S3;zqbPV_vb2I} HTй;qy�:) vd^K(7tx('wjlc=:\6ٸ-a^ے&@H 썔9uo- Վ;h[L?x~3ط#u0!D`ڎh?rGo~?�+A/u;Ykۋ:+@u&r(^}AU/cy0떅ӫy��x�<��zLS}����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/module.py�����������������������������������������������������������0000664�0000000�0000000�00000003147�13034501105�0020646�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import LyricsplanetBrowser __all__ = ['LyricsplanetModule'] class LyricsplanetModule(Module, CapLyrics): NAME = 'lyricsplanet' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Lyricsplanet.com song lyrics website' LICENSE = 'AGPLv3+' BROWSER = LyricsplanetBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, pattern.encode('utf-8')) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/pages.py������������������������������������������������������������0000664�0000000�0000000�00000006060�13034501105�0020455�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class HomePage(HTMLPage): def search_lyrics(self, criteria, pattern): form = self.get_form(xpath='//form[@class="form-inline"]') form['value'] = pattern form['field'] = criteria.replace('song','title') form.submit() class SearchPage(HTMLPage): @method class iter_song_lyrics(ListElement): item_xpath = '//div[@id="search"]//div[has-class("row")]//td/a' class item(ItemElement): klass = SongLyrics obj_id = Regexp(CleanText('./@href', default=NotAvailable), 'id=(.*)$') obj_title = Regexp(CleanText('.', default=NotAvailable), '(.*) - .*') obj_artist = Regexp(CleanText('.', default=NotAvailable), '.* - (.*)') obj_content = NotLoaded def get_artist_ids(self): artists_href = self.doc.xpath('//div[@id="search"]//div[has-class("row")]//td/a/@href') aids = [href.split('value=')[-1] for href in artists_href] return aids class ArtistPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[@id="search"]//div[has-class("row")]//td/a' class item(ItemElement): klass = SongLyrics obj_id = Regexp(CleanText('./@href', default=NotAvailable), 'id=(.*)$') obj_artist = Regexp(CleanText('.', default=NotAvailable), '(.*) - .*') obj_title = Regexp(CleanText('.', default=NotAvailable), '.* - (.*)') obj_content = NotLoaded class LyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): return self.page.url.split('id=')[-1] obj_content = CleanText(CleanHTML('//div[has-class("btn-toolbar")]/following-sibling::div[2]', default=NotAvailable), newlines=False) obj_artist = Regexp(CleanText('//title', default=NotAvailable), '(.*) - .* - .*') obj_title = Regexp(CleanText('//title', default=NotAvailable), '.* - (.*) - .*') ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/lyricsplanet/test.py�������������������������������������������������������������0000664�0000000�0000000�00000003242�13034501105�0020334�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class LyricsplanetTest(BackendTest): MODULE = 'lyricsplanet' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016430�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020542�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MailinatorModule __all__ = ['MailinatorModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/browser.py������������������������������������������������������������0000664�0000000�0000000�00000004131�13034501105�0020464�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.browsers import APIBrowser, ClientError from weboob.tools.date import datetime from weboob.tools.decorators import retry __all__ = ['MailinatorBrowser'] class MailinatorBrowser(APIBrowser): BASEURL = 'https://www.mailinator.com' ENCODING = 'utf-8' @retry(ClientError) def get_mails(self, boxid, after=None): mails = self.request('/api/webinbox2?x=0&public_to=%s' % boxid) for mail in mails['public_msgs']: d = { 'id': mail['id'], 'from': mail['fromfull'], 'to': mail['to'], 'from_name': mail['from'], 'datetime': frommillis(mail['time']), 'subject': mail['subject'], 'box': boxid } yield d @retry(ClientError) def get_mail_content(self, mailid): data = self.request('/fetchmail?msgid=%s&zone=public' % mailid)['data'] if 'parts' not in data: return 'text', '' for part in data['parts']: content_type = part['headers'].get('content-type', '') if content_type.startswith('text/plain'): return 'text', part['body'] elif content_type.startswith('text/html'): return 'html', part['body'] return 'text', '' def frommillis(millis): return datetime.fromtimestamp(millis / 1000) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000001027�13034501105�0020563�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME )1ڒ}���tEXtComment�Created with GIMPW��IDATxY[0 kPpb}mb-M&;K8����� 'iX9&55ek>=*D7w$xg|᭢M&=sMj?Ye_LXvb"̙�)osÝvH%& {@G%u٬R]8ƖإjnV!.-Ee|l:5ymP]lD^*xKdK Pm_yV#�!һxJ6,@ ra]zJ* hݱkSpgN�cMK`1x"@B O# (#/d_. L{�k�r}����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/module.py�������������������������������������������������������������0000664�0000000�0000000�00000006130�13034501105�0020267�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.messages import CapMessages, Thread, Message from weboob.tools.value import Value from .browser import MailinatorBrowser __all__ = ['MailinatorModule'] # There is only one thread per inbox, and the thread id is the inbox name # TODO but this can lead to bans if there are too many messages... class MailinatorModule(Module, CapMessages): NAME = 'mailinator' DESCRIPTION = u'mailinator temp mailbox' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = MailinatorBrowser CONFIG = BackendConfig(Value('inbox', label='Inbox', default='')) def iter_threads(self): inbox = self.config['inbox'].get() if not inbox: raise NotImplementedError() else: for d in self.browser.get_mails(inbox): thread = Thread(d['id']) thread.title = d['subject'] thread.flags = thread.IS_DISCUSSION msg = self.make_message(d, thread) if not msg.content: self._fetch_content(msg) thread.root = msg yield thread def _fetch_content(self, msg): msg_type, msg.content = self.browser.get_mail_content(msg.id) if msg_type == 'html': msg.flags = Message.IS_HTML def _get_messages_thread(self, inbox, thread): first = True for d in self.browser.get_mails(inbox): msg = self.make_message(d, thread) if first: first = False thread.root = msg else: msg.parent = thread.root msg.parent.children.append(msg) def get_thread(self, _id): thread = Thread(_id) thread.title = 'Mail for %s' % _id thread.flags = thread.IS_DISCUSSION self._get_messages_thread(_id, thread) return thread def make_message(self, d, thread): msg = Message(thread, d['id']) msg.children = [] msg.sender = d['from'] msg.flags = 0 msg.title = d['subject'] msg.date = d['datetime'] msg.receivers = [d['to']] return msg def fill_msg(self, msg, fields): if 'content' in fields: self._fetch_content(msg) return msg OBJECTS = {Message: fill_msg} ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mailinator/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002200�13034501105�0017753�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class MailinatorTest(BackendTest): MODULE = 'mailinator' def test_mailinator(self): t = self.backend.get_thread('qwerty') assert t assert t.root assert t.root.title assert t.root.date assert t.root.sender assert t.root.receivers self.backend.fillobj(t.root, ('content',)) assert t.root.content ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangafox/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016071�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangafox/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001435�13034501105�0020205�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MangafoxModule __all__ = ['MangafoxModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangafox/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000004467�13034501105�0020237�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��� pHYs�� �� ����tIME 9���iTXtComment�����Created with GIMPd.e���bKGD�����IDATx{LWOBT*e%.LFct,dd.&,.sF365yC|SdS'S|3iX)Z �q"s~ֶp/F{9wx}}%@MF"P3Q>P}TQa(ūd|{S;*yo?Lb?+Q!䁞0nah؍=yP4Gw2.}&]i ^3P.?0.)tVjuUx2{1~|v i�Y$9k^& rxwJTeCY $N_Q|WNFqdhG #8򂽥!C;P\DwyȃF+ =%;iK]1JH׆aݧ[�:!fY !TT/NX !!m/ gsgRhhu�f,` JpaQf�X93 *p�33� H '%P{l2vN7n1a�k2 -]]] c#`)] sC�@Q)ѐhةy|cwma_YK=%K!TZh߉;UY<!!S��6ub$A'� /f ŪFʰQ�lW@ bCq }W(pV@qc3ј9 yOkϚq'A�xpmMW6�~�<?óKz$O_AJMv_$G j8pP��׃>H>_0 =!(, ~�!(Z;�YOO!@9)ʚY"_'KcKBbaO`�x<u9 @ D�@ȵϡkeML4fS-\٧rHt;.Jw Ü�Dg!J~gȻ5Dz;޿΁S׺t `x3;?A]o7fi q(2~oP ΞU3OH.ZSm�UEqy.D�tq9bӷ8j7p(26$!=HgR4àĔh3"$ 磬@> [IcUKloD�} N`]�ɂ�{�7Ae\W,CIiۏN�_6ĦRA Xn�OHьRzPS$OsipN4HcjMk,�2ˇR]rZU,G�z`s�j}z _vKv7 pB,O@}B؍�ʬ� ?lOY�* h+ bC,(mrX0<G+2X0-4P� �PڜQ2wG g>| {6 r>#Qh ClcP5HYひY`l+#ԠN�!PWíG-m/TXP g Wbb7uMOx.υ0s@ f/eOR/_+@NsS]6]7IN]&-c7 q6|Ouӑ.4X8wse Dq&F>?sU廙Rc9rHJz1~׸X^U`e0P5̃qޒ: r'I7D7J #K iC9iB| j7x~}ۧ@ Pq:|ikd9aκDU#8X%W񼇶e_쬐̇Uh`,[c48 /4>pj QPӐ LlˬC|_ h/t ⍟7jmj1UvyNA`kbdn*o2^mR*s~&;+Uj{OA#9}7)^7b2rYN$HD!Xj% qf Ku_Ϳ-fEv !j4:w2/����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangafox/module.py���������������������������������������������������������������0000664�0000000�0000000�00000002707�13034501105�0017736�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['MangafoxModule'] class MangafoxModule(GenericComicReaderModule): NAME = 'mangafox' DESCRIPTION = 'Manga Fox manga reading website' BROWSER_PARAMS = dict( img_src_xpath="//img[@id='image']/attribute::src", page_list_xpath="(//select[@onchange='change_page(this)'])[1]/option[text()!='Comments']/@value", page_to_location="%s.html") ID_REGEXP = r'[^/]+/[^/]+(?:/[^/]+)?' URL_REGEXP = r'.+mangafox.(?:com|me)/manga/(%s).*' % ID_REGEXP ID_TO_URL = 'http://www.mangafox.me/manga/%s' PAGES = {r'http://(?:.+\.)?mangafox.\w+/manga/[^/]+/[^/]+/([^/]+/)?(.+\.html)?': DisplayPage} ���������������������������������������������������������weboob-1.2/modules/mangafox/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000001733�13034501105�0017426�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class MangafoxTest(GenericComicReaderTest): MODULE = 'mangafox' def test_download(self): return self._test_download('glass_no_kamen/v02/c000') �������������������������������������weboob-1.2/modules/mangago/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015702�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangago/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0020015�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MangagoModule __all__ = ['MangagoModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangago/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000016621�13034501105�0020043�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME +#|���iTXtComment�����Created with GIMPd.e��IDATx{Ɏ$ggkx,JP}U7=-t�hlS[.o{deUl u�pw?̒) ao_J,YJ*~x�08B'�δ̵RV iHJ�X":LY nR_REYde乽0Z+D+QgJZPN`ċjrU]8ƣq@buc@~_ectV-s%27ͪ&5rZ, `>8dN?ňc�Z ,#%qi|ax)yŪl,("uМPڅ�wӭ#$/9�IL�^)Lu,E^ֹ6VR?i=A{%300pd@ "0/�5'.J Z-,EUfz,-,8Ε$h -61�9 O?̷@bJ5"ArLSH)CL3%DQe(Ꜽܵ#LiH$hLqj{`�y%ggt|)DbH)IB^Y蠒-LF,0 vp�Tl ~2`2! EcLbJ!p SJJK( 4FBOcC �GL^gU�?w�h�FzCRTpC)D|#E9|}8:!}$���<~o~-Oo?8[@`q !nu9SatG78hH t4a?n/vmcvz`Yk!{#7:aD|w3#Bⴻ;ip;~ޅ] /ez/1Q7?'ѝG.gU_RDb!i)Bj0EzN^{9>xpw>Ľt7>~~_WϞޅc! @<Xz%d(u!X}jܥ +Ji})D&66ϥb"vcJqDi]{mٍ2DhC7~__sj]voEns]"3rfiJ8OWiVmžxBCRUfVʔ$.D:=@]Cnw>?[�m=c=WϞ_˜j @ezii,UgeUE]/,rT.Pper22>B!R ԵC antot;L`p> yS__]\l֗r4Ͳ^4^TU]yV(I!�,RM!!&B'? }+]{cwuyS]ۏ0 aε ps%8% V,R[4Zo,rSWe]eQlʲ\Vu]Wu]eYXksu2J -8 DǘRQzV ǃGH*G"(zʤS ͳea(FCHw�? 9R])\-g/^^n.MsU:˲Ͳlc kmnZ)4JH%II$H90sbN)% !mihOHx]ەm߭y}<pPQ]8$nVB GAV@SMٛ(/6W/_\\__Ջ**ˋdYJ)#TBJ%H "A ADD9fbL1#w\?=qw:txpwwX(NG}STBzQZ_4r|^?X,WrYQHmR D"M|l =98cD E:nK7R(+X1\,t;?g3B=k_}ŲyWM|qyybsyl\^UU*˪*RyB(DD )"}񧊘.d#ƑCV1FS1&S*ʎl沷V C`)@Z]]_\x/.\֯flVMS5"+Kb^J "!">>o/TI !0 db&f{;W2kUmUTx_}xHL) 0rU/uz_^ܼ~嗯.oՋf]eEfL(m@B bBgɞg2 ((f08%0$%Ic8++"!-J\imo҈VLJ+Howq?sWi1_\nfQ֫_-ͳ6e]-H-|�!<CJ%56Q D{E)Ex>G9$򲢬dZi@Dum15/H.i;gO' Oճ󫋫/WW/u4e]/�]޳G7"!%۲BRb,rrc"`Nge~q�= @b ,:ˠ\Dhb�`(}pwN>M8h6+Νڵ*Ϟ_<{ussr^?_m67jZ �{ǁ9%HmP4 ydUe@j=)Ŕ8?wGR4fP4 YU2+JR YY-U^^9J[E(i곬_岮yj|77׋fn몬k#B~۷ؽpص#�YjF/SRT SqmIh۷oy�gX\^3ЙWd\WUA^ڶ_oөYho �GXF"zzQׯ͋j4yQm-Hni=-N-m<y0B<Z\*e-YݔSBؿo0�YQb{HJ%ۢ*iDhay}h V]֏c޶C b.kE]֛/f\UUVuc?xpu ɜLQ`l[z&#"J{]mџN#a@ a 鈱"R !YxޙSxk;+ ͋/.rzyfsY/:jc�hl[v}O{`$Q~{DB�x#,{}? {<M }R+p `f7yJ!�$$®..pERR/ʥ�,(zrsylYEgE!1p:Q0Y!$LCKZ1|Q81s4N{q�- gl0BM9�?:N1 J[B./],M^lqVTeM\]^]]o67Y.2R0:cz؀"HhkҧԄ� 8C]>h?!%R�Rk, N)8 TyQz#9hZmw5 @K U]jQUUmr\jQVYM\\BL  tBJ,) آs,BG:g|x �w1B[*Ռ3$$2Y^ I)\y~{(dS,/Yԫu^Vkhk! %Y[ [ݶ,  ۢ@^S^P4 BO"J)!es48YJ H5)9*E@VPFCL 71 {uۗyU^v̇7frQMQrTz劲 sdu"\ARB̮󌲪B4( زHO!10E<D`fq$X*h llԛ y]C[sZRJpދ,<ϛ,Ͳݘ%Y._kEUU^rbd׌4L>œNPYe34آDXhkT5'n@B@i p&!g4Q 6PY[T b gJMe ms00 ey]EhֲڬWWJQWu"2y bL8%BdDP A)cur9Y%tCJ9 >uvsE8Ǹ2:Fآ@VUcBiIZ [ L*j[T& &˅V[kcUYRJ,)6ZR!"ctS@ >e^.4 w6`b,~L`I Pdz<pX( e3袄]�Sg%&c9o)%@JAjMm͊LU]=TyQ#1 .�!�!�1h&4 Y@(TLMeL=i|I~BJ(9+K� m- aA�:W2AI@IR� @ȩj)Hi%LfMQUC5˦RRY"ˬZ) !9G` DFIdy0f#t()@rp<cDZQ 7!BpQ Mk 0 P "&! -RXyQȬ%rb2BJM `BA)6`4A[I?yKpwa=ÅN 񘰉�Ab*R 2 ˬAARJ`r~Px`"`&9HnSwf^`te穌�@p0M IxAӻyND !H)%6d$#+"!dsJ8ߌ=氉&H!!*[љ &z�$c< '`(03RB)EhA@=\=;ȜXI"!q@cSf~}b`p#9)8̄pLgiA 0 C2F& �dJZMG @ 9;#V ?:?co*}` !!o}Ov&P$GJ)J�$ ̀!L5F J),SKĬfbZ ր1W!DN1@QHU߫ f4gD`}'jVk�RRq?*$<b!N+1A 6 f:%b'Ɣb?lBIb!mw#b k?)A5Ԭ~~13P(=<{#)Y9,CTOFO;}atØb PN*Edk۲Ji-6 i Exe�)RsD +3<))H1#` "%A 1(ŀ<{sna.X?|mý˲Ȗ5Dm yVr&'=E )]vIOۤhZN #NӘ=8Gލp<v ޳1"JX\ܧh$P<()ZJGG2?%< O#!{3q*5NNH! ~xz׶ioo}Rw~edeKhVr1d�) Oؖf |8wSb0O? y <qar ]m{:l֍cT۵hehvgۇ5$҅FT"x/DBL^B &~F��8CO>_ch 'T"@)N{]xxNtzn: <mݽfNC~ȫ:B,fR&>UQ6UrJ`NH1R a:|mqûݩ=vn(̌Ƙ]?~ơcߞ;u<hw2z'0mψ f[! aa6?‡8chOh{qM|7Ûo~wx7w߼Wnߞzw/煁G{t:kٵV{'DbF̶w"PQ0[='ԏ#ƾpj8?ܹ7o7O_o~xxݻqF6Ч YqTd9ikTT4HyGf,Jâx}i>yfZgKnu<=a޾kwﷇvZ OnިHo',|iܑ͠z$eߗ6ϭ2kVJ)%RJ)BJ)d)\C3ySHiZ#N)C1´Znqǡaﶻ~㷇=vy.$؍cwhZ*+va֥14Ƙc1FkZkJ))!G5|)&1C !B !BgsΟwѹ><Ƈnt n~g_J'=>|>4ZʵV(ɭY]]YYVy^E^Y,ck-d!8IBHDg NH)qc)x{;sэ8ٻ~a;ߏw~nd)�`d|L|x>"a�'�hQjjR4UUY*UUU,Ky<eYf4K)Ix#a*iܘscrq 0a}cvéOǶn; ~ (ʵ·q ? hUs%*y|c@ ) $k'9g n7¦NM!Ðnw6�~)Ph����IENDB`���������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangago/module.py����������������������������������������������������������������0000664�0000000�0000000�00000002542�13034501105�0017544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['MangagoModule'] class MangagoModule(GenericComicReaderModule): NAME = 'mangago' DESCRIPTION = 'Mangago manga reading site' DOMAIN = 'www.mangago.com' BROWSER_PARAMS = dict( img_src_xpath="//div[@id='pic_container']//img/@src", page_list_xpath="(//div[contains(@class,'page_select')]//select)[1]//@value") ID_REGEXP = r'(?:[^/]+/)+.+' URL_REGEXP = r'.+mangago.com/r/l_manga/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.mangago.com/r/l_manga/%s' PAGES = { URL_REGEXP: DisplayPage } ��������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangago/test.py������������������������������������������������������������������0000664�0000000�0000000�00000001751�13034501105�0017237�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class MangagoTest(GenericComicReaderTest): MODULE = 'mangago' def test_download(self): return self._test_download('manga/love_scar/mh/manga/love_scar/c001/') �����������������������weboob-1.2/modules/mangahere/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016220�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangahere/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0020336�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MangahereModule __all__ = ['MangahereModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangahere/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000020153�13034501105�0020354�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME ":L���tEXtComment�Created with GIMPW���bKGD������C��IDATx{y]u׷oop C,J-v+5v\5@ZYPhTKmbKMBn&V[cmI,$g}3wʮRw߽g97H>xQD2 M$RCQTb. ul{sc`׽?7sK1h*ǽTgDYj\Iޛ҂(B=?1#;/m^DpgD#%EeET=)<υ;:ZG?~LZJG}l4]͗\SPV"٨[[kr8mkcewkݲsUdN1͔("`yyE1G \[9Ͻ8x pCɁR*K9ahxR9?1Ċ  bm^Z?w.m ܿR0 E5EIVcIR8؀y:8]6 ڭZ\_h8Yy|X7ڟᑽО@qcz1/R2,[8ȇ!07wFm7ʥw[R[sbicz*UJiu#f6J(DItx"q]-ݬ9iΟz;7<GkN{45081P* iA\TM@!DHx#a&M3<%pYqͅS<<td:S*$8,cGxAQqJ$`8ȇ!ב!3w?t 1:q`TMDR{ XGJ~0<=1EF҇f8NN0 ܧNM%Sb.' s^ %+J!Că<^cd,lHRȲ[ >vk;`y318x|(3̞tUUՠ2ZOy(zF #uMβ:"gu Qx_yD?Jgb~:) 226= wdgk YT9^WDՒޟ_b'�):@ >1;-I|7HAF]qMZHFQ(� lء('Pޔ\ H^RRfF+KncQrd._,d!4#pLQ_G4ݐx4y?|ip{g=-Nx<ph!5}~Е+;Fwcػu-DŽC\, Xf$l~HAp+J°^<*Xȳb"9 b<Ȉt!t Lh[M52z>7m 08sҀ {d*!2(6׮Ϟv^dN)w+ rӒO~=UY}=0!)d*Je#COH3fIq\7.�Z(y˞*tj$%~2kfź,(<. bX4uFr1gdB0 O1帎փ 4.]:=>w!TR�'ĔmIb'x$J>>P)(D؃axZHÃַǟ$`Uýj. A묒Tju=^UV_qlqp`Y oLN�)[ _s<B?cL(oApͽi Zp \(pgaNjUV`=XR9A!ݵ|] 3M#eVN|/r#W;A00 Hf<f0edijD#ERQBϳ&5}WʎOTUˋ\'Ť=ѱ}1[պXGl64G,̟71qhEu'"\^ (djR6qJ?Wd[<5UoEa~ r\Rݕ$DTQPE!*H+wj2y,;j TYGQBX"\puSS ynLĈp92̈́73>PBĘ/<~db^ ቊ)B7(O=j&]ruCK:Ja+MڪdkrBaJ(!D|A ~*@)5:2˲R##�x8X{rs[{?Я|}MU $'/o9U1JuL#6k;C!J06rhT3x$zZ^4Lzqd_MsĒ!uYTdI\?/ȩL4(#كt?k"O:μp>ˢg= $B>Q hMt7InŁhÃnE:Ɇ1�|~h1DDK(NG88<E� %`S}5c CuZ2=Ϝ?{ =ETH\/_M1D˜/ww? q]HOPL#u�CҶx ̉7De8QnlSyBYΝ{c)N<_<EBv{_L?0jxЌ$ ZѦr=cסq\szJ "FTX[CUP[zh |AD jE\x=WUd r2~.˶MJ\ndg*? F<HЎ*xo]_x( /֋} {ƪ,Q8N'!@_D4lOdLa+KO3!+*wv0#c7?Nߝft e^h[ -{`m*ҋ 3cI"}ʐb)ݛAA^CQN$TMgrԯ1veíl&2Ł)nbQnzqkOywSg^]zD@MB) r6a(2�MO|G~ @HKK yv&f<T8 F~yʲNn/\[_(!% a!�r :Ϙ aP5cL2$HGbY9qey8i&|Qd"'\N(o펂c;r\ P =%׵^KzѴ(%h()ǁvRf0љ%dy_b#,7lto.,/y'Awzvƹ]4ylyt@?HJN[ _RDoqP]C20UI�9ZS,b\I e`p |aVٙ|[)JBqv#BA dnDVKs )�hdVSdE0tb4aje= 2)b;tP>V45Un E'awD [bFM,jBLBwPwD,�u� '/݁F,Bgj�CzV/ D94/g!7S="R8}7n9=FQ'聚"Y,%9y\|[#f"'Stsht) D^QPM13'8N8H#l|[yANP{_[65Fx\ծ䝑W͑|vNlBx]KG>Fϟq3!@Hm\ņ0P<}z̘MLeeſq]3|PboT*"swh ,Nz )Oٮ&OgXVk*-0و R!TVrOD3P{"hh @&-"`% Y&lp 3pc]DGL_i6+S|7h_ed?yW6!VgIp؎%YvK̤KB~͖b�jhd*Wx aN IJJBGtKu6d\{7$g>V\Hc6q^=7?3sWߚX^tLPY fSJv5`ߒN~8ҠY@ Vu5y6Arcemhs"ISL?@iYQh;1B%e1Bs{z} Q[": 4*&h(b/DH+w: SQZ޷'5;2L&ΦT ɼ-q7(Њ iS,XJݢfHQyh\([xj4D66vd[|<LD%}:wxıV �r!_V5EW]i4 �@UFd"(,B!|/ B~4?.JHsvw6,g xCW Qr.HJ:Nk(9]fFvoL| rR DŽ  '@ЁU"ErzaEPȚOQ�"I䃀ƴl%@f (p~qٍF\:F+ QӶݖC�Ml֙T$i!K8:k>{| ABadͭ$]A]:VRyKuZeňlm> xphOM$�Jޟǁp 6.)PHT2[~'- /_}Zf|]"~*@ {%h']'@lj \]0�&kz(Yl GP2x:f�qQ:Vr/<tZ҆\NC@occ5'= -snpYK,5y_wo-hĬs-dP}G7fspl=J8O%\DRtqfjJԞ=|{ aeꍍReV:Yp0zde/j.lthbekߎ: lV7k]^<m]5ZGy{xdzL0-HQ;vGZ.J�vj1la? y,@x~t aS0X,lV<RPoAǛhS+xa1H^.r\XM |!75>jq g6�݁t*:{>1g:u{ 9SWo@^/Y+/A@ٖW/gϿ6>_J̉x Pfr)jzQ6JYRljA|)th%%=zAF2dP,ON\�"i;pfԘ*`š*I( YYꃷJmpηD"}3gYMO1BxؐOBf�\ۍ"yv;n6Rp" n ÐfV Łf^M(O4]! ڪ*#Ýt&JNާf 'ڀèWF+FZgf&34pB`d3Ub!e iQ* 3[E܁~;͸~r+9:62UvB(Db$ zuB]],ͽs7OQu솁| lͩyjWSP8b(T�pyj} #Uw^epQ 㵣*jcdh4#Y20n-3i^GƉ.ƴJ^m6o2nʲѥ �0#TD]/rB>7P=~ mn-ЕeI3b!??0):g/~-qM9aflw|an'RieuAJ6LǬ<ęn{JN]^}=!! W8u[ӟw152*L]gxlQ+~=<mZ]1)"5`Lu {kbh-_86KvGZ!ϜVVl#�\ئfVjr`QO4.(u(w`Vyj?}ib\CZ~>|3 ZEQ@IV(Kzsc6S wtq�(QJS-MU&í~jzcii.G||b⠗LT4-ن?'#lyܙR4wkdnx'f8^8xHiLuf]fJ.S4])]BF6icsYtrؒ G_V"f?�܅fsԇG&+[ปLecx55&j.4IiB`Kͣ?~ӿ[hz(3(EԮ.\jf/EU1;{ʠF`.ixˉչ#PBMԀϸy,o !K~es*Z`M|U7Snf Aqɶ;X!*JW Gf]|i61]+ϊ(_e"lc7uuzhA'FR2qvڥ] +Jr u|7o8O`"vSEbҁ;޼kǻ=¨B˒F�DCkEZ,z9{őUpe]?rlb7W2[դh܆|n*vWh$t//0%Y YL0' QoN[ZE.DᏈ-shM#m~ 8u:Sm}'Rf6=sj"Kh٩[{<fjviyN PdA}z-;u4T/~2WC07:8;Mo>/Lo(#B S{;vQ[ŧns6ڿOά4qd.*{tl9!(W=rU^{HK[l) Rބ VT2?y4[h묯yҺF˿3w+on(jLʲ<>:ݍT=ka.^>7 i [٥1�i'.PFqyH36/�/TNGu;\_r#ײbk6ɬ˖�C v8=:%'q\2Y;|RȊYI$N@9煣OwK>vږ\[__/xK1vƑidc3jk:oV}iH|CO6 B4dd X!ѧ>Zpck ;Ԛeym]+l;=�EbGlZԵNHc.�_ɣE,x)|iYm4#5[k\ h(A~t@/bu 3 `FՇ8f>O= nm.~ B>=Ԅjy3#tWXSwg? wP`x\ :uo}y{S_ E�\x4GSlSt=GRS[ZX[^wk l  k>M9Wc{&!:[5VkT8ۏTw|6xZ _VzlQr C)xat=)xQ Ukj]hޣ:0ΏSDLFh*QȭhS2}RД)/sEYدg=ƣ]41o$<u����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangahere/module.py��������������������������������������������������������������0000664�0000000�0000000�00000002545�13034501105�0020065�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['MangahereModule'] class MangahereModule(GenericComicReaderModule): NAME = 'mangahere' DESCRIPTION = 'Manga Here manga reading website' DOMAIN = 'www.mangahere.com' BROWSER_PARAMS = dict( img_src_xpath="//img[@id='image']/@src", page_list_xpath="(//select[@onchange='change_page(this)'])[1]/option/@value") ID_REGEXP = r'[^/]+/[^/]+/[^/]+' URL_REGEXP = r'.+mangahere.com/manga/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.mangahere.com/manga/%s' PAGES = {URL_REGEXP: DisplayPage} �����������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangahere/test.py����������������������������������������������������������������0000664�0000000�0000000�00000001734�13034501105�0017556�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class MangahereTest(GenericComicReaderTest): MODULE = 'mangahere' def test_download(self): return self._test_download('glass_no_kamen/v02/c000') ������������������������������������weboob-1.2/modules/mangareader/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016537�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangareader/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001443�13034501105�0020652�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MangareaderModule __all__ = ['MangareaderModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangareader/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000011316�13034501105�0020674�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME *#e���iTXtComment�����Created with GIMPd.e��2IDATxiYgy>cOlYZ71Jڬ P ZT-"H ҂�D @*P/ $'NƱc3̽9jq^ifyY?gzE�.pbR* NHI_l< مrCu7:H4v|7� X�gi#+3n!S$Qkl^vY|G7~�F옾 G9>x_^ىOϷn`-k \e?rlpG=Q7i/bFGL6v3Y|<p?qzI;LD=#Mk Y=zg{!MKgnZSN6τ/t`G{2׮}7skHv­"f凴)ծU 2uZ{d=Sя(;[o-}~/Ӎ=4yqP$�Bqx]د R8ш)UEjcOdw茞cc[&fZ7Y!+degIœ4Y<OcMu3ê X P(9Z,ȟj^y3qlA|yUdnK%CI CPBFUughz1C='DEjH SxQV$[!%"PԒWeX2džu^oMBἣ*5eHsq".S>P*nt~ml7V&$J*$JAHP>3 U_hesŵ{9_١{�Ϗ ZYFdj4 lZ PJ3W(}D@kC[ gTSW|鋱7/79߿{K3!Bk#D֠AW 9ap.>5&kW2]$jBȫ.KGw溣[qLVPGhccRQ*BaP(0 H"t}[+S[zM  JyQͣhMot3 NWk5y�0WA@ Jbtr䮸s4z!JJIVfXѕUy5rLYgTUU0CP>PU~7PlR)d-U5PG!CoQcPX(tEAΏ(C C^ۿTmj" +ν {h͐&e ʃA9tB" D!$JHA)9SO}l 'KO9s0~(J!Sv~&ՠR$ &hն!jEUX_J jK>N,BDgTζӗrڋRZW(uܭ]_8+l3ş.ŕw4Y>"'Ǝ*ĜͿCd#hV}9"�"&IIZ.b=MU߳jKEUΕXc7NMw d)2 "97AĚi{O/{3 cP>QA$سZQx p3I!:t;hG_zy8^uoRsd(R ]흄{$(D"tn:\ogȇgTn@Vfޱ^~p|!6ҴFX!|[Y'~(A@Yu|Lzݻ/y ЮĚp6{>_zW[NcAr3؋Mʖ[7HO0bxEJ9I8߼;h/9W7(U]*7KA Fq%z׊ũW162p6,#/2  $fDjfJ/L㍴KG,x%UPQ!Zij$xrKɺ#y<\( bs^Q @sGS,գE|ܪSRjjve̵]c ˽'r [4>c,d/Yl |1یs~%=Dz7L0z&acuJlr%QS5z=C-enb)[EN4R.F Zr-S6lkB>5FqVϰ2x^,Z7⌿)\;<OdkI wRXkЄrK߇ 3# gP,6|㿾^B!_ 9r [=W<'nM<wIP^MƯ*�[+쳲$/M'M`Jԥ)-|eqsxEVVJ1(.9J+1+`' )7C*>Ľ�SQ K eydž_ܠ&ɉ"5P*U^)7.wO@3X\ +#y]MU#RmQ{ @-D+{>�|2=RV,A F ڎ92n#FW:v a5~icW�\Tue];^煕 J<tM Matm辏ݓۧOG5$l؅OUEUO5G91G7tKD<N*C$�bréD4Ӆ-]{Ckws=ZҬOctNsMNQ>ݴTa?fTqxJN*WcuisGgOY#>'ёs[gya/3?]'h#A8d ēp!,@TyW!>Ga\n۾V81x>J[tN3Tn7Wl'&^{@:z](jj U-RXkZlz~vQ2!:Yts~6OiUn7:t,de~z3{ȧX=BV^VcN_0UrJ)RBBP c~g74Qnu)K Kt۽?&uBef}pv7] >h۰:R @ΈIfj7L1:« ZGk?R1G~6K3^( is?>uʗX=ٙ9@)U:"c=OO aDjH7l"6m$ؾG}{8HT^;VKF Nz7ilBk1,[)_~,vV$gS+^|HH[B ͆N4QZ< >=˱lAw =|ֵZ{)qqD? hn+g=?L~+�mкne[&,~1eQ#*W|KrDYT"&f_7m["SuhW^Zo)bPu> T a#|(hĜZ8FP1ej}@= Ş7{wܷW>e5t#Op=#fPo7 $O vL~O{mW*YFTQh<e|f:~'4kjHFQfHgLI`scTBu*JeTt %֫_Xmh% h1k7ζ,B#{IcB͈*:yGӪtUbOҭ A2 -UmkL=6xD/|0uJ;VkMbT!3Z4eua'25x;xz<w֛^In^6P쓦z|yp,h6H l!'Jʢ$Kb5lm/lo]S2x# fl?.pO^x%=Ҥ8B,)2[NڵK6�%,�y97<DVvmVmF:5O ]lo߄s;p-BokdŒ"хE1Fox϶6f-ϯ~ #?g<(#U\$fduzQo>5_7ZKUX$P¸hu9&4r2*/6|~~9?rЪNH;dڥ4mKv$jMƺÐ~~ XT|#2ubEʲ+=MotZŅ!B@aNML`,^vhsڻ@gA:oUkHiۘj\Qq)'[hat 8Ss*l`T55b8ת4:E�.p�\"�Kqgj����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangareader/module.py������������������������������������������������������������0000664�0000000�0000000�00000002554�13034501105�0020404�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['MangareaderModule'] class MangareaderModule(GenericComicReaderModule): NAME = 'mangareader' DESCRIPTION = 'MangaReader manga reading website' DOMAIN = 'www.mangareader.net' BROWSER_PARAMS = dict( img_src_xpath="//img[@id='img']/@src", page_list_xpath="//select[@id='pageMenu']/option/@value") ID_REGEXP = r'[^/]+/[^/]+' URL_REGEXP = r'.+mangareader.net/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.mangareader.net/%s' PAGES = {r'http://.+\.mangareader.net/.+': DisplayPage} # oh well ����������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mangareader/test.py��������������������������������������������������������������0000664�0000000�0000000�00000001725�13034501105�0020075�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class MangareaderTest(GenericComicReaderTest): MODULE = 'mangareader' def test_download(self): return self._test_download('glass-mask/3') �������������������������������������������weboob-1.2/modules/manpower/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016121�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/manpower/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001436�13034501105�0020236�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import ManpowerModule __all__ = ['ManpowerModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/manpower/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000005067�13034501105�0020166�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import SearchPage, AdvertPage class ManpowerBrowser(PagesBrowser): BASEURL = 'https://www.manpower.fr' search_page = URL('/offre-emploi', '/offre-emploi/(?P<query>.*)', SearchPage) advert_page = URL('/candidats/detail-offre-d-emploi/(?P<_id>.*).html', AdvertPage) error_page = URL('/offre-emploi/offre-non-trouvee') def search_job(self, pattern=None): return self.call_search(query=pattern) def call_search(self, query=''): if not query: return self.search_page.go().iter_job_adverts() return self.search_page.go(query=query).iter_job_adverts() def advanced_search_job(self, job='', place='', contract='', activity_domain=''): query1 = [] query2 = [] if job != '': query1.append(job) if place != '': _query = place.rsplit('/', 1) if len(_query) >= 2: query2.append(_query[-1]) query1.append(_query[0]) if contract != '': _query = contract.rsplit('/', 1) if len(_query) >= 2: query2.append(_query[-1]) query1.append(_query[0]) if activity_domain != '': _query = activity_domain.rsplit('/', 1) if len(_query) >= 2: query2.append(_query[-1]) query1.append(_query[0]) squery1 = '/'.join(query1) squery2 = ''.join(query2) if squery2 != '': query = '%s/%s.html' % (squery1, squery2) else: query = squery1 return self.call_search(query=query) def get_job_advert(self, _id, advert): self.advert_page.go(_id=_id) if self.advert_page.is_here(): return self.page.get_job_advert(obj=advert) return advert �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/manpower/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000017751�13034501105�0020267�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��� pHYs�� �� ����tIME  �4?#���iTXtComment�����Created with GIMPd.e���bKGD�����MIDATx[ ~u}}2}!x3QU($(b4JšhVc4*6*QDaz>jap1&v{UIUU:_̙3iyĉ4vX$2#? àH$BcziӦMqꩧ/LgAk׮<i$x\\UUETQQA$Ivib흝E!v}Gׯe%Bި.ϟO`%Sl6xgxNg/L tttի顇pΛ7O8g�GX, ΁dpf3ܝN.BS FԐw\5`,Y; Qkk+Q8O?e2RZCbqԏZ7=GOtǫ $:F kw ak V*<{(**gV7<ӺݑDӔg}N 8.`Vz~}Ea$r$||╕gyƌ0M0&M$}kgg~o(AiIJ6ۺzqK[Ɔ^5vZ9`LUkƞ-~):?=-1#iҟ&"})R<oryc̳0 f�~C RY2nP(R"Gw5]ǯ83'zU}ɋa`]*ە:A.fb&9NuVK b2PYg͚E*GnXi(,Ŀ&e I >xDUYp+R=a]vDZ Ag pg�^iF~ٸ]\Xg�g+0RtprR*k2Y]w�N`GDnbJGcIG/;5ܻzL .h8/H^]8$0bMv* %ե~ϺN7fEm˃a䅭K{6%+X@TpQ;|S[Y!4HF8v߂93yZbiCdS)$C%]!4z*|O@ϭ8-NS` fT(sx׿tQolykhe[8yf{8I}t9 $R˽N j=W:oS#yMOYRKĊ}Dڟ fTY%4GtiqM+8.SA8 qTxVĞ5Fx3ILDEڙ|rC~tDF{adǣ{;i:>Ͻ[RӜ;Yy)~} gpR8%U$Jј\8\RsNiƒNpfJe)=@(X1ͯǯ"?J舤mє6@>iYuPq]>?4EsJKLx`5Rh)8$/RcS"%n)}2DTv/奤xsMZ8%LɌvrE3P0=})ꎦ)ք q)"ڨ2LeKsi흿K '=Cv']L 8An \&;NM]p֡݇g:Hʆ^SERl6ѷ Z1$„itґpJ+)KߜP"+a:ӣxJLNe%瘞Fċ4(ZFONeaؼ]cSR{Lqn[vtEwiWS:F~B&ه(BHF&')鳤ZO-dDZF 2AvrƨÍN 8N%͂_}{2bÇnP}lg89'YS0r$T J6pל;vgv'ೝKݷ`}Q,q@=L['%g(l!Ӥ5aHͤ5Kfnߵ0I.c"B~p]{8xNT2$rΙa9 93 `Ҹ+[JLYԟDRخlodmI& [%%v|'8'>Ot^+2{(Kqd18< xg"ނ=X~2]A&I#9v@Äۑ:S !�)o[\ÅI7Yx^Ǐ.Ư")ͅmGS~Aq;HD-e2$3K'FM9dF~Z2mm̔$)AvIdó0gS"zN2R!MwtrxNٶ-%(l H^$<{%np怬dDʽO35Eѣ k,A$j zh oTyr6Ur�"iotG ma,Qt^]Lc-d@x!EO\ �&%q+ih fCPvjb i6R'5 EQ"&/ ;1<$z\-jhnˈ=oLDi, ]= @ ҝ6p).@`7{(})]C98P>* iψIGBJ1*"֎k.ꌈ 5nHʕ|:YW,C{BI(@ci27Cqn>/G?h量XdDeG%=0VȠ$Oujt6J~aH1dpjZ8${M~Gmz G\E23N[_8v)xQL&wJ"U0tʱέ{hgG+uC(}5 z xR$";lpt Ax12<w 9ùsz>/)c1S{i'f{hmNC5E[eq+K{luB#>!}i>gp]Oob C}a>gӭBM8�ԒD&=LZd$$8/|-^'-8(sŒ[CZF)5thKk?}Qg{$1-4IPc_vR{Ǚ1nHF&Zqz7, ;�U_0R6 2EBe;? v@&j28yr �NÎSFj lmh<-Rere|tنr+d!]<B+M&{D&4q v'0Y٣TBݯLtqdZ5)CP|뿢H8۝#p�nKC^dftf-3Cto$EMwZx)]UC |k{vNWj`^}ȎXdwF8@VsD(wM]Nl%s C3 ~S6+?~Ga?M\<x`|8^@|tso<ϙefsD'n)a9N*@YDj eφ|~4PuF6/D{Pg>_8*jG pjAΛPLF%ԩ=&R%xRuAz<Av9 Ren{xEݚ"g)dd|GE@դȌw wṛ~X$ 28Gfp U*4>CqJGcp2لQhT|.uƲJS"7-bGq1Aw>|v-hpqvQ߿aAD#`dK+鉄5c$[E)@AkBV+�owPySl/W�J=Q�<3rRZ[ :USE \@l_hqrO0@"u rR^EaDrck gKη─S1 'p$>N*Kl+v; fI&3ɩ(Pn1>>N-"H4PS!XjQT[deQz`2J/R8Ke01Niq-A1b739K}IF!!;Hbu;UOUϫ8هs,H]wc'2Y! X`V] Өbݨ{PElnO/<8oMCgOmyziBYn/#!0s0Q)o%tf"FQefҍr9RpiSNE5ӸKIUd8se}xdLvx!}~IJ`h7(ZY,Sqʵ4ޫ̢9STF5jtk&ƻl?lXC>a$J;52 nH)͸% r Hk:^Tы*ws�P)" 1ND&` {|TL%j}gVxR�dL!N";\(@憍X"oOL2<4  {liM(q=&<8_\ޕEw|NޭG [ĀDZ{?.웤d^N&g: Ag}<R;ja @  ,pyOƶe'dtk"Ha+;'}< ɉn9Xd8�`nn L&;<|ՆIuzHra$y3=prp#T Oq\;8P!N_@ژb�"$bsҷ+UESq"lrZN`L⇏*C6 qD <-X*eaxߊب0޸*dyxCCT;&H;Ȭ� GD,EVVESIh;t4'_![|U<D{o{P)Sm8# Ò6fKENV?E{'墤uO&vF_N%uOE @k[qnQ + 2  R!N|s7>_*6<MB'!\yu/c\dծљc„tL<x.oo=%oUki# N䙅3Ӥ~`֗~ԂKmKB9CRI.S"u�n/]0+ėU!q%H=B_UTf8;&r30~VY4^Mٶ-7taed:۬fJj,AٛO?/qV1}]: 97`<N֌64/E.JAVŧUw{kY= Ys;S|ACTf8d9ѹu06ӿyE5HE%/CrEOя d�dw_:]$yI&DZ_rvZ0Dބ&_Ia\ Q9JM8xV�y~4n:9|y\Wu0 gk\6ٔd_@]b$#5H֏p8d #@~wXp*a*/CtV8u.z�c@DO>o"Ӭ B'~b}7B2}J I9QB9P+e7`-3OsK+7r!, <֭\nqݞ7WyV~ >h9x, pPtrUШD']8僬? _l0�Dga@P�zWQ 9v(3SEq_zwqZE iU^]秃4EGx8ƕ8&JPKK:jMh]5~+~/7Hߕ\Hu*DR|np&C/YwOz)+~3*￵ U+A�Y/s@/C=^(^zKeK]5]clǥs-G8xO^=pF,mOnpq%9'PB8̥:?'BcB }wK`7L`݃vtTEt$nj/Ȇ%nKO3^{ ԣ鐺1T'(wMRZObXZ1 EO[R gF8,N-kny8.wo( o|/=s4zwl%c?znXFƬ笎OfCXyp'!S+S}xCŷ]k9L/*&~XGSV+ŵ$9s 5K=I{} BeR<伷xAIUMd�G\|pD#/rL'A=jϩu#웇M!B)1F`Q`cԺ# fn6EW_<$2.W^JߧNR/}x!vpVN=�G'#z1mJ/~X!K'ܰZ^p"5lPa]xe~#_{ >0 'yPA=g|:&ss8dkٗBJ1㓐<I6tffȹ<'4 5bVfqe5 fâ 6sc "XU@eh}ªo+SJN"$F#=<%_@ ^FkTzEٳGl$໧i?߰ H?hT&sas&=;ڎg8<s.y>BXE׆JJwCuntssxn]vAvV(]vQmm{ݟS<<hFC4[Z c06ŵٴ$֯p`oي 32NOʫh\Y\;?m/x2P4ͿVy״{~4/<y9dwl3ۮ;4uTq82ՉgUS*=sſiAg);+.}3%'o꯫kԱi߶2wnqWɱzpmz< rJAN4u~W96l3.�;Nʻ, ViPc9{Д`^[ maQv%X϶l+< 4~x:xkUII]u':@=P׶yrع%/�Oh3;w_h^y0r;F(:iǣP:tG q{@;a1/sΞy_d09x_wqG~[?zxW~Ofq{ M|~a{uxd>`%y5k3g:Ӕ)SzOM6HCmF| 7OMau h����IENDB`�����������������������weboob-1.2/modules/manpower/module.py���������������������������������������������������������������0000664�0000000�0000000�00000027437�13034501105�0017775�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from weboob.capabilities.job import CapJob, BaseJobAdvert from .browser import ManpowerBrowser __all__ = ['ManpowerModule'] class ManpowerModule(Module, CapJob): NAME = 'manpower' DESCRIPTION = u'manpower website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ManpowerBrowser type_contract_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'All', 'cdi-interimaire/c11': u'Autre', 'formation-en-alternance/c4': u'Alternance', 'interim/c1': u'CDD', 'cdd/c2': u'CDI', 'cdi/c3': u'Mission en intérim', }.iteritems())]) activityDomain_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'All', 'accueil-secretariat/s69': u'Accueil - Secrétariat', 'achats-commerce-distribution/s66': u'Achats - Commerce - Distribution', 'agro-alimentaire/s65': u'Agro-Alimentaire', 'automobile/s2': u'Automobile', 'banque-assurances-immobilier/s3': u'Banque - Assurances - Immobilier', 'bijoux-horlogerie-lunetterie/s61': u'Bijoux - Horlogerie - Lunetterie', 'bureau-d-etudes-methodes-qualite/s4': u'Bureau d\études- Méthodes - Qualité', 'chimie-pharmacie-cosmetologie/s6': u'Chimie - Pharmacie - Cosmétologie', 'communication/s73': u'Communication', 'comptabilite-finance/s62': u'Comptabilité - Finance', 'construction-travaux-publics/s9': u'Construction - Travaux publics', 'electricite-electronique/s67': u'Electricité - Electronique', 'environnement-developpement-durable/s80': u'Environnement - Développement Durable', 'hotellerie-restauration-tourisme/s24': u'Hôtellerie- Restauration - Tourisme', 'it-commercial-conseil-amoa/s75': u'IT - Commercial - Conseil - AMOA', 'it-etude-et-developpement/s14': u'IT - Etude et Développement', 'it-exploitation-systeme-sgbd/s76': u'IT - Exploitation - Système - SGBD', 'it-reseau-telecom/s77': u'IT - Réseau - Telecom', 'it-support-maintenance-help-desk/s78': u'IT - Support - Maintenance - Help Desk', 'imprimerie/s12': u'Imprimerie', 'industrie-aeronautique/s79': u'Industrie aéronautique', 'logistique/s70': u'Logistique', 'maintenance-entretien/s53': u'Maintenance - Entretien', 'multimedia/s74': u'Multimédia', 'metallurgie-fonderie/s49': u'Métallurgie- Fonderie', 'naval/s47': u'Naval', 'nucleaire-autres-energies/s54': u'Nucléaire - Autres Énergies', 'papier-carton/s20': u'Papier - Carton', 'plasturgie/s22': u'Plasturgie', 'production-graphique/s72': u'Production Graphique', 'production-industrielle-mecanique/s16': u'Production industrielle - Mécanique', 'ressources-humaines-juridique/s63': u'Ressources humaines - Juridique', 'sante/s25': u'Santé', 'spectacle/s71': u'Spectacle', 'surveillance-securite/s68': u'Surveillance - Sécurité', 'textile-couture-cuir/s26': u'Textile - Couture - Cuir', 'transport/s64': u'Transport', 'transport-aerien/s52': u'Transport aérien', 'teleservices-marketing-vente/s21': u'Téléservices - Marketing - Vente', 'verre-porcelaine/s48': u'Verre - Porcelaine', 'vin-agriculture-paysagisme/s60': u'Vin - Agriculture - Paysagisme', }.iteritems())]) places_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'All', 'alsace/r01': u'Alsace', 'alsace/bas-rhin/r01d67': u'Bas-Rhin', 'alsace/haut-rhin/r01d68': u'Haut-Rhin', 'aquitaine/r02': u'Aquitaine', 'aquitaine/dordogne/r02d24': u'Dordogne', 'aquitaine/gironde/r02d33': u'Gironde', 'aquitaine/landes/r02d40': u'Landes', 'aquitaine/lot-et-garonne/r02d47': u'Lot-et-Garonne', 'aquitaine/pyrenees-atlantiques/r02d64': u'Pyrénées-Atlantiques', 'auvergne/r03': u'Auvergne', 'auvergne/allier/r03d3': u'Allier', 'auvergne/cantal/r03d15': u'Cantal', 'auvergne/haute-loire/r03d43': u'Haute-Loire', 'auvergne/puy-de-dome/r03d63': u'Puy-de-Dôme', 'basse-normandie/r04': u'Basse-Normandie', 'basse-normandie/calvados/r04d14': u'Calvados', 'basse-normandie/manche/r04d50': u'Manche', 'basse-normandie/orne/r04d61': u'Orne', 'bourgogne/r05': u'Bourgogne', 'bourgogne/cote-d-or/r05d21': u'Côte-d\'Or', 'bourgogne/nievre/r05d58': u'Nièvre', 'bourgogne/saone-et-loire/r05d71': u'Saône-et-Loire', 'bourgogne/yonne/r05d89': u'Yonne', 'bretagne/r06': u'Bretagne', 'bretagne/cotes-d-armor/r06d22': u'Côtes-d\'Armor', 'bretagne/finistere/r06d29': u'Finistère', 'bretagne/ille-et-vilaine/r06d35': u'Ille-et-Vilaine', 'bretagne/morbihan/r06d56': u'Morbihan', 'centre/r07': u'Centre', 'centre/cher/r07d18': u'Cher', 'centre/eure-et-loir/r07d28': u'Eure-et-Loir', 'centre/indre/r07d36': u'Indre', 'centre/indre-et-loire/r07d37': u'Indre-et-Loire', 'centre/loir-et-cher/r07d41': u'Loir-et-Cher', 'centre/loiret/r07d45': u'Loiret', 'champagne-ardennes/r08': u'Champagne-Ardennes', 'champagne-ardennes/ardennes/r08d8': u'Ardennes', 'champagne-ardennes/aube/r08d10': u'Aube', 'champagne-ardennes/haute-marne/r08d52': u'Haute-Marne', 'champagne-ardennes/marne/r08d51': u'Marne', 'dom-tom/r23': u'Dom Tom', 'dom-tom/nouvelle-caledonie/r23d98': u'Nouvelle Calédonie', 'franche-comte/r10': u'Franche-Comté', 'franche-comte/doubs/r10d25': u'Doubs', 'franche-comte/haute-saone/r10d70': u'Haute-Saône', 'franche-comte/jura/r10d39': u'Jura', 'franche-comte/territoire-de-belfort/r10d90': u'Territoire de Belfort', 'haute-normandie/r11': u'Haute-Normandie', 'haute-normandie/eure/r11d27': u'Eure', 'haute-normandie/seine-maritime/r11d76': u'Seine-Maritime', 'ile-de-france/r12': u'Île-de-France', 'ile-de-france/essonne/r12d91': u'Essonne', 'ile-de-france/hauts-de-seine/r12d92': u'Hauts-de-Seine', 'ile-de-france/paris/r12d75': u'Paris', 'ile-de-france/seine-st-denis/r12d93': u'Seine-St-Denis', 'ile-de-france/seine-et-marne/r12d77': u'Seine-et-Marne', 'ile-de-france/val-d-oise/r12d95': u'Val-d\'Oise', 'ile-de-france/val-de-marne/r12d94': u'Val-de-Marne', 'ile-de-france/yvelines/r12d78': u'Yvelines', 'languedoc-roussillon/r13': u'Languedoc-Roussillon', 'languedoc-roussillon/aude/r13d11': u'Aude', 'languedoc-roussillon/gard/r13d30': u'Gard', 'languedoc-roussillon/herault/r13d34': u'Hérault', 'languedoc-roussillon/lozere/r13d48': u'Lozère', 'languedoc-roussillon/pyrenees-orientales/r13d66': u'Pyrénées-Orientales', 'limousin/r14': u'Limousin', 'limousin/correze/r14d19': u'Corrèze', 'limousin/creuse/r14d23': u'Creuse', 'limousin/haute-vienne/r14d87': u'Haute-Vienne', 'lorraine/r15': u'Lorraine', 'lorraine/meurthe-et-moselle/r15d54': u'Meurthe-et-Moselle', 'lorraine/meuse/r15d55': u'Meuse', 'lorraine/moselle/r15d57': u'Moselle', 'lorraine/vosges/r15d88': u'Vosges', 'midi-pyrenees/r16': u'Midi-Pyrénées', 'midi-pyrenees/ariege/r16d9': u'Ariège', 'midi-pyrenees/aveyron/r16d12': u'Aveyron', 'midi-pyrenees/gers/r16d32': u'Gers', 'midi-pyrenees/haute-garonne/r16d31': u'Haute-Garonne', 'midi-pyrenees/hautes-pyrenees/r16d65': u'Hautes-Pyrénées', 'midi-pyrenees/lot/r16d46': u'Lot', 'midi-pyrenees/tarn/r16d81': u'Tarn', 'midi-pyrenees/tarn-et-garonne/r16d82': u'Tarn-et-Garonne', 'nord-pas-de-calais/r17': u'Nord-Pas-de-Calais', 'nord-pas-de-calais/nord/r17d59': u'Nord', 'nord-pas-de-calais/pas-de-calais/r17d62': u'Pas-de-Calais', 'pays-de-la-loire/r19': u'Pays de la Loire', 'pays-de-la-loire/loire-atlantique/r19d44': u'Loire-Atlantique', 'pays-de-la-loire/maine-et-loire/r19d49': u'Maine-et-Loire', 'pays-de-la-loire/mayenne/r19d53': u'Mayenne', 'pays-de-la-loire/sarthe/r19d72': u'Sarthe', 'pays-de-la-loire/vendee/r19d85': u'Vendée', 'picardie/r20': u'Picardie', 'picardie/aisne/r20d2': u'Aisne', 'picardie/oise/r20d60': u'Oise', 'picardie/somme/r20d80': u'Somme', 'poitou-charentes/r21': u'Poitou-Charentes', 'poitou-charentes/charente/r21d16': u'Charente', 'poitou-charentes/charente-maritime/r21d17': u'Charente-Maritime', 'poitou-charentes/deux-sevres/r21d79': u'Deux-Sèvres', 'poitou-charentes/vienne/r21d86': u'Vienne', 'provence-alpes-cote-d-azur/r18': u'Provence-Alpes-Côte d\'Azur', 'provence-alpes-cote-d-azur/alpes-maritimes/r18d6': u'Alpes-Maritimes', 'provence-alpes-cote-d-azur/alpes-de-haute-provence/r18d4': u'Alpes-de-Haute-Provence', 'provence-alpes-cote-d-azur/bouches-du-rhone/r18d13': u'Bouches-du-Rhône', 'provence-alpes-cote-d-azur/hautes-alpes/r18d5': u'Hautes-Alpes', 'provence-alpes-cote-d-azur/var/r18d83': u'Var', 'provence-alpes-cote-d-azur/vaucluse/r18d84': u'Vaucluse', 'rhone-alpes/r22': u'Rhône-Alpes', 'rhone-alpes/ain/r22d1': u'Ain', 'rhone-alpes/ardeche/r22d7': u'Ardèche', 'rhone-alpes/drome/r22d26': u'Drôme', 'rhone-alpes/haute-savoie/r22d74': u'Haute-Savoie', 'rhone-alpes/isere/r22d38': u'Isère', 'rhone-alpes/loire/r22d42': u'Loire', 'rhone-alpes/rhone/r22d69': u'Rhône', 'rhone-alpes/savoie/r22d73': u'Savoie', }.iteritems())]) CONFIG = BackendConfig(Value('job', label='Job name', masked=False, default=''), Value('place', label=u'County', choices=places_choices, default=''), Value('contract', labe=u'Contract type', choices=type_contract_choices, default=''), Value('activity_domain', label=u'Activity Domain', choices=activityDomain_choices, default=''), ) def advanced_search_job(self): for advert in self.browser.advanced_search_job(job=self.config['job'].get(), place=self.config['place'].get(), contract=self.config['contract'].get(), activity_domain=self.config['activity_domain'].get()): yield advert def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def search_job(self, pattern=None): for advert in self.browser.search_job(pattern): yield advert def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/manpower/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000006336�13034501105�0017602�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import date from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, Env, BrowserURL, Date, Format from weboob.browser.filters.html import CleanHTML from weboob.capabilities.job import BaseJobAdvert from weboob.capabilities.base import NotAvailable class SearchPage(HTMLPage): @method class iter_job_adverts(ListElement): item_xpath = '//div[has-class("item")]' class item(ItemElement): klass = BaseJobAdvert obj_id = Regexp(CleanText('./div/a[@class="title-link"]/@href'), '/candidats/detail-offre-d-emploi/(.*).html') obj_title = CleanText('./div/a[@class="title-link"]/h2') def obj_place(self): content = CleanText('./div[2]')(self) if len(content.split('|')) > 1: return content.split('|')[1] return '' def obj_publication_date(self): content = CleanText('./div[2]')(self) split_date = content.split('|')[0].split('/') if len(split_date) == 3: return date(int(split_date[2]) + 2000, int(split_date[1]), int(split_date[0])) return '' class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Env('_id') obj_url = BrowserURL('advert_page', _id=Env('_id')) obj_title = CleanText('//div[@class="infos-lieu"]/h1') obj_place = CleanText('//div[@class="infos-lieu"]/h2') obj_publication_date = Date(Regexp(CleanText('//div[@class="info-agency"]'), '.*Date de l\'annonce :(.*)', default='')) obj_job_name = CleanText('//div[@class="infos-lieu"]/h1') obj_description = Format('\n%s%s', CleanHTML('//article[@id="post-description"]/div'), CleanHTML('//article[@id="poste"]')) obj_contract_type = Regexp(CleanText('//article[@id="poste"]/div/ul/li'), 'Contrat : (\w*)', default=NotAvailable) obj_pay = Regexp(CleanText('//article[@id="poste"]/div/ul/li'), 'Salaire : (.*) par mois', default=NotAvailable) obj_experience = Regexp(CleanText('//article[@id="poste"]/div/ul/li'), u'Expérience : (.* ans)', default=NotAvailable) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/manpower/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002662�13034501105�0017460�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest import itertools class ManpowerTest(BackendTest): MODULE = 'manpower' def test_manpower(self): l = list(itertools.islice(self.backend.search_job(u'manutentionnaire'), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_manpower_advanced_search(self): l = list(itertools.islice(self.backend.advanced_search_job(), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) ������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016236�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020346�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MareeinfoModule __all__ = ['MareeinfoModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000002212�13034501105�0020270�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import IndexPage class MareeinfoBrowser(PagesBrowser): BASEURL = 'http://maree.info' harbor_page = URL('', '(?P<_id>.*)', IndexPage) def get_harbor_list(self, pattern): return self.harbor_page.go().get_harbor_list(pattern=pattern) def get_harbor_infos(self, gauge): return self.harbor_page.go(_id=gauge.id).get_harbor_infos(obj=gauge) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000001033�13034501105�0020366�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���gAMA�� a��� pHYs����(J���tEXtSoftware�paint.net 4.0.3P��IDATx^A Ѿg4F?L!;ĀAyRm؟qhIOG5Z-"O -6h@G }B% ݩJ.YNUz4tBwңKS }BБmh4|DiѰ#JU 4̖f 5h@*pd[Ehixv<ZZ;-֎GKkǣ2hK ߢ7Ё(?ޮ+VOћZz*O^Xw^?t.zCAX5aO^tWްrVtO󢻞n,tG{V1O-tVsZ >IZx>) MEjfv)`KUj4pT[z-8dk]Dn#����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004572�13034501105�0020105�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.base import find_object from weboob.capabilities.gauge import CapGauge, Gauge, SensorNotFound from .browser import MareeinfoBrowser __all__ = ['MareeinfoModule'] class MareeinfoModule(Module, CapGauge): NAME = 'mareeinfo' DESCRIPTION = u'Un module qui permet d\' aller a la pêche aux moules totalement informé' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = MareeinfoBrowser def get_last_measure(self, sensor_id): gauge_id = sensor_id.split('-')[0] return find_object(self.iter_sensors(gauge_id), id=sensor_id, error=SensorNotFound).lastvalue def iter_gauge_history(self, sensor_id): gauge_id = sensor_id.split('-')[0] return find_object(self.iter_sensors(gauge_id), id=sensor_id, error=SensorNotFound).history def iter_gauges(self, pattern=None): for _gauge in self.browser.get_harbor_list(pattern): if pattern is not None: gauge = self.browser.get_harbor_infos(_gauge) yield gauge else: yield _gauge def iter_sensors(self, gauge, pattern=None): if not isinstance(gauge, Gauge): gauge = find_object(self.iter_gauges(), id=gauge, error=SensorNotFound) gauge = self.browser.get_harbor_infos(gauge) if pattern is None: for sensor in gauge.sensors: yield sensor else: lowpattern = pattern.lower() for sensor in gauge.sensors: if lowpattern in sensor.name.lower(): yield sensor ��������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mareeinfo/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000020673�13034501105�0017717�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, DateTime, CleanDecimal, Regexp from weboob.browser.filters.html import Link, XPath from weboob.capabilities.gauge import Gauge, GaugeMeasure, GaugeSensor from datetime import timedelta import re class IndexPage(HTMLPage): @method class get_harbor_list(ListElement): item_xpath = "//a[@class='Port PP'] | //a[@class='Port PS']" class item(ItemElement): klass = Gauge obj_id = CleanText(Link('.'), replace=[('/', '')]) obj_name = CleanText('.') obj_city = CleanText('.') obj_object = u'Port' def validate(self, obj): if self.env['pattern']: return self.env['pattern'].lower() in obj.name.lower() return True @method class get_harbor_infos(ItemElement): klass = Gauge def _create_coef_sensor(self, gauge_id, AM=True): name = CleanText('//tr[@class="MJE"]/th[4]')(self) _name = 'matin' if AM else 'aprem' value = self._get_coef_value(AM=AM) if value: coef = GaugeSensor(u'%s-%s-%s' % (gauge_id, name, _name)) coef.name = '%s %s' % (name, _name) coef.lastvalue = value coef.gaugeid = gauge_id coef.history = [] for jour in range(0, 7): measure = self._get_coef_value(AM=AM, jour=jour) if measure: coef.history.append(measure) return coef def _get_coef_value(self, AM=True, jour=0): if AM: time = DateTime(CleanText('//tr[@id="MareeJours_%s"]/td[1]/b[1]' % jour))(self) value = CleanText('//tr[@id="MareeJours_%s"]/td[3]/b[1]' % jour)(self) else: time, value = None, None if len(XPath('//tr[@id="MareeJours_%s"]/td[1]/b' % jour)(self)) > 1: time = DateTime(CleanText('//tr[@id="MareeJours_%s"]/td[1]/b[2]' % jour))(self) value = CleanText('//tr[@id="MareeJours_%s"]/td[3]/b[2]' % jour)(self) if time and value: measure = GaugeMeasure() measure.level = float(value) measure.date = time + timedelta(days=jour) return measure def _create_high_tide(self, gauge_id, AM=True): name = CleanText('//tr[@class="MJE"]/th[3]')(self) _name = 'matin' if AM else 'aprem' value = self._get_high_tide_value(AM=AM) if value: tide = GaugeSensor(u'%s-%s-PM-%s' % (gauge_id, name, _name)) tide.name = u'Pleine Mer %s' % (_name) tide.unit = u'm' tide.lastvalue = value tide.gaugeid = gauge_id tide.history = [] for jour in range(0, 7): measure = self._get_high_tide_value(AM=AM, jour=jour) if measure: tide.history.append(measure) return tide def _get_high_tide_value(self, AM=True, jour=0): if AM: time = DateTime(CleanText('//tr[@id="MareeJours_%s"]/td[1]/b[1]' % jour))(self) value = CleanDecimal('//tr[@id="MareeJours_0"]/td[2]/b[1]', replace_dots=True)(self) else: time, value = None, None if len(XPath('//tr[@id="MareeJours_%s"]/td[1]/b' % jour)(self)) > 1: time = DateTime(CleanText('//tr[@id="MareeJours_%s"]/td[1]/b[2]' % jour), default=None)(self) value = CleanDecimal('//tr[@id="MareeJours_0"]/td[2]/b[2]', replace_dots=True, default=None)(self) if time and value: measure = GaugeMeasure() measure.level = float(value) measure.date = time + timedelta(days=jour) return measure def _create_low_tide(self, gauge_id, AM=True): name = CleanText('//tr[@class="MJE"]/th[3]')(self) _name = 'matin' if AM else 'aprem' value = self._get_low_tide_value(AM=AM) if value: tide = GaugeSensor(u'%s-%s-BM-%s' % (gauge_id, name, _name)) tide.name = u'Basse Mer %s' % (_name) tide.unit = u'm' tide.lastvalue = value tide.gaugeid = gauge_id tide.history = [] for jour in range(0, 7): measure = self._get_low_tide_value(AM=AM, jour=jour) if measure: tide.history.append(measure) return tide def _is_low_tide_first(self, jour): return XPath('//tr[@id="MareeJours_%s"]/td[1]' % jour)(self)[0].getchildren()[0].tag != 'b' def _get_low_tide_value(self, AM=True, jour=0): slow_tide_pos = 1 if self._is_low_tide_first(jour) else 2 m = re.findall('(\d{2}h\d{2})', CleanText('//tr[@id="MareeJours_%s"]/td[1]' % jour)(self)) re_time = '(\d{2}h\d{2}).*(\d{2}h\d{2}).*(\d{2}h\d{2})' re_value = '(.*)m(.*)m(.*)m' if len(m) > 3: re_time = '(\d{2}h\d{2}).*(\d{2}h\d{2}).*(\d{2}h\d{2}).*(\d{2}h\d{2})' re_value = '(.*)m(.*)m(.*)m(.*)m' if AM: time = DateTime(Regexp(CleanText('//tr[@id="MareeJours_%s"]/td[1]' % jour), re_time, '\\%s' % slow_tide_pos))(self) value = CleanDecimal(Regexp(CleanText('//tr[@id="MareeJours_%s"]/td[2]' % jour), re_value, '\\%s' % slow_tide_pos), replace_dots=True, default=None)(self) else: slow_tide_pos += 2 time, value = None, None if len(m) > slow_tide_pos - 1: time = DateTime(Regexp(CleanText('//tr[@id="MareeJours_%s"]/td[1]' % jour), re_time, '\\%s' % slow_tide_pos))(self) value = CleanDecimal(Regexp(CleanText('//tr[@id="MareeJours_%s"]/td[2]' % jour), re_value, '\\%s' % slow_tide_pos), replace_dots=True, default=None)(self) if time and value: measure = GaugeMeasure() measure.level = float(value) measure.date = time + timedelta(days=jour) return measure def obj_sensors(self): sensors = [] high_tide_PM = self._create_high_tide(self.obj.id) if high_tide_PM: sensors.append(high_tide_PM) high_tide_AM = self._create_high_tide(self.obj.id, AM=False) if high_tide_AM: sensors.append(high_tide_AM) low_tide_AM = self._create_low_tide(self.obj.id) if low_tide_AM: sensors.append(low_tide_AM) low_tide_PM = self._create_low_tide(self.obj.id, AM=False) if low_tide_PM: sensors.append(low_tide_PM) coef_AM = self._create_coef_sensor(self.obj.id) if coef_AM: sensors.append(coef_AM) coef_PM = self._create_coef_sensor(self.obj.id, AM=False) if coef_PM: sensors.append(coef_PM) return sensors ���������������������������������������������������������������������weboob-1.2/modules/mareeinfo/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002322�13034501105�0017566�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class MareeinfoTest(BackendTest): MODULE = 'mareeinfo' def test_mareeinfo(self): l = list(self.backend.iter_gauges()) self.assertTrue(len(l) > 0) gauge = l[0] s = list(self.backend.iter_sensors(gauge)) self.assertTrue(len(s) > 0) sensor = s[0] self.assertTrue(self.backend.get_last_measure(sensor.id) is not None) self.assertTrue(len(self.backend.iter_gauge_history(sensor.id)) > 0) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016117�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001435�13034501105�0020233�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MarmitonModule __all__ = ['MarmitonModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000003030�13034501105�0020150�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser, URL from .pages import RecipePage, ResultsPage __all__ = ['MarmitonBrowser'] class MarmitonBrowser(PagesBrowser): BASEURL = 'http://www.marmiton.org/' search = URL('recettes/recherche.aspx\?aqt=(?P<pattern>.*)', ResultsPage) recipe = URL('recettes/recette_(?P<id>.*).aspx', RecipePage) def iter_recipes(self, pattern): return self.search.go(pattern=pattern).iter_recipes() def get_recipe(self, id, recipe=None): try: recipe = self.recipe.go(id=id).get_recipe(obj=recipe) comments = list(self.page.get_comments()) if comments: recipe.comments = comments return recipe except BrowserHTTPNotFound: return ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000004173�13034501105�0020257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 6:m��IDATx{P\ǿMX� yGlXck715v4Ռ8R%&5:05:5Dq3q GB`eaeww ,ý9Bp _4`0�,s#\g�?xlnd"6|1Ko0>`ÓJ.9�3B !.L^{@2K Up-l*?=�!0�Y|HnP4<Rd\07*ѷe* tZ(Xw,,) y�u#ׁɭ�$|k=/ P4lw\d/ !`\- !wOˆv�kN�,uY\`YsX*6D !?hWALn)P,vv/]iݧ)Ñ 6O~e?Pͭ��n-J8?reypFh28}�z{+`M5(Z膳 hW$"ȡe,!y>ī6[1߿׃Hعm%Jwyj@$�0`V:yp ʾ�|K=gNIW,;][A.'^ ‡w�u*U*PD$23 @ }jۥ wb�4 88�KF69 $U 7hԳ;&E`r+eX#:OV#cߪi*tD C&BnDNM=*O t q J]`(w(;x 3A;bo u)8%H gv5Ɋմ �BVb-ρh(1�2b/v}2*ok@�ݭl(g622hwՏjwT/!fVhhHc�·^7R'F'W`L6o&Omt�>} ) ^ Bl4tDB<{ z!^iN(`&.~bl@�L�&�{͜'xߠvecR@�ln%7L}O�gZKAr02O,>ouY|fļ/\GsVp `Z߽/ZcDӭq+Pm 45ƀ뷀-\ʾR_;Gw"r,ioW{ly Qݧ1SZ�^P59uH՞ /oUnP1{= eX#| MMm·يGu0I >S19<:_.P wm߶7,˂x弼\wgkT8*zFQ>oqd;!iVrԘ9C͇AB|]`b g&ժ)LJybBBWβ'\*fy Qg.Sg�F5�QpEk�ZȱH 6;r%z(Q:XmrO�)n.�9�kY��T5jX`v#\#1iOY"9tV;Z �gm9ȰΟׂ9TH> ,lw qϬ@p B"#3;JJm $�*_<`?cDOjU)4EMC`Hʕ^10!|m'(xs�F:iX{߄z#3 $i  ݔz7MxΥT:HG۽/ijs۽o'<s t5�<c�+sLLȷيMIOAO$Nhv mh x(xSaRL4V>' :l0�Z;df����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/module.py���������������������������������������������������������������0000664�0000000�0000000�00000003170�13034501105�0017757�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.recipe import CapRecipe, Recipe from weboob.tools.backend import Module from .browser import MarmitonBrowser from urllib import quote_plus __all__ = ['MarmitonModule'] class MarmitonModule(Module, CapRecipe): NAME = 'marmiton' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'Marmiton French recipe website' LICENSE = 'AGPLv3+' BROWSER = MarmitonBrowser def get_recipe(self, id): return self.browser.get_recipe(id) def iter_recipes(self, pattern): return self.browser.iter_recipes(quote_plus(pattern.encode('utf-8'))) def fill_recipe(self, recipe, fields): if 'nb_person' in fields or 'instructions' in fields or 'thumbnail_url' in fields: recipe = self.browser.get_recipe(recipe.id, recipe) return recipe OBJECTS = { Recipe: fill_recipe, } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000007443�13034501105�0017600�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Regexp, CleanText, Format, Env, Type from weboob.browser.filters.html import CleanHTML from weboob.capabilities.recipe import Recipe, Comment from weboob.capabilities.base import NotAvailable class ResultsPage(HTMLPage): """ Page which contains results as a list of recipies """ @pagination @method class iter_recipes(ListElement): item_xpath = '//div[has-class("recette_classique")]' def next_page(self): return CleanText('//a[@id="ctl00_cphMainContent_m_ctrlSearchEngine_m_ctrlSearchListDisplay_m_ctrlSearchPagination_m_linkNextPage"]/@href', default=None)(self) class item(ItemElement): klass = Recipe obj_id = Regexp(CleanText('./div/div[@class="m_titre_resultat"]/a/@href'), '/recettes/recette_(.*).aspx') obj_title = CleanText('./div/div[@class="m_titre_resultat"]/a') obj_short_description = Format('%s. %s', CleanText('./div/div[@class="m_detail_recette"]'), CleanText('./div/div[@class="m_texte_resultat"]')) class RecipePage(HTMLPage): """ Page which contains a recipe """ @method class get_recipe(ItemElement): klass = Recipe obj_id = Env('id') obj_title = CleanText('//h1[has-class("m_title")]') obj_preparation_time = Type(CleanText('//span[@class="preptime"]'), type=int) obj_cooking_time = Type(CleanText('//span[@class="cooktime"]'), type=int) def obj_nb_person(self): nb_pers = Regexp(CleanText('//div[@class="m_content_recette_ingredients m_avec_substitution"]/span[1]'), '.*\(pour (\d+) personnes\)', default=0)(self) return [nb_pers] if nb_pers else NotAvailable def obj_ingredients(self): ingredients = CleanText('//div[@class="m_content_recette_ingredients m_avec_substitution"]', default='')(self).split('-') if len(ingredients) > 1: return ingredients[1:] return [] obj_instructions = CleanHTML('//div[@class="m_content_recette_todo"]') obj_thumbnail_url = CleanText('//a[@class="m_content_recette_illu"]/img/@src', default=NotAvailable) obj_picture_url = CleanText('//a[@class="m_content_recette_illu"]/img/@src', default=NotAvailable) @method class get_comments(ListElement): item_xpath = '//div[@class="m_commentaire_row"]' ignore_duplicate = True class item(ItemElement): klass = Comment obj_author = CleanText('./div[@class="m_commentaire_content"]/span[1]') obj_rate = CleanText('./div[@class="m_commentaire_note"]/span') obj_text = CleanText('./div[@class="m_commentaire_content"]/p[1]') obj_id = CleanText('./div[@class="m_commentaire_content"]/span[1]') �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/marmiton/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002313�13034501105�0017447�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest import itertools class MarmitonTest(BackendTest): MODULE = 'marmiton' def test_recipe(self): recipes = list(itertools.islice(self.backend.iter_recipes('fondue'), 0, 20)) for recipe in recipes: full_recipe = self.backend.get_recipe(recipe.id) assert full_recipe.instructions assert full_recipe.ingredients assert full_recipe.title assert full_recipe.preparation_time ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/materielnet/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016602�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/materielnet/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001452�13034501105�0020715�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MaterielnetModule __all__ = ['MaterielnetModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/materielnet/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000003340�13034501105�0020637�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, CaptchaPage, ProfilPage, DocumentsPage class MaterielnetBrowser(LoginBrowser): BASEURL = 'https://www.materiel.net' login = URL('/pm/client/login.html', LoginPage) captcha = URL('/pm/client/captcha.html', CaptchaPage) profil = URL('/pm/client/compte.html', ProfilPage) documents = URL('/pm/client/commande.html\?page=(?P<page>.*)', '/pm/client/commande.html', DocumentsPage) def do_login(self): self.login.go() self.page.login(self.username, self.password) if self.login.is_here() or self.captcha.is_here(): raise BrowserIncorrectPassword(self.page.get_error()) @need_login def get_subscription_list(self): return self.profil.stay_or_go().get_list() @need_login def iter_documents(self, subscription): return self.documents.stay_or_go(page=1).get_documents() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/materielnet/module.py������������������������������������������������������������0000664�0000000�0000000�00000004771�13034501105�0020452�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object, NotAvailable from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import MaterielnetBrowser __all__ = ['MaterielnetModule'] class MaterielnetModule(Module, CapDocument): NAME = 'materielnet' DESCRIPTION = u'Materiel.net' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='Identifiant'), ValueBackendPassword('password', label='Mot de passe')) BROWSER = MaterielnetBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) if document._url is NotAvailable: return return self.browser.open(document._url).content �������weboob-1.2/modules/materielnet/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000006626�13034501105�0020265�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.filters.standard import CleanText, CleanDecimal, Env, Format, TableCell, Date, Async, AsyncLoad from weboob.browser.elements import ListElement, ItemElement, TableElement, method from weboob.browser.filters.html import Attr from weboob.capabilities.bill import Bill, Subscription from weboob.capabilities.base import NotAvailable class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form('//form[contains(@action, "login")]') form['login'] = login form['pass'] = password form.submit() def get_error(self): return CleanText('//div[@class="ValidatorError"]')(self.doc) class CaptchaPage(HTMLPage): def get_error(self): return CleanText('//div[@class="captcha-block"]/p[1]/text()')(self.doc) class ProfilPage(LoggedPage, HTMLPage): @method class get_list(ListElement): class item(ItemElement): klass = Subscription obj_subscriber = Format('%s %s', Attr('//input[@id="prenom"]', 'value'), Attr('//input[@id="nom"]', 'value')) obj_id = Env('subid') obj_label = obj_id def parse(self, el): self.env['subid'] = self.page.browser.username class DocumentsPage(LoggedPage, HTMLPage): @pagination @method class get_documents(TableElement): item_xpath = '//table[@class="EpCmdList"]/tr[@class="Line1" or @class="Line2"]' head_xpath = '//table[@class="EpCmdList"]/tr/th' col_id = u'Référence' col_date = u'Date' col_price = u'Montant' def next_page(self): m = re.search('([^*]+page=)([^*]+)', self.page.url) if m: page = int(m.group(2)) + 1 if self.el.xpath('//a[contains(@href, "commande.html?page=' + str(page) + '")]'): next_page = u"%s%s" % (m.group(1), page) return next_page class item(ItemElement): klass = Bill load_details = Attr('./td/a', 'href') & AsyncLoad obj_id = Format('%s_%s', Env('email'), CleanDecimal(TableCell('id'))) obj__url = Async('details') & Attr('//a[contains(@href, "facture")]', 'href', default=NotAvailable) obj_date = Date(CleanText(TableCell('date'))) obj_format = u"pdf" obj_label = Async('details') & CleanText('//table/tr/td[@class="Prod"]') obj_type = u"bill" obj_price = CleanDecimal(TableCell('price'), replace_dots=True) obj_currency = u"€" def parse(self, el): self.env['email'] = self.page.browser.username ����������������������������������������������������������������������������������������������������������weboob-1.2/modules/materielnet/test.py��������������������������������������������������������������0000664�0000000�0000000�00000001622�13034501105�0020134�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class MaterielnetTest(BackendTest): MODULE = 'materielnet' def test_materielnet(self): raise NotImplementedError() ��������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016234�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001443�13034501105�0020347�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MediawikiModule __all__ = ['MediawikiModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000021266�13034501105�0020300�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from urlparse import urlsplit, urljoin import datetime import re from weboob.browser.browsers import DomainBrowser from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.content import Revision __all__ = ['MediawikiBrowser'] class APIError(Exception): pass # Browser class MediawikiBrowser(DomainBrowser): ENCODING = 'utf-8' def __init__(self, url, apiurl, username, password, *args, **kwargs): url_parsed = urlsplit(url) self.PROTOCOL = url_parsed.scheme self.DOMAIN = url_parsed.netloc self.BASEPATH = url_parsed.path if self.BASEPATH.endswith('/'): self.BASEPATH = self.BASEPATH[:-1] self.apiurl = apiurl self.username = username self.password = password DomainBrowser.__init__(self, *args, **kwargs) def url2page(self, page): baseurl = self.PROTOCOL + '://' + self.DOMAIN + self.BASEPATH m = re.match('^' + urljoin(baseurl, 'wiki/(.+)$'), page) if m: return m.group(1) else: return page def get_wiki_source(self, page, rev=None): assert isinstance(self.apiurl, basestring) page = self.url2page(page) data = {'action': 'query', 'prop': 'revisions|info', 'titles': page, 'rvprop': 'content|timestamp|ids', 'rvlimit': '1', 'intoken': 'edit', } if rev: data['rvstartid'] = rev result = self.API_get(data) pageid = result['query']['pages'].keys()[0] if pageid == "-1": # Page does not exist return "" if 'revisions' not in repr(result['query']['pages'][str(pageid)]): raise APIError('Revision %s does not exist' % rev) if rev and result['query']['pages'][str(pageid)]['revisions'][0]['revid'] != int(rev): raise APIError('Revision %s does not exist' % rev) return result['query']['pages'][str(pageid)]['revisions'][0]['*'] def get_token(self, page, _type): ''' _type can be edit, delete, protect, move, block, unblock, email or import''' if len(self.username) > 0 and not self.is_logged(): self.login() data = {'action': 'query', 'prop': 'info', 'titles': page, 'intoken': _type, } result = self.API_get(data) pageid = result['query']['pages'].keys()[0] return result['query']['pages'][str(pageid)][_type + 'token'] def set_wiki_source(self, content, message=None, minor=False): if len(self.username) > 0 and not self.is_logged(): self.login() page = content.id token = self.get_token(page, 'edit') data = {'action': 'edit', 'title': page, 'token': token, 'text': content.content.encode('utf-8'), 'summary': message, } if minor: data['minor'] = 'true' self.API_post(data) def get_wiki_preview(self, content, message=None): data = {'action': 'parse', 'title': content.id, 'text': content.content.encode('utf-8'), 'summary': message, } result = self.API_post(data) return result['parse']['text']['*'] def is_logged(self): data = {'action': 'query', 'meta': 'userinfo', } result = self.API_get(data) return result['query']['userinfo']['id'] != 0 def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert isinstance(self.apiurl, basestring) data = {'action': 'login', 'lgname': self.username, 'lgpassword': self.password, } result = self.API_post(data) if result['login']['result'] == 'WrongPass': raise BrowserIncorrectPassword() if result['login']['result'] == 'NeedToken': data['lgtoken'] = result['login']['token'] self.API_post(data) def iter_wiki_revisions(self, page): """ Yield 'Revision' objects. """ if len(self.username) > 0 and not self.is_logged(): self.login() MAX_RESULTS = 50 results = MAX_RESULTS last_id = None while results == MAX_RESULTS: data = {'action': 'query', 'titles': page, 'prop': 'revisions', 'rvprop': 'ids|timestamp|comment|user|flags', 'rvlimit': str(MAX_RESULTS), } if last_id is not None: data['rvstartid'] = last_id result = self.API_get(data) pageid = str(result['query']['pages'].keys()[0]) results = 0 if pageid != "-1": for rev in result['query']['pages'][pageid]['revisions']: rev_content = Revision(str(rev['revid'])) rev_content.comment = rev['comment'] rev_content.author = rev['user'] rev_content.timestamp = datetime.datetime.strptime(rev['timestamp'], '%Y-%m-%dT%H:%M:%SZ') rev_content.minor = 'minor' in rev yield rev_content last_id = rev_content.id results += 1 def _common_file_request(self): return {'action': 'query', 'prop': 'info|imageinfo', 'inprop': 'url', 'iiprop': 'extmetadata|size|url|canonicaltitle', 'iiurlwidth': 512, 'iiurlheight': 512, } def _common_parse_file(self, info): res = {'canonicalurl': info['canonicalurl'], 'title': info['title'], 'size': info['imageinfo'][0]['size'], } iinfo = info['imageinfo'][0] if 'url' in iinfo: res['original'] = iinfo['url'] if 'thumburl' in iinfo: res['thumbnail'] = iinfo['thumburl'] return res def search_file(self, pattern): data = self._common_file_request() data['generator'] = 'search' data['gsrnamespace'] = 6 # File: namespace data['gsrsearch'] = pattern while True: response = self.API_get(data) for fdict in response['query']['pages'].values(): yield self._common_parse_file(fdict) if 'continue' in response: data.update(response['continue']) else: break def get_image(self, page): page = self.url2page(page) data = self._common_file_request() data['titles'] = page response = self.API_get(data) pageid = response['query']['pages'].keys()[0] info = response['query']['pages'][pageid] return self._common_parse_file(info) def home(self): # We don't need to change location, we're using the JSON API here. pass def check_result(self, result): if 'error' in result: raise APIError(result['error']['info']) def API_get(self, data): """ Submit a GET request to the website The JSON data is parsed and returned as a dictionary """ data['format'] = 'json' result = self.open(self.apiurl, params=data).json() self.check_result(result) return result def API_post(self, data): """ Submit a POST request to the website The JSON data is parsed and returned as a dictionary """ data['format'] = 'json' result = self.open(self.apiurl, data=data).json() self.check_result(result) return result ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000002223�13034501105�0020366�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME@~S���tEXtComment�Created with GIMPW��IDATx[UUxiCpC`xEA, / Ճ$2RY =(#"(# ރj ihV0֐FH8Xw9{ p8}{}[_;xmp8^C#~Z%Lc)0ԯ F1,`0H1%p`&p4X a*\5p;W��Fq$m0 @[4Ѱ(u h4|4COmG uK0zp1zO/9{@ݦdǗ]#z0R+ X&/Ǎ/o*GgM{0ROܿe0"yʪ ikFzwE]cKeװO=J.a4)}8%ZZ( 4Td%sj#Jx=$s 2�!#+WT*"XOL~VGpJe\H` <7*!q8yJ%/J>LIiNs|+w W,C4Jnk&Oe�%B{N`Vd5U ܗ{yDşJ@r~]>00R}C]Vk'\?*ސL2U~,M2wM"oJErC{A_ @9pPG~.m٤Y*73J^ǰ[ ﷁ'#{Z,R�YղC>msf"y>( PZVh%5%lH^*]SX[F+AUL)=0w7;9חYLO.'J%�2iDTV'dPd +SE[HU7M2GܷӸ_U.JդmpȭGMؼM񎹩Wr=nm����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/module.py��������������������������������������������������������������0000664�0000000�0000000�00000010126�13034501105�0020073�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import os from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.content import CapContent, Content from weboob.capabilities.file import CapFile from weboob.capabilities.image import CapImage, BaseImage, Thumbnail from weboob.tools.value import ValueBackendPassword, Value from .browser import MediawikiBrowser __all__ = ['MediawikiModule'] class WikiImage(BaseImage): @property def page_url(self): return self._canonical_url class MediawikiModule(Module, CapContent, CapImage): NAME = 'mediawiki' MAINTAINER = u'Clément Schreiner' EMAIL = 'clemux@clemux.info' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Wikis running MediaWiki, like Wikipedia' CONFIG = BackendConfig(Value('url', label='URL of the Mediawiki website', default='https://en.wikipedia.org/', regexp='https?://.*'), Value('apiurl', label='URL of the Mediawiki website\'s API', default='https://en.wikipedia.org/w/api.php', regexp='https?://.*'), Value('username', label='Login', default=''), ValueBackendPassword('password', label='Password', default='')) BROWSER = MediawikiBrowser def create_default_browser(self): username = self.config['username'].get() if len(username) > 0: password = self.config['password'].get() else: password = '' return self.create_browser(self.config['url'].get(), self.config['apiurl'].get(), username, password) def get_content(self, _id, revision=None): _id = _id.replace(' ', '_') content = Content(_id) page = _id rev = revision.id if revision else None data = self.browser.get_wiki_source(page, rev) content.content = data return content def iter_revisions(self, _id): return self.browser.iter_wiki_revisions(_id) def push_content(self, content, message=None, minor=False): self.browser.set_wiki_source(content, message, minor) def get_content_preview(self, content): return self.browser.get_wiki_preview(content) def _make_image(self, info): img = WikiImage(info['title']) img.title, img.ext = os.path.splitext(info['title']) img.title = img.title.rsplit(':', 1)[-1] img.size = info['size'] if 'thumbnail' in info: thumb = Thumbnail(info['thumbnail']) img.thumbnail = thumb if 'original' in info: img.url = info['original'] img._canonical_url = info['canonicalurl'] return img def search_file(self, pattern, sortby=CapFile.SEARCH_RELEVANCE): for info in self.browser.search_file(pattern): yield self._make_image(info) def get_image(self, _id): _id = _id.replace(' ', '_') info = self.browser.get_image(_id) return self._make_image(info) def fill_img(self, obj, fields): if set(fields) & set(('url', 'thumbnail')): new = self.get_image(obj.id) if not obj.url: obj.url = new.url if not obj.thumbnail: obj.thumbnail = new.thumbnail if 'data' in fields: obj.data = self.browser.open(obj.url).content return obj OBJECTS = {BaseImage: fill_img, Thumbnail: fill_img} ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/mediawiki/test.py����������������������������������������������������������������0000664�0000000�0000000�00000004003�13034501105�0017562�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from datetime import datetime class MediawikiTest(BackendTest): MODULE = 'mediawiki' def test_get_content(self): obj = self.backend.get_content(u"Project:Sandbox") assert len(obj.content) > 0 def test_iter_revisions(self): for rev in zip(range(10), self.backend.iter_revisions(u"Project:Sandbox")): pass def test_push_content(self): content = self.backend.get_content(u"Project:Sandbox") content.content = "test "+str(datetime.now()) self.backend.push_content(content, message="test weboob", minor=True) new_content = self.backend.get_content(u"Project:Sandbox") assert content.content == new_content.content def test_content_preview(self): content = self.backend.get_content(u"Project:Sandbox") self.backend.get_content_preview(content) def test_search_image(self): it = iter(self.backend.search_file('logo')) for _, img in zip(xrange(3), it): assert img assert img.title assert img.ext assert img.page_url assert img.size if not img.url: img = self.backend.fillobj(img, ['url']) assert img.url assert img.thumbnail.url �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016561�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001447�13034501105�0020700�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MeteofranceModule __all__ = ['MeteofranceModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000002717�13034501105�0020625�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Cedric Defortis # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import WeatherPage, SearchCitiesPage __all__ = ['MeteofranceBrowser'] class MeteofranceBrowser(PagesBrowser): BASEURL = 'http://www.meteofrance.com' cities = URL('mf3-rpc-portlet/rest/lieu/facet/previsions/search/(?P<pattern>.*)', SearchCitiesPage) weather = URL('previsions-meteo-france/(?P<city_name>.*)/(?P<city_id>.*)', WeatherPage) def iter_city_search(self, pattern): return self.cities.go(pattern=pattern).iter_cities() def iter_forecast(self, city): return self.weather.go(city_id=city.id, city_name=city.name).iter_forecast() def get_current(self, city): return self.weather.go(city_id=city.id, city_name=city.name).get_current() �������������������������������������������������weboob-1.2/modules/meteofrance/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000007055�13034501105�0020723�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���% ���sRGB���� pHYs�� �� ����tIME  #=<���tEXtComment�Created with GIMPW�� IDAThޭZoTͽ]{m`06BKcF%)|HQ!RHjP> V$j+EJ+P۴j miZT$)`o}yҖ+ vs93̡ 28}p\u@z_<W:s<܅C 1PKw 7DbO P0((>H`@7NyHL�J>h<swYp4 )$`$Ġʐ4YڐZ8cʋ06/@ k Wa�Eu5**JuA/X"?O:S+6]X~�]^ 8&r$PS@s+WfL G4ʓ 01}:�(}Dp4uק�w2Y K:V=qD`9?S^rGܘ٥G?{ ?L8& /y}(Bnq-Mj>ΫMG $VH{Nr/Ug_sk7z0bD8�3A3 XR$iF^237HX\S;N8ػ|JK MkW\k 1xc3% b7y|<͡r)0i(#,φ["�P_oY{wt|>{;|ÃI*Zbubђf{x EP 56fRo+-rY/$<Zp?WfTf,ȿW[-sZt\`S_QE!% Œx)̘j=wB,!7y:5/gv1sZJ^J4{wwmR!44`s%J�} oF(pGN,iCֱ5yo|8s30ZlGJU9)f8xITVFa�O…�NWKE`p�pϖxzg+7O޿`uZZKY7dE�բTPľt3?S �Pr}R)j~cnPS"07 g1̚x09N{of}#dX |H. y|p"M�GܷkOk6lT_|h%3f|2 y/gmGBHz�wǫRi ?�RY^6�039F9 졷26?Vpu}bـa2WO:|cjj \Se{p'W#, k@i:8NbcLk�ѯɰTnXGrbl/H�^kjɶ /= ]z{}oi 1;0׿nt:5$,v$r6-k/ &nܹb+;i ݈W0a"hi 5+qW{C523!L-L{dYBY][_+'dr5;!y,bqz$wvdgBdFF pj뮽&býZVe^*{[(פdʘUaȆvW]9n`m!R(dGfL] mlapvO{lƴ7w?+~W O`" "Gx$W8Ga/K/ndž꓇< |> Rd%]fa3Y䗷N9q`צ-*>ګ'G3}G]+ lUl\JMÌy~k *Ɠ>N\zw'wZωx-@ueÖbKIrC1IRaEK:a0[8m>*wkP u)? o7lt"i-YeRY$j(ߪɾ ޒWym*nt N/RE7@!W<:K@z%و<YQ kJ`FSV\X*vZר9?IA?%Հ8rR2C `Wie J1G@LɖItSl̜лjf?|I)X.TfX&qAf &$PKJP;4[,. [k?S$HV}J FFV8i5iլFYvXM- C`z&kj!TQm4X5jt-eYy! 1ⷵh#.R+Ȉ=L'Kgޭ7D5?Db^X@NT + {0Q<L3F(pE*rMl i̝#xwBEDG"(i]a0lZh[MRAy> B2jIEYܪj6XyJaBH$'sVJ2i<6 lRXV|EQ+ksBv2v[WZ3ʰ=GFh)x? yy4>%':=(U_PqM0yx%G%j *12Ozu}H9hĉic?k�xw8.a#fODM$fDEr"{fz 5bq OҘ,E_2U:N{{/vаԓO<tj%`})4/Ge<Qv,Du;w/9CE2"?+Gfu.Dz,ԞtQ"K]xiLLo6E=a KR%h8iTۃ;~LYD~PX[ 8e /A&*2GTFJztƮ7yנ"jRRJ&2Yj{ÌzLG!QEY {lsNm)tt`J,Z/'ZgH>0y2\/DO$#q3y᠈)yE5(+Fk':NY gÚp(l`2'5VC:t:<ko6vh&:j@6_C �]E]@y xx/<n̅jЕ4b33&X>I\“Ϯ%}"FGY GXu4I8"o5ݥ0Zw6U5.F*aػ:&"!of>)4t|IĢ"6.d 4rvm.R i>V+ج4sm&2x;����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/module.py������������������������������������������������������������0000664�0000000�0000000�00000003443�13034501105�0020424�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Cedric Defortis # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.weather import CapWeather, CityNotFound from weboob.tools.backend import Module from weboob.capabilities.base import find_object from .browser import MeteofranceBrowser __all__ = ['MeteofranceModule'] class MeteofranceModule(Module, CapWeather): NAME = 'meteofrance' MAINTAINER = u'Cedric Defortis' EMAIL = 'cedric@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Get forecasts from the MeteoFrance website' LICENSE = 'AGPLv3+' BROWSER = MeteofranceBrowser def get_current(self, city_id): return self.browser.get_current(self.get_city(city_id)) def iter_forecast(self, city_id): return self.browser.iter_forecast(self.get_city(city_id)) def iter_city_search(self, pattern): return self.browser.iter_city_search(pattern) def get_city(self, _id): cities = list(self.iter_city_search(_id)) if len(cities) == 0: raise CityNotFound() try: return find_object(cities, id=_id, error=CityNotFound) except CityNotFound: return cities[0] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000007450�13034501105�0020240�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������ # -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Cedric Defortis # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import date from weboob.browser.pages import JsonPage, HTMLPage from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.capabilities.weather import Forecast, Current, City, Temperature from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import CleanText, CleanDecimal, Regexp, Format class SearchCitiesPage(JsonPage): @method class iter_cities(DictElement): ignore_duplicate = True class item(ItemElement): klass = City def condition(self): return Dict('type')(self) == "VILLE_FRANCE" obj_id = Dict('codePostal') obj_name = Dict('slug') class WeatherPage(HTMLPage): @method class iter_forecast(ListElement): item_xpath = '//div[@class="group-days-summary"]/article' class item(ItemElement): klass = Forecast obj_id = CleanText('./header/h4') obj_date = CleanText('./header/h4') def obj_low(self): temp = CleanDecimal(Regexp(CleanText('./ul/li[@class="day-summary-temperature"]'), '(.*) / .*'))(self) unit = Regexp(CleanText('./ul/li[@class="day-summary-temperature"]'), u'.*\xb0(\w) Minimale / .*')(self) return Temperature(float(temp), unit) def obj_high(self): temp = CleanDecimal(Regexp(CleanText('./ul/li[@class="day-summary-temperature"]'), '.* / (.*)'))(self) unit = Regexp(CleanText('./ul/li[@class="day-summary-temperature"]'), u'.* / .*\xb0(\w).*')(self) return Temperature(float(temp), unit) obj_text = Format('%s - %s - %s - %s', CleanText('./ul/li[@class="day-summary-temperature"]'), CleanText('./ul/li[@class="day-summary-image"]'), CleanText('./ul/li[@class="day-summary-uv"]'), CleanText('./ul/li[@class="day-summary-wind"]')) @method class get_current(ItemElement): klass = Current obj_id = date.today() obj_date = date.today() obj_text = Format('%s - %s - %s - %s', CleanText('(//div[@class="group-days-summary"])[1]/article[1]/ul/li[@class="day-summary-temperature"]'), CleanText('(//div[@class="group-days-summary"])[1]/article[1]/ul/li[@class="day-summary-image"]'), CleanText('(//div[@class="group-days-summary"])[1]/article[1]/ul/li[@class="day-summary-uv"]'), CleanText('(//div[@class="group-days-summary"])[1]/article[1]/ul/li[@class="day-summary-wind"]')) def obj_temp(self): temp = CleanDecimal('//div[@id="detail-day-01"]/table/tr[@class="in-between"]/td[1]')(self) unit = Regexp(CleanText('//div[@id="detail-day-01"]/table/tr[@class="in-between"]/td[1]'), u'.*\xb0(\w)')(self) return Temperature(float(temp), unit) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/meteofrance/test.py��������������������������������������������������������������0000664�0000000�0000000�00000002470�13034501105�0020115�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class MeteoFranceTest(BackendTest): MODULE = 'meteofrance' def test_meteofrance(self): l = list(self.backend.iter_city_search('paris')) self.assertTrue(len(l) > 0) city = l[0] current = self.backend.get_current(city.id) self.assertTrue(current.temp.value > -20 and current.temp.value < 50) forecasts = list(self.backend.iter_forecast(city.id)) self.assertTrue(len(forecasts) > 0) forecast2 = list(self.backend.iter_forecast('blagnac')) self.assertTrue(len(forecast2) > 0) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016117�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001522�13034501105�0020230�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"init of Newspaper20minutesModule" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import Newspaper20minutesModule __all__ = ['Newspaper20minutesModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000002251�13034501105�0020154�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"browser for 20minutes website" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import ArticlePage from weboob.browser import AbstractBrowser, URL class Newspaper20minutesBrowser(AbstractBrowser): "Newspaper20minutesBrowser class" BASEURL = 'http://www.20minutes.fr' PARENT = 'genericnewspaper' article_page = URL('/.+/?.*', ArticlePage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000006127�13034501105�0020260�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD����`��� pHYs�� �� ����tIMEC���tEXtComment�Created with GIMPW�� IDATx[{L[]cl+<HKFiiTdYZu j+ݦlݴVjҦV{IAb%jy86�{~)ݑ{}\ Caw3|]e>zu� ->�u8UDk`ԩ�fUWŬ*:ކy\\P^`\.`' BwƏ)8P02vC#ݗjw� �z\�45᏿;d1ܤUL*BFȏw{{y X9cK4h[UlMC"x,0!'lεڊoY1k6GmE rނVGZ2r'a@Scʊ(+6dž^+B{><(/ْ(q @KWYmjß�|>|<*jO-k[ʸ{QVd`"E>x\$DhjCR oA8Ec�OSbfUQ*ޕzoIq=ېu8=xJ^D\L(|^HR!E <$Be*sJŎOo؄irFޮTEAV粝9p{o$49 y15=q4I/wd��D-SI %Yh>\Edi?/o~WPEQ1<ZNI' A<FM|(&Ǔ6'+-WX#w`=& �  pwm�CU`CrNEK\",=;1w ;#KjK56S\̰Sodw.x;AV$h>/�$K, d5 HgWΚvGz㇛) y52 pdlP(f$�RpvRBpan03=eƓDs j2R*@zt9UDAm 2S<4ߪ7q) APH^eYx<f� گn7H`�@Qh8]n/VmqWӪD1=wOK!_.+2zW.vl\ B|,CCp{f?r{a@gH<j#dvB`<ei2I#QU q 0?8~-& ._l otJgA,'\S|AV*E]e i\G1miޏ^<'m 1FgPW0ڀp8=$j5[p#fŕ�ʊ16iR5BG{ ƝUq4ڻ5`nl%�S?~[x坶hw'h">ʊ ˎEjw@Ts_\EbEhߝE0/c&ˊb_K@gIEK^J+nsh1YNfk�o0c賆F&q_5pK6*>"sTH9<:{l14Ev^z݃qn�:{mSQ7XU\?�?IP\49">` &4U۬uJVvBIPk�?gm(�R�J3E[f*ρQR!uȉ+ŪT,}wwD[םVy |Q!"ϣ? V U V]6<@@UyZyj^ҷ~'R!?AeYNt-/[ ZG,�E:oU<(ǘ_a�Dvp=J(b Ӌz@!{, pp_;E 3#R"MR!šNQBfcS")E|b5 ėHW&t{ֆ�T[cKZܳ9f+ebH:�WLj))&H%1Bb&Ź\-DBNRe5᭦iA IIAwNHF`lZٝ ,Oq{P_w�&C/]�~\$ 4,!4yToQFJ6D#N"ʤJE 2�$ ZM |>8-jߝ(�:)lD۔@k1 , epV̲UsAV .rq"/urw,Hnc㓛19Nz?&37TFx x!:�$F�B,r{s>2 0;E-zId yf?4Tf*ݣGi|0pjwbl³bqJ;u(�M<3*Qs{.ToC,2_LMO /P89 kdk &ɝ#6^t> B&bg%[qkt% 'p8nt]dM4' HWl+Y7N-4nL/R8Gq)b1` K1c'!!1=CScT|Saw^o{kG/h_ދlup8=a"4�" �V^cZh[⛒+h5Cw)BRE6iw#|5<3E`ԫV,&`W% LƤY?N|;7f]f@SAVr(\ґ  M °;~L8J3 Z*`W R=m[b#c굡ds n.WPl5tX3s?LV~^y #w8cw����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/module.py��������������������������������������������������������������0000664�0000000�0000000�00000002514�13034501105�0017760�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://20minutes.fr" from weboob.capabilities.messages import CapMessages from weboob.tools.backend import AbstractModule from .browser import Newspaper20minutesBrowser from .tools import rssid class Newspaper20minutesModule(AbstractModule, CapMessages): MAINTAINER = u'Julien Hebert' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'minutes20' DESCRIPTION = u'20 Minutes French newspaper website' BROWSER = Newspaper20minutesBrowser RSS_FEED = 'http://www.20minutes.fr/rss/une.xml' RSSID = rssid PARENT = 'genericnewspaper' ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/minutes20/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000003730�13034501105�0017573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������ "ArticlePage object for minutes20" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.standard import CleanText from weboob.browser.filters.html import CSS class ArticlePage(AbstractPage): "ArticlePage object for minutes20" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_load(self): self.main_div = self.doc.getroot() self.element_title_selector = "h1" self.element_author_selector = "p.author-sign, span.author-name" self.element_body_selector = "div[role=main], div.mna-body" def get_body(self): try: element_body = self.get_element_body() except AttributeError: return None else: self.try_remove(element_body, "figure") self.try_remove(element_body, self.element_author_selector) self.try_remove(element_body, "ul[class^=content-related]") self.try_remove(element_body, "*.mt2") self.try_remove(element_body, "*[class^=index]") self.try_remove(element_body, "p > a.highlight") self.try_remove(element_body, "script") self.try_remove(element_body, "blockquote") return CleanText('.')(element_body) ����������������������������������������weboob-1.2/modules/minutes20/test.py����������������������������������������������������������������0000664�0000000�0000000�00000001667�13034501105�0017462�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class Minutes20Test(BackendTest): MODULE = 'minutes20' def test_new_messages(self): for message in self.backend.iter_unread_messages(): pass �������������������������������������������������������������������������weboob-1.2/modules/minutes20/tools.py���������������������������������������������������������������0000664�0000000�0000000�00000002034�13034501105�0017630�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"common tools for 20minutes backend" # -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re def url2id(url): "return an id from an url" regexp = re.compile("http://www.20min.fr/(\w+)/([0-9]+)/(.*$)") match = regexp.match(url) return '%s.%d.%s' % (match.group(1), int(match.group(2)), match.group(3)) def rssid(entry): return url2id(entry.id) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015760�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0020073�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MonsterModule __all__ = ['MonsterModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003536�13034501105�0020024�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.browser import PagesBrowser, URL from .pages import AdvertPage, AdvSearchPage, ExpiredAdvert __all__ = ['MonsterBrowser'] class MonsterBrowser(PagesBrowser): BASEURL = 'http://www.monster.fr' advert = URL('http://offre-emploi.monster.fr/v2/job/View\?JobID=(?P<_id>.*)', AdvertPage) expired_advert = URL('http://offre-emploi.monster.fr/v2/job/Expired\?JobId=(?P<_id>.*)', ExpiredAdvert) adv_search = URL('/emploi/recherche/(?P<search>.*)&page=(?P<page>\d*)', AdvSearchPage) def search_job(self, pattern=None): return self.adv_search.go(search='?q=%s' % urllib.quote_plus(pattern), page=1).iter_job_adverts() def advanced_search_job(self, job_name, place, contract, limit_date): search = '' if not contract else contract query = {'q': urllib.quote_plus(job_name), 'where': place, 'tm': limit_date} return self.adv_search.go(search='%s?%s' % (search, urllib.urlencode(query)), page=1).iter_job_adverts() def get_job_advert(self, _id, advert): return self.advert.go(_id=_id).get_job_advert(obj=advert) ������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000040312�13034501105�0020113�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��� pHYs�� �� ���� cHRM��z%��������u0��`��:��o_F��@PIDATx�@@X*������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������!*�jP��������������������������������������������������WoB�DW3��z������������������������������������������������������������������������������������������������������������������������������������������������������������������  �wZ�$� �Pf<�J^8�������������������������������������������������Mc:�Mc:� �&�wZ� �����������������������������������������������������������������������������������������������������������������������������������������������������������������Ma:� �Ƿ������<L-�������������������������������������������������<M.�����`�{� �La9��������������������������������������������������������������������������������������������������������������������������������������������������������������@R0�����������������������������������������������������������������������AS1��������������������������  ������������������������������������������������������������������������������������������������������*5 �  �����(�������������������������������������������������������������EX4������ �(3����������������������.;"��Lc:�����������������������������������������������������°�������������������������������������  �9H+���������wY�������������������������������������������������������� �^xG�������������6F*�������������������$�� � ����������������������������������������������������������6F)������������������������� �.;$�La9�",����������������������������������������������������� '�(2� ���������������������?P/��̿��������������~Z �2A&����������&/�'3������������������������������������������x�³������������������Nb;�AS0����������������������������������������������������������H[7�J`6���������������������������������������������X*���������������������������� �@Q0�G\6� ��������������������������������������^�����������c~J�3B&��������������������������������������������h���������������FX5�Nd;� ���������������������������������������������_����������,A�������������������������������4D(����������������������������������  �������/:$�gM�*6����������������������������������������#,�ȸ�������YpC�Qi=� �������������������������������������������ȹ������,A��������������������������������������*7���������������������������������������,7"�oT��ȷ�s���/=$�������������������������������  ����rU�>P.���v���"-����������������������������������ͽ�����������������������������������������������±�Lc7� ��������������������������������� �$�`{H� ����������[uD������������������������������ ��b|J�0?$�����e����������� �8H*���������������������������������",���,A�����������������������������������������1>&���������������������������������� )�DW3� �����������������BT1� ����������������������������� �>N/�4D'������������������������J_7���������������������������������������������������������������������������������������������������������������������� ����������������������������������������������������������&2�(5������������������������������������������������������������� � ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������ʼ�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������ò�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ������������������������������������²���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������̾�������������������������������������������������������������������������������������������������������������������������������������� ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������ ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������$����������������������������������������������������������������������� �������������������������������������������������������������������������� ���������<Ʈ����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/module.py����������������������������������������������������������������0000664�0000000�0000000�00000006132�13034501105�0017621�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.job import CapJob, BaseJobAdvert from weboob.tools.value import Value from weboob.tools.ordereddict import OrderedDict from .browser import MonsterBrowser __all__ = ['MonsterModule'] class MonsterModule(Module, CapJob): NAME = 'monster' DESCRIPTION = u'monster website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = MonsterBrowser type_contrat_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ 'Interim-ou-CDD-ou-mission_8': u'Interim ou CDD ou mission', 'CDI_8': u'CDI', 'Stage-Apprentissage-Alternance_8': u'Stage/Apprentissage/Alternance', ' ': u'Autres', 'Indépendant-Freelance-Saisonnier-Franchise_8': u'Indépendant/Freelance/Saisonnier/Franchise', 'Journalier_8': u'Journalier', 'Temps-Partiel_8': u'Temps Partiel', 'Temps-Plein_8': u'Temps Plein', }.iteritems())]) date_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '-1': u'N importe quelle date', '000000': u'Aujourd hui', '1': u'2 derniers jours', '3': u'3 derniers jours', '7': u'Les 7 derniers jours', '14': u'Les 14 derniers jours', '30': u'30 derniers jours', }.iteritems())]) CONFIG = BackendConfig( Value('job_name', label='Job name', masked=False, default=''), Value('place', label='Place', masked=False, default=''), Value('contract', label=u'Contract', choices=type_contrat_choices, default=''), Value('limit_date', label=u'Date', choices=date_choices, default='-1'), ) def search_job(self, pattern=None): return self.browser.search_job(pattern) def advanced_search_job(self): return self.browser.advanced_search_job(job_name=self.config['job_name'].get(), place=self.config['place'].get(), contract=self.config['contract'].get(), limit_date=self.config['limit_date'].get()) def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000010074�13034501105�0017433�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from datetime import datetime, time, timedelta from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, Filter, Env, BrowserURL, Format, DateTime from weboob.browser.filters.html import CleanHTML from weboob.capabilities.job import BaseJobAdvert from weboob.capabilities.base import NotAvailable class MonsterDate(Filter): def filter(self, date): now = datetime.now() number = re.search("\d+", date) if number: if 'heures' in date: date = now - timedelta(hours=int(number.group(0))) return datetime.combine(date, time()) elif 'jour' in date: date = now - timedelta(days=int(number.group(0))) return datetime.combine(date, time()) else: return datetime.combine(now, time.min) class AdvSearchPage(HTMLPage): @pagination @method class iter_job_adverts(ListElement): item_xpath = '//article[@class="js_result_row"]' def next_page(self): page = Regexp(CleanText('//link[@rel="next"]/@href', default=''), '.*page=(\d*)', default=None)(self) if page: return BrowserURL('adv_search', search=Env('search'), page=int(page))(self) class item(ItemElement): def condition(self): return u'Désolé' not in CleanText('//h1')(self) klass = BaseJobAdvert obj_id = CleanText('./div[@class="jobTitle"]/h2/a/@data-m_impr_j_jobid') obj_society_name = CleanText('./div[@class="company"]/span[@itemprop="name"]', replace=[(u'Trouvée sur : ', u'')], default=NotAvailable) obj_title = CleanText('./div[@class="jobTitle"]/h2/a/span[@itemprop="title"]', default=NotAvailable) obj_publication_date = DateTime(CleanText('./div[@class="extras"]/div[@class="postedDate"]/time/@datetime'), default=NotAvailable) obj_place = CleanText('./div[@class="location"]/span[@itemprop="name"]', default=NotAvailable) class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Env('_id') obj_url = BrowserURL('advert', _id=Env('_id')) obj_title = CleanText('//h2') obj_description = Format('%s\n%s', CleanHTML('//div[@id="JobDescription"]'), CleanText('//dl')) obj_contract_type = CleanText('(//dl/dt[text()="Type de contrat"]/following-sibling::dd)[1]') obj_society_name = CleanText('//div[@data-jsux="aboutCompany"]/div/dl/dd') obj_place = CleanText('//h3') obj_publication_date = MonsterDate(CleanText('(//dl/dt[starts-with(text(),"Publi")]/following-sibling::dd)[1]')) class ExpiredAdvert(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Env('_id') obj_url = BrowserURL('expired_advert', _id=Env('_id')) obj_title = CleanText('//div[@role="alert"]') obj_description = CleanText('//div[@role="alert"]') ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/monster/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002655�13034501105�0017321�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import itertools from weboob.tools.test import BackendTest class MonsterTest(BackendTest): MODULE = 'monster' def test_monster_search(self): l = list(itertools.islice(self.backend.search_job(u'marketing'), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_monster_advanced_search(self): l = list(itertools.islice(self.backend.advanced_search_job(), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) �����������������������������������������������������������������������������������weboob-1.2/modules/myhabit/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015726�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/myhabit/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020040�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import MyHabitModule __all__ = ['MyHabitModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/myhabit/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000015425�13034501105�0017772�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from requests.exceptions import Timeout from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.pages import HTMLPage from weboob.capabilities.base import Currency from weboob.capabilities.shop import OrderNotFound, Order, Payment, Item from weboob.exceptions import BrowserIncorrectPassword from datetime import datetime from decimal import Decimal __all__ = ['MyHabit'] def cleanup(s): return u' '.join(unicode(s).split()) class MyHabitPage(HTMLPage): @property def logged(self): return bool(self.doc.xpath('//a[text()="Sign Out"]')) class LoginPage(MyHabitPage): def login(self, username, password): form = self.get_form(name='signIn') form['email'] = username form['password'] = password form.submit() return self.browser.page class HistoryPage(MyHabitPage): def is_sane(self): # Occasionally MyHabit returns a page without this form. # In this case we must retry until correct page is retrieved. return bool(self.doc.xpath('//form[@id="viewOrdersHistory"]')) def to_year(self, year): form = self.get_form(xpath='//form[@id="viewOrdersHistory"]') form['orderRange'] = year form.submit() return self.browser.page def years(self): return self.doc.xpath('//option[contains(@value,"FULL_YEAR")]/@value') def iter_orders(self): return self.doc.xpath('//a[contains(@href,"ViewOrdersDetail")]/@href') class OrderPage(MyHabitPage): def order(self, url): order = Order(id=self.order_number()) order.url = unicode(url) order.date = self.order_date() order.tax = self.tax() order.shipping = self.shipping() order.discount = self.discount() order.total = self.total() return order def payments(self): method = self.doc.xpath('//div[@class="creditCard"]/text()')[0] pmt = Payment() pmt.date = self.order_date() pmt.method = cleanup(method) pmt.amount = self.total() yield pmt def items(self): for span in self.doc.xpath('//div[@class="shipmentItems1"]' '/span[@class="item"]'): url = span.xpath('span[@class="itemLink"]/a/@href')[0] label = span.xpath('span[@class="itemLink"]/a/text()')[0] qty = span.xpath('span[@class="itemQuantity"]/text()')[0] price = span.xpath('span[@class="itemPrice"]/text()')[0] price = Decimal(qty)*AmTr.decimal_amount(price) item = Item() item.url = unicode(url) item.label = cleanup(label) item.price = price yield item def order_date(self): date = self.doc.xpath(u'//span[text()="Order Placed:"]' '/following-sibling::span[1]/text()')[0].strip() return datetime.strptime(date, '%b %d, %Y') def order_number(self): return self.doc.xpath(u'//span[text()="MyHabit Order Number:"]' '/following-sibling::span[1]/text()')[0].strip() def order_amount(self, which): return AmTr.decimal_amount((self.doc.xpath( '//tr[@class="%s"]/td[2]/text()' % which) or ['0'])[0]) def tax(self): return self.order_amount('tax') def shipping(self): return self.order_amount('shippingCharge') def discount(self): TAGS = ['discount', 'gc'] return sum(self.order_amount(t) for t in TAGS) def total(self): return self.order_amount('total') class MyHabit(LoginBrowser): BASEURL = 'https://www.myhabit.com' MAX_RETRIES = 10 login = URL(r'/signin', r'https://www.amazon.com/ap/signin.*$', LoginPage) order = URL(r'/vieworders\?.*appAction=ViewOrdersDetail.*', OrderPage) history = URL(r'/vieworders$', r'/vieworders\?.*appAction=ViewOrdersHistory.*', HistoryPage) unknown = URL(r'/.*$', r'http://www.myhabit.com/.*$', MyHabitPage) def get_currency(self): # MyHabit uses only U.S. dollars. return Currency.get_currency(u'$') @need_login def get_order(self, id_): # MyHabit requires dynamically generated token each time you # request order details, which makes it problematic to get an order # by id. Hence this slow and painful stub. for year in self.to_history().years(): hist = self.to_history().to_year(year) for url in hist.iter_orders(): if id_ in url: self.location(url) assert self.order.is_here() o = self.page.order(url) if o.id == id_: return o raise OrderNotFound() @need_login def iter_orders(self): for year in self.to_history().years(): hist = self.to_history().to_year(year) for url in hist.iter_orders(): self.location(url) assert self.order.is_here() yield self.page.order(url) @need_login def iter_payments(self, order): if self.url != self.BASEURL+order.url: self.location(order.url) assert self.order.is_here() return self.page.payments() @need_login def iter_items(self, order): if self.url != self.BASEURL+order.url: self.location(order.url) assert self.order.is_here() return self.page.items() @need_login def to_history(self): for i in xrange(self.MAX_RETRIES): if self.history.is_here() and self.page.is_sane(): return self.page self.history.go() def do_login(self): if not self.login.go().login(self.username, self.password).logged: raise BrowserIncorrectPassword() def location(self, *args, **kwargs): for i in xrange(self.MAX_RETRIES): try: return super(MyHabit, self).location(*args, **kwargs) except Timeout as e: pass raise e �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/myhabit/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000007206�13034501105�0020066�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���u:���gAMA�� a��� cHRM��z&���������u0��`��:��pQ<���bKGD X���tIME9#k;|^�� IDATxy\TW6 M75Ђ"AMh4(DPh&DPQHq f̸ &D}"@7}ɣG4uԩss ?v0� �K *4)Lnt2 ^*RH!ձ$ `Ѓt / $0VC@`CHы^hjD���(A $,XCM%|hy@O+ 2!sƀ1b*bv7ی5X5hCڴ>Z1A D+ʵr-,303D+aKKW4'q!bh[6 !t2ܣQ%}�#E. 1;|KOctA_$`9$$�MhB#=`A|JԎAN&SI$.{\)'jp.ԙ [GgDDDDGQdX?LD"Ȉܢy:阎:؀M|}?qҐ9KuBl˟ꇓ8s`40 L޸qnn憤I.sz444NW+ŕW>y*TOOOfMRSSyQi23Gfz{ziɈg-d!{؅=&bfD_",rB<g3$N0TdońX6w&.コ%. ٸ($nJ썸obmbKձfޛu42g:6m~[z?Z1lX1ŋ6mM[4gŒw/Z>*yWWعNu.v2;J]+gcFXFXFq+VĭgyֶֶZkڷok7wSZc1M49,,?(ߚ)U6'5:KSt"d qnqa;a<irX ەvgm iTsO_؟+9j^]ؕPRQԨjh4ٵ}nyxz#|;f:OoZ-/mְzZMz/櫓"qZmFcfrW%>38'@oH7{4{4{t*;ՁWZrj̧O3[ZZ\$.ㆣϏ>?RJ)/// 4[lb!! [S2go̱{] -l﫷^mvdiӡvC&X.;>oaefWn=y2ѧF9F_e9E]OLhؼ۱^u̜,3w~'RCy?5׹2 L8Q u:(Iʤ7oO:<1c===&M>1.c\8"*B*PmmmQR[R[R+--}R 8 =ŖD-7/O ɷG M.toP}l|îNZVٯ!to'~Kc,6[L o^T( Mͤ]&͊)ӤKbJbwqmm8zK_xb xx8SsLO*3:Iѐ@-R69C%#:f[TI=(lم.t kš&FFFY,YLQ(PxN9!~_}e.], - - Hs9TM&o Y,H&[([([ذagΈֈֈ־6RE*ȣfU妔5_׼S~t]M}@~]qm@IY\=H-}.:,l}ۑeUn[ڬvU;\89? nb@yTWzxӕq~_lcvq;kѬ#6dc6{?}6':Ex *{+?{p9�@ododo+WүwwwO.>ҾK.9>v|xR˥=Y=Y=Y?\r9qNcrcrcg$IpM&TIZJFЩ$"R`HC-�L CZ(r 5M%to0]'. ڧMgfWŒ@)w/y'۷'1c0FE jTuU|ſ̢mf+GYJv}qzq`�cOQzƗ[^oD E(B9Qb#ؘTTTq3K%Ow/y]οX&Ꮷl [p<J%ߕߕG x/7Ƌ 6\aa춑r�b2j/sԞd4l^7ꄡ Oz߱mid9ۼjMj^i*{=˞ pþú4ۆߝ3q;B? TrfؽKݙh2IPIK#F߉NjCk5^bB K$ $)m,a7o@h.zBiP]Lv1-?0?0?طط(@C1Ǹ:{;{;=h:W+lc^c^c?z^JPŠi]fNna6t ]Y*Ӕծ},{TYq7QxNV]ׅs]v'/TZKnfS>jzUĄ|FNuVumuԇwEwSAeoNM!'wS];6~\Ӕ:8ïd-YO(D!Wc5V!6 0Ή Ÿ7'NLB qqH( '| P9E$r8v?^~xpEn[ f.3KGOD"}FG"n. r + Z!A$ 2 >} mG5!w0L$2dL\kMq 1i<`QkbJ#FDP\ :%1>f9*WP뼦o! a'Ċ 1V'74vE@ (`smF`Fp4 |)'T U|o4΋k=c<#hF3^׺,`7ڸ"{=;Ё_:(%G�m:'9,`wo�9A3GZqx���%tEXtdate:create�2015-02-23T22:57:35-06:00ؓ���%tEXtdate:modify�2015-02-23T22:57:35-06:00`/����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/myhabit/module.py����������������������������������������������������������������0000664�0000000�0000000�00000003531�13034501105�0017567�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.shop import CapShop from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import MyHabit __all__ = ['MyHabitModule'] class MyHabitModule(Module, CapShop): NAME = 'myhabit' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'MYHABIT' CONFIG = BackendConfig( ValueBackendPassword('email', label='E-mail', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = MyHabit def create_default_browser(self): return self.create_browser(self.config['email'].get(), self.config['password'].get()) def get_currency(self): return self.browser.get_currency() def get_order(self, id_): return self.browser.get_order(id_) def iter_orders(self): return self.browser.iter_orders() def iter_payments(self, order): return self.browser.iter_payments(order) def iter_items(self, order): return self.browser.iter_items(order) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/myhabit/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002150�13034501105�0017255�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class MyHabitTest(BackendTest): MODULE = 'myhabit' def test_history(self): """ Test that at least one item was ordered in the whole history. """ b = self.backend items = (i for o in b.iter_orders() for i in b.iter_items(o)) item = next(items, None) self.assertNotEqual(item, None) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/n26/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0014676�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/n26/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001445�13034501105�0017013�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Benjamin Bouvier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import Number26Module __all__ = ['Number26Module'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/n26/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000011356�13034501105�0016741�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Benjamin Bouvier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from decimal import Decimal from datetime import datetime from weboob.browser.browsers import DomainBrowser from weboob.capabilities.base import find_object from weboob.capabilities.bank import Account, Transaction, AccountNotFound from weboob.browser.filters.standard import CleanText # Do not use an APIBrowser since APIBrowser sends all its requests bodies as # JSON, although N26 only accepts urlencoded format. class Number26Browser(DomainBrowser): BASEURL = 'https://api.tech26.de' # Password encoded in base64 for the initial basic-auth scheme used to # get an access token. INITIAL_TOKEN = 'bXktdHJ1c3RlZC13ZHBDbGllbnQ6c2VjcmV0' def request(self, *args, **kwargs): """ Makes it more convenient to add the bearer token and convert the result body back to JSON. """ kwargs.setdefault('headers', {})['Authorization'] = self.auth_method + ' ' + self.bearer return self.open(*args, **kwargs).json() def __init__(self, username, password, *args, **kwargs): super(Number26Browser, self).__init__(*args, **kwargs) data = { 'username': username, 'password': password, 'grant_type': 'password' } self.auth_method = 'Basic' self.bearer = Number26Browser.INITIAL_TOKEN result = self.request('/oauth/token', data=data, method="POST") self.auth_method = 'bearer' self.bearer = result['access_token'] def get_accounts(self): account = self.request('/api/accounts') a = Account() # Number26 only provides a checking account (as of sept 19th 2016). a.type = Account.TYPE_CHECKING a.label = u'Checking account' a.id = account["id"] a.balance = Decimal(str(account["availableBalance"])) a.iban = account["iban"] return [a] def get_account(self, _id): return find_object(self.get_accounts(), id=_id, error=AccountNotFound) def get_categories(self): """ Generates a map of categoryId -> categoryName, for fast lookup when fetching transactions. """ categories = self.request('/api/smrt/categories') cmap = {} for c in categories: cmap[c["id"]] = c["name"] return cmap @staticmethod def is_past_transaction(t): return "userAccepted" in t or "confirmed" in t def get_transactions(self, categories): return self._internal_get_transactions(categories, Number26Browser.is_past_transaction) def get_coming(self, categories): filter = lambda x: not Number26Browser.is_past_transaction(x) return self._internal_get_transactions(categories, filter) def _internal_get_transactions(self, categories, filter_func): transactions = self.request('/api/smrt/transactions?limit=1000') for t in transactions: if not filter_func(t): continue new = Transaction() new.date = datetime.fromtimestamp(t["visibleTS"] / 1000) new.rdate = datetime.fromtimestamp(t["createdTS"] / 1000) new.id = t['id'] new.amount = Decimal(str(t["amount"])) if "merchantName" in t: new.raw = new.label = t["merchantName"] elif "partnerName" in t: new.raw = CleanText().filter(t["referenceText"]) if "referenceText" in t else CleanText().filter(t["partnerName"]) new.label = t["partnerName"] else: raise "unknown transaction label" if "originalCurrency" in t: new.original_currency = t["originalCurrency"] if "originalAmount"in t: new.original_amount = Decimal(str(t["originalAmount"])) if t["type"] == 'PT': new.type = Transaction.TYPE_CARD elif t["type"] == 'CT': new.type = Transaction.TYPE_TRANSFER if t["category"] in categories: new.category = categories[t["category"]] yield new ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/n26/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000000663�13034501105�0017036�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME ���iTXtComment�����Created with GIMPd.e��IDATx1jQECBll!+VE ;{{w2"=$i&03o{=|\~xx ""rLw 5S ̃3TuFMdL+X=eָl'`90RQw W`s`Tbn.l!9|ŒXO0ni@!7 j H6$bbh""ڠ6 jڠ6 jڠ6("Eء(n):qSMQPDD[~����IENDB`�����������������������������������������������������������������������������weboob-1.2/modules/n26/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000004244�13034501105�0016541�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Benjamin Bouvier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from .browser import Number26Browser __all__ = ['Number26Module'] class Number26Module(Module, CapBank): NAME = 'n26' DESCRIPTION = u'Bank N26' MAINTAINER = u'Benjamin Bouvier' EMAIL = 'public+weboob@benj.me' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = Number26Browser CONFIG = BackendConfig( Value('login', label='Email', regexp='.+'), ValueBackendPassword('password', label='Password') ) STORAGE = {'categories': {}} def get_categories(self): categories = self.storage.get("categories", None) if categories is None: categories = self.browser.get_categories() self.storage.set("categories", categories) return categories def create_default_browser(self): return Number26Browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts() def get_account(self, id): return self.browser.get_account(id) def iter_history(self, account): categories = self.get_categories() return self.browser.get_transactions(categories) def iter_coming(self, account): categories = self.get_categories() return self.browser.get_coming(categories) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/n26/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001510�13034501105�0016224�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Benjamin Bouvier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class Number26Test(BackendTest): MODULE = 'number26' ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016241�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000000103�13034501105�0020344�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import NectarineModule __all__ = ['NectarineModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000003164�13034501105�0020302�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from .pages import LivePage, StreamsPage __all__ = ['NectarineBrowser'] class NectarineBrowser(Browser): DOMAIN = 'www.scenemusic.net' PROTOCOL = 'https' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] PAGES = { 'https://www\.scenemusic\.net/demovibes/xml/streams/': StreamsPage, 'https://www\.scenemusic\.net/demovibes/xml/queue/': LivePage } def home(self): self.location('/demovibes/xml/streams/') assert self.is_on_page(StreamsPage) def iter_radios_list(self): self.location('/demovibes/xml/streams/') assert self.is_on_page(StreamsPage) return self.page.iter_radios_list() def get_current_emission(self): self.location('/demovibes/xml/queue/') assert self.is_on_page(LivePage) return self.page.get_current_emission() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000003521�13034501105�0020375�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���tEXtSoftware�Adobe ImageReadyqe<��IDATxYlTUϝ;iC;B1*BR*  DƄ(<HD .A$B*cUZ^ذڙ2YlwfN4LL}]_s; h.09٦E]jǴ'}՜z~#8{%�k=C1 +,sȑqv< m6vpw4<:Vyi0qί zm!s(8K*@![`h}Ⱦ]\h:ygKmoFy)鹏҄03'٧+88# ,`; r-*˜yKy3&sL|ωwyo,d +Ć5WS[d!�@R@%Ea,H0E8Nv:&+~x8&\fsS'>k0#xUZ@Z`� *p^x1]ٳo+55O;#D1'3;$& Y%p3D=2ZMH,-0``6tU ܈p;�r/e^:qQ.0d!*Zbhm磪fm^O316S Q?K Y йͺ3q< ݘw:- ڵk Q6پlU\\ n<\x#puSnsΙ_\~0[cy`dB)K(d˗/D9MXkŋ'ORZ`h5B |I̬*/6~$- -`YHFfw 7L kh&ϚF[|9Fw=^v-x,MΣࠠx>ąNA ,lKͺ".Bs g餭K_&'+++m>jj5;hfi ̟*,ͿߨtA$[ ` eI�ٜ?Uy$^C%EeHZ՛]'[1tpS#4g?&בb_C6vӧa`X黉KR2�Wχc#Ăq4M#=xP:4]myVX5BGKϴ;zK 46iW![(,L?.7+-0;BC͙IK M1UrtuZYBZ5)⩿X w]9ޡuMa76 PO8n㼭ֳS*@`� + &OIoEWGHJS4F=g!Mf!c_B%z /ފwQ٪��w+EZVRojq,0?߄=9ysH=lh{Rbcl2@+#IZhް. lOAyf1\gǺƺ_\Gd�pWjW5s>avF>¦P:MpN!BIk0}B3yՌ/Q.lIE�*6a(jGmi1h39з5ˈMhv苰M F`7A\] ,ـwXoX9=>ivmr%Fq% p; 5^<t_ hnȆ5,$vZt:,Q]R-NnO�8ԿU>,X@m{LI��{#�\甌����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004417�13034501105�0020106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from .browser import NectarineBrowser __all__ = ['NectarineModule'] class NectarineModule(Module, CapRadio, CapCollection): NAME = 'nectarine' MAINTAINER = u'Thomas Lecavelier' EMAIL = 'thomas-weboob@lecavelier.name' VERSION = '1.2' DESCRIPTION = u'Nectarine Demoscene Radio' # License of your module LICENSE = 'AGPLv3+' BROWSER = NectarineBrowser def iter_resources(self, objs, split_path): if Radio in objs: self._restrict_level(split_path) for radio in self.browser.iter_radios_list(): self.browser.get_current_emission() radio.current = self.browser.get_current_emission() yield radio def iter_radios_search(self, pattern): for radio in self.browser.iter_radios_list(): if pattern.lower() in radio.title.lower() or pattern.lower() in radio.description.lower(): self.browser.get_current_emission() radio.current = self.browser.get_current_emission() yield radio def get_radio(self, radio): if not isinstance(radio, Radio): for rad in self.browser.iter_radios_list(): if rad.id == radio: return rad return None def fill_radio(self, radio, fields): if 'current' in fields: return self.get_radio(radio.id) return radio OBJECTS = {Radio: fill_radio} �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000004021�13034501105�0017707�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page from weboob.capabilities.radio import Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo class StreamsPage(Page): def iter_radios_list(self): radio = Radio('necta') radio.title = u'Nectarine' radio.description = u'Nectarine Demoscene Radio' radio.streams = [] index = -1 for el in self.document.xpath('//stream'): index += 1 stream_url = unicode(el.findtext('url')) bitrate = el.findtext('bitrate') encode = unicode(el.findtext('type')) country = unicode(el.findtext('country')).upper() stream = BaseAudioStream(index) stream.bitrate = int(bitrate) stream.format = encode stream.title = ' '.join([radio.title, country, encode, unicode(bitrate), 'kbps']) stream.url = stream_url radio.streams.append(stream) yield radio class LivePage(Page): def get_current_emission(self): current = StreamInfo(0) current.who = unicode(self.document.xpath('//playlist/now/entry/artist')[0].text) current.what = unicode(self.document.xpath('//playlist/now/entry/song')[0].text) return current ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nectarine/test.py����������������������������������������������������������������0000664�0000000�0000000�00000001677�13034501105�0017605�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class NectarineTest(BackendTest): MODULE = 'nectarine' def test_nectarine(self): l = list(self.backend.iter_radios_search('')) self.assertTrue(len(l) > 0) �����������������������������������������������������������������weboob-1.2/modules/nettokom/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016131�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001436�13034501105�0020246�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NettoKomModule __all__ = ['NettoKomModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000006531�13034501105�0020173�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import HomePage, LoginPage, HistoryPage, DetailsPage, BillsPage __all__ = ['Nettokom'] class Nettokom(Browser): DOMAIN = 'konto.nettokom.de' PROTOCOL = 'https' ENCODING = None # refer to the HTML encoding PAGES = {'.*login.html.*': LoginPage, '.*start.html': HomePage, '.*guthabenverbrauch.html': DetailsPage, '.*/verbindungsnachweis/.*': HistoryPage, '.*verbindungsnachweis.html': BillsPage } def __init__(self, *args, **kwargs): Browser.__init__(self, *args, **kwargs) def home(self): self.location('/start.html') def is_logged(self): return not self.is_on_page(LoginPage) def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) assert self.username.isdigit() if not self.is_on_page(LoginPage): self.location('/login.html') self.page.login(self.username, self.password) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() def get_subscription_list(self): if not self.is_on_page(HomePage): self.location('/start.html') return self.page.get_list() def get_subscription(self, id): assert isinstance(id, basestring) if not self.is_on_page(HomePage): self.location('/start.html') l = self.page.get_list() for a in l: if a.id == id: return a return None def get_history(self): if not self.is_on_page(HistoryPage): self.location('/verbindungsnachweis/alle-verbindungen.html') return self.page.get_calls() def get_details(self): if not self.is_on_page(DetailsPage): self.location('/guthabenverbrauch.html') return self.page.get_details() def iter_documents(self, parentid): if not self.is_on_page(BillsPage): self.location('/verbindungsnachweis.html') return self.page.date_bills() def get_document(self, id): assert isinstance(id, basestring) if not self.is_on_page(BillsPage): self.location('/verbindungsnachweis.html') l = self.page.date_bills() for a in l: if a.id == id: return a # Todo : url depends of file format # def download_document(self, id): # assert isinstance(id, basestring) # date = id.split('.')[1] # # return self.readurl('/moncompte/ajax.php?page=facture&mode=html&date=' + date) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000012250�13034501105�0020264�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME  f/���iTXtComment�����Created with GIMPd.e�� IDATxrǕYՍ}ŤЖMk"lnL؏yy%|111cK־q)RA 66FwUe@þFTvWV<?g*^ @D7m խT cfDFGk9<eh_ 6g\$"c*4E$tIq /_!{sb&+мEgǟį1G[_DwXX$: 31})Bo|bfye[Z%{/760#C6MY3ZFƎ+'ڿn>2=謧2dx{$-*_<O�Ww6_B~KhY U"VB Z/ +Їw�̟T;R ?C/Gd ;D,8B 2 ōU<Ėu�hVuu\AiH^LY(q߂uk#óT^sѠg}7o`&Yc�Bq Ywz-[B3ŌAwP $�>*d/P7?#9"R><6puľ 7` 9 ڊ0vȯQ<�8Sm4F`F> 9!}E1 Â_Vp-_=sZ^=hQۛ77w�!}`ɖU7B&*{;9R�T"&N{@f I'@{.V;|*r|my3ؓt 7@[tt�ޤ V)0H%E @+�{gAFW/})~}WMts s̱Ʊ WDHehW8Z�Z ne |3 f�~Cb(@U9@(E~ne\^zT7u{CҀ7fFLx1 Uf| :naYxY0c2H^Cve;_㖖T0+wo >C&¸2Y^�dh35$ɓk5{%dpd\n`Ac&B qZu7S'v˛`c1� 韕n-)dv *O�hmR,bf iiB�Y{1M+nik@wp_?XL� 1q\݊PK)2R sF]f W6u H+a3$3GBW'f?6?XĞ-HEaKunMq[ f<z ( Vol;u0*r OA rA7 ^~&~v>ܧ%U8@h 2>D䓣g@WWq++hkC~N&삤&d@~NR3Ԡ¦ȡhWwKup $Gπ,ح@ ۓ`C$| oAr.kd^įmRwhQ %'choF�̱y<R=�.utcIsL 7vF(J/qi\%)On(pp6Ǟ(|>Fb@n,6A>9} գcU* 3MJ)>pAW&2 K!RA~:fCd")+= VnFY$}.d W[LP][�mwΟT;HӰCTOv-aJ@b& TL h[@j;0ۇ50�-f ّܰ!v3ڤ~OqK}W⑑y2@i:M$ØQ:{ %m4k`} @P|h^htqCB6n>k7-֢&j 7 fTz}=Am&T!* hiE3A }sBYܚ21=,0sBzAH_ p KPܼ[62V !� c ]<LL"3P<I�짴vn'IPؖZіP1 nUTe%.YЭ&9}St}ݮ!y`SR2ڎ|C*ǰO@F0�ESјv�8 ?�.Bq ܂Y bVs [nzwޯꊺ\}fzkY� 2�_Þ< UỀ GC/ӇMZ/ \'>L$DEĂ*0 YHF}Ϯ_ 3wu"s` [E#i4/.h^On ~tݡPCҾvH_R| nLL_.aC TWoD PHΝ#9d|l0ovI c% \ �-pka%:[NmhIE(C~ߣu?ojobO K~dn%h�m}fʏ~H0=n'/6f*(r n)SxZʀv NG; -qt'X`c߈Lj]RASCu i鰉|KTn-[_R\D�*McD7H ?Č7_30'1Rc>J�Wd dB�~ۂ5MZTG4Je@VDR_ DDPؠ[Z@f.]Xcx֧M%}P\Pf@%eˊ't[`XQ/սz&`![uW+WRpԓm,w+9B26vNQ헢ҳt7^W`oA,m19 Ak8"U/;l /AXL'PyE dՂ$yn`#gKijf AF|&:q;ux2@-( ˂=^)Iv(3pvjQo4?yiOrU9ݬe4 Cr<"Cڟ96{m6m UZhn@n`)YRJǠOIY:�{,`J$̼ӞRCmF|}=`NI-9yUM%)FܩK�qHiA&>;<!_JUL&%B5jهAO$g}DRf*X{<&Q@m)nSBQ $B#%߯L:h}: u)0}}j-R|w`{XiA� �kV@sAÖ خ3'�O{্સgikDt'Ax(gM+O9`ă\fTI׆K^B'�M(] ;ѳ?̜%w`�rpK! X`Nv@w* L e9hO]$Ц\8Oiwϐt`nߨb{_T�FJiiC–A׀\ zԡ!y) 9*ɻفVMmD2VLMF#Pڈ5*`kϊ4 c�ւnAqK0' { bK&+r}UDt3 @kI}!" Z ?HOi-M$&yK4""\z Zej=!gLKFpX=pGzo akx_m)/(h;L^`ΕV @9Zn!>azS?M]b&υA7HYq tK a<n�tpA.UAFt %P %uX:;9ww#Ӄnfoj~#LƳ �c<t <\H܋ݳg_uz"{5R^g\zh"P*vN٢P{]Ad@+s 8G| Lf3smE*�)0 M!L2%ٿ*>jB~I/KWYWZ[ v>g @j j>mϋ1(kctB~Q(bHlmJmL { DĀ�n J҆p7n0UF0| ١Io ~y0(IE˘>�{_(3>d+EYݒG d2 qŞ*> uR;P'yU=`&7QZJ#Ԙ6- jx: $RCAbOx*1U?o܂PVK_*1@w10d䡭`2v G%tWA*=_/6B>Sq>򋠻Jr l!� f@7cA 2?KoGHn Y󏂺"y Gߘpē"_|S1>3(p\XgmIE>[~U3'̄7_B䬒 c@}_o @[pp})n s^9;ol YM• ݝpR#8 y�ܦ\,u@g`gDT!=jrV"p "IpsAKZr㙒=Qʛ��k~%N02??}%ٯ,̀NI<{@|=@=:�quWڔ{٧>;eRRU~Izm�.tc7�2\Zn$ 6�qydçnLwVݗ ����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/module.py���������������������������������������������������������������0000664�0000000�0000000�00000005657�13034501105�0020005�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, SubscriptionNotFound, Detail from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Nettokom __all__ = ['NettoKomModule'] class NettoKomModule(Module, CapDocument): NAME = 'nettokom' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Nettokom website' CONFIG = BackendConfig(ValueBackendPassword('login', label='Account ID (phone number)', masked=False, regexp='^(\d{8,13}|)$'), ValueBackendPassword('password', label='Password') ) BROWSER = Nettokom def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): for subscription in self.browser.get_subscription_list(): yield subscription def get_subscription(self, _id): with self.browser: subscription = self.browser.get_subscription(_id) if subscription: return subscription else: raise SubscriptionNotFound() def iter_documents_history(self, subscription): with self.browser: for history in self.browser.get_history(): yield history # The subscription is actually useless, but maybe for the futur... def get_details(self, subscription): with self.browser: for detail in self.browser.get_details(): yield detail def get_balance(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) balance = Detail() balance.id = "%s-balance" % subscription.id balance.price = subscription._balance balance.label = u"Balance %s" % subscription.id balance.currency = u'EUR' return balance ���������������������������������������������������������������������������������weboob-1.2/modules/nettokom/pages/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017230�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/pages/__init__.py�������������������������������������������������������0000664�0000000�0000000�00000001643�13034501105�0021345�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .homepage import HomePage from .history import HistoryPage, DetailsPage, BillsPage from .login import LoginPage __all__ = ['LoginPage', 'HomePage', 'HistoryPage', 'DetailsPage', 'BillsPage'] ���������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/pages/history.py��������������������������������������������������������0000664�0000000�0000000�00000005700�13034501105�0021305�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime, date, time from decimal import Decimal from weboob.deprecated.browser import Page from weboob.capabilities.bill import Detail class DetailsPage(Page): def on_loaded(self): self.details = [] table = self.document.xpath('//table[@id="reportTable"]') if len(table) > 0: for tr in table[0].xpath('tbody/tr'): detail = Detail() # Skip global category if tr.find('td/a') is not None: continue if tr.attrib["class"] == "totalAmount": continue tds = tr.xpath('td') detail.label = unicode(tds[0].text.strip()) detail.infos = unicode(tds[1].text.strip()) detail.price = Decimal(tds[2].text.split(' ')[0].replace(',', '.')) self.details.append(detail) def get_details(self): return self.details def _get_date(detail): return detail.datetime class BillsPage(Page): def on_loaded(self): pass class HistoryPage(Page): def on_loaded(self): self.calls = [] for tr in self.document.xpath('//tr'): try: attrib = tr.attrib["class"] except: continue if attrib == "even" or attrib == "odd": label = u'' tddate = tr.find('td[@class="middle nowrap"]') for td in tr.xpath('td[@class="long"]'): label += unicode(td.text.strip()) + u' ' tdprice = tr.xpath('td[@class="price"]') label += u'(' + unicode(tdprice[0].text.strip()) + u')' price = Decimal(tdprice[1].text.strip().replace(',', '.')) detail = Detail() mydate = date(*reversed([int(x) for x in tddate.text.strip().split(' ')[0].split(".")])) mytime = time(*[int(x) for x in tddate.text.strip().split(' ')[1].split(":")]) detail.datetime = datetime.combine(mydate, mytime) detail.label = label detail.price = price self.calls.append(detail) def get_calls(self): return sorted(self.calls, key=_get_date, reverse=True) ����������������������������������������������������������������weboob-1.2/modules/nettokom/pages/homepage.py�������������������������������������������������������0000664�0000000�0000000�00000004234�13034501105�0021372�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import Subscription from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.deprecated.browser import Page from datetime import date from decimal import Decimal class HomePage(Page): def on_loaded(self): pass def get_list(self): l = [] divabo = self.document.xpath('//div[@id="accountSummary"]')[0] owner = divabo.xpath('a/h3')[0].text phone = divabo.xpath('dl/dd')[0].text credit = divabo.xpath('dl/dd')[1].text expiredate = divabo.xpath('dl/dd')[2].text phoneplan = divabo.xpath('dl/dd')[3].text self.browser.logger.debug('Found ' + owner + ' as subscriber') self.browser.logger.debug('Found ' + phone + ' as phone number') self.browser.logger.debug('Found ' + credit + ' as available credit') self.browser.logger.debug('Found ' + expiredate + ' as expire date ') self.browser.logger.debug('Found %s as subscription type', phoneplan) subscription = Subscription(phone) subscription.label = unicode(u'%s - %s - %s - %s' % (phone, credit, phoneplan, expiredate)) subscription.subscriber = unicode(owner) expiredate = date(*reversed([int(x) for x in expiredate.split(".")])) subscription.validity = expiredate subscription._balance = Decimal(FrenchTransaction.clean_amount(credit)) l.append(subscription) return l ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/pages/login.py����������������������������������������������������������0000664�0000000�0000000�00000002152�13034501105�0020712�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page class LoginPage(Page): def on_loaded(self): pass def login(self, login, password): self.browser.select_form(nr=0) self.browser.set_all_readonly(False) self.browser['number'] = login.encode('iso-8859-1') self.browser['password'] = password.encode('iso-8859-1') self.browser.submit(nologin=True) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nettokom/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002714�13034501105�0017466�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class NettokomTest(BackendTest): MODULE = 'nettokom' def test_details(self): for subscription in self.backend.iter_subscription(): details = list(self.backend.get_details(subscription)) self.assertTrue(len(details) > 1, msg="Not enough details") def test_history(self): for subscription in self.backend.iter_subscription(): self.assertTrue(len(list(self.backend.iter_documents_history(subscription))) > 0) def test_list(self): """ Test listing of subscriptions. """ subscriptions = list(self.backend.iter_subscription()) self.assertTrue(len(subscriptions) > 0, msg="Account listing failed") ����������������������������������������������������weboob-1.2/modules/newsfeed/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016071�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/newsfeed/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0020207�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewsfeedModule __all__ = ['NewsfeedModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/newsfeed/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000006625�13034501105�0020235�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ~���tIME  8(EH�� 'IDATxݛ{p\}?{W]OKe ۘ &dR -e:IKi-uȐ$iƱM@Rla6`[׻{9Ck{WFwFwwν}M;@B !Bp6mZkR�@;IH㥶gkZSPȌTV*)! aBʳ�J6x(h'v46m te]k[Lw<�Z=R) T65.+pIqT: m; @@ϳ#Y'^2 3C@?(ةT˟x�;4gwU65Q؀v\Cb@ș'rroc4A\fGifcj{MgaհZRC$pSIČ<'N Py-fA@ćGnᅮ)iG֮VeeĻIGѨs~:> !bbҺZ/h?#Դjj^hl$ݏ=z$0|*+Jݷ~~ʉF"ё^+*T'ni X5I~@켧z ZH1Gi*V4h%pj{8phFhug 2Ceex‡oYTA'FQ>Xx/B~B+855D;;T*iXY\U JMWfم=;xgN8ބЬ(5ZP EeF3`I N"P b.X`)�ݱ =M,)`D@(;)t|~N_uK1Š/NVpBZSa;MH0,C_w?;qv R^vl +VjB DkcJW%xFkVÅܗm71(,> zI=SgEy lb~ QTu'OoN�#rcaDa"X,D j##< ̰[%n9 !+0j/Ƭ`iE)-qww@9?uL0.:MDqڱ0uK1/\y*DlV˾�Շ&3k=/ŀ4n[?N~ԦQ}3/QE+V>5TiV(V~&CY3Ƨ@7֙BfĀ3ViTO;闾:t"" rDHq w_-DQ i+MNQh7j nnAPv~>y+F(iĺ^펟>ʙgO{QX,<cA؂:pVunV7-_<< 6wxu{]P7Qpz ׼uאj_yKّ;~sO "hKZP8[ u/+ȪKN~J f>~X9Fg-o4GPG~׸ց3 \YGvWy`8"X[:6/ס \Gsy<*�Q,q?k1 Y4gA03/ *斷QR' 7}uZ)(ZW!"GQ=o@|�ЈRs\*rp 5t(S\t ;ww@Zf$Hwwj0SG^Ge(,Xt5ƒȆ `xC{#Kmc/>6S[ 1kձ wV= -9b|=/l\?|_Уq=}}!Pu#`[w�?&t t]qKw7 X+<O4:1-.^+248+6n ?w 1g*1F~N&mJ<kߘ2I VVyl߈N=^B ޜqv|*[Zo*o([gQb̚Ox�ӄ2Vȡ]_\q#MP-d/o-D.Y(]J;^F{mzg*<zdܳUl:Eަ;ҍQVۀ]7W X3 !ADꐋV`z cUlH̸ٵR}¥+~f Rb޸ 2ͷ`\՜ LMoam"e}{{A UqNxAo7A6^. %{uHx'YX^UۧFEpInxrU<|c7GE+svo^A'cMmY w< ,[HuS l]۽ 2Vݽ'6=z50dwo;Iֵ}ޓ(RTgv!oTϩ(D|7o{=Α]K敋j{g;VM^KWᩎ:9ͳuƆc eY;6裇$2X ?&O>i!|孎/KZMrˣd̻oظc}>KbeY Ę"R!0R|zEt*q!BW#pߢŸw&J'' 3 �1'J!ձumຸqvT+6=[pNtp:YQĶ;yw=e%Ac;Z^_suK?#NޯAPwzljoI!hJox bv"]jN`Edby/ Iqs5hUY.vKH1ىR TF1 '&Qt28|">T[B(=$'vtu뱾>]|Eu vvrbh8-L˟<zf<6o,>14tPn*JZdX0׶=©-1,;{l`oC QҌ8NS.T[Lqs5Z(y,?YqdܡW PQv سNbw3<7*;eFg Nf78ÅJ(FzzF9!!mo?ʱŅ TzKaOCu=<II$FGI�A 0m?n*UmW"ĕl܄8b 6_Tg˯[U����IENDB`�����������������������������������������������������������������������������������������������������������weboob-1.2/modules/newsfeed/module.py���������������������������������������������������������������0000664�0000000�0000000�00000006076�13034501105�0017741�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.messages import CapMessages, Message, Thread from weboob.tools.newsfeed import Newsfeed from weboob.tools.value import Value __all__ = ['NewsfeedModule'] class NewsfeedModule(Module, CapMessages): NAME = 'newsfeed' MAINTAINER = u'Clément Schreiner' EMAIL = "clemux@clemux.info" VERSION = '1.2' DESCRIPTION = "Loads RSS and Atom feeds from any website" LICENSE = "AGPLv3+" CONFIG = BackendConfig(Value('url', label="Atom/RSS feed's url", regexp='https?://.*')) STORAGE = {'seen': []} def iter_threads(self): for article in Newsfeed(self.config['url'].get()).iter_entries(): yield self.get_thread(article.id, article) def get_thread(self, id, entry=None): if isinstance(id, Thread): thread = id id = thread.id else: thread = Thread(id) if entry is None: entry = Newsfeed(self.config['url'].get()).get_entry(id) if entry is None: return None flags = Message.IS_HTML if thread.id not in self.storage.get('seen', default=[]): flags |= Message.IS_UNREAD if len(entry.content) > 0: content = u"<p>Link %s</p> %s" % (entry.link, entry.content[0]) else: content = entry.link thread.title = entry.title thread.root = Message(thread=thread, id=0, url=entry.link, title=entry.title, sender=entry.author, receivers=None, date=entry.datetime, parent=None, content=content, children=[], flags=flags) return thread def iter_unread_messages(self): for thread in self.iter_threads(): for m in thread.iter_all_messages(): if m.flags & m.IS_UNREAD: yield m def set_message_read(self, message): self.storage.get('seen', default=[]).append(message.thread.id) self.storage.save() def fill_thread(self, thread, fields): return self.get_thread(thread) OBJECTS = {Thread: fill_thread} ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/newsfeed/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000001665�13034501105�0017432�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class NewsfeedTest(BackendTest): MODULE = 'newsfeed' def test_newsfeed(self): for message in self.backend.iter_unread_messages(): pass ���������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016463�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000000105�13034501105�0020570�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import NihonNoOtoModule __all__ = ['NihonNoOtoModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/browser.py������������������������������������������������������������0000664�0000000�0000000�00000003055�13034501105�0020523�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from .pages import LivePage, ProgramPage __all__ = ['NihonNoOtoBrowser'] class NihonNoOtoBrowser(Browser): DOMAIN = 'www.nihon-no-oto.com' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] PAGES = { 'http://www\.nihon-no-oto\.com/': LivePage, 'http://www\.nihon-no-oto\.com/app/playlist.php': ProgramPage, } def home(self): self.location('/') assert self.is_on_page(LivePage) def iter_radios_list(self): self.location('/') assert self.is_on_page(LivePage) return self.page.iter_radios_list() def get_current_emission(self): self.location('/app/playlist.php') assert self.is_on_page(ProgramPage) return self.page.get_current_emission() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000016673�13034501105�0020633�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���tEXtSoftware�Adobe ImageReadyqe<��]IDATx{ t\^WJXl˶C ؄8l==Lf@Iϙ$CO/!3CO&a'}&d@bVcxwɋ$k*UzW%c;2&ӝ9<kU[w{WipYc4c�W�d|̷=� ? 9N$/X.� 8F8Q$ Vu9>Z7miti2 ۦqer~mA,M"jڨNa_2 *'|kOsEL\&]G� Lۖ;v5ֶ/~JCMq7h~�yec?/f϶7ߴLM@ ҾK.�?lF9d\eZ 0Z "Mw킓oJF}#/#p|!_S 庞%CkQs뙁W$m}p9S�{,'n9fXa|c;wbٿ|Kn-ܽDh9�YUU Pa`g<3:y _%ɎáF$ՍOV6^-[Ƴ`*`1e *+nhX*J*[)_.^KR$TJEpqhbut*_i]I,WtV.κQڋ ϝ1}b==+�'0_c嗑!X,5<6: ,Zv,YIzkL+k1Iw.g�p=|&4S7bxf&AκfA-a('_||ȟ<iUdh5s\͜/G'?((`'h㵿}ka|tc175)T*b7D@pP\I_$@ܲuqJ3p7vzh|z_\}IҞèfOSB[u! |pqr jqR -K+Ԉ>�M͝(&J'wƏdjfZIL{+Po8o0 do눷eCd  oiiڟ?6m |2i^qgLR1Bw � W$oK <pb`7mHPA^䰆+kkSo$�s</Nxn' C?JnF 3F>PN�pWs8?cxO@d+N'&}7oFba� Fx2ϑN zR[3 yڱknZ櫯Ǐ'�Cք)iL/hZp-`jb1uRG߄ˠN!?Npmp{n4 ˈ͛$>~ʴ"@98¥c˗~R4Zeǻ�Coc1N,3le$H̘E":$$(Ԑȓ*Ujgْcb`[K)qd}vpJPsY 'C/ ib|p@ j jmmE=A^Pd sU q=|Cax:ATSj:[,ڸ}T?l=l >,�Nc&U5?486]Jz!OO":ZĖތO}O8@͋zHY^�$(WxqdE~`�j<c?A^+!1zHF6Bg'yw�1`G:29�=?k =D;\K1Jf{{9*hXit?q"{ӯv{g(dWjqbhn\ 6v ᅙF` Ξ 'INl=;Z4wh lY8-'Qa&a?yJ�B"K0OuVr9VO Z{ݍ�0Ĵ9ֺn=6S$G.QEnZPĹsRοKFG~or,"\ʦ%r (0d,) �B M,fxHϟGHUhg(f՚'(TؾoqZ_d6~�lwІη#$qg'w8<_~��FGhD LjQ4�6@"tE?sk#6t ##K fWK|+{uR^2 U-ݾy4A7k!IAU"hRzA̻dK~|Sx' gc m'Wtiy+� -'@NtgG/ηx[lqW,idX/CM$~a!ؽq P}=gյ{w܊Q05jr5Mʧ7d$>^ɸ۶<>oo:gKǬ#?JR1NJ3@JHh`$rnh!^{}-!D}z.�ICzx 5|g@Ǫg-3'[p}˴l`޽ȒL @7&([ 9V_Y!ׇ(=Jm!WRnS&"I~V@1fP6:ON$dwh8W=Q%Hi(_kLkZtc$8j4HҌrW["J8E|Z">g]ǵ 5{F\/rcX0&wC?-( g*ŎsIm*N a�!Δ'$/vtɴF\k1U^�0pGb{F;v/;w_jDg`1\x~d3ek6>@[jCoGX3,)/(RJ]aKNbahi�%d=s#H"E/R3 ^(]CpE ٖL6Ղܲ?i[yWFYa܋L*3#8~X'\8,.1H[EwGVy�*FRv{ƢfڵSxoKP[ЎSs)sUNBDڞ4v_jwCϦJoԾ~G-H=Vڗ3+a9 2k"XSVHV)V,Z@u4yITcշ@qj尩Oy; NcKI5抱Ȉ!CVz5L i=5;z/E^RUBߟ8Nsd襗rF, r<r ܁:f*~8=~յAbdTh |xc;&ydd ʹ |= WډaF=^ב%;|F鹠"(U H1)f_IBդn\&+n3r7IC;ߣs4̺}=cw_ 8(t͏ʺ=U#j^y#3EYSZٳ:+>P}"1�k=gtJRIXTF9Lf6ox22cٿk8޻uv6>|  BE>,3C*K,].fZH"}`,tγ2xeYkl{Lr5C4|`u"9‚7/t2 QRqjnep 8aO"v[ǍX[,9# Xڊ}NO8G42yZ?.N0#<P_ L9BJ9OCZUmM݇ȞW?;t'VHCK(ヨ]{0t?zu6m:�9:2|СSU [),[6ى`>o EqbayT^mM}!{>x< '�g� F~i?jZ]/ݙ<DBA%BlK;"p󘶼 5O�Rx46Q"cD�;=ϟ,\/Vwr2\V*riVZ+1iᥛMKtʡ8.JQ߿s##-R\;`�BQB17=̼[O亡H+1CFlRC-r3e*wAv8] KTm(R!-J8QmZ @,Kzgȷ7Akk)5Dj7|f|2f܂z RբIp(~RNHOFKBT.A7tЭdzSa0,lSNqڀ8$6D{8͢ ]&ZXپ27Cխ(YYG}()f AΈyΞpCbɉcN<\N^tQ8.1pN~f6;&/g[{olW.vkC3Ei7?2 z>] t~nV#yhʙX^\aexLbx[q9u@\+Ι=x=.{bAUıInk2vxڭ:\kS/΃W-{;}f<6؁ ux9;j! u0)g 5qx9A>X%t; A:{G<c*H~#N\ uEOMܾTMk^M_ǡW o@9Q;g&f5!cc(kOڂf٧S;9V4Sɉ1EV8}$ jT:=DEgIvwS1nv3L}+Hlfǽrnvt:ϥx=/0sN 탈eưLoGHCߖ^vJu; #Fk"44HEcSyGISA^ �DT oaY*Ltnӧ I+]3׷8Ka+0@~|g6#=Brl)ӏ 4?Kކ_~x?E*aey-u>Zдh|7JM8e/�VWXr,]wũݭs9Vк[鴿jXd)]}%v? } -hQO5cڀ?ҋ$<;cG%6B; z& \ DoEK[F | FN^33 epT_0ňv?s7]L'6sD$^7ߌ?J],Qk:HKч>!|!a2\oIѐj@b! !5bAFrh !X;Ks Ӊx$yX�_h6lFu KkM/"tK R1e:\{?v$oԄ!gs4 8&Vk�XQ7z#50 z`]ٴǽ^|1`z7 >a2Ab Em?OTeNߦe؏]wEgrl<_7f7L Qhƺ./B@eVPEwa<=ǠRp8Ac,ɏLdhh?(:|Iz@CWUlלY[$޾O{M}{L.&Yp0OAty!V}yNJWl^D|`U:Tҕ3HȤ(T)^%Ֆ!zgyչp&-4‹�{ɼ-N V䪵rDw!B*`z�tUPb*{ӛڕ+sFU/E嗒]͹EkcChn!꺆ۍ-7 d&'o Z2|:jLʏOD#E5h`sBb0w6EdX-Ǜf̩a2'o/{&٤;uS|%H ksl(NT]� rLKk?}9xOkF#tQ?OB 9q#B^)8`w:Uiͺ0n76Jbg~gqw#1KMnaI!:k%:ŌYxWNreqc/򜚪Z0CňXbJ 9.?PA bt vS@u|+Lj'EDO Co(*/'ﰝl5Xc>m+BOy|5B<~YAşy|Jf>) p&P>~H(W"iBd)*/b;8MjŲKk=!O;w6:n6TD1mj H`\[1d,S"üӷ!nD\ boSCcLex@#=kqEbX5ridc|�3V݈o{ R(3߇p(S_=\ru؉A?tjNkwi{` ֫P]hnn](Lݨ 9cV7St|4[T+"Bh_4׬]Cq 'u݃i?i?uHqufl00 V0 i[Q?Meaw<ZerI@ 3._kGԣCteѮ20::^՚cT|c.>o`Q8},=0 o�qAӴ4$ #%sMPz̞1ZmZ֩jUy[vT{,VܰS`Gܬ#?~ut} =!BxElD|\#s�Vz`cz9h#9`%/6yd)Gi|S,Q"S< W\F7IT}-�r;=6Ӈ>�GKbLjΐU̻Xc|D k?7)M tuKN% mkک@Qm]uUѵlzn V8tD0+յZ3I\̡E�lFynS_겮FmPi5Tkc;D֞ak-=']ȗ7i3$kbZu?A*MEJ/M`�<X+����IENDB`���������������������������������������������������������������������weboob-1.2/modules/nihonnooto/module.py�������������������������������������������������������������0000664�0000000�0000000�00000004573�13034501105�0020333�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from .browser import NihonNoOtoBrowser __all__ = ['NihonNoOtoModule'] class NihonNoOtoModule(Module, CapRadio, CapCollection): NAME = 'nihonnooto' MAINTAINER = u'Thomas Lecavelier' EMAIL = 'thomas-weboob@lecavelier.name' VERSION = '1.2' DESCRIPTION = u'« Le son du Japon » french operated web radio, diffusing japanese music' # License of your module LICENSE = 'AGPLv3+' BROWSER = NihonNoOtoBrowser _RADIOS = {'nihonnooto': (u'Nihon no OTO', True) } def iter_resources(self, objs, split_path): if Radio in objs: self._restrict_level(split_path) for radio in self.browser.iter_radios_list(): self.browser.get_current_emission() radio.current = self.browser.get_current_emission() yield radio def iter_radios_search(self, pattern): for radio in self.browser.iter_radios_list(): if pattern.lower() in radio.title.lower() or pattern.lower() in radio.description.lower(): self.browser.get_current_emission() radio.current = self.browser.get_current_emission() yield radio def get_radio(self, radio): if not isinstance(radio, Radio): for rad in self.browser.iter_radios_list(): if rad.id == radio: return rad return None def fill_radio(self, radio, fields): if 'current' in fields: return self.get_radio(radio.id) return radio OBJECTS = {Radio: fill_radio} �������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000004325�13034501105�0020140�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page from weboob.capabilities.radio import Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo class LivePage(Page): def iter_radios_list(self): radio = Radio('nihon') radio.title = u'Nihon no Oto' radio.description = u'Nihon no Oto: le son du Japon' radio.streams = [] index = -1 for el in self.document.xpath('//source'): index += 1 mime_type = unicode(el.attrib['type']) stream_url = unicode(el.attrib['src']) stream = BaseAudioStream(index) stream.bitrate = 128 if (mime_type == u'audio/mpeg'): stream.format = u'mp3' elif (mime_type == u'audio/ogg'): stream.format = u'vorbis' stream.title = radio.title + ' ' + mime_type stream.url = stream_url radio.streams.append(stream) yield radio class ProgramPage(Page): def get_current_emission(self): current = StreamInfo(0) two_or_more = unicode(self.document.xpath('//p')[0].text).split('/////')[0].split(' - ') # Consider that if String(' - ') appears it'll be in title rather in the artist name if len(two_or_more) > 2: current.who = two_or_more.pop(0) current.what = ' - '.join(two_or_more) else: current.who, current.what = two_or_more return current �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nihonnooto/test.py���������������������������������������������������������������0000664�0000000�0000000�00000001702�13034501105�0020014�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Thomas Lecavelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class NihonNoOtoTest(BackendTest): MODULE = 'nihonnooto' def test_nihonnooto(self): l = list(self.backend.iter_radios_search('')) self.assertTrue(len(l) > 0) ��������������������������������������������������������������weboob-1.2/modules/nolifetv/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016117�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001433�13034501105�0020231�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NolifeTVModule __all__ = ['NolifeTVModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000007156�13034501105�0020165�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserIncorrectPassword import urllib from weboob.deprecated.browser.decorators import id2url from .video import NolifeTVVideo from .pages import VideoPage, VideoListPage, FamilyPage, AboPage, LoginPage, HomePage __all__ = ['NolifeTVBrowser'] class NolifeTVBrowser(Browser): USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] DOMAIN = 'mobile.nolife-tv.com' PROTOCOL = 'http' PAGES = { r'http://mobile.nolife-tv.com/online/familles-\w+/': FamilyPage, r'http://mobile.nolife-tv.com/online/emission-(?P<id>\d+)/': VideoPage, 'http://mobile.nolife-tv.com/do.php': VideoListPage, 'http://mobile.nolife-tv.com/online/': VideoListPage, 'http://mobile.nolife-tv.com/abonnement/': AboPage, 'http://mobile.nolife-tv.com/login': LoginPage, 'http://mobile.nolife-tv.com/': HomePage, } AVAILABLE_VIDEOS = ['[Gratuit]'] def is_logged(self): return (self.username is None or (not self.is_on_page(HomePage)) or self.page.is_logged()) def login(self): if self.username is None: return if not self.is_on_page(LoginPage): self.location('/login', no_login=True) self.page.login(self.username, self.password) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() self.location('/abonnement/', no_login=True) assert self.is_on_page(AboPage) self.AVAILABLE_VIDEOS = self.page.get_available_videos() @id2url(NolifeTVVideo.id2url) def get_video(self, url, video=None): self.location(url) assert self.is_on_page(VideoPage) return self.page.get_video(video) def iter_family(self, type, sub): self.location('/online/familles-%s/' % type) assert self.is_on_page(FamilyPage) return self.page.iter_family(sub) def iter_category(self, type): self.location('/online/familles-%s/' % type) assert self.is_on_page(FamilyPage) return self.page.iter_category() def iter_video(self, family): data = { 'a': 'ge', 'famille': family, 'emissions': 0 } while True: self.location('/do.php', urllib.urlencode(data)) assert self.is_on_page(VideoListPage) if self.page.is_list_empty(): break for vid in self.page.iter_video(self.AVAILABLE_VIDEOS): yield vid data['emissions'] = data['emissions'] + 1 def get_latest(self): return self.iter_video(0) def search_videos(self, pattern): data = { 'search': pattern, 'submit': 'Rechercher' } self.location('/online/', urllib.urlencode(data)) assert self.is_on_page(VideoListPage) for vid in self.page.iter_video(self.AVAILABLE_VIDEOS): yield vid ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000005145�13034501105�0020257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������C��� pHYs�� �� ����tIME(yh���tEXtComment�Created with GIMPW�� IDATxYlSI!&KqHRBUH%6)mfQENUAtSVLȤ-"ai5Q-@5Yڊ`uB4ȏ q]fcvc;?bL}d{λ{y 2dȐ!C 2dȐ!C 2~JP9@KS #>UUU8�ftK��PᆪGy$6::w\�v�5 Rmm-i42軲qYVZqkKNN'ݻ7e�&DNDq:vzSqC__48ǡu9z?~UUU^�~i<|̙J@eeeFY\d;z==z�pyLLL߿'%jXeX}ݾ7F<9r�J�{^* Ҍr(yvT(p�@ ⽌k�6�x��z(+ACvvKl"^WY �^ZvmS}}}V|>~llr& q�\2ȴD}poo/ %""O,¹> D.f�Љ'$a2d4СCbXWWeB06� ǎ9Bxaqq<(}FCfI| % D"K9ϢKgDMNN.EڈyL"�!&qvH$<ݼy:D%~zvI_`1<<,v4 ?LOOK"}'=pm㻾3FE?H@  ɔ|*d4gVUzf�^2R�G%OئN&`xxX24;;+�% r\sR|׭,#ʲk׮GGGY=I'M+ғ:~ݺVUO?ӅD"H$fG~>66&pb/8ah4IN,n=!YD&-,DDz�XAv.Dncaa(++/e W_} 6H#m�d #P𦦦ܥ �Ѐh4uݵ%%%%�z=Ө~-xRʕ+DhIaIb[4Z-,cB? SEO=[ܜ�*|>x6D( o<Xj>scOgȅ�'p8B�ݻn+YD"#S�*А)KNss3zzzr"٪D_l? z^f(�L �Nm7LYSSSEqIޱcG[`ppP5 � @;3 `ҥ1o֭0L�PӳBggQ/w,A0m&s?s n9T_dOZL]]],#A<q{15b3`09F wޤDܹsҳ\A3�MBʕ+NCCCp8pW]A(�ob1S#DNFGGJbvxh4J/r y�886F/ TC,%vܿɉB4>>.f!�O)j?�Nx<tizRO8e2t:EY!FU}iϞ=[ѣGޞ*\%̣>Zj{r]0(:S^F"HgB8ub~~|/0VTTx<?cV`` qƩ 1-<kTOy:nIc :ضm)VtGFFl߾=~9զMyFSl�ĄGT*J҂`0�0B(//@YYYի|@.cݿ͛7_pA:P(|BTNK)ؿQIwGn/T|whhfI� 5^NLLp̄Fl2/F\ߏ9vx Z&vZ>%cju>;{Ѿ}~Fx@VW%oٳg㍍1"OP(yɓ>DV_|YkM)$.n:`?vNWd4jjj6XBh45[髩?xʕ:' ppYp:7nJ�iɼN+FZ ~h`cE (F 6eȐ!C 2dȐ!C Qͦ(U����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/module.py���������������������������������������������������������������0000664�0000000�0000000�00000012413�13034501105�0017757�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.backend import Module, BackendConfig from .browser import NolifeTVBrowser from .video import NolifeTVVideo import urllib import time from hashlib import md5 __all__ = ['NolifeTVModule'] class NolifeTVModule(Module, CapVideo, CapCollection): NAME = 'nolifetv' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'NolifeTV French video streaming website' LICENSE = 'AGPLv3+' BROWSER = NolifeTVBrowser CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), Value('quality', label='Quality', choices = { '1':'LQ', '2':'HQ', '3':'TV', '4':'720p', '5':'1080p' }, default = '5' )) def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(username, password) def iter_resources(self, objs, split_path): with self.browser: if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield Collection([u'theme'], u'Par theme') yield Collection([u'type'], u'Par type') yield Collection([u'latest'], u'Latest NolifeTV videos') if collection.path_level == 1: if split_path[0] == 'latest': for vid in self.browser.get_latest(): yield vid else: for cat in self.browser.iter_category(split_path[0]): yield cat if collection.path_level == 2: for cat in self.browser.iter_family(split_path[0], split_path[1]): yield cat if collection.path_level == 3: for cat in self.browser.iter_video(split_path[2]): yield cat def validate_collection(self, objs, collection): if BaseVideo in objs: if collection.path_level == 0: return if collection.path_level == 1 and collection.split_path[0] in [u'theme', u'type', u'latest']: return if collection.path_level > 1: return raise CollectionNotFound(collection.split_path) def get_video(self, _id): with self.browser: return self.browser.get_video(_id) def fill_video(self, video, fields): with self.browser: self.browser.get_video(NolifeTVVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) if 'url' in fields: with self.browser: video.url = self.get_url(video.id, self.config['quality'].get()) return video def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): with self.browser: return self.browser.search_videos(pattern) OBJECTS = { NolifeTVVideo: fill_video } SALT = 'a53be1853770f0ebe0311d6993c7bcbe' def genkey(self): # This website is really useful to get info: http://www.showmycode.com/ timestamp = str(int(time.time())) skey = md5(md5(timestamp).hexdigest() + self.SALT).hexdigest() return skey, timestamp def get_url(self, id, quality): skey, timestamp = self.genkey() self.browser.readurl('http://online.nolife-tv.com/_nlfplayer/api/api_player.php', 'quality=%s&a=EML&skey=%s&id%%5Fnlshow=%s×tamp=%s' % (quality, skey, id, timestamp)) skey, timestamp = self.genkey() data = self.browser.readurl('http://online.nolife-tv.com/_nlfplayer/api/api_player.php', 'quality=%s&a=UEM%%7CSEM%%7CMEM%%7CCH%%7CSWQ&skey=%s&id%%5Fnlshow=%s×tamp=%s' % (quality, skey, id, timestamp)) values = dict([urllib.splitvalue(s) for s in data.split('&')]) if 'url' not in values: return None return unicode(values['url']) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/pages.py����������������������������������������������������������������0000664�0000000�0000000�00000011460�13034501105�0017572�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.collection import Collection from weboob.capabilities.image import Thumbnail from weboob.deprecated.browser import Page import re from datetime import datetime, timedelta from .video import NolifeTVVideo class VideoPage(Page): def get_video(self, video): if not video: video = NolifeTVVideo(self.group_dict['id']) els = self.document.getroot().xpath('//div[@data-role="content"]') if els and els[0] is not None: h3 = els[0].find('h3') if h3 is not None and h3.text: video.title = unicode(h3.text) h4 = els[0].find('h4') if h4 is not None and h4.text: video.title = video.title + u' - ' + h4.text thumb = els[0].find('p/img') if thumb is not None and thumb.get('src'): video.thumbnail = Thumbnail(thumb.attrib['src']) video.thumbnail.url = video.thumbnail.id ps = els[0].findall('p') if len(ps) > 4: if ps[4].text: video.description = ps[4].text if ps[0].text and ps[0].text != u'∞': video.date = datetime.strptime(ps[0].text, '%d/%m/%Y').date() for text in ps[2].xpath('.//text()'): m = re.search(r'[^\d]*((\d+):)?(\d+)s?', text) if m: if m.group(2): minutes = int(m.group(2)) else: minutes = 0 video.duration = timedelta(minutes=minutes, seconds=int(m.group(3))) return video class VideoListPage(Page): def is_list_empty(self): return self.document.getroot() is None def iter_video(self, available_videos): for el in self.document.getroot().xpath('//li/a'): strongs = el.findall('p/strong') if len(strongs) > 3 and strongs[0].text not in ['Autopromo', 'Annonce'] and strongs[1].text in available_videos: m = re.search(r'emission-(\d+)', el.attrib['href']) if m and m.group(1): video = NolifeTVVideo(m.group(1)) h3 = el.find('h3') if h3 is not None and h3.text: video.title = unicode(h3.text) if strongs[3].text: video.title = video.title + ' - ' + strongs[3].text yield video class FamilyPage(Page): def iter_category(self): subs = list() for el in self.document.xpath('//ul/li[@data-role="list-divider"]'): if el.text not in subs: yield Collection([el.text], unicode(el.text)) subs.append(el.text) def iter_family(self, sub): for el in self.document.xpath('//ul/li[@data-role="list-divider"]'): if el.text != sub: continue while True: el = el.getnext() if el is None or el.get('data-role'): break h1 = el.find('.//h1') id = h1.getparent().attrib['href'] m = re.search(r'famille-(\d+)', id) if m and m.group(1): yield Collection([m.group(1)], unicode(h1.text)) class AboPage(Page): def get_available_videos(self): available = ['[Gratuit]'] for text in self.document.xpath('//div[@data-role="content"]/center/text()'): if 'Soutien' in text: available.append('[Archive]') available.append('[Standard]') if 'Standard' in text: available.append('[Standard]') return available class LoginPage(Page): def login(self, username, password): self.browser.select_form(name='login') self.browser['username'] = str(username) self.browser['password'] = str(password) self.browser.submit() class HomePage(Page): def is_logged(self): return len(self.document.xpath('//a[@href="deconnexion/"]')) == 1 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nolifetv/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000002701�13034501105�0017450�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class NolifeTVTest(BackendTest): MODULE = 'nolifetv' def test_search(self): l = list(self.backend.search_videos('nolife')) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) assert len(l) > 0 v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ���������������������������������������������������������������weboob-1.2/modules/nolifetv/video.py����������������������������������������������������������������0000664�0000000�0000000�00000002026�13034501105�0017577�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class NolifeTVVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.ext = u'mp4' @classmethod def id2url(cls, _id): return u'http://mobile.nolife-tv.com/online/emission-%s/' % _id ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nova/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015234�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nova/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001423�13034501105�0017345�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NovaModule __all__ = ['NovaModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/nova/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000000710�13034501105�0017365�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME/]���tEXtComment�Created with GIMPW��#IDATx1JA@ؤBZHeg!`6^iB+!N,N c6 fv0T;;ɼl6Y$I$I$I|X�/-02_)p9 8g)_o,8+x}Gp'؋  H@?jsɖӛ_P%V;|.9Q;`+nM� 6*C�'k w.;D OJZa(96^k~[�0 `��0 `�H9 ^����IENDB`��������������������������������������������������������weboob-1.2/modules/nova/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000006731�13034501105�0017102�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from cStringIO import StringIO from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from weboob.deprecated.browser import StandardBrowser from weboob.deprecated.browser.parsers import get_parser __all__ = ['NovaModule'] class NovaModule(Module, CapRadio, CapCollection): NAME = 'nova' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'Nova French radio' LICENSE = 'AGPLv3+' BROWSER = StandardBrowser _RADIOS = {'nova': (u'Radio Nova', u'Radio nova', u'http://broadcast.infomaniak.net:80/radionova-high.mp3'), } def create_default_browser(self): return self.create_browser(parser='json') def iter_resources(self, objs, split_path): if Radio in objs: self._restrict_level(split_path) for id in self._RADIOS.iterkeys(): yield self.get_radio(id) def iter_radios_search(self, pattern): for radio in self.iter_resources((Radio, ), []): if pattern.lower() in radio.title.lower() or pattern.lower() in radio.description.lower(): yield radio def get_radio(self, radio): if not isinstance(radio, Radio): radio = Radio(radio) if radio.id not in self._RADIOS: return None title, description, url = self._RADIOS[radio.id] radio.title = title radio.description = description artist, title = self.get_current() current = StreamInfo(0) current.who = artist current.what = title radio.current = current stream = BaseAudioStream(0) stream.bitrate=128 stream.format=u'mp3' stream.title = u'128kbits/s' stream.url = url radio.streams = [stream] return radio def get_current(self): doc = self.browser.location('http://www.novaplanet.com/radionova/ontheair?origin=/') html = doc['track']['markup'] parser = get_parser()() doc = parser.parse(StringIO(html)) artist = u' '.join([txt.strip() for txt in doc.xpath('//div[@class="artist"]')[0].itertext()]) title = u' '.join([txt.strip() for txt in doc.xpath('//div[@class="title"]')[0].itertext()]) return unicode(artist).strip(), unicode(title).strip() def fill_radio(self, radio, fields): if 'current' in fields: if not radio.current: radio.current = StreamInfo(0) radio.current.who, radio.current.what = self.get_current() return radio OBJECTS = {Radio: fill_radio} ���������������������������������������weboob-1.2/modules/nova/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000001737�13034501105�0016575�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.radio import Radio class NovaTest(BackendTest): MODULE = 'nova' def test_nova(self): l = list(self.backend.iter_resources((Radio, ), [])) self.assertTrue(len(l) > 0) ���������������������������������weboob-1.2/modules/okc/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015045�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001505�13034501105�0017157�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .browser import OkCBrowser from .module import OkCModule __all__ = ['OkCBrowser', 'OkCModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000007305�13034501105�0017107�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012-2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.json import json __all__ = ['OkCBrowser'] def need_login(func): def inner(browser, *args, **kwargs): if not browser.access_token: browser.do_login() return func(browser, *args, **kwargs) return inner class OkCBrowser(LoginBrowser): BASEURL = 'https://www.okcupid.com' login = URL('/login') threads = URL('/messages') messages = URL('/apitun/messages/conversations/global_messaging') thread_delete = URL(r'/1/apitun/messages/conversations/(?P<thread_id>\d+)/delete') message_send = URL('/apitun/messages/send') quickmatch = URL(r'/quickmatch\?okc_api=1') like = URL(r'/1/apitun/profile/(?P<user_id>\d+)/like') profile = URL(r'/apitun/profile/(?P<user_id>\d+)') full_profile = URL(r'/profile/(?P<username>.*)\?okc_api=1') access_token = None me = None def do_login(self): r = self.login.go(data={'username': self.username, 'password': self.password, 'okc_api': 1}).json() if not 'oauth_accesstoken' in r: raise BrowserIncorrectPassword(r['status_str']) self.access_token = r['oauth_accesstoken'] self.me = {'userid': r['userid'], 'username': r['screenname'], } self.session.headers['X-OkCupid-Platform'] = 'DESKTOP' self.session.headers['X-Requested-With'] = 'XMLHttpRequest' self.session.headers['Authorization'] = 'Bearer %s' % self.access_token @need_login def get_threads_list(self, folder=1): return self.threads.go(params={'okc_api': 1, 'folder': folder, 'messages_dropdown_ajax': 1}).json() @need_login def get_thread_messages(self, thread_id): r = self.messages.go(params={'access_token': self.access_token, '_json': '{"userids":["%s"]}' % thread_id}).json() return r[thread_id] @need_login def post_message(self, thread_id, content): data = {'body': content, 'profile_tab': '', 'receiverid': thread_id, 'service': 'mailbox', 'source': 'desktop_global'} self.message_send.go(params={'access_token': self.access_token}, data=json.dumps(data)) @need_login def delete_thread(self, thread_id): self.thread_delete.go(method='POST', thread_id=thread_id) @need_login def find_match_profile(self): r = self.quickmatch.go().json() return r['tracking_userid'] @need_login def do_rate(self, user_id): self.like.go(method='POST', user_id=user_id) @need_login def get_username(self, user_id): return self.profile.go(user_id=user_id).json()['username'] @need_login def get_profile(self, username): if username.isdigit(): username = self.get_username(username) return self.full_profile.go(username=username).json() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000005032�13034501105�0017200�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� B(x���tIMEk���tEXtComment�Created with GIMPW�� uIDATxIoqHdQŶ+41D �9h X[uΗ(K@k1 ;%;^Id-4IqiZDrI ݞ>;g;vixxm|M�"w 0#oC�|?i@HM_@C_7𑐚P�: w҇4l!�K1�e3IXR{ 4-oqx&RLu�=u�L<Cͥo~(s'j̽ PG�U7R=FdAj8"5i9*O\3<�Z~,rw*UjVe)yƣ:h?wv-EMH߳�@o؎m722E@๾)$Qi((t6YbDgf|U&4@ 8*^J\)/^"i}~3#"<||leǵ�(a&p\,c"/8JT[ @ͮP06ȔٕL-om -0sd ~+9fk>Q*f-,Ʃ؅,@"^J{b\1JޡЌR5.+wj M{!5 XG\7@ܝԶi?W=_D�mAM=XB>HMt=b$(}6f`:^Da׋R7�DaaG "ɶؒ�@y:;/*< �lP�xHR.cnE>O<0 x yh,Q<ߵbǺe� }9%E-ե _a40DTHͫ+!&N"ycnRPxp^TRWcL~ ɎdtBywSnLBNjsw ،҃,v.rBk{)j+j]vlzϲ;;(-#okE%{�zK'Zd{_ZZtߨHdN_d)?K 3Bǜwfds%* 'Bc]@ٲ50kghJ5㑆358p (USZ,*>lHUDAg�6>1xK1F3-2z[rt&UfMWL.l= sܱKnҍ&dn_z!$ NdDxBɅ ^~kp%YԔ˝P> ;T}2SsAV,to͎."3Uv2Vj2~fLUUa6g6M~+jO桚@6g54Ud02_VVk,?2c#�ِ߂ʶTHoDxtUm WpOE*^ۆu%bn ZS"2c#~"al"5_TENi$5b >a8,.-^b"[O&y΋gCȲ 2q:�(5[m|y=Ǘ7J3a:4�lQ@ ȼBwތ&CbdY@反Ѥ^:7o VM3ܙ@SC@ZGIxB/͕٣*oG1+~MUf''(ƫf8m? 4DGj �Rc4*ٻ|eГ>5)rHJpܽ.sJ|ɞb|gmCq?bb+D&�Yney6V WG@yn~T:4Uι&7W/M]S2XŎ(Л-Y% &z5w9>irYk*vR_r_9/+ x�@v\e7m #aXWX*\W/7]lT\rd.c | W>Z |e Rw(2w~gb4GD$Qv\̚To\j817",y!)4:Bz`8S`hcG92T˛x-=� Ezj丨ϲPMyٰ:D9w�8_Px\S[Ǖf|̈Bjz>pQHM;|xK!5}o[RospSإo=lV|ߑOoBjze����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000022516�13034501105�0016712�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012-2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime from html2text import unescape from weboob.capabilities.contact import CapContact, ContactPhoto, Contact, ProfileNode from weboob.capabilities.dating import CapDating from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message, Thread from weboob.tools.backend import Module, BackendConfig from weboob.tools.misc import to_unicode from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value, ValueBackendPassword from .browser import OkCBrowser from .optim.profiles_walker import ProfilesWalker __all__ = ['OkCModule'] class OkcContact(Contact): def set_profile(self, *args): section = self.profile for arg in args[:-2]: try: s = section[arg] except KeyError: s = section[arg] = ProfileNode(arg, arg.capitalize().replace('_', ' '), OrderedDict(), flags=ProfileNode.SECTION) section = s.value key = args[-2] value = args[-1] section[key] = ProfileNode(key, key.capitalize().replace('_', ' '), value) def __init__(self, profile): super(OkcContact, self).__init__(profile['userid'], profile['username'], self.STATUS_ONLINE if profile['is_online'] == '1' else self.STATUS_OFFLINE) self.url = 'https://www.okcupid.com/profile/%s' % self.name self.summary = profile.get('summary', '') self.status_msg = 'Last connection at %s' % profile['skinny']['last_online'] for no, photo in enumerate(profile['photos']): self.set_photo(u'image_%i' % no, url=photo['image_url'], thumbnail_url=photo['image_url']) self.profile = OrderedDict() self.set_profile('info', 'status', profile['status_str']) self.set_profile('info', 'orientation', profile['orientation_str']) self.set_profile('info', 'age', '%s yo' % profile['age']) self.set_profile('info', 'birthday', '%04d-%02d-%02d' % (profile['birthday']['year'], profile['birthday']['month'], profile['birthday']['day'])) self.set_profile('info', 'sex', profile['gender_str']) self.set_profile('info', 'location', profile['location']) self.set_profile('info', 'join_date', profile['skinny']['join_date']) self.set_profile('stats', 'match_percent', '%s%%' % profile['matchpercentage']) self.set_profile('stats', 'friend_percent', '%s%%' % profile['friendpercentage']) self.set_profile('stats', 'enemy_percent', '%s%%' % profile['enemypercentage']) for key, value in sorted(profile['skinny'].items()): self.set_profile('details', key, value or '-') for essay in profile['essays']: if len(essay['essay']) == 0: continue self.summary += '%s:\n' % essay['title'] self.summary += '-' * (len(essay['title']) + 1) self.summary += '\n' for text in essay['essay']: self.summary += text['rawtext'] self.summary += '\n\n' self.profile['info'].flags |= ProfileNode.HEAD self.profile['stats'].flags |= ProfileNode.HEAD class OkCModule(Module, CapMessages, CapContact, CapMessagesPost, CapDating): NAME = 'okc' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'OkCupid' CONFIG = BackendConfig(Value('username', label='Username'), ValueBackendPassword('password', label='Password')) STORAGE = {'profiles_walker': {'viewed': []}, 'sluts': {}, } BROWSER = OkCBrowser def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) # ---- CapDating methods --------------------- def init_optimizations(self): self.add_optimization('PROFILE_WALKER', ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)) # ---- CapMessages methods --------------------- def fill_thread(self, thread, fields): return self.get_thread(thread) def iter_threads(self): threads = self.browser.get_threads_list() for thread in threads: t = Thread(thread['userid']) t.flags = Thread.IS_DISCUSSION t.title = u'Discussion with %s' % thread['user']['username'] t.date = datetime.fromtimestamp(thread['timestamp']) yield t def get_thread(self, thread): if not isinstance(thread, Thread): thread = Thread(thread) thread.flags = Thread.IS_DISCUSSION messages = self.browser.get_thread_messages(thread.id) contact = self.storage.get('sluts', thread.id, default={'lastmsg': datetime(1970,1,1)}) thread.title = u'Discussion with %s' % messages['fields']['username'] me = OkcContact(self.browser.get_profile(self.browser.me['userid'])) other = OkcContact(self.browser.get_profile(thread.id)) parent = None for message in messages['messages']['messages']: date = datetime.fromtimestamp(message['timestamp']) flags = 0 if contact['lastmsg'] < date: flags = Message.IS_UNREAD if message['from'] == thread.id: sender = other receiver = me else: receiver = other sender = me msg = Message(thread=thread, id=message['id'], title=thread.title, sender=sender.name, receivers=[receiver.name], date=date, content=to_unicode(unescape(message['body'])), children=[], parent=parent, signature=sender.get_text(), flags=flags) if parent: parent.children = [msg] else: thread.root = msg parent = msg return thread def iter_unread_messages(self): for thread in self.iter_threads(): contact = self.storage.get('sluts', thread.id, default={'lastmsg': datetime(1970,1,1)}) if thread.date <= contact['lastmsg']: continue thread = self.get_thread(thread) for message in thread.iter_all_messages(): if message.flags & message.IS_UNREAD: yield message def set_message_read(self, message): contact = self.storage.get('sluts', message.thread.id, default={'lastmsg': datetime(1970,1,1)}) if contact['lastmsg'] < message.date: contact['lastmsg'] = message.date self.storage.set('sluts', message.thread.id, contact) self.storage.save() # ---- CapMessagesPost methods --------------------- def post_message(self, message): self.browser.post_message(message.thread.id, message.content) # ---- CapContact methods --------------------- def fill_contact(self, contact, fields): if 'profile' in fields: contact = self.get_contact(contact) if contact and 'photos' in fields: for name, photo in contact.photos.iteritems(): if photo.url and not photo.data: data = self.browser.open(photo.url).content contact.set_photo(name, data=data) if photo.thumbnail_url and not photo.thumbnail_data: data = self.browser.open(photo.thumbnail_url).content contact.set_photo(name, thumbnail_data=data) def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content if 'thumbnail_data' in fields and photo.thumbnail_url and not photo.thumbnail_data: photo.thumbnail_data = self.browser.open(photo.thumbnail_url).content return photo def get_contact(self, user_id): if isinstance(user_id, Contact): user_id = user_id.id info = self.browser.get_profile(user_id) return OkcContact(info) def iter_contacts(self, status=Contact.STATUS_ALL, ids=None): threads = self.browser.get_threads_list() for thread in threads: c = self.get_contact(thread['user']['username']) if c and (c.status & status) and (not ids or c.id in ids): yield c OBJECTS = {Thread: fill_thread, Contact: fill_contact, ContactPhoto: fill_photo } ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/optim/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016175�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/optim/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020274�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/okc/optim/profiles_walker.py�����������������������������������������������������0000664�0000000�0000000�00000007434�13034501105�0021747�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime from dateutil.relativedelta import relativedelta from random import randint from weboob.capabilities.dating import Optimization from weboob.tools.log import getLogger from weboob.tools.value import Value, ValuesDict class ProfilesWalker(Optimization): CONFIG = ValuesDict(Value('first_message', label='First message to send to matched profiles', default='')) def __init__(self, sched, storage, browser): super(ProfilesWalker, self).__init__() self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('walker', browser.logger) self._config = storage.get('profile_walker', 'config', default=None) if self._config == {}: self._config = None self._view_cron = None self._visited_profiles = set(storage.get('profiles_walker', 'viewed')) self._logger.info(u'Loaded %d already visited profiles from storage.', len(self._visited_profiles)) def save(self): if self._config is None: return False self._storage.set('profiles_walker', 'viewed', list(self._visited_profiles)) self._storage.save() def start(self): self._view_cron = self._sched.schedule(randint(5, 10), self.view_profile) return True def stop(self): self._sched.cancel(self._view_cron) self._view_cron = None return True def is_running(self): return self._view_cron is not None def set_config(self, params): self._config = params self._storage.set('profile_walker', 'config', self._config) self._storage.save() def get_config(self): return self._config def view_profile(self): try: # Remove old threads for thread in self._browser.get_threads_list(folder=2): # folder 2 is the sentbox last_message = datetime.fromtimestamp(thread['timestamp']) if not thread['replied'] and last_message < (datetime.now() - relativedelta(months=6)): self._logger.info('Removing old thread with %s from %s', thread['user']['username'], last_message) self._browser.delete_thread(thread['userid']) # Find a new profile user_id = self._browser.find_match_profile() if user_id in self._visited_profiles: return self._browser.do_rate(user_id) profile = self._browser.get_profile(user_id) if self._config['first_message'] != '': self._browser.post_message(user_id, self._config['first_message'] % {'name': profile['username']}) self._browser.delete_thread(user_id) self._logger.info(u'Visited profile of %s ', profile['username']) # do not forget that we visited this profile, to avoid re-visiting it. self._visited_profiles.add(user_id) self.save() finally: if self._view_cron is not None: self._view_cron = self._sched.schedule(randint(60, 120), self.view_profile) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015243�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001424�13034501105�0017355�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Budget Insight # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OneyModule __all__ = ['OneyModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/browser.py������������������������������������������������������������������0000664�0000000�0000000�00000004414�13034501105�0017303�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Budget Insight # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.exceptions import BrowserIncorrectPassword from weboob.browser import LoginBrowser, URL, need_login from .pages import LoginPage, IndexPage, OperationsPage __all__ = ['OneyBrowser'] class OneyBrowser(LoginBrowser): BASEURL = 'https://www.oney.fr' index = URL(r'/oney/client', IndexPage) login = URL(r'/site/s/login/login.html', LoginPage) operations = URL(r'/oney/client', OperationsPage) card_page = URL(r'/oney/client\?task=Synthese&process=SyntheseMultiCompte&indexSelectionne=(?P<acc_num>/d)') def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.go() self.page.login(self.username, self.password) if not self.index.is_here(): raise BrowserIncorrectPassword() @need_login def get_accounts_list(self): return self.index.stay_or_go().iter_accounts() @need_login def iter_history(self, account): if account._num: self.card_page.go(acc_num=account._num) post = {'task': 'Synthese', 'process': 'SyntheseCompte', 'taskid': 'Releve'} self.operations.go(data=post) return self.page.iter_transactions(seen=set()) @need_login def iter_coming(self, account): if account._num: self.card_page.go(acc_num=account._num) post = {'task': 'OperationRecente', 'process': 'OperationRecente', 'taskid': 'OperationRecente'} self.operations.go(data=post) return self.page.iter_transactions(seen=set()) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000005377�13034501105�0017412�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD��N�cQ��� pHYs�� �� ����tIME  %[}���tEXtComment�Created with GIMPW�� gIDATx[klf絳k.8!*PDe*$i+5i" E+G(Qd`(HچWU( jR+1k{mcfgfyɮ]xSZ;9{MiF]B8�K�4`Z�!�B6@@@'�N8FQ^rBՄC3VB�8A#uBHT><{2go%7ZoOo'&=,!Nd۶=n"�n{!K($N)ϏxDXE+X T1`w]m0 IdY~qΰ'Sh$ض M I% )] OEQt3BcTz( 4Meم'OlSKS7',R)(QD.^B xg!Ey|>x t{0`,>rb\;ƿq? 5D@zR~.K'C|ۃ(`h^)(U�Myy~e!(4x�. à|E^ &3h9m~Fqaَ;{mSjG{|j=XMӠiNc<̃/tJBTpb^p1;톅C �7#`�X?w⿲./VL5C̵p֣)t?>|t_~<7kxGAX`w&޺wf;;17oyp6�i�:7^k{Gx7{�@-7 3/_�5�`{Ww@ԣT oGy /ͮ | 4[Er6"$ j˖O_ .�މ~2 �2,beg#�{ê-ItURNhwgO܌'fnvpp{o$Ճ2vnHpAmrY[j�(]_> Ւ��K? s@�bz{b0uY7EE`<_%WVpUtԕ]cMe(c+I])Mr ?)�� 1P>�~jJPdM<#ds3b|=R,18V݆R"dzf| ��472LƞMYy=QknoŬɋٕlUİ~ُhZ5;ЗяOăͺi"*�i( s0|R\rp9ݗ_ȺbFg4�K懃�TqXU;>F>IDiB �!Vnx?S<q5a<TlA ߶B=X U\-V?SjF04XTX^+Lt jV?k L|C!l!D%R44( (0.^SO=(:K};϶mCuiZg8[2T6۶a6,˂iPU,G)|0`&LӄH$p mGa J9+I@IG !`6LӄPUB#۶mG[:`v<P *p ÀiV$֮iiZƾ}gsDeHdE"Q (*aYN/IJa@QX9{*W$AE |F7PVV!IT*MӠKD)H_4W(bpppo[[[G��˲u],0Me);,B'IX,+WlιfBMMͅT*Qe(0ܪ*s $8O$7KrV=ܹs'8[ qhv27 nT}̅(Dwwk֬YwxXլi%˲"S 7B #; /H`hhuwTϟߡX x@prd~$X,tww?u֎ўSn?B!yw>3,wH85xwG"tuu=ҲˣG %8wو^$8wfvN‹H$7P3CL0 2Ddz]%.b[~b=߿?A8<xƉyYH$܄M,˘]o7,=,t 8Rwæ:SNc``_~c,8vj^_UU嬨eYnE�n8NmH$7nذBԫ:?  z8 12.c"goOO;w^o'4Moٲ14WVV>hA ^oOUe8dw<YڊI7o۶c|]k}>ʪ`p40zz`NsK'ޓ$TU䑡H}Ϟ=u/9y x<�B!]IiFM )X,v*۷o_Mici:p}����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000004127�13034501105�0017106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Budget Insight # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import OneyBrowser __all__ = ['OneyModule'] class OneyModule(Module, CapBank): NAME = 'oney' MAINTAINER = u'Vincet Paredes' EMAIL = 'vparedes@budget-insight.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Oney' CONFIG = BackendConfig(ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = OneyBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def iter_history(self, account): # To prevent issues in calcul of actual balance and coming one, all # operations are marked as debited. for tr in self.browser.iter_coming(account): yield tr for tr in self.browser.iter_history(account): yield tr �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/oney/pages.py��������������������������������������������������������������������0000664�0000000�0000000�00000016770�13034501105�0016727�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Budget Insight # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from cStringIO import StringIO import requests from weboob.capabilities.bank import Account from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import Env, CleanDecimal, CleanText, Field, Format from weboob.browser.filters.html import Attr from weboob.exceptions import BrowserIncorrectPassword class Transaction(FrenchTransaction): PATTERNS = [(re.compile(ur'^(?P<text>Retrait .*?) - traité le \d+/\d+$'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(ur'^(?P<text>Prélèvement .*?) - traité le \d+/\d+$'), FrenchTransaction.TYPE_ORDER), (re.compile(ur'^(?P<text>.*?) - traité le \d+/\d+$'), FrenchTransaction.TYPE_CARD)] class VirtKeyboard(MappedVirtKeyboard): symbols={'0':('8664b9cdfa66b4c3a1ec99c35a2bf64b','9eb80c6e99410eaac32905b2c77e65e5','37717277dc2471c8a7bf37e2068a8f01'), '1':('1f36986f9d27dde54ce5b08e8e285476','9d0aa7a0a2bbab4f2c01ef1e820cb3f1'), '2':('b560b0cce2ca74d3d499d73775152ab7',), '3':('d16e426e71fc29b1b55d0fbded99a473',), '4':('19c68066e414e08d17c86fc5c4acc949','c43354a7f7739508f76c538d5b3bce26'), '5':('4b9abf98e30a1475997ec770cbe5e702','2059b4aa95c7b3156b171255fa10bbdd'), '6':('804be4171d61f9cc10e9978c43b1d2a0','a41b091d4a11a318406a5a8bd3ed3837'), '7':('8adf951f4eea5f446f714214e101d555',), '8':('568135f3844213c30f2c7880be867d3d',), '9':('a3750995c511ea1492ac244421109e77','eeb3a8ba804f19380dfe94a91a37595b'), } color=(0,0,0) def __init__(self, page): img = page.doc.find("//img[@usemap='#cv']") res = page.browser.open(img.attrib['src']) MappedVirtKeyboard.__init__(self, StringIO(res.content), page.doc, img, self.color, 'href', convert='RGB') self.check_symbols(self.symbols, page.browser.responses_dirname) def check_color(self, pixel): for p in pixel: if p >= 0xd5: return False return True def get_symbol_coords(self, coords): # strip borders x1, y1, x2, y2 = coords return MappedVirtKeyboard.get_symbol_coords(self, (x1+10, y1+10, x2-10, y2-10)) def get_symbol_code(self, md5sum_list): for md5sum in md5sum_list: try: code = MappedVirtKeyboard.get_symbol_code(self,md5sum) except VirtKeyboardError: continue else: return ''.join(re.findall(r"'(\d+)'", code)[-2:]) raise VirtKeyboardError('Symbol not found') def get_string_code(self, string): code = '' for c in string: code += self.get_symbol_code(self.symbols[c]) return code class LoginPage(HTMLPage): def login(self, login, password): vk = VirtKeyboard(self) form = self.get_form('//form[@id="formulaire-login"]') code = vk.get_string_code(password) try: assert len(code)==10 except AssertionError: raise BrowserIncorrectPassword("Wrong number of character") form['accordirect.identifiant'] = login form['accordirect.code'] = code form.submit() class IndexPage(LoggedPage, HTMLPage): is_here = "//div[@id='situation']" @method class iter_accounts(ListElement): item_xpath = '//div[@id="situation"]//div[@class="synthese-produit"]' class item(ItemElement): klass = Account obj_currency = u'EUR' obj_type = Account.TYPE_CARD obj_label = Env('label') obj__num = Env('_num') obj_id = Env('id') obj_balance = Env('balance') def parse(self, el): self.env['label'] = CleanText('./h3/a')(self) or u'Carte Oney' self.env['_num'] = Attr('%s%s%s' % ('//option[contains(text(), "', Field('label')(self).replace('Ma ', ''), '")]'), 'value', default=u'')(self) self.env['id'] = Format('%s%s' % (self.page.browser.username, Field('_num')(self)))(self) # On the multiple accounts page, decimals are separated with dots, and separated with commas on single account page. amount_due = CleanDecimal('./p[@class = "somme-due"]/span[@class = "synthese-montant"]', default=None)(self) if amount_due is None: amount_due = CleanDecimal('./div[@id = "total-sommes-dues"]/p[contains(text(), "sommes dues")]/span[@class = "montant"]', replace_dots=True)(self) self.env['balance'] = - amount_due class OperationsPage(LoggedPage, HTMLPage): is_here = "//div[@id='releve-reserve-credit'] | //div[@id='operations-recentes'] | //select[@id='periode']" @pagination @method class iter_transactions(ListElement): item_xpath = '//table[@class="tableau-releve"]/tbody/tr[not(node()//span[@class="solde-initial"])]' flush_at_end = True def flush(self): # As transactions are unordered on the page, we flush only at end # the sorted list of them. return sorted(self.objects.itervalues(), key=lambda tr: tr.rdate, reverse=True) def store(self, obj): # It stores only objects with an ID. To be sure it works, use the # uid of transaction as object ID. obj.id = obj.unique_id(seen=self.env['seen']) return ListElement.store(self, obj) class credit(ItemElement): klass = Transaction obj_type = Transaction.TYPE_CARD obj_date = Transaction.Date('./td[1]') obj_raw = Transaction.Raw('./td[2]') obj_amount = Env('amount') def condition(self): self.env['amount'] = Transaction.Amount('./td[3]')(self.el) return self.env['amount'] > 0 class debit(ItemElement): klass = Transaction obj_type = Transaction.TYPE_CARD obj_date = Transaction.Date('./td[1]') obj_raw = Transaction.Raw('./td[2]') obj_amount = Env('amount') def condition(self): self.env['amount'] = Transaction.Amount('', './td[4]')(self.el) return self.env['amount'] < 0 def next_page(self): options = self.page.doc.xpath('//select[@id="periode"]//option[@selected="selected"]/preceding-sibling::option[1]') if options: data = {'numReleve':options[0].values(),'task':'Releve','process':'Releve','eventid':'select','taskid':'','hrefid':'','hrefext':''} return requests.Request("POST", self.page.url, data=data) ��������weboob-1.2/modules/onlinenet/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016264�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/onlinenet/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0020402�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OnlinenetModule __all__ = ['OnlinenetModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/onlinenet/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000003137�13034501105�0020325�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, ProfilPage, DocumentsPage class OnlinenetBrowser(LoginBrowser): BASEURL = 'https://console.online.net/en/' login = URL('login', LoginPage) profil = URL('account/edit', ProfilPage) documents = URL('bill/list', DocumentsPage) def do_login(self): self.login.go() self.page.login(self.username, self.password) if self.login.is_here(): raise BrowserIncorrectPassword @need_login def get_subscription_list(self): return self.profil.stay_or_go().get_list() @need_login def iter_documents(self, subscription): for b in self.documents.stay_or_go().get_bills(): yield b for d in self.documents.stay_or_go().get_documents(): yield d ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/onlinenet/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004755�13034501105�0020136�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object, NotAvailable from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import OnlinenetBrowser __all__ = ['OnlinenetModule'] class OnlinenetModule(Module, CapDocument): NAME = 'onlinenet' DESCRIPTION = u'Online.net' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='Identifiant'), ValueBackendPassword('password', label='Mot de passe')) BROWSER = OnlinenetBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) if document._url is NotAvailable: return return self.browser.open(document._url).content �������������������weboob-1.2/modules/onlinenet/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000006706�13034501105�0017746�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.filters.standard import CleanText, CleanDecimal, Env, Format, TableCell, Date from weboob.browser.elements import ListElement, ItemElement, TableElement, method from weboob.browser.filters.html import Attr from weboob.capabilities.bill import Bill, Document, Subscription from weboob.capabilities.base import NotAvailable class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form('//form[@id="login"]') form['_username'] = login form['_password'] = password form.submit() class ProfilPage(LoggedPage, HTMLPage): @method class get_list(ListElement): class item(ItemElement): klass = Subscription obj_subscriber = Format('%s %s', CleanText('//label[@for="form_firstname"]/../following-sibling::div'), CleanText('//label[@for="form_firstname"]/../following-sibling::div')) obj_id = Env('username') obj_label = obj_id def parse(self, el): self.env['username'] = self.page.browser.username class DocumentsPage(LoggedPage, HTMLPage): @method class get_bills(TableElement): item_xpath = '//h3[contains(text(), "bills")]/following-sibling::table//tr[position() > 1]' head_xpath = '//h3[contains(text(), "bills")]/following-sibling::table//tr/th' col_id = u'Id' col_date = u'Date' col_price = u'Total TTC' class item(ItemElement): klass = Bill obj_id = Format('%s_%s', Env('username'), CleanDecimal(TableCell('id'))) obj__url = Attr('.//a[contains(text(), "PDF")]', 'href', default=NotAvailable) obj_date = Date(CleanText(TableCell('date'))) obj_format = u"pdf" obj_label = Format('Facture %s', CleanDecimal(TableCell('id'))) obj_type = u"bill" obj_price = CleanDecimal(TableCell('price')) obj_currency = u"€" def condition(self): return CleanText(TableCell('id'))(self) != "No bills" def parse(self, el): self.env['username'] = self.page.browser.username @method class get_documents(ListElement): item_xpath = '//a[contains(@href, ".pdf")]' class item(ItemElement): klass = Document obj_id = Format('%s_%s', Env('username'), Env('docid')) obj__url = Attr('.', 'href') obj_format = u"pdf" obj_label = CleanText('.') obj_type = u"other" def parse(self, el): self.env['username'] = self.page.browser.username self.env['docid'] = re.sub('[^a-zA-Z0-9-_*.]', '', CleanText('.')(self)) ����������������������������������������������������������weboob-1.2/modules/onlinenet/test.py����������������������������������������������������������������0000664�0000000�0000000�00000001614�13034501105�0017617�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class OnlinenetTest(BackendTest): MODULE = 'onlinenet' def test_onlinenet(self): raise NotImplementedError() ��������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016735�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001433�13034501105�0021047�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import AloesModule __all__ = ['AloesModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/browser.py����������������������������������������������������������0000664�0000000�0000000�00000005311�13034501105�0020772�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages import LoginPage, HomePage, RentedPage, HistoryPage, BookedPage __all__ = ['AloesBrowser'] # Browser class AloesBrowser(Browser): PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] #DEBUG_HTTP = True DEBUG_HTTP = False PAGES = { 'http://.*/index.aspx': LoginPage, 'http://.*/index.aspx\?IdPage=1': HomePage, 'http://.*/index.aspx\?IdPage=45': RentedPage, 'http://.*/index.aspx\?IdPage=429': HistoryPage, 'http://.*/index.aspx\?IdPage=44': BookedPage } def __init__(self, baseurl, *args, **kwargs): self.BASEURL = baseurl Browser.__init__(self, *args, **kwargs) def is_logged(self): return self.page \ and not self.page.document.getroot().xpath('//input[contains(@id, "ctl00_ContentPlaceHolder1_ctl00_ctl08_ctl00_TextSaisie")]') #return True def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.is_on_page(HomePage): self.location('%s://%s/index.aspx' % (self.PROTOCOL, self.BASEURL), no_login=True) if not self.page.login(self.username, self.password) or \ not self.is_logged() or \ (self.is_on_page(LoginPage) and self.page.is_error()): raise BrowserIncorrectPassword() def get_rented_books_list(self): if not self.is_on_page(RentedPage): self.location('%s://%s/index.aspx?IdPage=45' % (self.PROTOCOL, self.BASEURL) ) return self.page.get_list() def get_booked_books_list(self): if not self.is_on_page(BookedPage): self.location('%s://%s/index.aspx?IdPage=44' % (self.PROTOCOL, self.BASEURL)) return self.page.get_list() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000004763�13034501105�0021102�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 9") ���DtEXtComment�CREATOR: gd-jpeg v1.0 (using IJG JPEG v80), default quality Gul�� #IDATx[[lxu06vTQqZJ QLb"U.yI ^Z+JQ)Aj4Q8Ę $-{awg߇왙3]̜ws`1O!�`'-Ͻ*[mCRju=omz 2{!X(/{?鶍%*z쇢:)p<^U,D:{Kz+n :ojj‘#0%09"[\5b:[õ)\Nv@Uh1fPҲ""w8ۇ?8+1=<9i`񙍓3v N:?.ǬYNǁ 0gFp{q? \YD\XG aIELZї~X1"=&71sysѲfp466btt admfdJ!$e)κ\+ھ WG CZ8'q5s׏ s=Z$uvu&rܹw10/ݳر}' `W;@tBCC ޯLHа4XFk' BcQwɂ qK3B.É'd0u/;[I$蘜|&sHC~jCO>�d2_ XCb,3Q cْ%/,co 8TX ;~Ifbm (}J'J Io.$mܰ)$I9 l问9l߾*K'}$+W fJJvd 1%nD21v΁X+V&Iշ'ޮ+O""@QG|ٽhim19y^0FˋK9`dJR~(œ?.{Ty&w*Sx#b;Zۻ1D nJ֥U[nuMMt7444`h @DqJR Xaqw^Ư(nPC-C+$ݢCt; A%XU\~ x@ď!46:Ή04۷>S/a/t}T-4ꬎ (gVO/E:曧eC5U`SLߌgg/}t,{I"¿mDD5 2Nyc^3B@g\XNV<!ߣNrZ[۰B?m[tnݲ9AlVl^lٝ"i2F{c DGGGW}/<tyTe<lPSUI?}v=^-f܉͘C譎0 ?\XyӕߠXJJ%E &k)WjS|J:mVՠNbH؉u@46&ce;O.%?رj22E(f=sj\@*qZ]!tC4dٚ)FQʞvkK0e2xP"(掀qA }uj nꍔXq0 ,YQ,}X(/um!,sÅΡѾ68rUoꌜs "BG Z۰o}3Pv{u yp]rlS:uYZZµk<f"™`-b-:??o#^[[k$G@ Ah �*ڃXmܻW@> k3{Nd!A:uo:-U &7Ut]`ݶ;X] S`i�`cMի#iSU$%.,,` ֮[t]wi~~実YFM 59"y{?Rd5dvX{} ##/Tت)s$~<\~=V[jP 0{ /bx9+FA_  똞jȪ cϖǶ8nyEŃ00q;{{_N�N�ȝEYٗ0pH8.|LӴ bvAF|r_бbG+_9KU#)`#Ձbq(ArP_c|42\֦!L����IENDB`�������������weboob-1.2/modules/opacwebaloes/module.py�����������������������������������������������������������0000664�0000000�0000000�00000004252�13034501105�0020577�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.library import CapBook from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import AloesBrowser __all__ = ['AloesModule'] class AloesModule(Module, CapBook): NAME = 'opacwebaloes' MAINTAINER = u'Jeremy Monnet' EMAIL = 'jmonnet@gmail.com' VERSION = '1.2' DESCRIPTION = 'Aloes Library software' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('login', label='Account ID', regexp='^\d{1,8}\w$'), ValueBackendPassword('password', label='Password of account'), Value('baseurl', label='Base URL') ) BROWSER = AloesBrowser def create_default_browser(self): return self.create_browser(self.config['baseurl'].get(), self.config['login'].get(), self.config['password'].get()) def get_rented(self): for book in self.browser.get_rented_books_list(): yield book def get_booked(self): for book in self.browser.get_booked_books_list(): yield book def iter_books(self): for book in self.get_booked(): yield book for book in self.get_rented(): yield book def get_book(self, _id): raise NotImplementedError() def search_books(self, _string): raise NotImplementedError() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/pages.py������������������������������������������������������������0000664�0000000�0000000�00000012517�13034501105�0020414�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import date from weboob.capabilities.library import Book from weboob.deprecated.browser import Page, BrowserUnavailable from weboob.deprecated.mech import ClientForm class SkipPage(Page): pass class HomePage(Page): pass def txt2date(s): return date(*reversed([int(x) for x in s.split(' ')[-1].split('/')])) class RentedPage(Page): # TODO, table limited to 20 items, need to use pagination def get_list(self): for book in self.iter_books('//tr[contains(@id, "ctl00_ContentPlaceHolder1_ctl00_ctl07_COMPTE_PRET_1_1_GrillePrets_ctl00__")]', 1): book.late = False yield book for book in self.iter_books('//tr[contains(@id, "ctl00_ContentPlaceHolder1_ctl00_ctl08_COMPTE_RETARD_0_1_GrilleRetards_ctl00__")]', 0): book.late = True yield book def iter_books(self, el, start): for tr in self.document.getroot().xpath(el): book = Book(tr[start].text) book.name = tr[start+3].text book.author = tr[start+4].text book.date = txt2date(tr[start+5].text) yield book class HistoryPage(Page): pass class BookedPage(Page): # TODO, table limited to 20 items, need to use pagination def get_list(self): for tr in self.document.getroot().xpath('//tr[contains(@id, "ctl00_ContentPlaceHolder1_ctl00_ctl09_COMPTE_INFOS_0_GrilleInfos_ctl00__0")]'): username=tr[1].text+"_"+tr[0].text for i, tr in enumerate(self.document.getroot().xpath('//tr[contains(@id, "ctl00_ContentPlaceHolder1_ctl00_ctl10_COMPTE_RESA_1_1_GrilleResas_ctl00__")]')): book = Book('%s%d' % (username, i)) # if all the books booked are available, there are only 7 columns. # if (at least ?) one book is still not available, yous can cancel, and the first column does contain the checkbox. So 8 columns. if (len(tr) == 7): start = 2 if (len(tr) == 8): start = 3 book.name = tr[start].text book.author = tr[start+1].text book.date = txt2date(tr[start+3].text) book.late = False yield book class LoginPage(Page): def login(self, login, passwd): self.browser.select_form(predicate=lambda x: x.attrs.get('id','')=='aspnetForm') self.browser.form.set_all_readonly(False) self.browser['ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$TextSaisie'] = login self.browser['ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$TextPass'] = passwd self.browser['ctl00_ScriptManager1_TSM']="%3B%3BSystem.Web.Extensions%2C%20Version%3D1.0.61025.0%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D31bf3856ad364e35%3Afr-FR%3A1f0f78f9-0731-4ae9-b308-56936732ccb8%3Aea597d4b%3Ab25378d2%3BTelerik.Web.UI%2C%20Version%3D2009.3.1314.20%2C%20Culture%3Dneutral%2C%20PublicKeyToken%3D121fae78165ba3d4%3Afr-FR%3Aec1048f9-7413-49ac-913a-b3b534cde186%3A16e4e7cd%3Aed16cbdc%3Af7645509%3A24ee1bba%3A19620875%3A874f8ea2%3A33108d14%3Abd8f85e4" self.browser.controls.append(ClientForm.TextControl('text', 'RadAJAXControlID', {'value': ''})) self.browser['RadAJAXControlID']="ctl00_ContentPlaceHolder1_ctl00_ctl04_ctl00_RadAjaxPanelConnexion" self.browser.controls.append(ClientForm.TextControl('text', 'ctl00$ScriptManager1', {'value': ''})) self.browser['ctl00$ScriptManager1']="ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$RadAjaxPanelConnexionPanel|" self.browser.controls.append(ClientForm.TextControl('text', '__EVENTTARGET', {'value': ''})) self.browser.controls.append(ClientForm.TextControl('text', '__EVENTARGUMENT', {'value': ''})) self.browser.controls.append(ClientForm.TextControl('text', 'ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$btnImgConnexion.x', {'value': ''})) self.browser['ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$btnImgConnexion.x']="76" self.browser.controls.append(ClientForm.TextControl('text', 'ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$btnImgConnexion.y', {'value': ''})) self.browser['ctl00$ContentPlaceHolder1$ctl00$ctl04$ctl00$btnImgConnexion.y']="10" try: self.browser.submit() except BrowserUnavailable: # Login is not valid return False return True def is_error(self): for text in self.document.find('body').itertext(): text=text.strip() # Login seems valid, but password does not needle='Echec lors de l\'authentification' if text.startswith(needle.decode('utf-8')): return True return False ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opacwebaloes/test.py�������������������������������������������������������������0000664�0000000�0000000�00000001550�13034501105�0020267�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class AloestTest(BackendTest): MODULE = 'aloes' def test_aloes(self): pass ��������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015733�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001435�13034501105�0020047�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OpenEDXModule __all__ = ['OpenEDXModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000005472�13034501105�0020000�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.pages import RawPage, JsonPage, HTMLPage from weboob.browser.exceptions import ClientError from weboob.exceptions import BrowserIncorrectPassword class LoginPage(HTMLPage): def login(self, username, password): try: self.browser.login_result.open(data = { "email": username, "password": password, "remember": "false" }) except ClientError as e: if e.response.status_code == 403: raise BrowserIncorrectPassword() else: raise self.logged = True class OpenEDXBrowser(LoginBrowser): login = URL('/login', LoginPage) login_result = URL("/user_api/v1/account/login_session/", RawPage) threads = URL(r'/courses/(?P<course>.+)/discussion/forum/\?ajax=1&page=(?P<page>\d+)&sort_key=date&sort_order=desc', JsonPage) messages = URL(r'/courses/(?P<course>.+)/discussion/forum/(?P<topic>.+)/threads/(?P<id>.+)\?ajax=1&resp_skip=(?P<skip>\d+)&resp_limit=100', JsonPage) thread = URL(r'/courses/(?P<course>.+)/discussion/forum/(?P<topic>.+)/threads/(?P<id>.+)', HTMLPage) def __init__(self, url, course, *args, **kwargs): self.BASEURL = url self.course = course LoginBrowser.__init__(self, *args, **kwargs) def prepare_request(self, req): token = self.session.cookies.get("csrftoken") if token: req.headers.setdefault("X-CSRFToken", token) if self.threads.match(req.url) or self.messages.match(req.url): req.headers.setdefault("X-Requested-With", "XMLHttpRequest") return LoginBrowser.prepare_request(self, req) def do_login(self): self.login.stay_or_go() self.page.login(self.username, self.password) @need_login def get_threads(self, page=1): return self.threads.open(course = self.course, page = page) @need_login def get_thread(self, topic, id, skip): return self.messages.open(course = self.course, topic = topic, id = id, skip = skip) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000006037�13034501105�0020074�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME  2(���iTXtComment�����Created with GIMPd.e�� IDATxYipE~c\39fL<@FUkUQHe)JEEqQeEWEŒU�E$\c\ѽ?d2TT}_O)B )B )B )B 29̛os�@ W @/ Yg{Z>KʗBf2jϜ۵]< H(ZmO>Spx(7@: fїdt|g'VjN/MB�Ӫ�p0{q/i1Q}<]NOB!zZB%kIo'}BN�%Id6I%@'~?:!q 3 4jN)׭0_{HVm@ x_V[J%i@omr>bnXw]_s験IY 0}tٶPsږ݁S!8lXY=~Ϻkk�zx�XrǑVU λn"C-\ͺ˧%Wż`,ṕIg1WLgMgq@ gzn>Ɠ�xvGf) T�=tzE>tkVxS]kM挙ZE)6qcWD.X.^ؐ߱iD*w�\r?Ҩ� qB9ZNuri3lNBv$I}DKE 37RuuVXl\' ހN\S>Åv@# vAbБ*\WOX]h`7'8:er QV6mY>yT68+W,{� |hs\W|7\yCW޲Fktoy|e ):e4%w74t8HQ ��3eR}#5IwOE^z*3%l @H`h*W 0״n3uVk^ Ι(I6BH,"ɫnſ+Zk5g Yc*"|s#`eџH x̎j#:ty_jJ:7،A>)e�5M wA*t@8Gggf,|?I4\$+:%}ӌCFnU>ŘGؗrGXJǼgȎ0<xxe{u_gd_Ptu.#۔ѕv D|x]'V8Je볍g @iEOt<K}C"�jۮ-jp16Gp]45xxщ|o[[ `p=>%xRCɐ�Zv2YJjsk0k^p!5S^u�=-m!}ۦ$+Gl_9$PV@O@HfiMn%,Xpdf}28rP)EH-EqO}Z2%_ɚjZg邱�Dx+LIiς8~{@ѕNJ݅w7n/y ,\I-Yy95guY@ÃM I10"�S:i'']}[D/眡Rُܱ  >&S9"D* nT$ʵLC/1L@(_ѫ"0]ɾ \?jhR|_ 0Eo;3|-jM9)9w_ >i_.C.O"('j ]Q �K6JASm Dל+(kT[6Ē/z$7#%IFlg⎡�&#:ss&U5ԒybJ�p,zXgO}s)(37g T} �zAKeہS"},ȥkK�O9'%T IӴOmnjj<^8&L40/*..yO{,lSXC3E98dx?`'<|۩\ 9ANp>^)ecaX"_ 'Rć>P"I0f:{^\suS*o5-ZC&7ίޝԲҎ$mrY7GKj0 7(+7e"EޱRҮ 2u3 zc8h2e@N쨦i}s)--QeҒ%IHټd4G0#P8 6 n`Z8𷏆>`b޶ ] &*Zbw4O96I1CLE$9* \g rX |חCAu*KẮAuaI ݌]^N.HC=6�нmVU XDc 5ŖˌuB4nֽKޮsޫ| KAsιY=2ƒ�A$BH6x-4~0\Ӻ[O4[i+R @FvϪY~c)4ɯfZLkPfO-]2cMP%Soyk}~VҾ.Mw`cco< �@ _ow_xHx:־0D/\aMcT 9x&z`#oky07׿9f:fˮZme8�PQnkU+^=U#<.iՄ=-,uc8&Oq!RH!RH!RH!RH׆GufU ����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/module.py����������������������������������������������������������������0000664�0000000�0000000�00000014231�13034501105�0017573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import dateutil.parser from subprocess import Popen, PIPE from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.messages import CapMessages, Thread, Message from .browser import OpenEDXBrowser __all__ = ['OpenEDXModule'] class OpenEDXModule(Module, CapMessages): NAME = 'openedx' DESCRIPTION = u'Discussions on OpenEDX-powered coursewares' MAINTAINER = u'Simon Lipp' EMAIL = 'laiquo@hwold.net' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), Value('url', label='Site URL', default='https://courses.edx.org/'), Value('course', label='Course ID', default='edX/DemoX.1/2014')) BROWSER = OpenEDXBrowser STORAGE = {'seen': {}} def __init__(self, *args, **kwargs): Module.__init__(self, *args, **kwargs) def pandoc_formatter(text): return Popen(["pandoc", "-f", "markdown", "-t", "html", "--mathml", "-"], stdin=PIPE, stdout=PIPE).communicate(text.encode('utf-8'))[0].decode('utf-8') try: from markdown import Markdown except ImportError: Markdown = None self.default_flags = Message.IS_HTML try: Popen(["pandoc", "-v"], stdout=PIPE, stderr=PIPE).communicate() self.formatter = pandoc_formatter except OSError: if Markdown: self.formatter = Markdown().convert else: self.formatter = (lambda text: text) self.default_flags = 0 def create_default_browser(self): return self.create_browser(self.config['url'].get(), self.config['course'].get(), self.config['username'].get(), self.config['password'].get()) def _build_thread(self, data): thread = Thread("%s.%s" % (data["commentable_id"], data["id"])) thread.title = data["title"] thread.date = dateutil.parser.parse(data["created_at"]) thread.url = self.browser.thread.build(course=self.browser.course, topic=data["commentable_id"], id=data["id"]) thread.root = self._build_message(data, thread) thread._messages_count = data["comments_count"] + 1 return thread def _build_message(self, data, thread, parent = None): flags = self.default_flags if data["id"] not in self.storage.get("seen", thread.id, default=[]): flags |= Message.IS_UNREAD message = Message(thread = thread, id = data["id"], title = (parent and "Re: %s" or "%s") % thread.title, sender = data.get("username"), receivers = None, date = dateutil.parser.parse(data["created_at"]), content = self.formatter(data["body"]), flags = flags, parent = parent, url = thread.url) self._append_children(data, message, thread) return message def _append_children(self, data, message, thread): if "endorsed_responses" in data or "children" in data or "non_endorsed_responses" in data: message.children = [] for child in data.get("endorsed_responses", []) + data.get("children", []) + data.get('non_endorsed_responses', []): message.children.append(self._build_message(child, thread, message)) def fill_message(self, message, fields): # The only unfilled messages are the root messages of threads returned # by iter_threads(). Only `children` in unfilled. if 'children' in fields and message.thread.root.id == message.id: message.children = self.get_thread(message.id).root.children return message #### CapMessages ############################################## def get_thread(self, id): topic, id = id.rsplit(".", 1) thread = None skip = 0 while True: data = self.browser.get_thread(topic, id, skip).doc["content"] if thread is None: thread = self._build_thread(data) else: self._append_children(data, thread.root, thread) if data["resp_skip"] + data["resp_limit"] >= data["resp_total"]: return thread else: skip += 100 def iter_threads(self): page = 1 while True: tlist = self.browser.get_threads(page).doc for data in tlist["discussion_data"]: yield self._build_thread(data) if tlist["page"] < tlist["num_pages"]: page += 1 else: break def iter_unread_messages(self): for thread in self.iter_threads(): if thread._messages_count > len(self.storage.get('seen', thread.id, default=[])): thread = self.get_thread(thread.id) for m in thread.iter_all_messages(): if m.flags & m.IS_UNREAD: yield m def set_message_read(self, message): thread_seen = self.storage.get('seen', message.thread.id, default=[]) thread_seen.append(message.id) self.storage.set('seen', message.thread.id, thread_seen) self.storage.save() OBJECTS = {Message: fill_message} �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/openedx/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002407�13034501105�0017267�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class OpenEDXTest(BackendTest): MODULE = 'openedx' def test_openedx(self): thread = next(self.backend.iter_threads()) thread = self.backend.get_thread(thread.id) self.assertTrue(thread.id) self.assertTrue(thread.title) self.assertTrue(thread.url) self.assertTrue(thread.root.id) self.assertTrue(thread.root.content) self.assertTrue(thread.root.children) self.assertTrue(thread.root.url) self.assertTrue(thread.root.date) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017171�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000001447�13034501105�0021310�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OpensubtitlesModule __all__ = ['OpensubtitlesModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/browser.py���������������������������������������������������������0000664�0000000�0000000�00000004130�13034501105�0021224�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from weboob.applications.suboob.suboob import LANGUAGE_CONV from .pages import SubtitlesPage, SearchPage, SubtitlePage __all__ = ['OpensubtitlesBrowser'] class OpensubtitlesBrowser(Browser): DOMAIN = 'www.opensubtitles.org' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://www.opensubtitles.org.*search2/sublanguageid.*moviename.*': SearchPage, 'http://www.opensubtitles.org.*search/sublanguageid.*idmovie.*': SubtitlesPage, 'http://www.opensubtitles.org.*search/imdbid.*/sublanguageid.*/moviename.*': SubtitlesPage, 'http://www.opensubtitles.org.*subtitles/[0-9]*/.*': SubtitlePage } def iter_subtitles(self, language, pattern): lang = LANGUAGE_CONV[language] self.location('http://www.opensubtitles.org/search2/sublanguageid-%s/moviename-%s' % ( lang, pattern.encode('utf-8'))) assert self.is_on_page(SearchPage) or self.is_on_page(SubtitlesPage) or self.is_on_page(SubtitlePage) return self.page.iter_subtitles() def get_subtitle(self, id): try: self.location('http://www.opensubtitles.org/subtitles/%s' % id) except BrowserHTTPNotFound: return if self.is_on_page(SubtitlePage): return self.page.get_subtitle() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/favicon.png��������������������������������������������������������0000664�0000000�0000000�00000005567�13034501105�0021341�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME!m�� IDATxypU?s_^ŽIXQV"u\h:ک3mqcHY8cbTpP#Nk[ť,Z-A^K}y!I!g&r=QUf[7��\-p1 Q@b&clwA h4`8Ʝh^H6~=g�9o8V$~ԑlEPCG. �<ȡf`D9CU} W-HowU˳I;S%&3oL -5Mv*1 oY&b>H@$ ( Yhf\zdzi1d;Geh! &UƊ.CN+tH~hjK1S~b"n$Z/||mY-iɚVҀ#]Npohtoxӓ',}'vni5٣d1|9r8gS/nk:Rk˶7a cS"8/qIȑ&7/AV;4 �-.bd$o'/GdMJ͝Wzt3? xِQ2sdʼ)cyhxӲR8C_S}n0ݟJnw evYHls@M�⡨}0ɺ[G9c1Жv ]rs)2i^pq@9I^1WM|/3|!XXXIơ17�Jn?%2: Mp~`] 75ԓ6 |AS+4ml=^O7/Wɴz-~x% V dDŸ:>ѽ] _:kPz"H `T᜛}P-?إ5 a6 i:imn@~ne.TY\ ʃ٣p4ɫx#Jѿ%xʜ{g9doeCl׆]EQ)&*Qk7??-N$x1_�_:3뒍7G_壵5z1|-m{ Eu۷}"8XMG/K�8eO^}io^!t?$BG؟詧.O�ekΚ$M_4_80iOoJé"{C>h_+h;pܛL;Gű~z= AMf-g4ǰ4{#MC+x9[\icڅ 3Cyh[ծʏ] Ѐ5KRd!Ap vKIg\lNGs1A${.iE&eP31Ӈ@D*!:52$v<P(_yhb|P;eD0u \W++<1^Pnp0V-$�䊐 ܺF*By|سF�=`;;̪$Sp_m7R6QV ǻ+p`ϸף88O|N LXx5t�z?VyP#T$[mLVVCEXb"$gn\9㰳 �4s0? 8;Wl*#Dӓ_p"`y3<Q X[J|oU]qKl4-q(p%Ka�V-q8vca" Ƈ #Ҝ9FYyQC?][- S@?[b `;N6WgV9kb ǁz:Jx)+Rr܉U6uVP*He< <o,J [lU7aj)C:BmZq6„a kY$3[,'hֱj@ag_QRBK V8#^]o<MG*k>Fv2CEڈ%qu:�RإJBk1<9�vm6n۶d;Ʈ$Rŕ�.n�[ jp;J[Xa]I>]Y#W;T kvu4NkKՃJ!AS%*@eܖ k^x 7#^D_R<PoNpx �8}iAӀPkLwym L:Fy]#³.c$鶀HEDyI<-ˬi7烵6>'OC-"۷~u3fO;P9 k*Ϻ3n]wT�#>^}cxV{:<Po /` 4F4X$߈@eX籤]-N06[΋6O~t.FS Fx=r?jFu WQ B&pcD9EprQ{U8*O:Jܗ,`*wˍȬ4 ٶnj�B/5,;pr;C+kԊ>ʭ+ \yi&3F-eEj[Nw6O׽o�I5j�ԦV PЫ?BD+3\fOli5LdnV*k>wSY#C>Wg-2ʻTrZ8� E G�4e'5}Qa7ʂm(g[ОNG;Hj=޺Hn� O<����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/module.py����������������������������������������������������������0000664�0000000�0000000�00000004052�13034501105�0021031�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.subtitle import CapSubtitle, LanguageNotSupported, Subtitle from weboob.applications.suboob.suboob import LANGUAGE_CONV from weboob.tools.backend import Module from .browser import OpensubtitlesBrowser from urllib import quote_plus __all__ = ['OpensubtitlesModule'] class OpensubtitlesModule(Module, CapSubtitle): NAME = 'opensubtitles' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Opensubtitles subtitle website' LICENSE = 'AGPLv3+' BROWSER = OpensubtitlesBrowser def get_subtitle(self, id): return self.browser.get_subtitle(id) def get_subtitle_file(self, id): subtitle = self.browser.get_subtitle(id) if not subtitle: return None return self.browser.openurl(subtitle.url.encode('utf-8')).read() def iter_subtitles(self, language, pattern): if language not in LANGUAGE_CONV.keys(): raise LanguageNotSupported() return self.browser.iter_subtitles(language, quote_plus(pattern.encode('utf-8'))) def fill_subtitle(self, subtitle, fields): if 'description' in fields: sub = self.get_subtitle(subtitle.id) subtitle.description = sub.description return subtitle OBJECTS = { Subtitle: fill_subtitle, } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/pages.py�����������������������������������������������������������0000664�0000000�0000000�00000013445�13034501105�0020651�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.subtitle import Subtitle from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.deprecated.browser import Page from weboob.applications.suboob.suboob import LANGUAGE_CONV class SearchPage(Page): """ Page which contains results as a list of movies """ def iter_subtitles(self): tabresults = self.parser.select(self.document.getroot(), 'table#search_results') if len(tabresults) > 0: table = tabresults[0] # for each result line, explore the subtitle list page to iter subtitles for line in self.parser.select(table, 'tr'): links = self.parser.select(line, 'a') if len(links) > 0: a = links[0] url = a.attrib.get('href', '') if "ads2.opensubtitles" not in url: self.browser.location("http://www.opensubtitles.org%s" % url) assert self.browser.is_on_page(SubtitlesPage) or self.browser.is_on_page(SubtitlePage) # subtitles page does the job for subtitle in self.browser.page.iter_subtitles(): yield subtitle class SubtitlesPage(Page): """ Page which contains several subtitles for a single movie """ def iter_subtitles(self): tabresults = self.parser.select(self.document.getroot(), 'table#search_results') if len(tabresults) > 0: table = tabresults[0] # for each result line, get the data # why following line doesn't work all the time (for example 'search fr sopranos guy walks' ? # for line in self.parser.select(table,'tr'): for line in table.getiterator('tr'): # some tr are useless, specially ads if line.attrib.get('id', '').startswith('name'): yield self.get_subtitle_from_line(line) def get_subtitle_from_line(self, line): cells = self.parser.select(line, 'td') if len(cells) > 0: links = self.parser.select(line, 'a') a = links[0] name = u" ".join(a.text.strip().split()) first_cell = cells[0] spanlist = self.parser.select(first_cell, 'span') if len(spanlist) > 0: long_name = spanlist[0].attrib.get('title', '') else: texts = first_cell.itertext() long_name = texts.next() long_name = texts.next() if "Download at 25" in long_name: long_name = "---" name = "%s (%s)" % (name, long_name) second_cell = cells[1] link = self.parser.select(second_cell, 'a', 1) lang = link.attrib.get('href', '').split('/')[-1].split('-')[-1] for lshort, llong in LANGUAGE_CONV.items(): if lang == llong: lang = unicode(lshort) break nb_cd = int(cells[2].text.strip().lower().replace('cd', '')) cell_dl = cells[4] href = self.parser.select(cell_dl, 'a', 1).attrib.get('href', '') url = unicode('http://www.opensubtitles.org%s' % href) id = href.split('/')[-1] subtitle = Subtitle(id, name) subtitle.url = url subtitle.language = lang subtitle.nb_cd = nb_cd subtitle.description = NotLoaded return subtitle class SubtitlePage(Page): """ Page which contains a single subtitle for a movie """ def get_subtitle(self): desc = NotAvailable a = self.parser.select(self.document.getroot(), 'a#bt-dwl-bt', 1) id = a.attrib.get('data-product-id', '') m = re.match('Download \((\w+)\)', self.parser.tocleanstring(a)) if m: ext = m.group(1) else: ext = u'zip' url = unicode('http://www.opensubtitles.org/en/subtitleserve/sub/%s' % id) link = self.parser.select(self.document.getroot(), 'link[rel=bookmark]', 1) title = unicode(link.attrib.get('title', '')) nb_cd = int(title.lower().split('cd')[0].split()[-1]) lang = unicode(title.split('(')[1].split(')')[0]) file_names = self.parser.select(self.document.getroot(), "img[title~=filename]") if len(file_names) > 0: file_name = file_names[0].getparent().text_content() file_name = ' '.join(file_name.split()) desc = u'files :' for f in file_names: desc_line = f.getparent().text_content() desc += '\n'+' '.join(desc_line.split()) name = unicode('%s (%s)' % (title, file_name)) subtitle = Subtitle(id, name) subtitle.url = url subtitle.ext = ext for lshort, llong in LANGUAGE_CONV.items(): if lang == llong: lang = unicode(lshort) break subtitle.language = lang subtitle.nb_cd = nb_cd subtitle.description = desc return subtitle def iter_subtitles(self): yield self.get_subtitle() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/opensubtitles/test.py������������������������������������������������������������0000664�0000000�0000000�00000002452�13034501105�0020525�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from random import choice class OpensubtitlesTest(BackendTest): MODULE = 'opensubtitles' def test_subtitle(self): lsub = [] subtitles = self.backend.iter_subtitles('fr', 'spiderman') for i in range(5): subtitle = subtitles.next() lsub.append(subtitle) assert subtitle.url.startswith('http') assert (len(lsub) > 0) # get the file of a random sub if len(lsub): subtitle = choice(lsub) self.backend.get_subtitle_file(subtitle.id) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015544�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000000075�13034501105�0017657�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import OrangeModule __all__ = ['OrangeModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016466�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020565�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000005005�13034501105�0020523�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, ProfilPage, BillsPage __all__ = ['OrangeBillBrowser'] class OrangeBillBrowser(LoginBrowser): loginpage = URL('https://id.orange.fr/auth_user/bin/auth_user.cgi', LoginPage) profilpage = URL('https://espaceclientv3.orange.fr/\?page=profil-infosPerso', 'https://espaceclientv3.orange.fr/ajax.php', ProfilPage) billspage = URL('https://m.espaceclientv3.orange.fr/\?page=factures-archives', 'https://.*.espaceclientv3.orange.fr/\?page=factures-archives', 'https://espaceclientv3.orange.fr/\?page=factures-archives', 'https://espaceclientv3.orange.fr/\?page=facture-telecharger', 'https://espaceclientv3.orange.fr/maf.php', 'https://espaceclientv3.orange.fr/\?idContrat=(?P<subid>.*)&page=factures-archives', BillsPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.loginpage.stay_or_go().login(self.username, self.password) self.billspage.go() if self.loginpage.is_here(): raise BrowserIncorrectPassword() def get_nb_remaining_free_sms(self): raise NotImplementedError() def post_message(self, message, sender): raise NotImplementedError() @need_login def get_subscription_list(self): return self.billspage.go().get_list() @need_login def iter_documents(self, subscription): # Only if muti accounts subid = subscription.id if subscription._multi else "" return self.billspage.go(subid=subid).get_documents(subid=subscription.id) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/pages/���������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017565�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/pages/__init__.py����������������������������������������������������0000664�0000000�0000000�00000001533�13034501105�0021700�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .login import LoginPage from .bills import ProfilPage, BillsPage __all__ = ['LoginPage', 'ProfilPage', 'BillsPage'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/pages/bills.py�������������������������������������������������������0000664�0000000�0000000�00000007117�13034501105�0021252�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import json from weboob.browser.pages import HTMLPage, LoggedPage from weboob.capabilities.bill import Subscription from weboob.browser.elements import ListElement, ItemElement, SkipItem, method from weboob.browser.filters.standard import CleanDecimal, CleanText, Env, Format, Regexp, Date, Async, AsyncLoad, BrowserURL from weboob.browser.filters.html import Attr, Link from weboob.capabilities.base import NotAvailable from weboob.capabilities.bill import Bill class ProfilPage(HTMLPage): pass class BillsPage(LoggedPage, HTMLPage): @method class get_list(ListElement): def parse(self, el): if self.page.doc.xpath('//div[@ecareurl]'): self.item_xpath = '//div[@ecareurl]' class item(ItemElement): klass = Subscription load_details = BrowserURL('profilpage') & AsyncLoad obj_subscriber = Env('subscriber') obj_label = Env('subid') obj_id = obj_label obj__multi = Env('multi') def parse(self, el): subscriber = Async('details', CleanText(u'//span[contains(text(), "prénom / nom")]/following-sibling::span[1]'))(self) self.env['subscriber'] = subscriber if subscriber else \ Async('details', Format('%s %s %s', \ CleanText(u'//*[contains(text(), "civilité")]/following-sibling::*[1]'), \ CleanText(u'//*[contains(text(), "prénom")]/following-sibling::*[1]'), \ CleanText(u'//*[text() = "nom :"]/following-sibling::*[1]')))(self) subid = Regexp(Attr('.', 'ecareurl', default="None"), 'idContrat=(\d+)', default=None)(self) self.env['subid'] = subid if subid else self.page.browser.username self.env['multi'] = True if subid else False # Prevent from available account but no added in customer area if subid and not json.loads(self.page.browser.open(Attr('.', 'ecareurl')(self)).content)['html']: raise SkipItem() @method class get_documents(ListElement): item_xpath = '//ul[has-class("factures")]/li' class item(ItemElement): klass = Bill obj_id = Format('%s_%s', Env('subid'), CleanDecimal(CleanText('.//span[has-class("date")]'))) obj_url = Link('.//span[has-class("pdf")]/a', default=NotAvailable) obj_date = Date(CleanText('.//span[has-class("date")]'), dayfirst=True) obj_label = CleanText('.//span[has-class("date")]') obj_format = u"pdf" obj_type = u"bill" obj_price = CleanDecimal('span[@class="montant"]', replace_dots=True) def obj_currency(self): return Bill.get_currency(CleanText('span[@class="montant"]')(self)) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/bill/pages/login.py�������������������������������������������������������0000664�0000000�0000000�00000001754�13034501105�0021256�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form(xpath='//form[@id="AuthentForm"]') form['credential'] = login form['password'] = password form.submit() ��������������������weboob-1.2/modules/orange/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000005276�13034501105�0017613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. #~ from .pages.compose import ClosePage, ComposePage, ConfirmPage, SentPage #~ from .pages.login import LoginPage from .pages import LoginPage, ComposePage, ConfirmPage from weboob.deprecated.browser import Browser, BrowserIncorrectPassword __all__ = ['OrangeBrowser'] class OrangeBrowser(Browser): DOMAIN = 'orange.fr' PAGES = { 'http://id.orange.fr/auth_user/bin/auth_user.cgi.*': LoginPage, 'http://id.orange.fr/auth_user/bin/auth0user.cgi.*': LoginPage, 'https://id.orange.fr/auth_user/bin/auth_user.cgi.*': LoginPage, 'https://id.orange.fr/auth_user/bin/auth0user.cgi.*': LoginPage, 'https://authweb.orange.fr/auth_user/bin/auth_user.cgi.*': LoginPage, 'https://authweb.orange.fr/auth_user/bin/auth0user.cgi.*': LoginPage, 'http://smsmms1.orange.fr/./Sms/sms_write.php.*' : ComposePage, 'http://smsmms1.orange.fr/./Sms/sms_write.php?command=send' : ConfirmPage, 'https://smsmms1.orange.fr/./Sms/sms_write.php.*' : ComposePage, 'https://smsmms1.orange.fr/./Sms/sms_write.php?command=send' : ConfirmPage, } def get_nb_remaining_free_sms(self): self.location("http://smsmms1.orange.fr/M/Sms/sms_write.php") return self.page.get_nb_remaining_free_sms() def home(self): self.location("http://smsmms1.orange.fr/M/Sms/sms_write.php") def is_logged(self): self.location("http://smsmms1.orange.fr/M/Sms/sms_write.php", no_login=True) return not self.is_on_page(LoginPage) def login(self): if not self.is_on_page(LoginPage): self.location('https://authweb.orange.fr/auth_user/bin/auth_user.cgi?url=http://www.orange.fr', no_login=True) self.page.login(self.username, self.password) if not self.is_logged(): raise BrowserIncorrectPassword() def post_message(self, message, sender): if not self.is_on_page(ComposePage): self.home() self.page.post_message(message, sender) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000002216�13034501105�0017700�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME  ɲ���tEXtComment�Created with GIMPW��IDATxklUm-jb0񁦑JVh(hXdnTRJAC4[A#4hR,$QQR)-Kqvvs=sߞ{ͬO#Xr�P� @(��P� @(��P� @(��P� @(��P�W\uٯ _gڿ al]`-0yO�5pn־N= ǜ6,=L P MqѰ#] ƧΓѳ^ xSխ@�j U[`Vs.fMԗLUQh 1ˡ�sg�T;e/x C Kk ѐ,{7ڋ/,BDelODl{ tpjomwM$?g_/5E&csI<}, =,ފv a}O'<qxdw#iZi/.} r\RS]o֟yCuO^޳h9դ'K߰y]ޗwvA0=Y'Eo ~^&R�L,sF{é3aI数x0>s.݁ [<#``Ez!Pig~؝f>m }sPw^Za@QYf#rEDDdcFh:/bǛb4]"vr9dFG2%ϱ;nx6l'cpdd':Lz`A{_K"#﷧6_ԑqgK[~{ O`50;ojіo7KiCMVqk8B~X8ďHdhH= :EV{,n˲:V~ V_ zaOexi1� Ȍs����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000010532�13034501105�0017404�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.messages import CantSendMessage, CapMessages, CapMessagesPost from weboob.capabilities.base import find_object, NotAvailable from weboob.capabilities.account import CapAccount, StatusField from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import OrangeBrowser from .bill.browser import OrangeBillBrowser __all__ = ['OrangeModule'] # We need to have a switcher, CapMessages use a browser1 and # CapDocument use a browser2 # This will be remove when CapMessages use a browser2 def browser_switcher(b): def set_browser(func): def func_wrapper(*args, **kwargs): self = args[0] if self._browser is None or type(self._browser) != b: self.BROWSER = b try: self._browser = self._browsers[b] except KeyError: self._browsers[b] = self.create_default_browser() self._browser = self._browsers[b] return func(*args, **kwargs) return func_wrapper return set_browser class OrangeModule(Module, CapAccount, CapMessages, CapMessagesPost, CapDocument): NAME = 'orange' MAINTAINER = u'Lucas Nussbaum' EMAIL = 'lucas@lucas-nussbaum.net' VERSION = '1.2' DESCRIPTION = 'Orange French mobile phone provider' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('login', label='Login'), ValueBackendPassword('password', label='Password'), Value('phonenumber', label='Phone number', default='') ) ACCOUNT_REGISTER_PROPERTIES = None BROWSER = OrangeBrowser def __init__(self, *args, **kwargs): self._browsers = dict() super(OrangeModule, self).__init__(*args, **kwargs) def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) @browser_switcher(OrangeBrowser) def get_account_status(self): return (StatusField('nb_remaining_free_sms', 'Number of remaining free SMS', self.browser.get_nb_remaining_free_sms()),) @browser_switcher(OrangeBrowser) def post_message(self, message): if not message.content.strip(): raise CantSendMessage(u'Message content is empty.') self.browser.post_message(message, self.config['phonenumber'].get()) @browser_switcher(OrangeBillBrowser) def iter_subscription(self): return self.browser.get_subscription_list() @browser_switcher(OrangeBillBrowser) def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) @browser_switcher(OrangeBillBrowser) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) @browser_switcher(OrangeBillBrowser) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) @browser_switcher(OrangeBillBrowser) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) if document.url is NotAvailable: return return self.browser.open(document.url).content ����������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/pages/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016643�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/pages/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001543�13034501105�0020757�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .login import LoginPage from .compose import ComposePage, ConfirmPage __all__ = ['LoginPage', 'ComposePage', 'ConfirmPage'] �������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/pages/compose.py����������������������������������������������������������0000664�0000000�0000000�00000004237�13034501105�0020670�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.messages import CantSendMessage from weboob.deprecated.browser import Page class ConfirmPage(Page): def on_loaded(self): pass class ComposePage(Page): phone_regex = re.compile('^(\+33|0033|0)(6|7)(\d{8})$') def on_loaded(self): # Deal with bad encoding... for ie6... response = self.browser.response() response.set_data(response.get_data().decode('utf-8', 'ignore')) self.browser.set_response(response) def get_nb_remaining_free_sms(self): return "0" def post_message(self, message, sender): receiver = message.thread.id if self.phone_regex.match(receiver) is None: raise CantSendMessage(u'Invalid receiver: %s' % receiver) listetel = ",," + receiver #Fill the form self.browser.select_form(name="formulaire") self.browser.new_control("hidden", "autorize", {'value': ''}) self.browser.set_all_readonly(False) self.browser["corpsms"] = message.content.encode('utf-8') self.browser["pays"] = "33" self.browser["listetel"] = listetel self.browser["reply"] = "2" self.browser["typesms"] = "2" self.browser["produit"] = "1000" self.browser["destToKeep"] = listetel self.browser["NUMTEL"] = sender self.browser["autorize"] = "1" self.browser["msg"] = message.content.encode('utf-8') self.browser.submit() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/orange/pages/login.py������������������������������������������������������������0000664�0000000�0000000�00000004052�13034501105�0020326�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page import urllib class LoginPage(Page): def on_loaded(self): pass def login(self, user, pwd): post_data = {"credential" : str(user), "password" : str(pwd), "save_user": "false", "save_pwd" : "false", "save_TC" : "true", "action" : "valider", "usertype" : "", "service" : "", "url" : "http://www.orange.fr", "case" : "", "origin" : "", } post_data = urllib.urlencode(post_data) self.browser.addheaders = [('Referer', 'http://id.orange.fr/auth_user/template/auth0user/htm/vide.html'), ("Content-Type" , 'application/x-www-form-urlencoded') ] self.browser.open(self.browser.geturl(), data=post_data) #~ print "LOGIN!!!" #~ self.browser.select_form(predicate=lambda form: "id" in form.attrs and form.attrs["id"] == "authentication_form" ) #~ user_control = self.browser.find_control(id="user_credential") #~ user_control.value = user #~ pwd_control = self.browser.find_control(id="user_password") #~ pwd_control.value = pwd #~ self.browser.submit() ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ouifm/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015410�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ouifm/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001432�13034501105�0017521�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OuiFMModule __all__ = ['OuiFMModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ouifm/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000007574�13034501105�0017560�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME  66K���tEXtComment�Created with GIMPW��IDATxy|TUǿUedB�\"Hb#! B"#("#(4N#`KЀHF[XCVB%3WK Tpz˭wY.Mi$f.,pd ʼnC' jj:\nkr&=�|tɄVU"0dWh>%L0.0%ش5_h^.8~]d70Tm' @<�;7Afp}`Ji!�p%ChL1-c�́nm@yK ^KUǜ,1_U�Bz0/}'!7~7 Р%\U:oFL/һwƢ++UdUF{tm0)T VEBzW0Vj`3 ?Zu�sn#aғ( N%i(_@W \-[1Fl~<mՄ0j ӏ"hTq |;� mw d|Ďu[xJ2L�gaO֡$ nX1,Zjs r&Np2DqHQ-Eh!BsDh&BSIjŞQ̘ 17l4he ߶X3bb==.-3}sT)#Urh,iޜ~;'�~? = I*nm鄕` Ъ,zޛEEQ0.8v #pL#ڤWtu;IX_)dL#""ykHWѺB̓bv-fD }CYsv1f,/s$ښq{1ybF Tq*R MdՙBK@&Ly NϻCjS n nY;棼}!)^Sَ 6 Ǘo$Z @h j �,!4X٢;ySߒ@Yt`fT 5KGHxtdhċO:21 meY[R\qȄhp{jVצ4NvġG0r>9l Z8#"*;3nTyH9t�hW1!`GaG"P ?  ϳE5tJ>@/ |-m?k}{pC7HJ fH u24a-x.7&u΄6_=šȘ*.xLSgt}{}:ڗ >ǝss2}p~aĠk ��@q9V}z{U{xYuC`�cQ7 [Sܰ@vp 1Zhsx_ _] vJ�F5N-2'6uhUM18`^\p+*&rΡ۵ oˡGw*z ̛yEE4W�u�_n|.x&,ZZLVR>Nh{QPbۏbѰ:ݩ?XSgÁaaVfhGD.!kv6FimR@єviG5i: 8+|uhg ϷV]MpkO?Arx�_p?\Pi?1( yrNLG/Koѣ )]\%k�QSn\1n͜VfE r+( kkLw@C ^OvKkc�غ BKWS{߂Wþo[g-詳Zëҷ Z"=, \Kƭ`ANga g`FsN:^oܻKCl̦ @1罀$jl(34cPY0jR| |74|nٱ0hT 0 ՈϽO\,g0qAQptEg <PoDlӒxë i ":%4~lwxeh?SOVv i9NMW�ctLSg~Me3uUKt-yIr�û3!9h/u<+F?Pk{<J* i%m._xf #д|2Nj;%_WkFqN uy~I.]Zý `rHhW#^ $=x}8 8X{P͛A׎0z8kCK. 6b˻1ׄ֋G`ka.Hpﱴ;H2`#U9[3su|%|oϏ1_4<>G# Z6,y`>ۆ-�PP`1`+(N@ʝ:$5=ܖ.ˀY *6ޞ}Qv10㝰7!'L4dW]a~T 8TsנS+WBl!xف4έPd YUl>o:jZӅI "VugLX`/{ةYDž !8Vޭ1\`L)W`W] khNa֔ cgesQ &y$ /PYDBߪ,\q̆7]t-NfT}Fvy_�/!odw ꊘJ|:$*>HN}bL arsE5.a <j7q_;?S|/j/<۲}"�wӑV-a<L~Z~ﶨ_f/,p;V$Ff8-: U j,8Cߏjn;J Gi!1AfCkEm@5yP߬~3̪5y-;,ϏR|< jB%.1UWAOmk7V =%EbrM.Z<LJ-SuP}v_PI1B?{քCİ67ţz%cnیmZCkbr lV#g-9AcA @g}G^WT %cDXlEbC0J/f<f �`I $#-E4(/2@,_ӌ+| 4O;pٵ~HZ9A((",,%\(|)L5:?[h\불Xx/cX2q.20 cu+<ǐ><9kkT> P7h"Fi VFD ޠZ^l}Jo),>w4ͮ˻כCQ))QhAL [i'SY#[Ż(qPV: 2% !E?BhAz3NK>EEe8 '-gyh԰$Dp0AM`Zt/A8xbdi|z_1z'cmh7KӀ)8B#[r1,_jڥvU9t2����IENDB`������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ouifm/module.py������������������������������������������������������������������0000664�0000000�0000000�00000007731�13034501105�0017257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from weboob.deprecated.browser import StandardBrowser from weboob.tools.misc import to_unicode __all__ = ['OuiFMModule'] class OuiFMModule(Module, CapRadio, CapCollection): NAME = 'ouifm' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = u'OÜI FM French radio' LICENSE = 'AGPLv3+' BROWSER = StandardBrowser _RADIOS = {'general': (u"OÜI FM", u'OÜI FM', u'http://ouifm.ice.infomaniak.ch/ouifm-high.mp3', 128), 'alternatif': (u"OÜI FM Alternatif", u'OÜI FM - L\'Alternative Rock', u'http://ouifm.ice.infomaniak.ch/ouifm2.mp3', 128), 'classicrock': (u"OÜI FM Classic Rock", u'OÜI FM - Classic Rock', u'http://ouifm.ice.infomaniak.ch/ouifm3.mp3', 160), 'bluesnrock': (u"OÜI FM Blues'n'Rock", u'OÜI FM - Blues\'n\'Rock', u'http://ouifm.ice.infomaniak.ch/ouifm4.mp3', 128), 'rockinde': (u"OÜI FM Rock Indé", u'OÜI FM - Rock Indé', u'http://ouifm.ice.infomaniak.ch/ouifm5.mp3', 128), } def create_default_browser(self): return self.create_browser(parser='json') def iter_resources(self, objs, split_path): if Radio in objs: self._restrict_level(split_path) for id in self._RADIOS.iterkeys(): yield self.get_radio(id) def iter_radios_search(self, pattern): for radio in self.iter_resources((Radio, ), []): if pattern.lower() in radio.title.lower() or pattern.lower() in radio.description.lower(): yield radio def get_current(self, radio): document = self.browser.location('http://www.ouifm.fr/onair.json') rad = '' if radio == 'general': rad = 'rock' else: rad = radio last = document[rad][0] artist = to_unicode(last.get('artist', '').strip()) title = to_unicode(last.get('title', '').strip()) return artist, title def get_radio(self, radio): if not isinstance(radio, Radio): radio = Radio(radio) if radio.id not in self._RADIOS: return None title, description, url, bitrate = self._RADIOS[radio.id] radio.title = title radio.description = description artist, title = self.get_current(radio.id) current = StreamInfo(0) current.who = artist current.what = title radio.current = current stream = BaseAudioStream(0) stream.bitrate=bitrate stream.format=u'mp3' stream.title = u'%skbits/s' % (stream.bitrate) stream.url = url radio.streams = [stream] return radio def fill_radio(self, radio, fields): if 'current' in fields: if not radio.current: radio.current = StreamInfo(0) radio.current.who, radio.current.what = self.get_current(radio.id) return radio OBJECTS = {Radio: fill_radio} ���������������������������������������weboob-1.2/modules/ouifm/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000001747�13034501105�0016752�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.radio import Radio class OuiFMTest(BackendTest): MODULE = 'ouifm' def test_ouifm(self): l = list(self.backend.iter_resources((Radio, ), [])) self.assertTrue(len(l) > 0) �������������������������weboob-1.2/modules/ovh/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015065�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001432�13034501105�0017176�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OvhModule __all__ = ['OvhModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000003104�13034501105�0017120�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, ProfilePage, BillsPage class OvhBrowser(LoginBrowser): BASEURL = 'https://www.ovh.com' login = URL('/auth/', '/manager/web/index.html', LoginPage) profile = URL('/manager/dedicated/api/proxypass/me', ProfilePage) billspage = URL('/manager/web/api/billing/bills', BillsPage) def do_login(self): self.login.go().login(self.username, self.password) if not self.page.is_logged(): raise BrowserIncorrectPassword @need_login def get_subscription_list(self): return self.profile.stay_or_go().get_list() @need_login def iter_documents(self, subscription): return self.billspage.stay_or_go().get_documents(subid=subscription.id) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000011007�13034501105�0017217�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��� pHYs�� �� ����tIME 1(ry&���iTXtComment�����Created with GIMPd.e���bKGD�����kIDATx[ pUjKݝ$kKEDuePIN <SB |"сqАtWPqpԚutqg}PS[֌nm {N$Sw}s=;߽.|ݧ_{WP(*?`,CUAdnFV ,Љ GQB7^麂�vAy}/xE;=Ս׶baY\>2 g}g1Nbc}N G3Xd�OT[>Ǹ~coKY,l dS0:v)D2:q轄y <F ߮~sUHw0U`fcW? nM̝F1&qIqxjmC*sPE;ߎџk|9~RfZf`apI0g]2OG�K$ 08Gxf#NSG=E?6:Ok1TzcRG"&1*?$"#F(5>)`ލ@陋5~+*."=[ܖ`pRȻTXC:9 br2g0&cbN3I83-?ԃBݔͶ!LN%f8}7~ϯ�]-<u85'9/&uL.6Bi$|8y>l:;|gP(R)I4?Sf6J5t5\g߳ f,CF9V T=p>u_>+9y00h 1w6FGF[3R]͘@JNq{:)l*/>&~>928%F @*/zƻ6$L(/$e١]WHgrv3,Sp̧ByV*B"]"X-]JF22d'Ǹ(s J!5aB#s?C0~c!uY�L2 :яni,xzY>L^rC?E4q 2*NSֱE?1 !έ8xw s*GP\ЍV8h3Ϯz\aS[]~LȖyg8"&] R$D; %slp7;z)D7  n aЈu6fQ1ܵEOJ\G.|*)|-5Fi޳|'`K#2wĉ^"rSz a :^Z 5GRS/8BL#-''Y6)NK<#O54vڗb/bI"_fac[(cֲF4q,CpQtܑk-P%x{\UjkkH\Sb"A:-!0>fgأ*wfN)i~{ *u[Y&=ܙ40G*KO'VjS9""n DDق4`ɳGDѰ Jp6QR}V+}TQ>yyDy+Obpf@׆lP~ vӬ>mY]&ыqT~\*J: yHBs@IjpHJ_Q\3ty}k\5=|$ОtAדq:w"JPP߁ƶ~ :Y¬ ]UaRb6RُUܵ,śB DNʭajZZH �0 b&M*o:*<ދYS*A{X0fѸ3ȶ{}Jmx4)+D0NWn6BQ݌vTa1YU K$l}*OaJqtX.w Lc]>ss;2jZl7ξU0kˏm|!gZ;_0i8?sXMVL}FRhQqOkT=%&ƬL|#*;#^kR=t񬜹}v԰W\У͆7laܰZ 53=hd�dqdnl?e4ؓAFՇa18-gX=4"bԙۥw*% |4$%ϞWs_E0mKgpVLx̥%z=FIqT Lv5){n}?Č!Y*E4hQ`[^B 1lٗbO9W(.p m[&bw$U}oFLc gc1#4vUĴ]Mq$% *J';HR<B-Ō[-倌,m nvYEobVrofR) &twOҚ7M.۹egqE㈁5?Dhv Jb:@攛6U4M_dwgLK,U-�fBٯrVAڏ 2Z y8C  ]xqg!!z5"@k6&2 ?C 7X^}Z d[=gߋ.`WqFvb͞6}22i3pI-%cO4�o 2exhW5[x=p}05MNv+o <|O*qŠ~(WNKt5-XWH 'o "pB(JGp;PͿREICpt{mTަ9*&e`RnXdqAcs{ I*xlo V~kX^b ,,'cW•Cw1dR.aj6'6)•?3/>Aimr}F| &Lq7s,&gɘL'~'\]LzS};ƘLb2# έ^t~@t`.ҭA ΖXX;塑޷fK̋�ǡXlg;e =l3wk'opU~Lȑ. K'ZF)c^|X(整aL&iSu¸;%8[ALy',jqw@+z~OaXd5c.(>qOstlIoc: 1Rk 9J^d&֟Ǎc>P)jX"MoX3w0Y'X ySnOQ&tv 6} fP~)Rᕜ&aSy߼e{ nst/.Y"}2n|ϛhhk)1@:F+G08n*;s2SxnSDF 3r[nTԛ`@m`F*C(Aӝز$|4p|/+vCaC/Y`zN3[Eou !;F&d poy˫(y*ejf?i{y;?Lj1*լxp]�nٵ!do ?m[Ĝ~Y,hy1 ˃zDYjctǷw_Pxwi{A(zs D^�k2 f얐#ּ�,nY; wd0  Ib&֝<ZC%gR6k3yxɡ$�7&j[UARki54%=jҶ U?,O/} ܵ,[Tve0@.) bvvL <A pI?PMV~(IH%"^nE f!\5NMgl˪iP7.ޯgRr Y wgGI-<Z)V@2Ja!Ȱ1j4[ڊ1ܬlv"37oZ"݃|g[ᱯU/p; ZƺltJV+C[P7KIVR!5J;pK>W(Ӆ%TiCD=+Hl7ȹ[ 5*yW'Ic f}%UXVw ~^v]zraLfTXbe:?`4:A[MNW)MYL 7Anu^ Uޏ~Mo 멣{,l=+e*q{|JbsWII"X;W~9?=����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000004543�13034501105�0016732�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Bill, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import OvhBrowser __all__ = ['OvhModule'] class OvhModule(Module, CapDocument): NAME = 'ovh' DESCRIPTION = u'Ovh' MAINTAINER = u'Vincent Paredes' EMAIL = 'vparedes@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='Account ID'), ValueBackendPassword('password', label='Password')) BROWSER = OvhBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.split('.')[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, bill): if not isinstance(bill, Bill): bill = self.get_document(bill) return self.browser.open(bill.url).content �������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000004615�13034501105�0016544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import Bill, Subscription from weboob.browser.pages import HTMLPage, LoggedPage, JsonPage from weboob.browser.filters.standard import CleanDecimal, CleanText, Env, Format, Date from weboob.browser.filters.html import Attr from weboob.browser.filters.json import Dict from weboob.browser.elements import ListElement, ItemElement, method, DictElement class LoginPage(HTMLPage): def is_logged(self): return not self.doc.xpath('//div[has-class("error")]') def login(self, login, password): form = self.get_form('//form[@class="pagination-centered"]') user = Attr(None, 'name').filter(self.doc.xpath('//input[contains(@placeholder, "Account ID")]')) pwd = Attr(None, 'name').filter(self.doc.xpath('//input[@placeholder="Password"]')) form[user] = login form[pwd] = password form.submit() class ProfilePage(LoggedPage, JsonPage): @method class get_list(ListElement): class item(ItemElement): klass = Subscription obj_label = CleanText(Dict('nichandle')) obj_subscriber = Format("%s %s", CleanText(Dict('firstname')), CleanText(Dict('name'))) obj_id = CleanText(Dict('nichandle')) class BillsPage(LoggedPage, JsonPage): @method class get_documents(DictElement): item_xpath = 'list/results' class item(ItemElement): klass = Bill obj_id = Format('%s.%s', Env('subid'), Dict('orderId')) obj_date = Date(Dict('billingDate')) obj_format = u"pdf" obj_type = u"bill" obj_price = CleanDecimal(Dict('priceWithTax/value')) obj_url = Dict('pdfUrl') �������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovh/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001475�13034501105�0016425�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Vincent Paredes # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class OvhTest(BackendTest): MODULE = 'ovh' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015100�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001424�13034501105�0017212�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import OvsModule __all__ = ['OvsModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000011270�13034501105�0017136�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.deprecated.browser.parsers.iparser import IParser import BeautifulSoup from .pages import PagePrivateThreadsList, PagePrivateThread, PageLogin, PageIndex, DummyPage, PageUserProfile, PageCityList __all__ = ['OvsBrowser'] class SoupParser(IParser): def parse(self, data, encoding=None): return BeautifulSoup.BeautifulSoup(data.read().decode(encoding or 'utf-8'), convertEntities=BeautifulSoup.BeautifulStoneSoup.ALL_ENTITIES) class OvsBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'paris.onvasortir.com' ENCODING = 'cp1252' def __init__(self, city, username, password, *a, **kw): self.DOMAIN = '%s.onvasortir.com' % city self.PAGES = { '%s://%s/' % (self.PROTOCOL, self.DOMAIN): PageIndex, r'%s://%s/message_read.php\?Id=.+' % (self.PROTOCOL, self.DOMAIN): PagePrivateThread, '%s://%s/vue_messages_recus.php' % (self.PROTOCOL, self.DOMAIN): PagePrivateThreadsList, '%s://%s/vue_messages_envoyes.php' % (self.PROTOCOL, self.DOMAIN): PagePrivateThreadsList, '%s://%s/page_action_connect.php' % (self.PROTOCOL, self.DOMAIN): PageLogin, r'%s://%s/\?Langue=EN' % (self.PROTOCOL, self.DOMAIN): DummyPage, '%s://%s/page_action_boost.php' % (self.PROTOCOL, self.DOMAIN): DummyPage, '%s://%s/vue_profil_all.php.php' % (self.PROTOCOL, self.DOMAIN): DummyPage, r'%s://%s/message_msg_envoi_ok.php\?.*' % (self.PROTOCOL, self.DOMAIN): DummyPage, '%s://%s/message_action_envoi.php' % (self.PROTOCOL, self.DOMAIN): DummyPage, r'%s://%s/profil_read.php\?.+' % (self.PROTOCOL, self.DOMAIN): PageUserProfile, 'http://www.onvasortir.com/?': PageCityList, 'http://www.urbeez.com/?': PageCityList, } kw['parser'] = SoupParser() Browser.__init__(self, username, password, *a, **kw) self.city = city def iter_threads_list(self): self.location('/vue_messages_recus.php') assert self.is_on_page(PagePrivateThreadsList) for thread in self.page.iter_threads_list(): yield thread self.location('/vue_messages_envoyes.php') assert self.is_on_page(PagePrivateThreadsList) for thread in self.page.iter_threads_list(): yield thread def get_thread(self, _id): self.location('/message_read.php?Id=%s&AffMsg=all' % _id) assert self.is_on_page(PagePrivateThread) return self.page.get_thread(_id) def login(self): assert not self.is_logged() self.page.login(self.username, self.password) if not self.is_logged(): raise BrowserIncorrectPassword() self.location('/?Langue=EN') self.location('/page_action_boost.php') self.location('/') def is_logged(self): return (self.is_on_page(DummyPage) or self.page.is_logged()) def post_to_thread(self, thread_id, subject, body): self.location('/message_read.php?Id=%s' % thread_id.encode(self.ENCODING)) # FIXME assert self.is_on_page(PagePrivateThread) self.page.post_to_thread(thread_id, subject, body) def create_thread(self, recipient, subject, body): self.location('/profil_read.php?%s' % recipient.encode(self.ENCODING)) # FIXME assert self.is_on_page(PageUserProfile) self.page.create_thread(recipient, subject, body) def get_contact(self, id): self.location('/profil_read.php?%s' % id.encode(self.ENCODING)) # FIXME assert self.is_on_page(PageUserProfile) return self.page.get_contact() def get_french_cities(self): self.location('http://www.onvasortir.com') assert self.is_on_page(PageCityList) return self.page.get_cities('onvasortir.com') def get_world_cities(self): self.location('http://www.urbeez.com') assert self.is_on_page(PageCityList) return self.page.get_cities('urbeez.com') ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000006231�13034501105�0017235�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME8���tEXtComment�Created with GIMPW�� IDATxݛ{?݃Nފh0ʈ"F#╠B UZSHXJ0JRh +!!JB, J1FE̡N@n~czzfgw=bv{ۿQU,#> ԁ.QMձ5nxA(ߤcׯJ'*,SSdpixZ7fl P@b{aYJz\m_%g(:&�?WAԚ@WEO<NZH\n T:'ΓY@TID!q+h{H,f)S4(EUGI]9d q70M_h~RHX%~, ?C@Y�1s̖ *'qpZ ͹L*f^ gp<Y9dCOd׉omNtp(h b+za4\JW`qL@11y Q& 8&Mo ꁭ Rm41|iw�k"o5f Zl@e]d7>$_�4G2[QZJlkLyטcBGV_t6׸N($(:�-ǚpqor NV\k96cK9;U4vpbtm9KKX@:U[i$%@Ze>G,퓠d88<5傡aYC tf͓cutM(=$*i+Hk25J(|e=_*<OV\ d{WHZ wYQGVxq[X<@E q%dL@}b$g7p�5gP_ʆ]SQrAM<?!e\k?EopFx*Y*d0¥8RVHjH mh@pV@p2Xp=73F  H)"vz0KIoQE863@,VP~JZ_j�i)*4";AH`iSC8ʺX|a\Z|-7{V;_ YQ^ s` ^rr<;Lu&!dFv$u œ�8Rt�<Kq<�[mc+>o!| Gp|`\=Q.m� ݋� ̭}Jj-*ԪN(=j/N hwYZ;܈X5w5>ȲOM&3p�-hRe&Z _d/.lw9v\^h4ݔ<\, 4> L`I]KRF ,*�¹HIe<A{}`5u[ ux_0>}|d/$A\j^2MQ+7 $+h)@C 0G8ػ6OX�a0ɺuF[�ڈ$sY79Oe'ʣUA 8)+7ClWC}Ou#.W�CYb2tr8H=RTV݁61XS$!%":ykp[K` q(秨wݨEIv\Pn /I'&1:w �>8%?<CkIj9 (CaT�H:W!~!n5JTUѢ3tmp9<`!S"� d�rЂ-\nCُR#���O: j7Wf{L[|!@Sc(@{:G'qKZjL%x-zUez_LN)#FWN2^(9'%I|@v�xL)SPD|k2N, 0)!<m�6diwd%w ӽp9'ZP$4 jr2Qg}Tlxs|Q@"DfؘJ( # FYdUŭ[T{a'ΏVM~.1*+!L2ZQ2˴lYK`_ Ty.ז `kpIZ` ,F0 %gbی"Z)1d@πl`x6du!*wrp�#m;NzRsΤYQ`GV+IZF�[0Zc;SY's0Qna_> $̰NpxnN؂tURn@(_óФfZ\R0L¨\ELK^o2ڐJWySsJ,FZn3Qc_L LRAZR?ܟ NxO]X;-3'KS|Ë(=i*([P#+�^JZ:\BG[r i�n 8Yh` ԄȦ7x-{rq`B `tT] <!EIy1di˙2>W$g[:e$DZau\JD n|rjdzBrn$�v! P]ׇ鍉2jjJ]gP#t i<qwrKr,eUZ'}=(\|3 lˡqCA0[)pOdf@b,D8{P^e%!?0R뤷̒!ĘFi F_@R+-Z&s#4SGQch:R(O"Yx~QKB6$@JGϻ+[M*?̀^n ,$k^[,0{Ak9�Т >  Q�j�8؅r+>ڬ.h8�6!#fVS[?G����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000015312�13034501105�0016741�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.deprecated.browser import BrowserForbidden from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message from weboob.capabilities.contact import CapContact from .browser import OvsBrowser __all__ = ['OvsModule'] CITIES = {u'agen': u'Agen', u'ajaccio': u'Ajaccio', u'albi': u'Albi', u'amiens': u'Amiens', u'angers': u'Angers', u'angouleme': u'Angoul\xeame', u'annecy': u'Annecy', u'aurillac': u'Aurillac', u'auxerre': u'Auxerre', u'avignon': u'Avignon', u'bastia': u'Bastia', u'beauvais': u'Beauvais', u'belfort': u'Belfort', u'bergerac': u'Bergerac', u'besancon': u'Besan\xe7on', u'beziers': u'B\xe9ziers', u'biarritz': u'Biarritz', u'blois': u'Blois', u'bordeaux': u'Bordeaux', u'bourg-en-bresse': u'M\xe2con', u'bourges': u'Bourges', u'brest': u'Brest', u'brive-la-gaillarde': u'Brive', u'bruxelles': u'Bruxelles', u'caen': u'Caen', u'calais': u'Boulogne', u'carcassonne': u'Carcassonne', u'chalon-sur-saone': u'Chalon', u'chambery': u'Albertville', u'chantilly': u'Chantilly', u'charleroi': u'Charleroi', u'charleville-mezieres': u'Charleville', u'chartres': u'Chartres', u'chateauroux': u'Ch\xe2teauroux', u'cherbourg': u'Cherbourg', u'cholet': u'Cholet', u'clermont-ferrand': u'Clt-Ferrand', u'compiegne': u'Compi\xe8gne', u'dieppe': u'Dieppe', u'dijon': u'Dijon', u'dunkerque': u'Dunkerque', u'evreux': u'Evreux', u'frejus': u'Fr\xe9jus', u'gap': u'Gap', u'geneve': u'Gen\xe8ve', u'grenoble': u'Grenoble', u'la-roche-sur-yon': u'La Roche/Yon', u'la-rochelle': u'La Rochelle', u'lausanne': u'Lausanne', u'laval': u'Laval', u'le-havre': u'Le Havre', u'le-mans': u'Alen\xe7on', u'liege': u'Li\xe8ge', u'lille': u'Lille', u'limoges': u'Limoges', u'lorient': u'Lorient', u'luxembourg': u'Luxembourg', u'lyon': u'Lyon', u'marseille': u'Aix', u'metz': u'Metz', u'mons': u'Mons', u'mont-de-marsan': u'Mont de Marsan', u'montauban': u'Montauban', u'montlucon': u'Montlu\xe7on', u'montpellier': u'Montpellier', u'mulhouse': u'Colmar', u'namur': u'Namur', u'nancy': u'Nancy', u'nantes': u'Nantes', u'nevers': u'Nevers', u'nice': u'Cannes', u'nimes': u'N\xeemes', u'niort': u'Niort', u'orleans': u'Orl\xe9ans', u'paris': u'PARIS', u'pau': u'Pau', u'perigueux': u'P\xe9rigueux', u'perpignan': u'Perpignan', u'poitiers': u'Poitiers', u'quimper': u'Quimper', u'reims': u'Reims', u'rennes': u'Rennes', u'roanne': u'Roanne', u'rodez': u'Rodez', u'rouen': u'Rouen', u'saint-brieuc': u'St-Brieuc', u'saint-etienne': u'St-Etienne', u'saint-malo': u'St-Malo', u'saint-nazaire': u'St-Nazaire', u'saint-quentin': u'St-Quentin', u'saintes': u'Saintes', u'strasbourg': u'Strasbourg', u'tarbes': u'Tarbes', u'toulon': u'Toulon', u'toulouse': u'Toulouse', u'tours': u'Tours', u'troyes': u'Troyes', u'valence': u'Mont\xe9limar', u'vannes': u'Vannes', u'zurich': u'Zurich'} class OvsModule(Module, CapMessages, CapMessagesPost, CapContact): NAME = 'ovs' DESCRIPTION = u'OnVaSortir website. Handles private messages only' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), Value('city', label='City (subdomain)', default='paris', choices=CITIES)) BROWSER = OvsBrowser STORAGE = {'seen': {}} def create_default_browser(self): return self.create_browser(self.config['city'].get(), self.config['username'].get(), self.config['password'].get(), parser='raw') # CapMessages def iter_threads(self): with self.browser: for thread in self.browser.iter_threads_list(): yield thread def get_thread(self, id): with self.browser: thread = self.browser.get_thread(id) messages = [thread.root] + thread.root.children for message in messages: if not self.storage.get('seen', message.full_id, default=False): message.flags |= Message.IS_UNREAD return thread def iter_unread_messages(self): with self.browser: for thread in self.iter_threads(): # TODO reuse thread object? thread2 = self.get_thread(thread.id) messages = [thread2.root] + thread2.root.children for message in messages: if message.flags & Message.IS_UNREAD: yield message # TODO implement more efficiently by having a "last weboob seen" for # a thread and query a thread only if "last activity" returned by web # is later than "last weboob seen" def set_message_read(self, message): self.storage.set('seen', message.full_id, True) self.storage.save() # CapMessagesPost def post_message(self, message): if not self.browser.username: raise BrowserForbidden() with self.browser: thread = message.thread if message.parent: # ovs.<threadid>@* self.browser.post_to_thread(thread.id, message.title, message.content) else: # ovs.<recipient>@* self.browser.create_thread(thread.id, message.title, message.content) # CapContact def get_contact(self, id): return self.browser.get_contact(id) # FIXME known bug: parsing is done in "boosted mode" which is automatically disable after some time, the "boosted mode" should be re-toggled often # TODO support outing comments, forum messages # TODO make an CapOuting? ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/ovsparse.py������������������������������������������������������������������0000664�0000000�0000000�00000006231�13034501105�0017316�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import BeautifulSoup def nearest_parent(node, expected): return node.findParent(expected) while node and node.name != expected: node = node.parent return node def all_text_recursive(node): return ''.join(node.findAll(text=True)) def all_next_siblings(node): ret = [] while node: ret.append(node) node = node.nextSibling return ret def image_to_text(src): smileys = {'chat/e/grin.gif': ':D', 'chat/e/unsure.gif': ':s', 'chat/e/smile.gif': ':)', 'chat/e/shocked.gif': ':|', 'chat/e/sad.gif': ':(', 'chat/e/huh.gif': ':h', 'chat/e/suprised.gif': ':o', 'chat/e/cool.gif': 'B)', 'chat/e/redface.gif': ':red', 'chat/e/confused.gif': ':=', 'chat/e/razz.gif': ':p', 'chat/e/wink.gif': ';)', 'chat/e/mad.gif': ':x', 'chat/e/rolleyes.gif': ':b', 'chat/e/lol.gif': ':lol', 'chat/e/wub.gif': ':$', 'chat/e/bouche.gif': ':K', 'chat/e/sick.gif': '+o('} return smileys.get(src) def html_message_to_text(nodes): parts = [] for node in nodes: if isinstance(node, BeautifulSoup.NavigableString): parts.append(unicode(node).replace('\r', '')) elif node.name == 'img': parts.append(image_to_text(node['src'])) elif node.name == 'a': parts.append(node['href']) elif node.name == 'br': parts.append('\n') else: assert not ('%s not supported' % node.name) return ''.join(parts) def create_unique_id(proposed_id, used_ids): if proposed_id not in used_ids: return proposed_id def make_id(base, index): return '%s-%s' % (base, index) index = 1 while make_id(proposed_id, index) in used_ids: index += 1 return make_id(proposed_id, index) # public def private_message_form_fields(document): ret = {} form = document.find('form', attrs={'name': 'envoimail'}) def set_if_present(name): item = form.find('input', attrs={'name': name}) if item: ret[name] = item['value'] set_if_present('Pere') set_if_present('Sortie') set_if_present('Dest') set_if_present('Titre') return ret def is_logged(document): return (not document.find('form', attrs={'name': 'connection'})) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000023140�13034501105�0016551�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime import re import urllib from urlparse import urlsplit from weboob.deprecated.browser import Page from weboob.capabilities.messages import Message, Thread from weboob.capabilities.contact import Contact, ProfileNode from weboob.tools.date import parse_french_date from . import ovsparse class OvsPage(Page): def is_logged(self): return ovsparse.is_logged(self.document) def login(self, username, password): self.browser.select_form(name='connection') self.browser['Pseudo'] = username.encode(self.browser.ENCODING) self.browser['Password'] = password.encode(self.browser.ENCODING) self.browser['Retenir'] = ['ok'] self.browser.submit(nologin=True) class PagePrivateThreadsList(OvsPage): def iter_threads_list(self): # site is sorted from latest to oldest for message_a in reversed(self.document.findAll('a', href=re.compile(r'message_read.php\?'))): ovs_id = re.search(r'Id=(\d+)', message_a["href"]).group(1) id_ = ovs_id thread = Thread(id_) thread.title = ovsparse.all_text_recursive(message_a) thread.flags = Thread.IS_DISCUSSION #~ parent_tr = message_a.findParent('tr') #~ username = all_text_recursive(parent_tr.find('a', href=re.compile(r'profil_read.php\?.*'))) #~ notread_self = (parent_tr.get('class') == 'newmails') #~ notread_other = (parent_tr.find('span', **{'class': 'new_sortiepartenaire'}) is not None) yield thread class PagePrivateThread(OvsPage): def get_thread(self, _id): thread = Thread(_id) thread.title = self.document.find('div', 'PADtitreBlanc_txt').find('center').string thread.flags = Thread.IS_DISCUSSION root = True for message in self._get_messages(thread): if root: message.children = [] thread.root = message thread.date = message.date message.title = thread.title root = False else: message.title = 'Re: %s' % thread.title message.children = [] message.parent = thread.root thread.root.children.append(message) return thread def _get_messages(self, thread): thread_div = self.document.find(True, 'PADpost_txt') used_ids = set() rcpt = self.document.find('input', attrs={'type': 'hidden', 'name': 'Dest'})['value'] sender_to_receiver = {rcpt: self.browser.username, self.browser.username: rcpt} # site is sorted from latest to oldest message for message_table in reversed(thread_div.findAll('table')): for td in message_table.findAll('td'): profile_a = td.find('a', href=re.compile(r'profil_read.php\?.*')) if not profile_a: continue first_br = td.find('br') assert first_br.nextSibling.name == 'br' text_nodes = ovsparse.all_next_siblings(first_br.nextSibling.nextSibling) # TODO #~ print text_nodes # date will be used as id sitedate = profile_a.findParent('div').find(text=re.compile(',.*')).replace(', ', '') sysdate = parse_french_date(sitedate) compactdate = datetime.datetime.strftime(sysdate, '%Y%m%dT%H%M%S') # but make it unique msg_id = ovsparse.create_unique_id(compactdate, used_ids) used_ids.add(msg_id) message = Message(thread, msg_id) message.sender = re.search(r'\?(.+)', profile_a['href']).group(1) message.receivers = [sender_to_receiver[message.sender]] message.date = sysdate message.content = ovsparse.html_message_to_text(text_nodes) notread_self = bool(td.find('span', 'ColorSurligne')) notread_other = bool(td.find('span', 'new_sortiepartenaire')) if notread_other or notread_self: message.flags |= Message.IS_NOT_RECEIVED else: message.flags |= Message.IS_RECEIVED yield message def post_to_thread(self, thread_id, subject, body): form = ovsparse.private_message_form_fields(self.document) recode_dict(form, self.browser.ENCODING) form['Message'] = body.encode(self.browser.ENCODING) self.browser.location('/message_action_envoi.php', urllib.urlencode(form)) # html code is so broken that mechanize won't parse the forms #~ self.browser.select_form('envoimail') #~ self.browser['Message'] = body.encode(self.browser.ENCODING) #~ self.browser['Pere'] = thread_id.encode(self.browser.ENCODING) #~ self.browser['Titre'] = subject.encode(self.browser.ENCODING) #~ self.browser.submit() class PageLogin(Page): pass class PageIndex(OvsPage): pass class DummyPage(Page): pass class PagePostMessage(OvsPage): pass class PageUserProfile(OvsPage): def create_thread(self, recipient, subject, body): form = ovsparse.private_message_form_fields(self.document) recode_dict(form, self.browser.ENCODING) form['Message'] = body.encode(self.browser.ENCODING) form['Titre'] = subject.encode(self.browser.ENCODING) self.browser.location('/message_action_envoi.php', urllib.urlencode(form)) #~ self.browser.select_form('envoimail') #~ self.browser['Titre'] = subject.encode(self.browser.ENCODING) #~ self.browser['Message'] = body.encode(self.browser.ENCODING) #~ self.browser.submit() def get_contact(self): profile_a = self.document.find('a', href=re.compile(r'profil_read.php\?.*')) _id = re.search(r'\?(.*)', profile_a['href']).group(1) # not available in the 'boosted' version contact = Contact(_id, _id, Contact.STATUS_OFFLINE) contact.url = self.url contact.profile = {} thumbnail_url = 'http://photos.onvasortir.com/%s/photos/%s_resize.png' % (self.browser.city, _id) if self.document.find('img', attrs={'src': thumbnail_url}): photo_url = thumbnail_url.replace('_resize', '') contact.set_photo('main', thumbnail_url=thumbnail_url, url=photo_url, hidden=False) location_a = self.document.find('a', href=re.compile(r'vue_profil_carte\.php\?')) if location_a: lat = float(re.search(r'Lat=([\d.]+)', location_a['href']).group(1)) self._set_profile(contact, 'latitude', lat) lng = float(re.search(r'Lng=([\d.]+)', location_a['href']).group(1)) self._set_profile(contact, 'longitude', lng) div = self.document.find('div', attrs={'class': 'PADtitreBlanc_txt'}, text=re.compile('Personal Info')) td = div.findParent('tr').findNextSibling('tr').td infos_text = td.getText(separator='\n').strip() it = iter(infos_text.split('\n')) infos = dict(zip(it, it)) if infos['Sex :'] == 'Man': self._set_profile(contact, 'sex', 'M') elif infos['Sex :'] == 'Woman': self._set_profile(contact, 'sex', 'F') if infos['Birthday :'] != 'Unknown': self._set_profile(contact, 'birthday', parse_french_date(re.search(r'(\d+ \w+ \d+)', infos['Birthday :']).group(1))) self._try_attr(contact, infos, 'First Name :', 'first_name') self._try_attr(contact, infos, 'Status :', 'marriage') self._try_attr(contact, infos, 'Area :', 'area') div = self.document.find('div', attrs={'class': 'PADtitreBlanc_txt'}, text=re.compile('A few words')) td = div.findParent('tr').findNextSibling('tr').td summary = td.getText(separator='\n').strip() if summary == 'Unknown': contact.summary = u'' else: contact.summary = summary div = self.document.find('div', style=re.compile('dashed')) if div: # TODO handle html, links and smileys contact.status_msg = div.getText() else: contact.status_msg = u'' return contact def _set_profile(self, contact, key, value): contact.profile[key] = ProfileNode(key, key.capitalize(), value) def _try_attr(self, contact, infos, html_attr, obj_attr): if infos[html_attr] != 'Unknown': self._set_profile(contact, obj_attr, infos[html_attr].strip()) class PageCityList(DummyPage): def get_cities(self, master_domain='onvasortir.com'): cities = {} for home_a in self.document.findAll('a', href=re.compile(r'http://(.*)\.%s/?' % master_domain)): hostname = urlsplit(home_a['href']).hostname code = hostname.split('.')[0] if code == 'www': continue name = home_a.text cities[name] = {'code': code, 'hostname': hostname} return cities def recode_dict(dict_, encoding): for k in dict_: dict_[k] = dict_[k].encode(encoding) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ovs/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001467�13034501105�0016441�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class OvsTest(BackendTest): MODULE = 'ovs' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015051�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000000067�13034501105�0017165�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import PapModule __all__ = ['PapModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000005464�13034501105�0017117�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.browser import PagesBrowser, URL from weboob.capabilities.housing import Query, TypeNotSupported from .pages import SearchResultsPage, HousingPage, CitiesPage __all__ = ['PapBrowser'] class PapBrowser(PagesBrowser): BASEURL = 'http://www.pap.fr' search_page = URL('annonce/.*', SearchResultsPage) housing = URL('annonces/(?P<_id>.*)', HousingPage) cities = URL('index/ac-geo2\?q=(?P<pattern>.*)', CitiesPage) def search_geo(self, pattern): return self.cities.open(pattern=pattern).iter_cities() TYPES = {Query.TYPE_RENT: 'location', Query.TYPE_SALE: 'vente'} RET = {Query.HOUSE_TYPES.HOUSE: 'maison', Query.HOUSE_TYPES.APART: 'appartement', Query.HOUSE_TYPES.LAND: 'terrain', Query.HOUSE_TYPES.PARKING: 'garage-parking', Query.HOUSE_TYPES.OTHER: 'divers'} def search_housings(self, type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, house_types): if type not in self.TYPES: raise TypeNotSupported() self.session.headers.update({'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'}) data = {'geo_objets_ids': ','.join(cities), 'surface[min]': area_min or '', 'surface[max]': area_max or '', 'prix[min]': cost_min or '', 'prix[max]': cost_max or '', 'produit': self.TYPES.get(type, 'location'), 'recherche': 1, 'nb_resultats_par_page': 40, } if nb_rooms: data['nb_pieces[min]'] = nb_rooms data['nb_pieces[max]'] = nb_rooms ret = [] for house_type in house_types: if house_type in self.RET: ret.append(self.RET.get(house_type)) _data = '%s%s%s' % (urllib.urlencode(data), '&typesbien%5B%5D=', '&typesbien%5B%5D='.join(ret)) return self.search_page.go(data=_data).iter_housings() def get_housing(self, _id, housing=None): return self.housing.go(_id=_id).get_housing(obj=housing) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000001511�13034501105�0017202�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME .���tEXtComment�Created with GIMPW��IDATxOEܮx) #,!KqYR. QAE]T;CmUmta6 ?.h ]1`-z{Μ6»xg!      +*#YW=8 ,^S؊ </k|+Q€ĞftlA?slpbLΕQʀuqM_f6gr m!>bz8WM`v=;bM`^­MxϷ=6my|Ot3Kgp%:9[8uڛ߲t'[?RPG#C٢/^CܣYùYc6;K(izZŖ v۰OϽ#lC(a,册8}zT,V%(VU3~ "Ycq=<N,=ØwEﻖ~6K E<f?~;Klf_uw5%ƒW}0Y;bgO`Gft:‘SGG1ˎy8o9<-[ Xko#ϸLWد;KUp>LK }/=װeulEG,?eNAAAAA����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000004405�13034501105�0016713�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from weboob.tools.backend import Module from .browser import PapBrowser __all__ = ['PapModule'] class PapModule(Module, CapHousing): NAME = 'pap' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'French housing website' LICENSE = 'AGPLv3+' BROWSER = PapBrowser def search_housings(self, query): cities = ['%s' % c.id for c in query.cities if c.backend == self.name] if len(cities) == 0: return list() return self.browser.search_housings(query.type, cities, query.nb_rooms, query.area_min, query.area_max, query.cost_min, query.cost_max, query.house_types) def get_housing(self, housing): if isinstance(housing, Housing): id = housing.id else: id = housing housing = None return self.browser.get_housing(id, housing) def search_city(self, pattern): return self.browser.search_geo(pattern) def fill_housing(self, housing, fields): return self.browser.get_housing(housing.id, housing) def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo, } �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000013047�13034501105�0016527�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from decimal import Decimal from weboob.tools.date import parse_french_date from weboob.browser.pages import HTMLPage, JsonPage, pagination from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Regexp, Env, BrowserURL, Format from weboob.browser.filters.html import Link, XPath, CleanHTML from weboob.browser.filters.json import Dict from weboob.capabilities.base import NotAvailable from weboob.capabilities.housing import Housing, City, HousingPhoto from weboob.tools.capabilities.housing.housing import PricePerMeterFilter class CitiesPage(JsonPage): @method class iter_cities(DictElement): class item(ItemElement): klass = City obj_id = Dict('id') obj_name = Dict('name') class SearchResultsPage(HTMLPage): @pagination @method class iter_housings(ListElement): item_xpath = '//div[has-class("search-results-item")]' def next_page(self): return Link('//ul[@class="pagination"]/li[@class="next"]/a')(self) class item(ItemElement): klass = Housing def condition(self): return Regexp(Link('./div[has-class("box-header")]/a[@class="title-item"]'), '/annonces/(.*)', default=None)(self) obj_id = Regexp(Link('./div[has-class("box-header")]/a[@class="title-item"]'), '/annonces/(.*)') obj_title = CleanText('./div[has-class("box-header")]/a[@class="title-item"]') obj_area = CleanDecimal(Regexp(CleanText('./div[has-class("box-header")]/a/span[@class="h1"]'), '(.*?)(\d*) m\xb2(.*?)', '\\2'), default=NotAvailable) obj_cost = CleanDecimal(CleanText('./div[has-class("box-header")]/a/span[@class="price"]'), replace_dots=True, default=Decimal(0)) obj_currency = Regexp(CleanText('./div[has-class("box-header")]/a/span[@class="price"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') def obj_date(self): _date = Regexp(CleanText('./div[has-class("box-header")]/p[@class="date"]'), '.* / (.*)')(self) return parse_french_date(_date) obj_station = CleanText('./div/div/div[@cladd=metro]', default=NotAvailable) obj_location = CleanText('./div[@class="box-body"]/div/div/p[@class="item-description"]/strong') obj_text = CleanText('./div[@class="box-body"]/div/div/p[@class="item-description"]') def obj_photos(self): photos = [] for img in XPath('./div[@class="box-body"]/div/div/a/img/@src')(self): photos.append(HousingPhoto(u'%s' % img)) return photos class HousingPage(HTMLPage): @method class get_housing(ItemElement): klass = Housing obj_id = Env('_id') obj_title = CleanText('//h1[@class="clearfix"]/span[@class="title"]') obj_cost = CleanDecimal('//h1[@class="clearfix"]/span[@class="price"]', replace_dots=True) obj_currency = Regexp(CleanText('//h1[@class="clearfix"]/span[@class="price"]'), '.*([%s%s%s])' % (u'€', u'$', u'£'), default=u'€') obj_area = CleanDecimal(Regexp(CleanText('//h1[@class="clearfix"]/span[@class="title"]'), '(.*?)(\d*) m\xb2(.*?)', '\\2'), default=NotAvailable) obj_price_per_meter = PricePerMeterFilter() obj_location = CleanText('//div[@class="item-geoloc"]/h2') obj_text = CleanText(CleanHTML('//p[@class="item-description"]')) obj_station = CleanText('//div[@class="metro"]') obj_phone = CleanHTML('(//div[has-class("tel-wrapper")])[1]') obj_url = BrowserURL('housing', _id=Env('_id')) def obj_details(self): details = dict() for item in XPath('//ul[@class="item-summary"]/li')(self): key = CleanText('.', children=False)(item) value = CleanText('./strong')(item) if value and key: details[key] = value key = CleanText('//div[@class="box energy-box"]/div/div/p[@class="h3"]')(self) value = Format('%s(%s)', CleanText('(//div[@class="box energy-box"]/div/div/p)[2]'), CleanText('//div[@class="box energy-box"]/div/div/@class', replace=[('-', ''), ('rank', '')]))(self) if value and key: details[key] = value return details def obj_photos(self): photos = [] for img in XPath('//div[has-class("showcase-thumbnail")]/img/@src')(self): photos.append(HousingPhoto(u'%s' % img)) return photos �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pap/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000002462�13034501105�0016406�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.housing import Query from weboob.tools.test import BackendTest class PapTest(BackendTest): MODULE = 'pap' def test_pap(self): query = Query() query.area_min = 20 query.cost_max = 900 query.type = Query.TYPE_RENT query.cities = [] for city in self.backend.search_city('paris'): city.backend = self.backend.name query.cities.append(city) results = list(self.backend.search_housings(query)) self.assertTrue(len(results) > 0) self.backend.fillobj(results[0], 'phone') ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016273�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import ParisKiwiModule __all__ = ['ParisKiwiModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000010642�13034501105�0020333�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import unicode_literals from datetime import datetime, time import re from weboob.browser.browsers import APIBrowser __all__ = ['ParisKiwiBrowser'] CAL = 'Agenda/Detruire_Ennui_Paris' class ParisKiwiBrowser(APIBrowser): PROTOCOL = 'https' DOMAIN = 'pariskiwi.org' BASEURL = 'https://pariskiwi.org' ENCODING = 'utf-8' def list_events_all(self): ids = [] cont = '' # titles are in m-d-y format, so we're forced to fetch everything while True: data = self.request('/api.php?action=query&list=allpages&apprefix=%s&aplimit=500&format=json&apcontinue=%s' % (CAL, cont)) ids.extend(id_from_title(p['title']) for p in data['query']['allpages']) if 'continue' in data: cont = data['continue']['apcontinue'] else: break ids = [_id for _id in ids if _id and _id != 'style'] ids.sort(key=date_from_id) for _id in ids: yield { 'id': _id, 'date': date_from_id(_id), } def get_event(self, _id): _id = id_from_title(_id) j = self.request('/api.php?action=query&format=json&prop=revisions&rvprop=content&rvlimit=1&titles=%s/%s' % (CAL, _id)) pages = j['query']['pages'] page = pages[pages.keys()[0]] text = page['revisions'][0]['*'] res = { 'id': _id, 'date': date_from_id(_id), 'datetime': date_from_id(_id), 'url': 'https://pariskiwi.org/index.php/%s/%s' % (CAL, _id), 'description': text, 'summary': find_title(text), } match = re.search(r'\b(\d\d?)h(\d\d)?\b', text) if match: res['hour'] = time(int(match.group(1)), int(match.group(2) or '0')) res['datetime'] = combine(res['date'], res['hour']) text = text[:match.start(0)] + text[match.end(0):] match = re.search(ur'\b(\d+([,.]\d+)?)\s*(euros\b|euro\b|€)', text) if match: res['price'] = float(match.group(1).replace(',', '.')) text = text[:match.start(0)] + text[match.end(0):] res['address'] = find_address(text) if not res['address']: res.pop('address') return res def id_from_title(title): return title.rsplit('/', 1)[-1].replace(' ', '_') def date_from_id(_id): _id = id_from_title(_id).split('_', 1)[0] return datetime.strptime(_id, '%m-%d-%Y') def id_from_path(title): return title.replace(' ', '_').split('/')[-1] def combine(dt, t): return datetime(dt.year, dt.month, dt.day, t.hour, t.minute) def find_title(text): for line in text.split('\n'): line = line.strip() line = re.sub(r'^=+(.*)=+$', '', line) if line: return line def find_address(text): address = [] parts = text.split('\n') for n, p in enumerate(parts): match = re.search(r'\d+[\s,]+(rue|boulevard|avenue)\s+.+', p, re.I) if match: address.append(match.group(0)) p = parts[n] = p[:match.start(0)] + p[match.end(0):] match = re.search(r'\b(75|92|93|94|78|77|95|91)\d\d\d\b.*', p) if match: address.append(match.group(0)) p = parts[n] = p[:match.start(0)] + p[match.end(0):] match = re.search(r'\b(m.tro|rer)\b.*', p, re.I) if match: address.append(match.group(0)) p = parts[n] = p[:match.start(0)] + p[match.end(0):] match = re.search(r'@\s+\w+(\s+[^.]+.*)?', p) # refuse '@foo' or '@ foo . plop' if match: address.append(match.group(0)) p = parts[n] = p[:match.start(0)] + p[match.end(0):] return ' '.join(address) ����������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000001634�13034501105�0020432�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD�c�=�Xw��� pHYs�� �� ����tIME 6 '���tEXtComment�Created with GIMPW��IDATxZKNA@"eaqPv [#rYs΀@bkl&iVzơKiޫWjJӢE-Z|ڀ G~~2H{'Lo3wz &{P ~,U2? ]Pa s._H*0w}ȐnT)Wo!M~,0) /XOյD$LH$X),z6_^bm=Pd/ 5IE :D�n $ !{IxEDߏF8�oSLbG~͑c_^Q0-g?vm*X`j64T)?@5CX)kf9Cp,&(ZÌ_NaAHciR2$u?:[a=b㎓An}!Jן0: /� 5Ԯ@" @+M-R +n=`}n6gmhm7ԶHm%-KD ;s d̛]^q%#$k!R$r z@ wmg܆)qw Xw bW7AN9pdOn0Qjuf;TY>(V GC?<^d�Fr05!lL/tC0"uT�5 /ꪌ2Ǜۧ( y)_,% f,Ir@/MM*q2-Zhq rZN����IENDB`����������������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/module.py��������������������������������������������������������������0000664�0000000�0000000�00000005534�13034501105�0020141�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.calendar import CapCalendarEvent, BaseCalendarEvent, CATEGORIES, TRANSP, STATUS from datetime import datetime, time from .browser import ParisKiwiBrowser __all__ = ['ParisKiwiModule'] class ParisKiwiModule(Module, CapCalendarEvent): NAME = 'pariskiwi' DESCRIPTION = u'ParisKiwi website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ParisKiwiBrowser ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT] def search_events(self, query): if self.has_matching_categories(query): for event in self.list_events(query.start_date, query.end_date or None): yield event def list_events(self, date_from, date_to=None): for d in self.browser.list_events_all(): if date_from and d['date'] < date_from: continue if date_to and d['date'] > date_to: break event = self.get_event(d['id']) if event: yield event def get_event(self, _id): d = self.browser.get_event(_id) if not d: return None return self._make_event(d) def _make_event(self, d): event = BaseCalendarEvent(d['id']) event.city = u'Paris' event.url = d['url'] event.start_date = d['datetime'] event.end_date = datetime.combine(d['datetime'].date(), time.max) event.summary = d['summary'] event.category = CATEGORIES.CONCERT event.description = d['description'] event.status = STATUS.CONFIRMED event.transp = TRANSP.OPAQUE if 'price' in d: event.price = d['price'] if 'address' in d: event.location = d['address'] return event def _make_false_event(self): event = BaseCalendarEvent('0') event.start_date = event.end_date = datetime.utcfromtimestamp(0) event.summary = u'NON EXISTING EVENT' event.status = STATUS.CANCELLED event.category = CATEGORIES.CONCERT event.transp = TRANSP.OPAQUE return event ��������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pariskiwi/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002774�13034501105�0017636�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from datetime import datetime, date class ParisKiwiTest(BackendTest): MODULE = 'pariskiwi' def test_pariskiwi_event(self): event = self.backend.get_event('11-9-2013_-Event_2') self.assertTrue(event) self.assertIn('Belleville', event.location) self.assertEqual(event.price, 5) self.assertTrue(event.summary) self.assertEqual(event.start_date, datetime(2013, 11, 9, 20, 30)) self.assertEqual(event.url, 'https://pariskiwi.org/index.php/Agenda/Detruire_Ennui_Paris/11-9-2013_-Event_2') def test_pariskiwi_list(self): it = self.backend.list_events(datetime.now()) ev = next(it) self.assertTrue(ev) self.assertGreaterEqual(ev.start_date.date(), date.today()) ����weboob-1.2/modules/paroles2chansons/����������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017555�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/__init__.py�����������������������������������������������������0000664�0000000�0000000�00000000121�13034501105�0021660�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import Paroles2chansonsModule __all__ = ['Paroles2chansonsModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/browser.py������������������������������������������������������0000664�0000000�0000000�00000004431�13034501105�0021614�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchPage, LyricsPage, HomePage, ArtistPage import itertools __all__ = ['Paroles2chansonsBrowser'] class Paroles2chansonsBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.paroles2chansons.com/' home = URL('$', HomePage) search = URL('search', SearchPage) artist = URL('paroles-(?P<artistid>[^/]*)$', ArtistPage) lyrics = URL('paroles-(?P<artistid>[^/]*)/paroles-(?P<songid>[^/]*)\.html', LyricsPage) def iter_lyrics(self, criteria, pattern): self.home.stay_or_go() assert self.home.is_here() self.page.search_lyrics(pattern) assert self.search.is_here() if criteria == 'song': return self.page.iter_song_lyrics() elif criteria == 'artist': artist_ids = self.page.get_artist_ids() it = [] # we just take the 3 first artists to avoid too many page loadings for aid in artist_ids[:3]: it = itertools.chain(it, self.artist.go(artistid=aid).iter_lyrics()) return it def get_lyrics(self, id): ids = id.split('|') try: self.lyrics.go(artistid=ids[0], songid=ids[1]) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/favicon.png�����������������������������������������������������0000664�0000000�0000000�00000003261�13034501105�0021712�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME 9c0D��>IDATxilTUtޛ,"K)[KE@#J `1F%F@@1 q!@DH*B\ DXd)JiNK03PJ3̼w=۽ ,XH (nMs7UԊ x3o`K>׽l]$0iE)eHB\ooyѵy uN9/:Y~H3<kdg@m �Ȉ2~$`*Cf#tp͠.v<u�k� I|{�j$Γ_d&'5c cʁgM.:X \4̻ذM7כ`�8!z l~L TexİT oXjX.L .Vxc_</uN̨{͐6v`'!|q@\ /DM3y�cl)pN �٦]e./�Dn^�6ˆ~׀Q%0;EŽ;"�g[DɀT`48.4BQC ݛccS&L.C4J�,6jo/ ͡ m՘@10.o+<`,\�5Q N�ʪM*9 c[E.I|jpF A7Q'lB1ǀfBc~fa =^:Zg ,X^{S3N( U=TՐZ@qQ]kwyR&("!Xu6w8U# �Zgk?M[PGJa) ͛9z)MTzFkJ{Ҡ6F)1�~6߱6 :=xViU5i&* /oɲJ7TnPlKLm5uuDWu]&#IvKDi)]c\B?"CwZnĄ&~7֙OW\YnNǡvMK0OͰ}S%8wӭ0FU4ZĿҋḰai~�%Kzwpl(4& u_Udhמᶒ�WIlHA/j0`XR+ 8߶ܝ6AO*e`I"BHU/s,FFO[1H}_�='`eqq8ga#@VWz%zB'ʹX$yS|&uM/]we@-`/4 0k8 /7NT#-l(>-m&8+N?R~kpȜ![5 K'ヂA0t7̹<e* 5z%ѯ̣y,`>p=,%dN# 4Ұi 45n_>pcT22yvœf<@t<k4 ~R٢bc#jS b\pQl,X`$?#����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/module.py�������������������������������������������������������0000664�0000000�0000000�00000003177�13034501105�0021424�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import Paroles2chansonsBrowser __all__ = ['Paroles2chansonsModule'] class Paroles2chansonsModule(Module, CapLyrics): NAME = 'paroles2chansons' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Paroles2chansons.com song lyrics website' LICENSE = 'AGPLv3+' BROWSER = Paroles2chansonsBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, pattern.encode('utf-8')) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/pages.py��������������������������������������������������������0000664�0000000�0000000�00000007412�13034501105�0021232�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class HomePage(HTMLPage): def search_lyrics(self, pattern): form = self.get_form(xpath='//form[@class="search-block"]') form['query'] = pattern form.submit() class SearchPage(HTMLPage): @method class iter_song_lyrics(ListElement): item_xpath = '//p[text()="Chansons" and has-class("pull-left")]/../..//li[has-class("item")]' class item(ItemElement): klass = SongLyrics def obj_id(self): href = CleanText('.//a[has-class("link") and has-class("grey") and has-class("font-small")]/@href')(self) subid = href.replace('.html','').replace('paroles-','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id obj_title = CleanText('.//a[has-class("link") and has-class("grey") and has-class("font-small")]', default=NotAvailable) obj_artist = CleanText('.//a[has-class("link") and has-class("black") and has-class("font-default")]', default=NotAvailable) obj_content = NotLoaded def get_artist_ids(self): artists_href = self.doc.xpath('//p[text()="Artistes" and has-class("pull-left")]/../..//li[has-class("item")]//a[has-class("link")]/@href') aids = [href.split('/')[-1].replace('paroles-','') for href in artists_href] return aids class ArtistPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//p/text()[starts-with(.,"Toutes les")]/../../..//li[has-class("item") and has-class("clearfix")]' class item(ItemElement): klass = SongLyrics obj_title = CleanText('.//a[has-class("link")]', default=NotAvailable) obj_artist = Regexp(CleanText('//title'), 'Paroles (.*) :.*') obj_content = NotLoaded def obj_id(self): href = CleanText('.//a[has-class("link")]/@href')(self) subid = href.replace('.html','').replace('paroles-','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id class LyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): subid = self.page.url.replace('.html','').replace('paroles-','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id obj_content = CleanText(CleanHTML('//div[has-class("top-listing")]//div[has-class("text-center")]', default=NotAvailable), newlines=False) obj_title = Regexp(CleanText('//title', default=NotAvailable), 'Paroles (.*) - .*') obj_artist = Regexp(CleanText('//title', default=NotAvailable), 'Paroles .* - (.*) \(tra.*') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paroles2chansons/test.py���������������������������������������������������������0000664�0000000�0000000�00000003252�13034501105�0021110�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class Paroles2chansonsTest(BackendTest): MODULE = 'paroles2chansons' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016744�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0021062�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import ParolesmaniaModule __all__ = ['ParolesmaniaModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/browser.py����������������������������������������������������������0000664�0000000�0000000�00000004512�13034501105�0021003�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SearchSongPage, LyricsPage, SearchArtistPage, ArtistSongsPage import itertools __all__ = ['ParolesmaniaBrowser'] class ParolesmaniaBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.parolesmania.com/' searchSong = URL('recherche.php\?c=title&k=(?P<pattern>[^/]*).*', SearchSongPage) searchArtist = URL('recherche.php\?c=artist&k=(?P<pattern>[^/]*).*', SearchArtistPage) songLyrics = URL('paroles_(?P<artistid>[^/]*)/paroles_(?P<songid>[^/]*)\.html', LyricsPage) artistSongs = URL('paroles_(?P<artistid>[^/]*)\.html', ArtistSongsPage) def iter_lyrics(self, criteria, pattern): if criteria == 'artist': artist_ids = self.searchArtist.go(pattern=pattern).get_artist_ids() it = [] # we just take the 3 first artists to avoid too many page loadings for aid in artist_ids[:3]: it = itertools.chain(it, self.artistSongs.go(artistid=aid).iter_lyrics()) return it elif criteria == 'song': return self.searchSong.go(pattern=pattern).iter_lyrics() def get_lyrics(self, id): ids = id.split('|') try: self.songLyrics.go(artistid=ids[0], songid=ids[1]) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000012143�13034501105�0021100�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME5w~s��IDATx՛itu3,͒˶dY6%6+! %Є4 -Iڐ&) mB 큞' YZI Iz�oF6b.:, 1̝y̝6۶ Ð+{!{q[i\?<t|.yⷥ[ͺKX~w�_޲W:5(-߽�'-Ν[OٳҸpa5p\JJ$k/iz̹f)3h`ႅOI+UhY-SLN�f!ďr$WQ_W.̙ں5{'H'?�mQS倎P ̘QU_1&� q0 TZ#c$;` ׸~B*+R 4FV]or̙qa%%`N;(xU)`;)\5W-?KSַn];#q0F!@ �!� ]4;8WGRVSxfW YIaуhO#>lقgGG(� ¯D^GQiE>R� 0ZfA-[u͍0WZ#Lcsp�,>`$1\gg|5p)C{K)]'i ;=p{,u遇Oٍ][tqsa]N[S4OoTQa%sůR5:*5#ů5sJ,%/x JQ#J' Fb'GJk�p8ŚB 9D ڱ_ӜofY{g=�'.!@X\@8 ]o 7A-ed,M)^Ha�Bi[Ί% UmFs1:rWGyhl\(�?xN#t@l%g@h!Fѡ]�e9VV4�H/܈s={1j;˵L@ie̗>�ɋ"&c,!|}EZ1>1wJR9hn3,?OT^3Itu8n$i ,_w{�1f_( 1khDq%֝H$} Ə<[EKy7`ٴ  @@oߘ\\�2cKLPiEôH,s7_~u4 +qo$�ӼSƛc#Лdfo:M:њcD'YcJ-%KjGI.�De|^{|^X5 !�lSg.%1P0`f<W'JamHJ\ ;T%ʼnRqf\uY <~1: f#ژT װ>M㷏vi=ڥ`Nmp 0\Llş'M.}7~Eyweێ,jIile_݂Lspo13Wb$MpIv_T}Q? w ѱAi-4+}? p P Txq}08}B  6e}g�@ӜYH PXG ¢;wtŽ:I$K|]AX9Ėm?8H!4h,[PtyoKɊs9{n#FI|G{m;i?iZ,+ÿ5/7 :GA2~zLJ*h#)O]i b.;q։xG>i$CJfON�@ B<0~[wWz1,2 dNpPZ[8RKז�%x~Bb|KP4֓o`%]:^zMv,+#=IhL%uGUe%j&�Ypmmg۱Gw�1>;32jO~ 75!/1zj%fL. eض$ CHϡ#}s[y#<i'}itu1402!ͨg**&�UV:3  x<f~ݧSY0yޞ"b׮=TT‹{9�}r;w#Vڶ_cW+Σu~ )--ql >~H˜4MM|O;AT.g 4Zk(~s*)H)&�@D%sF P V"JH p\ 8qM%?ʶcʑ>@ +nb o h| ^keQ! " }<b:± E0̡3ftl/pUrx@k|/"#dR^BZeI$H6(qFH1Q0>+\ /ń!QDzx@$ Qʥ-3 4DFN20R "cE :f `^#P?\7YeM D! Hxyԕ)MsEp!( F&u WVb)Ken*_gnB^lx=IEY.[̺)|?i Q-!&a׉& W^WKMEg4ͮ: ,Ɯ �ߋ4ʟ}*_|eST%%q4IlҥP9 ]v1QDn^]ںE+bN{;c.Ys- jH%%r #}Hl/m32Ů rqe)zƵSV^aoؽ RLGZ6nId ,GU45NggSQYr/ee^[oz/ȑg1D{s?lYrN9յɖ3uj鏹 e:47p٥+R჻0;n?2 cF&f›:A;GԩS 4n߳rbH@\I)J :_?5s]-hH%<DD(o0 <.!FZ y}_!6G}}~}>n$7|J^# " | Yo�'Q Bĕ=H: tٺd=?Pgbeq=e(2<bCJYHSb$qlc$RZDFHXeyd):՜E硣$2JJKReYhmc RFQGc"PU&]P(6Yqn<5RqI)%BČ1Da`b@*>, i9DGttQ[{ ,F:f4;wvо|<h1_0/R @('1�J/3]F#b ##l{,pQ(h\sbfㆍ g�X/@ Bu@L,�v(Y @1& PBZ}zf ,XL2QŲC&a``; 5 FhF1\ ijj 2cD%m'PƇ?x%D CG)xX{njJJ " a`I q�'"˜ꕋV]ǯqF1yO32:B*Uʂ-L1l|E܄1\GwÍ7 ~(X[!,Q !mB"#2S@`.rÇ.eղVΙD9MEy!r8Jz H2C̮/W.r-� )8V:,JtIMMV%^ X|UIscH7B.< Il>\3n%ֱ{w'Me'sXAԤ:x|˺?=H$S_ƥR(&?0(!Ҋi$K*شu7Ǻ$Zū[;6JkYFvY@_>ʳn'& <6>aźUh4mٺm'q?#C";:º;l~ qNE)sL40/}^^f]N8x\fϫ̎Pȱ>VK?Eg`T!m:OrޫFiU.򵠆WKA{8i8 :2!x:Q5|n)Ncs)%TV'y*y_h:ZڝfC>8qF8OG̥Ru*]HS ЏBǰ1qD?V?2ժ9N+nw3JV-{a\^E87h5ȐD1s;]AϢb!Schu?V0??Fp} M}0X!؆8M`qh$)g +. +Pw\q�Idd䒱ZN('EFe,�B3d$GLYc>1TWASt%BF['.Bˈ5LTXc{8lIVm@;*|"B[3F}$:IVP=MD9ŚOkgxq>sRlq8!{cAmA5}FqTFa(ON3~-nl/C _����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/module.py�����������������������������������������������������������0000664�0000000�0000000�00000003212�13034501105�0020601�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import ParolesmaniaBrowser from urllib import quote_plus __all__ = ['ParolesmaniaModule'] class ParolesmaniaModule(Module, CapLyrics): NAME = 'parolesmania' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'Paroles Mania lyrics website' LICENSE = 'AGPLv3+' BROWSER = ParolesmaniaBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, quote_plus(pattern.encode('utf-8'))) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/pages.py������������������������������������������������������������0000664�0000000�0000000�00000006557�13034501105�0020432�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class SearchSongPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[has-class("elenco")]//div[has-class("col-left")]//li//a[starts-with(@href, "/paroles") and not(contains(@href, "alpha.html"))]' class item(ItemElement): klass = SongLyrics def obj_id(self): href = CleanText('./@href')(self) subid = href.replace('.html','').replace('paroles_','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id obj_title = Regexp(CleanText('.', default=NotAvailable), '(.*) - .*') obj_artist = Regexp(CleanText('.', default=NotAvailable), '.* - (.*)') obj_content = NotLoaded class SearchArtistPage(HTMLPage): def get_artist_ids(self): artists_href = self.doc.xpath('//div[has-class("elenco")]//div[has-class("col-left")]//li//a/@href') aids = [href.split('/')[-1].replace('paroles_', '').replace('.html', '') for href in artists_href] return aids class ArtistSongsPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[has-class("album")]//ul//li//a[starts-with(@href, "/paroles") and not(contains(@href, "alpha.html"))]' class item(ItemElement): klass = SongLyrics obj_title = CleanText('.', default=NotAvailable) obj_artist = Regexp(CleanText('//head/title'), 'Paroles (.*)') obj_content = NotLoaded def obj_id(self): href = CleanText('./@href')(self) subid = href.replace('.html','').replace('paroles_','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id class LyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): subid = self.page.url.replace('.html','').replace('paroles_','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id obj_content = CleanText(CleanHTML('//div[has-class("lyrics-body")]/*[not(contains(@id, "video"))]', default=NotAvailable), newlines=False) obj_title = Regexp(CleanText('//title', default=NotAvailable), 'Paroles (.*) - .*') obj_artist = Regexp(CleanText('//title', default=NotAvailable), 'Paroles .* - (.*)') �������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmania/test.py�������������������������������������������������������������0000664�0000000�0000000�00000003243�13034501105�0020277�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class ParolesmaniaTest(BackendTest): MODULE = 'parolesmania' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017347�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/__init__.py�������������������������������������������������������0000664�0000000�0000000�00000000115�13034501105�0021455�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import ParolesmusiqueModule __all__ = ['ParolesmusiqueModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/browser.py��������������������������������������������������������0000664�0000000�0000000�00000004676�13034501105�0021421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import SongResultsPage, SonglyricsPage, ArtistResultsPage, ArtistSongsPage, HomePage import itertools __all__ = ['ParolesmusiqueBrowser'] class ParolesmusiqueBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.paroles-musique.com/' home = URL('$', HomePage) songResults = URL('lyrics-paroles-0-.*,0.php', SongResultsPage) artistResults = URL('lyrics-paroles-.*-0,0.php', ArtistResultsPage) songLyrics = URL('paroles-(?P<songid>.*,p[0-9]*)', SonglyricsPage) artistSongs = URL('paroles-(?P<artistid>.*,a[0-9]*)', ArtistSongsPage) def iter_lyrics(self, criteria, pattern): self.home.stay_or_go() assert self.home.is_here() self.page.search_lyrics(criteria, pattern) if criteria == 'song': assert self.songResults.is_here() return self.page.iter_lyrics() elif criteria == 'artist': assert self.artistResults.is_here() artist_ids = self.page.get_artist_ids() it = [] # we just take the 3 first artists to avoid too many page loadings for aid in artist_ids[:3]: it = itertools.chain(it, self.artistSongs.go(artistid=aid).iter_lyrics()) return it def get_lyrics(self, id): try: self.songLyrics.go(songid=id) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return ������������������������������������������������������������������weboob-1.2/modules/parolesmusique/favicon.png�������������������������������������������������������0000664�0000000�0000000�00000003213�13034501105�0021501�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME/=yO�� IDATx{ebjEyK^: k0aR'*/e],9bڍB [ii㺌AcE%**>úgI{`ywo}ߙYPt)d& H�:յK8m :G`:#Cu��L@{`J=Svk]_8ϱ:9=VYp"�VS$ 9e,PG&g@_#vCEjgw`):'2lՆR)RwC:6&#?kЩL"�k.Ճ[Wt:%ŎEXG h�<+R cc^90X ^h - @/{K&}X? V\ke�56V/ 2FJ-~[Mn@J`Q'>b�}fSnjɣn|.RhiYŸ~#A B:T s4O(nV5p l&m-9w]oElٴqM9!0 Ƈ><T�Dv: I;ZEB/Չ)l_-n1pqDa&e˶1oR`6;pA3y |XkD!覕E3%ˀJV'\#V�RpX:~�8,B xXxUA#M<^];E0MIn [uZMǾv(:Ϩvtd*}g~[kU2QK$>׋{ke.IכI@il%7X3 [7tMJ~@i�%" l;y;DL�ZWcў:%S*4U%S2G59-߫16%8ZbJ^ߺFN(8E2t긶;`@E4ǸxTBGvslIM2u$y}'SQj^hE˒tK)IUԀ1VwlTzS:yFZC+2jl |t2n*xCIgg/ϧg*4}rn091|UsTEP C P))@6#B3Y-s*JMXa|-6(G!MhjƨT-pjY2*0#M)IF6T;E97S,B3|/1PLHL`荵6ԣ6N(-zy{T5# |oklW"45?hcG*}LQ;R&I=ښ)0;^>Eq)i{r1˹?H~N$S(4#Eu]QN ]zti; & q]{/Ŋ"@7B`aaaaaaaaaaaaq9SM����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/module.py���������������������������������������������������������0000664�0000000�0000000�00000003153�13034501105�0021210�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import ParolesmusiqueBrowser __all__ = ['ParolesmusiqueModule'] class ParolesmusiqueModule(Module, CapLyrics): NAME = 'parolesmusique' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'paroles-musique lyrics website' LICENSE = 'AGPLv3+' BROWSER = ParolesmusiqueBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, pattern.encode('utf-8')) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/pages.py����������������������������������������������������������0000664�0000000�0000000�00000007222�13034501105�0021023�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText, Format from weboob.browser.filters.html import CleanHTML import random class HomePage(HTMLPage): def search_lyrics(self, criteria, pattern): form = self.get_form(xpath='//form[@name="rechercher"]') form['query'] = pattern if criteria == 'artist': form['termes_a'] = pattern else: form['termes_t'] = pattern form.submit() class ArtistResultsPage(HTMLPage): def get_artist_ids(self): artists_href = self.doc.xpath('//div[has-class("cont_cat")]//a[has-class("matchA")]/@href') aids = [href.split('/')[-1].replace('paroles-','') for href in artists_href] return aids class ArtistSongsPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//td[has-class("art_titr")]//a' class item(ItemElement): klass = SongLyrics obj_title = CleanText('.', default=NotAvailable) obj_artist = CleanText('//h1[@id="art_title"]', default=NotAvailable) # little trick because the damn site potentially shows identical songs in results # the dummy added prefix number does not annoy the module # it seems this part of the URL is not red anyway def obj_id(self): res = Format('%s%s', int(random.random()*100000), Regexp(CleanText('./@href', default=NotAvailable), 'paroles-(.*)'))(self) return res obj_content = NotLoaded class SongResultsPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[has-class("cont_cat")]//table//tr[position() > 1]' class item(ItemElement): klass = SongLyrics obj_title = CleanText('.//a[has-class("matchT")]', default=NotAvailable) obj_id = Regexp(CleanText('.//a[has-class("matchT")]/@href', default=NotAvailable), 'paroles-(.*)') obj_artist = CleanText('.//a[has-class("matchA")]', default=NotAvailable) obj_content = NotLoaded class SonglyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics obj_content = CleanText(CleanHTML('//div[@id="lyr_scroll"]', default=NotAvailable), newlines=False) obj_title = CleanText('//div[@id="main_ct"]//ul[has-class("semiopaquemenu")]//li[position()=3]', default=NotAvailable) obj_artist = CleanText('//div[@id="main_ct"]//ul[has-class("semiopaquemenu")]//li[position()=2]', default=NotAvailable) obj_id = Regexp(CleanText('//div[@id="main_ct"]//ul[has-class("semiopaquemenu")]//li[position()=3]//a/@href', default=NotAvailable), 'paroles-(.*)') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesmusique/test.py�����������������������������������������������������������0000664�0000000�0000000�00000003247�13034501105�0020706�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class ParolesmusiqueTest(BackendTest): MODULE = 'parolesmusique' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016445�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000000105�13034501105�0020552�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import ParolesnetModule __all__ = ['ParolesnetModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/browser.py������������������������������������������������������������0000664�0000000�0000000�00000004361�13034501105�0020506�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import PagesBrowser from weboob.browser.url import URL from weboob.browser.profiles import Firefox from .pages import ResultsPage, SongLyricsPage, HomePage, ArtistSongsPage import itertools __all__ = ['ParolesnetBrowser'] class ParolesnetBrowser(PagesBrowser): PROFILE = Firefox() TIMEOUT = 30 BASEURL = 'http://www.paroles.net/' home = URL('$', HomePage) results = URL('search', ResultsPage) lyrics = URL('(?P<artistid>[^/]*)/paroles-(?P<songid>[^/]*)', SongLyricsPage) artist = URL('(?P<artistid>[^/]*)$', ArtistSongsPage) def iter_lyrics(self, criteria, pattern): self.home.stay_or_go() assert self.home.is_here() self.page.search_lyrics(pattern) assert self.results.is_here() if criteria == 'song': return self.page.iter_song_lyrics() else: artist_ids = self.page.get_artist_ids() it = [] # we just take the 3 first artists to avoid too many page loadings for aid in artist_ids[:3]: it = itertools.chain(it, self.artist.go(artistid=aid).iter_lyrics()) return it def get_lyrics(self, id): ids = id.split('|') try: self.lyrics.go(artistid=ids[0], songid=ids[1]) songlyrics = self.page.get_lyrics() return songlyrics except BrowserHTTPNotFound: return �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000002574�13034501105�0020610�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME  *'h(��IDATx[VU,S(gl`f3 Jbf]^Pm" */TD,Rbd+ 8i53(~c˷ٗ^ksq&es݀@);qKr& }+`q"q0<@ `,p8`P:7p?+֠yTiPm?Pks)XgRf0\ �js4}&`]"2A(@O`|"j7G@W`0;ٝ:WՔ.ѱKDC@܅ Im ҉q.0<!l`1 x1s Թ|NVhjƭv3{l TaN2 Xn|ViUf{uKZc^ךx<unV`0�hsD1un(�Chnچl-2vC99&"s95m&"z-RB�%";SW QHNz} Y]Ve&0}lHpX_ ŒL[}Ԩ}XZo�$unLnLgrW"T,tIM ϶cVkLۖ?�끥FDԹ-i:̋�{<#1S5{Xv?wiγwk |ssƷok)PV&/ ҀuDdB۲Lqm({C�|ʺ[p 0xU˝@5CsE@"�땈щJ:I끚D)y\�{�] x`F"!R})X὿]|wyKO`_P/% *=-p~+}5'81 Xoow6) �{e!!z9_B:}Y!@Qb~Fj=Lי z!.Eūf1UkN?o{Ú!۩^N7@g]v#{`yG׳I@&W!Yϱ~5{[=pSG�{A/{BKϐB•7ߞd0-9*}=-ȉ}t=u#""""""""""""""""""[\W8f����IENDB`������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/module.py�������������������������������������������������������������0000664�0000000�0000000�00000003134�13034501105�0020305�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import CapLyrics, SongLyrics from weboob.tools.backend import Module from .browser import ParolesnetBrowser __all__ = ['ParolesnetModule'] class ParolesnetModule(Module, CapLyrics): NAME = 'parolesnet' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'paroles.net lyrics website' LICENSE = 'AGPLv3+' BROWSER = ParolesnetBrowser def get_lyrics(self, id): return self.browser.get_lyrics(id) def iter_lyrics(self, criteria, pattern): return self.browser.iter_lyrics(criteria, pattern.encode('utf-8')) def fill_songlyrics(self, songlyrics, fields): if 'content' in fields: sl = self.get_lyrics(songlyrics.id) songlyrics.content = sl.content return songlyrics OBJECTS = { SongLyrics: fill_songlyrics } ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000007033�13034501105�0020121�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.lyrics import SongLyrics from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import Regexp, CleanText from weboob.browser.filters.html import CleanHTML class HomePage(HTMLPage): def search_lyrics(self, pattern): form = self.get_form(xpath='//form[@id="search-form-round"]') form['search'] = pattern form.submit() class ResultsPage(HTMLPage): @method class iter_song_lyrics(ListElement): item_xpath = '//h2[text()="Chansons"]/following-sibling::div[position() <= 2]//tr' class item(ItemElement): klass = SongLyrics def obj_id(self): href = CleanText('.//td[has-class("song-name")]//a/@href')(self) aid = href.split('/')[-2] sid = href.split('/')[-1].replace('paroles-','') id = '%s|%s'%(aid, sid) return id obj_title = CleanText('.//td[has-class("song-name")]', default=NotAvailable) obj_artist = CleanText('.//td[has-class("song-artist")]', default=NotAvailable) obj_content = NotLoaded def get_artist_ids(self): artists_href = self.doc.xpath('//h2[text()="Artiste"]/following-sibling::div[position() <= 2]//tr//a/@href') aids = [href.split('/')[-1] for href in artists_href] return aids class ArtistSongsPage(HTMLPage): @method class iter_lyrics(ListElement): item_xpath = '//div[@id="main"]//div[has-class("song-listing-extra")]//td[has-class("song-name")]//a' class item(ItemElement): klass = SongLyrics obj_title = CleanText('.', default=NotAvailable) obj_artist = Regexp(CleanText('//div[has-class("breadcrumb")]//span[has-class("breadcrumb-current")]'), 'Paroles (.*)') obj_content = NotLoaded def obj_id(self): href = CleanText('./@href')(self) aid = href.split('/')[-2] sid = href.split('/')[-1].replace('paroles-','') id = '%s|%s'%(aid, sid) return id class SongLyricsPage(HTMLPage): @method class get_lyrics(ItemElement): klass = SongLyrics def obj_id(self): subid = self.page.url.replace('paroles-','').split('/')[-2:] id = '%s|%s'%(subid[0], subid[1]) return id obj_content = CleanText(CleanHTML('//div[has-class("song-text")]', default=NotAvailable), newlines=False) obj_title = CleanText('//span[@property="v:name"]', default=NotAvailable) obj_artist = CleanText('//span[@property="v:artist"]', default=NotAvailable) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/parolesnet/test.py���������������������������������������������������������������0000664�0000000�0000000�00000003237�13034501105�0020003�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded class ParolesnetTest(BackendTest): MODULE = 'parolesnet' def test_search_song_n_get(self): l_lyrics = list(self.backend.iter_lyrics('song', 'chien')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded full_lyr = self.backend.get_lyrics(songlyrics.id) assert full_lyr.id assert full_lyr.title assert full_lyr.artist assert full_lyr.content is not NotLoaded def test_search_artist(self): l_lyrics = list(self.backend.iter_lyrics('artist', 'boris')) for songlyrics in l_lyrics: assert songlyrics.id assert songlyrics.title assert songlyrics.artist assert songlyrics.content is NotLoaded �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016563�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001445�13034501105�0020700�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PastealaconModule __all__ = ['PastealaconModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000007355�13034501105�0020632�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.paste import BasePaste, PasteNotFound from weboob.browser.filters.standard import CleanText, DateTime, Env, RawText, Regexp from weboob.browser.pages import HTMLPage from weboob.browser.browsers import PagesBrowser from weboob.browser.url import URL from weboob.browser.elements import ItemElement, method from weboob.exceptions import BrowserHTTPNotFound class Spam(Exception): def __init__(self): super(Spam, self).__init__("Detected as spam and unable to handle the captcha") class PastealaconPaste(BasePaste): # all pastes are public public = True # TODO perhaps move this logic elsewhere, remove this and id2url from capability # (page_url is required by pastoob) @property def page_url(self): return '%s%s' % (PastealaconBrowser.BASEURL, self.id) class PastePage(HTMLPage): @method class fill_paste(ItemElement): klass = PastealaconPaste obj_id = Env('id') obj_title = Regexp(CleanText('id("content")/h3'), r'Posted by (.+) on .+ \(') obj__date = DateTime(Regexp(CleanText('id("content")/h3'), r'Posted by .+ on (.+) \(')) obj_contents = RawText('//textarea[@id="code"]') def parse(self, el): # there is no 404, try to detect if there really is a content if len(el.xpath('id("content")/div[@class="syntax"]//ol')) != 1: raise PasteNotFound() class CaptchaPage(HTMLPage): pass class PostPage(HTMLPage): def post(self, paste, expiration=None): form = self.get_form(name='editor') form['code2'] = paste.contents form['poster'] = paste.title if expiration: form['expiry'] = expiration form.submit() class PastealaconBrowser(PagesBrowser): BASEURL = 'http://paste.alacon.org/' paste = URL(r'(?P<id>\d+)', PastePage) captcha = URL(r'%s' % re.escape('pastebin.php?captcha=1'), CaptchaPage) raw = URL(r'%s(?P<id>\d+)' % re.escape('pastebin.php?dl=')) post = URL(r'$', PostPage) @paste.id2url def get_paste(self, url): url = self.absurl(url, base=True) m = self.paste.match(url) if m: return PastealaconPaste(m.groupdict()['id']) def fill_paste(self, paste): """ Get as much as information possible from the paste page """ self.paste.stay_or_go(id=paste.id) return self.page.fill_paste(paste) def get_contents(self, _id): """ Get the contents from the raw URL This is the fastest and safest method if you only want the content. Returns unicode. """ try: # despite what the HTTP header says, it is iso8859 return self.raw.open(id=_id).content.decode('iso8859-15') except BrowserHTTPNotFound: raise PasteNotFound() def post_paste(self, paste, expiration=None): self.post.stay_or_go().post(paste, expiration=expiration) if self.captcha.is_here(): raise Spam() self.page.fill_paste(paste) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000004633�13034501105�0020724�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME7Uk���tEXtComment�Created with GIMPW��IDATx{T?;ٙ]XV\Qb.T j`F4GӐj1ilS[Fcc"ES1EQ4Z>hQ`]>f1gu2e7Mn2|={5PC 5PC GJA0P�: @ ,~|d G*`+PtZ_a`NUHGT,$|8`` cӋDz:z׊ <:? Pb` WŮLKUF| *x~V1̈́L56?>D(>2^<V_S1?dk2q>27c#vNP& ,?5D87"ij# =�dHL�HV3Y \Xύ9d(\W l !(xB�B&S&0B\qG+{4L/ld Upk8�`-Bϟ2zBU�flI4cg {mZ{{g/vA*{8�a_Zy{9S~=/_43ob}Wڶfo''CpgQv`۴aUypwBt;�כ<{Ułsύ 3|g u(-A t? -b`$yg"* M$O{Yk57ds m7IgW\xv3Å�'Qr},o!|u\+x{*?v€S`Ό545&x^.nk}HD"Dlw{p ht/IvNsِ4�],<bρ, `V&B׀diѪSZ\ew0aM-F+sE)zĥm,fvOI>0+]cAAklK=QEl2=oE6ś窼C87TG0dԓgk�c q.Ho(޻t^Ztt|#@X!'_79'hqNx rI+vudO_xj |l#4;|Ȝ|ޮ| <oKuؒŭٜG2a ):='�[h;h/c6Q[rCi'�@]ڂ(Ncf&ƌoI+MkO$imQFwu 18wЍ%oە~k�`; \K%C!@ I`BPn97ƭyjxkZ �qNf9z 㢯iM!x3?U N)h+} +a!:@y9/t_]E #hEO U4BZH+=Vvߊf AhFH I5k>wZ � N!d^+\B{QH +nv?v@ϿaTP,{Va#+H+~@N6|·:sP~gZ+4" !$*Ru'/} )@h5+ղvhKSZt$rra%R�;+h]*~^Oʇ;*hNy�^ؕjY=t 09teZAInf�`淓s.kDv)G!v\JMkY洫<Q"|�8][ړW�KB?!-AZqh@1ٗ��[Zִig!#a0Ғdg2f89AAɟ*Ϲ/ :p/{:�!$wnph^~5Nnb i'2Ri3{ ~:ޔ/C[k^)\5Fe'2H(/v<0z. JnQ^B&/ȑw[7{yPu*(b \OMK�z^]A9b~oxʗѐmw5#\ N%{|^6U`44ui5 t_x/<tjjj8.@=����IENDB`�����������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/module.py������������������������������������������������������������0000664�0000000�0000000�00000005271�13034501105�0020427�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.tools.capabilities.paste import BasePasteModule from weboob.tools.backend import Module from weboob.capabilities.base import NotLoaded from .browser import PastealaconBrowser, PastealaconPaste class PastealaconModule(Module, BasePasteModule): NAME = 'pastealacon' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' DESCRIPTION = u'Paste à la con text sharing tool' LICENSE = 'AGPLv3+' BROWSER = PastealaconBrowser EXPIRATIONS = { 24 * 3600: 'd', 24 * 3600 * 30: 'm', False: 'f', } def new_paste(self, *args, **kwargs): return PastealaconPaste(*args, **kwargs) def can_post(self, contents, title=None, public=None, max_age=None): try: contents.encode('ISO-8859-1') except UnicodeEncodeError: return 0 if public is False: return 0 if max_age is not None: if self.get_closest_expiration(max_age) is None: return 0 # the "title" is filtered (does not even accepts dots) if not title or re.match('^\w+$', title) and len(title) <= 24: return 2 return 1 def get_paste(self, _id): return self.browser.get_paste(_id) def fill_paste(self, paste, fields): # if we only want the contents if fields == ['contents']: if paste.contents is NotLoaded: contents = self.browser.get_contents(paste.id) paste.contents = contents # get all fields elif fields is None or len(fields): self.browser.fill_paste(paste) return paste def post_paste(self, paste, max_age=None): if max_age is not None: expiration = self.get_closest_expiration(max_age) else: expiration = None self.browser.post_paste(paste, expiration=self.EXPIRATIONS.get(expiration)) OBJECTS = {PastealaconPaste: fill_paste} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastealacon/test.py��������������������������������������������������������������0000664�0000000�0000000�00000007725�13034501105�0020127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest, skip_without_config from weboob.capabilities.base import NotLoaded from weboob.capabilities.paste import PasteNotFound class PastealaconTest(BackendTest): MODULE = 'pastealacon' def _get_paste(self, _id): # html method p = self.backend.get_paste(_id) self.backend.fillobj(p, ['title']) assert p.title == u'ouiboube' assert p.page_url.startswith('http://paste.alacon.org/') assert u'héhéhé' in p.contents assert p.public is True # raw method p = self.backend.get_paste(_id) self.backend.fillobj(p, ['contents']) assert p.title is NotLoaded assert p.page_url.startswith('http://paste.alacon.org/') assert u'héhéhé' in p.contents assert p.public is True @skip_without_config() def test_post(self): p = self.backend.new_paste(None, title=u'ouiboube', contents=u'Weboob Test héhéhé') self.backend.post_paste(p, max_age=3600*24) assert p.id self.backend.fill_paste(p, ['title']) assert p.title == 'ouiboube' assert p.id in p.page_url assert p.public is True # test all get methods from the Paste we just created self._get_paste(p.id) # same but from the full URL self._get_paste('http://paste.alacon.org/'+p.id) def test_spam(self): p = self.backend.new_paste(None, title=u'viagra', contents=u'http://example.com/') with self.assertRaises(Exception) as cm: self.backend.post_paste(p) self.assertEqual(cm.message, "Detected as spam and unable to handle the captcha") def test_notfound(self): for _id in ('424242424242424242424242424242424242', 'http://paste.alacon.org/424242424242424242424242424242424242'): # html method p = self.backend.get_paste(_id) self.assertRaises(PasteNotFound, self.backend.fillobj, p, ['title']) # raw method p = self.backend.get_paste(_id) self.assertRaises(PasteNotFound, self.backend.fillobj, p, ['contents']) def test_checkurl(self): # call with an URL we can't handle with this backend assert self.backend.get_paste('http://pastebin.com/nJG9ZFG8') is None # same even with correct domain (IDs are numeric) assert self.backend.get_paste('http://paste.alacon.org/nJG9ZFG8') is None assert self.backend.get_paste('nJG9ZFG8') is None def test_can_post(self): assert 0 == self.backend.can_post(u'hello', public=False) assert 1 <= self.backend.can_post(u'hello', public=True) assert 0 == self.backend.can_post(u'hello', public=True, max_age=600) assert 1 <= self.backend.can_post(u'hello', public=True, max_age=3600*24) assert 1 <= self.backend.can_post(u'hello', public=True, max_age=3600*24*3) assert 1 <= self.backend.can_post(u'hello', public=True, max_age=False) assert 1 <= self.backend.can_post(u'hello', public=None, max_age=False) assert 1 <= self.backend.can_post(u'hello', public=True, max_age=3600*24*40) assert 1 <= self.backend.can_post(u'héhé', public=True) assert 0 == self.backend.can_post(u'hello ♥', public=True) �������������������������������������������weboob-1.2/modules/pastebin/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016076�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastebin/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0020214�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PastebinModule __all__ = ['PastebinModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastebin/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000016517�13034501105�0020145�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.paste import BasePaste, PasteNotFound from weboob.browser import LoginBrowser, need_login, URL from weboob.browser.pages import HTMLPage, RawPage from weboob.browser.elements import ItemElement, method from weboob.browser.filters.standard import Base, CleanText, DateTime, Env, Filter, FilterError, RawText from weboob.browser.filters.html import Attr from weboob.exceptions import BrowserHTTPNotFound, BrowserIncorrectPassword, BrowserUnavailable class PastebinPaste(BasePaste): # TODO perhaps move this logic elsewhere, remove this and id2url from capability # (page_url is required by pastoob) @classmethod def id2url(cls, _id): return '%s%s' % (PastebinBrowser.BASEURL, _id) class BasePastebinPage(HTMLPage): @property def logged(self): for link in self.doc.xpath('//div[@id="header_bottom"]/ul[@class="top_menu"]//ul/li/a'): if link.text == 'logout': return True if link.text == 'login': return False raise BrowserUnavailable('Unable to determine login state') class LoginPage(BasePastebinPage): def login(self, username, password): form = self.get_form('myform') form['user_name'] = username form['user_password'] = password form.submit() class CleanVisibility(Filter): def filter(self, txt): if txt.startswith('Public'): return True if txt.startswith('Unlisted') or txt.startswith('Private'): return False return self.default_or_raise(FilterError('Unable to get the paste visibility')) class PastePage(BasePastebinPage): @method class fill_paste(ItemElement): klass = PastebinPaste def parse(self, el): self.env['header'] = el.find('//div[@id="content_left"]//div[@class="paste_box_info"]') obj_id = Env('id') obj_title = Base(Env('header'), CleanText('.//div[@class="paste_box_line1"]//h1')) obj_contents = RawText('//textarea[@id="paste_code"]') obj_public = Base( Env('header'), CleanVisibility(Attr('.//div[@class="paste_box_line1"]//img', 'title'))) obj__date = Base( Env('header'), DateTime(Attr('.//div[@class="paste_box_line2"]/span[1]', 'title'))) class PostPage(BasePastebinPage): def post(self, paste, expiration=None): form = self.get_form(name='myform') form['paste_code'] = paste.contents form['paste_name'] = paste.title if paste.public is True: form['paste_private'] = '0' elif paste.public is False: form['paste_private'] = '1' if expiration: form['paste_expire_date'] = expiration form.submit() class WarningPage(BasePastebinPage): def __init__(self, *args, **kwargs): raise LimitExceeded() class UserPage(BasePastebinPage): pass class BadAPIRequest(BrowserUnavailable): pass class LimitExceeded(BrowserUnavailable): pass class PastebinBrowser(LoginBrowser): BASEURL = 'http://pastebin.com/' warning = URL('warning\.php\?p=(?P<id>\d+)', WarningPage) api = URL('api/api_post\.php', RawPage) apilogin = URL('api/api_login\.php', RawPage) login = URL('login', LoginPage) userprofile = URL('u/(?P<username>.+)', UserPage) postpage = URL('$', PostPage) paste = URL('(?P<id>\w+)', PastePage) raw = URL('raw\.php\?i=(?P<id>\w+)', RawPage) def __init__(self, api_key, *args, **kwargs): super(PastebinBrowser, self).__init__(*args, **kwargs) self.api_key = api_key self.user_key = None # being connected is optionnal at the module level, so require # login only if an username is configured if self.username: self.post = need_login(self.post_paste) def fill_paste(self, paste): """ Get as much as information possible from the paste page """ try: return self.paste.stay_or_go(id=paste.id).fill_paste(paste) except BrowserHTTPNotFound: raise PasteNotFound() @paste.id2url def get_paste(self, url): m = self.paste.match(url) if m: return PastebinPaste(m.groupdict()['id']) def get_contents(self, _id): """ Get the contents from the raw URL This is the fastest and safest method if you only want the content. Returns unicode. """ try: return self.raw.open(id=_id).response.text except BrowserHTTPNotFound: raise PasteNotFound() def post_paste(self, paste, expiration=None): self.postpage.stay_or_go().post(paste, expiration=expiration) # We cannot call fill_paste because we often have a captcha # anti-spam page, and do not detect it. paste.id = self.page.params['id'] def api_post_paste(self, paste, expiration=None): data = {'api_dev_key': self.api_key, 'api_option': 'paste', 'api_paste_code': paste.contents} if self.password: data['api_user_key'] = self.api_login() if paste.public is True: data['api_paste_private'] = '0' elif paste.public is False: data['api_paste_private'] = '1' if paste.title: data['api_paste_name'] = paste.title if expiration: data['api_paste_expire_date'] = expiration res = self.open(self.api.build(), data=data, data_encoding='utf-8').text self._validate_api_response(res) paste.id = self.paste.match(res).groupdict()['id'] def api_login(self): # "The api_user_key does not expire." # TODO store it on disk if self.user_key: return self.user_key data = {'api_dev_key': self.api_key, 'api_user_name': self.username, 'api_user_password': self.password} res = self.open(self.apilogin.build(), data=data, data_encoding='utf-8').text try: self._validate_api_response(res) except BadAPIRequest as e: if str(e) == 'invalid login': raise BrowserIncorrectPassword() else: raise e self.user_key = res return res # TODO make it into a Page? def _validate_api_response(self, res): matches = re.match('Bad API request, (?P<error>.+)', res) if matches: raise BadAPIRequest(matches.groupdict().get('error')) def do_login(self): self.login.stay_or_go().login() self.page.login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastebin/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000004163�13034501105�0020235�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������C��� pHYs����od���tIME9 O ��IDATxMl\7o潱'2N*RG�"PYD 6:BeQ؄n*6,`Y  %$$nZ06I1|yg1@y3:ͻw |Iׯ�e`(@uttw9V?dÇURAA$c1F.WB`n lK%C(lFF[ot].FyꩧĚ>`ff)s%Iسl ls98s>}W/yiX⯼:tD)A>�l,G-Q8 YOWWxܠl#r=044$V<jtt>R$K6;E v킞`NK }w H6 <twof``3]]fff\.{墳{U+k}ǁlTuxYDॗbC2 q,k3رB!ϋ/cǎH$V PJ!|Ϗ6"vL&^:;]ˌy4R8nL$JQ(s^ XAP >qؾ}Qw�MlpQmL&d2XR"||ɵke ]J0`G0*LNBg)%B-Rja ZZ R4ssyFFh4x�FpmກeEH&cD"r龫>}۶16m`h7rpWӠ >@h4<om.&IR RJL@b`YQ:;;)8@)HH$u RuoTՐ /Yuq ]1MR0\HyffL�', Mߺ0]im0 )mLS =%ybICJ<ku6ͦxxDlr Q*8 |_13csȗWXIm\ԋuMv:u?^?4z… l|ޣZҨV% ضu&}6 /|PJQLO<޽ 4#RXIb�owx͓ I=.Ś@`B!}=RR1  ر19"U&'=^)45֯Oqc{)$vʖ-]4 Hllض8OiZN$7f8wnݻPm<x>f`gyݷ3J%xg~}/B 'N311O6[ʕYΜi~ڷĄD�K|\xbdbQU6lݝfbb+WJ4eb5Ν˳<�}4-B[[ݴ NgH&GI&'vpIWn�UmiR- @$/:G8mH䓿z$w {oL|;0 HDʉj[_;aPebMF!52.g.G)M 2n%`Æv 7e"t]'P(JK4VO,L GbR`6]Ks|D2oj"mFgg'r9|_2\WqrZ4XV˂hTbD"7)XyȈ8xzW)˚#^ Ccvb4C4'{c{CiSdP.WI[o)~gl~w@2AP/&X߰+_ P/ݲ:￟B];CLO(?~9a čuZٴ9{K71C طi<:BDH0V*CS*qR+ hx<ccǡ옂����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastebin/module.py���������������������������������������������������������������0000664�0000000�0000000�00000006376�13034501105�0017751�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.base import NotLoaded from weboob.tools.backend import BackendConfig, Module from weboob.tools.capabilities.paste import BasePasteModule from weboob.tools.value import Value, ValueBackendPassword from .browser import PastebinBrowser, PastebinPaste class PastebinModule(Module, BasePasteModule): NAME = 'pastebin' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' DESCRIPTION = 'Pastebin text sharing service' LICENSE = 'AGPLv3+' BROWSER = PastebinBrowser CONFIG = BackendConfig( Value('username', label='Optional username', default=''), ValueBackendPassword('password', label='Optional password', default=''), ValueBackendPassword('api_key', label='Optional API key', default='', noprompt=True), ) EXPIRATIONS = { 600: '10M', 3600: '1H', 3600 * 24: '1D', 3600 * 24 * 30: '1M', False: 'N', } def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(self.config['api_key'].get() or None, username, password) def new_paste(self, *args, **kwargs): return PastebinPaste(*args, **kwargs) def can_post(self, contents, title=None, public=None, max_age=None): if max_age is not None: if self.get_closest_expiration(max_age) is None: return 0 if not title or len(title) <= 60: return 2 return 1 def get_paste(self, _id): return self.browser.get_paste(_id) def fill_paste(self, paste, fields): # if we only want the contents if fields == ['contents']: if paste.contents is NotLoaded: paste.contents = self.browser.get_contents(paste.id) # get all fields elif fields is None or len(fields): self.browser.fill_paste(paste) return paste def post_paste(self, paste, max_age=None, use_api=True): if max_age is not None: expiration = self.get_closest_expiration(max_age) else: expiration = None if use_api and self.config.get('api_key').get(): self.browser.api_post_paste(paste, expiration=self.EXPIRATIONS.get(expiration)) else: self.browser.post_paste(paste, expiration=self.EXPIRATIONS.get(expiration)) OBJECTS = {PastebinPaste: fill_paste} ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pastebin/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000010764�13034501105�0017437�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.base import NotLoaded from weboob.capabilities.paste import PasteNotFound from weboob.tools.test import BackendTest, SkipTest from .browser import LimitExceeded class PastebinTest(BackendTest): MODULE = 'pastebin' def test_get_paste(self): for _id in ('7HmXwzyt', 'http://pastebin.com/7HmXwzyt'): # html method p = self.backend.get_paste(_id) self.backend.fillobj(p, ['title']) assert p.title == u'plop' assert p.page_url == 'http://pastebin.com/7HmXwzyt' assert p.contents == u'prout' assert p.public is True assert p._date.year == 2011 # raw method p = self.backend.get_paste(_id) self.backend.fillobj(p, ['contents']) assert p.title is NotLoaded assert p.page_url == 'http://pastebin.com/7HmXwzyt' assert p.contents == u'prout' assert p.public is NotLoaded def test_post(self): # we cannot test public pastes, as the website sometimes forces them as private # there seems to be a very low post per day limit, even when logged in p = self.backend.new_paste(None, title=u'ouiboube', contents=u'Weboob Test', public=False) try: self.backend.post_paste(p, max_age=600) except LimitExceeded: raise SkipTest("Limit exceeded") assert p.id assert not p.id.startswith('http://') self.backend.fill_paste(p, ['title']) assert p.title == u'ouiboube' assert p.id in p.page_url assert p.public is False def test_specialchars(self): # post a paste and get the contents through the HTML response p1 = self.backend.new_paste(None, title=u'ouiboube', contents=u'Weboob <test>¿¡', public=False) try: self.backend.post_paste(p1, max_age=600) except LimitExceeded: raise SkipTest("Limit exceeded") assert p1.id # not related to testing special chars, but check if the paste is # really private since test_post() tests the contrary assert p1.public is False # this should use the raw method to get the contents p2 = self.backend.get_paste(p1.id) self.backend.fillobj(p2, ['contents']) assert p2.contents == p1.contents assert p2.public is NotLoaded def test_notfound(self): for _id in ('weboooooooooooooooooooooooooob', 'http://pastebin.com/weboooooooooooooooooooooooooob'): # html method p = self.backend.get_paste(_id) self.assertRaises(PasteNotFound, self.backend.fillobj, p, ['title']) # raw method p = self.backend.get_paste(_id) self.assertRaises(PasteNotFound, self.backend.fillobj, p, ['contents']) def test_checkurl(self): # call with an URL we can't handle with this backend assert self.backend.get_paste('http://pastealacon.com/1') is None def test_can_post(self): assert self.backend.can_post(u'hello', public=None) > 0 assert self.backend.can_post(u'hello', public=True) > 0 assert self.backend.can_post(u'hello', public=False) > 0 assert self.backend.can_post(u'hello', public=True, max_age=600) > 0 assert self.backend.can_post(u'hello', public=True, max_age=3600*24) > 0 assert self.backend.can_post(u'hello', public=True, max_age=3600*24*3) > 0 assert self.backend.can_post(u'hello', public=True, max_age=False) > 0 assert self.backend.can_post(u'hello', public=None, max_age=False) > 0 assert self.backend.can_post(u'hello', public=True, max_age=3600*24*40) > 0 assert self.backend.can_post(u'héhé', public=True) > 0 assert self.backend.can_post(u'hello ♥', public=True) > 0 ������������weboob-1.2/modules/paypal/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015557�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001441�13034501105�0017670�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PaypalModule __all__ = ['PaypalModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000022401�13034501105�0017613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib import datetime from dateutil.relativedelta import relativedelta from weboob.exceptions import BrowserHTTPError, BrowserIncorrectPassword from weboob.browser.browsers import LoginBrowser, need_login from weboob.browser.url import URL from .pages import PromoPage, LoginPage, AccountPage, UselessPage, HomePage, ProHistoryPage, \ PartHistoryPage, HistoryDetailsPage, ErrorPage, OldWebsitePage, LandingPage, InfoPage __all__ = ['Paypal'] class Paypal(LoginBrowser): BASEURL = 'https://www.paypal.com' login = URL('https://\w+.paypal.com/signin.*', '/cgi-bin/webscr\?cmd=_login-submit.+$', '/fr/webapps/mpp/home', LoginPage) landing = URL('/home', '/$', LandingPage) useless = URL('/cgi-bin/webscr\?cmd=_login-processing.+$', '/cgi-bin/webscr\?cmd=_account.*$', '/cgi-bin/webscr\?cmd=_login-done.+$', UselessPage) info = URL('/fr/merchantsignup/personalInfo', InfoPage) home = URL('.*/cgi-bin/webscr\?cmd=_home&country_lang.x=true$', 'https://\w+.paypal.com/webapps/business/\?country_lang.x=true', 'https://\w+.paypal.com/myaccount/\?nav=0.0', 'https://\w+.paypal.com/webapps/business/\?nav=0.0', 'https://\w+.paypal.com/myaccount/$', '/businessexp/summary.*', '/myaccount/?\?country_lang.x=true', '/businessexp/fees/interchange-fees', HomePage) error = URL('/auth/validatecaptcha$', ErrorPage) history_details = URL('https://\w+.paypal.com/cgi-bin/webscr\?cmd=_history-details-from-hub&id=[\-A-Z0-9]+$', 'https://\w+.paypal.com/myaccount/transaction/details/[\-A-Z0-9]+$', HistoryDetailsPage) promo = URL('https://www.paypal.com/fr/webapps/mpp/clickthru/paypal-app-promo-2.*', '/fr/webapps/mpp/clickthru.*', PromoPage) account = URL('https://www.paypal.com/businessexp/money', AccountPage) pro_history = URL('https://\w+.paypal.com/businessexp/transactions/activity\?.*', ProHistoryPage) part_history = URL('https://\w+.paypal.com/myaccount/activity/.*', PartHistoryPage) old_website = URL('https://paypalmanager.paypal.com/login.do', OldWebsitePage) TIMEOUT = 180.0 def __init__(self, *args, **kwargs): self.BEGINNING = datetime.date.today() - relativedelta(months=24) self.account_type = None self.account_currencies = list() super(Paypal, self).__init__(*args, **kwargs) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.login.is_here(): self.location('/signin/') response = self.open(self.page.get_script_url()) token, csrf, key, value, sessionID = self.page.get_token_and_csrf(response.text) data = {} data['ads_token_js'] = token data['_csrf'] = csrf data['_sessionID'] = sessionID data[key] = value data = urllib.urlencode(data) self.open('/auth/verifychallenge', data=data) res = self.page.login(self.username, self.password) if 'LoginFailed' in res.content or 'Sorry, we can\'t log you in' in res.content or self.error.is_here(): raise BrowserIncorrectPassword() self.location('/') if self.old_website.is_here(): self.location('https://www.paypal.com/businessexp/summary') if self.login.is_here() or self.landing.is_here(): raise BrowserIncorrectPassword(u'Impossible de se connecter, le compte nécessite une étape de vérification supplémentaire.') self.detect_account_type() @need_login def detect_account_type(self): self.page.detect_account_type() @need_login def get_accounts(self): if self.account_type is None: self.detect_account_type() self.account.stay_or_go() return self.page.get_accounts() @need_login def get_account(self, _id): self.account.stay_or_go() return self.page.get_account(_id) @need_login def get_personal_history(self, account): s = self.BEGINNING.strftime('%Y-%m-%d') e = datetime.date.today().strftime('%Y-%m-%d') data = {'transactionType': 'ALL', 'timeFrame': '90', 'nextPageToken': '', 'freeTextSearch': '', 'startDate': s, 'endDate': e, } # The response is sometimes not the one we expect. for i in xrange(3): try: self.location('https://www.paypal.com/myaccount/activity/filter?%s' % urllib.urlencode(data), headers={'Accept' : 'application/json, text/javascript, */*; q=0.01'}) if self.page.transaction_left(): return self.page.iter_transactions(account) return iter([]) except KeyError as e: self.logger.warning("retrying to get activity ...") raise e @need_login def get_download_history(self, account, step_min=None, step_max=None): if self.account_type == "perso": for i in self.get_personal_history(account): yield i else: if step_min is None and step_max is None: step_min = 30 step_max = 180 def fetch_fn(start, end): p = self.download_history(start, end) transactions = [] # Iter on each page while self.location("https://www.paypal.com/businessexp/transactions/activity", \ params=p).page.transaction_left(): p['next_page_token'] = self.page.get_next_page_token() for t in self.page.iter_transactions(account): transactions.append(t) if not p['next_page_token']: break return transactions if len(transactions) else iter([]) assert step_max <= 365*2 # PayPal limitations as of 2014-06-16 try: for i in self.smart_fetch(beginning=self.BEGINNING, end=datetime.date.today(), step_min=step_min, step_max=step_max, fetch_fn=fetch_fn): yield i except BrowserHTTPError: self.logger.warning("Paypal timeout") def smart_fetch(self, beginning, end, step_min, step_max, fetch_fn): """ Fetches transactions in small chunks to avoid request timeouts. Time period of each requested chunk is adjusted dynamically. """ FACTOR = 1.5 step = step_min while end > beginning: start = end - datetime.timedelta(step) chunk = list(fetch_fn(start, end)) end = start - datetime.timedelta(1) if len(chunk) > 40: # If there're too much transactions in current period, decrease # the period. step = max(step_min, step/FACTOR) else: # If there's no transactions, or only a bit, in current period, # increase the period. step = min(step_max, step*FACTOR) for trans in chunk: yield trans def download_history(self, start, end): """ Download history. However, it is not normalized, and sometimes the download is refused and sent later by mail. """ params = {'transactiontype': "ALL_TRANSACTIONS", 'currency': "ALL_TRANSACTIONS_CURRENCY", 'limit': "", 'archive': "ACTIVE_TRANSACTIONS", 'fromdate_year': start.year, 'fromdate_month': start.month-1, # Months are from 0 to 11. 'fromdate_day': start.day, 'todate_year': end.year, 'todate_month': end.month-1, 'todate_day': end.day } return params def transfer(self, from_id, to_id, amount, reason=None): raise NotImplementedError() def convert_amount(self, account, trans, link): self.location('%s%s' % (self.BASEURL, link.replace(self.BASEURL, ''))) if self.history_details.is_here(): cc = self.page.get_converted_amount(account) return cc ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000003335�13034501105�0017716�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��IDATxYh\Uٗd5tbMi.Z "ڇOQ(>ЈTD#҅. QEVLifL&Üi朙;DAȽ{|߁EZ5i~ �Zۀ@ P L! |@pׯ]ru@t@0?4hb%;#702|`� @K��"<6�^{vkmX>St�21C3@:*wϱ x@U 2tUKF 2k6�[\rR0Z�tH�WfrJ$ sz:L';$&6;L3==URlWո6uݣ#cv'O0=B}SH. Ԩp8oΤ'f7.U߹/(Htߞ>aCdz;͹UsgYY_(p �4N4  HX۝>ӗ<f$ GH/ðuO%g׆BT` LɤcB="Pm*B�͌gP${EZ!09>ӎJTx/d g&Urӗ<�| |9&ғEiCV {sJ�KRN;shfs$b; lJoB�ZU8$v٩ׂ Zm t*B1ie*d�"͔.|N%g�NO(+2Q% ê]:1\VI@ <P'Y {UEH*� &ÖdTa^BIT�hUa #"#-*p(-*�tQYe**-hYF*�:v]=u GD)ҥ{ݰ&ݽkUK薶D(D%~k:4Ѷr5_V5xQ�".[g~^J$g ~Ea DžiTQlv;V7 *#*0nD]Y% ⇽؀!$ Ee4Zr=O%UeKյKذ95w"a\1�t:QUбa*F/yPM,^_S̷\|O ]PtXձyt_d91r{6fm^#?y>^ OM7< �,LuLӴqӅ`H?\)\WQYE]C øq\9Sqa`&U5u=^e+x|7HU~_T}�SQέk$Qz8 ktX{c%HvXr  :,bz�_�2. *1A@ ]Q-^/>j2�H˵=�Nc7jq@.'־+bCçI"$uT6F]m~'m/v:e*`f,GSlO%5Y�NΙ"�g% Bg!p8*Hv}�Q{,s.9х{O oiy#`"�R@wo۹qF_"-"/y Q6i����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000003613�13034501105�0017421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import Paypal __all__ = ['PaypalModule'] class PaypalModule(Module, CapBank): NAME = 'paypal' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'PayPal' CONFIG = BackendConfig(ValueBackendPassword('login', label='E-mail', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = Paypal def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.get_accounts().itervalues() def get_account(self, _id): account = self.browser.get_account(_id) if account: return account else: raise AccountNotFound() def iter_history(self, account): for history in self.browser.get_download_history(account): yield history ���������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/pages.py������������������������������������������������������������������0000664�0000000�0000000�00000034570�13034501105�0017241�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from decimal import Decimal, ROUND_DOWN import re from weboob.capabilities.bank import Account from weboob.capabilities.base import NotAvailable, Currency from weboob.exceptions import BrowserUnavailable, ActionNeeded from weboob.browser.exceptions import ServerError from weboob.browser.pages import HTMLPage, JsonPage, LoggedPage from weboob.browser.filters.standard import CleanText, CleanDecimal from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.date import parse_french_date from weboob.tools.js import Javascript class LandingPage(HTMLPage): pass class OldWebsitePage(LoggedPage, HTMLPage): pass class InfoPage(HTMLPage): def on_load(self): raise ActionNeeded(CleanText('//h1[@class="falconHeaderText"]')(self.doc)) class PromoPage(LoggedPage, HTMLPage): def on_load(self): # We land sometimes on this page, it's better to raise an unavailable browser # than an Incorrect Password raise BrowserUnavailable('Promo Page') class LoginPage(HTMLPage): def get_token_and_csrf(self, code): code1 = re.search('(function .*)\(function\(\)', code).group(1) # Another victory for the scrapper team # CommitStrip Data Wars code1 = re.sub('return typeof document!="undefined"&&typeof document.createAttribute!="undefined"', 'return 1==1', code1) # now it checks if some browsers-only builtin variables are defined: # e+=function(e,t){return typeof navigator!="undefined"?e:t} js = Javascript('var navigator = {}; var document = {}; document.lastModified = 1; \ document.getElementsByTagName = 1; document.implementation = 1; document.createAttribute = 1; document.getElementsByName = 1;'+ code1) func_name = re.search(r'function (\w+)\(\)', code1).group(1) token = str(js.call(func_name)) csrf = re.search(r'csrf="\+encodeURIComponent\("(.*?)"\)', code).group(1) key, value = re.search(r'"/auth/verifychallenge",t,"([^"]+)","([^"]+)"', code).groups() sessionID = re.search(r'_sessionID="\+encodeURIComponent\("(.*?)"\)', code).group(1) return token, csrf, key, value, sessionID def login(self, login, password): form = self.get_form(name='login') form['login_email'] = login form['login_password'] = password return form.submit(headers={'X-Requested-With': 'XMLHttpRequest'}) def get_script_url(self): body = self.doc.xpath('//body')[0] if 'data-ads-challenge-url' in body.attrib: return 'https://www.paypal.com%s' % body.attrib['data-ads-challenge-url'] # Paypal still use old method sometimes list1 = self.doc.xpath('//script') for s in list1: if 'src' in s.attrib and 'challenge' in s.attrib['src']: return s.attrib['src'] class ErrorPage(HTMLPage): pass class UselessPage(LoggedPage, HTMLPage): pass class HomePage(LoggedPage, HTMLPage): def detect_account_type(self): if self.doc.xpath('//a[contains(@href, "businessexp")] | //script[contains(text(), "business")]'): self.browser.account_type = "pro" elif self.doc.xpath('//a[contains(@href, "myaccount")]'): self.browser.account_type = "perso" class AccountPage(HomePage): def get_account(self, _id): return self.get_accounts().get(_id) def get_accounts(self): accounts = {} content = self.doc.xpath('//div[@id="moneyPage"]')[0] # Multiple accounts lines = content.xpath('(//div[@class="col-md-8 multi-currency"])[1]/ul/li') for li in lines: account = Account() account.iban = NotAvailable account.type = Account.TYPE_CHECKING currency_code = CleanText().filter((li.xpath('./span[@class="currencyUnit"]/span') or li.xpath('./span[1]'))[0]) currency = Currency.get_currency(currency_code) if not currency: self.logger.warning('Unable to find currency %r', currency_code) continue account.id = currency account.currency = currency account.balance = CleanDecimal(replace_dots=True).filter(li.xpath('./span[@class="amount"]/text()')) account.label = u'%s %s*' % (self.browser.username, account.currency) accounts[account.id] = account self.browser.account_currencies.append(account.currency) if not accounts: # Primary currency account primary_account = Account() primary_account.iban = NotAvailable primary_account.type = Account.TYPE_CHECKING try: balance = CleanText('.')(content.xpath('//div[contains(@class, "col-md-6")][contains(@class, "available")]')[0]) except IndexError: primary_account.id = 'EUR' primary_account.currency = u'EUR' primary_account.balance = NotAvailable primary_account.label = u'%s' % (self.browser.username) else: primary_account.currency = Account.get_currency(balance) primary_account.id = unicode(primary_account.currency) primary_account.balance = Decimal(FrenchTransaction.clean_amount(balance)) primary_account.label = u'%s %s*' % (self.browser.username, primary_account.currency) accounts[primary_account.id] = primary_account return accounts class HistoryPage(LoggedPage): def iter_transactions(self, account): for trans in self.parse(account): yield trans def parse(self, account): transactions = list() transacs = self.get_transactions() for t in transacs: for trans in self.parse_transaction(t, account): transactions.append(trans) for t in transactions: yield t def format_amount(self, amount, is_credit): """ This function takes a textual amount to convert it to Decimal. It tries to guess what is the decimal separator (, or .). """ if not isinstance(amount, Decimal): m = re.search(r"\D", amount.strip(u'€').strip(u'\xa0')[::-1]) amount = Decimal(re.sub(r'[^\d]', '', amount))/Decimal((10 ** m.start())) if m else Decimal(amount) if is_credit: return abs(amount) else: return -abs(amount) class ProHistoryPage(HistoryPage, JsonPage): def transaction_left(self): return 'transactions' in self.doc['data'] and self.doc['data']['transactions'] def get_next_page_token(self): if 'nextpageurl' in self.doc['data']: return self.doc['data']['nextpageurl'] return None def get_transactions(self): return self.doc['data']['transactions'] def parse_transaction(self, transaction, account): trans = [] # Add secondary transactions on label condition. for t in transaction['secondaryTransactions']: if t['transactionDescription']['description'] == u'Virement à partir de': trans.extend(self.parse_transaction(t, account)) if 'transactionStatus' in transaction and transaction['transactionStatus'] in [u'Créé', u'Annulé', u'Suspendu', u'Mis à jour', u'Actif', u'Payé', u'En attente', u'Rejeté', u'Expiré', u'Created', u'Brouillon', u'Paid', u'Pending', u'Canceled', u'Suspended']: return [] for pattern in [u'Commande à', u'Offre de remboursement', u'Bill to']: if 'description' not in transaction['transactionDescription'] or transaction['transactionDescription']['description'].startswith(pattern): return [] t = FrenchTransaction(transaction['transactionId']) # Those are not really transactions. if 'grossAmount' not in transaction or not 'currency' in transaction['grossAmount'] \ or transaction['transactionDescription']['description'].startswith("Conversion de devise"): return [] original_currency = unicode(transaction['grossAmount']['currency']) if not original_currency == account.currency: if original_currency in self.browser.account_currencies: return [] cc = [tr['grossAmount']['amountUnformatted'] for tr in transaction['secondaryTransactions'] \ if account.currency == tr['grossAmount']['currency'] \ and (tr['grossAmount']['amountUnformatted'] < 0) == (transaction['grossAmount']['amountUnformatted'] < 0) \ and tr['transactionDescription']['description'].startswith('Conversion de devise')] if not cc: return [] assert len(cc) == 1 t.original_amount = Decimal(str(transaction['netAmount']['amountUnformatted'])) t.original_currency = original_currency t.amount = Decimal(str(cc[0])) else: t.amount = Decimal(str(transaction['netAmount']['amountUnformatted'])) date = parse_french_date(transaction['transactionTime']) raw = "%s %s" % (transaction['transactionDescription']['description'], transaction['transactionDescription']['name']) if raw == "Transfert de Compte bancaire": t.type = FrenchTransaction.TYPE_TRANSFER if raw == u'Annulation des frais de PayPal': return [] # Dougs told us that commission should always be netAmount minus grossAmount grossAmount = Decimal(str(transaction['grossAmount']['amountUnformatted'])) t.commission = Decimal(str(transaction['feeAmount']['amountUnformatted'])) if t.commission: if original_currency == account.currency: assert abs(t.amount - grossAmount) == abs(t.commission) t.commission = t.amount - grossAmount else: t.commission = (t.commission * t.amount / t.original_amount).quantize(Decimal('.01'), rounding=ROUND_DOWN) t.parse(date=date, raw=raw) trans.append(t) return trans class PartHistoryPage(HistoryPage, JsonPage): def transaction_left(self): return self.doc['data']['activity']['hasTransactionsCompleted'] or self.doc['data']['activity']['hasTransactionsPending'] def get_transactions(self): return self.doc['data']['activity']['transactions'] def parse_transaction(self, transaction, account): if 'id' not in transaction or not transaction['date']: return [] t = FrenchTransaction(transaction['id']) if not transaction['isPrimaryCurrency']: if not 'txnCurrency' in transaction['amounts']: return [] original_currency = unicode(transaction['amounts']['txnCurrency']) if original_currency in self.browser.account_currencies: return [] if 'conversionFrom' in transaction['amounts'] and 'value' in transaction['amounts']['conversionFrom'] and account.currency == transaction['amounts']['conversionFrom']['currency']: cc = self.format_amount(transaction['amounts']['conversionFrom']['value'], transaction['isCredit']) else: try: cc = self.browser.convert_amount(account, transaction, transaction['detailsLink']) except ServerError: self.logger.warning('Unable to go on detail, transaction skipped.') return [] if not cc: return [] t.original_amount = self.format_amount(transaction['amounts']['net']['value'], transaction["isCredit"]) t.original_currency = original_currency t.amount = self.format_amount(cc, transaction['isCredit']) else: t.amount = self.format_amount(transaction['amounts']['net']['value'], transaction['isCredit']) date = parse_french_date(transaction['date']['formattedDate'] + ' ' + transaction['date']['year']) raw = transaction.get('counterparty', transaction['displayType']) t.parse(date=date, raw=raw) return [t] class HistoryDetailsPage(LoggedPage, HTMLPage): def get_converted_amount(self, account): find_td = self.doc.xpath('//td[contains(text(),"' + account.currency + '")] | //dd[contains(text(),"' + account.currency + '")]') if len(find_td) > 0 : # In case text is "12,34 EUR = 56.78 USD" or "-£115,62 GBP soit -€163,64 EUR" for f in find_td: for text in re.split('=|soit|equals', CleanText().filter(f)): if ' %s' % account.currency in text: return Decimal(FrenchTransaction.clean_amount(text.split(account.currency)[0])) return False ����������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/paypal/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000002034�13034501105�0017107�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PaypalTest(BackendTest): MODULE = 'paypal' def test_balance(self): for account in self.backend.iter_accounts(): balance = sum(t.amount for t in self.backend.iter_history(account)) self.assertEqual(balance, account.balance) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015364�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001477�13034501105�0017506�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .browser import PhpBB from .module import PhpBBModule __all__ = ['PhpBB', 'PhpBBModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000015025�13034501105�0017424�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re import urllib from urlparse import urlsplit from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from weboob.capabilities.messages import CantSendMessage from .pages.index import LoginPage from .pages.forum import ForumPage, TopicPage, PostingPage from .tools import id2url, url2id __all__ = ['PhpBB'] # Browser class PhpBB(Browser): PAGES = {'https?://.*/index.php': ForumPage, 'https?://.*/': ForumPage, 'https?://.*/viewforum.php\?f=(\d+)': ForumPage, 'https?://.*/search.php\?.*': ForumPage, 'https?://.*/viewtopic.php\?.*': TopicPage, 'https?://.*/posting.php\?.*': PostingPage, 'https?://.*/ucp.php\?mode=login.*': LoginPage, } last_board_msg_id = None def __init__(self, url, *args, **kwargs): self.url = url v = urlsplit(url) self.PROTOCOL = v.scheme self.DOMAIN = v.netloc self.BASEPATH = v.path[:v.path.rfind('/')] Browser.__init__(self, *args, **kwargs) def absurl(self, rel): return Browser.absurl(self, '%s/%s' % (self.BASEPATH, rel)) def home(self): self.location(self.url) def is_logged(self): return not self.page or self.page.is_logged() def login(self): data = {'login': 'Connexion', 'username': self.username, 'password': self.password, } self.location('%s/ucp.php?mode=login' % self.BASEPATH, urllib.urlencode(data), no_login=True) assert self.is_on_page(LoginPage) if not self.page.is_logged(): raise BrowserIncorrectPassword(self.page.get_error_message()) def get_root_feed_url(self): self.home() return self.page.get_feed_url() def iter_links(self, url): if url: self.location(url) else: self.home() assert self.is_on_page(ForumPage) return self.page.iter_links() def iter_posts(self, id, stop_id=None): if id.startswith('http'): self.location(id) else: self.location('%s/%s' % (self.BASEPATH, id2url(id))) assert self.is_on_page(TopicPage) parent = 0 while True: for post in self.page.iter_posts(): if stop_id and post.id >= stop_id: return post.parent = parent yield post parent = post.id if self.page.cur_page == self.page.tot_pages: return self.location(self.page.next_page_url()) def riter_posts(self, id, stop_id=None): if id.startswith('http'): self.location(id) else: self.location('%s/%s' % (self.BASEPATH, id2url(id))) assert self.is_on_page(TopicPage) child = None while True: for post in self.page.riter_posts(): if child: child.parent = post.id yield child if post.id <= stop_id: return child = post if self.page.cur_page == 1: if child: yield child return self.location(self.page.prev_page_url()) def get_post(self, id): if id.startswith('http'): self.location(id) id = url2id(id) else: self.location('%s/%s' % (self.BASEPATH, id2url(id))) assert self.is_on_page(TopicPage) post = self.page.get_post(int(id.split('.')[-1])) if not post: return None if post.parent == 0 and self.page.cur_page > 1: self.location(self.page.prev_page_url()) post.parent = self.page.get_last_post_id() return post def get_forums(self): self.home() return dict(self.page.iter_all_forums()) def post_answer(self, forum_id, topic_id, title, content): if topic_id == 0: if not forum_id: forums = self.get_forums() forums_prompt = 'Forums list:\n%s' % ('\n'.join(['\t- %s' % f for f in forums.itervalues()])) m = re.match('\[(.*)\] (.*)', title or '') if not m: raise CantSendMessage('Please enter a title formatted like that:\n\t"[FORUM] SUBJECT"\n\n%s' % forums_prompt) forum_id = None for k,v in forums.iteritems(): if v.lower() == m.group(1).lower(): forum_id = k break if not forum_id: raise CantSendMessage('Forum "%s" not found.\n\n%s' % (m.group(1), forums_prompt)) self.location('%s/posting.php?mode=post&f=%d' % (self.BASEPATH, forum_id)) assert self.is_on_page(PostingPage) self.page.post(title, content) assert self.is_on_page(PostingPage) error = self.page.get_error_message() if error: raise CantSendMessage(u'Unable to send message: %s' % error) else: self.location('%s/%s' % (self.BASEPATH, id2url(topic_id))) assert self.is_on_page(TopicPage) self.page.go_reply() assert self.is_on_page(PostingPage) # Don't send title because it isn't needed in real use case # and with monboob title is something like: # Re: [Forum Name] Re: Topic Name if title is not None and title.startswith('Re:'): title = None self.page.post(title, content) assert self.is_on_page(PostingPage) error = self.page.get_error_message() if error: raise CantSendMessage(u'Unable to send message: %s' % error) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000006620�13034501105�0017523�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME", T���tEXtComment�Created with GIMPW�� IDATxZk]u>{̝?xa#4&5UDiT%V͟JW\)i~PRUV JH>PJZ$ Nll#۱/=s{3csU%]\{5{}{ַeQeQeQeQeQ+Uq<~CDf �of "k#^_+]RLLa|Rzb1�*��8#"9$*§ 2̄Q(}DZK)�"�C[3p.aMwG6ƣ4.:�>JurmU?uVC0L`v r�g "A4)R. ca+@@d&3p tXcw_5604nA g'[^PJCP{DPZ-ܯɁؖ@;dZo}C'V\ֿj";gdAd@d{ ?Eo=ZP:R JgDOr1]$YٓO|a'&W]׋ `Y8g@i/qZX*%b}8ea@ _h|ɽ@-yxjwW&MA. �9Qa J{7?K<n/: |5ؾ$~̜d$3nO=O\z]\oP{S& p-u+y9ƫGZx8xh6߫ƽziɧF" m7_z=Z[9(M<8ՑU_Z"8k`Lt`Ltφ5Sn5k,e믞—_5W{A6>lrsYQמ~`�xW a& r;g.#4>uuZ?x1ll2@ 6}D" kS8 ΦJ4-/)]]WN׼ZsÉP5h쮑�:i"w? ep]'}DGw�2f`>*t) *W0u3Lʔmgu5~+19Q�<_hR Z}?\O}|( g12,ǍN1?f++]V3vpPT QTW@^7_o{b7z _VZ?1cK74�&a6Lց5ܳ?=6pTV.`а6WH^!J2KCQTC/:X4 a]=CW8~~\eK bwnl}OS#�a ٭Eo4XQZGO7p � Its"R [ŧzק=NI@0ۼz8y߰u#/ {A$d�dHJ%D>&qЃI%c) JԦ09q<NfN3_]74DX (Dڤ� =Z ¨ !A ?+%2di&<1cf D1'lof׬O\ñT'5$,�{T2K) '^<%gKn,X ARR� !!T 91OX#gYlIlɹkUj%kfꓗ-_@Zx$�|_@J! N<"$;vff"ð[ew*b>8AABH$n77u+EL#�}/h<+.*>9wtcyb �}? ]j:jaA7�OTC2#jcKbaVA`rYtg6o/ib�G)fA0?!UƓ>/P aT ߱%J}#a�+ĕ: ^>R'Y>|>]]31Jub=J)u*x\�jV�h\_BhW 1&׺g6m-0Ƈt{{=}ƨ"ǖC/EՍ � M7.չ=;E yexS"[M=1P6Esu=vU|�ˋt J]MPg taC\�dsyR Q\q&NR̾r.{vd�)w~A֥?PmK)rU7ajmc( 0i`m kF@xIюfg=At~kDc]zXEcT&:' >,\*${FN)gqXZofPA.' dI@ٌY=�""EpAspNjiQID8(^:`M�)Jka489,LPBhδt?j�A9 :&l2\Z!zwT>+}o\d7DeLw_̡:n4[SzJPZ*%mf�BW_&P"V>f:v덇fn;}`kԡ=N^cA[VS*ƣt߰z:D(!|??[ g6%]hFWTQ&<p#bh=zӿ;d`_"s�ZGQu0}?f*={eݷ a cLlN,'.ܵ>^N ]L nw�f_B2 _z$/'`v0Nk~К/WPn1oe&k|<gDv;w&sܘ%,mf]?}&5d4mmH nu6GzDM&sCm5I6eY*^WLi24m"M^OnJsw�5LgcWrC,6_}3=,m +iGiwtv<n};]� g{|I>y64&YOdM$s;#3F�ݤɺU&_X\EYEYEY"cz����IENDB`����������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/module.py������������������������������������������������������������������0000664�0000000�0000000�00000016065�13034501105�0017233�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.newsfeed import Newsfeed from weboob.tools.value import Value, ValueInt, ValueBackendPassword from weboob.tools.misc import limit from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message, Thread, CantSendMessage from .browser import PhpBB from .tools import rssid, url2id, id2url, id2topic __all__ = ['PhpBBModule'] class PhpBBModule(Module, CapMessages, CapMessagesPost): NAME = 'phpbb' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = "phpBB forum" CONFIG = BackendConfig(Value('url', label='URL of forum', regexp='https?://.*'), Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), ValueInt('thread_unread_messages', label='Limit number of unread messages to retrieve for a thread', default=500) ) STORAGE = {'seen': {}} BROWSER = PhpBB def create_default_browser(self): username = self.config['username'].get() if len(username) > 0: password = self.config['password'].get() else: password = None return self.create_browser(self.config['url'].get(), username, password) #### CapMessages ############################################## def _iter_threads(self, root_link=None): with self.browser: links = list(self.browser.iter_links(root_link.url if root_link else None)) for link in links: if link.type == link.FORUM: link.title = '%s[%s]' % (root_link.title if root_link else '', link.title) for thread in self._iter_threads(link): yield thread if link.type == link.TOPIC: thread = Thread(url2id(link.url)) thread.title = ('%s ' % root_link.title if root_link else '') + link.title thread.date = link.date thread.flags = thread.IS_DISCUSSION yield thread def iter_threads(self): return self._iter_threads() def get_thread(self, id): thread = None parent = None if isinstance(id, Thread): thread = id id = thread.id thread_id = url2id(id, nopost=True) or id try: last_seen_id = self.storage.get('seen', default={})[id2topic(thread_id)] except KeyError: last_seen_id = 0 with self.browser: for post in self.browser.iter_posts(id): if not thread: thread = Thread(thread_id) thread.title = post.title m = self._post2message(thread, post) m.parent = parent if last_seen_id < post.id: m.flags |= Message.IS_UNREAD if parent: parent.children = [m] else: thread.root = m parent = m return thread def _post2message(self, thread, post): signature = post.signature if signature: signature += '<br />' signature += 'URL: %s' % self.browser.absurl(id2url('%s.%s' % (thread.id, post.id))) return Message(thread=thread, id=post.id, title=post.title, sender=post.author, receivers=None, date=post.date, parent=None, content=post.content, signature=signature, children=[], flags=Message.IS_HTML) def iter_unread_messages(self): with self.browser: url = self.browser.get_root_feed_url() for article in Newsfeed(url, rssid).iter_entries(): id = url2id(article.link) thread = None try: last_seen_id = self.storage.get('seen', default={})[id2topic(id)] except KeyError: last_seen_id = 0 child = None iterator = self.browser.riter_posts(id, last_seen_id) if self.config['thread_unread_messages'].get() > 0: iterator = limit(iterator, self.config['thread_unread_messages'].get()) for post in iterator: if not thread: thread = Thread('%s.%s' % (post.forum_id, post.topic_id)) message = self._post2message(thread, post) if child: message.children.append(child) child.parent = message if post.parent: message.parent = Message(thread=thread, id=post.parent) else: thread.root = message yield message def set_message_read(self, message): try: last_seen_id = self.storage.get('seen', default={})[id2topic(message.thread.id)] except KeyError: last_seen_id = 0 if message.id > last_seen_id: self.storage.set('seen', id2topic(message.thread.id), message.id) self.storage.save() def fill_thread(self, thread, fields): return self.get_thread(thread) #### CapMessagesReply ######################################### def post_message(self, message): assert message.thread forum = 0 topic = 0 if message.thread: try: if '.' in message.thread.id: forum, topic = [int(i) for i in message.thread.id.split('.', 1)] else: forum = int(message.thread.id) except ValueError: raise CantSendMessage('Thread ID must be in form "FORUM_ID[.TOPIC_ID]".') with self.browser: return self.browser.post_answer(forum, topic, message.title, message.content) OBJECTS = {Thread: fill_thread} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/pages/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016463�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/pages/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020562�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/pages/forum.py�������������������������������������������������������������0000664�0000000�0000000�00000017657�13034501105�0020205�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from time import sleep from urlparse import urlsplit, parse_qs from weboob.deprecated.browser import BrokenPageError from .index import PhpBBPage from ..tools import parse_date class Link(object): (FORUM, TOPIC) = xrange(2) def __init__(self, type, url): self.type = type self.url = url self.title = u'' self.date = None class ForumPage(PhpBBPage): def iter_links(self): for li in self.parser.select(self.document.getroot(), 'ul.forums li.row'): title = li.cssselect('a.forumtitle')[0] link = Link(Link.FORUM, title.attrib['href']) link.title = title.text.strip() yield link for li in self.parser.select(self.document.getroot(), 'ul.topics li.row'): title = li.cssselect('a.topictitle')[0] link = Link(Link.TOPIC, title.attrib['href']) link.title = title.text.strip() for a in li.find('dl').find('dt').findall('a'): for text in (a.text, a.tail): if text is None: continue try: link.date = parse_date(text.strip(u'» \r\n')) except ValueError: continue else: break yield link def iter_all_forums(self): for option in self.parser.select(self.document.getroot(), 'select#f', 1).findall('option'): value = int(option.attrib['value']) if value < 0 or not option.text: continue yield value, option.text.strip(u'» \xa0\n\r') class Post(object): def __init__(self, forum_id, topic_id, id): self.id = int(id) self.forum_id = forum_id self.topic_id = topic_id self.title = u'' self.author = u'' self.date = None self.content = u'' self.signature = u'' self.parent = 0 class TopicPage(PhpBBPage): def on_loaded(self): div = self.document.getroot().cssselect('div.pagination')[0] strongs = div.cssselect('strong') self.cur_page = int(strongs[0].text.strip()) self.tot_pages = int(strongs[1].text.strip()) try: url = self.document.xpath('//h2/a')[-1].attrib['href'] except BrokenPageError: url = self.url v = urlsplit(url) args = parse_qs(v.query) self.topic_id = int(args['t'][0]) self.forum_id = int(args['f'][0]) if 'f' in args else 0 self.forum_title = u'' nav = self.parser.select(self.document.getroot(), 'li.icon-home') if len(nav) > 0: text = nav[0].findall('a')[-1].text.strip() if len(text) >= 20: text = text[:20] + u'…' self.forum_title = '[%s] ' % text def go_reply(self): self.browser.follow_link(url_regex='posting\.php') def next_page_url(self): try: return self.parser.select(self.document.getroot(), 'a.right-box', 1).attrib['href'] except BrokenPageError: a_list = self.parser.select(self.document.getroot(), 'div.pagination', 1).findall('a') if self.cur_page == self.tot_pages: return '#' return a_list[-1].attrib['href'] def prev_page_url(self): try: return self.parser.select(self.document.getroot(), 'a.left-box', 1).attrib['href'] except BrokenPageError: a_list = self.parser.select(self.document.getroot(), 'div.pagination', 1).findall('a') if self.cur_page == self.tot_pages: a = a_list[-1] else: a = a_list[-2] return a.attrib['href'] def iter_posts(self): for div in self.parser.select(self.document.getroot(), 'div.post'): yield self._get_post(div) def riter_posts(self): for div in reversed(self.parser.select(self.document.getroot(), 'div.post')): yield self._get_post(div) def get_post(self, id): parent = 0 for div in self.parser.select(self.document.getroot(), 'div.post'): if div.attrib['id'] == 'p%d' % id: post = self._get_post(div) post.parent = parent return post else: parent = int(div.attrib['id'][1:]) def _get_post(self, div): body = div.cssselect('div.postbody')[0] profile = div.cssselect('dl.postprofile')[0] id = div.attrib['id'][1:] post = Post(self.forum_id, self.topic_id, id) title_tags = body.xpath('//h3/a') if len(title_tags) == 0: title_tags = self.document.xpath('//h2/a') if len(title_tags) == 0: title = u'' self.logger.warning('Unable to parse title') else: title = title_tags[-1].text.strip() post.title = self.forum_title + title for a in profile.cssselect('dt a'): if a.text: post.author = a.text.strip() p_tags = body.cssselect('p.author') if len(p_tags) == 0: p_tags = body.find('p') if len(p_tags) == 0: post.date = None self.logger.warning('Unable to parse datetime') else: p = p_tags[0] text = p.find('strong') is not None and p.find('strong').tail if not text: text = p.text[4:] text = text.strip(u'» \n\r') try: post.date = parse_date(text) except ValueError: self.logger.warning(u'Unable to parse datetime "%s"' % text) post.content = self.parser.tostring(body.cssselect('div.content')[0]) signature = body.cssselect('div.signature') if len(signature) > 0: post.signature = self.parser.tostring(signature[0]) return post def get_last_post_id(self): id = 0 for div in self.parser.select(self.document.getroot(), 'div.post'): id = int(div.attrib['id'][1:]) return id class PostingPage(PhpBBPage): def post(self, title, content): self.browser.select_form(predicate=lambda form: form.attrs.get('id', '') == 'postform') self.browser.set_all_readonly(False) if title: self.browser['subject'] = title.encode('utf-8') self.browser['message'] = content.encode('utf-8') # This code on phpbb: # if ($cancel || ($current_time - $lastclick < 2 && $submit)) # { # /* ... */ # redirect($redirect); # } # To prevent that shit because weboob is too fast, we simulate # a value of lastclick 10 seconds before. self.browser['lastclick'] = str(int(self.browser['lastclick']) - 10) # Likewise for create_time, with this check: # $diff = time() - $creation_time; # // If creation_time and the time() now is zero we can assume it was not a human doing this (the check for if ($diff)... # if ($diff && ($diff <= $timespan || $timespan === -1)) # But as the form_token depends on the create_time value, I can't # change it. But I can wait a second before posting... sleep(1) self.browser.submit(name='post') ���������������������������������������������������������������������������������weboob-1.2/modules/phpbb/pages/index.py�������������������������������������������������������������0000664�0000000�0000000�00000002505�13034501105�0020146�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page class PhpBBPage(Page): def is_logged(self): return len(self.document.getroot().cssselect('li.icon-register')) == 0 def get_feed_url(self): links = self.document.getroot().cssselect('link[type="application/atom+xml"]') return links[-1].attrib['href'] def get_error_message(self): errors = [] for div in self.parser.select(self.document.getroot(), 'div.error,p.error'): if div.text: errors.append(div.text.strip()) return ', '.join(errors) class LoginPage(PhpBBPage): pass �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000002022�13034501105�0016711�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PhpBBTest(BackendTest): MODULE = 'phpbb' def testthreads(self): for thread in self.backend.iter_threads(): pass def test_unread_messages(self): for message in self.backend.iter_unread_messages(): pass ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/phpbb/tools.py�������������������������������������������������������������������0000664�0000000�0000000�00000004401�13034501105�0017075�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from dateutil.parser import parse as _parse_dt from urlparse import urlsplit, parse_qs from weboob.tools.date import local2utc def url2id(url, nopost=False): v = urlsplit(url) pagename = v.path.split('/')[-1] args = parse_qs(v.query) if pagename == 'viewforum.php': return '%d' % int(args['f'][0]) if pagename == 'viewtopic.php': if 'f' in args: s = '%d' % int(args['f'][0]) else: s = '0' s += '.%d' % int(args['t'][0]) if 'p' in args and not nopost: s += '.%d' % int(args['p'][0]) return s return None def id2url(id): v = id.split('.') if len(v) == 1: return 'viewforum.php?f=%d' % int(v[0]) if len(v) == 2: return 'viewtopic.php?f=%d&t=%d' % (int(v[0]), int(v[1])) if len(v) == 3: return 'viewtopic.php?f=%d&t=%d&p=%d#p%d' % (int(v[0]), int(v[1]), int(v[2]), int(v[2])) def id2topic(id): try: return int(id.split('.')[1]) except IndexError: return None def rssid(id): return id def parse_date(s): s = s.replace(u'Fév', 'Feb') \ .replace(u'Avr', 'Apr') \ .replace(u'Mai', 'May') \ .replace(u'Juin', 'Jun') \ .replace(u'Juil', 'Jul') \ .replace(u'Aoû', 'Aug') \ .replace(u'Ao\xfbt', 'Aug') \ .replace(u'Déc', 'Dec') return local2utc(_parse_dt(s)) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016251�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000000103�13034501105�0020354�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import PiratebayModule __all__ = ['PiratebayModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000003126�13034501105�0020310�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2015 Julien Veyssier, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages.index import IndexPage from .pages.torrents import TorrentPage, TorrentsPage, FilesPage __all__ = ['PiratebayBrowser'] class PiratebayBrowser(PagesBrowser): BASEURL = 'https://thepiratebay.se' index_page = URL('/$', IndexPage) torrents_page = URL('/search/(?P<query>.+)/0/7/0', TorrentsPage) torrent_page = URL('/torrent/(?P<id>.+)', TorrentPage) files_page = URL('/ajax_details_filelist.php\?id=(?P<id>.+)', FilesPage) def iter_torrents(self, pattern): self.torrents_page.go(query=pattern) return self.page.iter_torrents() def get_torrent(self, _id): self.torrent_page.go(id=_id) torrent = self.page.get_torrent() self.files_page.go(id=_id) files = self.page.get_files() torrent.files = files return torrent ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000003066�13034501105�0020411�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME  QML��IDATx{Tu?w\gGY4tA>? D#6% Ң!(B a%aVdaQJiCuݝUgݹ?;3wvfv~a߽;{!D!B"H?Onmlg-ow.Op`具�>_23qK1L}dB V9H|Ȁ5AP��*;ǘ=@DB p�x h4p7GZ `γX tTC] &oYwW{pHyk?9H3�,!"&RqpP 4 N%,hkRsj`h: Ӗ}1#L^> aX o&xL3H`sEp=LRW( 𒞢K)Eᙚ>Ee}[4gmSPdj 66,NrU}*^ w%<QӉ824Ά@9PSSr4͗97 h'Jl 5�ӁM9lYc74`*[bDm.ɷwx@0 8iy^sVǞQ�0(EՏX#z=Ijc@G<t{Siw {5�C(\f^?G3bVvϽ=Oo=yg sՌt6𨇟N-n Fe-p47�Q]FWsib3F` M4�cM2pa Pw�3=|4*91ȣb]zہMGPՆQgsD㿊yHz08 `[>{0h2vp&0N| a\َuCM-2EK7O \{3`E!^eU+đ�Z4L 1LFZӀ"a pTG)>m6Ł]F6]mi`GhY4?57m/ha ?L3+LEx/P#`9gR| < ,_ k6gfbFYjQ򍆤#nqNs%x).3i1G"r%L*Y̯t�Ձ5{W.-I}ϵlSχ]M7{p"Zt0xzpq}I2 ]ɡt6 y'f3KEB3OO-q^iqnnKTy7pr:$pZCͽw\PS۔I*O@<Br v.Zln2}Y"ih%Y ,Yh�FK5D{P}Jr1>Ѧ܋L,p.eZ&Y ϰa,BucDT 1A}֮A߯p(!B"D!B8*Z؛����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004502�13034501105�0020111�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.torrent import CapTorrent, MagnetOnly, Torrent from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.capabilities.base import NotAvailable from .browser import PiratebayBrowser __all__ = ['PiratebayModule'] class PiratebayModule(Module, CapTorrent): NAME = 'piratebay' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' DESCRIPTION = 'The Pirate Bay BitTorrent tracker' LICENSE = 'AGPLv3+' BROWSER = PiratebayBrowser CONFIG = BackendConfig(Value('proxybay', label='Use a Proxy Bay', regexp=r'https?://.*', default='', required=False)) def create_default_browser(self): return self.create_browser(self.config['proxybay'].get()) def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None if torrent.url is NotAvailable and torrent.magnet: raise MagnetOnly(torrent.magnet) return self.browser.open(torrent.url).content def iter_torrents(self, pattern): return self.browser.iter_torrents(pattern.replace(' ', '+')) def fill_torrent(self, torrent, fields): if 'description' in fields or 'files' in fields: tor = self.get_torrent(torrent.id) torrent.description = tor.description torrent.magnet = tor.magnet torrent.files = tor.files torrent.url = tor.url return torrent OBJECTS = {Torrent: fill_torrent} ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/pages/�����������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017350�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/pages/__init__.py������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021447�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/pages/index.py���������������������������������������������������������0000664�0000000�0000000�00000001573�13034501105�0021037�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage class IndexPage(HTMLPage): def is_logged(self): return 'id' in self.document.find('body').attrib �������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/pages/torrents.py������������������������������������������������������0000664�0000000�0000000�00000006256�13034501105�0021613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Julien Veyssier, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.misc import get_bytes_size from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotAvailable from weboob.browser.filters.standard import RawText, CleanText, Regexp, Date, Type class TorrentsPage(HTMLPage): @method class iter_torrents(ListElement): item_xpath = '//table[@id="searchResult"]/tr' class item(ItemElement): klass = Torrent obj_id = Regexp(CleanText('./td[2]/div/a[@class="detLink"]/@href'), r'^/torrent/(\d+)/', '\\1') obj_name = Regexp(CleanText('./td[2]/div/a[@class="detLink"]/@title'), r'Details for (.*)$', '\\1') obj_magnet = CleanText('./td[2]/a[title="Download this torrent using magnet"]/@href') obj_date = Date(Regexp(CleanText('./td[2]/font'), r'Uploaded ([^,]+),', '\\1'), fuzzy=True) obj_seeders = Type(CleanText('./td[3]'), type=int) obj_leechers = Type(CleanText('./td[4]'), type=int) def obj_size(self): value, unit = Regexp(CleanText('./td[2]/font'), r'Size ([\d\.]+ [^,]+),', '\\1')(self).split(' ') return get_bytes_size(float(value), unit) class TorrentPage(HTMLPage): @method class get_torrent(ItemElement): klass = Torrent def obj_id(self): return self.page.url.split('/')[-1] def obj_url(self): return NotAvailable obj_name = CleanText('//div[@id="title"]') obj_magnet = CleanText('//div[@class="download"]/a[starts-with(@href, "magnet:")]/@href') obj_date = Date(CleanText('//div[@id="details"]//dt[.="Uploaded:"]/following-sibling::dd[1]')) obj_size = Type(Regexp(CleanText('//div[@id="details"]//dt[.="Size:"]/following-sibling::dd[1]'), r'\((\d+) Bytes\)', '\\1'), type=float) obj_seeders = Type(CleanText('//div[@id="details"]//dt[.="Seeders:"]/following-sibling::dd[1]'), type=int) obj_leechers = Type(CleanText('//div[@id="details"]//dt[.="Leechers:"]/following-sibling::dd[1]'), type=int) obj_description = RawText('//div[@class="nfo"]/pre', children=True) class FilesPage(HTMLPage): def get_files(self): return [" ".join([td.text for td in tr.xpath('./td')]) for tr in self.doc.xpath('//table/tr')] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/piratebay/test.py����������������������������������������������������������������0000664�0000000�0000000�00000003155�13034501105�0017606�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Julien Veyssier, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.torrent import MagnetOnly from random import choice class PiratebayTest(BackendTest): MODULE = 'piratebay' def test_torrent(self): # try something popular so we sometimes get a magnet-only torrent l = list(self.backend.iter_torrents('ubuntu linux')) if len(l): torrent = choice(l) full_torrent = self.backend.get_torrent(torrent.id) assert torrent.name assert full_torrent.name == torrent.name # I assume descriptions can be empty assert isinstance(full_torrent.description, basestring) try: assert self.backend.get_torrent_file(torrent.id) except MagnetOnly as e: assert e.magnet.startswith('magnet:') assert e.magnet == full_torrent.magnet �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017144�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000001450�13034501105�0021255�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PixtoilelibreModule __all__ = ['PixtoilelibreModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/browser.py���������������������������������������������������������0000664�0000000�0000000�00000003612�13034501105�0021203�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from weboob.tools.capabilities.paste import image_mime from StringIO import StringIO from .pages import PageHome, PageImage, PageError __all__ = ['PixtoilelibreBrowser'] class PixtoilelibreBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'pix.toile-libre.org' ENCODING = None PAGES = {'%s://%s/' % (PROTOCOL, DOMAIN): PageHome, r'%s://%s/\?action=upload': PageError, r'%s://%s/\?img=(.+)' % (PROTOCOL, DOMAIN): PageImage} def post_image(self, filename, contents, private=False, description=''): self.location('/') assert self.is_on_page(PageHome) mime = image_mime(contents.encode('base64')) self.select_form(nr=0) self.form.find_control('private').items[0].selected = private self.form['description'] = description or '' self.form.find_control('img').add_file(StringIO(contents), filename=filename, content_type=mime) self.submit() assert self.is_on_page(PageImage) return self.page.get_info() def get_contents(self, id): return self.readurl('%s://%s/upload/original/%s' % (self.PROTOCOL, self.DOMAIN, id)) ����������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/favicon.png��������������������������������������������������������0000664�0000000�0000000�00000006517�13034501105�0021310�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs��@��@[���tIME 1 ?F�� IDATx}t?7I RP!T:<,]Zf@J7 30tbe=G9jOG=rBUhe]_Ӝn&2G"JTl5$yd0='$^wy_ș |( 4Xx @eX T759Sv <J99Oe%HVIY_Θq\ɓO݊.]Y(N,veu5X6XV4->%O~,־|#NYo"4⺮>q0@iU2 ֧\ִ]b׮çD".@U-h.mxAH |Hઊxe>2:cu@1?|8+rI&"1=(d@Dΰ8B 93 � @$}c'0qSr 0gu�/-%FeKQ}DD& ]o&>K\A>HRcpQ:s@$}p܆km~?6ܹ6rJq]c@߹>^;vT[[Drj�طoՈȝ"f8�޸�_zmf7o*Z[;7{O"ȏK<�~8]Hdk0=+,  oy:Q LJBقx@}y9O�ph4dHcnZn]Gl1)D&`�*@yYx3aYWaӦeOfߨǛo:u ]t-п{UUUꕪ31 m3gQ47[HaOtYϋ.嶶T&w 6%zbόoHh4J8^ہM"nF_dŹ@RUr'ԭ1c"X6`uMK [~d466D@}1^VՊkjjZY(c֪h4zU[{gQ"jcub8Ei\9P 1\YSSӘ+J"RcT>?'u.5PaQ{zR>@[sl5�Fdyf/_Kߕ%G'"*8""`#^)`[[ h)̠�n6z!W)guQ�<]g<}Y)D~_`JYgv7 l(-D" 3qU]F䩧:lܹtvvkm'"רNsPUk5kTX,m_R٪uu1Z7xr<<y7+gtRjjjBWX73^mFߎD"8sm?n (uL:hT>_x/|QU`A }ݻlW(߶-#JKFkkf&xbx "W)2?r�w�%y`ƌc1 0[U+Ej=[D~ԴB+  +%%lׇGP8'"U+"iM;&せ,X{|5Gp*yK.i//S&NDzz~{þ}˺`<~s眃\vYNQ;urtI$8C8fԨQ1+:txT= |UfΜISS ~?ݻHuz{'4wcT+?x8L3մs7a޽/R"'ocL&{WyUߛܜJ/NN68cƌ1fz]ss黹 /_ǿ9"ow`kYg(PAsى!"۷eekk{`YIUmulY_?HUD+BPSDe`<�o{@`Z6mɂ>g#I6O>{.cQw�vzU}/_�TǞ{R@2oos|+/p>;C`pgmG�[^VV6A9]&MSߋ-6�vsswس- DďxM^x@?_xhR*zs]GGGHWyw`ĉIR+STufNŋ8H$֥RZuր<0kرWaUFaaH$gYf ˖-Tm�Bk)�UuE4}%ρe^Plt' 0{쑉Dbpm֩w>Ŕ/W&$lSuuu˗/G�"PO`<iYQSS@̙3#+ lbm|>~-"Ezt}QjcL2bm&X R yXFw{L'`ǎ5ece?DOH1$=` P9\T[[=[n)H&uXX)((;wN6kY>}^ӨX,v#RZ3(҂<O ל(�DPDO${Ӫ%8B zH/v� i'W |f^nU`kjw$�\e*/_qRƘ -uuu!2Gj"H>><#rX]]8ۀ3N%Kԭ^E;}hYO'!mcv7ź}Nuvv{?7j|ƘD2nd=m.m{m[lVgmhtl@KI,X,m_A -Xp$o_ض]3@AX̍b9$ xIm#pJ_ޥ c/8A@&�Sτ/�Q$I �r֧ [l@,rgB kWKJJj@gbԙp^.))9Ei׬X,ёmI!#mxq>lb%K|$="f>mW',����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/module.py����������������������������������������������������������0000664�0000000�0000000�00000004267�13034501105�0021014�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.paste import CapPaste, BasePaste from weboob.tools.capabilities.paste import image_mime import re from .browser import PixtoilelibreBrowser __all__ = ['PixtoilelibreModule'] class PixPaste(BasePaste): @classmethod def id2url(cls, id): return 'http://pix.toile-libre.org/?img=%s' % id class PixtoilelibreModule(Module, CapPaste): NAME = 'pixtoilelibre' DESCRIPTION = u'toile-libre image hosting website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = PixtoilelibreBrowser def can_post(self, contents, title=None, public=None, max_age=None): if re.search(r'[^a-zA-Z0-9=+/\s]', contents): return 0 elif max_age: return 0 # expiration is not possible else: mime = image_mime(contents, ('gif', 'jpeg', 'png')) return 20 * int(mime is not None) def get_paste(self, id): paste = PixPaste(id) contents = self.browser.get_contents(id) if contents: paste.contents = contents.encode('base64') return paste def new_paste(self, *a, **kw): return PixPaste(*a, **kw) def post_paste(self, paste, max_age=None): d = self.browser.post_image(paste.title or '-', paste.contents.decode('base64'), private=(not paste.public), description=paste.title) paste.id = d['id'] return paste �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pixtoilelibre/pages.py�����������������������������������������������������������0000664�0000000�0000000�00000001752�13034501105�0020622�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page import re class PageHome(Page): pass class PageImage(Page): def get_info(self): id = re.search(r'img=([^&]+)', self.url).group(1) return {'url': self.url, 'id': id} class PageError(Page): pass ����������������������weboob-1.2/modules/pixtoilelibre/test.py������������������������������������������������������������0000664�0000000�0000000�00000002512�13034501105�0020475�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PixtoilelibreTest(BackendTest): MODULE = 'pixtoilelibre' # small gif file DATA = 'R0lGODlhAQABAIAAAP///wAAACH5BAAAAAAALAAAAAABAAEAAAICRAEAOw==\n' def test_pixtoilelibre(self): assert self.backend.can_post(self.DATA, max_age=0) post = self.backend.new_paste(None) post.contents = self.DATA post.public = True self.backend.post_paste(post, max_age=0) assert post.id got = self.backend.get_paste(post.id) assert got assert got.contents.decode('base64') == self.DATA.decode('base64') ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015560�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0017670�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PlayMeModule __all__ = ['PlayMeModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000012273�13034501105�0017622�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser import DomainBrowser from weboob.browser.exceptions import ClientError from weboob.browser.pages import HTMLPage from weboob.browser.profiles import Profile from weboob.exceptions import BrowserIncorrectPassword from weboob.tools.json import json __all__ = ['PlayMeBrowser', 'FacebookBrowser'] class NoCredits(Exception): pass class FacebookBrowser(DomainBrowser): BASEURL = 'https://graph.facebook.com' CLIENT_ID = "149987128492319" access_token = None info = None def login(self, username, password): self.location('https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=https://www.facebook.com/connect/login_success.html&scope=email,user_birthday,user_friends,public_profile,user_photos,user_likes&response_type=token' % self.CLIENT_ID) page = HTMLPage(self, self.response) form = page.get_form('//form[@id="login_form"]') form['email'] = username form['pass'] = password form['persistent'] = 1 form.submit(allow_redirects=False) if 'Location' not in self.response.headers: raise BrowserIncorrectPassword() self.location(self.response.headers['Location']) m = re.search('access_token=([^&]+)&', self.url) if m: self.access_token = m.group(1) self.info = self.request('/me') def request(self, url, *args, **kwargs): url += '?access_token=' + self.access_token r = self.location(self.absurl(url, base=True), *args, **kwargs) return json.loads(r.content) class IPhoneClient(Profile): def setup_session(self, session): session.headers["Accept-Language"] = "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5" session.headers["Accept"] = "*/*" session.headers["User-Agent"] = "PlayMe/3.0.2 (iPhone; iOS 7.1; Scale/2.00)" session.headers["Accept-Encoding"] = "gzip, deflate" session.headers["Content-Type"] = "application/json" class PlayMeBrowser(DomainBrowser): BASEURL = 'https://api2.goplayme.com/' PROFILE = IPhoneClient() VERIFY = False recs = [] def __init__(self, facebook, *args, **kwargs): super(PlayMeBrowser, self).__init__(*args, **kwargs) self.facebook = facebook profile_picture = 'http%3A%2F%2Fgraph.facebook.com%2F' + facebook.info['id'] + '%2Fpicture%3Fwidth%3D600%26height%3D600' me = self.request('/auth/facebook/callback?access_token=%s&profile_picture=%s' % (facebook.access_token, profile_picture)) self.session.headers['Authorization'] = 'Token token="%s"' % me['token'] self.my_id = me['id'] self.my_name = me['name'] self.credits = me['credits']['count'] def get_threads(self): r = self.request('/users/%s/contacts' % self.my_id) if 'status' in r: return [] return reversed(r) def get_thread_messages(self, contact_id): return self.request('/messages/%s' % contact_id) def get_user(self, contact_id): return self.request('/users/%s' % contact_id) def post_message(self, contact_id, content): self.request('/messages', data={'id': contact_id, 'msg': content}) def request(self, *args, **kwargs): if 'data' in kwargs: kwargs['data'] = json.dumps(kwargs['data']) r = self.location(*args, **kwargs) return json.loads(r.content) def find_users(self, lat, lon): r = self.request('/users/?lat=%s&lon=%s&type=full' % (lat, lon)) return r['pending'] + r['history'] def get_theme(self): r = self.request('/questions') for t in r: if t['theme']['is_vip']: continue return t def challenge(self, user_id): try: r = self.request('/users/%s/challenge/%s' % (self.my_id, user_id)) except ClientError as e: r = json.loads(e.response.content) self.credits = r['credits']['count'] raise NoCredits(r['credits']['next_restore_in_seconds']) if isinstance(r, list) and 'questions' in r[0]: t = r[0] else: t = self.get_theme() self.credits = r['credits']['count'] data = {} data['theme'] = {'id': t['theme']['id'], 'is_vip': 0} data['questions'] = [q['id'] for q in t['questions']][:5] data['answers'] = [{'duration': 1000, 'result': 1} for q in t['questions'][:5]] self.request('/users/%s/challenge/%s' % (self.my_id, user_id), data=data) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000002135�13034501105�0017714�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��� pHYs�� �� ����tIME 5 X ���tEXtComment�Created with GIMPW��IDATx[kUߛFViD" EkP+/1ȗP b66QLK$ͩx5q43μ4{翟Zk)RHbFTZσXU̡\ѝo+ƪ}&J`.E [\$TS 7cO,uD�gYU\ ?@kY p,~W8:WTnV�Nc 3XF�;c%D_\13WjU�A\q ?Mb.+)@9fH"y,4s4p6tmi| .6#e}0a܎[ZeLFn_G}1%wݿ9V\p<nB�Xڧ ;q/Q<яt(ABh d؃x �7f֤#pp2+Y( :;Ѓ;q�/'F#Ǥ X爚+z\$˧@ {�OlT4=Pi1t4DS4ܱsKwmin(pAwCx/Q]w⵨Pw+܃ǣVx6>QL!|WCx-nAmPx s�wpW8Uud$q.+Y(: |5,> �6'C!|dfmDc~+mr�ގ3ۤݞ C!|G[@�՛^B EF#NY}glu`$o�=Mj`49#aZB?QFB@!ͯԝ8۳r|FQ /bYD�5(v3m~ǧ>Hj˝xLtshw6$?yvxǿv%"4fm*Iu&X#H� H� H� H"E)Wh7q3p����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000017051�13034501105�0017423�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime from weboob.capabilities.messages import CapMessages, CapMessagesPost, Thread, Message from weboob.capabilities.dating import CapDating, Optimization from weboob.capabilities.account import CapAccount, StatusField from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.date import local2utc from weboob.tools.log import getLogger from .browser import PlayMeBrowser, FacebookBrowser, NoCredits __all__ = ['PlayMeModule'] class ProfilesWalker(Optimization): def __init__(self, sched, storage, browser): super(ProfilesWalker, self).__init__() self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('walker', browser.logger) self._view_cron = None def start(self): self._view_cron = self._sched.schedule(1, self.view_profile) return True def stop(self): self._sched.cancel(self._view_cron) self._view_cron = None return True def set_config(self, params): pass def is_running(self): return self._view_cron is not None def view_profile(self): delay = 900 try: challenged = self._storage.get('challenged', default=[]) for user in self._browser.find_users(48.883989, 2.367168): if user['id'] in challenged: continue try: self._browser.challenge(user['id']) except NoCredits as e: delay = int(str(e)) self._logger.info('No more credits (next try in %d minutes)', (delay/60)) else: self._logger.info('Challenged %s', user['name']) challenged.append(user['id']) self._storage.set('challenged', challenged) self._storage.save() break finally: if self._view_cron is not None: self._view_cron = self._sched.schedule(delay, self.view_profile) class PlayMeModule(Module, CapMessages, CapMessagesPost, CapDating, CapAccount): NAME = 'playme' DESCRIPTION = u'PlayMe dating mobile application' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Facebook email'), ValueBackendPassword('password', label='Facebook password')) BROWSER = PlayMeBrowser STORAGE = {'contacts': {}, 'challenged': [], } def create_default_browser(self): facebook = FacebookBrowser() facebook.login(self.config['username'].get(), self.config['password'].get()) return self.create_browser(facebook) # ---- CapDating methods ----------------------- def init_optimizations(self): self.add_optimization('PROFILE_WALKER', ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)) # ---- CapMessages methods --------------------- def fill_thread(self, thread, fields): return self.get_thread(thread) def iter_threads(self): for thread in self.browser.get_threads(): t = Thread(thread['id']) t.flags = Thread.IS_DISCUSSION t.title = u'Discussion with %s' % thread['name'] t.date = local2utc(datetime.datetime.fromtimestamp(thread['last_message']['utc_timestamp'])) yield t def get_thread(self, thread): if not isinstance(thread, Thread): thread = Thread(thread) thread.flags = Thread.IS_DISCUSSION user = self.browser.get_user(thread.id) thread.title = u'Discussion with %s' % user['name'] contact = self.storage.get('contacts', thread.id, default={'lastmsg': 0}) signature = u'Age: %s' % user['age'] signature += u'\nLast online: %s' % user['last_online'] signature += u'\nPhotos:\n\t%s' % '\n\t'.join([user['photo_host'] + photo['large'] for photo in user['photos']]) child = None for msg in self.browser.get_thread_messages(thread.id): flags = 0 if int(contact['lastmsg']) < msg['utc_timestamp']: flags = Message.IS_UNREAD if msg['type'] == 'msg': content = unicode(msg['msg']) elif msg['type'] == 'new_challenge': content = u'A new challenge has been proposed!' elif msg['type'] == 'serie': content = u"I've played" elif msg['type'] == 'end_game': content = u'%s is the winner! (%s VS %s)' % (self.browser.my_name if msg['score']['w'] == self.browser.my_id else user['name'], msg['score']['s'][0], msg['score']['s'][1]) else: content = u'Unknown action: %s' % msg['type'] msg = Message(thread=thread, id=msg['utc_timestamp'], title=thread.title, sender=unicode(self.browser.my_name if msg['from'] == self.browser.my_id else user['name']), receivers=[unicode(self.browser.my_name if msg['from'] != self.browser.my_id else user['name'])], date=local2utc(datetime.datetime.fromtimestamp(msg['utc_timestamp'])), content=content, children=[], parent=None, signature=signature if msg['from'] != self.browser.my_id else u'', flags=flags) if child: msg.children.append(child) child.parent = msg child = msg thread.root = child return thread def iter_unread_messages(self): for thread in self.iter_threads(): thread = self.get_thread(thread) for message in thread.iter_all_messages(): if message.flags & message.IS_UNREAD: yield message def set_message_read(self, message): contact = self.storage.get('contacts', message.thread.id, default={'lastmsg': 0}) if int(contact['lastmsg']) < int(message.id): contact['lastmsg'] = int(message.id) self.storage.set('contacts', message.thread.id, contact) self.storage.save() # ---- CapMessagesPost methods --------------------- def post_message(self, message): self.browser.post_message(message.thread.id, message.content) # ---- CapAccount methods --------------------- def get_account_status(self): return (StatusField(u'myname', u'My name', unicode(self.browser.my_name)), StatusField(u'credits', u'Credits', unicode(self.browser.credits)), ) OBJECTS = {Thread: fill_thread, } ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/playme/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000001646�13034501105�0017120�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PlayMeTest(BackendTest): MODULE = 'playme' def test_playme(self): for m in self.backend.iter_unread_messages(): pass ������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016257�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0020375�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PodnapisiModule __all__ = ['PodnapisiModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000003520�13034501105�0020314�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from .pages import SearchPage, SubtitlePage, LANGUAGE_NUMBERS __all__ = ['PodnapisiBrowser'] class PodnapisiBrowser(Browser): DOMAIN = 'www.podnapisi.net' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://www.podnapisi.net/fr/ppodnapisi/search\?sJ=[0-9]*&sK=.*&sS=downloads&sO=desc': SearchPage, 'http://www.podnapisi.net/fr/ppodnapisi/podnapis/i/[0-9]*': SubtitlePage } def iter_subtitles(self, language, pattern): nlang = LANGUAGE_NUMBERS[language] self.location('http://www.podnapisi.net/fr/ppodnapisi/search?sJ=%s&sK=%s&sS=downloads&sO=desc' % (nlang, pattern.encode('utf-8'))) assert self.is_on_page(SearchPage) return self.page.iter_subtitles(unicode(language)) def get_subtitle(self, id): try: self.location('http://www.podnapisi.net/fr/ppodnapisi/podnapis/i/%s' % id) except BrowserHTTPNotFound: return if self.is_on_page(SubtitlePage): return self.page.get_subtitle(id) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/favicon.png������������������������������������������������������������0000664�0000000�0000000�00000003101�13034501105�0020405�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME  )P˞���tEXtComment�Created with GIMPW��IDATxkTU6>j^b5JtfIYr44%")#B_JC_&J{#´@L4Au{5v\Gvsf̘1w΂ag^{@N9SN9ԨuD|h#ޘM @! +rƥ0\n t݀X�Iw4(֓񣯔jq}/�}�ڀg䚁'{ˀNJ9(@+5XiLPv�_|Pb6`1:G-Wz.c &,IIQSJ^Ѿ.B_I5N6s?0VX_gLpʘj|7u3&XmL|3`zL֘1NjbAS c60?߳4N�0&+ g8Еx@C></4}gR[0"7Ăf9*J'�',^S~ӿ+aU] �{c@K} l^6&X2zg9AӘ̃a5yA 0D}Ί]lro2 82792og2)c"/N~�8ILcքMϨL;0Řh]y/ZcPi�-�nuZ ֤GCl0&r "-`LЦ�<Gq/b>0Ę`ULO:zG+G`X-~zP(tG^,q$65B#sx""?[N ? 3EY+iEvZV+nEZ׭Hϸ.m"~+rŠ"f1h>`AB|%Nו1@wOsf]t2ޘf qH˴]W/wݤ/(�.@q;.Ҍ,�)Zi�XiZXݗ:scQ#^npw)�|hV �/mX.H֔v}[T"}6^u4y!*r`cj XQ ﵭ\R tvxaO9Uv/ j78(vmjػX/ ߎ: Cԋ8]3὎We@ߊLO 1_aW/ za8Y<KΊL;a_qY<$E/9+ҬF7�75y9}� ) )Ȭu[2O �ZKlo ˁG++Pe}U2ѯˉ�taA,{?�Nujd=JW?5 r_�Y{�j5 XA/NF5<r)r5}9j%����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004135�13034501105�0020121�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.subtitle import CapSubtitle, LanguageNotSupported, Subtitle from weboob.applications.suboob.suboob import LANGUAGE_CONV from weboob.tools.backend import Module from .browser import PodnapisiBrowser from urllib import quote_plus __all__ = ['PodnapisiModule'] class PodnapisiModule(Module, CapSubtitle): NAME = 'podnapisi' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Podnapisi movies and tv series subtitle website' LICENSE = 'AGPLv3+' BROWSER = PodnapisiBrowser def get_subtitle(self, id): return self.browser.get_subtitle(id) def get_subtitle_file(self, id): subtitle = self.browser.get_subtitle(id) if not subtitle: return None return self.browser.openurl(subtitle.url.encode('utf-8')).read() def iter_subtitles(self, language, pattern): if language not in LANGUAGE_CONV.keys(): raise LanguageNotSupported() return self.browser.iter_subtitles(language, quote_plus(pattern.encode('utf-8'))) def fill_subtitle(self, subtitle, fields): if 'description' in fields or 'url' in fields: sub = self.get_subtitle(subtitle.id) subtitle.description = sub.description subtitle.url = sub.url return subtitle OBJECTS = { Subtitle: fill_subtitle, } �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/pages.py���������������������������������������������������������������0000664�0000000�0000000�00000010010�13034501105�0017720�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.subtitle import Subtitle from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.deprecated.browser import Page LANGUAGE_NUMBERS = { 'sq': '29', 'de': '5', 'en': '2', 'ar': '12', 'bn': '59', 'be': '50', 'bg': '33', 'ca': '53', 'zh': '17', 'ko': '4', 'hr': '38', 'da': '24', 'es': '28', 'et': '20', 'fi': '31', 'fr': '8', 'gr': '16', 'hi': '42', 'nl': '23', 'hu': '15', 'iw': '22', 'id': '54', 'ga': '49', 'is': '6', 'it': '9', 'ja': '11', 'lv': '21', 'lt': '19', 'mk': '35', 'ms': '55', 'no': '3', 'pl': '26', 'pt': '32', 'ro': '13', 'ru': '27', 'sr': '36', 'sk': '37', 'sl': '1', 'sv': '25', 'cz': '7', 'th': '44', 'tr': '30', 'uk': '46', 'vi': '51' } class SearchPage(Page): """ Page which contains results as a list of movies """ def iter_subtitles(self, language): linksresults = self.parser.select(self.document.getroot(), 'a.subtitle_page_link') for link in linksresults: id = unicode(link.attrib.get('href', '').split('-p')[-1]) name = unicode(link.text_content()) tr = link.getparent().getparent().getparent() cdtd = self.parser.select(tr, 'td')[4] nb_cd = int(cdtd.text) description = NotLoaded subtitle = Subtitle(id, name) subtitle.nb_cd = nb_cd subtitle.language = language subtitle.description = description yield subtitle class SubtitlePage(Page): """ Page which contains a single subtitle for a movie """ def get_subtitle(self, id): language = NotAvailable url = NotAvailable nb_cd = NotAvailable links_info = self.parser.select(self.document.getroot(), 'fieldset.information a') for link in links_info: href = link.attrib.get('href', '') if '/fr/ppodnapisi/kategorija/jezik/' in href: nlang = href.split('/')[-1] for lang, langnum in LANGUAGE_NUMBERS.items(): if str(langnum) == str(nlang): language = unicode(lang) break desc = u'' infos = self.parser.select(self.document.getroot(), 'fieldset.information') for info in infos: for p in self.parser.select(info, 'p'): desc += '%s\n' % (u' '.join(p.text_content().strip().split())) spans = self.parser.select(info, 'span') for span in spans: if span.text is not None and 'CD' in span.text: nb_cd = int(self.parser.select(span.getparent(), 'span')[1].text) title = unicode(self.parser.select(self.document.getroot(), 'head title', 1).text) name = title.split(' - ')[0] dllinks = self.parser.select(self.document.getroot(), 'div.footer > a.download') for link in dllinks: href = link.attrib.get('href', '') if id in href: url = u'http://www.podnapisi.net%s' % href subtitle = Subtitle(id, name) subtitle.url = url subtitle.language = language subtitle.nb_cd = nb_cd subtitle.description = desc return subtitle ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/podnapisi/test.py����������������������������������������������������������������0000664�0000000�0000000�00000002524�13034501105�0017613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from random import choice class PodnapisiTest(BackendTest): MODULE = 'podnapisi' def test_subtitle(self): lsub = [] subtitles = self.backend.iter_subtitles('fr', 'spiderman') for i in range(5): subtitle = subtitles.next() lsub.append(subtitle) assert (len(lsub) > 0) # get the file of a random sub if len(lsub): subtitle = choice(lsub) self.backend.get_subtitle_file(subtitle.id) ss = self.backend.get_subtitle(subtitle.id) assert ss.url.startswith('http') ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015437�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001435�13034501105�0017553�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PoivyModule __all__ = ['PoivyModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000003375�13034501105�0017504�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import HomePage, LoginPage, HistoryPage, BillsPage, ErrorPage __all__ = ['PoivyBrowser'] class PoivyBrowser(LoginBrowser): BASEURL = 'https://www.poivy.com' login = URL('/login', LoginPage) homepage = URL('/buy_credit.*', HomePage) history = URL('/recent_calls', HistoryPage) bills = URL('/purchases', BillsPage) warning = URL('/warning.*', ErrorPage) def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) self.login.stay_or_go() self.page.login(self.username, self.password) if self.login.is_here() or self.warning.is_here(): raise BrowserIncorrectPassword() @need_login def get_subscription_list(self): return self.homepage.stay_or_go().get_list() @need_login def get_history(self): self.history.stay_or_go() return self.page.get_calls() �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000003504�13034501105�0017574�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME #47;���iTXtComment�����Created with GIMPd.e��IDATx{PTǿ>va#{,H"1ꂰ4/(3jLl !d$6$JHQL J Dh]]f2ɔ{;g.p8p8p8OKeE_PrAǘoF|Zn]c� Or"bo[Fw|ZR8V��mRF/EϧQ/o,~{IZ BA|hxޕE2x4>XVX,T]]s0% �ٵC?cYZ�RWڷ|~럨Ass3z?TܔEG =d͙+�<kɲO~OSuww>FDE1{<Ӓb۝;:rt,ɈlF%RSS|7&'ϟ7?kY{Kl޽ht _{M  &C"/:Qz^9sZ- @�DQpھmZz5i4ڀ~�H$Z~&�� ƆiE@Hz:0Rmhݷn8C̝;t:a0`6c0Nd`?aWWeeBE?&MBXXO˅.8r<͌F#uww㽊:3jN', N+/$ %%% (30"A6q` Yi玝~nE�PSC �%&&RdM �ي �F# @eGb/\*UAuy#^FMB_, $@N A3~zq7��ŀ[OQt:ݭN-D$W,S\T�8r^Z_>rrst�0{,DO~�0cLؚinPՀNUG'~D$|m.�q#�׷oáOʆ~ٜ;�X,غekdoF' `JRvP xPt *lnςx<-I_s)�U녋( ~qS|!A pi5 DЀ112tmI�pՀInw\oU =>}jҝOx뾾beFORmm-dYF+EH[K'>y]R�5T7xgx@`dY:Ą˲@3f̀pS,x [+ڙ:mmmhkk(Xy xB{" 0<=(Js�i40ax�WgB>L&1Fٜ\r|u&Ǎ#�TQ^qܶn2Ev%+z? ~Tos}-1 ,XrL7$IERRӬ_L }qءƦN9ܼ{E O:?o��;z]]$'gBf+*zlf&^ VЇ=Ǽhpաp!iVV ׋񪫫p8p8p8Ψ?g¿?o����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/module.py������������������������������������������������������������������0000664�0000000�0000000�00000005360�13034501105�0017302�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, SubscriptionNotFound, Detail from weboob.capabilities.base import find_object from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import PoivyBrowser __all__ = ['PoivyModule'] class PoivyModule(Module, CapDocument): NAME = 'poivy' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = 'Poivy website' CONFIG = BackendConfig(ValueBackendPassword('login', label='login', masked=False), ValueBackendPassword('password', label='Password') ) BROWSER = PoivyBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def iter_documents_history(self, subscription): # Try if we have a real subscription before to load the history if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.get_history() # No details on the website def get_details(self, subscription): raise NotImplementedError() def get_balance(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) balance = Detail() balance.id = "%s-balance" % subscription.id balance.price = subscription._balance balance.label = u"Balance %s" % subscription.id balance.currency = u'EUR' return balance ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/poivy/pages.py�������������������������������������������������������������������0000664�0000000�0000000�00000005767�13034501105�0017127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.exceptions import BrowserBanned from weboob.browser.pages import HTMLPage, LoggedPage, pagination from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Field, DateTime, Format from weboob.browser.filters.html import Attr, Link from weboob.capabilities.bill import Subscription, Detail class ErrorPage(HTMLPage): pass class LoginPage(HTMLPage): def login(self, login, password): captcha = self.doc.xpath('//label[@class="label_captcha_input"]') if len(captcha) > 0: raise BrowserBanned('Too many connections from you IP address: captcha enabled') xpath_hidden = '//form[@id="newsletter_form"]/input[@type="hidden"]' hidden_id = Attr(xpath_hidden, "value")(self.doc) hidden_name = Attr(xpath_hidden, "name")(self.doc) form = self.get_form(xpath="//form[@class='form-detail']") form['login[username]'] = login form['login[password]'] = password form[hidden_name] = hidden_id form.submit() class HomePage(LoggedPage, HTMLPage): @method class get_list(ListElement): item_xpath = '.' class item(ItemElement): klass = Subscription obj_id = CleanText('//span[@class="welcome-text"]/b') obj__balance = CleanDecimal(CleanText('//span[contains(@class, "balance")]'), replace_dots=False) obj_label = Format(u"Poivy - %s - %s €", Field('id'), Field('_balance')) class HistoryPage(LoggedPage, HTMLPage): @pagination @method class get_calls(ListElement): item_xpath = '//table/tbody/tr' next_page = Link("//div[@class='date-navigator center']/span/a[contains(text(), 'Previous')]", default=None) class item(ItemElement): klass = Detail obj_id = None obj_datetime = DateTime(CleanText('td[1] | td[2]')) obj_price = CleanDecimal('td[7]', replace_dots=False, default=0) obj_currency = u'EUR' obj_label = Format(u"%s from %s to %s - %s", CleanText('td[3]'), CleanText('td[4]'), CleanText('td[5]'), CleanText('td[6]')) #TODO class BillsPage(HTMLPage): pass ���������weboob-1.2/modules/poivy/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000002770�13034501105�0016776�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PoivyTest(BackendTest): MODULE = 'poivy' def test_list(self): """ Test listing of subscriptions . No support of multi-account on the website, we could assume to have only one subscription. Check the balance if the subscription is ok. """ subscriptions = list(self.backend.iter_subscription()) self.assertTrue(len(subscriptions) == 1, msg="Account listing failed") self.assertTrue(self.backend.get_balance(subscriptions[0]) > 0, msg="Get balance failed") def test_history(self): for subscription in self.backend.iter_subscription(): self.assertTrue(len(list(self.backend.iter_documents_history(subscription))) > 0) ��������weboob-1.2/modules/popolemploi/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016630�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001444�13034501105�0020744�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PopolemploiModule __all__ = ['PopolemploiModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000005442�13034501105�0020672�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import SearchPage, AdvertPage from weboob.browser import PagesBrowser, URL from urllib import quote __all__ = ['PopolemploiBrowser'] class PopolemploiBrowser(PagesBrowser): BASEURL = 'https://candidat.pole-emploi.fr/' advert = URL('candidat/rechercheoffres/detail/(?P<id>.*)', AdvertPage) search = URL('candidat/rechercheoffres/resultats/(?P<search>.*?)', SearchPage) decode_salary = { 'FOURCHETTE1': u'|15000|A', 'FOURCHETTE2': u'15000|18000|A', 'FOURCHETTE3': u'18000|21000|A', 'FOURCHETTE4': u'21000|24000|A', 'FOURCHETTE5': u'24000|36000|A', 'FOURCHETTE6': u'36000|60000|A', 'FOURCHETTE7': u'60000||A', } def search_job(self, pattern=None): search = "A_%s_____P__________INDIFFERENT_______________________" % \ quote(pattern.encode('utf-8')).replace('%', '$00') return self.search.go(search=search).iter_job_adverts() def advanced_search_job(self, metier='', place=None, contrat=None, salary=None, qualification=None, limit_date=None, domain=None): splitted_place = place.split('|') _domain = "%s-" % domain if domain else "" if salary in self.decode_salary: salary_time = self.decode_salary.get(salary).split('|')[2] salary_low = self.decode_salary.get(salary).split('|')[0] salary_hight = self.decode_salary.get(salary).split('|')[1] else: salary_time = "" salary_low = "" salary_hight = "" search = "A_%s_%s_%s__%s_P_%s_%s_%s_____________________%s_%s_%s_______" % ( quote(metier.encode('utf-8')).replace('%', '$00'), splitted_place[1], splitted_place[2], contrat, _domain, salary_time, qualification, limit_date, salary_low, salary_hight) return self.search.go(search=search).iter_job_adverts() def get_job_advert(self, id, advert): return self.advert.go(id=id).get_job_advert(obj=advert) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000012576�13034501105�0020776�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� Ǡ���tIME$kX�� IDATxyxUչ?k}>sNNf$H2(R^ŁrkQ?^jQk-`2 99>b˽?/Yϳd'{~}DC|@?~��W+j#24*0>.Fc Ci˩8 Z)ஓ<\Iawg{8@QMw1/ g( fn:@T aT` 7��GN:Og")<gKRC(�,i+Jex(gBaWD߷SvA[rE|? �i3N=)%*m�XsN[ōH?b`Z=5'2:= NvšT-AZ?+"di}jit+AMq%Lh&=X`Mcd7�G!7�`q-6 4lt $$R.$;"jI0ɸ qkkO+zRp"nJ0cf=1d$Ldhط]MZJskbc#N,?ƛ;P�q[p8q "XbÃ>4+C%HOIzI)EAqO|#VgsK&;KU՚$DqP?& e2/zC@@iЦEg6Z.YL?Xix5'Ϗ#N3ayȃf~M@5H*PHX_V $Y$Jf@YNir!YF<@@rq g|vL"B5e<q>w^܆6p"ҡ0А( QT)7~l,]Rg6ySc-Kl!v ν; _1Xi (M)t/X^?A0S~4ۋpy8J'jgp}$z.g+fǧXLFRswT z& Lb}E <3'?衑X.K'X7|sE<I56Q|a)5්<b N&{\ + OWm6+Ns;#°zC+Nj 砦O,V@ H O@W)~ihD1C<c 0DžR, 00bH9mZzݛ'#IBCCk).+""wP-;/Ys=W<Tl!}( (+(%,8'2H؎J$d=]<x%R4{ص+D<&lYy?UM]8糞4\BGW4U7Cjsx) _ |GC/h( p�TtVF$3ht`zmtÍa0LҊ&P`,݊Bu/Ieܼ5p} XhzW MJsB$bвqV1+F\`^ۃIiět6}X1Aܙ4z��~U(2r5p�< H)T.9|f�ڑ}# _•a 46ql -"+IIM1?IL CrkME"C5j~Ne^x/29:$r9 K@rv̴CĬ51z;e`'a�1B,F֧8�$gNIKÚKA`a�wMuT޼W~,%D%uBXӉ,C�6A3b�cXbB L2qq:Iu]"F?FB)$<* o) =N2vf2Jf n  &@ e@:PLQR8,!LT3䥘Qy|T&M*J%H&$R8?voo�2\X&JQ`wc)h-n9FȤ?jA1B`!p '\Nw�\ވ cG I,ұ�A F"B=xzbE\8Й:Kg=�3r a}v&9Ѻvw~? ` ["_yl&7[SKh`�“֢>#ԐLHGF(,q)~ҽ|6,X"ķ"bQͽ⩳v]LJrjGWRr#05.Fgfu*[0'&ʣnBJ@a ʬW8iA ]A<V=y+|K)y6.0Z&[L X6ѱJJ iA5{@1hDUF9M'6?*6ދRB< I2mt?nlJX|/- IaY0|3-Y#ϸ0FϠ20v|alu&A)hLzEI[ EdVh\pFMrBh&-ѬJHkAoO!zga[λӻ[9>|eWp \s9lOS"=EVh,z?dy |`Ҩ@ pGK$5mIGs阡(|= iR`dR:M" NzHI٭y!{9m '$A4 - 5:S$[6 qI:`�*fXTbjxi bBG:ʐ&Iҁ I4b!2QS>i0&REVyn'9bK NBhDv\58P'.Zf5b+<CwE! mᷮtɮ$l1vY8 vv{$iQѤWHT="rQMnF ڸC�~&dLUNK5Do:Uy=n*j<(%VDN['DD%ٟQh"X(e!5WP8y-;(Ey:0}Vͧ b!b!KEnDLD܈q 4`b b˶ǼOK5":Ѡ՗=dgnoÛADD  6PX#Z,.k4:IW J.{.3Ȑ ف*F ~�R1OVl^Y qzR!).EG`elaRr`4al܂1"*&JY]3jx1HQ5 3A"ąHh�> 2j-)ڃ^#|Uue":"�<t'>^tf7G:Z00: nw ق`T Fy!,BO|Dqa Q'h!b5%)"sH.,s'K}YAG?n�/0sʗz{X,l(dTzGvP �"^VI$@"BmXJʭE=BAgA՘,ЅE.WvWE;BkC H) !|"!NIxț}_RCx .[*vXיH6cȐ[3?nU|ϛ19giM.B n7"D9 ׺e&1 11U~9 H�UK~B% q?:�5e!t? @ɥh;&AGXl#Zy}38p-15wMƮtkvoqSDX]]I9DYjپ|S1�ti 8*23F7@%\Rw'p~Ə|gdZY%ՓOr-˞EL~aYJKtܿV΍K#!kgCY Nx0epǠ)>Ȥ[gϦspRV5s2wXf9]DS%q5C~u6]o-Z8gN:Y}-qػy.Y0[ bfB4a/`ȷEHrȟGO.{~/]O^O+}GZZ"~J8t'|_yz&z]-7C2jn>yWqU{9\Vرe;%g4Β.z6u)ݦ3%oZܭ9\{`lqFW8vTi!?|<$?Gw<s^ֹIvwޟoϦݝ< h.OaZ +|5\2V{\?$]pxD y?L<C=Ib3fRm-MoE2E$i%qDCimMD"06L6BMbɃ'Sh3DS0ƏV6ZαK0:�ߑn@iH ]'p"wq1c8}:-LԠ{<=k�;G$S+?cp?"Ά3g$vۮ}5I|B Q/Ly*F[ۺP}۱ #8۱}^=a4Vz=r1 c;y542í< 8g}!�T++_|d׬6[�,p>d _;h3W+MnqО;@=lHKZ R!|~qdNv>Tn@%tnbk(xݙ뼑M4̄;6zǡgo3?\"R18R{TLōr."xCrPx)Cl^B\*L+IxIBx4, P !aᢳa\7aADT!$xzjw![xFNE$YTh#aAsO re"^:9-~C+1#e.jR[a u /u ay�aHxKk)~L HC'+ʐT!F>DQ_?@?~��q#3=> Z����IENDB`����������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/module.py������������������������������������������������������������0000664�0000000�0000000�00000037310�13034501105�0020473�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.job import BaseJobAdvert from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.job import CapJob from weboob.tools.value import Value from weboob.tools.ordereddict import OrderedDict from .browser import PopolemploiBrowser __all__ = ['PopolemploiModule'] class PopolemploiModule(Module, CapJob): NAME = 'popolemploi' DESCRIPTION = u'Pole Emploi website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' BROWSER = PopolemploiBrowser places_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '100|PAYS|01': u'France entière', '102|REGION|42': u'Alsace', '103|DEPARTEMENT|67': u'-- Rhin (Bas) (67)', '104|DEPARTEMENT|68': u'-- Rhin (Haut) (68)', '105|REGION|72': u'Aquitaine', '106|DEPARTEMENT|24': u'-- Dordogne (24)', '107|DEPARTEMENT|33': u'-- Gironde (33)', '108|DEPARTEMENT|40': u'-- Landes (40)', '109|DEPARTEMENT|47': u'-- Lot et Garonne (47)', '110|DEPARTEMENT|64': u'-- Pyrénées Atlantiques (64)', '111|REGION|83': u'Auvergne', '112|DEPARTEMENT|03': u'-- Allier (03)', '113|DEPARTEMENT|15': u'-- Cantal (15)', '114|DEPARTEMENT|43': u'-- Loire (Haute) (43)', '115|DEPARTEMENT|63': u'-- Puy de Dôme (63)', '116|REGION|26': u'Bourgogne', '117|DEPARTEMENT|21': u'-- Côte d\'Or (21)', '118|DEPARTEMENT|58': u'-- Nièvre (58)', '119|DEPARTEMENT|71': u'-- Saône et Loire (71)', '120|DEPARTEMENT|89': u'-- Yonne (89)', '121|REGION|53': u'Bretagne', '122|DEPARTEMENT|22': u'-- Côtes d\'Armor (22)', '123|DEPARTEMENT|29': u'-- Finistère (29)', '124|DEPARTEMENT|35': u'-- Ille et Vilaine (35)', '125|DEPARTEMENT|56': u'-- Morbihan (56)', '126|REGION|24': u'Centre', '127|DEPARTEMENT|18': u'-- Cher (18)', '128|DEPARTEMENT|28': u'-- Eure et Loir (28)', '129|DEPARTEMENT|36': u'-- Indre (36)', '130|DEPARTEMENT|37': u'-- Indre et Loire (37)', '131|DEPARTEMENT|41': u'-- Loir et Cher (41)', '132|DEPARTEMENT|45': u'-- Loiret (45)', '133|REGION|21': u'Champagne Ardenne', '134|DEPARTEMENT|08': u'-- Ardennes (08)', '135|DEPARTEMENT|10': u'-- Aube (10)', '136|DEPARTEMENT|51': u'-- Marne (51)', '137|DEPARTEMENT|52': u'-- Marne (Haute) (52)', '138|REGION|94': u'Corse', '139|DEPARTEMENT|2A': u'-- Corse du Sud (2A)', '140|DEPARTEMENT|2B': u'-- Haute Corse (2B)', '141|REGION|43': u'Franche Comté', '142|DEPARTEMENT|90': u'-- Belfort (Territoire de) (90)', '143|DEPARTEMENT|25': u'-- Doubs (25)', '144|DEPARTEMENT|39': u'-- Jura (39)', '145|DEPARTEMENT|70': u'-- Saône (Haute) (70)', '146|REGION|11': u'Ile de France', '147|DEPARTEMENT|91': u'-- Essonne (91)', '148|DEPARTEMENT|92': u'-- Hauts de Seine (92)', '149|DEPARTEMENT|75': u'-- Paris (Dept.) (75)', '150|DEPARTEMENT|93': u'-- Seine Saint Denis (93)', '151|DEPARTEMENT|77': u'-- Seine et Marne (77)', '152|DEPARTEMENT|95': u'-- Val d\'Oise (95)', '153|DEPARTEMENT|94': u'-- Val de Marne (94)', '154|DEPARTEMENT|78': u'-- Yvelines (78)', '155|REGION|91': u'Languedoc Roussillon', '156|DEPARTEMENT|11': u'-- Aude (11)', '157|DEPARTEMENT|30': u'-- Gard (30)', '158|DEPARTEMENT|34': u'-- Hérault (34)', '159|DEPARTEMENT|48': u'-- Lozère (48)', '161|DEPARTEMENT|66': u'-- Pyrénées Orientales (66)', '162|REGION|74': u'Limousin', '163|DEPARTEMENT|19': u'-- Corrèze (19)', '164|DEPARTEMENT|23': u'-- Creuse (23)', '165|DEPARTEMENT|87': u'-- Vienne (Haute) (87)', '166|REGION|41': u'Lorraine', '167|DEPARTEMENT|54': u'-- Meurthe et Moselle (54)', '168|DEPARTEMENT|55': u'-- Meuse (55)', '169|DEPARTEMENT|57': u'-- Moselle (57)', '170|DEPARTEMENT|88': u'-- Vosges (88)', '171|REGION|73': u'Midi Pyrénées', '172|DEPARTEMENT|09': u'-- Ariège (09)', '173|DEPARTEMENT|12': u'-- Aveyron (12)', '174|DEPARTEMENT|31': u'-- Garonne (Haute) (31)', '175|DEPARTEMENT|32': u'-- Gers (32)', '176|DEPARTEMENT|46': u'-- Lot (46)', '177|DEPARTEMENT|65': u'-- Pyrénées (Hautes) (65)', '178|DEPARTEMENT|81': u'-- Tarn (81)', '179|DEPARTEMENT|82': u'-- Tarn et Garonne (82)', '180|REGION|31': u'Nord Pas de Calais', '181|DEPARTEMENT|59': u'-- Nord (59)', '182|DEPARTEMENT|62': u'-- Pas de Calais (62)', '183|REGION|25': u'Normandie (Basse)', '184|DEPARTEMENT|14': u'-- Calvados (14)', '185|DEPARTEMENT|50': u'-- Manche (50)', '186|DEPARTEMENT|61': u'-- Orne (61)', '187|REGION|23': u'Normandie (Haute)', '188|DEPARTEMENT|27': u'-- Eure (27)', '189|DEPARTEMENT|76': u'-- Seine Maritime (76)', '190|REGION|52': u'Pays de la Loire', '191|DEPARTEMENT|44': u'-- Loire Atlantique (44)', '192|DEPARTEMENT|49': u'-- Maine et Loire (49)', '193|DEPARTEMENT|53': u'-- Mayenne (53)', '194|DEPARTEMENT|72': u'-- Sarthe (72)', '195|DEPARTEMENT|85': u'-- Vendée (85)', '196|REGION|22': u'Picardie', '197|DEPARTEMENT|02': u'-- Aisne (02)', '198|DEPARTEMENT|60': u'-- Oise (60)', '199|DEPARTEMENT|80': u'-- Somme (80)', '200|REGION|54': u'Poitou Charentes', '201|DEPARTEMENT|16': u'-- Charente (16)', '202|DEPARTEMENT|17': u'-- Charente Maritime (17)', '203|DEPARTEMENT|79': u'-- Sèvres (Deux) (79)', '204|DEPARTEMENT|86': u'-- Vienne (86)', '205|REGION|93': u'Provence Alpes Côte d\'Azur', '206|DEPARTEMENT|05': u'-- Alpes (Hautes) (05)', '207|DEPARTEMENT|06': u'-- Alpes Maritimes (06)', '208|DEPARTEMENT|04': u'-- Alpes de Haute Provence (04)', '209|DEPARTEMENT|13': u'-- Bouches du Rhône (13)', '210|DEPARTEMENT|83': u'-- Var (83)', '211|DEPARTEMENT|84': u'-- Vaucluse (84)', '212|REGION|82': u'Rhône Alpes', '213|DEPARTEMENT|01': u'-- Ain (01)', '214|DEPARTEMENT|07': u'-- Ardèche (07)', '215|DEPARTEMENT|26': u'-- Drôme (26)', '216|DEPARTEMENT|38': u'-- Isère (38)', '217|DEPARTEMENT|42': u'-- Loire (42)', '218|DEPARTEMENT|69': u'-- Rhône (69)', '219|DEPARTEMENT|73': u'-- Savoie (73)', '220|DEPARTEMENT|74': u'-- Savoie (Haute) (74)', '221|REGION|03': u'Région Antilles / Guyane', '222|DEPARTEMENT|971': u'-- Guadeloupe (971)', '223|DEPARTEMENT|973': u'-- Guyane (973)', '224|DEPARTEMENT|972': u'-- Martinique (972)', '225|DEPARTEMENT|977': u'-- Saint Barthélémy (977)', '226|DEPARTEMENT|978': u'-- Saint Martin (978)', '227|REGION|98': u'Région Atlantique Nord', '228|DEPARTEMENT|975': u'-- Saint Pierre et Miquelon (975)', '229|REGION|95': u'Région Pacifique', '230|DEPARTEMENT|989': u'-- Ile de Clipperton (989)', '231|DEPARTEMENT|988': u'-- Nouvelle Calédonie (988)', '232|DEPARTEMENT|987': u'-- Polynésie française (987)', '233|DEPARTEMENT|984': u'-- Terres australes/antarctiques (984)', '234|DEPARTEMENT|986': u'-- Wallis et Futuna (986)', '235|REGION|97': u'Région Réunion / Mayotte', '236|DEPARTEMENT|976': u'-- Mayotte (976)', '237|DEPARTEMENT|974': u'-- Réunion (974)', }.iteritems())]) type_contrat_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Tous types de contrats', '11': u'CDI tout public', '14': u'CDI alternance', '13': u'CDI insertion', '12': u'CDD tout public', '16': u'CDD alternance', '15': u'CDD insertion', '10': u'CDD Senior', '3': u'Mission d\'intérim', '4': u'Contrat de travail saisonnier', '5': u'Contrat de travail intermittent', '8': u'Franchise', '7': u'Profession libérale', '9': u'Reprise d\'entreprise', '6': u'Profession commerciale', }.iteritems())]) salary_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Tout salaire annuel', 'FOURCHETTE1': u'Moins de 15000', 'FOURCHETTE2': u'Compris entre 15000 et 18000', 'FOURCHETTE3': u'Compris entre 18000 et 21000', 'FOURCHETTE4': u'Compris entre 21000 et 24000', 'FOURCHETTE5': u'Compris entre 24000 et 36000', 'FOURCHETTE6': u'Compris entre 36000 et 60000', 'FOURCHETTE7': u'Supérieur à 60000', }.iteritems())]) qualification_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Toute Qualification', '1': u'Manoeuvre', '2': u'Ouvrier spécialisé', '3': u'Ouvrier qualifié (P1,P2)', '4': u'Ouvrier qualifié (P3,P4,OHQ)', '5': u'Employé non qualifié', '6': u'Employé qualifié', '7': u'Technicien', '8': u'Agent de maîtrise', '9': u'Cadre', }.iteritems())]) limit_date_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Aucune limite', '1': u'Hier', '3': u'3 jours', '7': u'1 semaine', '14': u'2 semaines', '31': u'1 mois', '93': u'3 mois', }.iteritems())]) domain_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Tout secteur d\'activité', '88': u'Action sociale sans hebergt', '82': u'Activ.admin/soutien entreprise', '66': u'Activ. auxiliaire finance/assu', '90': u'Activ. crea/artistiq/spectacle', '77': u'Activ. de loc. et loc.-bail', '70': u'Activ. siege soc/conseil gest.', '93': u'Activ. sportive/recreat/loisir', '69': u'Activite juridique/comptable', '94': u'Activite organisations assoc.', '86': u'Activite pr la sante humaine', '53': u'Activites de poste/courrier', '64': u'Activite services financiers', '68': u'Activites immobilieres', '62': u'Activites informatiques', '78': u'Activites liees a l\'emploi', '75': u'Activites veterinaires', '84': u'Administration publiq/defense', '79': u'Agences voyage/activ. liees', '71': u'Archi/ing/control/analyse tech', '65': u'Assurance', '32': u'Autre industrie manufacturiere', '74': u'Autres activ.spe scientif/tech', '08': u'Autres industries extractives', '91': u'Biblio/ musée/ activ. culturel', '36': u'Captage/traitement/distrib.eau', '19': u'Cokefaction et raffinage', '37': u'Collecte/traitement eaux usees', '38': u'Collecte/traitnt/elimin dechet', '45': u'Commerce/reparation auto/moto', '47': u'Commerce detail sauf auto/moto', '46': u'Commerce gros sauf auto/moto', '41': u'Construction de batiments', '01': u'Cult./prod. animale, chasse', '39': u'Depollution/autre gest. dechet', '58': u'Edition', '80': u'Enquetes et securite', '85': u'Enseignement', '52': u'Entreposage/sce auxil. transp', '06': u'Extraction d\'hydrocarbures', '05': u'Extraction houille/ lignite', '07': u'Extraction minerais metalliq.', '26': u'Fab. prod. info/electro/optiq', '22': u'Fabr. prod. caoutchouc/plastiq', '30': u'Fabric. autre materiel transp.', '23': u'Fabric.autre produit non metal', '28': u'Fabric. autres machines/equip.', '27': u'Fabric. d\'equip. electriques', '31': u'Fabrication de meubles', '12': u'Fabrication produit base tabac', '25': u'Fabrication produits metalliq', '42': u'Genie civil', '55': u'Hebergement', '87': u'Hebergt médico-social/ social', '18': u'Imprimerie/reprod. enregistre.', '00': u'Indetermine', '29': u'Industrie automobile', '20': u'Industrie chimique', '14': u'Industrie de l\'habillement', '11': u'Industrie des boissons', '15': u'Industrie du cuir/la chaussure', '17': u'Industrie du papier/du carton', '21': u'Industrie pharmaceutique', '10': u'Industries alimentaires', '13': u'Industrie textile', '24': u'Metallurgie', '92': u'Orga. jeux hasard/argent', '99': u'Organisations et organismes', '03': u'Peche et aquaculture', '35': u'Prod./distrib.elec/gaz/vap/air', '59': u'Prod film cine/video/tv/musiq', '98': u'Production menage bien propre', '60': u'Programmation et diffusion', '73': u'Publicite et etudes de marche', '72': u'Rech.-dev. scientifique', '33': u'Repar./instal. machines/equip.', '95': u'Repar.pc/biens perso/domestiq', '56': u'Restauration', '97': u'Sce domestique pr particuliers', '81': u'Services bat/amenagnt paysager', '63': u'Services d\'information', '96': u'Services personnels', '09': u'Soutien industries extractives', '02': u'Sylvicult./exploit. forestiere', '61': u'Telecommunications', '51': u'Transports aeriens', '50': u'Transports par eau', '49': u'Transports terrestres', '16': u'Travail bois/fab. article bois', '43': u'Travaux constr.specialises', }.iteritems())]) CONFIG = BackendConfig(Value('metier', label='Job name', masked=False, default=''), Value('place', label=u'Place', choices=places_choices, default='100|FRANCE|01'), Value('contrat', label=u'Contract', choices=type_contrat_choices, default=''), Value('salary', label=u'Salary', choices=salary_choices, default=''), Value('qualification', label=u'Qualification', choices=qualification_choices, default=''), Value('limit_date', label=u'Date limite', choices=limit_date_choices, default=''), Value('domain', label=u'Domain', choices=domain_choices, default='')) def search_job(self, pattern=None): return self.browser.search_job(pattern=pattern) def advanced_search_job(self): return self.browser.advanced_search_job(metier=self.config['metier'].get(), place=self.config['place'].get(), contrat=self.config['contrat'].get(), salary=self.config['salary'].get(), qualification=self.config['qualification'].get(), limit_date=self.config['limit_date'].get(), domain=self.config['domain'].get()) def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000005514�13034501105�0020306�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.job import BaseJobAdvert from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Regexp, CleanText, Date, Env, BrowserURL from weboob.browser.filters.html import Link, CleanHTML class SearchPage(HTMLPage): @method class iter_job_adverts(ListElement): item_xpath = '//table[@class="definition-table ordered"]/tbody/tr' class item(ItemElement): klass = BaseJobAdvert obj_id = Regexp(Link('td[@headers="offre"]/a'), '.*detailoffre/(.*?)(?:\?|;).*') obj_contract_type = CleanText('td[@headers="contrat"]') obj_title = CleanText('td[@headers="offre"]/a') obj_society_name = CleanText('td/div/p/span[@class="company"]/span', default='') obj_place = CleanText('td[@headers="lieu"]') obj_publication_date = Date(CleanText('td[@headers="dateEmission"]')) class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_id = Env('id') obj_url = BrowserURL('advert', id=Env('id')) obj_title = CleanText('//div[@id="offre-body"]/h4[@itemprop="title"]') obj_job_name = CleanText('//div[@id="offre-body"]/h4[@itemprop="title"]') obj_description = CleanHTML('//div[@id="offre-body"]/p[@itemprop="description"]') obj_society_name = CleanText('//div[@id="offre-body"]/div[@class="vcard"]/p[@class="title"]/span', default='') obj_contract_type = CleanText('//div[@id="offre-body"]/dl/dd/span[@itemprop="employmentType"]') obj_place = CleanText('//div[@id="offre-body"]/dl/dd/ul/li[@itemprop="addressRegion"]') obj_formation = CleanText('//div[@id="offre-body"]/dl/dd/span[@itemprop="qualifications"]') obj_pay = CleanText('//div[@id="offre-body"]/dl/dd/span[@itemprop="baseSalary"]') obj_experience = CleanText('//div[@id="offre-body"]/dl/dd/span[@itemprop="experienceRequirements"]') obj_publication_date = Date(CleanText('//span[@itemprop="datePosted"]')) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/popolemploi/test.py��������������������������������������������������������������0000664�0000000�0000000�00000002576�13034501105�0020173�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PopolemploiTest(BackendTest): MODULE = 'popolemploi' def test_popolemploi_search(self): l = list(self.backend.search_job('infographiste')) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_popolemploi_advanced_search(self): l = list(self.backend.advanced_search_job()) assert len(l) advert = self.backend.get_job_advert(l[0].id, l[0]) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) ����������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015746�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001433�13034501105�0020060�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import PornhubModule __all__ = ['PornhubModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003114�13034501105�0020002�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import IndexPage, VideoPage __all__ = ['PornhubBrowser'] class PornhubBrowser(PagesBrowser): BASEURL = 'http://www.pornhub.com' index = URL(r'/$', IndexPage) search = URL(r'/video/search\?search=(?P<pattern>.+)\&page=(?P<pagenum>\d+)', IndexPage) video = URL(r'/view_video.php\?viewkey=(?P<id>.*)', VideoPage) @video.id2url def get_video(self, url, video=None): self.location(url) assert self.video.is_here() return self.page.get_video(video) def search_videos(self, pattern): self.search.go(pattern=pattern, pagenum=1) assert self.search.is_here(pattern=pattern, pagenum=1) return self.page.iter_videos() def latest_videos(self): self.index.go() assert self.index.is_here() return self.page.iter_videos() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000007624�13034501105�0020112�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME +ր���iTXtComment�����Created with GIMPd.e��IDATxWs[Iz>`V5[o:HDJ$oH+Wy- $/_;~|���?�_ѣN=5c�8_7c�3)޻Y#Ycf>w{ �{:̥9�?j7Jr?l=~odٍ!7Bo}߇Ap[�1jg] K=,Pxsomo6{*$a?B-nwnhW#g:RQ?¶=m-5%ԁ{-lۀ1ng!^Z ej>V֚DŽ~a\Ams}f*X:@^2y r eZ4j'&084N T�SnpAZntcFЉ :u %*(,zbĚ/4 p5d9pOKZ5B'J߆\N63O?'X! i ᎝ |B7K]=x9xqKs"[�u)pU))j<!$ :m8!dPA=tLeko�F8Pi0t2v/l5\Oލr*jŰ8p܂nT#@�"m kK0߮z*++J̥ D$`ԅLcCe H+7S wak\PNuPH3 sl.n!4Yjg8[d<�a!%-~Xd ;bݨBNLsy/%pfa>n0�ncXV2[�~߂!/5�L<+x(ZdL$4(\ %D\N[8ˉ i+Y -fs]@VvXú*8] ET>%y%llu^kFCzhi3M{8a ;2RB[I5 V+wRx2A"-`-!K4ZIs!�4RY&�FJFhفRd7nG0:n-8&c/ .o%t*܅soEskL$1ԎS =t^howƻb@ >O'ϓpz'Rrk^͍\_!Ǚ8#7G[Yv )[poӂn )tllTm7͘=�E :i^p+ r;{gROp9A q7sM6,k 4bI@ ^7]" fcx$ 9 ڞt  'L:ށvdKV9 I$c7H"vbxG n-714GR8*lGU}u`dH"MuC=w QKn<D>ZT۞@\F.p^Aad nTFoKժz#͡ -0eч3o+(bYx;^]ۆA[iEZ0+Zw_}h Y`| p>{f-waJD%||~C¼w@VWManKK̹N"ړwrۀ++a$7at%VKa6(Rbf'0 psCYA5|yF? Q5eE.j<BHb0 R\>l1dA`]jW3١0͠{skDQ-<it(@:>oVHF|Hm7{5Y-4ָ -莝r~:W;K]X<_.8*r]Br_,|l�h7iY+9�IOO@bVYQh&ݏhfSUNp1O/ᗧәnz<w=ӳq۱難wna~Y"nj؟ⴸv:\⿛r+)X1 ^ß« 8OReofk4r@NpU͔.-axh~ p4ҭ7yH %:WgΆtHLQu-]�r7?hOOlݶ,MaAHߏ12UTWq(.)<?;;!P@S8#rZ߂8W&cy1\kn*`oyy6ק2jgG:h oruTJ0°/u88>_Az L[ԮD~}_rQ*n1+Jg `}{�4.E3Mʾ>.|??30F/3ѰQǽ[Q"sh8.\OWpv, 9lT^6z`T(Po)ww1>}t\8`OG[R/+IYSj- d8C4D�5$.LZ.p 4;g@تKX`8*kmJG[k/V/QyKR&_7syZ麀))Y}.2*7ʿw2BlPk~b!܍‘;h WGTgW 'sXΡC<]np+m/kݶK;g)1Bc6 5Yu)ތ�ѦHi텡s<c fl`5W^qbc` WC7ncqV/veytE"JFtb`.C4y�f`sך@6t9X?ǁ "qIhUjԜt4mD5]ݯTqw@n�0Kntn7ĀUUL-<ZnTώ\p2M>�T^3u5F24@D`r߻ � :pY<x^π %-D>)E-G/YjN�Ԟ8w7@uT<Xr/(z@;Ͽ849JBBݑ4͟t![5t] ' &ʲ@ܰ OU+-/:hsu*`6bojW(mI^'mIP\!(!5sa!`iHLc/5fJs %,N2@|%Z<b1{ԃJ. Mzö$сSz,tC#FF|`' J4 FmH{nniB1<R8;Iq$bQl9Z1 zprAPzm JݽHYqPkC{>Гg(uPiZdjE8Dihl%2^)) Vg&ݢ"edPAV+7'^,~ߚN?~jJnCUބ_iwJujn4{Pki4Mmk]wS&R?|nޱo۝n(i$0 <Mj5*Ze}s$5dCM_ ^s}~f|o_9OխG ?:Iw�~�?\u����IENDB`������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/module.py����������������������������������������������������������������0000664�0000000�0000000�00000005203�13034501105�0017605�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module from .browser import PornhubBrowser __all__ = ['PornhubModule'] class PornhubModule(Module, CapVideo, CapCollection): NAME = 'pornhub' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = 'Pornhub pornographic video streaming website' LICENSE = 'AGPLv3+' BROWSER = PornhubBrowser def get_video(self, _id): video = self.browser.get_video(_id) return video def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): if not nsfw: return set() return self.browser.search_videos(pattern) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest_nsfw']) if collection.split_path == [u'latest_nsfw']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest_nsfw']: collection.title = u'Latest Pornhub videos (NSFW)' return raise CollectionNotFound(collection.split_path) OBJECTS = {BaseVideo: fill_video} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000004241�13034501105�0017420�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.html import Link, CSS, Attr from weboob.browser.filters.standard import CleanText, Duration, Regexp, Env from weboob.browser.pages import HTMLPage, pagination from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import BaseVideo class IndexPage(HTMLPage): @pagination @method class iter_videos(ListElement): item_xpath = '//li[has-class("videoblock")]' next_page = Link(u'//a[text()="Next"]') class item(ItemElement): klass = BaseVideo obj_id = CSS('a') & Link & Regexp(pattern=r'viewkey=(.+)') obj_title = Attr('.//span[has-class("title")]/a', 'title') & CleanText obj_duration = CSS('var.duration') & CleanText & Duration | NotAvailable obj_nsfw = True def obj_thumbnail(self): thumbnail = Thumbnail(Attr('.//img[has-class("js-videoThumb")]', 'data-path')(self).replace('{index}', '1')) thumbnail.url = thumbnail.id return thumbnail class VideoPage(HTMLPage): @method class get_video(ItemElement): klass = BaseVideo obj_id = Env('id') obj_title = CleanText('//title') obj_nsfw = True obj_ext = u'mp4' obj_url = CleanText('//script') & Regexp(pattern=r"(http://[^']+\.mp4[^']+)'") ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/pornhub/test.py������������������������������������������������������������������0000664�0000000�0000000�00000003320�13034501105�0017275�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.misc import limit from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class PornhubTest(BackendTest): MODULE = 'pornhub' def test_search(self): self.assertTrue(len(self.backend.search_videos('anus', nsfw=False)) == 0) l = list(limit(self.backend.search_videos('anus', nsfw=True), 100)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) r = self.backend.browser.open(v.url, stream=True) self.assertTrue(r.status_code == 200) def test_latest(self): l = list(limit(self.backend.iter_resources([BaseVideo], [u'latest_nsfw']), 100)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016500�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001526�13034501105�0020615�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"NewspaperPresseuropModule init" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewspaperPresseuropModule __all__ = ['NewspaperPresseuropModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/browser.py������������������������������������������������������������0000664�0000000�0000000�00000003377�13034501105�0020547�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"browser for presseurop website" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import date, datetime, time from .pages import PresseuropPage from weboob.browser.browsers import AbstractBrowser from weboob.browser.url import URL class NewspaperPresseuropBrowser(AbstractBrowser): "NewspaperPresseuropBrowser class" PARENT = 'genericnewspaper' BASEURL = 'http://www.voxeurop.eu' presseurop_page = URL("/.*", PresseuropPage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) def get_daily_date(self, _id): self.location(_id) return self.page.get_daily_date() def get_daily_infos(self, _id): url = "http://www.voxeurop.eu/fr/today/" + _id self.location(url) title = self.page.get_title() article_date = date(*[int(x) for x in _id.split('-')]) article_time = time(0, 0, 0) article_datetime = datetime.combine(article_date, article_time) return url, title, article_datetime �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000000553�13034501105�0020636�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���XGl���sRGB����PLTEnE�]y$~4��� pHYs�� �� ����tIME(S���IDATHŕa 0e7ExWjB[%ݠ{+>-(%XO�0I5'U1Z!�+@N9 aYUf�f\'�;�H : SzI- �|͊�\s)%7LŲBc l%|%ۀCrUIMH/XL }],Ea^VWo!l����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/module.py�������������������������������������������������������������0000664�0000000�0000000�00000006023�13034501105�0020340�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://www.presseurop.eu" from weboob.capabilities.messages import CapMessages, Thread from weboob.tools.backend import AbstractModule from weboob.tools.backend import BackendConfig from weboob.tools.value import Value from .browser import NewspaperPresseuropBrowser from .tools import rssid, url2id from weboob.tools.newsfeed import Newsfeed class NewspaperPresseuropModule(AbstractModule, CapMessages): MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'presseurop' DESCRIPTION = u'Presseurop website' BROWSER = NewspaperPresseuropBrowser RSSID = staticmethod(rssid) URL2ID = staticmethod(url2id) RSSSIZE = 300 PARENT = 'genericnewspaper' CONFIG = BackendConfig(Value('lang', label='Lang of articles', choices={'fr': 'fr', 'de': 'de', 'en': 'en', 'cs': 'cs', 'es': 'es', 'it': 'it', 'nl': 'nl', 'pl': 'pl', 'pt': 'pt', 'ro': 'ro'}, default='fr')) def __init__(self, *args, **kwargs): super(self.__class__, self).__init__(*args, **kwargs) self.RSS_FEED = 'http://www.voxeurop.eu/%s/rss.xml' % self.config['lang'].get() def iter_threads(self): daily = [] for article in Newsfeed(self.RSS_FEED, self.RSSID).iter_entries(): if "/news-brief/" in article.link: day = self.browser.get_daily_date(article.link) if day and (day not in daily): localid = url2id(article.link) daily.append(day) id, title, date = self.browser.get_daily_infos(day) id = id + "#" + localid thread = Thread(id) thread.title = title thread.date = date yield(thread) elif day is None: thread = Thread(article.link) thread.title = article.title thread.date = article.datetime yield(thread) else: thread = Thread(article.link) thread.title = article.title thread.date = article.datetime yield(thread) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000005134�13034501105�0020154�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"ArticlePage object for presseurope" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.html import CSS, CleanHTML class PresseuropPage(AbstractPage): "PresseuropPage object for presseurop" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_loaded(self): self.main_div = self.doc.getroot() self.element_title_selector = "title" self.element_author_selector = "a[rel=author], div.profilecartoontext>p>a" self.element_body_selector = "div.block, div.panel, div.bodytext" def get_body(self): element_body = self.get_element_body() self.try_drop_tree(element_body, "li.button-social") self.try_drop_tree(element_body, "div.sharecount") self.try_drop_tree(element_body, "p.ruledtop") self.try_drop_tree(element_body, "p.ctafeedback") self.try_drop_tree(element_body, "aside.articlerelated") self.try_drop_tree(element_body, "div.sharecount") self.try_drop_tree(element_body, "iframe") self.clean_relativ_urls(element_body, "http://presseurop.eu") return CleanHTML('.')(element_body) def get_title(self): title = super(self.__class__, self).get_title() title = title.split('|')[0] return title def get_author(self): author = super(self.__class__, self).get_author() try: source = self.doc.getroot().xpath( "//span[@class='sourceinfo']/a")[0] source = source.text author = author + " | " + source return author except: return author def get_daily_date(self): plink = self.doc.getroot().xpath("//p[@class='w200']") if len(plink) > 0: link = plink[0].xpath('a')[0] date = link.attrib['href'].split('/')[3] return date return None ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/presseurop/test.py���������������������������������������������������������������0000664�0000000�0000000�00000001665�13034501105�0020041�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class PresseuropTest(BackendTest): MODULE = 'presseurop' def test_new_messages(self): for message in self.backend.iter_unread_messages(): pass ���������������������������������������������������������������������������weboob-1.2/modules/presseurop/tools.py��������������������������������������������������������������0000664�0000000�0000000�00000002025�13034501105�0020211�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"tools for presseurop backend" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re def url2id(url): "return an id from an url" if "/today/" in url: return url.split("#")[1] else: regexp = re.compile(".*/([0-9]+)-.*") id = regexp.match(url).group(1) return id def rssid(entry): return url2id(entry.link) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017340�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/__init__.py�������������������������������������������������������0000664�0000000�0000000�00000000115�13034501105�0021446�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import PrixCarburantsModule __all__ = ['PrixCarburantsModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/browser.py��������������������������������������������������������0000664�0000000�0000000�00000004150�13034501105�0021375�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.capabilities.base import UserError from .pages import IndexPage, ComparisonResultsPage, ShopInfoPage __all__ = ['PrixCarburantsBrowser'] class PrixCarburantsBrowser(PagesBrowser): BASEURL = 'https://www.prix-carburants.gouv.fr' TOKEN = None result_page = URL('/recherche/', ComparisonResultsPage) shop_page = URL('/itineraire/infos/(?P<_id>\d+)', ShopInfoPage) index_page = URL('/', IndexPage) def iter_products(self): return self.index_page.go().iter_products() def get_token(self): self.TOKEN = self.index_page.stay_or_go().get_token() def iter_prices(self, zipcode, product): if self.TOKEN is None: self.get_token() data = { '_recherche_recherchertype[localisation]': '%s' % zipcode, '_recherche_recherchertype[choix_carbu]': '%s' % product.id, '_recherche_recherchertype[_token]': '%s' % self.TOKEN, } self.index_page.go(data=data) if not self.result_page.is_here(): raise UserError('Bad zip or product') if not product.name: product.name = self.page.get_product_name() return self.page.iter_results(product=product) def get_shop_info(self, id): self.session.headers.update({"X-Requested-With": "XMLHttpRequest"}) return self.shop_page.go(_id=id).get_info() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/favicon.png�������������������������������������������������������0000664�0000000�0000000�00000002563�13034501105�0021501�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME   k���IDATxoTUwfwswǶZ-#^MkSe.! "‚J0 BXfea1bq"d\5wuaeUwgs 99|Ç>|ÇBp�R}%A8eL. �tZR*F@+[F|CO 8~ 65X\c PLn�;qw遀1݀.] �p8q*)v+p@]{~Z_X"5APs @3pp|_kԶws*_t( $5FmK{Yb!�d+!gܯկC Q?0r-@L>![ܫЌ�V9\|ږt�̐ tL)uxB0Oޟ|PV2j[ecT)v�:3 $Ӆz`x<%r8pL)u~d wK:ྋJ88"B!f)c qNO !|\RSֿ6?l']zYk�~ڳ ƥq.=ڨ `DrpI{8N>t6[W.lk)xJ8ЮJp#` Lce.'3E9#r{fw?=A1:ch,uQz@#S@F<WZT/̰m)X ۖOfNy6rķ<h,dz!|PߟWz\~'ׯt}!dM M,y4l<b[4BU@ZHNk7Z#˳D Y7D 2 hJ!gn��K;}1K'@H<ZO?EcCEimR*gႌ^¬-x2)zhJH=P̐ u%6z�S99In�qJ?f4xSWr]{m0 m*2dfgRka3K\jm[怬~3m\Of=�WY3,-Xi`I{!X*ēͶe&I>@i#ξX� Jm.=�G䂥8x2c[`Q*�`#H㿻 ;h����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/module.py���������������������������������������������������������0000664�0000000�0000000�00000004370�13034501105�0021203�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value from weboob.capabilities.pricecomparison import CapPriceComparison, Price, Product, PriceNotFound from weboob.capabilities.base import find_object from .browser import PrixCarburantsBrowser __all__ = ['PrixCarburantsModule'] class PrixCarburantsModule(Module, CapPriceComparison): NAME = 'prixcarburants' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'French governement website to compare fuel prices' LICENSE = 'AGPLv3+' BROWSER = PrixCarburantsBrowser CONFIG = BackendConfig(Value('zipcode', label='Zipcode', regexp='\d+')) def search_products(self, pattern=None): for product in self.browser.iter_products(): if pattern is None or pattern.lower() in product.name.lower(): yield product def iter_prices(self, products): product = [product for product in products if product.backend == self.name] if product: return self.browser.iter_prices(self.config['zipcode'].get(), product[0]) def get_price(self, id, price=None): product = Product(id.split('.')[0]) product.backend = self.name price = find_object(self.iter_prices([product]), id=id, error=PriceNotFound) price.shop.info = self.browser.get_shop_info(price.id.split('.', 2)[-1]) return price def fill_price(self, price, fields): return self.get_price(price.id, price) OBJECTS = {Price: fill_price, } ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/pages.py����������������������������������������������������������0000664�0000000�0000000�00000005467�13034501105�0021025�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.filters.standard import CleanText, Env, Field, CleanDecimal, Date, Format from weboob.browser.elements import ItemElement, ListElement, method from weboob.capabilities.pricecomparison import Product, Shop, Price class IndexPage(HTMLPage): def get_token(self): return CleanText('//input[@id="recherche_recherchertype__token"]/@value')(self.doc) @method class iter_products(ListElement): item_xpath = '//div[@id="choix_carbu"]/ul/li' class item(ItemElement): klass = Product obj_id = CleanText('./input/@value') obj_name = CleanText('./label') class ComparisonResultsPage(HTMLPage): @method class iter_results(ListElement): item_xpath = '//table[@id="tab_resultat"]/tr' class item(ItemElement): klass = Price def condition(self): return CleanText('./@id', default=False)(self) obj_product = Env('product') def obj_id(self): product = Field('product')(self) _id = CleanText('./@id')(self) return u"%s.%s" % (product.id, _id) def obj_shop(self): _id = Field('id')(self) shop = Shop(_id) shop.name = CleanText('(./td)[4]')(self) shop.location = CleanText('(./td)[3]')(self) return shop obj_date = Date(CleanText('(./td)[7]'), dayfirst=True) obj_currency = u'€' obj_cost = CleanDecimal('(./td)[6]') def get_product_name(self): return CleanText('(//table[@id="tab_resultat"]/tr/th)[6]', default='')(self.doc) class ShopInfoPage(HTMLPage): def get_info(self): return Format('%s / %s / %s: %s', CleanText('(//div[@class="infos"]/p)[1]/text()'), CleanText('(//div[@class="infos"]/p)[2]'), CleanText('(//div[@class="infos"]/p)[3]'), CleanText('(//div[@class="infos"]/p)[3]/img/@alt'))(self.doc) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/prixcarburants/test.py�����������������������������������������������������������0000664�0000000�0000000�00000002533�13034501105�0020674�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.tools.value import Value import itertools class PrixCarburantsTest(BackendTest): MODULE = 'prixcarburants' def setUp(self): if not self.is_backend_configured(): self.backend.config['Zipcode'] = Value(value='59000') def test_prixcarburants(self): products = list(self.backend.search_products('gpl')) self.assertTrue(len(products) == 1) product = products[0] product.backend = self.backend.name prices = list(itertools.islice(self.backend.iter_prices([product]), 0, 20)) self.backend.fillobj(prices[0]) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015255�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001456�13034501105�0017374�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import QuviModule, QuviVideo __all__ = ['QuviModule', 'QuviVideo'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000005547�13034501105�0017423�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@������ pHYs�� �� ����tIME |���tEXtComment�Created with GIMPW��PLTE�%z%z&{'{(|)|*}*}+~ ,~ - . / 001234566789:;<<=>?? @!A"B#B$C%D&E'F)H*H+I,J,K-K.L/M0N1N2O3P4Q5R6S8T9U:V;W<X=Y?Z@[A\B]C^D_E_F`GaHbIcJdKeLeMfNgOhQjRkSkVnWoXpYqZq[r\s]t^u_v`wawcydzd{e{f|g}h}i~jkklmnopqrsuvwxyz{|}~ŒÍÎďĐőŒƓƔǕǖȗȘəɚʛʝ˞̟̠͢͡ΣϤϥЦЧѨѩҪҫӬӭԮԯհձֲֳ״׵ضطٸٹںۻۼܾܽݿ"���bKGDLdW��IDATXÍ{\TUk�<TQ1 fVdd[ZFQn.A$d`x <ag뿽>fa};@4( KHѷ N%-,,*?BfyK�ޱ1P"AJ@N3L~!ðIx;kխꐤ@bS\;+1-%ZRE0{ďS:Xڄ\0 `/(3Q֓�E}h% otR�5+}k${yu�Oٜ(jm<&P*0'�XB13'5{=�Ci,v.-�_;e�ILl`M*6l=ہ ):V6*�FX=�x q]7l/Z^̱lv15ɾ.-߾B&00awq8OC0Bjkv � >AzׂC`%kyE"յR惀Flx@r!CɆ+ I'E8uDz 4{eKO@! aH�u30b~sB0a' gSm9h#�NY:E+"{"WkWjob�Mz:Ǭ{W/H\kaFGf-L,M .HGS?v=Nn(SC}cv->\v.(>/M Cai|kik^~˜;sa'.uV/Fi1`3fa~uw8OeUrwnj0puTfJ|bJ۰&M|Yo{9أCe%7fB!ɵ3J 8[.fy!vwɔȈ w^*; �W_˻aK"B` 멧\Tz@AL ^z]oa2UxYX m|N!U]dͼ3X)JN |l"lA|y. n@WZ~unGq "q'(J^q̧mEͣӎQc,:.F봀 4.Ӕ>! 0Ɍ Em,YRՕ}ں\p{-9@Z P0S{jw#01sK l##oɌQ!H&!z:=*CNю:D$D'aUBaMpITCĻ _g !<&RظL{\-CcR PEhj}0;:4ԥenؿWn6uEeY]D2 $ SxmRBjSyr³-F3ȟv =jŭe{fWt>"D8MWs{D[) C&�qcd_grR!-ky�n~$bl9˃VgS$[đZuf )b)W+ 8`xN&\IԪfXyٮdgv] ;qcl ʽSUe8u4,kO̠8o  ˚݇ \�頩+܍^w˙C3jy�^T%^[#DC3D(npAvSűJOYia$[>7 hNkC#M>ނW/um_ MiΣ|OXH¼"<LX%<0mK nRsrLǠ4á X?#)HGǰe3c&Uj*/+%.zfDDD. t波a'^ޥ9×U`U68o/Q~9 ����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000010053�13034501105�0017113�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. # sample usage: youtube.XXXXXX@quvi # or also: https://www.youtube.com/watch?v=XXXXXX@quvi # shortened URLs are also supported # this backend requires the quvi 0.4 C library to be installed import datetime from weboob.capabilities.base import UserError, StringField from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.image import Thumbnail from weboob.tools.backend import Module from weboob.tools.misc import to_unicode from .quvi import LibQuvi, QuviError __all__ = ['QuviModule', 'QuviVideo'] class QuviModule(Module, CapVideo): NAME = 'quvi' DESCRIPTION = u'Multi-website video helper with quvi. Handles Youtube, BBC, and a lot more' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = None def get_video(self, _id): video = QuviVideo(_id) parser = LibQuvi() if not parser.load(): raise UserError('Make sure libquvi 0.4 is installed') try: info = parser.get_info(video.page_url) except QuviError as qerror: raise UserError(qerror.message) video.url = to_unicode(info.get('url')) if not video.url: raise NotImplementedError() video.ext = to_unicode(info.get('suffix')) video.title = to_unicode(info.get('title')) video.page = to_unicode(info.get('page')) duration = int(info.get('duration', 0)) if duration: video.duration = datetime.timedelta(milliseconds=duration) if info.get('thumbnail'): video.thumbnail = Thumbnail(info.get('thumbnail')) video.thumbnail.url = video.thumbnail.id return video class QuviVideo(BaseVideo): BACKENDS = { 'youtube': 'https://www.youtube.com/watch?v=%s', 'vimeo': 'http://vimeo.com/%s', 'dailymotion': 'http://www.dailymotion.com/video/%s', 'metacafe': 'http://www.metacafe.com/watch/%s/', 'arte': 'http://videos.arte.tv/fr/videos/plop--%s.html', 'videa': 'http://videa.hu/videok/%s/', 'wimp': 'http://www.wimp.com/%s/', 'funnyordie': 'http://www.funnyordie.com/videos/%s/', 'tapuz': 'http://flix.tapuz.co.il/v/watch-%s-.html', 'liveleak': 'http://www.liveleak.com/view?i=%s', # nsfw 'xhamster': 'https://xhamster.com/movies/%s/plop.html', 'xvideos': 'http://www.xvideos.com/video%s/', 'redtube': 'http://www.redtube.com/%s', 'xnxx': 'http://video.xnxx.com/video%s/', # more websites are supported, but <service>.<id> isn't always enough # however, URLs are supported, like this: # https://www.youtube.com/watch?v=BaW_jenozKc@quvi } page = StringField('Page URL of the video') @classmethod def id2url(cls, _id): if _id.startswith('http'): return _id if '.' not in _id: raise UserError('Please give an ID in form WEBSITE.ID (for example youtube.BaW_jenozKc). Supported websites are: %s' % ', '.join(cls.BACKENDS.keys())) sub_backend, sub_id = _id.split('.', 1) try: return cls.BACKENDS[sub_backend] % sub_id except KeyError: raise NotImplementedError() @property def page_url(self): if self.page: return self.page else: return self.id2url(self.id) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/quvi.py���������������������������������������������������������������������0000664�0000000�0000000�00000007352�13034501105�0016622�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from ctypes import cdll, c_char_p, c_double, c_void_p, byref from ctypes.util import find_library class QuviError(Exception): pass class LibQuvi04(object): QUVI_VERSION = 0 QUVIOPT_FORMAT = 0 QUVIOPT_CATEGORY = 4 QUVI_OK = 0 QUVI_LAST = 5 QUVIPROP_PAGEURL = 0x100002 QUVIPROP_PAGETITLE = 0x100003 QUVIPROP_MEDIAID = 0x100004 QUVIPROP_MEDIAURL = 0x100005 QUVIPROP_FILESUFFIX = 0x100008 #~ QUVIPROP_FORMAT = 0x10000A QUVIPROP_MEDIATHUMBNAILURL = 0x10000C QUVIPROP_MEDIACONTENTLENGTH = 0x300006 QUVIPROP_MEDIADURATION = 0x30000D QUVIPROTO_HTTP = 1 QUVIPROTO_RTMP = 8 def __init__(self, lib=None): self.lib = lib self.qh = c_void_p() self.qmh = c_void_p() def load(self): path = find_library('quvi') if not path: return False self.lib = cdll.LoadLibrary(path) if self.lib is None: return False self.lib.quvi_version.restype = c_char_p version_str = self.lib.quvi_version(self.QUVI_VERSION) if version_str.startswith('v0.4'): return True else: return False def _cleanup(self): if self.qmh: self.lib.quvi_parse_close(byref(self.qmh)) self.qmh = c_void_p() if self.qh: self.lib.quvi_close(byref(self.qh)) self.qh = c_void_p() def get_info(self, url): try: return self._get_info(url) finally: self._cleanup() def _get_info(self, url): status = self.lib.quvi_init(byref(self.qh)) self._assert_ok(status) status = self.lib.quvi_setopt(self.qh, self.QUVIOPT_FORMAT, 'best') self._assert_ok(status) status = self.lib.quvi_parse(self.qh, c_char_p(url), byref(self.qmh)) self._assert_ok(status) info = {} info['url'] = self._get_str(self.QUVIPROP_MEDIAURL) info['title'] = self._get_str(self.QUVIPROP_PAGETITLE) info['suffix'] = self._get_str(self.QUVIPROP_FILESUFFIX) info['page'] = self._get_str(self.QUVIPROP_PAGEURL) # uncut! info['media_id'] = self._get_str(self.QUVIPROP_MEDIAID) info['thumbnail'] = self._get_str(self.QUVIPROP_MEDIATHUMBNAILURL) info['duration'] = self._get_double(self.QUVIPROP_MEDIADURATION) info['size'] = self._get_double(self.QUVIPROP_MEDIACONTENTLENGTH) return info def _assert_ok(self, status): if status != self.QUVI_OK: self.lib.quvi_strerror.restype = c_char_p c_msg = self.lib.quvi_strerror(self.qh, status) raise QuviError(c_msg) def _get_str(self, prop): c_value = c_char_p() status = self.lib.quvi_getprop(self.qmh, prop, byref(c_value)) self._assert_ok(status) return c_value.value def _get_double(self, prop): c_value = c_double() status = self.lib.quvi_getprop(self.qmh, prop, byref(c_value)) self._assert_ok(status) return c_value.value LibQuvi = LibQuvi04 ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/quvi/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000003137�13034501105�0016612�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class QuviTest(BackendTest): MODULE = 'quvi' def test_get_id(self): v = self.backend.get_video('youtube.BaW_jenozKc') assert len(v.url) assert len(v.title) assert (v.page_url == 'https://www.youtube.com/watch?v=BaW_jenozKc') def test_get_url(self): v = self.backend.get_video('https://www.youtube.com/watch?v=BaW_jenozKc') assert len(v.url) assert len(v.title) # did we retrieve more? assert len(v.ext) assert v.duration assert v.thumbnail assert v.page_url == 'https://www.youtube.com/watch?v=BaW_jenozKc' def test_get_shortened(self): v = self.backend.get_video('http://youtu.be/BaW_jenozKc') assert len(v.url) assert len(v.title) assert v.page_url.startswith('http://www.youtube.com/watch?v=BaW_jenozKc') ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016546�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001477�13034501105�0020670�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Johann Broudin, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import RadioFranceModule __all__ = ['RadioFranceModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000011677�13034501105�0020617�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Johann Broudin, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import RadioPage, JsonPage, PodcastPage __all__ = ['RadioFranceBrowser'] class RadioFranceBrowser(PagesBrowser): json_page = URL('sites/default/files/(?P<json_url>.*).json', 'player-json/reecoute/(?P<json_url_fip>.*)', 'station/(?P<fbplayer>.*)', 'programmes\?xmlHttpRequest=1', 'autocomplete/emissions.json', JsonPage) podcast_page = URL('podcast09/rss_(?P<podcast_id>.*)\.xml', PodcastPage) radio_page = URL('(?P<page>.*)', RadioPage) def get_radio_url(self, radio, player): self.fill_base_url(radio) if radio == 'francebleu': return self.json_page.go(fbplayer=player).get_fburl() return self.radio_page.go(page=player).get_url() def fill_base_url(self, radio): if radio in ['franceinter', 'francebleu', 'franceculture']: self.BASEURL = 'https://www.%s.fr/' % radio else: self.BASEURL = 'http://www.%s.fr/' % radio def get_current(self, radio, url): self.fill_base_url(radio) if radio == 'francebleu': return self.radio_page.go(page=url).get_current() if radio in ['franceculture', 'franceinter']: return self.json_page.go().get_culture_inter_current() if radio == 'francetvinfo': return u'', u'' return self.json_page.go(json_url=url).get_current() def get_selection(self, radio_url, json_url, radio_id): self.BASEURL = 'http://www.%s.fr/' % radio_url if radio_id == 'fipradio': return self.json_page.go(json_url_fip=json_url).get_selection(radio_id=radio_id) elif radio_id == 'franceculture': self.fill_base_url(radio_id) return self.radio_page.go(page='').get_france_culture_selection(radio_id=radio_id) elif radio_id == 'francetvinfo': self.fill_base_url(radio_id) selection_list = self.radio_page.go(page=json_url).get_francetvinfo_selection_list() sel = [] for item in selection_list: sel.append(self.radio_page.go(page=item).get_francetvinfo_selection()) return sel return self.json_page.go(json_url=json_url).get_selection(radio_id=radio_id) def get_audio(self, _id, radio_url, json_url, radio_id): for item in self.get_selection(radio_url, json_url, radio_id): if item.id == _id: return item return [] def search_audio(self, pattern, radio_url, json_url, radio_id): for item in self.get_selection(radio_url, json_url, radio_id): if pattern.upper() in item.title.upper(): yield item def get_podcast_emissions(self, radio_url, podcast_url, split_path): self.fill_base_url(radio_url) if split_path[0] == 'franceinter': return self.radio_page.go(page=podcast_url).get_france_inter_podcast_emissions(split_path=split_path) elif split_path[0] == 'franceculture': self.location('%s%s' % (self.BASEURL, podcast_url)) return self.page.get_france_culture_podcast_emissions(split_path=split_path) elif split_path[0] == 'francetvinfo': return self.radio_page.go(page=podcast_url).get_france_info_podcast_emissions(split_path=split_path) elif split_path[0] == 'francemusique': return self.radio_page.go(page=podcast_url).get_france_musique_podcast_emissions(split_path=split_path) elif split_path[0] == 'mouv': return self.radio_page.go(page=podcast_url).get_mouv_podcast_emissions(split_path=split_path) def get_podcasts(self, podcast_id): self.BASEURL = 'http://radiofrance-podcast.net/' return self.podcast_page.go(podcast_id=podcast_id).iter_podcasts() def get_france_culture_podcasts_url(self, url): self.BASEURL = 'https://www.franceculture.fr/' return self.radio_page.go(page='emissions/%s' % url).get_france_culture_podcasts_url() def get_francetvinfo_podcasts_url(self, url): self.BASEURL = 'http://www.francetvinfo.fr/' return self.radio_page.go(page='replay-radio/%s' % url).get_francetvinfo_podcasts_url() �����������������������������������������������������������������weboob-1.2/modules/radiofrance/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000003657�13034501105�0020714�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���% ���sRGB����bKGD������ pHYs�� �� ����tIMEr ���iTXtComment�����Created with GIMPd.e��IDATh{PSW7$! 0 Ĕ,C"EWS+>ЭeL�LgR]V* 4J!� }R`tw} BQD"7`\�`X�`X�`XGغc&3�<ތ8qg5ﮍ}.Qşs�R7gD$Pn{ۿwmU!%⟚+b˺w4N薙T=Hx@~x3mvڽ%)sac˱L9^)jkʿ5�+$� )}NܸNOT2-2HYl|iv)2 V)z˭}f+NO،!}&rZ�(x6J2Ko.(xCkxB_VȉDUɌU4~w�lOrFDɖx ՟o=�l0;~+� \Dw Wh3[K-w;q$l8R1 Y C}��Y3BZ:V$?SbBVNB<J/hs✎{3v� ь9V[s){sR�PPvIwtvAeQ^/#�0"iy22ЉN�, rl\�ld(fA"rc~Í ;-p.UoI^<�|K}8�8QP�JWV8}SV<XW=&LҘK-wj{(!Txvpf2cGj4|Z{oضs{,6|pvW>f'/lj #Zu]rçM|BN�,Ӯa�xBP((j49BN4A@ HV^*M&S'p@KVjwww� I҅ EQ7%%E vww*$ɢR)ZqݎgϞX h4[Sf!o\*1ԎonbVl\Ԝ0A4N|8isr¢d7O_qyȂ9 !K(qAg쳂E~~6Xk�7.0u<*IUH}�A`iI,ЫҧO k׮-[p8m۶uttz^izB_?ygژuʜ KIV3WuCyrg�`MjXV\oU4\ņGJ?ŕM]]]###f`0LO3+m! ?2hLf'+�Umz 7T�ia�yg˺{6+F4ig�d&8 !%I$I.K'k=HQŏAB+~ (ǿe&g6N7.gJa4d3kC[ȻY3%O7EV5bֲª뚁듛4�P>nӊ?<7nRD+v�䦅<<.ύ{M*]y({&(=۞#Moߔ}Scב\_VO|T(hR}E͗rp`v\]Z9#";yBMuӭcןmٰL7\sCB}Q:q|ttjҭԣNOxK$b&`51  /Ѷ����IENDB`���������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/module.py������������������������������������������������������������0000664�0000000�0000000�00000042200�13034501105�0020403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# * -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Johann Broudin, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.base import NotLoaded from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audio import CapAudio, BaseAudio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.tools.backend import Module from .browser import RadioFranceBrowser import re import time from datetime import datetime __all__ = ['RadioFranceModule'] class RadioFranceModule(Module, CapRadio, CapCollection, CapAudio): NAME = 'radiofrance' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' DESCRIPTION = u'Radios of Radio France: Inter, Info, Bleu, Culture, Musique, FIP, Le Mouv\'' LICENSE = 'AGPLv3+' BROWSER = RadioFranceBrowser _RADIOS = { 'franceinter': {u'title': u'France Inter', u'player': u'', u'live': u'programmes?xmlHttpRequest=1', u'podcast': u'podcasts'}, 'franceculture': {u'title': u'France Culture', u'player': u'', u'live': u'programmes?xmlHttpRequest=1', u'podcast': u'programmes?xmlHttpRequest=1', u'selection': u''}, 'francetvinfo': {u'title': u'France Info', u'player': u'en-direct/radio.html', u'live': u'', u'podcast': u'replay-radio', u'selection': u'en-direct/radio.html'}, 'fbidf': {u'title': u'France Bleu Île-de-France (Paris)', u'player': u'107-1', u'live': u'grid/107-1'}, 'fipradio': {u'title': u'FIP', u'player': u'player', u'live': 'import_si/si_titre_antenne/FIP_player_current', u'selection': u'%s' % int(time.mktime(datetime.utcnow().replace(hour=12, minute=0, second=0).timetuple()))}, 'francemusique': {u'title': u'France Musique', u'player': u'player', u'live': u'lecteur_commun_json/reecoute-%s' % int(time.mktime(datetime.utcnow().replace(hour=14, minute=0, second=0).timetuple())), u'podcast': u'emissions', u'selection': u'lecteur_commun_json/reecoute-%s' % int(time.mktime(datetime.utcnow().replace(hour=14, minute=0, second=0).timetuple()))}, 'mouv': {u'title': u'Le Mouv\'', u'player': u'player', u'live': u'lecteur_commun_json/timeline', u'podcast': u'podcasts', u'selection': u'lecteur_commun_json/reecoute-%s' % int(time.mktime(datetime.utcnow().replace(hour=14, minute=0, second=0).timetuple()))}, 'fbalsace': {u'title': u'France Bleu Alsace (Strasbourg)', u'player': u'alsace', u'live': u'grid/alsace'}, 'fbarmorique': {u'title': u'France Bleu Armorique (Rennes)', u'player': u'armorique', u'live': u'grid/armorique'}, 'fbauxerre': {u'title': u'France Bleu Auxerre', u'player': u'auxerre', u'live': u'grid/auxerre'}, 'fbazur': {u'title': u'France Bleu Azur (Nice)', u'player': u'azur', u'live': u'grid/azur'}, 'fbbearn': {u'title': u'France Bleu Bearn (Pau)', u'player': u'bearn', u'live': u'grid/bearn'}, 'fbbelfort': {u'title': u'France Bleu Belfort', u'player': u'belfort-montbeliard', u'live': u'grid/belfort-montbeliard'}, 'fbberry': {u'title': u'France Bleu Berry (Châteauroux)', u'player': u'berry', u'live': u'grid/berry'}, 'fbbesancon': {u'title': u'France Bleu Besancon', u'player': u'besancon', u'live': u'grid/besancon'}, 'fbbourgogne': {u'title': u'France Bleu Bourgogne (Dijon)', u'player': u'bourgogne', u'live': u'grid/bourgogne'}, 'fbbreihzizel': {u'title': u'France Bleu Breizh Izel (Quimper)', u'player': u'breizh-izel', u'live': u'grid/breizh-izel'}, 'fbchampagne': {u'title': u'France Bleu Champagne (Reims)', u'player': u'champagne-ardenne', u'live': u'grid/champagne-ardenne'}, 'fbcotentin': {u'title': u'France Bleu Cotentin (Cherbourg)', u'player': u'cotentin', u'live': u'grid/cotentin'}, 'fbcreuse': {u'title': u'France Bleu Creuse (Gueret)', u'player': u'creuse', u'live': u'grid/creuse'}, 'fbdromeardeche': {u'title': u'France Bleu Drome Ardeche (Valence)', u'player': u'drome-ardeche', u'live': u'grid/drome-ardeche'}, 'fbelsass': {u'title': u'France Bleu Elsass', u'player': 'elsass', u'live': u'grid/elsass'}, 'fbgardlozere': {u'title': u'France Bleu Gard Lozère (Nîmes)', u'player': u'gard-lozere', u'live': u'grid/gard-lozere'}, 'fbgascogne': {u'title': u'France Bleu Gascogne (Mont-de-Marsan)', u'player': u'gascogne', u'live': u'grid/gascogne'}, 'fbgironde': {u'title': u'France Bleu Gironde (Bordeaux)', u'player': u'gironde', u'live': u'grid/gironde'}, 'fbherault': {u'title': u'France Bleu Hérault (Montpellier)', u'player': u'herault', u'live': u'grid/herault'}, 'fbisere': {u'title': u'France Bleu Isère (Grenoble)', u'player': u'isere', u'live': u'grid/isere'}, 'fblarochelle': {u'title': u'France Bleu La Rochelle', u'player': u'la-rochelle', u'live': u'grid/la-rochelle'}, 'fblimousin': {u'title': u'France Bleu Limousin (Limoges)', u'player': u'limousin', u'live': u'grid/limousin'}, 'fbloireocean': {u'title': u'France Bleu Loire Océan (Nantes)', u'player': u'loire-ocean', u'live': u'grid/loire-ocean'}, 'fblorrainenord': {u'title': u'France Bleu Lorraine Nord (Metz)', u'player': u'lorraine-nord', u'live': u'grid/lorraine-nord'}, 'fbmaine': {u'title': u'France Bleu Maine', u'player': 'maine', u'live': u'grid/maine'}, 'fbmayenne': {u'title': u'France Bleu Mayenne (Laval)', u'player': u'mayenne', u'live': u'grid/mayenne'}, 'fbnord': {u'title': u'France Bleu Nord (Lille)', u'player': u'nord', u'live': u'grid/nord'}, 'fbcaen': {u'title': u'France Bleu Normandie (Calvados - Orne)', u'player': u'normandie-caen', u'live': u'grid/normandie-caen'}, 'fbrouen': {u'title': u'France Bleu Normandie (Seine-Maritime - Eure)', u'player': u'normandie-rouen', u'live': u'grid/normandie-rouen'}, 'fborleans': {u'title': u'France Bleu Orléans', u'player': u'orleans', u'live': u'grid/orleans'}, 'fbpaysbasque': {u'title': u'France Bleu Pays Basque (Bayonne)', u'player': u'pays-basque', u'live': u'grid/pays-basque'}, 'fbpaysdauvergne': {u'title': u'France Bleu Pays d\'Auvergne (Clermont-Ferrand)', u'player': u'pays-d-auvergne', u'live': u'grid/pays-d-auvergne'}, 'fbpaysdesavoie': {u'title': u'France Bleu Pays de Savoie (Chambery)', u'player': u'pays-de-savoie', u'live': u'grid/pays-de-savoie'}, 'fbperigord': {u'title': u'France Bleu Périgord (Périgueux)', u'player': u'perigord', u'live': u'grid/perigord'}, 'fbpicardie': {u'title': u'France Bleu Picardie (Amiens)', u'player': u'picardie', u'live': u'grid/picardie'}, 'fbpoitou': {u'title': u'France Bleu Poitou (Poitiers)', u'player': u'poitou', u'live': u'grid/poitou'}, 'fbprovence': {u'title': u'France Bleu Provence (Aix-en-Provence)', u'player': u'provence', u'live': u'grid/provence'}, 'fbrcfm': {u'title': u'France Bleu RCFM', u'player': u'rcfm', u'live': u'grid/rcfm'}, 'fbsaintetienneloire': {u'title': u'France Bleu Saint-Etienne Loire', u'player': u'saint-etienne-loire', u'live': u'grid/saint-etienne-loire'}, 'fbroussillon': {u'title': u'France Bleu Roussillon', u'player': u'roussillon', u'live': u'grid/roussillon'}, 'fbsudlorraine': {u'title': u'France Bleu Sud Lorraine (Nancy)', u'player': u'sud-lorraine', u'live': u'grid/sud-lorraine'}, 'fbtoulouse': {u'title': u'France Bleu Toulouse', u'player': u'toulouse', u'live': u'grid/toulouse'}, 'fbtouraine': {u'title': u'France Bleu Touraine (Tours)', u'player': u'touraine', u'live': u'grid/touraine'}, 'fbvaucluse': {u'title': u'France Bleu Vaucluse (Avignon)', u'player': u'vaucluse', u'live': u'grid/vaucluse'}, } def iter_resources(self, objs, split_path): if len(split_path) == 0: for _id, item in sorted(self._RADIOS.iteritems()): if not _id.startswith('fb'): yield Collection([_id], item['title']) yield Collection([u'francebleu'], u'France Bleu') elif split_path[0] == u'francebleu': if len(split_path) == 1: for _id, item in sorted(self._RADIOS.iteritems()): if _id.startswith('fb'): yield Collection([_id], item['title']) elif len(split_path) > 1 and split_path[1] in self._RADIOS: if len(split_path) == 2: yield Collection([split_path[0], u'direct'], u'Direct') if 'selection' in self._RADIOS[split_path[1]]: yield Collection([split_path[0], u'selection'], u'Selection') elif len(split_path) == 3 and split_path[2] == 'selection': selection_url = self._RADIOS[split_path[1]]['selection'] for item in self.browser.get_selection('francebleu', selection_url, split_path[1]): yield item elif len(split_path) == 3 and split_path[2] == 'direct': yield self.get_radio(split_path[1]) else: raise CollectionNotFound(split_path) elif len(split_path) == 1: yield Collection([split_path[0], u'direct'], u'Direct') if 'selection' in self._RADIOS[split_path[0]]: yield Collection([split_path[0], u'selection'], u'Selection') if 'podcast' in self._RADIOS[split_path[0]]: yield Collection([split_path[0], u'podcasts'], u'Podcast') elif len(split_path) == 2 and split_path[1] == 'selection': for _id, item in sorted(self._RADIOS.iteritems()): if _id == split_path[0]: if 'selection' in self._RADIOS[_id]: selection_url = self._RADIOS[_id]['selection'] for item in self.browser.get_selection(_id, selection_url, _id): yield item break elif len(split_path) == 2 and split_path[1] == 'podcasts': for item in self.browser.get_podcast_emissions(split_path[0], self._RADIOS[split_path[0]]['podcast'], split_path): yield item elif len(split_path) == 2 and split_path[1] == 'direct': yield self.get_radio(split_path[0]) elif len(split_path) == 3: podcasts_url = split_path[-1] if split_path[0] == 'franceculture': podcasts_url = self.browser.get_france_culture_podcasts_url(split_path[-1]) elif split_path[0] == 'francetvinfo': podcasts_url = self.browser.get_francetvinfo_podcasts_url(split_path[-1]) if podcasts_url: for item in self.browser.get_podcasts(podcasts_url): yield item else: raise CollectionNotFound(split_path) def get_radio(self, radio): def create_stream(url, hd=True): stream = BaseAudioStream(0) if hd: stream.bitrate = 128 else: stream.bitrate = 32 url = url.replace('midfi', 'lofi') stream.format = u'mp3' stream.title = u'%s kbits/s' % (stream.bitrate) stream.url = url return stream if not isinstance(radio, Radio): radio = Radio(radio) if radio.id not in self._RADIOS: return None title = self._RADIOS[radio.id]['title'] player_url = self._RADIOS[radio.id]['player'] radio.title = title radio.description = title radio_name = radio.id if not radio.id.startswith('fb') else 'francebleu' url = self.browser.get_radio_url(radio_name, player_url) self.fillobj(radio, ('current', )) radio.streams = [create_stream(url), create_stream(url, False)] return radio def fill_radio(self, radio, fields): if 'current' in fields: title = self._RADIOS[radio.id]['title'] live_url = self._RADIOS[radio.id]['live'] radio_name = radio.id if not radio.id.startswith('fb') else 'francebleu' artist, title = self.browser.get_current(radio_name, live_url) if not radio.current or radio.current is NotLoaded: radio.current = StreamInfo(0) radio.current.what = title radio.current.who = artist return radio def fill_audio(self, audio, fields): if 'thumbnail' in fields and audio.thumbnail: audio.thumbnail.data = self.browser.open(audio.thumbnail.url) return audio def get_radio_id(self, audio_id): m = re.match('^\w+\.(\w+)\..*', audio_id) if m: return m.group(1) return '' def search_audio(self, pattern, sortby=CapAudio.SEARCH_RELEVANCE): for radio in self._RADIOS: if 'selection' in self._RADIOS[radio]: selection_url = self._RADIOS[radio]['selection'] radio_url = radio if not radio.startswith('fb') else 'francebleu' for item in self.browser.search_audio(pattern, radio_url, selection_url, radio): yield item def get_audio(self, _id): radio = self.get_radio_id(_id) if radio in self._RADIOS: if 'selection' in self._RADIOS[radio]: selection_url = self._RADIOS[radio]['selection'] radio_url = radio if not radio.startswith('fb') else 'francebleu' return self.browser.get_audio(_id, radio_url, selection_url, radio) elif radio == 'podcast': m = re.match('audio\.podcast\.(\d*)-.*', _id) if m: for item in self.browser.get_podcasts(m.group(1)): if _id == item.id: return item def iter_radios_search(self, pattern): for key, radio in self._RADIOS.iteritems(): if pattern.lower() in radio['title'].lower() or pattern.lower() in key.lower(): yield self.get_radio(key) OBJECTS = {Radio: fill_radio, BaseAudio: fill_audio} ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000034073�13034501105�0020226�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.elements import ItemElement, DictElement, ListElement, method from weboob.browser.pages import HTMLPage, JsonPage, XMLPage from weboob.browser.filters.json import Dict from weboob.browser.filters.html import XPath from weboob.browser.filters.standard import Format, CleanText, Join, Env, Regexp, Duration, Time from weboob.capabilities.audio import BaseAudio from weboob.tools.capabilities.audio.audio import BaseAudioIdFilter from weboob.capabilities.image import Thumbnail from weboob.capabilities.collection import Collection import time from datetime import timedelta, datetime, date class PodcastPage(XMLPage): @method class iter_podcasts(ListElement): item_xpath = '//item' class item(ItemElement): klass = BaseAudio obj_id = BaseAudioIdFilter(Format('podcast.%s', Regexp(CleanText('./guid'), 'http://media.radiofrance-podcast.net/podcast09/(.*).mp3'))) obj_title = CleanText('title') obj_format = u'mp3' obj_ext = u'mp3' obj_url = CleanText('enclosure/@url') obj_description = CleanText('description') def obj_author(self): author = self.el.xpath('itunes:author', namespaces={'itunes': 'http://www.itunes.com/dtds/podcast-1.0.dtd'}) return CleanText('.')(author[0]) def obj_duration(self): duration = self.el.xpath('itunes:duration', namespaces={'itunes': 'http://www.itunes.com/dtds/podcast-1.0.dtd'}) return Duration(CleanText('.'))(duration[0]) def obj_thumbnail(self): thumbnail = Thumbnail(CleanText('//image[1]/url')(self)) thumbnail.url = thumbnail.id return thumbnail class RadioPage(HTMLPage): def get_url(self): url = Regexp(CleanText('//script'), '.*liveUrl: \'(.*)\', timeshiftUrl.*', default=None)(self.doc) if not url: url = CleanText('//a[@id="player"][1]/@href')(self.doc) if not url: url = CleanText('//div[@id="audio"][1]/@data-url-live')(self.doc) if not url: url = Regexp(CleanText('//script'), '.*urlLive:\'(.*)\',urlTS.*', default=None)(self.doc) return url def get_francetvinfo_selection_list(self): for item in XPath('//div[@class="flowItem"]/a')(self.doc): yield Regexp(CleanText('./@href'), 'http://www.francetvinfo.fr/(.*)')(item) @method class get_francetvinfo_selection(ItemElement): klass = BaseAudio obj_id = BaseAudioIdFilter(Format(u'francetvinfo.%s', Regexp(CleanText('//div[@class="player-selector"]/@data-url'), 'http://media.radiofrance-podcast.net/podcast09/(.*).mp3'))) obj_ext = u'mp3' obj_format = u'mp3' obj_url = CleanText('//div[@class="player-selector"]/@data-url') obj_title = Format(u'%s %s', CleanText('//div[@class="player-selector"]/@data-emission-title'), CleanText('//div[@class="player-selector"]/@data-diffusion-title')) obj_description = CleanText('//h2[1]') @method class get_france_culture_selection(ListElement): item_xpath = '//div[@id="sidebar"]/div[has-class("expression")]/div' class item(ItemElement): klass = BaseAudio obj_id = BaseAudioIdFilter(Format(u'%s.%s', Env('radio_id'), Regexp(CleanText('./div/div/a/@href'), 'http://media.radiofrance-podcast.net/podcast09/(.*).mp3'))) obj_ext = u'mp3' obj_format = u'mp3' obj_url = CleanText('./div/div/a/@href') obj_title = Format(u'%s : %s', CleanText('./a/div[@class="subtitle"]'), CleanText('./a/div[@class="title"]')) obj_description = CleanText('./div/div/a/@data-asset-xtname') def obj_duration(self): _d = CleanText('./div/div/a/@data-duration')(self) return timedelta(seconds=int(_d)) def get_france_culture_podcasts_url(self): for a in XPath('//a[@class="podcast"]')(self.doc): emission_id = Regexp(CleanText('./@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml', default=None)(a) if emission_id: return emission_id def get_france_culture_url(self): return CleanText('//a[@id="lecteur-commun"]/@href')(self.doc) def get_francetvinfo_podcasts_url(self): return Regexp(CleanText('//a[@class="btn rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self.doc) @method class get_france_info_podcast_emissions(ListElement): item_xpath = '//section[@class="magazine"]' ignore_duplicate = True class item(ItemElement): klass = Collection def obj_split_path(self): _id = Regexp(CleanText('./a/@href'), '/replay-radio/(.*)/')(self) self.env['split_path'].append(_id) return self.env['split_path'] obj_id = Regexp(CleanText('./a/@href'), '/replay-radio/(.*)/') obj_title = CleanText('./a') @method class get_mouv_podcast_emissions(ListElement): item_xpath = '//div[@class="view-content"]/div' ignore_duplicate = True class item(ItemElement): klass = Collection def condition(self): return CleanText('./div/a[@class="podcast-rss"]/@href')(self) and \ Regexp(CleanText('./div/a[@class="podcast-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) def obj_split_path(self): _id = Regexp(CleanText('./div/a[@class="podcast-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) self.env['split_path'].append(_id) return self.env['split_path'] obj_id = Regexp(CleanText('./div/a[@class="podcast-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml') obj_title = CleanText('./h2') @method class get_france_musique_podcast_emissions(ListElement): item_xpath = '//div[@class="liste-emissions"]/ul/li' ignore_duplicate = True class item(ItemElement): klass = Collection def condition(self): return CleanText('./div/ul/li/a[@class="ico-rss"]/@href')(self) and\ Regexp(CleanText('./div/ul/li/a[@class="ico-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) def obj_split_path(self): _id = Regexp(CleanText('./div/ul/li/a[@class="ico-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) self.env['split_path'].append(_id) return self.env['split_path'] obj_id = Regexp(CleanText('./div/ul/li/a[@class="ico-rss"]/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml') obj_title = CleanText('./div/h3') @method class get_france_inter_podcast_emissions(ListElement): ignore_duplicate = True item_xpath = '//article/div' class item(ItemElement): klass = Collection def condition(self): return CleanText('./div/footer/div[has-class("rss")]/a/@href')(self) and\ Regexp(CleanText('./div/footer/div[has-class("rss")]/a/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) def obj_split_path(self): _id = Regexp(CleanText('./div/footer/div[has-class("rss")]/a/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml')(self) self.env['split_path'].append(_id) return self.env['split_path'] obj_id = Regexp(CleanText('./div/footer/div[has-class("rss")]/a/@href'), 'http://radiofrance-podcast.net/podcast09/rss_(.*).xml') obj_title = CleanText('./div/div[@class="rich-section-list-item-content-show"]') def get_current(self): now = datetime.now() today = date.today() emission_title = u'' for el in self.doc.xpath('//li[@class="chronique clear"]'): emission_time = Time(CleanText('./div[@class="quand"]', replace=[(u'à', '')]))(el) emission_datetime = datetime.combine(today, emission_time) if emission_datetime > now: return u'', emission_title emission_title = CleanText('./h3[@class="titre"]')(el) return u'', u'' class JsonPage(JsonPage): @method class get_france_culture_podcast_emissions(DictElement): class item(ItemElement): klass = Collection def condition(self): return Dict('conceptLink')(self) != '' def obj_split_path(self): _id = Regexp(Dict('conceptLink'), '/emissions/(.*)')(self) self.env['split_path'].append(_id) return self.env['split_path'] obj_id = Regexp(Dict('conceptLink'), '/emissions/(.*)') obj_title = Dict('conceptTitle') def get_culture_inter_current(self): for item in self.doc: now = int(time.time()) for item in self.doc: if int(item['start']) < now and int(item['end']) > now: emission = item['surtitle'] if 'surtitle' in item else None title = item['title'] if 'title' in item else item['conceptTitle'] if emission: title = u'%s: %s' % (title, emission) return u'', title return u'', u'' @method class get_selection(DictElement): def __init__(self, *args, **kwargs): super(DictElement, self).__init__(*args, **kwargs) if 'json_url' not in self.env or \ self.env['json_url'] != u'lecteur_commun_json/selection': self.item_xpath = 'diffusions' ignore_duplicate = True class item(ItemElement): klass = BaseAudio def condition(self): return Dict('path_mp3')(self) obj_id = BaseAudioIdFilter(Format(u'%s.%s', Env('radio_id'), Dict('nid'))) obj_format = u'mp3' obj_ext = u'mp3' obj_title = Format(u'%s : %s', Dict('title_emission'), Dict('title_diff')) obj_description = Dict('desc_emission', default=u'') obj_author = Join(u', ', Dict('personnes', default=u'')) obj_url = Dict('path_mp3') def obj_thumbnail(self): if 'path_img_emission' in self.el: thumbnail = Thumbnail(Dict('path_img_emission')(self)) thumbnail.url = thumbnail.id return thumbnail def obj_duration(self): fin = Dict('fin')(self) debut = Dict('debut')(self) if debut and fin: return timedelta(seconds=int(fin) - int(debut)) def get_current(self): if 'current' in self.doc: emission_title = self.doc['current']['emission']['titre'] song_title = self.doc['current']['song']['titre'] title = u'%s: %s' % (emission_title, song_title) person = self.doc['current']['song']['interpreteMorceau'] return person, title elif 'diffusions' in self.doc: now = int(time.time()) for item in self.doc['diffusions']: if item['debut'] < now and item['fin'] > now: title = u'%s: %s' % (item['title_emission'], item['title_diff'] if 'title_diff' in item else '') person = u'' return person, title return u'', u'' else: now = int(time.time()) for item in self.doc: if int(item['debut']) < now and int(item['fin']) > now: emission = u'' if 'diffusions' in item and item['diffusions'] and 'title' in item['diffusions'][0]: emission = item['diffusions'][0]['title'] title = item['title_emission'] if emission: title = u'%s: %s' % (title, emission) person = u'' if 'personnes' in item and item['personnes'] and item['personnes'][0]: person = u','.join(item['personnes']) return person, title return u'', u'' def get_fburl(self): for el in self.doc['url']: if el['type'] == 'live' and el['bitrate'] == 128: return Dict('url')(el) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/radiofrance/test.py��������������������������������������������������������������0000664�0000000�0000000�00000006621�13034501105�0020104�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.audio import BaseAudio from weboob.capabilities.radio import Radio class RadioFranceTest(BackendTest): MODULE = 'radiofrance' def test_ls_radios_and_selections(self): l = list(self.backend.iter_resources(objs=[Radio], split_path=[])) self.assertTrue(0 < len(l) < 30) for radio in l: name = radio.split_path[-1] if name != 'francebleu': streams = self.backend.get_radio(name).streams self.assertTrue(len(streams) > 0) l_sel = list(self.backend.iter_resources(objs=[BaseAudio], split_path=[name, 'selection'])) if len(l_sel) > 0: self.assertTrue(len(l_sel[0].url) > 0) l = list(self.backend.iter_resources(objs=[Radio], split_path=['francebleu'])) self.assertTrue(len(l) > 30) for radio in l: streams = self.backend.get_radio(radio.split_path[-1]).streams self.assertTrue(len(streams) > 0) l_sel1 = list(self.backend.iter_resources(objs=[BaseAudio], split_path=['francebleu', radio.split_path[-1]])) if 'Selection' in [el.title for el in l_sel1]: l_sel = list(self.backend.iter_resources(objs=[BaseAudio], split_path=['francebleu', radio.split_path[-1], 'selection'])) if len(l_sel) > 0: self.assertTrue(len(l_sel[0].url) > 0) def test_podcasts(self): for key, item in self.backend._RADIOS.iteritems(): if 'podcast' in item: emissions = list(self.backend.iter_resources(objs=[BaseAudio], split_path=[key, 'podcasts'])) self.assertTrue(len(emissions) > 0) podcasts = list(self.backend.iter_resources(objs=[BaseAudio], split_path=emissions[0].split_path)) self.assertTrue(len(podcasts) > 0) podcast = self.backend.get_audio(podcasts[0].id) self.assertTrue(podcast.url) def test_search_radio(self): l = list(self.backend.iter_radios_search('bleu')) self.assertTrue(len(l) > 0) self.assertTrue(len(l[0].streams) > 0) def test_search_get_audio(self): l = list(self.backend.search_audio('jou')) self.assertTrue(len(l) > 0) a = self.backend.get_audio(l[0].id) self.assertTrue(a.url) ���������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015750�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0020063�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import RazibusModule __all__ = ['RazibusModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003370�13034501105�0020010�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import EventListPage, EventPage __all__ = ['RazibusBrowser'] class RazibusBrowser(PagesBrowser): BASEURL = 'http://razibus.net/' TIMEOUT = 20 event_list_page = URL('evenements-a-venir.php\?region=(?P<region>.*)', EventListPage) event_page = URL('(?P<_id>.*).html', EventPage) region = None def __init__(self, region, *args, **kwargs): super(RazibusBrowser, self).__init__(*args, **kwargs) self.region = region def get_event(self, _id, event=None): return self.event_page.go(_id=_id).get_event(obj=event) def list_events(self, date_from, date_to, city=None, categories=None): return self.event_list_page.go(region=self.region).list_events(date_from=date_from, date_to=date_to, city=city, categories=categories) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/calendar.py��������������������������������������������������������������0000664�0000000�0000000�00000002134�13034501105�0020073�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES class RazibusCalendarEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.category = CATEGORIES.CONCERT self.timezone = 'Europe/Paris' ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000007670�13034501105�0020115�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@��R ���tEXtSoftware�Adobe ImageReadyqe<��ZIDATxb`!|>0×O T _kadׯ 6?XHz~~Qp�H232-J_ "v�.䘙!�S+Pb-t {:/?@Ν'𗁁{1L`;90QhX21- ȏ?u.0t|?|x( gb`eE8 =@;hZ1 ϟg!?$ExooaTdf``/І?{�n7XgJOf/g ` (B/K e UH"ga5We9?& 8fbz;D01a2Z�tTJ1%<QFA�[,Al'Xd7dٌL2kbrsέCWaL+b J~rǹi+E q~owWkaŽᰪ\7 W(ػ6ƣ]O8Yg vر)eaY]>̲׉#?,Dg08Ԃf {JG+°]y r�Ae}x4*(tuEŬzov\M,?,V DSln8ߖ2I)'Ȥ3#J!U(R-(;?[30܏-b9_e#: i;aRӲYi#I2Eb)ܺ vQ*[ܵv}ą|I$3s厣wbf{&99C>4w]h0?;m"o|*u5M+ & H8!eW?:,yB qi0ꟅB]) Nq.9kX4"5$@箋ЊS'yJ g$U[+#^) ,8J&€܅g)E-`UʞR�۱OGmx^*=]JHjld ioZjQ 0 21vG*߆l_DUǡidC @J y۶BLΤ6^'Fzv1xbq9,$x} P#fs7Ž:"%O3�} ùZmٴ_t <<@; Y (4_o= kX2heӶflXd|S nyu,M5/lj}`6^ry{tMOLFv q(1(( wW*hJεAC\AЅJD1kQD\d#NN !8D΅&]tא^Vk~4S {/ S?F/P铇en-WBL8 p FOjAFA4 �wא+݊ҌJ! U ]vB`¶-)EuL< B̔A4cYR \Q 9Vt}!&n7vʲԁ,E49wOT:d[I?[B1. 79N~OTZTVs&D& -W\M0wmy%�*boATNpr<Ĕa z^ )!MmZ99 ēO,)/ѩyqb Y=B'Y7ð$eR>y#%=3fTq+˾e9Yig}A)ImRk4/KV_1AS)oA BwX�T.(U-\ˢRRt(Rr)'7(''W <FǻLQP㏠\6o-1Ax<W i4-G+g<PcWR!P/56 ,)BP?6ݕ.fmi{5㮻cDet28x^g2r'ێ ]YH"DVFcU)|q 75)z&iգ (kYRJwUMLKxܲYz'RyO esB#b~wpB.O BC̶7ȋmyu)hdsQڲ>Z݂EhM͂(ݰ _l۔@@{%L$ U5 :8bF]7Θ !$ov4lAUE0j6(;<? H$bU$(`U TThE*WC- BH �E%B* Q5 ڦ x23v{ [[?6s|9XWD`mVv<5 >#=I<`Op>mR]"$Ѐ?eQ4-1ۆG, d3D %Jrgl[CPwJΥRnVH}ļO&ACB`J'*rNؿA bP"1NkǸQ"n_O$1͕0ݑ؉΋}jwўq!Ȱ^?س7YR$+aX巷>lY8ZץDz~ڧ=J/K (^{ !�k[YIM8eYYQBKn*@ABRn0gvxМ)8D-ܣɓ]UB2 !!3AdwЋ#$p)KmƬ,ԞILְ+R>xcll%zy©prw 6B6+xZ Y_yUDæY$A0 ӵh } X 5q X(,sV/UEy:jhO/ZV|.jӓi 0]Eq0*-}j(tk?VԲ_+] 2蠤mH c@ij�CK}Dx@QX Was _/݀5H>NڃF:W\ |"TX*QoB6gY7u}P m) %<pC*dD 95Jq!%NZֽ9u?..QRrX!_rLZf@GUoTRBiJ84RmryH怞ۄ'_2�.̪Q4z☀yhVLpZgR)}ڂEݡ�3.Cm$6aP <7ѻTR)KkMC0hZ(4 1UD(x:vJS7jSSw9C_H`T{l,iߔJa([)589Z(7(bQ>bz@1>n)PyNGJJ% R Jݾ1 (;iaG8|q ߦSBPs %6o)O _RBt tLb60^PwFo cne oljMS6DՎ> �?{@|!@,M~?&H!Vh (& JU}K|+-S@j7 05vґ�;(tC 8~߅(z X˼>A~UtI!`۞'gCK;h>;D'4F@P77YsmCYn`ዮ{mH xE*nPu Aؼ&;鸓k5~r 4 ,Xx0Da=/xQӼ"s1<\sf:!F ov U����IENDB`������������������������������������������������������������������������weboob-1.2/modules/razibus/module.py����������������������������������������������������������������0000664�0000000�0000000�00000006255�13034501105�0017617�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from .browser import RazibusBrowser from .calendar import RazibusCalendarEvent __all__ = ['RazibusModule'] class RazibusModule(Module, CapCalendarEvent): NAME = 'razibus' DESCRIPTION = u'site annonçant les évènements attendus par les punks a chiens' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT] BROWSER = RazibusBrowser region_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'-- Indifférent --', 'Alsace': u'Alsace', 'Aquitaine': u'Aquitaine', 'Auvergne': u'Auvergne', 'Basse-Normandie': u'Basse-Normandie', 'Bourgogne': u'Bourgogne', 'Bretagne': u'Bretagne', 'Centre': u'Centre', 'Champagne-Ardenne': u'Champagne-Ardenne', 'Franche-Comte': u'Franche-Comté', 'Haute-Normandie': u'Haute-Normandie', 'Ile-de-France': u'Île-de-France', 'Languedoc-Roussillon': u'Languedoc-Roussillon', 'Limousin': u'Limousin', 'Lorraine': u'Lorraine', 'Midi-Pyrenees': u'Midi-Pyrénées', 'Nord-Pas-de-Calais': u'Nord-Pas-de-Calais', 'Pays-de-la-Loire': u'Pays de la Loire', 'Picardie': u'Picardie', 'Poitou-Charentes': u'Poitou-Charentes', 'PACA': u'PACA', 'Rhone-Alpes': u'Rhône-Alpes', 'Belgique': u'Belgique', 'Suisse': u'Suisse', }.iteritems())]) CONFIG = BackendConfig(Value('region', label=u'Region', choices=region_choices, default='')) def create_default_browser(self): region = self.config['region'].get() return self.create_browser(region) def search_events(self, query): return self.browser.list_events(query.start_date, query.end_date, query.city, query.categories) def get_event(self, _id): return self.browser.get_event(_id) def list_events(self, date_from, date_to=None): return self.browser.list_events(date_from, date_to) def fill_obj(self, event, fields): return self.browser.get_event(event.id, event) OBJECTS = {RazibusCalendarEvent: fill_obj} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000006643�13034501105�0017432�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .calendar import RazibusCalendarEvent from datetime import time from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.html import CleanHTML, Link from weboob.browser.filters.standard import Regexp, CleanText, DateTime, CombineDate, Filter, Env class EndTime(Filter): def filter(self, el): return time.max class EventListPage(HTMLPage): @method class list_events(ListElement): item_xpath = '//div[@class="item"]' class item(ItemElement): klass = RazibusCalendarEvent def validate(self, obj): return (self.is_valid_event(obj, self.env['city'], self.env['categories']) and self.is_event_in_valid_period(obj.start_date, self.env['date_from'], self.env['date_to'])) def is_valid_event(self, event, city, categories): if city and city != '' and city.upper() != event.city.upper(): return False if categories and len(categories) > 0 and event.category not in categories: return False return True def is_event_in_valid_period(self, event_date, date_from, date_to): if event_date >= date_from: if not date_to: return True else: if event_date <= date_to: return True return False obj_id = Regexp(Link('./p/strong/a[@itemprop="url"]'), 'http://razibus.net/(.*).html') obj_summary = CleanText('./p/strong/a[@itemprop="url"]') obj_start_date = DateTime(CleanText('./p/span[@itemprop="startDate"]/@content')) obj_end_date = CombineDate(DateTime(CleanText('./p/span[@itemprop="startDate"]/@content')), EndTime('.')) obj_location = CleanText('./p/span[@itemprop="location"]/@content') obj_city = CleanText('./p/span[@itemprop="location"]') class EventPage(HTMLPage): @method class get_event(ItemElement): klass = RazibusCalendarEvent obj_id = Env('_id') obj_summary = CleanText('//h2[@itemprop="name"]') obj_start_date = DateTime(CleanText('//span[@itemprop="startDate"]/time/@datetime')) obj_end_date = CombineDate(DateTime(CleanText('//span[@itemprop="startDate"]/time/@datetime')), EndTime('.')) obj_location = CleanText('//meta[@property="og:street-address"]/@content') obj_city = CleanText('//meta[@property="og:locality"]/@content') obj_url = CleanText('//meta[@property="og:url"]/@content') obj_description = CleanHTML('//div[@itemprop="description"]') ���������������������������������������������������������������������������������������������weboob-1.2/modules/razibus/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002132�13034501105�0017277�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from datetime import datetime class RazibusTest(BackendTest): MODULE = 'razibus' def test_razibus(self): l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015714�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0020032�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import RedmineModule __all__ = ['RedmineModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000025656�13034501105�0017767�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from urlparse import urlsplit import urllib import re import lxml.html from weboob.capabilities.bugtracker import IssueError from weboob.deprecated.browser import Browser, BrowserIncorrectPassword from .pages.index import LoginPage, IndexPage, MyPage, ProjectsPage from .pages.wiki import WikiPage, WikiEditPage from .pages.issues import IssuesPage, IssuePage, NewIssuePage, IssueLogTimePage, \ IssueTimeEntriesPage __all__ = ['RedmineBrowser'] # Browser class RedmineBrowser(Browser): ENCODING = 'utf-8' PAGES = { 'https?://[^/]+/': IndexPage, 'https?://[^/]+/login': LoginPage, # compatibility with redmine 0.9 'https?://[^/]+/login\?back_url.*': MyPage, 'https?://[^/]+/my/page': MyPage, 'https?://[^/]+/projects': ProjectsPage, 'https?://[^/]+/projects/([\w-]+)/wiki/([^\/]+)/edit(?:\?version=\d+)?': WikiEditPage, 'https?://[^/]+/projects/[\w-]+/wiki/[^\/]*': WikiPage, 'https?://[^/]+/projects/[\w-]+/issues/new': NewIssuePage, 'https?://[^/]+/projects/[\w-]+/issues': IssuesPage, 'https?://[^/]+/issues(|/?\?.*)': IssuesPage, 'https?://[^/]+/issues/(\d+)': IssuePage, 'https?://[^/]+/issues/(\d+)/time_entries/new': IssueLogTimePage, 'https?://[^/]+/projects/[\w-]+/time_entries': IssueTimeEntriesPage, } def __init__(self, url, *args, **kwargs): self._userid = 0 v = urlsplit(url) self.PROTOCOL = v.scheme self.DOMAIN = v.netloc self.BASEPATH = v.path if self.BASEPATH.endswith('/'): self.BASEPATH = self.BASEPATH[:-1] Browser.__init__(self, *args, **kwargs) self.projects = {} def is_logged(self): return self.is_on_page(LoginPage) or self.page and len(self.page.document.getroot().cssselect('a.my-account')) == 1 def login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.is_on_page(LoginPage): self.location('%s/login' % self.BASEPATH, no_login=True) self.page.login(self.username, self.password) if self.is_on_page(LoginPage): raise BrowserIncorrectPassword() divs = self.page.document.getroot().cssselect('div#loggedas') if len(divs) > 0: parts = divs[0].find('a').attrib['href'].split('/') self._userid = int(parts[2]) def get_userid(self): return self._userid def get_wiki_source(self, project, page, version=None): url = '%s/projects/%s/wiki/%s/edit' % (self.BASEPATH, project, urllib.quote(page.encode('utf-8'))) if version: url += '?version=%s' % version self.location(url) return self.page.get_source() def set_wiki_source(self, project, page, data, message): self.location('%s/projects/%s/wiki/%s/edit' % (self.BASEPATH, project, urllib.quote(page.encode('utf-8')))) self.page.set_source(data, message) def get_wiki_preview(self, project, page, data): if (not self.is_on_page(WikiEditPage) or self.page.groups[0] != project or self.page.groups[1] != page): self.location('%s/projects/%s/wiki/%s/edit' % (self.BASEPATH, project, urllib.quote(page.encode('utf-8')))) url = '%s/projects/%s/wiki/%s/preview' % (self.BASEPATH, project, urllib.quote(page.encode('utf-8'))) params = {} params['content[text]'] = data.encode('utf-8') params['authenticity_token'] = "%s" % self.page.get_authenticity_token() preview_html = lxml.html.fragment_fromstring(self.readurl(url, urllib.urlencode(params)), create_parent='div') preview_html.find("fieldset").drop_tag() preview_html.find("legend").drop_tree() return lxml.html.tostring(preview_html) METHODS = {'POST': {'project_id': 'project_id', 'column': 'query[column_names][]', 'value': 'values[%s][]', 'field': 'fields[]', 'operator': 'operators[%s]', }, 'GET': {'project_id': 'project_id', 'column': 'c[]', 'value': 'v[%s][]', 'field': 'f[]', 'operator': 'op[%s]', } } def query_issues(self, project_name, **kwargs): self.location('/projects/%s/issues' % project_name) token = self.page.get_authenticity_token() method = self.page.get_query_method() data = ((self.METHODS[method]['project_id'], project_name), (self.METHODS[method]['column'], 'tracker'), ('authenticity_token', token), (self.METHODS[method]['column'], 'status'), (self.METHODS[method]['column'], 'priority'), (self.METHODS[method]['column'], 'subject'), (self.METHODS[method]['column'], 'assigned_to'), (self.METHODS[method]['column'], 'updated_on'), (self.METHODS[method]['column'], 'category'), (self.METHODS[method]['column'], 'fixed_version'), (self.METHODS[method]['column'], 'done_ratio'), (self.METHODS[method]['column'], 'author'), (self.METHODS[method]['column'], 'start_date'), (self.METHODS[method]['column'], 'due_date'), (self.METHODS[method]['column'], 'estimated_hours'), (self.METHODS[method]['column'], 'created_on'), ) for key, value in kwargs.iteritems(): if value: value = self.page.get_value_from_label(self.METHODS[method]['value'] % key, value) data += ((self.METHODS[method]['value'] % key, value),) data += ((self.METHODS[method]['field'], key),) data += ((self.METHODS[method]['operator'] % key, '~'),) if method == 'POST': self.location('/issues?set_filter=1&per_page=100', urllib.urlencode(data)) else: data += (('set_filter', '1'), ('per_page', '100')) self.location(self.buildurl('/issues', *data)) assert self.is_on_page(IssuesPage) return {'project': self.page.get_project(project_name), 'iter': self.page.iter_issues(), } def get_project(self, project): self.location('/projects/%s/issues/new' % project) assert self.is_on_page(NewIssuePage) return self.page.get_project(project) def get_issue(self, id): self.location('/issues/%s' % id) assert self.is_on_page(IssuePage) return self.page.get_params() def logtime_issue(self, id, hours, message): self.location('/issues/%s/time_entries/new' % id) assert self.is_on_page(IssueLogTimePage) self.page.logtime(hours.seconds/3600, message) def comment_issue(self, id, message): self.location('/issues/%s' % id) assert self.is_on_page(IssuePage) self.page.fill_form(note=message) def get_custom_fields(self, project): self.location('/projects/%s/issues/new' % project) assert self.is_on_page(NewIssuePage) fields = {} for key, div in self.page.iter_custom_fields(): if 'value' in div.attrib: fields[key] = div.attrib['value'] else: olist = div.xpath('.//option[@selected="selected"]') fields[key] = ', '.join([i.attrib['value'] for i in olist]) return fields def create_issue(self, project, **kwargs): self.location('/projects/%s/issues/new' % project) assert self.is_on_page(NewIssuePage) self.page.fill_form(**kwargs) error = self.page.get_errors() if len(error) > 0: raise IssueError(error) assert self.is_on_page(IssuePage) return int(self.page.groups[0]) def edit_issue(self, id, **kwargs): self.location('/issues/%s' % id) assert self.is_on_page(IssuePage) self.page.fill_form(**kwargs) assert self.is_on_page(IssuePage) return int(self.page.groups[0]) def remove_issue(self, id): self.location('/issues/%s' % id) assert self.is_on_page(IssuePage) token = self.page.get_authenticity_token() data = (('authenticity_token', token),) self.openurl('/issues/%s/destroy' % id, urllib.urlencode(data)) def iter_projects(self): self.location('/projects') return self.page.iter_projects() def create_category(self, project, name, token): data = {'issue_category[name]': name.encode('utf-8')} headers = {'X-CSRF-Token': token, 'X-Prototype-Version': '1.7', 'X-Requested-With': 'XMLHttpRequest', 'Accept': 'text/javascript, text/html, application/xml, text/xml, */*', } request = self.request_class(self.absurl(self.buildurl('%s/projects/%s/issue_categories' % (self.BASEPATH, project), **data)), '', headers) r = self.readurl(request) # Element.replace("issue_category_id", "\u003Cselect id=\"issue_category_id\" name=\"issue[category_id]\"\u003E\u003Coption\u003E\u003C/option\u003E\u003Coption value=\"28\"\u003Ebnporc\u003C/option\u003E\n\u003Coption value=\"31\"\u003Ebp\u003C/option\u003E\n\u003Coption value=\"30\"\u003Ecrag2r\u003C/option\u003E\n\u003Coption value=\"29\"\u003Ecragr\u003C/option\u003E\n\u003Coption value=\"27\"\u003Ei\u003C/option\u003E\n\u003Coption value=\"32\"\u003Elol\u003C/option\u003E\n\u003Coption value=\"33\" selected=\"selected\"\u003Elouiel\u003C/option\u003E\u003C/select\u003E"); m = re.search('''value=\\\\"(\d+)\\\\" selected''', r) if m: return m.group(1) ����������������������������������������������������������������������������������weboob-1.2/modules/redmine/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000004062�13034501105�0020051�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ~���tIME 918f=��IDATxZo NqEEY-۰H-;HSiւ-=r)zjhkM&ȑbɕlZ+NhRdT޼Pm,4\d�d�d�d�ZmC?2$9^Os޻ 1 fR<Rb"]IXd_qSZ<6 N[es}�*h*I I!u=a#]l= f2Vx e{s1棧 udpią!qOh'r@XyɹBOGkh\!n&܎2kW+ <$ٕf fVIcό.4M!˕1=�܆qgvV :톳C � ؍,N=zz[q 62dFڴ<N]}HÍADS:u ˬ%,WwI$7Uk=?ZNB劀`M,V_cYo}2?]F8 XfTyZInb11X M!.`YOM01ZǾZb$G wg( uCl2D*_{ЬSLd�Ja Z4Irߩ\ep8<h{w⽛  w8$GPTOW0TMĢW57�v $s%Uu |#X0l:CM=1*n"m��OYܛ̐]4~x'<bMɻ̶ ]H yzĘJFɕm ){��lFFkz,Sk-s4EzAەV8/éd3Ptoj<5cMa2u{wxz^sN"MIH .ZvE e* S"7(Z EcJ2~"B)Ly2+` E0zЂ4hY\!/âa6# 46N{ce- *v Kpe Q-Ud ehߥG<SDXA)bu#nyR4zz[pfȎуV:uYJ⯟ '8Tvm*qfȆcI=N$IV[zZmc[ڸ= Q0}&2nbct&Jp;CN[c "Y|8h'[߅o\d�d�\vn҈ A,3oW+S+KdY,Wƭ``7*] pDŽS~ ,R3KI|2/�GM8f$u;Ig~+lF\$UDo(pߊN=֒q \o~I$Yw( s�](4N�>/s:Җ+hA> e Tm4<6-{kdOAQMAs+iO B`~-MsJ<fŬIn'Vucbv|b7땸<b(Uч:5Ԥ\Πm-nn|ey#SDMg3/ ݲ@~4oJZ39m}þ(rTbګXEҜ7O'�+~KcpD"q-�nkjdŠ"tvmUWI^ݭKjݚm6Lm2TyM8;de*i\<މX{sj<O/.SGvmnONG~{;"%?E ŭɐȬ���������VtyB����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/module.py����������������������������������������������������������������0000664�0000000�0000000�00000027532�13034501105�0017564�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.content import CapContent, Content from weboob.capabilities.bugtracker import CapBugTracker, Issue, Project, User, \ Version, Status, Update, Attachment, \ Query, Change from weboob.capabilities.collection import CapCollection, Collection, CollectionNotFound from weboob.tools.backend import Module, BackendConfig from weboob.exceptions import BrowserHTTPNotFound from weboob.tools.value import ValueBackendPassword, Value from .browser import RedmineBrowser __all__ = ['RedmineModule'] class RedmineModule(Module, CapContent, CapBugTracker, CapCollection): NAME = 'redmine' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'The Redmine project management web application' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('url', label='URL of the Redmine website', regexp=r'https?://.*'), Value('username', label='Login'), ValueBackendPassword('password', label='Password')) BROWSER = RedmineBrowser def create_default_browser(self): return self.create_browser(self.config['url'].get(), self.config['username'].get(), self.config['password'].get()) ############# CapContent ###################################################### def id2path(self, id): return id.split('/', 2) def get_content(self, id, revision=None): if isinstance(id, basestring): content = Content(id) else: content = id id = content.id try: _type, project, page = self.id2path(id) except ValueError: return None version = revision.id if revision else None with self.browser: data = self.browser.get_wiki_source(project, page, version) content.content = data return content def push_content(self, content, message=None, minor=False): try: _type, project, page = self.id2path(content.id) except ValueError: return with self.browser: return self.browser.set_wiki_source(project, page, content.content, message) def get_content_preview(self, content): try: _type, project, page = self.id2path(content.id) except ValueError: return with self.browser: return self.browser.get_wiki_preview(project, page, content.content) ############# CapCollection ################################################### def iter_resources(self, objs, split_path): if Project in objs or Issue in objs: self._restrict_level(split_path, 1) if len(split_path) == 0: return [Collection([project.id], project.name) for project in self.iter_projects()] elif len(split_path) == 1: query = Query() query.project = unicode(split_path[0]) return self.iter_issues(query) def validate_collection(self, objs, collection): if collection.path_level == 0: return if Issue in objs and collection.path_level == 1: for project in self.iter_projects(): if collection.basename == project.id: return Collection([project.id], project.name) # if the project is not found by ID, try again by name for project in self.iter_projects(): if collection.basename == project.name: return Collection([project.id], project.name) raise CollectionNotFound(collection.split_path) ############# CapBugTracker ################################################### @classmethod def _build_project(cls, project_dict): project = Project(project_dict['name'], project_dict['name']) project.members = [User(int(u[0]), u[1]) for u in project_dict['members']] project.versions = [Version(int(v[0]), v[1]) for v in project_dict['versions']] project.categories = [c[1] for c in project_dict['categories']] # TODO set the value of status project.statuses = [Status(int(s[0]), s[1], 0) for s in project_dict['statuses']] return project @staticmethod def _attr_to_id(availables, text): if not text: return None if isinstance(text, basestring) and text.isdigit(): return text for value, key in availables: if key.lower() == text.lower(): return value return text def iter_issues(self, query): """ Iter issues with optionnal patterns. @param query [Query] @return [iter(Issue)] issues """ params = self.browser.get_project(query.project) kwargs = {'subject': query.title, 'author_id': self._attr_to_id(params['members'], query.author), 'assigned_to_id': self._attr_to_id(params['members'], query.assignee), 'fixed_version_id': self._attr_to_id(params['versions'], query.version), 'category_id': self._attr_to_id(params['categories'], query.category), 'status_id': self._attr_to_id(params['statuses'], query.status), } r = self.browser.query_issues(query.project, **kwargs) project = self._build_project(r['project']) for issue in r['iter']: obj = Issue(issue['id']) obj.project = project obj.title = issue['subject'] obj.creation = issue['created_on'] obj.updated = issue['updated_on'] obj.start = issue['start_date'] obj.due = issue['due_date'] if isinstance(issue['author'], tuple): obj.author = project.find_user(*issue['author']) else: obj.author = User(0, issue['author']) if isinstance(issue['assigned_to'], tuple): obj.assignee = project.find_user(*issue['assigned_to']) else: obj.assignee = issue['assigned_to'] obj.tracker = issue['tracker'] obj.category = issue['category'] if issue['fixed_version'] is not None: obj.version = project.find_version(*issue['fixed_version']) else: obj.version = None obj.status = project.find_status(issue['status']) obj.priority = issue['priority'] yield obj def get_issue(self, issue): if isinstance(issue, Issue): id = issue.id else: id = issue issue = Issue(issue) try: with self.browser: params = self.browser.get_issue(id) except BrowserHTTPNotFound: return None issue.project = self._build_project(params['project']) issue.title = params['subject'] issue.body = params['body'] issue.creation = params['created_on'] issue.updated = params['updated_on'] issue.start = params['start_date'] issue.due = params['due_date'] issue.fields = {} for key, value in params['fields'].iteritems(): issue.fields[key] = value issue.attachments = [] for a in params['attachments']: attachment = Attachment(a['id']) attachment.filename = a['filename'] attachment.url = a['url'] issue.attachments.append(attachment) issue.history = [] for u in params['updates']: update = Update(u['id']) update.author = issue.project.find_user(*u['author']) update.date = u['date'] update.message = u['message'] update.changes = [] for i, (field, last, new) in enumerate(u['changes']): change = Change(i) change.field = field change.last = last change.new = new update.changes.append(change) issue.history.append(update) issue.author = issue.project.find_user(*params['author']) issue.assignee = issue.project.find_user(*params['assignee']) issue.tracker = params['tracker'][1] issue.category = params['category'][1] issue.version = issue.project.find_version(*params['version']) issue.status = issue.project.find_status(params['status'][1]) issue.priority = params['priority'][1] return issue def create_issue(self, project): try: with self.browser: r = self.browser.get_project(project) except BrowserHTTPNotFound: return None issue = Issue(0) issue.project = self._build_project(r) with self.browser: issue.fields = self.browser.get_custom_fields(project) return issue def post_issue(self, issue): project = issue.project.id kwargs = {'title': issue.title, 'version': issue.version.id if issue.version else None, 'assignee': issue.assignee.id if issue.assignee else None, 'tracker': issue.tracker if issue.tracker else None, 'category': issue.category, 'status': issue.status.id if issue.status else None, 'priority': issue.priority if issue.priority else None, 'start': issue.start if issue.start else None, 'due': issue.due if issue.due else None, 'body': issue.body, 'fields': issue.fields, } with self.browser: if int(issue.id) < 1: id = self.browser.create_issue(project, **kwargs) else: id = self.browser.edit_issue(issue.id, **kwargs) if id is None: return None issue.id = id return issue def update_issue(self, issue, update): if isinstance(issue, Issue): issue = issue.id with self.browser: if update.hours: return self.browser.logtime_issue(issue, update.hours, update.message) else: return self.browser.comment_issue(issue, update.message) def remove_issue(self, issue): """ Remove an issue. """ if isinstance(issue, Issue): issue = issue.id with self.browser: return self.browser.remove_issue(issue) def iter_projects(self): """ Iter projects. @return [iter(Project)] projects """ with self.browser: for project in self.browser.iter_projects(): yield Project(project['id'], project['name']) def get_project(self, id): try: with self.browser: params = self.browser.get_project(id) except BrowserHTTPNotFound: return None return self._build_project(params) def fill_issue(self, issue, fields): return self.get_issue(issue) OBJECTS = {Issue: fill_issue} ����������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/pages/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017013�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/pages/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021112�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/pages/index.py�����������������������������������������������������������0000664�0000000�0000000�00000003023�13034501105�0020472�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page class LoginPage(Page): def login(self, username, password): self.browser.select_form(predicate=lambda f: f.attrs.get('method', '') == 'post') self.browser['username'] = username.encode(self.browser.ENCODING) self.browser['password'] = password.encode(self.browser.ENCODING) self.browser.submit() class IndexPage(Page): pass class MyPage(Page): pass class ProjectsPage(Page): def iter_projects(self): for ul in self.parser.select(self.document.getroot(), 'ul.projects'): for li in ul.findall('li'): prj = {} link = li.find('div').find('a') prj['id'] = link.attrib['href'].split('/')[-1] prj['name'] = link.text yield prj �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/pages/issues.py����������������������������������������������������������0000664�0000000�0000000�00000051246�13034501105�0020710�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re import datetime from weboob.capabilities.bugtracker import IssueError from weboob.deprecated.browser import Page, BrokenPageError from weboob.tools.date import parse_french_date from weboob.tools.misc import to_unicode from weboob.deprecated.mech import ClientForm from weboob.tools.json import json class BaseIssuePage(Page): def parse_datetime(self, text): m = re.match('(\d+)/(\d+)/(\d+) (\d+):(\d+) (\w+)', text) if m: date = datetime.datetime(int(m.group(3)), int(m.group(1)), int(m.group(2)), int(m.group(4)), int(m.group(5))) if m.group(6) == 'pm': date += datetime.timedelta(0,12*3600) return date m = re.match('(\d+)-(\d+)-(\d+) (\d+):(\d+)', text) if m: return datetime.datetime(int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4)), int(m.group(5))) m = re.match('(\d+) (\w+) (\d+) (\d+):(\d+)', text) if m: return parse_french_date(text) self.logger.warning('Unable to parse "%s"' % text) return text PROJECT_FIELDS = {'members': 'values_assigned_to_id', 'categories': 'values_category_id', 'versions': 'values_fixed_version_id', 'statuses': 'values_status_id', } def iter_choices(self, name): try: select = self.parser.select(self.document.getroot(), 'select#%s' % name, 1) except BrokenPageError: try: select = self.parser.select(self.document.getroot(), 'select#%s_1' % name, 1) except BrokenPageError: return for option in select.findall('option'): if option.attrib['value'].isdigit(): yield (option.attrib['value'], option.text) def get_project(self, project_name): project = {} project['name'] = project_name for field, elid in self.PROJECT_FIELDS.iteritems(): project[field] = list(self.iter_choices(elid)) return project def get_authenticity_token(self): tokens = self.parser.select(self.document.getroot(), 'input[name=authenticity_token]') if len(tokens) == 0: tokens = self.document.xpath('//meta[@name="csrf-token"]') if len(tokens) == 0: raise IssueError("You don't have rights to remove this issue.") try: token = tokens[0].attrib['value'] except KeyError: token = tokens[0].attrib['content'] return token def get_errors(self): errors = [] for li in self.document.xpath('//div[@id="errorExplanation"]//li'): errors.append(li.text.strip()) return ', '.join(errors) def get_value_from_label(self, name, label): for option in self.document.xpath('//select[@name="%s"]/option' % name): if option.text.strip().lower() == label.lower(): return option.attrib['value'] return label class IssuesPage(BaseIssuePage): PROJECT_FIELDS = {'members': 'values_assigned_to_id', 'categories': 'values_category_id', 'versions': 'values_fixed_version_id', 'statuses': 'values_status_id', } def get_from_js(self, pattern, end, is_list=False): """ find a pattern in any javascript text """ value = None for script in self.document.xpath('//script'): txt = script.text if txt is None: continue start = txt.find(pattern) if start < 0: continue while True: if value is None: value = '' else: value += ',' value += txt[start+len(pattern):start+txt[start+len(pattern):].find(end)+len(pattern)] if not is_list: break txt = txt[start+len(pattern)+txt[start+len(pattern):].find(end):] start = txt.find(pattern) if start < 0: break return value def get_project(self, project_name): project = super(IssuesPage, self).get_project(project_name) if len(project['statuses']) > 0: return project args = self.get_from_js('var availableFilters = ', ';') if args is None: return project args = json.loads(args) def get_values(key): values = [] if key not in args: return values for key, value in args[key]['values']: if value.isdigit(): values.append((value, key)) return values project['members'] = get_values('assigned_to_id') project['categories'] = get_values('category_id') project['versions'] = get_values('fixed_version_id') project['statuses'] = get_values('status_id') return project def get_query_method(self): return self.document.xpath('//form[@id="query_form"]')[0].attrib['method'].upper() def iter_issues(self): try: issues = self.parser.select(self.document.getroot(), 'table.issues', 1) except BrokenPageError: # No results. return for tr in issues.getiterator('tr'): if not tr.attrib.get('id', '').startswith('issue-'): continue issue = {'id': tr.attrib['id'].replace('issue-', '')} for td in tr.getiterator('td'): field = td.attrib.get('class', '') if field in ('checkbox','todo',''): continue a = td.find('a') if a is not None: if a.attrib['href'].startswith('/users/') or \ a.attrib['href'].startswith('/versions/'): text = (int(a.attrib['href'].split('/')[-1]), a.text) else: text = a.text else: text = td.text if field.endswith('_on'): text = self.parse_datetime(text) elif field.endswith('_date') and text is not None: m = re.match('(\d+)-(\d+)-(\d+)', text) if m: text = datetime.datetime(int(m.group(1)), int(m.group(2)), int(m.group(3))) if isinstance(text, str): text = to_unicode(text) issue[field] = text if len(issue) != 0: yield issue class NewIssuePage(BaseIssuePage): PROJECT_FIELDS = {'members': 'issue_assigned_to_id', 'categories': 'issue_category_id', 'versions': 'issue_fixed_version_id', 'statuses': 'issue_status_id', } def get_project_name(self): m = re.search('/projects/([^/]+)/', self.url) return m.group(1) def iter_custom_fields(self): for div in self.document.xpath('//form//input[starts-with(@id, "issue_custom_field")]|//form//select[starts-with(@id, "issue_custom_field")]'): if 'type' in div.attrib and div.attrib['type'] == 'hidden': continue label = self.document.xpath('//label[@for="%s"]' % div.attrib['id'])[0] yield self.parser.tocleanstring(label), div def set_title(self, title): self.browser['issue[subject]'] = title.encode('utf-8') def set_body(self, body): self.browser['issue[description]'] = body.encode('utf-8') def set_assignee(self, member): if member: self.browser['issue[assigned_to_id]'] = [str(member)] else: self.browser['issue[assigned_to_id]'] = [''] def set_version(self, version): try: if version: self.browser['issue[fixed_version_id]'] = [str(version)] else: self.browser['issue[fixed_version_id]'] = [''] except ClientForm.ItemNotFoundError: self.logger.warning('Version not found: %s' % version) def set_tracker(self, tracker): if tracker: select = self.parser.select(self.document.getroot(), 'select#issue_tracker_id', 1) for option in select.findall('option'): if option.text and option.text.strip() == tracker: self.browser['issue[tracker_id]'] = [option.attrib['value']] return # value = None # if len(self.document.xpath('//a[@title="New tracker"]')) > 0: # value = self.browser.create_tracker(self.get_project_name(), tracker, self.get_authenticity_token()) # if value: # control = self.browser.find_control('issue[tracker_id]') # ClientForm.Item(control, {'name': tracker, 'value': value}) # self.browser['issue[tracker_id]'] = [value] # else: # self.logger.warning('Tracker "%s" not found' % tracker) self.logger.warning('Tracker "%s" not found' % tracker) else: try: self.browser['issue[tracker_id]'] = [''] except ClientForm.ControlNotFoundError: self.logger.warning('Tracker item not found') def set_category(self, category): if category: select = self.parser.select(self.document.getroot(), 'select#issue_category_id', 1) for option in select.findall('option'): if option.text and option.text.strip() == category: self.browser['issue[category_id]'] = [option.attrib['value']] return value = None if len(self.document.xpath('//a[@title="New category"]')) > 0: value = self.browser.create_category(self.get_project_name(), category, self.get_authenticity_token()) if value: control = self.browser.find_control('issue[category_id]') ClientForm.Item(control, {'name': category, 'value': value}) self.browser['issue[category_id]'] = [value] else: self.logger.warning('Category "%s" not found' % category) else: try: self.browser['issue[category_id]'] = [''] except ClientForm.ControlNotFoundError: self.logger.warning('Category item not found') def set_status(self, status): assert status is not None self.browser['issue[status_id]'] = [str(status)] def set_priority(self, priority): if priority: select = self.parser.select(self.document.getroot(), 'select#issue_priority_id', 1) for option in select.findall('option'): if option.text and option.text.strip() == priority: self.browser['issue[priority_id]'] = [option.attrib['value']] return # value = None # if len(self.document.xpath('//a[@title="New priority"]')) > 0: # value = self.browser.create_priority(self.get_project_name(), priority, self.get_authenticity_token()) # if value: # control = self.browser.find_control('issue[priority_id]') # ClientForm.Item(control, {'name': priority, 'value': value}) # self.browser['issue[priority_id]'] = [value] # else: # self.logger.warning('Priority "%s" not found' % priority) self.logger.warning('Priority "%s" not found' % priority) else: try: self.browser['issue[priority_id]'] = [''] except ClientForm.ControlNotFoundError: self.logger.warning('Priority item not found') def set_start(self, start): if start is not None: self.browser['issue[start_date]'] = start.strftime("%Y-%m-%d") #XXX: else set to "" ? def set_due(self, due): if due is not None: self.browser['issue[due_date]'] = due.strftime("%Y-%m-%d") #XXX: else set to "" ? def set_note(self, message): try: self.browser['notes'] = message.encode('utf-8') except ClientForm.ControlNotFoundError: self.browser['issue[notes]'] = message.encode('utf-8') def set_fields(self, fields): for key, div in self.iter_custom_fields(): try: control = self.browser.find_control(div.attrib['name'], nr=0) if isinstance(control, ClientForm.TextControl): control.value = fields[key] else: item = control.get(label=fields[key].encode("utf-8"), nr=0) if item and fields[key] != '': item.selected = True except KeyError: continue def fill_form(self, **kwargs): self.browser.select_form(predicate=lambda form: form.attrs.get('id', '') == 'issue-form') for key, value in kwargs.iteritems(): if value is not None: getattr(self, 'set_%s' % key)(value) self.browser.submit() class IssuePage(NewIssuePage): def _parse_selection(self, id): try: select = self.parser.select(self.document.getroot(), 'select#%s' % id, 1) except BrokenPageError: # not available for this project return ('', None) else: options = select.findall('option') for option in options: if 'selected' in option.attrib: return (int(option.attrib['value']), to_unicode(option.text)) return ('', None) def get_params(self): params = {} content = self.parser.select(self.document.getroot(), 'div#content', 1) issue = self.parser.select(content, 'div.issue', 1) params['project'] = self.get_project(to_unicode(self.parser.select(self.document.getroot(), 'h1', 1).text)) params['subject'] = to_unicode(self.parser.select(issue, 'div.subject', 1).find('div').find('h3').text.strip()) params['body'] = to_unicode(self.parser.select(self.document.getroot(), 'textarea#issue_description', 1).text) author = self.parser.select(issue, 'p.author', 1) # check issue 666 on symlink.me i = 0 alist = author.findall('a') if 'title' not in alist[i].attrib: params['author'] = (int(alist[i].attrib['href'].split('/')[-1]), to_unicode(alist[i].text)) i += 1 else: params['author'] = (0, 'Anonymous') params['created_on'] = self.parse_datetime(alist[i].attrib['title']) if len(alist) > i+1: params['updated_on'] = self.parse_datetime(alist[i+1].attrib['title']) else: params['updated_on'] = None params['status'] = self._parse_selection('issue_status_id') params['priority'] = self._parse_selection('issue_priority_id') params['assignee'] = self._parse_selection('issue_assigned_to_id') params['tracker'] = self._parse_selection('issue_tracker_id') params['category'] = self._parse_selection('issue_category_id') params['version'] = self._parse_selection('issue_fixed_version_id') div = self.parser.select(self.document.getroot(), 'input#issue_start_date', 1) if 'value' in div.attrib: params['start_date'] = datetime.datetime.strptime(div.attrib['value'], "%Y-%m-%d") else: params['start_date'] = None div = self.parser.select(self.document.getroot(), 'input#issue_due_date', 1) if 'value' in div.attrib: params['due_date'] = datetime.datetime.strptime(div.attrib['value'], "%Y-%m-%d") else: params['due_date'] = None params['fields'] = {} for key, div in self.iter_custom_fields(): value = '' if 'value' in div.attrib: value = div.attrib['value'] else: # XXX: use _parse_selection()? olist = div.xpath('.//option[@selected="selected"]') value = ', '.join([opt.attrib['value'] for opt in olist]) params['fields'][key] = value params['attachments'] = [] try: for p in self.parser.select(content, 'div.attachments', 1).findall('p'): attachment = {} a = p.find('a') attachment['id'] = int(a.attrib['href'].split('/')[-2]) attachment['filename'] = p.find('a').text attachment['url'] = '%s://%s%s' % (self.browser.PROTOCOL, self.browser.DOMAIN, p.find('a').attrib['href']) params['attachments'].append(attachment) except BrokenPageError: pass params['updates'] = [] for div in self.parser.select(content, 'div.journal'): update = {} update['id'] = div.xpath('.//h4//a')[0].text[1:] user_link = div.xpath('.//h4//a[contains(@href, "/users/")]') if len(user_link) > 0: update['author'] = (int(user_link[0].attrib['href'].split('/')[-1]), to_unicode(user_link[0].text)) else: for txt in div.xpath('.//h4//text()'): m = re.search('Updated by (.*)', txt.strip()) if m: update['author'] = (0, to_unicode(m.group(1))) update['date'] = self.parse_datetime(div.xpath('.//h4//a[last()]')[0].attrib['title']) comments = div.xpath('.//div[starts-with(@id, "journal-")]') if len(comments) > 0: comment = comments[0] subdiv = comment.find('div') if subdiv is not None: # a subdiv which contains changes is found, move the tail text # of this div to comment text, and remove it. comment.text = (comment.text or '') + (subdiv.tail or '') comment.remove(comment.find('div')) update['message'] = self.parser.tostring(comment).strip() else: update['message'] = None changes = [] try: details = self.parser.select(div, 'ul.details', 1) except BrokenPageError: pass else: for li in details.findall('li'): field = li.find('strong').text#.decode('utf-8') i = li.findall('i') new = None last = None if len(i) > 0: if len(i) == 2: last = i[0].text.decode('utf-8') new = i[-1].text#.decode('utf-8') elif li.find('strike') is not None: last = li.find('strike').find('i').text.decode('utf-8') elif li.find('a') is not None: new = li.find('a').text.decode('utf-8') else: self.logger.warning('Unable to handle change for %s' % field) changes.append((field, last, new)) update['changes'] = changes params['updates'].append(update) return params class IssueLogTimePage(Page): def logtime(self, hours, message): self.browser.select_form(predicate=lambda form: form.attrs.get('action', '').endswith('/edit')) self.browser['time_entry[hours]'] = '%.2f' % hours self.browser['time_entry[comments]'] = message.encode('utf-8') self.browser['time_entry[activity_id]'] = ['8'] self.browser.submit() class IssueTimeEntriesPage(Page): pass ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/redmine/pages/wiki.py������������������������������������������������������������0000664�0000000�0000000�00000002576�13034501105�0020342�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page class WikiEditPage(Page): def get_source(self): return self.parser.select(self.document.getroot(), 'textarea#content_text', 1).text def set_source(self, data, message): self.browser.select_form(nr=1) self.browser['content[text]'] = data.encode('utf-8') if message: self.browser['content[comments]'] = message.encode('utf-8') self.browser.submit() def get_authenticity_token(self): wiki_form = self.parser.select(self.document.getroot(), 'form#wiki_form', 1) return wiki_form.xpath('div/input')[0].get('value') class WikiPage(Page): pass ����������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/regionsjob/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016432�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/regionsjob/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import RegionsjobModule __all__ = ['RegionsjobModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/regionsjob/browser.py������������������������������������������������������������0000664�0000000�0000000�00000004345�13034501105�0020475�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import SearchPage, AdvertPage, LocationPage __all__ = ['RegionsjobBrowser'] class RegionsjobBrowser(PagesBrowser): search_page = URL('emplois/recherche.html\?.*', SearchPage) advert_page = URL('emplois/(?P<_id>.*)\.html', AdvertPage) location_page = URL('search/getloc\?term=(?P<place>.*)', LocationPage) def __init__(self, website, *args, **kwargs): self.BASEURL = 'http://%s/' % website PagesBrowser.__init__(self, *args, **kwargs) def search_job(self, pattern='', fonction='', secteur='', contract='', experience='', qualification='', enterprise_type='', place=''): params = {'k': pattern.encode('utf-8')} if fonction: params['f'] = fonction if qualification: params['q'] = qualification if contract: params['c'] = contract if experience: params['e'] = experience if secteur: params['s'] = secteur if enterprise_type: params['et'] = enterprise_type if place: location = self.location_page.go(place=place).get_location() params['l'] = location return self.search_page.go(params=params).iter_job_adverts(domain=self.BASEURL) def get_job_advert(self, _id, advert): splitted_id = _id.split('#') self.BASEURL = 'http://www.%s.com/' % splitted_id[0] return self.advert_page.go(_id=splitted_id[1]).get_job_advert(obj=advert) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/regionsjob/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000000736�13034501105�0020573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs����+���tIME/-B��kIDATxڽMA b Qrj )F!9CϷp烙Wog/u]gKl /������������Xh]x,_>^ֻz xjwcK1B 6xDmp/�^23͇粥YmtiGei<!7^`3W_t@SW_t�������u'pLSvASt`<6O A8t8+x*,;03Ý*x5+ӡ4EY3�SěH=�MlJnC DBPN!6Cn�����������������o' l+����IENDB`����������������������������������weboob-1.2/modules/regionsjob/module.py�������������������������������������������������������������0000664�0000000�0000000�00000021777�13034501105�0020307�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.job import CapJob, BaseJobAdvert from .browser import RegionsjobBrowser from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value __all__ = ['RegionsjobModule'] class RegionsjobModule(Module, CapJob): NAME = 'regionsjob' DESCRIPTION = u'regionsjob website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = RegionsjobBrowser website_choices = OrderedDict([(k, u'%s (%s)' % (v, k)) for k, v in sorted({ 'www.centrejob.com': u'CentreJob', 'www.estjob.com': u'EstJob', 'www.nordjob.com': u'NordJob', 'www.ouestjob.com': u'OuestJob', 'www.pacajob.com': u'PacaJob', 'www.parisjob.com': u'ParisJob', 'www.rhonealpesjob.com': u'RhoneAlpesJob', 'www.sudouestjob.com': u'SudOuestJob', 'www.jobtrotter.com': u'JobTrotter', }.iteritems())]) fonction_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'Indifferent', 'Assistanat_admin_accueil': u'Assistanat/Adm.ventes/Accueil', 'BTP_gros_second_oeuvre': u'BTP - Gros Oeuvre/Second Oeuvre', 'Bureau_etude_R_D': u'Bureau d\'Etudes/R & D/BTP archi/conception', 'Commercial_technico_com': u'Commercial - Technico-Commercial', 'Commercial_particulier': u'Commercial auprès des particuliers', 'Commercial_professionnel': u'Commercial auprès des professionnels', 'Commercial_vendeur': u'Commercial-Vendeur en magasin', 'Compta_gestion_finance_audit': u'Compta/Gestion/Finance/Audit', 'Dir_resp_centre_profit': u'Direction/Resp. Co. et Centre de Profit', 'Import_export_inter': u'Import/Export/International', 'Informatique_dev_hard': u'Informatique - Dével. Hardware', 'Informatique_dev': u'Informatique - Développement', 'Informatique_syst_info': u'Informatique - Systèmes d\'Information', 'Informatique_syst_reseaux': u'Informatique - Systèmes/Réseaux', 'Ingenierie_agro_agri': u'Ingénierie - Agro/Agri', 'Ingenierie_chimie_pharma_bio': u'Ingénierie - Chimie/Pharmacie/Bio.', 'Ingenierie_electro_tech': u'Ingénierie - Electro-tech./Automat.', 'Ingenierie_meca_aero': u'Ingénierie - Mécanique/Aéron.', 'Ingenierie_telecom': u'Ingénierie - Telecoms/Electronique', 'Juridique_droit': u'Juridique/Droit', 'Logistique_metiers_transport': u'Logistique/Métiers du Transport', 'Marketing_com_graphisme': u'Marketing/Communication/Graphisme', 'Dir_management_resp': u'Métiers de la distribution - Management/Resp.', 'Metiers_fonction_publique': u'Métiers de la Fonction Publique', 'Negociation_gest_immo': u'Négociation/Gestion immobilière', 'Production_gestion': u'Production - Gestion/Maintenance', 'Production_operateur': u'Production - Opérateur/Manoeuvre', 'Qualite_securite_environnement': u'Qualité/Hygiène/Sécurité/Environnement', 'Restauration_hotellerie_tourisme': u'Restauration/Tourisme/Hôtellerie/Loisirs', 'RH_Personnel_Formation': u'RH/Personnel/Formation', 'Sante_social': u'Santé/Social', 'SAV_Hotline': u'SAV/Hotline/Téléconseiller', 'Services_pers_entreprises': u'Services à la personne/aux entreprises', }.iteritems())]) secteur_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'Indifferent', 'Agri_peche': u'Agriculture/Pêche', 'Banq_assur_finan': u'Banque/Assurance/Finance', 'BTP': u'BTP', 'Distrib_commerce': u'Distribution/Commerce de gros', 'Enseign_forma': u'Enseignement/Formation', 'Immo': u'Immobilier', 'Ind_aero': u'Industrie Aéronautique/Aérospatial', 'Ind_agro': u'Industrie Agro-alimentaire', 'Ind_auto_meca_nav': u'Industrie Auto/Meca/Navale', 'Ind_hightech_telecom': u'Industrie high-tech/Telecom', 'Ind_manufact': u'Industrie Manufacturière', 'Ind_petro': u'Industrie Pétrolière/Pétrochimie', 'Ind_pharma_bio_chim': u'Industrie Pharmaceutique/Biotechn./Chimie', 'Media_internet_com': u'Média/Internet/Communication', 'Resto': u'Restauration', 'Sante_social': u'Santé/Social/Association', 'Energie_envir': u'Secteur Energie/Environnement', 'Inform_SSII': u'Secteur informatique/SSII', 'Serv_public_autre': u'Service public autres', 'Serv_public_collec_terri': u'Service public des collectivités territoriales', 'Serv_public_etat': u'Service public d\'état', 'Serv_public_hosp': u'Service public hospitalier', 'Serv_entreprise': u'Services aux Entreprises', 'Serv_pers_part': u'Services aux Personnes/Particuliers', 'Tourism_hotel_loisir': u'Tourisme/Hôtellerie/Loisirs', 'Transport_logist': u'Transport/Logistique', }.iteritems())]) experience_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ ' ': u'Indifferent', 'Inf_1': u'- 1 an', '1_7': u'1 à 7 ans', 'Sup_7': u'+ 7 ans', }.iteritems())]) contract_choices = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'Tous types de contrat', 'CDD': u'CDD', 'CDI': u'CDI', 'Stage': u'Stage', 'Travail_temp': u'Travail temporaire', 'Alternance': u'Alternance', 'Independant': u'Indépendant', 'Franchise': u'Franchise', }.iteritems())]) qualification_choice = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'Indifferent', 'BEP_CAP': u'BEP/CAP', 'Employe_Operateur': u'Employé/Opérateur/Ouvrier Spe/Bac', 'Technicien_B2': u'Technicien/Employé Bac +2', 'Agent_maitrise_B3': u'Agent de maîtrise/Bac +3/4', 'Ingenieur_B5': u'Ingénieur/Cadre/Bac +5', 'Cadre_dirigeant': u'> Bac + 5 (cadre dirigeant)', }.iteritems())]) enterprise_type_choice = OrderedDict([(k, u'%s' % (v)) for k, v in sorted({ '': u'Tous types d\'entreprises', 'Cabinet_recr': u'Cabinets de recrutement', 'Entreprises': u'Entreprises', 'SSII': u'SSII', 'Travail_temporaire': u'Travail temporaire', }.iteritems())]) CONFIG = BackendConfig(Value('website', label=u'Region', choices=website_choices), Value('place', label='Place', masked=False, default=''), Value('metier', label='Job name', masked=False, default=''), Value('fonction', label=u'Fonction', choices=fonction_choices, default=''), Value('secteur', label=u'Secteur', choices=secteur_choices, default=''), Value('contract', label=u'Contract', choices=contract_choices, default=''), Value('experience', label=u'Experience', choices=experience_choices, default=''), Value('qualification', label=u'Qualification', choices=qualification_choice, default=''), Value('enterprise_type', label=u'Enterprise type', choices=enterprise_type_choice, default='')) def create_default_browser(self): return self.create_browser(self.config['website'].get()) def search_job(self, pattern=''): return self.browser.search_job(pattern=pattern) def advanced_search_job(self): return self.browser.search_job(pattern=self.config['metier'].get(), fonction=self.config['fonction'].get(), secteur=self.config['secteur'].get(), contract=self.config['contract'].get(), experience=self.config['experience'].get().strip(), qualification=self.config['qualification'].get(), enterprise_type=self.config['enterprise_type'].get(), place=self.config['place'].get()) def get_job_advert(self, _id, advert=None): return self.browser.get_job_advert(_id, advert) def fill_obj(self, advert, fields): return self.get_job_advert(advert.id, advert) OBJECTS = {BaseJobAdvert: fill_obj} �weboob-1.2/modules/regionsjob/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000010017�13034501105�0020102�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, pagination, JsonPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import CleanText, Regexp, Env, Date, BrowserURL, Join from weboob.browser.filters.html import CleanHTML, Link from weboob.browser.filters.json import Dict from weboob.capabilities.job import BaseJobAdvert from weboob.exceptions import ParseError from datetime import date, timedelta from weboob.capabilities import NotAvailable class LocationPage(JsonPage): def get_location(self): return Dict('0/value', default='')(self.doc) class SearchPage(HTMLPage): @pagination @method class iter_job_adverts(ListElement): item_xpath = '//section[@class="annonce"]' def next_page(self): return Link('//a[@class="picto picto-nextsmall"]')(self) class item(ItemElement): klass = BaseJobAdvert def condition(self): return Regexp(CleanText('h1/a[@class="lien-annonce"]/@href'), '/emplois/(.*)\.html', default=None)(self) def obj_id(self): site = Regexp(CleanText('h1/a[@class="lien-annonce"]/@href'), 'http://www\.(.*)\.com', default=None)(self) if site is None: site = Regexp(Env('domain'), 'http://www\.(.*)\.com')(self) _id = Regexp(CleanText('h1/a[@class="lien-annonce"]/@href'), '/emplois/(.*)\.html')(self) return u'%s#%s' % (site, _id) obj_title = CleanText('h1/a[2]') obj_society_name = CleanText('figure/span[@itemprop="name"]') obj_place = CleanText('p[@class="inlineblock max-width-75"]') obj_contract_type = CleanText('p[@class="max-width-75"]') def obj_publication_date(self): _date = CleanText('p[@class="infos"]') try: return Date(_date)(self) except ParseError: str_date = _date(self) if 'hier' in str_date: return date.today() - timedelta(days=1) else: return date.today() class AdvertPage(HTMLPage): @method class get_job_advert(ItemElement): klass = BaseJobAdvert obj_description = Join('\n', '//div[@id="annonce-detail"]/p[@class="text"]', textCleaner=CleanHTML) obj_id = Env('_id') obj_url = BrowserURL('advert_page', _id=Env('_id')) obj_publication_date = Date(Regexp(CleanText('//div[@id="annonce-detail"]/p[@class="infos"]'), '(\d{2}/\d{2}/\d{4})', default=NotAvailable), default=NotAvailable) obj_title = CleanText('//div[@id="annonce"]/div/div/h1') obj_society_name = CleanText('//section[@class="entp-resume"]/h1/a') obj_contract_type = CleanText('//dl[@class="infos-annonce"]/dt[span[@class="picto picto-contrat-grey"]]/following-sibling::dd[1]') obj_place = CleanText('//dl[@class="infos-annonce"]/dt[span[@class="picto picto-geolocalisation-grey"]]/following-sibling::dd[1]') obj_pay = CleanText('//div[@id="annonce-detail"]/p[@class="infos"]/preceding-sibling::p[1]', replace=[('Salaire : ', '')]) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/regionsjob/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002674�13034501105�0017774�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest import itertools class RegionsjobTest(BackendTest): MODULE = 'regionsjob' def test_regionjob_search(self): l = list(itertools.islice(self.backend.search_job(u'informaticien'), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) def test_regionjob_advanced_search(self): l = list(itertools.islice(self.backend.advanced_search_job(), 0, 20)) assert len(l) advert = self.backend.get_job_advert(l[0].id, None) self.assertTrue(advert.url, 'URL for announce "%s" not found: %s' % (advert.id, advert.url)) ��������������������������������������������������������������������weboob-1.2/modules/residentadvisor/�����������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017476�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/residentadvisor/__init__.py������������������������������������������������������0000664�0000000�0000000�00000001465�13034501105�0021615�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Alexandre Morignot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import ResidentadvisorModule __all__ = ['ResidentadvisorModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/residentadvisor/browser.py�������������������������������������������������������0000664�0000000�0000000�00000007747�13034501105�0021552�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Alexandre Morignot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, EventPage, ListPage, SearchPage from datetime import datetime class ResidentadvisorBrowser(LoginBrowser): BASEURL = 'http://www.residentadvisor.net' # this ID is used by Resident Advisor ALBANIA_ID = 223 login = URL('https://www.residentadvisor.net/login', LoginPage) event = URL('/event.aspx\?(?P<id>\d+)', EventPage) list_events = URL('/events.aspx\?ai=(?P<city>\d+)&v=(?P<v>.+)&yr=(?P<year>\d{4})&mn=(?P<month>\d\d?)&dy=(?P<day>\d\d?)', ListPage) search_page = URL('/search.aspx\?searchstr=(?P<query>.+)§ion=events&titles=1', SearchPage) attends = URL('/Output/addhandler.ashx') def do_login(self): self.login.stay_or_go() self.page.login(self.username, self.password) # in case of successful connection, we are redirected to the home page if self.login.is_here(): raise BrowserIncorrectPassword() def get_events(self, city, v = 'week', date = datetime.now()): self.list_events.go(v = v, year = date.year, month = date.month, day = date.day, city = city) assert self.list_events.is_here() for event in self.page.get_events(): yield event def get_event(self, _id): self.event.go(id = _id) if not self.event.is_here(): return None event = self.page.get_event() event.id = _id event.url = self.event.build(id = _id) return event def search_events_by_summary(self, pattern): self.search_page.go(query = pattern) assert self.search_page.is_here() for event in self.page.get_events(): yield event def get_country_city_id(self, country, city): now = datetime.now() self.list_events.go(v = 'day', year = now.year, month = now.month, day = now.day, city = self.ALBANIA_ID) assert self.list_events.is_here() country_id = self.page.get_country_id(country) if country_id is None: return None self.list_events.go(v = 'day', year = now.year, month = now.month, day = now.day, city = country_id) assert self.list_events.is_here() city_id = self.page.get_city_id(city) if city_id is None: return None return city_id def get_city_id(self, city): now = datetime.now() country_id = self.ALBANIA_ID city_id = None while True: self.list_events.go(v = 'day', year = now.year, month = now.month, day = now.day, city = country_id) assert self.list_events.is_here() city_id = self.page.get_city_id(city) country_id = self.page.get_country_id_next_to(country_id) # city_id != None => city found # country_id = None => no more country, city not found if city_id is not None or country_id is None: break return city_id @need_login def attends_event(self, id, is_attending): data = {'type': 'saveFavourite', 'action':'attending', 'id': id} if not is_attending: data['type'] = 'deleteFavourite' self.attends.open(data = data) �������������������������weboob-1.2/modules/residentadvisor/favicon.png������������������������������������������������������0000664�0000000�0000000�00000001454�13034501105�0021635�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME %)d���iTXtComment�����Created with GIMPd.e��IDATx횽A{ rX=V ? ^!Xb#b)z F/ **xy0ff6}Y6of~ޛ7F@%�P� @(��P� @(I% pJj[6{BG7 MC0ԀrYsop�p 8 ,:\˺q �ve1Z @sc8h˽_SMk`/ؚ19cR~φ'Wbk2@xCZNR@8!l'hC)a| lr!0\`,v1INH*Pn;<&a`\57q]!DF:#kz/wFQ+ GwVOdEp椸i4 o�C $ 6l0 < Alymr,1Od!-0pn-?ݳ5z,z=@.0*$MN*AB(O=.s' 9:jhӖ4+rX?V� @(��P� @(��P�%f% ����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/residentadvisor/module.py��������������������������������������������������������0000664�0000000�0000000�00000011614�13034501105�0021340�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Alexandre Morignot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.backend import BackendConfig from weboob.capabilities.calendar import CapCalendarEvent, BaseCalendarEvent, CATEGORIES from .browser import ResidentadvisorBrowser from datetime import timedelta __all__ = ['ResidentadvisorModule'] class ResidentadvisorModule(Module, CapCalendarEvent): NAME = 'residentadvisor' DESCRIPTION = u'residentadvisor website' MAINTAINER = u'Alexandre Morignot' EMAIL = 'erdnaxeli@cervoi.se' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = ResidentadvisorBrowser CONFIG = BackendConfig(Value('username', label='Username or email', default=''), ValueBackendPassword('password', label='Password', default=''), Value('country', required=True), Value('city', required=True)) ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT] _city_id = None @property def city_id(self): if not self._city_id: self._city_id = self.browser.get_country_city_id( country = self.config['country'].get(), city = self.config['city'].get()) return self._city_id def create_default_browser(self): password = None username = self.config['username'].get() if len(username) > 0: password = self.config['password'].get() return self.create_browser(username, password) def attends_event(self, event, is_attending): """ Attends or not to an event :param event : the event :type event : BaseCalendarEvent :param is_attending : is attending to the event or not :type is_attending : bool """ self.browser.attends_event(event.id, is_attending) def get_event(self, _id): """ Get an event from an ID. :param _id: id of the event :type _id: str :rtype: :class:`BaseCalendarEvent` or None is fot found. """ return self.browser.get_event(_id) def list_events(self, date_from, date_to): """ list coming event. :param date_from: date of beguinning of the events list :type date_from: date :param date_to: date of ending of the events list :type date_to: date :rtype: iter[:class:`BaseCalendarEvent`] """ # we check if date_to is defined try: date_to.date() except: # default is week date_to = date_from + timedelta(days = 7) delta = date_to - date_from while delta.days >= 0 : v = 'week' if delta.days > 7: v = 'month' for event in self.browser.get_events(v = v, date = date_from, city = self.city_id): if event.start_date <= date_to: yield event if v == 'week': date_from += timedelta(days = 7) else: date_from += timedelta(days = 30) delta = date_to - date_from def search_events(self, query): """ Search event :param query: search query :type query: :class:`Query` :rtype: iter[:class:`BaseCalendarEvent`] """ if not self.has_matching_categories(query): return events = None if query.city: # FIXME # we need the country to search the city_id in an efficient way city_id = self.browser.get_city_id(query.city) events = self.browser.get_events(city = city_id) elif query.summary: events = self.browser.search_events_by_summary(query.summary) else: events = self.list_events(query.start_date, query.end_date) for event in events: event = self.fillobj(event, ['ticket']) if event.ticket in query.ticket: yield event def fill_event(self, event, fields): if set(fields) & set(('end_date', 'price', 'description', 'ticket')): return self.get_event(event.id) return event OBJECTS = {BaseCalendarEvent: fill_event} ��������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/residentadvisor/pages.py���������������������������������������������������������0000664�0000000�0000000�00000011410�13034501105�0021144�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Alexandre Morignot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.calendar import CATEGORIES, STATUS, TICKET from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.html import Attr, CleanHTML, Link from weboob.browser.filters.standard import CleanDecimal, CleanText, Date, CombineDate, DateTime, Regexp, Time, Type from weboob.browser.pages import HTMLPage from weboob.capabilities.calendar import BaseCalendarEvent from datetime import timedelta class BasePage(HTMLPage): @property def logged(self): return bool(self.doc.xpath('//li[@id="profile"]/span[contains(text(), "Welcome")]')) class LoginPage(BasePage): def login(self, username, password): form = self.get_form() form['UsernameOrEmailAddress'] = username form['Password'] = password form.submit() class ListPage(BasePage): @method class get_events(ListElement): item_xpath = '//ul[@id="items"]/li/article' class item(ItemElement): klass = BaseCalendarEvent obj_url = Link('./div[@class="bbox"]/h1/a') obj_id = Regexp(Link('./div[@class="bbox"]/h1/a'), r'aspx\?(.+)') obj_location = CleanText('./div[@class="bbox"]/span/a') obj_start_date = DateTime(Attr('.//time', 'datetime')) obj_summary = Regexp(Attr('./div[@class="bbox"]/h1/a', 'title'), r'details of (.+)') obj_category = CATEGORIES.CONCERT obj_status = STATUS.CONFIRMED def get_country_id(self, country): return Regexp(Link('//li[@id="liCountry"]/ul/li/a[./text()="%s"]' % country, default=''), r'ai=([^&]+)&?', default=None)(self.doc) def get_city_id(self, city): return Regexp(Link('//li[@id="liArea"]/ul/li/a[./text()="%s"]' % city, default=''), r'ai=([^&]+)&?', default=None)(self.doc) def get_country_id_next_to(self, country_id): return Regexp(Link('//li[@id="liCountry"]/ul/li[./a[contains(@href, "ai=%s&")]]/following-sibling::li/a' % country_id, default=''), r'ai=([^&]+)&?', default=None)(self.doc) class EventPage(BasePage): @method class get_event(ItemElement): klass = BaseCalendarEvent obj_summary = CleanText('//div[@id="sectionHead"]/h1') obj_description = CleanHTML('//div[@id="event-item"]/div[3]/p[2]') obj_price = CleanDecimal(Regexp(CleanText('//aside[@id="detail"]/ul/li[3]'), r'Cost /[^\d]*([\d ,.]+).', default=''), default=None) obj_location = Regexp(CleanText('//aside[@id="detail"]/ul/li[2]'), r'Venue / (.+)') obj_booked_entries = Type(CleanText('//h1[@id="MembersFavouriteCount"]'), type=int) obj_status = STATUS.CONFIRMED obj_category = CATEGORIES.CONCERT _date = Date(CleanText('//aside[@id="detail"]/ul/li[1]/a[1]')) def obj_start_date(self): start_time = Time(Regexp(CleanText('//aside[@id="detail"]/ul/li[1]'), r'(\d{2}:\d{2}) -'))(self) return CombineDate(self._date, start_time)(self) def obj_end_date(self): end_time = Time(Regexp(CleanText('//aside[@id="detail"]/ul/li[1]'), r'- (\d{2}:\d{2})'))(self) end_date = CombineDate(self._date, end_time)(self) if end_date > self.obj_start_date(): end_date += timedelta(days = 1) return end_date def obj_ticket(self): li_class = Attr('//li[@id="tickets"]//li[1]', 'class', default=None)(self) if li_class: if li_class == 'closed': return TICKET.CLOSED else: return TICKET.AVAILABLE return TICKET.NOTAVAILABLE class SearchPage(BasePage): @method class get_events(ListElement): item_xpath = '//main/ul/li/section/div/div/ul/li' class item(ItemElement): klass = BaseCalendarEvent obj_url = Link('./a[1]') obj_id = Regexp(Link('./a[1]'), r'\?(\d+)') obj_summary = CleanText('./a[1]') obj_start_date = Date(CleanText('./span[1]')) obj_category = CATEGORIES.CONCERT obj_status = STATUS.CONFIRMED ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/residentadvisor/test.py����������������������������������������������������������0000664�0000000�0000000�00000003105�13034501105�0021026�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Alexandre Morignot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.calendar import Query from datetime import datetime, timedelta class ResidentadvisorTest(BackendTest): MODULE = 'residentadvisor' def test_searchcity(self): query = Query() query.city = u'Melbourne' self.assertTrue(len(list(self.backend.search_events(query))) > 0) event = self.backend.search_events(query).next() self.assertTrue(self.backend.get_event(event.id)) def test_datefrom(self): query = Query() later = (datetime.now() + timedelta(days=31)) query.start_date = later event = self.backend.search_events(query).next() self.assertTrue(later.date() <= event.start_date.date()) event = self.backend.get_event(event.id) self.assertTrue(later.date() <= event.start_date.date()) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015237�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/TODO������������������������������������������������������������������������0000664�0000000�0000000�00000000132�13034501105�0015723�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������Add following pseudo channels: most-viewed most-commented count on latest add API_KEY ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001430�13034501105�0017346�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import RmllModule __all__ = ['RmllModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/browser.py������������������������������������������������������������������0000664�0000000�0000000�00000004670�13034501105�0017303�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.browser.exceptions import HTTPNotFound from .pages import RmllCollectionPage, RmllVideoPage, RmllChannelsPage, RmllSearchPage, RmllLatestPage __all__ = ['RmllBrowser'] class RmllBrowser(PagesBrowser): BASEURL = 'http://video.rmll.info' index_page = URL(r'channels/content/(?P<id>.+)', RmllCollectionPage) latest_page = URL(r'api/v2/latest/', RmllLatestPage) video_page = URL(r'permalink/(?P<id>.+)/', RmllVideoPage) channels_page = URL(r'api/v2/channels/content/\?parent_oid=(?P<oid>.*)', RmllChannelsPage) search_page = URL(r'api/v2/search/\?search=(?P<pattern>.+)', RmllSearchPage) def __init__(self, *args, **kwargs): self.channels = None PagesBrowser.__init__(self, *args, **kwargs) @video_page.id2url def get_video(self, url, video=None): self.location(url) assert self.video_page.is_here() video = self.page.get_video(obj=video) return video def search_videos(self, pattern): url = self.search_page.build(pattern=pattern) self.location(url) return self.page.iter_resources() def get_latest_videos(self): url = self.latest_page.build() self.location(url) assert self.latest_page.is_here() return self.page.iter_resources() def get_channel_videos(self, split_path): oid = '' if len(split_path) > 0: oid = split_path[-1] try: url = self.channels_page.build(oid=oid) self.location(url) assert self.channels_page.is_here() for video in self.page.iter_resources(split_path): yield video except HTTPNotFound: pass ������������������������������������������������������������������������weboob-1.2/modules/rmll/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000011171�13034501105�0017373�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���J���M������sRGB����bKGD������ pHYs��M��Mħ���tIME3GA��IDATx{p]uʒcI<mI40H \bPJ L(BxSBDCZBhH %9N4 F?=孭}-:3gַ ,-j{DF(O 뀊TVG( 1 4#PjnU]'S;׆y2,x@ʀӀ3=l9g�O?5\|�٦;γୈo6�0Ϥ2p ?~q>/DNj1l&v'v< 3~$�5I�[LY`iom�6ڌuOyo&p?�]ZRp@[-v@�x=äS<7b3 |yw�T,& ,1U8y'e X lM>55A&oFS?ya�ߜ9� ]b<2+K;Qs[.^3W n`֏dP#O^t4u`UT&�:Y[X7ع�n�N?6CSzf\b5V!"U/"LjW4#"cEBU+< fUݬ[7D!yRUQՏ 5{Q =L/ B; O qjaNMJ_fR/\`UEdۥEDû]FUIU-?Qէ_ g^Tr7_ӥU[vc1?vlHX+"gIDUUنTTU ȋ&yXzOװ盌Ro zX@2o0i:p""clqj W @H_�DDH#^({´bm:X 2f{*?4O:v7� ל $FUWLŜ4RK�U$I.vDxs&ѱ=jksbw "۟TK{M @<b-^ewj$y'q8u< xȢ~4hX@�_ʑ>ySkUe$ר5!84*@K;~[vE.U~7,4Mȅ u $`4qqmJ(91.Ł *U;I7= "_wZhnk^lj9=is2Lj]MU3z{HvC- <j_ ?|nWhQXybmGwK{ ]4u:3ZD.w`>ow9umj7:8PuԲh0_ׁlP(,"r w>G՟lZVJ9_mLUW/:Fi}ODW$Ad\%AZi`@%FD;5s$O4O<`+EخX۱VD4} H97BēkUHZSn@MVH"ܩRs[UjxQuNBjmFлf 7<Z56ė˖~ڢ\U'MHUUbZu`o֪yÌMuˋ1"2ߝ3~@Yd\$l@}K}sDdpOK{ n!ͭ}rXߵҺjK֪It:M�XCF#MK-SDT̷UnZ7VPm#u"r%Мx?0I@Xy+M#+j%b螠9/ [h$=mǺ�](NŅ0j#964M!nUpM6$j:Y6D`aW�!j܀jӁ4<*v^Ùǽȃ<`U?[1@fFp!P"2k% sJ;8RU=XDlj:r*vUoco=+ei91r}JUz&#IWbh!i勁P-"ש$w<}k<FU۪}k$jkDMPdN_D6DΒ(/P&"׷׬+ve.cj=E8s?M,IUu%Xo@U2zꀜwqQPR40)R4M{\2we<a m e~sKP<&L\N6 H⊗ބEҮ*">YGsۤX%ټyE_ -IxcKԖt\QbkvqVlVݖ 7U3SVY\J >c>!Á)"+D䍜M)[АBTw H2iZ/5Iz~ y@.O PI8x3�/I)T^wrl/F@�}d> NDqUi8w:V b_NP$"'xN;gt/k;4@UuN!"rq^Ao=k[Hx_´P�пm YF?֪- I�xK{LU} Y +$0К}Zjd RÈU]"2:II3m"Xj=Ƚg4Upt[{_U}iw=d0x9>p Ǿ/" 쀚i Yu)C5U'D5\so/| %#Tӄ/ne@U5]vzFU-v#*\i8>x˖jzJ % ˜16X.V}E>IŪWB.Uk]:+KSK{ I8u;RD~S976cAB@?&C�ވȚJ,Wz<Q{1ᤃNP 뻖Ux!K udR3ϒ5m %*5+znYfӨivyLs_z~c}q*Zu7b ~z xyq[ye;J8k;\UC&U+0 ';\$Vs̺DWq%_Xߵ8 vKF?6Yۢ z}W~J:k򸪾_X^:N;v-^𢓦8/]FE&f{is\s(cyX۱<=p$".,S{F ]usepJf-lwH )4|ÉqEd桃ZS_E۪};sjy\Bv�ޯdpCTK~w]w"r.> I\Z5]G5"}U$4l* @VU4M7 uG|.Y]Dֈ5d]WKֵV%ڎ_"դoz@NZk�$U}DU�7sXED:UwI٤_|y]I1]Ow׼E%"g8wn'זJB�' dyOs wTQ+jֳlKL<}^EX{ v;mО "FDp3]P~�@c󼪡sI8u F=rS;,�x'iU9<`Fyɖi49:5zLUUIuUERS[U/}@"wޯnǂɶKzz4Mwx1iUIVq^(2<G>7EX۱2"E}?sk)\OXA<s6?!d%"'w!"3Xq6%"?V#Sm>59e8wm0oR6yQgD�KNBl%+ei.:Orsf!ҿ+-4M%yRKOGjAU%I4MK@)IR4ع "Uj.CmWeRn_ ł�x8O}{3N%)f)JL%s.4_fVd+kx?{5]tL{X <h lc߆*F6E�F;4W +6]c r,1>[aVMpW7 l^ numv686[2B�l dϷbTm%9+nT` mUǤ;?U5 C,6[fm#vTZ|7@-l8ҷm;<q >Iֺ|Y|eIһw5͚gl<N|2,f_&u70p26����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000005565�13034501105�0017111�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, Collection from weboob.tools.backend import Module from .browser import RmllBrowser from .video import RmllVideo __all__ = ['RmllModule'] class RmllModule(Module, CapVideo, CapCollection): NAME = 'rmll' # The name of module MAINTAINER = u'Guyou' # Name of maintainer of this module EMAIL = 'guilhem.bonnefille@gmail.com' # Email address of the maintainer VERSION = '1.2' # Version of weboob DESCRIPTION = 'Videos from RMLL' # Description of your module LICENSE = 'AGPLv3+' # License of your module BROWSER = RmllBrowser def create_default_browser(self): return self.create_browser() def get_video(self, _id): self.logger.debug("Getting video for %s", _id) return self.browser.get_video(_id) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern) def fill_video(self, video, fields): self.logger.debug("Fill video %s for fields %s", video.id, fields) if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def iter_resources(self, objs, split_path): if BaseVideo in objs: if len(split_path) == 0: # Add fake Collection yield Collection(['latest'], u'Latest') if len(split_path) == 1 and split_path[0] == 'latest': for video in self.browser.get_latest_videos(): yield video else: channel_videos = self.browser.get_channel_videos(split_path) if channel_videos: for content in channel_videos: yield content OBJECTS = {RmllVideo: fill_video} �������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/pages.py��������������������������������������������������������������������0000664�0000000�0000000�00000012775�13034501105�0016724�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage, JsonPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Regexp, Format, CleanText, Duration, DateTime, Filter from weboob.browser.filters.html import Link, XPath, CleanHTML from weboob.browser.filters.json import Dict from weboob.capabilities import NotAvailable, NotLoaded from weboob.capabilities.image import Thumbnail from weboob.capabilities.collection import Collection from .video import RmllVideo BASE_URL = 'http://video.rmll.info' class NormalizeThumbnail(Filter): def filter(self, thumbnail): if not thumbnail.startswith('http'): thumbnail = BASE_URL + thumbnail if thumbnail == "http://rmll.ubicast.tv/statics/mediaserver/images/video_icon.png": # This is the default: remove it as any frontend default should be better thumbnail = None return thumbnail class RmllDuration(Duration): _regexp = re.compile(r'((?P<hh>\d+) h )?((?P<mm>\d+) m )?(?P<ss>\d+) s') kwargs = {'hours': 'hh', 'minutes': 'mm', 'seconds': 'ss'} def create_video(metadata): video = RmllVideo(metadata['oid']) video.title = unicode(metadata['title']) video.date = DateTime(Dict('creation'), default=NotLoaded)(metadata) video.duration = RmllDuration(Dict('duration', default=''), default=NotLoaded)(metadata) thumbnail = NormalizeThumbnail(Dict('thumb'))(metadata) video.thumbnail = Thumbnail(thumbnail) video.thumbnail.url = video.thumbnail.id video.url = NotLoaded return video class RmllVideoPage(HTMLPage): @method class get_video(ItemElement): klass = RmllVideo obj_id = CleanHTML('/html/head/meta[@property="og:url"]/@content') & CleanText() & Regexp(pattern=r'.*/permalink/(.+)/$') obj_title = Format(u'%s', CleanHTML('/html/head/meta[@name="DC.title"]/@content') & CleanText()) obj_description = Format(u'%s', CleanHTML('/html/head/meta[@property="og:description"]/@content') & CleanText()) def obj_thumbnail(self): url = NormalizeThumbnail(CleanText('/html/head/meta[@property="og:image"]/@content'))(self) if url: thumbnail = Thumbnail(url) thumbnail.url = thumbnail.id return thumbnail obj_duration = CleanText('/html/head/script[not(@src)]') & Regexp(pattern=r'media_duration: ([^,.]+),?.*,', default='') & Duration(default=NotAvailable) def obj_url(self): links = XPath('//div[@id="tab_sharing_content"]/div/div/div[@class="paragraph"]/div[@class="share"]/a[@target="_blank"]/@href')(self) for link in links: ext = str(link).split('.')[-1] self.logger.debug("Link:%s Ext:%s", link, ext) if ext in ['mp4', 'webm']: return unicode(link) class RmllCollectionPage(HTMLPage): @method class iter_videos(ListElement): item_xpath = '//div[@class="item-entry type-video " or @class="item-entry type-vod "]' class item(ItemElement): klass = RmllVideo obj_id = Link('a') & Regexp(pattern=r'.*/videos/(.+)/$') obj_title = Format(u'%s', CleanHTML('a/span/span/span[@class="item-entry-title"]') & CleanText()) obj_url = NotLoaded #obj_date = XPath('a/span/span/span[@class="item-entry-creation"]') obj_duration = CleanText('a/span/span/span[@class="item-entry-duration"]') & RmllDuration() def obj_thumbnail(self): thumbnail = NormalizeThumbnail(CleanText('a/span[@class="item-entry-preview"]/img/@src'))(self) if thumbnail: thumbnail = Thumbnail(thumbnail) thumbnail.url = thumbnail.id return thumbnail class RmllChannelsPage(JsonPage): def iter_resources(self, split_path): if 'channels' in self.doc: for metadata in self.doc['channels']: collection = Collection(split_path+[metadata['oid']], metadata['title']) yield collection if 'videos' in self.doc: for metadata in self.doc['videos']: video = create_video(metadata) yield video class RmllLatestPage(JsonPage): def iter_resources(self): for metadata in self.doc['items']: if metadata['type'] == 'c': collection = Collection([metadata['oid']], metadata['title']) yield collection if metadata['type'] == 'v': video = create_video(metadata) yield video class RmllSearchPage(JsonPage): def iter_resources(self): if 'videos' in self.doc: for metadata in self.doc['videos']: video = create_video(metadata) yield video ���weboob-1.2/modules/rmll/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000005161�13034501105�0016573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class RmllTest(BackendTest): MODULE = 'rmll' def test_video_search(self): videos = self.backend.search_videos('test') self.assertTrue(videos) for video in videos: self.assertTrue(video.id, 'ID for video not found') def test_video_page(self): for slug in ["v124f0bc409e704d92cf", "http://video.rmll.info/permalink/v124f0bc409e704d92cf/"]: video = self.backend.browser.get_video(slug) self.assertTrue(video.id, 'ID for video not found') self.assertTrue(video.url, 'URL for video "%s" not found' % (video.id)) self.assertTrue(video.thumbnail, 'Thumbnail for video "%s" not found' % (video.id)) self.assertTrue(video.title, 'Title for video "%s" not found' % (video.id)) #self.assertTrue(video.description, 'Description for video "%s" not found' % (video.id)) self.assertTrue(video.duration, 'Duration for video "%s" not found' % (video.id)) #help(video) def test_video_fill(self): slug = "v124f0bc409e704d92cf" video = self.backend.browser.get_video(slug) video = self.backend.fill_video(video, ["url"]) self.assertTrue(video) self.assertTrue(video.url, 'URL for video "%s" not found' % (video.id)) def test_browse(self): for path in [[], ['latest']]: videos = self.backend.iter_resources([BaseVideo],path) self.assertTrue(videos) for video in videos: self.assertTrue(video.id, 'ID for video not found') def test_missing_duration(self): videos = self.backend.search_videos('weboob') self.assertTrue(videos) for video in videos: self.assertTrue(video.id, 'ID for video not found') video = self.backend.fill_video(video, ["$full"]) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/rmll/video.py��������������������������������������������������������������������0000664�0000000�0000000�00000001744�13034501105�0016725�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Guilhem Bonnefille # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class RmllVideo(BaseVideo): @classmethod def id2url(cls, _id): if _id.startswith('http'): return _id else: return 'http://video.rmll.info/permalink/%s/' % (_id) ����������������������������weboob-1.2/modules/s2e/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0014762�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001426�13034501105�0017076�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import S2eModule __all__ = ['S2eModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000011325�13034501105�0017021�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountsPage, AMFCodePage, HistoryPage, ErrorPage class S2eBrowser(LoginBrowser): login = URL('/portal/salarie-(?P<slug>\w+)/authentification', '/portal/j_security_check', LoginPage) accounts = URL('/portal/salarie-(?P<slug>\w+)/monepargne/mesavoirs\?language=(?P<lang>)', '/portal/salarie-(?P<slug>\w+)/monepargne/mesavoirs', AccountsPage) amfcode = URL('https://www.assetmanagement.hsbc.com/feedRequest', AMFCodePage) history = URL('/portal/salarie-(?P<slug>\w+)/operations/consulteroperations', HistoryPage) error = URL('/maintenance/HSBC/', ErrorPage) def __init__(self, *args, **kwargs): self.secret = None if 'secret' in kwargs: self.secret = kwargs['secret'] del kwargs['secret'] super(S2eBrowser, self).__init__(*args, **kwargs) self.cache = {} self.cache['invs'] = {} self.cache['trs'] = {} self.cache['details'] = {} def do_login(self): self.login.go(slug=self.SLUG).login(self.username, self.password, self.secret) if self.login.is_here(): error = self.page.get_error() raise BrowserIncorrectPassword(error) @need_login def iter_accounts(self): if 'accs' not in self.cache.keys(): self.accounts.stay_or_go(slug=self.SLUG, lang=self.LANG) # weird wrongpass if not self.accounts.is_here(): raise BrowserIncorrectPassword() multi = self.page.get_multi() if len(multi): # Handle multi entreprise accounts accs = [] for id in multi: self.page.go_multi(id) for a in self.accounts.go(slug=self.SLUG).iter_accounts(): a._multi = id accs.append(a) else: accs = [a for a in self.page.iter_accounts()] self.cache['accs'] = accs return self.cache['accs'] @need_login def iter_investment(self, account): if account.id not in self.cache['invs']: self.accounts.stay_or_go(slug=self.SLUG) # Handle multi entreprise accounts if hasattr(account, '_multi'): self.page.go_multi(account._multi) self.accounts.go(slug=self.SLUG) # Select account self.page.get_investment_pages(account.id) invs = [i for i in self.page.iter_investment()] # Get page with quantity self.page.get_investment_pages(account.id, False) self.cache['invs'][account.id] = self.page.update_quantity(invs) return self.cache['invs'][account.id] @need_login def iter_history(self, account): if account.id not in self.cache['trs']: self.history.stay_or_go(slug=self.SLUG) # Handle multi entreprise accounts if hasattr(account, '_multi'): self.page.go_multi(account._multi) self.history.go(slug=self.SLUG) # Get more transactions on each page self.page.show_more("50") trs = [t for t in self.page.iter_history(accid=account.id)] self.cache['trs'][account.id] = trs # Go back to first page self.page.go_start() return self.cache['trs'][account.id] class EsaliaBrowser(S2eBrowser): BASEURL = 'https://salaries.esalia.com' SLUG = 'sg' LANG = 'fr' # ['fr', 'en'] class CapeasiBrowser(S2eBrowser): BASEURL = 'https://www.capeasi.com' SLUG = 'axa' LANG = 'fr' # ['fr', 'en'] class ErehsbcBrowser(S2eBrowser): BASEURL = 'https://epargnant.ere.hsbc.fr' SLUG = 'hsbc' LANG = 'fr' # ['fr', 'en'] class BnppereBrowser(S2eBrowser): BASEURL = 'https://personeo.epargne-retraite-entreprises.bnpparibas.com' SLUG = 'bnp' LANG = 'fr' # ['fr', 'en'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000015266�13034501105�0017127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME (���iTXtComment�����Created with GIMPd.e��IDATx{Y\i~Yb͌tfznDh#Fsp@nwH\sŏpFH# ajg\.[ڹFzo;]{5H<q|<~s�6{{W/mn]nnvzRg8Y2;Ofxx4gŋgLJOO�8PoTƒ�ԥ7UI/ۖ@Q'f2w٤oq!zJ=(nf-U#2:B)4ݠ۸mk6c|Dx7:3ENdt'e>�*�Wm@naܗ2Bnc; lczqiu6;h$�ID1&9s.眈(Fu66bU1͹0BJc4Te>&cs928 IOF�^U�8dK]ڹrCV' ͧfm"\vUoE(nFqB�"#"F�c >~%(ENl>*,ageYh�a8ٽ<L9yUfxNOգwʗO�0�h;A.6bVgs Y�+ Hc1*s@u`--,10a'"BEUs-J9eI2ͳ9wʈ 9R:t7S_CmLn*Qs+[[a%U8 潹yj�b G`|3`WO>"־y9 C3EUHY't>:\Wzdt5fo?4Tv2iJiAc<d"h vȁYzkmQޡ_Gc "/,yYd)`RJ48*b& 7ZƠ*oڣ,MӪLUiT:G"1eXk|1DbD8E| Sm?V2Mjc/qiS8g�,1I!+Z )"(n6[([AgZqιR) ݰYs0Ƣ,2Mt:4ό10BK 2t10s'o0D hE"WEX Ƙ (ja5ōfh9LjqΥ#c uι c)% Ee(تTJ$ c 10#r:k5Ysf8gqZc3pN1s)$81N`H!5ђy@>ʬ:o.:s`L͎ F&W@$sLk<Og8M(K]im %d2hpb|IXmUiY2Ϧ֘1΢(۝FFqъv�` �c .$;g];qB$sdgmm ID r=׮g-1iHeJWpy@L(8k΋ls*8c>F�g @ )Dh d뜃3vp"% c#kD',~+t`Bِqcm *c$jblxӼ(֥oGqs+P*n4Z 8KD9g 8g:gKos)7˭��a�s7pL(.X*Ϫ*$~9BHbR _hg/ܫN#a#uy5BD 1Z40ց1G,*jqYrnAnu]w ZKd#]l AJ[?)|gA^jT@9ƹBI{]&}E4" ^ Hv`P# a$G!y-_<rT"Z'{!|iy|. <A:PΘץ(Yip{3CIP ]0RJ<{H5Xg`Np5y-?!"V%<9hcADHPqW0_kB0%9`p�M2֬5,;"V,k9"S !jGHG( HJ" 88cP3BBŐ5 9W`!VsK0QpNppuV(C%XQ#Ru( CM7+[/z04B8`(J kr k0ƒٲ L%$'kZ_ua4Cڍ/^ _!%hS`�EDk:Js(%!g@Yi|0fTm<%6!5"`!!8-ph 5b3"pÉO(luDp .fU ""k>@-[bO^L�h a j4q4rS1��m,-`D8䕡1c'Yf @q�2@m"@kk RZ[x3·m,l]w(d{0>Yh)XWR_\3Rh5(! �EekimP kkmu(jٽ�UӅ & Ku14Bf(}:809X<yYV~:<Z[8p+$DW(`�B3yAoAAWyi`(,-CJ9hkw[ (AQid_�H+0Ҧõ"XkFrqy^֠�!TF)`b21J8g܅ƢI^!+*D\Ak,A F8 KB3RJdE|hKɐW,+̳`4BġY%XRvB yZ`P3y>_|<G[p. (˧@Њ>*mF^TH qjH0FB8>;ߍYa2) ]6Q|#p YVQ()|0O2< ;}Xd9F9J(Fj@p3(BF$I Nx{k1FPB 8 y�hI 4)Ւ2`vPe%//pc<}q MTO zm()A<~~?){xwA�@H(%|Y-bFG>Bq S\/0+?)9dfi c-f9`</Pfħ79cAo~)#K|_<;p4CPR�EQ!KYsJ 4f@rnƪٌ@uGJ4#(``dT +#_hkQUE8AIv+ġDQ G@e IYZ`1Ww?kcd8:ps<~~ycAoic8gOppx4+pp4' r70w1u]ֻo@zf3U[VI6[w6Kؼޢq W W`! us�^sX$YPj B10[チd,i 7! N$Y'xVN!+`:K {h7ChIӄ6 jyVz&buf;独(hDJ 6P!/Z0=x 6b|>FS6�ŕwzȋ;}lmv`-D<!{Onob෿6hbhmpr6t*0vYbR&<ɽ3ଭE` ь�S+�RxHV碷Pi/X1akKW$f; ս-l7w<g ioqm@ ch1<iV`2M0uЌ#nm7m(*<}qv3B$@aD*0 6#^.f@7oRZkQ>yJ 瀢|-Gsb!?] yyV5OO;on{owpeo ^YH!%Ƴ_.\(+;ݺ vZ:ȗ́ۊ4� %H/NJGxX;y[dhеRD!|3|ps4!6:Mp{ѓCq0P*4AAk(PB׻^^I@+#8AWss<?e(Kw�Kew<h"M~yxzOp>/}Jk|oc�O#{�~gS&3<?<: 6/a}3GH"|88LZEcfubs1 KD<+`1q%bn7ÕK[l"rܾCR;߼ �8:88<7\NyN10K2R`4nbw.AV/ӌқshc1e'9t<"8_&Wo,Jjj}E(iR, CH;[[W%R1?ѝh7cs ~2>3Y_gO0MέgquW�yopA(*#̳@YȎѳ^,`؆< `P�G?>>V~Dx!8xq4D^'s|[޷>HΗi<58)d5Yi$+a0rM[m=?X#@,0ڢ t޼q 'FD#ev (ğR֍}~~ _MV)ʃN?@HQpiJkc%`4܂rBbm ^l YDXup�8F+>c00&8Ni5p8g}1ΐzϽE)8ƓeUAJn Wx ,/V:d:6]z-ݫCzzu{A\;+*m0ewpr:³'Olڸ;8ÿzg/OPi?9>o^JkezO-,C1"1 W[vkыZ`e?D$h0Irlm4o\G~uFh0q[wf_(&)8휟<zz,lbׅqnxa!.Az:o{uC@J[k2870R`}#"#0X |uNPRsgȋ$y?}̓%߸-tZr+J~AV\\x6>>)ngcA cYaV֘XɃav&ZW܂cw{ۛb89FҼnY=~~c,eU݌kp4~[p鱣0 HP.,?}?G>� 1:}dxkB7[I'\ �H'5gЈCT^5o} pᜟ {حWva]oSȒy/ãOT=g3*@ ��?�`ۻ`~Y;[{IVӼ˜3@kp@99JARzt%ŗv{�Ós8) 5+ g 5F*s"F"?Ng'O_=xt.ßĈp&,cm ?<wg0ͯZZ^oshltZhhD.F1*]-(�xe3(NѨt<O&qu{/ ,)g}dʼGo�w�>x޻\f12*CUEjcQ8R)KJ)ZhU z 0U[kRb¹fT}]SOL�aQicH)htIdo4D񛛽^ݎj$!1F#"ƈ10μ*0elt KZ5cklVeb2O�;?HϹox{>ƍ{ɾAȅ3!C8(J.ҖE6y(>?>xtѝ}�Ytx4Wu,ɱ_<an2Il2l6zdu}ƭ,fڤȒ@_u�އnykf̎Γ?}z/?>}�ƿy7o?L{\����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000005353�13034501105�0016627�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from .browser import EsaliaBrowser, CapeasiBrowser, ErehsbcBrowser, BnppereBrowser __all__ = ['S2eModule'] class S2eModule(Module, CapBank): NAME = 's2e' DESCRIPTION = u'Épargne Salariale' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig( ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Code secret', regexp='^(\d{6}|)$'), ValueBackendPassword('secret', label=u'Réponse secrète (optionnel)', default=''), Value('website', label='Banque', default='', choices={'esalia': u'Esalia', 'capeasi': u'Capeasi', 'erehsbc': u'ERE HSBC', 'bnppere': u'BNPP ERE'})) BROWSERS = { 'esalia': EsaliaBrowser, 'capeasi': CapeasiBrowser, 'erehsbc': ErehsbcBrowser, 'bnppere': BnppereBrowser, } def create_default_browser(self): self.BROWSER = self.BROWSERS[self.config['website'].get()] return self.create_browser(self.config['login'].get(), self.config['password'].get(), secret=self.config['secret'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_history(self, account): return self.browser.iter_history(account) def iter_investment(self, account): return self.browser.iter_investment(account) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000037432�13034501105�0016444�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re, requests from cStringIO import StringIO from decimal import Decimal from lxml import objectify from weboob.browser.pages import HTMLPage, XMLPage, LoggedPage, pagination from weboob.browser.elements import ItemElement, TableElement, SkipItem, method from weboob.browser.filters.standard import CleanText, Date, Regexp, Eval, CleanDecimal, Env, TableCell, Field from weboob.browser.filters.html import Attr from weboob.capabilities.bank import Account, Investment, Transaction from weboob.capabilities.base import NotAvailable from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard from weboob.exceptions import NoAccountsException, BrowserUnavailable def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class ErrorPage(HTMLPage): def on_load(self): raise BrowserUnavailable() class S2eVirtKeyboard(MappedVirtKeyboard): symbols = {'0':('8adee734aaefb163fb008d26bb9b3a42', '922d79345bf824b1186d0aa523b37a7c'), '1':('b815d6ce999910d48619b5912b81ddf1', '4730473dcd86f205dff51c59c97cf8c0'), '2':('54255a70694787a4e1bd7dd473b50228', '2d8b1ab0b5ce0b88abbc0170d2e85b7e'), '3':('ba06373d2bfba937d00bf52a31d475eb', '08e7e7ab7b330f3cfcb819b95eba64c6'), '4':('3fa795ac70247922048c514115487b10', 'ffb3d035a3a335cfe32c59d8ee1302ad'), '5':('788963d15fa05832ee7640f7c2a21bc3', 'c4b12545020cf87223901b6b35b9a9e2'), '6':('c8bf62dfaed9feeb86934d8617182503', '473357666949855a0794f68f3fc40127'), '7':('f7543fdda3039bdd383531954dd4fc46', '5f3a71bd2f696b8dc835dfeb7f32f92a'), '8':('5c4210e2d8e39f7667d7a9e5534b18b7', 'b9a1a73430f724541108ed5dd862431b'), '9':('94520ac801883fbfb700f43cd4172d41', '12c18ca3d4350acd077f557ac74161e5'), } color = (0, 0, 0) def __init__(self, page, vkid): img = page.doc.find('//img[@id="clavier_virtuel"]') res = page.browser.open("/portal/rest/clavier_virtuel/%s" % vkid) MappedVirtKeyboard.__init__(self, StringIO(res.content), page.doc, img, self.color, convert='RGB') self.check_symbols(self.symbols, None) def get_symbol_code(self, md5sum): code = MappedVirtKeyboard.get_symbol_code(self, md5sum) m = re.search('(\d+)', code) if m: return m.group(1) def get_string_code(self, string): return ''.join([self.get_symbol_code(self.symbols[c]) for c in string]) class LoginPage(HTMLPage): def get_password(self, password, secret): vkid = Attr('//input[@id="identifiantClavierVirtuel"]', 'value')(self.doc) code = S2eVirtKeyboard(self, vkid).get_string_code(password) tcc = Attr('//input[@id="codeTCC"]', 'value')(self.doc) password = "%s|%s|#%s#" % (code, vkid, tcc) if secret: password = "%s%s" % (password, secret) return password def login(self, login, password, secret): data = {} data['initialURI'] = "/portal/private/salarie-%s" % self.browser.SLUG data['password'] = self.get_password(password, secret) data['username'] = login self.browser.location('/portal/login', data=data) def get_error(self): cgu = CleanText('//h1[contains(text(), "Conditions")]', default=None)(self.doc) if cgu: cgu = u"Veuillez accepter les conditions générales d'utilisation." if self.browser.LANG == "fr" \ else "Please accept the general conditions of use." if self.browser.LANG == 'en' \ else cgu return cgu or CleanText('//div[contains(text(), "Erreur")]', default='')(self.doc) class AMFCodePage(XMLPage): ENCODING = "UTF-8" def build_doc(self, content): doc = super(AMFCodePage, self).build_doc(content).getroot() # Remove namespaces for el in doc.getiterator(): if not hasattr(el.tag, 'find'): continue i = el.tag.find('}') if i >= 0: el.tag = el.tag[i+1:] objectify.deannotate(doc, cleanup_namespaces=True) return doc class ItemInvestment(ItemElement): klass = Investment obj_unitvalue = Env('unitvalue') obj_vdate = Env('vdate') def obj_label(self): return CleanText(TableCell('label')(self)[0].xpath('.//div[contains(@style, \ "text-align")][1]'))(self).rsplit(' - ', 1)[0] def obj_valuation(self): return MyDecimal(TableCell('valuation')(self)[0].xpath('.//div[not(.//div)]'))(self) def obj_code(self): link_id = Attr(u'.//a[contains(@title, "détail du fonds")]', 'id', default=None)(self) inv_id = Attr('.//a[contains(@id, "linkpdf")]', 'id', default=None)(self) if link_id and inv_id: form = self.page.get_form('//div[@id="operation"]//form') form['idFonds'] = inv_id.split('-', 1)[-1] form['org.richfaces.ajax.component'] = form[link_id] = link_id page = self.page.browser.open(form['javax.faces.encodedURL'], data=dict(form)).page m = re.search('fundid=(\w+).+SH=(\w+)', CleanText('//complete', default="")(page.doc)) if m: page = page.browser.open('https://www.assetmanagement.hsbc.com/feedRequest?feed_data=gfcFundData&cod=FR&client=FCPE&fId=%s&SH=%s&lId=fr' % m.groups()).page return CleanText('//AMF_Code', default=NotAvailable)(page.doc) return NotAvailable def parse(self, el): # Trying to find vdate and unitvalue unitvalue, vdate = None, None for span in TableCell('label')(self)[0].xpath('.//span'): if unitvalue is None: unitvalue = Regexp(CleanText('.'), '^([\d,]+)$', default=None)(span) if vdate is None: vdate = None if any(x in CleanText('./parent::div')(span) for x in [u"échéance", "Maturity"]) else \ Regexp(CleanText('.'), '^([\d\/]+)$', default=None)(span) self.env['unitvalue'] = MyDecimal().filter(unitvalue) if unitvalue else NotAvailable self.env['vdate'] = Date(dayfirst=True).filter(vdate) if vdate else NotAvailable class MultiPage(HTMLPage): def get_multi(self): return [Attr('.', 'value')(option) for option in \ self.doc.xpath('//select[@class="ComboEntreprise"]/option')] def go_multi(self, id): if Attr('//select[@class="ComboEntreprise"]/option[@selected]', 'value')(self.doc) != id: form = self.get_form('//select[@class="ComboEntreprise"]/ancestor::form[1]') key = [k for k, v in dict(form).iteritems() if "SelectItems" in k][0] form[key] = id form['javax.faces.source'] = key form.submit() class AccountsPage(LoggedPage, MultiPage): def on_load(self): no_accounts_message = CleanText('//span[contains(text(), "On this date, you still have no employee savings in this company.")] | \ //span[contains(text(), "On this date, you do not yet have any employee savings in this company.")] | \ //span[contains(text(), "On this date, you no longer have any employee savings in this company.")]')(self.doc) if no_accounts_message: raise NoAccountsException(no_accounts_message) TYPES = {'PEE': Account.TYPE_PEE, 'PEI': Account.TYPE_PEE, 'PEEG': Account.TYPE_PEE, 'PEG': Account.TYPE_PEE, 'PLAN': Account.TYPE_PEE, 'PERCO': Account.TYPE_PERCO, 'PERCOI': Account.TYPE_PERCO, 'SWISS': Account.TYPE_MARKET } @method class iter_accounts(TableElement): item_xpath = '//div[contains(@id, "Dispositif")]//table/tbody/tr[td[2]]' head_xpath = '//div[contains(@id, "Dispositif")]//table/thead/tr/th' col_label = [u'My schemes', u'Mes dispositifs'] col_balance = [re.compile(u'Total'), re.compile(u'Montant')] class item(ItemElement): klass = Account obj_id = Env('id') obj_label = Env('label') def obj_type(self): return self.page.TYPES.get(Field('label')(self).split()[0].upper(), Account.TYPE_UNKNOWN) def obj_balance(self): return MyDecimal(TableCell('balance')(self)[0].xpath('.//div[has-class("nowrap")]'))(self) def obj_currency(self): return Account.get_currency(CleanText(TableCell('balance')(self)[0].xpath('.//div[has-class("nowrap")]'))(self)) def parse(self, el): id, label = CleanText(TableCell('label'))(self).split(' ', 1) self.env['id'] = id self.env['label'] = label def get_investment_pages(self, accid, valuation=True): form = self.get_form('//div[@id="operation"]//form') div_xpath = '//div[contains(@id, "ongletDetailParSupport")]' input_id = Attr('//input[contains(@id, "onglets")]', 'name')(self.doc) select_id = Attr('%s//select' % div_xpath, 'id')(self.doc) option_value = Attr('//option[contains(text(), "%s")]' % accid, 'value')(self.doc) form[select_id] = option_value form[input_id] = "onglet2" # Select display : amount or quantity radio_txt = ("En montant" if valuation else [u"Quantité", "En parts"]) if self.browser.LANG == "fr" else \ ("In amount" if valuation else ["Quantity", "In units"]) if isinstance(radio_txt, list): radio_txt = '" or text()="'.join(radio_txt) input_id = Regexp(Attr('%s//span[text()="%s"]/preceding-sibling::a[1]' \ % (div_xpath, radio_txt), 'onclick'), '"([^"]+)')(self.doc) form[input_id] = input_id form['javax.faces.source'] = input_id form['valorisationMontant'] = "true" if valuation else "false" data = {k: v for k, v in dict(form).iteritems() if "blocages" not in v} self.browser.location(form.url, data=data) def update_quantity(self, invs): for inv in invs: inv.quantity = MyDecimal().filter(CleanText('//div[contains(@id, "ongletDetailParSupport")] \ //tr[.//div[contains(text(), "%s")]]/td[last()]//div/text()' % inv.label)(self.doc)) return invs @method class iter_investment(TableElement): item_xpath = '//div[contains(@id, "ongletDetailParSupport")]//table/tbody/tr[td[4]]' head_xpath = '//div[contains(@id, "ongletDetailParSupport")]//table/thead/tr/th' col_label = [re.compile(u'My investment'), re.compile(u'Mes supports')] col_valuation = [re.compile(u'Gross amount'), re.compile(u'Montant brut')] col_portfolio_share = [u'Distribution', u'Répartition'] class item(ItemInvestment): def obj_portfolio_share(self): return Eval(lambda x: x / 100, MyDecimal(TableCell('portfolio_share')(self)[0] \ .xpath('.//div[has-class("nowrap")]'))(self))(self) class HistoryPage(LoggedPage, MultiPage): XPATH_FORM = '//div[@id="operation"]//form' def get_history_form(self, idt, args={}): form = self.get_form(self.XPATH_FORM) form[idt] = idt form['javax.faces.source'] = idt form.update(args) return form def show_more(self, nb): form = self.get_form(self.XPATH_FORM) for select in self.doc.xpath('//select'): if Attr('./option[@selected]', 'value')(select) == nb: return idt = Attr('.', 'id')(select) form[idt] = nb if 'javax.faces.source' not in form: form['javax.faces.source'] = idt form.submit() def go_start(self): idt = Attr('//a[@title="debut" or @title="precedent"]', 'id', default=None)(self.doc) if idt: form = self.get_history_form(idt) form.submit() @method class get_investments(TableElement): item_xpath = '//table//table/tbody/tr[td[4]]' head_xpath = '//table//table/thead/tr/th' col_scheme = [u'Scheme', u'Dispositif'] col_label = [re.compile(u'Investment'), re.compile('My investment'), u'fund', re.compile(u'Support')] col_quantity = [re.compile(u'Quantity'), re.compile(u'Quantité'), re.compile('En parts')] col_valuation = [u'Gross amount', u'Net amount', re.compile(u'Montant brut'), u'Montant Net'] class item(ItemInvestment): def obj_quantity(self): return MyDecimal(TableCell('quantity')(self)[0].xpath('./text()'))(self) def condition(self): return Env('accid')(self) in CleanText(TableCell('scheme'))(self) @pagination @method class iter_history(TableElement): item_xpath = '//table/tbody/tr[td[4]]' head_xpath = '//table/thead/tr/th' col_id = [re.compile(u'Ref'), re.compile(u'Réf')] col_date = [re.compile(u'Date'), re.compile('Creation date')] col_label = [re.compile('Transaction'), re.compile(u'Type')] def next_page(self): idt = Attr('//a[@title="suivant"]', 'id', default=None)(self.page.doc) if idt: form = self.page.get_history_form(idt) return requests.Request("POST", form.url, data=dict(form)) class item(ItemElement): klass = Transaction obj_id = CleanText(TableCell('id')) obj_label = CleanText(TableCell('label')) obj_type = Transaction.TYPE_BANK obj_date = Date(CleanText(TableCell('date')), dayfirst=True) obj_amount = Env('amount') obj_investments = Env('investments') def parse(self, el): # We have only one history for all accounts... # And we know only on details page if it match current account. trid = CleanText(TableCell('id'))(self) if trid not in self.page.browser.cache['details']: # Thanks to stateful website : first go on details page... idt = Attr(TableCell('id')(self)[0].xpath('./a'), 'id', default=None)(self) typeop = Regexp(Attr(TableCell('id')(self)[0].xpath('./a'), 'onclick'), 'Operation.+?([A-Z_]+)')(self) form = self.page.get_history_form(idt, {'referenceOp': trid, 'typeOperation': typeop}) page = self.page.browser.open(form.url, data=dict(form)).page self.page.browser.cache['details'][trid] = page # ...then go back to history list. idt = Attr('//input[@title="Retour"]', 'id', default=None)(page.doc) form = self.page.get_history_form(idt) self.page.browser.open(form.url, data=dict(form)).page else: page = self.page.browser.cache['details'][trid] # Check if page is related to the account if not len(page.doc.xpath('//td[contains(text(), "%s")]' % Env('accid')(self))): raise SkipItem() self.env['investments'] = list(page.get_investments(accid=Env('accid')(self))) self.env['amount'] = sum([i.valuation or Decimal('0') for i in self.env['investments']]) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/s2e/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001736�13034501105�0016322�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class S2eTest(BackendTest): MODULE = 's2e' def test_bank(self): l = list(self.backend.iter_accounts()) if len(l) > 0: a = l[0] list(self.backend.iter_history(a)) ����������������������������������weboob-1.2/modules/sachsen/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015715�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sachsen/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001452�13034501105�0020030�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SachsenLevelModule __all__ = ['SachsenLevelModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sachsen/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000002542�13034501105�0017755�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import ListPage, HistoryPage __all__ = ['SachsenBrowser'] class SachsenBrowser(PagesBrowser): BASEURL = 'http://www.umwelt.sachsen.de' homepage = URL('/umwelt/infosysteme/hwims/portal/web/wasserstand-uebersicht.*', ListPage) history = URL('/umwelt/infosysteme/hwims/portal/web/wasserstand-pegel-(?P<idgauge>.*)', HistoryPage) def get_rivers_list(self): return self.homepage.stay_or_go().get_rivers_list() def iter_history(self, sensor, **kwargs): self.history.go(idgauge=sensor.gaugeid) return self.page.iter_history(sensor=sensor) ��������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sachsen/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000011740�13034501105�0020053�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� B(x���tIME _P��mIDATx՛wՕB瞞ܓH ,0L0^{c۬vװik.fN,FK0lP@YiY3ӓz:UW?^ Xd9t^[~+ ]Hh$ @Jqա3ml_�n4t7Ґr+rkG8$@X]V 2-c`Ss7 [ æx CSƒ@a�UBǘr!Hd2=?ke|a�$,w /VT'_E9mU !Um}8�> 3 uT)\xK9~夅Q2oYwAc oW=ٖVT*D| 7&0@RowAm>\�zKAyى,:a0T+0m? riί<5H SQUqz`uTGo(xR'jQ1˅0~C;fx޳~*PY~( wõ`h;@k9b‡V<B=P>( y*E^M/>虀`]+m"w֟0)}߅[ܽ�&UWE`e tM=T}2Ea mw@_?_^A҅{hQ>m0J`"Hf!y+ܙD.t ѯ5<}L�a~\׊.P%(d*MJC|!41Ii l Wxy|?L 54쌙3ι}~?(=| -rU-z"?(qS) T:gQJe;ѺX$ќ\I @6B$`ٺ'�ǧIU` nqCB >lz>,PsK,K@G[$+e:l=++5WWBDC Zai̇IU%W)T{_)d,Θȅ~x<QUy B:{n&r}Z‹]  T<ѪpaF;$0ձ%*Z6ME&}*g _?Q- GiӌctFŬCCJkym1=;N0�z3 (¤W{pnX`KfuDEĒrEcI}- KtrVS J P L/޶sxee g<Gܟߨx ]pIJiŠ*V!kJ0 0K!"=;f'9བྷIꦓQr imsԁ';~WqאRP(t!l;\6LUZw|kG9 c:}ՏK;mPR]UI:6E1[c?G<- :<%Z-"`'=l;H I �|o]nVH>WCp]ܭdrm>mM@4<4<‡-X(P߼N(2 :ܫB=*&V˗Cy]| c,yOg58* +K4D8 "y*c;4XB-|E5$MC2غ_ǿ5Ⱥ,_(z{ mlKbD ٮ+"7&WOʜKZlG`_.jﱎ 14#|~y߷P5Em,{ib<i)(nEJ 0mgΡb<˳Ou΅F@(,"ue?px=r߹)4ᐢG6bg[5>}ld#N63:&HS[zwl>kAdBA f!YYȉwG[ ;W4uȎ}M84W=C@b̯d`6 YˠfRq((4�4X!ȯH F@s <^}8^rO@:i|~0g#է~\ >7W;5 5$pb#d7sǭ'T‚2jZw կHC~'t/t�C2OI10-(x iT iXLA;p~Njщ%/�:vb=@^'Զ.ѹ/ mBI?&`d4u@hza: \|3sٗsZilDҨ{ djQBE;DFaSw)8\9*,VCTVQ$uApvSH ;o`֠sP:B~ء(냽v' Y1*Scpvp:.)58�ă&xMB"j~EC8{c;N[\ Rp-4nGTJpf/Փᘊ$!W.A#IwQ2ކfk2=s Nuq]a1qhLpm ! {PC9g< ÕP4IBw%ὗ굜j)rOD^N1U)!V=ګPj=� i~rBGx�΁Pͬܯ@P>Y XԨ+5NC~#tͥMHzrb> ĩŀyr NCҕ:_yF{4셮f E\:z$  у=f⹼)i`�� ӯ�gI?%Ϳ_=wD _e#d!DRޜt�މ˛&Rj8B`Uݴ\l(Wsg1uo9NC X/aa9ZAsq{5whʝ "c!KQ%MaC,`R TIC�Ts]1( U]F*;~Jfb%3j I0{9jTup4CncU LhP]]@-8& J(icD# gtULD!V(/ڛ+ ŇIt dB*C)U5ZWL�Z)hk-`Yݧ^ E0R Qٯ4EhJ`^ #*0\_ |,|Ay۔NH7$HVG"X"(k K 2S ChDD [n=x$rQ;3-&xf7M 8R`ECWeOC4IL!+H#K5USc1cphmH/Pd*㇢.>g xai^4a!Nt;k8`3# 9/0ۯ.8}.u >fz:ԧ !\GʁM`9T&L㋠et^w\E LO_kg+?o p_}eH[ 2) k?#F_&#.F?DP85J\]6Dvwڰ҇, ;G7YzsC巭:?SfauסO~wF-$:3֤ "ys8&MgEI2S ף+tMUSaOacQΞJ+/O8Rv 27w73v? ykkj6; laFTäbR9;n U׾ Ocr>l'B>'ɛ/]#j3�y;GI�#Pѥ(qJ H$c!ۗ =tP-H/;�0y"X^mA2F@ !]LNRwtХE(;Ԙ^Pt�J|C^- }*RY&!8W s ~:tEGOsL#0Ep,YQ$Cr0JFODE۽}к>rWdDִЄMpvAj:$vLfC0xjRǡmzǘ~<u<&rbol 2Uէaٷ@1@3p-$om='={x?i&� 5I A*h>T2`/hnK:SSBJC W#-o܈[M/l'@DBKN "}¶R欼.; xDm@sq-['L'owFWO4b, lC =vLO>O'}CvWz#ܽEpC3B@ǝg{3 go3ڽO)u,' :N$xz)B#]x߃6Ml9{3<WCl΋V++I7őxgmK.ڃ5RC|T.? /^m4/F |GaI-*G9H+OPE..5އ{`|TNױakس 잍Ea{RYVpkxԽE9<! JC٢B�t@Jd2Ċe11Gptڅ+kDͬ:=e^ql�Q§8<7d D?vM]Nk4GpH}~$m ����IENDB`��������������������������������weboob-1.2/modules/sachsen/module.py����������������������������������������������������������������0000664�0000000�0000000�00000005246�13034501105�0017563�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .browser import SachsenBrowser from weboob.capabilities.gauge import CapGauge, GaugeSensor, Gauge,\ SensorNotFound from weboob.capabilities.base import find_object from weboob.tools.backend import Module __all__ = ['SachsenLevelModule'] class SachsenLevelModule(Module, CapGauge): NAME = 'sachsen' MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"Level of Sachsen river" BROWSER = SachsenBrowser def iter_gauges(self, pattern=None): if pattern is None: for gauge in self.browser.get_rivers_list(): yield gauge else: lowpattern = pattern.lower() for gauge in self.browser.get_rivers_list(): if lowpattern in gauge.name.lower()\ or lowpattern in gauge.object.lower(): yield gauge def _get_sensor_by_id(self, id): for gauge in self.browser.get_rivers_list(): for sensor in gauge.sensors: if id == sensor.id: return sensor raise SensorNotFound() def iter_sensors(self, gauge, pattern=None): if not isinstance(gauge, Gauge): gauge = find_object(self.browser.get_rivers_list(), id=gauge, error=SensorNotFound) if pattern is None: for sensor in gauge.sensors: yield sensor else: lowpattern = pattern.lower() for sensor in gauge.sensors: if lowpattern in sensor.name.lower(): yield sensor def iter_gauge_history(self, sensor): if not isinstance(sensor, GaugeSensor): sensor = self._get_sensor_by_id(sensor) return self.browser.iter_history(sensor) def get_last_measure(self, sensor): if not isinstance(sensor, GaugeSensor): sensor = self._get_sensor_by_id(sensor) return sensor.lastvalue ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sachsen/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000013274�13034501105�0017375�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import lxml.html from weboob.browser.pages import HTMLPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import Env, CleanText, Regexp, Field, DateTime, Map from weboob.browser.filters.html import Attr from weboob.capabilities.gauge import Gauge, GaugeMeasure, GaugeSensor from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.exceptions import ParseError import re class ListPage(HTMLPage): @method class get_rivers_list(ListElement): item_xpath = ".//div[@class='msWrapper']/script" class item(ItemElement): klass = Gauge extract = re.compile(".*content:(?P<html>.*),show") forecasts = {'pfeil_gerade.png.jsf': u'stable', 'pfeil_unten.png.jsf': u'Go down', 'pfeil_oben.png.jsf': u'Go up', } alarmlevel = {"as1.gif": u"Alarmstufe 1", "as2.gif": u"Alarmstufe 2", "as3.gif": u"Alarmstufe 3", "as4.gig": u"Alarmstufe 4", "qua_grau.gif": u"No alarm function", "p_gruen.gif": u"", "qua_weiss.gif": u"no data", "as0.gif": u"", "MNW.gif": u""} obj_id = CleanText(Env('id')) obj_name = CleanText(Env('name'), "'") obj_city = Regexp(Field('name'), '^([^\s]+).*') obj_object = Env('object') def parse(self, el): raw = self.extract.match(el.text).group("html") raw = raw.replace('\\"', '"').replace('\\n', '').replace('\\/', '/') parsed = lxml.html.fromstring(raw) self.env['name'] = CleanText('.//span[@class="popUpTitleBold"]')(parsed) self.env['object'] = CleanText('.//span[@class="popUpTitleNormal"]')(parsed).strip(' /') url = Attr('.//div[@class="popUpMsDiagramm"]/img', 'src')(parsed) self.env['id'] = url.split('_')[1] for tr in parsed.xpath('.//tr'): td = tr.xpath('.//td') if len(td) == 1 and "Datum" in td[0].text: l = td[0].text.split()[1:3] self.env['datetime'] = "%s %s" % (l[0], l[1]) elif len(td) == 2: if "Wasserstand" in td[0].text: self.env['levelvalue'] = td[1].text.split()[0] elif "Durchfluss" in td[0].text: self.env['flowvalue'] = td[1].text.split()[0] elif "Tendenz" in td[0].text: try: self.env['forecast'] = Attr('img', 'src')(td[1]).split("/")[-1] except ParseError: self.env['forecast'] = None # TODO self.env['alarm'] = None def add_sensor(self, sensors, name, unit, value, forecast, alarm, date): sensor = GaugeSensor("%s-%s" % (self.obj.id, name.lower())) sensor.name = name sensor.unit = unit sensor.forecast = forecast lastvalue = GaugeMeasure() lastvalue.alarm = alarm try: lastvalue.level = float(value) except ValueError: lastvalue.level = NotAvailable lastvalue.date = date sensor.lastvalue = lastvalue sensor.history = NotLoaded sensor.gaugeid = self.obj.id sensors.append(sensor) def obj_sensors(self): sensors = [] lastdate = DateTime(Regexp(Env('datetime'), r'(\d+)\.(\d+)\.(\d+) (\d+):(\d+)', r'\3-\2-\1 \4:\5', default=NotAvailable), default=NotAvailable)(self) forecast = Map(Env('forecast'), self.forecasts, default=NotAvailable)(self) alarm = Map(Env('alarm'), self.alarmlevel, default=u'')(self) self.add_sensor(sensors, u"Level", u"cm", self.env['levelvalue'], forecast, alarm, lastdate) self.add_sensor(sensors, u"Flow", u"m3/s", self.env['flowvalue'], forecast, alarm, lastdate) return sensors class HistoryPage(HTMLPage): @method class iter_history(ListElement): item_xpath = '//table[@class="quickbarTable"]/tbody/tr' class item(ItemElement): klass = GaugeMeasure verif = re.compile("\d\d.\d\d.\d+ \d\d:\d\d") obj_date = DateTime(Regexp(CleanText('.'), r'(\d+)\.(\d+)\.(\d+) (\d+):(\d+)', r'\3-\2-\1 \4:\5')) sensor_types = [u'Level', u'Flow'] def obj_level(self): index = self.sensor_types.index(self.env['sensor'].name) + 1 try: return float(self.el[index].text_content()) except ValueError: return NotAvailable # TODO: history.alarm ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sachsen/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002640�13034501105�0017250�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import find_object class SachsenTest(BackendTest): MODULE = 'sachsen' def test_sachsen(self): l = list(self.backend.iter_gauges()) self.assertTrue(len(l) > 0) gauge = find_object(l, id=u'501060') sensors = list(self.backend.iter_sensors(gauge)) self.assertTrue(len(sensors) > 0) for sensor in sensors: self.assertTrue(sensor.lastvalue.level > 0) sensor = sensors[0] history = list(self.backend.iter_gauge_history(sensor)) self.assertTrue(len(history) > 0) self.assertTrue(self.backend.get_last_measure(sensor) is not None) ������������������������������������������������������������������������������������������������weboob-1.2/modules/seloger/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015731�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/seloger/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000000077�13034501105�0020046�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import SeLogerModule __all__ = ['SeLogerModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/seloger/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000005703�13034501105�0017773�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.capabilities.housing import Query, TypeNotSupported from weboob.browser import PagesBrowser, URL from .pages import SearchResultsPage, HousingPage, CitiesPage from weboob.browser.profiles import Android __all__ = ['SeLogerBrowser'] class SeLogerBrowser(PagesBrowser): BASEURL = 'http://www.seloger.com' PROFILE = Android() cities = URL('js,ajax,villequery_v3.htm\?ville=(?P<pattern>.*)', CitiesPage) search = URL('http://ws.seloger.com/search.xml\?(?P<request>.*)', SearchResultsPage) housing = URL('http://ws.seloger.com/annonceDetail.xml\?idAnnonce=(?P<_id>\d+)&noAudiotel=(?P<noAudiotel>\d)', HousingPage) def search_geo(self, pattern): return self.cities.open(pattern=pattern).iter_cities() TYPES = {Query.TYPE_RENT: 1, Query.TYPE_SALE: 2, } RET = {Query.HOUSE_TYPES.HOUSE: '2', Query.HOUSE_TYPES.APART: '1', Query.HOUSE_TYPES.LAND: '4', Query.HOUSE_TYPES.PARKING: '3', Query.HOUSE_TYPES.OTHER: '10'} def search_housings(self, type, cities, nb_rooms, area_min, area_max, cost_min, cost_max, house_types): if type not in self.TYPES: raise TypeNotSupported() data = {'ci': ','.join(cities), 'idtt': self.TYPES.get(type, 1), 'org': 'advanced_search', 'surfacemax': area_max or '', 'surfacemin': area_min or '', 'tri': 'd_dt_crea', } if type == Query.TYPE_SALE: data['pxmax'] = cost_max or '' data['pxmin'] = cost_min or '' else: data['px_loyermax'] = cost_max or '' data['px_loyermin'] = cost_min or '' if nb_rooms: data['nb_pieces'] = nb_rooms ret = [] for house_type in house_types: if house_type in self.RET: ret.append(self.RET.get(house_type)) if ret: data['idtypebien'] = ','.join(ret) return self.search.go(request=urllib.urlencode(data)).iter_housings() def get_housing(self, _id, obj=None): return self.housing.go(_id=_id, noAudiotel=1).get_housing(obj=obj) �������������������������������������������������������������weboob-1.2/modules/seloger/module.py����������������������������������������������������������������0000664�0000000�0000000�00000005143�13034501105�0017573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.housing import CapHousing, Housing, HousingPhoto from weboob.tools.backend import Module from .browser import SeLogerBrowser __all__ = ['SeLogerModule'] class SeLogerModule(Module, CapHousing): NAME = 'seloger' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'French housing website' LICENSE = 'AGPLv3+' ICON = 'http://static.poliris.com/z/portail/svx/portals/sv6_gen/favicon.png' BROWSER = SeLogerBrowser def search_housings(self, query): cities = [c.id for c in query.cities if c.backend == self.name] if len(cities) == 0: return list([]) return self.browser.search_housings(query.type, cities, query.nb_rooms, query.area_min, query.area_max, query.cost_min, query.cost_max, query.house_types) def get_housing(self, housing): if isinstance(housing, Housing): id = housing.id else: id = housing housing = None return self.browser.get_housing(id, housing) def search_city(self, pattern): return self.browser.search_geo(pattern) def fill_housing(self, housing, fields): if fields != ['photos'] or not housing.photos: housing = self.browser.get_housing(housing.id, housing) if 'photos' in fields: for photo in housing.photos: if not photo.data: photo.data = self.browser.open(photo.url) return housing def fill_photo(self, photo, fields): if 'data' in fields and photo.url and not photo.data: photo.data = self.browser.open(photo.url).content return photo OBJECTS = {Housing: fill_housing, HousingPhoto: fill_photo, } �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/seloger/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000007504�13034501105�0017410�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import XMLPage, JsonPage, pagination from weboob.browser.elements import ItemElement, ListElement, DictElement, method from weboob.browser.filters.json import Dict from weboob.browser.filters.html import XPath from weboob.browser.filters.standard import CleanText, CleanDecimal, DateTime from weboob.capabilities.base import NotAvailable from weboob.capabilities.housing import Housing, HousingPhoto, City from weboob.tools.capabilities.housing.housing import PricePerMeterFilter class CitiesPage(JsonPage): @method class iter_cities(DictElement): item_xpath = '*/values' ignore_duplicate = True class item(ItemElement): klass = City def condition(self): return Dict('value', default=None)(self) obj_id = Dict('value') obj_name = Dict('label') class SeLogerItem(ItemElement): klass = Housing obj_id = CleanText('idAnnonce') obj_title = CleanText('titre') obj_date = DateTime(CleanText('dtFraicheur')) obj_cost = CleanDecimal('prix') obj_currency = CleanText('prixUnite') obj_area = CleanDecimal('surface', default=NotAvailable) obj_price_per_meter = PricePerMeterFilter() obj_text = CleanText('descriptif') obj_location = CleanText('ville') obj_station = CleanText('proximite', default=NotAvailable) obj_url = CleanText('permaLien') class SearchResultsPage(XMLPage): @pagination @method class iter_housings(ListElement): item_xpath = "//annonce" def next_page(self): page = CleanText('//pageSuivante', default=None, replace=[('http://ws.seloger.com/', '')])(self) if page: return page class item(SeLogerItem): def obj_photos(self): photos = [] for photo in XPath('./photos/photo/stdUrl')(self): photos.append(HousingPhoto(CleanText('.')(photo))) return photos class HousingPage(XMLPage): @method class get_housing(SeLogerItem): def obj_photos(self): photos = [] for photo in XPath('./photos/photo')(self): url = CleanText('bigUrl', default=None)(photo) if not url: url = CleanText('stdUrl', default=None)(photo) photos.append(HousingPhoto(url)) return photos def obj_location(self): location = CleanText('//detailAnnonce/adresse')(self) quartier = CleanText('//detailAnnonce/quartier', default=None)(self) if not location and quartier is not None: location = quartier ville = CleanText('ville')(self) return u'%s %s' % (location, ville) def obj_details(self): details = {} for detail in XPath('//detailAnnonce/details/detail')(self): details[CleanText('libelle')(detail)] = CleanText('valeur', default='N/A')(detail) details['Reference'] = CleanText('//detailAnnonce/reference')(self) return details obj_phone = CleanText('//contact/telephone') ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/seloger/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002552�13034501105�0017266�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import itertools from weboob.capabilities.housing import Query from weboob.tools.test import BackendTest class SeLogerTest(BackendTest): MODULE = 'seloger' def test_seloger(self): query = Query() query.area_min = 20 query.cost_max = 1000 query.type = Query.TYPE_RENT query.cities = [] for city in self.backend.search_city(u'Ferté'): city.backend = self.backend.name query.cities.append(city) results = list(itertools.islice(self.backend.search_housings(query), 0, 20)) self.assertTrue(len(results) > 0) self.backend.fillobj(results[0], 'phone') ������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017007�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0021125�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SenscritiqueModule __all__ = ['SenscritiqueModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/browser.py����������������������������������������������������������0000664�0000000�0000000�00000005035�13034501105�0021047�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.base import UserError from weboob.browser import PagesBrowser, URL from .pages import FilmsPage, EventPage, JsonResumePage from weboob.browser.profiles import Firefox __all__ = ['SenscritiqueBrowser'] class SenscritiqueBrowser(PagesBrowser): BASEURL = 'http://www.senscritique.com' films_page = URL('/everymovie/programme-tv/cette-semaine', FilmsPage) event_page = URL('/film/(?P<_id>.*)', EventPage) json_page = URL('/sc2/product/storyline/(?P<_id>.*).json', JsonResumePage) def set_json_header(self): self.session.headers.update({"User-Agent": "Mozilla/5.0 (Windows; U; Windows " "NT 5.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8" " GTB7.1 (.NET CLR 3.5.30729)", "Accept": "application/json, text/javascript, */*; q=0.01", "X-Requested-With": "XMLHttpRequest", }) def list_events(self, date_from, date_to=None): return self.films_page.go().iter_films(date_from=date_from, date_to=date_to) def get_event(self, _id, event=None): if not event: try: event = self.films_page.go().iter_films(_id=_id).next() except StopIteration: raise UserError('This event (%s) does not exists' % _id) film_id = _id.split('#')[0] event = self.event_page.go(_id=film_id).get_event(obj=event) resume = self.get_resume(film_id) event.description += resume return event def get_resume(self, film_id): self.set_json_header() _id = film_id.split('/')[-1] resume = self.json_page.go(_id=_id).get_resume() self.set_profile(Firefox()) return resume if resume else '' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/calendar.py���������������������������������������������������������0000664�0000000�0000000�00000002070�13034501105�0021131�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.calendar import BaseCalendarEvent, TRANSP, STATUS, CATEGORIES class SensCritiquenCalendarEvent(BaseCalendarEvent): def __init__(self): BaseCalendarEvent.__init__(self) self.sequence = 1 self.transp = TRANSP.TRANSPARENT self.status = STATUS.CONFIRMED self.category = CATEGORIES.TELE ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000012210�13034501105�0021136�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����gAMA�� a��� pHYs����+��IDATx^[ Uev~B7 F'4I1JM*L@^!5L&DrL2,iDDF#Uq "h٧̰4{'w=Kwt wH鹦5yooH'锤tKgosq]Rz)qnI >)NI%T9=ҝ�crjK/Y\l}2@J&: -9NyhD.#A^ kE1)@THX)ߎ[Ϧz1v'�D/�N6+`^.S$�"�$ҡ{CAT}wn`$)v<ҥKjurkkk \-o�N\Waq:8�A@Zy`eINM@6wvv::H=ꟾtuAwfCvI[{3땺&lceUr}*@ .GS*,:]DP˕Vim|QCZZu/466,zlDPXQF�8Q5jjj!Z�ӱUb>8'5`c%6̏s04LC Yq3E ȏ$_@'L0Aʔ+**\KKdoKiE?O{*'JyIL/esRZZ*e%㥲b/- +b<BǝPZ!` 2~b$ 1ecq䁌�`…P("`hPp{`H $kh "@BϓDby0LF?p-q9#KH�IB襗^҉J@(r0z? +�HQ$A\#�a_ 0�h , X#_PP>t %92\5UVΛ#_!` 1Z@pP 4@@8K1X=ϒpB'd,Qb|sRq󆣑k� (Gd5@+l9Yv'1b# Es`9({YQp<$y#^(4D ˨<G! Yh{KplsjḰPWݻWe|�K[!P LK2茢QH(Z$6|ӏHv Dd\dN[+9y\wF=sseʀnfWŝy#(|`2=skE&L*DP.^(3g/+r-`f|?+>|ס�[]N_|XFOi(SΕ=rV #P3QF0 t}5 � XbN$;u b8<D±dt0\8z<X2f_ɕX.�SKWW˱(tRwj|׬}]1g!#�PyʇQxaðRD'eַlZSSs"D JB6 .ێ0qc`-s|zϖd͚2xtNi>kR^**5{gZt4俿Wc8=�'O(#�,1Id,_^Yz,>DfcCgNӯ\@]BE{T I g|['#G ?+YI~<w$|H87_%&71 S(ٳ*�%՜ (m-D$p.A1J >LVo|V-䳏vɁ'3gΔ_(%1zC%ZA6;GIAb>t"DNuߔ+ D|w|kz@.X28c}zm\+"�LtvIu'^FvPZw~1-= 7ˁ{姵V,_…7ɹyumWH�rV (3"w}Ϥ6ф\$á<W/ˁ N2w>}ZԩSj7�%-;ĄH~Yq> ,wXyk637ݕcX%pXBQ$N W)5>ky RY*K�HgU-ɏԔzum�ИͰT/} ꂚ4QD4*٨4VY@Q'g8״24 ,x?kq.,q808W<18^sl>P&�EiB&R׉=&"$Κ !.QXRT &)3Y!7qoA CJ;+S <xPZZZT�PaOpm]hs1,\K|�`9 Ty=@5?]~Zڀ `\S\˓9Ή'u`�p��*sD-pQ,qhK E˱&Q/uGhZYGч; Z;$ܘGb yf+�1 l/駟jje hRmOհob#�ZCHFwex8Nk9 bYzy!Bw8e2h*MPsCྂy\q: @}@Lbn]l+AX Pgƒ]*|^ע|#on|[jwVȇ,eA>]("-i �Gπ  n99v�’ _�[%M^\�B&ICPeܔ%?g$'<T@Ƀcq51 Řу1x@m45*(ve'<VJ�Jaߛ!DήV\'?zffŲv', fҚ6{\|q&YaݷSsi쀼pZL˿:T9mOII89g*[6կ@9^"dTO0_pMd :::�Ty�<[yn_f8ɾ]$n;u:y{I {$HƍRZ4:(s`3RZyޕkH49 yrRV` ޴RV[)O=Xw-|"o�7P;�S�5mgy:)c6=\xB:Z^fƜ{2mQ[ʅdd" &5oW@dȮGjƳRYV,rHn ;?^!,9?} !O`|Vc(P]%?2CI'I�I.�\[AԬoeMonA?<&>gg͒|$X+IOe9Y|YQ!u9%Ҍyx{w݉qj_}vP^߰V‰rRtB?y@W$dK~?N?y |C)Or\ת_jS84+<̰ 05;;$fɐqjoUh> "_iUK�`?"Vx@diDz|\nC|;zVh<%�zAgG\i뒹RU ;(09AP5煅SSn|0rZV&OOUy7c*̉J[opPodWy2 C*4c`qLb|@vew5{F `1/ߘx</ESz9z'HD ,Y݁&l[eWxrK]\]]vכMyB!~5DWҕ+W�Nd l^HoP<7,tҴl#~S <R9w jժd74}v (#�n/O}דMkBb|y-)2dJp#y#[^`~#drD~oBZK&Tf} A Bb\-^ɼBd⁞45{2ǴдI�vݺuLe,e}@c5;7  GQY?W,[L2eUeb hJܫT~vع]qyN웥}T�l0.^A y_k}ll={$?a �B8@e hVve(D{羈y;t1Wj �l2�lss=sr+{-;vl?$H:`ixw hJqh"-l"aĔżirիW'd^7ek}Hx̟<eK 6o,s'6>aҠ#)Nͯ:dlx3Yfh'V1غҕU*srJ(#�Rv[NFf! mq}B^p=sȶ}=zosr<&KĀ9V'�#N̖1?\ޱc>w ieSؼ;83ҿ� XAsCc: ?JOeTR@jر|r"\Z}y<uT3[@o}| :QqKFfo]CG֚%LY^ lIg|Ν~8,[@؇qm@+̊ [Zk;C,Lc Z yGZ?,z& gh& HphGyDj&)Ö$_~$BI1 }$a>?�!'כ$3Fky(ACD?6&ٸqj�0]qz#x7H<[o|ed""v]7y౟7}也%A&y{W(#�t#_zF�APNK\z/ Ybلכ l267\O65ʙ" Kֆ1Rn����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/module.py�����������������������������������������������������������0000664�0000000�0000000�00000003623�13034501105�0020652�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.calendar import CapCalendarEvent, CATEGORIES from .browser import SenscritiqueBrowser from .calendar import SensCritiquenCalendarEvent __all__ = ['SenscritiqueModule'] class SenscritiqueModule(Module, CapCalendarEvent): NAME = 'senscritique' DESCRIPTION = u'senscritique website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' ASSOCIATED_CATEGORIES = [CATEGORIES.TELE] BROWSER = SenscritiqueBrowser def search_events(self, query): if self.has_matching_categories(query): return self.list_events(query.start_date, query.end_date) def list_events(self, date_from, date_to=None): items = [] for item in self.browser.list_events(date_from, date_to): items.append(item) items.sort(key=lambda o: o.start_date) return items def get_event(self, _id, event=None): return self.browser.get_event(_id, event) def fill_obj(self, event, fields): return self.get_event(event.id, event) OBJECTS = {SensCritiquenCalendarEvent: fill_obj} �������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/pages.py������������������������������������������������������������0000664�0000000�0000000�00000012126�13034501105�0020462�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from .calendar import SensCritiquenCalendarEvent from datetime import date, datetime, timedelta from weboob.capabilities.base import empty from weboob.browser.pages import HTMLPage, JsonPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.standard import Filter, CleanText, Regexp, Join, Format, BrowserURL, Env from weboob.browser.filters.html import Link class Description(Filter): def filter(self, el): header = "//div[@class='pvi-hero-product']" section = "//section[@class='pvi-productDetails']" return Format(u'\n%s\n\n%s%s\n', CleanText("(%s/div[@class='d-rubric-inner']/h1)[1]" % header), Join(u'- ', "%s/ul/li" % section, newline=True, addBefore='- '), Join(u'- Avec ', "%s/div[@class='pvi-productDetails-workers']/a" % section, newline=True, addBefore='- Avec '))(el[0]) class FormatDate(Filter): def __init__(self, pattern, selector): super(FormatDate, self).__init__(selector) self.pattern = pattern def filter(self, _date): return _date.strftime(self.pattern) class Date(Filter): def filter(self, el): str_time = el[0].xpath("time")[0].attrib['datetime'][:-6] _time = datetime.strptime(str_time, '%H:%M:%S') str_date = CleanText('.')(el[0]) _date = date.today() m = re.search('\w* (\d\d?) .*', str_date) if (('Demain' in str_date and str_time[0] != "0") or ('Ce soir' in str_date and str_time[0] == "0")): _date += timedelta(days=1) elif 'Demain' in str_date: _date += timedelta(days=2) elif m: day_number = int(m.group(1)) month = _date.month year = _date.year if day_number < _date.day: month = _date.month % 12 + 1 if _date.month == 12: year = _date.year + 1 _date = date(day=day_number, month=month, year=year) return datetime.combine(_date, _time.time()) class JsonResumePage(JsonPage): def get_resume(self): if self.doc['json']['success']: return self.doc['json']['data'] class EventPage(HTMLPage): @method class get_event(ItemElement): klass = SensCritiquenCalendarEvent obj_url = BrowserURL('event_page', _id=Env('_id')) obj_description = Description('.') class FilmsPage(HTMLPage): @method class iter_films(ListElement): item_xpath = '//li[@class="elgr-mosaic "]/a' class item(ItemElement): klass = SensCritiquenCalendarEvent def condition(self): if '_id' in self.env and self.env['_id']: return Format(u'%s#%s#%s', Regexp(Link('.'), '/film/(.*)'), FormatDate("%Y%m%d%H%M", Date('div[@class="elgr-guide-details"]/div[@class="elgr-data-diffusion"]')), CleanText('./div/span[@class="d-offset"]', replace=[(' ', '-')]))(self) == self.env['_id'] return True def validate(self, obj): if 'date_from' in self.env and self.env['date_from'] and obj.start_date > self.env['date_from']: if not self.env['date_to']: return True else: if empty(obj.end_date): if obj.start_date < self.env['date_to']: return True elif obj.end_date <= self.env['date_to']: return True if '_id' in self.env: return True return False obj_id = Format(u'%s#%s#%s', Regexp(Link('.'), '/film/(.*)'), FormatDate("%Y%m%d%H%M", Date('div/div[@class="elgr-data-diffusion"]')), CleanText('./div/span[@class="d-offset"]', replace=[(' ', '-')])) obj_start_date = Date('div/div[@class="elgr-data-diffusion"]') obj_summary = Format('%s - %s', Regexp(CleanText('./div/img/@alt'), '^Affiche(.*)'), CleanText('./div/span[@class="d-offset"]')) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/senscritique/test.py�������������������������������������������������������������0000664�0000000�0000000�00000002151�13034501105�0020337�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from datetime import datetime class SenscritiqueTest(BackendTest): MODULE = 'senscritique' def test_senscritique(self): l = list(self.backend.list_events(datetime.now())) assert len(l) event = self.backend.get_event(l[0].id) self.assertTrue(event.url, 'URL for event "%s" not found: %s' % (event.id, event.url)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015063�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000000067�13034501105�0017177�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import SfrModule __all__ = ['SfrModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000004576�13034501105�0017134�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from .pages.compose import ClosePage, ComposePage, ConfirmPage, SentPage from .pages.login import LoginPage from weboob.deprecated.browser import Browser, BrowserIncorrectPassword __all__ = ['SfrBrowser'] class SfrBrowser(Browser): DOMAIN = 'www.sfr.fr' PAGES = { 'http://messagerie-.+.sfr.fr/webmail/close_xms_tab.html': ClosePage, 'http://www.sfr.fr/xmscomposer/index.html\?todo=compose': ComposePage, 'http://www.sfr.fr/xmscomposer/mc/envoyer-texto-mms/confirm.html': ConfirmPage, 'https://www.sfr.fr/cas/login\?service=.*': LoginPage, 'http://www.sfr.fr/xmscomposer/mc/envoyer-texto-mms/send.html': SentPage, } def get_nb_remaining_free_sms(self): if not self.is_on_page(ComposePage): self.home() return self.page.get_nb_remaining_free_sms() def home(self): self.location('http://www.sfr.fr/xmscomposer/index.html?todo=compose') def is_logged(self): return 'loginForm' not in [form.name for form in self.forms()] def login(self): service_url = 'http://www.sfr.fr/xmscomposer/j_spring_cas_security_check' self.location('https://www.sfr.fr/cas/login?service=%s' % urllib.quote_plus(service_url), no_login=True) self.page.login(self.username, self.password) if not self.is_logged(): raise BrowserIncorrectPassword() def post_message(self, message): if not self.is_on_page(ComposePage): self.home() self.page.post_message(message) if self.is_on_page(ConfirmPage): self.page.confirm() assert self.is_on_page(ClosePage) or self.is_on_page(SentPage) ����������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000002517�13034501105�0017223�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD���� K��� pHYs�� �� ����tIME  !���tEXtComment�Created with GIMPW��IDATx_h[U?'M[ڎuM 76DTYUt>}PaPdaV'lvXqJ4['Mܟ7mۦiCm;${ d@YT/! Y@|.Y�A�>B`26k#Sn`2;;bo͕-އI`ĺNz[�![K$ѝ%ID5� @�4� @�4� !3}L?\^&=̧g_"ܵ)=>EO5/�U(OLߞl7Cy(e>@/ .8w`* ހZo  ع͙?`7Gvkmӊ*}jX|CDm?r/ 8j~4MҀD7PBX;BcNLk&0FVxAD�ϴZ+O#%kjdS=r{]a+Vz^ڏ߉ g~[$U+)x��.eU;tv tnF:X}Ϟo/D@tUg\=*�J[eUXmK"3Aon1GFc}b޳řake2UJTRӀATA#�=As#ycPp> H @חM"w? zfOCnE{ sgBxuhPS σuk=iǭZK^D0 +Dm1_|%y @?Hq楈I#r4 HLQ.㬖aVͭDD$46.+6H˯#[@>;E  ˽HHmzK~AB!k|0<ϑo?EN xew:}3d9 33X&w#4� @�4� @�4� @�4� ,ɟf1�V�H4؏;QS̯ ,0LٯP�1d&COq@`r5\(rmaAab2QǗ Ѝc}TrQi+x/!1+#3Ptk!p����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000004121�13034501105�0016720�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.messages import CantSendMessage, CapMessages, CapMessagesPost from weboob.capabilities.account import CapAccount, StatusField from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from .browser import SfrBrowser __all__ = ['SfrModule'] class SfrModule(Module, CapAccount, CapMessages, CapMessagesPost): NAME = 'sfr' MAINTAINER = u'Christophe Benz' EMAIL = 'christophe.benz@gmail.com' VERSION = '1.2' DESCRIPTION = 'SFR French mobile phone provider' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('login', label='Login'), ValueBackendPassword('password', label='Password')) BROWSER = SfrBrowser ACCOUNT_REGISTER_PROPERTIES = None def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) # CapMessagesPost methods def get_account_status(self): with self.browser: return (StatusField('nb_remaining_free_sms', 'Number of remaining free SMS', self.browser.get_nb_remaining_free_sms()),) def post_message(self, message): if not message.content.strip(): raise CantSendMessage(u'Message content is empty.') with self.browser: self.browser.post_message(message) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/pages/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016162�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/pages/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020261�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/pages/compose.py�������������������������������������������������������������0000664�0000000�0000000�00000003164�13034501105�0020205�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.messages import CantSendMessage from weboob.deprecated.browser import Page class ClosePage(Page): pass class ComposePage(Page): phone_regex = re.compile('^(\+33|0033|0)(6|7)(\d{8})$') def get_nb_remaining_free_sms(self): nbSms = self.parser.select(self.document.getroot(), '#freeSms',1).text.strip() return nbSms def post_message(self, message): receiver = message.thread.id if self.phone_regex.match(receiver) is None: raise CantSendMessage(u'Invalid receiver: %s' % receiver) self.browser.select_form(nr=0) self.browser['msisdns'] = receiver self.browser['textMessage'] = message.content.encode('utf-8') self.browser.submit() class ConfirmPage(Page): def confirm(self): self.browser.select_form(nr=0) self.browser.submit() class SentPage(Page): pass ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/pages/login.py���������������������������������������������������������������0000664�0000000�0000000�00000002023�13034501105�0017641�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page class LoginPage(Page): def login(self, login, password): self.browser.select_form(nr=0) self.browser['username'] = login self.browser['password'] = password self.browser['remember-me'] = ['on'] self.browser.submit() �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sfr/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000002173�13034501105�0016417�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class SFRTest(BackendTest): MODULE = 'sfr' def test_sfr(self): pass def test_create_default_browser(self): connect = self.backend.create_default_browser() assert connect def test_get_account_status(self): nbSms = self.backend.get_account_status() assert nbSms assert isinstance(nbSms[0].value, str) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/simplyreadit/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016777�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/simplyreadit/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001445�13034501105�0021114�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SimplyreaditModule __all__ = ['SimplyreaditModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/simplyreadit/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000017323�13034501105�0021140�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs����+���tIME 9So���tEXtComment�Created with GIMPW���bKGD������C��.IDATx{i%W}y/ʬ4U۸`C[PAH `-j~�"G4M fM1.גUvUn˗o_bܗe06tK&쨗/{wι>^[}ӿm(DJ.ǓiSU5MEA;enW\3 lD`"dS7(^g컕WFRёC( %JQH(@yv{}~y{z7Hf&'ɪ!)*jFBٶz{,#G2l!놦q!+L1H%3bYJڶ+նⷼW_,H15ȪXPU]p0h0bԍ8h9 OzwҡD1&S8<r г8#&_a+zegCӨ,.|bdH 7iRqU5tUEY"YVt-A{\;zvv'N)oH92z/ O� E 5".\2"/Y|ȃ"ܝJY\Z:uZ5wou?Xt~qR*ξH&IX7ie„t):z"Cl VU]YScuH/[o-$JC~ nX B 6q _dI"^7ⱄ |&ݕ.kPح%Rb>O |IDQDQ oYd1fT<Bl` r<]?x XsqpX|lXX6ffOxByM� <Ah #R"@}""da&<*h5:6lV>B7xuB2etfEQe(̮)>П F<PVgUv1΄/_'bqS@lMi: C@x�8ľQ` "ӌNz] um(X}ߏܢOO҅b!7ƆEM}E4kw&Z,ĝN<p/�/̜xm>~͖,+&$ЂtZF@A2BPNqX| paQW^lqT",Ӄ10p+! UsFQAעpE!3t- x"9/7J�yhᓿHGc,/]zDL>CN,OlB57B~f:>Gr6?FTAu89 D0'<pP)#ON bMV8fH~^sZn2aԬH(V7/3 #s^pg3NN.# rVΞ}<_Y:y#{05!̄Y;ob100QH2ļ !ur1HBJNh4R^3 .l+*4L#ك. BO(4ֶ4u _w*|~H8 CG m~`^rvbD$R #r6၇ܭÔ~${NF€IP  hi(f84yƸ ՛Jjzj;Ѷ 㫥Cek Hn M0e9@ߗDD71o~tC,40!#m(p划҂ `n WڃFrR8B:PuV􉘙JfSs Z]a`_|ft}NQBa'AR?ڵg` # Qʚ%*~ຽú+?X231y<4FV>:vɣS#͡qK\Gieb (�YA#3[a!2HD Ťf&!4"U;frm¡zcw< BS㲦,Qa$8A@Hq$ʙ>|j ; f"JNT$<İ?fYqeP Gʇϵ4U$CQbDbqwn{f:=>Pbb/?U:uꦦF"܀JjLJRN>7+<~!|#zX? kT͖h[z5ECa9ϏT\ S$!ARC!_{?sţ R2 |I<_h%/̜LOld)"~(p&(P^bըIv$+8Ŏ̭w11<9Q kDq{յ IP``sk#50(Ad0K;@ ZhC@"(V]s!%!ȿtr@y1C?2=Fc SЈ('oeD@9~\a 'N=UH׭Z2^niih`6 )46|-@؉\nxgh Sr8@I S@q@_vkVu^lue($Rgv36ah:Ip#|6K *DCa(TS("BD<bXNi4Z}'W#?Oy"OG t-:�AfIU(a@NܠT2G5...)uffNɲH9DfqƢʐؙrDK9i뙩T.� \(׳kg1łm"&<kԸ.)b~lƀHa)B| A) g44әHanŞeŐM"*Vihб( k`<m'\>]_TJ뺋!q>�O5E^eu5#US3 A7 DY<qP7e�vP XAB$O�S4J<`q4J(n\H (0:~7 RUUϞ=3;~1i{crnDRr,'o 54eaَ- yRP˞hnZ,!es 81Dj#ԁHIZNi�!s` ɩ/^ {p OB́X%? YVjt5rvWG!ngdIPFvy-Hbnhh|nTsFغg\•:*vDx%2x5#݉HSÍ^ɺ!*On}B UV>0IV^)b�fS=(FG~W4 X1䍍X;)yd}"Μ`eg#@L:AaJhN8X9zV:~Iګnڝz{(I k++g0²A!=B<\Y):x VQyF RXy'h&:u*bC|/AvHR.PYy,Hv 699ɶ^q7x#,ٶ}o}+իWri677w 7T*f<{(^{W< {ꩧ~#u9ד'O~vO|<|Gi7� ]Ya?я={^|3?ӟ4}{җ}Q}ߣĄ׿Ύ9rk_:'$Amn388Ȓ$w뮻3y{؏cO~)rPԦnc#< dE}c PGz׻ط-s=###u]7ov.=-oaox}Ї>ľo w KmA)Bޗei } =կ~moH~I0_Wg?Yy.8U:j>!Cj>f? ˗G?Qjy|sϥK׾5|}_dos$,C5M>}~ɀ\Ŏ%NMM8 Rĵ_{tu;GI>O$/|#B K\6IJ9sGǹ0Yqff5 VV9r>OqŸP4z ܹs?ϯ|~&H b\0bY8u{ GO<]:b<�{衇؅ �O(  H>K}tI&RDx"D“1(=jX._\qiy 6t5"ZSD(?!(cF-=zts`D|=6!&sfO.9z AvZ3~401 +reetig_).Rq:;FO<zڲ:OB~/2}`5gճZ}YIZe(lm/)[W:6rl9}V? ,Zb$0tgGO>sscK(E"JF|OH&gN]pZd$յ%eoo[GC݁Њ'QD8}+D %uU*mwN<s4 Pg:=vy9QoVLN*/OM-Axai ﳽf 3R|dy,anum;j;B̌zG s|(ꥩFZ=$%ŌD'_(U[/ V=/FmQN--_-åJY|eZ36s %_ ۦ,`y;PU)Fx.˦}#ٸ,+9-'ᩅKD A=,EC7[pSd/gwȮՉ=80nIɩh[BVQKxf:8Ai.A0 #go@lN/xr%D$"[vG,Ah7+ƪn]ijDcK@ =||&=T4r6JVNI\2B~ /> >^vJll|c}jg Q&|_Bt.~7OO==W%"'C� VӄD&78'kN::JSKB#(\\4dМ#ۉ[]8k[Stn@ xVn7Xy*%`."!IqBXt,|5HgY|J.]e4ydz ob|4.,E~Ǡ#.mr١ک;ώ_TKWkn,'n=}YRgt]Kh-?@F jj;V<Wl\*EE\|cf=d4K�bڝFc3ճ4F2z]~(9fOtM!2DVlzu<[O:r`X@xn, Z]_4{q\M=?9Pu*x߼yGBnս@<ȅojP!,4`(^z~ݱ^b`u]mm6աC/p={PmB&*/ <Ru Ɛǃ>,.4wH ]]3A|O-ERӇ_yJqzj0hzWӍR''dʆ>"w`F1`+,-],/"DsY=}V1Om[ �jJs_GD=w暭2[q`;BG0HI{ՍKtFiz6KyK ZџYۊ=\]{:]oå 'O+jhubZѶWS{Zia=w\u <Wctw,7}CTJ A 57sӐOY^}jbn.yGX贐k8{F:80Y{PCK0E{^0F$ԫzm/i& #}ٲmdbT ZnΟu?K#Hfa, ? |U H2./baq D=!m"r0 nUGKv.3tubtv w%+7d_Cdw1/A`/#MQHjn%V~Ym7Xj_?X0tN,Nkh@"q}1 Nݐ$8-$,oírj}bO\:9>y]a;@lB^{wtX,w-ݖUm7W{߿k׷~2bvh#s8QKy`@]U зH8=OA1Ֆl+�_ssJnsr܋E剑c[@lWahp?] ޓ)�E3lL9_'Md0pi[7b(Lt4" V^:h|".S"Ar`HQ Wa^ {s?*_"δ?B?W+V"ج7:SKSe0Wg!#m{+!�2a9EE{٭eT4xڈ2K _E!�i9xqw +)V@ =NND!(ZxzgbE(z HItZΊ22|d ~33 s?of@ {lU=D Y!_֯~ R]oUb@PGߵ·#K^bbIes|�T0Ax~^eM*eT]g@*yi+oWV(ZVpBbg@xJq6_(f#`<_J%oDЯ)ܑ�癠Meppw6WݿW4{W�{r YUmJkHX2a_ 4'\~ h]pyb̯W/_q+YsH@3Lv{z]M#i';kL- :7ߨU3(\(ynPA }VֺHp_eRB] +U|{F{?/7 _x%StL71KXP5?A6mGm/8܎<_+[ht����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/simplyreadit/module.py�����������������������������������������������������������0000664�0000000�0000000�00000002626�13034501105�0020644�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['SimplyreaditModule'] class SimplyreaditModule(GenericComicReaderModule): NAME = 'simplyreadit' DESCRIPTION = 'SimplyReadIt manga reading website' BROWSER_PARAMS = dict( img_src_xpath="//img[@class='open']/@src", page_list_xpath="(//div[contains(@class,'dropdown_right')]/ul[@class='dropdown'])[1]/li/a/@href") ID_TO_URL = 'http://www.simplyread.it/reader/read/%s' ID_REGEXP = r'[^/]+(?:/[^/]+)*' URL_REGEXP = r'.+simplyread.it/reader/read/(%s)/page/.+' % ID_REGEXP PAGES = {r'http://.+\.simplyread.it/reader/read/.+': DisplayPage} ����������������������������������������������������������������������������������������������������������weboob-1.2/modules/simplyreadit/test.py�������������������������������������������������������������0000664�0000000�0000000�00000001732�13034501105�0020333�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class SimplyreaditTest(GenericComicReaderTest): MODULE = 'simplyreadit' def test_download(self): return self._test_download('bonnouji/en/1/3') ��������������������������������������weboob-1.2/modules/societegenerale/�����������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017427�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/__init__.py������������������������������������������������������0000664�0000000�0000000�00000001461�13034501105�0021542�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SocieteGeneraleModule __all__ = ['SocieteGeneraleModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/browser.py�������������������������������������������������������0000664�0000000�0000000�00000015102�13034501105�0021463�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword, BrowserUnavailable from weboob.capabilities.bank import Account from weboob.browser.exceptions import BrowserHTTPNotFound from .pages.accounts_list import AccountsList, AccountHistory, CardsList, LifeInsurance, \ LifeInsuranceHistory, LifeInsuranceInvest, Market, ListRibPage, AdvisorPage from .pages.transfer import RecipientsPage, TransferPage from .pages.login import LoginPage, BadLoginPage, ReinitPasswordPage __all__ = ['SocieteGenerale'] class SocieteGenerale(LoginBrowser): BASEURL = 'https://particuliers.secure.societegenerale.fr' login = URL('https://particuliers.societegenerale.fr/index.html', LoginPage) bad_login = URL('\/acces/authlgn.html', '/error403.html', BadLoginPage) reinit = URL('/acces/changecodeobligatoire.html', ReinitPasswordPage) accounts = URL('/restitution/cns_listeprestation.html', AccountsList) cards_list = URL('/restitution/cns_listeCartes.*.html', CardsList) account_history = URL('/restitution/cns_detail.*\.html', '/lgn/url.html', AccountHistory) market = URL('/brs/cct/comti20.html', Market) life_insurance = URL('/asv/asvcns10.html', '/asv/AVI/asvcns10a.html', '/brs/fisc/fisca10a.html', LifeInsurance) life_insurance_invest = URL('/asv/AVI/asvcns20a.html', LifeInsuranceInvest) life_insurance_history = URL('/asv/AVI/asvcns2[0-9]c.html', LifeInsuranceHistory) list_rib = URL('/restitution/imp_listeRib.html', ListRibPage) advisor = URL('/com/contacts.html', AdvisorPage) recipients = URL('/personnalisation/per_cptBen_modifier_liste.html', RecipientsPage) transfer = URL('/virement/pas_vipon_saisie.html', '/lgn/url.html\?dup', TransferPage) accounts_list = None def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.password.isdigit() or len(self.password) != 6: raise BrowserIncorrectPassword() self.username = self.username[:8] self.login.stay_or_go() try: self.page.login(self.username, self.password) except BrowserHTTPNotFound: raise BrowserIncorrectPassword() if self.login.is_here(): raise BrowserIncorrectPassword() if self.bad_login.is_here(): error = self.page.get_error() if error is None: raise BrowserIncorrectPassword() elif error.startswith('Votre session a'): raise BrowserUnavailable('Session has expired') elif error.startswith('Le service est momentan'): raise BrowserUnavailable(error) else: raise BrowserIncorrectPassword(error) @need_login def get_accounts_list(self): if self.accounts_list is None: self.accounts.stay_or_go() self.accounts_list = [acc for acc in self.page.get_list()] self.list_rib.go() if self.list_rib.is_here(): # Caching rib url, so we don't have to go back and forth for each account for account in self.accounts_list: account._rib_url = self.page.get_rib_url(account) for account in self.accounts_list: if account._rib_url: self.location(account._rib_url) account.iban = self.page.get_iban() return iter(self.accounts_list) @need_login def iter_history(self, account): if not account._link_id: return self.location(account._link_id) if self.cards_list.is_here(): for card_link in self.page.iter_cards(): self.location(card_link) for trans in self.page.iter_transactions(): yield trans elif self.account_history.is_here(): for trans in self.page.iter_transactions(): yield trans elif self.life_insurance.is_here(): self.location('/asv/AVI/asvcns20c.html') for trans in self.page.iter_transactions(): yield trans # go to next page while self.page.doc.xpath('//div[@class="net2g_asv_tableau_pager"]/a[contains(@href, "actionSuivPage")]'): form = self.page.get_form('//form[@id="operationForm"]') form['a100_asv_action'] = 'actionSuivPage' form.submit() for trans in self.page.iter_transactions(): yield trans else: self.logger.warning('This account is not supported') @need_login def iter_investment(self, account): if account.type == Account.TYPE_MARKET: self.location(account._link_id) elif account.type == Account.TYPE_LIFE_INSURANCE: self.location(account._link_id) self.location('/asv/AVI/asvcns20a.html') else: self.logger.warning('This account is not supported') return for invest in self.page.iter_investment(): yield invest @need_login def get_advisor(self): return self.advisor.stay_or_go().get_advisor() @need_login def iter_recipients(self, account): # Seems like all accounts can transfer on all recipients. for recipient in self.recipients.go().iter_recipients(): yield recipient for recipient in self.transfer.go().iter_recipients(account_id=account): yield recipient @need_login def init_transfer(self, account, recipient, transfer): self.transfer.go().init_transfer(account, recipient, transfer) self.page.check_data_consistency(transfer) return self.page.create_transfer(account, recipient, transfer) @need_login def execute_transfer(self, transfer): self.page.confirm() return transfer ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/captcha.py�������������������������������������������������������0000664�0000000�0000000�00000010440�13034501105�0021403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import hashlib from PIL import Image from weboob.tools.log import getLogger class TileError(Exception): def __init__(self, msg, tile=None): Exception.__init__(self, msg) self.tile = tile class Captcha(object): def __init__(self, file, infos): self.inim = Image.open(file) self.infos = infos self.nbr = int(infos["nbrows"]) self.nbc = int(infos["nbcols"]) (self.nx, self.ny) = self.inim.size self.inmat = self.inim.load() self.map = {} self.tiles = [[Tile(y * self.nbc + x) for y in xrange(4)] for x in xrange(4)] def __getitem__(self, coords): x, y = coords return self.inmat[x % self.nx, y % self.ny] def all_coords(self): for y in xrange(self.ny): for x in xrange(self.nx): yield x, y def get_codes(self, code): s = '' num = 0 for c in code: index = self.map[int(c)].id keycode = str(self.infos["grid"][num * self.nbr * self.nbc + index]) s += keycode if num < 5: s += ',' num += 1 return s def build_tiles(self): for ty in xrange(0, self.nbc): y = ty * 23 for tx in xrange(0, self.nbr): x = tx * 24 tile = self.tiles[tx][ty] for yy in xrange(y, y + 23): for xx in xrange(x, x + 24): tile.map.append(self[xx, yy]) num = tile.get_num() if num > -1: tile.valid = True self.map[num] = tile class Tile(object): hash = {'ff1441b2c5f90703ef04e688e399aca5': 1, '53d7f3dfd64f54723b231fc398b6be57': 2, '5bcba7fa2107ba9a606e8d0131c162eb': 3, '9db6e7ed063e5f9a69ab831e6cc0d721': 4, '30ebb75bfa5077f41ccfb72e8c9cc15b': 5, '61e27275e494038e524bc9fbbd0be130': 6, '0e0816f1b743f320ca561f090df0fbb1': 7, '11e7d4a6d447e66a5a112c1d9f7fc442': 8, '2ea3c82768030d91571d360acf7a0f75': 9, '28a834ebbf0238b46d3fffae1a0b781b': 0, '04211db029ce488e07010f618a589c71': -1, '9a1bdf493d4067e98d3f364586c81e9d': 1, '932032493860463bb4a3df7c99a900ad': 2, '59cd90f1fa0b416ecdb440bc16d0b8e7': 3, '53fe822c5efebe5f6fdef0f272c29638': 4, '2082a9c830c0c7c9c22e9c809c6cadf7': 5, '7f24aa97f0037bddcf2a4c8c2dbf5948': 6, '725b6f11f44ecc2e9f6e79e86e3a82a5': 7, '61d57da23894b96fab11f7b83c055bba': 8, '18f6290c1cfaecadc5992e7ef6047a49': 9, '1ce77709ec1d7475685d7b50d6f1c89e': 0, '6718858a509fff4b86604f3096cf65e1': -1, } def __init__(self, _id): self.id = _id self.valid = False self.logger = getLogger('societegenerale.captcha') self.map = [] def __repr__(self): return "<Tile(%02d) valid=%s>" % (self.id, self.valid) def checksum(self): s = '' for pxls in self.map: for pxl in pxls: s += '%02d' % pxl return hashlib.md5(s).hexdigest() def get_num(self): sum = self.checksum() try: return self.hash[sum] except KeyError: self.display() raise TileError('Tile not found ' + sum, self) def display(self): self.logger.debug(self.checksum()) #im = Image.new('RGB', (24, 23)) #im.putdata(self.map) #im.save('/tmp/%s.png' % self.checksum()) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/favicon.png������������������������������������������������������0000664�0000000�0000000�00000001672�13034501105�0021570�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME  2I���tEXtComment�Created with GIMPW��'IDATxKAƿ5[0C-VE*MzQAZHo^_ mⵉYsAzO%ll2DWwCbB޼߾D#@q@7 ���@�� ���@��Pk `#�O5�C�s5č(^&4%v]we�<pG}~ƿanb6|#y5&O\Ee{7wJTR}?kz�`](D鏪�\`o+53}Ʌ?JW'SvKe-r{}f@ sQd@-|'Le>oIo%h- oJ"ꌇ՝:~zu>�U5x*}�iZc@g�m s vuu!ɠT*annCe!";sfYD,slX $q||p89UZU_b4MNLLTb�pll$,Iryyٱ$.gOOOFo܀I, ?0 1 1 14Mvww&�ym΀͊ص5GGGIDʇNSSS3n|cHdTrhjt:bI9D*0 -9Z��mZ夶Ԝ&IW=33:D*A)��@�� ���@�� �@{Z dz#����IENDB`����������������������������������������������������������������������weboob-1.2/modules/societegenerale/module.py��������������������������������������������������������0000664�0000000�0000000�00000012235�13034501105�0021271�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # Copyright(C) 2012-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from decimal import Decimal from datetime import timedelta from weboob.capabilities.bank import CapBankTransfer, AccountNotFound, Account, RecipientNotFound from weboob.capabilities.contact import CapContact from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.base import find_object from .browser import SocieteGenerale from .sgpe.browser import SGEnterpriseBrowser, SGProfessionalBrowser __all__ = ['SocieteGeneraleModule'] class SocieteGeneraleModule(Module, CapBankTransfer, CapContact): NAME = 'societegenerale' MAINTAINER = u'Jocelyn Jaubert' EMAIL = 'jocelyn.jaubert@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Société Générale' CONFIG = BackendConfig( ValueBackendPassword('login', label='Code client', masked=False), ValueBackendPassword('password', label='Code secret'), Value('website', label='Type de compte', default='par', choices={'par': 'Particuliers', 'pro': 'Professionnels', 'ent': 'Entreprises'})) def create_default_browser(self): b = {'par': SocieteGenerale, 'pro': SGProfessionalBrowser, 'ent': SGEnterpriseBrowser} self.BROWSER = b[self.config['website'].get()] return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_accounts(self): for account in self.browser.get_accounts_list(): yield account def get_account(self, _id): return find_object(self.browser.get_accounts_list(), id=_id, error=AccountNotFound) def key(self, tr): # Can't compare datetime and date, so cast them. try: return tr.rdate.date() except AttributeError: return tr.rdate def iter_coming(self, account): if hasattr(self.browser, 'get_cb_operations'): transactions = list(self.browser.get_cb_operations(account)) else: transactions = [tr for tr in self.browser.iter_history(account) if tr._coming] transactions.sort(key=self.key, reverse=True) return transactions def iter_history(self, account): if hasattr(self.browser, 'get_cb_operations'): transactions = list(self.browser.iter_history(account)) else: transactions = [tr for tr in self.browser.iter_history(account) if not tr._coming] transactions.sort(key=self.key, reverse=True) return transactions def iter_investment(self, account): return self.browser.iter_investment(account) def iter_contacts(self): if not hasattr(self.browser, 'get_advisor'): raise NotImplementedError() return self.browser.get_advisor() def iter_transfer_recipients(self, origin_account): if self.config['website'].get() != 'par': raise NotImplementedError() if isinstance(origin_account, Account): origin_account = origin_account.id return self.browser.iter_recipients(origin_account) def init_transfer(self, transfer, **params): if self.config['website'].get() != 'par': raise NotImplementedError() transfer.label = ' '.join(w for w in re.sub('[^0-9a-zA-Z ]+', '', transfer.label).split()) self.logger.info('Going to do a new transfer') if transfer.account_iban: account = find_object(self.iter_accounts(), iban=transfer.account_iban, error=AccountNotFound) else: account = find_object(self.iter_accounts(), id=transfer.account_id, error=AccountNotFound) if transfer.recipient_iban: recipient = find_object(self.iter_transfer_recipients(account.id), iban=transfer.recipient_iban, error=RecipientNotFound) else: recipient = find_object(self.iter_transfer_recipients(account.id), id=transfer.recipient_id, error=RecipientNotFound) transfer.amount = transfer.amount.quantize(Decimal('1.')) return self.browser.init_transfer(account, recipient, transfer) def execute_transfer(self, transfer, **params): if self.config['website'].get() != 'par': raise NotImplementedError() return self.browser.execute_transfer(transfer) def transfer_check_exec_date(self, old_exec_date, new_exec_date): return old_exec_date == new_exec_date or old_exec_date + timedelta(days=1) == new_exec_date �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/pages/�����������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020526�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/pages/__init__.py������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0022625�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/pages/accounts_list.py�������������������������������������������0000664�0000000�0000000�00000044776�13034501105�0023774�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib import datetime from urlparse import parse_qs, urlparse from lxml.etree import XML from lxml.html import fromstring from decimal import Decimal, InvalidOperation import re from weboob.capabilities.base import empty, NotAvailable from weboob.capabilities.bank import Account, Investment from weboob.capabilities.contact import Advisor from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.filters.standard import CleanText, CleanDecimal, Regexp from weboob.browser.pages import LoggedPage from .base import BasePage def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class AccountsList(LoggedPage, BasePage): LINKID_REGEXP = re.compile(".*ch4=(\w+).*") TYPES = {u'Compte Bancaire': Account.TYPE_CHECKING, u'Compte Epargne': Account.TYPE_SAVINGS, u'Compte Sur Livret': Account.TYPE_SAVINGS, u'Compte Titres': Account.TYPE_MARKET, u'Crédit': Account.TYPE_LOAN, u'Ldd': Account.TYPE_SAVINGS, u'Livret': Account.TYPE_SAVINGS, u'PEA': Account.TYPE_SAVINGS, u'PEL': Account.TYPE_SAVINGS, u'Plan Epargne': Account.TYPE_SAVINGS, u'Prêt': Account.TYPE_LOAN, } def get_list(self): def check_valid_url(url): pattern = ['/restitution/cns_detailAVPAT.html', '/restitution/cns_detailPea.html', '/restitution/cns_detailAlterna.html', ] for p in pattern: if url.startswith(p): return False return True for tr in self.doc.getiterator('tr'): if 'LGNTableRow' not in tr.attrib.get('class', '').split(): continue account = Account() for td in tr.getiterator('td'): if td.attrib.get('headers', '') == 'TypeCompte': a = td.find('a') if a is None: break account.label = CleanText('.')(a) account._link_id = a.get('href', '') for pattern, actype in self.TYPES.iteritems(): if account.label.startswith(pattern): account.type = actype break else: if account._link_id.startswith('/asv/asvcns10.html'): account.type = Account.TYPE_LIFE_INSURANCE # Website crashes when going on theses URLs if not check_valid_url(account._link_id): account._link_id = None elif td.attrib.get('headers', '') == 'NumeroCompte': account.id = CleanText(u'.', replace=[(' ', '')])(td) elif td.attrib.get('headers', '') == 'Libelle': text = CleanText('.')(td) if text != '': account.label = text elif td.attrib.get('headers', '') == 'Solde': div = td.xpath('./div[@class="Solde"]') if len(div) > 0: balance = CleanText('.')(div[0]) if len(balance) > 0 and balance not in ('ANNULEE', 'OPPOSITION'): try: account.balance = Decimal(FrenchTransaction.clean_amount(balance)) except InvalidOperation: self.logger.error('Unable to parse balance %r' % balance) continue account.currency = account.get_currency(balance) else: account.balance = NotAvailable if not account.label or empty(account.balance): continue if account._link_id and 'CARTE_' in account._link_id: account.type = account.TYPE_CARD if account.type == Account.TYPE_UNKNOWN: self.logger.debug('Unknown account type: %s', account.label) yield account class CardsList(LoggedPage, BasePage): def iter_cards(self): for tr in self.doc.getiterator('tr'): tds = tr.findall('td') if len(tds) < 4 or tds[0].attrib.get('class', '') != 'tableauIFrameEcriture1': continue yield tr.xpath('.//a')[0].attrib['href'] class Transaction(FrenchTransaction): PATTERNS = [(re.compile(r'^CARTE \w+ RETRAIT DAB.*? (?P<dd>\d{2})/(?P<mm>\d{2})( (?P<HH>\d+)H(?P<MM>\d+))? (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^CARTE \w+ (?P<dd>\d{2})/(?P<mm>\d{2})( A (?P<HH>\d+)H(?P<MM>\d+))? RETRAIT DAB (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^CARTE \w+ REMBT (?P<dd>\d{2})/(?P<mm>\d{2})( A (?P<HH>\d+)H(?P<MM>\d+))? (?P<text>.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile(r'^(?P<category>CARTE) \w+ (?P<dd>\d{2})/(?P<mm>\d{2}) (?P<text>.*)'), FrenchTransaction.TYPE_CARD), (re.compile(r'^(?P<dd>\d{2})(?P<mm>\d{2})/(?P<text>.*?)/?(-[\d,]+)?$'), FrenchTransaction.TYPE_CARD), (re.compile(r'^(?P<category>(COTISATION|PRELEVEMENT|TELEREGLEMENT|TIP)) (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(r'^(\d+ )?VIR (PERM )?POUR: (.*?) (REF: \d+ )?MOTIF: (?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^(?P<category>VIR(EMEN)?T? \w+) (?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^(CHEQUE) (?P<text>.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(r'^(FRAIS) (?P<text>.*)'), FrenchTransaction.TYPE_BANK), (re.compile(r'^(?P<category>ECHEANCEPRET)(?P<text>.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile(r'^(?P<category>REMISE CHEQUES)(?P<text>.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(r'^CARTE RETRAIT (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^TOTAL DES FACTURES (?P<text>.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile(r'^DEBIT MENSUEL CARTE (?P<text>.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile(r'^CREDIT MENSUEL CARTE (?P<text>.*)'), FrenchTransaction.TYPE_CARD_SUMMARY), ] class AccountHistory(LoggedPage, BasePage): def is_here(self): return not CleanText('//h1[contains(text(), "Effectuer un virement")]')(self.doc) debit_date = None def get_part_url(self): for script in self.doc.getiterator('script'): if script.text is None: continue m = re.search('var listeEcrCavXmlUrl="(.*)";', script.text) if m: return m.group(1) return None def iter_transactions(self): url = self.get_part_url() if url is None: # There are no transactions in this kind of account return is_deferred_card = bool(self.doc.xpath(u'//div[contains(text(), "Différé")]')) while True: d = XML(self.browser.open(url).content) el = d.xpath('//dataBody') if not el: return el = el[0] s = unicode(el.text).encode('iso-8859-1') doc = fromstring(s) for tr in self._iter_transactions(doc): if is_deferred_card and tr.type is Transaction.TYPE_CARD: tr.type = Transaction.TYPE_DEFERRED_CARD yield tr el = d.xpath('//dataHeader')[0] if int(el.find('suite').text) != 1: return url = urlparse(url) p = parse_qs(url.query) args = {} args['n10_nrowcolor'] = 0 args['operationNumberPG'] = el.find('operationNumber').text args['operationTypePG'] = el.find('operationType').text args['pageNumberPG'] = el.find('pageNumber').text args['idecrit'] = el.find('idecrit').text or '' args['sign'] = p['sign'][0] args['src'] = p['src'][0] url = '%s?%s' % (url.path, urllib.urlencode(args)) def _iter_transactions(self, doc): t = None for i, tr in enumerate(doc.xpath('//tr')): try: raw = tr.attrib['title'].strip() except KeyError: raw = CleanText('./td[@headers="Libelle"]//text()')(tr) date = CleanText('./td[@headers="Date"]')(tr) if date == '': m = re.search(r'(\d+)/(\d+)', raw) if not m: continue self.debit_date = t.date if t else datetime.date.today() self.debit_date = self.debit_date.replace(day=int(m.group(1)), month=int(m.group(2))) if not t: continue t = Transaction() if 'EnTraitement' in tr.get('class', ''): t._coming = True else: t._coming = False t.set_amount(*reversed([el.text for el in tr.xpath('./td[@class="right"]')])) if date == '': # Credit from main account. t.amount = -t.amount date = self.debit_date t.rdate = t.parse_date(date) t.parse(raw=raw, date=(self.debit_date or date)) yield t def get_iban(self): return CleanText().filter(self.doc.xpath("//font[contains(text(),'IBAN')]/b[1]")[0]).replace(' ', '') class Invest(object): def create_investement(self, cells): inv = Investment() inv.quantity = MyDecimal('.', replace_dots=True, default=NotAvailable)(cells[self.COL_QUANTITY]) inv.unitvalue = MyDecimal('.', replace_dots=True, default=NotAvailable)(cells[self.COL_UNITVALUE]) inv.unitprice = NotAvailable inv.valuation = MyDecimal('.', replace_dots=True, default=NotAvailable)(cells[self.COL_VALUATION]) inv.diff = NotAvailable link = cells[self.COL_LABEL].xpath('a[contains(@href, "CDCVAL=")]')[0] m = re.search('CDCVAL=([^&]+)', link.attrib['href']) if m: inv.code = m.group(1) else: inv.code = NotAvailable return inv class Market(LoggedPage, BasePage, Invest): COL_LABEL = 0 COL_QUANTITY = 1 COL_UNITPRICE = 2 COL_VALUATION = 3 COL_DIFF = 4 def iter_investment(self): doc = self.browser.open('/brs/fisc/fisca10a.html').page.doc num_page = None try: num_page = int(CleanText('.')(doc.xpath(u'.//tr[contains(td[1], "Relevé des plus ou moins values latentes")]/td[2]')[0]).split('/')[1]) except IndexError: pass docs = [doc] if num_page: for n in range(2, num_page + 1): docs.append(self.browser.open('%s%s' % ('/brs/fisc/fisca10a.html?action=12&numPage=', str(n))).page.doc) for doc in docs: # There are two different tables possible depending on the market account type. is_detailed = bool(doc.xpath(u'//span[contains(text(), "Années d\'acquisition")]')) tr_xpath = '//tr[@height and td[@colspan="6"]]' if is_detailed else '//tr[count(td)>5]' for tr in doc.xpath(tr_xpath): cells = tr.findall('td') inv = Investment() inv.label = unicode(cells[self.COL_LABEL].xpath('.//span')[0].attrib['title'].split(' - ')[0]) inv.code = unicode(cells[self.COL_LABEL].xpath('.//span')[0].attrib['title'].split(' - ')[1]) if is_detailed: inv.quantity = MyDecimal('.')(tr.xpath('./following-sibling::tr/td[2]')[0]) inv.unitprice = MyDecimal('.', replace_dots=True)(tr.xpath('./following-sibling::tr/td[3]')[1]) inv.unitvalue = MyDecimal('.', replace_dots=True)(tr.xpath('./following-sibling::tr/td[3]')[0]) inv.valuation = MyDecimal('.')(tr.xpath('./following-sibling::tr/td[4]')[0]) inv.diff = MyDecimal('.')(tr.xpath('./following-sibling::tr/td[5]')[0]) else: inv.quantity = MyDecimal('.')(cells[self.COL_QUANTITY]) inv.diff = MyDecimal('.')(cells[self.COL_DIFF]) inv.unitprice = MyDecimal('.')(cells[self.COL_UNITPRICE].xpath('.//tr[1]/td[2]')[0]) inv.unitvalue = MyDecimal('.')(cells[self.COL_VALUATION].xpath('.//tr[1]/td[2]')[0]) inv.valuation = MyDecimal('.')(cells[self.COL_VALUATION].xpath('.//tr[2]/td[2]')[0]) yield inv class LifeInsurance(LoggedPage, BasePage): def get_error(self): try: return self.doc.xpath("//div[@class='net2g_asv_error_full_page']")[0].text.strip() except IndexError: return super(LifeInsurance, self).get_error() class LifeInsuranceInvest(LifeInsurance, Invest): COL_LABEL = 0 COL_QUANTITY = 1 COL_UNITVALUE = 2 COL_VALUATION = 3 def iter_investment(self): for tr in self.doc.xpath("//table/tbody/tr[starts-with(@class, 'net2g_asv_tableau_ligne_')]"): cells = tr.findall('td') inv = self.create_investement(cells) inv.label = cells[self.COL_LABEL].xpath('a/span')[0].text.strip() inv.description = cells[self.COL_LABEL].xpath('a//div/b[last()]')[0].tail yield inv class LifeInsuranceHistory(LifeInsurance): COL_DATE = 0 COL_LABEL = 1 COL_AMOUNT = 2 COL_STATUS = 3 def iter_transactions(self): for tr in self.doc.xpath("//table/tbody/tr[starts-with(@class, 'net2g_asv_tableau_ligne_')]"): cells = tr.findall('td') link = cells[self.COL_LABEL].xpath('a')[0] # javascript:detailOperation('operationForm', '2'); m = re.search(", '([0-9]+)'", link.attrib['href']) if m: id_trans = m.group(1) else: id_trans = '' trans = Transaction() trans._temp_id = id_trans trans.parse(raw=link.attrib['title'], date=cells[self.COL_DATE].text) trans.set_amount(cells[self.COL_AMOUNT].text) # search for 'Réalisé' trans._coming = 'alis' not in cells[self.COL_STATUS].text.strip() self.set_date(trans) if u'Annulé' in cells[self.COL_STATUS].text.strip(): continue yield trans def set_date(self, trans): """fetch date and vdate from another page""" form = self.doc.xpath('//form[@id="operationForm"]')[0] # go to the page containing the dates data = dict((item.name, item.value or '') for item in form.inputs) data['a100_asv_action'] = 'detail' data['a100_asv_indexOp'] = trans._temp_id doc = self.browser.open('/asv/AVI/asvcns21c.html', data=data).page.doc # process the data date_xpath = '//td[@class="net2g_asv_suiviOperation_element1"]/following-sibling::td' vdate_xpath = '//td[@class="net2g_asv_tableau_cell_date"]' trans.date = self.parse_date(doc, trans, date_xpath, 1) trans.rdate = trans.date trans.vdate = self.parse_date(doc, trans, vdate_xpath, 0) @staticmethod def parse_date(doc, trans, xpath, index): elem = doc.xpath(xpath)[index] if elem.text: return trans.parse_date(elem.text.strip()) else: return NotAvailable class ListRibPage(LoggedPage, BasePage): def get_rib_url(self, account): for div in self.doc.xpath('//table//td[@class="fond_cellule"]//div[@class="tableauBodyEcriture1"]//table//tr'): if account.id == CleanText().filter(div.xpath('./td[2]//div/div')).replace(' ', ''): href = CleanText().filter(div.xpath('./td[4]//a/@href')) m = re.search("javascript:windowOpenerRib\('(.*?)'(.*)\)", href) if m: return m.group(1) class AdvisorPage(BasePage): def get_advisor(self): fax = CleanText('//div[contains(text(), "Fax")]/following-sibling::div[1]', replace=[(' ', '')])(self.doc) agency = CleanText('//div[contains(@class, "agence")]/div[last()]')(self.doc) address = CleanText('//div[contains(text(), "Adresse")]/following-sibling::div[1]')(self.doc) for div in self.doc.xpath('//div[div[text()="Contacter mon conseiller"]]'): a = Advisor() a.name = CleanText('./div[2]')(div) a.phone = Regexp(CleanText(u'./following-sibling::div[div[contains(text(), "Téléphone")]][1]/div[last()]', replace=[(' ', '')]), '([+\d]+)')(div) a.fax = fax a.agency = agency a.address = address a.mobile = a.email = NotAvailable a.role = u"wealth" if "patrimoine" in CleanText('./div[1]')(div) else u"bank" yield a ��weboob-1.2/modules/societegenerale/pages/base.py����������������������������������������������������0000664�0000000�0000000�00000002547�13034501105�0022022�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from decimal import Decimal from weboob.capabilities.base import NotAvailable from weboob.browser.pages import HTMLPage from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.browser.filters.standard import CleanText class BasePage(HTMLPage): def get_error(self): try: return self.doc.xpath('//span[@class="error_msg"]')[0].text.strip() except IndexError: return None def parse_decimal(self, td): value = CleanText('.')(td) if value: return Decimal(FrenchTransaction.clean_amount(value)) else: return NotAvailable ���������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/pages/login.py���������������������������������������������������0000664�0000000�0000000�00000010376�13034501105�0022217�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from cStringIO import StringIO from base64 import b64decode from logging import error import re from weboob.tools.json import json from weboob.exceptions import BrowserUnavailable from weboob.exceptions import BrowserPasswordExpired from .base import BasePage from ..captcha import Captcha, TileError class PasswordPage(object): STRANGE_KEY = ["180","149","244","125","115","058","017","071","075","119","167","040","066","083","254","151","212","245","193","224","006","068","139","054","089","083","111","208","105","235","109","030","130","226","155","245","157","044","061","233","036","101","145","103","185","017","126","142","007","192","239","140","133","250","194","222","079","178","048","184","158","158","086","160","001","114","022","158","030","210","008","067","056","026","042","113","043","169","128","051","107","112","063","240","108","003","079","059","053","127","116","084","157","203","244","031","062","012","062","093"] strange_map = None def decode_grid(self, infos): grid = b64decode(infos['grid']) grid = map(int, re.findall('[0-9]{3}', grid)) n = int(infos['nbrows']) * int(infos['nbcols']) self.strange_map = list(grid[:n]) grid = list(grid[n:]) new_grid = list(grid) s = n u = list(infos['crypto']) for j in xrange(s): u[j] = '%02d' % ord(u[j]) for i in xrange(5, 0, -1): for j in xrange(s): new_grid[i*s+j] = '%03d' % (new_grid[i*s+j]^new_grid[(i-1)*s+j]) for j in xrange(s): new_grid[j] = '%03d' % (new_grid[j]^int(self.STRANGE_KEY[j])^self.strange_map[j]) for j in xrange(s): self.strange_map[j] = int(u[j])^self.strange_map[j] return new_grid class LoginPage(BasePage, PasswordPage): def on_load(self): for td in self.doc.getroot().cssselect('td.LibelleErreur'): if td.text is None: continue msg = td.text.strip() if 'indisponible' in msg: raise BrowserUnavailable(msg) def login(self, login, password): url = self.browser.BASEURL + '//sec/vkm/gen_crypto?estSession=0' headers = {'Referer': 'https://particuliers.societegenerale.fr/index.html'} infos_data = self.browser.open(url, headers=headers).content infos_data = re.match('^_vkCallback\((.*)\);$', infos_data).group(1) infos = json.loads(infos_data.replace("'", '"')) infos['grid'] = self.decode_grid(infos) url = self.browser.BASEURL + '//sec/vkm/gen_ui?modeClavier=0&cryptogramme=' + infos["crypto"] content = self.browser.open(url).content img = Captcha(StringIO(content), infos) try: img.build_tiles() except TileError as err: error("Error: %s" % err) if err.tile: err.tile.display() form = self.get_form(id='n2g_authentification') pwd = img.get_codes(password[:6]) t = pwd.split(',') newpwd = ','.join([t[self.strange_map[j]] for j in xrange(6)]) form['codcli'] = login.encode('iso-8859-1') form['user_id'] = login.encode('iso-8859-1') form['codsec'] = newpwd form['cryptocvcs'] = infos["crypto"].encode('iso-8859-1') form['vkm_op'] = 'auth' form.url = 'https://particuliers.secure.societegenerale.fr//acces/authlgn.html' del form['button'] form.submit() class BadLoginPage(BasePage): pass class ReinitPasswordPage(BasePage): def on_load(self): raise BrowserPasswordExpired() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/pages/transfer.py������������������������������������������������0000664�0000000�0000000�00000023617�13034501105�0022735�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime, timedelta import re from cStringIO import StringIO from logging import error from weboob.tools.json import json from weboob.browser.pages import LoggedPage from weboob.browser.elements import method, ListElement, ItemElement from weboob.capabilities.bank import Recipient, TransferError, Transfer from weboob.capabilities.base import find_object, NotAvailable from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, \ Env, Date from weboob.browser.filters.html import Attr from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.tools.ordereddict import OrderedDict from ..captcha import Captcha, TileError from .base import BasePage from .login import PasswordPage class MyRecipient(ItemElement): klass = Recipient # Assume all recipients currency is euros. obj_currency = u'EUR' def obj_enabled_at(self): return datetime.now().replace(microsecond=0) class RecipientsPage(LoggedPage, BasePage): @method class iter_recipients(ListElement): item_xpath = '//div[@class="items-groups"]/a' class Item(MyRecipient): obj_id = obj_iban = CleanText('./div[1]/div[1]/span[1]') obj_bank_name = CleanText('./div[1]/div[2]/span[1]') obj_category = u'Externe' def obj_label(self): first_label = CleanText('./div[1]/div[3]/span[1]')(self) second_label = CleanText('./div[1]/div[3]/span[2]')(self) return first_label if first_label == second_label else ('%s %s' % (first_label, second_label)).strip() class TransferPage(LoggedPage, BasePage, PasswordPage): def on_load(self): error_msg = CleanText('//span[@class="error_msg"]')(self.doc) if error_msg: raise TransferError(error_msg) @method class iter_recipients(ListElement): item_xpath = '//select[@id="SelectDest"]/optgroup[@label="Vos comptes"]/option | //select[@id="SelectDest"]/optgroup[@label="Procurations"]/option' class Item(MyRecipient): validate = lambda self, obj: self.obj_id(self) != self.env['account_id'] obj_id = Env('id') obj_label = Env('label') obj_bank_name = u'Société Générale' obj_category = u'Interne' obj_iban = Env('iban') def parse(self, el): _id = Regexp(CleanText('.', replace=[(' ', '')]), '(\d+)', default=NotAvailable)(self) if _id: account = find_object(self.page.browser.get_accounts_list(), id=_id) if not account: accounts = [acc for acc in self.page.browser.get_accounts_list() if acc.id in _id] assert len(accounts) == 1 account = accounts[0] self.env['id'] = _id else: account = find_object(self.page.browser.get_accounts_list(), label=Regexp(CleanText('.'), '- (.*)')(self)) self.env['id']= account.id self.env['label'] = account.label self.env['iban'] = account.iban def get_params(self, _id, _type): for script in [sc for sc in self.doc.xpath('//script') if sc.text]: accounts = re.findall('TableauComptes%s.*?\)' % _type, script.text) if accounts: break for account in accounts: params = re.findall('"(.*?)"', account) if params[2] + params[3] == _id or params[3] + params[4] == _id or params[-2] == _id: return params def get_account_value(self, _id): for option in self.doc.xpath('//select[@id="SelectEmet"]//option'): if _id in CleanText('.', replace=[(' ', '')])(option): attr = Attr('.', 'value')(option) return attr def init_transfer(self, account, recipient, transfer): try: assert account.currency == recipient.currency == 'EUR' except AssertionError: raise TransferError('wrong currency') origin_params = self.get_params(account.id, 'Emetteurs') recipient_params = self.get_params(recipient.id, 'Destinataires') data = OrderedDict() value = self.get_account_value(account.id) data['dup'] = re.search('dup=(.*?)(&|$)', value).group(1) data['src'] = re.search('src=(.*?)(&|$)', value).group(1) data['sign'] = re.search('sign=(.*?)(&|$)', value).group(1) data['cdbqem'] = origin_params[1] data['cdguem'] = origin_params[2] data['nucpem'] = origin_params[3] data['clriem'] = origin_params[4] data['libeem'] = origin_params[5] data['grroem'] = origin_params[6] data['cdprem'] = origin_params[7] data['liprem'] = origin_params[8] # This one seem to be set in stone. data['inrili'] = 'N' data['toprib'] = '0' if recipient_params[0] == 'pic_a_recuperer' else '1' if recipient_params[1]: data['idprde'] = recipient_params[1] data['cdbqde'] = recipient_params[2] data['cdgude'] = recipient_params[3] data['nucpde'] = recipient_params[4] data['clride'] = recipient_params[5] data['libede'] = recipient_params[6] data['grrode'] = recipient_params[7] if recipient_params[8]: data['cdprde'] = recipient_params[8] if recipient_params[9]: data['liprde'] = recipient_params[9] if recipient_params[10]: data['tycpde'] = recipient_params[10] data['formatCompteBenef'] = recipient_params[12] data['nomBenef'] = recipient_params[13] data['codeBICBenef'] = recipient_params[15] data['codeIBANBenef'] = recipient_params[16] # This needs the currency to be euro. data['mntval'] = transfer.amount * 100 data['mntcdc'] = '2' data['mntcdv'] = 'EUR' data['datvir'] = transfer.exec_date.strftime('%Y%m%d') data['motvir'] = transfer.label # Initiate transfer self.browser.location('/lgn/url.html?%s' % '&'.join(['%s=%s' % (k, v) for k, v in data.iteritems()])) def check_data_consistency(self, transfer): amount = CleanDecimal('.//td[@headers="virement montant"]', replace_dots=True)(self.doc) label = CleanText('.//td[@headers="virement motif"]')(self.doc) exec_date = Date(CleanText('.//td[@headers="virement date"]'), dayfirst=True)(self.doc) try: assert transfer.amount == amount except AssertionError: raise TransferError('data consistency failed, %s changed from %s to %s' % ('amount', transfer.amount, amount)) try: assert transfer.label in label except AssertionError: raise TransferError('data consistency failed, %s changed from %s to %s' % ('label', transfer.label, label)) try: assert transfer.exec_date == exec_date or transfer.exec_date + timedelta(days=1) == exec_date except AssertionError: raise TransferError('data consistency failed, %s changed from %s to %s' % ('exec_date', transfer.exec_date, exec_date)) def create_transfer(self, account, recipient, transfer): transfer = Transfer() transfer.currency = FrenchTransaction.Currency('.//td[@headers="virement montant"]')(self.doc) transfer.amount = CleanDecimal('.//td[@headers="virement montant"]', replace_dots=True)(self.doc) transfer.account_iban = CleanText('//td[@headers="emetteur IBAN"]', replace=[(' ', '')])(self.doc) transfer.recipient_iban = CleanText('//td[@headers="beneficiaire IBAN"]', replace=[(' ','')])(self.doc) transfer.account_id = account.id transfer.recipient_id = recipient.id transfer.exec_date = Date(CleanText('.//td[@headers="virement date"]'), dayfirst=True)(self.doc) transfer.label = CleanText('.//td[@headers="virement motif"]')(self.doc) transfer.account_label = account.label transfer.recipient_label = recipient.label transfer._account = account transfer._recipient = recipient transfer.account_balance = account.balance return transfer def confirm(self): form = self.get_form(id='authentification') url = self.browser.BASEURL + '//sec/vkm/gen_crypto?estSession=0' infos_data = self.browser.open(url).content infos_data = re.match('^_vkCallback\((.*)\);$', infos_data).group(1) infos = json.loads(infos_data.replace("'", '"')) infos['grid'] = self.decode_grid(infos) url = self.browser.BASEURL + '/sec/vkm/gen_ui?modeClavier=0&cryptogramme=' + infos["crypto"] content = self.browser.open(url).content img = Captcha(StringIO(content), infos) try: img.build_tiles() except TileError as err: error("Error: %s" % err) if err.tile: err.tile.display() pwd = img.get_codes(self.browser.password[:6]) t = pwd.split(',') newpwd = ','.join([t[self.strange_map[j]] for j in xrange(6)]) form['codsec'] = newpwd form['cryptocvcs'] = infos["crypto"].encode('iso-8859-1') form['vkm_op'] = 'sign' form.submit() �����������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/sgpe/������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020365�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/sgpe/__init__.py�������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0022464�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/sgpe/browser.py��������������������������������������������������0000664�0000000�0000000�00000011715�13034501105�0022427�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.browsers import LoginBrowser, need_login from weboob.browser.url import URL from weboob.browser.exceptions import ClientError from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, CardsPage, CardHistoryPage, ChangePassPage from .json_pages import AccountsJsonPage, BalancesJsonPage, HistoryJsonPage __all__ = ['SGProfessionalBrowser', 'SGEnterpriseBrowser'] class SGPEBrowser(LoginBrowser): login = URL('$', LoginPage) cards = URL('/Pgn/.+PageID=Cartes&.+', CardsPage) cards_history = URL('/Pgn/.+PageID=ReleveCarte&.+', CardHistoryPage) change_pass = URL('/gao/changer-code-secret-expire-saisie.html', '/gao/changer-code-secret-inscr-saisie.html', '/gao/inscrire-utilisateur-saisie.html', ChangePassPage) def is_logged(self): if not self.page or self.login.is_here(): return False error = self.page.get_error() if error is None: return True return False def do_login(self): assert isinstance(self.username, basestring) assert isinstance(self.password, basestring) if not self.password.isdigit(): raise BrowserIncorrectPassword('Password must be 6 digits long.') self.login.stay_or_go() self.session.cookies.set('PILOTE_OOBA', 'true') try: self.page.login(self.username, self.password) except ClientError: raise BrowserIncorrectPassword() # force page change if not self.accounts.is_here(): self.go_accounts() if not self.is_logged(): raise BrowserIncorrectPassword() def card_history(self, account, coming): page = 1 while page: self.location('/Pgn/NavigationServlet?PageID=ReleveCarte&MenuID=%sOPF&Classeur=1&Rib=%s&Carte=%s&Date=%s&PageDetail=%s&Devise=%s' % \ (self.MENUID, account.id, coming['carte'], coming['date'], page, account.currency)) for transaction in self.page.iter_transactions(date=coming['date']): yield transaction if self.page.has_next(): page += 1 else: page = False @need_login def get_cb_operations(self, account): self.location('/Pgn/NavigationServlet?PageID=Cartes&MenuID=%sOPF&Classeur=1&NumeroPage=1&Rib=%s&Devise=%s' % (self.MENUID, account.id, account.currency)) for coming in self.page.get_coming_list(): for tr in self.card_history(account, coming): yield tr def iter_investment(self, account): raise NotImplementedError() class SGEnterpriseBrowser(SGPEBrowser): BASEURL = 'https://entreprises.secure.societegenerale.fr' LOGIN_FORM = 'auth' MENUID = 'BANREL' CERTHASH = '2231d5ddb97d2950d5e6fc4d986c23be4cd231c31ad530942343a8fdcc44bb99' accounts = URL('/icd/syd-front/data/syd-comptes-accederDepuisMenu.json', AccountsJsonPage) balances = URL('/icd/syd-front/data/syd-comptes-chargerSoldes.json', BalancesJsonPage) history = URL('/icd/syd-front/data/syd-comptes-chargerReleve.json', '/icd/syd-front/data/syd-intraday-chargerDetail.json', HistoryJsonPage) history_next = URL('/icd/syd-front/data/syd-comptes-chargerProchainLotEcriture.json', HistoryJsonPage) def go_accounts(self): self.accounts.go() @need_login def get_accounts_list(self): accounts = [] accounts.extend(self.accounts.stay_or_go().iter_accounts()) for acc in self.balances.go().populate_balances(accounts): yield acc @need_login def iter_history(self, account): transactions = [] transactions.extend(self.history.go(data={'cl500_compte': account._id, 'cl200_typeReleve': 'valeur'}).iter_history()) transactions.extend(self.location('/icd/syd-front/data/syd-intraday-chargerDetail.json', data={'cl500_compte': account._id}).page.iter_history()) return iter(transactions) class SGProfessionalBrowser(SGEnterpriseBrowser): BASEURL = 'https://professionnels.secure.societegenerale.fr' LOGIN_FORM = 'auth_reco' MENUID = 'SBOREL' CERTHASH = '9f5232c9b2283814976608bfd5bba9d8030247f44c8493d8d205e574ea75148e' ���������������������������������������������������weboob-1.2/modules/societegenerale/sgpe/json_pages.py�����������������������������������������������0000664�0000000�0000000�00000012310�13034501105�0023064�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Baptiste Delpey # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import requests from weboob.browser.pages import LoggedPage, JsonPage, pagination from weboob.browser.elements import ItemElement, method, DictElement from weboob.browser.filters.standard import CleanDecimal, CleanText, Date, Format, BrowserURL from weboob.browser.filters.json import Dict from weboob.capabilities.base import Currency from weboob.capabilities import NotAvailable from weboob.capabilities.bank import Account from .pages import Transaction class AccountsJsonPage(LoggedPage, JsonPage): TYPES = {u'COMPTE COURANT': Account.TYPE_CHECKING, u'COMPTE PERSONNEL': Account.TYPE_CHECKING, u'CPTE PRO': Account.TYPE_CHECKING, u'CPTE PERSO': Account.TYPE_CHECKING, u'CODEVI': Account.TYPE_SAVINGS, u'CEL': Account.TYPE_SAVINGS, u'Ldd': Account.TYPE_SAVINGS, u'Livret': Account.TYPE_SAVINGS, u'PEA': Account.TYPE_SAVINGS, u'PEL': Account.TYPE_SAVINGS, u'Plan Epargne': Account.TYPE_SAVINGS, u'Prêt': Account.TYPE_LOAN, } def iter_accounts(self): for classeur in self.doc['donnees']['classeurs']: title = classeur['title'] for compte in classeur['comptes']: a = Account() a.label = CleanText().filter(compte['libelle']) a._id = compte['id'] a.iban = compte['iban'].replace(' ', '') # id based on iban to match ids in database. a.id = a.iban[4:-2] if len(a.iban) == 27 else a.iban a.type = self.obj_type(a.label) a._agency = compte['agenceGestionnaire'] a._title = title yield a def obj_type(self, label): for wording, acc_type in self.TYPES.iteritems(): if wording.lower() in label.lower(): return acc_type return Account.TYPE_CHECKING def get_error(self): if self.doc['commun']['statut'] == 'nok': return self.doc['commun']['raison'] return None class BalancesJsonPage(LoggedPage, JsonPage): def populate_balances(self, accounts): for account in accounts: acc_dict = self.doc['donnees']['compteSoldesMap'][account._id] account.balance = CleanDecimal(replace_dots=True).filter(acc_dict['soldeComptable']) account.currency = Currency.get_currency(acc_dict['deviseSoldeComptable']) yield account class HistoryJsonPage(LoggedPage, JsonPage): @pagination @method class iter_history(DictElement): def __init__(self, *args, **kwargs): super(DictElement, self).__init__(*args, **kwargs) self.item_xpath = 'donnees/compte/operations' if not 'Prochain' in self.page.url else 'donnees/ecritures' def condition(self): return 'donnees' in self.page.doc def next_page(self): d = self.page.doc['donnees']['compte'] if not 'Prochain' in self.page.url else self.page.doc['donnees'] if 'ecrituresRestantes' in d: next_ope = d['ecrituresRestantes'] next_data = d['sceauEcriture'] else: next_ope = d['operationsRestantes'] next_data = d['sceauOperation'] if next_ope: data = {} data['b64e4000_sceauEcriture'] = next_data if not 'intraday' in self.page.url: data['cl200_typeReleve'] = 'valeur' return requests.Request("POST", BrowserURL('history_next')(self), data=data) class item(ItemElement): klass = Transaction # This is 'Date de valeur' obj_date = Date(Dict('dVl'), dayfirst=True) obj__date = Date(Dict('date', default=None), dayfirst=True, default=NotAvailable) obj__coming = False obj_raw = Transaction.Raw(Format('%s %s %s', Dict('l1'), Dict('l2'), Dict('l3'))) # We have l4 and l5 too most of the time, but it seems to be unimportant and would make label too long. #tr.label = ' '.join([' '.join(transaction[l].strip().split()) for l in ['l1', 'l2', 'l3']]) def obj_amount(self): return CleanDecimal(Dict('c', default=None), replace_dots=True, default=None)(self) or \ CleanDecimal(Dict('d'), replace_dots=True)(self) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/sgpe/pages.py����������������������������������������������������0000664�0000000�0000000�00000016412�13034501105�0022042�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from logging import error import re from cStringIO import StringIO from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, CleanDecimal, Date, Env from weboob.browser.filters.html import Attr from weboob.tools.capabilities.bank.transactions import FrenchTransaction from weboob.exceptions import ActionNeeded from weboob.tools.json import json from weboob.capabilities.base import NotAvailable from ..captcha import Captcha, TileError class Transaction(FrenchTransaction): PATTERNS = [(re.compile(r'^CARTE \w+ RETRAIT DAB.* (?P<dd>\d{2})/(?P<mm>\d{2})( (?P<HH>\d+)H(?P<MM>\d+))? (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^CARTE \w+ (?P<dd>\d{2})/(?P<mm>\d{2})( A (?P<HH>\d+)H(?P<MM>\d+))? RETRAIT DAB (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), (re.compile(r'^CARTE \w+ REMBT (?P<dd>\d{2})/(?P<mm>\d{2})( A (?P<HH>\d+)H(?P<MM>\d+))? (?P<text>.*)'), FrenchTransaction.TYPE_PAYBACK), (re.compile(r'^DEBIT MENSUEL CARTE.*'), FrenchTransaction.TYPE_CARD_SUMMARY), (re.compile(r'^(?P<category>CARTE) \w+ (?P<dd>\d{2})/(?P<mm>\d{2}) (?P<text>.*)'), FrenchTransaction.TYPE_CARD), (re.compile(r'^(?P<dd>\d{2})(?P<mm>\d{2})/(?P<text>.*?)/?(-[\d,]+)?$'), FrenchTransaction.TYPE_CARD), (re.compile(r'^REMISE CB /(?P<dd>\d{2})/(?P<mm>\d{2}) (?P<text>.*?)/?(-[\d,]+)?$'), FrenchTransaction.TYPE_CARD), (re.compile(r'^(?P<category>(COTISATION|PRELEVEMENT|TELEREGLEMENT|TIP|PRLV)) (?P<text>.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(r'^(\d+ )?VIR (PERM )?POUR: (.*?) (REF: \d+ )?MOTIF: (?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^(?P<category>VIR(EMEN)?T? \w+) (?P<text>.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^(CHEQUE) (?P<text>.*)'), FrenchTransaction.TYPE_CHECK), (re.compile(r'^(FRAIS) (?P<text>.*)'), FrenchTransaction.TYPE_BANK), (re.compile(r'^(?P<category>ECHEANCEPRET)(?P<text>.*)'), FrenchTransaction.TYPE_LOAN_PAYMENT), (re.compile(r'^(?P<category>REMISE CHEQUES)(?P<text>.*)'), FrenchTransaction.TYPE_DEPOSIT), (re.compile(r'^CARTE RETRAIT (?P<text>.*)'), FrenchTransaction.TYPE_WITHDRAWAL), ] _coming = False class SGPEPage(HTMLPage): def get_error(self): err = self.doc.getroot().cssselect('div.ngo_mire_reco_message') \ or self.doc.getroot().cssselect('#nge_zone_centre .nge_cadre_message_utilisateur') \ or self.doc.xpath(u'//div[contains(text(), "Echec de connexion à l\'espace Entreprises")]') \ or self.doc.xpath(u'//div[contains(@class, "waitAuthJetonMsg")]') if err: return err[0].text.strip() class ChangePassPage(SGPEPage): def on_load(self): message = (CleanText('//div[@class="ngo_gao_message_intro"]')(self.doc) or CleanText('//div[@class="ngo_gao_intro"]')(self.doc)) raise ActionNeeded(message) class LoginPage(SGPEPage): def login(self, login, password): infos_data = self.browser.open('/sec/vk/gen_crypto?estSession=0').content infos_data = re.match('^_vkCallback\((.*)\);$', infos_data).group(1) infos = json.loads(infos_data.replace("'", '"')) url = '/sec/vk/gen_ui?modeClavier=0&cryptogramme=' + infos["crypto"] img = Captcha(StringIO(self.browser.open(url).content), infos) try: img.build_tiles() except TileError as err: error("Error: %s" % err) if err.tile: err.tile.display() form = self.get_form(name=self.browser.LOGIN_FORM) form['user_id'] = login form['codsec'] = img.get_codes(password[:6]) form['cryptocvcs'] = infos['crypto'] form['vk_op'] = 'auth' form.url = '/authent.html' try: form.pop('button') except KeyError: pass form.submit() class CardsPage(LoggedPage, SGPEPage): def get_coming_list(self): coming_list = [] for a in self.doc.xpath('//a[contains(@onclick, "changeCarte")]'): m = re.findall("'([^']+)'", Attr(a.xpath('.'), 'onclick')(self)) params = {} params['carte'] = m[1] params['date'] = m[2] coming_list.append(params) return coming_list class CardHistoryPage(LoggedPage, SGPEPage): @method class iter_transactions(ListElement): item_xpath = '//table[@id="tab-corps"]//tr' class item(ItemElement): klass = Transaction obj_rdate = Date(CleanText('./td[1]'), dayfirst=True) obj_date = Date(Env('date'), dayfirst=True, default=NotAvailable) obj_raw = Transaction.Raw(CleanText('./td[2]')) obj_type = Transaction.TYPE_DEFERRED_CARD obj__coming = True obj_nopurge = True def obj_amount(self): return CleanDecimal('./td[3]', replace_dots=True, default=NotAvailable)(self) \ or CleanDecimal('./td[2]', replace_dots=True)(self) def condition(self): return CleanText('./td[2]')(self) def has_next(self): current = None total = None for script in self.doc.xpath('//script'): if script.text is None: continue m = re.search('var pageActive\s+= (\d+)', script.text) if m: current = int(m.group(1)) m = re.search("var nombrePage\s+= (\d+)", script.text) if m: total = int(m.group(1)) if all((current, total)) and current < total: return True return False ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/societegenerale/test.py����������������������������������������������������������0000664�0000000�0000000�00000002063�13034501105�0020761�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jocelyn Jaubert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class SocieteGeneraleTest(BackendTest): MODULE = 'societegenerale' def test_societegenerale(self): l = list(self.backend.iter_accounts()) self.assertTrue(len(l) > 0) a = l[0] list(self.backend.iter_coming(a)) list(self.backend.iter_history(a)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/somafm/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015553�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/somafm/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001433�13034501105�0017665�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SomaFMModule __all__ = ['SomaFMModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/somafm/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000007170�13034501105�0017713�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME3 A����tEXtComment�Created with GIMPW�� IDATx{\u?Νξ۳f񲀱Ā5DQDIFBE"Ap%R(R(Qj+uUE4$ۀ^ޗw;?ιeX$f>~wXcy,<l�9{ԆUi48?{y5h<Ϋ5ߗw4/&|� `pGIo >8NS+OYQhik}s8:; yWG` L0BɇjBڄ 5Ț<gF�$wn 9{v[@vVos%l>^'6LQ%[qпyKkq}U@Ԩ,8é*);z`2K-qȤ9_M"!C;Er8 3jB!'0=vCCxi�@D[$g]]ew n. 0<#2<_U$F1[[\?&�V(w_|ܴ-NOx(xR=ӂe;ugJ{aZS="_eY^tG#')ؙ֐l_�0 jXW}(Pt lO@[.I.x�||XgOáb<@47}ɦASFN/#o]L<W 㐝?{ TAX}8b /7�豠߀Q Q' h2cDT|<Go}I�M`~uF-g}_sA2)N=0p�nkiF`,N$slǧ9ͬ3ʫF]l=`M0ׇEa#0IGٓ[a�x ƶNز Kλ  n_ /ēf=ܶ( � dظQf= X !-`ƅ_;.ke(z%qH:}xEmalp0y yxm<k{zJ1{uleJC11^P6% M8ps N8pԮDgf< WKܻ's1�%9O"* }7Z >!#oH+9Z-XմIÚkaC(`K|Q VCl-YE[R#`8"}cΥ*kj8Aޓ�Ͽ+X2AK`8USi,I.ׅRA V`I2%u<[)]a._< @WMq$KCɇ| /m@+~Dcnw.+^gp lBqკ)|% \VjܡwxTDIW�q޳9.cՀT.+|z8M?tusPiXՆdHS�|H$b,`:3$\D.#[ syR!*2BHČk I .<QG|ʕVDي$a�v?'ᅉ|mS,ɣp-qו*C 7AnV Tbx1/;U9n~H4SJV%V[pJýEkz_k|XkXzL R+^`xe/v%zni:,n s+*,l9)05&jq7a \V*,]yZLX[RoBj~T H@Ҁ[** OHK3 rt`mbrB.+ NQ2&HkdTlnNKoMn`<DԐdƒk&B-{%MM1)R?{~& ܫx;ߟx;9wA`✤S"w-ml:{> -)"F7{8~[Z{|@pXQw`g"#LwtEF%n0BLtfxןVE`x 3?yjކ|Kly%w42Goxg~0!kh\+ O�VH0F^�0!|Qoix@^զ㱾a*f�E|vP>߁\v^CN }X0qOFA>gERKkܤFzCC~z7ڤQӼH?)Q/07B}_H+ �8Qs@KR^/Z2p8 u?X^6MՊ60�[:wRu7WR\)XX�k5]J0;i` �>}U!`?p۠h,�z }W�K=_S{Y5 f5 .z$pS,�dt5:.*�D(0^^pox[ c)58!<Ʒ@躚4luBzWjd4C SZ�m#^b:N�<9iiܥym0D_j!)7TYj@ߢ 484a)Z6X#+X:%橫%)Wg.]YLI�^)d|*zjP_P$ 9b\@<@^2&^"*E[݆zo9T('T247(Q Uy%yֺWN* 4_VڂcPN-~A JTxgÚj eZDt|1Y:�r 7`MEn�C@7#Xk,':�PnW ZU"urj�`ͳ2p x8ﳉ CnܕPn "JK=AZ~tOѾaF؁jT8jt$`m4lM@}{up*7@�Jnz7aN nuO)Rh�Աr�Ly ]`i)N'}Ṗ0W%U)<vxbfP]C`�Q]*9cP)(u|*tE�*POw鹣b0Ϧ^ROy5n{!ꔊV/3kNh-qn16/t1tkUnP۵ZidL g5F=\vAhI̅52~`�Bñ +iEiG(ֹ.ʀ~_$Kr/<(7Dt7����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/somafm/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000010113�13034501105�0017406�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import lxml.etree from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from weboob.deprecated.browser import StandardBrowser from weboob.deprecated.browser.parsers.iparser import IParser __all__ = ['SomaFMModule'] class LxmlParser(IParser): def parse(self, data, encoding=None): return lxml.etree.fromstring(data.read()) class SomaFMModule(Module, CapRadio, CapCollection): NAME = 'somafm' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = u'SomaFM web radio' LICENSE = 'AGPLv3+' BROWSER = StandardBrowser QUALITIES = ['fast', 'slow', 'highest'] ALLINFO = 'http://api.somafm.com/channels.xml' def create_default_browser(self): return self.create_browser(parser=LxmlParser()) def _parse_current(self, data): current = data.split(' - ') if len(current) == 2: return current else: return ('Unknown', 'Unknown') def _fetch_radio_list(self): radios = [] document = self.browser.location(self.ALLINFO) for channel in document.iter('channel'): id=channel.get('id') radio = Radio(id) radio.title = channel.findtext('title') radio.description = channel.findtext('description') current_data = channel.findtext('lastPlaying') current = StreamInfo(0) current.what, current.who = self._parse_current(current_data) radio.current = current radio.streams = [] stream_id = 0 for subtag in channel: if subtag.tag.endswith('pls'): stream = BaseAudioStream(stream_id) bitrate=subtag.text.replace('http://somafm.com/'+id,'').replace('.pls','') if(bitrate != ''): stream.bitrate=int(bitrate) bitrate+='Kbps' else: stream.bitrate=0 bitrate=subtag.tag.replace('pls','') stream.format=subtag.get('format') stream.title = '%s/%s' % (bitrate, stream.format) stream.url = subtag.text radio.streams.append(stream) stream_id += 1 radios.append(radio) return radios def iter_radios_search(self, pattern): radios = self._fetch_radio_list() pattern = pattern.lower() for radio in radios: if pattern in radio.title.lower() or pattern in radio.description.lower(): yield radio def iter_resources(self, objs, split_path): radios = self._fetch_radio_list() if Radio in objs: self._restrict_level(split_path) for radio in radios: yield radio def get_radio(self, radio_id): radios = self._fetch_radio_list() for radio in radios: if radio_id == radio.id: return radio def fill_radio(self, radio, fields): if 'current' in fields: return self.get_radio(radio.id) return radio OBJECTS = {Radio: fill_radio} �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/somafm/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000002666�13034501105�0017116�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.radio import Radio class SomaFMTest(BackendTest): MODULE = 'somafm' def test_difm(self): ls = list(self.backend.iter_resources((Radio, ), [])) self.assertTrue(len(ls) > 0) search = list(self.backend.iter_radios_search('doom')) self.assertTrue(len(search) > 0) self.assertTrue(len(search) < len(ls)) radio = self.backend.get_radio('doomed') self.assertTrue(radio.title) self.assertTrue(radio.description) self.assertTrue(radio.current.who) self.assertTrue(radio.current.what) self.assertTrue(radio.streams[0].url) self.assertTrue(radio.streams[0].title) ��������������������������������������������������������������������������weboob-1.2/modules/spirica/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015723�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/spirica/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020035�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SpiricaModule __all__ = ['SpiricaModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/spirica/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000006167�13034501105�0017772�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from .pages import LoginPage, AccountsPage, DetailsPage, MaintenancePage class SpiricaBrowser(LoginBrowser): TIMEOUT = 60 login = URL('/securite/login.xhtml', LoginPage) accounts = URL('/sylvea/client/synthese.xhtml', AccountsPage) details = URL('/sylvea/contrat/consultationContratEpargne.xhtml', DetailsPage) maintenance = URL('/maintenance.html', MaintenancePage) def __init__(self, website, *args, **kwargs): super(SpiricaBrowser, self).__init__(*args, **kwargs) self.BASEURL = website self.cache = {} self.cache['invs'] = {} self.cache['trs'] = {} def do_login(self): self.login.go().login(self.username, self.password) if self.login.is_here(): error = self.page.get_error() raise BrowserIncorrectPassword(error) def get_subscription_list(self): return iter([]) @need_login def iter_accounts(self): if 'accs' not in self.cache.keys(): self.cache['accs'] = [a for a in self.accounts.stay_or_go().iter_accounts()] return self.cache['accs'] @need_login def iter_investment(self, account): if account.id not in self.cache['invs']: # Get form to show PRM form = self.location(account._link).page.get_investment_form() invs = [i for i in self.location(form.url, data=dict(form)).page.iter_investment()] self.cache['invs'][account.id] = invs return self.cache['invs'][account.id] @need_login def iter_history(self, account): if account.id not in self.cache['trs']: # Get form to go to History's tab form = self.location(account._link).page.get_historytab_form() # Get form to show all transactions form = self.location(form.url, data=dict(form)).page.get_historyallpages_form() if form: self.location(form.url, data=dict(form)) # Get forms to expand details of all transactions for form in self.page.get_historyexpandall_form(): # Can't async because of ReadTimeout self.location(form.url, data=dict(form)) trs = [t for t in self.page.iter_history()] self.cache['trs'][account.id] = trs return self.cache['trs'][account.id] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/spirica/module.py����������������������������������������������������������������0000664�0000000�0000000�00000003671�13034501105�0017571�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from .browser import SpiricaBrowser __all__ = ['SpiricaModule'] class SpiricaModule(Module, CapBank): NAME = 'spirica' DESCRIPTION = u'Spirica' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig( ValueBackendPassword('login', label='Identifiant', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = SpiricaBrowser def create_default_browser(self): return self.create_browser("https://www.sylvea.fr", self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_history(self, account): return self.browser.iter_history(account) def iter_investment(self, account): return self.browser.iter_investment(account) �����������������������������������������������������������������������weboob-1.2/modules/spirica/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000023555�13034501105�0017406�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage, LoggedPage from weboob.browser.elements import ItemElement, TableElement, method from weboob.browser.filters.standard import CleanText, Date, Regexp, CleanDecimal, \ TableCell, Field, Async, AsyncLoad, Eval from weboob.browser.filters.html import Attr, Link from weboob.capabilities.bank import Account, Investment, Transaction from weboob.capabilities.base import NotAvailable from weboob.exceptions import BrowserUnavailable def MyDecimal(*args, **kwargs): kwargs.update(replace_dots=True, default=NotAvailable) return CleanDecimal(*args, **kwargs) class MaintenancePage(HTMLPage): def on_load(self): raise BrowserUnavailable(CleanText().filter(self.doc.xpath('//p'))) class LoginPage(HTMLPage): def login(self, login, password): form = self.get_form('//form[@id="loginForm"]') form['loginForm:name'] = login form['loginForm:password'] = password form['loginForm:login'] = "loginForm:login" form.submit() def get_error(self): return CleanText('//li[@class="erreurBox"]')(self.doc) class AccountsPage(LoggedPage, HTMLPage): TYPES = {'Assurance Vie': Account.TYPE_LIFE_INSURANCE, 'Unknown': Account.TYPE_UNKNOWN} @method class iter_accounts(TableElement): item_xpath = '//table[@role]/tbody/tr' head_xpath = '//table[@role]/thead/tr/th' col_label = u'Produit' col_id = u'Numéro de contrat' col_balance = u'Montant (€)' class item(ItemElement): klass = Account load_details = Field('_link') & AsyncLoad obj_id = CleanText(TableCell('id'), replace=[(' ', '')]) obj_label = CleanText(TableCell('label')) obj_balance = MyDecimal(TableCell('balance')) obj_valuation_diff = Async('details') & MyDecimal('//tr[1]/td[contains(text(), \ "value du contrat")]/following-sibling::td') obj__link = Link('.//a') def obj_type(self): return self.page.TYPES[Async('details', CleanText('//td[contains(text(), \ "Option fiscale")]/following-sibling::td', default="Unknown"))(self)] class TableInvestment(TableElement): col_label = u'Support' col_vdate = u'Date de valeur' col_unitvalue = u'Valeur de part' col_quantity = u'Nombre de parts' col_portfolio_share = u'%' class ItemInvestment(ItemElement): klass = Investment obj_label = CleanText(TableCell('label')) obj_quantity = MyDecimal(TableCell('quantity', default=None)) obj_unitvalue = MyDecimal(TableCell('unitvalue', default=None)) obj_vdate = Date(CleanText(TableCell('vdate', default="")), dayfirst=True, default=NotAvailable) def obj_valuation(self): valuation = MyDecimal(TableCell('valuation', default=None))(self) h2 = CleanText('./ancestor::div[contains(@id, "Histo")][1]/preceding-sibling::h2[1]')(self) return -valuation if valuation and any(word in h2.lower() for word in self.page.DEBIT_WORDS) else valuation def obj_portfolio_share(self): ps = MyDecimal(TableCell('portfolio_share', default=None))(self) return Eval(lambda x: x / 100, ps)(self) if ps else NotAvailable class TableTransactionsInvestment(TableInvestment): item_xpath = './tbody/tr' head_xpath = './thead/tr/th' col_code = u'ISIN' col_valuation = [u'Montant brut', u'Montant net'] class item(ItemInvestment): obj_code = Regexp(CleanText(TableCell('code')), pattern='([A-Z]{2}\d{10})', default=NotAvailable) class ProfileTableInvestment(TableInvestment): # used only when portfolio is divided in multiple "profiles" head_xpath = '//thead[ends-with(@id, ":contratProfilTable_head")]/tr/th' class DetailsPage(LoggedPage, HTMLPage): DEBIT_WORDS = [u'arrêté', 'rachat', 'frais', u'désinvestir'] def get_investment_form(self): form = self.get_form('//form[contains(@id, "j_idt")]') form['ongletSituation:ongletContratTab_newTab'] = \ Link().filter(self.doc.xpath('//a[contains(text(), "Prix de revient moyen")]'))[1:] form['javax.faces.source'] = "ongletSituation:ongletContratTab" form['javax.faces.behavior.event'] = "tabChange" return form @method class iter_investment(TableInvestment): item_xpath = '//div[contains(@id,"INVESTISSEMENT")]//div[ends-with(@id, ":tableDetailSituationCompte")]//table/tbody/tr[@data-ri]' head_xpath = '//div[contains(@id,"INVESTISSEMENT")]//div[ends-with(@id, ":tableDetailSituationCompte")]//table/thead/tr/th' col_valuation = re.compile('Contre') class item(ItemInvestment): obj_code = Regexp(CleanText('.//td[contains(text(), "Isin")]'), ':[\s]+([\w]+)', default=NotAvailable) def obj_unitprice(self): return MyDecimal('//div[contains(@id, "PRIX_REVIENT")]//a[contains(text(), \ "%s")]/ancestor::tr/td[5]' % Field('label')(self))(self) def obj_diff(self): return MyDecimal('//div[contains(@id, "PRIX_REVIENT")]//a[contains(text(), \ "%s")]/ancestor::tr/td[6]' % Field('label')(self))(self) def obj_portfolio_share(self): inv_share = ItemInvestment.obj_portfolio_share(self) if self.xpath('ancestor::tbody[ends-with(@id, "contratProfilTable_data")]'): # investments are nested in profiles, row share is relative to profile share profile_table_el = self.xpath('ancestor::tr/ancestor::table[position() = 1]')[0] profile_table = ProfileTableInvestment(self.page, self, profile_table_el) share_idx = profile_table.get_colnum('portfolio_share') assert share_idx path = 'ancestor::tr/preceding-sibling::tr[@data-ri][position() = 1][1]/td[%d]' % (share_idx + 1) profile_share = MyDecimal(path)(self) assert profile_share #raise Exception('dtc') profile_share = Eval(lambda x: x / 100, profile_share)(self) return inv_share * profile_share else: return inv_share def get_historytab_form(self): form = self.get_form('//form[contains(@id, "j_idt")]') idt = Attr(None, 'name').filter(self.doc.xpath('//input[contains(@name, "j_idt") \ and contains(@name, "activeIndex")]')).rsplit('_', 1)[0] form['%s_contentLoad' % idt] = "true" form['%s_newTab' % idt] = Link().filter(self.doc.xpath('//a[contains(@href, "HISTORIQUE")]'))[1:] form['%s_activeIndex' % idt] = "1" form['javax.faces.source'] = idt form['javax.faces.behavior.event'] = "tabChange" return form def get_historyallpages_form(self): onclick = self.doc.xpath('//a[contains(text(), "Tout")]/@onclick') if onclick: idt = re.search('{[^\w]+([\w\d:]+)', onclick[0]).group(1) form = self.get_form('//form[contains(@id, "j_idt")]') form[idt] = idt return form return False def get_historyexpandall_form(self): form = self.get_form('//form[contains(@id, "j_idt")]') form['javax.faces.source'] = "ongletHistoOperations:newoperations" form['javax.faces.behavior.event'] = "rowToggle" form['ongletHistoOperations:newoperations_rowExpansion'] = "true" for data in self.doc.xpath('//tr[@data-ri]/@data-ri'): form['ongletHistoOperations:newoperations_expandedRowIndex'] = data yield form @method class iter_history(TableElement): item_xpath = '//table/tbody[@id and not(contains(@id, "j_idt"))]/tr[@data-ri]' head_xpath = '//table/thead[@id and not(contains(@id, "j_idt"))]/tr/th' col_label = u'Type' col_status = u'Etat' col_brut = [u'Montant brut', u'Brut'] col_net = [u'Montant net', u'Net'] col_date = u'Date de réception' col_vdate = u'Date de valeur' class item(ItemElement): klass = Transaction obj_label = CleanText(TableCell('label')) obj_vdate = Date(CleanText(TableCell('vdate')), dayfirst=True) obj_type = Transaction.TYPE_BANK def obj_amount(self): amount = MyDecimal(TableCell('net') if not CleanText(TableCell('brut'))(self) else TableCell('brut'))(self) return -amount if amount and any(word in Field('label')(self).lower() for word in self.page.DEBIT_WORDS) else amount def obj_date(self): return Date(CleanText(TableCell('date')), dayfirst=True, default=Field('vdate')(self))(self) def condition(self): return u"Validé" in CleanText(TableCell('status'))(self) def obj_investments(self): investments = [] for table in self.el.xpath('./following-sibling::tr[1]//span[contains(text(), "ISIN")]/ancestor::table[1]'): investments.extend(TableTransactionsInvestment(self.page, el=table)()) return investments ���������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/spirica/test.py������������������������������������������������������������������0000664�0000000�0000000�00000001606�13034501105�0017257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class SpiricaTest(BackendTest): MODULE = 'spirica' def test_spirica(self): raise NotImplementedError() ��������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/��������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016770�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/__init__.py���������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0021106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SueurDeMetalModule __all__ = ['SueurDeMetalModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/browser.py����������������������������������������������������������0000664�0000000�0000000�00000004620�13034501105�0021027�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from .pages import PageCity, PageConcert, PageCityList, PageDate, PageDates __all__ = ['SueurDeMetalBrowser'] class SueurDeMetalBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'www.sueurdemetal.com' ENCODING = 'iso-8859-15' PAGES = { '%s://%s/ville-metal-.+.htm' % (PROTOCOL, DOMAIN): PageCity, r'%s://%s/detail-concert-metal.php\?c=.+' % (PROTOCOL, DOMAIN): PageConcert, '%s://%s/recherchemulti.php' % (PROTOCOL, DOMAIN): PageCityList, '%s://%s/liste-dates-concerts.php' % (PROTOCOL, DOMAIN): PageDates, r'%s://%s/date-metal-.+.htm' % (PROTOCOL, DOMAIN): PageDate, } def get_concerts_city(self, city): self.location('%s://%s/ville-metal-%s.htm' % (self.PROTOCOL, self.DOMAIN, city)) assert self.is_on_page(PageCity) return self.page.get_concerts() def get_concerts_date(self, date_from, date_end=None): self.location('%s://%s/liste-dates-concerts.php' % (self.PROTOCOL, self.DOMAIN)) assert self.is_on_page(PageDates) for day in self.page.get_dates_filtered(date_from, date_end): self.location(day['url']) assert self.is_on_page(PageDate) for data in self.page.get_concerts(): yield data def get_concert(self, _id): self.location('%s://%s/detail-concert-metal.php?c=%s' % (self.PROTOCOL, self.DOMAIN, _id)) assert self.is_on_page(PageConcert) return self.page.get_concert() def get_cities(self): self.location('%s://%s/recherchemulti.php' % (self.PROTOCOL, self.DOMAIN)) assert self.is_on_page(PageCityList) return self.page.get_cities() ����������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/favicon.png���������������������������������������������������������0000664�0000000�0000000�00000001575�13034501105�0021133�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME ,y"���tEXtComment�Created with GIMPW��IDATxZAr0 )>g){rS<<dˀbnu%3LX1΃�7@�<uݝ&bhrOPgXO<@To.99�E%%TDude0TH}+w �סp-n[>ߧwK+[܇<wc?e;4R�*-_]EkxP0kM/ݖ7LG~竁߲go+o@F7OۥY@c>>�Z~HM.�%?#Zݞe@Py\1�9\=u:R�ݍa@v Zm ꠒ^/XBL�Y1gs4acr=]aF=fHC"==&hS9Ӯ'yh;[8JoA04{@1\U쟾9; _ae6 -)ÇǶSӪ"ǥABUʲװi_{ P͑dAP@ܮ5a_ٕF[e&ʠ<Q_: >01o9j+ }M*̽Cϓ/nPa}u- BDLO40d#<�O-]`=mqJE  S ;ү auǦ']/+cX[6����IENDB`�����������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/module.py�����������������������������������������������������������0000664�0000000�0000000�00000010123�13034501105�0020624�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.calendar import CapCalendarEvent, BaseCalendarEvent, CATEGORIES, TRANSP, STATUS import datetime from .browser import SueurDeMetalBrowser __all__ = ['SueurDeMetalModule'] class Concert(BaseCalendarEvent): @classmethod def id2url(cls, _id): return 'http://www.sueurdemetal.com/detail-concert-metal.php?c=%s' % _id class SueurDeMetalModule(Module, CapCalendarEvent): NAME = 'sueurdemetal' DESCRIPTION = u'SueurDeMetal French concerts list website' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = SueurDeMetalBrowser ASSOCIATED_CATEGORIES = [CATEGORIES.CONCERT] def __init__(self, *a, **kw): super(SueurDeMetalModule, self).__init__(*a, **kw) self.cities = {} def list_events(self, from_date, to_date=None): for d in self.browser.get_concerts_date(from_date, date_end=to_date): yield self._make_event(d) def search_events(self, query): if not self.has_matching_categories(query): return if query.city: city_id = self.find_city_id(query.city) for d in self.browser.get_concerts_city(city_id): if self._date_matches(d['date'], query): yield self._make_event(d) else: for e in self.list_events(query.start_date, query.end_date): yield e def get_event(self, _id): d = self.browser.get_concert(_id) return self._make_event(d) def _make_event(self, d): event = Concert(d['id']) event.category = CATEGORIES.CONCERT event.start_date = d['date'] event.end_date = datetime.datetime.combine(event.start_date.date(), datetime.time.max) event.summary = d['summary'] event.url = d['url'] if 'price' in d: event.price = d['price'] if d['active']: event.status = STATUS.CONFIRMED else: event.status = STATUS.CANCELLED if 'city' in d: event.city = d['city'] else: event.city = self.find_city_name(d['city_id']) event.transp = TRANSP.OPAQUE # "room, address" or "room" or "address" or "" location = ', '.join(filter(None, (d.get('room', ''), d.get('address', '')))) if location: event.location = location return event def _fetch_cities(self): if self.cities: return self.cities = self.browser.get_cities() def find_city_id(self, name): self._fetch_cities() name = name.lower() for c in self.cities: if c.lower() == name: return self.cities[c]['id'] def find_city_name(self, _id): self._fetch_cities() for c in self.cities.values(): if c['id'] == _id: return c['name'] def _date_matches(self, date, query): return ((not query.start_date or query.start_date <= date) and (not query.end_date or date <= query.end_date)) def fill_concert(self, obj, fields): if set(fields) & set(('price', 'location', 'description')): new_obj = self.get_event(obj.id) for field in fields: setattr(obj, field, getattr(new_obj, field)) return obj OBJECTS = {Concert: fill_concert} ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/pages.py������������������������������������������������������������0000664�0000000�0000000�00000013221�13034501105�0020440�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page from weboob.tools.date import parse_french_date import re from urlparse import urljoin class PageWithConcerts(Page): def extract_concert(self, concert_table): d = {} date_h3 = concert_table.iter('h3').next() d['date'] = parse_french_date(date_h3.text) cancel_h2 = next(date_h3.itersiblings('h2'), None) if cancel_h2 is not None and cancel_h2.text.startswith('ANNUL'): d['active'] = False else: d['active'] = True performers_table = concert_table.iterdescendants('table').next() d['performers'] = list(self.extract_performers(performers_table)) d['summary'] = ' + '.join(p['name'] for p in d['performers']) d['description'] = d['summary'] return d def extract_performers(self, performers_table): for performer_tr in performers_table.findall('tr'): performer_td = performer_tr.find('td') d = {'name': performer_td.find('strong').text.strip(' \t\r\n+')} # handle '+ GUESTS' rest = performer_td.tail if rest: d['genre'] = rest yield d def extract_id_from_url(self, url): return re.search(r'c=(\d+)', url).group(1) def extract_city_from_url(self, url): return re.search('metal-(.+).htm$', url).group(1) def extract_concert_link(self, concert_table, d): infos_a = concert_table.xpath('.//a[starts-with(@href, "detail-concert-metal.php")]')[0] infos_a = concert_table.xpath('.//a[starts-with(@href, "detail-concert-metal.php")]')[0] d['id'] = self.extract_id_from_url(infos_a.get('href')) d['url'] = 'http://www.sueurdemetal.com/detail-concert-metal.php?c=%s' % d['id'] class PageCity(PageWithConcerts): def get_concerts(self): for concert_table in self.document.xpath('//div[@id="centre-page"]//div/table'): yield self.extract_concert(concert_table) def extract_concert(self, concert_table): d = PageWithConcerts.extract_concert(self, concert_table) self.extract_concert_link(concert_table, d) d['city_id'] = self.extract_city_from_url(self.url) return d class PageDate(PageWithConcerts): def get_concerts(self): for concert_table in self.document.xpath('//div[@id="centre-page"]//div/table'): yield self.extract_concert(concert_table) def extract_concert(self, concert_table): d = PageWithConcerts.extract_concert(self, concert_table) self.extract_concert_link(concert_table, d) city_a = concert_table.xpath('.//a[starts-with(@href, "ville-metal-")]')[0] d['city_id'] = self.extract_city_from_url(city_a.get('href')) return d class PageConcert(PageWithConcerts): def get_concert(self): concert_table = self.document.xpath('//div[@id="centre-page"]//div/table')[0] d = self.extract_concert(concert_table) d['id'] = self.extract_id_from_url(self.url) d['url'] = self.url it = concert_table.iterdescendants('table') it.next() # ignore performers table infos_table = it.next() self.infos_table = infos_table info_trs = infos_table.findall('tr') d['room'] = (info_trs[3].findall('td')[1].text or '').strip() d['address'] = (info_trs[4].findall('td')[1].text or '').strip() price = self.parse_price(info_trs[5].findall('td')[1].text) if price is not None: # "None" is different from "0€" d['price'] = price city_a = self.document.xpath('//a[starts-with(@href, "ville-metal-")]')[0] d['city_id'] = self.extract_city_from_url(city_a.get('href')) d['city'] = city_a.text return d def parse_price(self, s): if not s: return parts = filter(None, re.split(r'[^\d.]+', s.strip())) if not parts: return return float(parts[-1]) class PageCityList(Page): def get_cities(self): cities = {} for option in self.document.xpath('//select[@name="ville"]/option'): v = option.get('value') if not v: continue d = {} d['code'], d['dept'] = re.search(r'ville-metal-(.*)-([0-9AB]+).htm$', v).groups() # french dept d['id'] = '%s-%s' % (d['code'], d['dept']) d['name'] = option.text.split('(')[0].strip() cities[d['name']] = d return cities class PageDates(Page): def get_dates(self): for a in self.document.xpath('//div[@id="dateconcerts"]//a'): d = {} d['date'] = parse_french_date(a.text.strip()) d['url'] = urljoin(self.url, a.get('href')) yield d def get_dates_filtered(self, date_from=None, date_end=None): for d in self.get_dates(): date = d['date'] if (not date_from or date_from <= date) and \ (not date_end or date <= date_end): yield d �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/sueurdemetal/test.py�������������������������������������������������������������0000664�0000000�0000000�00000003154�13034501105�0020324�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.calendar import Query, CATEGORIES from datetime import datetime, timedelta class SueurDeMetalTest(BackendTest): MODULE = 'sueurdemetal' def test_sueurdemetal_searchcity(self): q = Query() q.city = u'paris' self.assertTrue(len(list(self.backend.search_events(q))) > 0) ev = self.backend.search_events(q).next() self.assertTrue(self.backend.get_event(ev.id)) def test_sueurdemetal_datefrom(self): q = Query() later = (datetime.now() + timedelta(days=31)) q.start_date = later ev = self.backend.search_events(q).next() self.assertTrue(later.date() <= ev.start_date.date()) def test_sueurdemetal_nocategory(self): q = Query() q.categories = [CATEGORIES.CINE] self.assertTrue(len(list(self.backend.search_events(q))) == 0) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017203�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000001447�13034501105�0021322�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import SupertoinetteModule __all__ = ['SupertoinetteModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/browser.py���������������������������������������������������������0000664�0000000�0000000�00000003223�13034501105�0021240�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from .pages import RecipePage, ResultsPage __all__ = ['SupertoinetteBrowser'] class SupertoinetteBrowser(Browser): DOMAIN = 'www.supertoinette.com' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] PAGES = { 'http://www.supertoinette.com/liste-recettes/.*': ResultsPage, 'http://www.supertoinette.com/recette/[0-9]*.*': RecipePage, } def iter_recipes(self, pattern): self.location('http://www.supertoinette.com/liste-recettes/%s/' % (pattern)) assert self.is_on_page(ResultsPage) return self.page.iter_recipes() def get_recipe(self, id): try: self.location('http://www.supertoinette.com/recette/%s/' % id) except BrowserHTTPNotFound: return if self.is_on_page(RecipePage): return self.page.get_recipe(id) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/favicon.png��������������������������������������������������������0000664�0000000�0000000�00000012003�13034501105�0021332�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME g ���tEXtComment�Created with GIMPW��^IDATx[ixUչ~װs2Y@@EW "VrCkUʃWz:PSm R[#"EP @3ى/zY}a�dpn�?S,�VfgguݑﻕKbDľW[[�*%{~|ɧ=xO\'X׮]SXX8ZJ=۱cdž? 뮻8sL$ `IҰQҥK8SO=n+V%K/o$[IEJJg̝;� Hӈ}^|fĉܰa[3iҤ)L2 7jk-ZID3g/eI ,PM!`SO=o0˖-c,;Xyyy9y8y衇_N" e˖Ʊ � &L�u"rrry;ebY xkd�wtnJJ!7HBJyVG!Xku 9jԨh˱~D/A|p$:d;:DB$}#h42L6DV @ap}XS�<`t'iսwC\^>VVV[[l5kW`bۼ56ӂmVP0@H3~6o̶C,㣏>#FΙ3qƭ 8G68>8֋ ODiӦ98{=wG#w8^"}� Ҟlzv &w8ZcȐ!�PG}G�:_pjb"HGMœ2�Iot&i@'J(@ fn )�Hk;:ܫX,iixվ]m �9&hs|`hRv.*BtGnE߼A΃w֔5+^<:ۇ%ָ7Sݴ̬6| "i7E@Qɒm5k>\{//}yԢ5JFa4X-5[:x/ V~`�x6D³�ۚ s�TW^/d)f'og`o% @uz NẫGoٌ!} ̖?MW`{[j*KQR D(lxd(Dz!^ c^q7DP"=hǘX' 1 pcRpLǕcn$Um}{;b[т[+~xj(rmseq!ЊKW͘LI ؚJ]߄P5)%cR0ZsV\eR3}ƅ`nzʽ6UIX# !-+̞=�pW?4kۦ:lf6T.К^ڜP<O*njû侩,gB 4j'b8&~b\;UeRb[~ÎdVxnNfO[pSÊQTI%2f4ݙlFK-@qX?NeKXg1CYlwY"w4=ЌZcR1 NIuX'm&$M9Bk~5O Ѭ6ܐOGQ ~ط]ccTk8Uz4WqhDW̯Y@!C|*fae912 bBHFl͉mQ%Kp L7ʼb,{9`wMe^: y 8g\{IpFXm30Msn>2�4�Bp\0b6s91 R=?"˸JaYoF-!fҌkm6dQ+xnLp_ %?|}Cfz[r*$I1pt3f% e )6{0ĮqR1@`BF,jc*2Yf�OVLҀ1Gp49a Oݿ` SCg1!4i[Ah{Bba41Р � �YuwJJ^e}$�l8 yNA3MyIݹ* R½@2/:`?…DCz`>j5`p.mokī q-n �Yns M/%gP'kqcWJΆ Z{$Rq`oLVb)è, Tc�t.0 ?I  @W1z[gBc {pFy͑�^hoq<}�'YUr_u i0xlhβ I R\j<0{v@G@NqͲQIJCo@ - P sl)^8 os~" �.oo&i:,*pʘG-Ҳfe\6(FxQX֬Ӛۄ?>47@q47Cq7@5h.#�gؒCqS:dtXT^Ɵ"D*5ÒpʺW׿UҔeVIF<I?`X=�vݐ~K*\Q|KЦE!dG�)+B:a=e+5PCkz5'z9g)0C]3wbDiHyȗoo_)[LqPM{uOFm3a)3ݏGa�T~)q25pS u<D*Ͽ5B5iy۹)X^14?՚QK-&8ܠL/d6RWbq>7x_q=R $$�_ _Hĕ!ap!| ᒄq\|eG#u�: -fڔEI*U$7\^b7k`lNP4Bl<&Xh|*jՅFIsrٳenVj?&U>ʗ^t�`PE?Blyr8X N ç͕ǵFhn-4?Nh)1$oDFD Ro[.$,ґEYIC#}x񒜢*.Ne{Lk3d4{woa<g*v {a03$߅bD~ 4jQDk3 :�xouoNipLk+Z��;�w� .'n&>zGQiJli00HTʃ pJz0a]4pPT/+;]nfgj]Q-(X}3 OvB@e,vSJRyNqCGTlOg-Q1鰀.{0%tlMc9Yزֲ80vw7Uw$ 凕[*vxRӄܯoi굮c,>ekHXDZ,o<Ypb2E0"Gd72w| w�68Ϛz$幗gG7=H۪<y0Ha�8cҤKGH2eoZ14� I :;?et|ʬ|&>b<q%Ix٧WWݰU=M `G$O?" i;nڢ/4e$(WW:.$g(ȶ96 )6q\nS!F`L+57ͽ'�ߋ$H$U={@CGZq/!bD]rOs#`THFb\ +pθXdʋ4]<PSSNuLĖL�JͨL1`\4OZ1. e{!g͎OvJ2׶ O:~Ԛ_ymo> qQ%)cav(gBSRxѲ-!-ǶW'buƜ>gHUvi4GҞdX)R[Dz(mKs^]2{EpܣF^^�g1h`w*/L"v'��{S1q)i컨{2a7!Y77Kjm\ G{Lګl|L5RJӗ@;Wf,|j`*{8(ԒmC?�wޯҍiI}J{&\gU}˭fdVlS)[Bi3uJ:b. O<Y,l[ {W\x'5dw[ /,}�;Gٌ[iЊz}vc4mRr怒kvftS>34[#Gd(x#R3P[5|ж+s'dZ7ϻc2�<hi઎.Ǐ$:tY~f{4B10UPh7N0| C[ѝG#SΝ| 2 2u`^qCԻGܜb#_~^hcJ[.RjrⰔptԩ�@E<ڙͮFL7(O%eFzo|WgO|`Ɨ<|@�TUWTf۷W٨tΜ5$Cws/NX69+�?ö݊7lbomÿY]޼_Ih?/`����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/module.py����������������������������������������������������������0000664�0000000�0000000�00000003725�13034501105�0021051�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.recipe import CapRecipe, Recipe from weboob.tools.backend import Module from .browser import SupertoinetteBrowser __all__ = ['SupertoinetteModule'] class SupertoinetteModule(Module, CapRecipe): NAME = 'supertoinette' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = u'Super Toinette, la cuisine familiale French recipe website' LICENSE = 'AGPLv3+' BROWSER = SupertoinetteBrowser def get_recipe(self, id): return self.browser.get_recipe(id) def iter_recipes(self, pattern): return self.browser.iter_recipes(pattern.encode('utf-8')) def fill_recipe(self, recipe, fields): if 'nb_person' in fields or 'instructions' in fields: rec = self.get_recipe(recipe.id) recipe.picture_url = rec.picture_url recipe.instructions = rec.instructions recipe.ingredients = rec.ingredients recipe.comments = rec.comments recipe.author = rec.author recipe.nb_person = rec.nb_person recipe.cooking_time = rec.cooking_time recipe.preparation_time = rec.preparation_time return recipe OBJECTS = { Recipe: fill_recipe, } �������������������������������������������weboob-1.2/modules/supertoinette/pages.py�����������������������������������������������������������0000664�0000000�0000000�00000012157�13034501105�0020662�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.recipe import Recipe from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.deprecated.browser import Page import string class ResultsPage(Page): """ Page which contains results as a list of recipies """ def iter_recipes(self): for div in self.parser.select(self.document.getroot(), 'div.result-recipe'): thumbnail_url = NotAvailable short_description = NotAvailable imgs = self.parser.select(div, 'a.pull-image-left img') if len(imgs) > 0: url = unicode(imgs[0].attrib.get('src', '')) if url.startswith('http://'): thumbnail_url = url link = self.parser.select(div, 'div.result-text a', 1) title = unicode(link.text) id = unicode(link.attrib.get('href', '').split('/')[2]) txt = self.parser.select(div, 'div.result-text p', 1) short_description = unicode(txt.text_content()) recipe = Recipe(id, title) recipe.thumbnail_url = thumbnail_url recipe.short_description = short_description recipe.instructions = NotLoaded recipe.ingredients = NotLoaded recipe.nb_person = NotLoaded recipe.cooking_time = NotLoaded recipe.preparation_time = NotLoaded recipe.author = NotLoaded yield recipe class RecipePage(Page): """ Page which contains a recipe """ def get_recipe(self, id): title = NotAvailable preparation_time = NotAvailable cooking_time = NotAvailable author = NotAvailable nb_person = NotAvailable ingredients = NotAvailable picture_url = NotAvailable instructions = NotAvailable comments = NotAvailable title = unicode(self.parser.select(self.document.getroot(), 'h1 span[property$=name]', 1).text) main = self.parser.select(self.document.getroot(), 'div[typeof$=Recipe]', 1) imgillu = self.parser.select(main, 'div.image-with-credit img') if len(imgillu) > 0: picture_url = unicode(imgillu[0].attrib.get('src', '')) l_spanprep = self.parser.select(self.document.getroot(), 'span.preptime[property$=prepTime]') if len(l_spanprep) > 0: preparation_time = 0 prep = l_spanprep[0].attrib.get('content','') if 'H' in prep: preparation_time += 60 * (int(prep.split('PT')[-1].split('H')[0])) if 'M' in prep: preparation_time += int(prep.split('PT')[-1].split('H')[-1].split('M')[0]) l_cooktime = self.parser.select(main, 'span.cooktime[property$=cookTime]') if len(l_cooktime) > 0: cooking_time = 0 cook = l_cooktime[0].attrib.get('content','') if 'H' in cook: cooking_time += 60 * (int(cook.split('PT')[-1].split('H')[0])) if 'M' in cook: cooking_time += int(cook.split('PT')[-1].split('H')[-1].split('M')[0]) l_nbpers = self.parser.select(main, 'div.ingredients p.servings') if len(l_nbpers) > 0: rawnb = l_nbpers[0].text.strip(string.letters+' ') if '/' in rawnb: nbs = rawnb.split('/') nb_person = [int(nbs[0]), int(nbs[1])] else: nb_person = [int(rawnb)] ingredients = [] l_ing = self.parser.select(main, 'div.ingredients ul.dotlist') for ing in l_ing: sublists = self.parser.select(ing, 'li') for i in sublists: ingtxt = unicode(i.text_content().strip()) if ingtxt != '': ingredients.append(' '.join(ingtxt.split())) instructions = u'' num_inst = 1 l_divinst = self.parser.select(self.document.getroot(), 'div#recipe-steps-list p.step-details') for inst in l_divinst: instructions += '%s: %s\n' % (num_inst, inst.text_content()) num_inst += 1 recipe = Recipe(id, title) recipe.preparation_time = preparation_time recipe.cooking_time = cooking_time recipe.nb_person = nb_person recipe.ingredients = ingredients recipe.instructions = instructions recipe.picture_url = picture_url recipe.comments = comments recipe.author = author recipe.thumbnail_url = NotLoaded return recipe �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/supertoinette/test.py������������������������������������������������������������0000664�0000000�0000000�00000002165�13034501105�0020540�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class SupertoinetteTest(BackendTest): MODULE = 'supertoinette' def test_recipe(self): recipes = self.backend.iter_recipes('fondue') for recipe in recipes: full_recipe = self.backend.get_recipe(recipe.id) assert full_recipe.instructions assert full_recipe.ingredients assert full_recipe.title �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0014762�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000000071�13034501105�0017071�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import T411Module __all__ = ['T411Module'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/browser.py������������������������������������������������������������������0000664�0000000�0000000�00000004440�13034501105�0017021�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.exceptions import BrowserHTTPNotFound from weboob.browser import LoginBrowser, need_login from weboob.browser.url import URL from weboob.browser.profiles import Wget from weboob.exceptions import BrowserIncorrectPassword from .pages.index import HomePage from .pages.torrents import TorrentPage, SearchPage, DownloadPage __all__ = ['T411Browser'] class T411Browser(LoginBrowser): PROFILE = Wget() TIMEOUT = 30 BASEURL = 'https://www.t411.lI/' home = URL('$', HomePage) search = URL('torrents/search/\?search=(?P<pattern>.*)&order=seeders&type=desc', SearchPage) # Order matters here: 'torrents/[^&]*' would match '/torrents/download/\?id...' and # TorrentPage would crash on the bencode data, so DownloadPage must be listed before # TorrentPage download = URL('/torrents/download/\?id=(?P<id>.*)', DownloadPage) torrent = URL('/torrents/details/\?id=(?P<id>.*)&r=1', 'torrents/[^&]*', TorrentPage) def do_login(self): self.home.go() if not self.page.logged: self.page.login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() @need_login def iter_torrents(self, pattern): return self.search.go(pattern=pattern).iter_torrents() @need_login def get_torrent(self, fullid, torrent=None): try: self.torrent.go(id=fullid) torrent = self.page.get_torrent() return torrent except BrowserHTTPNotFound: return ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000001245�13034501105�0017117�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME  �5v>��2IDATx[;0XA!pG)bAOUVl7?*2۱MTT5Sju[^|,J "DG(A.k~s*�VHA+BJXD*v䓔� M.Ԋm<`/ iD^P�A^|{kctjZ }| zl?p"|M|q>$r;QpĀplA |`S FdCM1L۩D!YɠW"ڨ :>I,vUJi*;.LͼCȿADJI*@Nk)ustәo;S}WۚDuneh_SFc-tXO1YP,=9C9(8`q>㍒n 6DƓsn U{ӳ<(�%#tbŔ5z8a.r!R�ʀr:"0;b^g[2o Ui&@l+#hׁ0dD_N]@cOS."p'<�_|W"0k����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000004232�13034501105�0016622�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from urllib import quote_plus from weboob.capabilities.torrent import CapTorrent, Torrent from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import T411Browser __all__ = ['T411Module'] class T411Module(Module, CapTorrent): NAME = 't411' MAINTAINER = u'Julien Veyssier' EMAIL = 'eneiluj@gmx.fr' VERSION = '1.2' DESCRIPTION = 'T411 BitTorrent tracker' LICENSE = 'AGPLv3+' CONFIG = BackendConfig(Value('username', label='Username'), ValueBackendPassword('password', label='Password')) BROWSER = T411Browser def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): torrent = self.browser.get_torrent(id) if not torrent: return None resp = self.browser.open(torrent.url) return resp.content def iter_torrents(self, pattern): return self.browser.iter_torrents(quote_plus(pattern.encode('utf-8'))) def fill_torrent(self, torrent, fields): if 'description' in fields or 'files' in fields: torrent = self.browser.get_torrent(torrent.id, torrent) return torrent OBJECTS = { Torrent: fill_torrent } ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/pages/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016061�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/pages/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/pages/index.py��������������������������������������������������������������0000664�0000000�0000000�00000002112�13034501105�0017536�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage class HomePage(HTMLPage): def login(self, login, password): form = self.get_form(xpath='//form[contains(@id, "loginform")]') form['login'] = login form['password'] = password form.submit() @property def logged(self): return self.doc.xpath('//a[@class="logout"]') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/pages/torrents.py�����������������������������������������������������������0000664�0000000�0000000�00000010021�13034501105�0020305�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015-2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.tools.misc import get_bytes_size from weboob.capabilities.torrent import Torrent from weboob.capabilities.base import NotLoaded, NotAvailable from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.pages import HTMLPage, LoggedPage, RawPage from weboob.browser.filters.standard import Regexp, CleanText, Type, Format from weboob.browser.filters.html import CleanHTML class SearchPage(LoggedPage, HTMLPage): @method class iter_torrents(ListElement): item_xpath = '//table[@class="results"]/tbody/tr' class item(ItemElement): klass = Torrent obj_id = Regexp(CleanText('./td[3]/a/@href'), '/torrents/nfo/\?id=(.*)') obj_name = CleanText('./td[2]/a/@title') obj_seeders = CleanText('./td[8]') & Type(type=int) obj_leechers = CleanText('./td[9]') & Type(type=int) obj_description = NotLoaded obj_files = NotLoaded obj_filename = Format('%s.torrent', CleanText('./td[2]/a/@title')) obj_magnet = NotAvailable def obj_url(self): fullid = Regexp(CleanText('./td[3]/a/@href'), '/torrents/nfo/\?id=(.*)')(self) downurl = 'https://www.t411.ch/torrents/download/?id=%s' % fullid return downurl def obj_size(self): rawsize = CleanText('./td[6]')(self) nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper() size = get_bytes_size(nsize, usize) return size class TorrentPage(LoggedPage, HTMLPage): @method class get_torrent(ItemElement): klass = Torrent def obj_description(self): desctxt = CleanHTML('//div[has-class("description")]/article')(self) strippedlines = '\n'.join([s.strip() for s in desctxt.split('\n') if re.search(r'\[[0-9]+\]', s) is None]) description = re.sub(r'\s\s+', '\n\n', strippedlines) return description obj_name = CleanText('//div[has-class("torrentDetails")]/h2/span/text()') obj_id = CleanText('//input[@id="torrent-id"][1]/@value') def obj_url(self): fullid = CleanText('//input[@id="torrent-id"][1]/@value')(self) downurl = 'https://www.t411.ch/torrents/download/?id=%s' % fullid return downurl obj_filename = CleanText('//div[@class="accordion"]//tr[th="Torrent"]/td') def obj_size(self): rawsize = CleanText('//div[@class="accordion"]//tr[th="Taille totale"]/td')(self) nsize = float(rawsize.split()[0]) usize = rawsize.split()[-1].upper() size = get_bytes_size(nsize, usize) return size def obj_files(self): res = [] for f in Type('//div[@class="accordion"]/h3[text()="Liste des Fichiers"]\ /following-sibling::div[1]//tr', type=list)(self)[1:]: res.append(CleanText(f)(self)) return res obj_seeders = CleanText('//div[@class="details"]//td[@class="up"]') & Type(type=int) obj_leechers = CleanText('//div[@class="details"]//td[@class="down"]') & Type(type=int) obj_magnet = NotAvailable class DownloadPage(LoggedPage, RawPage): pass ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/t411/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000003311�13034501105�0016311�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.base import NotLoaded import urllib from random import choice class T411Test(BackendTest): MODULE = 't411' def test_torrent(self): torrents = list(self.backend.iter_torrents('spiderman'))[:10] for torrent in torrents: path, qs = urllib.splitquery(torrent.url) assert qs.endswith(torrent.id) if qs: assert torrent.filename assert torrent.id assert torrent.name assert torrent.description is NotLoaded full_torrent = self.backend.get_torrent(torrent.id) assert full_torrent.name assert full_torrent.url assert full_torrent.description is not NotLoaded # get the file of a random torrent # from the list (getting them all would be too long) if len(torrents): torrent = choice(torrents) self.backend.get_torrent_file(torrent.id) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tapatalk/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016072�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tapatalk/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000001437�13034501105�0020210�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TapatalkModule __all__ = ['TapatalkModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tapatalk/module.py���������������������������������������������������������������0000664�0000000�0000000�00000021710�13034501105�0017732�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Simon Lipp # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import dateutil.parser import datetime import requests import re import xmlrpclib from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.capabilities.messages import CapMessages, Thread, Message from weboob.exceptions import BrowserIncorrectPassword __all__ = ['TapatalkModule'] class TapatalkError(Exception): pass class RequestsTransport(object): def __init__(self, uri): self._uri = uri self._session = requests.Session() def request(self, host, handler, request, verbose): response = self._session.post(self._uri, data = request, headers={"Content-Type": "text/xml; charset=UTF-8"}) p, u = xmlrpclib.getparser() p.feed(response.content) p.close() response.close() return u.close() class TapatalkServerProxy(xmlrpclib.ServerProxy): def __init__(self, uri): transport = RequestsTransport(uri) xmlrpclib.ServerProxy.__init__(self, uri, transport) def __getattr__(self, name): method = xmlrpclib.ServerProxy.__getattr__(self, name) return self._wrap(method) def _wrap(self, method): def call(*args, **kwargs): res = method(*args, **kwargs) if 'result' in res and not res['result']: raise TapatalkError(str(res.get('result_text'))) return res return call class TapatalkModule(Module, CapMessages): NAME = 'tapatalk' DESCRIPTION = u'Tapatalk-compatible sites' MAINTAINER = u'Simon Lipp' EMAIL = 'laiquo@hwold.net' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), Value('url', label='Site URL', default="https://support.tapatalk.com/")) def __init__(self, *args, **kwargs): super(TapatalkModule, self).__init__(*args, **kwargs) self._xmlrpc_client = None @property def _conn(self): if self._xmlrpc_client is None: url = self.config['url'].get().rstrip('/') + "/mobiquo/mobiquo.php" username = self.config['username'].get() password = self.config['password'].get() self._xmlrpc_client = TapatalkServerProxy(url) try: self._xmlrpc_client.login(xmlrpclib.Binary(username), xmlrpclib.Binary(password)) except TapatalkError as e: raise BrowserIncorrectPassword(e.message) return self._xmlrpc_client def _get_time(self, post): if 'post_time' in post: return dateutil.parser.parse(str(post['post_time'])) else: return datetime.datetime.now() def _format_content(self, post): msg = unicode(str(post['post_content']), 'utf-8') msg = re.sub(r'\[url=(.+?)\](.*?)\[/url\]', r'<a href="\1">\2</a>', msg) msg = re.sub(r'\[quote\s?.*\](.*?)\[/quote\]', r'<blockquote><p>\1</p></blockquote>', msg) msg = re.sub(r'\[img\](.*?)\[/img\]', r'<img src="\1">', msg) if post.get('icon_url'): return u'<img style="float:right;position:relative" src="%s"> %s' % (post['icon_url'], msg) else: return msg def _process_post(self, thread, post, is_root): # Tapatalk app seems to have hardcoded this construction... I don't think we can do better :( url = u'%s/index.php?/topic/%s-%s#entry%s' % ( self.config["url"].get().rstrip('/'), thread.id, re.sub(r'[^a-zA-Z0-9-]', '', re.sub(r'\s+', '-', thread.title)), post['post_id'] ) message = Message( id = is_root and "0" or str(post['post_id']), thread = thread, sender = unicode(str(post['post_author_name']), 'utf-8'), title = is_root and thread.title or u"Re: %s"%thread.title, url = url, receivers = None, date = self._get_time(post), content = self._format_content(post),#bbcode(), signature = None, parent = thread.root or None, children = [], flags = Message.IS_HTML) if thread.root: thread.root.children.append(message) elif is_root: thread.root = message else: # First message in the thread is not the root message, # because we asked only for unread messages. Create a non-loaded root # message to allow monboob to fill correctly the References: header thread.root = Message(id="0", parent=None, children=[message], thread=thread) message.parent = thread.root return message def fill_thread(self, thread, fields, unread=False): def fill_root(thread, start, count, first_unread): while True: topic = self._conn.get_thread(thread.id, start, start+count-1, True) for i, post in enumerate(topic['posts']): message = self._process_post(thread, post, start*count+i == 0) if start+i >= first_unread: message.flags |= Message.IS_UNREAD start += count if start >= topic['total_post_num']: return thread count = 50 topic = self._conn.get_thread_by_unread(thread.id, count) if 'title' in fields: thread.title = unicode(str(topic['topic_title']), 'utf-8') if 'date' in fields: thread.date = self._get_time(topic) if 'root' in fields: # "position" starts at 1, whereas the "start" argument of get_thread starts at 0 pos = topic['position']-1 if unread: # start must be on a page boundary, or various (unpleasant) things will happen, # like get_threads returning nothing start = (pos//count)*count thread = fill_root(thread, start, count, pos) else: thread = fill_root(thread, 0, count, pos) return thread #### CapMessages ############################################## def get_thread(self, id): return self.fill_thread(Thread(id), ['title', 'root', 'date']) def iter_threads(self, unread=False): def browse_forum_mode(forum, prefix, mode): start = 0 count = 50 while True: if mode: topics = self._conn.get_topic(forum['forum_id'], start, start+count-1, mode) else: topics = self._conn.get_topic(forum['forum_id'], start, start+count-1) all_ignored = True for topic in topics['topics']: t = Thread(topic['topic_id']) t.title = unicode(str(topic['topic_title']), 'utf-8') t.date = self._get_time(topic) if not unread or topic.get('new_post'): all_ignored = False yield t start += count if start >= topics['total_topic_num'] or all_ignored: break def process_forum(forum, prefix): if (not unread or forum.get('new_post')) and not forum['sub_only']: for mode in ('TOP', 'ANN', None): for thread in browse_forum_mode(forum, prefix, mode): yield thread for child in forum.get('child', []): for thread in process_forum(child, "%s.%s" % (prefix, child['forum_name'])): yield thread for forum in self._conn.get_forum(): for thread in process_forum(forum, "%s" % forum['forum_name']): yield thread def iter_unread_messages(self): for thread in self.iter_threads(unread=True): self.fill_thread(thread, ['root'], unread=True) for message in thread.iter_all_messages(): if message.flags & Message.IS_UNREAD: yield message def set_message_read(self, message): # No-op: the underlying forum will mark topics as read as we read them pass OBJECTS = {Thread: fill_thread} ��������������������������������������������������������weboob-1.2/modules/taz/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015067�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/taz/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001501�13034501105�0017175�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"NewspaperTazModule init" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import NewspaperTazModule __all__ = ['NewspaperTazModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/taz/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000002265�13034501105�0017131�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"browser for taz website" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .pages import ArticlePage from weboob.browser.browsers import AbstractBrowser from weboob.browser.url import URL class NewspaperTazBrowser(AbstractBrowser): "NewspaperTazBrowser class" PARENT = 'genericnewspaper' BASEURL = 'http://www.taz.de' article_page = URL('/.*', ArticlePage) def __init__(self, weboob, *args, **kwargs): self.weboob = weboob super(self.__class__, self).__init__(*args, **kwargs) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/taz/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000000725�13034501105�0017226�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���% ���sRGB���� pHYs�� �� ����tIME3酧a��gIDATh 0 E5`�`2`#G{gL<_]5]G%kWV+&�U�zBQ(QL q:HL8R_+BQ8Rz{K�շd~[� C:КДQ WaXOV佮)t3Gan�(^ԩ e xF*d b̻jpT�>A lۮC),Z==vE�qs�ƨUZ:Tbn, އ z*@7it~!�$xg� V�> IȄtf OXhϬK 7$n% � �'k����IENDB`�������������������������������������������weboob-1.2/modules/taz/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000002563�13034501105�0016734�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://www.taz.de" from weboob.capabilities.messages import CapMessages from weboob.tools.backend import AbstractModule from .browser import NewspaperTazBrowser from .tools import rssid, url2id class NewspaperTazModule(AbstractModule, CapMessages): MAINTAINER = u'Florent Fourcot' EMAIL = 'weboob@flo.fourcot.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} NAME = 'taz' DESCRIPTION = u'Taz newspaper website' BROWSER = NewspaperTazBrowser RSSID = staticmethod(rssid) URL2ID = staticmethod(url2id) RSSSIZE = 30 RSS_FEED = "http://www.taz.de/!p3270;rss/" PARENT = 'genericnewspaper' ���������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/taz/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000002701�13034501105�0016540�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"ArticlePage object for Taz newspaper" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import AbstractPage from weboob.browser.filters.html import CSS, CleanHTML class ArticlePage(AbstractPage): "ArticlePage object for taz" _selector = CSS PARENT = 'genericnewspaper' PARENT_URL = 'generic_news_page' def on_loaded(self): self.main_div = self.doc.getroot() self.element_title_selector = "title" self.element_author_selector = 'a[rel="author"]>h4' def get_body(self): div = self.doc.getroot().find('.//div[@class="sectbody"]') self.try_drop_tree(div, "div.anchor") self.try_drop_tree(div, "script") self.clean_relativ_urls(div, "http://taz.de") return CleanHTML('.')(div) ���������������������������������������������������������������weboob-1.2/modules/taz/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001647�13034501105�0016430�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class TazTest(BackendTest): MODULE = 'taz' def test_new_messages(self): for message in self.backend.iter_unread_messages(): pass �����������������������������������������������������������������������������������������weboob-1.2/modules/taz/tools.py���������������������������������������������������������������������0000664�0000000�0000000�00000001674�13034501105�0016611�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"tools for taz module" # -*- coding: utf-8 -*- # Copyright(C) 2012 Florent Fourcot # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re def url2id(url): "return an id from an url" regexp = re.compile(".*/!([0-9]+)/") id = regexp.match(url).group(1) return id def rssid(entry): return url2id(entry.link) ��������������������������������������������������������������������weboob-1.2/modules/tinder/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015556�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tinder/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0017666�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TinderModule __all__ = ['TinderModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tinder/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000011523�13034501105�0017615�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.browsers import DomainBrowser, APIBrowser from weboob.browser.filters.standard import CleanText from weboob.browser.pages import HTMLPage from weboob.browser.profiles import IPhone, Android from weboob.exceptions import BrowserIncorrectPassword, ParseError from weboob.tools.json import json __all__ = ['TinderBrowser', 'FacebookBrowser'] class FacebookBrowser(DomainBrowser): BASEURL = 'https://graph.facebook.com' PROFILE = Android() CLIENT_ID = "464891386855067" access_token = None info = None def login(self, username, password): self.location('https://www.facebook.com/v2.6/dialog/oauth?redirect_uri=fb464891386855067%3A%2F%2Fauthorize%2F&display=touch&state=%7B%22challenge%22%3A%22IUUkEUqIGud332lfu%252BMJhxL4Wlc%253D%22%2C%220_auth_logger_id%22%3A%2230F06532-A1B9-4B10-BB28-B29956C71AB1%22%2C%22com.facebook.sdk_client_state%22%3Atrue%2C%223_method%22%3A%22sfvc_auth%22%7D&scope=user_birthday%2Cuser_photos%2Cuser_education_history%2Cemail%2Cuser_relationship_details%2Cuser_friends%2Cuser_work_history%2Cuser_likes&response_type=token%2Csigned_request&default_audience=friends&return_scopes=true&auth_type=rerequest&client_id=' + self.CLIENT_ID + '&ret=login&sdk=ios&logger_id=30F06532-A1B9-4B10-BB28-B29956C71AB1&ext=1470840777&hash=AeZqkIcf-NEW6vBd') page = HTMLPage(self, self.response) form = page.get_form() form['email'] = username form['pass'] = password form.submit(allow_redirects=False) if 'Location' not in self.response.headers: raise BrowserIncorrectPassword() self.location(self.response.headers['Location']) page = HTMLPage(self, self.response) if len(page.doc.xpath('//td/div[has-class("s")]')) > 0: raise BrowserIncorrectPassword(CleanText('//td/div[has-class("s")]')(page.doc)) form = page.get_form(nr=0, submit='//input[@name="__CONFIRM__"]') form.submit() m = re.search('access_token=([^&]+)&', self.response.text) if m: self.access_token = m.group(1) else: raise ParseError('Unable to find access_token') self.info = self.request('/me') def request(self, url, *args, **kwargs): url += '?access_token=' + self.access_token self.location(self.absurl(url, base=True), *args, **kwargs) return json.loads(self.response.content) class TinderBrowser(APIBrowser): BASEURL = 'https://api.gotinder.com/' PROFILE = IPhone('Tinder/3.0.2') recs = [] def __init__(self, facebook, *args, **kwargs): super(TinderBrowser, self).__init__(*args, **kwargs) self.facebook = facebook me = self.request('/auth', data={'facebook_id': facebook.info['id'], 'facebook_token': facebook.access_token}) self.session.headers['Authorization'] = 'Token token="%s"' % me['token'] self.session.headers['X-Auth-Token'] = me['token'] self.my_id = me['user']['_id'] self.my_name = me['user']['name'] def get_threads(self): resp = self.request('/updates', data={'last_activity_date': '2014-05-01T06:13:16.971Z'}) return sorted(resp['matches'], key=lambda m: m['last_activity_date'], reverse=True) def post_message(self, match_id, content): self.request('/user/matches/%s' % match_id, data={'message': content}) def update_recs(self): resp = self.request('/user/recs') try: self.recs = resp['results'] except KeyError: self.recs = [] def like_profile(self): if len(self.recs) == 0: self.update_recs() if len(self.recs) == 0: return 60 profile = self.recs.pop() if 'tinder_rate_limited' in profile['_id']: self.logger.info(profile['bio']) return 600 resp = self.request('/like/%s' % profile['_id']) if resp['match']: self.logger.error('Match with %s!' % profile['name']) else: self.logger.info('Liked %s (%r)' % (profile['name'], profile['common_likes'])) if len(self.recs) > 0: return 1 else: return 60 �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tinder/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000002230�13034501105�0017706�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs��.#��.#x?v���tIME 974���tEXtComment�Created with GIMPW���IDATxOhW?Qj`zlEhIm$[מA*cK) YbBzKT6Г4hZHj}=/tnvwvޛy6}3߼%yq6^*==$�V'~GJ0\O�Vy`,wՐUK\7P0� �:`�_x�x&[zyEj_,w�/I/W3G}:6ˣ �4WgV(}Wwߴ:B mZ{c|FPcN+5p+]�r'о:\^fK!.wPmg�Hv_SRڊU`ħ>'�۟sOqxKoվڜ(�@CƷ�Fϵjp\HCQsF$%DWR5'q,30 ^2q@Rx�ێg xЬ$ 2% pK3q@_x-.)oqyMnE*; 1uMLĴJ.@bɡ Q7l*9�l ` Ԗ�Xjӑ ]*/Qp_YҶ�<^†r�pF�4/�6�>O&bnȴ'p&)(�.9wo]<�/p<U;G@5�x0pw�)@\�x~?_Һ �grAS}I~u�b)rYصNMۀIƇ"$Fpڤe^*=|_ߛ4>tC٥r0 G& %>0|)}Ȝi#@8 li)r^ok(�dD<Og28-NCME_^/Sdch� �"]f=]#W/OƦUUUUU l Ңd����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tinder/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000014667�13034501105�0017433�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime from dateutil.parser import parse as parse_date from weboob.capabilities.messages import CapMessages, CapMessagesPost, Thread, Message from weboob.capabilities.dating import CapDating, Optimization from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.log import getLogger from .browser import TinderBrowser, FacebookBrowser __all__ = ['TinderModule'] class ProfilesWalker(Optimization): def __init__(self, sched, storage, browser): self._sched = sched self._storage = storage self._browser = browser self._logger = getLogger('walker', browser.logger) self._view_cron = None def start(self): self._view_cron = self._sched.schedule(1, self.view_profile) return True def stop(self): self._sched.cancel(self._view_cron) self._view_cron = None return True def set_config(self, params): pass def is_running(self): return self._view_cron is not None def view_profile(self): next_try = 1 try: next_try = self._browser.like_profile() finally: if self._view_cron is not None: self._view_cron = self._sched.schedule(next_try, self.view_profile) class TinderModule(Module, CapMessages, CapMessagesPost, CapDating): NAME = 'tinder' DESCRIPTION = u'Tinder dating mobile application' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('username', label='Facebook email'), ValueBackendPassword('password', label='Facebook password')) BROWSER = TinderBrowser STORAGE = {'contacts': {}, } def create_default_browser(self): facebook = self.create_browser(klass=FacebookBrowser) facebook.login(self.config['username'].get(), self.config['password'].get()) return self.create_browser(facebook) # ---- CapDating methods ----------------------- def init_optimizations(self): self.add_optimization('PROFILE_WALKER', ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)) # ---- CapMessages methods --------------------- def fill_thread(self, thread, fields): return self.get_thread(thread) def iter_threads(self): for thread in self.browser.get_threads(): if 'person' not in thread: # The account has been removed, probably because it was a # spammer. continue t = Thread(thread['_id']) t.flags = Thread.IS_DISCUSSION t.title = u'Discussion with %s' % thread['person']['name'] contact = self.storage.get('contacts', t.id, default={'lastmsg': 0}) birthday = parse_date(thread['person']['birth_date']).date() signature = u'Age: %d (%s)' % ((datetime.date.today() - birthday).days / 365.25, birthday) signature += u'\nLast ping: %s' % parse_date(thread['person']['ping_time']).strftime('%Y-%m-%d %H:%M:%S') signature += u'\nPhotos:\n\t%s' % '\n\t'.join([photo['url'] for photo in thread['person']['photos']]) signature += u'\n\n%s' % thread['person']['bio'] t.root = Message(thread=t, id=1, title=t.title, sender=unicode(thread['person']['name']), receivers=[self.browser.my_name], date=parse_date(thread['created_date']), content=u'Match!', children=[], signature=signature, flags=Message.IS_UNREAD if int(contact['lastmsg']) < 1 else 0) parent = t.root for msg in thread['messages']: flags = 0 if int(contact['lastmsg']) < msg['timestamp']: flags = Message.IS_UNREAD msg = Message(thread=t, id=msg['timestamp'], title=t.title, sender=unicode(self.browser.my_name if msg['from'] == self.browser.my_id else thread['person']['name']), receivers=[unicode(self.browser.my_name if msg['to'] == self.browser.my_id else thread['person']['name'])], date=parse_date(msg['sent_date']), content=unicode(msg['message']), children=[], parent=parent, signature=signature if msg['to'] == self.browser.my_id else u'', flags=flags) parent.children.append(msg) parent = msg yield t def get_thread(self, _id): for t in self.iter_threads(): if t.id == _id: return t def iter_unread_messages(self): for thread in self.iter_threads(): for message in thread.iter_all_messages(): if message.flags & message.IS_UNREAD: yield message def set_message_read(self, message): contact = self.storage.get('contacts', message.thread.id, default={'lastmsg': 0}) if int(contact['lastmsg']) < int(message.id): contact['lastmsg'] = int(message.id) self.storage.set('contacts', message.thread.id, contact) self.storage.save() # ---- CapMessagesPost methods --------------------- def post_message(self, message): self.browser.post_message(message.thread.id, message.content) OBJECTS = {Thread: fill_thread, } �������������������������������������������������������������������������weboob-1.2/modules/tinder/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000001613�13034501105�0017110�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class TinderTest(BackendTest): MODULE = 'tinder' def test_tinder(self): self.backend.browser.like_profile() ���������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016160�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/__init__.py�������������������������������������������������������������0000664�0000000�0000000�00000000101�13034501105�0020261�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import TorrentzModule __all__ = ['TorrentzModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/browser.py��������������������������������������������������������������0000664�0000000�0000000�00000001514�13034501105�0020216�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- from weboob.browser import PagesBrowser, URL from .pages.index import IndexPage from .pages.torrents import TorrentsPage, TorrentPage __all__ = ['TorrentzBrowser'] class TorrentzBrowser(PagesBrowser): BASEURL = 'https://torrentz2.eu/' index_page = URL('/$', IndexPage) torrents_page = URL('/search\?f=(?P<query>.+)', TorrentsPage) torrent_page = URL('/(?P<hash>[0-9a-f]+)', TorrentPage) def home(self): return self.index_page.go() def iter_torrents(self, pattern): self.torrents_page.go(query=pattern) return self.page.iter_torrents() def get_torrent(self, id): self.torrent_page.go(hash=id) return self.page.get_torrent() def get_torrent_file(self, id): self.torrent_page.go(hash=id) return self.page.get_torrent_file() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/favicon.png�������������������������������������������������������������0000664�0000000�0000000�00000001246�13034501105�0020316�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD�3�f�K[��� pHYs�� �� ����tIME/_8���tEXtComment�Created with GIMPW��IDATx;H[Q5>PQ!֡M⫋lK4>B%FC hWIKK7Kk\` !EE^&(Blʹo͏s*M>5 WL^ �@�� �@�3V],"O{B\A)ټ.Z\ �6䢲1{i�s p 7C͊+\v sB|m~E2Ȝ�}nتʅ~p"�}h~lzS QFv'Boe}_nۮBUqF%S:T �E/z.!l+#fb&&$t(x?[1z3G5FZnB/4Ď> );F}ƾ-p6؄l7~pCou;/WWbL%CR 3t<Oq!L|A#o^ۥ syn� �@�� �VW݊.����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/module.py���������������������������������������������������������������0000664�0000000�0000000�00000001420�13034501105�0020014�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- from weboob.capabilities.torrent import CapTorrent from weboob.tools.backend import Module from .browser import TorrentzBrowser __all__ = ['TorrentzModule'] class TorrentzModule(Module, CapTorrent): NAME = 'torrentz' MAINTAINER = u'Matthieu Weber' EMAIL = 'weboob@weber.fi.eu.org' VERSION = '1.2' DESCRIPTION = 'Torrentz Search Engine.' LICENSE = 'AGPL' BROWSER = TorrentzBrowser def create_default_browser(self): return self.create_browser() def get_torrent(self, id): return self.browser.get_torrent(id) def get_torrent_file(self, id): return self.browser.get_torrent_file(id) def iter_torrents(self, pattern): return self.browser.iter_torrents(pattern.replace(' ', '+')) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/pages/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017257�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/pages/__init__.py�������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021356�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/pages/index.py����������������������������������������������������������0000664�0000000�0000000�00000000151�13034501105�0020735�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- from weboob.browser.pages import HTMLPage class IndexPage(HTMLPage): pass �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/pages/torrents.py�������������������������������������������������������0000664�0000000�0000000�00000010105�13034501105�0021506�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- from datetime import datetime from urllib import quote_plus from weboob.tools.misc import get_bytes_size from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.capabilities.base import NotAvailable from weboob.capabilities.torrent import Torrent, MagnetOnly from weboob.browser.filters.standard import CleanText, Regexp, Date, Type def parse_timestamp(txt, **kwargs): try: ts = int(txt) return datetime.fromtimestamp(ts) except: return None class TorrentsPage(HTMLPage): @method class iter_torrents(ListElement): item_xpath = '//div[@class="results"]/dl[count(dt/a) > 0]' class item(ItemElement): klass = Torrent obj_id = Regexp(CleanText('./dt/a/@href'), r'/([0-9a-f]+)', '\\1') obj_name = CleanText('./dt/a') obj_date = CleanText('./dd/span[2]/@title') & Date(default=None, parse_func=parse_timestamp) obj_seeders = CleanText('./dd/span[4]', replace=[(',', '')]) & Type(type=int) obj_leechers = CleanText('./dd/span[5]', replace=[(',', '')]) & Type(type=int) def obj_size(self): data = CleanText('./dd/span[3]')(self) if data: value, unit = data.split() return get_bytes_size(float(value), unit) else: return float("NaN") class TorrentPage(HTMLPage): @method class get_torrent(ItemElement): klass = Torrent obj_id = Regexp(CleanText('//div[@class="trackers"]/h2'), r'hash ([0-9a-f]+)', '\\1') obj_name = CleanText('//div[@class="download"]/h2/span') obj_date = CleanText('//div[@class="download"]/div/span/@title') & Date(default=None) obj_size = CleanText('//div[@class="files"]/div/@title', replace=[(',', ''), ('b', '')]) & \ Type(type=float) def obj_seeders(self): try: return max([int(_.text.replace(',', '')) for _ in self.xpath('//div[@class="trackers"]/dl/dd/span[@class="u"]')]) except ValueError: return NotAvailable def obj_leechers(self): try: return max([int(_.text.replace(',', '')) for _ in self.xpath('//div[@class="trackers"]/dl/dd/span[@class="d"]')]) except ValueError: return NotAvailable def obj_url(self): return self.page.browser.BASEURL + \ Regexp(CleanText('//div[@class="trackers"]/h2'), r'hash ([0-9a-f]+)', '\\1')(self) def obj_files(self): def traverse_nested_lists(ul, result, depth=0): for li in ul.xpath('./li'): sub_uls = li.xpath('./ul') if sub_uls: result.append(("| " * depth) + ("%s" % li.text)) for sub_ul in sub_uls: traverse_nested_lists(sub_ul, result, depth+1) else: try: size = li.xpath('span')[0].text except: size = "" result.append(("| " * depth) + ("%s [%s]" % (li.text, size))) result = [] traverse_nested_lists(self.xpath('//div[@class="files"]/ul')[0], result) return result def obj_magnet(self): hsh = Regexp(CleanText('//div[@class="trackers"]/h2'), r'hash ([0-9a-f]+)', '\\1')(self) name = "dn=%s" % quote_plus(CleanText('//div[@class="download"]/h2/span')(self)) trackers = ["tr=%s" % _.text for _ in self.xpath('//div[@class="trackers"]/dl/dt')] return "&".join(["magnet:?xt=urn:btih:%s" % hsh, name] + trackers) def obj_description(self): return u"Torrent files available at:\n" + \ u"\n\n".join(self.xpath('//div[@class="download"]/dl/dt/a/@href')) def get_torrent_file(self): raise MagnetOnly(self.get_torrent().magnet) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/torrentz/test.py�����������������������������������������������������������������0000664�0000000�0000000�00000003102�13034501105�0017505�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- from weboob.tools.test import BackendTest from weboob.capabilities.torrent import MagnetOnly from weboob.tools.date import date from random import choice class TorrentzTest(BackendTest): MODULE = 'torrentz' def test_iter_torrents(self): # try something popular so we sometimes get a magnet-only torrent l = list(self.backend.iter_torrents('ubuntu linux')) self.assertEquals(50, len(l)) for torrent in l: assert torrent.id assert torrent.name assert not(torrent.size == torrent.size) or torrent.size >= 0 assert (torrent.date is None or type(torrent.date) is date) assert torrent.seeders >= 0 assert torrent.leechers >= 0 self.assertEquals(40, len(torrent.id)) def test_get_random_torrentfile(self): torrent = choice(list(self.backend.iter_torrents('ubuntu linux'))) full_torrent = self.backend.get_torrent(torrent.id) try: self.backend.get_torrent_file(torrent.id) except MagnetOnly as e: assert e.magnet.startswith("magnet:") assert e.magnet == full_torrent.magnet def test_get_special_torrent(self): torrent = self.backend.get_torrent("abd1d2648c97958789d62f6a6a1f5d33f4eff5be") assert torrent.name == u'Ubuntu Linux Toolbox - 1000+ Commands for Ubuntu and Debian Power Users' assert len(torrent.files) == 4 assert torrent.size == float(7010361.0) dt = torrent.date assert dt.year == 2013 assert dt.month == 12 ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trainline/�����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016256�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trainline/__init__.py������������������������������������������������������������0000664�0000000�0000000�00000001446�13034501105�0020374�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TrainlineModule __all__ = ['TrainlineModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trainline/browser.py�������������������������������������������������������������0000664�0000000�0000000�00000007256�13034501105�0020325�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime from dateutil.relativedelta import relativedelta from weboob.browser.browsers import APIBrowser from weboob.exceptions import BrowserIncorrectPassword from weboob.browser.filters.standard import CleanDecimal, Date from weboob.browser.exceptions import ClientError from weboob.capabilities.bill import Bill, Subscription class TrainlineBrowser(APIBrowser): BASEURL = 'https://www.trainline.fr/api/v5/' def __init__(self, email, password, *args, **kwargs): super(TrainlineBrowser, self).__init__(*args, **kwargs) self.session.headers['X-Requested-With'] = 'XMLHttpRequest' try: me = self.request('account/signin', data={'email': email, 'password': password}) except ClientError: raise BrowserIncorrectPassword self.session.headers['Authorization'] = 'Token token="%s"' % me['meta']['token'] def get_subscription_list(self): me = self.request('user')['user'] sub = Subscription() sub.subscriber = '%s %s' % (me['first_name'], me['last_name']) sub.id = me['id'] sub.label = me['email'] yield sub def iter_documents(self, subscription): docs, docs_len, check, month_back, date = list(), -1, 0, 6, None # First request is known bills = self.request('pnrs') while check < month_back: # If not first if docs_len > -1: if check > 0: # If nothing, we try 4 weeks back date = (datetime.strptime(date, '%Y-%m-%d') - relativedelta(weeks=4)).strftime('%Y-%m-%d') else: # Add 8 weeks to last date to be sure to get all date = (datetime.combine(date, datetime.min.time()) + relativedelta(weeks=8)).strftime('%Y-%m-%d') bills = self.request('pnrs?date=%s' % date) docs_len = len(docs) for proof, pnr, trip in zip(bills['proofs'], bills['pnrs'], bills['trips']): # Check if not already in docs list for doc in docs: if vars(doc)['id'].split('_', 1)[1] == pnr['id']: break else: b = Bill() b.id = '%s_%s' % (subscription.id, pnr['id']) b._url = proof['url'] b.date = Date().filter(proof['created_at']) b.format = u"pdf" b.label = u'Trajet du %s' % Date().filter(trip['departure_date']) b.type = u"bill" b.vat = CleanDecimal().filter('0') b.price = CleanDecimal().filter(format(pnr['cents']/float(100), '.2f')) b.currency = pnr['currency'] docs.append(b) check += 1 # If a new bill is found, we reset check if docs_len < len(docs): date = b.date check = 0 return iter(docs) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trainline/module.py��������������������������������������������������������������0000664�0000000�0000000�00000004766�13034501105�0020132�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bill import CapDocument, Subscription, Document, SubscriptionNotFound, DocumentNotFound from weboob.capabilities.base import find_object, NotAvailable from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword, Value from .browser import TrainlineBrowser __all__ = ['TrainlineModule'] class TrainlineModule(Module, CapDocument): NAME = 'trainline' DESCRIPTION = u'trainline website' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('login', label='Adresse email'), ValueBackendPassword('password', label='Mot de passe')) BROWSER = TrainlineBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def iter_subscription(self): return self.browser.get_subscription_list() def get_subscription(self, _id): return find_object(self.iter_subscription(), id=_id, error=SubscriptionNotFound) def get_document(self, _id): subid = _id.rsplit('_', 1)[0] subscription = self.get_subscription(subid) return find_object(self.iter_documents(subscription), id=_id, error=DocumentNotFound) def iter_documents(self, subscription): if not isinstance(subscription, Subscription): subscription = self.get_subscription(subscription) return self.browser.iter_documents(subscription) def download_document(self, document): if not isinstance(document, Document): document = self.get_document(document) if document._url is NotAvailable: return return self.browser.open(document._url).content ����������weboob-1.2/modules/trainline/test.py����������������������������������������������������������������0000664�0000000�0000000�00000001614�13034501105�0017611�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class TrainlineTest(BackendTest): MODULE = 'trainline' def test_trainline(self): raise NotImplementedError() ��������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/transilien/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016441�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/transilien/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001465�13034501105�0020560�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Hébert, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TransilienModule __all__ = ['TransilienModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/transilien/browser.py������������������������������������������������������������0000664�0000000�0000000�00000006037�13034501105�0020504�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Hébert, Romain Bignon # Copyright(C) 2014 Benjamin Carton # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime from weboob.browser import PagesBrowser, URL from .pages import StationsPage, DeparturesPage, DeparturesPage2, HorairesPage, RoadMapPage class Transilien(PagesBrowser): BASEURL = 'http://www.transilien.com' TIMEOUT = 20 stations_page = URL('aidesaisie/autocompletion\?saisie=(?P<pattern>.*)', StationsPage) departures_page = URL('gare/pagegare/chargerGare\?nomGare=(?P<station>.*)', 'gare/.*', DeparturesPage) departures_page2 = URL('fichehoraire/fichehoraire/(?P<url>.*)', 'fichehoraire/fichehoraire/.*', DeparturesPage2) horaires_page = URL('fiche-horaire/(?P<station>.*)--(?P<arrival>.*)-(?P<station2>.*)-(?P<arrival2>)-(?P<date>)', 'fiche-horaire/.*', HorairesPage) roadmap_page = URL('itineraire/trajet', RoadMapPage) def get_roadmap(self, departure, arrival, filters): dep = self.get_stations(departure, False).next() arr = self.get_stations(arrival, False).next() self.roadmap_page.go().request_roadmap(dep, arr, filters.departure_time, filters.arrival_time) if self.page.is_ambiguous(): self.page.fix_ambiguity() return self.page.get_roadmap() def get_stations(self, pattern, only_station=True): return self.stations_page.go(pattern=pattern).get_stations(only_station=only_station) def get_station_departues(self, station, arrival_id, date): if arrival_id is not None: arrival_name = arrival_id.replace('-', ' ') self.departures_page2.go(url='init').init_departure(station) arrival = self.page.get_potential_arrivals().get(arrival_name) if arrival: station_id = self.page.get_station_id() if date is None: date = datetime.now() _date = datetime.strftime(date, "%d/%m/%Y-%H:%M") self.horaires_page.go(station=station.replace(' ', '-'), arrival=arrival_id, station2=station_id, arrival2=arrival, date=_date) return self.page.get_departures(station, arrival_name, date) return [] else: return self.departures_page.go(station=station).get_departures(station=station) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/transilien/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000002721�13034501105�0020576�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME#_0��cIDATx_h[U?7MMG\D1pn3PPE18&T}ȋ: ({_3Y>8ahjmkCmiܤM89ɽs;{~^"X oTvGzɬE{hWqh�X0MB>h"�Oeas-K@29Odp0 8$ +6MUP%+Ev(`S2e6�&qh_{'n �Ο4+/uo74�8 zbxdCLf*󓾠w.'j4~AA<f-.�% ?U+V}Ao/%�e-WR/>Y(S@Go=C²c)> :ڨ#.I0̝/�� z'IΟURVi]Yu6Pks03?MRK i%  ׿ɜN�FdyvxG,顖yvls;5߿^G2P'X+N`@6ٚG%nm&O Jv.�&I,&ֵll(XWэu<a=ip7"I"Uu/$\b4>—W>[`f~,O oқF0rge/r#G ]ηRiOL[7qv cf.6_gOʩ$% _zwZxZ=m{Hʲe&QT�<z^.O@�!?lɰQޖkUڴײ`^�,ͱF1,�t#JRDr @W]㡺f*J/Wĺg=oKG,=o� [b-�q$R^ <LR!2)<Bq Wa"K/]U/ҧ4�}P&�dyQ^e.#RbZCG5?,.5lRFޱ[}�Dzg8vi~v~Fܓʈp.xX`,,-V+աRb1$ JOQy[% GD'͖&R4� He Bfc@^#u"TH8�2!3[T2j]Eu2-C=*[I=} ܣ|BzA�Xc< <O꽀y6p7 0A`њy0pg|ep]7C kM%bfpI(*G�v r (�ц�r@T_#�j"omٖmYVf񺫶'����IENDB`�����������������������������������������������weboob-1.2/modules/transilien/module.py�������������������������������������������������������������0000664�0000000�0000000�00000003042�13034501105�0020277�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Hébert, Romain Bignon # Copyright(C) 2014 Benjamin Carton # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.travel import CapTravel from weboob.tools.backend import Module from .browser import Transilien class TransilienModule(Module, CapTravel): NAME = 'transilien' MAINTAINER = u'Julien Hébert' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u"Public transportation in the Paris area" BROWSER = Transilien def iter_station_search(self, pattern): return self.browser.get_stations(pattern) def iter_station_departures(self, station_id, arrival_id=None, date=None): return self.browser.get_station_departues(station_id.replace('-', ' '), arrival_id, date) def iter_roadmap(self, departure, arrival, filters): return self.browser.get_roadmap(departure, arrival, filters) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/transilien/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000020702�13034501105�0020113�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Hébert, Romain Bignon # Copyright(C) 2014 Benjamin Carton # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import JsonPage, HTMLPage from weboob.browser.elements import TableElement, ItemElement, DictElement, method from weboob.capabilities.travel import Station, Departure, RoadStep from weboob.capabilities import NotAvailable from weboob.browser.filters.standard import CleanText, TableCell, Filter, Time, Env, Regexp, Duration,\ Format, Join, DateTime from weboob.browser.filters.json import Dict from weboob.browser.filters.html import Link class RoadMapDuration(Duration): _regexp = re.compile(r'(?P<mn>\d+)') kwargs = {'minutes': 'mn'} class DepartureTypeFilter(Filter): def filter(self, el): result = [] for img in el[0].getiterator(tag='img'): result.append(img.attrib['alt']) return u' '.join(result) class Child(Filter): def filter(self, el): return list(el[0].iterchildren()) class RoadMapPage(HTMLPage): def request_roadmap(self, station, arrival, departure_date, arrival_date): form = self.get_form('//form[@id="form_rechercheitineraire"]') form['depart'] = '%s' % station.name.replace(' ', '+') form['coordDepart'] = station._coord form['typeDepart'] = station._type_point form['arrivee'] = '%s' % arrival.name.replace(' ', '+') form['coordArrivee'] = arrival._coord form['typeArrivee'] = arrival._type_point if departure_date: form['jour'] = departure_date.strftime('%d/%m/%Y') form['horaire'] = departure_date.strftime('%H:%M') form['sens'] = 1 elif arrival_date: form['jour'] = arrival_date.strftime('%d/%m/%Y') form['horaire'] = arrival_date.strftime('%H:%M') form['sens'] = -1 form.submit() def is_ambiguous(self): return self.doc.xpath('//span[has-class("errormsg")]') def fix_ambiguity(self): form = self.get_form('//form[@id="cRechercheItineraire"]') if self.doc.xpath('//select[@id="gare_arrivee_ambigu"]'): form['coordArrivee'] = self.doc.xpath('//select[@id="gare_arrivee_ambigu"]/option[@cat="STOP_AREA"]/@value')[0] if self.doc.xpath('//select[@id="gare_depart_ambigu"]'): form['coordDepart'] = self.doc.xpath('//select[@id="gare_depart_ambigu"]/option[@cat="STOP_AREA"]/@value')[0] form.submit() def get_roadmap(self): roadstep = None for step in self.doc.xpath('(//ol[@class="trajet_feuilleDeRoute transport"])[1]/li'): if step.attrib and 'class' in step.attrib and step.attrib['class'] == 'odd': if roadstep: roadstep.end_time = Time(CleanText('./div/div[has-class("temps")]'))(step) roadstep.arrival = CleanText('./div/div/div/div[@class="step_infos clearfix"]', default=None)(step) yield roadstep roadstep = RoadStep() roadstep.start_time = Time(CleanText('./div/div[has-class("temps")]'))(step) roadstep.departure = CleanText('./div/div/div/div[@class="step_infos clearfix"]', default=None)(step) if not step.attrib: roadstep.line = CleanText('./div/div/div/div/div/div[@class="transport"]', default=None)(step) or\ CleanText('./div/div/div/div[@class="step_infos clearfix"]', default=None)(step) or\ Join('\n', './div/div/div/div/div/ul/li/text()')(step) roadstep.duration = RoadMapDuration(CleanText('./div/div[has-class("temps")]'))(step) del roadstep class HorairesPage(HTMLPage): def get_departures(self, station, arrival, date): for table in self.doc.xpath('//table[@class="trajet_horaires trajet_etapes"]'): lignes = table.xpath('./tr[@class="ligne"]/th') arrives = table.xpath('./tr[@class="arrivee"]/td') departs = table.xpath('./tr[@class="depart"]/td') items = zip(lignes, arrives, departs) for item in items: departure = Departure() departure.id = Regexp(Link('./div/a'), '.*?vehicleJourneyExternalCode=(.*?)&.*?')(item[1]) departure.departure_station = station departure.arrival_station = arrival hour, minute = CleanText('./div/a')(item[1]).split('h') departure.time = date.replace(hour=int(hour), minute=int(minute)) hour, minute = CleanText('./div/a')(item[2]).split('h') departure.arrival_time = date.replace(hour=int(hour), minute=int(minute)) departure.information = CleanText('.')(item[0]) departure.type = DepartureTypeFilter(item)(self) yield departure class StationsPage(JsonPage): @method class get_stations(DictElement): item_xpath = 'gares' class item(ItemElement): klass = Station MapCategorieToTypePoint = {"StopArea": "STOP_AREA", "City": "CITY", "Site": "SITE_SEUL", "Address": "ADRESSE"} def condition(self): if self.env['only_station']: return Dict('entryPointType')(self.el) == 'StopArea' and Dict('reseau')(self.el)[0] return True obj_name = CleanText(Dict('gare')) obj_id = CleanText(Dict('gare'), replace=[(' ', '-')]) obj__coord = Format('%s_%s', Dict('coordLambertX'), Dict('coordLambertY')) def obj__type_point(self): key = Dict('entryPointType', default=None)(self) if key: return self.MapCategorieToTypePoint[key] class DeparturesPage2(HTMLPage): def get_potential_arrivals(self): arrivals = {} for el in self.doc.xpath('//select[@id="gare_arrive_ambigu"]/option'): arrivals[el.text] = el.attrib['value'] return arrivals def get_station_id(self): form = self.get_form('//form[@id="cfichehoraire"]') return form['departExternalCode'] def init_departure(self, station): form = self.get_form('//form[@id="cfichehoraire"]') form['depart'] = station form.submit() def get_departures(self, arrival, date): form = self.get_form('//form[@id="cfichehoraire"]') form['arrive'] = arrival if date: form['jourHoraire'] = date.day form['moiHoraire'] = '%s|%s' % (date.month, date.year) form['heureHoraire'] = date.hour form['minuteHoraire'] = date.minute self.logger.debug(form) form.submit() class DeparturesPage(HTMLPage): @method class get_departures(TableElement): head_xpath = u'//table[@class="etat_trafic"][1]/thead/tr/th[@scope="col"]/text()' item_xpath = u'//table[@class="etat_trafic"]/tr' col_type = u'Ligne' col_info = u'Nom du train' col_time = u'Heure de départ' col_arrival = u'Destination' col_plateform = u'Voie/quai' col_id = u'Gares desservies' class item(ItemElement): klass = Departure def condition(self): return len(self.el.xpath('./td')) >= 6 obj_time = TableCell('time') & CleanText & DateTime | NotAvailable obj_type = DepartureTypeFilter(TableCell('type')) obj_departure_station = CleanText(Env('station')) obj_arrival_station = CleanText(TableCell('arrival')) obj_information = TableCell('time') & CleanText & Regexp(pattern='([^\d:]+)') | u'' obj_plateform = CleanText(TableCell('plateform')) obj_id = Regexp(Link(Child(TableCell('id'))), '.*?numeroTrain=(.*?)&.*?') ��������������������������������������������������������������weboob-1.2/modules/transilien/test.py���������������������������������������������������������������0000664�0000000�0000000�00000003511�13034501105�0017772�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime from weboob.capabilities.travel import RoadmapFilters from weboob.tools.test import BackendTest class TransilienTest(BackendTest): MODULE = 'transilien' def test_stations(self): stations = list(self.backend.iter_station_search('aul')) self.assertTrue(len(stations) > 0) def test_departures(self): stations = list(self.backend.iter_station_search('paris')) self.assertTrue(len(stations) > 0) list(self.backend.iter_station_departures(stations[0].id)) def test_roadmap(self): filters = RoadmapFilters() roadmap = list(self.backend.iter_roadmap('aul', u'aub', filters)) self.assertTrue(len(roadmap) > 0) filters.arrival_time = datetime.datetime.now() + datetime.timedelta(days=1) roadmap = list(self.backend.iter_roadmap('aul', u'bag', filters)) self.assertTrue(len(roadmap) > 0) filters.departure_time = datetime.datetime.now() + datetime.timedelta(days=1) roadmap = list(self.backend.iter_roadmap('gare du nord', u'stade de boulogne', filters)) self.assertTrue(len(roadmap) > 0) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016456�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000000105�13034501105�0020563�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import TricTracTVModule __all__ = ['TricTracTVModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/browser.py������������������������������������������������������������0000664�0000000�0000000�00000005011�13034501105�0020510�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier, Benjamin Drieu # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.deprecated.browser import Browser from weboob.deprecated.browser.decorators import id2url from .pages import IndexPage, VideoPage from .video import TricTracTVVideo __all__ = ['TricTracTVBrowser'] class TricTracTVBrowser(Browser): DOMAIN = 'trictrac.tv' ENCODING = 'ISO-8859-1' PAGES = {r'http://[w\.]*trictrac.tv/': IndexPage, r'http://[w\.]*trictrac.tv/home/listing.php.*': IndexPage, r'http://[w\.]*trictrac.tv/video-(.+)': VideoPage, } @id2url(TricTracTVVideo.id2url) def get_video(self, url, video=None): self.location(url) assert self.is_on_page(VideoPage) _id = self.page.get_id() if video is None: video = TricTracTVVideo(_id) infourl = self.page.get_info_url() if infourl is not None: self.parse_info(self.openurl(infourl).read(), video) return video def home(self): self.location(self.buildurl('http://www.trictrac.tv/home/listing.php', mot='%')) def search_videos(self, pattern): if not pattern: self.home() else: self.location(self.buildurl('http://www.trictrac.tv/home/listing.php', mot=pattern.encode('utf-8'))) assert self.is_on_page(IndexPage) return self.page.iter_videos() def parse_info(self, data, video): m = re.match ( '.*fichier=(.*?)&', data ) video.url = unicode ( r'http://src.povcon.net/videos/%s' % m.group ( 1 ) ) video.description = self.page.get_descriptif() video.duration = self.page.get_duration() video.title = self.page.get_title() video.date = self.page.get_date() video.rating = self.page.get_rating() video.rating_max = 5 return video �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000002345�13034501105�0020615�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME6 m��rIDATx]h[e[XIgKq!(J;pH wcÉ)BBgވPqJ^42DR#ùycUiM]ؓt!'<y?F;3/W uW k ZugFB6p H8퀆*jh LQ9" *JˬFgv3/;Fae ro/(WXhQRZTGL&C@@8`H.H/@8Ldp8}9@ߨ|}Er0SL�ۋʪDUV/ߩ7 ͔>IM&#NOxZ6v;IU3'b\LU_VBnc5ŋB! = `&xwMcCSO-3Bn3`lqi~Ɨ4Rq7s@fj,Rn7C" ;s(4Rۭ3glR3iCdr#ƭ64[RE13^a$Fc\6SzH!cˮ ΜK%/[ؔ$ĄS!n",͒c9`PU3'$fJ6E֞kWS{kW$Ƙv* ~ɞ>eMӚkdE&J[sCe'd*S^W84.~fw=}?4i�,--oZDž7>řK/浌@=V�.~fK^O?��2>@/W^}~M4S`bvxp?'zynlcu "|,s;B�}[80޽<g봺̔-(Ymzrb;N>=NúoWh,k_#w4,8'/8tӇ,kr^Pr8LH.Prh&́q)fc!m{4S+a>kiZ!n4 r%v ȓ@g[yca_eg%8`gjD5�l?B cU…2[ZiVwB����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/module.py�������������������������������������������������������������0000664�0000000�0000000�00000003751�13034501105�0020323�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # Copyright(C) 2012 Benjamin Drieu # # This file is *not yet* part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo from weboob.tools.backend import Module from .browser import TricTracTVBrowser from .video import TricTracTVVideo __all__ = ['TricTracTVModule'] class TricTracTVModule(Module, CapVideo): NAME = 'trictractv' MAINTAINER = u'Benjamin Drieu' EMAIL = 'benjamin@drieu.org' VERSION = '1.2' DESCRIPTION = u'TricTrac.tv video website' LICENSE = 'AGPLv3+' BROWSER = TricTracTVBrowser def get_video(self, _id): with self.browser: return self.browser.get_video(_id) def search_videos(self, pattern=None, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): with self.browser: return self.browser.search_videos(pattern) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields with self.browser: video = self.browser.get_video(TricTracTVVideo.id2url(video.id), video) if 'thumbnail' in fields and video.thumbnail: with self.browser: video.thumbnail.data = self.browser.readurl(video.thumbnail.url) return video OBJECTS = {TricTracTVVideo: fill_video} �����������������������weboob-1.2/modules/trictractv/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000011042�13034501105�0020125�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import datetime import re from weboob.capabilities.image import Thumbnail from weboob.deprecated.browser import Page, BrokenPageError from .video import TricTracTVVideo class IndexPage(Page): def iter_videos(self): for div in self.parser.select(self.document.getroot(), 'li#contentsearch'): title = self.parser.select(div, '#titlesearch span', 1) a = self.parser.select(div, 'a', 1) url = a.attrib['href'] m = re.match('/video-(.*)', url) if not m: self.logger.debug('url %s does not match' % url) continue _id = m.group(1) video = TricTracTVVideo(_id) video.title = unicode(title.text) url = self.parser.select(div, 'img', 1).attrib['src'] stars = self.parser.select(div, '.etoile_on') video.rating = len(stars) video.rating_max = 5 video.thumbnail = Thumbnail('http://www.trictrac.tv/%s' % url) video.thumbnail.url = video.thumbnail.id yield video class VideoPage(Page): def on_loaded(self): p = self.parser.select(self.document.getroot(), 'p.alert') if len(p) > 0: raise Exception(p[0].text) def get_info_url(self): try: div = self.parser.select(self.document.getroot(), '#Content_Video object', 1) except BrokenPageError: return None else: for param in self.parser.select(div, 'param', None): if param.get('name') == 'flashvars': m = re.match('varplaymedia=([0-9]*)', param.attrib['value']) if m: return r'http://www.trictrac.tv/swf/listelement.php?idfile=%s' % m.group(1) def get_title(self): try: title = self.parser.select(self.document.getroot(), 'title', 1) except BrokenPageError: return None else: return title.text def get_descriptif(self): try: descriptif = self.parser.select(self.document.getroot(), '.video_descriptif p', 1) except BrokenPageError: return None else: return descriptif.text def get_duration(self): try: details = self.parser.select(self.document.getroot(), 'div#video_detail div') except BrokenPageError: return None else: duration = details[2] duration_string = duration.text [ duration.text.rfind ( ' ' ) + 1 : ] tokens = duration_string.split(':') if len(tokens) > 2: return datetime.timedelta(hours=int(tokens[0]), minutes=int(tokens[1]), seconds=int(tokens[2])) else: return datetime.timedelta(minutes=int(tokens[0]), seconds=int(tokens[1])) def get_date(self): try: date = self.parser.select(self.document.getroot(), 'div#video_detail div.date', 1) except BrokenPageError: return None else: string = date.text string = string [ string.rfind('le ') + 3 : ] months = [ 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec' ] words = string.split ( ' ' ) month_no = months.index ( words [ 1 ] ) + 1 return datetime.datetime.strptime ( ( '%s %s %s %s' % ( words [ 0 ], month_no, words [ 2 ], words [ 3 ] ) ), '%d %m %Y, %H:%M:%S') def get_rating(self): try: stars = self.parser.select(self.document.getroot(), '#video_info .etoile_on') except BrokenPageError: return None else: return len(stars) def get_id(self): return self.groups[0] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002210�13034501105�0020002�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class TricTracTVTest(BackendTest): MODULE = 'trictractv' def test_trictractv(self): l = list(self.backend.search_videos('TricTrac')) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/trictractv/video.py��������������������������������������������������������������0000664�0000000�0000000�00000002010�13034501105�0020127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class TricTracTVVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.ext = u'flv' @classmethod def id2url(cls, _id): return 'http://www.trictrac.tv/video-%s' % _id ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016661�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001443�13034501105�0020774�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TvsubtitlesModule __all__ = ['TvsubtitlesModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000003713�13034501105�0020722�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser, BrowserHTTPNotFound from .pages import SeriePage, SearchPage, SeasonPage, HomePage __all__ = ['TvsubtitlesBrowser'] LANGUAGE_LIST = ['en', 'es', 'fr', 'de', 'br', 'ru', 'ua', 'it', 'gr', 'ar', 'hu', 'pl', 'tr', 'nl', 'pt', 'sv', 'da', 'fi', 'ko', 'cn', 'jp', 'bg', 'cz', 'ro'] class TvsubtitlesBrowser(Browser): DOMAIN = 'www.tvsubtitles.net' PROTOCOL = 'http' ENCODING = 'utf-8' USER_AGENT = Browser.USER_AGENTS['wget'] PAGES = { 'http://www.tvsubtitles.net': HomePage, 'http://www.tvsubtitles.net/search.php': SearchPage, 'http://www.tvsubtitles.net/tvshow-.*.html': SeriePage, 'http://www.tvsubtitles.net/subtitle-[0-9]*-[0-9]*-.*.html': SeasonPage } def iter_subtitles(self, language, pattern): self.location('http://www.tvsubtitles.net') assert self.is_on_page(HomePage) return self.page.iter_subtitles(language, pattern) def get_subtitle(self, id): try: self.location('http://www.tvsubtitles.net/subtitle-%s.html' % id) except BrowserHTTPNotFound: return if self.is_on_page(SeasonPage): return self.page.get_subtitle() �����������������������������������������������������weboob-1.2/modules/tvsubtitles/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000002512�13034501105�0021014�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� ����tIME 6��IDATx_U?G:dl%$e ,ycZV(A>ԃHT[,-d$ee&Z=YY*6JZڸN<ͽ3wg\w~=wΜYE ^5~{49r*3�l Iy�eS-� .g�UƸoueQ~�PugSP9ske)Tg ;P܂LEcU'8`٤ŵ$C`ͬ/[eh:o+�P-�HS/}Ƞ:c< z?})ݖ}@ v_NZw-P]@ưS S�hK!!^S|[NeJ-kia ЦĪ \e@ @@C/bX6K3zC[tSuE] FNJjIir 7Qb҇@V'-MQ;䀘6?Df;�E*JO*-})k7.qǀyn|k5}hʍ_ < ,.�>m"[;dT |6. xX,�F׀F8X=;8.6ʟ9Wg bC+Vzpl mmVԺڸѶG=f[}}h{B= zߛUbgNWߕ @F #[�y_]).d{(; 1Q�x;'yuߕ`FA!sDQЕ�eEC޹;P;//(Pa, R.}<y1=fBpmF7�<#�|^ ,UKφQSF� a @= :vp )Bc� Ȋ%sEƟ0 2n�umO�Ox%sG(B xΞU }~lO�y9 zV\D;~6 M y@/7k_mnu#"A > |<~; �'\V@�&0d[£@`T k7h+QuO8ׁ[FۑSQEIXڐ$F����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/module.py������������������������������������������������������������0000664�0000000�0000000�00000003327�13034501105�0020525�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.subtitle import CapSubtitle, LanguageNotSupported from weboob.tools.backend import Module from .browser import TvsubtitlesBrowser, LANGUAGE_LIST from urllib import quote_plus __all__ = ['TvsubtitlesModule'] class TvsubtitlesModule(Module, CapSubtitle): NAME = 'tvsubtitles' MAINTAINER = u'Julien Veyssier' EMAIL = 'julien.veyssier@aiur.fr' VERSION = '1.2' DESCRIPTION = 'Tvsubtitles subtitle website' LICENSE = 'AGPLv3+' BROWSER = TvsubtitlesBrowser def get_subtitle(self, id): return self.browser.get_subtitle(id) def get_subtitle_file(self, id): subtitle = self.browser.get_subtitle(id) if not subtitle: return None return self.browser.openurl(subtitle.url.encode('utf-8')).read() def iter_subtitles(self, language, pattern): if language not in LANGUAGE_LIST: raise LanguageNotSupported() return self.browser.iter_subtitles(language, quote_plus(pattern.encode('utf-8'))) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000011224�13034501105�0020332�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.capabilities.subtitle import Subtitle from weboob.deprecated.browser import Page class HomePage(Page): def iter_subtitles(self, language, pattern): self.browser.select_form(nr=0) self.browser['q'] = pattern.encode('utf-8') self.browser.submit() assert self.browser.is_on_page(SearchPage) for subtitle in self.browser.page.iter_subtitles(language): yield subtitle class SearchPage(Page): """ Page which contains results as a list of series """ def iter_subtitles(self, language): list_result = self.parser.select(self.document.getroot(), 'div.left_articles ul') if len(list_result) > 0: li_result = self.parser.select(list_result[0], 'li') for line in li_result: if len(self.parser.select(line, 'img[alt=%s]' % language)) > 0: link = self.parser.select(line, 'a', 1) href = link.attrib.get('href', '') self.browser.location("http://%s%s" % (self.browser.DOMAIN, href)) assert self.browser.is_on_page(SeriePage) for subtitle in self.browser.page.iter_subtitles(language): yield subtitle class SeriePage(Page): """ Page of all seasons """ def iter_subtitles(self, language, only_one_season=False): # handle the current season last_table_line = self.parser.select(self.document.getroot(), 'table#table5 tr')[-1] amount = int(self.parser.select(last_table_line, 'td')[2].text_content()) if amount > 0: my_lang_img = self.parser.select(last_table_line, 'img[alt=%s]' % language) if len(my_lang_img) > 0: url_current_season = self.browser.geturl().split('/')[-1].replace( 'tvshow', 'subtitle').replace('.html', '-%s.html' % language) self.browser.location(url_current_season) assert self.browser.is_on_page(SeasonPage) yield self.browser.page.iter_subtitles() if not only_one_season: # handle the other seasons by following top links other_seasons_links = self.parser.select(self.document.getroot(), 'p.description a') for link in other_seasons_links: href = link.attrib.get('href', '') self.browser.location("http://%s/%s" % (self.browser.DOMAIN, href)) assert self.browser.is_on_page(SeriePage) for subtitle in self.browser.page.iter_subtitles(language, True): yield subtitle class SeasonPage(Page): """ Page of a season with the right language """ def get_subtitle(self): filename_line = self.parser.select(self.document.getroot(), 'img[alt=filename]', 1).getparent().getparent() name = unicode(self.parser.select(filename_line, 'td')[2].text) id = self.browser.geturl().split('/')[-1].replace('.html', '').replace('subtitle-', '') url = unicode('http://%s/download-%s.html' % (self.browser.DOMAIN, id)) amount_line = self.parser.select(self.document.getroot(), 'tr[title~=amount]', 1) nb_cd = int(self.parser.select(amount_line, 'td')[2].text) lang = unicode(url.split('-')[-1].split('.html')[0]) filenames_line = self.parser.select(self.document.getroot(), 'tr[title~=list]', 1) file_names = self.parser.select(filenames_line, 'td')[2].text_content().strip().replace('.srt', '.srt\n') desc = u"files :\n" desc += file_names m = re.match('(.*?)\.(\w+)$', name) if m: name = m.group(1) ext = m.group(2) else: ext = 'zip' subtitle = Subtitle(id, name) subtitle.url = url subtitle.ext = ext subtitle.language = lang subtitle.nb_cd = nb_cd subtitle.description = desc return subtitle def iter_subtitles(self): return self.get_subtitle() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/tvsubtitles/test.py��������������������������������������������������������������0000664�0000000�0000000�00000002717�13034501105�0020221�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from random import choice class TvsubtitlesTest(BackendTest): MODULE = 'tvsubtitles' def test_subtitle(self): subtitles = list(self.backend.iter_subtitles('fr', 'sopranos')) assert (len(subtitles) > 0) for subtitle in subtitles: assert subtitle.url.startswith('http') # get the file of a random sub if len(subtitles): subtitle = choice(subtitles) self.backend.get_subtitle_file(subtitle.id) def test_get_subtitle(self): subtitles = list(self.backend.iter_subtitles('fr', 'sopranos')) assert (len(subtitles) > 0) subtitle = choice(subtitles) assert self.backend.get_subtitle(subtitle.id) �������������������������������������������������weboob-1.2/modules/twitter/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015773�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001434�13034501105�0020106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import TwitterModule __all__ = ['TwitterModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000011554�13034501105�0020036�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.messages import Message from .pages import LoginPage, LoginErrorPage, ThreadPage, Tweet, TrendsPage,\ TimelinePage, HomeTimelinePage, SearchTimelinePage, SearchPage __all__ = ['TwitterBrowser'] class TwitterBrowser(LoginBrowser): BASEURL = 'https://twitter.com/' authenticity_token = None thread_page = URL(u'(?P<user>.+)/status/(?P<_id>.+)', ThreadPage) login_error = URL(u'login/error.+', LoginErrorPage) tweet = URL(u'i/tweet/create', Tweet) trends = URL(u'i/trends\?pc=true&show_context=false&src=search-home&k=(?P<token>.*)', TrendsPage) search = URL(u'i/search/timeline', SearchTimelinePage) search_page = URL(u'search\?q=(?P<pattern>.+)&src=sprv', u'search-home', SearchPage) profil = URL(u'i/profiles/show/(?P<path>.+)/timeline/with_replies', HomeTimelinePage) timeline = URL(u'i/timeline', TimelinePage) login = URL(u'', LoginPage) def do_login(self): self.login.stay_or_go() if not self.authenticity_token: self.authenticity_token = self.page.login(self.username, self.password) if not self.page.logged or self.login_error.is_here(): raise BrowserIncorrectPassword() @need_login def get_me(self): return self.login.stay_or_go().get_me() @need_login def iter_threads(self): return self.timeline.go().iter_threads() def get_trendy_subjects(self): if self.username: return self.get_logged_trendy_subject() else: return self.trends.open(token="").get_trendy_subjects() def get_logged_trendy_subject(self): if not self.authenticity_token: self.do_login() trends_token = self.search_page.open().get_trends_token() return self.trends.open(token=trends_token).get_trendy_subjects() @need_login def post(self, thread, message): datas = {'place_id': '', 'tagged_users': ''} datas['authenticity_token'] = self.authenticity_token datas['status'] = message if thread: datas['in_reply_to_status_id'] = thread.id.split('#')[-1] self.tweet.open(data=datas) def get_thread(self, _id, thread=None, seen=None): splitted_id = _id.split('#') if not thread: thread = self.thread_page.go(_id=splitted_id[1].split('.')[-1], user=splitted_id[0]).get_thread(obj=thread) title_content = thread.title.split('\n\t')[-1] thread.root = Message(thread=thread, id=splitted_id[1].split('.')[-1], title=title_content[:50] if len(title_content) > 50 else title_content, sender=splitted_id[0], receivers=None, date=thread.date, parent=thread.root, content=title_content, signature=u'', children=[] ) if seen and (_id not in seen): thread.root.flags = Message.IS_UNREAD comments = self.thread_page.stay_or_go(_id=splitted_id[1].split('.')[-1], user=splitted_id[0]).iter_comments() for comment in comments: comment.thread = thread comment.parent = thread.root if seen and comment.id not in seen.keys(): comment.flags = Message.IS_UNREAD thread.root.children.append(comment) return thread def get_tweets_from_profil(self, path): return self.profil.go(path=path).iter_threads() def get_tweets_from_hashtag(self, path): return self.get_tweets_from_search(u'#%s' % path if not path.startswith('#') else path) def get_tweets_from_search(self, path): min_position = self.search_page.go(pattern=path).get_min_position() params = {'q': "%s" % path.encode('utf-8'), 'src': 'sprv'} return self.search.go(params=params).iter_threads(params=params, min_position=min_position) ����������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000015457�13034501105�0020142�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sBIT|d��� pHYs����+��IDATxy]UgX!UJBB(b Ԁ(' ؾ'`-(( abHU$dPsy89y/ *xqϾwZ{<E�| |?�>nzVއ[;٧ĎXW�ia;ͲQ0z}_dժUfT]"DUa+V048¦7ߤSO=Gwb bw۾mZ93*[aww7</| i:;;ykغu+VxW|Yfa5\&;7o[(2*&xG|�Ew_:BѪ8n47tx衇HVu׌j)]bB1#fE/"7p/ϯʽK&aӖ<S,\ JY6Hq([oŖ͛ho_fddիWsĮ]Xf wy'Lq"5-I)UtZZ߹lULE!x?_dOmvDA  p-uG L&b̝; /DUD",˘ל xqvɌ'vRK_bɒ%c1L4 t]'RTH&tuK%:t.dYfq%PPOJQ<? �GeŊ}=5 e_r7K#?8N(~0Mx2d;G]B2Esֆ88C8�G ~WWk�bY_Ï︓TLh&UCGk+aPUfΜSQ>!KSS[OoUϿB{M3zۼK\uU_K L�G?!ƒEo~s|f9||Du C7u:gK/eZg'D@$B̙T5#+> '<L@0AӟM6 Th�8xG咋/>=2&Vދ Q4Lp$K�'|5o.(Jb xr<5$Od'2H6Oy76oCFIh�mv_A!_+6#߸dk5f| /2onQӏAT} t_>n`:C<(JTftwfn6ng-r\~:m= 3?H,H t$l2ʵ4 7*|>'MF3M/w`6DN:Ν˔iS9vlz{{YJ~Waj:6xww':&3y1YC<^: N?hPؼy3`rx O=$]ʥ^JMC$8FZYhcn|%t*˼OɌ30*a�EPlx<N6g3+p _DN=}>׮cO3뤓\HV3<z/Ǟm{s93B ӥT16g}.<# eT] '%(&eH@$N8YM0@wXu朾f#XH8ȜlK+V~lo:(G 3X.sͷ>Om;x86 DqRU5" >"ŊB0uDDɇyQr2rxm=${ |.CDIe$ 'L4  hN1q see䮻~޽{NX 2cLV\ Hx@ Dpm< uX?H._d[ٴm;]y ,#|aEm6Bt#-_��Xş'R5I~9^^ͯO,[o]Q__O4%}(ศERtvp($)i�YUزmB%L`8 HB>0 y4===(o(J|<ԓ|?fɣKɡA6l̼yx'xW>ie.aܹE[Fxc6(J֬Yܹs%@"vL$I0 DQdr{[غu+{n:::7X,F"op!6nm߽G) ;4t͗hli w`SӃ8ehȊ(,JzЈF" ݨp%K\E%R*pt|63`ٲeJ\Ԁݛ!~MLh7r Mdy"JEQTRȬ'qϽX u UUq]UlJehK̝;d"n[daBA?_:)Ξ3eF1ʃ>HMM*U eʥaca:6Z(M-ͤjk-d|A\EP4B$Ih?agc9&#(`|r U\TUB.O]]:ȿ|6jQ۶[eYYM[6swK$!*$h>+ARf4% SXQӦҳf5OF35DE\\,]G`Ϟ=r'% Rd3?Iz_"S M�rуBw E#rCX)#*2Hh"N\U8i"ol|O|躎C<$I@ ϐJ$slr.HOEft^x!|bahtuurJ"#7�ER"h])y~'D%پg;Վ8c+x `�3WD\dObNeCC P}WT0U/ j4 # SdtՏ.k9sǛnkB̙kr<1zO�bkp׺Eӧ:###ttL3 /3˗S][ǃ>HP@$4MP(ĶmۨȢeCA\A7-YSNcѝضvݵlBTereYcY&'Jꊦi�dӴ)lܸ! d}C[h_Ԁ={1<2B(& "Im-ظJ";xOsXRDD<"*OAD`aۉb4770H:>(/pM AX'a;dRB0dv&NȄv_%>Ep mmmrc^)s 'pA DF^]ô.:z:\UU `;$QD5-ʅ"zB +$Ďvt]{a"*˟}du O' bYS˔J~ӟBE(UQu\s,z~4~WW0DB.gz.R*'B,VaxaWgqU^a$=q466bCHҌa(%HgrĄ6B꧿|@BD $ Q,1MUUEP(DiYMCtd֯[KŶi :8.,ׯzIDcTg˖-L>j (O$\>DI8<,m FUy+Ҋi<elڼ sN9qIUOUd2 :>$I@%\A<f8M7RGc3WN>:0zk/̕YYiaħpsWLxU ٧s4MCX.SIӻ~=>y Ѫ*>{R]brG,Nͤ B~J@`0ah☑566ؾu '8\.W^- [&m|.$tc!$]xK ykBT}8ட4ycF8$Isģ1TUedd, ۶N~\EDQ$1m4zzzD"<̙3&r@ (8$B*DDVw ڵkƸdYQ"0XrL&K/ `d2ɢD<KUݔEپu~GKS ۶vLI, L=hvuIg& ǧXNP�媫bk+xlR֯_e[ovTl.iۈDCs,w�(LCCk֬AE&N(:Fu]UJLΆ7z9Yg5*KOQE Ap<t6C]C=eEbE�,!0 fnvn?BIVST4h4BǤ00�npٽs'i܌$IcA@%22i2#;& ) `6PhlOk`)mnf2R&ÄzrR!O0' Q5‰=H,'dd4 *&"PWWcGd9AQ+9^hkk>z"0w$r.^|9+1AX6MN6v=iLщzzZ;[^LQW[K ⫯y6.b,bU)2xudôQ}b$Ej;r PDT,MӐ{ۻ`0]pLY8K~{f9===,Y07n$Q4Mb`iG͗*Mm;6n${`2<:.E1 }~k<]$ ۶0aJh"8xժZH<%N#"7l " kUJLjW}|vڧLX*c̝G:~;B Ect7d ǟǖkپ}+7004H4YWc"Lհk^X xxL2~}19G<laxxZrxqكn:<cԩl߾ɓ&a&i2-M\}s}L;h<LL*""BYQQU(nr 7R4,aѢE4G" E4FQ5c.|p8H*&"LA`ekW\Y^|UnV~ς3`^;TWn(8D+,Y-SL&J10MH(W,jjH &Oi!_KAAp5"(8].R"&,JFxsmt]R) sBf͚E˄ LAs*FE+s9 \Y}"[o  hllih4Sr!$i,K}RD&3J!T*Q*(mضM<GeGaGdla[kYpƧ8oхȊ_0@>Ǵ-C ࣬k}a Z06#"Dueyl׸Ra1i$g]'acㆆuuu 4a,RL&o'x"g-@4r �4}%,4 ʺ$Il߹rH8%RSS8_1a<~G6%˰`f3(Jiعs'he˖a܌iAl0 \EE$I"^~.X~BL$Cį&ikk#_*cv iooCp%\ץtCgU7֭Ư],Ice.Ǥn6ar7qL2\&KG/eUW#8& <95_JY#(覃 XE8C,E\.S|x `[H@MmT"Ah<$>1(zUF1)Gb be O vlD,~r‡gcDY;KD#rZ[[`cㅫ|HH0dP{p8aHʻoub̛7R6;>OH)r"eӳvuɿ; k% zNɇ7`mc{.4^K> p==*}  3jAV}Hoxsq#H(edFߣvWWMg8}iWPͮ8p�A&Me„q#`T Nvp6۶l"1q"ZZ4MBgŬXO?.B; fedئc{Ȳ*8K$KԌ /":{%K:oLcS=H"bX ;DG*Bꠊ4MlA@D(:(I3gY/_ߏ(puh�.c$EA?�'a``ry99N>qfChÏv ouv˹>.P(D>ŶM0@�P;U"y�OESz=����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/module.py����������������������������������������������������������������0000664�0000000�0000000�00000017543�13034501105�0017644�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime, timedelta from weboob.tools.value import Value, ValueBackendPassword from weboob.tools.backend import Module, BackendConfig from weboob.capabilities.messages import CapMessages, Thread, CapMessagesPost from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.capabilities.base import find_object from weboob.exceptions import BrowserForbidden from .browser import TwitterBrowser import itertools __all__ = ['TwitterModule'] class TwitterModule(Module, CapMessages, CapMessagesPost, CapCollection): NAME = 'twitter' DESCRIPTION = u'twitter website' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = TwitterBrowser STORAGE = {'seen': {}} CONFIG = BackendConfig(Value('username', label='Username', default=''), ValueBackendPassword('password', label='Password', default=''), Value('hashtags_subscribe', label='Hashtags subscribe', default=''), Value('search_subscribe', label='Search subscribe', default=''), Value('profils_subscribe', label='Profils subscribe', default='')) def create_default_browser(self): username = self.config['username'].get() if username: password = self.config['password'].get() else: password = None return self.create_browser(username, password) def iter_threads(self): if self.config['username'].get(): return self.browser.iter_threads() else: profils = self.config['profils_subscribe'].get() hashtags = self.config['hashtags_subscribe'].get() searchs = self.config['search_subscribe'].get() tweets = [] if profils: for profil in profils.split(','): for tweet in itertools.islice(self.browser.get_tweets_from_profil(profil), 0, 20): tweets.append(tweet) if hashtags: for hashtag in hashtags.split(','): for tweet in itertools.islice(self.browser.get_tweets_from_hashtag(hashtag), 0, 20): tweets.append(tweet) if searchs: for search in searchs.split(','): for tweet in itertools.islice(self.browser.get_tweets_from_search(search), 0, 20): tweets.append(tweet) tweets.sort(key=lambda o: o.date, reverse=True) return tweets def get_thread(self, _id, thread=None, getseen=True): seen = None if getseen: seen = self.storage.get('seen', default={}) return self.browser.get_thread(_id, thread, seen) def fill_thread(self, thread, fields, getseen=True): return self.get_thread(thread.id, thread, getseen) def set_message_read(self, message): self.storage.set('seen', message.thread.id, message.thread.date) self.storage.save() self._purge_message_read() def _purge_message_read(self): lastpurge = self.storage.get('lastpurge', default=datetime.now() - timedelta(days=60)) if datetime.now() - lastpurge > timedelta(days=60): self.storage.set('lastpurge', datetime.now() - timedelta(days=60)) self.storage.save() # we can't directly delete without a "RuntimeError: dictionary changed size during iteration" todelete = [] for id, date in self.storage.get('seen', default={}).iteritems(): # if no date available, create a new one (compatibility with "old" storage) if not date: self.storage.set('seen', id, datetime.now()) elif lastpurge > date: todelete.append(id) for id in todelete: self.storage.delete('seen', id) self.storage.save() def post_message(self, message): if not self.config['username'].get(): raise BrowserForbidden() self.browser.post(find_object(self.iter_threads(), id=message.full_id.split('.')[0]), message.content) def iter_resources(self, objs, split_path): collection = self.get_collection(objs, split_path) if collection.path_level == 0: if self.config['username'].get(): yield Collection([u'me'], u'me') yield Collection([u'profils'], u'profils') yield Collection([u'trendy'], u'trendy') yield Collection([u'hashtags'], u'hashtags') yield Collection([u'search'], u'search') if collection.path_level == 1: if collection.split_path[0] == u'me': for el in self.browser.get_tweets_from_profil(self.browser.get_me()): yield el if collection.split_path[0] == u'profils': profils = self.config['profils_subscribe'].get() if profils: for profil in profils.split(','): yield Collection([profil], profil) if collection.split_path[0] == u'hashtags': hashtags = self.config['hashtags_subscribe'].get() if hashtags: for hashtag in hashtags.split(','): yield Collection([hashtag], hashtag) if collection.split_path[0] == u'search': searchs = self.config['search_subscribe'].get() if searchs: for search in searchs.split(','): yield Collection([search], search) if collection.split_path[0] == u'trendy': for obj in self.browser.get_trendy_subjects(): yield Collection([obj.id], obj.id) if collection.path_level == 2: if collection.split_path[0] == u'profils': for el in self.browser.get_tweets_from_profil(collection.split_path[1]): yield el if collection.split_path[0] == u'trendy': if collection.split_path[1].startswith('#'): for el in self.browser.get_tweets_from_hashtag(collection.split_path[1]): yield el else: for el in self.browser.get_tweets_from_search(collection.split_path[1]): yield el if collection.split_path[0] == u'hashtags': for el in self.browser.get_tweets_from_hashtag(collection.split_path[1]): yield el if collection.split_path[0] == u'search': for el in self.browser.get_tweets_from_search(collection.split_path[1]): yield el def validate_collection(self, objs, collection): if collection.path_level == 0: return if collection.path_level == 1 and collection.split_path[0] in \ [u'profils', u'trendy', u'me', u'hashtags', u'search']: return if collection.path_level == 2: return raise CollectionNotFound(collection.split_path) OBJECTS = {Thread: fill_thread} �������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000017452�13034501105�0017455�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from datetime import datetime from weboob.tools.date import DATE_TRANSLATE_FR from io import StringIO import lxml.html as html import urllib from weboob.tools.json import json from weboob.browser.pages import HTMLPage, JsonPage, FormNotFound, pagination, LoggedPage from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Format, Regexp, Env, DateTime, Filter from weboob.browser.filters.html import Link, Attr from weboob.capabilities.messages import Thread, Message from weboob.capabilities.base import BaseObject class DatetimeFromTimestamp(Filter): def filter(self, el): return datetime.fromtimestamp(float(el)) class TwitterJsonHTMLPage(JsonPage): ENCODING = None has_next = None scroll_cursor = None def __init__(self, browser, response, *args, **kwargs): super(TwitterJsonHTMLPage, self).__init__(browser, response, *args, **kwargs) self.encoding = self.ENCODING or response.encoding parser = html.HTMLParser(encoding=self.encoding) if 'module_html' in self.doc: self.doc = html.parse(StringIO(self.doc['module_html']), parser) else: self.has_next = self.doc['has_more_items'] self.min_position = None if 'min_position' in self.doc: self.min_position = self.doc['min_position'] if self.doc['items_html']: el = html.parse(StringIO(self.doc['items_html']), parser) self.doc = el if el.getroot() is not None else html.Element('brinbrin') else: self.doc = html.Element('brinbrin') class LoginPage(HTMLPage): def login(self, login, passwd): try: form = self.get_form(xpath='//form[@action="https://twitter.com/sessions"]') form['session[username_or_email]'] = login form['session[password]'] = passwd form.submit() return form['authenticity_token'] except FormNotFound: return CleanText('(//input[@id="authenticity_token"])[1]/@value')(self.doc) @property def logged(self): try: self.get_form(xpath='//form[@action="https://twitter.com/sessions"]') return False except FormNotFound: return True def get_me(self): return Regexp(Link('//a[@data-nav="view_profile"]'), '/(.+)')(self.doc) class ThreadPage(HTMLPage): @method class get_thread(ItemElement): klass = Thread obj_id = Format('%s#%s', Env('user'), Env('_id')) obj_title = Format('%s \n\t %s', CleanText('//div[has-class("permalink-inner permalink-tweet-container")]/div/div/div/a', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')]), CleanText('//div[has-class("permalink-inner permalink-tweet-container")]/div/div/p', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')])) obj_date = DateTime(Regexp(CleanText('//div[has-class("permalink-inner permalink-tweet-container")]/div/div/div[@class="client-and-actions"]/span/span'), '(\d+:\d+).+- (.+\d{4})', '\\2 \\1'), translations=DATE_TRANSLATE_FR) @method class iter_comments(ListElement): item_xpath = '//ol[@id="stream-items-id"]/li/ol/div/li/div' class item(ItemElement): klass = Message obj_id = Regexp(Link('./div/div/small/a', default=''), '/.+/status/(.+)', default=None) obj_title = Regexp(CleanText('./div[@class="content"]/div/p[has-class("tweet-text")]', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')]), '(.{50}|.+).+') obj_content = CleanText('./div[@class="content"]/div/p[has-class("tweet-text")]', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')]) obj_sender = Regexp(Link('./div/div/small/a', default=''), '/(.+)/status/.+', default=None) obj_date = DatetimeFromTimestamp(Attr('./div/div[@class="stream-item-header"]/small/a/span | ./div/div[@class="ProfileTweet-authorDetails"]/span/a/span', 'data-time')) def validate(self, obj): return obj.id is not None class SearchPage(HTMLPage): def get_trends_token(self): json_data = CleanText('//input[@id="init-data"]/@value')(self.doc) return json.loads(json_data)['trendsCacheKey'] def get_min_position(self): return CleanText('//div[@class="stream-container "]/@data-min-position')(self.doc) class TrendsPage(TwitterJsonHTMLPage): @method class get_trendy_subjects(ListElement): item_xpath = '//li[@class="trend-item js-trend-item "]' class item(ItemElement): klass = BaseObject obj_id = Attr('.', 'data-trend-name') class TimelineListElement(ListElement): item_xpath = '//*[@data-item-type="tweet"]/div[@data-tweet-id]' ignore_duplicate = True def get_last_id(self): _el = self.page.doc.xpath('//*[@data-item-type="tweet"]/div')[-1] return CleanText('./@data-tweet-id')(_el) class item(ItemElement): klass = Thread obj_id = Format('%s#%s', CleanText('./@data-screen-name'), CleanText('./@data-tweet-id')) obj_title = Format('%s \n\t %s', CleanText('./div/div[@class="stream-item-header"]/a|./div/div[@class="ProfileTweet-authorDetails"]/a', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')]), CleanText('./div/div/p', replace=[('@ ', '@'), ('# ', '#'), ('http:// ', 'http://')])) obj_date = DatetimeFromTimestamp(Attr('./div/div[@class="stream-item-header"]/small/a/span | ./div/div[@class="ProfileTweet-authorDetails"]/span/a/span', 'data-time')) class TimelinePage(TwitterJsonHTMLPage): @pagination @method class iter_threads(TimelineListElement): def next_page(self): if self.page.has_next: return u'%s?max_position=%s' % (self.page.url.split('?')[0], self.get_last_id()) class HomeTimelinePage(TwitterJsonHTMLPage, LoggedPage): @pagination @method class iter_threads(TimelineListElement): def next_page(self): if self.page.has_next: return u'%s?max_id=%s' % (self.page.url.split('?')[0], self.get_last_id()) class SearchTimelinePage(TwitterJsonHTMLPage): @pagination @method class iter_threads(TimelineListElement): def next_page(self): params = self.env['params'] params['max_position'] = self.page.min_position if 'min_position' in self.env and not params['max_position']: params['max_position'] = self.env['min_position'] if self.page.has_next: return u'%s?%s' % (self.page.url.split('?')[0], urllib.urlencode(params)) class LoginErrorPage(HTMLPage): pass class Tweet(JsonPage): pass ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/twitter/test.py������������������������������������������������������������������0000664�0000000�0000000�00000005671�13034501105�0017335�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import itertools from weboob.capabilities.base import BaseObject from weboob.tools.test import BackendTest, SkipTest class TwitterTest(BackendTest): MODULE = 'twitter' def test_twitter_logged(self): if self.backend.browser.username: assert(self.backend.browser.get_me()) else: raise SkipTest("User credentials not defined") def test_twitter_list(self): if self.backend.browser.username: l = list(itertools.islice(self.backend.iter_threads(), 0, 20)) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) else: raise SkipTest("User credentials not defined") def test_ls_me(self): if self.backend.browser.username: l = list(itertools.islice(self.backend.iter_resources([BaseObject], ['me']), 0, 20)) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) else: raise SkipTest("User credentials not defined") def test_ls_search(self): l = list(itertools.islice(self.backend.iter_resources([BaseObject], ['search', 'weboob']), 0, 20)) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) def test_ls_hashtag(self): l = list(itertools.islice(self.backend.iter_resources([BaseObject], ['hashtags', 'weboob']), 0, 20)) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) def test_ls_profils(self): l = list(itertools.islice(self.backend.iter_resources([BaseObject], ['profils', 'jf_cope']), 0, 20)) assert len(l) thread = self.backend.get_thread(l[0].id) assert len(thread.root.content) def test_ls_trend(self): l = list(self.backend.iter_resources([BaseObject], ['trendy'])) assert len(l) l1 = list(itertools.islice(self.backend.iter_resources([BaseObject], ['trendy', u'%s' % l[0].split_path[0]]), 0, 20)) assert len(l1) thread = self.backend.get_thread(l1[0].id) assert len(thread.root.content) �����������������������������������������������������������������������weboob-1.2/modules/unsee/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015410�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/unsee/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001430�13034501105�0017517�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import UnseeModule __all__ = ['UnseeModule'] ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/unsee/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000006141�13034501105�0017447�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser import os from uuid import uuid4 from urllib2 import Request from urlparse import urljoin from weboob.tools.json import json from weboob.deprecated.browser.parsers.lxmlparser import LxmlHtmlParser __all__ = ['UnseeBrowser'] def to_bytes(s): if isinstance(s, unicode): return s.encode('utf-8') else: return s class FileField(object): def __init__(self, filename, contents=None, headers=None): self.filename = to_bytes(os.path.basename(filename)) self.headers = headers or {} if contents is not None: self.contents = contents else: self.contents = open(filename).read() class UnseeBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'unsee.cc' ENCODING = 'utf-8' def _make_boundary(self): return '==%s==' % uuid4() def _make_multipart(self, pairs, boundary): s = [] for k, v in pairs: s.append('--%s' % boundary) if isinstance(v, FileField): s.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (k, v.filename)) for hk, hv in v.headers.items(): s.append('%s: %s' % (hk, hv)) v = v.contents else: s.append('Content-Disposition: form-data; name="%s"' % k) s.append('') s.append(to_bytes(v)) s.append('--%s--' % boundary) s.append('') return '\r\n'.join(s) def _multipart(self, url, fields): b = self._make_boundary() data = self._make_multipart(fields, b) headers = {'Content-type': 'multipart/form-data; boundary=%s' % b, 'Content-length': len(data)} return Request(url, data=self._make_multipart(fields, b), headers=headers) def post_image(self, name, contents, time): # time='first' for one-shot view params = [('time', time), ('image[]', FileField(name or '-', contents))] request = self._multipart('https://unsee.cc/upload/', params) d = json.loads(self.readurl(request)) return {'id': d['hash']} def get_image(self, id): doc = self.get_document(self.openurl('https://unsee.cc/%s/' % id)) images = LxmlHtmlParser.select(doc, '//img/@src[starts-with(., "/image/")]', method='xpath') url = urljoin('https://unsee.cc', images[0]) return self.readurl(url) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/unsee/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000000743�13034501105�0017547�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME�I���tEXtComment�Created with GIMPW��KIDATxK C'&C-;2'%!B!~SE5_av< p"+�x E!F`z='NL( * BKX;|F^PڮỦyàR"l44nDȓ[TP怴9}*}tsnIQNxDjUk-@X""RxeɍU;=G鋴y5EwM11$d@<u eFTg!財(pk5A8�[c|Addd I!B!B|ǀ&_8����IENDB`�����������������������������weboob-1.2/modules/unsee/module.py������������������������������������������������������������������0000664�0000000�0000000�00000004636�13034501105�0017260�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.paste import BasePaste from weboob.tools.capabilities.paste import BasePasteModule from weboob.tools.capabilities.paste import image_mime from weboob.deprecated.browser.decorators import check_url import re from .browser import UnseeBrowser __all__ = ['UnseeModule'] class UnPaste(BasePaste): @classmethod def id2url(cls, id): return 'https://unsee.cc/%s' % id class UnseeModule(Module, BasePasteModule): NAME = 'unsee' DESCRIPTION = u'unsee.cc expiring image hosting' MAINTAINER = u'Vincent A' EMAIL = 'dev@indigo.re' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = UnseeBrowser EXPIRATIONS = {3600: 'hour', 86400: 'day', 86400 * 7: 'week'} def can_post(self, contents, title=None, public=None, max_age=None): if re.search(r'[^a-zA-Z0-9=+/\s]', contents): return 0 elif max_age is not None and not self.get_closest_expiration(max_age): return 0 else: mime = image_mime(contents, ('gif', 'jpeg', 'png')) return 20 * int(mime is not None) @check_url('https?://unsee.cc/.+') def get_paste(self, id): paste = UnPaste(id) paste.contents = self.browser.get_image(id).encode('base64') return paste def new_paste(self, *a, **kw): return UnPaste(*a, **kw) def post_paste(self, paste, max_age=None): if max_age is None: max_code = 'week' else: max_code = self.EXPIRATIONS[self.get_closest_expiration(max_age)] d = self.browser.post_image(paste.title, paste.contents.decode('base64'), max_code) paste.id = d['id'] return paste ��������������������������������������������������������������������������������������������������weboob-1.2/modules/unsee/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000002544�13034501105�0016746�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Vincent A # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class UnseeTest(BackendTest): MODULE = 'unsee' # small gif file DATA = 'R0lGODlhAQABAIAAAP///wAAACH5BAAAAAAALAAAAAABAAEAAAICRAEAOw==\n' def test_unsee(self): assert self.backend.can_post(self.DATA, max_age=3600) post = self.backend.new_paste(None) post.contents = self.DATA post = self.backend.post_paste(post, max_age=3600) assert post.id got = self.backend.get_paste(post.id) assert got assert got.contents.decode('base64').startswith('GIF89a') # we can't check the exact data, unsee.cc modifies the images... ������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015100�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/__init__.py������������������������������������������������������������������0000664�0000000�0000000�00000001424�13034501105�0017212�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import UpsModule __all__ = ['UpsModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/browser.py�������������������������������������������������������������������0000664�0000000�0000000�00000002761�13034501105�0017143�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.deprecated.browser import Browser from .pages import TrackPage __all__ = ['UpsBrowser'] class UpsBrowser(Browser): PROTOCOL = 'https' DOMAIN = 'wwwapps.ups.com' ENCODING = None PAGES = { 'https://wwwapps.ups.com/WebTracking/track': TrackPage, } def get_tracking_info(self, _id): data = {'HTMLVersion': '5.0', 'USER_HISTORY_LIST': '', 'loc': 'en_US', 'track.x': 'Track', 'trackNums': _id.encode('utf-8'), } self.location('https://wwwapps.ups.com/WebTracking/track', urllib.urlencode(data)) assert self.is_on_page(TrackPage) return self.page.get_info(_id) ���������������weboob-1.2/modules/ups/favicon.png������������������������������������������������������������������0000664�0000000�0000000�00000012444�13034501105�0017240�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq��IDATx՛y]W}?s[zoւeY-kbYm�6! bLJ0IH2Ibec8LX,’ZjR{?}{- U3W=w�,>/}� 1Q1V5b,*c#XcFA4Q #!x4{T=Z~v�>'!KCB <w >-誷>管ߧ'".Han!ւ:{Xc-bUlN R�$@5 -&/�!EԻpXWua`�Wb"^1EE$kXLtPc1ƀ2�j @e%�J�<A=Z 'NڮC KA )C/O}0�m#J]Q_JݣJUOo%MeE[/5Www-�*+%k2D"z'Aڄ YQ%bڼ-]$UH�o?T`!0(H~"%VPDjtڄAT E%bT5(+bEmm=olNU6N%&iݮ-A .8P%J+m#ZV%HIPo,ӕ4\\kJ:EC2w*Ky!jT_~IDH,@d >03=gG8wvR, !EΟenn=>(k:0K̟'!y=bԂǎ+<8HA/o̭{הe(c\,ع)NO099ωr(B<p!9gDMLLJ`=wx} ڸRxp'v> �<�S30=QmBGDGdRRB2 Q&&zJk \4b₯#c&& XLrHo ڀUA) B0`"FR- ҙȚjE`r?:x(sEO(^obT|=[5Y J5ZbD͌p4_!FF1`ڈ(2DIFhii|즔4Z65-�HV7tV,Oz~QMv-ۑ|.Hs.}9h#4CR[�@hLdYXt;y~p?{l6C&cwҪvZeb&JܺP2|CϜSB.ELժe b[м6H{k&.%/F>ЩI2/3e3OK@5V j\֔@jظzBc˸RHRF.kp9[T4ʮo+WܶkS㨚JP3 cִe˻$A@WEJ[-hZp8@/<-ZJvoEqjwYlH+@k2hi,զ*V[W+e7ZjYYrX'nB]BgmQn^Z k[/׮Jhs(s>d%qB�.8*&H�S !!6VuXtplB\ XL0v:1,b@|P.fZcSLʿ6˾{n39:<ϳ'j -ϐGʕy~r AL'yByb[.aǥ|PEoɖnοBΛ/g,w?>6]^_n$d'1OGYHV:Jµ;:x޵ ϼ0Ș7϶5sSxb 6 Ak\d�1=9\mMks||gq[ym36OS3gm<2ޗ]6Q9BeĚZVw[M?8[ (.+7gU<rb3ʎm)?=:!7+ְ M3:Ubg'=Gj8g}HƜsjXT ΖLUߛa"Od!B>k8>3 ˷u^, 1B[."t#W}8s1}y^ k`vܖ|:UYLCjQA&Km#vF-G0=F]u.(:WV iz#{d";l͝{^G޹_CUѳ`HSK(JV+Jj 仍 jw"f= Q(;~=z z-};희*TOZMaRWG$C{U;mS�flSCꢫ#9P l̟:_cǎNT3Hs hPWwcU[p\!鿁y@MDDD˩qZ iURQA᥃LŌ/"FP h�c{0=o 9|pqmr1*"|V!iYC<WX<W\ځ5P,&t mx4N؅@<B`qV*qȰ;3] ͱqM{YC!VB]2]Z/E !ȹH2<s5|b-.-g9ppѱc GM2tjKͷE5 :F0.鮰~m;xN Ocf|K'CN캸Ss-d}u}YIN/bSꤰ #,>yɄUL = ϟ*ƫVq.XFb X+ �|(a5\uņXdl|g&8925I$4Xdxt֐ 3Ӌ<s|{C0<C6#x/( n^u8f~k?41*Sk.%OpUXBOw. /rCc䳓twfp{Io D`a!__=BRr.k 1Lpk6.zKB6o}o {Tb(=8]{vGT :$`$UmWؔcv!J@EOoZ&W.P,qUJ$\;b`(CYXزȖٓoFHyu떵hH*9} DҤ~-2pcKfK]2Ŧ$IYՕI }xk$*:ڳiOfP]>lI#ǧ8qf!k1g3]-,}�}w<U�6nxK7V/ ,}#]ud Mqbdbf:Jƥ \ ޻R6x)M G[,mRkV+Ɓəsq:6-J 3( kc֮;=.?瀱4oFoOGrZCWKc6 *Pj?:)%n*VwO �&Ə?,gٽk΅'Wn<ݥQZZX}6Vcm)G�#r/_ '$ ܳkLӼ,PohRdٷ{5%Oﻒ"{ ^nܷcLUġœѼ#M|1xI{ְqm!$ST{AZ³Rw`Dkٲ!e>Vr9Y/]b|L ?u> 5;[R'0T vljR ѕsrPaGVش=zq.G s㟻fi��?>p\RO}\w*dI'}˲*,ki8޴9I&.]})�d_Af+rovkovFUQ[ts-@R+m}/!vAVѥx%@0A/ĀMk^wUR+rv/}M|Wc)?:~��?A;njO{*{.پ;6]`^kENu;b6"Z:7��$v} ! ӹ|^qVWqa|nsK3MBPeS;wy'o>[G&'?rV  swd6Lfӯ}~@4 ѻ, åu<#M2W\ǞѤ,(?pr%ЛD&(GKvUWTWIvFc>>z/zԢu^�e-;rv㻧{8*dzM\u*,ѮL g#nOq< }g^07.poA8T?cmE2I=ȡ|9NpBVHN$7z/jO{Gw񪽛(h.=W^Cmiʟp3e##bEѠdnbOq<$FSb\^@�uc_!6mtwXw sW|gğ"f[7jeH6kCe9|:5(wKs@r E:2-lKGβ'C.^Q/=疯NL}Vzn�07&7mB/-NwgBbD(K9WfC;w`Ej /̾bx}W~Рf@n.* 1QCChpZrvdoIf-IuvK{[lZK%VWUһ_Tt~^� L~J߽҅GѠjz*\L9&=+WU&1U_<̷ϿL�go?rZUjkV�|֗oIz^z}zモ_܋^_pȆ4nJ&?ވBթ?wS¢3@N81w7Q tZ.��6+y?Ɍjؤ:%xI/U5 K @b :Q�gگ=_ sݒ'����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/module.py��������������������������������������������������������������������0000664�0000000�0000000�00000002265�13034501105�0016744�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.parcel import CapParcel from weboob.tools.backend import Module from .browser import UpsBrowser __all__ = ['UpsModule'] class UpsModule(Module, CapParcel): NAME = 'ups' DESCRIPTION = u'UPS website' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' BROWSER = UpsBrowser def get_parcel_tracking(self, id): with self.browser: return self.browser.get_tracking_info(id) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/pages.py���������������������������������������������������������������������0000664�0000000�0000000�00000004241�13034501105�0016552�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from dateutil.parser import parse as parse_date from weboob.capabilities.parcel import Parcel, Event from weboob.deprecated.browser import Page class TrackPage(Page): def get_info(self, id): if len(self.parser.tocleanstring(self.document.xpath('//p[@class="error"]')[0])) > 0: return None p = Parcel(id) for dl in self.document.xpath('//dl'): dt = dl.find('dt') dd = dl.find('dd') if dt is None or dd is None: continue label = self.parser.tocleanstring(dt) if label == 'Scheduled Delivery:': p.status = p.STATUS_IN_TRANSIT elif label == u'Delivered On:': p.status = p.STATUS_ARRIVED else: continue m = re.search('(\d+/\d+/\d+)', dd.text) if m: p.arrival = parse_date(m.group(1)) p.history = [] for i, tr in enumerate(self.document.xpath('//table[@class="dataTable"]//tr')): tds = tr.findall('td') if len(tds) < 4: continue ev = Event(i) ev.location = self.parser.tocleanstring(tds[0]) ev.activity = self.parser.tocleanstring(tds[-1]) ev.date = parse_date('%s %s' % (tds[1].text, tds[2].text)) p.history.append(ev) p.info = self.document.xpath('//a[@id="tt_spStatus"]')[0].text.strip() return p ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/ups/test.py����������������������������������������������������������������������0000664�0000000�0000000�00000001473�13034501105�0016436�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class UpsTest(BackendTest): MODULE = 'ups' �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicsec/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015545�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicsec/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0017655�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VicSecModule __all__ = ['VicSecModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicsec/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000016400�13034501105�0017603�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.pages import HTMLPage from weboob.capabilities.base import Currency from weboob.capabilities.shop import Order, Item, Payment from weboob.exceptions import BrowserIncorrectPassword from datetime import datetime from decimal import Decimal from itertools import chain import re __all__ = ['VicSec'] class VicSecPage(HTMLPage): @property def logged(self): return bool(self.doc.xpath( '//a[@href="https://www.victoriassecret.com/account/profile"]')) class LoginPage(VicSecPage): def login(self, email, password): form = self.get_form(name='accountLogonForm') form['j_username'] = email form['j_password'] = password form.submit() class HistoryPage(VicSecPage): def iter_orders(self): return [onum for date, onum in sorted( chain(self.orders(), self.returns()), reverse=True)] def orders(self): for tr in self.doc.xpath('//table[@class="order-status"]/tbody[1]/tr'): date = datetime.strptime(tr.xpath('td[1]/text()')[0], '%m/%d/%Y') num = tr.xpath('td[2]/a/text()')[0] status = tr.xpath('td[4]/span/text()')[0] if status == u'Delivered': yield date, num def returns(self): for tr in self.doc.xpath('//table[@class="order-status"]/tbody[3]/tr'): num = tr.xpath('td[1]/a/text()')[0] date = datetime.strptime(tr.xpath('td[2]/text()')[0], '%m/%d/%Y') status = tr.xpath('td[4]/span/text()')[0] if status == u'Complete': yield date, num class OrderPage(VicSecPage): def order(self): order = Order(id=self.order_number()) order.date = self.order_date() order.tax = self.tax() order.discount = self.discount() order.shipping = self.shipping() return order def payments(self): for tr in self.doc.xpath('//tbody[@class="payment-summary"]' '//th[text()="Payment Summary"]/../../../tbody/tr'): method = tr.xpath('td[1]/text()')[0] amnode = tr.xpath('td[2]')[0] amsign = -1 if amnode.xpath('em') else 1 amount = amnode.text_content().strip() pmt = Payment() pmt.date = self.order_date() pmt.method = unicode(method) pmt.amount = amsign * AmTr.decimal_amount(amount) if pmt.method not in [u'Funds to be applied on backorder']: yield pmt def items(self): for tr in self.doc.xpath('//tbody[@class="order-items"]/tr'): label = tr.xpath('*//h1')[0].text_content().strip() price = AmTr.decimal_amount(re.match(r'^\s*([^\s]+)(\s+.*)?', tr.xpath('*//div[@class="price"]')[0].text_content(), re.DOTALL).group(1)) url = 'http:' + tr.xpath('*//img/@src')[0] item = Item() item.label = unicode(label) item.url = unicode(url) item.price = price yield item def is_void(self): return not self.doc.xpath('//tbody[@class="order-items"]/tr') def order_number(self): return self.order_info(u'Order Number') def order_date(self): return datetime.strptime(self.order_info(u'Order Date'), '%m/%d/%Y') def tax(self): return self.payment_part(u'Sales Tax') def shipping(self): return self.payment_part(u'Shipping & Handling') def order_info(self, which): info = self.doc.xpath('//p[@class="orderinfo"]' )[0].text_content() return re.match(u'.*%s:\\s+([^\\s]+)\\s'%which,info,re.DOTALL).group(1) def discount(self): # Sometimes subtotal doesn't add up with items. # I saw that "Special Offer" was actually added to the item's price, # instead of being subtracted. Looks like a bug on VS' side. # To compensate for it I'm correcting discount value. dcnt = self.payment_part(u'Special Offer') subt = self.payment_part(u'Merchandise Subtotal') rett = self.payment_part(u'Return Merchandise Total') items = sum(i.price for i in self.items()) return dcnt + subt + rett - items def payment_part(self, which): # The numbers notation on VS is super wierd. # Sometimes negative amounts are represented by <em> element. for node in self.doc.xpath('//tbody[@class="payment-summary"]' '//td[contains(text(),"%s")]/../td[2]' % which): strv = node.text_content().strip() v = Decimal(0) if strv == u'FREE' else AmTr.decimal_amount(strv) return -v if node.xpath('em') and v > 0 else v return Decimal(0) class VicSec(LoginBrowser): BASEURL = 'https://www.victoriassecret.com' login = URL(r'/account/signin/overlay$', LoginPage) history = URL(r'/account/orderlist$', HistoryPage) order = URL(r'/account/orderstatus/submit/view\?emailId=(?P<email>.*)&orderNumber=(?P<order_num>\d+)$', r'/account/orderstatus.*$', OrderPage) unknown = URL(r'/.*$', VicSecPage) def get_currency(self): # Victoria's Secret uses only U.S. dollars. return Currency.get_currency(u'$') def get_order(self, id_): return self.to_order(id_).order() def iter_orders(self): for order in self.to_history().iter_orders(): if not self.to_order(order).is_void(): yield self.page.order() def iter_payments(self, order): return self.to_order(order.id).payments() def iter_items(self, order): return self.to_order(order.id).items() @need_login def to_history(self): self.history.stay_or_go() assert self.history.is_here() return self.page @need_login def to_order(self, order_num): self.order.stay_or_go(order_num=order_num, email=self.username.upper()) assert self.order.is_here(order_num=order_num, email=self.username.upper()) return self.page def do_login(self): self.session.cookies.clear() # Need to go there two times. Perhaps because of cookies... self.location('/secureoverlay/wrapper/medium/account/signin/overlay/show') self.login.go() self.login.go().login(self.username, self.password) self.history.go() if not self.history.is_here(): raise BrowserIncorrectPassword() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicsec/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000003752�13034501105�0017707�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@����.���gAMA�� a��� cHRM��z&���������u0��`��:��pQ<���bKGD�̿���tIME7��IDATXW{PU[1MҦX3ԭf3MrMJM9FQ{IYM\ P): 05 +pIdT8<Γ^kF0SP4sל>k[!F�J!oPXJ*oK��tnRa�Z]xX0eH�-1̿{ Cw27B[`ٿ$djDQ\Nn%`i7~<K>:!JkatBu±+i(l>C`_�G?u.z<98D=L>-PPhn u^@@vD,rr IvDw�4U]cNP,PjN?&4(Gd@=i#�E&6Ս+SA7͗L&PJEĴ)tx~sTPiƒ=#(̠ѵ,OxŢT)q�f־M> r�t�e^̒O}hA;@2e'S돳ߥ a@ ,9('�(xhK{5&`"FClvF�8-'t$!C .y$ &xڈ5L ʄ^`AQ\٤yJ  قVOLŻOGe { *�#ڝBwHvm'*SW7 BǁAwmoWmŵ{9.7,>4Զxrk}l?kyvGNY.<V<WX5^TW#ݍ;Wo'}(aaڀ#rZUzL]#w RwU0G(sϸ ;#4J[^&n4.xy,Y =;{÷'G@Q�ywq GH/;zfא0,8c̦0'rD#XyFw[f"Ŵ8 F:9/_ؔq=wt2o/KJSpF+/i fJ+?oQ; ^*Jo? - sx_9pF!F〺EM*Oz&ҽ5Y#N'49y&s9@󄵞M y-8`ʽZpka p`9ou|RGɤv:[;R/x:wV>L!UkbO`ʴJIV(z57$/LҘVf:ض3) ֲRg) ϔNRb6.Ժ$ϯ�i-"aW3X5/!' ;P,ǜ 7<.'iMçc8 Xz{bڦ,t5DR(ex PtcA]xqh%{Ȝ439+Y 0Cr E҇bZ-bˆ�ËN_HE4-�򤄣ѭ()ȒƏcpp�J(Yk^wFgG6&Bf0 u0�w/3fE)HU*p]lt5̸>-uegnd_ �KuwT �o<E Q?D7���%tEXtdate:create�2015-02-22T22:25:33-06:00e���%tEXtdate:modify�2015-02-22T22:25:33-06:00:����IENDB`����������������������weboob-1.2/modules/vicsec/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000003543�13034501105�0017411�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.shop import CapShop from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import VicSec __all__ = ['VicSecModule'] class VicSecModule(Module, CapShop): NAME = 'vicsec' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Victoria\'s Secret' CONFIG = BackendConfig( ValueBackendPassword('email', label='Username', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = VicSec def create_default_browser(self): return self.create_browser(self.config['email'].get(), self.config['password'].get()) def get_currency(self): return self.browser.get_currency() def get_order(self, id_): return self.browser.get_order(id_) def iter_orders(self): return self.browser.iter_orders() def iter_payments(self, order): return self.browser.iter_payments(order) def iter_items(self, order): return self.browser.iter_items(order) �������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicsec/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000002146�13034501105�0017101�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class VicSecTest(BackendTest): MODULE = 'vicsec' def test_history(self): """ Test that at least one item was ordered in the whole history. """ b = self.backend items = (i for o in b.iter_orders() for i in b.iter_items(o)) item = next(items, None) self.assertNotEqual(item, None) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016377�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001447�13034501105�0020516�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VicSecCardModule __all__ = ['VicSecCardModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/browser.py������������������������������������������������������������0000664�0000000�0000000�00000011177�13034501105�0020443�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from requests.exceptions import Timeout, ConnectionError from weboob.capabilities.bank import AccountNotFound, Account, Transaction from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.browser import LoginBrowser, URL, need_login from weboob.browser.exceptions import ServerError from weboob.browser.pages import HTMLPage from weboob.exceptions import BrowserIncorrectPassword from datetime import datetime __all__ = ['VicSecCard'] class SomePage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//span[text()="Sign Out"]')) class LoginPage(SomePage): def login(self, username, password): form = self.get_form(name='frmLogin') form['username_input'] = username form['userName'] = username form['password_input'] = password form['hiddenPassword'] = password form['btnLogin'] = 'btnLogin' form.submit() class HomePage(SomePage): def account(self): id_ = self.doc.xpath(u'//strong[contains(text(),' '"Credit Card account ending in")]/text()')[0].strip()[-4:] balance = self.doc.xpath(u'//span[@class="description" and text()="Current Balance"]/../span[@class="total"]/text()')[0].strip() cardlimit = self.doc.xpath(u'//span[contains(text(),"Credit limit")]' '/text()')[0].split()[-1] paymin = self.doc.xpath(u'//section[@id=" account_summary"]' '//strong[text()="Minimum Payment Due"]/../../span[2]/text()' )[0].strip() a = Account() a.id = id_ a.label = u'ACCOUNT ENDING IN %s' % id_ a.currency = Account.get_currency(balance) a.balance = -AmTr.decimal_amount(balance) a.type = Account.TYPE_CARD a.cardlimit = AmTr.decimal_amount(cardlimit) a.paymin = AmTr.decimal_amount(paymin) #TODO: Add paydate. #Oleg: I don't have an account with scheduled payment. # Need to wait for a while... return a class RecentPage(SomePage): def iter_transactions(self): for li in self.doc.xpath('//section[@class="transactions"]//div/li'): date = li.xpath('p[@data-type="date"]//text()')[0].strip() label = li.xpath('p[@data-type="description"]//text()')[0].strip() amount = li.xpath('p[@data-type="amount"]//text()')[0].strip() t = Transaction() t.date = datetime.strptime(date, '%m/%d/%Y') t.rdate = datetime.strptime(date, '%m/%d/%Y') t.type = Transaction.TYPE_UNKNOWN t.raw = unicode(label) t.label = unicode(label) t.amount = -AmTr.decimal_amount(amount) yield t class VicSecCard(LoginBrowser): BASEURL = 'https://c.comenity.net' MAX_RETRIES = 10 TIMEOUT = 30.0 login = URL(r'/victoriassecret/$', LoginPage) home = URL(r'/victoriassecret/secure/SecureHome.xhtml', HomePage) recent = URL(r'/victoriassecret/secure/accountactivity/Transactions.xhtml', RecentPage) unknown = URL('.*', SomePage) def get_account(self, id_): a = next(self.iter_accounts()) if (a.id != id_): raise AccountNotFound() return a @need_login def iter_accounts(self): yield self.home.stay_or_go().account() @need_login def iter_history(self, account): for trans in self.recent.stay_or_go().iter_transactions(): yield trans def do_login(self): self.session.cookies.clear() self.login.go().login(self.username, self.password) if not self.page.logged: raise BrowserIncorrectPassword() def location(self, *args, **kwargs): for i in xrange(self.MAX_RETRIES): try: return super(VicSecCard, self).location(*args, **kwargs) except (ServerError, Timeout, ConnectionError) as e: pass raise e �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000036461�13034501105�0020544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@������gAMA�� a��� cHRM��z&���������u0��`��:��pQ<���bKGD X���tIME,/��<5IDATxգUgιQP)QPP $-@IT)A HIK#)% 'k9{ ܎q\!Z/%/////9<53vozc<u+ȕy]V3etp¯V2 :6\)C_%Fz MXM, ӑ& zYdk aX jcy+;-vPl`glyK&dqFچxf:fO!C rMn ]⺸ʧ*`X@o<d~u.rm-I9) fWYg'Kz*T O@MAju>�1O)OB\lQ>|mzQHTS5'oz=DD?ߓ iw7Gg7\9w-!k.;|C= 9grq'@m7;ޜ qӮ3pj eYj U;SU7z~Z : g # 62"�3K0H7/M~t,gzNioM!wEN~('ɷ?vgK8`qM0!jA2l^0Am0{"#pv#ZM)(+@ Sd ܆E7!9 r>U_~9mL/ :`Fj<a"Ne`& BܠCwӖksP8L[B_4 r[9*%z''L#|k(d佼+xKCl.sÖഉfDA|cva;2xN,Ps <dĮM'#zj?:AL4A1{m H,Z"X+CjVxEe<vA4 Kc`s]c>E t0DR )*iYx(i[XqF, ӱ[j] ;2r=vXdz(vq٧;{# z8ݺ8%'9#`GtB7rd�xMnesSWCtH}1Jp6َ:0$Xs<b[NtiT9ﮇ yԧA4$Tp׃|;ȧu5\kZAڍS҆9 "vDFʼG=+Z9 .@ɂjgN3uuK ދy`ʞ:= I&_^ZWc3 NKJN=gj>@|+= S43Y-Bp5@lo/9@ogME8Kh0 D#,"aU+&֫@5Q^PCM)նAU �^|): 3ρ1V)@SQMֈ7(\| %Pm%X=,a xY[Oۀvvk${; )x [3"H*̎HQ:@#P֮6?- aʂӍb dSL)9RG Jk?ˊ+u{/!H= b HH5!<O B" t_HU0u*lXR02{yt%P;\yDGN)0{ X_>扅 ʅ>vnSNC(' u? ^H�w8ū!) ?h .nR BPfyB@I!m"΋YB,θ4-6;ԁ}6{,8U8Y TeZU&LqQP8ցWCw KLGM.Og0xcܓӠ_ePwTU7jɛ@t@M|wdK< ';&Lr cd�n2o#= ESAp6:@SOv4#b2@vŁ VoI { V<zT7͝ a~A[ݠ ^Sv(zv`^q 8(" gfXBQSnY+; f4C@K D_ *+z ƙ4` YZQ,ѣ2z8-ܹnzDF&&B;AO0!0P3�?΃s} ~-f]Af \9 "Ǣ@.w8y!-vor= 6.w) ]~ I",?g,W"c/|:A?:v0,pXF6=پl+�_PU[l$ r*Z۴3G!K>4o,w>oi..e l>j߰wT-0}IWLs Mh lDJi]2K�Y8خ*is;E4QB|g" Bp^#`#v9w~0�졷( AE.l A~<A(dǂbK؇Al[3@6T#f%8©[@f �UܗSAwl agg*v6H=*,"QD#yd!ݞՐ�KH%tb}�t\u[΀l#Ku@yt3~~036ee"j,`޵BX,B /fDQ\v<87<P}EkDs5QUl3[87nʮ6  Uf*ؓ pa#+[s=׎ՀFjwT<59P|rl-%m-iLk2&b([xH טno V�޴4`RfTuN^O�ca~p$;_B79�'4)CZ=, n{@*pv *3geq=pҜj8+>t`ƓY7w,SSY LuÏE&0I2C.T;lUQ0" o AȢ'p7dmAgfp@Wyu ҳ 3Dް>`ŏ:E9 vxtNvkz rac9i'؅)vzn"(/kιܟn]J(y6J77ث&) @o WA>=E uŷ{48bFhSMA:L.s qt| Աm5d>=w1 µ~8lٳ /ʪR! ɱ%"R_2#+zρ; oo$΀Q|˅eBw{Cvw!8 rB8*&?pz︍!Ea$QT1.Mk#8/nAg">W`)wi-gR thJ`;]ۃq`&r4( g;8kf>a{=Z~~0'ZX$nncLۆ]e[t@vNn!KXfqS8o{QlYl9;EKgv1EmYqc<PS56ù zҌ!X< 2Un=A4E[A DUYSLg{]"߀n3ixW¸4yX@ck ~^tL,1!+ۥ`DpKXeE.P <<`.gi<*A3^ZTi dm r$P;%i/}Bl>L�;Vȇr`.U ;RΚ>o߇g9TjM- "Uo� BP;8Asꉊ{r]`.eamq;I.k@xX ]j+9a?�bc 0]LoerǤB`Uaj^fBX1gap ?ڊvx-xf>bX<'ׂ b0\s9tSI!-}&yLj\%.=.8F{rF ' =eALķȫ 0Gk \5SbnANԣN*/|n?z%~3 H<n](.^'@]TӠA2_ 7ׅ'-ʦjC"">v=M 00%[TS@T FIqR�ֲ|!&Bo:w xU=yQ@71g7 <rDJP9Q Fb)ȧUV}v@{>ml;U+@~ P�w/_V!8CY;xy 4Io!pSN6Vҭ@P.A~&x\T4KN_]Hv\)߆\(b$7w |( a΁f0An\5!\/[4-A~&AVd50mE[QskC?$F/qnA*"g`UPrx? ' {7Ӡnʣ[a V `xG tn*dЏl)"6: K ~ڤM`&kÁ=EM9 tu8B9WM:eWZS +WupWNWFYZl!Ǧ�D}Q]ε )gn  'KZ qWÐ~ JڕTԷ#u%oȵo)s]'s v;EA>=,3!T6h :TQt\_-r(#JRVEt ':MCR`^�^t�$^s� &pz,�UVS}-5c5t(PŜ[Ηq7٭ +o@'�~vlW8cN;uB<"#ս�}AӦ]`"/ wGr@fy!> u! ʛM({�}B8Tx~Q.\W/xK@NE545̛61N D@6rH /bs_CdOS & DKLShwn}R?Ep Bn 0 bxo7 I^u&ȸ%7Nڋhc>!�l92,Lm!Bm]d CL6prwOX!Q[O@xWW߁]fW D߮sO6v ]ۖ mL}+j Qһr2F@¤tѐ% KJNu$CMuHY-VmiS[L.$)1NpeZ'?rj/F!"׍?@\ bA)M([~b683-b#3|/[2Ӟ&D :j0M_ "Op W4_ $;qJNSSDpLg49"@}L}&RYUKYTOllۋAV DqHQ@,S 84[G`߱Mn,pr(rA렌`a@L, _fM1pav{M bl5iT䩴p o~"2Mən JX? >IH.kľs7AެDA4賑KIC2J@B&s"9i;gɟq쳜"LP%-Kٝ:d�S=$z` H%`w{*yvm` z5,6@tLҖh*'=7@ɉ~:wak03 58Iu�Ƌ| ns6p<6%I'[۩ <)@0L )~.X7J/\:ex0">6;܂<^3?%-#w@ /v Ѵh׮nd&=$e)QlzׁUsnZR|".eO8b?ք/&$_!sr_2v0 d',_m\X>{آIِ|'yZCy-UzҠ>ys7i`t.ؚ"A`61},<l,]$g[߉+?ף!]O?AL3@=2 .S|~^xu# `A`!v 8/ b~^975cu 3~a||Zx4Cl@}& lNxoiae0W_ [) ]HZC u! Bb{t}A^ i*8(7I ˁW(Z~`DÍezۜ d}:oadUz`ٕBe6>r|y\zI .U^+PP4;!J'p̶W)1qA @<$Pi*�j衦ol2K@qiYQT?pGGDB"AgB:r5T-Vs1Fg=޲ }0Nd(&J2vuB䟓Բ, NKyX.??�g.!ix%ț9)Cɧȳ7pS`߾=BWlgs5|&%(@M{>uO)pgzX(FX7Rxbw[C7ޣحQ|oW {BtBT]7$ܳճ!37s[6f |; s=k߀Ht0!Tp-v+ce}`MIQ[5AVed95:bMmPVYeAȏG t-]K U@ąۦ2v _żb_ZVRH)85n>??༨^SAp&VY `#`oR. N@<W M~fNuoI_ } /jt'nW vCNǜo!%'; 6<Ӣ2% ˅I3{΃O;q% -Q`lj޽9nʱ7v`]JHo(RU߁\­=UtZpH*_!20H ~Z†]8_9p^=tz2YhG;p; ^oNU H5JQmR6k0uM]S ֺ9#DI =lvaV,pP?yj-߀*:@MTT 8 xP_9oeE@nsn*rj,�8boċ$ȭgձ-H'|ۙY7CNG},ƹzS/r/5Z;?$$OM=x}­JqԽzwR]Pb-kcy pIouf=D׈?N1 4$:CǓJi {\;@qS]N{f6Kd&bVTV�}_S06%l oYYΑmuD=QUV{⾷1qNTV'TU9"CTqLLAvW "E4yW\Ax.0gm};yXSv;qT2 gvy\r��K?wc;!ȓ)Ah'~Est;MN{mk+WB3D8+!?C(R x^"k.yn٪[8"C 5|ʙ& eԌuAtqFBbbFC!Ly7+I}mNLM:~1Q 0s ',f9i~W%ЕE@z q`_sG<+� 8.p kv]O>QF#LUG[L.gs9I4HDUY&CJ;U"($9@W TAd&h;CV[?hU-kۀlHeߐ]; P<9ghy$MHywUiąYɫ r!'o [k@v,sZ\.Z2]xpOn ^}e-9ݦ*v&Y{mWx$3> Ɖc ټ4o7M~?%?x�M/| hn�بSygW”~"B\yƽǁY-]]k9F ARUS>f+q%AlGblģAxN_e@qQ+^:D)|&&FH 1&e6&G,%!w[)!�S~7Ł0o �ƒY+!+p%dԊ/It}\;1v LUcutCu,ΝR,AIM)BP3OAWG$AŴQ6w@0~7& mеz6MU q0]A|* qRtQ^A7yPj7ȥ<H}sĈG�g|OM+>vKp:93zMhHdٕR CFP'L8euRJ8!6طlܖQ֦Dq@5ψ R燠n&|j\$>٦`W�wB$&rv~|rU䔹;AjM[@ NހHH߽*>YiyI^rƀ.efսyCVkAL3 ƇHF)U@ ȝ< o3APET!e�XtAaHɛmg ޢ pθ{SXQD/Q\h@|>B& e:HZr;$JsrN9OO R!i}p&/y!1z+z"SBQ86 2geOCt"%D,'4%{K i%1%与;&/ g0CxN&)\ YD쵷{XϔŇV|YHJN#|W u*oz)fXvֳsλ |P gֺm1z "H |p~Nc$+bcEbAt4#6WP<=p}0A|( l<I ,6: Mg9y)L9IL& :7\VlC Slf)t~]A "^Nb# \5F6pe8 Ja!(:<HhH0hGafH]/.'g $77!zx/9 P t39~#6w;At-;p?oYd]:_ r:HK gUK�&(__97{ù7p>d؇@N_}{} ě 6&d9@!z||@_LJl0!5T03 \ 9`st9]V#sŏ`t;#x<Rc IKP @D\K@׀ܵ |<9xZJpyݾ Trtgba% \x.< "D/S)M4u`@? N`=άpYhStcSΔ \zmM ~S9x]M)l|, ^=@{1GqDWE7DAwhUUU>l]W?jḿ<p9e EHȧPo(t!2@gS`p?@17 sd&{FDCX$,,?k'o6Oc N* NiqWܔ0}A1=cc;fT!‡@HqpsLw5`>ׁ{ݻXHY{G|zǿ92'3TUZ=F'a\+Sbh fhL7V)W![Φ dpɁ"YqWhS38yM4ocFT~gZ "5|\{`1^N2Ns=N9f ~`706�JVb.8睶`y̟''e+\ {jjho3@ETj\mEA~ e\'"#_ ] Tg bm[@9 j,ZzgU50}LAs$rD[yVRϵAVyl n2T!wtz!&_@x)?@\' LAw) x's=^0DG rYԀH\[܏=SN]jwkm-ÏA)+Cnp/3ZVr dgU=Yd.j*9ꪸrxOMK T!gz(a5}Z &=h*TUJ{ oc rP | =v;⨨`pW?#2 e7)pR-T/wk= *pcsHr:7d;/un?}!/ èv8Kx@U|`o_;; A f xUÂ@փl-3@5wz9Ak"7GO  v`d;0o$,]rM@ =  [b:x"Ýb7|`_ }=r#8c S,'5r^rÜ:SQF=fAe6Htt:BDig+(>=H8A�8�ۂ9{ i B$Sȫ O}-PK݊ح4 *@~.rQ r]^7na+[yH<G*xxytCxQ)8%.ঞ <!Vĝ-}Ĝ>PɴA3d 7 TwIlPcːX&s!DFÞ�%*'!~lS w?Wp=A|CqXTnTȂ|yE tn@v;ղ; n;TsFml!/1A$`f=k6CL0 %5囲&r%/y`+1~ ~ ֑Q V%Z>A>tv- , 7/&"pGQP4߁3PFU@N8*fyZi 4t�.xϑ2)m'H Wn)HY@ֺąD&]":(�e};~*HYJ0K!.ːS>lTjapu_6!WuU lMI&+Ab+ƶy패y݆s{VWσxKvѧ\)N0d<WbQN >4.y[P-a _7y!φ`yVmSP LO|B @U A}`,;Ƚn#1.Rh!A[Vuz.H[#"WwA<>H w+r= wD/5%fn0!᷉"5q i|kYdf5 ˕C}ٰ/Bp'/ i prԓ;�+@4!8S [9 S N2 :%8f_}d`eȩ^u?'yR1@�UUO2'A8S[ogم>55di(<y3]l[*p 2�-A c<ыyX䅰l#ȂpY^NUR%@TtU5v `%5ߙ,"TL jcՠu&o]1xY e0u3AU  j�iYP2PU ŰfX  %pLO@59>$W"-НD{'*WpHg+J9^'ܽ?y^R Մ{t!X^_g`$:x Q𿔝*`ӣcBcgσbP"S\Di[|gtQ25JԄwTpFz87˲pO4ӀD( EUX(®6{f[) UOf8HۜҪ.r<qخ x#1"8ԷtSGsNfy^O OCe�i9#6 `W1ל!>/eERd]H۪Ե5!Aj0xόK.+8sw=zȾM냶SnNQ惝8�?|\ >GLYř'zrdH~2m/$)W~07\E 7LAv#ͅUbM4 C~m}8 /D:?-_17@#uAd#p{vnC;gv5RDI0ljee5o֞�yw;oB؛xAWm@ CsįfR%$;7@.xg#u~ښ JV^LRvO0#XWA|@֗9m!Xw/ (ag|pgU>\g@Ɲg2l}_Al^q D[*;g8IQC5y֥]w;Ӑ60G|ݝI"Άt~0ueN+g{ 7D[6ߛ.UlzYVj֫9Y* bpğv;;x&*ژ@pfx BRȶAp֚`Prr;Yq˃ʹ1[ȱ)<LE7ulSeOr w;L} 9c@ԏ�kl哏< ;E=]0Ns"pS{sfՏ<VwxbY$GΎh_ du j*C|  ,`+\;[o^�5k/Խ22ƝvEpSʞ bkwZr%~-Ȯx|˅<~Z:}!<$1"}D7?YdH{f3#Ph|ᤀ2x АR@]mYy gUxL歡`եw{z} P@S  |DpMIt1 dʉ%`Ϋb+1䗙,p�bq /'!)Gy_ F0+<HvJ+8 /NG5;C @?JAt5̒Hs7 5'Eb%@}nQ DEHlN�^ vDE֝XLH^hmfKrJf8$?;:v(V܍8qENjt!hYQ-{{Tv!?rK+&@u�g>g-L>)! oHWՁZ^R/_O7B,m LQYw } )^!A<5sP>i/'ol�zw&S*'o [JCng!o/rW@LԻ`."Ò>}PP;{2M u*) ߥ/////G%Q!E���%tEXtdate:create�2015-02-23T12:44:47-06:00N���%tEXtdate:modify�2015-02-23T12:44:47-06:00?S9����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/module.py�������������������������������������������������������������0000664�0000000�0000000�00000003372�13034501105�0020243�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import VicSecCard __all__ = ['VicSecCardModule'] class VicSecCardModule(Module, CapBank): NAME = 'vicseccard' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Victoria\'s Secret Angel Card' CONFIG = BackendConfig( ValueBackendPassword('username', label='User name', masked=False), ValueBackendPassword('password', label='Password')) BROWSER = VicSecCard def create_default_browser(self): return self.create_browser(self.config['username'].get(), self.config['password'].get()) def iter_accounts(self): return self.browser.iter_accounts() def get_account(self, id_): return self.browser.get_account(id_) def iter_history(self, account): return self.browser.iter_history(account) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vicseccard/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002202�13034501105�0017724�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from itertools import chain class VicSecCardTest(BackendTest): MODULE = 'vicseccard' def test_history(self): """ Test that there's at least one transaction in the whole history. """ b = self.backend ts = chain(*[b.iter_history(a) for a in b.iter_accounts()]) t = next(ts, None) self.assertNotEqual(t, None) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015410�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000000073�13034501105�0017521�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import VimeoModule __all__ = ['VimeoModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000010211�13034501105�0017440�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.browser.exceptions import HTTPNotFound from weboob.capabilities.base import NotAvailable from .pages import SearchPage, VideoPage, VideoJsonPage, CategoriesPage, ChannelsPage, ListPage, APIPage import urllib from urlparse import urljoin __all__ = ['VimeoBrowser'] class VimeoBrowser(PagesBrowser): BASEURL = 'https://vimeo.com' search_page = URL(r'search/page:(?P<page>.*)/sort:(?P<sortby>.*)/format:thumbnail\?type=videos&q=(?P<pattern>.*)', r'search/.*', SearchPage) list_page = URL(r'channels/(?P<channel>.*)/videos/.*?', r'categories/(?P<category>.*)/videos/.*?', ListPage) categories_page = URL('categories', CategoriesPage) channels_page = URL('channels', ChannelsPage) video_url = URL(r'https://player.vimeo.com/video/(?P<_id>.*)/config', VideoJsonPage) video_page = URL('https://vimeo.com/(?P<_id>.*)', VideoPage) api_page = URL('https://api.vimeo.com/search\?filter_mature=191&filter_type=clip&sort=featured&direction=desc&page=(?P<page>\d*)&per_page=20&sizes=590x332&_video_override=true&c=b&query=&filter_category=(?P<category>\w*)&fields=search_web%2Cmature_hidden_count&container_fields=parameters%2Ceffects%2Csearch_id%2Cstream_id%2Cmature_hidden_count', APIPage) def __init__(self, method, quality, *args, **kwargs): self.method = method self.quality = quality PagesBrowser.__init__(self, *args, **kwargs) def get_video(self, _id, video=None): try: video = self.video_page.go(_id=_id).get_video(video) video._quality = self.quality video._method = self.method video = self.video_url.open(_id=_id).fill_url(obj=video) if self.method == u'hls': streams = [] for item in self.read_url(video.url): if not item.startswith('#'): streams.append(item) if streams: streams.reverse() url = streams[self.quality] if self.quality < len(streams) else streams[0] if url.startswith('..'): video.url = urljoin(video.url, url) else: video.url = url else: video.url = NotAvailable return video except HTTPNotFound: return None def read_url(self, url): r = self.open(url, stream=True) buf = r.iter_lines() return buf def search_videos(self, pattern, sortby): return self.search_page.go(pattern=urllib.quote_plus(pattern.encode('utf-8')), sortby=sortby, page=1).iter_videos() def get_categories(self): return self.categories_page.go().iter_categories() def get_channels(self): return self.channels_page.go().iter_channels() def get_channel_videos(self, channel): return self.list_page.go(channel=channel).iter_videos() def get_category_videos(self, category): token = self.list_page.go(category=category).get_token() self.session.headers.update({"Authorization": "jwt %s" % token, "Accept": "application/vnd.vimeo.*+json;version=3.3"}) return self.api_page.go(page=1, category=category).iter_videos() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000003571�13034501105�0017551�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD�Q��X��� pHYs�� �� ����tIME ֺ���tEXtComment�Created with GIMPW��IDATxklSeގ:v7.:Ec $/~ ނ#~`DLD"KbBH0e!b(Q.[i{zzZҝntOɻ9<  ok!rd \@nLH1 ) b3~* EH1t/Xb}V0xh2Y}Ot껹Bj˧]Ƶ1�TaczY}׺}<QYxa &zN0h2^p_Hz 7.'tK4`ct}@{}*hnҐza.Yg2a��+Rqx|hԫPUIN: �z;jR` UwDj`kU/sw%Hf*R/+bxy{a,t ]a -�{аu-dhPUy^)�W8 `@Z0b)DK{A¨b]Pm$+y 'tc~}!ݘ6G�āԛ6Xx8R|Q-[�mH =e_OmeШW�'}nЫՌ4#0N x߇1龑[T/uG77K8Ⱥ5ixjN\'Y#hay��tɞ|g0vJJH q ]tysLV_."CBq0ܕ1J& mh Cppϱ0#R]@"9֋q..[SPRp @waD޵> r/�\89X^5-i57טv'qx%�K� _o]CI]o5c094z#A{A?o�F 6dph}e\^cq<^p9 .ņr=\!rޓx u:(x7M884"gm8ߟZDhh7buE {q$'-ANwL<eZW ZNܙ¢WźTQs^#DW;J4h񈚝ĩ^)YҴ IxS6Up Pb!UVN�iγQĚ}0㗁UY A~ ILFRP"Vs }FLYڬF'1�B0I˷Kke8EȻñxTkb ֈ׆R-ĵ; U�w&cGͥZ,;=Ò I#hMmZ#'DbYMF20v+J;MQ[D`D|IZ"R'R.m-d)H7v#\/yt:tX"_kҫ€b ih'b|bym4Ey&~6'Gk{LZhq'=,^,)�7\5R`VWLŎ+IpD\ł\)MMdke ScQaI=` �q?67U�bL/U¤Qb$Ň7pp]=>߲ Epg=Zl]:4SQ[r:8 <�bև*\7TT ����IENDB`���������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/favicon.xcf����������������������������������������������������������������0000664�0000000�0000000�00000014124�13034501105�0017541�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������gimp xcf file����@���@����������������BB�����������������������m��� gimp-comment�������Created with GIMP����gimp-image-grid�������(style solid) (fgcolor (color-rgba 0.000000 0.000000 0.000000 1.000000)) (bgcolor (color-rgba 1.000000 1.000000 1.000000 1.000000)) (xspacing 10.000000) (yspacing 10.000000) (spacing-unit inches) (xoffset 0.000000) (yoffset 0.000000) (offset-unit inches) ����gamma�������0.45454999804496765������������� �������������6��� ������i ...���������������������� �������������������� ���������� ���������� ���������� �������������������������������������������P���gimp-text-layer������4(text "i \nménon") (font "Serif Bold Italic") (font-size 14.000000) (font-size-unit pixels) (antialias yes) (language "fr-fr") (base-direction ltr) (color (color-rgb 1.000000 1.000000 1.000000)) (justify right) (box-mode fixed) (box-width 54.000000) (box-height 32.000000) (box-unit pixels) (hinting yes) ������������������6��� ������������6��� ���������1��1��/��0��0��1��1��0��0��0�����/��0��U�������������������������������������������������������������������������������������������������������1��1��/��0��0��1��1��0��0��0�����/��0��U�������������������������������������������������������������������������������������������������������1��1��/��0��0��1��1��0��0��0�����/��0��U������������������������������������������������������������������������������������������������������go�1��1�"�/�-5�0�_�0��1��1��0�[�0�@.�0�m��� �/� �0� �U�K/��2R��q��*�q���~l���84&�c�2({�c��C0����<<�C6�h ���C6�h ��p����@a�p��@b���p���̜���^�xʨ^�����x4�������� e�C1�y��7S�����q�N:�����u�39�o�4I�;�u�_�#2�u�_��"I�^���^Ű:��"I�>-�EY��"I�>-������7���@������v���������������������� �������������������� ���������� ���������� ���������� ����������������������������������������E���gimp-text-layer������)(text "v") (font "Serif Bold Italic") (font-size 50.000000) (font-size-unit pixels) (antialias yes) (language "fr-fr") (base-direction ltr) (color (color-rgb 1.000000 1.000000 1.000000)) (justify left) (box-mode fixed) (box-width 55.000000) (box-height 64.000000) (box-unit pixels) (hinting yes) ������������������7���@������������7���@�������������������� ���� ���� ������������������������������!����!����"����"����#���%��%��&� �'� �'��)��*��+����������������� ���� ���� ������������������������������!����!����"����"����#���%��%��&� �'� �'��)��*��+����������������� ���� ���� ������������������������������!����!����"����"����#���%��%��&� �'� �'��)��*��+����W1��RL��q��^����D��� �?��� ���E� ���`z��5V��+�� ��������)��N����X��0��$��1�!����!�#��|�"�X��I�"�Q��"7�#�� �%�  �%�0�&�~ p�'�I �'� �)��*�'�+�v)�����@���@������ favicon.png����������������������������� �������������������� ���������� ���������� ���������� ���������������������������������������������������������@���@������������@���@������ A)QA-QA1Q A3Q A5QA7QA9QA;QA;QA=QA=QAQ QAQ<QA=QA;QA;QA9QA7QA5Q A3Q A1QA-QA)Q A )-1 3 579;;== <=;;975 3 1-) )-1 3 579;;== <=;;975 3 1-) �)�-�1 �3 �5�7�9�;�;�=�=� �<�=�;�;�9�7�5 �3 �1�-�) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/module.py������������������������������������������������������������������0000664�0000000�0000000�00000010414�13034501105�0017247�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound, Collection from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from .browser import VimeoBrowser import re __all__ = ['VimeoModule'] class VimeoModule(Module, CapVideo, CapCollection): NAME = 'vimeo' MAINTAINER = u'François Revol' EMAIL = 'revol@free.fr' VERSION = '1.2' DESCRIPTION = 'Vimeo video streaming website' LICENSE = 'AGPLv3+' BROWSER = VimeoBrowser SORTBY = ['relevance', 'rating', 'views', 'time'] quality_choice = OrderedDict([(k, v) for k, v in sorted( {u'0': u'hight', u'1': u'medium', u'2': u'low'}.iteritems())]) method_choice = [u'hls', u'progressive'] CONFIG = BackendConfig(Value('method', label='Choose a stream method', choices=method_choice), Value('quality', label='Choosen a quality', choices=quality_choice)) def create_default_browser(self): return self.create_browser(method=self.config['method'].get(), quality=int(self.config['quality'].get())) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): return self.browser.search_videos(pattern, self.SORTBY[sortby]) def get_video(self, _id): return self.browser.get_video(self.parse_id(_id)) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def parse_id(self, _id): m = re.match('https?://vimeo.com/(.*)', _id) if m: return m.group(1) return _id def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield Collection([u'vimeo-categories'], u'Vimeo categories') yield Collection([u'vimeo-channels'], u'Vimeo channels') if collection.path_level == 1: if collection.split_path == [u'vimeo-categories']: for category in self.browser.get_categories(): yield category if collection.split_path == [u'vimeo-channels']: for channel in self.browser.get_channels(): yield channel if collection.path_level == 2: if collection.split_path[0] == u'vimeo-channels': for video in self.browser.get_channel_videos(collection.split_path[1]): yield video if collection.split_path[0] == u'vimeo-categories': for video in self.browser.get_category_videos(collection.split_path[1]): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and (collection.split_path[0] == u'vimeo-categories' or collection.split_path[0] == u'vimeo-channels'): return raise CollectionNotFound(collection.split_path) OBJECTS = {BaseVideo: fill_video} ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/pages.py�������������������������������������������������������������������0000664�0000000�0000000�00000015614�13034501105�0017070�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo from weboob.capabilities.image import Thumbnail from weboob.capabilities.collection import Collection from weboob.exceptions import ParseError from weboob.browser.elements import ItemElement, ListElement, method, DictElement from weboob.browser.pages import HTMLPage, pagination, JsonPage from weboob.browser.filters.standard import Regexp, Env, CleanText, DateTime, Duration, Field, BrowserURL from weboob.browser.filters.html import Attr, Link, CleanHTML, XPath from weboob.browser.filters.json import Dict import re class VimeoDuration(Duration): _regexp = re.compile(r'PT(?P<hh>\d+)H(?P<mm>\d+)M(?P<ss>\d+)S') class ListPage(HTMLPage): def get_token(self): return Regexp(CleanText('//script'), '"jwt":"(.*)","url"', default=None)(self.doc) @pagination @method class iter_videos(ListElement): item_xpath = '//div[@id="browse_content"]/ol/li' next_page = Link(u'//a[text()="Next"]') class item(ItemElement): klass = BaseVideo obj_id = Regexp(Attr('.', 'id'), 'clip_(.*)') obj_title = Attr('./a', 'title') def obj_thumbnail(self): thumbnail = Thumbnail(self.xpath('./a/img')[0].attrib['src']) thumbnail.url = thumbnail.id return thumbnail class APIPage(JsonPage): @pagination @method class iter_videos(DictElement): item_xpath = 'data' def parse(self, el): self.env['next_page'] = Regexp(Dict('paging/next'), 'page=(\d*)', default=None)(el) def next_page(self): if Env('next_page')(self) is not None: return BrowserURL('api_page', page=int(Env('next_page')(self)), category=Env('category'))(self) class item(ItemElement): klass = BaseVideo obj_id = Regexp(Dict('clip/uri'), '/videos/(.*)') obj_title = Dict('clip/name') def obj_thumbnail(self): thumbnail = Thumbnail(Dict('clip/pictures/sizes/0/link')(self)) thumbnail.url = thumbnail.id return thumbnail class SearchPage(HTMLPage): @pagination @method class iter_videos(ListElement): item_xpath = '//ul[@class="small-block-grid-3"]/li/div[has-class("clip_thumbnail")]' ignore_duplicate = True next_page = Link(u'//a[text()="Next"]') class item(ItemElement): klass = BaseVideo obj_id = Attr('.', 'data-clip-id') obj_title = Attr('./a/span', 'title') def obj_thumbnail(self): thumbnail = Thumbnail(self.xpath('./a/div/img')[0].attrib['src']) thumbnail.url = thumbnail.id return thumbnail class VideoPage(HTMLPage): def __init__(self, *args, **kwargs): super(VideoPage, self).__init__(*args, **kwargs) from weboob.tools.json import json jsoncontent = XPath('//script[@type="application/ld+json"]/text()')(self.doc)[0] self.doc = json.loads(jsoncontent)[0] @method class get_video(ItemElement): klass = BaseVideo obj_id = Env('_id') obj_title = CleanText(CleanHTML(Dict('name'))) obj_description = CleanHTML(Dict('description')) obj_date = DateTime(Dict('uploadDate')) obj_duration = VimeoDuration(Dict('duration')) obj_author = CleanText(Dict('author/name')) def obj_nsfw(self): _sfw = Dict('isFamilyFriendly', default="True")(self) return _sfw != "True" def obj_thumbnail(self): thumbnail = Thumbnail(Dict('thumbnailUrl')(self.el)) thumbnail.url = thumbnail.id return thumbnail class VideoJsonPage(JsonPage): @method class fill_url(ItemElement): klass = BaseVideo def obj_url(self): data = self.el if not data['request']['files']: raise ParseError('Unable to detect any stream method for id: %r (available: %s)' % (int(Field('id')(self)), data['request']['files'].keys())) # Choosen method is not available, we choose an other one method = self.obj._method if method not in data['request']['files']: method = data['request']['files'].keys()[0] streams = data['request']['files'][method] if not streams: raise ValueError('There is no url available for id: %r' % (int(Field('id')(self)))) # stream is single for hls, just return the url stream = streams['url'] if method == 'hls' else None # ...but a list for progressive # we assume the list is sorted by quality with best first if not stream: quality = self.obj._quality stream = streams[quality]['url'] if quality < len(streams) else streams[0]['url'] return stream obj_ext = Regexp(Field('url'), '.*\.(.*)\?.*$', '\\1') class CategoriesPage(HTMLPage): @method class iter_categories(ListElement): item_xpath = '//div[@class="category_grid"]/div/a' class item(ItemElement): klass = Collection obj_id = CleanText('./@href') obj_title = CleanText('./div/div/p') def obj_split_path(self): split_path = ['vimeo-categories'] category = CleanText('./@href', replace=[('/categories/', '')])(self) split_path.append(category) return split_path class ChannelsPage(HTMLPage): @pagination @method class iter_channels(ListElement): item_xpath = '//div[@id="browse_content"]/ol/li' next_page = Link('//li[@class="pagination_next"]/a') class item(ItemElement): klass = Collection obj_title = CleanText('div/a/div/p[@class="title"]') obj_id = CleanText('./@id') def obj_split_path(self): split_path = ['vimeo-channels'] channel = CleanText('div/a/@href', replace=[('/channels/', '')])(self) split_path.append(channel) return split_path ��������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vimeo/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000004261�13034501105�0016744�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # Copyright(C) 2012 François Revol # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo from weboob.tools.test import BackendTest import itertools class VimeoTest(BackendTest): MODULE = 'vimeo' def test_search(self): l = list(itertools.islice(self.backend.search_videos('boobs'), 0, 20)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('https://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_channels(self): l = list(itertools.islice(self.backend.iter_resources([BaseVideo], [u'vimeo-channels']), 0, 20)) self.assertTrue(len(l) > 0) l1 = list(itertools.islice(self.backend.iter_resources([BaseVideo], l[0].split_path), 0, 20)) self.assertTrue(len(l1) > 0) v = l1[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('https://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) def test_categories(self): l = list(itertools.islice(self.backend.iter_resources([BaseVideo], [u'vimeo-categories']), 0, 20)) self.assertTrue(len(l) > 0) l1 = list(itertools.islice(self.backend.iter_resources([BaseVideo], l[0].split_path), 0, 20)) self.assertTrue(len(l1) > 0) v = l1[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('https://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015232�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/__init__.py�����������������������������������������������������������������0000664�0000000�0000000�00000001424�13034501105�0017344�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 P4ncake # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VineModule __all__ = ['VineModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/browser.py������������������������������������������������������������������0000664�0000000�0000000�00000002357�13034501105�0017276�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 P4ncake # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import SearchPage, PostPage import urllib class VineBrowser(PagesBrowser): BASEURL = 'https://vine.co' search_page = URL(r'/api/posts/search/(?P<pattern>.*)',SearchPage) post_page = URL('r/api/timelines/posts/s/(?P<_id>.*)', PostPage) def search_videos(self, pattern): return self.search_page.go(pattern=urllib.quote_plus(pattern.encode('utf-8'))).iter_videos() def get_video(self, _id): return self.post_page.go(_id=_id).get_video() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/favicon.png�����������������������������������������������������������������0000664�0000000�0000000�00000006650�13034501105�0017374�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���tEXtSoftware�Adobe ImageReadyqe<��"iTXtXML:com.adobe.xmp�����<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.3-c011 66.145661, 2012/02/06-14:56:27 "> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/" xmlns:stRef="http://ns.adobe.com/xap/1.0/sType/ResourceRef#" xmp:CreatorTool="Adobe Photoshop CS6 (Windows)" xmpMM:InstanceID="xmp.iid:3628AE485B6A11E3878FFC9BDA611944" xmpMM:DocumentID="xmp.did:3628AE495B6A11E3878FFC9BDA611944"> <xmpMM:DerivedFrom stRef:instanceID="xmp.iid:3628AE465B6A11E3878FFC9BDA611944" stRef:documentID="xmp.did:3628AE475B6A11E3878FFC9BDA611944"/> </rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end="r"?>*��� IDATx[{TW;wvfw]! Pti  hR5mDV(K&Dkbh+Z yF..]sfuw9w>f6'9ǽs=va[ؑ�8)D:EIVr!#Ȧ%xR%3 x -]M~L3'f֘E3m,8  FC4g+1`fp@8M!ff8Ālfdwqm{WB0�wZu0`!Ck-`1nj@.Zm-XC1s/^:k2$; !Gr2 A U҆_|aPA2eYXhDO/@eRx[ 0r8-@KQ\y] yt2/-w*٢IMjh;I4;jQeճT Vlۗ( tZs ǥ'�mrf?]37_g`*\rTt>fKHjM{u5T|~Z#ͭot0}{v[ucZfKh4�Jg�ie47CZxKyZC/} IEx`{:DoF'Π( afm@Z Ƭ ٮ^ 7l&6r"/5 _gpb dpƂw,f0 =n;?\,M*hr>>J311�Ty->.Ф@"婹.r�Wk׍յ> 9I9ǹ>g(P.\@<YW?Pr?]D pKI7c!CV_Gr9A%s}>?w�yw)f_y8͎y=>D9"8H-#8 y}>@+Y+nޛ΍KMITT~f|sqf/;w16}%1H2$MjيILH8% 0ن#pO�p*r4}1Q<׾֥zk}vmx?:!9y5='^,E|7̤sa:Y:|oi<x kAxZvglb %uJz#KW^=0'_H!#?O`NJ}~] e`42rα5 dwV\}Cw_ {L$P-6mbL?>�+( 8r zC=: `'ƴ&N a[x0R>3'?g LUgH:N@<tf͖ -Ruo}|k.+C1 P`NC96\";�R P0)d9) [֗p 7Q`MZgas9FG̀R9t||Ծz]\ڥʐUX9g= :!RaPuD�W;`  ]qc$o0jP5Ūƅe$ᱭ\9% $Hlgϖ"(z +흇|~뿏U52^SV݀k#gy8~}-1hS!JlȳB>@)c4eK\f`M\}.X0oAkmzmV4 }ĥ�3pM}Q6o~@k7FRrx~\=] j걇PS[OEj>tdc(h=wHm nrdBpGQ 7$#/7m=]xCٱ^?H(HUlDݳ1ǮG"SgH16 8VKPf(#mGF-43 Ky|ST(x+Bp<-1QV(|BRE.YM(_EwEu׶]Ӱ".mr^Lgfᙏ3ޔ!{ ʁ)t#r>{=x(.LBb;<b9wS\C!E*D}"(rwN dB\>sX>ۓ Q<`,�fFe!<OQa#U{k󅚢(E.د[Ye[iKgXV#/-($��9 eXL_5x! zmGs&޴1|)di?NLI{JˇV͑mJ-.,cݝ&dq2ìƙ7YHYvqp[BuXCu/WhDe?Hi)QK @kSkMYI_�}o1����IENDB`����������������������������������������������������������������������������������������weboob-1.2/modules/vine/module.py�������������������������������������������������������������������0000664�0000000�0000000�00000003044�13034501105�0017072�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 P4ncake # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module from weboob.capabilities.video import CapVideo from .browser import VineBrowser __all__ = ['VineModule'] class VineModule(Module, CapVideo): NAME = 'vine' DESCRIPTION = u'vine website' MAINTAINER = u'P4ncake' EMAIL = 'me@p4ncake.fr' LICENSE = 'AGPLv3+' VERSION = '1.2' BROWSER = VineBrowser def search_videos(self, pattern, nsfw=False): return self.browser.search_videos(pattern) def get_video(self, _id): return self.browser.get_video(_id=_id) def fill_video(self, video, fields): if fields != ['thumbnail']: video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/pages.py��������������������������������������������������������������������0000664�0000000�0000000�00000003061�13034501105�0016703�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 P4ncake # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo from weboob.browser.elements import ItemElement, DictElement, method from weboob.browser.pages import JsonPage from weboob.browser.filters.standard import Regexp from weboob.browser.filters.json import Dict class SearchPage(JsonPage): @method class iter_videos(DictElement): item_xpath ='data/records' class item(ItemElement): klass = BaseVideo obj_id = Regexp(Dict('shareUrl'), '/([a-zA-Z0-9]*)$') obj_title = Dict('description') obj_url = Dict('videoUrl') obj_ext = Regexp(Dict('videoUrl'), r'.*\.(.*?)\?.*') obj_author = Dict('username') class PostPage(JsonPage): @method class get_video(ItemElement): klass = BaseVideo obj_id = Dict('postId') obj_url = Dict('videoUrl') �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vine/test.py���������������������������������������������������������������������0000664�0000000�0000000�00000002206�13034501105�0016563�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2015 P4ncake # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest import itertools class VineTest(BackendTest): MODULE = 'vine' def test_search(self): l = list(itertools.islice(self.backend.search_videos('coucou'), 0, 20)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/virginradio/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016606�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/virginradio/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001442�13034501105�0020720�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Johann Broudin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VirginRadioModule __all__ = ['VirginRadioModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/virginradio/favicon.licence������������������������������������������������������0000664�0000000�0000000�00000002006�13034501105�0021555�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������favicon.png is licenced under the Créative Commons Attribution-Share Alike 4.0 International licence Original picture attribution: © Hans Hillewaert You are free to: Share — copy and redistribute the material in any medium or format Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms. Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/virginradio/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000011625�13034501105�0020746�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������C��� pHYs�� �� ����tIME ��IDATx[y|T޷̾f2IBX" QDۂOkتRZZ7jբUk?b*u-Y  !$d&ٗ7f{IX$w߽|Ϲg{3hdcw<q3q#~v%;es�oeuw0U/Z%,:;0 Z{_"@:`pB0߼hԯ �}3o�H~p-`DN 5�9 M@:_�0I$, lsE�%f)�(t\;`F%�>F'�`+= ":*Fݼ^vY}LD24g暪DBDZƙGn] -;Fr-ԩ8KD AlK \I]O~3 qj&"R{!Y`�:}'Ui-WKr+?�9q4z�%\CPxԽ0O,ܢc�錱йI֕e.RO�$2#~�g􍕞vHoZC Pwl19svdɌ#|ӽϘ:`~@��MQn%|H@i\4@oL[L�&Jc1Ҵ-F@:@AZY@ソ@߽buc'/U,&%x\"F|l�e�~ `gO<> r@zs�n�//cl�K2ơ ̄0lp&Au"."zQ;�g{J<`e* h+h{Xg] `deI-emPly =4=ןg"d7IF<h˨zO\yhu0ȖT:x+anX}uc8�CnGiI7 mW!xo&M;q\,( 5 JApYn?άqYc1"&XNDc/ +�_1x0֝fvA!B*. 0߭< c`G& f0A�m-{p빽u]`� 9O"-W{ti^I�.GNOȇ.'Z81E�5fEˏyKѧ}� k+٠p#ԎC M7z}�P6c|UwWɱU/vJR8c|zP2 ^sC0;uNσde^!(Ml3\tLim#I802(&"{ۯ햽�e(J>`vU%س-YrɂRx�X2O1bܹb(#܌ K?q1 @U�DlzfHiԆ#R hQ?4v4`=:o}cAo 2�Xy͡rB[L7 NW2#}y.EU[={Plzssz*]/sYׁ/(j]}䔊,Y:dx3G>|6 hppoߑdu �n��ʌCFTB\_8o1AܹӝWzdPU!r ]uiI-|kj瞵Mu1s3�A'dBUr8 i!qx/8G. f7Ɲ4 tK�<�^`+�?�XwEEDcy+ 0qOZ'?<Giɩ9*G,)iELuW6 rN?JDiG:g d<:a{&((L)q�ͼ_%lás~ZX _Ҧ-+`rU�LuCzQaq0) x>/ t @ t& sP}B!a*?E(c?D>X9m>kފS._"tW܁Цa5Jx `Іbk��;|=^d#Ěas w9%ڑ-׬<Qi9qx+?3~[j:59q:oHǔ#h&@ aWm:j, d?79(7vA`( CEY߄Zyw9Ls`e#u'e'RїE]>PF޿ 5;1(;mCo L'G?P ުGD#)!\4?0{|ľ х� e5rO]Muo0úbpQCjA@"2�Ɣ.nD `+.gq )֕)A.�sO^ˎq`t &7谱D,1J@Uieڹ*j1b1v0� AڄTS%YF��' J'\H �b,)R  PvVKR fJz=Z:e#FîHB0Q8o&X scP+0L=|$թHSbTCZ L򂒻ݼ1/CN>']}=,e @*H�׿A`˲�4 A[AG]dˁ�0$ C<Q˧[f.Qc=[8AɩDŽV9BӻFZZ!2㋳"(JKJT~�A_}ܩ{J h0?;Ť?mOSq_y95%ƃ0IJ*TDRIC"rQК$eS ӄgv3s{>a👫7L-/޸Ӿ6xִz^IaQsw冞a]Ru5)Ʀmu+4l>sJ{ `mXwi:Y'Z6>pME  gI�,Ǵڮ((f`JJ1ztp;wℊ438j"-c]yz / xw=Ctjm*G2~6D " /z1[IHq!)0*w#wdܶ&0=ňב;ҽe#@S��C�^E!ykXIB$ I 'vg|kϛ{X~(QDO0Dhnm1RItc`/3N`CK+eko~co#;qPl5AHIH+IvҞල vƵ% i)B.Nqx2x&<QVjC:懎7rh b+|W# 7}Y$!X@0e8r:J! .۷߯~XqN`rZx[a60LT -]͙Ez8/rF %O J'vE"DR<�A=yf>�lgl=25ufb ֖6x<ȱl1o_zg#�=>�9V%%$hYF= d- S1rmw) hl: hE';I�h v=p5#�m(i9g"al&~ hz> s_JRqM85?BKc+XT$@kSѫ'<k-q$c}k޸脇ݼ|}$`njn"`>Ek&'0/YJE&O(rS`E 3̬KA鵏=3@f1N͗4dx&kDP;^+\?._鍳栌B�AFQX]?r �N�{ cܱPX0d: B,FP߼v]ю5;SW!N0ÿ�L3SN5s6׮s �cǴmvγ,x4iwfcng>ls % .މZO2pO+GMqs挳R"R&AdO='2sc JM>"w>l�O P"&?Qn$~;_ȷ0yOď�sKrU<+ϖ 0xp' ː Cup?*͢ (HB��skM(Qur3A)"R^?C[-\ۛ-rrKJ 0 (o6dϬ:j#>*{cXs)6uè��<wDMлlЕ/�~hZ?sw/iGW{OX-ʩr$@B*`Ae&k\qo=coʎ81k)@CSd;&/<M6ɒQZ4q(MbKNZ@ݲWk,B4(,f*&wS*#gku>×^3@0%8q;'K@G8hב-M\n-CCD8B����IENDB`�����������������������������������������������������������������������������������������������������������weboob-1.2/modules/virginradio/module.py������������������������������������������������������������0000664�0000000�0000000�00000010663�13034501105�0020453�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Johann Broudin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audiostream import BaseAudioStream from weboob.tools.capabilities.streaminfo import StreamInfo from weboob.capabilities.collection import CapCollection from weboob.tools.backend import Module from weboob.browser import Browser from weboob.tools.misc import to_unicode __all__ = ['VirginRadioModule'] class VirginRadioModule(Module, CapRadio, CapCollection): NAME = 'virginradio' MAINTAINER = u'Johann Broudin' EMAIL = 'Johann.Broudin@6-8.fr' VERSION = '1.2' DESCRIPTION = u'VirginRadio french radio' LICENSE = 'AGPLv3+' BROWSER = Browser _RADIOS = { 'officiel': ( u'Virgin Radio', u'Virgin Radio', u'http://mp3lg3.scdn.arkena.com/10490/virginradio.mp3', 64), 'new': ( u'Virgin Radio New', u'Virgin Radio New', u'http://mp3lg3.tdf-cdn.com/9145/lag_103228.mp3', 64), 'classics': ( u'Virgin Radio Classics', u'Virgin Radio Classics', u'http://mp3lg3.tdf-cdn.com/9146/lag_103325.mp3', 64), 'electroshock': ( u'Virgin Radio Electroshock', u'Virgin Radio Electroshock', u'http://mp3lg3.tdf-cdn.com/9148/lag_103401.mp3', 64), 'hits': ( u'Virgin Radio Hits', u'Virgin Radio Hits', u'http://mp3lg3.tdf-cdn.com/9150/lag_103440.mp3', 64), 'rock': ( u'Virgin Radio Rock', u'Virgin Radio Rock', u'http://mp3lg3.scdn.arkena.com/9151/lag_103523.mp3', 64) } def get_stream_info(self, radio, url): stream = BaseAudioStream(0) current = StreamInfo(0) r = self.browser.open(url, stream=True, headers={'Icy-Metadata':1}) stream.bitrate = int(r.headers['icy-br'].split(',')[0]) r.raw.read(int(r.headers['icy-metaint'])) size = ord(r.raw.read(1)) content = r.raw.read(size*16) r.close() for s in content.split("\x00")[0].split(";"): a = s.split("=") if a[0] == "StreamTitle": stream.title = to_unicode(a[1].split("'")[1]) res = stream.title.split(" - ") current.who = to_unicode(res[0]) if(len(res) == 1): current.what = "" else: current.what = to_unicode(res[1]) stream.format=u'mp3' stream.url = url return [stream], current def get_radio(self, radio): if not isinstance(radio, Radio): radio = Radio(radio) if radio.id not in self._RADIOS: return None title, description, url, bitrate = self._RADIOS[radio.id] radio.title = title radio.description = description radio.streams, radio.current = self.get_stream_info(radio.id, url) return radio def iter_resources(self, objs, split_path): if Radio in objs: self._restrict_level(split_path) for id in self._RADIOS.iterkeys(): yield self.get_radio(id) def iter_radios_search(self, pattern): for radio in self.iter_resources((Radio, ), []): if pattern.lower() in radio.title.lower() or pattern.lower() in radio.description.lower(): yield radio def fill_radio(self, radio, fields): if 'current' in fields: if not radio.current: radio = self.get_radio(radio.id) return radio OBJECTS = {Radio: fill_radio} �����������������������������������������������������������������������������weboob-1.2/modules/virginradio/test.py��������������������������������������������������������������0000664�0000000�0000000�00000001765�13034501105�0020150�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Johann Broudin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.radio import Radio class VirginRadioTest(BackendTest): MODULE = 'virginradio' def test_virginradio(self): l = list(self.backend.iter_resources((Radio, ), [])) self.assertTrue(len(l) > 0) �����������weboob-1.2/modules/vlille/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015560�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001432�13034501105�0017671�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VlilleModule __all__ = ['VlilleModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000002436�13034501105�0017622�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import ListStationsPage, InfoStationPage __all__ = ['VlilleBrowser'] class VlilleBrowser(PagesBrowser): BASEURL = 'http://www.vlille.fr' list_page = URL('/stations/les-stations-vlille.aspx', ListStationsPage) info_page = URL('/stations/xml-station.aspx\?borne=(?P<idgauge>.*)', InfoStationPage) def get_station_list(self): return self.list_page.go().get_station_list() def get_station_infos(self, gauge): return self.info_page.go(idgauge=gauge.id).get_station_infos(obj=gauge) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000001031�13034501105�0017706�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME#��IDATxZ }bY \AJ !9v|l>�WDGf1WYZ�@ @R~7 ^RyL^RylLi@MiKDu #|R3߿)޺ @yx>@^t,cJAg_!4{}KpE:G pj1rJDgAcbBn9=k{8Ke.uJcCFw[!lYC1�Kl6\|Rԩd7j8Od�L~@X'� @�l;~R喷%֍ƀ O*Z}!K<W+$l͐AQ ZuOsh6L5C(rںJ��0~$Jgb����IENDB`�������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000005601�13034501105�0017421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.tools.backend import Module from weboob.capabilities.base import find_object from weboob.capabilities.gauge import CapGauge, GaugeSensor, Gauge, SensorNotFound from .browser import VlilleBrowser __all__ = ['VlilleModule'] class VlilleModule(Module, CapGauge): NAME = 'vlille' DESCRIPTION = u'Lille bike renting availability information' MAINTAINER = u'Bezleputh' EMAIL = 'carton_ben@yahoo.fr' VERSION = '1.2' BROWSER = VlilleBrowser def iter_gauges(self, pattern=None): if pattern is None: for gauge in self.browser.get_station_list(): yield gauge else: lowpattern = pattern.lower() for gauge in self.browser.get_station_list(): if lowpattern in gauge.name.lower(): yield gauge def iter_sensors(self, gauge, pattern=None): if not isinstance(gauge, Gauge): gauge = find_object(self.iter_gauges(), id=gauge, error=SensorNotFound) gauge = self.browser.get_station_infos(gauge).next() if pattern is None: for sensor in gauge.sensors: yield sensor else: lowpattern = pattern.lower() for sensor in gauge.sensors: if lowpattern in sensor.name.lower(): yield sensor def get_last_measure(self, sensor): if not isinstance(sensor, GaugeSensor): sensor = self._get_sensor_by_id(sensor) if sensor is None: raise SensorNotFound() return sensor.lastvalue """ def _get_gauge_by_id(self, id): for gauge in self.browser.get_station_list(): if id == gauge.id: return gauge return None """ def _get_sensor_by_id(self, id): reSensorId = re.search('(\d+)-((bikes|attach|status))', id, re.IGNORECASE) if reSensorId: gauge = reSensorId.group(1) pattern = reSensorId.group(2) sensor_generator = self.iter_sensors(gauge, pattern) if sensor_generator: return next(sensor_generator) else: return None return None �������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/pages.py������������������������������������������������������������������0000664�0000000�0000000�00000012602�13034501105�0017232�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, XMLPage from weboob.browser.elements import ListElement, ItemElement, TableElement, method from weboob.browser.filters.standard import CleanText, TableCell, Filter from weboob.capabilities.gauge import Gauge, GaugeMeasure, GaugeSensor from weboob.capabilities.base import NotLoaded import datetime import re class LastDateFilter(Filter): def filter(self, last_update): return datetime.datetime.now() - datetime.timedelta(seconds=int(re.match(r'\d+', last_update).group(0))) class InfoStationPage(XMLPage): ENCODING = 'utf-8' @method class get_station_infos(ListElement): item_xpath = "." class item(ItemElement): klass = Gauge def _create_bikes_sensor(self, value, gauge_id, last_update, adresse): levelbikes = GaugeSensor(gauge_id + '-bikes') levelbikes.name = u'Bikes' levelbikes.address = u'%s' % adresse lastvalue = GaugeMeasure() lastvalue.level = float(value) lastvalue.date = last_update if lastvalue.level < 1: lastvalue.alarm = u'Empty station' levelbikes.lastvalue = lastvalue levelbikes.history = NotLoaded levelbikes.gaugeid = gauge_id return levelbikes def _create_attach_sensor(self, value, gauge_id, last_update, adresse): levelattach = GaugeSensor(gauge_id + '-attach') levelattach.name = u'Attach' levelattach.address = u'%s' % adresse lastvalue = GaugeMeasure() if lastvalue.level < 1: lastvalue.alarm = u'Full station' lastvalue.level = float(value) lastvalue.date = last_update levelattach.lastvalue = lastvalue levelattach.history = NotLoaded levelattach.gaugeid = gauge_id return levelattach def _create_status_sensor(self, value, gauge_id, last_update, adresse): levelstatus = GaugeSensor(gauge_id + '-status') levelstatus.name = u'Status' levelstatus.address = u'%s' % adresse lastvalue = GaugeMeasure() status = float(value) if status == 0: status = 1 else: status = -1 if lastvalue.level < 1: lastvalue.alarm = u'Not available station' lastvalue.level = float(status) lastvalue.date = last_update levelstatus.lastvalue = lastvalue levelstatus.history = NotLoaded levelstatus.gaugeid = gauge_id return levelstatus def parse(self, el): self.obj = self.env['obj'] def obj_sensors(self): sensors = [] last_update = LastDateFilter(CleanText('lastupd'))(self) adresse = CleanText('adress')(self) sensors.append(self._create_bikes_sensor(CleanText('bikes')(self), self.env['idgauge'], last_update, adresse)) sensors.append(self._create_attach_sensor(CleanText('attachs')(self), self.env['idgauge'], last_update, adresse)) sensors.append(self._create_status_sensor(CleanText('status')(self), self.env['idgauge'], last_update, adresse)) return sensors class ListStationsPage(HTMLPage): @method class get_station_list(TableElement): item_xpath = "//table[@id='ctl00_Contenu_ListeStations1_ListViewStations_itemPlaceholderContainer']/tr" head_xpath = "//table[@id='ctl00_Contenu_ListeStations1_ListViewStations_itemPlaceholderContainer']/tr/th/@id" col_id = 'ctl00_Contenu_ListeStations1_ListViewStations_Th1' col_name = 'ctl00_Contenu_ListeStations1_ListViewStations_Th2' col_city = 'ctl00_Contenu_ListeStations1_ListViewStations_Th9' class item(ItemElement): klass = Gauge condition = lambda self: (len(self.el.xpath('td/span')) > 4 and not ('id' in self.el.attrib)) obj_id = CleanText(TableCell('id')) obj_name = CleanText(TableCell('name')) obj_city = CleanText(TableCell('city')) obj_object = u'vLille' ������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/vlille/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000002173�13034501105�0017114�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class VlilleTest(BackendTest): MODULE = 'vlille' def test_vlille(self): l = list(self.backend.iter_gauges()) self.assertTrue(len(l) > 0) gauge = l[0] s = list(self.backend.iter_sensors(gauge)) self.assertTrue(len(s) > 0) sensor = s[0] self.assertTrue(self.backend.get_last_measure(sensor.id) is not None) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016620�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000001450�13034501105�0020731�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import VoyagesSNCFModule __all__ = ['VoyagesSNCFModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/browser.py�����������������������������������������������������������0000664�0000000�0000000�00000004242�13034501105�0020657�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from random import randint from weboob.deprecated.browser import Browser from .pages import CitiesPage, SearchPage, SearchErrorPage, \ SearchInProgressPage, ResultsPage, ForeignPage __all__ = ['VoyagesSNCFBrowser'] class VoyagesSNCFBrowser(Browser): PROTOCOL = 'http' DOMAIN = 'www.voyages-sncf.com' ENCODING = 'utf-8' PAGES = { 'http://www.voyages-sncf.com/completion/VSC/FR/fr/cityList.js': (CitiesPage, 'raw'), 'http://www.voyages-sncf.com/billet-train': SearchPage, 'http://www.voyages-sncf.com/billet-train\?.+': SearchErrorPage, 'http://www.voyages-sncf.com/billet-train/recherche-en-cours.*': SearchInProgressPage, 'http://www.voyages-sncf.com/billet-train/resultat.*': ResultsPage, 'http://(?P<country>\w{2})\.voyages-sncf.com/\w{2}/.*': ForeignPage, } def __init__(self, *args, **kwargs): Browser.__init__(self, *args, **kwargs) self.addheaders += (('X-Forwarded-For', '82.228.147.%s' % randint(1,254)),) def get_stations(self): self.location('/completion/VSC/FR/fr/cityList.js') return self.page.get_stations() def iter_departures(self, departure, arrival, date, age, card, comfort_class): self.location('/billet-train') self.page.search(departure, arrival, date, age, card, comfort_class) return self.page.iter_results() ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/favicon.png����������������������������������������������������������0000664�0000000�0000000�00000012346�13034501105�0020761�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������ pHYs�� �� ����tIME ,M���tEXtComment�Created with GIMPW��NIDATx{eu׿k9{yy?j'ı؉ *MP(⡀DQQ@ DA!"PZZDEUQ$hIΌ]>oĐĉ=Lɒ>wcc,t+Cg.ߧ�Џ>|Tx3 (�x�n�Ľk~#t+omk�ĺFT8D&* ` |q(LGTE"(4szd>#\p @:Tt1KpD{ B$RP` �A1 4W^8](LޏDeȴB&JP�IUtm-s=~/6\�18Pz�I)Hdl ]F+ec#ʃ~"w$ɣIxs O kbR=VUC$dF"IEI�q/T/{s1OB^Y3CJ BD䡘W�se/G�c."[e\'݇~=YeW],@(AXHhm\Wk~?iG5{^%aeM 7vˀTnm?+q]�_&o ~rSP�+ w!)rH" URQ,i[6ыey<{2Ni9Z%% F tXVmR9IKVZC VRYLKpABU{ VwXff + @6e(tP6%nKሺ<@ΔDUg,+,)eH4d31gYcgl&oɹyͽbiX?ݬCX[JӀ Ka@5`2$mKk6I=tp\Ǘ1ixaXYFf:)h @A+PK �@I.x. h@ Zlk@G*Q>8.#<@$q!F (  pD>tпZfZ=׽&�G#!, c:&LRBחv, n'I>hvzk@�?#)gX,p7dyY\T΢*#-i`/D ql U(S:ճ<RM{\x-oһtp֡p*v'K7-[-P�b )!4a" +"et("1p 00A-iO.ш׷<bN'ozS+;,<?4p2uCt/T c <+anfAv5! :r7Hl4k$ϗg(?5+U0:HUBe5+AfG{jf?~>Y廛 du =�G#)) Bdu59TK0.UtMK,"y[jş-a�q, O診0WWU㠚 :Ӑ7YefßLO@kqz¼#CL5 BY[z̖bA./`Ijy ^_] ss|+|XƠ sй=Tn~-9-,(b/)ntwԕWBw| oH^avAp"#SSAf` �2,*iiNJ̏~um7c�ۏ<'ޡsG YG4(ծh*+ >� � OhmSC.۟T:sO.J 2*4 r *XQ9VR{Lw0�,wQ[,Oh=A<2cYQ85T tNYԢ' 9'>sx7 ^wf׌),20iAP(a{p�K"M"z}U =&~9/ٮ{MdgLnY;Lz.Nz 9f3PCG6;D66Q'$En(jLy`Dߜ;=�/U7)} )EN% DmIhҁE�s[Q噥/v'{ISg<\1:x̦@̻ʻ3ye̜R`J:7Dsyاs?/Êt{p7Is@dd�d@lЃBN,;HIkKP<M"︭插m` V%"CIͦ@L�[d;@*ŦO("v\BkA6Y%ĶɁW`z,h'{;L$e ^%b8C)e=a췩Oܗ�P Bj.B�mQ`T"]Fi@Lh@G j`vձ/KT0J�C&�Y4B$>yGD^,Bu;Pm "g-3.OX23PH�}a(6*9$6p)kA54-f0fqGN_DOiQVRq[|Dm&(fr Ut 8-CR1[>`g::iXxҰ) |dacڴ#`mes=Gye vj[д dܮ."iHY;[E.@a1jΧap#ኈMfBc܁sblTp`z�rL.䪚 f.j+&'-]5-8πCsI@ L8^!=Mav6/o�P|Dmp"B=>5yN%ާޗ9 SD8T}"\Wl\Fa�%|fL}Y1XD %.@PpCE�ePkUA_\d/S^4oG9~e13[K"w+c)k$hIfrs1>43J7b*Qz`q;Ӱo(R <b f}7vG]#Jz^(6h؋=kZϡ;p9sPw!Ab$bLNȈ2BW-cQplC!Lf73tR,%bx2W*Va쓕Osϖr|2W9 pn%tzQ'%;Kӥ9^2/tǧ8Д~֢!X4NL؝Xߏ+T9t!BHD΂FYKbgَ{ݎPϤpۃMskDJ~'LXb>{jq|2 a /S(`] ,�r@"g$s蹄uܣG]j4ɩOݏ|Þ`yG? E"-6N@*o܀!4%2fs.7u4Ȣ+ *pȬ61)Ip:um'Y1�de Ȓ i&.=TGVڊ%@BZ0t &J KA[ BeRD[a= bEo{$nԆql"Tr{!*țn﷧{TRĒ֛A2HA;I <\Ohʀh36ةfa3=V}vYH<](�o<~cJ<g"'vuO+&SlݤCs9 2$!bcAvgz;I7cT^62g*�O}W>O_QW$*$c.jFE\MPFa\G!aq;)A h?lDO=+#J#M> /Z,HtЧȹ� fs3cM: a@=GzϹL�٨׿k ��|7~Ecݒb% F&C]n254R�ȀD|2Hv)LLʱayj~d�wo~qÔ9, &F1J) )@l Xa@$6Is]駗tM�cŵ'o D5/ݵ t .$ $͐k^Ӯ\[/j6O}9ik:(SM�j+�I5j0w,[kulz5swt;a*߉}|$S}WxL+ʡ5{  NAa\@5b` ͸/øt-m9(v%^}FSwJUf>U9 `Y#4H f4CSu6y*v&UИrͻeĮ)0}6Vqp< Z I8`K(c Ia5(u\=њq Kw@6�ٗkN`fQL`Bd`e̅' (Eg[ul*J`6+�ǿ?񇛁_O2 hps<)-g-.e̘uhJ*\<0Mx]�C~n5q1P1%YzK `J2By6\wk�vWR2F@@e!6 b [1BOͼgz`oؙ=k[i�;Czz5w㸮>C3ϸ?ֳ {  B"B/uۏ/]:2gI -#KwN<1t"jwgIVˌп8OV[09CU{kLy0@K3ޚD:"U%IKl)ut;Upz_ 7 v!OtN]NG����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/module.py������������������������������������������������������������0000664�0000000�0000000�00000014122�13034501105�0020457�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from weboob.capabilities.travel import CapTravel, Station, Departure from weboob.capabilities import UserError from .browser import VoyagesSNCFBrowser __all__ = ['VoyagesSNCFModule'] class VoyagesSNCFModule(Module, CapTravel): NAME = 'voyagessncf' DESCRIPTION = u'Voyages SNCF' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig(Value('age', label='Passenger age', default='ADULT', choices=OrderedDict((('ADULT', '26-59 ans'), ('SENIOR', '60 et +'), ('YOUNG', '12-25 ans'), ('CHILD_UNDER_FOUR', '0-3 ans'), ('CHILDREN', '4-11 ans')))), Value('card', label='Passenger card', default='default', choices=OrderedDict((('default', u'Pas de carte'), ('YOUNGS', u'Carte Jeune'), ('ESCA', u'Carte Escapades'), ('WEEKE', u'Carte Week-end'), ('FQ2ND', u'Abo Fréquence 2e'), ('FQ1ST', u'Abo Fréquence 1e'), ('FF2ND', u'Abo Forfait 2e'), ('FF1ST', u'Abo Forfait 1e'), ('ACCWE', u'Accompagnant Carte Week-end'), ('ACCCHD', u'Accompagnant Carte Enfant+'), ('ENFAM', u'Carte Enfant Famille'), ('FAM30', u'Carte Familles Nombreuses 30%'), ('FAM40', u'Carte Familles Nombreuses 40%'), ('FAM50', u'Carte Familles Nombreuses 50%'), ('FAM75', u'Carte Familles Nombreuses 75%'), ('MI2ND', u'Carte Militaire 2e'), ('MI1ST', u'Carte Militaire 1e'), ('MIFAM', u'Carte Famille Militaire'), ('THBIZ', u'Thalys ThePass Business'), ('THPREM', u'Thalys ThePass Premium'), ('THWE', u'Thalys ThePass Weekend')))), Value('class', label='Comfort class', default='2', choices=OrderedDict((('1', u'1e classe'), ('2', u'2e classe'))))) BROWSER = VoyagesSNCFBrowser STATIONS = [] def _populate_stations(self): if len(self.STATIONS) == 0: with self.browser: self.STATIONS = self.browser.get_stations() def iter_station_search(self, pattern): self._populate_stations() pattern = pattern.lower() already = set() # First stations whose name starts with pattern... for _id, name in enumerate(self.STATIONS): if name.lower().startswith(pattern): already.add(_id) yield Station(_id, unicode(name)) # ...then ones whose name contains pattern. for _id, name in enumerate(self.STATIONS): if pattern in name.lower() and _id not in already: yield Station(_id, unicode(name)) def iter_station_departures(self, station_id, arrival_id=None, date=None): self._populate_stations() if arrival_id is None: raise UserError('The arrival station is required') try: station = self.STATIONS[int(station_id)] arrival = self.STATIONS[int(arrival_id)] except (IndexError, ValueError): try: station = list(self.iter_station_search(station_id))[0].name arrival = list(self.iter_station_search(arrival_id))[0].name except IndexError: raise UserError('Unknown station') with self.browser: for i, d in enumerate(self.browser.iter_departures(station, arrival, date, self.config['age'].get(), self.config['card'].get(), self.config['class'].get())): departure = Departure(i, d['type'], d['time']) departure.departure_station = d['departure'] departure.arrival_station = d['arrival'] departure.arrival_time = d['arrival_time'] departure.price = d['price'] departure.currency = d['currency'] departure.information = d['price_info'] yield departure ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/pages.py�������������������������������������������������������������0000664�0000000�0000000�00000010547�13034501105�0020300�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from decimal import Decimal from datetime import time, datetime, timedelta from weboob.deprecated.browser import Page from weboob.tools.json import json from weboob.deprecated.mech import ClientForm from weboob.capabilities.base import UserError, Currency class ForeignPage(Page): def on_loaded(self): raise UserError('Your IP address is localized in a country not supported by this module (%s). Currently only the French website is supported.' % self.group_dict['country']) class CitiesPage(Page): def get_stations(self): result = json.loads(self.document[self.document.find('{'):-2]) return result['CITIES'] class SearchPage(Page): def search(self, departure, arrival, date, age, card, comfort_class): self.browser.select_form(name='saisie') self.browser['ORIGIN_CITY'] = departure.encode(self.browser.ENCODING) self.browser['DESTINATION_CITY'] = arrival.encode(self.browser.ENCODING) if date is None: date = datetime.now() + timedelta(hours=1) elif date < datetime.now(): raise UserError("You cannot look for older departures") self.browser['OUTWARD_DATE'] = date.strftime('%d/%m/%y') self.browser['OUTWARD_TIME'] = [str(date.hour)] self.browser['PASSENGER_1'] = [age] self.browser['PASSENGER_1_CARD'] = [card] self.browser['COMFORT_CLASS'] = [str(comfort_class)] self.browser.controls.append(ClientForm.TextControl('text', 'nbAnimalsForTravel', {'value': ''})) self.browser['nbAnimalsForTravel'] = '0' self.browser.submit() class SearchErrorPage(Page): def on_loaded(self): p = self.document.getroot().cssselect('div.messagesError p') if len(p) > 0: message = p[0].text.strip() raise UserError(message) class SearchInProgressPage(Page): def on_loaded(self): link = self.document.xpath('//a[@id="url_redirect_proposals"]')[0] self.browser.location(link.attrib['href']) class ResultsPage(Page): def get_value(self, div, name, last=False): i = -1 if last else 0 p = div.cssselect(name)[i] sub = p.find('p') if sub is not None: txt = sub.tail.strip() if txt == '': p.remove(sub) else: return unicode(txt) return unicode(self.parser.tocleanstring(p)) def parse_hour(self, div, name, last=False): txt = self.get_value(div, name, last) hour, minute = map(int, txt.split('h')) return time(hour, minute) def iter_results(self): for div in self.document.getroot().cssselect('div.train_info'): info = None price = None currency = None for td in div.cssselect('td.price'): txt = self.parser.tocleanstring(td) p = Decimal(re.sub('([^\d\.]+)', '', txt)) if price is None or p < price: info = list(div.cssselect('strong.price_label')[0].itertext())[-1].strip().strip(':') price = p currency = Currency.get_currency(txt) yield {'type': self.get_value(div, 'div.transporteur-txt'), 'time': self.parse_hour(div, 'div.departure div.hour'), 'departure': self.get_value(div, 'div.departure div.station'), 'arrival': self.get_value(div, 'div.arrival div.station', last=True), 'arrival_time': self.parse_hour(div, 'div.arrival div.hour', last=True), 'price': price, 'currency': currency, 'price_info': info, } ���������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/voyagessncf/test.py��������������������������������������������������������������0000664�0000000�0000000�00000002476�13034501105�0020162�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class VoyagesSNCFTest(BackendTest): MODULE = 'voyagessncf' def test_stations(self): stations = list(self.backend.iter_station_search('paris')) self.assertTrue(len(stations) > 0) self.assertTrue('Paris Massy' in stations[-1].name) def test_departures(self): departure = list(self.backend.iter_station_search('paris'))[0] arrival = list(self.backend.iter_station_search('lyon'))[0] prices = list(self.backend.iter_station_departures(departure.id, arrival.id)) self.assertTrue(len(prices) > 0) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015730�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001432�13034501105�0020041�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Arno Renevier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import WeatherModule __all__ = ['WeatherModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000004455�13034501105�0017775�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Arno Renevier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import urllib from weboob.deprecated.browser import Browser from .pages import ForecastPage, WeatherPage, CityPage __all__ = ['WeatherBrowser'] class WeatherBrowser(Browser): DOMAIN = 'www.weather.com' PROTOCOL = 'http' ENCODING = 'utf-8' PAGES = {} SEARCH_URL = 'http://www.weather.com/search/enhancedlocalsearch?where=%s' WEATHER_URL = 'http://www.weather.com/weather/today/%s' FORECAST_URL = 'http://www.weather.com/weather/tenday/%s' RIGHTNOW_URL = 'http://www.weather.com/weather/right-now/%s' USER_AGENT = Browser.USER_AGENTS['desktop_firefox'] PAGES = { (SEARCH_URL.replace('.', '\\.').replace('?', '\\?') % '.*'): CityPage, (WEATHER_URL.replace('.', '\\.').replace('?', '\\?') % '.*'): WeatherPage, (FORECAST_URL.replace('.', '\\.').replace('?', '\\?') % '.*'): ForecastPage, (RIGHTNOW_URL.replace('.', '\\.').replace('?', '\\?') % '.*'): WeatherPage, } def iter_city_search(self, pattern): self.location(self.SEARCH_URL % urllib.quote_plus(pattern.encode('utf-8'))) if self.is_on_page(CityPage): return self.page.iter_city_search() elif self.is_on_page(WeatherPage): return [self.page.get_city()] def get_current(self, city_id): self.location(self.WEATHER_URL % urllib.quote_plus(city_id.encode('utf-8'))) return self.page.get_current() def iter_forecast(self, city_id): self.location(self.FORECAST_URL % urllib.quote_plus(city_id.encode('utf-8'))) assert self.is_on_page(ForecastPage) return self.page.iter_forecast() �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000002743�13034501105�0020071�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@�����@PLTE��  !!##$$%%&&(())**,,..//001122445566778899::;;<<??AABBDDIIJJKKMMNNQQRRWWXXYYZZ[[]]^^``aabbccddeeffiijjllmmnnppqqrruuwwyyzz{{~~ĀŁłƄDždžLJȈȋʌʎ˔ΗϘЙКћќҞҠӡԢԤե֦֧רשتثج٭ٮگڰ۱۲ܴܳݵݶ޷޻༼྾῿Vn��^IDATXoUǿyq'JLI*QITRN*'$\�BQ "4 I4?&l`rC]!sڧ}ٙ7y;0&ABNΞK~<}rt\Xq1$LLDbY.U8%PItEs��PaIUH~~@ermP];[�쭻:STn:~w?:!Z`}=[HY7o~31sDlwNz!~-= 3N"H]~뎏Rsljk*�j>sG!#ޡD��|KtQg^\h{O 5@fZyTJ!̕@Wlj>_^&L&i\>L}ξ Ra*V(P:9XiRl`f/tۺ}Y@lmجВ� y_Up%./JC*oִLa FFmO4ZiI:h>5\�ャP` W(*&F@na; \#$@7BC(@UD~Bk ?$p{)G~yV9:�fL_Vd–+ @.7{{wn6>8,@jt8|v\GӇ,[j׍4o:{ APŢFT NvUE}&TP*`js Z/}?lw*����IENDB`�����������������������������weboob-1.2/modules/weather/module.py����������������������������������������������������������������0000664�0000000�0000000�00000002606�13034501105�0017573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Arno Renevier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.weather import CapWeather from weboob.tools.backend import Module from .browser import WeatherBrowser __all__ = ['WeatherModule'] class WeatherModule(Module, CapWeather): NAME = 'weather' MAINTAINER = u'Arno Renevier' EMAIL = 'arno@renevier.net' VERSION = '1.2' DESCRIPTION = 'Get forecasts from weather.com' LICENSE = 'AGPLv3+' BROWSER = WeatherBrowser def iter_city_search(self, pattern): return self.browser.iter_city_search(pattern) def get_current(self, city_id): return self.browser.get_current(city_id) def iter_forecast(self, city_id): return self.browser.iter_forecast(city_id) ��������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000005260�13034501105�0017404�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Arno Renevier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Page from weboob.capabilities.weather import Forecast, Current, City import datetime class CityPage(Page): def iter_city_search(self): for item in self.document.findall('//div[@class="searchResultsList"]/ul/li'): if item.attrib.get('class', '') == 'searchResultsMoreLink': continue city_name = unicode(item.text_content().strip()) city_id = item.find('a').attrib.get("href", "").split("+")[-1] yield City(city_id, city_name) class WeatherPage(Page): def get_city(self): parts = self.url.split('/')[-1].split('+') return City(parts[-1], u', '.join(parts[:-1])) def get_current(self): date = datetime.datetime.now() text = unicode(self.document.findall('//p[@class="wx-narrative"]')[0].text_content().strip()) temp = float(self.document.find('//p[@class="wx-temp"]').text_content().strip().split(u'°')[0]) return Current(date, temp, text, u'F') class ForecastPage(Page): def iter_forecast(self): divs = self.document.findall('//div[@class="wx-daypart"]') for day in range (0, len(divs)): div = divs[day].find('div[@class="wx-conditions"]') text = unicode(div.find('p[@class="wx-phrase"]').text_content().strip()) try: thigh = float(div.find('p[@class="wx-temp"]').text_content().strip().split(u'°')[0]) except: thigh = None try: tlow = float(div.find('p[@class="wx-temp-alt"]').text_content().strip().split(u'°')[0]) except: tlow = None date = divs[day].find('h3/span').text_content().strip() #date = self.document.findall('//table[@class="twc-forecast-table twc-first"]//th')[day].text #if len (date.split(' ')) > 3: # date = " ".join(date.split(' ', 3)[:3]) yield Forecast(date, tlow, thigh, text, u'F') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/weather/test.py������������������������������������������������������������������0000664�0000000�0000000�00000002643�13034501105�0017266�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Arno Renevier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class WeatherTest(BackendTest): MODULE = 'weather' def test_cities(self): paris = self.backend.iter_city_search('crappything¶m=;drop database') self.assertTrue(len(list(paris)) == 0) paris = self.backend.iter_city_search('paris') self.assertTrue(len(list(paris)) >= 1) paris = self.backend.iter_city_search('paris france') self.assertTrue(len(list(paris)) == 1) current = self.backend.get_current(paris[0].id) self.assertTrue(current.temp.value is float(current.temp.value)) forecasts = list(self.backend.iter_forecast(paris[0].id)) self.assertTrue(len(forecasts) == 10) ���������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016436�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/__init__.py�����������������������������������������������������������0000664�0000000�0000000�00000001450�13034501105�0020547�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import WellsFargoModule __all__ = ['WellsFargoModule'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/browser.py������������������������������������������������������������0000664�0000000�0000000�00000025606�13034501105�0020504�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import AccountNotFound from weboob.browser import LoginBrowser, URL, need_login from weboob.exceptions import BrowserIncorrectPassword, BrowserUnavailable import ssl import json import os from tempfile import mkstemp from subprocess import STDOUT, CalledProcessError from weboob.tools.compat import check_output from urllib import unquote from .pages import LoginProceedPage, LoginRedirectPage, \ SummaryPage, ActivityCashPage, ActivityCardPage, \ DocumentsPage, StatementPage, StatementsPage, \ StatementsEmbeddedPage, LoggedInPage, CodeRequestPage, \ CodeSubmitPage __all__ = ['WellsFargo'] class WellsFargo(LoginBrowser): BASEURL = 'https://online.wellsfargo.com' TIMEOUT = 30 MAX_RETRIES = 10 login_proceed = URL('/das/cgi-bin/session.cgi\?screenid=SIGNON.*$', '/login\?ERROR_CODE=.*LOB=CONS&$', LoginProceedPage) login_redirect = URL('/das/cgi-bin/session.cgi\?screenid=SIGNON.*$', '/login\?ERROR_CODE=.*LOB=CONS&$', LoginRedirectPage) code_request = URL('https://oam.wellsfargo.com/oam/access' '/twoFAAARDisplay\?OAM_TKN=.+$', CodeRequestPage) code_submit = URL('https://oam.wellsfargo.com/oam/access' '/twoFAAARDisplay\?OAM_TKN=.+$', 'https://oam.wellsfargo.com/oam/access' '/twoFAAARSubmitCode\?OAM_TKN=.+$', CodeSubmitPage) summary = URL('/das/channel/accountSummary$', SummaryPage) activity_cash = URL('/das/cgi-bin/session.cgi\?sessargs=.+$', ActivityCashPage) activity_card = URL('/das/cgi-bin/session.cgi\?sessargs=.+$', ActivityCardPage) documents = URL('https://connect.secure.wellsfargo.com' '/accounts/start\?.+$', DocumentsPage) statements_embedded = URL('https://connect.secure.wellsfargo.com' '/accounts/start\?.+$', StatementsEmbeddedPage) statements = URL('https://connect.secure.wellsfargo.com' '/accounts/documents/statement/list.+$', StatementsPage) statement = URL('https://connect.secure.wellsfargo.com' '/accounts/documents/retrieve/.+$', StatementPage) unknown = URL('/.*$', LoggedInPage) # E.g. random advertisement pages. def __init__(self, question1, answer1, question2, answer2, question3, answer3, phone_last4, code_file, *args, **kwargs): super(WellsFargo, self).__init__(*args, **kwargs) self.question1 = question1 self.answer1 = answer1 self.question2 = question2 self.answer2 = answer2 self.question3 = question3 self.answer3 = answer3 self.phone_last4 = phone_last4 self.code_file = code_file def do_login(self): ''' There's a bunch of dynamically generated obfuscated JavaScript, which uses DOM. For now the easiest option seems to be to run it in PhantomJs. ''' for i in xrange(self.MAX_RETRIES): scrf, scrn = mkstemp('.js') cookf, cookn = mkstemp('.json') os.write(scrf, LOGIN_JS % { 'scriptTimeout': self.TIMEOUT*2, 'resourceTimeout': self.TIMEOUT, 'username': self.username, 'password': self.password, 'output': cookn, 'question1': self.question1, 'answer1': self.answer1, 'question2': self.question2, 'answer2': self.answer2, 'question3': self.question3, 'answer3': self.answer3}) os.close(scrf) os.close(cookf) try: check_output(["phantomjs", scrn], stderr=STDOUT) with open(cookn) as cookf: cookies = json.loads(cookf.read()) except CalledProcessError: continue finally: os.remove(scrn) os.remove(cookn) self.session.cookies.clear() for c in cookies: for k in ['expiry', 'expires', 'httponly']: c.pop(k, None) c['value'] = unquote(c['value']) self.session.cookies.set(**c) self.summary.go() if self.page.logged: break else: raise BrowserIncorrectPassword def location(self, *args, **kwargs): """ Wells Fargo inserts redirecting pages from time to time, so we should follow them whenever we see them. """ r = super(WellsFargo, self).location(*args, **kwargs) if self.login_proceed.is_here(): return self.page.proceed() elif self.login_redirect.is_here(): return self.page.redirect() elif self.code_request.is_here(): return self.page.request_code() elif self.code_submit.is_here(): return self.page.submit_code() else: return r def prepare_request(self, req): """ Wells Fargo uses TLS v1.0. See issue #1647 for details. """ preq = super(WellsFargo, self).prepare_request(req) conn = self.session.adapters['https://'].get_connection(preq.url) conn.ssl_version = ssl.PROTOCOL_TLSv1 return preq def get_account(self, id_): self.to_activity() if id_ not in self.page.accounts_ids(): raise AccountNotFound() else: self.to_activity(id_) return self.page.get_account() def iter_accounts(self): self.to_activity() for id_ in self.page.accounts_ids(): self.to_activity(id_) yield self.page.get_account() @need_login def to_summary(self): self.summary.stay_or_go() assert self.summary.is_here() def is_activity(self): return self.activity_cash.is_here() or self.activity_card.is_here() @need_login def to_activity(self, id_=None): if not self.is_activity(): self.to_summary() self.page.to_activity() assert self.is_activity() if id_ and self.page.account_id() != id_: self.page.to_account(id_) assert self.is_activity() assert self.page.account_id() == id_ @need_login def to_statements(self, id_=None, year=None): if not self.statements.is_here() \ and not self.statements_embedded.is_here(): self.to_summary() self.page.to_documents() if self.documents.is_here(): self.page.to_statements() assert self.statements.is_here() else: assert self.statements_embedded.is_here() if id_ and self.page.parser().account_id() != id_: self.page.parser().to_account(id_) assert self.statements.is_here() assert self.page.parser().account_id() == id_ if year and self.page.parser().year() != year: self.page.parser().to_year(year) assert self.statements.is_here() assert self.page.parser().year() == year @need_login def to_statement(self, uri): for i in xrange(self.MAX_RETRIES): self.location(uri) if self.statement.is_here(): break else: raise BrowserUnavailable() def iter_history(self, account): self.to_activity(account.id) # Skip transactions on web page if we cannot apply # "since last statement" filter. # This might be the case, for example, if Wells Fargo # is processing the current statement: # "Since your credit card account statement is being processed, # transactions grouped by statement period will not be available # for up to seven days." # (www.wellsfargo.com, 2014-07-20) if self.page.since_last_statement(): assert self.page.account_id() == account.id while True: for trans in self.page.iter_transactions(): yield trans if not self.page.next_(): break self.to_statements(account.id) for year in self.page.parser().years(): self.to_statements(account.id, year) for stmt in self.page.parser().statements(): self.to_statement(stmt) for trans in self.page.iter_transactions(): yield trans LOGIN_JS = u'''\ var page = require('webpage').create(); page.settings.resourceTimeout = %(resourceTimeout)s*1000; page.open('https://www.wellsfargo.com/'); var waitForForm = function() { var hasForm = page.evaluate(function(){ return !!document.getElementById('frmSignon') }); if (hasForm) { page.evaluate(function(){ document.getElementById('userid').value = '%(username)s'; document.getElementById('password').value = '%(password)s'; document.getElementById('frmSignon').submit(); }); } else { setTimeout(waitForForm, 1000); } } var waitForQuestions = function() { var isQuestion = page.content.indexOf('Confirm Your Identity') != -1; if (isQuestion) { var questions = { "%(question1)s": "%(answer1)s", "%(question2)s": "%(answer2)s", "%(question3)s": "%(answer3)s" }; for (var question in questions) { if (page.content.indexOf(question)) { page.evaluate(function(answer){ document.getElementById('answer').value = answer; document.getElementById('command').submit.click(); }, questions[question]); } } } setTimeout(waitForQuestions, 2000); } var waitForLogin = function() { var isSplash = page.content.indexOf('Splash Page') != -1; var hasSignOff = page.content.indexOf('Sign Off') != -1; if (isSplash || hasSignOff) { var cookies = JSON.stringify(phantom.cookies); require('fs').write('%(output)s', cookies, 'w'); phantom.exit(); } else { setTimeout(waitForLogin, 2000); } } waitForForm(); waitForQuestions(); waitForLogin(); setTimeout(function(){phantom.exit(-1);}, %(scriptTimeout)s*1000); ''' ��������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/favicon.png�����������������������������������������������������������0000664�0000000�0000000�00000000435�13034501105�0020573�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@������PLTE�KXP���IDAT(c`#`o�30f~e C[Y l ';E00\> T�d|<p a@Fbl=>a q<<G Dxj v99dϟ? ,!0`}axg1L<P]Ll`{;4�T*G!,T����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/module.py�������������������������������������������������������������0000664�0000000�0000000�00000005527�13034501105�0020306�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import CapBank from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from .browser import WellsFargo __all__ = ['WellsFargoModule'] class WellsFargoModule(Module, CapBank): NAME = 'wellsfargo' MAINTAINER = u'Oleg Plakhotniuk' EMAIL = 'olegus8@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' DESCRIPTION = u'Wells Fargo' CONFIG = BackendConfig( ValueBackendPassword('login', label='Username', masked=False), ValueBackendPassword('password', label='Password'), ValueBackendPassword('question1', label='Question 1', masked=False), ValueBackendPassword('answer1', label='Answer 1', masked=False), ValueBackendPassword('question2', label='Question 2', masked=False), ValueBackendPassword('answer2', label='Answer 2', masked=False), ValueBackendPassword('question3', label='Question 3', masked=False), ValueBackendPassword('answer3', label='Answer 3', masked=False), ValueBackendPassword('phone_last4', label='Last 4 digits of phone number to request access code to', masked=False), ValueBackendPassword('code_file', label='File to read access code from', masked=False)) BROWSER = WellsFargo def create_default_browser(self): return self.create_browser( username = self.config['login'].get(), password = self.config['password'].get(), question1 = self.config['question1'].get(), answer1 = self.config['answer1'].get(), question2 = self.config['question2'].get(), answer2 = self.config['answer2'].get(), question3 = self.config['question3'].get(), answer3 = self.config['answer3'].get(), phone_last4 = self.config['phone_last4'].get(), code_file = self.config['code_file'].get()) def iter_accounts(self): return self.browser.iter_accounts() def get_account(self, id_): return self.browser.get_account(id_) def iter_history(self, account): return self.browser.iter_history(account) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/pages.py��������������������������������������������������������������0000664�0000000�0000000�00000040427�13034501105�0020116�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import Account, Transaction from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.browser.pages import HTMLPage, LoggedPage, RawPage from decimal import Decimal from requests.cookies import morsel_to_cookie from .parsers import StatementParser, clean_label from time import sleep import itertools import json import re import os import datetime import Cookie class LoginProceedPage(LoggedPage, HTMLPage): is_here = '//script[contains(text(),"setAndCheckCookie")]' def proceed(self): script = self.doc.xpath('//script/text()')[0] cookieStr = re.match('.*document\.cookie = "([^"]+)".*', script, re.DOTALL).group(1) morsel = Cookie.Cookie(cookieStr).values()[0] self.browser.session.cookies.set_cookie(morsel_to_cookie(morsel)) form = self.get_form() return form.submit() class LoginRedirectPage(LoggedPage, HTMLPage): is_here = 'contains(//meta[@http-equiv="Refresh"]/@content,' \ '"SIGNON_PORTAL_PAUSE")' def redirect(self): refresh = self.doc.xpath( '//meta[@http-equiv="Refresh"]/@content')[0] url = re.match(r'^.*URL=(.*)$', refresh).group(1) return self.browser.location(url) class LoggedInPage(HTMLPage): @property def logged(self): return bool(self.doc.xpath(u'//a[text()="Sign Off"]')) \ or bool(self.doc.xpath(u'//title[text()="Splash Page"]')) \ or bool(self.doc.xpath(u'//title[contains(text(),' u'"Advanced Access Required")]')) class CodeRequestPage(LoggedInPage): is_here = u'contains(//label/text(),"Send Code to")' def request_code(self): phone = self.browser.page.doc.xpath( '//select[@name="telephone"]/option[contains(text(),"%s")]/@value' % self.browser.phone_last4)[0] form = self.get_form(name='otp') form['telephone'] = [phone] form['deliveryMode'] = ['sms'] form['sendcode'] = ['Request Code'] del form['cancelBtn'] del form['sendcode2'] return form.submit() class CodeSubmitPage(LoggedInPage): is_here = u'contains(//title/text(),"Enter and Submit Advanced Access Code")' def submit_code(self): self.browser.logger.warning( 'The code has been sent to the phone number ending with %s. ' 'Please write this code into the file %s' % (self.browser.phone_last4, self.browser.code_file)) while True: try: with open(self.browser.code_file) as f: code = f.read().strip() break except IOError: sleep(1) os.remove(self.browser.code_file) self.browser.logger.info('The code %s has been successfully read'%code) form = self.get_form(name='otp') form['passcode'] = [code] del form['cancelBtn'] del form['btnSubmit'] return form.submit() class SummaryPage(LoggedInPage): is_here = u'//title[contains(text(),"Account Summary")]' def to_activity(self): href = self.doc.xpath(u'//a[text()="Account Activity"]/@href')[0] self.browser.location(href) def to_documents(self): href = self.doc.xpath('//a[text()="Statements & Documents"]' '/@href')[0] self.browser.location(href) class AccountPage(object): def account_id(self, name=None): if name: return name[-4:] # Last 4 digits of "BLAH XXXXXXX1234" else: return self.account_id(self.account_name()) class ActivityPage(AccountPage, LoggedInPage): def is_here(self): return bool(self.doc.xpath( u'contains(//title/text(),"Account Activity")')) def accounts_names(self): return self.doc.xpath( u'//select[@name="selectedAccountUID"]/option/text()') def accounts_ids(self): return [self.account_id(name) for name in self.accounts_names()] def account_uid(self, id_=None): if id_: return self.doc.xpath( u'//select[@name="selectedAccountUID"]' u'/option[contains(text(),"%s")]/@value' % id_)[0] else: return self.doc.xpath( u'//select[@name="selectedAccountUID"]' u'/option[@selected="selected"]/@value')[0] def account_name(self): for name in self.doc.xpath(u'//select[@name="selectedAccountUID"]' u'/option[@selected="selected"]/text()'): return name return u'' def account_type(self, name=None): raise NotImplementedError() def account_balance(self): raise NotImplementedError() def account_paydatemin(self): return None, None def account_cardlimit(self): return None def to_account(self, id_): form = self.get_form(xpath='//form[@name="AccountActivityForm"]') form['selectedAccountUID'] = [self.account_uid(id_)] form.submit() def get_account(self): name = self.account_name() balance = self.account_balance() currency = Account.get_currency(balance) id_ = self.account_id() type_ = self.account_type() paydate, paymin = self.account_paydatemin() cardlimit = self.account_cardlimit() account = Account() account.id = id_ account.label = name account.currency = currency account.balance = AmTr.decimal_amount(balance) account.type = type_ if paydate is not None: account.paydate = paydate if paymin is not None: account.paymin = paymin if cardlimit is not None: account.cardlimit = AmTr.decimal_amount(cardlimit) return account def since_last_statement(self): raise NotImplementedError() def iter_transactions(self): raise NotImplementedError() def next_(self): raise NotImplementedError() class ActivityCashPage(ActivityPage): def is_here(self): return super(ActivityCashPage, self).is_here() and \ (u'CHECKING' in self.account_name() or u'SAVINGS' in self.account_name()) def account_type(self, name=None): name = name or self.account_name() if u'CHECKING' in name: return Account.TYPE_CHECKING elif u'SAVINGS' in name: return Account.TYPE_SAVINGS else: return Account.TYPE_UNKNOWN def account_balance(self): return self.doc.xpath( u'//td[@headers="currentPostedBalance"]/span/text()')[0] def since_last_statement(self): if not self.doc.xpath(u'//p[@id="noactivitymessage"]'): form = self.get_form(xpath='//form[@id="ddaShowForm"]') form['showTabDDACommand.transactionTypeFilterValue'] = [ u'All Transactions'] form['showTabDDACommand.timeFilterValue'] = ['8'] form.submit() return True def iter_transactions(self): for row in self.doc.xpath('//tr/th[@headers=' '"postedHeader dateHeader"]/..'): date = row.xpath('th[@headers="postedHeader ' 'dateHeader"]/text()')[0] desc = row.xpath('td[@headers="postedHeader descriptionHeader"]' '//span[@class="OneLinkNoTx"]/text()')[0] deposit = row.xpath('td[@headers="postedHeader ' 'depositsConsumerHeader"]/span/text()')[0] withdraw = row.xpath('td[@headers="postedHeader ' 'withdrawalsConsumerHeader"]/span/text()')[0] date = datetime.datetime.strptime(date, '%m/%d/%y') desc = clean_label(desc) deposit = deposit.strip() deposit = AmTr.decimal_amount(deposit or '0') withdraw = withdraw.strip() withdraw = AmTr.decimal_amount(withdraw or '0') amount = deposit - withdraw trans = Transaction(u'') trans.date = date trans.rdate = date trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = amount yield trans def next_(self): links = self.doc.xpath('//a[@title="Go To Next Page"]/@href') if links: self.browser.location(links[0]) return True else: return False class ActivityCardPage(ActivityPage): def is_here(self): return super(ActivityCardPage, self).is_here() and \ u'CARD' in self.account_name() def account_type(self, name=None): return Account.TYPE_CARD def account_balance(self): return self.doc.xpath( u'//td[@headers="outstandingBalance"]/text()')[0] def account_cardlimit(self): return self.doc.xpath( u'//td[@headers="totalCreditLimit"]/text()')[0] def account_paydatemin(self): if self.doc.xpath(u'//p[contains(text(),' '"payment due date is not yet scheduled")]'): # If payment date is not scheduled yet, set it somewhere in a # distant future, so that we always have a valid date. return datetime.datetime.now() + datetime.timedelta(days=999), \ Decimal(0) else: date = self.doc.xpath(u'//span[contains(text(),"Minimum Payment")]' '/span/text()')[0] date = re.match(u'.*(../../..).*', date).group(1) date = datetime.datetime.strptime(date, '%m/%d/%y') amount = self.doc.xpath(u'//td[@headers="minimumPaymentDue"]' '//text()')[0].strip() return date, AmTr.decimal_amount(amount) def get_account(self): account = ActivityPage.get_account(self) # Credit card is essentially a liability. # Negative amount means there's a payment due. account.balance = -account.balance return account def since_last_statement(self): if self.doc.xpath('//select[@name="showTabCommand.' 'transactionTypeFilterValue"]' '/option[@value="sincelastStmt"]'): form = self.get_form(xpath='//form[@id="creditCardShowForm"]') form['showTabCommand.transactionTypeFilterValue'] = [ 'sincelastStmt'] form.submit() return True def iter_transactions(self): for row in self.doc.xpath('//tr/th[@headers=' '"postedHeader transactionDateHeader"]/..'): tdate = row.xpath('th[@headers="postedHeader ' 'transactionDateHeader"]/text()')[0] pdate = row.xpath('td[@headers="postedHeader ' 'postingDateHeader"]/text()')[0] desc = row.xpath('td[@headers="postedHeader ' 'descriptionHeader"]/span/text()')[0] ref = row.xpath('td[@headers="postedHeader ' 'descriptionHeader"]/text()')[0] amount = row.xpath('td[@headers="postedHeader ' 'amountHeader"]/text()')[0] tdate = datetime.datetime.strptime(tdate, '%m/%d/%y') pdate = datetime.datetime.strptime(pdate, '%m/%d/%y') desc = clean_label(desc) ref = re.match('.*<REFERENCE ([^>]+)>.*', ref).group(1) if amount.startswith('+'): amount = AmTr.decimal_amount(amount[1:]) else: amount = -AmTr.decimal_amount(amount) trans = Transaction(ref) trans.date = tdate trans.rdate = pdate trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = amount yield trans def next_(self): # As of 2014-07-05, there's only one page for cards history. return False class DocumentsPage(LoggedInPage): HEADER_XPATH = u'//h1[contains(text(),"Statements and Documents")]' LINK_XPATH = u'//a[@data-async-load-template="stmtdisc.html"]' \ u'/@data-async-load-url' def is_here(self): return self.doc.xpath(self.HEADER_XPATH) \ and self.doc.xpath(self.LINK_XPATH) def to_statements(self): url = self.doc.xpath(self.LINK_XPATH)[0] self.browser.location(url) class StatementsEmbeddedPage(LoggedInPage): HEADER_XPATH = u'//h1[contains(text(),"Statements and Documents")]' SCRIPT_XPATH = '//script[contains(text(),"stmtdisc.html")]/text()' def is_here(self): return self.doc.xpath(self.HEADER_XPATH) \ and self.doc.xpath(self.SCRIPT_XPATH) def get_embedded_data(self): scr = self.doc.xpath(self.SCRIPT_XPATH)[0] data = json.loads('\n'.join(scr.split('\n')[2:-2]).replace( "'appendTo'",'"appendTo"')) return json.loads(data['data']) def parser(self): return StatementsPageParser(self.get_embedded_data(), self.browser) class WfJsonPage(LoggedPage, RawPage): def __init__(self, *args, **kwArgs): RawPage.__init__(self, *args, **kwArgs) clean = self.doc.replace('"/*WellFargoProprietary%','') \ .replace('%WellFargoProprietary*/"','').decode('string_escape') self.doc = json.loads(clean) class StatementsPage(WfJsonPage): def parser(self): return StatementsPageParser(self.doc, self.browser) class StatementsPageParser(AccountPage): def __init__(self, doc, browser): self.doc = doc self.browser = browser def account_name(self): for account in self.accounts(): if account['selected']: return account['accountDisplayName'] def to_account(self, id_): for account in self.accounts(): if account['accountDisplayName'].endswith(id_): self.browser.location(account['url']) def year(self): for period in self.periods(): if period['selected']: try: return int(period['timePeriod']) except ValueError: pass def years(self): for period in self.periods(): try: yield int(period['timePeriod']) except ValueError: pass def to_year(self, year): for period in self.periods(): if period['timePeriod'] == str(year): self.browser.location(period['url']) def statements(self): stmts = self.doc['statementsDisclosuresInfo'].get('statements', []) for stmt in stmts: yield stmt['url'] def accounts(self): return self.doc['statementsDisclosuresInfo']['accountList'] def periods(self): return self.doc['statementsDisclosuresInfo']['timePeriodList'] class StatementPage(LoggedPage, RawPage): def __init__(self, *args, **kwArgs): RawPage.__init__(self, *args, **kwArgs) self._parser = StatementParser(self.doc) def is_here(self): return self.doc[:4] == '%PDF' def iter_transactions(self): # Maintain a nice consistent newer-to-older order of transactions. return sorted( itertools.chain( self._parser.read_cash_transactions(), self._parser.read_card_transactions()), cmp=lambda t1, t2: cmp(t2.date, t1.date) or cmp(t1.label, t2.label) or cmp(t1.amount, t2.amount)) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/parsers.py������������������������������������������������������������0000664�0000000�0000000�00000025152�13034501105�0020474�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.bank import Transaction from weboob.tools.capabilities.bank.transactions import \ AmericanTransaction as AmTr from weboob.tools.date import closest_date from weboob.tools.pdf import decompress_pdf from weboob.tools.tokenizer import ReTokenizer import re import datetime def clean_label(text): """ Web view and statements use different label formatting. User shouldn't be able to see the difference, so we need to make labels from both sources look the same. """ return re.sub(u' +', u' ', text.strip().upper(), re.UNICODE) class StatementParser(object): """ Each "read_*" method takes position as its argument, and returns next token position if read was successful, or the same position if it was not. """ LEX = [ ('amount', r'^\[\(([0-9,]+\.\d+)\)\] TJ$'), ('date', r'^\[\((\d+/\d+)\)\] TJ$'), ('date_range_1', r'^\[\(([A-z]+ \d+, \d{4})' r' - ([A-z]+ \d+, \d{4})\)\] TJ$'), ('date_range_2', r'^\[\((\d{2}/\d{2}/\d{4})' r' to (\d{2}/\d{2}/\d{4})\)\] TJ$'), ('layout_tz', r'^(\d+\.\d{2}) Tz$'), ('layout_tc', r'^(\d+\.\d{2}) Tc$'), ('layout_tw', r'^(\d+\.\d{2}) Tw$'), ('layout_tf', r'^/F(\d) (\d+\.\d{2}) Tf$'), ('layout_tm', r'^' + (r'(\d+\.\d+ )'*6) + r'Tm$'), ('ref', r'^\[\(([0-9A-Z]{17})\)\] TJ$'), ('text', r'^\[\(([^\)]+)\)\] TJ$') ] def __init__(self, pdf): self._pdf = decompress_pdf(pdf) self._tok = ReTokenizer(self._pdf, '\n', self.LEX) def read_card_transactions(self): # Early check if this is a card account statement at all. if '[(Transactions)] TJ' not in self._pdf: return # Read statement dates range. date_from, date_to = self.read_first_date_range() # Read transactions. pos = 0 while not self._tok.tok(pos).is_eof(): pos, trans = self.read_card_transaction(pos, date_from, date_to) if trans: yield trans else: pos += 1 def read_cash_transactions(self): # Early check if this is a cash account statement at all. if '[(Transaction history)] TJ' not in self._pdf: return # Read statement dates range. date_from, date_to = self.read_first_date_range() # Read transactions. pos = 0 while not self._tok.tok(pos).is_eof(): pos, trans = self.read_cash_transaction(pos, date_from, date_to) if trans: yield trans else: pos += 1 def read_first_date_range(self): pos = 0 while not self._tok.tok(pos).is_eof(): pos, date_range = self.read_date_range(pos) if date_range is not None: return date_range else: pos += 1 def read_card_transaction(self, pos, date_from, date_to): INDENT_CHARGES = 520 startPos = pos pos, tdate = self.read_date(pos) pos, pdate_layout = self.read_layout_tm(pos) pos, pdate = self.read_date(pos) pos, ref_layout = self.read_layout_tm(pos) pos, ref = self.read_ref(pos) pos, desc = self.read_multiline_desc(pos) pos, amount = self.read_indent_amount( pos, range_minus = (INDENT_CHARGES, 9999), range_plus = (0, INDENT_CHARGES)) if tdate is None or pdate_layout is None or pdate is None \ or ref_layout is None or ref is None or desc is None or amount is None: return startPos, None else: tdate = closest_date(tdate, date_from, date_to) pdate = closest_date(pdate, date_from, date_to) trans = Transaction(ref) trans.date = tdate trans.rdate = pdate trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = amount return pos, trans def read_cash_transaction(self, pos, date_from, date_to): INDENT_BALANCE = 520 INDENT_WITHDRAWAL = 470 startPos = pos pos, date = self.read_date(pos) pos, _ = self.read_star(pos) pos, desc = self.read_multiline_desc(pos) pos, amount = self.read_indent_amount( pos, range_plus = (0, INDENT_WITHDRAWAL), range_minus = (INDENT_WITHDRAWAL, INDENT_BALANCE), range_skip = (INDENT_BALANCE, 9999)) if desc is None or date is None or amount is None: return startPos, None else: date = closest_date(date, date_from, date_to) trans = Transaction(u'') trans.date = date trans.rdate = date trans.type = Transaction.TYPE_UNKNOWN trans.raw = desc trans.label = desc trans.amount = amount return pos, trans def read_multiline_desc(self, pos): startPos = pos descs = [] while True: prevPos = pos pos, layout = self.read_layout_tm(pos) pos, desc = self.read_text(pos) if layout is None or desc is None: pos = prevPos break else: descs.append(desc) if descs: return pos, clean_label(' '.join(descs)) else: return startPos, None def read_indent_amount(self, pos, range_skip=(0,0), range_plus=(0,0), range_minus=(0,0)): startPos = pos # Read layout-amount pairs. amounts = [] while True: prevPos = pos pos, layout = self.read_layout_tm(pos) pos, amount = self.read_amount(pos) if layout is None or amount is None: pos = prevPos break else: amounts.append((layout, amount)) if not amounts: return startPos, None else: # Infer amount type by its indentation in the layout. amount_total = AmTr.decimal_amount('0') for (_, _, _, _, indent, _), amount in amounts: within = lambda xmin_xmax: xmin_xmax[0] <= indent <= xmin_xmax[1] if within(range_skip): continue elif within(range_plus): amount_total += amount elif within(range_minus): amount_total -= amount return pos, amount_total def read_star(self, pos): pos1, star1 = self.read_star_1(pos) pos2, star2 = self.read_star_2(pos) if star1 is not None: return pos1, star1 else: return pos2, star2 def read_star_1(self, pos): startPos = pos vals = list() pos, v = self.read_layout_tz(pos); vals.append(v) pos, v = self.read_layout_tc(pos); vals.append(v) pos, v = self.read_layout_tw(pos); vals.append(v) pos, v = self.read_layout_tf(pos); vals.append(v) pos, v = self.read_layout_tm(pos); vals.append(v) pos, star = self.read_text(pos) pos, v = self.read_layout_tz(pos); vals.append(v) pos, v = self.read_layout_tc(pos); vals.append(v) pos, v = self.read_layout_tw(pos); vals.append(v) pos, v = self.read_layout_tf(pos); vals.append(v) if star == 'S' and None not in vals: return pos, star else: return startPos, None def read_star_2(self, pos): startPos = pos vals = list() pos, v = self.read_layout_tf(pos); vals.append(v) pos, v = self.read_layout_tm(pos); vals.append(v) pos, star = self.read_text(pos) pos, v = self.read_layout_tf(pos); vals.append(v) if star == 'S' and None not in vals: return pos, star else: return startPos, None def read_date(self, pos): def parse_date(v): for year in [1900, 1904]: # try leap and non-leap years fullstr = '%s/%i' % (v, year) try: return datetime.datetime.strptime(fullstr, '%m/%d/%Y') except ValueError as e: pass raise e return self._tok.simple_read('date', pos, parse_date) def read_text(self, pos): t = self._tok.tok(pos) #TODO: handle PDF encodings properly. return (pos+1, unicode(t.value(), errors='ignore')) \ if t.is_text() else (pos, None) def read_amount(self, pos): t = self._tok.tok(pos) return (pos+1, AmTr.decimal_amount(t.value())) \ if t.is_amount() else (pos, None) def read_date_range(self, pos): t = self._tok.tok(pos) if t.is_date_range_1(): return (pos+1, [datetime.datetime.strptime(v, '%B %d, %Y') for v in t.value()]) elif t.is_date_range_2(): return (pos+1, [datetime.datetime.strptime(v, '%m/%d/%Y') for v in t.value()]) else: return (pos, None) def read_ref(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_ref() else (pos, None) def read_layout_tz(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_layout_tz() else (pos, None) def read_layout_tc(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_layout_tc() else (pos, None) def read_layout_tw(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_layout_tw() else (pos, None) def read_layout_tf(self, pos): t = self._tok.tok(pos) return (pos+1, t.value()) if t.is_layout_tf() else (pos, None) def read_layout_tm(self, pos): t = self._tok.tok(pos) return (pos+1, [float(v) for v in t.value()]) \ if t.is_layout_tm() else (pos, None) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wellsfargo/test.py���������������������������������������������������������������0000664�0000000�0000000�00000002202�13034501105�0017763�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from itertools import chain class WellsFargoTest(BackendTest): MODULE = 'wellsfargo' def test_history(self): """ Test that there's at least one transaction in the whole history. """ b = self.backend ts = chain(*[b.iter_history(a) for a in b.iter_accounts()]) t = next(ts, None) self.assertNotEqual(t, None) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017123�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000001503�13034501105�0021233�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������"WordReferenceModule init" # -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import WordReferenceModule __all__ = ['WordReferenceModule'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/browser.py���������������������������������������������������������0000664�0000000�0000000�00000002546�13034501105�0021167�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import TranslatePage __all__ = ['WordReferenceBrowser'] class WordReferenceBrowser(PagesBrowser): BASEURL = 'http://www.wordreference.com' translation_page = URL('(?P<sl>[a-z]{2})(?P<tl>[a-z]{2})/(?P<pattern>.*)', TranslatePage) def translate(self, source, to, text): """ translate 'text' from 'source' language to 'to' language """ return self.translation_page.go(sl=source.encode('utf-8'), tl=to.encode('utf-8'), pattern=text.encode('utf-8')).get_translation() ����������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/favicon.png��������������������������������������������������������0000664�0000000�0000000�00000004110�13034501105�0021252�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB����bKGD������ pHYs�� �� B(x���tIME  sC2���iTXtComment�����Created with GIMPd.e��IDATx}e?Wr'H̕@m(-&-`btmD !cTL'PC`k(PǦԥml I]yIx rEԡwr=gannyk$_y>gC:)s_( [r�?fwO�Q~&�p8& ܝ'7�K,.񣁏 �U;˯^r,r,+͂ѸfZl3�6 h~~4wA<b./D;0 �!{ }jΡVMrE]a7� K) + irEgl ��LN;wkDl�X~7 X׎c?HKϯsy�T<AGͭk@t�i^�TShFF?�%F<bt �*F{)|w?gUѭ]l' S\g/$?e5NepS.F! �״scp�OG] -hDŎTsO*fe5FE(rwbR]�إ_2 O\O'{T4j_<SnIJ e(,k"؅� �U\ (7֘)b5,;58>_q ˉ4fJ '#)cmNx>n�}7cA>M`%QV�d1܉E:3X<H1*ZUO+ $*d_(n�," pQ?+jMIbys 6)âF6-Bn)<59/֭6aP׊[uWHFqv]ڪiM0Y)o厫<Wu{Y0T %@֙Ib({ ԕƪ#$M20Ը/o� h3rhmfZ g7nJjMUz{։d2L`"@xۺn;4߰X Xf:�U{P| 򽩻;3;v'D۩[҉dRKOOӾCjc/M<{swh4eefX[[ʌ'̯-55YDh7kEh޹5-뀷M6>@U SٿLFɺ_EQnIWE_yrL.*'#$%/ʺ: `{\ӿ6ԗ(+�&;~(pw�2xn'4%}A$ {`u K/ 퉱KMPe/OѷCO7+*ہnAt?\jAeWh <Q[ZJEjʭ@`J zj�Gpo)pD&l۫?0;nˁm'66@lh/?4lP.$NlFz/S�@)p?S !gU,^i|GԾ4-/A&JZ&FTWET9;~Tqk 2�dmǁK1[7IG8ӄ [nXv;~t^JהffCE,wwQ1&}8Oz`In>n/S© li38,׉!& }ل~#QeYTCxS:w(o &U$-sZD$bscR 6 enl?^(\}vS良�+%M L>z <] =+loCP:ԡuCg(TB����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/module.py����������������������������������������������������������0000664�0000000�0000000�00000004061�13034501105�0020763�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. "backend for http://www.wordreference.com" from weboob.capabilities.translate import CapTranslate, TranslationFail, LanguageNotSupported from weboob.tools.backend import Module from .browser import WordReferenceBrowser __all__ = ['WordReferenceModule'] class WordReferenceModule(Module, CapTranslate): MAINTAINER = u'Lucien Loiseau' EMAIL = 'loiseau.lucien@gmail.com' VERSION = '1.2' LICENSE = 'AGPLv3+' NAME = 'wordreference' DESCRIPTION = u'Free online translator' BROWSER = WordReferenceBrowser WRLANGUAGE = { 'Arabic': 'ar', 'Chinese': 'zh', 'Czech': 'cz', 'English': 'en', 'French': 'fr', 'Greek': 'gr', 'Italian': 'it', 'Japanese': 'ja', 'Korean': 'ko', 'Polish': 'pl', 'Portuguese': 'pt', 'Romanian': 'ro', 'Spanish': 'es', 'Turkish': 'tr', } def translate(self, lan_from, lan_to, text): if lan_from not in self.WRLANGUAGE.keys(): raise LanguageNotSupported() if lan_to not in self.WRLANGUAGE.keys(): raise LanguageNotSupported() translations = self.browser.translate(self.WRLANGUAGE[lan_from], self.WRLANGUAGE[lan_to], text) has_translation = False for translation in translations: has_translation = True yield translation if not has_translation: raise TranslationFail() �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/pages.py�����������������������������������������������������������0000664�0000000�0000000�00000002553�13034501105�0020601�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.capabilities.translate import Translation from weboob.browser.filters.standard import CleanText, Regexp, Env class TranslatePage(HTMLPage): @method class get_translation(ListElement): item_xpath = '//table[@class="WRD" and not(@id)]/tr[@id]' class item(ItemElement): klass = Translation obj_id = Regexp(CleanText('./@id'), '.*:(.*)') obj_lang_src = Env('sl') obj_lang_dst = Env('tl') obj_text = CleanText('./td[@class="ToWrd"]', children=False) �����������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/wordreference/test.py������������������������������������������������������������0000664�0000000�0000000�00000001736�13034501105�0020463�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- CODing: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class WordReferenceTest(BackendTest): MODULE = 'wordreference' def test_translate(self): tr = list(self.backend.translate('French', 'English', 'chat'))[0] self.assertEqual(tr.text, u'cat') ����������������������������������weboob-1.2/modules/yahoo/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015410�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yahoo/__init__.py����������������������������������������������������������������0000664�0000000�0000000�00000001433�13034501105�0017522�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import YahooModule __all__ = ['YahooModule'] �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yahoo/browser.py�����������������������������������������������������������������0000664�0000000�0000000�00000003253�13034501105�0017450�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Cedric Defortis # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from .pages import YahooPage __all__ = ['YahooBrowser'] class YahooBrowser(PagesBrowser): BASEURL = 'https://query.yahooapis.com' yahoo = URL('/v1/public/yql\?q=(?P<query>.*)&format=json', YahooPage) def __init__(self, unit, *args, **kwargs): self.unit = unit PagesBrowser.__init__(self, *args, **kwargs) def iter_city_search(self, pattern): query = 'select name, country, woeid, admin1 from geo.places where text="%s"' % pattern.encode('utf-8') return self.yahoo.go(query=query).iter_cities() def iter_forecast(self, city): query = 'select * from weather.forecast where woeid = %s and u="%s"' % (city, self.unit) return self.yahoo.go(query=query).iter_forecast() def get_current(self, city): query = 'select * from weather.forecast where woeid = %s and u="%s"' % (city, self.unit) return self.yahoo.go(query=query).get_current() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yahoo/favicon.png����������������������������������������������������������������0000664�0000000�0000000�00000000562�13034501105�0017546�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME(Ԯ\���tEXtComment�Created with GIMPW���IDATx 07y@)e2R{7ɧɬ#B<q.טqzȥr.[` {P0��T�0��@0��T�0��c�1Ʊ%poŀ[�5�(+#H�`� T P xUҩ)�1|Sl:d P |q֊_nw�����������!e<c,!yF����IENDB`����������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yahoo/module.py������������������������������������������������������������������0000664�0000000�0000000�00000003447�13034501105�0017257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.weather import CapWeather from weboob.tools.backend import Module, BackendConfig from weboob.tools.ordereddict import OrderedDict from weboob.tools.value import Value from .browser import YahooBrowser __all__ = ['YahooModule'] class YahooModule(Module, CapWeather): NAME = 'yahoo' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'Yahoo! Weather.' LICENSE = 'AGPLv3+' BROWSER = YahooBrowser units_choice = OrderedDict([('c', 'International System of Units'), ('f', 'U.S. System of Units')]) CONFIG = BackendConfig(Value('units', label=u'System of Units', choices=units_choice)) def create_default_browser(self): return self.create_browser(unit=self.config['units'].get()) def iter_city_search(self, pattern): return self.browser.iter_city_search(pattern) def get_current(self, city_id): return self.browser.get_current(city_id) def iter_forecast(self, city_id): return self.browser.iter_forecast(city_id) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yahoo/pages.py�������������������������������������������������������������������0000664�0000000�0000000�00000005713�13034501105�0017067�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������ # -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Cedric Defortis # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import JsonPage from weboob.browser.elements import ItemElement, DictElement, method from weboob.capabilities.weather import Forecast, Current, City, Temperature from weboob.browser.filters.json import Dict from weboob.browser.filters.standard import CleanText, CleanDecimal, Format, Date, Env class YahooPage(JsonPage): @method class iter_cities(DictElement): item_xpath = 'query/results/place' class item(ItemElement): klass = City obj_id = Dict('woeid') obj_name = Format(u'%s, %s, %s', Dict('name'), Dict('admin1/content'), Dict('country/content')) @method class get_current(ItemElement): klass = Current def parse(self, el): self.env['pct'] = u'%' obj_id = Date(Dict('query/results/channel/item/condition/date')) obj_date = Date(Dict('query/results/channel/item/condition/date')) obj_text = Format('%s - wind: %s%s - humidity:%s%s', Dict('query/results/channel/item/condition/text'), Dict('query/results/channel/wind/speed'), Dict('query/results/channel/units/speed'), Dict('query/results/channel/atmosphere/humidity'), Env('pct')) def obj_temp(self): temp = CleanDecimal(Dict('query/results/channel/item/condition/temp'))(self) unit = CleanText(Dict('query/results/channel/units/temperature'))(self) return Temperature(float(temp), unit) @method class iter_forecast(DictElement): item_xpath = 'query/results/channel/item/forecast' def parse(self, el): self.env['unit'] = Dict('query/results/channel/units/temperature')(el) class item(ItemElement): klass = Forecast obj_id = Dict('date') obj_date = Date(Dict('date')) obj_text = Dict('text') def obj_low(self): temp = CleanDecimal(Dict('low'))(self) return Temperature(float(temp), Env('unit')(self)) def obj_high(self): temp = CleanDecimal(Dict('high'))(self) return Temperature(float(temp), Env('unit')(self)) �����������������������������������������������������weboob-1.2/modules/yahoo/test.py��������������������������������������������������������������������0000664�0000000�0000000�00000002571�13034501105�0016746�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class YahooTest(BackendTest): MODULE = 'yahoo' def test_meteo(self): l = list(self.backend.iter_city_search('paris')) self.assertTrue(len(l) > 0) city = l[0] current = self.backend.get_current(city.id) self.assertTrue(current.temp.unit in ['C', 'F']) if current.temp.unit == 'F': self.assertTrue(current.temp.value > -4 and current.temp.value < 122) else: self.assertTrue(current.temp.value > -20 and current.temp.value < 50) forecasts = list(self.backend.iter_forecast(city.id)) self.assertTrue(len(forecasts) > 0) ���������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015603�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/__init__.py���������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0017713�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import YomoniModule __all__ = ['YomoniModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/browser.py����������������������������������������������������������������0000664�0000000�0000000�00000012654�13034501105�0017650�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from functools import wraps import json import re from weboob.browser.browsers import APIBrowser from weboob.browser.exceptions import ClientError from weboob.browser.filters.standard import CleanDecimal, Date from weboob.exceptions import BrowserIncorrectPassword from weboob.capabilities.bank import Account, Investment, Transaction from weboob.capabilities.base import NotAvailable def need_login(func): @wraps(func) def wrapper(self, *args, **kwargs): if self.users is None: self.do_login() return func(self, *args, **kwargs) return wrapper class YomoniBrowser(APIBrowser): BASEURL = 'https://yomoni.fr' def __init__(self, username, password, *args, **kwargs): super(YomoniBrowser, self).__init__(*args, **kwargs) self.username = username self.password = password self.users = None self.accounts = [] self.investments = {} self.histories = {} def build_request(self, *args, **kwargs): if 'data' in kwargs: kwargs['data'] = json.dumps(kwargs['data']) if 'headers' not in kwargs: kwargs['headers'] = {} kwargs['headers']['Content-Type'] = 'application/vnd.yomoni.v1+json; charset=UTF-8' return super(APIBrowser, self).build_request(*args, **kwargs) def do_login(self): self.open('auth/init') data = { 'username': self.username, 'password': self.password.encode('base64').strip(), } try: self.users = self.request('auth/login', data=data) except ClientError: raise BrowserIncorrectPassword() @need_login def iter_accounts(self): if self.accounts: for account in self.accounts: yield account return for project in self.users['projects']: me = self.request('/user/%s/project/%s/' % (self.users['userId'], project['projectId'])) # Check project in progress if not me['numeroContrat'] or not me['dateAdhesion']: continue a = Account() a.id = "".join(me['numeroContrat'].split()) a.label = " ".join(me['supportEpargne'].split("_")) a.type = Account.TYPE_LIFE_INSURANCE if "assurance vie" in a.label.lower() else \ Account.TYPE_MARKET if "compte titre" in a.label.lower() else \ Account.TYPE_UNKNOWN a.balance = CleanDecimal().filter(me['solde']) a.iban = me['ibancompteTitre'] or NotAvailable a.number = project['projectId'] a.valuation_diff = CleanDecimal().filter(me['performanceEuro']) a._startbalance = me['montantDepart'] self.accounts.append(a) self.iter_investment(a, me['sousJacents']) yield a @need_login def iter_investment(self, account, invs=None): if account not in self.investments and invs is not None: self.investments[account] = [] for inv in invs: i = Investment() i.label = "%s - %s" % (inv['classification'], inv['description']) i.code = inv['isin'] i.quantity = CleanDecimal().filter(inv['nombreParts']) i.unitprice = CleanDecimal().filter(inv['prixMoyenAchat']) i.unitvalue = CleanDecimal().filter(inv['valeurCotation']) i.valuation = CleanDecimal().filter(inv['montantEuro']) i.vdate = Date().filter(inv['datePosition']) i.diff = CleanDecimal().filter(inv['performanceEuro']) self.investments[account].append(i) return self.investments[account] @need_login def iter_history(self, account): if account not in self.histories: histories = [] for activity in [acc for acc in self.request('/user/%s/project/%s/activity' % (self.users['userId'], account.number))['activities'] \ if acc['details'] is not None]: m = re.search(u'([\d\.]+)(?=[\s]+€|[\s]+euro)', activity['details']) if "Souscription" not in activity['title'] and not m: continue t = Transaction() t.label = "%s - %s" % (" ".join(activity['type'].split("_")), activity['title']) t.date = Date().filter(activity['date']) t.type = Transaction.TYPE_BANK amount = account._startbalance if not m else "-%s" % m.group(1) if "FRAIS" in activity['type'] else m.group(1) t.amount = CleanDecimal().filter(amount) histories.append(t) self.histories[account] = histories return self.histories[account] ������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/favicon.png���������������������������������������������������������������0000664�0000000�0000000�00000010043�13034501105�0017734�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���bKGD������C��� pHYs�� �� ����tIME '+J���iTXtComment�����Created with GIMPd.e��IDATxɏuV՛{h(( *,dm�?del8F LIqPwljdqu(d(5 }oU-ajTWMߙ ~] Z@x" AicwnpֿfɤpzH*͆:bPë7D@L amDӓMX:%ՙe ªAUx *ՙecSw7a� ֐R1Ay^Ag1:a!&�X_UTA˟^ѡ6!ӗA#ٺ҉L�UpNUbc^ S0!2P }Ԧ#J<@ Bxҍ/RgCVf"ND8ǽ~NwTJd +g+|BWj�wRm%z˱ևdzTSꕀU{_[4Џ a8u<%�OU MuɅ & sa '(fpLgkԏ\3:_]QX}vwbP Zl-ӣfkZ@h(@J 0O5nX�KSH2nݭnu+Ca;):i嫀!20HP(caU弿:܈e3% !+J3_ h NEX3n>/sK N[8S޽a!ZF:TxґVv{+29u0-ݱ0rB$^bhO)XWvb䨺sgžjPKUU 0N)ؒWP*PN GB+Jo^lpv�N 2#>\*\W+VB7:!.d?T#5:[=�͚^5|6"3A/r ;5NpAunpn_5˟_bٺ?pz6b-6PK5B J+ *Rv �` DW f#:T+> suC/P#@(5eD ЀSed0ZY| Fl=<`dTzB#d4-ߤZNҲJ@|Eb7%sPlr=7"][9޸82;*xtǶ>B/W 0h/^wf/fpTŦ|gTpo3^N9(ݼ ;0SJegNϧ�8iU@Ļ76;Υ&gID|AYά ԗ9`hF~\h'ͱ0|И/ǨPΓ~l-f77b6zYAp@a^ aIA0̄$=y|2ǓvFXP6̑YD9a8&ǪOuw'"~f:@JJ,r/e'F:$nU[bKC4) ͚AD<whY;@s)N'2tHL^ˍhBVf#gCg"@w\/f bJ-Fʤ;e⨺rM#*6BR(\P W|,oٹ*a΍c޽F8vdWeX;4VaOv,c^h(sUҌc樗bh [au`覥kl�Ԃ:_{oMZ@?)kl `3!U-Sur4C-Bc \H *Jq8UW . 2A3Ĺ`_TUUU7K-v#Fĕ:vٟmrڣHVE97B 4BLSDJn(ʹXk&Y)K58y)#=G^8J(Pa X|JhpT`&#ewx"yVzDRGD!DЌ|q�@LO/s*bн%~r {֩`ϛ<ewT5ȜQj s߽g B}'{oBlU) aT*$ tU>]e1]񨝒@xpmG A(aba'RUN"lŞ@`'Xd7$o|?X؍=p"5ߏr-'/J꿐ʣNǏb.. sL1Bw\pF| Pu\ @ȿ('a36: !J-b\۱`i <"MT]N6xmju+o*SFc˝G1CFnvJgX`�dVO \l7C°3V(80̄>+<ɴf&.?5 j:U̬kgR-ZVwR"Bg\0LIpNBba\=`O{n V!nԂ ȟpWĚ `sU�/ՙM} 6͗0CA\!KGg)$騐w?8U7)~v_|{g[ͺnq5`BpEvzݸMKmv yKMέ#-`ʯԟZ{bɁ>g';M]7nq>C)〡Yxj6!R⊄Ӡ2W,rvr78s֍pgv} nX7DMw=LTYoIXkbBovgk>? ֪ȍwg(n?w!�/kHlšf#Tg%^r:Snh:؝O޷7{] /=�)~5 6/ҰR#| Ԧ1h׍w?7Z<|K>L#; {CnSz!fWƖowb9 +&<Z\:uݾwtnpİeԲeuĘ(|a̍HG~(&!ͼɣ%=QEuꊄ<nk膛,Pw =b&;ꎜHseK&.!+51ߍ/buݽ+׾#{]7xk{xrr 4x'xw׿?N]!u+hF^{W9>~|/lYHq@Χݺaqk^ʃ ґhq{UnuWohXjZ/IufHpPū>C}/ M7\w~Iww5>Ѹ?vןN{e9ǭ_1Ko]?2Ko\ /}Uj_!.- *b'9&W:Wd.Cl֤vW?qwok~�:^'C 5Ě\w7nD5ޔi,fgKX 0ft[ZV~':T];#Mw fgix޶9ןN1Vv,ihjg%/3.&;l+byvIm{vbp=or'ĿHMMz]& kR{]KLsDY�kIfU]"}IMpE d/xzٜ"s~Ӭ\zxY|U*%�͆m߻o}\<~;g+V*����IENDB`���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/module.py�����������������������������������������������������������������0000664�0000000�0000000�00000003634�13034501105�0017450�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.backend import Module, BackendConfig from weboob.tools.value import ValueBackendPassword from weboob.capabilities.bank import CapBank, AccountNotFound from weboob.capabilities.base import find_object from .browser import YomoniBrowser __all__ = ['YomoniModule'] class YomoniModule(Module, CapBank): NAME = 'yomoni' DESCRIPTION = u'Yomoni' MAINTAINER = u'Edouard Lambert' EMAIL = 'elambert@budget-insight.com' LICENSE = 'AGPLv3+' VERSION = '1.2' CONFIG = BackendConfig( ValueBackendPassword('login', label='Adresse email', masked=False), ValueBackendPassword('password', label='Mot de passe')) BROWSER = YomoniBrowser def create_default_browser(self): return self.create_browser(self.config['login'].get(), self.config['password'].get()) def get_account(self, _id): return find_object(self.browser.iter_accounts(), id=_id, error=AccountNotFound) def iter_accounts(self): return self.browser.iter_accounts() def iter_history(self, account): return self.browser.iter_history(account) def iter_investment(self, account): return self.browser.iter_investment(account) ����������������������������������������������������������������������������������������������������weboob-1.2/modules/yomoni/test.py�������������������������������������������������������������������0000664�0000000�0000000�00000001503�13034501105�0017133�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2016 Edouard Lambert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest class YomoniTest(BackendTest): MODULE = 'yomoni' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016014�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020124�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import YoujizzModule __all__ = ['YoujizzModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003456�13034501105�0020061�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.capabilities.base import UserError from .pages.index import IndexPage from .pages.video import VideoPage __all__ = ['YoujizzBrowser'] class YoujizzBrowser(PagesBrowser): BASEURL = 'https://www.youjizz.com' index = URL(r'/?(index.php)?$', r'/page/\d+.html', IndexPage) search = URL(r'/search/(?P<pattern>.+)-(?P<pagenum>\d+).html', IndexPage) video = URL(r'/videos/(?P<id>.*).html', VideoPage) @video.id2url def get_video(self, url, video=None): self.location(url) assert self.video.is_here() return self.page.get_video(video) def search_videos(self, pattern): if len(pattern) < 3: raise UserError('Pattern to short (min length : 3 characters)') self.search.go(pattern=pattern, pagenum=1) assert self.search.is_here(pattern=pattern, pagenum=1) return self.page.iter_videos() def latest_videos(self): self.index.go() assert self.index.is_here() return self.page.iter_videos() ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000006164�13034501105�0020156�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME -i�� IDATx]l[>c'M|M(!!ka#㣄 $h (EBH `Lf傋EUҤ!MH6X+R2Z6)K&M{' ;'c;Yd|}yW]g~]]j@ [;K2s!s?}83S- �fC-h"@5fmy X,ex ~E`tum~+/;`bTUqTQ<&0jKdR( XUU4ݮ醑JR] dҧ뺠=sIS �I֬lZO߿o!� d>yrp|Ju0 �!I2YDѭz\8@Jݪʂ(ɤ[uO}.c713$Β�08phTD߯z;7m] un3�H1(+0�%)0E�ɹ&u &t'`rǎY_E0is`YNǁ @Za9MR\j-Mx@vknmHO1MץQEm��DqIE+zWZ3l965屼b d2ùܮW0f4TV^47!^p$R)`y3= Zv[fW]zok1�~cJ%p `[V(u@,%aN}Ʋ‚/Ȫ_ }L  :) ϳOyϹKjI}g'eB'^׉֙KhMw#p&jMl'a?nU(re%89hŋWlp N\O0t%?0:<JV[l| 0?顡,--\kjz=F>[n茶H#~dfj�8Ȳ ̹{ٰc$qb~ݛ&XJJ٧XjOp|T|lڽ-'/]w _1-$dx*9G]Eadd$}#<5:ʶ>bKSUEöm ʨHQ0TMK/ӧN͞=<&! "--yYдep ϲH<.qܵ\ +߼*~Z dC\6�a]fNJ($k9 ko#?N2N?P8Ctb U*?TZ�9 -#G=wođ#~ܵ }h9dK/w @(| @k)Ce/)ooC8ݍ G2;56>$wڻG9CjfzlDlSɛjV_ӃtSZ@577طT0,�r4 oHA?gfhh6lڽ{YYF+,􄣿?` FM$LcPIRY<Ζ ɋe r ߽ ^/ 뛗! T7-$LË^MO=20KsϱqNR0_JH)Nfb1bbh3,C\};Ux˸jj�[\`JJcRcC9ATUĶO>㲔VUف) bR ~m$P([qMÖ-K jjl cyt[ !t]'O3nu]�,Q"gļ>_\Ϣ9,'p{Șa?+ݱH{@NL4wzI']haX1�NYfY.1ݮ^bz@/]],E¸jjhlᬨYQAbz'<~ hmSWL bt) 1/\^f7ș 쒤2wQ;:套Fl=R\s |c:a%J'qj<Y^Npp0 8*f`tJIc1@0H$嚛VWW#�b $H\Vţ3ח/�3K2�=j:t.{׭㮷ʫ&S>E;J6U"R::pϩ Ұ-$Ifʶ6ΦcH eS N@�h[_[`as}}}u݆(f/l ޹ZV^(r,CUA0?e<8'Z �'2YVc)J}}oƝ;iݾ|7p8dxՉĜa TKU<Lrǘ ;$3&.7aժX0dX8:wov:,CL]wߍw:OVɲիv,#SȔ;"hmmIK?^B箮0q𚋉[]>[{ %8+*cAUU͆iahj"A"޽{|555gnom [? ~J*�-e ? #hr//]-r;o |x~Skkrxj/.0#禧 1 n3Q_Y9۵aEY0)ec?9\ Y%%%%vǓII7 &(H)/4l*lUH ȑx\:39 b(ꪦ a 6Ql6avIҜ&TՖL$fӽ.W]8hVaQ3\I*Ҋa kSED歕-cnf? �53yw`�aSU:J|4M uw][����IENDB`������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/module.py����������������������������������������������������������������0000664�0000000�0000000�00000005210�13034501105�0017651�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module from .browser import YoujizzBrowser __all__ = ['YoujizzModule'] class YoujizzModule(Module, CapVideo, CapCollection): NAME = 'youjizz' MAINTAINER = u'Roger Philibert' EMAIL = 'roger.philibert@gmail.com' VERSION = '1.2' DESCRIPTION = 'YouJizz pornographic video streaming website' LICENSE = 'AGPLv3+' BROWSER = YoujizzBrowser def get_video(self, _id): video = self.browser.get_video(_id) return video def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): if not nsfw: return set() return self.browser.search_videos(pattern) def fill_video(self, video, fields): if fields != ['thumbnail']: # if we don't want only the thumbnail, we probably want also every fields video = self.browser.get_video(video.id, video) if 'thumbnail' in fields and video.thumbnail: video.thumbnail.data = self.browser.open(video.thumbnail.url).content return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest_nsfw']) if collection.split_path == [u'latest_nsfw']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest_nsfw']: collection.title = u'Latest Youjizz videos (NSFW)' return raise CollectionNotFound(collection.split_path) OBJECTS = {BaseVideo: fill_video} ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/pages/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017113�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/pages/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021212�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/pages/index.py�����������������������������������������������������������0000664�0000000�0000000�00000003461�13034501105�0020600�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage, pagination from weboob.browser.elements import ListElement, ItemElement, method from weboob.browser.filters.standard import CleanText, Duration, Regexp from weboob.browser.filters.html import Link, CSS from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import BaseVideo class IndexPage(HTMLPage): @pagination @method class iter_videos(ListElement): item_xpath = '//span[@id="miniatura"]' next_page = Link(u'//a[text()="Next »"]') class item(ItemElement): klass = BaseVideo obj_id = CSS('a') & Link & Regexp(pattern=r'/videos/(.+)\.html') obj_title = CSS('span#title1') & CleanText obj_duration = CSS('span.thumbtime span') & CleanText & Duration | NotAvailable obj_nsfw = True def obj_thumbnail(self): thumbnail = Thumbnail(self.xpath('.//img')[0].attrib['data-original']) thumbnail.url = thumbnail.id return thumbnail ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/pages/video.py�����������������������������������������������������������0000664�0000000�0000000�00000003534�13034501105�0020600�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, method from weboob.browser.filters.standard import CleanText, Env, Duration from weboob.capabilities.video import BaseVideo from weboob.tools.misc import to_unicode class VideoPage(HTMLPage): @method class get_video(ItemElement): klass = BaseVideo obj_id = Env('id') obj_title = CleanText('//title') obj_nsfw = True obj_ext = u'flv' obj_duration = CleanText('//div[@id="video_text"]') & Duration def obj_url(self): real_id = int(self.env['id'].split('-')[-1]) response = self.page.browser.open('https://www.youjizz.com/videos/embed/%s' % real_id) data = response.text video_file_urls = re.findall(r'"(https?://[^",]+\.(?:flv|mp4)(?:\?[^"]*)?)"', data) if len(video_file_urls) == 0: raise ValueError('Video URL not found') elif len(video_file_urls) > 1: raise ValueError('Many video file URL found') else: return to_unicode(video_file_urls[0]) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youjizz/test.py������������������������������������������������������������������0000664�0000000�0000000�00000003323�13034501105�0017346�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.misc import limit from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class YoujizzTest(BackendTest): MODULE = 'youjizz' def test_search(self): self.assertTrue(len(self.backend.search_videos('anus', nsfw=False)) == 0) l = list(limit(self.backend.search_videos('anus', nsfw=True), 100)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) r = self.backend.browser.open(v.url, stream=True) self.assertTrue(r.status_code == 200) def test_latest(self): l = list(limit(self.backend.iter_resources([BaseVideo], [u'latest_nsfw']), 100)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016004�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000000077�13034501105�0020121�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������from .module import YoupornModule __all__ = ['YoupornModule'] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000003226�13034501105�0020044�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser import PagesBrowser, URL from weboob.capabilities.base import UserError from .pages.index import IndexPage from .pages.video import VideoPage __all__ = ['YoupornBrowser'] class YoupornBrowser(PagesBrowser): BASEURL = 'http://www.youporn.com' home = URL('/$', IndexPage) search = URL('/search/\?query=(?P<query>.*)', IndexPage) video = URL('/watch/(?P<id>[0-9]+)/.*', VideoPage) def get_video(self, _id): self.video.go(id=_id) assert self.video.is_here() return self.page.get_video() def search_videos(self, pattern, sortby): if pattern == 'a' or pattern == 'i': raise UserError('this pattern is not supported'); self.search.go(query=pattern) assert self.search.is_here() return self.page.iter_videos() def latest_videos(self): self.home.go() assert self.home.is_here() return self.page.iter_videos() ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000002176�13034501105�0020145�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME5 ���tEXtComment�Created with GIMPW��IDATxOHQ?VDY)CObCFPiPIXuH nCA%I <NB%#2-u- q/,}H AEa0m@?`�c@rP p@+RO@0õU.`zh:{҆t?{:Upo,�qTCSs$Bӡ8S vԄ] Ɓ�{WJ?㤀>~S'PN,X#9!4 p>ZJW@gP $ӡ9Hx.*6YH �GnP#.Ue>1T<1S4,�4m/e6ߟ|ڽzMCǽ&!`&*TL8DZmU,\g_MR>]UB|TR6pe3%�=Z!Q x\p4FH gi; mnX Uͦc@}F_rRvj- >KjaY� ^˴~M3m, aQj Oo5N:+Y~롊M`Kݎ[S4 ӹ(Kgv)WtPB/hJܦ~:ƞ9QVG4Vs?5*5T/ Y]dPqϟ]i%[L[ n;{<x,4r J: P1XKE&xe6fC}T@ɵ+j:̮$Y }BʝEj( -#h3/{늨G6$d"`rL-QYVHB.&FsTXkRRV|]a'D )6jS`XH/j}z ႄ]ix 4�$H AA)����IENDB`��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/module.py����������������������������������������������������������������0000664�0000000�0000000�00000004662�13034501105�0017653�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import CapVideo, BaseVideo from weboob.tools.backend import Module from weboob.capabilities.collection import CapCollection, CollectionNotFound from .browser import YoupornBrowser from .video import YoupornVideo __all__ = ['YoupornModule'] class YoupornModule(Module, CapVideo, CapCollection): NAME = 'youporn' MAINTAINER = u'Romain Bignon' EMAIL = 'romain@weboob.org' VERSION = '1.2' DESCRIPTION = 'YouPorn pornographic video streaming website' LICENSE = 'AGPLv3+' BROWSER = YoupornBrowser def get_video(self, _id): return self.browser.get_video(_id) SORTBY = ['relevance', 'rating', 'views', 'time'] def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): if not nsfw: return set() return self.browser.search_videos(pattern, self.SORTBY[sortby]) def fill_video(self, video, fields): return self.browser.get_video(video.id) def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest_nsfw']) if collection.split_path == [u'latest_nsfw']: for video in self.browser.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest_nsfw']: collection.title = u'Latest YouPorn videos (NSFW)' return raise CollectionNotFound(collection.split_path) OBJECTS = {YoupornVideo: fill_video} ������������������������������������������������������������������������������weboob-1.2/modules/youporn/pages/�������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017103�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/pages/__init__.py��������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0021202�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/pages/index.py�����������������������������������������������������������0000664�0000000�0000000�00000003566�13034501105�0020576�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, ListElement, method from weboob.browser.filters.html import Attr, CSS from weboob.browser.filters.standard import CleanText, Duration, Regexp, Type from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from ..video import YoupornVideo class IndexPage(HTMLPage): @method class iter_videos(ListElement): item_xpath = '//div[@id="content"]/div/div/ul/li/div/a' class item(ItemElement): klass = YoupornVideo def obj_thumbnail(self): thumbnail_url = Attr('./img', 'src')(self) thumbnail = Thumbnail(thumbnail_url) thumbnail.url = thumbnail.id return thumbnail obj_author = NotAvailable obj_duration = CSS('span.duration') & CleanText() & Duration() obj_id = Attr('../..', 'data-video-id') obj_rating = CleanText('./span/i') & Regexp(pattern=r'(..)%') & Type(type=int) obj_rating_max = 100 obj_title = CleanText('./p') obj_url = NotAvailable ������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/pages/video.py�����������������������������������������������������������0000664�0000000�0000000�00000003174�13034501105�0020570�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.browser.pages import HTMLPage from weboob.browser.elements import ItemElement, method from weboob.browser.filters.html import Link from weboob.browser.filters.standard import CleanText, Env, Regexp, Type from weboob.capabilities.base import NotAvailable from ..video import YoupornVideo class VideoPage(HTMLPage): @method class get_video(ItemElement): klass = YoupornVideo obj_author = CleanText('//div[@class="author-block--line"][1]') & Regexp(pattern=r'By: (.*)') #obj_date = Date('//div[@id="stats-date"]') obj_duration = NotAvailable obj_ext = 'mp4' obj_id = Env('id') obj_rating = CleanText('//div[@class="rating-percentage"]') & Regexp(pattern=r'(..)%') & Type(type=int) obj_rating_max = 100 obj_thumbnail = NotAvailable obj_title = CleanText('//h1') obj_url = Link('//ul[@class="downloadList"]/li[2]/a') ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/test.py������������������������������������������������������������������0000664�0000000�0000000�00000003153�13034501105�0017337�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class YoupornTest(BackendTest): MODULE = 'youporn' def test_search(self): self.assertTrue(len(self.backend.search_videos('ass to mouth', nsfw=False)) == 0) l = list(self.backend.search_videos('ass to mouth', nsfw=True)) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) self.backend.browser.openurl(v.url) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest_nsfw'])) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youporn/video.py�����������������������������������������������������������������0000664�0000000�0000000�00000002156�13034501105�0017470�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Roger Philibert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class YoupornVideo(BaseVideo): def __init__(self, *args, **kwargs): BaseVideo.__init__(self, *args, **kwargs) self.nsfw = True self.ext = u'flv' @classmethod def id2url(cls, _id): if _id.isdigit(): return u'http://www.youporn.com/watch/%d' % int(_id) else: return None ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0015765�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/__init__.py��������������������������������������������������������������0000664�0000000�0000000�00000001440�13034501105�0020075�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .module import YoutubeModule __all__ = ['YoutubeModule'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/browser.py���������������������������������������������������������������0000664�0000000�0000000�00000004435�13034501105�0020030�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.deprecated.browser import Browser from .pages import BaseYoutubePage, VideoPage, ForbiddenVideoPage, \ VerifyAgePage, VerifyControversyPage, \ LoginPage, LoginRedirectPage __all__ = ['YoutubeBrowser'] class YoutubeBrowser(Browser): DOMAIN = u'youtube.com' ENCODING = 'utf-8' PAGES = {r'https?://.*youtube\.com/': BaseYoutubePage, r'https?://.*youtube\.com/watch\?v=(?P<id>.+)': VideoPage, r'https?://.*youtube\.com/index\?ytsession=.+': ForbiddenVideoPage, r'https?://.*youtube\.com/verify_age\?next_url=(?P<next_url>.+)': VerifyAgePage, r'https?://.*youtube\.com/verify_controversy\?next_url(?P<next_url>.+)': VerifyControversyPage, r'https?://accounts.google.com/ServiceLogin.*': LoginPage, r'https?://accounts.google.fr/accounts/SetSID.*': LoginRedirectPage, } def is_logged(self): logged = not self.is_on_page(BaseYoutubePage) or self.page.is_logged() return logged def login(self): self.location('https://accounts.google.com/ServiceLogin?uilel=3&service=youtube&passive=true&continue=https%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26nomobiletemp%3D1%26hl%3Den_US%26next%3D%252F&hl=en_US<mpl=sso') self.page.login(self.username, self.password) def get_video_url(self, video, player_url): self.location(player_url + '&has_verified=1&bpctr=9999999999') assert self.is_on_page(VideoPage) return self.page.get_video_url(video) �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/favicon.png��������������������������������������������������������������0000664�0000000�0000000�00000005451�13034501105�0020125�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������PNG  ��� IHDR���@���@���iq���sRGB���� pHYs�� �� ����tIME 4[���tEXtComment�Created with GIMPW�� IDATx[oKR%R,!˱M i..zhRžszl"@oR@^&H 8h&.RIa9DqZrggz,5HCr�˝~gx@?WB$"Y\Rz!�Yu= QT�AhS�L�'T�kճ,� =@.b �D�B�~ uKy /c ,W[b1Z,֛MKHɹqL(hX�0bo/ZLƚhrB…0~!H@Ř87;>�"k7oNBJY�keSAiB� 朧j}!{19O�@%ӑ� �^�?--[-'GٟO[z $�(Uj8q+4�ʅJխyP���쳩w\�%�ʣ$.}r7 ۈ p{�R)m�sI# �Ɔ;NߊC-ǙS�भ4B2o=#.!57B)em#.9p]pa |w>Bk}͍ 􀓘|!j919Yp. �E�.ca&}ASaa F6 #2kj 酅O?sj~xP0N�@= �m'Uo+T6> r=1?~ccc @JqA)az[zaïVŕ+.- 'C]DM]e=@hzj,sO=Ocl] 76b)LNNBR5,~rpH&SҌY�[>fg9@&Ʃ'nT1|>T G.]Jne]zෂT{&|c2 +uxixGb%u~/ m�ҫYj 9! \hl*V*yv: M�ȦRaw$pyyZ{~f~FLkF ATyD)X[YԶ&!mhjϒVױ|,N!D^G&+ wReڭ[pu!H>Ք.ztjN jos.t1 jƪ*2۶a,c {yJ@fOU�aq#G0tL5()%-Y%k9T*d+j 0f<ar^ԙQBH;#!3OTo5y�c;eL3 ="JF^�0l;Q0 C* -L/h^"I0d1։dĊAz=)h͎d.\~Զ[V u !l�*�K}LiqgLHj…>=.A`mOjm0�*Nvk&Lb]*$,TOAd$TWTA'n~5ABPWTl{TRs; xOA z3¤PR6x�)XATJ$;q>Coɢc_yGnE%Z@qR5B:iso }?ԼDh+Möcr"ɉjqN�Jqht{F\`7{f/�P@$[;9~ 8PU#z}D<h SSS�zExh='>3{`cǒ.HM*`3@C~ŰmT*B*K#)]+Hh4=j ,krc^R�Բ1M6= RU3RRT? 88bbt4`#`'0d:RH!}?BUx# �nfd$( `�FÍel Ɛ0b])D;/&?Λo wh|&3%"˸] Z8W%H ;"4zt�p�fǫq>X^ƭ^+W u r}իf:P�>ӎZ@gW_Lθ;oWؿ_ ֛sTK)w." xll l|Mn@h+J\_9}6[ZPc^S^z�^C�Q* իGQo[sԚڊQR*q$O;\lDd.ׯ֚\}S".*ZsSSk*ZbK%!wΜ٢ɸދ3G怱w-S/|I1=Ƒ<`o?e|E GF&&vmWSv6>2s=�~ܥˮˊjz.+V*ibBJbP*)!B`6| pfG (!벵\Ѱ$@ J}"$iP)%1GM΍1fbԶ_9==]�~}CSx <5�W ڻH|0 ` #rof-4Tw�M0b2b rDUn7g>o%b3P<%Oxa����IENDB`�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/module.py����������������������������������������������������������������0000664�0000000�0000000�00000014772�13034501105�0017637�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz, Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. import re import urllib from weboob.capabilities.base import NotAvailable from weboob.capabilities.image import Thumbnail from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.collection import CapCollection, CollectionNotFound from weboob.tools.backend import Module, BackendConfig from weboob.tools.misc import to_unicode from weboob.tools.value import ValueBackendPassword, Value from .browser import YoutubeBrowser from .video import YoutubeVideo try: # weboob modifies HTTPSConnection, which conflicts with apiclient # so apiclient must be imported after from apiclient.discovery import build as ytbuild except ImportError: raise ImportError("Please install python-googleapi") __all__ = ['YoutubeModule'] class YoutubeModule(Module, CapVideo, CapCollection): NAME = 'youtube' MAINTAINER = u'Laurent Bachelier' EMAIL = 'laurent@bachelier.name' VERSION = '1.2' DESCRIPTION = 'YouTube video streaming website' LICENSE = 'AGPLv3+' BROWSER = YoutubeBrowser CONFIG = BackendConfig(Value('username', label='Email address', default=''), ValueBackendPassword('password', label='Password', default='')) URL_RE = re.compile(r'^https?://(?:\w*\.?youtube(?:|-nocookie)\.com/(?:watch\?v=|embed/|v/)|youtu\.be\/|\w*\.?youtube\.com\/user\/\w+#p\/u\/\d+\/)([^\?&]+)') def create_default_browser(self): password = None username = self.config['username'].get() if len(username) > 0: password = self.config['password'].get() return self.create_browser(username, password) def _entry2video(self, entry): """ Parse an entry returned by googleapi and return a Video object. """ snippet = entry['snippet'] video = YoutubeVideo(to_unicode(entry['id']['videoId'])) video.title = to_unicode(snippet['title'].strip()) # duration does not seem to be available with api video.thumbnail = Thumbnail(snippet['thumbnails']['default']['url']) video.author = to_unicode(snippet['channelTitle'].strip()) return video def _set_video_url(self, video): """ In the case of a download, if the user-chosen format is not available, the next available format will be used. Much of the code for this method is borrowed from youtubeservice.py of Cutetube http://maemo.org/packages/view/cutetube/. """ if video.url: return player_url = YoutubeVideo.id2url(video.id) with self.browser: url, ext = self.browser.get_video_url(video, player_url) video.url = unicode(url) video.ext = unicode(ext) def get_video(self, _id): m = self.URL_RE.match(_id) if m: _id = m.group(1) params = {'id': _id, 'part': 'id,snippet'} youtube = self._build_yt() response = youtube.videos().list(**params).execute() items = response.get('items', []) if not items: return None video = self._entry2video(items[0]) self._set_video_url(video) video.set_empty_fields(NotAvailable) # Youtube video url is https, using ssl encryption # so we need to use the "play_proxy" method using urllib2 proxy streaming to handle this video._play_proxy = True return video def _build_yt(self): DEVELOPER_KEY = "AIzaSyApVVeZ03XkKDYHX8T5uOn8Eizfe9CMDbs" YOUTUBE_API_SERVICE_NAME = "youtube" YOUTUBE_API_VERSION = "v3" return ytbuild(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY) def search_videos(self, pattern, sortby=CapVideo.SEARCH_RELEVANCE, nsfw=False): YOUTUBE_MAX_RESULTS = 50 youtube = self._build_yt() params = {'part': 'id,snippet', 'maxResults': YOUTUBE_MAX_RESULTS} if pattern is not None: if isinstance(pattern, unicode): pattern = pattern.encode('utf-8') params['q'] = pattern params['safeSearch'] = 'none' if nsfw else 'strict' # or 'moderate' params['order'] = ('relevance', 'rating', 'viewCount', 'date')[sortby] nb_yielded = 0 while True: search_response = youtube.search().list(**params).execute() items = search_response.get('items', []) for entry in items: if entry["id"]["kind"] != "youtube#video": continue yield self._entry2video(entry) nb_yielded += 1 params['pageToken'] = search_response.get('nextPageToken') if not params['pageToken']: return if nb_yielded < YOUTUBE_MAX_RESULTS: return def latest_videos(self): return self.search_videos(None, CapVideo.SEARCH_DATE) def fill_video(self, video, fields): if 'thumbnail' in fields: video.thumbnail.data = urllib.urlopen(video.thumbnail.url).read() if 'url' in fields: self._set_video_url(video) return video def iter_resources(self, objs, split_path): if BaseVideo in objs: collection = self.get_collection(objs, split_path) if collection.path_level == 0: yield self.get_collection(objs, [u'latest']) if collection.split_path == [u'latest']: for video in self.latest_videos(): yield video def validate_collection(self, objs, collection): if collection.path_level == 0: return if BaseVideo in objs and collection.split_path == [u'latest']: collection.title = u'Latest YouTube videos' return raise CollectionNotFound(collection.split_path) OBJECTS = {YoutubeVideo: fill_video} ������weboob-1.2/modules/youtube/pages.py�����������������������������������������������������������������0000664�0000000�0000000�00000145176�13034501105�0017454�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz, Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. # Some parts are taken from youtube-dl, licensed under the UNLICENSE. from urlparse import urlparse, parse_qs import codecs import zlib import re import os import string import struct import collections import traceback import urllib import io from weboob.capabilities.base import UserError from weboob.deprecated.browser import Page, BrokenPageError, BrowserIncorrectPassword from weboob.tools.json import json class LoginPage(Page): def on_loaded(self): errors = [] for errdiv in self.parser.select(self.document.getroot(), 'div.errormsg'): errors.append(errdiv.text.encode('utf-8').strip()) if len(errors) > 0: raise BrowserIncorrectPassword(', '.join(errors)) def login(self, username, password): self.browser.select_form(predicate=lambda form: form.attrs.get('id', '') == 'gaia_loginform') self.browser['Email'] = username.encode(self.browser.ENCODING) self.browser['Passwd'] = password.encode(self.browser.ENCODING) self.browser.submit() class LoginRedirectPage(Page): pass class ForbiddenVideo(UserError): pass class BaseYoutubePage(Page): def is_logged(self): try: self.parser.select(self.document.getroot(), 'span#yt-masthead-account-picker', 1) except BrokenPageError: return False else: return True class ForbiddenVideoPage(BaseYoutubePage): def on_loaded(self): element = self.parser.select(self.document.getroot(), '.yt-alert-content', 1) raise ForbiddenVideo(element.text.strip()) class VerifyAgePage(BaseYoutubePage): def on_loaded(self): if not self.is_logged(): raise ForbiddenVideo('This video or group may contain content that is inappropriate for some users') self.browser.select_form(predicate=lambda form: form.attrs.get('id', '') == 'confirm-age-form') self.browser.submit() class VerifyControversyPage(BaseYoutubePage): def on_loaded(self): self.browser.select_form(predicate=lambda form: 'verify_controversy' in form.attrs.get('action', '')) self.browser.submit() def determine_ext(url, default_ext=u'unknown_video'): guess = url.partition(u'?')[0].rpartition(u'.')[2] if re.match(r'^[A-Za-z0-9]+$', guess): return guess else: return default_ext def uppercase_escape(s): unicode_escape = codecs.getdecoder('unicode_escape') return re.sub( r'\\U[0-9a-fA-F]{8}', lambda m: unicode_escape(m.group(0))[0], s) _NO_DEFAULT = object() class VideoPage(BaseYoutubePage): _formats = { '5': {'ext': 'flv', 'width': 400, 'height': 240}, '6': {'ext': 'flv', 'width': 450, 'height': 270}, '13': {'ext': '3gp'}, '17': {'ext': '3gp', 'width': 176, 'height': 144}, '18': {'ext': 'mp4', 'width': 640, 'height': 360}, '22': {'ext': 'mp4', 'width': 1280, 'height': 720}, '34': {'ext': 'flv', 'width': 640, 'height': 360}, '35': {'ext': 'flv', 'width': 854, 'height': 480}, '36': {'ext': '3gp', 'width': 320, 'height': 240}, '37': {'ext': 'mp4', 'width': 1920, 'height': 1080}, '38': {'ext': 'mp4', 'width': 4096, 'height': 3072}, '43': {'ext': 'webm', 'width': 640, 'height': 360}, '44': {'ext': 'webm', 'width': 854, 'height': 480}, '45': {'ext': 'webm', 'width': 1280, 'height': 720}, '46': {'ext': 'webm', 'width': 1920, 'height': 1080}, # 3d videos '82': {'ext': 'mp4', 'height': 360, 'resolution': '360p', 'format_note': '3D', 'preference': -20}, '83': {'ext': 'mp4', 'height': 480, 'resolution': '480p', 'format_note': '3D', 'preference': -20}, '84': {'ext': 'mp4', 'height': 720, 'resolution': '720p', 'format_note': '3D', 'preference': -20}, '85': {'ext': 'mp4', 'height': 1080, 'resolution': '1080p', 'format_note': '3D', 'preference': -20}, '100': {'ext': 'webm', 'height': 360, 'resolution': '360p', 'format_note': '3D', 'preference': -20}, '101': {'ext': 'webm', 'height': 480, 'resolution': '480p', 'format_note': '3D', 'preference': -20}, '102': {'ext': 'webm', 'height': 720, 'resolution': '720p', 'format_note': '3D', 'preference': -20}, # Apple HTTP Live Streaming '92': {'ext': 'mp4', 'height': 240, 'resolution': '240p', 'format_note': 'HLS', 'preference': -10}, '93': {'ext': 'mp4', 'height': 360, 'resolution': '360p', 'format_note': 'HLS', 'preference': -10}, '94': {'ext': 'mp4', 'height': 480, 'resolution': '480p', 'format_note': 'HLS', 'preference': -10}, '95': {'ext': 'mp4', 'height': 720, 'resolution': '720p', 'format_note': 'HLS', 'preference': -10}, '96': {'ext': 'mp4', 'height': 1080, 'resolution': '1080p', 'format_note': 'HLS', 'preference': -10}, '132': {'ext': 'mp4', 'height': 240, 'resolution': '240p', 'format_note': 'HLS', 'preference': -10}, '151': {'ext': 'mp4', 'height': 72, 'resolution': '72p', 'format_note': 'HLS', 'preference': -10}, # DASH mp4 video '133': {'ext': 'mp4', 'height': 240, 'resolution': '240p', 'format_note': 'DASH video', 'preference': -40}, '134': {'ext': 'mp4', 'height': 360, 'resolution': '360p', 'format_note': 'DASH video', 'preference': -40}, '135': {'ext': 'mp4', 'height': 480, 'resolution': '480p', 'format_note': 'DASH video', 'preference': -40}, '136': {'ext': 'mp4', 'height': 720, 'resolution': '720p', 'format_note': 'DASH video', 'preference': -40}, '137': {'ext': 'mp4', 'height': 1080, 'resolution': '1080p', 'format_note': 'DASH video', 'preference': -40}, '138': {'ext': 'mp4', 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40}, # Height can vary (https://github.com/rg3/youtube-dl/issues/4559) '160': {'ext': 'mp4', 'height': 192, 'resolution': '192p', 'format_note': 'DASH video', 'preference': -40}, '264': {'ext': 'mp4', 'height': 1440, 'resolution': '1440p', 'format_note': 'DASH video', 'preference': -40}, '298': {'ext': 'mp4', 'height': 720, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'h264'}, '299': {'ext': 'mp4', 'height': 1080, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'h264'}, '266': {'ext': 'mp4', 'height': 2160, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'vcodec': 'h264'}, # Dash mp4 audio '139': {'ext': 'm4a', 'format_note': 'DASH audio', 'acodec': 'aac', 'vcodec': 'none', 'abr': 48, 'preference': -50, 'container': 'm4a_dash'}, '140': {'ext': 'm4a', 'format_note': 'DASH audio', 'acodec': 'aac', 'vcodec': 'none', 'abr': 128, 'preference': -50, 'container': 'm4a_dash'}, '141': {'ext': 'm4a', 'format_note': 'DASH audio', 'acodec': 'aac', 'vcodec': 'none', 'abr': 256, 'preference': -50, 'container': 'm4a_dash'}, # Dash webm '167': {'ext': 'webm', 'height': 360, 'width': 640, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '168': {'ext': 'webm', 'height': 480, 'width': 854, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '169': {'ext': 'webm', 'height': 720, 'width': 1280, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '170': {'ext': 'webm', 'height': 1080, 'width': 1920, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '218': {'ext': 'webm', 'height': 480, 'width': 854, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '219': {'ext': 'webm', 'height': 480, 'width': 854, 'format_note': 'DASH video', 'container': 'webm', 'vcodec': 'VP8', 'acodec': 'none', 'preference': -40}, '278': {'ext': 'webm', 'height': 144, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'container': 'webm', 'vcodec': 'VP9'}, '242': {'ext': 'webm', 'height': 240, 'resolution': '240p', 'format_note': 'DASH webm', 'preference': -40}, '243': {'ext': 'webm', 'height': 360, 'resolution': '360p', 'format_note': 'DASH webm', 'preference': -40}, '244': {'ext': 'webm', 'height': 480, 'resolution': '480p', 'format_note': 'DASH webm', 'preference': -40}, '245': {'ext': 'webm', 'height': 480, 'resolution': '480p', 'format_note': 'DASH webm', 'preference': -40}, '246': {'ext': 'webm', 'height': 480, 'resolution': '480p', 'format_note': 'DASH webm', 'preference': -40}, '247': {'ext': 'webm', 'height': 720, 'resolution': '720p', 'format_note': 'DASH webm', 'preference': -40}, '248': {'ext': 'webm', 'height': 1080, 'resolution': '1080p', 'format_note': 'DASH webm', 'preference': -40}, '271': {'ext': 'webm', 'height': 1440, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40}, '272': {'ext': 'webm', 'height': 2160, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40}, '302': {'ext': 'webm', 'height': 720, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'VP9'}, '303': {'ext': 'webm', 'height': 1080, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'VP9'}, '308': {'ext': 'webm', 'height': 1440, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'VP9'}, '313': {'ext': 'webm', 'height': 2160, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'vcodec': 'VP9'}, '315': {'ext': 'webm', 'height': 2160, 'format_note': 'DASH video', 'acodec': 'none', 'preference': -40, 'fps': 60, 'vcodec': 'VP9'}, # Dash webm audio '171': {'ext': 'webm', 'vcodec': 'none', 'format_note': 'DASH audio', 'abr': 128, 'preference': -50}, '172': {'ext': 'webm', 'vcodec': 'none', 'format_note': 'DASH webm audio', 'abr': 256, 'preference': -50}, # Dash webm audio with opus inside '249': {'ext': 'webm', 'vcodec': 'none', 'format_note': 'DASH audio', 'acodec': 'opus', 'abr': 50, 'preference': -50}, '250': {'ext': 'webm', 'vcodec': 'none', 'format_note': 'DASH audio', 'acodec': 'opus', 'abr': 70, 'preference': -50}, '251': {'ext': 'webm', 'vcodec': 'none', 'format_note': 'DASH audio', 'acodec': 'opus', 'abr': 160, 'preference': -50}, # RTMP (unnamed) '_rtmp': {'protocol': 'rtmp'}, } def __init__(self, *args, **kwargs): Page.__init__(self, *args, **kwargs) self._player_cache = {} def _signature_cache_id(self, example_sig): """ Return a string representation of a signature """ return '.'.join(unicode(len(part)) for part in example_sig.split('.')) def _extract_signature_function(self, video_id, player_url, example_sig): id_m = re.match( r'.*?-(?P<id>[a-zA-Z0-9_-]+)(?:/watch_as3|/html5player)?\.(?P<ext>[a-z]+)$', player_url) if not id_m: raise BrokenPageError('Cannot identify player %r' % player_url) player_type = id_m.group('ext') player_id = id_m.group('id') # Read from filesystem cache func_id = '%s_%s_%s' % ( player_type, player_id, self._signature_cache_id(example_sig)) assert os.path.basename(func_id) == func_id if player_type == 'js': code = self.browser.readurl(player_url) res = self._parse_sig_js(code) elif player_type == 'swf': urlh = self.browser.openurl(player_url) code = urlh.read() res = self._parse_sig_swf(code) else: assert False, 'Invalid player type %r' % player_type return res def _parse_sig_js(self, jscode): funcname = self._search_regex( r'\.sig\|\|([a-zA-Z0-9$]+)\(', jscode, u'Initial JS player signature function name') functions = {} objects = {} code = jscode def argidx(varname): return string.lowercase.index(varname) def interpret_statement(stmt, local_vars, allow_recursion=20): if allow_recursion < 0: raise BrokenPageError(u'Recursion limit reached') if stmt.startswith(u'var '): stmt = stmt[len(u'var '):] ass_m = re.match(r'^(?P<out>[a-z]+)(?:\[(?P<index>[^\]]+)\])?' + r'=(?P<expr>.*)$', stmt) if ass_m: if ass_m.groupdict().get('index'): def assign(val): lvar = local_vars[ass_m.group('out')] idx = interpret_expression(ass_m.group('index'), local_vars, allow_recursion) assert isinstance(idx, int) lvar[idx] = val return val expr = ass_m.group('expr') else: def assign(val): local_vars[ass_m.group('out')] = val return val expr = ass_m.group('expr') elif stmt.startswith(u'return '): assign = lambda v: v expr = stmt[len(u'return '):] else: # Try interpreting it as an expression expr = stmt assign = lambda v: v v = interpret_expression(expr, local_vars, allow_recursion) return assign(v) def extract_object(objname): obj = {} obj_m = re.search( (r'(?:var\s+)?%s\s*=\s*\{' % re.escape(objname)) + r'\s*(?P<fields>([a-zA-Z$0-9]+\s*:\s*function\(.*?\)\s*\{.*?\})*)' + r'\}\s*;', code) fields = obj_m.group('fields') # Currently, it only supports function definitions fields_m = re.finditer( r'(?P<key>[a-zA-Z$0-9]+)\s*:\s*function' r'\((?P<args>[a-z,]+)\){(?P<code>[^}]+)}', fields) for f in fields_m: argnames = f.group('args').split(',') obj[f.group('key')] = build_function(argnames, f.group('code')) return obj def build_function(argnames, code): def resf(args): local_vars = dict(zip(argnames, args)) for stmt in code.split(';'): res = interpret_statement(stmt, local_vars) return res return resf def interpret_expression(expr, local_vars, allow_recursion): if expr.isdigit(): return int(expr) if expr.isalpha(): return local_vars[expr] try: return json.loads(expr) except ValueError: pass m = re.match(r'^(?P<var>[a-zA-Z0-9_]+)\.(?P<member>[^(]+)(?:\(+(?P<args>[^()]*)\))?$', expr) if m: variable = m.group('var') member = m.group('member') arg_str = m.group('args') if variable in local_vars: obj = local_vars[variable] else: if variable not in objects: objects[variable] = extract_object(variable) obj = objects[variable] if arg_str is None: # Member access if member == 'length': return len(obj) return obj[member] assert expr.endswith(')') # Function call if arg_str == '': argvals = tuple() else: argvals = tuple([ interpret_expression(v, local_vars, allow_recursion) for v in arg_str.split(',')]) if member == 'split': assert argvals == ('',) return list(obj) if member == 'join': assert len(argvals) == 1 return argvals[0].join(obj) if member == 'reverse': assert len(argvals) == 0 obj.reverse() return obj if member == 'slice': assert len(argvals) == 1 return obj[argvals[0]:] if member == 'splice': assert isinstance(obj, list) index, howMany = argvals res = [] for i in range(index, min(index + howMany, len(obj))): res.append(obj.pop(index)) return res return obj[member](argvals) m = re.match( r'^(?P<in>[a-z]+)\[(?P<idx>.+)\]$', expr) if m: val = local_vars[m.group('in')] idx = interpret_expression( m.group('idx'), local_vars, allow_recursion - 1) return val[idx] m = re.match(r'^(?P<a>.+?)(?P<op>[%])(?P<b>.+?)$', expr) if m: a = interpret_expression( m.group('a'), local_vars, allow_recursion) b = interpret_expression( m.group('b'), local_vars, allow_recursion) return a % b m = re.match( r'^(?P<func>[a-zA-Z$]+)\((?P<args>[a-z0-9,]+)\)$', expr) if m: fname = m.group('func') argvals = tuple([ int(v) if v.isdigit() else local_vars[v] for v in m.group('args').split(',')]) if fname not in functions: functions[fname] = extract_function(fname) return functions[fname](argvals) raise BrokenPageError(u'Unsupported JS expression %r' % expr) def extract_function(funcname): func_m = re.search( r'function ' + re.escape(funcname) + r'\((?P<args>[a-z,]+)\){(?P<code>[^}]+)}', jscode) argnames = func_m.group('args').split(',') def resf(args): local_vars = dict(zip(argnames, args)) for stmt in func_m.group('code').split(';'): res = interpret_statement(stmt, local_vars) return res return resf initial_function = extract_function(funcname) return lambda s: initial_function([s]) def _parse_sig_swf(self, file_contents): if file_contents[1:3] != b'WS': raise BrokenPageError( u'Not an SWF file; header is %r' % file_contents[:3]) if file_contents[:1] == b'C': content = zlib.decompress(file_contents[8:]) else: raise NotImplementedError(u'Unsupported compression format %r' % file_contents[:1]) def extract_tags(content): pos = 0 while pos < len(content): header16 = struct.unpack('<H', content[pos:pos+2])[0] pos += 2 tag_code = header16 >> 6 tag_len = header16 & 0x3f if tag_len == 0x3f: tag_len = struct.unpack('<I', content[pos:pos+4])[0] pos += 4 assert pos+tag_len <= len(content) yield (tag_code, content[pos:pos+tag_len]) pos += tag_len code_tag = next(tag for tag_code, tag in extract_tags(content) if tag_code == 82) p = code_tag.index(b'\0', 4) + 1 code_reader = io.BytesIO(code_tag[p:]) # Parse ABC (AVM2 ByteCode) def read_int(reader=None): if reader is None: reader = code_reader res = 0 shift = 0 for _ in range(5): buf = reader.read(1) assert len(buf) == 1 b = struct.unpack('<B', buf)[0] res = res | ((b & 0x7f) << shift) if b & 0x80 == 0: break shift += 7 return res def u30(reader=None): res = read_int(reader) assert res & 0xf0000000 == 0 return res u32 = read_int def s32(reader=None): v = read_int(reader) if v & 0x80000000 != 0: v = - ((v ^ 0xffffffff) + 1) return v def read_string(reader=None): if reader is None: reader = code_reader slen = u30(reader) resb = reader.read(slen) assert len(resb) == slen return resb.decode('utf-8') def read_bytes(count, reader=None): if reader is None: reader = code_reader resb = reader.read(count) assert len(resb) == count return resb def read_byte(reader=None): resb = read_bytes(1, reader=reader) res = struct.unpack('<B', resb)[0] return res # minor_version + major_version read_bytes(2 + 2) # Constant pool int_count = u30() for _c in range(1, int_count): s32() uint_count = u30() for _c in range(1, uint_count): u32() double_count = u30() read_bytes((double_count-1) * 8) string_count = u30() constant_strings = [u''] for _c in range(1, string_count): s = read_string() constant_strings.append(s) namespace_count = u30() for _c in range(1, namespace_count): read_bytes(1) # kind u30() # name ns_set_count = u30() for _c in range(1, ns_set_count): count = u30() for _c2 in range(count): u30() multiname_count = u30() MULTINAME_SIZES = { 0x07: 2, # QName 0x0d: 2, # QNameA 0x0f: 1, # RTQName 0x10: 1, # RTQNameA 0x11: 0, # RTQNameL 0x12: 0, # RTQNameLA 0x09: 2, # Multiname 0x0e: 2, # MultinameA 0x1b: 1, # MultinameL 0x1c: 1, # MultinameLA } multinames = [u''] for _c in range(1, multiname_count): kind = u30() assert kind in MULTINAME_SIZES, u'Invalid multiname kind %r' % kind if kind == 0x07: u30() # namespace_idx name_idx = u30() multinames.append(constant_strings[name_idx]) else: multinames.append('[MULTINAME kind: %d]' % kind) for _c2 in range(MULTINAME_SIZES[kind]): u30() # Methods method_count = u30() MethodInfo = collections.namedtuple( 'MethodInfo', ['NEED_ARGUMENTS', 'NEED_REST']) method_infos = [] for method_id in range(method_count): param_count = u30() u30() # return type for _ in range(param_count): u30() # param type u30() # name index (always 0 for youtube) flags = read_byte() if flags & 0x08 != 0: # Options present option_count = u30() for c in range(option_count): u30() # val read_bytes(1) # kind if flags & 0x80 != 0: # Param names present for _ in range(param_count): u30() # param name mi = MethodInfo(flags & 0x01 != 0, flags & 0x04 != 0) method_infos.append(mi) # Metadata metadata_count = u30() for _c in range(metadata_count): u30() # name item_count = u30() for _c2 in range(item_count): u30() # key u30() # value def parse_traits_info(): trait_name_idx = u30() kind_full = read_byte() kind = kind_full & 0x0f attrs = kind_full >> 4 methods = {} if kind in [0x00, 0x06]: # Slot or Const u30() # Slot id u30() # type_name_idx vindex = u30() if vindex != 0: read_byte() # vkind elif kind in [0x01, 0x02, 0x03]: # Method / Getter / Setter u30() # disp_id method_idx = u30() methods[multinames[trait_name_idx]] = method_idx elif kind == 0x04: # Class u30() # slot_id u30() # classi elif kind == 0x05: # Function u30() # slot_id function_idx = u30() methods[function_idx] = multinames[trait_name_idx] else: raise BrokenPageError(u'Unsupported trait kind %d' % kind) if attrs & 0x4 != 0: # Metadata present metadata_count = u30() for _c3 in range(metadata_count): u30() # metadata index return methods # Classes TARGET_CLASSNAME = u'SignatureDecipher' searched_idx = multinames.index(TARGET_CLASSNAME) searched_class_id = None class_count = u30() for class_id in range(class_count): name_idx = u30() if name_idx == searched_idx: # We found the class we're looking for! searched_class_id = class_id u30() # super_name idx flags = read_byte() if flags & 0x08 != 0: # Protected namespace is present u30() # protected_ns_idx intrf_count = u30() for _c2 in range(intrf_count): u30() u30() # iinit trait_count = u30() for _c2 in range(trait_count): parse_traits_info() if searched_class_id is None: raise BrokenPageError(u'Target class %r not found' % TARGET_CLASSNAME) method_names = {} method_idxs = {} for class_id in range(class_count): u30() # cinit trait_count = u30() for _c2 in range(trait_count): trait_methods = parse_traits_info() if class_id == searched_class_id: method_names.update(trait_methods.items()) method_idxs.update(dict( (idx, name) for name, idx in trait_methods.items())) # Scripts script_count = u30() for _c in range(script_count): u30() # init trait_count = u30() for _c2 in range(trait_count): parse_traits_info() # Method bodies method_body_count = u30() Method = collections.namedtuple('Method', ['code', 'local_count']) methods = {} for _c in range(method_body_count): method_idx = u30() u30() # max_stack local_count = u30() u30() # init_scope_depth u30() # max_scope_depth code_length = u30() code = read_bytes(code_length) if method_idx in method_idxs: m = Method(code, local_count) methods[method_idxs[method_idx]] = m exception_count = u30() for _c2 in range(exception_count): u30() # from u30() # to u30() # target u30() # exc_type u30() # var_name trait_count = u30() for _c2 in range(trait_count): parse_traits_info() assert p + code_reader.tell() == len(code_tag) assert len(methods) == len(method_idxs) method_pyfunctions = {} def extract_function(func_name): if func_name in method_pyfunctions: return method_pyfunctions[func_name] if func_name not in methods: raise BrokenPageError(u'Cannot find function %r' % func_name) m = methods[func_name] def resfunc(args): registers = ['(this)'] + list(args) + [None] * m.local_count stack = [] coder = io.BytesIO(m.code) while True: opcode = struct.unpack('!B', coder.read(1))[0] if opcode == 36: # pushbyte v = struct.unpack('!B', coder.read(1))[0] stack.append(v) elif opcode == 44: # pushstring idx = u30(coder) stack.append(constant_strings[idx]) elif opcode == 48: # pushscope # We don't implement the scope register, so we'll just # ignore the popped value stack.pop() elif opcode == 70: # callproperty index = u30(coder) mname = multinames[index] arg_count = u30(coder) args = list(reversed( [stack.pop() for _ in range(arg_count)])) obj = stack.pop() if mname == u'split': assert len(args) == 1 assert isinstance(args[0], unicode) assert isinstance(obj, unicode) if args[0] == u'': res = list(obj) else: res = obj.split(args[0]) stack.append(res) elif mname == u'slice': assert len(args) == 1 assert isinstance(args[0], int) assert isinstance(obj, list) res = obj[args[0]:] stack.append(res) elif mname == u'join': assert len(args) == 1 assert isinstance(args[0], unicode) assert isinstance(obj, list) res = args[0].join(obj) stack.append(res) elif mname in method_pyfunctions: stack.append(method_pyfunctions[mname](args)) else: raise NotImplementedError( u'Unsupported property %r on %r' % (mname, obj)) elif opcode == 72: # returnvalue res = stack.pop() return res elif opcode == 79: # callpropvoid index = u30(coder) mname = multinames[index] arg_count = u30(coder) args = list(reversed( [stack.pop() for _ in range(arg_count)])) obj = stack.pop() if mname == u'reverse': assert isinstance(obj, list) obj.reverse() else: raise NotImplementedError( u'Unsupported (void) property %r on %r' % (mname, obj)) elif opcode == 93: # findpropstrict index = u30(coder) mname = multinames[index] res = extract_function(mname) stack.append(res) elif opcode == 97: # setproperty index = u30(coder) value = stack.pop() idx = stack.pop() obj = stack.pop() assert isinstance(obj, list) assert isinstance(idx, int) obj[idx] = value elif opcode == 98: # getlocal index = u30(coder) stack.append(registers[index]) elif opcode == 99: # setlocal index = u30(coder) value = stack.pop() registers[index] = value elif opcode == 102: # getproperty index = u30(coder) pname = multinames[index] if pname == u'length': obj = stack.pop() assert isinstance(obj, list) stack.append(len(obj)) else: # Assume attribute access idx = stack.pop() assert isinstance(idx, int) obj = stack.pop() assert isinstance(obj, list) stack.append(obj[idx]) elif opcode == 128: # coerce u30(coder) elif opcode == 133: # coerce_s assert isinstance(stack[-1], (type(None), unicode)) elif opcode == 164: # modulo value2 = stack.pop() value1 = stack.pop() res = value1 % value2 stack.append(res) elif opcode == 208: # getlocal_0 stack.append(registers[0]) elif opcode == 209: # getlocal_1 stack.append(registers[1]) elif opcode == 210: # getlocal_2 stack.append(registers[2]) elif opcode == 211: # getlocal_3 stack.append(registers[3]) elif opcode == 214: # setlocal_2 registers[2] = stack.pop() elif opcode == 215: # setlocal_3 registers[3] = stack.pop() else: raise NotImplementedError( u'Unsupported opcode %d' % opcode) method_pyfunctions[func_name] = resfunc return resfunc initial_function = extract_function(u'decipher') return lambda s: initial_function([s]) def _decrypt_signature(self, s, video_id, player_url, age_gate=False): """Turn the encrypted s field into a working signature""" if player_url is not None: if player_url.startswith(u'//'): player_url = u'https:' + player_url try: player_id = (player_url, len(s)) if player_id not in self._player_cache: func = self._extract_signature_function( video_id, player_url, s ) self._player_cache[player_id] = func func = self._player_cache[player_id] return func(s) except Exception: tb = traceback.format_exc() raise BrokenPageError(u'Automatic signature extraction failed: %s' % tb) return self._static_decrypt_signature( s, video_id, player_url, age_gate) def _static_decrypt_signature(self, s, video_id, player_url, age_gate): if age_gate: # The videos with age protection use another player, so the # algorithms can be different. if len(s) == 86: return s[2:63] + s[82] + s[64:82] + s[63] if len(s) == 93: return s[86:29:-1] + s[88] + s[28:5:-1] elif len(s) == 92: return s[25] + s[3:25] + s[0] + s[26:42] + s[79] + s[43:79] + s[91] + s[80:83] elif len(s) == 91: return s[84:27:-1] + s[86] + s[26:5:-1] elif len(s) == 90: return s[25] + s[3:25] + s[2] + s[26:40] + s[77] + s[41:77] + s[89] + s[78:81] elif len(s) == 89: return s[84:78:-1] + s[87] + s[77:60:-1] + s[0] + s[59:3:-1] elif len(s) == 88: return s[7:28] + s[87] + s[29:45] + s[55] + s[46:55] + s[2] + s[56:87] + s[28] elif len(s) == 87: return s[6:27] + s[4] + s[28:39] + s[27] + s[40:59] + s[2] + s[60:] elif len(s) == 86: return s[80:72:-1] + s[16] + s[71:39:-1] + s[72] + s[38:16:-1] + s[82] + s[15::-1] elif len(s) == 85: return s[3:11] + s[0] + s[12:55] + s[84] + s[56:84] elif len(s) == 84: return s[78:70:-1] + s[14] + s[69:37:-1] + s[70] + s[36:14:-1] + s[80] + s[:14][::-1] elif len(s) == 83: return s[80:63:-1] + s[0] + s[62:0:-1] + s[63] elif len(s) == 82: return s[80:37:-1] + s[7] + s[36:7:-1] + s[0] + s[6:0:-1] + s[37] elif len(s) == 81: return s[56] + s[79:56:-1] + s[41] + s[55:41:-1] + s[80] + s[40:34:-1] + s[0] + s[33:29:-1] + s[34] + s[28:9:-1] + s[29] + s[8:0:-1] + s[9] elif len(s) == 80: return s[1:19] + s[0] + s[20:68] + s[19] + s[69:80] elif len(s) == 79: return s[54] + s[77:54:-1] + s[39] + s[53:39:-1] + s[78] + s[38:34:-1] + s[0] + s[33:29:-1] + s[34] + s[28:9:-1] + s[29] + s[8:0:-1] + s[9] else: raise BrokenPageError(u'Unable to decrypt signature, key length %d not supported; retrying might work' % (len(s))) def _parse_dash_manifest(self, video_id, dash_manifest_url, player_url, age_gate): def decrypt_sig(mobj): s = mobj.group(1) dec_s = self._decrypt_signature(s, video_id, player_url, age_gate) return '/signature/%s' % dec_s def int_or_none(v, default=None): try: return int(v) except (ValueError,TypeError): return default dash_manifest_url = re.sub(r'/s/([\w\.]+)', decrypt_sig, dash_manifest_url) dash_doc = self.browser.get_document(self.browser.openurl(dash_manifest_url)) formats = [] for r in dash_doc.findall('.//{urn:mpeg:DASH:schema:MPD:2011}Representation'): url_el = r.find('{urn:mpeg:DASH:schema:MPD:2011}BaseURL') if url_el is None: continue format_id = r.attrib['id'] video_url = url_el.text filesize = int_or_none(url_el.attrib.get('{http://youtube.com/yt/2012/10/10}contentLength')) f = { 'format_id': format_id, 'url': video_url, 'width': int_or_none(r.attrib.get('width')), 'height': int_or_none(r.attrib.get('height')), 'tbr': int_or_none(r.attrib.get('bandwidth'), 1000), 'asr': int_or_none(r.attrib.get('audioSamplingRate')), 'filesize': filesize, 'fps': int_or_none(r.attrib.get('frameRate')), } try: existing_format = next( fo for fo in formats if fo['format_id'] == format_id) except StopIteration: f.update(self._formats.get(format_id, {}).items()) formats.append(f) else: existing_format.update(f) return formats def _extract_from_m3u8(self, manifest_url, video_id): url_map = {} def _get_urls(_manifest): lines = _manifest.split('\n') urls = filter(lambda l: l and not l.startswith('#'), lines) return urls manifest = self.browser.readurl(manifest_url) formats_urls = _get_urls(manifest) for format_url in formats_urls: itag = self._search_regex(r'itag/(\d+?)/', format_url, 'itag') url_map[itag] = format_url return url_map def get_video_url(self, video): video_id = video.id video_webpage = ' '.join([self.parser.tocleanstring(el) for el in self.document.xpath('//script')]) # Attempt to extract SWF player URL mobj = re.search(r'swfConfig.*?"(https?:\\/\\/.*?watch.*?-.*?\.swf)"', video_webpage) if mobj is not None: player_url = re.sub(r'\\(.)', r'\1', mobj.group(1)) else: player_url = None # Get video info if re.search(r'player-age-gate-content">', video_webpage) is not None: age_gate = True # We simulate the access to the video from www.youtube.com/v/{video_id} # this can be viewed without login into Youtube url = 'https://www.youtube.com/embed/%s' % video_id embed_webpage = self.browser.readurl(url) data = urllib.urlencode({ 'video_id': video_id, 'eurl': 'https://youtube.googleapis.com/v/' + video_id, 'sts': self._search_regex( r'"sts"\s*:\s*(\d+)', embed_webpage, 'sts', default=''), }) video_info_url = 'https://www.youtube.com/get_video_info?' + data video_info_webpage = self.browser.readurl(video_info_url) video_info = parse_qs(video_info_webpage) else: age_gate = False try: # Try looking directly into the video webpage mobj = re.search(r';ytplayer\.config\s*=\s*({.*?});', video_webpage) if not mobj: raise ValueError('Could not find ytplayer.config') # caught below json_code = uppercase_escape(mobj.group(1)) ytplayer_config = json.loads(json_code) args = ytplayer_config['args'] # Convert to the same format returned by compat_parse_qs video_info = dict((k, [v]) for k, v in args.items()) if 'url_encoded_fmt_stream_map' not in args: raise ValueError('No stream_map present') # caught below except ValueError: # We fallback to the get_video_info pages (used by the embed page) for el_type in ['&el=embedded', '&el=detailpage', '&el=vevo', '']: video_info_url = ('https://www.youtube.com/get_video_info?&video_id=%s%s&ps=default&eurl=&gl=US&hl=en' % (video_id, el_type)) video_info_webpage = self.browser.readurl(video_info_url) video_info = parse_qs(video_info_webpage) if 'token' in video_info: break if 'token' not in video_info: if 'reason' in video_info: raise UserError(video_info['reason'][0]) else: raise BrokenPageError(u'"token" parameter not in video info for unknown reason') # Check for "rental" videos if 'ypc_video_rental_bar_text' in video_info and 'author' not in video_info: raise UserError(u'"rental" videos not supported') def _map_to_format_list(urlmap): formats = [] for itag, video_real_url in urlmap.items(): dct = { 'format_id': itag, 'url': video_real_url, 'player_url': player_url, } if itag in self._formats: dct.update(self._formats[itag]) formats.append(dct) return formats if 'conn' in video_info and video_info['conn'][0].startswith('rtmp'): formats = [{ 'format_id': '_rtmp', 'protocol': 'rtmp', 'url': video_info['conn'][0], 'player_url': player_url, }] elif len(video_info.get('url_encoded_fmt_stream_map', [''])[0]) >= 1 or len(video_info.get('adaptive_fmts', [''])[0]) >= 1: encoded_url_map = video_info.get('url_encoded_fmt_stream_map', [''])[0] + ',' + video_info.get('adaptive_fmts', [''])[0] if 'rtmpe%3Dyes' in encoded_url_map: raise BrokenPageError('rtmpe downloads are not supported, see https://github.com/rg3/youtube-dl/issues/343 for more information.', expected=True) url_map = {} for url_data_str in encoded_url_map.split(','): url_data = parse_qs(url_data_str) if 'itag' not in url_data or 'url' not in url_data: continue format_id = url_data['itag'][0] url = url_data['url'][0] if 'sig' in url_data: url += '&signature=' + url_data['sig'][0] elif 's' in url_data: encrypted_sig = url_data['s'][0] jsplayer_url_json = self._search_regex( r'"assets":.+?"js":\s*("[^"]+")', embed_webpage if age_gate else video_webpage, 'JS player URL') player_url = json.loads(jsplayer_url_json) if player_url is None: player_url_json = self._search_regex( r'ytplayer\.config.*?"url"\s*:\s*("[^"]+")', video_webpage, 'age gate player URL') player_url = json.loads(player_url_json) signature = self._decrypt_signature( encrypted_sig, video_id, player_url, age_gate) url += '&signature=' + signature if 'ratebypass' not in url: url += '&ratebypass=yes' url_map[format_id] = url formats = _map_to_format_list(url_map) elif video_info.get('hlsvp'): manifest_url = video_info['hlsvp'][0] url_map = self._extract_from_m3u8(manifest_url, video_id) formats = _map_to_format_list(url_map) else: raise BrokenPageError(u'no conn, hlsvp or url_encoded_fmt_stream_map information found in video info') dash_mpd = video_info.get('dashmpd') if dash_mpd: dash_manifest_url = dash_mpd[0] try: dash_formats = self._parse_dash_manifest( video_id, dash_manifest_url, player_url, age_gate) except (BrokenPageError, KeyError) as e: self.logger.info( 'Skipping DASH manifest: %r' % e) else: # Hide the formats we found through non-DASH dash_keys = set(df['format_id'] for df in dash_formats) for f in formats: if f['format_id'] in dash_keys: f['format_id'] = 'nondash-%s' % f['format_id'] f['preference'] = f.get('preference', 0) - 10000 formats.extend(dash_formats) # Check for malformed aspect ratio stretched_m = re.search( r'<meta\s+property="og:video:tag".*?content="yt:stretch=(?P<w>[0-9]+):(?P<h>[0-9]+)">', video_webpage) if stretched_m: ratio = float(stretched_m.group('w')) / float(stretched_m.group('h')) for f in formats: if f.get('vcodec') != 'none': f['stretched_ratio'] = ratio self._sort_formats(formats) best = formats[-1] return best['url'], best['ext'] def _sort_formats(self, formats): if not formats: raise BrokenPageError(u'No video formats found') def _formats_key(f): # TODO remove the following workaround if not f.get('ext') and 'url' in f: f['ext'] = determine_ext(f['url']) preference = f.get('preference') if preference is None: proto = f.get('protocol') if proto is None: proto = urlparse(f.get('url', '')).scheme preference = 0 if proto in ['http', 'https'] else -0.1 if f.get('ext') in ['f4f', 'f4m']: # Not yet supported preference -= 0.5 if f.get('vcodec') == 'none': # audio only ORDER = [u'webm', u'opus', u'ogg', u'mp3', u'aac', u'm4a'] ext_preference = 0 try: audio_ext_preference = ORDER.index(f['ext']) except ValueError: audio_ext_preference = -1 else: ORDER = [u'webm', u'flv', u'mp4'] try: ext_preference = ORDER.index(f['ext']) except ValueError: ext_preference = -1 audio_ext_preference = 0 return ( preference, f.get('quality') if f.get('quality') is not None else -1, f.get('height') if f.get('height') is not None else -1, f.get('width') if f.get('width') is not None else -1, ext_preference, f.get('tbr') if f.get('tbr') is not None else -1, f.get('vbr') if f.get('vbr') is not None else -1, f.get('abr') if f.get('abr') is not None else -1, audio_ext_preference, f.get('filesize') if f.get('filesize') is not None else -1, f.get('format_id'), ) formats.sort(key=_formats_key) def _search_regex(self, pattern, text, name, default=_NO_DEFAULT, fatal=True, flags=0): """ Perform a regex search on the given string, using a single or a list of patterns returning the first matching group. In case of failure return a default value or raise a WARNING or a RegexNotFoundError, depending on fatal, specifying the field name. """ if isinstance(pattern, (str, unicode, type(re.compile('')))): mobj = re.search(pattern, text, flags) else: for p in pattern: mobj = re.search(p, text, flags) if mobj: break if mobj: # return the first matching group return next(g for g in mobj.groups() if g is not None) elif default is not _NO_DEFAULT: return default elif fatal: raise BrokenPageError(u'Unable to extract %s' % name) else: self.logger.warning(u'unable to extract %s' % name) return None ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/test.py������������������������������������������������������������������0000664�0000000�0000000�00000003540�13034501105�0017320�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.tools.test import BackendTest from weboob.capabilities.video import BaseVideo class YoutubeTest(BackendTest): MODULE = 'youtube' def test_search(self): l = list(self.backend.search_videos('lol')) self.assertTrue(len(l) > 0) v = l[0] self.backend.fillobj(v, ('url',)) self.assertTrue(v.url and v.url.startswith('https://'), 'URL for video "%s" not found: %s' % (v.id, v.url)) assert self.backend.get_video(v.shorturl) self.backend.browser.openurl(v.url) def test_latest(self): l = list(self.backend.iter_resources([BaseVideo], [u'latest'])) assert len(l) > 0 def test_drm(self): v = self.backend.get_video('http://youtu.be/UxxajLWwzqY') self.backend.fillobj(v, ('url',)) assert len(v.url) try: self.backend.browser.openurl(v.url) except: self.fail('can\'t open url') def test_weirdchars(self): v = self.backend.get_video('https://www.youtube.com/watch?v=BaW_jenozKc') self.backend.fillobj(v, ('title', 'url',)) assert unicode(v.title) ����������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/modules/youtube/video.py�����������������������������������������������������������������0000664�0000000�0000000�00000002015�13034501105�0017443�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from weboob.capabilities.video import BaseVideo class YoutubeVideo(BaseVideo): @classmethod def id2url(cls, _id): return 'https://www.youtube.com/watch?v=%s' % _id def _get_shorturl(self): return 'https://youtu.be/%s' % self.id shorturl = property(_get_shorturl) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/release.sh�������������������������������������������������������������������������������0000775�0000000�0000000�00000003004�13034501105�0014575�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/bash # This script is used to release a version. set -e function set_version { echo -n "Replacing version in source files to $1" for fl in $(find . -iregex ".*\.\(py\|rst\)$" ! -path "./contrib/*"); do sed "s/^\(\s*\)\(VERSION\|version\|release\)\( *\)=\( *\)[\"'][0-9]\+\..\+[\"']\(,\?\)$/\1\2\3=\4'$1'\5/g" $fl > $fl.new diff $fl.new $fl >/dev/null && echo -n "." || echo -n "+" cp -f $fl.new $fl rm -f $fl.new done echo -e " done.\n" } if [ "$1" = "" ]; then echo "Syntax: $0 VERSION" exit 1 fi VERSION=$1 export LANG=en_US.utf8 mv ChangeLog ChangeLog.old echo -e "Weboob $VERSION (`date +%Y-%m-%d`)\n\t \n\n" > ChangeLog cat ChangeLog.old >> ChangeLog rm -f ChangeLog.old vi +2 ChangeLog set_version $VERSION echo "Building Qt applications..." ./setup.py --qt sdist bdist clean -a || exit 1 echo "Generating manpages..." tools/make_man.sh echo -e "done!\n" # in case there are new manpages not included in the git tree. git add man/* echo "Release commit:" git commit -a -m "Weboob $VERSION released" echo -ne "\n" echo "Release tag:" git tag $VERSION -s -m "Weboob $VERSION" echo -ne "\n" echo -n "Generating archive.. " git archive HEAD --prefix=weboob-$VERSION/ -o weboob-$VERSION.tar gzip -f weboob-$VERSION.tar md5sum weboob-$VERSION.tar.gz echo -ne "\nDo you want to change the version number (y/n) " read change_version if [ "$change_version" = "y" ]; then echo -n "Enter the new version number: " read NEW_VERSION set_version $NEW_VERSION git commit -a -m "bump to $NEW_VERSION" fi ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/���������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0014310�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/boobank��������������������������������������������������������������������������0000775�0000000�0000000�00000001700�13034501105�0015647�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2009-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobank import Boobank if __name__ == '__main__': Boobank.run() ����������������������������������������������������������������weboob-1.2/scripts/boobathon������������������������������������������������������������������������0000775�0000000�0000000�00000001702�13034501105�0016211�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobathon import Boobathon if __name__ == '__main__': Boobathon.run() ��������������������������������������������������������������weboob-1.2/scripts/boobcoming�����������������������������������������������������������������������0000775�0000000�0000000�00000001574�13034501105�0016363�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobcoming import Boobcoming if __name__ == '__main__': Boobcoming.run() ������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/boobill��������������������������������������������������������������������������0000775�0000000�0000000�00000001675�13034501105�0015671�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2012 Fourcot Florent # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobill import Boobill if __name__ == '__main__': Boobill.run() �������������������������������������������������������������������weboob-1.2/scripts/booblyrics�����������������������������������������������������������������������0000775�0000000�0000000�00000001706�13034501105�0016411�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.booblyrics import Booblyrics if __name__ == '__main__': Booblyrics.run() ����������������������������������������������������������weboob-1.2/scripts/boobmsg��������������������������������������������������������������������������0000775�0000000�0000000�00000001703�13034501105�0015667�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobmsg import Boobmsg if __name__ == '__main__': Boobmsg.run() �������������������������������������������������������������weboob-1.2/scripts/boobooks�������������������������������������������������������������������������0000775�0000000�0000000�00000001677�13034501105�0016066�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobooks import Boobooks if __name__ == '__main__': Boobooks.run() �����������������������������������������������������������������weboob-1.2/scripts/boobsize�������������������������������������������������������������������������0000775�0000000�0000000�00000001701�13034501105�0016051�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobsize import Boobsize if __name__ == '__main__': Boobsize.run() ���������������������������������������������������������������weboob-1.2/scripts/boobtracker����������������������������������������������������������������������0000775�0000000�0000000�00000001603�13034501105�0016533�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.boobtracker import BoobTracker if __name__ == '__main__': BoobTracker.run() �����������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/cineoob��������������������������������������������������������������������������0000775�0000000�0000000�00000001675�13034501105�0015665�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.cineoob import Cineoob if __name__ == '__main__': Cineoob.run() �������������������������������������������������������������������weboob-1.2/scripts/comparoob������������������������������������������������������������������������0000775�0000000�0000000�00000000377�13034501105�0016226�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai from __future__ import absolute_import from weboob.applications.comparoob import Comparoob if __name__ == '__main__': Comparoob.run() �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/cookboob�������������������������������������������������������������������������0000775�0000000�0000000�00000001700�13034501105�0016031�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.cookboob import Cookboob if __name__ == '__main__': Cookboob.run() ����������������������������������������������������������������weboob-1.2/scripts/flatboob�������������������������������������������������������������������������0000775�0000000�0000000�00000001677�13034501105�0016041�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.flatboob import Flatboob if __name__ == '__main__': Flatboob.run() �����������������������������������������������������������������weboob-1.2/scripts/galleroob������������������������������������������������������������������������0000775�0000000�0000000�00000001604�13034501105�0016205�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.galleroob import Galleroob if __name__ == '__main__': Galleroob.run() ����������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/geolooc��������������������������������������������������������������������������0000775�0000000�0000000�00000001701�13034501105�0015664�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.geolooc import Geolooc if __name__ == '__main__': Geolooc.run() ���������������������������������������������������������������weboob-1.2/scripts/handjoob�������������������������������������������������������������������������0000775�0000000�0000000�00000001567�13034501105�0016033�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.handjoob import Handjoob if __name__ == '__main__': Handjoob.run() �����������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/havedate�������������������������������������������������������������������������0000775�0000000�0000000�00000001704�13034501105�0016021�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.havedate import HaveDate if __name__ == '__main__': HaveDate.run() ������������������������������������������������������������weboob-1.2/scripts/masstransit����������������������������������������������������������������������0000775�0000000�0000000�00000001716�13034501105�0016613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Julien Hébert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.masstransit import Masstransit if __name__ == '__main__': Masstransit.run() ��������������������������������������������������weboob-1.2/scripts/monboob��������������������������������������������������������������������������0000775�0000000�0000000�00000001701�13034501105�0015670�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.monboob import Monboob if __name__ == '__main__': Monboob.run() ���������������������������������������������������������������weboob-1.2/scripts/parceloob������������������������������������������������������������������������0000775�0000000�0000000�00000001702�13034501105�0016204�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.parceloob import Parceloob if __name__ == '__main__': Parceloob.run() ��������������������������������������������������������������weboob-1.2/scripts/pastoob��������������������������������������������������������������������������0000775�0000000�0000000�00000001677�13034501105�0015720�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.pastoob import Pastoob if __name__ == '__main__': Pastoob.run() �����������������������������������������������������������������weboob-1.2/scripts/qbooblyrics����������������������������������������������������������������������0000775�0000000�0000000�00000001715�13034501105�0016572�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qbooblyrics import QBooblyrics if __name__ == '__main__': QBooblyrics.run() ���������������������������������������������������weboob-1.2/scripts/qboobmsg�������������������������������������������������������������������������0000775�0000000�0000000�00000001704�13034501105�0016051�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qboobmsg import QBoobMsg if __name__ == '__main__': QBoobMsg.run() ������������������������������������������������������������weboob-1.2/scripts/qcineoob�������������������������������������������������������������������������0000775�0000000�0000000�00000001700�13034501105�0016033�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qcineoob import QCineoob if __name__ == '__main__': QCineoob.run() ����������������������������������������������������������������weboob-1.2/scripts/qcookboob������������������������������������������������������������������������0000775�0000000�0000000�00000001703�13034501105�0016215�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qcookboob import QCookboob if __name__ == '__main__': QCookboob.run() �������������������������������������������������������������weboob-1.2/scripts/qflatboob������������������������������������������������������������������������0000775�0000000�0000000�00000001707�13034501105�0016214�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qflatboob import QFlatBoob if __name__ == '__main__': QFlatBoob.run() ���������������������������������������������������������weboob-1.2/scripts/qhandjoob������������������������������������������������������������������������0000775�0000000�0000000�00000001712�13034501105�0016204�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2012 Sébastien Monel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qhandjoob import QHandJoob if __name__ == '__main__': QHandJoob.run() ������������������������������������������������������weboob-1.2/scripts/qhavedate������������������������������������������������������������������������0000775�0000000�0000000�00000001707�13034501105�0016205�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qhavedate import QHaveDate if __name__ == '__main__': QHaveDate.run() ���������������������������������������������������������weboob-1.2/scripts/qvideoob�������������������������������������������������������������������������0000775�0000000�0000000�00000001704�13034501105�0016050�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qvideoob import QVideoob if __name__ == '__main__': QVideoob.run() ������������������������������������������������������������weboob-1.2/scripts/qwebcontentedit������������������������������������������������������������������0000775�0000000�0000000�00000001731�13034501105�0017437�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qwebcontentedit import QWebContentEdit if __name__ == '__main__': QWebContentEdit.run() ���������������������������������������weboob-1.2/scripts/radioob��������������������������������������������������������������������������0000775�0000000�0000000�00000001701�13034501105�0015654�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.radioob import Radioob if __name__ == '__main__': Radioob.run() ���������������������������������������������������������������weboob-1.2/scripts/shopoob��������������������������������������������������������������������������0000775�0000000�0000000�00000001676�13034501105�0015721�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/python2.7 # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.shopoob import Shopoob if __name__ == '__main__': Shopoob.run() ������������������������������������������������������������������weboob-1.2/scripts/suboob���������������������������������������������������������������������������0000775�0000000�0000000�00000001673�13034501105�0015536�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.suboob import Suboob if __name__ == '__main__': Suboob.run() ���������������������������������������������������������������������weboob-1.2/scripts/translaboob����������������������������������������������������������������������0000775�0000000�0000000�00000001604�13034501105�0016545�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.translaboob import Translaboob if __name__ == '__main__': Translaboob.run() ����������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/traveloob������������������������������������������������������������������������0000775�0000000�0000000�00000001707�13034501105�0016240�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.traveloob import Traveloob if __name__ == '__main__': Traveloob.run() ���������������������������������������������������������weboob-1.2/scripts/videoob��������������������������������������������������������������������������0000775�0000000�0000000�00000001703�13034501105�0015666�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.videoob import Videoob if __name__ == '__main__': Videoob.run() �������������������������������������������������������������weboob-1.2/scripts/webcontentedit�������������������������������������������������������������������0000775�0000000�0000000�00000001726�13034501105�0017262�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.webcontentedit import WebContentEdit if __name__ == '__main__': WebContentEdit.run() ������������������������������������������weboob-1.2/scripts/weboob���������������������������������������������������������������������������0000775�0000000�0000000�00000014103�13034501105�0015512�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2009-2017 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import import re import os import sys import inspect from datetime import datetime, timedelta import weboob.applications from weboob.tools.ordereddict import OrderedDict from weboob.tools.application.console import ConsoleApplication __all__ = ['Weboob'] class Weboob(ConsoleApplication): APPNAME = 'weboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR The Weboob Team' DESCRIPTION = "This is a console script to launch weboob applications," SHORT_DESCRIPTION = "launch weboob applications" UPDATE_DAYS_DELAY = 20 def main(self): self.update() capApplicationDict = self.init_CapApplicationDict() if len(sys.argv) >= 2: try: cmd = getattr(self, 'cmd_%s' % sys.argv[1]) except AttributeError: pass else: cmd() return cap = sys.argv.pop(1) if cap not in capApplicationDict: print('Unknown capability, please choose one in the following list') cap = self.choose_capability(capApplicationDict) else: cap = self.choose_capability(capApplicationDict) applications = capApplicationDict[cap] application = applications[0] if len(applications) == 1 else self.choose_application(applications) application.run() def cmd_update(self): self.weboob.update() def update(self): for repository in self.weboob.repositories.repositories: update_date = datetime.strptime(str(repository.update), '%Y%m%d%H%M') if (datetime.now() - timedelta(days=self.UPDATE_DAYS_DELAY)) > update_date: update = self.ask('The repositories have not been updated for %s days, do you want to update them ? (y/n)' % self.UPDATE_DAYS_DELAY, default='n') if update.upper() == 'Y': self.weboob.repositories.update() break def choose_application(self, applications): application = None while not application: for app in applications: print(' %s%2d)%s %s: %s' % (self.BOLD, applications.index(app) + 1, self.NC, app.APPNAME, app.DESCRIPTION)) r = self.ask(' Select an application', regexp='(\d+|)', default='') if not r.isdigit(): continue r = int(r) if r <= 0 or r > len(applications): continue application = applications[r - 1] return application def choose_capability(self, capApplicationDict): cap = None while cap not in capApplicationDict.keys(): for _cap in capApplicationDict.keys(): print(' %s%2d)%s %s' % (self.BOLD, capApplicationDict.keys().index(_cap) + 1, self.NC, _cap)) r = self.ask(' Select a capability', regexp='(\d+|)', default='') if not r.isdigit(): continue r = int(r) if r <= 0 or r > len(capApplicationDict.keys()): continue cap = capApplicationDict.keys()[r - 1] return cap def init_CapApplicationDict(self): capApplicationDict = {} for path in weboob.applications.__path__: regexp = re.compile('^%s/([\w\d_]+)$' % path) for root, dirs, files in os.walk(path): m = regexp.match(root) if not (m and '__init__.py' in files): continue application = self.get_applicaction_from_filename(m.group(1)) if not application: continue capabilities = self.get_application_capabilities(application) if not capabilities: continue for capability in capabilities: if capability in capApplicationDict: capApplicationDict[capability].append(application) else: capApplicationDict[capability] = [application] return OrderedDict([(k, v) for k, v in sorted(capApplicationDict.iteritems())]) def get_application_capabilities(self, application): if hasattr(application, 'CAPS') and application.CAPS: _capabilities = list(application.CAPS) if isinstance(application.CAPS, tuple) else [application.CAPS] return [os.path.splitext(os.path.basename(inspect.getfile(x)))[0] for x in _capabilities] def get_applicaction_from_filename(self, name): module = 'weboob.applications.%s.%s' % (name, name) try: _module = __import__(module, fromlist=['*']) except ImportError: return _application = [x for x in dir(_module) if x.lower() == name] if _application: return getattr(_module, _application[0]) if __name__ == '__main__': try: Weboob().main() except KeyboardInterrupt: print('') �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/scripts/weboob-cli�����������������������������������������������������������������������0000775�0000000�0000000�00000001707�13034501105�0016265�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.weboobcli import WeboobCli if __name__ == '__main__': WeboobCli.run() ���������������������������������������������������������weboob-1.2/scripts/weboob-config��������������������������������������������������������������������0000775�0000000�0000000�00000001707�13034501105�0016763�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.weboobcfg import WeboobCfg if __name__ == '__main__': WeboobCfg.run() ���������������������������������������������������������weboob-1.2/scripts/weboob-config-qt�����������������������������������������������������������������0000775�0000000�0000000�00000001712�13034501105�0017401�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.qweboobcfg import QWeboobCfg if __name__ == '__main__': QWeboobCfg.run() ������������������������������������������������������weboob-1.2/scripts/weboob-debug���������������������������������������������������������������������0000775�0000000�0000000�00000001717�13034501105�0016605�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.weboobdebug import WeboobDebug if __name__ == '__main__': WeboobDebug.run() �������������������������������������������������weboob-1.2/scripts/weboob-repos���������������������������������������������������������������������0000775�0000000�0000000�00000001710�13034501105�0016640�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.weboobrepos import WeboobRepos if __name__ == '__main__': WeboobRepos.run() ��������������������������������������������������������weboob-1.2/scripts/weboorrents����������������������������������������������������������������������0000775�0000000�0000000�00000001715�13034501105�0016613�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.weboorrents import Weboorrents if __name__ == '__main__': Weboorrents.run() ���������������������������������������������������weboob-1.2/scripts/wetboobs�������������������������������������������������������������������������0000775�0000000�0000000�00000001704�13034501105�0016064�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import from weboob.applications.wetboobs import WetBoobs if __name__ == '__main__': WetBoobs.run() ������������������������������������������������������������weboob-1.2/setup.cfg��������������������������������������������������������������������������������0000664�0000000�0000000�00000001405�13034501105�0014442�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������[nosetests] verbosity = 2 detailed-errors = 1 with-doctest = 1 with-coverage = 1 where = weboob tests = weboob.tools.capabilities.bank.iban, weboob.tools.capabilities.bank.transactions, weboob.tools.capabilities.paste, weboob.tools.application.formatters.json, weboob.tools.application.formatters.table, weboob.tools.date, weboob.tools.misc, weboob.tools.path, weboob.tools.tokenizer, weboob.browser.browsers, weboob.browser.pages, weboob.browser.filters.standard, weboob.browser.tests.form, weboob.browser.tests.url [isort] known_first_party = weboob line_length = 120 [flake8] max-line-length = 120 exclude = dist,*.egg-info,build,.git,__pycache__ [easy_install] �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/setup.py���������������������������������������������������������������������������������0000775�0000000�0000000�00000016247�13034501105�0014350�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#! /usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Christophe Benz, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function import glob import os import subprocess import sys from setuptools import find_packages, setup def find_executable(name, names): envname = '%s_EXECUTABLE' % name.upper() if os.getenv(envname): return os.getenv(envname) paths = os.getenv('PATH', os.defpath).split(os.pathsep) exts = os.getenv('PATHEXT', os.pathsep).split(os.pathsep) for name in names: for path in paths: for ext in exts: fpath = os.path.join(path, name) + ext if os.path.exists(fpath) and os.access(fpath, os.X_OK): return fpath print('Could not find executable: %s' % name, file=sys.stderr) def build_qt(): print('Building Qt applications...', file=sys.stderr) make = find_executable('make', ('gmake', 'make')) pyuic5 = find_executable('pyuic5', ('python2-pyuic5', 'pyuic5-python2.7', 'pyuic5')) if not pyuic5 or not make: print('Install missing component(s) (see above) or disable Qt applications (with --no-qt).', file=sys.stderr) sys.exit(1) subprocess.check_call( [make, '-f', 'build.mk', '-s', '-j2', 'all', 'PYUIC=%s%s' % (pyuic5, ' WIN32=1' if sys.platform == 'win32' else '')]) def install_weboob(): scripts = set(os.listdir('scripts')) packages = set(find_packages(exclude=['modules'])) hildon_scripts = set(('masstransit',)) qt_scripts = set(('qboobmsg', 'qhavedate', 'qvideoob', 'weboob-config-qt', 'qwebcontentedit', 'qflatboob', 'qcineoob', 'qcookboob', 'qbooblyrics', 'qhandjoob')) if not options.hildon: scripts = scripts - hildon_scripts if options.qt: build_qt() else: scripts = scripts - qt_scripts hildon_packages = set(( 'weboob.applications.masstransit', )) qt_packages = set(( 'weboob.applications.qboobmsg', 'weboob.applications.qboobmsg.ui', 'weboob.applications.qcineoob', 'weboob.applications.qcineoob.ui', 'weboob.applications.qcookboob', 'weboob.applications.qcookboob.ui', 'weboob.applications.qbooblyrics', 'weboob.applications.qbooblyrics.ui', 'weboob.applications.qhandjoob', 'weboob.applications.qhandjoob.ui', 'weboob.applications.qhavedate', 'weboob.applications.qhavedate.ui', 'weboob.applications.qvideoob', 'weboob.applications.qvideoob.ui', 'weboob.applications.qweboobcfg', 'weboob.applications.qweboobcfg.ui', 'weboob.applications.qwebcontentedit', 'weboob.applications.qwebcontentedit.ui' 'weboob.applications.qflatboob', 'weboob.applications.qflatboob.ui' )) if not options.hildon: packages = packages - hildon_packages if not options.qt: packages = packages - qt_packages data_files = [ ('share/man/man1', glob.glob('man/*')), ] if options.xdg: data_files.extend([ ('share/applications', glob.glob('desktop/*')), ('share/icons/hicolor/64x64/apps', glob.glob('icons/*')), ]) # Do not put PyQt, it does not work properly. requirements = [ 'lxml', 'cssselect', 'feedparser', 'requests>=2.0.0', 'python-dateutil', 'PyYAML', 'prettytable', 'google-api-python-client', ] try: import Image except ImportError: requirements.append('Pillow') else: # detect Pillow-only feature, or weird Debian stuff if hasattr(Image, 'alpha_composite') or 'PILcompat' in Image.__file__: requirements.append('Pillow') else: requirements.append('PIL') if sys.version_info < (3, 0): requirements.append('mechanize') if sys.version_info < (3, 2): requirements.append('futures') if sys.version_info < (2, 7): print('Python older than 2.7 is not supported.', file=sys.stderr) sys.exit(1) if not options.deps: requirements = [] try: if sys.argv[1] == 'requirements': print('\n'.join(requirements)) sys.exit(0) except IndexError: pass setup( name='weboob', version='1.2', description='Weboob, Web Outside Of Browsers', long_description=open('README').read(), author='Romain Bignon', author_email='weboob@weboob.org', maintainer='Romain Bignon', maintainer_email='romain@weboob.org', url='http://weboob.org/', license='GNU AGPL 3', classifiers=[ 'Environment :: Console', 'Environment :: X11 Applications :: Qt', 'License :: OSI Approved :: GNU Affero General Public License v3', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python', 'Topic :: Communications :: Email', 'Topic :: Internet :: WWW/HTTP', ], packages=packages, scripts=[os.path.join('scripts', script) for script in scripts], data_files=data_files, install_requires=requirements, ) class Options(object): hildon = False qt = False xdg = True deps = True options = Options() args = list(sys.argv) if '--hildon' in args and '--no-hildon' in args: print('--hildon and --no-hildon options are incompatible', file=sys.stderr) sys.exit(1) if '--qt' in args and '--no-qt' in args: print('--qt and --no-qt options are incompatible', file=sys.stderr) sys.exit(1) if '--xdg' in args and '--no-xdg' in args: print('--xdg and --no-xdg options are incompatible', file=sys.stderr) sys.exit(1) if '--hildon' in args or os.environ.get('HILDON') == 'true': options.hildon = True if '--hildon' in args: args.remove('--hildon') elif '--no-hildon' in args: options.hildon = False args.remove('--no-hildon') if '--qt' in args: options.qt = True args.remove('--qt') elif '--no-qt' in args: options.qt = False args.remove('--no-qt') if '--xdg' in args: options.xdg = True args.remove('--xdg') elif '--no-xdg' in args: options.xdg = False args.remove('--no-xdg') if '--nodeps' in args: options.deps = False args.remove('--nodeps') sys.argv = args install_weboob() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/�����������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0013761�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate.py���������������������������������������������������������������������0000775�0000000�0000000�00000017406�13034501105�0016650�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#! /usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function import argparse import subprocess import datetime import importlib import os import sys import codecs from mako.lookup import TemplateLookup MODULE_PATH = os.getenv( 'MODULE_PATH', os.path.realpath(os.path.join(os.path.dirname(__file__), '../modules'))) TEMPLATE_PATH = os.getenv( 'TEMPLATE_PATH', os.path.realpath(os.path.join(os.path.dirname(__file__), 'boilerplate_data'))) VERSION = '1.2' TEMPLATES = TemplateLookup(directories=[TEMPLATE_PATH]) def u8(s): if isinstance(s, unicode): return s return s.decode('utf-8') def gitconfig(entry): return u8(subprocess.check_output('git config -z --get %s' % entry, shell=True)[:-1]) def write(target, contents): if not os.path.isdir(os.path.dirname(target)): os.makedirs(os.path.dirname(target)) if os.path.exists(target): print("%s already exists." % target, file=sys.stderr) sys.exit(4) with codecs.open(target, mode='w', encoding='utf-8') as f: f.write(contents) print('Created %s' % target) class Recipe(object): @classmethod def configure_subparser(cls, subparsers): subparser = subparsers.add_parser(cls.NAME) subparser.add_argument('name', help='Module name') subparser.set_defaults(recipe=cls) return subparser def __init__(self, args): self.name = args.name.lower().replace(' ', '') self.classname = args.name.title().replace(' ', '') self.year = datetime.date.today().year self.author = args.author self.email = args.email self.version = VERSION def write(self, filename, contents): return write(os.path.join(MODULE_PATH, self.name, filename), contents) def template(self, name, **kwargs): if '.' not in name: name += '.py' return TEMPLATES.get_template(name) \ .render(r=self, # workaround, as it's also a mako directive coding='# -*- coding: utf-8 -*-', **kwargs) def generate(self): raise NotImplementedError() class BaseRecipe(Recipe): NAME = 'base' def generate(self): self.write('__init__.py', self.template('init')) self.write('module.py', self.template('base_module')) self.write('browser.py', self.template('base_browser')) self.write('pages.py', self.template('base_pages')) self.write('test.py', self.template('base_test')) class CapRecipe(Recipe): NAME = 'cap' LINES = {'def' : ' def %s%s:', 'docbound': ' """', 'docline': ' %s', 'body' : ' raise NotImplementedError()' } def __init__(self, args): super(CapRecipe, self).__init__(args) self.capname = args.capname @classmethod def configure_subparser(cls, subparsers): subparser = super(CapRecipe, cls).configure_subparser(subparsers) subparser.add_argument('capname', help='Capability name') return subparser def find_module_cap(self): if '.' not in self.capname: return self.search_cap() PREFIX = 'weboob.capabilities.' if not self.capname.startswith(PREFIX): self.capname = PREFIX + self.capname try: self.capmodulename, self.capname = self.capname.rsplit('.', 1) except ValueError: self.error('Cap name must be in format module.CapSomething or CapSomething') try: module = importlib.import_module(self.capmodulename) except ImportError: self.error('Module %r not found' % self.capmodulename) try: cap = getattr(module, self.capname) except AttributeError: self.error('Module %r has no such capability %r' % (self.capmodulename, self.capname)) return cap def search_cap(self): import pkgutil import weboob.capabilities modules = pkgutil.walk_packages(weboob.capabilities.__path__, prefix='weboob.capabilities.') for _, capmodulename, __ in modules: module = importlib.import_module(capmodulename) if hasattr(module, self.capname): self.capmodulename = capmodulename return getattr(module, self.capname) self.error('Capability %r not found' % self.capname) def error(self, message): print(message, file=sys.stderr) sys.exit(1) def methods_code(self, klass): import inspect import re codes = [] for name, member in inspect.getmembers(klass): if inspect.ismethod(member): argspec = inspect.getargspec(member) args = inspect.formatargspec(*argspec) code = [] code.append(self.LINES['def'] % (name, args)) doc = inspect.getdoc(member) if doc: code.append(self.LINES['docbound']) for line in doc.split('\n'): if line: line = re.sub('"""', '\\"\\"\\"', line) code.append(self.LINES['docline'] % line) else: code.append('') code.append(self.LINES['docbound']) code.append(self.LINES['body']) codes.append('\n'.join(code)) return '\n\n'.join(codes) def generate(self): cap = self.find_module_cap() self.methods_code = self.methods_code(cap) self.write('__init__.py', self.template('init')) self.write('module.py', self.template('cap_module')) self.write('browser.py', self.template('base_browser')) self.write('pages.py', self.template('base_pages')) self.write('test.py', self.template('base_test')) class ComicRecipe(Recipe): NAME = 'comic' def generate(self): self.write('__init__.py', self.template('init')) self.write('module.py', self.template('comic_module')) class ComicTestRecipe(Recipe): NAME = 'comic.test' @classmethod def configure_subparser(cls, subparsers): subparser = super(ComicTestRecipe, cls).configure_subparser(subparsers) subparser.add_argument('download_id', help='Download ID') return subparser def __init__(self, args): super(ComicTestRecipe, self).__init__(args) self.download_id = args.download_id def generate(self): self.write('test.py', self.template('comic_test')) def main(): parser = argparse.ArgumentParser() parser.add_argument( '-a', '--author', default=gitconfig('user.name'), type=u8) parser.add_argument( '-e', '--email', default=gitconfig('user.email'), type=u8) subparsers = parser.add_subparsers() recipes = [BaseRecipe, ComicRecipe, ComicTestRecipe, CapRecipe] for recipe in recipes: recipe.configure_subparser(subparsers) args = parser.parse_args() recipe = args.recipe(args) recipe.generate() if __name__ == '__main__': main() ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0017254�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/base_browser.py���������������������������������������������������0000664�0000000�0000000�00000000743�13034501105�0022307�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.browser import PagesBrowser, URL from .pages import Page1, Page2 class ${r.classname}Browser(PagesBrowser): BASEURL = 'http://www.${r.name}.com' page1 = URL('/page1\?id=(?P<id>.+)', Page1) page2 = URL('/page2', Page2) def get_stuff(self, _id): self.page1.go(id=_id) assert self.page1.is_here() self.page.do_stuff(_id) assert self.page2.is_here() return self.page.do_more_stuff() �����������������������������weboob-1.2/tools/boilerplate_data/base_module.py����������������������������������������������������0000664�0000000�0000000�00000000614�13034501105�0022106�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.tools.backend import Module from .browser import ${r.classname}Browser __all__ = ['${r.classname}Module'] class ${r.classname}Module(Module): NAME = '${r.name}' DESCRIPTION = u'${r.name} website' MAINTAINER = u'${r.author}' EMAIL = '${r.email}' LICENSE = 'AGPLv3+' VERSION = '${r.version}' BROWSER = ${r.classname}Browser ��������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/base_pages.py�����������������������������������������������������0000664�0000000�0000000�00000000373�13034501105�0021722�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.browser.pages import HTMLPage class Page1(HTMLPage): def do_stuff(self, _id): raise NotImplementedError() class Page2(HTMLPage): def do_more_stuff(self): raise NotImplementedError() ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/base_test.py������������������������������������������������������0000664�0000000�0000000�00000000211�13034501105�0021571�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.tools.test import BackendTest class ${r.classname}Test(BackendTest): MODULE = '${r.name}' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/cap_module.py�����������������������������������������������������0000664�0000000�0000000�00000000732�13034501105�0021740�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.tools.backend import Module from ${r.capmodulename} import ${r.capname} from .browser import ${r.classname}Browser __all__ = ['${r.classname}Module'] class ${r.classname}Module(Module, ${r.capname}): NAME = '${r.name}' DESCRIPTION = u'${r.name} website' MAINTAINER = u'${r.author}' EMAIL = '${r.email}' LICENSE = 'AGPLv3+' VERSION = '${r.version}' BROWSER = ${r.classname}Browser ${r.methods_code} ��������������������������������������weboob-1.2/tools/boilerplate_data/comic_module.py���������������������������������������������������0000664�0000000�0000000�00000001377�13034501105�0022275�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.tools.capabilities.gallery.genericcomicreader import GenericComicReaderModule, DisplayPage __all__ = ['${r.classname}Module'] class ${r.classname}Module(GenericComicReaderModule): NAME = '${r.name}' DESCRIPTION = u'${r.name} manga reading site' MAINTAINER = u'${r.author}' EMAIL = '${r.email}' VERSION = '${r.version}' LICENSE = 'AGPLv3+' DOMAIN = 'www.${r.name}.com' BROWSER_PARAMS = dict( img_src_xpath="//img[@id='comic_page']/@src", page_list_xpath="(//select[@id='page_select'])[1]/option/@value") ID_REGEXP = r'[^/]+/[^/]+' URL_REGEXP = r'.+${r.name}.com/(%s).+' % ID_REGEXP ID_TO_URL = 'http://www.${r.name}.com/%s' PAGES = {URL_REGEXP: DisplayPage} �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/comic_test.py�����������������������������������������������������0000664�0000000�0000000�00000000442�13034501105�0021757�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from weboob.tools.capabilities.gallery.genericcomicreadertest import GenericComicReaderTest class ${r.classname}BackendTest(GenericComicReaderTest): MODULE = '${r.name}' def test_download(self): return self._test_download('${r.download_id}') ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/init.py�����������������������������������������������������������0000664�0000000�0000000�00000000153�13034501105�0020570�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<%inherit file="layout.py"/> from .module import ${r.classname}Module __all__ = ['${r.classname}Module'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/boilerplate_data/layout.py���������������������������������������������������������0000664�0000000�0000000�00000001344�13034501105�0021145�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������${coding} # Copyright(C) ${r.year} ${r.author} # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. ${self.body()}\ ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/certhash.py������������������������������������������������������������������������0000775�0000000�0000000�00000000256�13034501105�0016142�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python from __future__ import print_function import sys from weboob.deprecated.browser import StandardBrowser print(StandardBrowser()._certhash(sys.argv[1])) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/debpydep.py������������������������������������������������������������������������0000775�0000000�0000000�00000002360�13034501105�0016133�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import print_function import os import subprocess import sys selection = set() dependencies = set() for root, dirs, files in os.walk(sys.argv[1]): for f in files: if f.endswith('.py') and f != '__init__.py': s = "from %s import %s" % (root.strip('/').replace('/', '.'), f[:-3]) try: exec(s) except ImportError as e: print(str(e), file=sys.stderr) else: m = eval(f[:-3]) for attrname in dir(m): try: attr = getattr(m, attrname) selection.add(attr.__file__) except AttributeError: pass for f in selection: f = f.replace('.pyc', '.py') try: f = os.path.abspath(os.path.join(os.path.split(f)[0], os.readlink(f))) except OSError: pass p = subprocess.Popen(['/usr/bin/dpkg', '-S', f], stdout=subprocess.PIPE, stderr=subprocess.PIPE) if p.wait() == 0: for line in p.stdout.readlines(): dependencies.add(line.strip().split(':')[0]) else: print('not found: %s' % f) for d in dependencies: print(d) ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/generate_changelog_modules.sh������������������������������������������������������0000775�0000000�0000000�00000001101�13034501105�0021642�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/bash # Usage: weboob$ tools/generate_changelog_modules.sh TAG "list of hash" [show] BEGIN=$1 EXCLUDE=$2 SHOW=$3 for a in modules/* do if [ -d $a ] then MODULE=`basename $a` LOG=`git log --format="%H:::* %s" --date-order --reverse "$BEGIN..HEAD" -- $a` for b in $EXCLUDE do LOG=$(echo "$LOG" |grep -v $b) done if [ -n "$LOG" ] then if [ -n "$SHOW" ] then echo "$LOG" | awk -F ":::" '{print $1}' | git show --stdin else echo -e "\tModules: $MODULE" echo "$LOG" | awk -F ":::" '{print "\t"$2}' echo "" fi fi fi done ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/local_install.py�������������������������������������������������������������������0000664�0000000�0000000�00000003255�13034501105�0017160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import print_function import os import subprocess import sys if '--deps' in sys.argv: sys.argv.remove('--deps') deps = [] else: deps = ['--nodeps'] print("Weboob local installer") print() if len(sys.argv) < 2: print("This tool will install Weboob to be usuable without requiring") print("messing with your system, which should only be touched by a package manager.") print() print("Usage: %s DESTINATION [OPTIONS]" % sys.argv[0]) print() print("By default, no dependencies are installed, as you should try") print("to install them from your package manager as much as possible.") print("To install all the missing dependencies, add the option --deps") print("at the end of the command line.") print() print("Error: Please provide a destination, " "for example ‘%s/bin’" % os.getenv('HOME'), file=sys.stderr) sys.exit(1) else: dest = os.path.expanduser(sys.argv[1]) print("Installing weboob applications into ‘%s’." % dest) subprocess.check_call( [sys.executable, 'setup.py', 'install', '--user', '--install-scripts=%s' % dest] + sys.argv[2:] + deps, cwd=os.path.join(os.path.dirname(__file__), os.pardir)) subprocess.check_call([sys.executable, os.path.join(dest, 'weboob-config'), 'update']) print() print("Installation done. Applications are available in ‘%s’." % dest) print("You can remove the source files.") print() print("To have easy access to the Weboob applications,") print("you should add the following line to your ~/.bashrc or ~/.zshrc file:") print("export PATH=\"$PATH:%s\"" % dest) print("And then restart your shells.") ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/local_install.sh�������������������������������������������������������������������0000775�0000000�0000000�00000000512�13034501105�0017136�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e if [ -z "${PYTHON}" ]; then which python >/dev/null 2>&1 && PYTHON=$(which python) which python2 >/dev/null 2>&1 && PYTHON=$(which python2) which python2.7 >/dev/null 2>&1 && PYTHON=$(which python2.7) fi ${PYTHON} "$(dirname $0)/stale_pyc.py" exec "${PYTHON}" "$(dirname $0)/local_install.py" "$@" ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/local_run.py�����������������������������������������������������������������������0000664�0000000�0000000�00000003160�13034501105�0016311�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import print_function import subprocess import sys import os if len(sys.argv) < 2: print("Usage: %s SCRIPTNAME [args]" % sys.argv[0]) sys.exit(1) else: script = sys.argv[1] args = sys.argv[2:] project = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) wd = os.path.join(project, 'localconfig') if not os.path.isdir(wd): os.makedirs(wd) paths = os.getenv('PYTHONPATH', None) if not paths: paths = sys.path else: paths = paths.split(':') if project not in paths: paths.insert(0, project) env = os.environ.copy() env['PYTHONPATH'] = ':'.join(p for p in paths if p) env['WEBOOB_WORKDIR'] = wd env['WEBOOB_DATADIR'] = wd env['WEBOOB_BACKENDS'] = os.getenv('WEBOOB_LOCAL_BACKENDS', os.getenv('WEBOOB_BACKENDS', os.path.join(os.environ.get('XDG_CONFIG_HOME', os.path.join(os.path.expanduser('~'), '.config')), 'weboob', 'backends'))) modpath = os.getenv('WEBOOB_MODULES', os.path.join(project, 'modules')) with open(os.path.join(wd, 'sources.list'), 'w') as f: f.write("file://%s\n" % modpath) # Hide output unless there is an error p = subprocess.Popen( [sys.executable, os.path.join(project, 'scripts', 'weboob-config'), 'update'], env=env, stdout=subprocess.PIPE) s = p.communicate() if p.returncode != 0: print(s[0]) sys.exit(p.returncode) if os.path.exists(script): spath = script else: spath = os.path.join(project, 'scripts', script) os.execvpe( sys.executable, ['-Wall', '-s', spath] + args, env) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/local_run.sh�����������������������������������������������������������������������0000775�0000000�0000000�00000000506�13034501105�0016277�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e if [ -z "${PYTHON}" ]; then which python >/dev/null 2>&1 && PYTHON=$(which python) which python2 >/dev/null 2>&1 && PYTHON=$(which python2) which python2.7 >/dev/null 2>&1 && PYTHON=$(which python2.7) fi ${PYTHON} "$(dirname $0)/stale_pyc.py" exec "${PYTHON}" "$(dirname $0)/local_run.py" "$@" ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/make_man.py������������������������������������������������������������������������0000775�0000000�0000000�00000022404�13034501105�0016110�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import absolute_import, print_function import imp import inspect import optparse import os import re import sys import tempfile import time from datetime import datetime from weboob.tools.application.base import Application BASE_PATH = os.path.join(os.path.dirname(__file__), os.pardir) DEST_DIR = 'man' class ManpageHelpFormatter(optparse.HelpFormatter): def __init__(self, app, indent_increment=0, max_help_position=0, width=80, short_first=1): optparse.HelpFormatter.__init__(self, indent_increment, max_help_position, width, short_first) self.app = app def format_heading(self, heading): return ".SH %s\n" % heading.upper() def format_usage(self, usage): txt = '' for line in usage.split('\n'): line = line.lstrip().split(' ', 1) if len(txt) > 0: txt += '.br\n' txt += '.B %s\n' % line[0] arg_re = re.compile(r'([\[\s])([\w_]+)') args = re.sub(arg_re, r"\1\\fI\2\\fR", line[1]) txt += args txt += '\n' return '.SH SYNOPSIS\n%s' % txt def format_description(self, description): desc = u'.SH DESCRIPTION\n.LP\n\n%s\n' % description if hasattr(self.app, 'CAPS'): self.app.weboob.modules_loader.load_all() caps = self.app.CAPS if isinstance(self.app.CAPS, tuple) else (self.app.CAPS,) modules = [] for name, module in self.app.weboob.modules_loader.loaded.iteritems(): if module.has_caps(*caps): modules.append(u'* %s (%s)' % (name, module.description)) if len(modules) > 0: desc += u'\n.SS Supported websites:\n' desc += u'\n.br\n'.join(sorted(modules)) return desc def format_commands(self, commands): s = u'' for section, cmds in commands.iteritems(): if len(cmds) == 0: continue s += '.SH %s COMMANDS\n' % section.upper() for cmd in sorted(cmds): s += '.TP\n' h = cmd.split('\n') if ' ' in h[0]: cmdname, args = h[0].split(' ', 1) arg_re = re.compile(r'([A-Z_]+)') args = re.sub(arg_re, r"\\fI\1\\fR", args) s += '\\fB%s\\fR %s' % (cmdname, args) else: s += '\\fB%s\\fR' % h[0] s += '%s\n' % '\n.br\n'.join(h[1:]) return s def format_option_strings(self, option): opts = optparse.HelpFormatter.format_option_strings(self, option).split(", ") return ".TP\n" + ", ".join("\\fB%s\\fR" % opt for opt in opts) def main(): scripts_path = os.path.join(BASE_PATH, "scripts") files = os.listdir(scripts_path) # Create a fake "scripts" modules to import the scripts into sys.modules["scripts"] = imp.new_module("scripts") for fname in files: fpath = os.path.join(scripts_path, fname) if os.path.isfile(fpath) and os.access(fpath, os.X_OK): with open(fpath) as f: # Python will likely want create a compiled file, we provide a place tmpdir = os.path.join(tempfile.gettempdir(), "weboob", "make_man") if not os.path.isdir(tmpdir): os.makedirs(tmpdir) tmpfile = os.path.join(tmpdir, fname) desc = ("", "U", imp.PY_SOURCE) try: script = imp.load_module("scripts.%s" % fname, f, tmpfile, desc) except ImportError as e: print("Unable to load the %s script (%s)" % (fname, e), file=sys.stderr) else: print("Loaded %s" % fname) # Find the applications we can handle for klass in script.__dict__.itervalues(): if inspect.isclass(klass) and issubclass(klass, Application) and klass.VERSION: analyze_application(klass, fname) finally: # Cleanup compiled files if needed if (os.path.isfile(tmpfile + "c")): os.unlink(tmpfile + "c") def format_title(title): return re.sub(r'^(.+):$', r'.SH \1\n.TP', title.group().upper()) # XXX useful because the PyQt QApplication destructor crashes sometimes. By # keeping every applications until program end, it prevents to stop before # every manpages have been generated. If it crashes at exit, it's not a # really a problem. applications = [] def analyze_application(app, script_name): application = app() applications.append(application) formatter = ManpageHelpFormatter(application) # patch the application application._parser.prog = "%s" % script_name application._parser.formatter = formatter helptext = application._parser.format_help(formatter) cmd_re = re.compile(r'^.+ Commands:$', re.MULTILINE) helptext = re.sub(cmd_re, format_title, helptext) helptext = helptext.replace("-", r"\-") coding = r'.\" -*- coding: utf-8 -*-' comment = r'.\" This file was generated automatically by tools/make_man.sh.' header = '.TH %s 1 "%s" "%s %s"' % (script_name.upper(), time.strftime("%d %B %Y"), script_name, app.VERSION.replace('.', '\\&.')) name = ".SH NAME\n%s \- %s" % (script_name, application.SHORT_DESCRIPTION) condition = """.SH CONDITION The \-c and \-\-condition is a flexible way to filter and get only interesting results. It supports conditions on numerical values, dates, durations and strings. Dates are given in YYYY\-MM\-DD or YYYY\-MM\-DD HH:MM format. Durations look like XhYmZs where X, Y and Z are integers. Any of them may be omitted. For instance, YmZs, XhZs or Ym are accepted. The syntax of one expression is "\\fBfield operator value\\fR". The field to test is always the left member of the expression. .LP The field is a member of the objects returned by the command. For example, a bank account has "balance", "coming" or "label" fields. .SS The following operators are supported: .TP = Test if object.field is equal to the value. .TP != Test if object.field is not equal to the value. .TP > Test if object.field is greater than the value. If object.field is date, return true if value is before that object.field. .TP < Test if object.field is less than the value. If object.field is date, return true if value is after that object.field. .TP | This operator is available only for string fields. It works like the Unix standard \\fBgrep\\fR command, and returns True if the pattern specified in the value is in object.field. .SS Expression combination .LP You can make a expression combinations with the keywords \\fB" AND "\\fR, \\fB" OR "\\fR an \\fB" LIMIT "\\fR. .LP The \\fBLIMIT\\fR keyword can be used to limit the number of items upon which running the expression. \\fBLIMIT\\fR can only be placed at the end of the expression followed by the number of elements you want. .SS Examples: .nf .B boobank ls \-\-condition 'label=Livret A' .fi Display only the "Livret A" account. .PP .nf .B boobank ls \-\-condition 'balance>10000' .fi Display accounts with a lot of money. .PP .nf .B boobank history account@backend \-\-condition 'label|rewe' .fi Get transactions containing "rewe". .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 AND date<2013\-12\-09' .fi Get transactions betweens the 2th December and 8th December 2013. .PP .nf .B boobank history account@backend \-\-condition 'date>2013\-12\-01 LIMIT 10' .fi Get transactions after the 2th December in the last 10 transactions """ footer = """.SH COPYRIGHT %s .LP For full copyright information see the COPYING file in the weboob package. .LP .RE .SH FILES "~/.config/weboob/backends" """ % application.COPYRIGHT.replace('YEAR', '%d' % datetime.today().year) if len(app.CONFIG) > 0: footer += '\n\n "~/.config/weboob/%s"' % app.APPNAME # Skip internal applications. footer += "\n\n.SH SEE ALSO\nHome page: http://weboob.org/applications/%s" % application.APPNAME mantext = u"%s\n%s\n%s\n%s\n%s\n%s\n%s" % (coding, comment, header, name, helptext, condition, footer) with open(os.path.join(BASE_PATH, DEST_DIR, "%s.1" % script_name), 'w+') as manfile: for line in mantext.split('\n'): manfile.write('%s\n' % line.lstrip().encode('utf-8')) print("wrote %s/%s.1" % (DEST_DIR, script_name)) if __name__ == '__main__': sys.exit(main()) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/make_man.sh������������������������������������������������������������������������0000775�0000000�0000000�00000001432�13034501105�0016070�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh # stop on failure set -e # Use C local to avoid local dates in headers export LANG=en_US.utf8 # disable termcolor export ANSI_COLORS_DISABLED=1 [ -z "${TMPDIR}" ] && TMPDIR="/tmp" # do not allow undefined variables anymore set -u WEBOOB_TMPDIR=$(mktemp -d "${TMPDIR}/weboob_man.XXXXX") # path to sources WEBOOB_DIR=$(cd $(dirname $0)/.. && pwd -P) touch "${WEBOOB_TMPDIR}/backends" chmod 600 "${WEBOOB_TMPDIR}/backends" echo "file://$WEBOOB_DIR/modules" > "${WEBOOB_TMPDIR}/sources.list" export WEBOOB_WORKDIR="${WEBOOB_TMPDIR}" export WEBOOB_DATADIR="${WEBOOB_TMPDIR}" export PYTHONPATH="${WEBOOB_DIR}" "${WEBOOB_DIR}/scripts/weboob-config" update "${WEBOOB_DIR}/tools/make_man.py" # allow failing commands past this point STATUS=$? rm -rf "${WEBOOB_TMPDIR}" exit $STATUS ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/modules_testing_grid.py������������������������������������������������������������0000775�0000000�0000000�00000004010�13034501105�0020543�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python # -*- coding: utf-8 -*- """ Script to format XUNIT output from unittests as a JSON string ready to be sent to a [Weboob-CI](https://github.com/Phyks/weboob-ci) instance. * `XUNIT` is the XUNIT file to handle. * `ORIGIN` is an origin string as described in the Weboob-CI documentation (basically just a string to identify the source of the unittests results). """ from __future__ import print_function import json import sys import xunitparser def main(xunit, origin): with open(xunit, "r") as fh: ts, tr = xunitparser.parse(fh) # Get test results for each module modules = {} other_testcases = [] for tc in ts: if not tc.classname.startswith("modules."): other_testcases.append(repr(tc)) continue module = tc.classname.split(".")[1] # In the following, we consider # bad > skipped > good # and only make update of a module status according to this order if tc.good: if tc.skipped: # Set to skipped only if previous test was good if module not in modules or modules[module] == "good": modules[module] = "skipped" else: # Set to good only if no previous result if module not in modules: modules[module] = "good" else: # Always set to bad on failed test modules[module] = "bad" # Agregate results by test result rather than module results = { "good": [], "bad": [], "skipped": [] } for module in modules: results[modules[module]].append(module) return { "origin": origin, "modules": results, "others": other_testcases } if __name__ == "__main__": if len(sys.argv) < 3: sys.exit("Usage: %s XUNIT_FILE ORIGIN" % (sys.argv[0])) print( json.dumps( main(sys.argv[1], sys.argv[2]), sort_keys=True, indent=4, separators=(',', ': ') ) ) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/pyflakes.sh������������������������������������������������������������������������0000775�0000000�0000000�00000004154�13034501105�0016142�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/bash -u cd $(dirname $0) cd .. PYFILES=$(git ls-files|grep '^scripts\|\.py$'|grep -v boilerplate_data|tr '\n' ' ') grep -n 'class [^( ]\+:$' ${PYFILES} && echo 'Error: old class style found, always inherit object' && exit 3 grep -n $'\t\|\s$' $PYFILES && echo 'Error: tabs or trailing whitespace found, remove them' && exit 4 grep -Fn '.setlocale' ${PYFILES} && echo 'Error: do not use setlocale' && exit 5 grep -Fn '__future__ import with_statement' ${PYFILES} && echo 'Error: with_statement useless as we do not support Python 2.5' && exit 6 grep -nE '^[[:space:]]+except [[:alnum:] ]+,[[:alnum:] ]+' ${PYFILES} && echo 'Error: use new "as" way of naming exceptions' && exit 7 grep -nE "^ *print " ${PYFILES} && echo 'Error: Use the print function' && exit 8 grep -Fn ".has_key" ${PYFILES} && echo 'Error: Deprecated, use operator "in"' && exit 9 grep -Fn "os.isatty" ${PYFILES} && echo 'Error: Use stream.isatty() instead of os.isatty(stream.fileno())' && exit 10 grep -Fn "raise StopIteration" ${PYFILES} && echo 'Error: PEP 479' && exit 11 MODULE_FILES=$(git ls-files|grep '^modules/.*\.py$'|tr '\n' ' ') grep -nE "^ *print(\(| )" ${MODULE_FILES} && echo 'Error: Use of print in modules is forbidden, use logger instead' && exit 20 FLAKE8="" if which flake8 >/dev/null 2>&1; then FLAKE8=$(which flake8) fi if which flake8-python2 >/dev/null 2>&1; then FLAKE8=$(which flake8-python2) fi if [ -n "${FLAKE8}" ]; then exec env python2 ${FLAKE8} --select=E9,F *.py $PYFILES else PYFLAKES="" if which pyflakes >/dev/null 2>&1; then PYFLAKES=pyflakes fi if which pyflakes-python2 >/dev/null 2>&1; then PYFLAKES=pyflakes-python2 fi if [ -z "${PYFLAKES}" ] then echo "pyflakes not found" exit 1 fi # check for modern pyflakes if ${PYFLAKES} --version >/dev/null 2>&1; then exec ${PYFLAKES} $PYFILES else # hide error reported by mistake. # grep will return 0 only if it founds something, but our script # wants to return 0 when it founds nothing! ${PYFLAKES} $PYFILES | grep -v redefinition && exit 1 || exit 0 fi fi ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/pyreverse.sh�����������������������������������������������������������������������0000775�0000000�0000000�00000000433�13034501105�0016344�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/bash # # Examples: # pyreverse.sh weboob.backends.aum # # pyreverse is included in pylint Debian package function usage() { echo "pyreverse.sh <package_name>" exit } [ -z "$1" ] && usage PYTHONPATH="$(dirname $0)/../modules/$1" pyreverse -p "$1" -o pdf -a1 -s1 "." �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/run_tests.sh�����������������������������������������������������������������������0000775�0000000�0000000�00000012530�13034501105�0016347�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/bash # Mai available environment variables # * RSYNC_TARGET: target on which to rsync the xunit output. # * XUNIT_OUT: file in which xunit output should be saved. # * WEBOOB_BACKENDS: path to the Weboob backends file to use. # * WEBOOB_CI_TARGET: URL of your Weboob-CI instance. # * WEBOOB_CI_ORIGIN: origin for the Weboob-CI data. # stop on failure set -e # path to sources WEBOOB_DIR=$(cd $(dirname $0)/.. && pwd -P) BACKEND="${1}" if [ -z "${WEBOOB_WORKDIR}" ]; then # use the old workdir by default WEBOOB_WORKDIR="${HOME}/.weboob" # but if we can find a valid xdg workdir, switch to it [ "${XDG_CONFIG_HOME}" != "" ] || XDG_CONFIG_HOME="${HOME}/.config" [ -d "${XDG_CONFIG_HOME}/weboob" ] && WEBOOB_WORKDIR="${XDG_CONFIG_HOME}/weboob" fi [ -z "${TMPDIR}" ] && TMPDIR="/tmp" WEBOOB_TMPDIR=$(mktemp -d "${TMPDIR}/weboob_test.XXXXX") [ -z "${WEBOOB_BACKENDS}" ] && WEBOOB_BACKENDS="${WEBOOB_WORKDIR}/backends" [ -z "${WEBOOB_MODULES}" ] && WEBOOB_MODULES="${WEBOOB_DIR}/modules" [ -z "${PYTHONPATH}" ] && PYTHONPATH="" # allow private environment setup [ -f "${WEBOOB_WORKDIR}/pre-test.sh" ] && source "${WEBOOB_WORKDIR}/pre-test.sh" # setup xunit reporting (buildbot slaves only) if [ -n "${RSYNC_TARGET}" ]; then # by default, builder name is containing directory name [ -z "${BUILDER_NAME}" ] && BUILDER_NAME=$(basename $(readlink -e $(dirname $0)/../..)) XUNIT_OUT="${WEBOOB_TMPDIR}/xunit.xml" else RSYNC_TARGET="" fi # Avoid undefined variables if [ ! -n "${XUNIT_OUT}" ]; then XUNIT_OUT="" fi # Handle Weboob-CI variables if [ -n "${WEBOOB_CI_TARGET}" ]; then if [ ! -n "${WEBOOB_CI_ORIGIN}" ]; then WEBOOB_CI_ORIGIN="Weboob unittests run" fi # Set up xunit reporting XUNIT_OUT="${WEBOOB_TMPDIR}/xunit.xml" else WEBOOB_CI_TARGET="" fi # find executables if [ -z "${PYTHON}" ]; then which python >/dev/null 2>&1 && PYTHON=$(which python) which python2 >/dev/null 2>&1 && PYTHON=$(which python2) which python2.7 >/dev/null 2>&1 && PYTHON=$(which python2.7) fi if [ -z "${NOSE}" ]; then which nosetests >/dev/null 2>&1 && NOSE=$(which nosetests) which nosetests2 >/dev/null 2>&1 && NOSE=$(which nosetests2) which nosetests-python2.7 >/dev/null 2>&1 && NOSE=$(which nosetests-python2.7) fi if [ -z "${PYTHON}" ]; then echo "Python required" exit 1 fi if [ -z "${NOSE}" ]; then echo "python-nose required" exit 1 fi # do not allow undefined variables anymore set -u if [ -f "${WEBOOB_BACKENDS}" ]; then cp "${WEBOOB_BACKENDS}" "${WEBOOB_TMPDIR}/backends" else touch "${WEBOOB_TMPDIR}/backends" chmod go-r "${WEBOOB_TMPDIR}/backends" fi # xunit nose setup if [ -n "${XUNIT_OUT}" ]; then XUNIT_ARGS="--with-xunit --xunit-file=${XUNIT_OUT}" else XUNIT_ARGS="" fi ${PYTHON} "$(dirname $0)/stale_pyc.py" echo "file://${WEBOOB_MODULES}" > "${WEBOOB_TMPDIR}/sources.list" export WEBOOB_WORKDIR="${WEBOOB_TMPDIR}" export WEBOOB_DATADIR="${WEBOOB_TMPDIR}" export PYTHONPATH="${WEBOOB_DIR}:${PYTHONPATH}" export NOSE_NOPATH="1" ${PYTHON} "${WEBOOB_DIR}/scripts/weboob-config" update # allow failing commands past this point set +e set -o pipefail if [ -n "${BACKEND}" ]; then ${PYTHON} ${NOSE} -c /dev/null -sv "${WEBOOB_MODULES}/${BACKEND}/test.py" ${XUNIT_ARGS} STATUS=$? STATUS_CORE=0 else echo "=== Weboob ===" CORE_TESTS=$(mktemp) ${PYTHON} ${NOSE} --cover-package weboob -c ${WEBOOB_DIR}/setup.cfg -sv 2>&1 | tee "${CORE_TESTS}" STATUS_CORE=$? echo "=== Modules ===" MODULES_TESTS=$(mktemp) MODULES_TO_TEST=$(find "${WEBOOB_MODULES}" -name "test.py" | sort | xargs echo) ${PYTHON} ${NOSE} --with-coverage --cover-package modules -c /dev/null -sv ${XUNIT_ARGS} ${MODULES_TO_TEST} 2>&1 | tee ${MODULES_TESTS} STATUS=$? # Compute total coverage echo "=== Total coverage ===" CORE_STMTS=$(grep "TOTAL" ${CORE_TESTS} | awk '{ print $2; }') CORE_MISS=$(grep "TOTAL" ${CORE_TESTS} | awk '{ print $3; }') CORE_COVERAGE=$(grep "TOTAL" ${CORE_TESTS} | awk '{ print $4; }') MODULES_STMTS=$(grep "TOTAL" ${MODULES_TESTS} | awk '{ print $2; }') MODULES_MISS=$(grep "TOTAL" ${MODULES_TESTS} | awk '{ print $3; }') MODULES_COVERAGE=$(grep "TOTAL" ${MODULES_TESTS} | awk '{ print $4; }') echo "CORE COVERAGE: ${CORE_COVERAGE}" echo "MODULES COVERAGE: ${MODULES_COVERAGE}" TOTAL_STMTS=$((${CORE_STMTS} + ${MODULES_STMTS})) TOTAL_MISS=$((${CORE_MISS} + ${MODULES_MISS})) TOTAL_COVERAGE=$((100 * (${TOTAL_STMTS} - ${TOTAL_MISS}) / ${TOTAL_STMTS})) echo "TOTAL: ${TOTAL_COVERAGE}%" # removal of temp files rm ${CORE_TESTS} rm ${MODULES_TESTS} fi # Rsync xunit transfer if [ -n "${RSYNC_TARGET}" ]; then rsync -iz "${XUNIT_OUT}" "${RSYNC_TARGET}/${BUILDER_NAME}-$(date +%s).xml" rm "${XUNIT_OUT}" fi # Weboob-CI upload if [ -n "${WEBOOB_CI_TARGET}" ]; then JSON_MODULE_MATRIX=$(${PYTHON} "${WEBOOB_DIR}/tools/modules_testing_grid.py" "${XUNIT_OUT}" "${WEBOOB_CI_ORIGIN}") curl -H "Content-Type: application/json" --data "${JSON_MODULE_MATRIX}" "${WEBOOB_CI_TARGET}/api/v1/modules" rm "${XUNIT_OUT}" fi # safe removal rm -r "${WEBOOB_TMPDIR}/icons" "${WEBOOB_TMPDIR}/repositories" "${WEBOOB_TMPDIR}/modules" "${WEBOOB_TMPDIR}/keyrings" rm "${WEBOOB_TMPDIR}/backends" "${WEBOOB_TMPDIR}/sources.list" rmdir "${WEBOOB_TMPDIR}" [ $STATUS_CORE -gt 0 ] && exit $STATUS_CORE exit $STATUS ������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/stale_pyc.py�����������������������������������������������������������������������0000775�0000000�0000000�00000001324�13034501105�0016321�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python from __future__ import print_function import os import sys root = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) verbose = '-v' in sys.argv excludes = ('.git', '.svn', '__pycache__') for dirpath, dirnames, filenames in os.walk(root): for exclude in excludes: try: dirnames.remove(exclude) except ValueError: pass for filename in filenames: if filename.endswith('.pyc') or filename.endswith('pyo'): if not os.path.exists(os.path.join(dirpath, filename[:-1])): os.unlink(os.path.join(dirpath, filename)) if verbose: print(os.path.join(dirpath, filename)) ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/weboob_bash_completion�������������������������������������������������������������0000664�0000000�0000000�00000002422�13034501105�0020407�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Weboob completion for Bash # # vim: filetype=sh expandtab softtabstop=4 shiftwidth=4 # # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. # # This script can be distributed under the same license as the # weboob or bash packages. _weboob() { local cur args COMPREPLY=() cur=${COMP_WORDS[COMP_CWORD]} args="$(${COMP_WORDS[0]} --shell-completion)" COMPREPLY=( $(compgen -o default -W "${args}" -- "$cur" ) ) } hash weboob-config 2>/dev/null && weboob_applications=$(weboob-config applications 2>/dev/null) hash -d weboob-config for application in $weboob_applications do complete -F _weboob $application done ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/weboob_lint.py���������������������������������������������������������������������0000775�0000000�0000000�00000002115�13034501105�0016640�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/env python from __future__ import print_function # Hint: use this script with file:///path/to/local/modules/ in sources.list # if you want to correctly check all modules. from weboob.core import Weboob import os import sys weboob = Weboob() weboob.modules_loader.load_all() backends_without_tests = [] backends_without_icons = [] for name, backend in weboob.modules_loader.loaded.iteritems(): path = backend.package.__path__[0] if not os.path.exists(os.path.join(path, 'test.py')): backends_without_tests.append(name) if not os.path.exists(os.path.join(path, 'favicon.png')) and \ not os.path.exists(os.path.join(weboob.repositories.icons_dir, '%s.png' % name)) and \ not backend.icon: backends_without_icons.append(name) if backends_without_tests: backends_without_tests.sort() print('Modules without tests: %s' % backends_without_tests) if backends_without_icons: backends_without_icons.sort() print('Modules without icons: %s' % backends_without_icons) if backends_without_tests or backends_without_icons: sys.exit(1) ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/tools/weboob_lint.sh���������������������������������������������������������������������0000775�0000000�0000000�00000001245�13034501105�0016625�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh # stop on failure set -e [ -z "${TMPDIR}" ] && TMPDIR="/tmp" # do not allow undefined variables anymore set -u WEBOOB_TMPDIR=$(mktemp -d "${TMPDIR}/weboob_lint.XXXXX") # path to sources WEBOOB_DIR=$(cd $(dirname $0)/.. && pwd -P) touch "${WEBOOB_TMPDIR}/backends" chmod 600 "${WEBOOB_TMPDIR}/backends" echo "file://$WEBOOB_DIR/modules" > "${WEBOOB_TMPDIR}/sources.list" export WEBOOB_WORKDIR="${WEBOOB_TMPDIR}" export WEBOOB_DATADIR="${WEBOOB_TMPDIR}" export PYTHONPATH="${WEBOOB_DIR}" "${WEBOOB_DIR}/scripts/weboob-config" update "${WEBOOB_DIR}/tools/weboob_lint.py" # allow failing commands past this point STATUS=$? rm -rf "${WEBOOB_TMPDIR}" exit $STATUS �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/����������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0014076�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/__init__.py�����������������������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0016175�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0016564�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/__init__.py����������������������������������������������������������0000664�0000000�0000000�00000000000�13034501105�0020663�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobank/�������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020177�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobank/__init__.py��������������������������������������������������0000664�0000000�0000000�00000001426�13034501105�0022313�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .boobank import Boobank __all__ = ['Boobank'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobank/boobank.py���������������������������������������������������0000664�0000000�0000000�00000062452�13034501105�0022175�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2009-2011 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function import datetime import uuid from dateutil.relativedelta import relativedelta from dateutil.parser import parse as parse_date from decimal import Decimal, InvalidOperation from weboob.browser.browsers import APIBrowser from weboob.browser.profiles import Weboob from weboob.exceptions import BrowserHTTPError from weboob.capabilities.base import empty from weboob.capabilities.bank import CapBank, Account, Transaction, CapBankTransfer, \ Transfer, TransferStep from weboob.capabilities.contact import CapContact, Advisor from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['Boobank'] class OfxFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'raw', 'amount', 'category') TYPES_ACCTS = ['', 'CHECKING', 'SAVINGS', 'DEPOSIT', 'LOAN', 'MARKET', 'JOINT', 'CARD'] TYPES_TRANS = ['', 'DIRECTDEP', 'PAYMENT', 'CHECK', 'DEP', 'OTHER', 'ATM', 'POS', 'INT', 'FEE'] TYPES_CURRS = ['', 'EUR', 'CHF', 'USD'] balance = Decimal(0) coming = Decimal(0) def start_format(self, **kwargs): account = kwargs['account'] self.balance = account.balance self.coming = account.coming self.output(u'OFXHEADER:100') self.output(u'DATA:OFXSGML') self.output(u'VERSION:102') self.output(u'SECURITY:NONE') self.output(u'ENCODING:USASCII') self.output(u'CHARSET:1252') self.output(u'COMPRESSION:NONE') self.output(u'OLDFILEUID:NONE') self.output(u'NEWFILEUID:%s\n' % uuid.uuid1()) self.output(u'<OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>0<SEVERITY>INFO</STATUS>') self.output(u'<DTSERVER>%s113942<LANGUAGE>ENG</SONRS></SIGNONMSGSRSV1>' % datetime.date.today().strftime('%Y%m%d')) self.output(u'<BANKMSGSRSV1><STMTTRNRS><TRNUID>%s' % uuid.uuid1()) self.output(u'<STATUS><CODE>0<SEVERITY>INFO</STATUS><CLTCOOKIE>null<STMTRS>') self.output(u'<CURDEF>%s<BANKACCTFROM>' % (account.currency or 'EUR')) self.output(u'<BANKID>null') self.output(u'<BRANCHID>null') self.output(u'<ACCTID>%s' % account.id) try: account_type = self.TYPES_ACCTS[account.type] except IndexError: account_type = '' self.output(u'<ACCTTYPE>%s' % (account_type or 'CHECKING')) self.output(u'<ACCTKEY>null</BANKACCTFROM>') self.output(u'<BANKTRANLIST>') self.output(u'<DTSTART>%s' % datetime.date.today().strftime('%Y%m%d')) self.output(u'<DTEND>%s' % datetime.date.today().strftime('%Y%m%d')) def format_obj(self, obj, alias): # special case of coming operations with card ID if hasattr(obj, '_coming') and obj._coming and hasattr(obj, 'obj._cardid') and not empty(obj._cardid): result = u'<STMTTRN><TRNTYPE>%s\n' % obj._cardid elif obj.type != 0: result = u'<STMTTRN><TRNTYPE>%s\n' % self.TYPES_TRANS[obj.type] else: result = u'<STMTTRN><TRNTYPE>%s\n' % ('DEBIT' if obj.amount < 0 else 'CREDIT') result += u'<DTPOSTED>%s\n' % obj.date.strftime('%Y%m%d') result += u'<TRNAMT>%s\n' % obj.amount result += u'<FITID>%s\n' % obj.unique_id() if hasattr(obj, 'label') and not empty(obj.label): result += u'<NAME>%s</STMTTRN>' % obj.label.replace('&', '&') else: result += u'<NAME>%s</STMTTRN>' % obj.raw.replace('&', '&') return result def flush(self): self.output(u'</BANKTRANLIST>') self.output(u'<LEDGERBAL><BALAMT>%s' % self.balance) self.output(u'<DTASOF>%s</LEDGERBAL>' % datetime.date.today().strftime('%Y%m%d')) try: self.output(u'<AVAILBAL><BALAMT>%s' % (self.balance + self.coming)) except TypeError: self.output(u'<AVAILBAL><BALAMT>%s' % self.balance) self.output(u'<DTASOF>%s</AVAILBAL>' % datetime.date.today().strftime('%Y%m%d')) self.output(u'</STMTRS></STMTTRNRS></BANKMSGSRSV1></OFX>') class QifFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'raw', 'amount') def start_format(self, **kwargs): self.output(u'!Type:Bank') def format_obj(self, obj, alias): result = u'D%s\n' % obj.date.strftime('%d/%m/%y') result += u'T%s\n' % obj.amount if hasattr(obj, 'category') and not empty(obj.category): result += u'N%s\n' % obj.category result += u'M%s\n' % obj.raw result += u'^' return result class PrettyQifFormatter(QifFormatter): MANDATORY_FIELDS = ('id', 'date', 'raw', 'amount', 'category') def start_format(self, **kwargs): self.output(u'!Type:Bank') def format_obj(self, obj, alias): if hasattr(obj, 'rdate') and not empty(obj.rdate): result = u'D%s\n' % obj.rdate.strftime('%d/%m/%y') else: result = u'D%s\n' % obj.date.strftime('%d/%m/%y') result += u'T%s\n' % obj.amount if hasattr(obj, 'category') and not empty(obj.category): result += u'N%s\n' % obj.category if hasattr(obj, 'label') and not empty(obj.label): result += u'M%s\n' % obj.label else: result += u'M%s\n' % obj.raw result += u'^' return result class TransactionsFormatter(IFormatter): MANDATORY_FIELDS = ('date', 'label', 'amount') TYPES = ['', 'Transfer', 'Order', 'Check', 'Deposit', 'Payback', 'Withdrawal', 'Card', 'Loan', 'Bank'] def start_format(self, **kwargs): self.output(' Date Category Label Amount ') self.output('------------+------------+---------------------------------------------------+-----------') def format_obj(self, obj, alias): if hasattr(obj, 'category') and obj.category: _type = obj.category else: try: _type = self.TYPES[obj.type] except (IndexError, AttributeError): _type = '' label = obj.label if not label and hasattr(obj, 'raw'): label = obj.raw date = obj.date.strftime('%Y-%m-%d') if not empty(obj.date) else '' amount = obj.amount or Decimal('0') return ' %s %s %s %s' % (self.colored('%-10s' % date, 'blue'), self.colored('%-12s' % _type[:12], 'magenta'), self.colored('%-50s' % label[:50], 'yellow'), self.colored('%10.2f' % amount, 'green' if amount >= 0 else 'red')) class TransferFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'origin', 'recipient', 'amount') DISPLAYED_FIELDS = ('reason', ) def format_obj(self, obj, alias): result = u'------- Transfer %s -------\n' % obj.fullid result += u'Date: %s\n' % obj.date result += u'Origin: %s\n' % obj.origin result += u'Recipient: %s\n' % obj.recipient result += u'Amount: %.2f\n' % obj.amount if obj.reason: result += u'Reason: %s\n' % obj.reason return result class InvestmentFormatter(IFormatter): MANDATORY_FIELDS = ('label', 'quantity', 'unitvalue') DISPLAYED_FIELDS = ('code', 'diff') tot_valuation = Decimal(0) tot_diff = Decimal(0) def start_format(self, **kwargs): self.output(' Label Code Quantity Unit Value Valuation diff ') self.output('-------------------------------+--------------+------------+------------+------------+---------') def check_emptyness(self, obj): if not empty(obj): return (obj, '%11.2f') return ('---', '%11s') def format_obj(self, obj, alias): label = obj.label if not empty(obj.diff): diff = obj.diff elif not empty(obj.quantity) and not empty(obj.unitprice): diff = obj.valuation - (obj.quantity * obj.unitprice) else: diff = '---' format_diff = '%8s' if isinstance(diff, Decimal): format_diff = '%8.2f' self.tot_diff += diff if not empty(obj.quantity): quantity = obj.quantity format_quantity = '%11.2f' if obj.quantity._isinteger(): format_quantity = '%11d' else: format_quantity = '%11s' quantity = '---' unitvalue, format_unitvalue = self.check_emptyness(obj.unitvalue) valuation, format_valuation = self.check_emptyness(obj.valuation) if isinstance(valuation, Decimal): self.tot_valuation += obj.valuation if empty(obj.code) and not empty(obj.description): code = obj.description else: code = obj.code return u' %s %s %s %s %s %s' % \ (self.colored('%-30s' % label[:30], 'red'), self.colored('%-12s' % code[:12], 'yellow') if not empty(code) else ' ' * 12, self.colored(format_quantity % quantity, 'yellow'), self.colored(format_unitvalue % unitvalue, 'yellow'), self.colored(format_valuation % valuation, 'yellow'), self.colored(format_diff % diff, 'green' if diff >= 0 else 'red') ) def flush(self): self.output(u'-------------------------------+--------------+------------+------------+------------+---------') self.output(u' Total %s %s' % (self.colored('%11.2f' % self.tot_valuation, 'yellow'), self.colored('%9.2f' % self.tot_diff, 'green' if self.tot_diff >= 0 else 'red')) ) self.tot_valuation = Decimal(0) self.tot_diff = Decimal(0) class RecipientListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'label') def start_format(self, **kwargs): self.output('Available recipients:') def get_title(self, obj): return obj.label class AdvisorListFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'name') def start_format(self, **kwargs): self.output(' Bank Name Contacts') self.output('-----------------+------------------------------+-----------------------------------------') def format_obj(self, obj, alias): bank = obj.backend phones = "" contacts = [] if not empty(obj.phone): phones += obj.phone if not empty(obj.mobile): if phones != "": phones += " or %s" % obj.mobile else: phones += obj.mobile if phones: contacts.append(phones) for attr in ('email', 'agency', 'address'): value = getattr(obj, attr) if not empty(value): contacts.append(value) if len(contacts) > 0: first_contact = contacts.pop(0) else: first_contact = "" result = u" %s %s %s " % (self.colored('%-15s' % bank, 'yellow'), self.colored('%-30s' % obj.name, 'red'), self.colored("%-30s" % first_contact, 'green')) for contact in contacts: result += "\n %s %s" % ((" ") * 47, self.colored("%-25s" % contact, 'green')) return result class AccountListFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'label', 'balance', 'coming', 'type') totals = {} def start_format(self, **kwargs): self.output(' %s Account Balance Coming ' % ((' ' * 15) if not self.interactive else '')) self.output('------------------------------------------%s+----------+----------' % (('-' * 15) if not self.interactive else '')) def format_obj(self, obj, alias): if alias is not None: id = '%s (%s)' % (self.colored('%3s' % ('#' + alias), 'red', 'bold'), self.colored(obj.backend, 'blue', 'bold')) clean = '#%s (%s)' % (alias, obj.backend) if len(clean) < 15: id += (' ' * (15 - len(clean))) else: id = self.colored('%30s' % obj.fullid, 'red', 'bold') balance = obj.balance or Decimal('0') coming = obj.coming or Decimal('0') currency = obj.currency or 'EUR' result = u'%s %s %s %s' % (id, self.colored('%-25s' % obj.label[:25], 'yellow' if obj.type != Account.TYPE_LOAN else 'blue'), self.colored('%9.2f' % obj.balance, 'green' if balance >= 0 else 'red') if not empty(obj.balance) else ' ' * 9, self.colored('%9.2f' % obj.coming, 'green' if coming >= 0 else 'red') if not empty(obj.coming) else '') currency_totals = self.totals.setdefault(currency, {}) currency_totals.setdefault('balance', Decimal(0)) currency_totals.setdefault('coming', Decimal(0)) if obj.type != Account.TYPE_LOAN: currency_totals['balance'] += balance currency_totals['coming'] += coming return result def flush(self): self.output(u'------------------------------------------%s+----------+----------' % (('-' * 15) if not self.interactive else '')) for currency, currency_totals in sorted(self.totals.iteritems(), key=lambda (k,v): (v,k)): balance = currency_totals['balance'] coming = currency_totals['coming'] self.output(u'%s Total (%s) %s %s' % ( (' ' * 15) if not self.interactive else '', currency, self.colored('%8.2f' % balance, 'green' if balance >= 0 else 'red'), self.colored('%8.2f' % coming, 'green' if coming >= 0 else 'red')) ) self.totals.clear() class Boobank(ReplApplication): APPNAME = 'boobank' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon, Christophe Benz' CAPS = CapBank, CapBankTransfer DESCRIPTION = "Console application allowing to list your bank accounts and get their balance, " \ "display accounts history and coming bank operations, and transfer money from an account to " \ "another (if available)." SHORT_DESCRIPTION = "manage bank accounts" EXTRA_FORMATTERS = {'account_list': AccountListFormatter, 'recipient_list': RecipientListFormatter, 'transfer': TransferFormatter, 'qif': QifFormatter, 'pretty_qif': PrettyQifFormatter, 'ofx': OfxFormatter, 'ops_list': TransactionsFormatter, 'investment_list': InvestmentFormatter, 'advisor_list': AdvisorListFormatter, } DEFAULT_FORMATTER = 'table' COMMANDS_FORMATTERS = {'ls': 'account_list', 'list': 'account_list', 'transfer': 'transfer', 'history': 'ops_list', 'coming': 'ops_list', 'investment': 'investment_list', 'advisor': 'advisor_list', } COLLECTION_OBJECTS = (Account, Transaction, ) def bcall_error_handler(self, backend, error, backtrace): if isinstance(error, TransferStep): params = {} for key, value in error.fields.iteritems(): if key and value: params[key] = self.ask(value) #backend.config['accept_transfer'].set(v) params['backends'] = backend self.start_format() for transfer in self.do('transfer', error.transfer, **params): self.format(transfer) else: return ReplApplication.bcall_error_handler(self, backend, error, backtrace) def load_default_backends(self): self.load_backends(CapBank, storage=self.create_storage()) def _complete_account(self, exclude=None): if exclude: exclude = '%s@%s' % self.parse_id(exclude) return [s for s in self._complete_object() if s != exclude] def do_list(self, line): """ list [-U] List accounts. Use -U to disable sorting of results. """ return self.do_ls(line) def show_history(self, command, line): id, end_date = self.parse_command_args(line, 2, 1) account = self.get_object(id, 'get_account', []) if not account: print('Error: account "%s" not found (Hint: try the command "list")' % id, file=self.stderr) return 2 if end_date is not None: try: end_date = parse_date(end_date) except ValueError: print('"%s" is an incorrect date format (for example "%s")' % (end_date, (datetime.date.today() - relativedelta(months=1)).strftime('%Y-%m-%d')), file=self.stderr) return 3 old_count = self.options.count self.options.count = None self.start_format(account=account) for transaction in self.do(command, account, backends=account.backend): if end_date is not None and transaction.date < end_date: break self.format(transaction) if end_date is not None: self.options.count = old_count def complete_history(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_account() @defaultcount(10) def do_history(self, line): """ history ID [END_DATE] Display history of transactions. If END_DATE is supplied, list all transactions until this date. """ return self.show_history('iter_history', line) def complete_coming(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_account() @defaultcount(10) def do_coming(self, line): """ coming ID [END_DATE] Display future transactions. If END_DATE is supplied, show all transactions until this date. """ return self.show_history('iter_coming', line) def complete_transfer(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_account() if len(args) == 3: return self._complete_account(args[1]) def do_transfer(self, line): """ transfer ACCOUNT [RECIPIENT AMOUNT [REASON]] Make a transfer beetwen two account - ACCOUNT the source account - RECIPIENT the recipient - AMOUNT amount to transfer - REASON reason of transfer If you give only the ACCOUNT parameter, it lists all the available recipients for this account. """ id_from, id_to, amount, reason = self.parse_command_args(line, 4, 1) account = self.get_object(id_from, 'get_account', []) if not account: print('Error: account %s not found' % id_from, file=self.stderr) return 1 if not id_to: self.objects = [] self.set_formatter('table') self.set_formatter_header(u'Available recipients') self.start_format() for recipient in self.do('iter_transfer_recipients', account.id, backends=account.backend, caps=CapBankTransfer): self.cached_format(recipient) return 0 id_to, backend_name_to = self.parse_id(id_to) if account.backend != backend_name_to: print("Transfer between different backends is not implemented", file=self.stderr) return 4 try: amount = Decimal(amount) except (TypeError, ValueError, InvalidOperation): print('Error: please give a decimal amount to transfer', file=self.stderr) return 2 exec_date = datetime.date.today() if self.interactive: # Try to find the recipient label. It can be missing from # recipients list, for example for banks which allow transfers to # arbitrary recipients. to = id_to for recipient in self.do('iter_transfer_recipients', account.id, backends=account.backend, caps=CapBankTransfer): if recipient.id == id_to: to = recipient.label break print('Amount: %s%s' % (amount, account.currency_text)) print('From: %s' % account.label) print('To: %s' % to) print('Reason: %s' % (reason or '')) print('Date: %s' % exec_date) if not self.ask('Are you sure to do this transfer?', default=True): return self.start_format() transfer = Transfer() transfer.account_id = account.id transfer.recipient_id = id_to transfer.amount = amount transfer.label = reason transfer.exec_date = exec_date next(iter(self.do('transfer', transfer, backends=account.backend))) def do_investment(self, id): """ investment ID Display investments of an account. """ account = self.get_object(id, 'get_account', []) if not account: print('Error: account "%s" not found (Hint: try the command "list")' % id, file=self.stderr) return 2 self.start_format() for investment in self.do('iter_investment', account, backends=account.backend): self.format(investment) def do_budgea(self, line): """ budgea USERNAME PASSWORD Export your bank accounts and transactions to Budgea. Budgea is an online web and mobile application to manage your bank accounts. To avoid giving your credentials to this service, you can use this command. https://www.budgea.com """ username, password = self.parse_command_args(line, 2, 2) client = APIBrowser(baseurl='https://budgea.biapi.pro/2.0/') client.set_profile(Weboob(self.VERSION)) client.TIMEOUT = 60 try: r = client.request('auth/token', data={'username': username, 'password': password, 'application': 'weboob'}) except BrowserHTTPError as r: error = r.response.json() print('Error: {}'.format(error.get('message', error['code'])), file=self.stderr) return 1 client.session.headers['Authorization'] = 'Bearer %s' % r['token'] accounts = {} for account in client.request('users/me/accounts')['accounts']: if account['id_connection'] is None: accounts[account['number']] = account for account in self.do('iter_accounts'): if account.id not in accounts: r = client.request('users/me/accounts', data={'name': account.label, 'balance': account.balance, 'number': account.id, }) self.logger.debug(r) account_id = r['id'] else: account_id = accounts[account.id]['id'] transactions = [] for tr in self.do('iter_history', account, backends=account.backend): transactions.append({'original_wording': tr.raw, 'simplified_wording': tr.label, 'value': tr.amount, 'date': tr.date.strftime('%Y-%m-%d'), }) r = client.request('users/me/accounts/%s/transactions' % account_id, data={'transactions': transactions}) client.request('users/me/accounts/%s' % account_id, data={'balance': account.balance}) print('- %s (%s%s): %s new transactions' % (account.label, account.balance, account.currency_text, len(r))) def do_advisor(self, line): """ advisor Display advisors. """ self.start_format() found = 0 for advisor in self.do('iter_contacts', caps=CapContact): if isinstance(advisor, Advisor): self.format(advisor) found = 1 if not found: print('Error: no advisor found', file=self.stderr) ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobathon/�����������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020537�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobathon/__init__.py������������������������������������������������0000664�0000000�0000000�00000001426�13034501105�0022653�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .boobathon import Boobathon __all__ = ['Boobathon'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobathon/boobathon.py�����������������������������������������������0000664�0000000�0000000�00000066013�13034501105�0023072�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function from datetime import datetime, timedelta import re import sys from urlparse import urlsplit from random import choice from weboob.capabilities.content import CapContent from weboob.tools.application.repl import ReplApplication from weboob.tools.ordereddict import OrderedDict __all__ = ['Boobathon'] class Task(object): STATUS_NONE = 0 STATUS_PROGRESS = 1 STATUS_DONE = 2 def __init__(self, backend, capability): self.backend = backend self.capability = capability self.status = self.STATUS_NONE self.date = None self.branch = u'' def __repr__(self): return '<Task (%s,%s)>' % (self.backend, self.capability) class Member(object): def __init__(self, id, name): self.name = name self.id = id self.tasks = [] self.availabilities = u'' self.repository = None self.hardware = u'' self.is_me = False def shortname(self): name = self.name if len(name) > 20: name = '%s..' % name[:18] return name class Event(object): def __init__(self, name, backend): self.my_id = backend.browser.get_userid() self.name = 'wiki/weboob/%s' % name self.description = None self.date = None self.begin = None self.end = None self.location = None self.winner = None self.backend = backend self.members = OrderedDict() self.load() def get_me(self): return self.members.get(self.backend.browser.get_userid(), None) def currently_in_event(self): if not self.date or not self.begin or not self.end: return False return self.begin < datetime.now() < self.end def is_closed(self): return self.end < datetime.now() def format_duration(self): if not self.begin or not self.end: return None delta = self.end - self.begin return '%02d:%02d' % (delta.seconds/3600, delta.seconds%3600) def check_time_coherence(self): """ Check if the end's day is before the begin's one, in case it stops at the next day (eg. 15h->1h). If it occures, add a day. """ if self.begin > self.end: self.end = self.end + timedelta(1) def load(self): self.content = self.backend.get_content(self.name) self.members.clear() member = None for line in self.content.content.split('\n'): line = line.strip() if line.startswith('h1. '): self.title = line[4:] elif line.startswith('h3=. '): m = re.match('h3=. Event finished. Winner is "(.*)":/users/(\d+)\!', line) if not m: print('Unable to parse h3=: %s' % line, file=self.stderr) continue self.winner = Member(int(m.group(2)), m.group(1)) elif line.startswith('h2. '): continue elif line.startswith('h3. '): m = re.match('h3. "(.*)":/users/(\d+)', line) if not m: print('Unable to parse user "%s"' % line, file=self.stderr) continue member = Member(int(m.group(2)), m.group(1)) if member.id == self.my_id: member.is_me = True if self.winner is not None and member.id == self.winner.id: self.winner = member self.members[member.id] = member elif self.description is None and len(line) > 0 and line != '{{TOC}}': self.description = line elif line.startswith('* '): m = re.match('\* \*(\w+)\*: (.*)', line) if not m: continue key, value = m.group(1), m.group(2) if member is None: if key == 'Date': self.date = self.parse_date(value) elif key == 'Start' or key == 'Begin': self.begin = self.parse_time(value) elif key == 'End': self.end = self.parse_time(value) self.check_time_coherence() elif key == 'Location': self.location = value else: if key == 'Repository': m = re.match('"(.*.git)":.*', value) if m: member.repository = m.group(1) else: member.repository = value elif key == 'Hardware': member.hardware = value elif key == 'Availabilities': member.availabilities = value elif line.startswith('[['): m = re.match('\[\[(\w+)\]\]\|\[\[(\w+)\]\]\|(.*)\|', line) if not m: print('Unable to parse task: "%s"' % line, file=self.stderr) continue task = Task(m.group(1), m.group(2)) member.tasks.append(task) if m.group(3) == '!/img/weboob/_progress.png!': task.status = task.STATUS_PROGRESS continue mm = re.match('!/img/weboob/_done.png! (\d+):(\d+) (\w+)', m.group(3)) if mm and self.date: task.status = task.STATUS_DONE task.date = datetime(self.date.year, self.date.month, self.date.day, int(mm.group(1)), int(mm.group(2))) task.branch = mm.group(3) def parse_date(self, value): try: return datetime.strptime(value, '%Y-%m-%d') except ValueError: return None def parse_time(self, value): m = re.match('(\d+):(\d+)', value) if not m: return try: return self.date.replace(hour=int(m.group(1)), minute=int(m.group(2))) except ValueError: return None def save(self, message): if self.winner: finished = u'\nh3=. Event finished. Winner is "%s":/users/%d!\n' % (self.winner.name, self.winner.id) else: finished = u'' s = u"""h1. %s {{TOC}} %s h2. Event %s * *Date*: %s * *Begin*: %s * *End*: %s * *Duration*: %s * *Location*: %s h2. Attendees """ % (self.title, finished, self.description, self.date.strftime('%Y-%m-%d') if self.date else '_Unknown_', self.begin.strftime('%H:%M') if self.begin else '_Unknown_', self.end.strftime('%H:%M') if self.end else '_Unknown_', self.format_duration() or '_Unknown_', self.location or '_Unknown_') for member in self.members.itervalues(): if self.date: availabilities = '' else: availabilities = '* *Availabilities*: %s' % member.availabilities if member.repository is None: repository = '_Unknown_' elif member.repository.endswith('.git'): repository = '"%s":git://git.symlink.me/pub/%s ("http":http://git.symlink.me/?p=%s;a=summary)' repository = repository.replace('%s', member.repository) else: repository = member.repository s += u"""h3. "%s":/users/%d * *Repository*: %s * *Hardware*: %s %s |_.Backend|_.Capabilities|_.Status|""" % (member.name, member.id, repository, member.hardware, availabilities) for task in member.tasks: if task.status == task.STATUS_DONE: status = '!/img/weboob/_done.png! %02d:%02d %s' % (task.date.hour, task.date.minute, task.branch) elif task.status == task.STATUS_PROGRESS: status = '!/img/weboob/_progress.png!' else: status = ' ' s += u""" |=.!/img/weboob/%s.png!:/projects/weboob/wiki/%s [[%s]]|[[%s]]|%s|""" % (task.backend.lower(), task.backend, task.backend, task.capability, status) s += '\n\n' self.content.content = s self.backend.push_content(self.content, message) class Boobathon(ReplApplication): APPNAME = 'boobathon' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2011-YEAR Romain Bignon' DESCRIPTION = 'Console application to participate to a Boobathon.' SHORT_DESCRIPTION = "participate in a Boobathon" CAPS = CapContent SYNOPSIS = 'Usage: %prog [-dqv] [-b backends] [-cnfs] boobathon\n' SYNOPSIS += ' %prog [--help] [--version]' radios = [] def __init__(self, *args, **kwargs): ReplApplication.__init__(self, *args, **kwargs) def main(self, argv): if len(argv) < 2: print('Please give the name of the boobathon', file=self.stderr) return 1 self.event = Event(argv[1], choice(self.weboob.backend_instances.values())) if self.event.description is None: if not self.ask("This event doesn't seem to exist. Do you want to create it?", default=True): return 1 self.edit_event() self.save_event('Event created') return ReplApplication.main(self, [argv[0]]) def save_event(self, message): if self.ask("Do you confirm your changes?", default=True): self.event.save(message) return True return False def edit_event(self): self.event.title = self.ask('Enter a title', default=self.event.title) self.event.description = self.ask('Enter a description', default=self.event.description) self.event.date = self.ask('Enter a date (yyyy-mm-dd)', default=self.event.date.strftime('%Y-%m-%d') if self.event.date else '', regexp='^(\d{4}-\d{2}-\d{2})?$') if self.event.date: self.event.date = datetime.strptime(self.event.date, '%Y-%m-%d') s = self.ask('Begin at (HH:MM)', default=self.event.begin.strftime('%H:%M') if self.event.begin else '', regexp='^(\d{2}:\d{2})?$') if s: h, m = s.split(':') self.event.begin = self.event.date.replace(hour=int(h), minute=int(m)) s = self.ask('End at (HH:MM)', default=self.event.end.strftime('%H:%M') if self.event.end else '', regexp='^(\d{2}:\d{2})?$') if s: h, m = s.split(':') self.event.end = self.event.date.replace(hour=int(h), minute=int(m)) self.event.check_time_coherence() self.event.location = self.ask('Enter a location', default=self.event.location) def edit_member(self, member): if member.name is None: firstname = self.ask('Enter your firstname') lastname = self.ask('Enter your lastname') member.name = '%s %s' % (firstname, lastname) else: member.name = self.ask('Enter your name', default=member.name) if self.event.date is None: member.availabilities = self.ask('Enter availabilities', default=member.availabilities) member.repository = self.ask('Enter your repository (ex. romain/weboob.git)', default=member.repository) member.hardware = self.ask('Enter your hardware', default=member.hardware) def do_progress(self, line): """ progress Display progress of members. """ self.event.load() for member in self.event.members.itervalues(): if member.is_me and member is self.event.winner: status = '\o/ ->' elif member.is_me: status = ' ->' elif member is self.event.winner: status = ' \o/' else: status = ' ' s = u' %s%20s %s|' % (status, member.shortname(), self.BOLD) for task in member.tasks: if task.status == task.STATUS_DONE: s += '##' elif task.status == task.STATUS_PROGRESS: s += u'=>' else: s += ' ' s += '|%s' % self.NC print(s) print('') now = datetime.now() if self.event.begin > now: d = self.event.begin - now msg = 'The event will start in %d days, %02d:%02d:%02d' elif self.event.end < now: d = now - self.event.end msg = 'The event is finished since %d days, %02d:%02d:%02d' else: tot = (self.event.end - self.event.begin).seconds cur = (datetime.now() - self.event.begin).seconds pct = cur*20/tot progress = '' for i in xrange(20): if i < pct: progress += '=' elif i == pct: progress += '>' else: progress += ' ' print('Event started: %s |%s| %s' % (self.event.begin.strftime('%H:%M'), progress, self.event.end.strftime('%H:%M'))) d = self.event.end - now msg = 'The event will be finished in %d days, %02d:%02d:%02d' print(msg % (d.days, d.seconds/3600, d.seconds%3600/60, d.seconds%60)) def do_tasks(self, line): """ tasks Display all tasks of members. """ self.event.load() stop = False i = -2 while not stop: if i >= 0 and not i%2: self.stdout.write(' #%-2d' % (i/2)) else: self.stdout.write(' ') if i >= 0 and i%2: # second line of task, see if we'll stop stop = True for mem in self.event.members.itervalues(): if len(mem.tasks) > (i/2+1): # there are more tasks, don't stop now stop = False if i == -2: self.stdout.write(' %s%-20s%s' % (self.BOLD, mem.shortname().encode('utf-8'), self.NC)) elif i == -1: self.stdout.write(' %s%-20s%s' % (self.BOLD, '-' * len(mem.shortname()), self.NC)) elif len(mem.tasks) <= (i/2): self.stdout.write(' ' * (20+1)) else: task = mem.tasks[i/2] if task.status == task.STATUS_DONE: status = u'#' elif task.status == task.STATUS_PROGRESS: if not i%2: status = u'|' #1st line else: status = u'v' #2nd line else: status = u' ' if not i%2: #1st line line = u'%s %s' % (status, task.backend) else: #2nd line line = u'%s `-%s' % (status, task.capability[3:]) self.stdout.write((u' %-20s' % line).encode('utf-8')) self.stdout.write('\n') i += 1 def complete_close(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: self.event.load() return [member.name for member in self.event.members.itervalues()] def do_close(self, name): """ close WINNER Close the event and set the winner. """ self.event.load() for member in self.event.members.itervalues(): if member.name == name: self.event.winner = member if self.save_event('Close event'): print('Event is now closed. Winner is %s!' % self.event.winner.name) return print('"%s" not found' % name, file=self.stderr) return 3 def complete_edit(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return ['event', 'me'] def do_edit(self, line): """ edit [event | me] Edit information about you or about event. """ if not line: print('Syntax: edit [event | me]', file=self.stderr) return 2 self.event.load() if line == 'event': self.edit_event() self.save_event('Event edited') elif line == 'me': mem = self.event.get_me() if not mem: print('You haven\'t joined the event.', file=self.stderr) return 1 self.edit_member(mem) self.save_event('Member edited') else: print('Unable to edit "%s"' % line, file=self.stderr) return 1 def do_info(self, line): """ info Display information about this event. """ self.event.load() print(self.event.title) print('-' * len(self.event.title)) print(self.event.description) print('') print('Date:', self.event.date.strftime('%Y-%m-%d') if self.event.date else 'Unknown') print('Begin:', self.event.begin.strftime('%H:%M') if self.event.begin else 'Unknown') print('End:', self.event.end.strftime('%H:%M') if self.event.end else 'Unknown') print('Duration:', self.event.format_duration() or 'Unknown') print('Location:', self.event.location or 'Unknown') print('') print('There are %d members, use the "members" command to list them' % len(self.event.members)) if self.event.get_me() is None: print('To join this event, use the command "join".') def do_members(self, line): """ members Display members information. """ self.event.load() for member in self.event.members.itervalues(): print(member.name) print('-' * len(member.name)) print('Repository:', member.repository) if self.event.date is None: print('Availabilities:', member.availabilities) print('Hardware:', member.hardware) accompl = 0 for task in member.tasks: if task.status == task.STATUS_DONE: accompl += 1 print('%d tasks (%d accomplished)' % (len(member.tasks), accompl)) if member is self.event.winner: print('=== %s is the winner!' % member.name) print('') print('Use the "tasks" command to display all tasks') def do_join(self, line): """ join Join this event. """ self.event.load() if self.event.backend.browser.get_userid() in self.event.members: print('You have already joined this event.', file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 m = Member(self.event.backend.browser.get_userid(), None) self.edit_member(m) self.event.members[m.id] = m self.save_event('Joined the event') def do_leave(self, line): """ leave Leave this event. """ self.event.load() if self.event.currently_in_event(): print('Unable to leave during the event, loser!', file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 try: self.event.members.pop(self.event.backend.browser.get_userid()) except KeyError: print("You have not joined this event.", file=self.stderr) return 1 else: self.save_event('Left the event') def do_remtask(self, line): """ remtask TASK_ID Remove a task. """ self.event.load() mem = self.event.get_me() if not mem: print("You have not joined this event.", file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 try: task_id = int(line) except ValueError: print('The task ID should be a number', file=self.stderr) return 2 try: task = mem.tasks.pop(task_id) except IndexError: print('Unable to find task #%d' % task_id, file=self.stderr) return 1 else: print('Removing task #%d (%s,%s).' % (task_id, task.backend, task.capability)) self.save_event('Remove task') def do_addtask(self, line): """ addtask BACKEND CAPABILITY Add a new task. """ self.event.load() mem = self.event.get_me() if not mem: print("You have not joined this event.", file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 backend, capability = self.parse_command_args(line, 2, 2) if not backend[0].isupper(): print('The backend name "%s" needs to start with a capital.' % backend, file=self.stderr) return 2 if not capability.startswith('Cap') or not capability[3].isupper(): print('"%s" is not a proper capability name (must start with Cap).' % capability, file=self.stderr) return 2 for task in mem.tasks: if (task.backend,task.capability) == (backend,capability): print("A task already exists for that.", file=self.stderr) return 1 task = Task(backend, capability) mem.tasks.append(task) self.save_event('New task') def do_start(self, line): """ start [TASK_ID] Start a task. If you don't give a task ID, the first available task will be taken. """ self.event.load() mem = self.event.get_me() if not mem: print("You have not joined this event.", file=self.stderr) return 1 if len(mem.tasks) == 0: print("You don't have any task to do.", file=self.stderr) return 1 if not self.event.currently_in_event(): print("You can't start a task, we are not in event.", file=self.stderr) return 1 if line.isdigit(): task_id = int(line) else: task_id = -1 last_done = -1 for i, task in enumerate(mem.tasks): if task.status == task.STATUS_DONE: last_done = i elif task.status == task.STATUS_PROGRESS: task.status = task.STATUS_NONE print('Task #%s (%s,%s) canceled.' % (i, task.backend, task.capability)) if (i == task_id or task_id < 0) and task.status == task.STATUS_NONE: break else: print('Task not found.', file=self.stderr) return 3 if task.status == task.STATUS_DONE: print('Task is already done.', file=self.stderr) return 1 task.status = task.STATUS_PROGRESS mem.tasks.remove(task) mem.tasks.insert(last_done + 1, task) self.save_event('Started a task') def do_done(self, line): """ done Set the current task as done. """ self.event.load() mem = self.event.get_me() if not mem: print("You have not joined this event.", file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 for i, task in enumerate(mem.tasks): if task.status == task.STATUS_PROGRESS: print('Task (%s,%s) done! (%d%%)' % (task.backend, task.capability, (i+1)*100/len(mem.tasks))) if self.event.currently_in_event(): task.status = task.STATUS_DONE task.date = datetime.now() task.branch = self.ask('Enter name of branch') self.save_event('Task accomplished') else: task.status = task.STATUS_NONE print('Oops, you are out of event. Canceling the task...', file=self.stderr) self.save_event('Cancel task') return 1 return print("There isn't any task in progress.", file=self.stderr) return 1 def do_cancel(self, line): """ cancel Cancel the current task. """ self.event.load() mem = self.event.get_me() if not mem: print("You have not joined this event.", file=self.stderr) return 1 if self.event.is_closed(): print("Boobathon is closed.", file=self.stderr) return 1 for task in mem.tasks: if task.status == task.STATUS_PROGRESS: print('Task (%s,%s) canceled.' % (task.backend, task.capability)) task.status = task.STATUS_NONE self.save_event('Cancel task') return print("There isn't any task in progress.", file=self.stderr) return 1 def load_default_backends(self): """ Overload a Application method. """ for backend_name, module_name, params in self.weboob.backends_config.iter_backends(): if module_name != 'redmine': continue v = urlsplit(params['url']) if v.netloc == 'symlink.me': self.load_backends(names=[backend_name]) return if not self.check_loaded_backends({'url': 'https://symlink.me'}): print("Ok, so leave now, fag.") sys.exit(0) def is_module_loadable(self, module): """ Overload a ConsoleApplication method. """ return module.name == 'redmine' ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobcoming/����������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020702�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobcoming/__init__.py�����������������������������������������������0000664�0000000�0000000�00000001423�13034501105�0023013�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .boobcoming import Boobcoming __all__ = ['Boobcoming'] ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobcoming/boobcoming.py���������������������������������������������0000664�0000000�0000000�00000037573�13034501105�0023411�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function from datetime import time, datetime from dateutil import tz from weboob.tools.date import parse_date from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.capabilities.base import empty from weboob.capabilities.calendar import CapCalendarEvent, Query, CATEGORIES, BaseCalendarEvent, TICKET from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.config.yamlconfig import YamlConfig __all__ = ['Boobcoming'] class UpcomingSimpleFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'start_date', 'category', 'summary') def format_obj(self, obj, alias): result = u'%s - %s' % (obj.backend, obj.category) if not empty(obj.start_date): result += u' - %s' % obj.start_date.strftime('%H:%M') result += u' - %s' % obj.summary return result class ICalFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'start_date', 'end_date', 'summary', 'status') def start_format(self, **kwargs): result = u'BEGIN:VCALENDAR\r\n' result += u'VERSION:2.0\r\n' result += u'PRODID:-//hacksw/handcal//NONSGML v1.0//EN\r\n' self.output(result) def format_obj(self, obj, alias): result = u'BEGIN:VEVENT\r\n' utc_zone = tz.gettz('UTC') event_timezone = tz.gettz(obj.timezone) start_date = obj.start_date if not empty(obj.start_date) else datetime.now() start_date = start_date.replace(tzinfo=event_timezone) utc_start_date = start_date.astimezone(utc_zone) result += u'DTSTART:%s\r\n' % utc_start_date.strftime("%Y%m%dT%H%M%SZ") end_date = obj.end_date if not empty(obj.end_date) else datetime.combine(start_date, time.max) end_date = end_date.replace(tzinfo=event_timezone) utc_end_date = end_date.astimezone(utc_zone) result += u'DTEND:%s\r\n' % utc_end_date.strftime("%Y%m%dT%H%M%SZ") result += u'SUMMARY:%s\r\n' % obj.summary result += u'UID:%s\r\n' % obj.id result += u'STATUS:%s\r\n' % obj.status location = '' if hasattr(obj, 'location') and not empty(obj.location): location += obj.location + ' ' if hasattr(obj, 'city') and not empty(obj.city): location += obj.city + ' ' if not empty(location): result += u'LOCATION:%s\r\n' % location if hasattr(obj, 'categories') and not empty(obj.categories): result += u'CATEGORIES:%s\r\n' % obj.categories if hasattr(obj, 'description') and not empty(obj.description): result += u'DESCRIPTION:%s\r\n' % obj.description.strip(' \t\n\r')\ .replace('\r', '')\ .replace('\n', r'\n')\ .replace(',', '\,') if hasattr(obj, 'transp') and not empty(obj.transp): result += u'TRANSP:%s\r\n' % obj.transp if hasattr(obj, 'sequence') and not empty(obj.sequence): result += u'SEQUENCE:%s\r\n' % obj.sequence if hasattr(obj, 'url') and not empty(obj.url): result += u'URL:%s\r\n' % obj.url result += u'END:VEVENT\r\n' return result def flush(self, **kwargs): self.output(u'END:VCALENDAR') class UpcomingListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'start_date', 'end_date', 'summary', 'category') def get_title(self, obj): return ' %s - %s ' % (obj.category, obj.summary) def get_description(self, obj): result = u'' if not empty(obj.start_date): result += u'\tDate: %s\n' % obj.start_date.strftime('%A %d %B %Y') result += u'\tHour: %s' % obj.start_date.strftime('%H:%M') if not empty(obj.end_date): result += ' - %s' % obj.end_date.strftime('%H:%M') days_diff = (obj.end_date - obj.start_date).days if days_diff >= 1: result += ' (%i day(s) later)' % (days_diff) result += '\n' return result.strip('\n\t') class UpcomingFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'start_date', 'end_date', 'summary', 'category') def format_obj(self, obj, alias): result = u'%s%s - %s%s\n' % (self.BOLD, obj.category, obj.summary, self.NC) if not empty(obj.start_date): if not empty(obj.end_date): days_diff = (obj.end_date - obj.start_date).days if days_diff >= 1: result += u'From: %s to %s ' % (obj.start_date.strftime('%A %d %B %Y'), obj.end_date.strftime('%A %d %B %Y')) else: result += u'Date: %s\n' % obj.start_date.strftime('%A %d %B %Y') result += u'Hour: %s' % obj.start_date.strftime('%H:%M') result += ' - %s' % obj.end_date.strftime('%H:%M') else: result += u'Date: %s\n' % obj.start_date.strftime('%A %d %B %Y') result += u'Hour: %s' % obj.start_date.strftime('%H:%M') result += '\n' if hasattr(obj, 'location') and not empty(obj.location): result += u'Location: %s\n' % obj.location if hasattr(obj, 'city') and not empty(obj.city): result += u'City: %s\n' % obj.city if hasattr(obj, 'event_planner') and not empty(obj.event_planner): result += u'Event planner: %s\n' % obj.event_planner if hasattr(obj, 'booked_entries') and not empty(obj.booked_entries) and \ hasattr(obj, 'max_entries') and not empty(obj.max_entries): result += u'Entry: %s/%s \n' % (obj.booked_entries, obj.max_entries) elif hasattr(obj, 'booked_entries') and not empty(obj.booked_entries): result += u'Entry: %s \n' % (obj.booked_entries) elif hasattr(obj, 'max_entries') and not empty(obj.max_entries): result += u'Max entries: %s \n' % (obj.max_entries) if hasattr(obj, 'description') and not empty(obj.description): result += u'Description:\n %s\n\n' % obj.description if hasattr(obj, 'price') and not empty(obj.price): result += u'Price: %.2f\n' % obj.price if hasattr(obj, 'ticket') and not empty(obj.ticket): result += u'Ticket: %s\n' % obj.ticket if hasattr(obj, 'url') and not empty(obj.url): result += u'url: %s\n' % obj.url return result class Boobcoming(ReplApplication): APPNAME = 'boobcoming' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Bezleputh' DESCRIPTION = "Console application to see upcoming events." SHORT_DESCRIPTION = "see upcoming events" CAPS = CapCalendarEvent EXTRA_FORMATTERS = {'upcoming_list': UpcomingListFormatter, 'upcoming': UpcomingFormatter, 'simple_upcoming': UpcomingSimpleFormatter, 'ical_formatter': ICalFormatter, } COMMANDS_FORMATTERS = {'list': 'upcoming_list', 'search': 'upcoming_list', 'load': 'upcoming_list', 'ls': 'upcoming_list', 'info': 'upcoming', 'export': 'ical_formatter' } def main(self, argv): self.load_config(klass=YamlConfig) return ReplApplication.main(self, argv) def comp_object(self, obj1, obj2): if isinstance(obj1, BaseCalendarEvent) and isinstance(obj2, BaseCalendarEvent): if obj1.start_date == obj2.start_date: return 0 if obj1.start_date > obj2.start_date: return 1 return -1 else: return super(Boobcoming, self).comp_object(obj1, obj2) def select_values(self, values_from, values_to, query_str): r = 'notempty' while r != '': for value in values_from.values: print(' %s%2d)%s [%s] %s' % (self.BOLD, values_from.index[value] + 1, self.NC, 'x' if value in values_to else ' ', value)) r = self.ask(query_str, regexp='(\d+|)', default='') if not r.isdigit(): continue r = int(r) if r <= 0 or r > len(values_from.values): continue value = values_from.values[r - 1] if value in values_to: values_to.remove(value) else: values_to.append(value) @defaultcount(10) def do_search(self, line): """ search search for an event. Parameters interactively asked """ query = Query() self.select_values(CATEGORIES, query.categories, ' Select categorie (or empty to stop)') self.select_values(TICKET, query.ticket, ' Select tickets status (or empty to stop)') if query.categories and len(query.categories) > 0 and query.ticket and len(query.ticket) > 0: query.city = self.ask('Enter a city', default='') query.summary = self.ask('Enter a title', default='') start_date = self.ask_date('Enter a start date', default='today') end_date = self.ask_date('Enter a end date', default='') if end_date: if end_date == start_date: end_date = datetime.combine(start_date, time.max) else: end_date = datetime.combine(end_date, time.max) query.start_date = datetime.combine(start_date, time.min) query.end_date = end_date save_query = self.ask('Save query (y/n)', default='n') if save_query.upper() == 'Y': name = '' while not name: name = self.ask('Query name') self.config.set('queries', name, query) self.config.save() self.complete_search(query) def complete_search(self, query): self.change_path([u'events']) self.start_format() for event in self.do('search_events', query): if event: self.cached_format(event) def ask_date(self, txt, default=''): r = self.ask(txt, default=default) return parse_date(r) @defaultcount(10) def do_load(self, query_name): """ load [query name] without query name : list loadable queries with query name laod query """ queries = self.config.get('queries') if not queries: print('There is no saved queries', file=self.stderr) return 2 if not query_name: for name in queries.keys(): print(' %s* %s %s' % (self.BOLD, self.NC, name)) query_name = self.ask('Which one') if query_name in queries: self.complete_search(queries.get(query_name)) else: print('Unknown query', file=self.stderr) return 2 @defaultcount(10) def do_list(self, line): """ list [PATTERN] List upcoming events, pattern can be an english or french week day, 'today' or a date (dd/mm/yy[yy]) """ self.change_path([u'events']) if line: _date = parse_date(line) if not _date: print('Invalid argument: %s' % self.get_command_help('list'), file=self.stderr) return 2 date_from = datetime.combine(_date, time.min) date_to = datetime.combine(_date, time.max) else: date_from = datetime.now() date_to = None for event in self.do('list_events', date_from, date_to): self.cached_format(event) def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, _id): """ info ID Get information about an event. """ if not _id: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 event = self.get_object(_id, 'get_event') if not event: print('Upcoming event not found: %s' % _id, file=self.stderr) return 3 self.start_format() self.format(event) def do_export(self, line): """ export FILENAME [ID1 ID2 ID3 ...] ID is the identifier of the event. If no ID every events are exported FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. Export event in ICALENDAR format """ if not line: print('This command takes at leat one argument: %s' % self.get_command_help('export'), file=self.stderr) return 2 _file, args = self.parse_command_args(line, 2, req_n=1) l = self.retrieve_events(args) if l == 3: return 3 if not _file == "-": dest = self.check_file_ext(_file) self.formatter.outfile = dest self.formatter.start_format() for item in l: self.format(item) def retrieve_events(self, args): l = [] if not args: _ids = [] for event in self.do('list_events', datetime.now(), None): _ids.append(event.id) else: _ids = args.strip().split(' ') for _id in _ids: event = self.get_object(_id, 'get_event') if not event: print('Upcoming event not found: %s' % _id, file=self.stderr) return 3 l.append(event) return l def check_file_ext(self, _file): splitted_file = _file.split('.') if splitted_file[-1] != 'ics': return "%s.ics" % _file else: return _file def do_attends(self, line): """ attends ID1 [ID2 ID3 ...] Register as participant of an event. ID is the identifier of the event. """ if not line: print('This command takes at leat one argument: %s' % self.get_command_help('attends'), file=self.stderr) return 2 args = self.parse_command_args(line, 1, req_n=1) l = self.retrieve_events(args[0]) for event in l: # we wait till the work be done, else the errors are not handled self.do('attends_event', event, True).wait() def do_unattends(self, line): """ unattends ID1 [ID2 ID3 ...] Unregister you participation for an event. ID is the identifier of the event. """ if not line: print('This command takes at leat one argument: %s' % self.get_command_help('unattends'), file=self.stderr) return 2 args = self.parse_command_args(line, 1, req_n=1) l = self.retrieve_events(args[0]) for event in l: self.do('attends_event', event, False) �������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobill/�������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020206�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobill/__init__.py��������������������������������������������������0000664�0000000�0000000�00000001424�13034501105�0022320�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .boobill import Boobill __all__ = ['Boobill'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobill/boobill.py���������������������������������������������������0000664�0000000�0000000�00000020374�13034501105�0022210�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2012-2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function from decimal import Decimal from weboob.capabilities.bill import CapDocument, Detail, Subscription from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import PrettyFormatter from weboob.tools.application.base import MoreResultsAvailable from weboob.core import CallErrors __all__ = ['Boobill'] class SubscriptionsFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'label') def get_title(self, obj): if obj.renewdate: return u"%s - %s" % (obj.label, obj.renewdate.strftime('%d/%m/%y')) return obj.label class Boobill(ReplApplication): APPNAME = 'boobill' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Florent Fourcot' DESCRIPTION = 'Console application allowing to get/download documents and bills.' SHORT_DESCRIPTION = "get/download documents and bills" CAPS = CapDocument COLLECTION_OBJECTS = (Subscription, ) EXTRA_FORMATTERS = {'subscriptions': SubscriptionsFormatter, } DEFAULT_FORMATTER = 'table' COMMANDS_FORMATTERS = {'subscriptions': 'subscriptions', 'ls': 'subscriptions', } def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def exec_method(self, id, method): l = [] id, backend_name = self.parse_id(id) if not id: for subscrib in self.get_object_list('iter_subscription'): l.append((subscrib.id, subscrib.backend)) else: l.append((id, backend_name)) more_results = [] not_implemented = [] self.start_format() for id, backend in l: names = (backend,) if backend is not None else None try: for result in self.do(method, id, backends=names): self.format(result) except CallErrors as errors: for backend, error, backtrace in errors: if isinstance(error, MoreResultsAvailable): more_results.append(id + u'@' + backend.name) elif isinstance(error, NotImplementedError): if backend not in not_implemented: not_implemented.append(backend) else: self.bcall_error_handler(backend, error, backtrace) if len(more_results) > 0: print('Hint: There are more results available for %s (use option -n or count command)' % (', '.join(more_results)), file=self.stderr) for backend in not_implemented: print(u'Error(%s): This feature is not supported yet by this backend.' % backend.name, file=self.stderr) def do_subscriptions(self, line): """ subscriptions List all subscriptions. """ return self.do_ls(line) def do_details(self, id): """ details [ID] Get details of subscriptions. If no ID given, display all details of all backends. """ l = [] id, backend_name = self.parse_id(id) if not id: for subscrib in self.get_object_list('iter_subscription'): l.append((subscrib.id, subscrib.backend)) else: l.append((id, backend_name)) for id, backend in l: names = (backend,) if backend is not None else None # XXX: should be generated by backend? -Flo # XXX: no, but you should do it in a specific formatter -romain # TODO: do it, and use exec_method here. Code is obsolete mysum = Detail() mysum.label = u"Sum" mysum.infos = u"Generated by boobill" mysum.price = Decimal("0.") self.start_format() for detail in self.do('get_details', id, backends=names): self.format(detail) mysum.price = detail.price + mysum.price self.format(mysum) def do_balance(self, id): """ balance [ID] Get balance of subscriptions. If no ID given, display balance of all backends. """ self.exec_method(id, 'get_balance') @defaultcount(10) def do_history(self, id): """ history [ID] Get the history of subscriptions. If no ID given, display histories of all backends. """ self.exec_method(id, 'iter_bills_history') @defaultcount(10) def do_documents(self, id): """ documents [ID] Get the list of documents for subscriptions. If no ID given, display documents of all backends """ self.exec_method(id, 'iter_documents') @defaultcount(10) def do_bills(self, id): """ bills [ID] Get the list of bills documents for subscriptions. If no ID given, display bills of all backends """ self.exec_method(id, 'iter_bills') def do_download(self, line): """ download [ID | all] [FILENAME] download ID [FILENAME] download the document id is the identifier of the document (hint: try documents command) FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. download all [ID] You can use special word "all" and download all documents of subscription identified by ID. If Id not given, download documents of all subscriptions. """ id, dest = self.parse_command_args(line, 2, 1) id, backend_name = self.parse_id(id) if not id: print('Error: please give a document ID (hint: use documents command)', file=self.stderr) return 2 names = (backend_name,) if backend_name is not None else None # Special keywords, download all documents of all subscriptions if id == "all": if dest is None: for subscription in self.do('iter_subscription', backends=names): self.download_all(subscription.id, names) return else: self.download_all(dest, names) return if dest is None: for document in self.do('get_document', id, backends=names): dest = id + "." + document.format for buf in self.do('download_document', id, backends=names): if buf: if dest == "-": print(buf) else: try: with open(dest, 'wb') as f: f.write(buf) except IOError as e: print('Unable to write document in "%s": %s' % (dest, e), file=self.stderr) return 1 return def download_all(self, id, names): id, backend_name = self.parse_id(id) for document in self.do('iter_documents', id, backends=names): dest = document.id + "." + document.format for buf in self.do('download_document', document.id, backends=names): if buf: if dest == "-": print(buf) else: try: with open(dest, 'wb') as f: f.write(buf) except IOError as e: print('Unable to write bill in "%s": %s' % (dest, e), file=self.stderr) return 1 return ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/booblyrics/����������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020733�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/booblyrics/__init__.py�����������������������������������������������0000664�0000000�0000000�00000001432�13034501105�0023044�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .booblyrics import Booblyrics __all__ = ['Booblyrics'] ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/booblyrics/booblyrics.py���������������������������������������������0000664�0000000�0000000�00000007037�13034501105�0023463�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function from weboob.capabilities.lyrics import CapLyrics from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['Booblyrics', 'LyricsGetFormatter', 'LyricsListFormatter'] class LyricsGetFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'title', 'artist', 'content') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.title, self.NC) result += 'ID: %s\n' % obj.fullid result += 'Title: %s\n' % obj.title result += 'Artist: %s\n' % obj.artist result += '\n%sContent%s\n' % (self.BOLD, self.NC) result += '%s' % obj.content return result class LyricsListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'artist') def get_title(self, obj): return obj.title def get_description(self, obj): artist = u'' if not empty(obj.artist): artist = obj.artist return '%s' % artist class Booblyrics(ReplApplication): APPNAME = 'booblyrics' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Julien Veyssier' DESCRIPTION = "Console application allowing to search for song lyrics on various websites." SHORT_DESCRIPTION = "search and display song lyrics" CAPS = CapLyrics EXTRA_FORMATTERS = {'lyrics_list': LyricsListFormatter, 'lyrics_get': LyricsGetFormatter, } COMMANDS_FORMATTERS = {'search': 'lyrics_list', 'get': 'lyrics_get', } SEARCH_CRITERIAS = ['artist', 'song'] def complete_get(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_get(self, id): """ get ID Display lyrics of the song. """ songlyrics = self.get_object(id, 'get_lyrics') if not songlyrics: print('Song lyrics not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(songlyrics) def complete_search(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self.SEARCH_CRITERIAS @defaultcount(10) def do_search(self, line): """ search [artist | song] [PATTERN] Search lyrics by artist name or by song title. """ criteria, pattern = self.parse_command_args(line, 2, 2) self.change_path([u'search']) if not pattern: pattern = None self.start_format(pattern=pattern) for songlyrics in self.do('iter_lyrics', criteria, pattern): self.cached_format(songlyrics) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobmsg/�������������������������������������������������������������0000775�0000000�0000000�00000000000�13034501105�0020214�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobmsg/__init__.py��������������������������������������������������0000664�0000000�0000000�00000001426�13034501105�0022330�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from .boobmsg import Boobmsg __all__ = ['Boobmsg'] ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������weboob-1.2/weboob/applications/boobmsg/boobmsg.py���������������������������������������������������0000664�0000000�0000000�00000043130�13034501105�0022217�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see <http://www.gnu.org/licenses/>. from __future__ import print_function import os import datetime import hashlib from tempfile import NamedTemporaryFile from lxml import etree from weboob.core import CallErrors from weboob.capabilities.base import empty from weboob.capabilities.messages import CapMessages, Message, Thread from weboob.capabilities.account import CapAccount from weboob.capabilities.contact import CapContact from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter from weboob.tools.html import html2text __all__ = ['Boobmsg'] class AtomFormatter(IFormatter): MANDATORY_FIELDS = ('title', 'date', 'sender', 'content') def start_format(self, **kwargs): self.output(u'<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"') self.output(u'xmlns:dc="http://purl.org/dc/elements/1.1/">\n') self.output(u'<title type="text">Atom feed by Weboob') # TODO : get backend name self.output(u'%s' % datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")) m = hashlib.md5() m.update(datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")) self.output(u'urn:md5:%s' % m.hexdigest()) def format_obj(self, obj, alias): elem = etree.Element('entry') title = etree.Element('title') title.text = obj.title elem.append(title) id = etree.Element('id') m = hashlib.md5() m.update(obj.content.encode('utf8', 'ascii')) id.text = "urn:md5:%s" % m.hexdigest() elem.append(id) link = etree.Element('link') link.attrib["href"] = obj.thread.id link.attrib["title"] = obj.title link.attrib["type"] = "text/html" elem.append(link) author = etree.Element('author') name = etree.Element('name') if obj.sender: name.text = obj.sender else: name.text = obj.backend author.append(name) elem.append(author) date = etree.Element('updated') date.text = obj.date.strftime("%Y-%m-%dT%H:%M:%SZ") elem.append(date) content = etree.Element('content') content.text = obj.content content.attrib["type"] = "html" elem.append(content) return etree.tostring(elem, pretty_print=True) def flush(self): self.output(u'') class XHtmlFormatter(IFormatter): MANDATORY_FIELDS = ('title', 'date', 'sender', 'signature', 'content') def format_obj(self, obj, alias): result = "
\n" result += "

%s

" % (obj.title) result += "
" result += "
Date
%s
" % (obj.date.strftime('%Y-%m-%d %H:%M')) result += "
Sender
%s
" % (obj.sender) result += "
Signature
%s
" % (obj.signature) result += "
" result += "
%s
" % (obj.content) result += "
\n" return result class MessageFormatter(IFormatter): MANDATORY_FIELDS = ('title', 'date', 'sender', 'signature', 'content') def format_obj(self, obj, alias): result = u'%sTitle:%s %s\n' % (self.BOLD, self.NC, obj.title) result += u'%sDate:%s %s\n' % (self.BOLD, self.NC, obj.date.strftime('%Y-%m-%d %H:%M')) result += u'%sFrom:%s %s\n' % (self.BOLD, self.NC, obj.sender) if hasattr(obj, 'receivers') and obj.receivers: result += u'%sTo:%s %s\n' % (self.BOLD, self.NC, ', '.join(obj.receivers)) if obj.flags & Message.IS_HTML: content = html2text(obj.content) else: content = obj.content result += '\n%s' % content if obj.signature: if obj.flags & Message.IS_HTML: signature = html2text(obj.signature) else: signature = obj.signature result += '\n-- \n%s' % signature return result class MessagesListFormatter(IFormatter): MANDATORY_FIELDS = () count = 0 _list_messages = False def flush(self): self.count = 0 def format_obj(self, obj, alias): if not self._list_messages: return self.format_dict_thread(obj, alias) else: return self.format_dict_messages(obj, alias) def format_dict_thread(self, obj, alias): self.count += 1 if self.interactive: result = u'%s* (%d) %s (%s)%s' % (self.BOLD, self.count, obj.title, obj.backend, self.NC) else: result = u'%s* (%s) %s%s' % (self.BOLD, obj.id, obj.title, self.NC) if obj.date: result += u'\n %s' % obj.date return result def format_dict_messages(self, obj, alias): if obj.flags == Thread.IS_THREADS: depth = 0 else: depth = -1 result = self.format_message(obj.backend, obj.root, depth) return result def format_message(self, backend, message, depth=0): if not message: return u'' self.count += 1 flags = '[' if message.flags & message.IS_UNREAD: flags += 'N' else: flags += '-' if message.flags & message.IS_NOT_RECEIVED: flags += 'U' elif message.flags & message.IS_RECEIVED: flags += 'R' else: flags += '-' flags += ']' if self.interactive: result = u'%s%s* (%d)%s %s <%s> %s (%s)\n' % (depth * ' ', self.BOLD, self.count, self.NC, flags, message.sender, message.title, backend) else: result = u'%s%s* (%s.%s@%s)%s %s <%s> %s\n' % (depth * ' ', self.BOLD, message.thread.id, message.id, backend, self.NC, flags, message.sender, message.title) if message.children: if depth >= 0: depth += 1 for m in message.children: result += self.format_message(backend, m, depth) return result class ProfileFormatter(IFormatter): def flush(self): pass def format_obj(self, obj, alias=None): return obj.get_text() class Boobmsg(ReplApplication): APPNAME = 'boobmsg' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Christophe Benz' DESCRIPTION = "Console application allowing to send messages on various websites and " \ "to display message threads and contents." SHORT_DESCRIPTION = "send and receive message threads" CAPS = CapMessages EXTRA_FORMATTERS = {'msglist': MessagesListFormatter, 'msg': MessageFormatter, 'xhtml': XHtmlFormatter, 'atom': AtomFormatter, 'profile' : ProfileFormatter, } COMMANDS_FORMATTERS = {'list': 'msglist', 'show': 'msg', 'export_thread': 'msg', 'export_all': 'msg', 'ls': 'msglist', 'profile': 'profile', } def add_application_options(self, group): group.add_option('-E', '--accept-empty', action='store_true', help='Send messages with an empty body.') group.add_option('-t', '--title', action='store', help='For the "post" command, set a title to message', type='string', dest='title') def load_default_backends(self): self.load_backends(CapMessages, storage=self.create_storage()) def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def do_status(self, line): """ status Display status information about a backend. """ if len(line) > 0: backend_name = line else: backend_name = None results = {} for field in self.do('get_account_status', backends=backend_name, caps=CapAccount): if field.backend in results: results[field.backend].append(field) else: results[field.backend] = [field] for name, fields in results.iteritems(): print(':: %s ::' % name) for f in fields: if f.flags & f.FIELD_HTML: value = html2text(f.value) else: value = f.value print('%s: %s' % (f.label, value)) print('') def do_post(self, line): """ post RECEIVER@BACKEND[,RECEIVER@BACKEND[...]] [TEXT] Post a message to the specified receivers. Multiple receivers are separated by a comma. If no text is supplied on command line, the content of message is read on stdin. """ receivers, text = self.parse_command_args(line, 2, 1) if text is None: text = self.acquire_input() if not self.options.accept_empty and not text.strip(): self.logger.warning(u'The message body is empty, use option --accept_empty to send empty messages') return for receiver in receivers.strip().split(','): receiver, backend_name = self.parse_id(receiver.strip(), unique_backend=True) if not backend_name and len(self.enabled_backends) > 1: self.logger.warning(u'No backend specified for receiver "%s": message will be sent with all the ' 'enabled backends (%s)' % (receiver, ','.join(backend.name for backend in self.enabled_backends))) if '.' in receiver: # It's a reply thread_id, parent_id = receiver.rsplit('.', 1) else: # It's an original message thread_id = receiver parent_id = None try: thread_id = self.threads[int(thread_id) - 1].id except (IndexError,ValueError): pass thread = Thread(thread_id) message = Message(thread, 0, title=self.options.title, parent=Message(thread, parent_id) if parent_id else None, content=text) try: self.do('post_message', message, backends=backend_name).wait() except CallErrors as errors: self.bcall_errors_handler(errors) else: if self.interactive: print('Message sent sucessfully to %s' % receiver) threads = [] messages = [] @defaultcount(10) def do_list(self, arg): """ list Display all threads. """ if len(arg) > 0: try: thread = self.threads[int(arg) - 1] except (IndexError, ValueError): id, backend_name = self.parse_id(arg) else: id = thread.id backend_name = thread.backend self.messages = [] cmd = self.do('get_thread', id, backends=backend_name) self.formatter._list_messages = True else: self.threads = [] cmd = self.do('iter_threads') self.formatter._list_messages = False self.start_format() for thread in cmd: if not thread: continue if len(arg) > 0: for m in thread.iter_all_messages(): if not m.backend: m.backend = thread.backend self.messages.append(m) else: self.threads.append(thread) self.format(thread) def do_export_all(self, arg): """ export_all Export All threads """ def func(backend): for thread in backend.iter_threads(): if not thread: continue t = backend.fillobj(thread, None) for msg in t.iter_all_messages(): yield msg self.start_format() for msg in self.do(func): self.format(msg) def do_export_thread(self, arg): """ export_thread ID Export the thread identified by ID """ _id, backend_name = self.parse_id(arg) cmd = self.do('get_thread', _id, backends=backend_name) self.start_format() for thread in cmd: if thread is not None: for msg in thread.iter_all_messages(): self.format(msg) def do_show(self, arg): """ show MESSAGE Read a message """ message = None if len(arg) == 0: print('Please give a message ID.', file=self.stderr) return 2 try: message = self.messages[int(arg) - 1] except (IndexError, ValueError): # The message is not is the cache, we have now two cases: # 1) the user uses a number to get a thread in the cache # 2) the user gives a thread id try: thread = self.threads[int(arg) - 1] if not empty(thread.root): message = thread.root else: for thread in self.do('get_thread', thread.id, backends=thread.backend): if thread is not None: message = thread.root except (IndexError, ValueError): _id, backend_name = self.parse_id(arg) for thread in self.do('get_thread', _id, backends=backend_name): if thread is not None: message = thread.root if message is not None: self.start_format() self.format(message) self.weboob.do('set_message_read', message, backends=message.backend) return else: print('Message not found', file=self.stderr) return 3 def do_profile(self, id): """ profile ID Display a profile """ _id, backend_name = self.parse_id(id, unique_backend=True) found = 0 for contact in self.do('get_contact', _id, backends=backend_name, caps=CapContact): if contact: self.format(contact) found = 1 if not found: self.logger.error(u'Profile not found') def do_photos(self, id): """ photos ID Display photos of a profile """ photo_cmd = self.config.get('photo_viewer') if photo_cmd is None: print("Configuration error: photo_viewer is undefined", file=self.stderr) return _id, backend_name = self.parse_id(id, unique_backend=True) found = 0 for contact in self.do('get_contact', _id, backends=backend_name): if contact: # Write photo to temporary files tmp_files = [] for photo in contact.photos.values(): suffix = '.jpg' if '.' in photo.url.split('/')[-1]: suffix = '.%s' % photo.url.split('/')[-1].split('.')[-1] f = NamedTemporaryFile(suffix=suffix) photo = self.weboob[contact.backend].fillobj(photo, 'data') f.write(photo.data) tmp_files.append(f) os.system(photo_cmd % ' '.join([file.name for file in tmp_files])) found = 1 if not found: self.logger.error(u'Profile not found') weboob-1.2/weboob/applications/boobooks/000077500000000000000000000000001303450110500204015ustar00rootroot00000000000000weboob-1.2/weboob/applications/boobooks/__init__.py000066400000000000000000000014311303450110500225110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Jérémy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .boobooks import Boobooks __all__ = ['Boobooks'] weboob-1.2/weboob/applications/boobooks/boobooks.py000066400000000000000000000045731303450110500226010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.library import CapBook, Book from weboob.tools.application.repl import ReplApplication from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['Boobooks'] class RentedListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'date', 'author', 'name', 'late') RED = '' def get_title(self, obj): s = u'%s — %s (%s)' % (obj.author, obj.name, obj.date) if obj.late: s += u' %sLATE!%s' % (self.RED, self.NC) return s class Boobooks(ReplApplication): APPNAME = 'boobooks' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Jeremy Monnet' CAPS = CapBook DESCRIPTION = "Console application allowing to list your books rented or booked at the library, " \ "book and search new ones, get your booking history (if available)." SHORT_DESCRIPTION = "manage rented books" EXTRA_FORMATTERS = {'rented_list': RentedListFormatter, } DEFAULT_FORMATTER = 'table' COMMANDS_FORMATTERS = {'ls': 'rented_list', 'list': 'rented_list', } COLLECTION_OBJECTS = (Book, ) def do_renew(self, id): """ renew ID Renew a book """ id, backend_name = self.parse_id(id) if not id: print('Error: please give a book ID (hint: use ls command)', file=self.stderr) return 2 names = (backend_name,) if backend_name is not None else None for renew in self.do('renew_book', id, backends=names): self.format(renew) weboob-1.2/weboob/applications/boobsize/000077500000000000000000000000001303450110500204005ustar00rootroot00000000000000weboob-1.2/weboob/applications/boobsize/__init__.py000066400000000000000000000014251303450110500225130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .boobsize import Boobsize __all__ = ['Boobsize'] weboob-1.2/weboob/applications/boobsize/boobsize.py000066400000000000000000000156661303450110500226040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.base import empty from weboob.capabilities.gauge import CapGauge, SensorNotFound from weboob.tools.application.repl import ReplApplication from weboob.tools.application.formatters.iformatter import IFormatter __all__ = ['Boobsize'] class GaugeFormatter(IFormatter): MANDATORY_FIELDS = ('name', 'object', 'sensors') DISPLAYED_FIELDS = ('city', ) def start_format(self, **kwargs): # Name = 27 Object = 10 City = 10 Sensors = 33 self.output(' Name and ID Object City Sensors ') self.output('----------------------------+----------+----------+---------------------------------') def format_obj(self, obj, alias): name = obj.name city = u"" if not empty(obj.city): city = obj.city if not obj.sensors or (len(obj.sensors) == 0): result = u' %s %s %s \n' %\ (self.colored('%-27s' % name[:27], 'red'), self.colored('%-10s' % obj.object[:10], 'yellow'), self.colored('%-10s' % city[:10], 'yellow') ) result += u' %s \n' % self.colored('%-47s' % obj.fullid[:47], 'blue') else: first = True firstaddress = obj.sensors[0].address for sensor in obj.sensors: sensorname = sensor.name # This is a int value, do not display it as a float if not empty(sensor.lastvalue.level): if int(sensor.lastvalue.level) == sensor.lastvalue.level: lastvalue = "%d " % sensor.lastvalue.level else: lastvalue = "%r " % sensor.lastvalue.level if not empty(sensor.unit): lastvalue += "%s" % sensor.unit else: lastvalue = u"? " if first: result = u' %s %s %s ' %\ (self.colored('%-27s' % name[:27], 'red'), self.colored('%-10s' % obj.object[:10], 'yellow'), self.colored('%-10s' % city[:10], 'yellow'), ) if not empty(firstaddress): result += u'%s' % self.colored('%-33s' % sensor.address[:33], 'yellow') result += u'\n' result += u' %s' % self.colored('%-47s' % obj.fullid[:47], 'blue') result += u' %s %s\n' %\ (self.colored('%-20s' % sensorname[:20], 'magenta'), self.colored('%-13s' % lastvalue[:13], 'red') ) first = False else: result += u' %s %s\n' %\ (self.colored('%-20s' % sensorname[:20], 'magenta'), self.colored('%-13s' % lastvalue[:13], 'red') ) if not empty(sensor.address) and sensor.address != firstaddress: result += u' %s \n' %\ self.colored('%-33s' % sensor.address[:33], 'yellow') return result class Boobsize(ReplApplication): APPNAME = 'Boobsize' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Florent Fourcot' DESCRIPTION = "Console application allowing to display various sensors and gauges values." SHORT_DESCRIPTION = "display sensors and gauges values" CAPS = (CapGauge) DEFAULT_FORMATTER = 'table' EXTRA_FORMATTERS = {'gauge_list': GaugeFormatter, } COMMANDS_FORMATTERS = {'search': 'gauge_list', } def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def bcall_error_handler(self, backend, error, backtrace): if isinstance(error, SensorNotFound): msg = unicode(error) or 'Sensor not found (hint: try details command)' print('Error(%s): %s' % (backend.name, msg), file=self.stderr) else: return ReplApplication.bcall_error_handler(self, backend, error, backtrace) def do_search(self, pattern): """ search [PATTERN] Display all gauges. If PATTERN is specified, search on a pattern. """ self.change_path([u'gauges']) self.start_format() for gauge in self.do('iter_gauges', pattern or None, caps=CapGauge): self.cached_format(gauge) def complete_search(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_details(self, line): """ details GAUGE_ID Display details of all sensors of the gauge. """ gauge, pattern = self.parse_command_args(line, 2, 1) _id, backend_name = self.parse_id(gauge) self.start_format() for sensor in self.do('iter_sensors', _id, pattern=pattern, backends=backend_name, caps=CapGauge): self.format(sensor) def do_history(self, line): """ history SENSOR_ID Get history of a specific sensor (use 'search' to find a gauge, and sensors GAUGE_ID to list sensors attached to the gauge). """ gauge, = self.parse_command_args(line, 1, 1) _id, backend_name = self.parse_id(gauge) self.start_format() for measure in self.do('iter_gauge_history', _id, backends=backend_name, caps=CapGauge): self.format(measure) def complete_last_sensor_measure(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_last_sensor_measure(self, line): """ last_sensor_measure SENSOR_ID Get last measure of a sensor. """ gauge, = self.parse_command_args(line, 1, 1) _id, backend_name = self.parse_id(gauge) self.start_format() for measure in self.do('get_last_measure', _id, backends=backend_name, caps=CapGauge): self.format(measure) weboob-1.2/weboob/applications/boobtracker/000077500000000000000000000000001303450110500210615ustar00rootroot00000000000000weboob-1.2/weboob/applications/boobtracker/__init__.py000066400000000000000000000014331303450110500231730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .boobtracker import BoobTracker __all__ = ['BoobTracker'] weboob-1.2/weboob/applications/boobtracker/boobtracker.py000066400000000000000000000426561303450110500237450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from datetime import timedelta from email import message_from_string, message_from_file from email.Header import decode_header from email.mime.text import MIMEText from smtplib import SMTP import os import re import unicodedata from weboob.capabilities.base import empty, BaseObject from weboob.capabilities.bugtracker import CapBugTracker, Query, Update, Project, Issue, IssueError from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.tools.html import html2text from weboob.tools.date import parse_french_date __all__ = ['BoobTracker'] class IssueFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'project', 'title', 'body', 'author') def format_attr(self, obj, attr): if not hasattr(obj, attr) or empty(getattr(obj, attr)): return u'' value = getattr(obj, attr) if isinstance(value, BaseObject): value = value.name return self.format_key(attr.capitalize(), value) def format_key(self, key, value): return '%s %s\n' % (self.colored('%s:' % key, 'green'), value) def format_obj(self, obj, alias): result = u'%s %s %s %s %s\n' % (self.colored(obj.project.name, 'blue', 'bold'), self.colored(u'—', 'cyan', 'bold'), self.colored(obj.fullid, 'red', 'bold'), self.colored(u'—', 'cyan', 'bold'), self.colored(obj.title, 'yellow', 'bold')) result += '\n%s\n\n' % obj.body result += self.format_key('Author', '%s (%s)' % (obj.author.name, obj.creation)) result += self.format_attr(obj, 'status') result += self.format_attr(obj, 'priority') result += self.format_attr(obj, 'version') result += self.format_attr(obj, 'tracker') result += self.format_attr(obj, 'category') result += self.format_attr(obj, 'assignee') if hasattr(obj, 'fields') and not empty(obj.fields): for key, value in obj.fields.iteritems(): result += self.format_key(key.capitalize(), value) if hasattr(obj, 'attachments') and obj.attachments: result += '\n%s\n' % self.colored('Attachments:', 'green') for a in obj.attachments: result += '* %s%s%s <%s>\n' % (self.BOLD, a.filename, self.NC, a.url) if hasattr(obj, 'history') and obj.history: result += '\n%s\n' % self.colored('History:', 'green') for u in obj.history: result += '%s %s %s %s\n' % (self.colored('*', 'red', 'bold'), self.colored(u.date, 'yellow', 'bold'), self.colored(u'—', 'cyan', 'bold'), self.colored(u.author.name, 'blue', 'bold')) for change in u.changes: result += ' - %s %s %s %s\n' % (self.colored(change.field, 'green'), change.last, self.colored('->', 'magenta'), change.new) if u.message: result += ' %s\n' % html2text(u.message).strip().replace('\n', '\n ') return result class IssuesListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'project', 'status', 'title', 'category') def get_title(self, obj): return '%s - [%s] %s' % (obj.project.name, obj.status.name, obj.title) def get_description(self, obj): return obj.category class BoobTracker(ReplApplication): APPNAME = 'boobtracker' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2011-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to create, edit, view bug tracking issues." SHORT_DESCRIPTION = "manage bug tracking issues" CAPS = CapBugTracker EXTRA_FORMATTERS = {'issue_info': IssueFormatter, 'issues_list': IssuesListFormatter, } COMMANDS_FORMATTERS = {'get': 'issue_info', 'post': 'issue_info', 'edit': 'issue_info', 'search': 'issues_list', 'ls': 'issues_list', } COLLECTION_OBJECTS = (Project, Issue, ) def add_application_options(self, group): group.add_option('--author') group.add_option('--title') group.add_option('--assignee') group.add_option('--target-version', dest='version') group.add_option('--tracker') group.add_option('--category') group.add_option('--status') group.add_option('--priority') group.add_option('--start') group.add_option('--due') @defaultcount(10) def do_search(self, line): """ search PROJECT List issues for a project. You can use these filters from command line: --author AUTHOR --title TITLE_PATTERN --assignee ASSIGNEE --target-version VERSION --category CATEGORY --status STATUS """ query = Query() path = self.working_path.get() backends = [] if line.strip(): query.project, backends = self.parse_id(line, unique_backend=True) elif len(path) > 0: query.project = path[0] else: print('Please enter a project name', file=self.stderr) return 1 query.author = self.options.author query.title = self.options.title query.assignee = self.options.assignee query.version = self.options.version query.category = self.options.category query.status = self.options.status self.change_path([query.project, u'search']) for issue in self.do('iter_issues', query, backends=backends): self.add_object(issue) self.format(issue) def complete_get(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_get(self, line): """ get ISSUE Get an issue and display it. """ if not line: print('This command takes an argument: %s' % self.get_command_help('get', short=True), file=self.stderr) return 2 issue = self.get_object(line, 'get_issue') if not issue: print('Issue not found: %s' % line, file=self.stderr) return 3 self.format(issue) def complete_comment(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_comment(self, line): """ comment ISSUE [TEXT] Comment an issue. If no text is given, enter it in standard input. """ id, text = self.parse_command_args(line, 2, 1) if text is None: text = self.acquire_input() id, backend_name = self.parse_id(id, unique_backend=True) update = Update(0) update.message = text self.do('update_issue', id, update, backends=backend_name).wait() def do_logtime(self, line): """ logtime ISSUE HOURS [TEXT] Log spent time on an issue. """ id, hours, text = self.parse_command_args(line, 3, 2) if text is None: text = self.acquire_input() try: hours = float(hours) except ValueError: print('Error: HOURS parameter may be a float', file=self.stderr) return 1 id, backend_name = self.parse_id(id, unique_backend=True) update = Update(0) update.message = text update.hours = timedelta(hours=hours) self.do('update_issue', id, update, backends=backend_name).wait() def complete_remove(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_remove(self, line): """ remove ISSUE Remove an issue. """ id, backend_name = self.parse_id(line, unique_backend=True) self.do('remove_issue', id, backends=backend_name).wait() ISSUE_FIELDS = (('title', (None, False)), ('assignee', ('members', True)), ('version', ('versions', True)), ('tracker', (None, False)),#XXX ('category', ('categories', False)), ('status', ('statuses', True)), ('priority', (None, False)),#XXX ('start', (None, False)), ('due', (None, False)), ) def get_list_item(self, objects_list, name): if name is None: return None for obj in objects_list: if obj.name.lower() == name.lower(): return obj if not name: return None raise ValueError('"%s" is not found' % name) def sanitize_key(self, key): if isinstance(key, str): key = unicode(key, "utf8") key = unicodedata.normalize('NFKD', key).encode("ascii", "ignore") return key.replace(' ', '-').capitalize() def issue2text(self, issue, backend=None): if backend is not None and 'username' in backend.config: sender = backend.config['username'].get() else: sender = os.environ.get('USERNAME', 'boobtracker') output = u'From: %s\n' % sender for key, (list_name, is_list_object) in self.ISSUE_FIELDS: value = None if not self.interactive: value = getattr(self.options, key) if not value: value = getattr(issue, key) if not value: value = '' elif hasattr(value, 'name'): value = value.name if list_name is not None: objects_list = getattr(issue.project, list_name) if len(objects_list) == 0: continue output += '%s: %s\n' % (self.sanitize_key(key), value) if list_name is not None: availables = ', '.join(['<%s>' % (o if isinstance(o, basestring) else o.name) for o in objects_list]) output += 'X-Available-%s: %s\n' % (self.sanitize_key(key), availables) for key, value in issue.fields.iteritems(): output += '%s: %s\n' % (self.sanitize_key(key), value or '') # TODO: Add X-Available-* for lists output += '\n%s' % (issue.body or 'Please write your bug report here.') return output def text2issue(self, issue, m): # XXX HACK to support real incoming emails if 'Subject' in m: m['Title'] = m['Subject'] for key, (list_name, is_list_object) in self.ISSUE_FIELDS: value = m.get(key) if value is None: continue new_value = u'' for part in decode_header(value): if part[1]: new_value += unicode(part[0], part[1]) else: new_value += part[0].decode('utf-8') value = new_value if is_list_object: objects_list = getattr(issue.project, list_name) value = self.get_list_item(objects_list, value) # FIXME: autodetect if key in ['start', 'due']: if len(value) > 0: #value = datetime.strptime(value, "%Y-%m-%d %H:%M:%S") value = parse_french_date(value) else: value = None setattr(issue, key, value) for key in issue.fields.keys(): value = m.get(self.sanitize_key(key)) if value is not None: issue.fields[key] = value.decode('utf-8') content = u'' for part in m.walk(): if part.get_content_type() == 'text/plain': s = part.get_payload(decode=True) charsets = part.get_charsets() + m.get_charsets() for charset in charsets: try: if charset is not None: content += unicode(s, charset) else: content += unicode(s, encoding='utf-8') except UnicodeError as e: self.logger.warning('Unicode error: %s' % e) continue except Exception as e: self.logger.exception(e) continue else: break issue.body = content m = re.search('([^< ]+@[^ >]+)', m['From'] or '') if m: return m.group(1) def edit_issue(self, issue, edit=True): backend = self.weboob.get_backend(issue.backend) content = self.issue2text(issue, backend) while True: if self.stdin.isatty(): content = self.acquire_input(content, {'vim': "-c 'set ft=mail'"}) m = message_from_string(content.encode('utf-8')) else: m = message_from_file(self.stdin) try: email_to = self.text2issue(issue, m) except ValueError as e: if not self.stdin.isatty(): raise raw_input("%s -- Press Enter to continue..." % unicode(e).encode("utf-8")) continue try: issue = backend.post_issue(issue) print('Issue %s %s' % (self.formatter.colored(issue.fullid, 'red', 'bold'), 'updated' if edit else 'created')) if edit: self.format(issue) elif email_to: self.send_notification(email_to, issue) return 0 except IssueError as e: if not self.stdin.isatty(): raise raw_input("%s -- Press Enter to continue..." % unicode(e).encode("utf-8")) def send_notification(self, email_to, issue): text = """Hi, You have successfuly created this ticket on the Weboob tracker: %s You can follow your bug report on this page: https://symlink.me/issues/%s Regards, Weboob Team """ % (issue.title, issue.id) msg = MIMEText(text, 'plain', 'utf-8') msg['Subject'] = 'Issue #%s reported' % issue.id msg['From'] = 'Weboob ' msg['To'] = email_to s = SMTP('localhost') s.sendmail('weboob@weboob.org', [email_to], msg.as_string()) s.quit() def do_post(self, line): """ post PROJECT Post a new issue. If you are not in interactive mode, you can use these parameters: --title TITLE --assignee ASSIGNEE --target-version VERSION --category CATEGORY --status STATUS """ if not line.strip(): print('Please give the project name') return 1 project, backend_name = self.parse_id(line, unique_backend=True) backend = self.weboob.get_backend(backend_name) issue = backend.create_issue(project) issue.backend = backend.name return self.edit_issue(issue, edit=False) def complete_edit(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() if len(args) == 3: return dict(self.ISSUE_FIELDS).keys() def do_edit(self, line): """ edit ISSUE [KEY [VALUE]] Edit an issue. If you are not in interactive mode, you can use these parameters: --title TITLE --assignee ASSIGNEE --target-version VERSION --category CATEGORY --status STATUS """ _id, key, value = self.parse_command_args(line, 3, 1) issue = self.get_object(_id, 'get_issue') if not issue: print('Issue not found: %s' % _id, file=self.stderr) return 3 return self.edit_issue(issue, edit=True) def complete_attach(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_attach(self, line): """ attach ISSUE FILENAME Attach a file to an issue (Not implemented yet). """ print('Not implemented yet.', file=self.stderr) weboob-1.2/weboob/applications/cineoob/000077500000000000000000000000001303450110500202025ustar00rootroot00000000000000weboob-1.2/weboob/applications/cineoob/__init__.py000066400000000000000000000014211303450110500223110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .cineoob import Cineoob __all__ = ['Cineoob'] weboob-1.2/weboob/applications/cineoob/cineoob.py000066400000000000000000000640111303450110500221740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from datetime import datetime from weboob.applications.weboorrents.weboorrents import TorrentInfoFormatter, TorrentListFormatter from weboob.applications.suboob.suboob import SubtitleInfoFormatter, SubtitleListFormatter from weboob.capabilities.torrent import CapTorrent, MagnetOnly from weboob.capabilities.cinema import CapCinema from weboob.capabilities.subtitle import CapSubtitle from weboob.capabilities.base import empty, NotAvailable from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.core import CallErrors __all__ = ['Cineoob'] ROLE_LIST = ['actor', 'director', 'writer', 'composer', 'producer'] COUNTRY_LIST = ['us', 'fr', 'de', 'jp'] class MovieInfoFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'original_title', 'release_date', 'other_titles', 'duration', 'pitch', 'note', 'roles', 'country') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.original_title, self.NC) result += 'ID: %s\n' % obj.fullid if not empty(obj.release_date): result += 'Released: %s\n' % obj.release_date.strftime('%Y-%m-%d') result += 'Country: %s\n' % obj.country if not empty(obj.duration): result += 'Duration: %smin\n' % obj.duration result += 'Note: %s\n' % obj.note if not empty(obj.genres): result += '\n%sGenres%s\n' % (self.BOLD, self.NC) for g in obj.genres: result += ' * %s\n' % g if not empty(obj.roles): result += '\n%sRelated persons%s\n' % (self.BOLD, self.NC) for role, lpersons in obj.roles.items(): result += ' -- %s\n' % role for person in lpersons: result += ' * %s\n' % person[1] if not empty(obj.other_titles): result += '\n%sOther titles%s\n' % (self.BOLD, self.NC) for t in obj.other_titles: result += ' * %s\n' % t if not empty(obj.pitch): result += '\n%sStory%s\n' % (self.BOLD, self.NC) result += '%s' % obj.pitch return result class MovieListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'original_title', 'short_description') def get_title(self, obj): return obj.original_title def get_description(self, obj): result = u'' if not empty(obj.short_description): result = obj.short_description return result class MovieReleasesFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'original_title', 'all_release_dates') def get_title(self, obj): return u'Releases of %s' % obj.original_title def get_description(self, obj): return u'\n%s' % obj.all_release_dates def yearsago(years, from_date=None): if from_date is None: from_date = datetime.now() try: return from_date.replace(year=from_date.year - years) except: # Must be 2/29 assert from_date.month == 2 and from_date.day == 29 return from_date.replace(month=2, day=28, year=from_date.year-years) def num_years(begin, end=None): if end is None: end = datetime.now() num_years = int((end - begin).days / 365.25) if begin > yearsago(num_years, end): return num_years - 1 else: return num_years class PersonInfoFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'name', 'birth_date', 'birth_place', 'short_biography') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.name, self.NC) result += 'ID: %s\n' % obj.fullid if not empty(obj.real_name): result += 'Real name: %s\n' % obj.real_name if not empty(obj.birth_place): result += 'Birth place: %s\n' % obj.birth_place if not empty(obj.birth_date): result += 'Birth date: %s\n' % obj.birth_date.strftime('%Y-%m-%d') if not empty(obj.death_date): age = num_years(obj.birth_date, obj.death_date) result += 'Death date: %s at %s years old\n' % (obj.death_date.strftime('%Y-%m-%d'), age) else: age = num_years(obj.birth_date) result += 'Age: %s\n' % age if not empty(obj.gender): result += 'Gender: %s\n' % obj.gender if not empty(obj.nationality): result += 'Nationality: %s\n' % obj.nationality if not empty(obj.roles): result += '\n%sRelated movies%s\n' % (self.BOLD, self.NC) for role, lmovies in obj.roles.items(): result += ' -- %s\n' % role for movie in lmovies: result += ' * %s\n' % movie[1] if not empty(obj.short_biography): result += '\n%sShort biography%s\n' % (self.BOLD, self.NC) result += '%s' % obj.short_biography return result class PersonListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name', 'short_description') def get_title(self, obj): return obj.name def get_description(self, obj): result = u'' if not empty(obj.short_description): result = obj.short_description return result class PersonBiographyFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name', 'biography') def get_title(self, obj): return u'Biography of %s' % obj.name def get_description(self, obj): result = u'\n%s' % obj.biography return result class Cineoob(ReplApplication): APPNAME = 'cineoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Julien Veyssier' DESCRIPTION = "Console application allowing to search for movies and persons on various cinema databases " \ ", list persons related to a movie, list movies related to a person and list common movies " \ "of two persons." SHORT_DESCRIPTION = "search movies and persons around cinema" CAPS = (CapCinema, CapTorrent, CapSubtitle) EXTRA_FORMATTERS = {'movie_list': MovieListFormatter, 'movie_info': MovieInfoFormatter, 'movie_releases': MovieReleasesFormatter, 'person_list': PersonListFormatter, 'person_info': PersonInfoFormatter, 'person_bio': PersonBiographyFormatter, 'torrent_list': TorrentListFormatter, 'torrent_info': TorrentInfoFormatter, 'subtitle_list': SubtitleListFormatter, 'subtitle_info': SubtitleInfoFormatter } COMMANDS_FORMATTERS = {'search_movie': 'movie_list', 'info_movie': 'movie_info', 'search_person': 'person_list', 'info_person': 'person_info', 'casting': 'person_list', 'filmography': 'movie_list', 'biography': 'person_bio', 'releases': 'movie_releases', 'movies_in_common': 'movie_list', 'persons_in_common': 'person_list', 'search_torrent': 'torrent_list', 'search_movie_torrent': 'torrent_list', 'info_torrent': 'torrent_info', 'search_subtitle': 'subtitle_list', 'search_movie_subtitle': 'subtitle_list', 'info_subtitle': 'subtitle_info' } def complete_filmography(self, text, line, *ignored): args = line.split(' ') if len(args) == 3: return ROLE_LIST def complete_casting(self, text, line, *ignored): return self.complete_filmography(text, line, ignored) def do_movies_in_common(self, line): """ movies_in_common person_ID person_ID Get the list of common movies between two persons. """ id1, id2 = self.parse_command_args(line, 2, 1) person1 = self.get_object(id1, 'get_person', caps=CapCinema) if not person1: print('Person not found: %s' % id1, file=self.stderr) return 3 person2 = self.get_object(id2, 'get_person', caps=CapCinema) if not person2: print('Person not found: %s' % id2, file=self.stderr) return 3 initial_count = self.options.count self.options.count = None lid1 = [] for id in self.do('iter_person_movies_ids', person1.id, caps=CapCinema): lid1.append(id) lid2 = [] for id in self.do('iter_person_movies_ids', person2.id, caps=CapCinema): lid2.append(id) self.options.count = initial_count inter = list(set(lid1) & set(lid2)) chrono_list = [] for common in inter: movie = self.get_object(common, 'get_movie', caps=CapCinema) role1 = movie.get_roles_by_person_id(person1.id) if not role1: role1 = movie.get_roles_by_person_name(person1.name) role2 = movie.get_roles_by_person_id(person2.id) if not role2: role2 = movie.get_roles_by_person_name(person2.name) if (movie.release_date != NotAvailable): year = movie.release_date.year else: year = '????' movie.short_description = '(%s) %s as %s ; %s as %s'%(year, person1.name, ', '.join(role1), person2.name, ', '.join(role2)) if movie: i = 0 while (i chrono_list[i].release_date.year)): i += 1 chrono_list.insert(i, movie) for movie in chrono_list: self.cached_format(movie) def do_persons_in_common(self, line): """ persons_in_common movie_ID movie_ID Get the list of common persons between two movies. """ id1, id2 = self.parse_command_args(line, 2, 1) movie1 = self.get_object(id1, 'get_movie', caps=CapCinema) if not movie1: print('Movie not found: %s' % id1, file=self.stderr) return 3 movie2 = self.get_object(id2, 'get_movie', caps=CapCinema) if not movie2: print('Movie not found: %s' % id2, file=self.stderr) return 3 initial_count = self.options.count self.options.count = None lid1 = [] for id in self.do('iter_movie_persons_ids', movie1.id, caps=CapCinema): lid1.append(id) lid2 = [] for id in self.do('iter_movie_persons_ids', movie2.id, caps=CapCinema): lid2.append(id) self.options.count = initial_count inter = list(set(lid1) & set(lid2)) for common in inter: person = self.get_object(common, 'get_person', caps=CapCinema) role1 = person.get_roles_by_movie_id(movie1.id) if not role1: role1 = person.get_roles_by_movie_title(movie1.original_title) role2 = person.get_roles_by_movie_id(movie2.id) if not role2: role2 = person.get_roles_by_movie_title(movie2.original_title) person.short_description = '%s in %s ; %s in %s'%(', '.join(role1), movie1.original_title, ', '.join(role2), movie2.original_title) self.cached_format(person) def do_info_movie(self, id): """ info_movie movie_ID Get information about a movie. """ movie = self.get_object(id, 'get_movie', caps=CapCinema) if not movie: print('Movie not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(movie) def do_info_person(self, id): """ info_person person_ID Get information about a person. """ person = self.get_object(id, 'get_person', caps=CapCinema) if not person: print('Person not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(person) @defaultcount(10) def do_search_movie(self, pattern): """ search_movie [PATTERN] Search movies. """ self.change_path([u'search movies']) if not pattern: pattern = None self.start_format(pattern=pattern) for movie in self.do('iter_movies', pattern=pattern, caps=CapCinema): self.cached_format(movie) @defaultcount(10) def do_search_person(self, pattern): """ search_person [PATTERN] Search persons. """ self.change_path([u'search persons']) if not pattern: pattern = None self.start_format(pattern=pattern) for person in self.do('iter_persons', pattern=pattern, caps=CapCinema): self.cached_format(person) def do_casting(self, line): """ casting movie_ID [ROLE] List persons related to a movie. If ROLE is given, filter by ROLE """ movie_id, role = self.parse_command_args(line, 2, 1) movie = self.get_object(movie_id, 'get_movie', caps=CapCinema) if not movie: print('Movie not found: %s' % id, file=self.stderr) return 3 for person in self.do('iter_movie_persons', movie.id, role, backends=movie.backend, caps=CapCinema): self.cached_format(person) def do_filmography(self, line): """ filmography person_ID [ROLE] List movies of a person. If ROLE is given, filter by ROLE """ person_id, role = self.parse_command_args(line, 2, 1) person = self.get_object(person_id, 'get_person', caps=CapCinema) if not person: print('Person not found: %s' % id, file=self.stderr) return 3 for movie in self.do('iter_person_movies', person.id, role, backends=person.backend, caps=CapCinema): self.cached_format(movie) def do_biography(self, person_id): """ biography person_ID Show the complete biography of a person. """ person = self.get_object(person_id, 'get_person', ('name', 'biography'), caps=CapCinema) if not person: print('Person not found: %s' % person_id, file=self.stderr) return 3 self.start_format() self.format(person) def complete_releases(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() if len(args) == 3: return COUNTRY_LIST def do_releases(self, line): """ releases movie_ID [COUNTRY] Get releases dates of a movie. If COUNTRY is given, show release in this country. """ id, country = self.parse_command_args(line, 2, 1) movie = self.get_object(id, 'get_movie', ('original_title'), caps=CapCinema) if not movie: print('Movie not found: %s' % id, file=self.stderr) return 3 # i would like to clarify with fillobj but how could i fill the movie AND choose the country ? for release in self.do('get_movie_releases', movie.id, country, caps=CapCinema, backends=movie.backend): if not empty(release): movie.all_release_dates = u'%s' % (release) else: print('Movie releases not found for %s' % movie.original_title, file=self.stderr) return 3 self.start_format() self.format(movie) # ================== TORRENT ================== def complete_info_torrent(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info_torrent(self, id): """ info_torrent ID Get information about a torrent. """ torrent = self.get_object(id, 'get_torrent', caps=CapTorrent) if not torrent: print('Torrent not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(torrent) def complete_getfile_torrent(self, text, line, *ignored): args = line.split(' ', 2) if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_getfile_torrent(self, line): """ getfile_torrent ID [FILENAME] Get the .torrent file. FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. """ id, dest = self.parse_command_args(line, 2, 1) _id, backend_name = self.parse_id(id) if dest is None: dest = '%s.torrent' % _id try: for buf in self.do('get_torrent_file', _id, backends=backend_name, caps=CapTorrent): if buf: if dest == '-': print(buf) else: try: with open(dest, 'wb') as f: f.write(buf) except IOError as e: print('Unable to write .torrent in "%s": %s' % (dest, e), file=self.stderr) return 1 return except CallErrors as errors: for backend, error, backtrace in errors: if isinstance(error, MagnetOnly): print(u'Error(%s): No direct URL available, ' u'please provide this magnet URL ' u'to your client:\n%s' % (backend, error.magnet), file=self.stderr) return 4 else: self.bcall_error_handler(backend, error, backtrace) print('Torrent "%s" not found' % id, file=self.stderr) return 3 @defaultcount(10) def do_search_torrent(self, pattern): """ search_torrent [PATTERN] Search torrents. """ self.change_path([u'search torrent']) if not pattern: pattern = None self.start_format(pattern=pattern) for torrent in self.do('iter_torrents', pattern=pattern, caps=CapTorrent): self.cached_format(torrent) @defaultcount(10) def do_search_movie_torrent(self, id): """ search_movie_torrent movie_ID Search torrents of movie_ID. """ movie = self.get_object(id, 'get_movie', ('original_title'), caps=CapCinema) if not movie: print('Movie not found: %s' % id, file=self.stderr) return 3 pattern = movie.original_title self.change_path([u'search torrent']) if not pattern: pattern = None self.start_format(pattern=pattern) for torrent in self.do('iter_torrents', pattern=pattern, caps=CapTorrent): self.cached_format(torrent) # ================== SUBTITLE ================== def complete_info_subtitle(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info_subtitle(self, id): """ info_subtitle subtitle_ID Get information about a subtitle. """ subtitle = self.get_object(id, 'get_subtitle', caps=CapCinema) if not subtitle: print('Subtitle not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(subtitle) def complete_getfile_subtitle(self, text, line, *ignored): args = line.split(' ', 2) if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_getfile_subtitle(self, line): """ getfile_subtitle subtitle_ID [FILENAME] Get the subtitle or archive file. FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. """ id, dest = self.parse_command_args(line, 2, 1) _id, backend_name = self.parse_id(id) if dest is None: dest = '%s' % _id for buf in self.do('get_subtitle_file', _id, backends=backend_name, caps=CapSubtitle): if buf: if dest == '-': print(buf) else: try: with open(dest, 'w') as f: f.write(buf) except IOError as e: print('Unable to write file in "%s": %s' % (dest, e), file=self.stderr) return 1 return print('Subtitle "%s" not found' % id, file=self.stderr) return 3 @defaultcount(10) def do_search_subtitle(self, line): """ search_subtitle language [PATTERN] Search subtitles. Language Abbreviation ---------------------- Arabic ar Esperanto eo Irish ga Russian ru Afrikaans af Estonian et Italian it Serbian sr Albanian sq Filipino tl Japanese ja Slovak sk Armenian hy Finnish fi Kannada kn Slovenian sl Azerbaijani az French fr Korean ko Spanish es Basque eu Galician gl Latin la Swahili sw Belarusian be Georgian ka Latvian lv Swedish sv Bengali bn German de Lithuanian lt Tamil ta Bulgarian bg Greek gr Macedonian mk Telugu te Catalan ca Gujarati gu Malay ms Thai th Chinese zh Haitian ht Maltese mt Turkish tr Croatian hr Hebrew iw Norwegian no Ukrainian uk Czech cz Hindi hi Persian fa Urdu ur Danish da Hungaric hu Polish pl Vietnamese vi Dutch nl Icelandic is Portuguese pt Welsh cy English en Indonesian id Romanian ro Yiddish yi ---------------------- """ language, pattern = self.parse_command_args(line, 2, 1) self.change_path([u'search subtitle']) if not pattern: pattern = None self.start_format(pattern=pattern) for subtitle in self.do('iter_subtitles', language=language, pattern=pattern, caps=CapSubtitle): self.cached_format(subtitle) @defaultcount(10) def do_search_movie_subtitle(self, line): """ search_movie_subtitle language movie_ID Search subtitles of movie_ID. Language Abbreviation ---------------------- Arabic ar Esperanto eo Irish ga Russian ru Afrikaans af Estonian et Italian it Serbian sr Albanian sq Filipino tl Japanese ja Slovak sk Armenian hy Finnish fi Kannada kn Slovenian sl Azerbaijani az French fr Korean ko Spanish es Basque eu Galician gl Latin la Swahili sw Belarusian be Georgian ka Latvian lv Swedish sv Bengali bn German de Lithuanian lt Tamil ta Bulgarian bg Greek gr Macedonian mk Telugu te Catalan ca Gujarati gu Malay ms Thai th Chinese zh Haitian ht Maltese mt Turkish tr Croatian hr Hebrew iw Norwegian no Ukrainian uk Czech cz Hindi hi Persian fa Urdu ur Danish da Hungaric hu Polish pl Vietnamese vi Dutch nl Icelandic is Portuguese pt Welsh cy English en Indonesian id Romanian ro Yiddish yi ---------------------- """ language, id = self.parse_command_args(line, 2, 2) movie = self.get_object(id, 'get_movie', ('original_title'), caps=CapCinema) if not movie: print('Movie not found: %s' % id, file=self.stderr) return 3 pattern = movie.original_title self.change_path([u'search subtitle']) if not pattern: pattern = None self.start_format(pattern=pattern) for subtitle in self.do('iter_subtitles', language=language, pattern=pattern, caps=CapSubtitle): self.cached_format(subtitle) weboob-1.2/weboob/applications/comparoob/000077500000000000000000000000001303450110500205455ustar00rootroot00000000000000weboob-1.2/weboob/applications/comparoob/__init__.py000066400000000000000000000001231303450110500226520ustar00rootroot00000000000000# -*- coding: utf-8 -*- from .comparoob import Comparoob __all__ = ['Comparoob'] weboob-1.2/weboob/applications/comparoob/comparoob.py000066400000000000000000000140421303450110500231010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.pricecomparison import CapPriceComparison from weboob.tools.html import html2text from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.tools.application.base import MoreResultsAvailable from weboob.tools.application.console import ConsoleApplication __all__ = ['Comparoob'] class PriceFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'cost', 'currency', 'shop', 'product') def format_obj(self, obj, alias): if hasattr(obj, 'message') and obj.message: message = obj.message else: message = u'%s (%s)' % (obj.shop.name, obj.shop.location) result = u'%s%s%s\n' % (self.BOLD, message, self.NC) result += u'ID: %s\n' % obj.fullid result += u'Product: %s\n' % obj.product.name result += u'Cost: %s%s\n' % (obj.cost, obj.currency) if hasattr(obj, 'date') and obj.date: result += u'Date: %s\n' % obj.date.strftime('%Y-%m-%d') result += u'\n%sShop:%s\n' % (self.BOLD, self.NC) result += u'\tName: %s\n' % obj.shop.name if obj.shop.location: result += u'\tLocation: %s\n' % obj.shop.location if obj.shop.info: result += u'\n\t' + html2text(obj.shop.info).replace('\n', '\n\t').strip() return result class PricesFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'cost', 'currency') def get_title(self, obj): if hasattr(obj, 'message') and obj.message: message = obj.message elif hasattr(obj, 'shop') and obj.shop: message = '%s (%s)' % (obj.shop.name, obj.shop.location) else: return u'%s%s' % (obj.cost, obj.currency) return u'%s%s - %s' % (obj.cost, obj.currency, message) def get_description(self, obj): if obj.date: return obj.date.strftime('%Y-%m-%d') class Comparoob(ReplApplication): APPNAME = 'comparoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Romain Bignon' DESCRIPTION = "Console application to compare products." SHORT_DESCRIPTION = "compare products" DEFAULT_FORMATTER = 'table' EXTRA_FORMATTERS = {'prices': PricesFormatter, 'price': PriceFormatter, } COMMANDS_FORMATTERS = {'prices': 'prices', 'info': 'price', } CAPS = CapPriceComparison @defaultcount(10) def do_prices(self, pattern): """ prices [PATTERN] Display prices for a product. If a pattern is supplied, do not prompt what product to compare. """ products = {} for product in self.do('search_products', pattern): double = False for prod in products.keys(): if product.name == prod: double = True products[product.name].append(product) break if not double: products[product.name] = [product] products_type = None products_names = products.keys() if len(products_names) == 0: print('Error: no product found with this pattern', file=self.stderr) return 1 elif len(products_names) == 1: products_type = products.keys()[0] else: print('What product do you want to compare?') for i, p in enumerate(products_names): print(' %s%2d)%s %s' % (self.BOLD, i+1, self.NC, p)) r = int(self.ask(' Select a product', regexp='\d+')) while products_type is None: if r <= 0 or r > len(products): print('Error: Please enter a valid ID') continue products_type = products_names[r-1] self.change_path([u'prices']) _products = self.get_object_list('iter_prices', products.get(products_type)) self._sort_display_products(_products) def _sort_display_products(self, products): if products: self.start_format() for price in sorted(products, key=self._get_price): self.cached_format(price) def bcall_errors_handler(self, errors, debugmsg='Use --debug option to print backtraces', ignore=()): for backend, error, backtrace in errors.errors: if isinstance(error, MoreResultsAvailable): products = self.get_object_list() self._sort_display_products(products) ConsoleApplication.bcall_errors_handler(self, errors, debugmsg, ignore) def _get_price(self, price): return price.cost def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, _id): """ info ID Get information about a product. """ if not _id: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 price = self.get_object(_id, 'get_price') if not price: print('Price not found: %s' % _id, file=self.stderr) return 3 self.start_format() self.format(price) weboob-1.2/weboob/applications/cookboob/000077500000000000000000000000001303450110500203615ustar00rootroot00000000000000weboob-1.2/weboob/applications/cookboob/__init__.py000066400000000000000000000014231303450110500224720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .cookboob import Cookboob __all__ = ['Cookboob'] weboob-1.2/weboob/applications/cookboob/cookboob.py000066400000000000000000000124721303450110500225360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import codecs from weboob.capabilities.recipe import CapRecipe from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['Cookboob'] class RecipeInfoFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'title', 'preparation_time', 'ingredients', 'instructions') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.title, self.NC) result += 'ID: %s\n' % obj.fullid if not empty(obj.author): result += 'Author: %s\n' % obj.author if not empty(obj.preparation_time): result += 'Preparation time: %smin\n' % obj.preparation_time if not empty(obj.cooking_time): result += 'Cooking time: %smin\n' % obj.cooking_time if not empty(obj.nb_person): nbstr = '-'.join(str(num) for num in obj.nb_person) result += 'Amount of people: %s\n' % nbstr result += '\n%sIngredients%s\n' % (self.BOLD, self.NC) for i in obj.ingredients: result += ' * %s\n' % i result += '\n%sInstructions%s\n' % (self.BOLD, self.NC) result += '%s\n' % obj.instructions if not empty(obj.comments): result += '\n%sComments%s\n' % (self.BOLD, self.NC) for c in obj.comments: result += ' * %s\n' % c return result class RecipeListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'short_description', 'preparation_time') def get_title(self, obj): return obj.title def get_description(self, obj): result = u'' if not empty(obj.preparation_time): result += 'prep time: %smin' % obj.preparation_time if not empty(obj.short_description): result += 'description: %s\n' % obj.short_description return result.strip() class Cookboob(ReplApplication): APPNAME = 'cookboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Julien Veyssier' DESCRIPTION = "Console application allowing to search for recipes on various websites." SHORT_DESCRIPTION = "search and consult recipes" CAPS = CapRecipe EXTRA_FORMATTERS = {'recipe_list': RecipeListFormatter, 'recipe_info': RecipeInfoFormatter } COMMANDS_FORMATTERS = {'search': 'recipe_list', 'info': 'recipe_info' } def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, id): """ info ID Get information about a recipe. """ recipe = self.get_object(id, 'get_recipe') if not recipe: print('Recipe not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(recipe) def complete_export(self, text, line, *ignored): args = line.split(' ', 2) if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_export(self, line): """ export ID [FILENAME] Export the recipe to a KRecipes XML file FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. """ id, dest = self.parse_command_args(line, 2, 1) _id, backend_name = self.parse_id(id) if dest is None: dest = '%s.kreml' % _id recipe = self.get_object(id, 'get_recipe') if recipe: xmlstring = recipe.toKrecipesXml(backend_name or None) if dest == '-': print(xmlstring) else: if not dest.endswith('.kreml'): dest += '.kreml' try: with codecs.open(dest, 'w', 'utf-8') as f: f.write(xmlstring) except IOError as e: print('Unable to write .kreml in "%s": %s' % (dest, e), file=self.stderr) return 1 return print('Recipe "%s" not found' % id, file=self.stderr) return 3 @defaultcount(10) def do_search(self, pattern): """ search [PATTERN] Search recipes. """ self.change_path([u'search']) self.start_format(pattern=pattern) for recipe in self.do('iter_recipes', pattern=pattern): self.cached_format(recipe) weboob-1.2/weboob/applications/flatboob/000077500000000000000000000000001303450110500203545ustar00rootroot00000000000000weboob-1.2/weboob/applications/flatboob/__init__.py000066400000000000000000000000671303450110500224700ustar00rootroot00000000000000from .flatboob import Flatboob __all__ = ['Flatboob'] weboob-1.2/weboob/applications/flatboob/flatboob.py000066400000000000000000000225201303450110500225170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.housing import CapHousing, Query from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.tools.config.yamlconfig import YamlConfig __all__ = ['Flatboob'] class HousingFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'title', 'cost', 'currency', 'area', 'date', 'text') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.title, self.NC) result += 'ID: %s\n' % obj.fullid if hasattr(obj, 'url') and obj.url: result += 'URL: %s\n' % obj.url result += 'Cost: %s%s\n' % (obj.cost, obj.currency) area = u'%.2fm²' % (obj.area) if obj.area else u'%s' % obj.area result += u'Area: %s\n' % area if hasattr(obj, 'price_per_meter') and not empty(obj.price_per_meter): result += u'Price per square meter: %.2f %s/m²\n' % (obj.price_per_meter, obj.currency) if obj.date: result += 'Date: %s\n' % obj.date.strftime('%Y-%m-%d') result += 'Phone: %s\n' % obj.phone if hasattr(obj, 'location') and obj.location: result += 'Location: %s\n' % obj.location if hasattr(obj, 'station') and obj.station: result += 'Station: %s\n' % obj.station if hasattr(obj, 'photos') and obj.photos: result += '\n%sPhotos%s\n' % (self.BOLD, self.NC) for photo in obj.photos: result += ' * %s\n' % photo.url result += '\n%sDescription%s\n' % (self.BOLD, self.NC) result += obj.text if hasattr(obj, 'details') and obj.details: result += '\n\n%sDetails%s\n' % (self.BOLD, self.NC) for key, value in obj.details.iteritems(): result += ' %s: %s\n' % (key, value) return result class HousingListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'cost', 'text') def get_title(self, obj): return '%s%s - %s' % (obj.cost, obj.currency, obj.title) def get_description(self, obj): result = u'' if hasattr(obj, 'date') and obj.date: result += '%s - ' % obj.date.strftime('%Y-%m-%d') result += obj.text return result class Flatboob(ReplApplication): APPNAME = 'flatboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Romain Bignon' DESCRIPTION = "Console application to search for housing." SHORT_DESCRIPTION = "search for housing" CAPS = CapHousing CONFIG = {'queries': {}} EXTRA_FORMATTERS = {'housing_list': HousingListFormatter, 'housing': HousingFormatter, } COMMANDS_FORMATTERS = {'search': 'housing_list', 'info': 'housing', 'load': 'housing_list' } def main(self, argv): self.load_config(klass=YamlConfig) return ReplApplication.main(self, argv) @defaultcount(10) def do_search(self, line): """ search Search for housing. Parameters are interactively asked. """ pattern = 'notempty' query = Query() query.cities = [] while pattern: if len(query.cities) > 0: print('\n%sSelected cities:%s %s' % (self.BOLD, self.NC, ', '.join([c.name for c in query.cities]))) pattern = self.ask('Enter a city pattern (or empty to stop)', default='') if not pattern: break cities = [] for city in self.weboob.do('search_city', pattern): cities.append(city) if len(cities) == 0: print(' Not found!') continue if len(cities) == 1: if city in query.cities: query.cities.remove(city) else: query.cities.append(city) continue r = 'notempty' while r != '': for i, city in enumerate(cities): print(' %s%2d)%s [%s] %s (%s)' % (self.BOLD, i+1, self.NC, 'x' if city in query.cities else ' ', city.name, city.backend)) r = self.ask(' Select cities (or empty to stop)', regexp='(\d+|)', default='') if not r.isdigit(): continue r = int(r) if r <= 0 or r > len(cities): continue city = cities[r-1] if city in query.cities: query.cities.remove(city) else: query.cities.append(city) r = 'notempty' while r != '': for good in Query.HOUSE_TYPES.values: print(' %s%2d)%s [%s] %s' % (self.BOLD, Query.HOUSE_TYPES.index[good] + 1, self.NC, 'x' if good in query.house_types else ' ', good)) r = self.ask(' Select type of house (or empty to stop)', regexp='(\d+|)', default='') if not r.isdigit(): continue r = int(r) if r <= 0 or r > len(Query.HOUSE_TYPES.values): continue value = Query.HOUSE_TYPES.values[r - 1] if value in query.house_types: query.house_types.remove(value) else: query.house_types.append(value) _type = None while _type not in [query.TYPE_RENT, query.TYPE_SALE, query.TYPE_SHARING]: print(' %s%2d)%s %s' % (self.BOLD, query.TYPE_RENT, self.NC, "Rent")) print(' %s%2d)%s %s' % (self.BOLD, query.TYPE_SALE, self.NC, "Sale")) print(' %s%2d)%s %s' % (self.BOLD, query.TYPE_SHARING, self.NC, "Sharing")) _type = self.ask_int('Type of query') query.type = _type query.area_min = self.ask_int('Enter min area') query.area_max = self.ask_int('Enter max area') query.cost_min = self.ask_int('Enter min cost') query.cost_max = self.ask_int('Enter max cost') query.nb_rooms = self.ask_int('Enter number of rooms') save_query = self.ask('Save query (y/n)', default='n') if save_query.upper() == 'Y': name = '' while not name: name = self.ask('Query name') self.config.set('queries', name, query) self.config.save() self.complete_search(query) def complete_search(self, query): self.change_path([u'housings']) self.start_format() for housing in self.do('search_housings', query): self.cached_format(housing) def ask_int(self, txt): r = self.ask(txt, default='', regexp='(\d+|)') if r: return int(r) return None @defaultcount(10) def do_load(self, query_name): """ load [query name] without query name : list loadable queries with query name laod query """ queries = self.config.get('queries') if not queries: print('There is no saved queries', file=self.stderr) return 2 if not query_name: for name in queries.keys(): print(' %s* %s %s' % (self.BOLD, self.NC, name)) query_name = self.ask('Which one') if query_name in queries: self.complete_search(queries.get(query_name)) else: print('Unknown query', file=self.stderr) return 2 def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, _id): """ info ID Get information about a housing. """ if not _id: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 housing = self.get_object(_id, 'get_housing') if not housing: print('Housing not found: %s' % _id, file=self.stderr) return 3 self.start_format() self.format(housing) weboob-1.2/weboob/applications/galleroob/000077500000000000000000000000001303450110500205325ustar00rootroot00000000000000weboob-1.2/weboob/applications/galleroob/__init__.py000066400000000000000000000014341303450110500226450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .galleroob import Galleroob __all__ = ['Galleroob'] weboob-1.2/weboob/applications/galleroob/galleroob.py000066400000000000000000000112711303450110500230540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os from re import search, sub from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.capabilities.base import empty from weboob.capabilities.gallery import CapGallery, BaseGallery, BaseImage from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['Galleroob'] class GalleryListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title') def get_title(self, obj): s = obj.title if hasattr(obj, 'cardinality') and not empty(obj.cardinality): s += u' (%d pages)' % obj.cardinality return s def get_description(self, obj): if hasattr(obj, 'description') and obj.description: return obj.description class Galleroob(ReplApplication): APPNAME = 'galleroob' VERSION = '1.2' COPYRIGHT = u'Copyright(C) 2011-2014 Noé Rubinstein' DESCRIPTION = 'galleroob browses and downloads web image galleries' SHORT_DESCRIPTION = 'browse and download web image galleries' CAPS = CapGallery EXTRA_FORMATTERS = {'gallery_list': GalleryListFormatter} COMMANDS_FORMATTERS = {'search': 'gallery_list', 'ls': 'gallery_list'} COLLECTION_OBJECTS = (BaseGallery, BaseImage, ) def __init__(self, *args, **kwargs): ReplApplication.__init__(self, *args, **kwargs) @defaultcount(10) def do_search(self, pattern): """ search PATTERN List galleries matching a PATTERN. """ if not pattern: print('This command takes an argument: %s' % self.get_command_help('search', short=True), file=self.stderr) return 2 self.start_format(pattern=pattern) for gallery in self.do('search_galleries', pattern=pattern): self.cached_format(gallery) def do_download(self, line): """ download ID [FIRST [FOLDER]] Download a gallery. Begins at page FIRST (default: 0) and saves to FOLDER (default: title) """ _id, first, dest = self.parse_command_args(line, 3, 1) if first is None: first = 0 else: first = int(first) gallery = None _id, backend = self.parse_id(_id) for result in self.do('get_gallery', _id, backends=backend): if result: gallery = result if not gallery: print('Gallery not found: %s' % _id, file=self.stderr) return 3 self.weboob[backend].fillobj(gallery, ('title',)) if dest is None: dest = sub('/', ' ', gallery.title) print("Downloading to %s" % dest) try: os.mkdir(dest) except OSError: pass # ignore error on existing directory os.chdir(dest) # fail here if dest couldn't be created i = 0 for img in self.weboob[backend].iter_gallery_images(gallery): i += 1 if i < first: continue self.weboob[backend].fillobj(img, ('url', 'data')) if img.data is None: self.weboob[backend].fillobj(img, ('url', 'data')) if img.data is None: print("Couldn't get page %d, exiting" % i, file=self.stderr) break ext = search(r"\.([^\.]{1,5})$", img.url) if ext: ext = ext.group(1) else: ext = "jpg" name = '%03d.%s' % (i, ext) print('Writing file %s' % name) with open(name, 'wb') as f: f.write(img.data) os.chdir(os.path.pardir) def do_info(self, line): """ info ID Get information about a gallery. """ _id, = self.parse_command_args(line, 1, 1) gallery = self.get_object(_id, 'get_gallery') if not gallery: print('Gallery not found: %s' % _id, file=self.stderr) return 3 self.start_format() self.format(gallery) weboob-1.2/weboob/applications/geolooc/000077500000000000000000000000001303450110500202135ustar00rootroot00000000000000weboob-1.2/weboob/applications/geolooc/__init__.py000066400000000000000000000014241303450110500223250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .geolooc import Geolooc __all__ = ['Geolooc'] weboob-1.2/weboob/applications/geolooc/geolooc.py000066400000000000000000000031301303450110500222110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.geolocip import CapGeolocIp from weboob.tools.application.repl import ReplApplication __all__ = ['Geolooc'] class Geolooc(ReplApplication): APPNAME = 'geolooc' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to geolocalize IP addresses." SHORT_DESCRIPTION = "geolocalize IP addresses" CAPS = CapGeolocIp def main(self, argv): if len(argv) < 2: print('Syntax: %s ipaddr' % argv[0], file=self.stderr) return 2 for location in self.do('get_location', argv[1]): if location.lt and location.lg: location.osmlink = u'http://www.openstreetmap.org/?mlat=%s&mlon=%s#map=13/%s/%s' % (location.lt, location.lg, location.lt, location.lg) self.format(location) weboob-1.2/weboob/applications/handjoob/000077500000000000000000000000001303450110500203505ustar00rootroot00000000000000weboob-1.2/weboob/applications/handjoob/__init__.py000066400000000000000000000014151303450110500224620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .handjoob import Handjoob __all__ = ['Handjoob'] weboob-1.2/weboob/applications/handjoob/handjoob.py000066400000000000000000000115461303450110500225150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.job import CapJob from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['Handjoob'] class JobAdvertFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'url', 'publication_date', 'title') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.title, self.NC) result += 'url: %s\n' % obj.url if hasattr(obj, 'publication_date') and obj.publication_date: result += 'Publication date : %s\n' % obj.publication_date.strftime('%Y-%m-%d') if hasattr(obj, 'place') and obj.place: result += 'Location: %s\n' % obj.place if hasattr(obj, 'society_name') and obj.society_name: result += 'Society : %s\n' % obj.society_name if hasattr(obj, 'job_name') and obj.job_name: result += 'Job name : %s\n' % obj.job_name if hasattr(obj, 'contract_type') and obj.contract_type: result += 'Contract : %s\n' % obj.contract_type if hasattr(obj, 'pay') and obj.pay: result += 'Pay : %s\n' % obj.pay if hasattr(obj, 'formation') and obj.formation: result += 'Formation : %s\n' % obj.formation if hasattr(obj, 'experience') and obj.experience: result += 'Experience : %s\n' % obj.experience if hasattr(obj, 'description') and obj.description: result += 'Description : %s\n' % obj.description return result class JobAdvertListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title') def get_title(self, obj): return '%s' % (obj.title) def get_description(self, obj): result = u'' if hasattr(obj, 'publication_date') and obj.publication_date: result += '\tPublication date : %s\n' % obj.publication_date.strftime('%Y-%m-%d') if hasattr(obj, 'place') and obj.place: result += '\tLocation: %s\n' % obj.place if hasattr(obj, 'society_name') and obj.society_name: result += '\tSociety : %s\n' % obj.society_name if hasattr(obj, 'contract_type') and obj.contract_type: result += '\tContract : %s\n' % obj.contract_type return result.strip('\n\t') class Handjoob(ReplApplication): APPNAME = 'handjoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Bezleputh' DESCRIPTION = "Console application to search for a job." SHORT_DESCRIPTION = "search for a job" CAPS = CapJob EXTRA_FORMATTERS = {'job_advert_list': JobAdvertListFormatter, 'job_advert': JobAdvertFormatter, } COMMANDS_FORMATTERS = {'search': 'job_advert_list', 'ls': 'job_advert_list', 'info': 'job_advert', } @defaultcount(10) def do_search(self, pattern): """ search PATTERN Search for an advert matching a PATTERN. """ self.change_path([u'search']) self.start_format(pattern=pattern) for job_advert in self.do('search_job', pattern): self.cached_format(job_advert) @defaultcount(10) def do_ls(self, line): """ advanced search Search for an advert matching to advanced filters. """ self.change_path([u'advanced']) for job_advert in self.do('advanced_search_job'): self.cached_format(job_advert) def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, _id): """ info ID Get information about an advert. """ if not _id: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 job_advert = self.get_object(_id, 'get_job_advert') if not job_advert: print('Job advert not found: %s' % _id, file=self.stderr) return 3 self.start_format() self.format(job_advert) weboob-1.2/weboob/applications/havedate/000077500000000000000000000000001303450110500203455ustar00rootroot00000000000000weboob-1.2/weboob/applications/havedate/__init__.py000066400000000000000000000014271303450110500224620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .havedate import HaveDate __all__ = ['HaveDate'] weboob-1.2/weboob/applications/havedate/havedate.py000066400000000000000000000242021303450110500225000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from copy import copy from weboob.core import CallErrors from weboob.tools.application.repl import ReplApplication from weboob.applications.boobmsg import Boobmsg from weboob.capabilities.dating import CapDating, OptimizationNotFound from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['HaveDate'] class EventListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('date', 'type') def get_title(self, event): s = u'(%s) %s' % (event.date, event.type) if hasattr(event, 'contact') and event.contact: s += u' — %s (%s)' % (event.contact.name, event.contact.id) return s def get_description(self, event): if hasattr(event, 'message'): return event.message class HaveDate(Boobmsg): APPNAME = 'havedate' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to interact with various dating websites " \ "and to optimize seduction algorithmically." SHORT_DESCRIPTION = "interact with dating websites" STORAGE_FILENAME = 'dating.storage' STORAGE = {'optims': {}} CAPS = CapDating EXTRA_FORMATTERS = copy(Boobmsg.EXTRA_FORMATTERS) EXTRA_FORMATTERS['events'] = EventListFormatter COMMANDS_FORMATTERS = copy(Boobmsg.COMMANDS_FORMATTERS) COMMANDS_FORMATTERS['optim'] = 'table' COMMANDS_FORMATTERS['events'] = 'events' def load_default_backends(self): self.load_backends(CapDating, storage=self.create_storage(self.STORAGE_FILENAME)) def main(self, argv): self.load_config() try: self.do('init_optimizations').wait() except CallErrors as e: self.bcall_errors_handler(e) optimizations = self.storage.get('optims') for optim, backends in optimizations.iteritems(): self.optims('start', backends, optim, store=False) return ReplApplication.main(self, argv) def do_query(self, id): """ query ID Send a query to someone. """ _id, backend_name = self.parse_id(id, unique_backend=True) for query in self.do('send_query', _id, backends=backend_name): print('%s' % query.message) def edit_optims(self, backend_names, optims_names, stop=False): if optims_names is None: print('Error: missing parameters.', file=self.stderr) return 2 for optim_name in optims_names.split(): backends_optims = {} for optim in self.do('get_optimization', optim_name, backends=backend_names): if optim: backends_optims[optim.backend] = optim for backend_name, optim in backends_optims.iteritems(): if len(optim.CONFIG) == 0: print('%s.%s does not require configuration.' % (backend_name, optim_name)) continue was_running = optim.is_running() if stop and was_running: print('Stopping %s: %s' % (optim_name, backend_name)) optim.stop() params = optim.get_config() if params is None: params = {} print('Configuration of %s.%s' % (backend_name, optim_name)) print('-----------------%s-%s' % ('-' * len(backend_name), '-' * len(optim_name))) for key, value in optim.CONFIG.iteritems(): params[key] = self.ask(value, default=params[key] if (key in params) else value.default) optim.set_config(params) if stop and was_running: print('Starting %s: %s' % (optim_name, backend_name)) optim.start() def optims(self, function, backend_names, optims, store=True): if optims is None: print('Error: missing parameters.', file=self.stderr) return 2 for optim_name in optims.split(): try: if store: storage_optim = set(self.storage.get('optims', optim_name, default=[])) self.stdout.write('%sing %s:' % (function.capitalize(), optim_name)) for optim in self.do('get_optimization', optim_name, backends=backend_names): if optim: # It's useless to start a started optim, or to stop a stopped one. if (function == 'start' and optim.is_running()) or \ (function == 'stop' and not optim.is_running()): continue # Optim is not configured and would be, ask user to do it. if function == 'start' and len(optim.CONFIG) > 0 and optim.get_config() is None: self.edit_optims(optim.backend, optim_name) ret = getattr(optim, function)() self.stdout.write(' ' + optim.backend) if not ret: self.stdout.write('(failed)') self.stdout.flush() if store: if function == 'start' and ret: storage_optim.add(optim.backend) elif function == 'stop': try: storage_optim.remove(optim.backend) except KeyError: pass self.stdout.write('.\n') except CallErrors as errors: for backend, error, backtrace in errors: if isinstance(error, OptimizationNotFound): self.logger.error(u'Error(%s): Optimization "%s" not found' % (backend.name, optim_name)) else: self.bcall_error_handler(backend, error, backtrace) if store: if len(storage_optim) > 0: self.storage.set('optims', optim_name, list(storage_optim)) else: self.storage.delete('optims', optim_name) if store: self.storage.save() def complete_optim(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return ['list', 'start', 'stop', 'edit'] elif len(args) == 3: return [backend.name for backend in self.enabled_backends] elif len(args) >= 4: if args[2] == '*': backend = None else: backend = args[2] optims = set() for optim in self.do('iter_optimizations', backends=backend): optims.add(optim.id) return sorted(optims - set(args[3:])) def do_optim(self, line): """ optim [list | start | edit | stop] BACKEND [OPTIM1 [OPTIM2 ...]] All dating backends offer optimization services. This command can be manage them. Use * us BACKEND value to apply command to all backends. Commands: * list list all available optimizations of a backend * start start optimization services on a backend * edit configure an optimization service for a backend * stop stop optimization services on a backend """ cmd, backend_name, optims_names = self.parse_command_args(line, 3) if backend_name == '*': backend_name = None elif backend_name is not None and backend_name not in [b.name for b in self.enabled_backends]: print('Error: No such backend "%s"' % backend_name, file=self.stderr) return 1 if cmd == 'start': return self.optims('start', backend_name, optims_names) if cmd == 'stop': return self.optims('stop', backend_name, optims_names) if cmd == 'edit': self.edit_optims(backend_name, optims_names, stop=True) return if cmd == 'list' or cmd is None: if optims_names is not None: optims_names = optims_names.split() optims = {} backends = set() for optim in self.do('iter_optimizations', backends=backend_name): if optims_names is not None and optim.id not in optims_names: continue if optim.is_running(): status = 'RUNNING' else: status = '-------' if optim.id not in optims: optims[optim.id] = {optim.backend: status} else: optims[optim.id][optim.backend] = status backends.add(optim.backend) backends = sorted(backends) for name, backends_status in optims.iteritems(): line = [('name', name)] for b in backends: try: status = backends_status[b] except KeyError: status = '' line.append((b, status)) self.format(tuple(line)) return print("No such command '%s'" % cmd, file=self.stderr) return 1 def do_events(self, line): """ events Display dating events. """ self.change_path([u'events']) self.start_format() for event in self.do('iter_events'): self.cached_format(event) weboob-1.2/weboob/applications/masstransit/000077500000000000000000000000001303450110500211345ustar00rootroot00000000000000weboob-1.2/weboob/applications/masstransit/__init__.py000066400000000000000000000014401303450110500232440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .masstransit import Masstransit __all__ = ['Masstransit'] weboob-1.2/weboob/applications/masstransit/masstransit.py000066400000000000000000000243441303450110500240650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Julien Hébert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.travel import CapTravel from weboob.tools.application.base import Application from logging import warning import gtk class FakeConic(object): STATUS_CONNECTED = None STATUS_DISCONNECTED = None CONNECT_FLAG_NONE = None def Connection(self): raise NotImplementedError() try: import hildon except ImportError: toolkit = gtk else: toolkit = hildon try: import conic except ImportError: warning("conic is not found") conic = FakeConic() from logging import debug __all__ = ['Masstransit'] class MasstransitHildon(): "hildon interface" def connect_event(self, connection, event=None, c=None, d=None): debug("DBUS-DEBUG a: %s, b:%s, c:%s,d: %s" % (connection, event, c, d)) status = event.get_status() if status == conic.STATUS_CONNECTED: self.connected = True if not self.touch_selector_entry_filled: debug("connected, now fill") self.fill_touch_selector_entry() if self.refresh_in_progress: self.refresh() elif status == conic.STATUS_DISCONNECTED: self.connected = False def __init__(self, weboob): self.touch_selector_entry_filled = False self.refresh_in_progress = False self.connected = False self.weboob = weboob try: self.connection = conic.Connection() self.connection.connect("connection-event", self.connect_event) self.connection.set_property("automatic-connection-events", True) self.connection.request_connection(conic.CONNECT_FLAG_NONE) except NotImplementedError: pass horizontal_box = gtk.HBox() self.main_window = toolkit.Window() try: self.refresh_button = toolkit.Button( gtk.HILDON_SIZE_AUTO_WIDTH | gtk.HILDON_SIZE_FINGER_HEIGHT, hildon.BUTTON_ARRANGEMENT_HORIZONTAL, "Actualiser" ) self.retour_button = hildon.Button( gtk.HILDON_SIZE_AUTO_WIDTH | gtk.HILDON_SIZE_FINGER_HEIGHT, hildon.BUTTON_ARRANGEMENT_HORIZONTAL, "Retour" ) self.combo_source = hildon.TouchSelectorEntry(text=True) self.combo_dest = hildon.TouchSelectorEntry(text=True) self.picker_button_source = hildon.PickerButton( gtk.HILDON_SIZE_AUTO, hildon.BUTTON_ARRANGEMENT_VERTICAL) self.picker_button_dest = hildon.PickerButton( gtk.HILDON_SIZE_AUTO, hildon.BUTTON_ARRANGEMENT_VERTICAL ) self.picker_button_source.set_sensitive(False) self.picker_button_dest.set_sensitive(False) self.picker_button_source.set_title("Gare de Depart") self.picker_button_dest.set_title("Gare d'arrivee") self.picker_button_source.set_selector(self.combo_source) self.picker_button_dest.set_selector(self.combo_dest) horizontal_box.pack_start(self.picker_button_source) horizontal_box.pack_start(self.picker_button_dest) except AttributeError: self.refresh_button = gtk.Button("Actualiser") self.retour_button = gtk.Button("Retour") self.combo_source = gtk.combo_box_entry_new_text() self.combo_dest = gtk.combo_box_entry_new_text() horizontal_box.pack_start(self.combo_source) horizontal_box.pack_start(self.combo_dest) self.main_window.set_title("Horaires des Prochains Trains") self.main_window.connect("destroy", self.on_main_window_destroy) self.refresh_button.connect("clicked", self.on_refresh_button_clicked) self.retour_button.set_sensitive(False) self.retour_button.connect("clicked", self.on_retour_button_clicked) self.treestore = gtk.TreeStore(str, str, str, str, str) treeview = gtk.TreeView(self.treestore) treeview.append_column( gtk.TreeViewColumn( 'Train', gtk.CellRendererText(), text=0 )) treeview.append_column( gtk.TreeViewColumn( 'Horaire', gtk.CellRendererText(), text=1 )) treeview.append_column( gtk.TreeViewColumn( 'Destination', gtk.CellRendererText(), text=2 )) treeview.append_column( gtk.TreeViewColumn( 'Voie', gtk.CellRendererText(), text=3 )) treeview.append_column( gtk.TreeViewColumn( 'Information', gtk.CellRendererText(), text=4 )) vertical_box = gtk.VBox() vertical_box.pack_start(horizontal_box) horizontal_box.pack_start(self.retour_button) vertical_box.pack_start(treeview) vertical_box.pack_start(self.refresh_button) self.main_window.add(vertical_box) self.main_window.show_all() self.fill_touch_selector_entry() if toolkit != gtk: self.picker_button_source.connect("value-changed", self.check_station_input, self.picker_button_source) self.picker_button_dest.connect("value-changed", self.check_station_input, self.picker_button_dest) def fill_touch_selector_entry(self): liste = [] for backend in self.weboob.iter_backends(): for station in backend.iter_station_search(""): liste.append(station.name.capitalize()) liste.sort() for station in liste: self.combo_source.append_text(station) self.combo_dest.append_text(station) self.touch_selector_entry_filled = True if toolkit != gtk: self.picker_button_source.set_sensitive(True) def on_main_window_destroy(self, widget): "exit application at the window close" gtk.main_quit() def on_main_window_show(self, param): self.fill_touch_selector_entry() def on_retour_button_clicked(self, widget): "the button is clicked" debug("on_retour_button_clicked") self.refresh_in_progress = True col_source = self.combo_source.get_active(0) col_dest = self.combo_dest.get_active(0) self.combo_source.set_active(0, col_dest) self.combo_dest.set_active(0, col_source) self.refresh() def on_refresh_button_clicked(self, widget): "the refresh button is clicked" debug("on_refresh_button_clicked") self.refresh_in_progress = True try: self.connection.request_connection(conic.CONNECT_FLAG_NONE) except AttributeError: if isinstance(conic, FakeConic): self.refresh() else: raise def check_station_input(self, widget, user_data): if self.combo_source.get_current_text() is None: self.picker_button_dest.set_sensitive(False) self.refresh_button.set_sensitive(False) self.retour_button.set_sensitive(False) else: self.picker_button_dest.set_sensitive(True) if self.combo_dest.get_current_text() is None: self.refresh_button.set_sensitive(False) self.retour_button.set_sensitive(False) else: self.refresh_button.set_sensitive(True) self.retour_button.set_sensitive(True) def refresh(self): "update departures" banner = hildon.hildon_banner_show_information(self.main_window, "", "Chargement en cours") banner.set_timeout(10000) hildon.hildon_gtk_window_set_progress_indicator(self.main_window, 1) self.treestore.clear() try: source_text = self.combo_source.get_current_text() dest_text = self.combo_dest.get_current_text() except AttributeError: source_text = self.combo_source.child.get_text() dest_text = self.combo_dest.child.get_text() for backend in self.weboob.iter_backends(): for station in backend.iter_station_search(source_text): for arrival in \ backend.iter_station_search(dest_text): for departure in \ backend.iter_station_departures(station.id, arrival.id): self.treestore.append(None, [departure.type, departure.time, departure.arrival_station, departure.plateform, departure.information]) self.refresh_in_progress = False banner.set_timeout(1) hildon.hildon_gtk_window_set_progress_indicator(self.main_window, 0) class Masstransit(Application): "Application Class" APPNAME = 'masstransit' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Julien Hébert' DESCRIPTION = "Maemo application allowing to search for train stations and get departure times." SHORT_DESCRIPTION = "search for train stations and departures" def main(self, argv): self.load_backends(CapTravel) MasstransitHildon(self.weboob) gtk.main() weboob-1.2/weboob/applications/monboob/000077500000000000000000000000001303450110500202175ustar00rootroot00000000000000weboob-1.2/weboob/applications/monboob/__init__.py000066400000000000000000000014241303450110500223310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .monboob import Monboob __all__ = ['Monboob'] weboob-1.2/weboob/applications/monboob/monboob.py000066400000000000000000000325351303450110500222340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2011 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from email.mime.text import MIMEText from smtplib import SMTP from email.Header import Header, decode_header from email.Utils import parseaddr, formataddr, formatdate from email import message_from_file, message_from_string from smtpd import SMTPServer import time import re import logging import asyncore import subprocess import socket from weboob.core import Weboob, CallErrors from weboob.core.scheduler import Scheduler from weboob.capabilities.messages import CapMessages, CapMessagesPost, Thread, Message from weboob.tools.application.repl import ReplApplication from weboob.tools.date import utc2local from weboob.tools.html import html2text from weboob.tools.misc import get_backtrace, to_unicode __all__ = ['Monboob'] class FakeSMTPD(SMTPServer): def __init__(self, app, bindaddr, port): SMTPServer.__init__(self, (bindaddr, port), None) self.app = app def process_message(self, peer, mailfrom, rcpttos, data): msg = message_from_string(data) self.app.process_incoming_mail(msg) class MonboobScheduler(Scheduler): def __init__(self, app): Scheduler.__init__(self) self.app = app def run(self): if self.app.options.smtpd: if ':' in self.app.options.smtpd: host, port = self.app.options.smtpd.split(':', 1) else: host = '127.0.0.1' port = self.app.options.smtpd try: FakeSMTPD(self.app, host, int(port)) except socket.error as e: self.logger.error('Unable to start the SMTP daemon: %s' % e) return False # XXX Fuck, we shouldn't copy this piece of code from # weboob.scheduler.Scheduler.run(). try: while True: self.stop_event.wait(0.1) if self.app.options.smtpd: asyncore.loop(timeout=0.1, count=1) except KeyboardInterrupt: self._wait_to_stop() raise else: self._wait_to_stop() return True class Monboob(ReplApplication): APPNAME = 'monboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = 'Daemon allowing to regularly check for new messages on various websites, ' \ 'and send an email for each message, and post a reply to a message on a website.' SHORT_DESCRIPTION = "daemon to send and check messages" CONFIG = {'interval': 300, 'domain': 'weboob.example.org', 'recipient': 'weboob@example.org', 'smtp': 'localhost', 'pipe': '', 'html': 0} CAPS = CapMessages DISABLE_REPL = True def add_application_options(self, group): group.add_option('-S', '--smtpd', help='run a fake smtpd server and set the port') def create_weboob(self): return Weboob(scheduler=MonboobScheduler(self)) def load_default_backends(self): self.load_backends(CapMessages, storage=self.create_storage()) def main(self, argv): self.load_config() try: self.config.set('interval', int(self.config.get('interval'))) if self.config.get('interval') < 1: raise ValueError() except ValueError: print('Configuration error: interval must be an integer >0.', file=self.stderr) return 1 try: self.config.set('html', int(self.config.get('html'))) if self.config.get('html') not in (0, 1): raise ValueError() except ValueError: print('Configuration error: html must be 0 or 1.', file=self.stderr) return 2 return ReplApplication.main(self, argv) def get_email_address_ident(self, msg, header): s = msg.get(header) if not s: return None m = re.match('.*<([^@]*)@(.*)>', s) if m: return m.group(1) else: try: return s.split('@')[0] except IndexError: return s def do_post(self, line): """ post Pipe with a mail to post message. """ msg = message_from_file(self.stdin) return self.process_incoming_mail(msg) def process_incoming_mail(self, msg): to = self.get_email_address_ident(msg, 'To') sender = msg.get('From') reply_to = self.get_email_address_ident(msg, 'In-Reply-To') title = msg.get('Subject') if title: new_title = u'' for part in decode_header(title): if part[1]: new_title += unicode(part[0], part[1]) else: new_title += unicode(part[0]) title = new_title content = u'' for part in msg.walk(): if part.get_content_type() == 'text/plain': s = part.get_payload(decode=True) charsets = part.get_charsets() + msg.get_charsets() for charset in charsets: try: if charset is not None: content += unicode(s, charset) else: content += unicode(s) except UnicodeError as e: self.logger.warning('Unicode error: %s' % e) continue except Exception as e: self.logger.exception(e) continue else: break if len(content) == 0: print('Unable to send an empty message', file=self.stderr) return 1 # remove signature content = content.split(u'\n-- \n')[0] parent_id = None if reply_to is None: # This is a new message if '.' in to: backend_name, thread_id = to.split('.', 1) else: backend_name = to thread_id = None else: # This is a reply try: backend_name, id = reply_to.split('.', 1) thread_id, parent_id = id.rsplit('.', 1) except ValueError: print('In-Reply-To header might be in form ', file=self.stderr) return 1 # Default use the To header field to know the backend to use. if to and backend_name != to: backend_name = to try: backend = self.weboob.backend_instances[backend_name] except KeyError: print('Backend %s not found' % backend_name, file=self.stderr) return 1 if not backend.has_caps(CapMessagesPost): print('The backend %s does not implement CapMessagesPost' % backend_name, file=self.stderr) return 1 thread = Thread(thread_id) message = Message(thread, 0, title=title, sender=sender, receivers=[to], parent=Message(thread, parent_id) if parent_id else None, content=content) try: backend.post_message(message) except Exception as e: content = u'Unable to send message to %s:\n' % thread_id content += u'\n\t%s\n' % to_unicode(e) if logging.root.level <= logging.DEBUG: content += u'\n%s\n' % to_unicode(get_backtrace(e)) self.send_email(backend.name, Message(thread, 0, title='Unable to send message', sender='Monboob', parent=Message(thread, parent_id) if parent_id else None, content=content)) def do_run(self, line): """ run Run the fetching daemon. """ self.weboob.repeat(self.config.get('interval'), self.process) self.weboob.loop() def do_once(self, line): """ once Send mails only once, then exit. """ return self.process() def process(self): try: for message in self.weboob.do('iter_unread_messages'): if self.send_email(message.backend, message): self.weboob[message.backend].set_message_read(message) except CallErrors as e: self.bcall_errors_handler(e) def send_email(self, backend_name, mail): domain = self.config.get('domain') recipient = self.config.get('recipient') parent_message = mail.parent references = [] while parent_message: references.append(u'<%s.%s@%s>' % (backend_name, mail.parent.full_id, domain)) parent_message = parent_message.parent subject = mail.title sender = u'"%s" <%s@%s>' % (mail.sender.replace('"', '""') if mail.sender else '', backend_name, domain) # assume that .date is an UTC datetime date = formatdate(time.mktime(utc2local(mail.date).timetuple()), localtime=True) msg_id = u'<%s.%s@%s>' % (backend_name, mail.full_id, domain) if self.config.get('html') and mail.flags & mail.IS_HTML: body = mail.content content_type = 'html' else: if mail.flags & mail.IS_HTML: body = html2text(mail.content) else: body = mail.content content_type = 'plain' if body is None: body = '' if mail.signature: if self.config.get('html') and mail.flags & mail.IS_HTML: body += u'

--
%s

' % mail.signature else: body += u'\n\n-- \n' if mail.flags & mail.IS_HTML: body += html2text(mail.signature) else: body += mail.signature # Header class is smart enough to try US-ASCII, then the charset we # provide, then fall back to UTF-8. header_charset = 'ISO-8859-1' # We must choose the body charset manually for body_charset in 'US-ASCII', 'ISO-8859-1', 'UTF-8': try: body.encode(body_charset) except UnicodeError: pass else: break # Split real name (which is optional) and email address parts sender_name, sender_addr = parseaddr(sender) recipient_name, recipient_addr = parseaddr(recipient) # We must always pass Unicode strings to Header, otherwise it will # use RFC 2047 encoding even on plain ASCII strings. sender_name = str(Header(unicode(sender_name), header_charset)) recipient_name = str(Header(unicode(recipient_name), header_charset)) # Make sure email addresses do not contain non-ASCII characters sender_addr = sender_addr.encode('ascii') recipient_addr = recipient_addr.encode('ascii') # Create the message ('plain' stands for Content-Type: text/plain) msg = MIMEText(body.encode(body_charset), content_type, body_charset) msg['From'] = formataddr((sender_name, sender_addr)) msg['To'] = formataddr((recipient_name, recipient_addr)) msg['Subject'] = Header(unicode(subject), header_charset) msg['Message-Id'] = msg_id msg['Date'] = date if references: msg['In-Reply-To'] = references[0] msg['References'] = u" ".join(reversed(references)) self.logger.info('Send mail from <%s> to <%s>' % (sender, recipient)) if len(self.config.get('pipe')) > 0: p = subprocess.Popen(self.config.get('pipe'), shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) p.stdin.write(msg.as_string()) p.stdin.close() if p.wait() != 0: self.logger.error('Unable to deliver mail: %s' % p.stdout.read().strip()) return False else: # Send the message via SMTP to localhost:25 try: smtp = SMTP(self.config.get('smtp')) smtp.sendmail(sender, recipient, msg.as_string()) except Exception as e: self.logger.error('Unable to deliver mail: %s' % e) return False else: smtp.quit() return True weboob-1.2/weboob/applications/parceloob/000077500000000000000000000000001303450110500205325ustar00rootroot00000000000000weboob-1.2/weboob/applications/parceloob/__init__.py000066400000000000000000000014251303450110500226450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .parceloob import Parceloob __all__ = ['Parceloob'] weboob-1.2/weboob/applications/parceloob/parceloob.py000066400000000000000000000156541303450110500230650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.base import empty from weboob.capabilities.parcel import CapParcel, Parcel, ParcelNotFound from weboob.tools.application.repl import ReplApplication from weboob.tools.application.formatters.iformatter import IFormatter __all__ = ['Parceloob'] STATUS = {Parcel.STATUS_PLANNED: ('PLANNED', 'red'), Parcel.STATUS_IN_TRANSIT: ('IN TRANSIT', 'yellow'), Parcel.STATUS_ARRIVED: ('ARRIVED', 'green'), Parcel.STATUS_UNKNOWN: ('', 'white'), } def get_backend_name(backend): return backend.name class HistoryFormatter(IFormatter): MANDATORY_FIELDS = () def format_obj(self, obj, alias): if isinstance(obj, Parcel): result = u'Parcel %s (%s)\n' % (self.colored(obj.id, 'red', 'bold'), self.colored(obj.backend, 'blue', 'bold')) result += u'%sArrival:%s %s\n' % (self.BOLD, self.NC, obj.arrival) status, status_color = STATUS[obj.status] result += u'%sStatus:%s %s\n' % (self.BOLD, self.NC, self.colored(status, status_color)) result += u'%sInfo:%s %s\n\n' % (self.BOLD, self.NC, obj.info) result += u' Date Location Activity \n' result += u'---------------------+-----------------+---------------------------------------------------' return result return ' %s %s %s' % (self.colored('%-19s' % obj.date, 'blue'), self.colored('%-17s' % (obj.location or ''), 'magenta'), self.colored(obj.activity or '', 'yellow')) class StatusFormatter(IFormatter): MANDATORY_FIELDS = ('id',) def format_obj(self, obj, alias): if alias is not None: id = '%s (%s)' % (self.colored('%3s' % ('#' + alias), 'red', 'bold'), self.colored(obj.backend, 'blue', 'bold')) clean = '#%s (%s)' % (alias, obj.backend) if len(clean) < 15: id += (' ' * (15 - len(clean))) else: id = self.colored('%30s' % obj.fullid, 'red', 'bold') status, status_color = STATUS[obj.status] arrival = obj.arrival.strftime('%Y-%m-%d') if not empty(obj.arrival) else '' result = u'%s %s %s %s %s' % (id, self.colored(u'—', 'cyan'), self.colored('%-10s' % status, status_color), self.colored('%-10s' % arrival, 'blue'), self.colored('%-20s' % obj.info, 'yellow')) return result class Parceloob(ReplApplication): APPNAME = 'parceloob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Romain Bignon' CAPS = CapParcel DESCRIPTION = "Console application to track your parcels." SHORT_DESCRIPTION = "manage your parcels" EXTRA_FORMATTERS = {'status': StatusFormatter, 'history': HistoryFormatter, } DEFAULT_FORMATTER = 'table' COMMANDS_FORMATTERS = {'status': 'status', 'info': 'history', } STORAGE = {'tracking': []} def do_track(self, line): """ track ID Track a parcel. """ parcel = self.get_object(line, 'get_parcel_tracking') if not parcel: print('Error: the parcel "%s" is not found' % line, file=self.stderr) return 2 parcels = set(self.storage.get('tracking', default=[])) parcels.add(parcel.fullid) self.storage.set('tracking', list(parcels)) self.storage.save() print('Parcel "%s" has been tracked.' % parcel.fullid) def do_untrack(self, line): """ untrack ID Stop tracking a parcel. """ removed = False # Always try to first remove the parcel, the untrack should always success parcels = set(self.storage.get('tracking', default=[])) try: parcels.remove(line) removed = True except KeyError: pass if not removed: try: parcel = self.get_object(line, 'get_parcel_tracking') except ParcelNotFound: parcel = False if not parcel: print('Error: the parcel "%s" is not found. Did you provide the full id@backend parameter?' % line, file=self.stderr) return 2 try: parcels.remove(parcel.fullid) except KeyError: print("Error: parcel \"%s\" wasn't tracked" % parcel.fullid, file=self.stderr) return 2 self.storage.set('tracking', list(parcels)) self.storage.save() if removed: print("Parcel \"%s\" isn't tracked anymore." % line) else: print("Parcel \"%s\" isn't tracked anymore." % parcel.fullid) def do_status(self, line): """ status Display status for all of the tracked parcels. """ backends = map(get_backend_name, self.enabled_backends) self.start_format() # XXX cleaning of cached objects may be by start_format()? self.objects = [] for id in self.storage.get('tracking', default=[]): # It should be safe to do it here, since all objects in storage # are stored with the fullid _id, backend_name = id.rsplit('@', 1) # If the user use the -b or -e option, do not try to get # the status of parcel of not loaded backends if backend_name not in backends: continue p = self.get_object(id, 'get_parcel_tracking') if p is None: continue self.cached_format(p) def do_info(self, id): """ info ID Get information about a parcel. """ parcel = self.get_object(id, 'get_parcel_tracking', []) if not parcel: print('Error: parcel not found', file=self.stderr) return 2 self.start_format() self.format(parcel) for event in parcel.history: self.format(event) weboob-1.2/weboob/applications/pastoob/000077500000000000000000000000001303450110500202335ustar00rootroot00000000000000weboob-1.2/weboob/applications/pastoob/__init__.py000066400000000000000000000014231303450110500223440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .pastoob import Pastoob __all__ = ['Pastoob'] weboob-1.2/weboob/applications/pastoob/pastoob.py000066400000000000000000000166161303450110500222660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os import codecs import re from random import choice from weboob.capabilities.paste import CapPaste, PasteNotFound from weboob.tools.application.repl import ReplApplication __all__ = ['Pastoob'] class Pastoob(ReplApplication): APPNAME = 'pastoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2011-YEAR Laurent Bachelier' DESCRIPTION = "Console application allowing to post and get pastes from pastebins." SHORT_DESCRIPTION = "post and get pastes from pastebins" CAPS = CapPaste def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def do_info(self, line): """ info ID [ID2 [...]] Get information about pastes. """ if not line: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 self.start_format() for _id in line.split(' '): paste = self.get_object(_id, 'get_paste', ['id', 'title', 'language', 'public', 'contents']) if not paste: print('Paste not found: %s' % _id, file=self.stderr) self.format(paste) def do_get(self, line): """ get ID Get a paste contents. """ return self._get_op(line, binary=False, command='get') def do_get_bin(self, line): """ get_bin ID Get a paste contents. File will be downloaded from binary services. """ return self._get_op(line, binary=True, command='get_bin') def _get_op(self, _id, binary, command='get'): if not _id: print('This command takes an argument: %s' % self.get_command_help(command, short=True), file=self.stderr) return 2 try: paste = self.get_object(_id, 'get_paste', ['contents']) except PasteNotFound: print('Paste not found: %s' % _id, file=self.stderr) return 3 if not paste: print('Unable to handle paste: %s' % _id, file=self.stderr) return 1 if binary: if self.interactive: if not self.ask('The console may become messed up. Are you sure you want to show a binary file on your terminal?', default=False): print('Aborting.', file=self.stderr) return 1 output = self.stdout output.write(paste.contents.decode('base64')) else: output = codecs.getwriter(self.encoding)(self.stdout) output.write(paste.contents) # add a newline unless we are writing # in a file or in a pipe if output.isatty(): output.write('\n') def do_post(self, line): """ post [FILENAME] Submit a new paste. The filename can be '-' for reading standard input (pipe). If 'bin' is passed, file will be uploaded to binary services. """ return self._post(line, binary=False) def do_post_bin(self, line): """ post_bin [FILENAME] Submit a new paste. The filename can be '-' for reading standard input (pipe). File will be uploaded to binary services. """ return self._post(line, binary=True) def _post(self, filename, binary): use_stdin = (not filename or filename == '-') if use_stdin: if binary: contents = self.stdin.read() else: contents = self.acquire_input() if not len(contents): print('Empty paste, aborting.', file=self.stderr) return 1 else: try: if binary: m = open(filename) else: m = codecs.open(filename, encoding=self.options.encoding or self.encoding) with m as fp: contents = fp.read() except IOError as e: print('Unable to open file "%s": %s' % (filename, e.strerror), file=self.stderr) return 1 if binary: contents = contents.encode('base64') # get and sort the backends able to satisfy our requirements params = self.get_params() backends = {} for backend in self.weboob.iter_backends(): score = backend.can_post(contents, **params) if score: backends.setdefault(score, []).append(backend) # select a random backend from the best scores if len(backends): backend = choice(backends[max(backends.keys())]) else: print('No suitable backend found.', file=self.stderr) return 1 p = backend.new_paste(_id=None) p.public = params['public'] if self.options.title is not None: p.title = self.options.title else: p.title = os.path.basename(filename) p.contents = contents backend.post_paste(p, max_age=params['max_age']) print('Successfuly posted paste: %s' % p.page_url) def get_params(self): return {'public': self.options.public, 'max_age': self.str_to_duration(self.options.max_age), 'title': self.options.title} def str_to_duration(self, s): if s.strip().lower() == 'never': return False parts = re.findall(r'(\d*(?:\.\d+)?)\s*([A-z]+)', s) argsmap = {'Y|y|year|years|yr|yrs': 365.25 * 24 * 3600, 'M|o|month|months': 30.5 * 24 * 3600, 'W|w|week|weeks': 7 * 24 * 3600, 'D|d|day|days': 24 * 3600, 'H|h|hours|hour|hr|hrs': 3600, 'm|i|minute|minutes|min|mins': 60, 'S|s|second|seconds|sec|secs': 1} seconds = 0 for number, unit in parts: for rx, secs in argsmap.iteritems(): if re.match('^(%s)$' % rx, unit): seconds += float(number) * float(secs) return int(seconds) def add_application_options(self, group): group.add_option('-p', '--public', action='store_true', help='Make paste public.') group.add_option('-t', '--title', action='store', help='Paste title', type='string') group.add_option('-m', '--max-age', action='store', help='Maximum age (duration), default "1 month", "never" for infinite', type='string', default='1 month') group.add_option('-E', '--encoding', action='store', help='Input encoding', type='string') weboob-1.2/weboob/applications/qbooblyrics/000077500000000000000000000000001303450110500211145ustar00rootroot00000000000000weboob-1.2/weboob/applications/qbooblyrics/__init__.py000066400000000000000000000001001303450110500232140ustar00rootroot00000000000000from .qbooblyrics import QBooblyrics __all__ = ['QBooblyrics'] weboob-1.2/weboob/applications/qbooblyrics/main_window.py000066400000000000000000000270111303450110500240020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os from PyQt5.QtCore import pyqtSlot as Slot, Qt from PyQt5.QtGui import QKeySequence from PyQt5.QtWidgets import QApplication, QFrame, QShortcut from weboob.capabilities.lyrics import CapLyrics from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.models import BackendListModel from weboob.tools.application.qt5.search_history import HistoryCompleter from weboob.applications.qbooblyrics.ui.main_window_ui import Ui_MainWindow from weboob.applications.qbooblyrics.ui.result_ui import Ui_Result from .minisonglyrics import MiniSonglyrics from .songlyrics import Songlyrics MAX_TAB_TEXT_LENGTH=30 class Result(QFrame): def __init__(self, weboob, app, parent=None): super(Result, self).__init__(parent) self.ui = Ui_Result() self.ui.setupUi(self) self.parent = parent self.weboob = weboob self.app = app self.minis = [] self.current_info_widget = None # action history is composed by the last action and the action list # An action is a function, a list of arguments and a description string self.action_history = {'last_action': None, 'action_list': []} self.ui.backButton.clicked.connect(self.doBack) self.ui.backButton.setShortcut(QKeySequence('Alt+Left')) self.ui.backButton.hide() def doAction(self, description, fun, args): ''' Call fun with args as arguments and save it in the action history ''' self.ui.currentActionLabel.setText(description) if self.action_history['last_action'] is not None: self.action_history['action_list'].append(self.action_history['last_action']) self.ui.backButton.setToolTip('%s (Alt+Left)'%self.action_history['last_action']['description']) self.ui.backButton.show() self.action_history['last_action'] = {'function': fun, 'args': args, 'description': description} # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = description if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, description) return fun(*args) @Slot() def doBack(self): ''' Go back in action history Basically call previous function and update history ''' if len(self.action_history['action_list']) > 0: todo = self.action_history['action_list'].pop() self.ui.currentActionLabel.setText(todo['description']) self.action_history['last_action'] = todo if len(self.action_history['action_list']) == 0: self.ui.backButton.hide() else: self.ui.backButton.setToolTip(self.action_history['action_list'][-1]['description']) # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = todo['description'] if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, todo['description']) return todo['function'](*todo['args']) def processFinished(self): self.parent.ui.searchEdit.setEnabled(True) QApplication.restoreOverrideCursor() self.process = None self.parent.ui.stopButton.hide() @Slot() def stopProcess(self): if self.process is not None: self.process.stop() def searchSonglyrics(self,pattern): if not pattern: return self.doAction(u'Search lyrics "%s"' % pattern, self.searchSonglyricsAction, [pattern]) def searchSonglyricsAction(self, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addSonglyrics, fb=self.processFinished) self.process.do(self.app._do_complete, self.parent.getCount(), ('title'), 'iter_lyrics', self.parent.ui.typeCombo.currentText(), pattern, backends=backend_name, caps=CapLyrics) self.parent.ui.stopButton.show() def addSonglyrics(self, songlyrics): minisonglyrics = MiniSonglyrics(self.weboob, self.weboob[songlyrics.backend], songlyrics, self) self.ui.list_content.layout().insertWidget(self.ui.list_content.layout().count()-1,minisonglyrics) self.minis.append(minisonglyrics) def displaySonglyrics(self, songlyrics, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wsonglyrics = Songlyrics(songlyrics, backend, self) self.ui.info_content.layout().addWidget(wsonglyrics) self.current_info_widget = wsonglyrics QApplication.restoreOverrideCursor() def searchId(self, id): QApplication.setOverrideCursor(Qt.WaitCursor) if '@' in id: backend_name = id.split('@')[1] id = id.split('@')[0] else: backend_name = None for backend in self.weboob.iter_backends(): if (backend_name and backend.name == backend_name) or not backend_name: songlyrics = backend.get_lyrics(id) if songlyrics: self.doAction('Lyrics of "%s" (%s)' % (songlyrics.title, songlyrics.artist), self.displaySonglyrics, [songlyrics, backend]) QApplication.restoreOverrideCursor() class MainWindow(QtMainWindow): def __init__(self, config, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.app = app self.minis = [] self.current_info_widget = None # search history is a list of patterns which have been searched history_path = os.path.join(self.weboob.workdir, 'qbooblyrics_history') qc = HistoryCompleter(history_path, self) qc.load() qc.setCaseSensitivity(Qt.CaseInsensitive) self.ui.searchEdit.setCompleter(qc) self.ui.typeCombo.addItem('song') self.ui.typeCombo.addItem('artist') self.ui.searchEdit.returnPressed.connect(self.search) self.ui.idEdit.returnPressed.connect(self.searchId) count = self.config.get('settings', 'maxresultsnumber') self.ui.countSpin.setValue(int(count)) showT = self.config.get('settings', 'showthumbnails') self.ui.showTCheck.setChecked(showT == '1') self.ui.stopButton.hide() self.ui.actionBackends.triggered.connect(self.backendsConfig) q = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_Q), self) q.activated.connect(self.close) n = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageDown), self) n.activated.connect(self.nextTab) p = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageUp), self) p.activated.connect(self.prevTab) w = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_W), self) w.activated.connect(self.closeCurrentTab) l = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_L), self) l.activated.connect(self.ui.searchEdit.setFocus) l.activated.connect(self.ui.searchEdit.selectAll) self.ui.resultsTab.tabCloseRequested.connect(self.closeTab) self.loadBackendsList() if self.ui.backendEdit.count() == 0: self.backendsConfig() @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapLyrics, ), self) if bckndcfg.run(): self.loadBackendsList() def loadBackendsList(self): model = BackendListModel(self.weboob) model.addBackends() self.ui.backendEdit.setModel(model) current_backend = self.config.get('settings', 'backend') idx = self.ui.backendEdit.findData(current_backend) if idx >= 0: self.ui.backendEdit.setCurrentIndex(idx) if self.ui.backendEdit.count() == 0: self.ui.searchEdit.setEnabled(False) else: self.ui.searchEdit.setEnabled(True) def getCount(self): num = self.ui.countSpin.value() if num == 0: return None else: return num @Slot(int) def closeTab(self, index): if self.ui.resultsTab.widget(index) != 0: self.ui.resultsTab.removeTab(index) @Slot() def closeCurrentTab(self): self.closeTab(self.ui.resultsTab.currentIndex()) @Slot() def prevTab(self): index = self.ui.resultsTab.currentIndex() - 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) @Slot() def nextTab(self): index = self.ui.resultsTab.currentIndex() + 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) def newTab(self, txt, backend, songlyrics=None): id = '' if songlyrics is not None: id = songlyrics.id new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, txt) new_res.searchId('%s@%s'%(id,backend.NAME)) self.ui.stopButton.clicked.connect(new_res.stopProcess) @Slot() def search(self): pattern = self.ui.searchEdit.text() self.ui.searchEdit.completer().addString(pattern) new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, pattern) self.ui.resultsTab.setCurrentWidget(new_res) new_res.searchSonglyrics(pattern) @Slot() def searchId(self): id = self.ui.idEdit.text() new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, id) self.ui.resultsTab.setCurrentWidget(new_res) new_res.searchId(id) def closeEvent(self, ev): self.config.set('settings', 'backend', self.ui.backendEdit.itemData( self.ui.backendEdit.currentIndex())) self.ui.searchEdit.completer().save() self.config.set('settings', 'maxresultsnumber', self.ui.countSpin.value()) self.config.save() ev.accept() weboob-1.2/weboob/applications/qbooblyrics/minisonglyrics.py000066400000000000000000000063421303450110500245440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from PyQt5.QtWidgets import QFrame, QApplication from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.applications.qbooblyrics.ui.minisonglyrics_ui import Ui_MiniSonglyrics from weboob.capabilities.base import empty class MiniSonglyrics(QFrame): def __init__(self, weboob, backend, songlyrics, parent=None): super(MiniSonglyrics, self).__init__(parent) self.parent = parent self.ui = Ui_MiniSonglyrics() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.songlyrics = songlyrics self.ui.titleLabel.setText(songlyrics.title) if not empty(songlyrics.artist): if len(songlyrics.artist) > 300: self.ui.artistLabel.setText('%s [...]'%songlyrics.artist[:300]) else: self.ui.artistLabel.setText(songlyrics.artist) else: self.ui.artistLabel.setText('') self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) @Slot() def viewPressed(self): QApplication.setOverrideCursor(Qt.WaitCursor) songlyrics = self.backend.get_lyrics(self.songlyrics.id) if songlyrics: self.parent.doAction('Lyrics of "%s" (%s)' % (songlyrics.title, songlyrics.artist), self.parent.displaySonglyrics, [songlyrics, self.backend]) @Slot() def newTabPressed(self): songlyrics = self.backend.get_lyrics(self.songlyrics.id) self.parent.parent.newTab(u'Lyrics of "%s" (%s)' % (songlyrics.title, songlyrics.artist), self.backend, songlyrics=songlyrics) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qbooblyrics/qbooblyrics.py000066400000000000000000000027441303450110500240250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.lyrics import CapLyrics from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QBooblyrics(QtApplication): APPNAME = 'qbooblyrics' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2016 Julien Veyssier' DESCRIPTION = "Qt application allowing to search song lyrics." SHORT_DESCRIPTION = "search lyrics" CAPS = CapLyrics CONFIG = {'settings': {'backend': '', 'maxresultsnumber': '10' } } def main(self, argv): self.load_backends([CapLyrics]) self.load_config() self.main_window = MainWindow(self.config, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qbooblyrics/songlyrics.py000066400000000000000000000035161303450110500236670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from PyQt5.QtCore import Qt from PyQt5.QtWidgets import QFrame from weboob.applications.qbooblyrics.ui.songlyrics_ui import Ui_Songlyrics from weboob.capabilities.base import empty class Songlyrics(QFrame): def __init__(self, songlyrics, backend, parent=None): super(Songlyrics, self).__init__(parent) self.parent = parent self.ui = Ui_Songlyrics() self.ui.setupUi(self) self.songlyrics = songlyrics self.backend = backend self.ui.idEdit.setText(u'%s@%s' % (songlyrics.id, backend.name)) if not empty(songlyrics.title): self.ui.titleLabel.setText(songlyrics.title) if not empty(songlyrics.artist): self.ui.artistLabel.setText(songlyrics.artist) else: self.ui.artistLabel.parent().hide() if not empty(songlyrics.content): self.ui.lyricsPlain.setPlainText(songlyrics.content) else: self.ui.lyricsPlain.parent().hide() self.ui.verticalLayout.setAlignment(Qt.AlignTop) self.ui.verticalLayout_2.setAlignment(Qt.AlignTop) weboob-1.2/weboob/applications/qbooblyrics/ui/000077500000000000000000000000001303450110500215315ustar00rootroot00000000000000weboob-1.2/weboob/applications/qbooblyrics/ui/Makefile000066400000000000000000000002651303450110500231740ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qbooblyrics/ui/__init__.py000066400000000000000000000000001303450110500236300ustar00rootroot00000000000000weboob-1.2/weboob/applications/qbooblyrics/ui/main_window.ui000066400000000000000000000105721303450110500244100ustar00rootroot00000000000000 MainWindow 0 0 748 463 QBooblyrics 4 QFrame::StyledPanel QFrame::Raised 0 0 Search: 35 16777215 background-color: qlineargradient(spread:pad, x1:1, y1:1, x2:1, y2:1, stop:0 rgba(255, 0, 0, 255), stop:0.479904 rgba(255, 0, 0, 255), stop:0.522685 rgba(255, 255, 255, 255), stop:1 rgba(255, 255, 255, 255)); stop <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> max results <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> -1 true true Show thumbnails search by ID: 0 0 748 25 toolBar TopToolBarArea false Backends weboob-1.2/weboob/applications/qbooblyrics/ui/minisonglyrics.ui000066400000000000000000000121551303450110500251450ustar00rootroot00000000000000 MiniSonglyrics 0 0 578 136 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 TextLabel true 0 0 50 true false TextLabel true 75 true Artist 75 true Where 75 true Title 120 20 View 120 20 View in new tab Qt::Horizontal 40 20 16777215 20 PushButton true Qt::Horizontal 40 20 weboob-1.2/weboob/applications/qbooblyrics/ui/result.ui000066400000000000000000000111721303450110500234100ustar00rootroot00000000000000 Result 0 0 645 733 0 0 10000 10000 Frame QFrame::StyledPanel QFrame::Raised QFrame::NoFrame QFrame::Raised 0 0 Qt::Horizontal 40 20 60 20 <<back 75 true Qt::AlignCenter true Qt::Horizontal 40 20 true 0 0 605 667 Qt::Vertical 20 40 true 0 0 605 667 weboob-1.2/weboob/applications/qbooblyrics/ui/songlyrics.ui000066400000000000000000000155211303450110500242700ustar00rootroot00000000000000 Songlyrics 0 0 645 889 0 0 8000 5000 Frame QFrame::StyledPanel QFrame::Raised 16777215 3000 QFrame::NoFrame QFrame::Raised 16777215 4000 QFrame::NoFrame QFrame::Raised 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Title: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Artist: 0 0 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Lyrics 0 100 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse weboob-1.2/weboob/applications/qboobmsg/000077500000000000000000000000001303450110500203755ustar00rootroot00000000000000weboob-1.2/weboob/applications/qboobmsg/__init__.py000066400000000000000000000000671303450110500225110ustar00rootroot00000000000000from .qboobmsg import QBoobMsg __all__ = ['QBoobMsg'] weboob-1.2/weboob/applications/qboobmsg/main_window.py000066400000000000000000000036021303450110500232630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtCore import pyqtSlot as Slot from weboob.tools.application.qt5 import QtMainWindow from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.capabilities.messages import CapMessages from .ui.main_window_ui import Ui_MainWindow from .messages_manager import MessagesManager class MainWindow(QtMainWindow): def __init__(self, config, weboob, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.manager = MessagesManager(weboob, self) self.setCentralWidget(self.manager) self.ui.actionBackends.triggered.connect(self.backendsConfig) self.ui.actionRefresh.triggered.connect(self.refresh) if self.weboob.count_backends() == 0: self.backendsConfig() else: self.centralWidget().load() @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapMessages,), self) if bckndcfg.run(): self.centralWidget().load() @Slot() def refresh(self): self.centralWidget().refreshThreads() weboob-1.2/weboob/applications/qboobmsg/messages_manager.py000066400000000000000000000245301303450110500242540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import time import logging from PyQt5.QtWidgets import QWidget, QTreeWidgetItem, QListWidgetItem, QMessageBox from PyQt5.QtGui import QBrush from PyQt5.QtCore import Qt, pyqtSignal as Signal, pyqtSlot as Slot from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message from weboob.tools.application.qt5 import QtDo from weboob.tools.misc import to_unicode from .ui.messages_manager_ui import Ui_MessagesManager class MessagesManager(QWidget): display_contact = Signal(object) def __init__(self, weboob, parent=None): super(MessagesManager, self).__init__(parent) self.ui = Ui_MessagesManager() self.ui.setupUi(self) self.weboob = weboob self.ui.backendsList.setCurrentRow(0) self.backend = None self.thread = None self.message = None self.ui.replyButton.setEnabled(False) self.ui.replyWidget.hide() self.ui.backendsList.itemSelectionChanged.connect(self._backendChanged) self.ui.threadsList.itemSelectionChanged.connect(self._threadChanged) self.ui.messagesTree.itemClicked.connect(self._messageSelected) self.ui.messagesTree.itemActivated.connect(self._messageSelected) self.ui.profileButton.clicked.connect(self._profilePressed) self.ui.replyButton.clicked.connect(self._replyPressed) self.ui.sendButton.clicked.connect(self._sendPressed) def load(self): self.ui.backendsList.clear() self.ui.backendsList.addItem('(All)') for backend in self.weboob.iter_backends(): if not backend.has_caps(CapMessages): continue item = QListWidgetItem(backend.name.capitalize()) item.setData(Qt.UserRole, backend) self.ui.backendsList.addItem(item) self.refreshThreads() @Slot() def _backendChanged(self): selection = self.ui.backendsList.selectedItems() if not selection: self.backend = None return self.backend = selection[0].data(Qt.UserRole) self.refreshThreads() def refreshThreads(self): self.ui.messagesTree.clear() self.ui.threadsList.clear() self.hideReply() self.ui.profileButton.hide() self.ui.replyButton.setEnabled(False) self.ui.backendsList.setEnabled(False) self.ui.threadsList.setEnabled(False) self.process_threads = QtDo(self.weboob, self._gotThread, fb=self._gotThreadsEnd) self.process_threads.do('iter_threads', backends=self.backend, caps=CapMessages) def _gotThreadsEnd(self): self.process_threads = None self.ui.backendsList.setEnabled(True) self.ui.threadsList.setEnabled(True) def _gotThread(self, thread): item = QListWidgetItem(thread.title) item.setData(Qt.UserRole, (thread.backend, thread.id)) self.ui.threadsList.addItem(item) @Slot() def _threadChanged(self): self.ui.messagesTree.clear() selection = self.ui.threadsList.selectedItems() if not selection: return t = selection[0].data(Qt.UserRole) self.refreshThreadMessages(*t) def refreshThreadMessages(self, backend, id): self.ui.messagesTree.clear() self.ui.messageBody.clear() self.ui.backendsList.setEnabled(False) self.ui.threadsList.setEnabled(False) self.ui.replyButton.setEnabled(False) self.ui.profileButton.hide() self.hideReply() self.process = QtDo(self.weboob, self._gotThreadMessages, fb=self._gotThreadMessagesEnd) self.process.do('get_thread', id, backends=backend) def _gotThreadMessagesEnd(self): self.ui.backendsList.setEnabled(True) self.ui.threadsList.setEnabled(True) self.process = None def _gotThreadMessages(self, thread): self.thread = thread if thread.flags & thread.IS_THREADS: top = self.ui.messagesTree.invisibleRootItem() else: top = None self._insert_message(thread.root, top) self.showMessage(thread.root) self.ui.messagesTree.expandAll() def _insert_message(self, message, top): item = QTreeWidgetItem(None, [message.title or '', message.sender or 'Unknown', time.strftime('%Y-%m-%d %H:%M:%S', message.date.timetuple())]) item.setData(0, Qt.UserRole, message) if message.flags & message.IS_UNREAD: item.setForeground(0, QBrush(Qt.darkYellow)) item.setForeground(1, QBrush(Qt.darkYellow)) item.setForeground(2, QBrush(Qt.darkYellow)) if top is not None: # threads top.addChild(item) else: # discussion self.ui.messagesTree.invisibleRootItem().insertChild(0, item) if message.children is not None: for child in message.children: self._insert_message(child, top and item) @Slot(QTreeWidgetItem, int) def _messageSelected(self, item, column): message = item.data(0, Qt.UserRole) self.showMessage(message, item) def showMessage(self, message, item=None): backend = self.weboob.get_backend(message.thread.backend) if backend.has_caps(CapMessagesPost): self.ui.replyButton.setEnabled(True) self.message = message if message.title.startswith('Re:'): self.ui.titleEdit.setText(message.title) else: self.ui.titleEdit.setText('Re: %s' % message.title) if message.flags & message.IS_HTML: content = message.content else: content = message.content.replace('&', '&').replace('<', '<').replace('>', '>').replace('\n', '
') extra = u'' if message.flags & message.IS_NOT_RECEIVED: extra += u'Status: Unread
' elif message.flags & message.IS_RECEIVED: extra += u'Status: Read
' elif message.flags & message.IS_UNREAD: extra += u'Status: New
' self.ui.messageBody.setText("

%s

" "Date: %s
" "From: %s
" "%s" "

%s

" % (message.title, str(message.date), message.sender, extra, content)) if item and message.flags & message.IS_UNREAD: backend.set_message_read(message) message.flags &= ~message.IS_UNREAD item.setForeground(0, QBrush()) item.setForeground(1, QBrush()) item.setForeground(2, QBrush()) if message.thread.flags & message.thread.IS_DISCUSSION: self.ui.profileButton.show() else: self.ui.profileButton.hide() @Slot() def _profilePressed(self): print(self.thread.id) self.display_contact.emit(self.thread.id) def displayReply(self): self.ui.replyButton.setText(self.tr('Cancel')) self.ui.replyWidget.show() def hideReply(self): self.ui.replyButton.setText(self.tr('Reply')) self.ui.replyWidget.hide() self.ui.replyEdit.clear() self.ui.titleEdit.clear() @Slot() def _replyPressed(self): if self.ui.replyWidget.isVisible(): self.hideReply() else: self.displayReply() @Slot() def _sendPressed(self): if not self.ui.replyWidget.isVisible(): return text = self.ui.replyEdit.toPlainText() title = self.ui.titleEdit.text() self.ui.backendsList.setEnabled(False) self.ui.threadsList.setEnabled(False) self.ui.messagesTree.setEnabled(False) self.ui.replyButton.setEnabled(False) self.ui.replyWidget.setEnabled(False) self.ui.sendButton.setText(self.tr('Sending...')) flags = 0 if self.ui.htmlBox.currentIndex() == 0: flags = Message.IS_HTML m = Message(thread=self.thread, id=0, title=title, sender=None, receivers=None, content=text, parent=self.message, flags=flags) self.process_reply = QtDo(self.weboob, None, self._postReply_eb, self._postReply_fb) self.process_reply.do('post_message', m, backends=self.thread.backend) def _postReply_fb(self): self.ui.backendsList.setEnabled(True) self.ui.threadsList.setEnabled(True) self.ui.messagesTree.setEnabled(True) self.ui.replyButton.setEnabled(True) self.ui.replyWidget.setEnabled(True) self.ui.sendButton.setEnabled(True) self.ui.sendButton.setText(self.tr('Send')) self.hideReply() self.process_reply = None self.refreshThreadMessages(self.thread.backend, self.thread.id) def _postReply_eb(self, backend, error, backtrace): content = self.tr('Unable to send message:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while posting reply'), content, QMessageBox.Ok) self.ui.backendsList.setEnabled(True) self.ui.threadsList.setEnabled(True) self.ui.messagesTree.setEnabled(True) self.ui.replyButton.setEnabled(True) self.ui.replyWidget.setEnabled(True) self.ui.sendButton.setText(self.tr('Send')) self.process_reply = None weboob-1.2/weboob/applications/qboobmsg/qboobmsg.py000066400000000000000000000026211303450110500225610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.messages import CapMessages from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QBoobMsg(QtApplication): APPNAME = 'qboobmsg' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Qt application allowing to read messages on various websites and reply to them." SHORT_DESCRIPTION = "send and receive message threads" CAPS = CapMessages def main(self, argv): self.load_backends(CapMessages, storage=self.create_storage()) self.main_window = MainWindow(self.config, self.weboob) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qboobmsg/ui/000077500000000000000000000000001303450110500210125ustar00rootroot00000000000000weboob-1.2/weboob/applications/qboobmsg/ui/Makefile000066400000000000000000000002651303450110500224550ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qboobmsg/ui/__init__.py000066400000000000000000000000001303450110500231110ustar00rootroot00000000000000weboob-1.2/weboob/applications/qboobmsg/ui/main_window.ui000066400000000000000000000043071303450110500236700ustar00rootroot00000000000000 MainWindow 0 0 763 580 QBoobMsg 0 0 763 20 File toolBar TopToolBarArea false Backends Quit Quit Ctrl+Q Refresh actionQuit triggered() MainWindow close() -1 -1 381 289 weboob-1.2/weboob/applications/qboobmsg/ui/messages_manager.ui000066400000000000000000000212051303450110500246520ustar00rootroot00000000000000 MessagesManager 0 0 696 591 Qt::Horizontal 0 0 150 16777215 Qt::Horizontal Qt::Vertical 0 0 5 0 QFrame::StyledPanel QFrame::Raised 9 2 9 2 0 0 + 0 0 Qt::Horizontal 40 20 QAbstractItemView::NoEditTriggers true true true true 150 true Title From Date Qt::Vertical 0 5 true Qt::LinksAccessibleByKeyboard|Qt::LinksAccessibleByMouse|Qt::TextBrowserInteraction|Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse QFrame::StyledPanel QFrame::Raised Profile Reply 0 With HTML Without HTML Send expandButton clicked() messagesTree expandAll() 733 31 527 150 collapseButton clicked() messagesTree collapseAll() 733 60 527 150 weboob-1.2/weboob/applications/qcineoob/000077500000000000000000000000001303450110500203635ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcineoob/__init__.py000066400000000000000000000000671303450110500224770ustar00rootroot00000000000000from .qcineoob import QCineoob __all__ = ['QCineoob'] weboob-1.2/weboob/applications/qcineoob/main_window.py000066400000000000000000000602301303450110500232510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os from PyQt5.QtCore import Qt, pyqtSlot as Slot from PyQt5.QtGui import QKeySequence from PyQt5.QtWidgets import QApplication, QFrame, QShortcut from weboob.capabilities.base import NotAvailable from weboob.capabilities.cinema import CapCinema from weboob.capabilities.torrent import CapTorrent from weboob.capabilities.subtitle import CapSubtitle from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.models import BackendListModel from weboob.tools.application.qt5.search_history import HistoryCompleter from weboob.applications.suboob.suboob import LANGUAGE_CONV from weboob.applications.qcineoob.ui.main_window_ui import Ui_MainWindow from weboob.applications.qcineoob.ui.result_ui import Ui_Result from .minimovie import MiniMovie from .miniperson import MiniPerson from .minitorrent import MiniTorrent from .minisubtitle import MiniSubtitle from .movie import Movie from .person import Person from .torrent import Torrent from .subtitle import Subtitle MAX_TAB_TEXT_LENGTH=30 class Result(QFrame): def __init__(self, weboob, app, parent=None): super(Result, self).__init__(parent) self.ui = Ui_Result() self.ui.setupUi(self) self.parent = parent self.weboob = weboob self.app = app self.minis = [] self.current_info_widget = None # action history is composed by the last action and the action list # An action is a function, a list of arguments and a description string self.action_history = {'last_action': None, 'action_list': []} self.ui.backButton.clicked.connect(self.doBack) self.ui.backButton.setShortcut(QKeySequence('Alt+Left')) self.ui.backButton.hide() def doAction(self, description, fun, args): ''' Call fun with args as arguments and save it in the action history ''' self.ui.currentActionLabel.setText(description) if self.action_history['last_action'] is not None: self.action_history['action_list'].append(self.action_history['last_action']) self.ui.backButton.setToolTip('%s (Alt+Left)'%self.action_history['last_action']['description']) self.ui.backButton.show() self.action_history['last_action'] = {'function': fun, 'args': args, 'description': description} # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = description if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, description) return fun(*args) @Slot() def doBack(self): ''' Go back in action history Basically call previous function and update history ''' if len(self.action_history['action_list']) > 0: todo = self.action_history['action_list'].pop() self.ui.currentActionLabel.setText(todo['description']) self.action_history['last_action'] = todo if len(self.action_history['action_list']) == 0: self.ui.backButton.hide() else: self.ui.backButton.setToolTip(self.action_history['action_list'][-1]['description']) # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = todo['description'] if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, todo['description']) return todo['function'](*todo['args']) def castingAction(self, backend_name, id, role): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) self.process = QtDo(self.weboob, self.addPerson, fb=self.processFinished) self.process.do('iter_movie_persons', id, role, backends=backend_name, caps=CapCinema) self.parent.ui.stopButton.show() def moviesInCommonAction(self, backend_name, id1, id2): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) for a_backend in self.weboob.iter_backends(): if (backend_name and a_backend.name == backend_name): backend = a_backend person1 = backend.get_person(id1) person2 = backend.get_person(id2) lid1 = [] for p in backend.iter_person_movies_ids(id1): lid1.append(p) lid2 = [] for p in backend.iter_person_movies_ids(id2): lid2.append(p) inter = list(set(lid1) & set(lid2)) chrono_list = [] for common in inter: movie = backend.get_movie(common) movie.backend = backend_name role1 = movie.get_roles_by_person_id(person1.id) role2 = movie.get_roles_by_person_id(person2.id) if (movie.release_date != NotAvailable): year = movie.release_date.year else: year = '????' movie.short_description = '(%s) %s as %s ; %s as %s'%(year , person1.name, ', '.join(role1), person2.name, ', '.join(role2)) i = 0 while (i chrono_list[i].release_date.year)): i += 1 chrono_list.insert(i, movie) for movie in chrono_list: self.addMovie(movie) self.processFinished() def personsInCommonAction(self, backend_name, id1, id2): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) for a_backend in self.weboob.iter_backends(): if (backend_name and a_backend.name == backend_name): backend = a_backend movie1 = backend.get_movie(id1) movie2 = backend.get_movie(id2) lid1 = [] for p in backend.iter_movie_persons_ids(id1): lid1.append(p) lid2 = [] for p in backend.iter_movie_persons_ids(id2): lid2.append(p) inter = list(set(lid1) & set(lid2)) for common in inter: person = backend.get_person(common) person.backend = backend_name role1 = movie1.get_roles_by_person_id(person.id) role2 = movie2.get_roles_by_person_id(person.id) person.short_description = '%s in %s ; %s in %s'%(', '.join(role1), movie1.original_title, ', '.join(role2), movie2.original_title) self.addPerson(person) self.processFinished() def filmographyAction(self, backend_name, id, role): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) self.process = QtDo(self.weboob, self.addMovie, fb=self.processFinished) self.process.do('iter_person_movies', id, role, backends=backend_name, caps=CapCinema) self.parent.ui.stopButton.show() def search(self, tosearch, pattern, lang): if tosearch == 'person': self.searchPerson(pattern) elif tosearch == 'movie': self.searchMovie(pattern) elif tosearch == 'torrent': self.searchTorrent(pattern) elif tosearch == 'subtitle': self.searchSubtitle(lang, pattern) def searchMovie(self, pattern): if not pattern: return self.doAction(u'Search movie "%s"' % pattern, self.searchMovieAction, [pattern]) def searchMovieAction(self, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addMovie, fb=self.processFinished) #self.process.do('iter_movies', pattern, backends=backend_name, caps=CapCinema) self.process.do(self.app._do_complete, self.parent.getCount(), ('original_title'), 'iter_movies', pattern, backends=backend_name, caps=CapCinema) self.parent.ui.stopButton.show() def addMovie(self, movie): minimovie = MiniMovie(self.weboob, self.weboob[movie.backend], movie, self) self.ui.list_content.layout().insertWidget(self.ui.list_content.layout().count()-1,minimovie) self.minis.append(minimovie) def displayMovie(self, movie, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wmovie = Movie(movie, backend, self) self.ui.info_content.layout().addWidget(wmovie) self.current_info_widget = wmovie QApplication.restoreOverrideCursor() def searchPerson(self, pattern): if not pattern: return self.doAction(u'Search person "%s"' % pattern, self.searchPersonAction, [pattern]) def searchPersonAction(self, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addPerson, fb=self.processFinished) #self.process.do('iter_persons', pattern, backends=backend_name, caps=CapCinema) self.process.do(self.app._do_complete, self.parent.getCount(), ('name'), 'iter_persons', pattern, backends=backend_name, caps=CapCinema) self.parent.ui.stopButton.show() def addPerson(self, person): miniperson = MiniPerson(self.weboob, self.weboob[person.backend], person, self) self.ui.list_content.layout().insertWidget(self.ui.list_content.layout().count()-1,miniperson) self.minis.append(miniperson) def displayPerson(self, person, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wperson = Person(person, backend, self) self.ui.info_content.layout().addWidget(wperson) self.current_info_widget = wperson QApplication.restoreOverrideCursor() def searchTorrent(self, pattern): if not pattern: return self.doAction(u'Search torrent "%s"' % pattern, self.searchTorrentAction, [pattern]) def searchTorrentAction(self, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addTorrent, fb=self.processFinished) #self.process.do('iter_torrents', pattern, backends=backend_name, caps=CapTorrent) self.process.do(self.app._do_complete, self.parent.getCount(), ('name'), 'iter_torrents', pattern, backends=backend_name, caps=CapTorrent) self.parent.ui.stopButton.show() def processFinished(self): self.parent.ui.searchEdit.setEnabled(True) QApplication.restoreOverrideCursor() self.process = None self.parent.ui.stopButton.hide() @Slot() def stopProcess(self): if self.process is not None: self.process.stop() def addTorrent(self, torrent): minitorrent = MiniTorrent(self.weboob, self.weboob[torrent.backend], torrent, self) positionToInsert = self.ui.list_content.layout().count()-1 # if possible, we insert the torrent keeping a sort by seed if torrent.seeders != NotAvailable: seeders = torrent.seeders positionToInsert = 0 while positionToInsert < len(self.minis) and\ (self.minis[positionToInsert].torrent.seeders != NotAvailable and\ seeders <= self.minis[positionToInsert].torrent.seeders): positionToInsert += 1 self.ui.list_content.layout().insertWidget(positionToInsert, minitorrent) self.minis.insert(positionToInsert, minitorrent) def displayTorrent(self, torrent, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wtorrent = Torrent(torrent, backend, self) self.ui.info_content.layout().addWidget(wtorrent) self.current_info_widget = wtorrent def searchSubtitle(self, lang, pattern): if not pattern: return self.doAction(u'Search subtitle "%s" (lang:%s)' % (pattern, lang), self.searchSubtitleAction, [lang, pattern]) def searchSubtitleAction(self, lang, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addSubtitle, fb=self.processFinished) #self.process.do('iter_subtitles', lang, pattern, backends=backend_name, caps=CapSubtitle) self.process.do(self.app._do_complete, self.parent.getCount(), ('name'), 'iter_subtitles', lang, pattern, backends=backend_name, caps=CapSubtitle) self.parent.ui.stopButton.show() def addSubtitle(self, subtitle): minisubtitle = MiniSubtitle(self.weboob, self.weboob[subtitle.backend], subtitle, self) self.ui.list_content.layout().insertWidget(self.ui.list_content.layout().count()-1,minisubtitle) self.minis.append(minisubtitle) def displaySubtitle(self, subtitle, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wsubtitle = Subtitle(subtitle, backend, self) self.ui.info_content.layout().addWidget(wsubtitle) self.current_info_widget = wsubtitle def searchId(self, id, stype): QApplication.setOverrideCursor(Qt.WaitCursor) title_field = 'name' if stype == 'movie': cap = CapCinema title_field = 'original_title' elif stype == 'person': cap = CapCinema elif stype == 'torrent': cap = CapTorrent elif stype == 'subtitle': cap = CapSubtitle if '@' in id: backend_name = id.split('@')[1] id = id.split('@')[0] else: backend_name = None for backend in self.weboob.iter_backends(): if backend.has_caps(cap) and ((backend_name and backend.name == backend_name) or not backend_name): exec('object = backend.get_%s(id)' % (stype)) if object: func_display = 'self.display' + stype[0].upper() + stype[1:] exec("self.doAction('Details of %s \"%%s\"' %% object.%s, %s, [object, backend])" % (stype, title_field, func_display)) QApplication.restoreOverrideCursor() class MainWindow(QtMainWindow): def __init__(self, config, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.app = app # search history is a list of patterns which have been searched history_path = os.path.join(self.weboob.workdir, 'qcineoob_history') qc = HistoryCompleter(history_path, self) qc.load() qc.setCaseSensitivity(Qt.CaseInsensitive) self.ui.searchEdit.setCompleter(qc) self.ui.searchEdit.returnPressed.connect(self.search) self.ui.idEdit.returnPressed.connect(self.searchId) self.ui.typeCombo.currentIndexChanged[unicode].connect(self.typeComboChanged) count = self.config.get('settings', 'maxresultsnumber') self.ui.countSpin.setValue(int(count)) showT = self.config.get('settings', 'showthumbnails') self.ui.showTCheck.setChecked(showT == '1') self.ui.stopButton.hide() self.ui.actionBackends.triggered.connect(self.backendsConfig) q = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_Q), self) q.activated.connect(self.close) n = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageDown), self) n.activated.connect(self.nextTab) p = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageUp), self) p.activated.connect(self.prevTab) w = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_W), self) w.activated.connect(self.closeCurrentTab) l = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_L), self) l.activated.connect(self.ui.searchEdit.setFocus) l.activated.connect(self.ui.searchEdit.selectAll) self.ui.resultsTab.tabCloseRequested.connect(self.closeTab) self.loadBackendsList() if self.ui.backendEdit.count() == 0: self.backendsConfig() langs = sorted(LANGUAGE_CONV.keys()) for lang in langs: self.ui.langCombo.addItem(lang) self.ui.langCombo.hide() self.ui.langLabel.hide() @Slot(int) def closeTab(self, index): if self.ui.resultsTab.widget(index) != 0: self.ui.resultsTab.removeTab(index) @Slot() def closeCurrentTab(self): self.closeTab(self.ui.resultsTab.currentIndex()) @Slot() def prevTab(self): index = self.ui.resultsTab.currentIndex() - 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) @Slot() def nextTab(self): index = self.ui.resultsTab.currentIndex() + 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) def newTab(self, txt, backend, person=None, movie=None, torrent=None, subtitle=None): id = '' if person is not None: id = person.id stype = 'person' elif movie is not None: id = movie.id stype = 'movie' elif subtitle is not None: id = subtitle.id stype = 'subtitle' elif torrent is not None: id = torrent.id stype = 'torrent' new_res = Result(self.weboob, self.app, self) self.ui.stopButton.clicked.connect(new_res.stopProcess) tabtxt = txt if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] index = self.ui.resultsTab.addTab(new_res, tabtxt) self.ui.resultsTab.setTabToolTip(index, txt) new_res.searchId(id, stype) @Slot() def search(self): pattern = self.ui.searchEdit.text() self.ui.searchEdit.completer().addString(pattern) tosearch = self.ui.typeCombo.currentText() lang = self.ui.langCombo.currentText() new_res = Result(self.weboob, self.app, self) txt = 'search %s "%s"'%(tosearch, pattern) tabtxt = txt if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] index = self.ui.resultsTab.addTab(new_res, tabtxt) self.ui.resultsTab.setTabToolTip(index, txt) self.ui.resultsTab.setCurrentWidget(new_res) new_res.search(tosearch, pattern, lang) @Slot() def searchId(self): id = self.ui.idEdit.text() stype = self.ui.idTypeCombo.currentText() new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, id) self.ui.resultsTab.setCurrentWidget(new_res) new_res.searchId(id, stype) @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapCinema, CapTorrent, CapSubtitle,), self) if bckndcfg.run(): self.loadBackendsList() def loadBackendsList(self): model = BackendListModel(self.weboob) model.addBackends() self.ui.backendEdit.setModel(model) current_backend = self.config.get('settings', 'backend') idx = self.ui.backendEdit.findData(current_backend) if idx >= 0: self.ui.backendEdit.setCurrentIndex(idx) if self.ui.backendEdit.count() == 0: self.ui.searchEdit.setEnabled(False) else: self.ui.searchEdit.setEnabled(True) @Slot(unicode) def typeComboChanged(self, value): if value == 'subtitle': self.ui.langCombo.show() self.ui.langLabel.show() else: self.ui.langCombo.hide() self.ui.langLabel.hide() def getCount(self): num = self.ui.countSpin.value() if num == 0: return None else: return num def closeEvent(self, ev): self.config.set('settings', 'backend', self.ui.backendEdit.itemData( self.ui.backendEdit.currentIndex())) self.ui.searchEdit.completer().save() self.config.set('settings', 'maxresultsnumber', self.ui.countSpin.value()) self.config.set('settings', 'showthumbnails', '1' if self.ui.showTCheck.isChecked() else '0') self.config.save() ev.accept() weboob-1.2/weboob/applications/qcineoob/minimovie.py000066400000000000000000000070631303450110500227370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from PyQt5.QtWidgets import QFrame, QApplication from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.applications.qcineoob.ui.minimovie_ui import Ui_MiniMovie from weboob.capabilities.base import empty, NotAvailable class MiniMovie(QFrame): def __init__(self, weboob, backend, movie, parent=None): super(MiniMovie, self).__init__(parent) self.parent = parent self.ui = Ui_MiniMovie() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.movie = movie self.ui.titleLabel.setText(movie.original_title) self.ui.shortDescLabel.setText(movie.short_description) self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) self.ui.viewThumbnailButton.clicked.connect(self.gotThumbnail) if self.parent.parent.ui.showTCheck.isChecked(): self.gotThumbnail() @Slot() def gotThumbnail(self): if empty(self.movie.thumbnail_url) and self.movie.thumbnail_url != NotAvailable: self.backend.fill_movie(self.movie, ('thumbnail_url')) if not empty(self.movie.thumbnail_url): data = urllib.urlopen(self.movie.thumbnail_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToHeight(100,Qt.SmoothTransformation)) @Slot() def viewPressed(self): QApplication.setOverrideCursor(Qt.WaitCursor) movie = self.backend.get_movie(self.movie.id) if movie: self.parent.doAction('Movie "%s"' % movie.original_title, self.parent.displayMovie, [movie, self.backend]) @Slot() def newTabPressed(self): movie = self.backend.get_movie(self.movie.id) self.parent.parent.newTab(u'Movie "%s"' % movie.original_title, self.backend, movie=movie) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 2: self.gotThumbnail() elif event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qcineoob/miniperson.py000066400000000000000000000075511303450110500231300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from PyQt5.QtWidgets import QFrame, QApplication from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.applications.qcineoob.ui.miniperson_ui import Ui_MiniPerson from weboob.capabilities.base import empty, NotAvailable class MiniPerson(QFrame): def __init__(self, weboob, backend, person, parent=None): super(MiniPerson, self).__init__(parent) self.parent = parent self.ui = Ui_MiniPerson() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.person = person self.ui.nameLabel.setText('%s' % person.name) if not empty(person.short_description): if self.parent.ui.currentActionLabel.text().startswith('Casting'): self.ui.shortDescTitleLabel.setText(u'Role') self.ui.shortDescLabel.setText('%s' % person.short_description) else: self.ui.shortDescTitleLabel.hide() self.ui.shortDescLabel.hide() self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) self.ui.viewThumbnailButton.clicked.connect(self.gotThumbnail) if self.parent.parent.ui.showTCheck.isChecked(): self.gotThumbnail() @Slot() def gotThumbnail(self): if empty(self.person.thumbnail_url) and self.person.thumbnail_url != NotAvailable: self.backend.fill_person(self.person, ('thumbnail_url')) if not empty(self.person.thumbnail_url): data = urllib.urlopen(self.person.thumbnail_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToHeight(100,Qt.SmoothTransformation)) @Slot() def viewPressed(self): QApplication.setOverrideCursor(Qt.WaitCursor) person = self.backend.get_person(self.person.id) if person: self.parent.doAction(u'Person "%s"' % person.name, self.parent.displayPerson, [person, self.backend]) @Slot() def newTabPressed(self): person = self.backend.get_person(self.person.id) self.parent.parent.newTab(u'Person "%s"' % person.name, self.backend, person=person) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 2: self.gotThumbnail() elif event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qcineoob/minisubtitle.py000066400000000000000000000055531303450110500234550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QFrame from PyQt5.QtCore import pyqtSlot as Slot from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from weboob.applications.qcineoob.ui.minisubtitle_ui import Ui_MiniSubtitle from weboob.capabilities.base import empty class MiniSubtitle(QFrame): def __init__(self, weboob, backend, subtitle, parent=None): super(MiniSubtitle, self).__init__(parent) self.parent = parent self.ui = Ui_MiniSubtitle() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.subtitle = subtitle self.ui.nameLabel.setText(subtitle.name) if not empty(subtitle.nb_cd): self.ui.nbcdLabel.setText(u'%s' % subtitle.nb_cd) self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) @Slot() def viewPressed(self): subtitle = self.backend.get_subtitle(self.subtitle.id) if subtitle: self.parent.doAction('Subtitle "%s"' % subtitle.name, self.parent.displaySubtitle, [subtitle, self.backend]) @Slot() def newTabPressed(self): subtitle = self.backend.get_subtitle(self.subtitle.id) self.parent.parent.newTab(u'Subtitle "%s"' % subtitle.name, self.backend, subtitle=subtitle) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qcineoob/minitorrent.py000066400000000000000000000061001303450110500233040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QFrame from PyQt5.QtCore import pyqtSlot as Slot from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from weboob.applications.qcineoob.ui.minitorrent_ui import Ui_MiniTorrent from weboob.applications.weboorrents.weboorrents import sizeof_fmt from weboob.capabilities.base import empty class MiniTorrent(QFrame): def __init__(self, weboob, backend, torrent, parent=None): super(MiniTorrent, self).__init__(parent) self.parent = parent self.ui = Ui_MiniTorrent() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.torrent = torrent self.ui.nameLabel.setText(torrent.name) if not empty(torrent.seeders) and not empty(torrent.leechers): self.ui.seedLeechLabel.setText('%s/%s' % (torrent.seeders, torrent.leechers)) if not empty(torrent.size): self.ui.sizeLabel.setText(u'%s' % sizeof_fmt(torrent.size)) self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) @Slot() def viewPressed(self): torrent = self.backend.get_torrent(self.torrent.id) if torrent: self.parent.doAction('Torrent "%s"' % torrent.name, self.parent.displayTorrent, [torrent, self.backend]) @Slot() def newTabPressed(self): torrent = self.backend.get_torrent(self.torrent.id) self.parent.parent.newTab(u'Torrent "%s"' % torrent.name, self.backend, torrent=torrent) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qcineoob/movie.py000066400000000000000000000136041303450110500220600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from PyQt5.QtCore import Qt, pyqtSlot as Slot from PyQt5.QtGui import QImage, QPixmap from PyQt5.QtWidgets import QFrame, QMessageBox from weboob.applications.qcineoob.ui.movie_ui import Ui_Movie from weboob.capabilities.base import empty from weboob.applications.suboob.suboob import LANGUAGE_CONV class Movie(QFrame): def __init__(self, movie, backend, parent=None): super(Movie, self).__init__(parent) self.parent = parent self.ui = Ui_Movie() self.ui.setupUi(self) langs = sorted(LANGUAGE_CONV.keys()) for lang in langs: self.ui.langCombo.addItem(lang) self.ui.castingButton.clicked.connect(self.casting) self.ui.torrentButton.clicked.connect(self.searchTorrent) self.ui.subtitleButton.clicked.connect(self.searchSubtitle) self.ui.personsInCommonButton.clicked.connect(self.personsInCommon) self.movie = movie self.backend = backend self.ui.titleLabel.setText(movie.original_title) self.ui.durationLabel.setText(unicode(movie.duration)) self.gotThumbnail() self.putReleases() self.ui.idEdit.setText(u'%s@%s' % (movie.id, backend.name)) if not empty(movie.other_titles): self.ui.otherTitlesPlain.setPlainText('\n'.join(movie.other_titles)) else: self.ui.otherTitlesPlain.parent().hide() if not empty(movie.genres): genres = u'' for g in movie.genres: genres += '%s, ' % g genres = genres[:-2] self.ui.genresLabel.setText(genres) else: self.ui.genresLabel.parent().hide() if not empty(movie.release_date): self.ui.releaseDateLabel.setText(movie.release_date.strftime('%Y-%m-%d')) else: self.ui.releaseDateLabel.parent().hide() if not empty(movie.duration): self.ui.durationLabel.setText('%s min' % movie.duration) else: self.ui.durationLabel.parent().hide() if not empty(movie.pitch): self.ui.pitchPlain.setPlainText('%s' % movie.pitch) else: self.ui.pitchPlain.parent().hide() if not empty(movie.country): self.ui.countryLabel.setText('%s' % movie.country) else: self.ui.countryLabel.parent().hide() if not empty(movie.note): self.ui.noteLabel.setText('%s' % movie.note) else: self.ui.noteLabel.parent().hide() for role in movie.roles.keys(): self.ui.castingCombo.addItem('%s' % role) self.ui.verticalLayout.setAlignment(Qt.AlignTop) self.ui.verticalLayout_2.setAlignment(Qt.AlignTop) def putReleases(self): rel = self.backend.get_movie_releases(self.movie.id) if not empty(rel): self.ui.allReleasesPlain.setPlainText(rel) else: self.ui.allReleasesPlain.parent().hide() def gotThumbnail(self): if not empty(self.movie.thumbnail_url): data = urllib.urlopen(self.movie.thumbnail_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToWidth(220,Qt.SmoothTransformation)) @Slot() def searchSubtitle(self): tosearch = unicode(self.movie.original_title) lang = self.ui.langCombo.currentText() desc = 'Search subtitles for "%s" (lang:%s)' % (tosearch, lang) self.parent.doAction(desc, self.parent.searchSubtitleAction, [lang, tosearch]) @Slot() def searchTorrent(self): tosearch = self.movie.original_title if not empty(self.movie.release_date): tosearch += ' %s' % self.movie.release_date.year desc = 'Search torrents for "%s"' % tosearch self.parent.doAction(desc, self.parent.searchTorrentAction, [tosearch]) @Slot() def casting(self): role = None tosearch = self.ui.castingCombo.currentText() role_desc = '' if tosearch != 'all': role = tosearch role_desc = ' as %s' % role self.parent.doAction('Casting%s of movie "%s"' % (role_desc, self.movie.original_title), self.parent.castingAction, [self.backend.name, self.movie.id, role]) @Slot() def personsInCommon(self): my_id = self.movie.id my_title = self.movie.original_title other_id = self.ui.personsInCommonEdit.text().split('@')[0] other_movie = self.backend.get_movie(other_id) if other_id == self.movie.id: QMessageBox.critical(None, self.tr('"Persons in common" error'), self.tr('Nice try\nThe movies must be different'), QMessageBox.Ok) elif not other_movie: QMessageBox.critical(None, self.tr('"Persons in common" error'), self.tr('Movie not found: %s' % other_id), QMessageBox.Ok) else: other_title = other_movie.original_title desc = 'Persons in common %s, %s'%(my_title, other_title) self.parent.doAction(desc, self.parent.personsInCommonAction, [self.backend.name, my_id, other_id]) weboob-1.2/weboob/applications/qcineoob/person.py000066400000000000000000000110371303450110500222450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from PyQt5.QtCore import pyqtSlot as Slot, Qt from PyQt5.QtGui import QImage, QPixmap from PyQt5.QtWidgets import QFrame, QApplication, QMessageBox from weboob.applications.qcineoob.ui.person_ui import Ui_Person from weboob.capabilities.base import empty class Person(QFrame): def __init__(self, person, backend, parent=None): super(Person, self).__init__(parent) self.parent = parent self.ui = Ui_Person() self.ui.setupUi(self) self.ui.filmographyButton.clicked.connect(self.filmography) self.ui.biographyButton.clicked.connect(self.biography) self.ui.moviesInCommonButton.clicked.connect(self.moviesInCommon) self.person = person self.backend = backend self.gotThumbnail() self.ui.nameLabel.setText(person.name) self.ui.idEdit.setText(u'%s@%s' % (person.id, backend.name)) if not empty(person.real_name): self.ui.realNameLabel.setText('%s' % person.real_name) else: self.ui.realNameLabel.parent().hide() if not empty(person.birth_place): self.ui.birthPlaceLabel.setText('%s' % person.birth_place) else: self.ui.birthPlaceLabel.parent().hide() if not empty(person.birth_date): self.ui.birthDateLabel.setText(person.birth_date.strftime('%Y-%m-%d')) else: self.ui.birthDateLabel.parent().hide() if not empty(person.death_date): self.ui.deathDateLabel.setText(person.death_date.strftime('%Y-%m-%d')) else: self.ui.deathDateLabel.parent().hide() self.ui.shortBioPlain.setPlainText('%s' % person.short_biography) for role in person.roles.keys(): self.ui.filmographyCombo.addItem(role) self.ui.verticalLayout_2.setAlignment(Qt.AlignTop) def gotThumbnail(self): if not empty(self.person.thumbnail_url): data = urllib.urlopen(self.person.thumbnail_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToWidth(220,Qt.SmoothTransformation)) @Slot() def filmography(self): role = None tosearch = self.ui.filmographyCombo.currentText() role_desc = '' if tosearch != 'all': role = tosearch role_desc = ' as %s' % role self.parent.doAction('Filmography of "%s"%s' % (self.person.name, role_desc), self.parent.filmographyAction, [self.backend.name, self.person.id, role]) @Slot() def biography(self): QApplication.setOverrideCursor(Qt.WaitCursor) self.backend.fill_person(self.person, 'biography') bio = self.person.biography self.ui.shortBioPlain.setPlainText(u'%s' % bio) self.ui.biographyLabel.setText('Full biography:') self.ui.biographyButton.hide() QApplication.restoreOverrideCursor() @Slot() def moviesInCommon(self): my_id = self.person.id my_name = self.person.name other_id = self.ui.moviesInCommonEdit.text().split('@')[0] other_person = self.backend.get_person(other_id) if other_id == self.person.id: QMessageBox.critical(None, self.tr('"Moviess in common" error'), self.tr('Nice try\nThe persons must be different'), QMessageBox.Ok) elif not other_person: QMessageBox.critical(None, self.tr('"Movies in common" error'), self.tr('Person not found: %s' % other_id), QMessageBox.Ok) else: other_name = other_person.name desc = 'Movies in common %s, %s'%(my_name, other_name) self.parent.doAction(desc, self.parent.moviesInCommonAction, [self.backend.name, my_id, other_id]) weboob-1.2/weboob/applications/qcineoob/qcineoob.py000066400000000000000000000033501303450110500225350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.cinema import CapCinema from weboob.capabilities.torrent import CapTorrent from weboob.capabilities.subtitle import CapSubtitle from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QCineoob(QtApplication): APPNAME = 'qcineoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Julien Veyssier' DESCRIPTION = "Qt application allowing to search movies, people, torrent and subtitles." SHORT_DESCRIPTION = "search movies, people, torrent and subtitles" CAPS = CapCinema, CapTorrent, CapSubtitle CONFIG = {'settings': {'backend': '', 'maxresultsnumber': '10', 'showthumbnails': '0' } } def main(self, argv): self.load_backends([CapCinema, CapTorrent, CapSubtitle]) self.load_config() self.main_window = MainWindow(self.config, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qcineoob/subtitle.py000066400000000000000000000065571303450110500226050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from PyQt5.QtCore import Qt, pyqtSlot as Slot from PyQt5.QtWidgets import QFrame, QFileDialog from weboob.applications.qcineoob.ui.subtitle_ui import Ui_Subtitle from weboob.capabilities.base import empty class Subtitle(QFrame): def __init__(self, subtitle, backend, parent=None): super(Subtitle, self).__init__(parent) self.parent = parent self.backend = backend self.ui = Ui_Subtitle() self.ui.setupUi(self) self.ui.downloadButton.clicked.connect(self.download) self.subtitle = subtitle self.ui.idEdit.setText(u'%s@%s' % (subtitle.id, backend.name)) self.ui.nameLabel.setText(u'%s' % subtitle.name) if not empty(subtitle.nb_cd): self.ui.nbcdLabel.setText(u'%s' % subtitle.nb_cd) else: self.ui.nbcdLabel.parent().hide() if not empty(subtitle.language): self.ui.langLabel.setText(u'%s' % subtitle.language) else: self.ui.langLabel.parent().hide() if not empty(subtitle.description): self.ui.descriptionPlain.setPlainText(u'%s' % subtitle.description) else: self.ui.descriptionPlain.parent().hide() if not empty(subtitle.url): self.ui.urlEdit.setText(u'%s' % subtitle.url) else: self.ui.downloadButton.setDisabled(True) self.ui.downloadButton.setText('Impossible to download this subtitle') self.ui.verticalLayout.setAlignment(Qt.AlignTop) @Slot() def download(self): if not empty(self.subtitle.url): if self.subtitle.url.endswith('.rar'): ext = '.rar' elif self.subtitle.url.endswith('.srt'): ext = '.srt' else: ext = '.zip' fileDial = QFileDialog(self, 'Save "%s" subtitle file' % self.subtitle.name, '%s%s' % (self.subtitle.name, ext), 'all files (*)') fileDial.setAcceptMode(QFileDialog.AcceptSave) fileDial.setLabelText(QFileDialog.Accept, 'Save subtitle file') fileDial.setLabelText(QFileDialog.FileName, 'Subtitle file name') ok = (fileDial.exec_() == 1) if not ok: return result = fileDial.selectedFiles() if len(result) > 0: dest = result[0] data = self.backend.get_subtitle_file(self.subtitle.id) try: with open(dest, 'w') as f: f.write(data) except IOError as e: print('Unable to write subtitle file in "%s": %s' % (dest, e), file=self.stderr) return 1 return weboob-1.2/weboob/applications/qcineoob/torrent.py000066400000000000000000000072761303450110500224460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from PyQt5.QtCore import Qt, pyqtSlot as Slot from PyQt5.QtWidgets import QFrame, QFileDialog from weboob.applications.qcineoob.ui.torrent_ui import Ui_Torrent from weboob.applications.weboorrents.weboorrents import sizeof_fmt from weboob.capabilities.base import empty class Torrent(QFrame): def __init__(self, torrent, backend, parent=None): super(Torrent, self).__init__(parent) self.parent = parent self.backend = backend self.ui = Ui_Torrent() self.ui.setupUi(self) self.ui.downloadButton.clicked.connect(self.download) self.torrent = torrent self.ui.idEdit.setText(u'%s@%s' % (torrent.id, backend.name)) self.ui.nameLabel.setText(u'%s' % torrent.name) if not empty(torrent.url): self.ui.urlEdit.setText(u'%s' % torrent.url) else: self.ui.urlFrame.hide() self.ui.downloadButton.setDisabled(True) if not empty(torrent.magnet): self.ui.downloadButton.setText(u'Download not available\nbut magnet link provided') self.ui.downloadButton.setToolTip(u'Use the magnet link') if not empty(torrent.description): self.ui.descriptionPlain.setText(u'%s' % torrent.description) else: self.ui.descriptionPlain.parent().hide() if not empty(torrent.files): files = u'' for f in torrent.files: files += '%s\n' % f self.ui.filesPlain.setText(u'%s' % files) else: self.ui.filesPlain.parent().hide() if not empty(torrent.magnet): self.ui.magnetEdit.setText(u'%s' % torrent.magnet) else: self.ui.magnetFrame.hide() if not empty(torrent.seeders) and not empty(torrent.leechers): self.ui.seedLeechLabel.setText(u'%s/%s' % (torrent.seeders, torrent.leechers)) if not empty(torrent.size): self.ui.sizeLabel.setText(u'%s' % sizeof_fmt(torrent.size)) self.ui.verticalLayout.setAlignment(Qt.AlignTop) @Slot() def download(self): fileDial = QFileDialog(self, 'Save "%s" torrent file' % self.torrent.name, '%s.torrent' % self.torrent.name, 'Torrent file (*.torrent);;all files (*)') fileDial.setAcceptMode(QFileDialog.AcceptSave) fileDial.setLabelText(QFileDialog.Accept, 'Save torrent file') fileDial.setLabelText(QFileDialog.FileName, 'Torrent file name') ok = (fileDial.exec_() == 1) if not ok: return result = fileDial.selectedFiles() if len(result) > 0: dest = result[0] data = self.backend.get_torrent_file(self.torrent.id) try: with open(unicode(dest), 'w') as f: f.write(data) except IOError as e: print('Unable to write .torrent in "%s": %s' % (dest, e), file=self.stderr) return 1 return weboob-1.2/weboob/applications/qcineoob/ui/000077500000000000000000000000001303450110500210005ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcineoob/ui/Makefile000066400000000000000000000002651303450110500224430ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qcineoob/ui/__init__.py000066400000000000000000000000001303450110500230770ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcineoob/ui/main_window.ui000066400000000000000000000137151303450110500236610ustar00rootroot00000000000000 MainWindow 0 0 748 463 QCineoob 4 QFrame::StyledPanel QFrame::Raised 0 0 Search: 45 16777215 75 true background-color: qlineargradient(spread:pad, x1:1, y1:1, x2:1, y2:1, stop:0 rgba(255, 0, 0, 255), stop:0.479904 rgba(255, 0, 0, 255), stop:0.522685 rgba(255, 255, 255, 255), stop:1 rgba(255, 255, 255, 255)); stop movie person torrent subtitle 8 language <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> max results <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> -1 true true Show thumbnails Search by id: movie person torrent subtitle 0 0 748 23 toolBar TopToolBarArea false Backends weboob-1.2/weboob/applications/qcineoob/ui/minimovie.ui000066400000000000000000000127361303450110500233440ustar00rootroot00000000000000 MiniMovie 0 0 563 138 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 75 true Title 75 true Short description: 0 0 50 true false TextLabel true 75 true Where TextLabel false 120 20 View 120 20 View in new tab 120 20 View thumbnail Qt::Horizontal 40 20 16777215 20 PushButton true Qt::Horizontal 40 20 weboob-1.2/weboob/applications/qcineoob/ui/miniperson.ui000066400000000000000000000130731303450110500235260ustar00rootroot00000000000000 MiniPerson 0 0 594 136 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 75 true Name 0 0 50 true false TextLabel true 75 true Short description TextLabel false 75 true Where 0 120 20 View 120 20 View in new tab 120 20 View thumbnail Qt::Horizontal 40 20 16777215 20 PushButton true Qt::Horizontal 40 20 weboob-1.2/weboob/applications/qcineoob/ui/minisubtitle.ui000066400000000000000000000103051303450110500240460ustar00rootroot00000000000000 MiniSubtitle 0 0 464 136 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 QFormLayout::ExpandingFieldsGrow 75 true Name 0 0 50 true false TextLabel true 75 true Nb CD TextLabel 75 true Where 16777215 20 View in new tab 16777215 20 View 16777215 20 PushButton true weboob-1.2/weboob/applications/qcineoob/ui/minitorrent.ui000066400000000000000000000111771303450110500237200ustar00rootroot00000000000000 MiniTorrent 0 0 464 138 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 QFormLayout::ExpandingFieldsGrow 75 true Name 0 0 50 true false TextLabel 75 true Seed/leech: TextLabel 75 true Size: TextLabel 75 true Where 16777215 20 View 16777215 20 View in new tab 16777215 20 PushButton true weboob-1.2/weboob/applications/qcineoob/ui/movie.ui000066400000000000000000000430301303450110500224560ustar00rootroot00000000000000 Movie 0 0 655 733 0 0 10000 10000 Frame QFrame::StyledPanel QFrame::Raised 16777215 3000 QFrame::NoFrame QFrame::Raised QFrame::NoFrame QFrame::Raised 0 0 view casting all QFrame::NoFrame QFrame::Raised 0 0 search torrents QFrame::NoFrame QFrame::Raised 0 0 search subtitles QFrame::StyledPanel QFrame::Raised Search persons in common with 0 0 Movie ID : 0 0 16777215 9000 QFrame::NoFrame QFrame::Raised 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Title: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Duration: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Release date: 16777215 150 QFrame::StyledPanel QFrame::Raised 75 true Pitch: 0 0 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Country: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Genres: true 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Note: 16777215 200 QFrame::StyledPanel QFrame::Raised 75 true All release dates: Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse 0 0 16777215 200 QFrame::StyledPanel QFrame::Raised 100 16777215 75 true Other titles: Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse weboob-1.2/weboob/applications/qcineoob/ui/person.ui000066400000000000000000000273031303450110500226520ustar00rootroot00000000000000 Person 0 0 857 629 Frame QFrame::StyledPanel QFrame::Raised QFrame::NoFrame QFrame::Raised QFrame::NoFrame QFrame::Raised view filmography as all QFrame::StyledPanel QFrame::Raised Search movies in common with 0 0 Person ID : 0 0 QFrame::NoFrame QFrame::Raised 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true QFrame::StyledPanel QFrame::Raised 75 true name: QFrame::StyledPanel QFrame::Raised 75 true Birth date: QFrame::StyledPanel QFrame::Raised 75 true Real Name: QFrame::StyledPanel QFrame::Raised 75 true Birth place: QFrame::StyledPanel QFrame::Raised 75 true Death date: QFrame::StyledPanel QFrame::Raised QFrame::NoFrame QFrame::Raised 75 true Short biography: view full biography 0 0 16777215 250 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse weboob-1.2/weboob/applications/qcineoob/ui/result.ui000066400000000000000000000111731303450110500226600ustar00rootroot00000000000000 Result 0 0 645 733 0 0 10000 10000 Frame QFrame::StyledPanel QFrame::Raised QFrame::NoFrame QFrame::Raised 0 0 Qt::Horizontal 40 20 60 20 <<back 75 true Qt::AlignCenter true Qt::Horizontal 40 20 true 0 0 605 667 Qt::Vertical 20 40 true 0 0 605 667 weboob-1.2/weboob/applications/qcineoob/ui/subtitle.ui000066400000000000000000000204371303450110500232000ustar00rootroot00000000000000 Subtitle 0 0 629 552 0 0 2000 600 Frame QFrame::StyledPanel QFrame::Raised 16777215 600 QFrame::NoFrame QFrame::Raised 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Name: true 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Nb CD: 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Lang: 16777215 200 QFrame::StyledPanel QFrame::Raised 75 true Description: 16777215 120 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Url: Download weboob-1.2/weboob/applications/qcineoob/ui/torrent.ui000066400000000000000000000245631303450110500230460ustar00rootroot00000000000000 Torrent 0 0 629 571 0 0 10000 10000 Frame QFrame::StyledPanel QFrame::Raised 16777215 10000 QFrame::NoFrame QFrame::Raised 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Name: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Seed/leech: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Size: 0 150 16777215 500 QFrame::StyledPanel QFrame::Raised 75 true Description: 0 120 16777215 200 QFrame::StyledPanel QFrame::Raised 75 true Files: 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Url: 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Magnet: Download weboob-1.2/weboob/applications/qcookboob/000077500000000000000000000000001303450110500205425ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcookboob/__init__.py000066400000000000000000000000721303450110500226520ustar00rootroot00000000000000from .qcookboob import QCookboob __all__ = ['QCookboob'] weboob-1.2/weboob/applications/qcookboob/main_window.py000066400000000000000000000263301303450110500234330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os from PyQt5.QtCore import pyqtSlot as Slot, Qt from PyQt5.QtGui import QKeySequence from PyQt5.QtWidgets import QApplication, QFrame, QShortcut from weboob.capabilities.recipe import CapRecipe from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.models import BackendListModel from weboob.tools.application.qt5.search_history import HistoryCompleter from weboob.applications.qcookboob.ui.main_window_ui import Ui_MainWindow from weboob.applications.qcookboob.ui.result_ui import Ui_Result from .minirecipe import MiniRecipe from .recipe import Recipe MAX_TAB_TEXT_LENGTH=30 class Result(QFrame): def __init__(self, weboob, app, parent=None): super(Result, self).__init__(parent) self.ui = Ui_Result() self.ui.setupUi(self) self.parent = parent self.weboob = weboob self.app = app self.minis = [] self.current_info_widget = None # action history is composed by the last action and the action list # An action is a function, a list of arguments and a description string self.action_history = {'last_action': None, 'action_list': []} self.ui.backButton.clicked.connect(self.doBack) self.ui.backButton.setShortcut(QKeySequence('Alt+Left')) self.ui.backButton.hide() def doAction(self, description, fun, args): ''' Call fun with args as arguments and save it in the action history ''' self.ui.currentActionLabel.setText(description) if self.action_history['last_action'] is not None: self.action_history['action_list'].append(self.action_history['last_action']) self.ui.backButton.setToolTip('%s (Alt+Left)'%self.action_history['last_action']['description']) self.ui.backButton.show() self.action_history['last_action'] = {'function': fun, 'args': args, 'description': description} # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = description if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, description) return fun(*args) @Slot() def doBack(self): ''' Go back in action history Basically call previous function and update history ''' if len(self.action_history['action_list']) > 0: todo = self.action_history['action_list'].pop() self.ui.currentActionLabel.setText(todo['description']) self.action_history['last_action'] = todo if len(self.action_history['action_list']) == 0: self.ui.backButton.hide() else: self.ui.backButton.setToolTip(self.action_history['action_list'][-1]['description']) # manage tab text mytabindex = self.parent.ui.resultsTab.indexOf(self) tabtxt = todo['description'] if len(tabtxt) > MAX_TAB_TEXT_LENGTH: tabtxt = '%s...'%tabtxt[:MAX_TAB_TEXT_LENGTH] self.parent.ui.resultsTab.setTabText(mytabindex, tabtxt) self.parent.ui.resultsTab.setTabToolTip(mytabindex, todo['description']) return todo['function'](*todo['args']) def processFinished(self): self.parent.ui.searchEdit.setEnabled(True) QApplication.restoreOverrideCursor() self.process = None self.parent.ui.stopButton.hide() @Slot() def stopProcess(self): if self.process is not None: self.process.stop() def searchRecipe(self,pattern): if not pattern: return self.doAction(u'Search recipe "%s"' % pattern, self.searchRecipeAction, [pattern]) def searchRecipeAction(self, pattern): self.ui.stackedWidget.setCurrentWidget(self.ui.list_page) for mini in self.minis: self.ui.list_content.layout().removeWidget(mini) mini.hide() mini.deleteLater() self.minis = [] self.parent.ui.searchEdit.setEnabled(False) QApplication.setOverrideCursor(Qt.WaitCursor) backend_name = self.parent.ui.backendEdit.itemData(self.parent.ui.backendEdit.currentIndex()) self.process = QtDo(self.weboob, self.addRecipe, fb=self.processFinished) self.process.do(self.app._do_complete, self.parent.getCount(), ('title'), 'iter_recipes', pattern, backends=backend_name, caps=CapRecipe) self.parent.ui.stopButton.show() def addRecipe(self, recipe): minirecipe = MiniRecipe(self.weboob, self.weboob[recipe.backend], recipe, self) self.ui.list_content.layout().insertWidget(self.ui.list_content.layout().count()-1,minirecipe) self.minis.append(minirecipe) def displayRecipe(self, recipe, backend): self.ui.stackedWidget.setCurrentWidget(self.ui.info_page) if self.current_info_widget is not None: self.ui.info_content.layout().removeWidget(self.current_info_widget) self.current_info_widget.hide() self.current_info_widget.deleteLater() wrecipe = Recipe(recipe, backend, self) self.ui.info_content.layout().addWidget(wrecipe) self.current_info_widget = wrecipe QApplication.restoreOverrideCursor() def searchId(self, id): QApplication.setOverrideCursor(Qt.WaitCursor) if '@' in id: backend_name = id.split('@')[1] id = id.split('@')[0] else: backend_name = None for backend in self.weboob.iter_backends(): if (backend_name and backend.name == backend_name) or not backend_name: recipe = backend.get_recipe(id) if recipe: self.doAction('Recipe "%s"' % recipe.title, self.displayRecipe, [recipe, backend]) QApplication.restoreOverrideCursor() class MainWindow(QtMainWindow): def __init__(self, config, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.app = app self.minis = [] self.current_info_widget = None # search history is a list of patterns which have been searched history_path = os.path.join(self.weboob.workdir, 'qcookboob_history') qc = HistoryCompleter(history_path, self) qc.load() qc.setCaseSensitivity(Qt.CaseInsensitive) self.ui.searchEdit.setCompleter(qc) self.ui.searchEdit.returnPressed.connect(self.search) self.ui.idEdit.returnPressed.connect(self.searchId) count = self.config.get('settings', 'maxresultsnumber') self.ui.countSpin.setValue(int(count)) showT = self.config.get('settings', 'showthumbnails') self.ui.showTCheck.setChecked(showT == '1') self.ui.stopButton.hide() self.ui.actionBackends.triggered.connect(self.backendsConfig) q = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_Q), self) q.activated.connect(self.close) n = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageDown), self) n.activated.connect(self.nextTab) p = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_PageUp), self) p.activated.connect(self.prevTab) w = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_W), self) w.activated.connect(self.closeCurrentTab) l = QShortcut(QKeySequence(Qt.CTRL + Qt.Key_L), self) l.activated.connect(self.ui.searchEdit.setFocus) l.activated.connect(self.ui.searchEdit.selectAll) self.ui.resultsTab.tabCloseRequested.connect(self.closeTab) self.loadBackendsList() if self.ui.backendEdit.count() == 0: self.backendsConfig() @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapRecipe, ), self) if bckndcfg.run(): self.loadBackendsList() def loadBackendsList(self): model = BackendListModel(self.weboob) model.addBackends() self.ui.backendEdit.setModel(model) current_backend = self.config.get('settings', 'backend') idx = self.ui.backendEdit.findData(current_backend) if idx >= 0: self.ui.backendEdit.setCurrentIndex(idx) if self.ui.backendEdit.count() == 0: self.ui.searchEdit.setEnabled(False) else: self.ui.searchEdit.setEnabled(True) def getCount(self): num = self.ui.countSpin.value() if num == 0: return None else: return num @Slot(int) def closeTab(self, index): if self.ui.resultsTab.widget(index) != 0: self.ui.resultsTab.removeTab(index) @Slot() def closeCurrentTab(self): self.closeTab(self.ui.resultsTab.currentIndex()) @Slot() def prevTab(self): index = self.ui.resultsTab.currentIndex() - 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) @Slot() def nextTab(self): index = self.ui.resultsTab.currentIndex() + 1 size = self.ui.resultsTab.count() if size != 0: self.ui.resultsTab.setCurrentIndex(index % size) def newTab(self, txt, backend, recipe=None): id = '' if recipe is not None: id = recipe.id new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, txt) new_res.searchId('%s@%s'%(id,backend.NAME)) @Slot() def search(self): pattern = self.ui.searchEdit.text() self.ui.searchEdit.completer().addString(pattern) new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, pattern) self.ui.resultsTab.setCurrentWidget(new_res) new_res.searchRecipe(pattern) self.ui.stopButton.clicked.connect(new_res.stopProcess) @Slot() def searchId(self): id = self.ui.idEdit.text() new_res = Result(self.weboob, self.app, self) self.ui.resultsTab.addTab(new_res, id) self.ui.resultsTab.setCurrentWidget(new_res) new_res.searchId(id) def closeEvent(self, ev): self.config.set('settings', 'backend', self.ui.backendEdit.itemData( self.ui.backendEdit.currentIndex())) self.ui.searchEdit.completer().save() self.config.set('settings', 'maxresultsnumber', self.ui.countSpin.value()) self.config.save() ev.accept() weboob-1.2/weboob/applications/qcookboob/minirecipe.py000066400000000000000000000072371303450110500232510ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import urllib from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache from PyQt5.QtWidgets import QFrame, QApplication from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.applications.qcookboob.ui.minirecipe_ui import Ui_MiniRecipe from weboob.capabilities.base import empty class MiniRecipe(QFrame): def __init__(self, weboob, backend, recipe, parent=None): super(MiniRecipe, self).__init__(parent) self.parent = parent self.ui = Ui_MiniRecipe() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.recipe = recipe self.ui.titleLabel.setText(recipe.title) if not empty(recipe.short_description): if len(recipe.short_description) > 300: self.ui.shortDescLabel.setText('%s [...]'%recipe.short_description[:300]) else: self.ui.shortDescLabel.setText(recipe.short_description) else: self.ui.shortDescLabel.setText('') self.ui.backendButton.setText(backend.name) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) self.ui.backendButton.setIcon(QIcon(pixmap)) self.ui.newTabButton.clicked.connect(self.newTabPressed) self.ui.viewButton.clicked.connect(self.viewPressed) self.ui.viewThumbnailButton.clicked.connect(self.gotThumbnail) if self.parent.parent.ui.showTCheck.isChecked(): self.gotThumbnail() @Slot() def gotThumbnail(self): if not empty(self.recipe.thumbnail_url): data = urllib.urlopen(self.recipe.thumbnail_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToHeight(100,Qt.SmoothTransformation)) @Slot() def viewPressed(self): QApplication.setOverrideCursor(Qt.WaitCursor) recipe = self.backend.get_recipe(self.recipe.id) if recipe: self.parent.doAction('Recipe "%s"' % recipe.title, self.parent.displayRecipe, [recipe, self.backend]) @Slot() def newTabPressed(self): recipe = self.backend.get_recipe(self.recipe.id) self.parent.parent.newTab(u'Recipe "%s"' % recipe.title, self.backend, recipe=recipe) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) if event.button() == 2: self.gotThumbnail() elif event.button() == 4: self.newTabPressed() else: self.viewPressed() weboob-1.2/weboob/applications/qcookboob/qcookboob.py000066400000000000000000000027421303450110500230770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.recipe import CapRecipe from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QCookboob(QtApplication): APPNAME = 'qcookboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-2014 Julien Veyssier' DESCRIPTION = "Qt application allowing to search recipes." SHORT_DESCRIPTION = "search recipes" CAPS = CapRecipe CONFIG = {'settings': {'backend': '', 'maxresultsnumber': '10' } } def main(self, argv): self.load_backends([CapRecipe]) self.load_config() self.main_window = MainWindow(self.config, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qcookboob/recipe.py000066400000000000000000000106441303450110500223700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import urllib import codecs from PyQt5.QtCore import Qt, pyqtSlot as Slot from PyQt5.QtGui import QImage, QPixmap from PyQt5.QtWidgets import QFrame, QFileDialog from weboob.applications.qcookboob.ui.recipe_ui import Ui_Recipe from weboob.capabilities.base import empty class Recipe(QFrame): def __init__(self, recipe, backend, parent=None): super(Recipe, self).__init__(parent) self.parent = parent self.ui = Ui_Recipe() self.ui.setupUi(self) self.ui.exportButton.clicked.connect(self.export) self.recipe = recipe self.backend = backend self.gotThumbnail() self.ui.idEdit.setText(u'%s@%s' % (recipe.id, backend.name)) if not empty(recipe.title): self.ui.titleLabel.setText(recipe.title) if not empty(recipe.nb_person): nbstr = '-'.join(str(num) for num in recipe.nb_person) self.ui.nbPersonLabel.setText(nbstr) else: self.ui.nbPersonLabel.parent().hide() if not empty(recipe.preparation_time): self.ui.prepTimeLabel.setText('%s min' % recipe.preparation_time) else: self.ui.prepTimeLabel.parent().hide() if not empty(recipe.cooking_time): self.ui.cookingTimeLabel.setText('%s min' % recipe.cooking_time) else: self.ui.cookingTimeLabel.parent().hide() if not empty(recipe.ingredients): txt = u'' for ing in recipe.ingredients: txt += '* %s\n' % ing self.ui.ingredientsPlain.setPlainText('%s' % txt) else: self.ui.ingredientsPlain.parent().hide() if not empty(recipe.author): self.ui.authorLabel.setText('%s' % recipe.author) else: self.ui.authorLabel.parent().hide() if not empty(recipe.instructions): self.ui.instructionsPlain.setPlainText('%s' % recipe.instructions) else: self.ui.instructionsPlain.parent().hide() if not empty(recipe.comments): txt = u'' for com in recipe.comments: txt += '* %s\n' % com self.ui.commentsPlain.setPlainText('%s' % txt) else: self.ui.commentsPlain.parent().hide() self.ui.verticalLayout.setAlignment(Qt.AlignTop) self.ui.verticalLayout_2.setAlignment(Qt.AlignTop) def gotThumbnail(self): if self.recipe.picture_url and not empty(self.recipe.picture_url): data = urllib.urlopen(self.recipe.picture_url).read() img = QImage.fromData(data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img).scaledToWidth(250, Qt.SmoothTransformation)) @Slot() def export(self): fileDial = QFileDialog(self, 'Export "%s" recipe' % self.recipe.title, '%s.kreml' % self.recipe.title.replace('/', ','), 'Krecipe file (*.kreml);;all files (*)') fileDial.setAcceptMode(QFileDialog.AcceptSave) fileDial.setLabelText(QFileDialog.Accept, 'Export recipe') fileDial.setLabelText(QFileDialog.FileName, 'Recipe file name') ok = (fileDial.exec_() == 1) if not ok: return result = fileDial.selectedFiles() if len(result) > 0: dest = result[0] if not dest.endswith('.kreml'): dest += '.kreml' data = self.recipe.toKrecipesXml(author=self.backend.name) try: with codecs.open(dest, 'w', 'utf-8') as f: f.write(data) except IOError as e: print('Unable to write Krecipe file in "%s": %s' % (dest, e), file=self.stderr) return 1 return weboob-1.2/weboob/applications/qcookboob/ui/000077500000000000000000000000001303450110500211575ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcookboob/ui/Makefile000066400000000000000000000002651303450110500226220ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qcookboob/ui/__init__.py000066400000000000000000000000001303450110500232560ustar00rootroot00000000000000weboob-1.2/weboob/applications/qcookboob/ui/main_window.ui000066400000000000000000000104471303450110500240370ustar00rootroot00000000000000 MainWindow 0 0 748 463 QCookboob 4 QFrame::StyledPanel QFrame::Raised 0 0 Search: 35 16777215 background-color: qlineargradient(spread:pad, x1:1, y1:1, x2:1, y2:1, stop:0 rgba(255, 0, 0, 255), stop:0.479904 rgba(255, 0, 0, 255), stop:0.522685 rgba(255, 255, 255, 255), stop:1 rgba(255, 255, 255, 255)); stop <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> max results <html><head/><body><p>Maximum results by backend</p><p>0 = no limit</p></body></html> -1 true true Show thumbnails search by ID: 0 0 748 27 toolBar TopToolBarArea false Backends weboob-1.2/weboob/applications/qcookboob/ui/minirecipe.ui000066400000000000000000000127371303450110500236540ustar00rootroot00000000000000 MiniRecipe 0 0 578 136 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 TextLabel true 0 0 50 true false TextLabel true 75 true Short description: 75 true Where 75 true Title 120 20 View 120 20 View in new tab 120 20 View thumbnail Qt::Horizontal 40 20 16777215 20 PushButton true Qt::Horizontal 40 20 weboob-1.2/weboob/applications/qcookboob/ui/recipe.ui000066400000000000000000000325341303450110500227740ustar00rootroot00000000000000 Recipe 0 0 645 855 0 0 8000 5000 Frame QFrame::StyledPanel QFrame::Raised 16777215 3000 QFrame::NoFrame QFrame::Raised 16777215 4000 QFrame::NoFrame QFrame::Raised Export to KRecipesML file 16777215 40 QFrame::StyledPanel QFrame::Raised 3 3 75 true Id: Qt::AlignCenter true 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Title: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Author: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Number of people: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Preparation time: 16777215 35 QFrame::StyledPanel QFrame::Raised 75 true Cook time: 0 0 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Ingredients: 0 200 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse 0 0 16777215 1000 QFrame::StyledPanel QFrame::Raised 100 16777215 75 true Instructions: 0 200 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse 0 0 16777215 1000 QFrame::StyledPanel QFrame::Raised 75 true Comments: 0 100 Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse weboob-1.2/weboob/applications/qcookboob/ui/result.ui000066400000000000000000000111731303450110500230370ustar00rootroot00000000000000 Result 0 0 645 733 0 0 10000 10000 Frame QFrame::StyledPanel QFrame::Raised QFrame::NoFrame QFrame::Raised 0 0 Qt::Horizontal 40 20 60 20 <<back 75 true Qt::AlignCenter true Qt::Horizontal 40 20 true 0 0 605 667 Qt::Vertical 20 40 true 0 0 605 667 weboob-1.2/weboob/applications/qflatboob/000077500000000000000000000000001303450110500205355ustar00rootroot00000000000000weboob-1.2/weboob/applications/qflatboob/__init__.py000066400000000000000000000000721303450110500226450ustar00rootroot00000000000000from .qflatboob import QFlatBoob __all__ = ['QFlatBoob'] weboob-1.2/weboob/applications/qflatboob/main_window.py000066400000000000000000000367751303450110500234440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtGui import QImage, QPixmap, QIcon, QBrush, QColor from PyQt5.QtWidgets import QLabel, QListWidgetItem from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.tools.application.qt5 import QtMainWindow, QtDo, HTMLDelegate from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.capabilities.housing import CapHousing, Query, City from weboob.capabilities.base import NotLoaded, NotAvailable, empty from .ui.main_window_ui import Ui_MainWindow from .query import QueryDialog class HousingListWidgetItem(QListWidgetItem): def __init__(self, housing, *args, **kwargs): super(HousingListWidgetItem, self).__init__(*args, **kwargs) self.housing = housing self.read = True def __lt__(self, other): return '%s%s' % (self.read, self.housing.price_per_meter) < \ '%s%s' % (other.read, other.housing.price_per_meter) def setAttrs(self, storage): text = u'

%s

' % self.housing.title _area = u'%.0fm²' % self.housing.area if self.housing.area else self.housing.area text += u'%s — %s — %s%s — %.0f %s/m2 (%s)' % ( self.housing.date.strftime('%Y-%m-%d') if self.housing.date else 'Unknown', _area, self.housing.cost, self.housing.currency, self.housing.price_per_meter, self.housing.currency, self.housing.backend) text += u'
%s' % self.housing.text.strip() text += u'
%s' % storage.get('notes', self.housing.fullid, default='').strip().replace('\n', '
') self.setText(text) if self.housing.fullid not in storage.get('read'): self.setBackground(QBrush(QColor(200, 200, 255))) self.read = False elif self.housing.fullid in storage.get('bookmarks'): self.setBackground(QBrush(QColor(255, 200, 200))) elif self.background().color() != QColor(0, 0, 0): self.setBackground(QBrush()) class MainWindow(QtMainWindow): def __init__(self, config, storage, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.storage = storage self.weboob = weboob self.app = app self.process = None self.housing = None self.displayed_photo_idx = 0 self.process_photo = {} self.process_bookmarks = {} self.ui.housingsList.setItemDelegate(HTMLDelegate()) self.ui.housingFrame.hide() self.ui.actionBackends.triggered.connect(self.backendsConfig) self.ui.queriesList.currentIndexChanged.connect(self.queryChanged) self.ui.addQueryButton.clicked.connect(self.addQuery) self.ui.editQueryButton.clicked.connect(self.editQuery) self.ui.removeQueryButton.clicked.connect(self.removeQuery) self.ui.bookmarksButton.clicked.connect(self.displayBookmarks) self.ui.housingsList.currentItemChanged.connect(self.housingSelected) self.ui.previousButton.clicked.connect(self.previousClicked) self.ui.nextButton.clicked.connect(self.nextClicked) self.ui.bookmark.stateChanged.connect(self.bookmarkChanged) self.reloadQueriesList() self.refreshHousingsList() if self.weboob.count_backends() == 0: self.backendsConfig() if len(self.config.get('queries')) == 0: self.addQuery() def closeEvent(self, event): self.setHousing(None) QtMainWindow.closeEvent(self, event) @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapHousing,), self) if bckndcfg.run(): pass def reloadQueriesList(self, select_name=None): self.ui.queriesList.currentIndexChanged.disconnect(self.queryChanged) self.ui.queriesList.clear() for name in self.config.get('queries', default={}).iterkeys(): self.ui.queriesList.addItem(name) if name == select_name: self.ui.queriesList.setCurrentIndex(len(self.ui.queriesList)-1) self.ui.queriesList.currentIndexChanged.connect(self.queryChanged) if select_name is not None: self.queryChanged() @Slot() def removeQuery(self): name = self.ui.queriesList.itemText(self.ui.queriesList.currentIndex()) queries = self.config.get('queries') queries.pop(name, None) self.config.set('queries', queries) self.config.save() self.reloadQueriesList() self.queryChanged() @Slot() def editQuery(self): name = self.ui.queriesList.itemText(self.ui.queriesList.currentIndex()) self.addQuery(name) @Slot() def addQuery(self, name=None): querydlg = QueryDialog(self.weboob, self) if name is not None: query = self.config.get('queries', name) querydlg.ui.nameEdit.setText(name) querydlg.ui.nameEdit.setEnabled(False) for c in query['cities']: city = City(c['id']) city.backend = c['backend'] city.name = c['name'] item = querydlg.buildCityItem(city) querydlg.ui.citiesList.addItem(item) querydlg.ui.typeBox.setCurrentIndex(int(query.get('type', 0))) querydlg.ui.areaMin.setValue(query['area_min']) querydlg.ui.areaMax.setValue(query['area_max']) querydlg.ui.costMin.setValue(query['cost_min']) querydlg.ui.costMax.setValue(query['cost_max']) querydlg.selectComboValue(querydlg.ui.nbRooms, query['nb_rooms']) if querydlg.exec_(): name = querydlg.ui.nameEdit.text() query = {} query['type'] = querydlg.ui.typeBox.currentIndex() query['cities'] = [] for i in xrange(len(querydlg.ui.citiesList)): item = querydlg.ui.citiesList.item(i) city = item.data(Qt.UserRole) query['cities'].append({'id': city.id, 'backend': city.backend, 'name': city.name}) query['area_min'] = querydlg.ui.areaMin.value() query['area_max'] = querydlg.ui.areaMax.value() query['cost_min'] = querydlg.ui.costMin.value() query['cost_max'] = querydlg.ui.costMax.value() try: query['nb_rooms'] = int(querydlg.ui.nbRooms.itemText(querydlg.ui.nbRooms.currentIndex())) except ValueError: query['nb_rooms'] = 0 self.config.set('queries', name, query) self.config.save() self.reloadQueriesList(name) @Slot(int) def queryChanged(self, i=None): self.refreshHousingsList() def refreshHousingsList(self): name = self.ui.queriesList.itemText(self.ui.queriesList.currentIndex()) q = self.config.get('queries', name) if q is None: return q self.ui.housingsList.clear() self.ui.queriesList.setEnabled(False) self.ui.bookmarksButton.setEnabled(False) query = Query() query.type = int(q.get('type', 0)) query.cities = [] for c in q['cities']: city = City(c['id']) city.backend = c['backend'] city.name = c['name'] query.cities.append(city) query.area_min = int(q['area_min']) or None query.area_max = int(q['area_max']) or None query.cost_min = int(q['cost_min']) or None query.cost_max = int(q['cost_max']) or None query.nb_rooms = int(q['nb_rooms']) or None self.process = QtDo(self.weboob, self.addHousing, fb=self.addHousingEnd) self.process.do(self.app._do_complete, 20, None, 'search_housings', query) @Slot() def displayBookmarks(self): self.ui.housingsList.clear() self.ui.queriesList.setEnabled(False) self.ui.queriesList.setCurrentIndex(-1) self.ui.bookmarksButton.setEnabled(False) self.processes = {} for id in self.storage.get('bookmarks'): _id, backend_name = id.rsplit('@', 1) self.process_bookmarks[id] = QtDo(self.weboob, self.addHousing, fb=self.addHousingEnd) self.process_bookmarks[id].do('get_housing', _id, backends=backend_name) def addHousingEnd(self): self.ui.queriesList.setEnabled(True) self.ui.bookmarksButton.setEnabled(True) self.process = None def addHousing(self, housing): if not housing: return item = HousingListWidgetItem(housing) item.setAttrs(self.storage) if housing.photos is NotLoaded: process = QtDo(self.weboob, lambda c: self.setPhoto(c, item)) process.do('fillobj', housing, ['photos'], backends=housing.backend) self.process_photo[housing.id] = process elif housing.photos is not NotAvailable and len(housing.photos) > 0: if not self.setPhoto(housing, item): photo = housing.photos[0] process = QtDo(self.weboob, lambda p: self.setPhoto(housing, item)) process.do('fillobj', photo, ['data'], backends=housing.backend) self.process_photo[housing.id] = process self.ui.housingsList.addItem(item) if housing.fullid in self.process_bookmarks: self.process_bookmarks.pop(housing.fullid) @Slot(QListWidgetItem, QListWidgetItem) def housingSelected(self, item, prev): if item is not None: housing = item.housing self.ui.queriesFrame.setEnabled(False) read = set(self.storage.get('read')) read.add(housing.fullid) self.storage.set('read', list(read)) self.storage.save() self.process = QtDo(self.weboob, self.gotHousing) self.process.do('fillobj', housing, backends=housing.backend) else: housing = None self.setHousing(housing) if prev: prev.setAttrs(self.storage) def setPhoto(self, housing, item): if not housing: return False try: self.process_photo.pop(housing.id, None) except KeyError: pass if not housing.photos: return False img = None for photo in housing.photos: if photo.data: img = QImage.fromData(photo.data) break if img: item.setIcon(QIcon(QPixmap.fromImage(img))) return True return False def setHousing(self, housing, nottext='Loading...'): if self.housing is not None: self.saveNotes() self.housing = housing if self.housing is None: self.ui.housingFrame.hide() return self.ui.housingFrame.show() self.display_photo() self.ui.bookmark.setChecked(housing.fullid in self.storage.get('bookmarks')) self.ui.titleLabel.setText('

%s

' % housing.title) _area = u'%.2f m²' % housing.area if housing.area else housing.area self.ui.areaLabel.setText(u'%s' % _area) self.ui.costLabel.setText(u'%s %s' % (housing.cost, housing.currency)) self.ui.pricePerMeterLabel.setText(u'%.2f %s/m²' % (housing.price_per_meter, housing.currency)) self.ui.dateLabel.setText(housing.date.strftime('%Y-%m-%d') if housing.date else nottext) self.ui.phoneLabel.setText(housing.phone or nottext) self.ui.locationLabel.setText(housing.location or nottext) self.ui.stationLabel.setText(housing.station or nottext) self.ui.urlLabel.setText('%s' % (housing.url or nottext, housing.url or nottext)) text = housing.text.replace('\n', '
') if housing.text else nottext self.ui.descriptionEdit.setText(text) self.ui.notesEdit.setText(self.storage.get('notes', housing.fullid, default='')) while self.ui.detailsFrame.layout().count() > 0: child = self.ui.detailsFrame.layout().takeAt(0) child.widget().hide() child.widget().deleteLater() if housing.details: for key, value in housing.details.iteritems(): if empty(value): continue label = QLabel(value) label.setTextInteractionFlags(Qt.TextSelectableByMouse | Qt.LinksAccessibleByMouse) self.ui.detailsFrame.layout().addRow('%s:' % key, label) def gotHousing(self, housing): self.setHousing(housing, nottext='') self.ui.queriesFrame.setEnabled(True) self.process = None @Slot(int) def bookmarkChanged(self, state): bookmarks = set(self.storage.get('bookmarks')) if state == Qt.Checked: bookmarks.add(self.housing.fullid) elif self.housing.fullid in bookmarks: bookmarks.remove(self.housing.fullid) self.storage.set('bookmarks', list(bookmarks)) self.storage.save() def saveNotes(self): if not self.housing: return txt = self.ui.notesEdit.toPlainText().strip() if len(txt) > 0: self.storage.set('notes', self.housing.fullid, txt) else: self.storage.delete('notes', self.housing.fullid) self.storage.save() @Slot() def previousClicked(self): if not self.housing.photos or len(self.housing.photos) == 0: return self.displayed_photo_idx = (self.displayed_photo_idx - 1) % len(self.housing.photos) self.display_photo() @Slot() def nextClicked(self): if not self.housing.photos or len(self.housing.photos) == 0: return self.displayed_photo_idx = (self.displayed_photo_idx + 1) % len(self.housing.photos) self.display_photo() def display_photo(self): if not self.housing.photos: self.ui.photosFrame.hide() return if self.displayed_photo_idx >= len(self.housing.photos): self.displayed_photo_idx = len(self.housing.photos) - 1 if self.displayed_photo_idx < 0: self.ui.photosFrame.hide() return self.ui.photosFrame.show() photo = self.housing.photos[self.displayed_photo_idx] if photo.data: data = photo.data if photo.id in self.process_photo: self.process_photo.pop(photo.id) else: self.process_photo[photo.id] = QtDo(self.weboob, lambda p: self.display_photo()) self.process_photo[photo.id].do('fillobj', photo, ['data'], backends=self.housing.backend) return img = QImage.fromData(data) img = img.scaledToWidth(self.width()/3) self.ui.photoLabel.setPixmap(QPixmap.fromImage(img)) if photo.url is not NotLoaded: text = '%s' % (photo.url, photo.url) self.ui.photoUrlLabel.setText(text) weboob-1.2/weboob/applications/qflatboob/qflatboob.py000066400000000000000000000030411303450110500230560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.housing import CapHousing from weboob.tools.application.qt5 import QtApplication from weboob.tools.config.yamlconfig import YamlConfig from .main_window import MainWindow class QFlatBoob(QtApplication): APPNAME = 'qflatboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-2014 Romain Bignon' DESCRIPTION = "Qt application to search for housing." SHORT_DESCRIPTION = "search for housing" CAPS = CapHousing CONFIG = {'queries': {}} STORAGE = {'bookmarks': [], 'read': [], 'notes': {}} def main(self, argv): self.load_backends(CapHousing) self.create_storage() self.load_config(klass=YamlConfig) self.main_window = MainWindow(self.config, self.storage, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qflatboob/query.py000066400000000000000000000071051303450110500222570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QDialog, QListWidgetItem, QMessageBox from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.tools.application.qt5 import QtDo, HTMLDelegate from .ui.query_ui import Ui_QueryDialog class QueryDialog(QDialog): def __init__(self, weboob, parent=None): super(QueryDialog, self).__init__(parent) self.ui = Ui_QueryDialog() self.ui.setupUi(self) self.weboob = weboob self.ui.resultsList.setItemDelegate(HTMLDelegate()) self.ui.citiesList.setItemDelegate(HTMLDelegate()) self.search_process = None self.ui.cityEdit.returnPressed.connect(self.searchCity) self.ui.resultsList.itemDoubleClicked.connect(self.insertCity) self.ui.citiesList.itemDoubleClicked.connect(self.removeCity) self.ui.buttonBox.accepted.connect(self.okButton) if hasattr(self.ui.cityEdit, "setPlaceholderText"): self.ui.cityEdit.setPlaceholderText("Press enter to search city") def keyPressEvent(self, event): """ Disable handler and to prevent closing the window. """ event.ignore() def selectComboValue(self, box, value): for i in xrange(box.count()): if box.itemText(i) == str(value): box.setCurrentIndex(i) break @Slot() def searchCity(self): pattern = self.ui.cityEdit.text() self.ui.resultsList.clear() self.ui.cityEdit.clear() self.ui.cityEdit.setEnabled(False) self.search_process = QtDo(self.weboob, self.addResult, fb=self.addResultEnd) self.search_process.do('search_city', pattern) def addResultEnd(self): self.search_process = None self.ui.cityEdit.setEnabled(True) def addResult(self, city): if not city: return item = self.buildCityItem(city) self.ui.resultsList.addItem(item) self.ui.resultsList.sortItems() def buildCityItem(self, city): item = QListWidgetItem() item.setText('%s (%s)' % (city.name, city.backend)) item.setData(Qt.UserRole, city) return item @Slot(QListWidgetItem) def insertCity(self, i): item = QListWidgetItem() item.setText(i.text()) item.setData(Qt.UserRole, i.data(Qt.UserRole)) self.ui.citiesList.addItem(item) @Slot(QListWidgetItem) def removeCity(self, item): self.ui.citiesList.removeItemWidget(item) @Slot() def okButton(self): if not self.ui.nameEdit.text(): QMessageBox.critical(self, self.tr('Error'), self.tr('Please enter a name to your query.'), QMessageBox.Ok) return if self.ui.citiesList.count() == 0: QMessageBox.critical(self, self.tr('Error'), self.tr('Please add at least one city.'), QMessageBox.Ok) return self.accept() weboob-1.2/weboob/applications/qflatboob/ui/000077500000000000000000000000001303450110500211525ustar00rootroot00000000000000weboob-1.2/weboob/applications/qflatboob/ui/Makefile000066400000000000000000000002651303450110500226150ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qflatboob/ui/__init__.py000066400000000000000000000000001303450110500232510ustar00rootroot00000000000000weboob-1.2/weboob/applications/qflatboob/ui/main_window.ui000066400000000000000000000502151303450110500240270ustar00rootroot00000000000000 MainWindow 0 0 709 572 QFlatBoob Qt::Horizontal 0 0 QFrame::StyledPanel QFrame::Raised + edit - 0 0 QAbstractItemView::NoEditTriggers 128 128 true Bookmarks Qt::Horizontal 1 0 QFrame::StyledPanel QFrame::Raised Qt::Vertical 0 0 20 0 <h1>Loading...</h1> QFormLayout::ExpandingFieldsGrow 75 true Cost Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse 75 true Area 75 true Price per square meter 75 true Date 75 true Phone 75 true Location 75 true Station 75 true URL Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Qt::Vertical QSizePolicy::Expanding 20 5 QFrame::StyledPanel QFrame::Raised 0 0 20 16777215 < Qt::AlignCenter 0 0 20 16777215 > Qt::AlignCenter Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Qt::Vertical QSizePolicy::Expanding 13 5 Qt::Horizontal Qt::Vertical 0 50 true 0 0 false QFrame::NoFrame QFrame::Plain QFormLayout::ExpandingFieldsGrow 20 0 0 709 24 toolBar TopToolBarArea false Backends addQueryButton queriesList editQueryButton removeQueryButton housingsList bookmarksButton bookmark previousButton nextButton descriptionEdit weboob-1.2/weboob/applications/qflatboob/ui/query.ui000066400000000000000000000214411303450110500226600ustar00rootroot00000000000000 QueryDialog 0 0 551 457 Add a query QFrame::StyledPanel QFrame::Raised Name of this query: Rent Sale Cities Number of rooms N/A 1 2 3 4 5 Area Min Max 9999 5 9999 5 Cost Min Max 99999999 100 99999999 100 Qt::Horizontal QDialogButtonBox::Cancel|QDialogButtonBox::Ok nameEdit cityEdit resultsList citiesList nbRooms areaMin areaMax costMin costMax buttonBox buttonBox rejected() QueryDialog reject() 316 260 286 274 weboob-1.2/weboob/applications/qhandjoob/000077500000000000000000000000001303450110500205315ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhandjoob/__init__.py000066400000000000000000000000721303450110500226410ustar00rootroot00000000000000from .qhandjoob import QHandJoob __all__ = ['QHandJoob'] weboob-1.2/weboob/applications/qhandjoob/main_window.py000066400000000000000000000151361303450110500234240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Sébastien Monel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QListWidgetItem, QApplication from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.search_history import HistoryCompleter from weboob.capabilities.job import CapJob from .ui.main_window_ui import Ui_MainWindow import os class JobListWidgetItem(QListWidgetItem): def __init__(self, job, *args, **kwargs): super(JobListWidgetItem, self).__init__(*args, **kwargs) self.job = job def __lt__(self, other): if self.job.publication_date and other.job.publication_date: return self.job.publication_date < other.job.publication_date else: return False def setAttrs(self, storage): text = u'%s - %s' % (self.job.backend, self.job.title) self.setText(text) class MainWindow(QtMainWindow): def __init__(self, config, storage, weboob, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.storage = storage self.weboob = weboob self.process = None self.displayed_photo_idx = 0 self.process_photo = {} self.process_bookmarks = {} # search history is a list of patterns which have been searched history_path = os.path.join(self.weboob.workdir, 'qhandjoob_history') qc = HistoryCompleter(history_path, self) qc.load() qc.setCaseSensitivity(Qt.CaseInsensitive) self.ui.searchEdit.setCompleter(qc) self.ui.jobFrame.hide() self.ui.actionBackends.triggered.connect(self.backendsConfig) self.ui.searchEdit.returnPressed.connect(self.doSearch) self.ui.jobList.currentItemChanged.connect(self.jobSelected) self.ui.searchButton.clicked.connect(self.doSearch) self.ui.refreshButton.clicked.connect(self.doAdvancedSearch) self.ui.queriesTabWidget.currentChanged.connect(self.tabChange) self.ui.jobListAdvancedSearch.currentItemChanged.connect(self.jobSelected) self.ui.idEdit.returnPressed.connect(self.openJob) if self.weboob.count_backends() == 0: self.backendsConfig() @Slot(int) def tabChange(self, index): if index == 1: self.doAdvancedSearch() def searchFinished(self): self.process = None QApplication.restoreOverrideCursor() @Slot() def doAdvancedSearch(self): QApplication.setOverrideCursor(Qt.WaitCursor) self.ui.jobListAdvancedSearch.clear() self.process = QtDo(self.weboob, self.addJobAdvancedSearch, fb=self.searchFinished) self.process.do('advanced_search_job') @Slot() def doSearch(self): QApplication.setOverrideCursor(Qt.WaitCursor) pattern = self.ui.searchEdit.text() self.ui.searchEdit.completer().addString(pattern) self.ui.jobList.clear() self.process = QtDo(self.weboob, self.addJobSearch, fb=self.searchFinished) self.process.do('search_job', pattern) def addJobSearch(self, job): item = self.addJob(job) if item: self.ui.jobList.addItem(item) def addJobAdvancedSearch(self, job): item = self.addJob(job) if item: self.ui.jobListAdvancedSearch.addItem(item) def addJob(self, job): if not job: return item = JobListWidgetItem(job) item.setAttrs(self.storage) return item def closeEvent(self, event): self.ui.searchEdit.completer().save() QtMainWindow.closeEvent(self, event) @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapJob,), self) if bckndcfg.run(): pass @Slot(QListWidgetItem, QListWidgetItem) def jobSelected(self, item, prev): QApplication.setOverrideCursor(Qt.WaitCursor) if item is not None: job = item.job self.ui.queriesTabWidget.setEnabled(False) self.process = QtDo(self.weboob, self.gotJob) self.process.do('fillobj', job, backends=job.backend) else: job = None self.setJob(job) if prev: prev.setAttrs(self.storage) @Slot() def openJob(self): QApplication.setOverrideCursor(Qt.WaitCursor) url = self.ui.idEdit.text() if not url: return for backend in self.weboob.iter_backends(): job = backend.get_job_advert(url) if job: self.process = QtDo(self.weboob, self.gotJob) self.process.do('fillobj', job, backends=job.backend) break self.setJob(job) self.ui.idEdit.clear() QApplication.restoreOverrideCursor() def gotJob(self, job): self.setJob(job) self.ui.queriesTabWidget.setEnabled(True) self.process = None def setJob(self, job): if job: self.ui.descriptionEdit.setText("%s" % job.description) self.ui.titleLabel.setText("

%s

" % job.title) self.ui.idLabel.setText("%s" % job.id) self.ui.jobNameLabel.setText("%s" % job.job_name) self.ui.publicationDateLabel.setText("%s" % job.publication_date) self.ui.societyNameLabel.setText("%s" % job.society_name) self.ui.placeLabel.setText("%s" % job.place) self.ui.payLabel.setText("%s" % job.pay) self.ui.contractTypeLabel.setText("%s" % job.contract_type) self.ui.formationLabel.setText("%s" % job.formation) self.ui.experienceLabel.setText("%s" % job.experience) self.ui.urlLabel.setText("%s" % (job.url, job.url)) self.ui.jobFrame.show() else: self.ui.jobFrame.hide() QApplication.restoreOverrideCursor() weboob-1.2/weboob/applications/qhandjoob/qhandjoob.py000066400000000000000000000030051303450110500230460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Sébastien Monel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.job import CapJob from weboob.tools.application.qt5 import QtApplication from weboob.tools.config.yamlconfig import YamlConfig from .main_window import MainWindow class QHandJoob(QtApplication): APPNAME = 'qhandjoob' VERSION = '1.2' COPYRIGHT = u'Copyright(C) 2013-2014 Sébastien Monel' DESCRIPTION = "Qt application to search for job." SHORT_DESCRIPTION = "search for job" CAPS = CapJob CONFIG = {'queries': {}} STORAGE = {'bookmarks': [], 'read': [], 'notes': {}} def main(self, argv): self.load_backends(CapJob) self.create_storage() self.load_config(klass=YamlConfig) self.main_window = MainWindow(self.config, self.storage, self.weboob) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qhandjoob/ui/000077500000000000000000000000001303450110500211465ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhandjoob/ui/Makefile000066400000000000000000000002651303450110500226110ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qhandjoob/ui/__init__.py000066400000000000000000000000001303450110500232450ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhandjoob/ui/main_window.ui000066400000000000000000000413211303450110500240210ustar00rootroot00000000000000 MainWindow 0 0 709 572 QHandJoob Qt::Horizontal 0 0 0 Search search 0 0 QAbstractItemView::NoEditTriggers 128 128 true id : Advanced search 0 0 QAbstractItemView::NoEditTriggers 128 128 true Refresh Qt::Horizontal 1 0 QFrame::StyledPanel QFrame::Raised Qt::Vertical <h1>Loading...</h1> QFormLayout::ExpandingFieldsGrow 75 true Id Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse 75 true Job Name 75 true Publication Date 75 true Society Name 75 true Place 75 true Pay 75 true Contract Type 75 true Formation 75 true Experience 75 true URL Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Loading... true Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Qt::Vertical QSizePolicy::Expanding 20 5 0 50 true 0 0 709 23 toolBar TopToolBarArea false Backends searchButton jobList weboob-1.2/weboob/applications/qhavedate/000077500000000000000000000000001303450110500205265ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhavedate/__init__.py000066400000000000000000000000721303450110500226360ustar00rootroot00000000000000from .qhavedate import QHaveDate __all__ = ['QHaveDate'] weboob-1.2/weboob/applications/qhavedate/contacts.py000066400000000000000000000534661303450110500227340ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time import logging from PyQt5.QtGui import QImage, QIcon, QPixmap from PyQt5.QtWidgets import QWidget, QListWidgetItem, QFrame, \ QMessageBox, QTabWidget, QVBoxLayout, \ QFormLayout, QLabel, QPushButton from PyQt5.QtCore import Qt, pyqtSlot as Slot from weboob.tools.application.qt5 import QtDo, HTMLDelegate from weboob.tools.application.qt5.models import BackendListModel from weboob.tools.misc import to_unicode from weboob.capabilities.contact import CapContact, Contact from weboob.capabilities.chat import CapChat from weboob.capabilities.messages import CapMessages, CapMessagesPost, Message from weboob.capabilities.base import NotLoaded from .ui.contacts_ui import Ui_Contacts from .ui.contact_thread_ui import Ui_ContactThread from .ui.thread_message_ui import Ui_ThreadMessage from .ui.profile_ui import Ui_Profile from .ui.notes_ui import Ui_Notes class ThreadMessage(QFrame): """ This class represents a message in the thread tab. """ def __init__(self, message, parent=None): super(ThreadMessage, self).__init__(parent) self.ui = Ui_ThreadMessage() self.ui.setupUi(self) self.set_message(message) def set_message(self, message): self.message = message self.ui.nameLabel.setText(message.sender) header = time.strftime('%Y-%m-%d %H:%M:%S', message.date.timetuple()) if message.flags & message.IS_NOT_RECEIVED: header += u' — Unread' elif message.flags & message.IS_RECEIVED: header += u' — Read' self.ui.headerLabel.setText(header) if message.flags & message.IS_HTML: content = message.content else: content = message.content.replace('&', '&').replace('<', '<').replace('>', '>').replace('\n', '
') self.ui.contentLabel.setText(content) def __eq__(self, m): if not isinstance(m, Message): return False return self.message == m.message class ContactThread(QWidget): """ The thread of the selected contact. """ def __init__(self, weboob, contact, support_reply, parent=None): super(ContactThread, self).__init__(parent) self.ui = Ui_ContactThread() self.ui.setupUi(self) self.weboob = weboob self.contact = contact self.thread = None self.messages = [] self.process_msg = None self.ui.refreshButton.clicked.connect(self.refreshMessages) if support_reply: self.ui.sendButton.clicked.connect(self.postReply) else: self.ui.frame.hide() self.refreshMessages() @Slot() def refreshMessages(self, fillobj=False): if self.process_msg: return self.ui.refreshButton.setEnabled(False) def finished(): #v = self.ui.scrollArea.verticalScrollBar() #print v.minimum(), v.value(), v.maximum(), v.sliderPosition() #self.ui.scrollArea.verticalScrollBar().setValue(self.ui.scrollArea.verticalScrollBar().maximum()) self.process_msg = None self.process_msg = QtDo(self.weboob, self.gotThread, self.gotError, finished) if fillobj and self.thread: self.process_msg.do('fillobj', self.thread, ['root'], backends=self.contact.backend) else: self.process_msg.do('get_thread', self.contact.id, backends=self.contact.backend) def gotError(self, backend, error, backtrace): self.ui.textEdit.setEnabled(False) self.ui.sendButton.setEnabled(False) self.ui.refreshButton.setEnabled(True) def gotThread(self, thread): self.ui.textEdit.setEnabled(True) self.ui.sendButton.setEnabled(True) self.ui.refreshButton.setEnabled(True) self.thread = thread if thread.root is NotLoaded: self._insert_load_button(0) else: for message in thread.iter_all_messages(): self._insert_message(message) def _insert_message(self, message): widget = ThreadMessage(message) if widget in self.messages: old_widget = self.messages[self.messages.index(widget)] if old_widget.message.flags != widget.message.flags: old_widget.set_message(widget.message) return for i, m in enumerate(self.messages): if widget.message.date > m.message.date: self.ui.scrollAreaContent.layout().insertWidget(i, widget) self.messages.insert(i, widget) if message.parent is NotLoaded: self._insert_load_button(i) return self.ui.scrollAreaContent.layout().addWidget(widget) self.messages.append(widget) if message.parent is NotLoaded: self._insert_load_button(-1) def _insert_load_button(self, pos): button = QPushButton(self.tr('More messages...')) button.clicked.connect(self._load_button_pressed) if pos >= 0: self.ui.scrollAreaContent.layout().insertWidget(pos, button) else: self.ui.scrollAreaContent.layout().addWidget(button) @Slot() def _load_button_pressed(self): button = self.sender() self.ui.scrollAreaContent.layout().removeWidget(button) button.hide() button.deleteLater() self.refreshMessages(fillobj=True) @Slot() def postReply(self): text = self.ui.textEdit.toPlainText() self.ui.textEdit.setEnabled(False) self.ui.sendButton.setEnabled(False) m = Message(thread=self.thread, id=0, title=u'', sender=None, receivers=None, content=text, parent=self.messages[0].message if len(self.messages) > 0 else None) self.process_reply = QtDo(self.weboob, None, self._postReply_eb, self._postReply_fb) self.process_reply.do('post_message', m, backends=self.contact.backend) def _postReply_fb(self): self.ui.textEdit.clear() self.ui.textEdit.setEnabled(True) self.ui.sendButton.setEnabled(True) self.refreshMessages() self.process_reply = None def _postReply_eb(self, backend, error, backtrace): content = self.tr('Unable to send message:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while posting reply'), content, QMessageBox.Ok) self.process_reply = None class ContactProfile(QWidget): def __init__(self, weboob, contact, parent=None): super(ContactProfile, self).__init__(parent) self.ui = Ui_Profile() self.ui.setupUi(self) self.ui.previousButton.clicked.connect(self.previousClicked) self.ui.nextButton.clicked.connect(self.nextClicked) self.weboob = weboob self.contact = contact self.loaded_profile = False self.displayed_photo_idx = 0 self.process_photo = {} missing_fields = self.gotProfile(contact) if len(missing_fields) > 0: self.process_contact = QtDo(self.weboob, self.gotProfile, self.gotError) self.process_contact.do('fillobj', self.contact, missing_fields, backends=self.contact.backend) def gotError(self, backend, error, backtrace): self.ui.frame_photo.hide() self.ui.descriptionEdit.setText('

Unable to show profile

%s

' % to_unicode(error)) def gotProfile(self, contact): missing_fields = set() self.display_photo() self.ui.nicknameLabel.setText('

%s

' % contact.name) if contact.status == Contact.STATUS_ONLINE: status_color = 0x00aa00 elif contact.status == Contact.STATUS_OFFLINE: status_color = 0xff0000 elif contact.status == Contact.STATUS_AWAY: status_color = 0xffad16 else: status_color = 0xaaaaaa self.ui.statusLabel.setText('%s' % (status_color, contact.status_msg)) self.ui.contactUrlLabel.setText('URL: %s' % (contact.url, contact.url)) if contact.summary is NotLoaded: self.ui.descriptionEdit.setText('

Description

Receiving...

') missing_fields.add('summary') else: self.ui.descriptionEdit.setText('

Description

%s

' % contact.summary.replace('\n', '
')) if not contact.profile: missing_fields.add('profile') elif not self.loaded_profile: self.loaded_profile = True for head in contact.profile.itervalues(): if head.flags & head.HEAD: widget = self.ui.headWidget else: widget = self.ui.profileTab self.process_node(head, widget) return missing_fields def process_node(self, node, widget): # Set the value widget value = None if node.flags & node.SECTION: value = QWidget() value.setLayout(QFormLayout()) for sub in node.value.itervalues(): self.process_node(sub, value) elif isinstance(node.value, list): value = QLabel('
'.join(unicode(s) for s in node.value)) value.setWordWrap(True) elif isinstance(node.value, tuple): value = QLabel(', '.join(unicode(s) for s in node.value)) value.setWordWrap(True) elif isinstance(node.value, (basestring,int,long,float)): value = QLabel(unicode(node.value)) else: logging.warning('Not supported value: %r' % node.value) return if isinstance(value, QLabel): value.setTextInteractionFlags(Qt.TextSelectableByMouse|Qt.TextSelectableByKeyboard|Qt.LinksAccessibleByMouse) # Insert the value widget into the parent widget, depending # of its type. if isinstance(widget, QTabWidget): widget.addTab(value, node.label) elif isinstance(widget.layout(), QFormLayout): label = QLabel(u'%s: ' % node.label) widget.layout().addRow(label, value) elif isinstance(widget.layout(), QVBoxLayout): widget.layout().addWidget(QLabel(u'

%s

' % node.label)) widget.layout().addWidget(value) else: logging.warning('Not supported widget: %r' % widget) @Slot() def previousClicked(self): if len(self.contact.photos) == 0: return self.displayed_photo_idx = (self.displayed_photo_idx - 1) % len(self.contact.photos) self.display_photo() @Slot() def nextClicked(self): if len(self.contact.photos) == 0: return self.displayed_photo_idx = (self.displayed_photo_idx + 1) % len(self.contact.photos) self.display_photo() def display_photo(self): if self.displayed_photo_idx >= len(self.contact.photos) or self.displayed_photo_idx < 0: self.displayed_photo_idx = len(self.contact.photos) - 1 if self.displayed_photo_idx < 0: self.ui.photoUrlLabel.setText('') return photo = self.contact.photos.values()[self.displayed_photo_idx] if photo.data: data = photo.data if photo.id in self.process_photo: self.process_photo.pop(photo.id) else: self.process_photo[photo.id] = QtDo(self.weboob, lambda p: self.display_photo()) self.process_photo[photo.id].do('fillobj', photo, ['data'], backends=self.contact.backend) if photo.thumbnail_data: data = photo.thumbnail_data else: return img = QImage.fromData(data) img = img.scaledToWidth(self.width()/3) self.ui.photoLabel.setPixmap(QPixmap.fromImage(img)) if photo.url is not NotLoaded: text = '%s' % (photo.url, photo.url) if photo.hidden: text += '
(Hidden photo)' self.ui.photoUrlLabel.setText(text) class ContactNotes(QWidget): """ Widget for storing notes about a contact """ def __init__(self, weboob, contact, parent=None): super(ContactNotes, self).__init__(parent) self.ui = Ui_Notes() self.ui.setupUi(self) self.weboob = weboob self.contact = contact self.ui.textEdit.setEnabled(False) self.ui.saveButton.setEnabled(False) def finished(): self.process = None self.ui.textEdit.setEnabled(True) self.ui.saveButton.setEnabled(True) self.process = QtDo(self.weboob, self._getNotes_cb, self._getNotes_eb, finished) self.process.do('get_notes', self.contact.id, backends=(self.contact.backend,)) self.ui.saveButton.clicked.connect(self.saveNotes) def _getNotes_cb(self, data): if data: self.ui.textEdit.setText(data) def _getNotes_eb(self, backend, error, backtrace): if isinstance(error, NotImplementedError): return self.ui.textEdit.setEnabled(True) self.ui.saveButton.setEnabled(True) content = self.tr('Unable to load notes:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while loading notes'), content, QMessageBox.Ok) @Slot() def saveNotes(self): text = self.ui.textEdit.toPlainText() self.ui.saveButton.setEnabled(False) self.ui.textEdit.setEnabled(False) self.process = QtDo(self.weboob, None, self._saveNotes_eb, self._saveNotes_fb) self.process.do('save_notes', self.contact.id, text, backends=(self.contact.backend,)) def _saveNotes_fb(self): self.ui.saveButton.setEnabled(True) self.ui.textEdit.setEnabled(True) def _saveNotes_eb(self, backend, error, backtrace): content = self.tr('Unable to save notes:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while saving notes'), content, QMessageBox.Ok) class IGroup(object): def __init__(self, weboob, id, name): self.id = id self.name = name self.weboob = weboob def iter_contacts(self, cb): raise NotImplementedError() class MetaGroup(IGroup): def iter_contacts(self, cb): if self.id == 'online': status = Contact.STATUS_ONLINE|Contact.STATUS_AWAY elif self.id == 'offline': status = Contact.STATUS_OFFLINE else: status = Contact.STATUS_ALL self.process = QtDo(self.weboob, lambda d: self.cb(cb, d), fb=lambda: self.fb(cb)) self.process.do('iter_contacts', status, caps=CapContact) def cb(self, cb, contact): if contact: cb(contact) def fb(self, fb): self.process = None if fb: fb(None) class ContactsWidget(QWidget): def __init__(self, weboob, parent=None): super(ContactsWidget, self).__init__(parent) self.ui = Ui_Contacts() self.ui.setupUi(self) self.weboob = weboob self.contact = None self.ui.contactList.setItemDelegate(HTMLDelegate()) self.url_process = None self.photo_processes = {} self.ui.groupBox.addItem('All', MetaGroup(self.weboob, 'all', self.tr('All'))) self.ui.groupBox.addItem('Online', MetaGroup(self.weboob, 'online', self.tr('Online'))) self.ui.groupBox.addItem('Offline', MetaGroup(self.weboob, 'offline', self.tr('Offline'))) self.ui.groupBox.setCurrentIndex(1) self.ui.groupBox.currentIndexChanged.connect(self.groupChanged) self.ui.contactList.itemClicked.connect(self.contactChanged) self.ui.refreshButton.clicked.connect(self.refreshContactList) self.ui.urlButton.clicked.connect(self.urlClicked) def load(self): self.refreshContactList() model = BackendListModel(self.weboob) model.addBackends(entry_all=False) self.ui.backendsList.setModel(model) @Slot() def groupChanged(self): self.refreshContactList() @Slot() def refreshContactList(self): self.ui.contactList.clear() self.ui.refreshButton.setEnabled(False) i = self.ui.groupBox.currentIndex() group = self.ui.groupBox.itemData(i) group.iter_contacts(self.addContact) def setPhoto(self, contact, item): if not contact: return False try: self.photo_processes.pop(contact.id, None) except KeyError: pass img = None for photo in contact.photos.itervalues(): if photo.thumbnail_data: img = QImage.fromData(photo.thumbnail_data) break if img: item.setIcon(QIcon(QPixmap.fromImage(img))) return True return False def addContact(self, contact): if not contact: self.ui.refreshButton.setEnabled(True) return status = '' if contact.status == Contact.STATUS_ONLINE: status = u'Online' status_color = 0x00aa00 elif contact.status == Contact.STATUS_OFFLINE: status = u'Offline' status_color = 0xff0000 elif contact.status == Contact.STATUS_AWAY: status = u'Away' status_color = 0xffad16 else: status = u'Unknown' status_color = 0xaaaaaa if contact.status_msg: status += u' — %s' % contact.status_msg item = QListWidgetItem() item.setText('

%s

%s
%s' % (contact.name, status_color, status, contact.backend)) item.setData(Qt.UserRole, contact) if contact.photos is NotLoaded: process = QtDo(self.weboob, lambda c: self.setPhoto(c, item)) process.do('fillobj', contact, ['photos'], backends=contact.backend) self.photo_processes[contact.id] = process elif len(contact.photos) > 0: if not self.setPhoto(contact, item): photo = contact.photos.values()[0] process = QtDo(self.weboob, lambda p: self.setPhoto(contact, item)) process.do('fillobj', photo, ['thumbnail_data'], backends=contact.backend) self.photo_processes[contact.id] = process for i in xrange(self.ui.contactList.count()): if self.ui.contactList.item(i).data(Qt.UserRole).status > contact.status: self.ui.contactList.insertItem(i, item) return self.ui.contactList.addItem(item) @Slot(QListWidgetItem) def contactChanged(self, current): if not current: return contact = current.data(Qt.UserRole) self.setContact(contact) def setContact(self, contact): if not contact or contact == self.contact: return if not isinstance(contact, Contact): return self.retrieveContact(contact) self.ui.tabWidget.clear() self.contact = contact backend = self.weboob.get_backend(self.contact.backend) self.ui.tabWidget.addTab(ContactProfile(self.weboob, self.contact), self.tr('Profile')) if backend.has_caps(CapMessages): self.ui.tabWidget.addTab(ContactThread(self.weboob, self.contact, backend.has_caps(CapMessagesPost)), self.tr('Messages')) if backend.has_caps(CapChat): self.ui.tabWidget.setTabEnabled(self.ui.tabWidget.addTab(QWidget(), self.tr('Chat')), False) self.ui.tabWidget.setTabEnabled(self.ui.tabWidget.addTab(QWidget(), self.tr('Calendar')), False) self.ui.tabWidget.addTab(ContactNotes(self.weboob, self.contact), self.tr('Notes')) @Slot() def urlClicked(self): url = self.ui.urlEdit.text() if not url: return self.retrieveContact(url) def retrieveContact(self, url): backend_name = self.ui.backendsList.currentText() self.ui.urlButton.setEnabled(False) def finished(): self.url_process = None self.ui.urlButton.setEnabled(True) self.url_process = QtDo(self.weboob, self.retrieveContact_cb, self.retrieveContact_eb, finished) self.url_process.do('get_contact', url, backends=backend_name) def retrieveContact_cb(self, contact): self.ui.urlEdit.clear() self.setContact(contact) def retrieveContact_eb(self, backend, error, backtrace): content = self.tr('Unable to get contact:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += u'\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while getting contact'), content, QMessageBox.Ok) weboob-1.2/weboob/applications/qhavedate/events.py000066400000000000000000000135431303450110500224120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from PyQt5.QtGui import QImage, QIcon, QPixmap from PyQt5.QtWidgets import QWidget, QTreeWidgetItem from PyQt5.QtCore import Qt, pyqtSignal as Signal, pyqtSlot as Slot from weboob.capabilities.base import NotLoaded from weboob.tools.application.qt5 import QtDo, HTMLDelegate from .ui.events_ui import Ui_Events class EventsWidget(QWidget): display_contact = Signal(object) def __init__(self, weboob, parent=None): super(EventsWidget, self).__init__(parent) self.ui = Ui_Events() self.ui.setupUi(self) self.weboob = weboob self.photo_processes = {} self.event_filter = None self.ui.eventsList.itemDoubleClicked.connect(self.eventDoubleClicked) self.ui.typeBox.currentIndexChanged.connect(self.typeChanged) self.ui.refreshButton.clicked.connect(self.refreshEventsList) self.ui.eventsList.setItemDelegate(HTMLDelegate()) self.ui.eventsList.sortByColumn(1, Qt.DescendingOrder) def load(self): self.refreshEventsList() @Slot(int) def typeChanged(self, i): if self.ui.refreshButton.isEnabled(): self.refreshEventsList() @Slot() def refreshEventsList(self): self.ui.eventsList.clear() self.ui.refreshButton.setEnabled(False) if self.ui.typeBox.currentIndex() >= 0: # XXX strangely, in gotEvent() in the loop to check if there is already the # event type to try to introduce it in list, itemData() returns the right value. # But, I don't know why, here, it will ALWAYS return None... # So the filter does not work currently. self.events_filter = self.ui.typeBox.itemData(self.ui.typeBox.currentIndex()) else: self.event_filter = None self.ui.typeBox.setEnabled(False) self.ui.typeBox.clear() self.ui.typeBox.addItem('All', None) def finished(): self.ui.refreshButton.setEnabled(True) self.ui.typeBox.setEnabled(True) self.process = QtDo(self.weboob, self.gotEvent, fb=finished) self.process.do('iter_events') def setPhoto(self, contact, item): if not contact: return False try: self.photo_processes.pop(contact.id, None) except KeyError: pass img = None for photo in contact.photos.itervalues(): if photo.thumbnail_data: img = QImage.fromData(photo.thumbnail_data) break if img: item.setIcon(0, QIcon(QPixmap.fromImage(img))) self.ui.eventsList.resizeColumnToContents(0) return True return False def gotEvent(self, event): found = False for i in xrange(self.ui.typeBox.count()): s = self.ui.typeBox.itemData(i) if s == event.type: found = True if not found: print(event.type) self.ui.typeBox.addItem(event.type.capitalize(), event.type) if event.type == self.event_filter: self.ui.typeBox.setCurrentIndex(self.ui.typeBox.count()-1) if self.event_filter and self.event_filter != event.type: return if not event.contact: return contact = event.contact contact.backend = event.backend status = '' if contact.status == contact.STATUS_ONLINE: status = u'Online' status_color = 0x00aa00 elif contact.status == contact.STATUS_OFFLINE: status = u'Offline' status_color = 0xff0000 elif contact.status == contact.STATUS_AWAY: status = u'Away' status_color = 0xffad16 else: status = u'Unknown' status_color = 0xaaaaaa if contact.status_msg: status += u' — %s' % contact.status_msg name = '

%s

%s
%s' % (contact.name, status_color, status, event.backend) date = event.date.strftime('%Y-%m-%d %H:%M') type = event.type message = event.message item = QTreeWidgetItem(None, [name, date, type, message]) item.setData(0, Qt.UserRole, event) if contact.photos is NotLoaded: process = QtDo(self.weboob, lambda c: self.setPhoto(c, item)) process.do('fillobj', contact, ['photos'], backends=contact.backend) self.photo_processes[contact.id] = process elif len(contact.photos) > 0: if not self.setPhoto(contact, item): photo = contact.photos.values()[0] process = QtDo(self.weboob, lambda p: self.setPhoto(contact, item)) process.do('fillobj', photo, ['thumbnail_data'], backends=contact.backend) self.photo_processes[contact.id] = process self.ui.eventsList.addTopLevelItem(item) self.ui.eventsList.resizeColumnToContents(0) self.ui.eventsList.resizeColumnToContents(1) @Slot(QTreeWidgetItem, int) def eventDoubleClicked(self, item, col): event = item.data(0, Qt.UserRole) self.display_contact.emit(event.contact) weboob-1.2/weboob/applications/qhavedate/main_window.py000066400000000000000000000064341303450110500234220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QWidget from PyQt5.QtCore import pyqtSlot as Slot from weboob.tools.application.qt5 import QtMainWindow from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.capabilities.dating import CapDating try: from weboob.applications.qboobmsg.messages_manager import MessagesManager HAVE_BOOBMSG = True except ImportError: HAVE_BOOBMSG = False from .ui.main_window_ui import Ui_MainWindow from .status import AccountsStatus from .contacts import ContactsWidget from .events import EventsWidget from .search import SearchWidget class MainWindow(QtMainWindow): def __init__(self, config, weboob, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.loaded_tabs = {} self.ui.actionBackends.triggered.connect(self.backendsConfig) self.ui.tabWidget.currentChanged.connect(self.tabChanged) self.addTab(AccountsStatus(self.weboob), self.tr('Status')) self.addTab(MessagesManager(self.weboob) if HAVE_BOOBMSG else None, self.tr('Messages')) self.addTab(ContactsWidget(self.weboob), self.tr('Contacts')) self.addTab(EventsWidget(self.weboob), self.tr('Events')) self.addTab(SearchWidget(self.weboob), self.tr('Search')) self.addTab(None, self.tr('Calendar')) self.addTab(None, self.tr('Optimizations')) if self.weboob.count_backends() == 0: self.backendsConfig() @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapDating,), self) if bckndcfg.run(): self.loaded_tabs.clear() widget = self.ui.tabWidget.widget(self.ui.tabWidget.currentIndex()) widget.load() def addTab(self, widget, title): if widget: if getattr(widget, 'display_contact', None): widget.display_contact.connect(self.display_contact) self.ui.tabWidget.addTab(widget, title) else: index = self.ui.tabWidget.addTab(QWidget(), title) self.ui.tabWidget.setTabEnabled(index, False) @Slot(int) def tabChanged(self, i): widget = self.ui.tabWidget.currentWidget() if hasattr(widget, 'load') and i not in self.loaded_tabs: widget.load() self.loaded_tabs[i] = True @Slot(object) def display_contact(self, contact): self.ui.tabWidget.setCurrentIndex(2) widget = self.ui.tabWidget.currentWidget() widget.setContact(contact) weboob-1.2/weboob/applications/qhavedate/qhavedate.py000066400000000000000000000026661303450110500230540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.dating import CapDating from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QHaveDate(QtApplication): APPNAME = 'qhavedate' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-2014 Romain Bignon' DESCRIPTION = "Qt application allowing to interact with various dating websites." SHORT_DESCRIPTION = "interact with dating websites" CAPS = CapDating STORAGE_FILENAME = 'dating.storage' def main(self, argv): self.create_storage(self.STORAGE_FILENAME) self.load_backends(CapDating) self.main_window = MainWindow(self.config, self.weboob) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qhavedate/search.py000066400000000000000000000062771303450110500223610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QWidget from PyQt5.QtCore import pyqtSlot as Slot from weboob.tools.application.qt5 import QtDo from .ui.search_ui import Ui_Search from .contacts import ContactProfile from .status import Account class SearchWidget(QWidget): def __init__(self, weboob, parent=None): super(SearchWidget, self).__init__(parent) self.ui = Ui_Search() self.ui.setupUi(self) self.weboob = weboob self.contacts = [] self.accounts = [] self.current = None self.ui.nextButton.clicked.connect(self.next) self.ui.queryButton.clicked.connect(self.sendQuery) def load(self): while self.ui.statusFrame.layout().count() > 0: item = self.ui.statusFrame.layout().takeAt(0) if item.widget(): item.widget().deinit() item.widget().hide() item.widget().deleteLater() self.accounts = [] for backend in self.weboob.iter_backends(): account = Account(self.weboob, backend) account.title.setText(u'

%s

' % backend.name) self.accounts.append(account) self.ui.statusFrame.layout().addWidget(account) self.ui.statusFrame.layout().addStretch() self.getNewProfiles() def updateStats(self): for account in self.accounts: account.updateStats() def getNewProfiles(self): self.newprofiles_process = QtDo(self.weboob, self.retrieveNewContacts_cb) self.newprofiles_process.do('iter_new_contacts') def retrieveNewContacts_cb(self, contact): self.contacts.insert(0, contact) self.ui.queueLabel.setText('%d' % len(self.contacts)) if self.current is None: self.next() @Slot() def next(self): try: contact = self.contacts.pop() except IndexError: contact = None self.ui.queueLabel.setText('%d' % len(self.contacts)) self.setContact(contact) self.updateStats() def setContact(self, contact): self.current = contact if contact is not None: widget = ContactProfile(self.weboob, contact) self.ui.scrollArea.setWidget(widget) else: self.ui.scrollArea.setWidget(None) @Slot() def sendQuery(self): self.newprofiles_process = QtDo(self.weboob, None, fb=self.next) self.newprofiles_process.do('send_query', self.current.id, backends=[self.current.backend]) weboob-1.2/weboob/applications/qhavedate/status.py000066400000000000000000000107671303450110500224360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtGui import QImage, QPixmap from PyQt5.QtWidgets import QScrollArea, QWidget, QHBoxLayout, QVBoxLayout, QFrame, QLabel from weboob.capabilities.account import CapAccount, StatusField from weboob.tools.application.qt5 import QtDo from weboob.tools.misc import to_unicode class Account(QFrame): def __init__(self, weboob, backend, parent=None): super(Account, self).__init__(parent) self.setFrameShape(QFrame.StyledPanel) self.setFrameShadow(QFrame.Raised) self.weboob = weboob self.backend = backend self.setLayout(QVBoxLayout()) self.timer = None head = QHBoxLayout() headw = QWidget() headw.setLayout(head) self.title = QLabel(u'

%s — %s

' % (backend.name, backend.DESCRIPTION)) self.body = QLabel() minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: self.icon = QLabel() img = QImage(icon_path) self.icon.setPixmap(QPixmap.fromImage(img)) head.addWidget(self.icon) head.addWidget(self.title) head.addStretch() self.layout().addWidget(headw) if backend.has_caps(CapAccount): self.body.setText(u'Waiting...') self.layout().addWidget(self.body) self.timer = self.weboob.repeat(60, self.updateStats) def deinit(self): if self.timer is not None: self.weboob.stop(self.timer) def updateStats(self): self.process = QtDo(self.weboob, self.updateStats_cb, self.updateStats_eb, self.updateStats_fb) self.process.body = u'' self.process.in_p = False self.process.do('get_account_status', backends=self.backend) def updateStats_fb(self): if self.process.in_p: self.process.body += u"

" self.body.setText(self.process.body) self.process = None def updateStats_cb(self, field): if field.flags & StatusField.FIELD_HTML: value = u'%s' % field.value else: value = (u'%s' % field.value).replace('&', '&').replace('<', '<').replace('>', '>') if field.flags & StatusField.FIELD_TEXT: if self.process.in_p: self.process.body += u'

' self.process.body += u'

%s

' % value self.process.in_p = False else: if not self.process.in_p: self.process.body += u"

" self.process.in_p = True else: self.process.body += u"
" self.process.body += u'%s: %s' % (field.label, field.value) def updateStats_eb(self, backend, err, backtrace): self.body.setText(u'Unable to connect: %s' % to_unicode(err)) self.title.setText(u'%s' % self.title.text()) class AccountsStatus(QScrollArea): def __init__(self, weboob, parent=None): super(AccountsStatus, self).__init__(parent) self.weboob = weboob self.setFrameShadow(self.Plain) self.setFrameShape(self.NoFrame) self.setWidgetResizable(True) widget = QWidget(self) widget.setLayout(QVBoxLayout()) widget.show() self.setWidget(widget) def load(self): while self.widget().layout().count() > 0: item = self.widget().layout().takeAt(0) if item.widget(): item.widget().deinit() item.widget().hide() item.widget().deleteLater() for backend in self.weboob.iter_backends(): account = Account(self.weboob, backend) self.widget().layout().addWidget(account) self.widget().layout().addStretch() weboob-1.2/weboob/applications/qhavedate/ui/000077500000000000000000000000001303450110500211435ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhavedate/ui/Makefile000066400000000000000000000002651303450110500226060ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qhavedate/ui/__init__.py000066400000000000000000000000001303450110500232420ustar00rootroot00000000000000weboob-1.2/weboob/applications/qhavedate/ui/contact_thread.ui000066400000000000000000000111221303450110500244610ustar00rootroot00000000000000 ContactThread 0 0 578 429 Form Qt::Vertical 0 0 QFrame::StyledPanel QFrame::Raised false false 0 0 Send 0 5 0 Qt::Horizontal 40 20 ../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png 0 1 Qt::ScrollBarAsNeeded true Qt::AlignBottom|Qt::AlignLeading|Qt::AlignLeft 0 0 556 154 QWidget#scrollAreaContent { background-color: rgb(255, 255, 255); } weboob-1.2/weboob/applications/qhavedate/ui/contacts.ui000066400000000000000000000073071303450110500233270ustar00rootroot00000000000000 Contacts 0 0 478 374 Form Qt::Horizontal 0 0 QFrame::StyledPanel QFrame::Raised ../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png 120 120 1 true false true From URL Display 1 0 weboob-1.2/weboob/applications/qhavedate/ui/events.ui000066400000000000000000000045251303450110500230140ustar00rootroot00000000000000 Events 0 0 497 318 Form ../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../usr/share/icons/oxygen/16x16/actions/view-refresh.png QAbstractItemView::NoEditTriggers true 64 64 false true true Contact Date Type Message weboob-1.2/weboob/applications/qhavedate/ui/main_window.ui000066400000000000000000000041701303450110500240170ustar00rootroot00000000000000 MainWindow 0 0 763 580 QHaveDate -1 0 0 763 24 File toolBar TopToolBarArea false Backends Quit Quit actionQuit triggered() MainWindow close() -1 -1 381 289 weboob-1.2/weboob/applications/qhavedate/ui/notes.ui000066400000000000000000000012351303450110500226330ustar00rootroot00000000000000 Notes 0 0 430 323 Form Save weboob-1.2/weboob/applications/qhavedate/ui/profile.ui000066400000000000000000000212111303450110500231370ustar00rootroot00000000000000 Profile 0 0 755 647 Form 0 QFrame::NoFrame QFrame::Plain true 0 0 755 647 0 0 0 0 0 QFrame::StyledPanel QFrame::Raised <h1>Loading...</h1> Qt::LinksAccessibleByMouse|Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse <b>URL:</b> true Qt::LinksAccessibleByMouse|Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse 0 9 0 0 Qt::Vertical 20 0 QFrame::StyledPanel QFrame::Raised 0 0 20 16777215 < Qt::AlignCenter 0 0 20 16777215 > Qt::AlignCenter Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Qt::Vertical 13 0 Qt::Horizontal 0 0 true 2 0 -1 weboob-1.2/weboob/applications/qhavedate/ui/search.ui000066400000000000000000000063031303450110500227510ustar00rootroot00000000000000 Search 0 0 438 349 Form 4 0 300 16777215 QFrame::StyledPanel QFrame::Raised <b>Profiles in queue:</b> Qt::Horizontal 40 20 QFrame::StyledPanel QFrame::Raised Query Next true 0 0 178 235 weboob-1.2/weboob/applications/qhavedate/ui/thread_message.ui000066400000000000000000000053701303450110500244620ustar00rootroot00000000000000 ThreadMessage 0 0 552 76 Frame QFrame::StyledPanel QFrame::Raised Qt::Vertical 20 1 0 0 true Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse true Qt::LinksAccessibleByMouse|Qt::TextSelectableByMouse Qt::Vertical 20 1 weboob-1.2/weboob/applications/qvideoob/000077500000000000000000000000001303450110500203745ustar00rootroot00000000000000weboob-1.2/weboob/applications/qvideoob/__init__.py000066400000000000000000000000671303450110500225100ustar00rootroot00000000000000from .qvideoob import QVideoob __all__ = ['QVideoob'] weboob-1.2/weboob/applications/qvideoob/main_window.py000066400000000000000000000123741303450110500232700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtCore import pyqtSlot as Slot from weboob.capabilities.video import CapVideo from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.models import BackendListModel from weboob.applications.qvideoob.ui.main_window_ui import Ui_MainWindow from .video import Video from .minivideo import MiniVideo class MainWindow(QtMainWindow): def __init__(self, config, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.minivideos = [] self.app = app self.ui.sortbyEdit.setCurrentIndex(int(self.config.get('settings', 'sortby'))) self.ui.nsfwCheckBox.setChecked(int(self.config.get('settings', 'nsfw'))) self.ui.sfwCheckBox.setChecked(int(self.config.get('settings', 'sfw'))) self.ui.searchEdit.returnPressed.connect(self.search) self.ui.urlEdit.returnPressed.connect(self.openURL) self.ui.nsfwCheckBox.stateChanged.connect(self.nsfwChanged) self.ui.sfwCheckBox.stateChanged.connect(self.sfwChanged) self.ui.actionBackends.triggered.connect(self.backendsConfig) self.loadBackendsList() if self.ui.backendEdit.count() == 0: self.backendsConfig() @Slot() def backendsConfig(self): bckndcfg = BackendCfg(self.weboob, (CapVideo,), self) if bckndcfg.run(): self.loadBackendsList() def loadBackendsList(self): model = BackendListModel(self.weboob) model.addBackends() self.ui.backendEdit.setModel(model) current_backend = self.config.get('settings', 'backend') idx = self.ui.backendEdit.findData(current_backend) if idx >= 0: self.ui.backendEdit.setCurrentIndex(idx) if self.ui.backendEdit.count() == 0: self.ui.searchEdit.setEnabled(False) self.ui.urlEdit.setEnabled(False) else: self.ui.searchEdit.setEnabled(True) self.ui.urlEdit.setEnabled(True) @Slot(int) def nsfwChanged(self, state): self.config.set('settings', 'nsfw', int(self.ui.nsfwCheckBox.isChecked())) self.updateVideosDisplay() @Slot(int) def sfwChanged(self, state): self.config.set('settings', 'sfw', int(self.ui.sfwCheckBox.isChecked())) self.updateVideosDisplay() def updateVideosDisplay(self): for minivideo in self.minivideos: if (minivideo.video.nsfw and self.ui.nsfwCheckBox.isChecked() or not minivideo.video.nsfw and self.ui.sfwCheckBox.isChecked()): minivideo.show() else: minivideo.hide() @Slot() def search(self): pattern = self.ui.searchEdit.text() if not pattern: return for minivideo in self.minivideos: self.ui.scrollAreaContent.layout().removeWidget(minivideo) minivideo.hide() minivideo.deleteLater() self.minivideos = [] self.ui.searchEdit.setEnabled(False) backend_name = self.ui.backendEdit.itemData(self.ui.backendEdit.currentIndex()) def finished(): self.ui.searchEdit.setEnabled(True) self.process = None self.process = QtDo(self.weboob, self.addVideo, fb=finished) self.process.do(self.app._do_complete, 20, (), 'search_videos', pattern, self.ui.sortbyEdit.currentIndex(), nsfw=True, backends=backend_name) def addVideo(self, video): minivideo = MiniVideo(self.weboob, self.weboob[video.backend], video) self.ui.scrollAreaContent.layout().addWidget(minivideo) self.minivideos.append(minivideo) if (video.nsfw and not self.ui.nsfwCheckBox.isChecked() or not video.nsfw and not self.ui.sfwCheckBox.isChecked()): minivideo.hide() @Slot() def openURL(self): url = self.ui.urlEdit.text() if not url: return for backend in self.weboob.iter_backends(): video = backend.get_video(url) if video: video_widget = Video(video, self) video_widget.show() self.ui.urlEdit.clear() def closeEvent(self, ev): self.config.set('settings', 'backend', self.ui.backendEdit.itemData(self.ui.backendEdit.currentIndex())) self.config.set('settings', 'sortby', self.ui.sortbyEdit.currentIndex()) self.config.save() ev.accept() weboob-1.2/weboob/applications/qvideoob/minivideo.py000066400000000000000000000047731303450110500227440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtGui import QImage, QPixmap from PyQt5.QtWidgets import QFrame from weboob.tools.application.qt5 import QtDo from weboob.applications.qvideoob.ui.minivideo_ui import Ui_MiniVideo from .video import Video class MiniVideo(QFrame): def __init__(self, weboob, backend, video, parent=None): super(MiniVideo, self).__init__(parent) self.ui = Ui_MiniVideo() self.ui.setupUi(self) self.weboob = weboob self.backend = backend self.video = video self.ui.titleLabel.setText(video.title) self.ui.backendLabel.setText(backend.name) self.ui.durationLabel.setText(unicode(video.duration)) self.ui.authorLabel.setText(unicode(video.author)) self.ui.dateLabel.setText(video.date and unicode(video.date) or '') if video.rating_max: self.ui.ratingLabel.setText('%s / %s' % (video.rating, video.rating_max)) else: self.ui.ratingLabel.setText('%s' % video.rating) self.process_thumbnail = QtDo(self.weboob, self.gotThumbnail) self.process_thumbnail.do('fillobj', self.video, ['thumbnail'], backends=backend) def gotThumbnail(self, video): if video.thumbnail and video.thumbnail.data: img = QImage.fromData(video.thumbnail.data) self.ui.imageLabel.setPixmap(QPixmap.fromImage(img)) def enterEvent(self, event): self.setFrameShadow(self.Sunken) QFrame.enterEvent(self, event) def leaveEvent(self, event): self.setFrameShadow(self.Raised) QFrame.leaveEvent(self, event) def mousePressEvent(self, event): QFrame.mousePressEvent(self, event) video = self.backend.fillobj(self.video) if video: video_widget = Video(video, self) video_widget.show() weboob-1.2/weboob/applications/qvideoob/qvideoob.py000066400000000000000000000031001303450110500225500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.video import CapVideo from weboob.tools.application.qt5 import QtApplication from .main_window import MainWindow class QVideoob(QtApplication): APPNAME = 'qvideoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-2014 Romain Bignon' DESCRIPTION = "Qt application allowing to search videos on various websites and play them." SHORT_DESCRIPTION = "search and play videos" CAPS = CapVideo CONFIG = {'settings': {'nsfw': 1, 'sfw': 1, 'sortby': 0, 'backend': '' } } def main(self, argv): self.load_backends(CapVideo) self.load_config() self.main_window = MainWindow(self.config, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qvideoob/ui/000077500000000000000000000000001303450110500210115ustar00rootroot00000000000000weboob-1.2/weboob/applications/qvideoob/ui/Makefile000066400000000000000000000002651303450110500224540ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qvideoob/ui/__init__.py000066400000000000000000000000001303450110500231100ustar00rootroot00000000000000weboob-1.2/weboob/applications/qvideoob/ui/main_window.ui000066400000000000000000000124111303450110500236620ustar00rootroot00000000000000 MainWindow 0 0 582 463 QVideoob QFrame::StyledPanel QFrame::Raised Search: 0 0 Relevance Rating Duration Date 10 0 0 0 Display: SFW true NSFW true Qt::Horizontal 40 20 true 0 0 560 230 QFrame::StyledPanel QFrame::Raised URL: 0 0 582 25 toolBar TopToolBarArea false Backends weboob-1.2/weboob/applications/qvideoob/ui/minivideo.ui000066400000000000000000000113401303450110500233320ustar00rootroot00000000000000 MiniVideo 0 0 464 132 Form QFrame::StyledPanel QFrame::Raised 9 5 5 0 0 QFormLayout::ExpandingFieldsGrow 75 true Title 0 0 50 true false TextLabel 75 true Duration TextLabel 75 true Author TextLabel 75 true Date TextLabel 75 true Rating TextLabel 75 true Where TextLabel weboob-1.2/weboob/applications/qvideoob/ui/video.ui000066400000000000000000000131751303450110500224650ustar00rootroot00000000000000 Video 0 0 647 404 Video 12 75 true QFrame::Box QFrame::Raised TextLabel Qt::AlignCenter 0 0 QFrame::StyledPanel QFrame::Sunken 0 0 Qt::Horizontal QFrame::StyledPanel QFrame::Raised QFormLayout::ExpandingFieldsGrow 75 true URL true ArrowCursor true false true 75 true Duration TextLabel 75 true Author TextLabel 75 true Date TextLabel 75 true Rating TextLabel QVideoWidget QWidget

PyQt5.QtMultimediaWidgets
1 weboob-1.2/weboob/applications/qvideoob/video.py000066400000000000000000000051111303450110500220520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtCore import QUrl, pyqtSlot as Slot from PyQt5.QtWidgets import QDialog from PyQt5.QtMultimedia import QMediaContent, QMediaPlayer from weboob.applications.qvideoob.ui.video_ui import Ui_Video class Video(QDialog): def __init__(self, video, parent=None): super(Video, self).__init__(parent) self.ui = Ui_Video() self.ui.setupUi(self) self.video = video self.setWindowTitle("Video - %s" % video.title) self.ui.urlEdit.setText(video.url) self.ui.titleLabel.setText(video.title) self.ui.durationLabel.setText(unicode(video.duration)) self.ui.authorLabel.setText(unicode(video.author)) self.ui.dateLabel.setText(unicode(video.date)) if video.rating_max: self.ui.ratingLabel.setText('%s / %s' % (video.rating, video.rating_max)) else: self.ui.ratingLabel.setText('%s' % video.rating) self.mediaPlayer = QMediaPlayer() self.mediaPlayer.durationChanged.connect(self._setMax) self.mediaPlayer.seekableChanged.connect(self.ui.seekSlider.setEnabled) self.mediaPlayer.positionChanged.connect(self._slide) self.ui.seekSlider.valueChanged.connect(self.mediaPlayer.setPosition) mc = QMediaContent(QUrl(video.url)) self.mediaPlayer.setMedia(mc) self.ui.videoPlayer.setMediaObject(self.mediaPlayer) self.mediaPlayer.play() @Slot('qint64') def _slide(self, pos): blocking = self.ui.seekSlider.blockSignals(True) self.ui.seekSlider.setValue(pos) self.ui.seekSlider.blockSignals(blocking) @Slot('qint64') def _setMax(self, duration): self.ui.seekSlider.setMaximum(duration) def closeEvent(self, event): self.mediaPlayer.stop() event.accept() def hideEvent(self, event): self.mediaPlayer.stop() event.accept() weboob-1.2/weboob/applications/qwebcontentedit/000077500000000000000000000000001303450110500217635ustar00rootroot00000000000000weboob-1.2/weboob/applications/qwebcontentedit/__init__.py000066400000000000000000000001151303450110500240710ustar00rootroot00000000000000from .qwebcontentedit import QWebContentEdit __all__ = ['QWebContentEdit'] weboob-1.2/weboob/applications/qwebcontentedit/main_window.py000066400000000000000000000242471303450110500246610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import logging from copy import deepcopy from PyQt5.QtCore import pyqtSlot as Slot from PyQt5.QtWidgets import QMessageBox, QTableWidgetItem from PyQt5.QtCore import Qt from weboob.tools.application.base import MoreResultsAvailable from weboob.tools.application.qt5 import QtMainWindow, QtDo from weboob.tools.application.qt5.backendcfg import BackendCfg from weboob.tools.application.qt5.models import BackendListModel from weboob.capabilities.content import CapContent from weboob.tools.misc import to_unicode from .ui.main_window_ui import Ui_MainWindow class MainWindow(QtMainWindow): def __init__(self, config, weboob, app, parent=None): super(MainWindow, self).__init__(parent) self.ui = Ui_MainWindow() self.ui.setupUi(self) self.config = config self.weboob = weboob self.backend = None self.app = app self.ui.idEdit.returnPressed.connect(self.loadPage) self.ui.loadButton.clicked.connect(self.loadPage) self.ui.tabWidget.currentChanged.connect(self._currentTabChanged) self.ui.saveButton.clicked.connect(self.savePage) self.ui.actionBackends.triggered.connect(self.backendsConfig) self.ui.contentEdit.textChanged.connect(self._textChanged) self.ui.loadHistoryButton.clicked.connect(self.loadHistory) if hasattr(self.ui.descriptionEdit, "setPlaceholderText"): self.ui.descriptionEdit.setPlaceholderText("Edit summary") if self.weboob.count_backends() == 0: self.backendsConfig() else: self.loadBackends() @Slot() def backendsConfig(self): """ Opens backends configuration dialog when 'Backends' is clicked """ bckndcfg = BackendCfg(self.weboob, (CapContent,), self) if bckndcfg.run(): self.loadBackends() def loadBackends(self): """ Fills the backends comboBox with available backends """ model = BackendListModel(self.weboob) model.addBackends(entry_all=False) self.ui.backendBox.setModel(model) @Slot() def _currentTabChanged(self): """ Loads history or preview when the corresponding tabs are shown """ if self.ui.tabWidget.currentIndex() == 1: if self.backend is not None: self.loadPreview() elif self.ui.tabWidget.currentIndex() == 2: if self.backend is not None: self.loadHistory() @Slot() def _textChanged(self): """ The text in the content QPlainTextEdit has changed """ if self.backend: self.ui.saveButton.setEnabled(True) self.ui.saveButton.setText('Save') @Slot() def loadPage(self): """ Loads a page's source into the 'content' QPlainTextEdit """ _id = self.ui.idEdit.text() if not _id: return self.ui.loadButton.setEnabled(False) self.ui.loadButton.setText('Loading...') self.ui.contentEdit.setReadOnly(True) backend = self.ui.backendBox.currentText() def finished(): self.process = None if self.backend: self.ui.contentEdit.setReadOnly(False) self.ui.loadButton.setEnabled(True) self.ui.loadButton.setText('Load') self.process = QtDo(self.weboob, self._loadedPage, self._errorLoadPage, finished) self.process.do('get_content', _id, backends=(backend,)) def _loadedPage(self, data): """ Callback for loadPage """ if not data: self.content = None self.backend = None QMessageBox.critical(self, self.tr('Unable to open page'), 'Unable to open page "%s" on %s: it does not exist.' % (self.ui.idEdit.text(), self.ui.backendBox.currentText()), QMessageBox.Ok) return self.content = data self.ui.contentEdit.setPlainText(self.content.content) self.setWindowTitle("QWebcontentedit - %s@%s" %(self.content.id, self.content.backend)) self.backend = self.weboob[self.content.backend] def _errorLoadPage(self, backend, error, backtrace): """ Error callback for loadPage """ content = self.tr('Unable to load page:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while loading page'), content, QMessageBox.Ok) self.ui.loadButton.setEnabled(True) self.ui.loadButton.setText("Load") @Slot() def savePage(self): """ Saves the current page to the remote site """ if self.backend is None: return new_content = self.ui.contentEdit.toPlainText() minor = self.ui.minorBox.isChecked() if new_content != self.content.content: self.ui.saveButton.setEnabled(False) self.ui.saveButton.setText('Saving...') self.ui.contentEdit.setReadOnly(True) self.content.content = new_content message = self.ui.descriptionEdit.text() def finished(): self.process = None self.ui.saveButton.setText('Saved') self.ui.descriptionEdit.clear() self.ui.contentEdit.setReadOnly(False) self.process = QtDo(self.weboob, None, self._errorSavePage, finished) self.process.do('push_content', self.content, message, minor=minor, backends=self.backend) def _errorSavePage(self, backend, error, backtrace): """ """ content = self.tr('Unable to save page:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while saving page'), content, QMessageBox.Ok) self.ui.saveButton.setEnabled(True) self.ui.saveButton.setText("Save") def loadPreview(self): """ Loads the current page's preview into the preview QTextEdit """ tmp_content = deepcopy(self.content) tmp_content.content = self.ui.contentEdit.toPlainText() self.ui.previewEdit.setHtml(self.backend.get_content_preview(tmp_content)) @Slot() def loadHistory(self): """ Loads the page's log into the 'History' tab """ if self.backend is None: return self.ui.loadHistoryButton.setEnabled(False) self.ui.loadHistoryButton.setText("Loading...") self.ui.historyTable.clear() self.ui.historyTable.setRowCount(0) self.ui.historyTable.setHorizontalHeaderLabels(["Revision", "Time", "Author", "Summary"]) self.ui.historyTable.setColumnWidth(3, 1000) def finished(): self.process = None self.ui.loadHistoryButton.setEnabled(True) self.ui.loadHistoryButton.setText("Reload") self.process = QtDo(self.weboob, self._gotRevision, self._errorHistory, finished) self.process.do(self.app._do_complete, self.ui.nbRevBox.value(), (), 'iter_revisions', self.content.id, backends=(self.backend,)) def _gotRevision(self, revision): """ Callback for loadHistory's QtDo """ # we set the flags to Qt.ItemIsEnabled so that the items # are not modifiable (they are modifiable by default) item_revision = QTableWidgetItem(revision.id) item_revision.setFlags(Qt.ItemIsEnabled) item_time = QTableWidgetItem(revision.timestamp.strftime('%Y-%m-%d %H:%M:%S')) item_time.setFlags(Qt.ItemIsEnabled) item_author = QTableWidgetItem(revision.author) item_author.setFlags(Qt.ItemIsEnabled) item_summary = QTableWidgetItem(revision.comment) item_summary.setFlags(Qt.ItemIsEnabled) row = self.ui.historyTable.currentRow() + 1 self.ui.historyTable.insertRow(row) self.ui.historyTable.setItem(row, 0, item_revision) self.ui.historyTable.setItem(row, 1, item_time) self.ui.historyTable.setItem(row, 2, item_author) self.ui.historyTable.setItem(row, 3, item_summary) self.ui.historyTable.setCurrentCell(row, 0) def _errorHistory(self, backend, error, backtrace): """ Loading the history has failed """ if isinstance(error, MoreResultsAvailable): return content = self.tr('Unable to load history:\n%s\n') % to_unicode(error) if logging.root.level <= logging.DEBUG: content += '\n%s\n' % to_unicode(backtrace) QMessageBox.critical(self, self.tr('Error while loading history'), content, QMessageBox.Ok) self.ui.loadHistoryButton.setEnabled(True) self.ui.loadHistoryButton.setText("Reload") weboob-1.2/weboob/applications/qwebcontentedit/qwebcontentedit.py000066400000000000000000000026141303450110500255370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.application.qt5 import QtApplication from weboob.capabilities.content import CapContent from .main_window import MainWindow class QWebContentEdit(QtApplication): APPNAME = 'qwebcontentedit' VERSION = '1.2' COPYRIGHT = u'Copyright(C) 2011-2014 Clément Schreiner' DESCRIPTION = "Qt application allowing to manage content of various websites." SHORT_DESCRIPTION = "manage websites content" CAPS = CapContent def main(self, argv): self.load_backends(CapContent, storage=self.create_storage()) self.main_window = MainWindow(self.config, self.weboob, self) self.main_window.show() return self.weboob.loop() weboob-1.2/weboob/applications/qwebcontentedit/ui/000077500000000000000000000000001303450110500224005ustar00rootroot00000000000000weboob-1.2/weboob/applications/qwebcontentedit/ui/Makefile000066400000000000000000000002651303450110500240430ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/applications/qwebcontentedit/ui/__init__.py000066400000000000000000000000001303450110500244770ustar00rootroot00000000000000weboob-1.2/weboob/applications/qwebcontentedit/ui/main_window.ui000066400000000000000000000140561303450110500252600ustar00rootroot00000000000000 MainWindow 0 0 608 609 QWebcontentedit false Load 0 Edit true Preview true History New Column Time Author Summary Qt::Horizontal 40 20 Number of revisions 10 Reload Minor edit false No changes 0 0 608 22 File toolBar TopToolBarArea false Exit Backends actionExit triggered() MainWindow close() -1 -1 343 306 weboob-1.2/weboob/applications/qweboobcfg/000077500000000000000000000000001303450110500207025ustar00rootroot00000000000000weboob-1.2/weboob/applications/qweboobcfg/__init__.py000066400000000000000000000014351303450110500230160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .qweboobcfg import QWeboobCfg __all__ = ['QWeboobCfg'] weboob-1.2/weboob/applications/qweboobcfg/qweboobcfg.py000066400000000000000000000024501303450110500233730ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.application.qt5 import BackendCfg, QtApplication class QWeboobCfg(QtApplication): APPNAME = 'qweboobcfg' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-2014 Romain Bignon' DESCRIPTION = "weboob-config-qt is a graphical application to add/edit/remove backends, " \ "and to register new website accounts." SHORT_DESCRIPTION = "manage backends or register new accounts" def main(self, argv): self.load_backends() self.dlg = BackendCfg(self.weboob) self.dlg.show() return self.weboob.loop() weboob-1.2/weboob/applications/radioob/000077500000000000000000000000001303450110500202035ustar00rootroot00000000000000weboob-1.2/weboob/applications/radioob/__init__.py000066400000000000000000000014241303450110500223150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .radioob import Radioob __all__ = ['Radioob'] weboob-1.2/weboob/applications/radioob/radioob.py000066400000000000000000000371511303450110500222030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import subprocess import os import re import requests from weboob.capabilities.radio import CapRadio, Radio from weboob.capabilities.audio import CapAudio, BaseAudio, Playlist, Album from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.media_player import InvalidMediaPlayer, MediaPlayer, MediaPlayerNotFound from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['Radioob'] class RadioListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title') def get_title(self, obj): return obj.title def get_description(self, obj): result = '' if hasattr(obj, 'description') and not empty(obj.description): result += '%-30s' % obj.description if hasattr(obj, 'current') and not empty(obj.current): if obj.current.who: result += ' (Current: %s - %s)' % (obj.current.who, obj.current.what) else: result += ' (Current: %s)' % obj.current.what return result class SongListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title') def get_title(self, obj): result = obj.title if hasattr(obj, 'author') and not empty(obj.author): result += ' (%s)' % obj.author return result def get_description(self, obj): result = '' if hasattr(obj, 'description') and not empty(obj.description): result += '%-30s' % obj.description return result class AlbumTrackListInfoFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'tracks_list') def get_title(self, obj): result = obj.title if hasattr(obj, 'author') and not empty(obj.author): result += ' (%s)' % obj.author return result def get_description(self, obj): result = '' for song in obj.tracks_list: result += '- %s%-30s%s ' % (self.BOLD, song.title, self.NC) if hasattr(song, 'duration') and not empty(song.duration): result += '%-10s ' % song.duration else: result += '%-10s ' % ' ' result += '(%s)\r\n\t' % (song.id) return result class PlaylistTrackListInfoFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'tracks_list') def get_title(self, obj): return obj.title def get_description(self, obj): result = '' for song in obj.tracks_list: result += '- %s%-30s%s ' % (self.BOLD, song.title, self.NC) if hasattr(song, 'author') and not empty(song.author): result += '(%-15s) ' % song.author if hasattr(song, 'duration') and not empty(song.duration): result += '%-10s ' % song.duration else: result += '%-10s ' % ' ' result += '(%s)\r\n\t' % (song.id) return result class Radioob(ReplApplication): APPNAME = 'radioob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon\nCopyright(C) YEAR Pierre Maziere' DESCRIPTION = "Console application allowing to search for web radio stations, listen to them and get information " \ "like the current song." SHORT_DESCRIPTION = "search, show or listen to radio stations" CAPS = (CapRadio, CapAudio) EXTRA_FORMATTERS = {'radio_list': RadioListFormatter, 'song_list': SongListFormatter, 'album_tracks_list_info': AlbumTrackListInfoFormatter, 'playlist_tracks_list_info': PlaylistTrackListInfoFormatter, } COMMANDS_FORMATTERS = {'ls': 'radio_list', 'playlist': 'radio_list', } COLLECTION_OBJECTS = (Radio, BaseAudio, ) PLAYLIST = [] def __init__(self, *args, **kwargs): ReplApplication.__init__(self, *args, **kwargs) self.player = MediaPlayer(self.logger) def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def complete_download(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_download(self, line): """ download ID [DIRECTORY] Download an audio file """ _id, dest = self.parse_command_args(line, 2, 1) obj = self.retrieve_obj(_id) if obj is None: print('No object matches with this id:', _id, file=self.stderr) return 3 if isinstance(obj, BaseAudio): streams = [obj] else: streams = obj.tracks_list if len(streams) == 0: print('Radio or Audio file not found:', _id, file=self.stderr) return 3 for stream in streams: self.download_file(stream, dest) def download_file(self, audio, dest): _obj = self.get_object(audio.id, 'get_audio', ['url', 'title']) if not _obj: print('Audio file not found: %s' % audio.id, file=self.stderr) return 3 if not _obj.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 audio.url = _obj.url def check_exec(executable): with open(os.devnull, 'w') as devnull: process = subprocess.Popen(['which', executable], stdout=devnull) if process.wait() != 0: print('Please install "%s"' % executable, file=self.stderr) return False return True def audio_to_file(_audio): ext = _audio.ext if not ext: ext = 'audiofile' title = _audio.title if _audio.title else _audio.id return '%s.%s' % (re.sub('[?:/]', '-', title), ext) if dest is not None and os.path.isdir(dest): dest += '/%s' % audio_to_file(audio) if dest is None: dest = audio_to_file(audio) if audio.url.startswith('rtmp'): if not check_exec('rtmpdump'): return 1 args = ('rtmpdump', '-e', '-r', audio.url, '-o', dest) elif audio.url.startswith('mms'): if not check_exec('mimms'): return 1 args = ('mimms', '-r', audio.url, dest) else: if check_exec('wget'): args = ('wget', '-c', audio.url, '-O', dest) elif check_exec('curl'): args = ('curl', '-C', '-', audio.url, '-o', dest) else: return 1 os.spawnlp(os.P_WAIT, args[0], *args) def complete_play(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_play(self, line): """ play ID [stream_id] Play a radio or a audio file with a found player (optionnaly specify the wanted stream). """ _id, stream_id = self.parse_command_args(line, 2, 1) if not _id: print('This command takes an argument: %s' % self.get_command_help('play', short=True), file=self.stderr) return 2 try: stream_id = int(stream_id) except (ValueError, TypeError): stream_id = 0 obj = self.retrieve_obj(_id) if obj is None: print('No object matches with this id:', _id, file=self.stderr) return 3 if isinstance(obj, Radio): try: streams = [obj.streams[stream_id]] except IndexError: print('Stream %d not found' % stream_id, file=self.stderr) return 1 elif isinstance(obj, BaseAudio): streams = [obj] else: streams = obj.tracks_list if len(streams) == 0: print('Radio or Audio file not found:', _id, file=self.stderr) return 3 try: player_name = self.config.get('media_player') media_player_args = self.config.get('media_player_args') if not player_name: self.logger.debug(u'You can set the media_player key to the player you prefer in the radioob ' 'configuration file.') for stream in streams: if isinstance(stream, BaseAudio) and not stream.url: stream = self.get_object(stream.id, 'get_audio') else: r = requests.get(stream.url, stream=True) buf = r.iter_content(512).next() r.close() playlistFormat = None for line in buf.split("\n"): if playlistFormat is None: if line == "[playlist]": playlistFormat = "pls" elif line == "#EXTM3U": playlistFormat = "m3u" else: break elif playlistFormat == "pls": if line.startswith('File'): stream.url = line.split('=', 1).pop(1).strip() break elif playlistFormat == "m3u": if line[0] != "#": stream.url = line.strip() break self.player.play(stream, player_name=player_name, player_args=media_player_args) except (InvalidMediaPlayer, MediaPlayerNotFound) as e: print('%s\nRadio URL: %s' % (e, stream.url)) def retrieve_obj(self, _id): obj = None if self.interactive: try: obj = self.objects[int(_id) - 1] _id = obj.id except (IndexError, ValueError): pass m = CapAudio.get_object_method(_id) if m: obj = self.get_object(_id, m) return obj if obj is not None else self.get_object(_id, 'get_radio') def do_playlist(self, line): """ playlist cmd [args] playlist add ID [ID2 ID3 ...] playlist remove ID [ID2 ID3 ...] playlist export [FILENAME] playlist display """ if not line: print('This command takes an argument: %s' % self.get_command_help('playlist'), file=self.stderr) return 2 cmd, args = self.parse_command_args(line, 2, req_n=1) if cmd == "add": _ids = args.strip().split(' ') for _id in _ids: audio = self.get_object(_id, 'get_audio') if not audio: print('Audio file not found: %s' % _id, file=self.stderr) return 3 if not audio.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 self.PLAYLIST.append(audio) elif cmd == "remove": _ids = args.strip().split(' ') for _id in _ids: audio_to_remove = self.get_object(_id, 'get_audio') if not audio_to_remove: print('Audio file not found: %s' % _id, file=self.stderr) return 3 if not audio_to_remove.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 for audio in self.PLAYLIST: if audio.id == audio_to_remove.id: self.PLAYLIST.remove(audio) break elif cmd == "export": filename = "playlist.m3u" if args: filename = args file = open(filename, 'w') for audio in self.PLAYLIST: file.write('%s\r\n' % audio.url) file.close() elif cmd == "display": for audio in self.PLAYLIST: self.cached_format(audio) else: print('Playlist command only support "add", "remove", "display" and "export" arguments.', file=self.stderr) return 2 def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, _id): """ info ID Get information about a radio or an audio file. """ if not _id: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 obj = self.retrieve_obj(_id) if isinstance(obj, Album): self.set_formatter('album_tracks_list_info') elif isinstance(obj, Playlist): self.set_formatter('playlist_tracks_list_info') if obj is None: print('No object matches with this id:', _id, file=self.stderr) return 3 self.format(obj) @defaultcount(10) def do_search(self, pattern=None): """ search (radio|song|file|album|playlist) PATTERN List (radio|song|file|album|playlist) matching a PATTERN. If PATTERN is not given, this command will list all the (radio|song|album|playlist). """ if not pattern: print('This command takes an argument: %s' % self.get_command_help('playlist'), file=self.stderr) return 2 cmd, args = self.parse_command_args(pattern, 2, req_n=1) if not args: args = "" self.set_formatter_header(u'Search pattern: %s' % pattern if pattern else u'All radios') self.change_path([u'search']) if cmd == "radio": self.set_formatter('radio_list') for radio in self.do('iter_radios_search', pattern=args): self.add_object(radio) self.format(radio) elif cmd == "song" or cmd == "file": self.set_formatter('song_list') for audio in self.do('search_audio', pattern=args): self.add_object(audio) self.format(audio) elif cmd == "album": self.set_formatter('song_list') for album in self.do('search_album', pattern=args): self.add_object(album) self.format(album) elif cmd == "playlist": self.set_formatter('song_list') for playlist in self.do('search_playlist', pattern=args): self.add_object(playlist) self.format(playlist) else: print('Search command only supports "radio", "song", "file", "album" and "playlist" arguments.', file=self.stderr) return 2 def do_ls(self, line): """ ls List radios """ ret = super(Radioob, self).do_ls(line) return ret weboob-1.2/weboob/applications/shopoob/000077500000000000000000000000001303450110500202355ustar00rootroot00000000000000weboob-1.2/weboob/applications/shopoob/__init__.py000066400000000000000000000014241303450110500223470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .shopoob import Shopoob __all__ = ['Shopoob'] weboob-1.2/weboob/applications/shopoob/shopoob.py000066400000000000000000000150441303450110500222640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Christophe Lampin # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from decimal import Decimal from weboob.capabilities.base import empty from weboob.capabilities.shop import CapShop, Order, Item from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter __all__ = ['Shopoob'] class OrdersFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'total') def start_format(self, **kwargs): self.output(' Id Date Total ') self.output('-----------------------------+------------+-----------') def format_obj(self, obj, alias): date = obj.date.strftime('%Y-%m-%d') if not empty(obj.date) else '' total = obj.total or Decimal('0') result = u'%s %s %s' % (self.colored('%-28s' % obj.fullid, 'yellow'), self.colored('%-10s' % date, 'blue'), self.colored('%9.2f' % total, 'green')) return result def flush(self): self.output(u'----------------------------+------------+-----------') class ItemsFormatter(IFormatter): MANDATORY_FIELDS = ('label', 'url', 'price') def start_format(self, **kwargs): self.output(' Label Url Price ') self.output('---------------------------------------------------------------------------+---------------------------------------------+----------') def format_obj(self, obj, alias): price = obj.price or Decimal('0') result = u'%s %s %s' % (self.colored('%-75s' % obj.label[:75], 'yellow'), self.colored('%-43s' % obj.url, 'magenta'), self.colored('%9.2f' % price, 'green')) return result def flush(self): self.output(u'---------------------------------------------------------------------------+---------------------------------------------+----------') class PaymentsFormatter(IFormatter): MANDATORY_FIELDS = ('date', 'method', 'amount') def start_format(self, **kwargs): self.output(' Date Method Amount ') self.output('-----------+-----------------+----------') def format_obj(self, obj, alias): date = obj.date.strftime('%Y-%m-%d') if not empty(obj.date) else '' amount = obj.amount or Decimal('0') result = u'%s %s %s' % (self.colored('%-10s' % date, 'blue'), self.colored('%-17s' % obj.method, 'yellow'), self.colored('%9.2f' % amount, 'green')) return result def flush(self): self.output(u'-----------+-----------------+----------') class Shopoob(ReplApplication): APPNAME = 'shopoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2015 Christophe Lampin' DESCRIPTION = 'Console application to obtain details and status of e-commerce orders.' SHORT_DESCRIPTION = "Obtain details and status of e-commerce orders" CAPS = CapShop COLLECTION_OBJECTS = (Order, ) EXTRA_FORMATTERS = {'orders': OrdersFormatter, 'items': ItemsFormatter, 'payments': PaymentsFormatter } DEFAULT_FORMATTER = 'table' COMMANDS_FORMATTERS = {'orders': 'orders', 'items': 'items', 'payments': 'payments', 'ls': 'orders', } def main(self, argv): self.load_config() return ReplApplication.main(self, argv) @defaultcount(10) def do_orders(self, line): """ orders [BACKEND_NAME] Get orders of a backend. If no BACKEND_NAME given, display all orders of all backends. """ if len(line) > 0: backend_name = line else: backend_name = None self.do_count(str(self.options.count)) # Avoid raise of MoreResultsAvailable l = [] for order in self.do('iter_orders', backends=backend_name): l.append(order) self.start_format() for order in sorted(l, self.comp_object): self.format(order) # Order by date DESC def comp_object(self, obj1, obj2): if obj1.date == obj2.date: return 0 elif obj1.date < obj2.date: return 1 else: return -1 def do_items(self, id): """ items [ID] Get items of orders. """ l = [] id, backend_name = self.parse_id(id, unique_backend=True) if not id: print('Error: please give a order ID (hint: use orders command)', file=self.stderr) return 2 else: l.append((id, backend_name)) for id, backend in l: names = (backend,) if backend is not None else None # TODO: Use specific formatter mysum = Item() mysum.label = u"Sum" mysum.url = u"Generated by shopoob" mysum.price = Decimal("0.") self.start_format() for item in self.do('iter_items', id, backends=names): self.format(item) mysum.price = item.price + mysum.price self.format(mysum) def do_payments(self, id): """ payments [ID] Get payments of orders. If no ID given, display payment of all backends. """ id, backend_name = self.parse_id(id, unique_backend=True) if not id: print('Error: please give a order ID (hint: use orders command)', file=self.stderr) return 2 self.start_format() for payment in self.do('iter_payments', id, backends=backend_name): self.format(payment)weboob-1.2/weboob/applications/suboob/000077500000000000000000000000001303450110500200555ustar00rootroot00000000000000weboob-1.2/weboob/applications/suboob/__init__.py000066400000000000000000000014151303450110500221670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .suboob import Suboob __all__ = ['Suboob'] weboob-1.2/weboob/applications/suboob/suboob.py000066400000000000000000000170501303450110500217230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.subtitle import CapSubtitle from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['Suboob'] LANGUAGE_CONV = { 'ar': 'ara', 'eo': 'epo', 'ga': '', 'ru': 'rus', 'af': '', 'et': 'est', 'it': 'ita', 'sr': 'scc', 'sq': 'alb', 'tl': '', 'ja': 'jpn', 'sk': 'slo', 'hy': 'arm', 'fi': 'fin', 'kn': '', 'sl': 'slv', 'az': '', 'fr': 'fre', 'ko': 'kor', 'es': 'spa', 'eu': 'baq', 'gl': 'glg', 'la': '', 'sw': 'swa', 'be': '', 'ka': 'geo', 'lv': 'lav', 'sv': 'swe', 'bn': 'ben', 'de': 'ger', 'lt': 'lit', 'ta': '', 'bg': 'bul', 'gr': 'ell', 'mk': 'mac', 'te': 'tel', 'ca': 'cat', 'gu': '', 'ms': 'may', 'th': 'tha', 'zh': 'chi', 'ht': '', 'mt': '', 'tr': 'tur', 'hr': 'hrv', 'iw': 'heb', 'no': 'nor', 'uk': 'ukr', 'cz': 'cze', 'hi': 'hin', 'fa': 'per', 'ur': 'urd', 'da': 'dan', 'hu': 'hun', 'pl': 'pol', 'vi': 'vie', 'nl': 'dut', 'is': 'ice', 'pt': 'por', 'cy': '', 'en': 'eng', 'id': 'ind', 'ro': 'rum', 'yi': ''} def sizeof_fmt(num): for x in ['bytes', 'KB', 'MB', 'GB', 'TB']: if num < 1024.0: return "%-4.1f%s" % (num, x) num /= 1024.0 class SubtitleInfoFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'name', 'url', 'description') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.name, self.NC) result += 'ID: %s\n' % obj.fullid result += 'URL: %s\n' % obj.url if not empty(obj.language): result += 'LANG: %s\n' % obj.language if not empty(obj.nb_cd): result += 'NB CD: %s\n' % obj.nb_cd result += '\n%sDescription%s\n' % (self.BOLD, self.NC) result += '%s' % obj.description return result class SubtitleListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name') def get_title(self, obj): return obj.name def get_description(self, obj): result = u'lang : %s' % obj.language result += ' ; %s CD' % obj.nb_cd if not empty(obj.url): result += ' ; url : %s' % obj.url return result class Suboob(ReplApplication): APPNAME = 'suboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2013-YEAR Julien Veyssier' DESCRIPTION = "Console application allowing to search for subtitles on various services " \ "and download them." SHORT_DESCRIPTION = "search and download subtitles" CAPS = CapSubtitle EXTRA_FORMATTERS = {'subtitle_list': SubtitleListFormatter, 'subtitle_info': SubtitleInfoFormatter } COMMANDS_FORMATTERS = {'search': 'subtitle_list', 'info': 'subtitle_info' } def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, id): """ info ID Get information about a subtitle. """ subtitle = self.get_object(id, 'get_subtitle') if not subtitle: print('Subtitle not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(subtitle) def complete_download(self, text, line, *ignored): args = line.split(' ', 2) if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_download(self, line): """ download ID [FILENAME] Get the subtitle or archive file. FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. """ id, dest = self.parse_command_args(line, 2, 1) subtitle = self.get_object(id, 'get_subtitle') if not subtitle: print('Subtitle not found: %s' % id, file=self.stderr) return 3 if dest is None: ext = subtitle.ext if empty(ext): ext = 'zip' dest = '%s.%s' % (subtitle.name, ext) for buf in self.do('get_subtitle_file', subtitle.id, backends=subtitle.backend): if buf: if dest == '-': self.stdout.write(buf) else: try: with open(dest, 'w') as f: f.write(buf) except IOError as e: print('Unable to write file in "%s": %s' % (dest, e), file=self.stderr) return 1 else: print('Saved to %s' % dest) return @defaultcount(10) def do_search(self, line): """ search language [PATTERN] Search subtitles. Language Abbreviation ---------------------- Arabic ar Esperanto eo Irish ga Russian ru Afrikaans af Estonian et Italian it Serbian sr Albanian sq Filipino tl Japanese ja Slovak sk Armenian hy Finnish fi Kannada kn Slovenian sl Azerbaijani az French fr Korean ko Spanish es Basque eu Galician gl Latin la Swahili sw Belarusian be Georgian ka Latvian lv Swedish sv Bengali bn German de Lithuanian lt Tamil ta Bulgarian bg Greek gr Macedonian mk Telugu te Catalan ca Gujarati gu Malay ms Thai th Chinese zh Haitian ht Maltese mt Turkish tr Croatian hr Hebrew iw Norwegian no Ukrainian uk Czech cz Hindi hi Persian fa Urdu ur Danish da Hungaric hu Polish pl Vietnamese vi Dutch nl Icelandic is Portuguese pt Welsh cy English en Indonesian id Romanian ro Yiddish yi ---------------------- """ language, pattern = self.parse_command_args(line, 2, 1) self.change_path([u'search']) if not pattern: pattern = None self.start_format(pattern=pattern) for subtitle in self.do('iter_subtitles', language=language, pattern=pattern): self.cached_format(subtitle) weboob-1.2/weboob/applications/translaboob/000077500000000000000000000000001303450110500210725ustar00rootroot00000000000000weboob-1.2/weboob/applications/translaboob/__init__.py000066400000000000000000000014341303450110500232050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .translaboob import Translaboob __all__ = ['Translaboob'] weboob-1.2/weboob/applications/translaboob/translaboob.py000066400000000000000000000134071303450110500237570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.translate import CapTranslate, TranslationFail, LanguageNotSupported from weboob.tools.application.repl import ReplApplication from weboob.tools.application.formatters.iformatter import IFormatter __all__ = ['Translaboob'] class TranslationFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'text') def format_obj(self, obj, alias): return u'%s* %s%s\n\t%s' % (self.BOLD, obj.backend, self.NC, obj.text.replace('\n', '\n\t')) class XmlTranslationFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'text') def start_format(self, **kwargs): if 'source' in kwargs: self.output('\n%s\n' % kwargs['source']) def format_obj(self, obj, alias): return u'\n%s\n' % (obj.backend, obj.text) class Translaboob(ReplApplication): APPNAME = 'translaboob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Lucien Loiseau' DESCRIPTION = "Console application to translate text from one language to another" SHORT_DESCRIPTION = "translate text from one language to another" CAPS = CapTranslate EXTRA_FORMATTERS = {'translation': TranslationFormatter, 'xmltrans': XmlTranslationFormatter, } COMMANDS_FORMATTERS = {'translate': 'translation', } LANGUAGE = { 'ar': 'Arabic', 'af': 'Afrikaans', 'sq': 'Albanian', 'hy': 'Armenian', 'az': 'Azerbaijani', 'eu': 'Basque', 'be': 'Belarusian', 'bn': 'Bengali', 'bg': 'Bulgarian', 'ca': 'Catalan', 'zh': 'Chinese', 'hr': 'Croatian', 'cz': 'Czech', 'da': 'Danish', 'nl': 'Dutch', 'en': 'English', 'eo': 'Esperanto', 'et': 'Estonian', 'tl': 'Filipino', 'fi': 'Finnish', 'fr': 'French', 'gl': 'Galician', 'ka': 'Georgian', 'de': 'German', 'gr': 'Greek', 'gu': 'Gujarati', 'ht': 'Haitian', 'iw': 'Hebrew', 'hi': 'Hindi', 'hu': 'Hungaric', 'is': 'Icelandic', 'id': 'Indonesian', 'ga': 'Irish', 'it': 'Italian', 'ja': 'Japanese', 'kn': 'Kannada', 'ko': 'Korean', 'la': 'Latin', 'lv': 'Latvian', 'lt': 'Lithuanian', 'mk': 'Macedonian', 'ms': 'Malay', 'mt': 'Maltese', 'no': 'Norwegian', 'fa': 'Persian', 'pl': 'Polish', 'pt': 'Portuguese', 'ro': 'Romanian', 'ru': 'Russian', 'sr': 'Serbian', 'sk': 'Slovak', 'sl': 'Slovenian', 'es': 'Spanish', 'sw': 'Swahili', 'sv': 'Swedish', 'ta': 'Tamil', 'te': 'Telugu', 'th': 'Thai', 'tr': 'Turkish', 'uk': 'Ukrainian', 'ur': 'Urdu', 'vi': 'Vietnamese', 'cy': 'Welsh', 'yi': 'Yiddish', 'nigger': 'Nigger!', } def do_translate(self, line): """ translate FROM TO [TEXT] Translate from one language to another. * FROM : source language * TO : destination language * TEXT : language to translate, standard input if - is given Language Abbreviation ---------------------- Arabic ar Esperanto eo Irish ga Russian ru Afrikaans af Estonian et Italian it Serbian sr Albanian sq Filipino tl Japanese ja Slovak sk Armenian hy Finnish fi Kannada kn Slovenian sl Azerbaijani az French fr Korean ko Spanish es Basque eu Galician gl Latin la Swahili sw Belarusian be Georgian ka Latvian lv Swedish sv Bengali bn German de Lithuanian lt Tamil ta Bulgarian bg Greek gr Macedonian mk Telugu te Catalan ca Gujarati gu Malay ms Thai th Chinese zh Haitian ht Maltese mt Turkish tr Croatian hr Hebrew iw Norwegian no Ukrainian uk Czech cz Hindi hi Persian fa Urdu ur Danish da Hungaric hu Polish pl Vietnamese vi Dutch nl Icelandic is Portuguese pt Welsh cy English en Indonesian id Romanian ro Yiddish yi ---------------------- """ lan_from, lan_to, text = self.parse_command_args(line, 3, 2) try: if lan_from not in self.LANGUAGE.keys(): raise LanguageNotSupported() if lan_to not in self.LANGUAGE.keys(): raise LanguageNotSupported() if not text or text == '-': text = self.acquire_input() self.start_format(source=text) for translation in self.do('translate', self.LANGUAGE[lan_from], self.LANGUAGE[lan_to], text): self.format(translation) except (TranslationFail, LanguageNotSupported) as error: print(error, file=self.stderr) pass weboob-1.2/weboob/applications/traveloob/000077500000000000000000000000001303450110500205615ustar00rootroot00000000000000weboob-1.2/weboob/applications/traveloob/__init__.py000066400000000000000000000014321303450110500226720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .traveloob import Traveloob __all__ = ['Traveloob'] weboob-1.2/weboob/applications/traveloob/traveloob.py000066400000000000000000000152261303450110500231360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Julien Hébert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import datetime from weboob.capabilities.base import Currency, empty from weboob.capabilities.travel import CapTravel, RoadmapFilters from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['Traveloob'] class DeparturesFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'type', 'departure_station', 'arrival_station', 'time') def get_title(self, obj): s = obj.type if hasattr(obj, 'price') and not empty(obj.price): s += u' %s %s' % (self.colored(u'—', 'cyan'), self.colored('%6.2f %s' % (obj.price, Currency.currency2txt(obj.currency)), 'green')) if hasattr(obj, 'late') and not empty(obj.late) and obj.late > datetime.time(): s += u' %s %s' % (self.colored(u'—', 'cyan'), self.colored('Late: %s' % obj.late, 'red', 'bold')) if hasattr(obj, 'information') and not empty(obj.information) and obj.information.strip() != '': s += u' %s %s' % (self.colored(u'—', 'cyan'), self.colored(obj.information, 'red')) return s def get_description(self, obj): if hasattr(obj, 'arrival_time') and not empty(obj.arrival_time): s = '(%s) %s%s\n\t(%s) %s' % (self.colored(obj.time.strftime('%H:%M') if obj.time else '??:??', 'cyan'), obj.departure_station, self.colored(' [Platform: %s]' % obj.platform, 'yellow') if (hasattr(obj, 'platform') and not empty(obj.platform)) else '', self.colored(obj.arrival_time.strftime('%H:%M'), 'cyan'), obj.arrival_station) else: s = '(%s) %20s -> %s' % (self.colored(obj.time.strftime('%H:%M') if obj.time else '??:??', 'cyan'), obj.departure_station, obj.arrival_station) return s class StationsFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name') def get_title(self, obj): return obj.name class Traveloob(ReplApplication): APPNAME = 'traveloob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to search for train stations and get departure times." SHORT_DESCRIPTION = "search for train stations and departures" CAPS = CapTravel DEFAULT_FORMATTER = 'table' EXTRA_FORMATTERS = {'stations': StationsFormatter, 'departures': DeparturesFormatter, } COMMANDS_FORMATTERS = {'stations': 'stations', 'departures': 'departures', } def add_application_options(self, group): group.add_option('--departure-time') group.add_option('--arrival-time') @defaultcount(10) def do_stations(self, pattern): """ stations PATTERN Search stations. """ for station in self.do('iter_station_search', pattern): self.format(station) @defaultcount(10) def do_departures(self, line): """ departures STATION [ARRIVAL [DATE]]] List all departures for a given station. The format for the date is "yyyy-mm-dd HH:MM" or "HH:MM". """ station, arrival, date = self.parse_command_args(line, 3, 1) station_id, backend_name = self.parse_id(station) if arrival: arrival_id, backend_name2 = self.parse_id(arrival) if backend_name and backend_name2 and backend_name != backend_name2: print('Departure and arrival aren\'t on the same backend', file=self.stderr) return 1 else: arrival_id = backend_name2 = None if backend_name: backends = [backend_name] elif backend_name2: backends = [backend_name2] else: backends = None if date is not None: try: date = self.parse_datetime(date) except ValueError as e: print('Invalid datetime value: %s' % e, file=self.stderr) print('Please enter a datetime in form "yyyy-mm-dd HH:MM" or "HH:MM".', file=self.stderr) return 1 for departure in self.do('iter_station_departures', station_id, arrival_id, date, backends=backends): self.format(departure) def do_roadmap(self, line): """ roadmap DEPARTURE ARRIVAL Display the roadmap to travel from DEPARTURE to ARRIVAL. Command-line parameters: --departure-time TIME requested departure time --arrival-time TIME requested arrival time TIME might be in form "yyyy-mm-dd HH:MM" or "HH:MM". Example: > roadmap Puteaux Aulnay-sous-Bois --arrival-time 22:00 """ departure, arrival = self.parse_command_args(line, 2, 2) filters = RoadmapFilters() try: filters.departure_time = self.parse_datetime(self.options.departure_time) filters.arrival_time = self.parse_datetime(self.options.arrival_time) except ValueError as e: print('Invalid datetime value: %s' % e, file=self.stderr) print('Please enter a datetime in form "yyyy-mm-dd HH:MM" or "HH:MM".', file=self.stderr) return 1 for route in self.do('iter_roadmap', departure, arrival, filters): self.format(route) def parse_datetime(self, text): if text is None: return None try: date = datetime.datetime.strptime(text, '%Y-%m-%d %H:%M') except ValueError: try: date = datetime.datetime.strptime(text, '%H:%M') except ValueError: raise ValueError(text) date = datetime.datetime.now().replace(hour=date.hour, minute=date.minute) return date weboob-1.2/weboob/applications/videoob/000077500000000000000000000000001303450110500202135ustar00rootroot00000000000000weboob-1.2/weboob/applications/videoob/__init__.py000066400000000000000000000014261303450110500223270ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .videoob import Videoob __all__ = ['Videoob'] weboob-1.2/weboob/applications/videoob/videoob.py000066400000000000000000000272611303450110500222240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Romain Bignon, John Obbele, Nicolas Duhamel # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from urlparse import urlparse import requests import subprocess import os from weboob.capabilities.video import CapVideo, BaseVideo from weboob.capabilities.base import empty from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.media_player import InvalidMediaPlayer, MediaPlayer, MediaPlayerNotFound from weboob.tools.application.formatters.iformatter import PrettyFormatter __all__ = ['Videoob'] class VideoListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'title', 'duration', 'date') DISPLAYED_FIELDS = ('author', 'rating') def get_title(self, obj): return obj.title def get_description(self, obj): if empty(obj.duration) and empty(obj.date): return None result = '%s' % (obj.duration or obj.date) if hasattr(obj, 'author') and not empty(obj.author): result += u' - %s' % obj.author if hasattr(obj, 'rating') and not empty(obj.rating): result += u' (%s/%s)' % (obj.rating, obj.rating_max) return result class Videoob(ReplApplication): APPNAME = 'videoob' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Christophe Benz, Romain Bignon, John Obbele' DESCRIPTION = "Console application allowing to search for videos on various websites, " \ "play and download them and get information." SHORT_DESCRIPTION = "search and play videos" CAPS = CapVideo EXTRA_FORMATTERS = {'video_list': VideoListFormatter} COMMANDS_FORMATTERS = {'search': 'video_list', 'ls': 'video_list', 'playlist': 'video_list'} COLLECTION_OBJECTS = (BaseVideo, ) PLAYLIST = [] nsfw = True def __init__(self, *args, **kwargs): ReplApplication.__init__(self, *args, **kwargs) self.player = MediaPlayer(self.logger) def main(self, argv): self.load_config() return ReplApplication.main(self, argv) def download(self, video, dest, default=None): if not video.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 def check_exec(executable): with open(os.devnull, 'w') as devnull: process = subprocess.Popen(['which', executable], stdout=devnull) if process.wait() != 0: print('Please install "%s"' % executable, file=self.stderr) return False return True dest = self.obj_to_filename(video, dest, default) if video.url.startswith('rtmp'): if not check_exec('rtmpdump'): return 1 args = ('rtmpdump', '-e', '-r', video.url, '-o', dest) elif video.url.startswith('mms'): if not check_exec('mimms'): return 1 args = ('mimms', '-r', video.url, dest) elif u'm3u8' == video.ext: _dest, _ = os.path.splitext(dest) dest = u'%s.%s' % (_dest, 'mp4') content = tuple() parsed_uri = urlparse(video.url) baseurl = '{uri.scheme}://{uri.netloc}'.format(uri=parsed_uri) for line in self.read_url(video.url): if not line.startswith('#'): if not line.startswith('http'): line = u'%s%s' % (baseurl, line) content += (line,) args = ('wget', '-nv',) + content + ('-O', dest) else: if check_exec('wget'): args = ('wget', '-c', video.url, '-O', dest) elif check_exec('curl'): args = ('curl', '-C', '-', video.url, '-o', dest) else: return 1 self.logger.debug(' '.join(args)) os.spawnlp(os.P_WAIT, args[0], *args) def read_url(self, url): r = requests.get(url, stream=True) return r.iter_lines() def complete_download(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_download(self, line): """ download ID [FILENAME] Download a video Braces-enclosed tags are replaced with data fields. Use the 'info' command to see what fields are available on a given video. Example: download KdRRge4XYIo@youtube '{title}.{ext}' """ _id, dest = self.parse_command_args(line, 2, 1) video = self.get_object(_id, 'get_video', ['url']) if not video: print('Video not found: %s' % _id, file=self.stderr) return 3 return self.download(video, dest) def complete_play(self, text, line, *ignored): args = line.split(' ') if len(args) >= 2: return self._complete_object() def do_play(self, line): """ play ID Play a video with a found player. """ if not line: print('This command takes an argument: %s' % self.get_command_help('play', short=True), file=self.stderr) return 2 ret = 0 for _id in line.split(' '): video = self.get_object(_id, 'get_video', ['url']) error = self.play(video, _id) if error is not None: ret = error return ret def play(self, video, _id): if not video: print('Video not found: %s' % _id, file=self.stderr) return 3 if not video.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 try: player_name = self.config.get('media_player') media_player_args = self.config.get('media_player_args') if not player_name: self.logger.info(u'You can set the media_player key to the player you prefer in the videoob ' 'configuration file.') self.player.play(video, player_name=player_name, player_args=media_player_args) except (InvalidMediaPlayer, MediaPlayerNotFound) as e: print('%s\nVideo URL: %s' % (e, video.url)) def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) >= 2: return self._complete_object() def do_info(self, line): """ info ID [ID2 [...]] Get information about a video. """ if not line: print('This command takes an argument: %s' % self.get_command_help('info', short=True), file=self.stderr) return 2 self.start_format() for _id in line.split(' '): video = self.get_object(_id, 'get_video') if not video: print('Video not found: %s' % _id, file=self.stderr) return 3 self.format(video) def complete_playlist(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return ['play', 'add', 'remove', 'export', 'display', 'download'] if len(args) >= 3: if args[1] in ('export', 'download'): return self.path_completer(args[2]) if args[1] in ('add', 'remove'): return self._complete_object() def do_playlist(self, line): """ playlist cmd [args] playlist add ID [ID2 ID3 ...] playlist remove ID [ID2 ID3 ...] playlist export [FILENAME] playlist display playlist download [PATH] playlist play """ if not self.interactive: print('This command can be used only in interactive mode.', file=self.stderr) return 1 if not line: print('This command takes an argument: %s' % self.get_command_help('playlist'), file=self.stderr) return 2 cmd, args = self.parse_command_args(line, 2, req_n=1) if cmd == "add": _ids = args.strip().split(' ') for _id in _ids: video = self.get_object(_id, 'get_video') if not video: print('Video not found: %s' % _id, file=self.stderr) return 3 if not video.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 self.PLAYLIST.append(video) elif cmd == "remove": _ids = args.strip().split(' ') for _id in _ids: video_to_remove = self.get_object(_id, 'get_video') if not video_to_remove: print('Video not found: %s' % _id, file=self.stderr) return 3 if not video_to_remove.url: print('Error: the direct URL is not available.', file=self.stderr) return 4 for video in self.PLAYLIST: if video.id == video_to_remove.id: self.PLAYLIST.remove(video) break elif cmd == "export": filename = "playlist.m3u" if args: filename = args file = open(filename, 'w') for video in self.PLAYLIST: file.write('%s\r\n' % video.url) file.close() elif cmd == "display": for video in self.PLAYLIST: self.cached_format(video) elif cmd == "download": for i, video in enumerate(self.PLAYLIST): self.download(video, args, '%02d-{id}-{title}.{ext}' % (i+1)) elif cmd == "play": for video in self.PLAYLIST: self.play(video, video.id) else: print('Playlist command only support "add", "remove", "display", "download" and "export" arguments.', file=self.stderr) return 2 def complete_nsfw(self, text, line, begidx, endidx): return ['on', 'off'] def do_nsfw(self, line): """ nsfw [on | off] If argument is given, enable or disable the non-suitable for work behavior. If no argument is given, print the current behavior. """ line = line.strip() if line: if line == 'on': self.nsfw = True elif line == 'off': self.nsfw = False else: print('Invalid argument "%s".' % line) return 2 else: print("on" if self.nsfw else "off") @defaultcount() def do_search(self, pattern): """ search PATTERN Search for videos matching a PATTERN. """ if not pattern: print('This command takes an argument: %s' % self.get_command_help('search', short=True), file=self.stderr) return 2 self.change_path([u'search']) self.start_format(pattern=pattern) for video in self.do('search_videos', pattern=pattern, nsfw=self.nsfw): self.cached_format(video) weboob-1.2/weboob/applications/webcontentedit/000077500000000000000000000000001303450110500216025ustar00rootroot00000000000000weboob-1.2/weboob/applications/webcontentedit/__init__.py000066400000000000000000000014511303450110500237140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .webcontentedit import WebContentEdit __all__ = ['WebContentEdit'] weboob-1.2/weboob/applications/webcontentedit/webcontentedit.py000066400000000000000000000164361303450110500252040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os import tempfile import codecs from distutils.spawn import find_executable from weboob.core.bcall import CallErrors from weboob.capabilities.content import CapContent, Revision from weboob.tools.application.repl import ReplApplication, defaultcount __all__ = ['WebContentEdit'] class WebContentEdit(ReplApplication): APPNAME = 'webcontentedit' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to display and edit contents on various websites." SHORT_DESCRIPTION = "manage websites content" CAPS = CapContent def do_edit(self, line): """ edit ID [ID...] Edit a content with $EDITOR, then push it on the website. """ contents = [] for id in line.split(): _id, backend_name = self.parse_id(id, unique_backend=True) backend_names = (backend_name,) if backend_name is not None else self.enabled_backends contents += [content for content in self.do('get_content', _id, backends=backend_names) if content] if len(contents) == 0: print('No contents found', file=self.stderr) return 3 if self.stdin.isatty(): paths = {} for content in contents: tmpdir = os.path.join(tempfile.gettempdir(), "weboob") if not os.path.isdir(tmpdir): os.makedirs(tmpdir) with tempfile.NamedTemporaryFile(prefix='%s_' % content.id.replace(os.path.sep, '_'), dir=tmpdir, delete=False) as f: data = content.content if isinstance(data, unicode): data = data.encode('utf-8') elif data is None: content.content = u'' data = '' f.write(data) paths[f.name.encode('utf-8')] = content params = '' editor = os.environ.get('EDITOR', 'vim') # check cases where /usr/bin/vi is a symlink to vim if 'vim' in (os.path.basename(editor), os.path.basename(os.path.realpath(find_executable(editor) or '/')).replace('.nox', '')): params = '-p' os.system("%s %s %s" % (editor, params, ' '.join(['"%s"' % path.replace('"', '\\"') for path in paths.iterkeys()]))) for path, content in paths.iteritems(): with open(path, 'r') as f: data = f.read() try: data = data.decode('utf-8') except UnicodeError: pass if content.content != data: content.content = data else: contents.remove(content) if len(contents) == 0: print('No changes. Abort.', file=self.stderr) return 1 print('Contents changed:\n%s' % ('\n'.join(' * %s' % content.id for content in contents))) message = self.ask('Enter a commit message', default='') minor = self.ask('Is this a minor edit?', default=False) if not self.ask('Do you want to push?', default=True): return errors = CallErrors([]) for content in contents: path = [path for path, c in paths.iteritems() if c == content][0] self.stdout.write('Pushing %s...' % content.id.encode('utf-8')) self.stdout.flush() try: self.do('push_content', content, message, minor=minor, backends=[content.backend]).wait() except CallErrors as e: errors.errors += e.errors self.stdout.write(' error (content saved in %s)\n' % path) else: self.stdout.write(' done\n') os.unlink(path) else: # stdin is not a tty if len(contents) != 1: print("Multiple ids not supported with pipe", file=self.stderr) return 2 message, minor = '', False data = self.stdin.read() contents[0].content = data.decode(self.guess_encoding(self.stdin)) errors = CallErrors([]) for content in contents: self.stdout.write('Pushing %s...' % content.id.encode(self.encoding)) self.stdout.flush() try: self.do('push_content', content, message, minor=minor, backends=[content.backend]).wait() except CallErrors as e: errors.errors += e.errors self.stdout.write(' error\n') else: self.stdout.write(' done\n') if len(errors.errors) > 0: raise errors @defaultcount(10) def do_log(self, line): """ log ID Display log of a page """ if not line: print('Error: please give a page ID', file=self.stderr) return 2 _id, backend_name = self.parse_id(line) backend_names = (backend_name,) if backend_name is not None else self.enabled_backends _id = _id.encode('utf-8') self.start_format() for revision in self.do('iter_revisions', _id, backends=backend_names): self.format(revision) def do_get(self, line): """ get ID [-r revision] Get page contents """ if not line: print('Error: please give a page ID', file=self.stderr) return 2 _part_line = line.strip().split(' ') revision = None if '-r' in _part_line: r_index = _part_line.index('-r') if len(_part_line) -1 > r_index: revision = Revision(_part_line[r_index+1]) _part_line.remove(revision.id) _part_line.remove('-r') if not _part_line: print('Error: please give a page ID', file=self.stderr) return 2 _id, backend_name = self.parse_id(" ".join(_part_line)) backend_names = (backend_name,) if backend_name is not None else self.enabled_backends _id = _id.encode('utf-8') output = codecs.getwriter(self.encoding)(self.stdout) for contents in [content for content in self.do('get_content', _id, revision, backends=backend_names) if content]: output.write(contents.content) # add a newline unless we are writing # in a file or in a pipe if output.isatty(): output.write('\n') weboob-1.2/weboob/applications/weboobcfg/000077500000000000000000000000001303450110500205215ustar00rootroot00000000000000weboob-1.2/weboob/applications/weboobcfg/__init__.py000066400000000000000000000014321303450110500226320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .weboobcfg import WeboobCfg __all__ = ['WeboobCfg'] weboob-1.2/weboob/applications/weboobcfg/weboobcfg.py000066400000000000000000000267221303450110500230410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os import re from weboob.capabilities.account import CapAccount from weboob.exceptions import ModuleLoadError from weboob.tools.application.repl import ReplApplication from weboob.tools.application.console import ConsoleProgress from weboob.tools.ordereddict import OrderedDict from weboob.tools.application.formatters.iformatter import IFormatter __all__ = ['WeboobCfg'] class CapabilitiesWrapper(list): """ A wrapper class to keep the list nature of capabilities, but provide a comma separated list representation for formaters unable to display a list by themselves. Useful for having an array representation in JSON and comma separated list for simple format. """ def __repr__(self): return ', '.join(self) class ModuleInfoFormatter(IFormatter): def format_dict(self, minfo): result = '.------------------------------------------------------------------------------.\n' result += '| Module %-69s |\n' % minfo['name'] result += "+-----------------.------------------------------------------------------------'\n" result += '| Version | %s\n' % minfo['version'] result += '| Maintainer | %s\n' % minfo['maintainer'] result += '| License | %s\n' % minfo['license'] result += '| Description | %s\n' % minfo['description'] result += '| Capabilities | %s\n' % ', '.join(minfo['capabilities']) result += '| Installed | %s\n' % minfo['installed'] result += '| Location | %s\n' % minfo['location'] if 'config' in minfo: first = True for key, field in minfo['config'].iteritems(): label = field['label'] if field['default'] is not None: label += ' (default: %s)' % field['default'] if first: result += '| | \n' result += '| Configuration | %s: %s\n' % (key, label) first = False else: result += '| | %s: %s\n' % (key, label) result += "'-----------------'\n" return result class WeboobCfg(ReplApplication): APPNAME = 'weboob-config' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Christophe Benz, Romain Bignon' DESCRIPTION = "Weboob-Config is a console application to add/edit/remove backends, " \ "and to register new website accounts." SHORT_DESCRIPTION = "manage backends or register new accounts" EXTRA_FORMATTERS = {'info_formatter': ModuleInfoFormatter} COMMANDS_FORMATTERS = {'modules': 'table', 'list': 'table', 'info': 'info_formatter', } DISABLE_REPL = True def load_default_backends(self): pass def do_add(self, line): """ add MODULE_NAME [BACKEND_NAME] [PARAMETERS ...] Create a backend from a module. By default, if BACKEND_NAME is omitted, that's the module name which is used. You can specify parameters from command line in form "key=value". """ if not line: print('You must specify a module name. Hint: use the "modules" command.', file=self.stderr) return 2 module_name, options = self.parse_command_args(line, 2, 1) if options: options = options.split(' ') else: options = () backend_name = None params = {} # set backend params from command-line arguments for option in options: try: key, value = option.split('=', 1) except ValueError: if backend_name is None: backend_name = option else: print('Parameters have to be formatted "key=value"', file=self.stderr) return 2 else: params[key] = value self.add_backend(module_name, backend_name or module_name, params) def do_register(self, line): """ register MODULE Register a new account on a module. """ self.register_backend(line) def do_confirm(self, backend_name): """ confirm BACKEND For a backend which support CapAccount, parse a confirmation mail after using the 'register' command to automatically confirm the subscribe. It takes mail from stdin. Use it with postfix for example. """ # Do not use the ReplApplication.load_backends() method because we # don't want to prompt user to create backend. self.weboob.load_backends(names=[backend_name]) try: backend = self.weboob.get_backend(backend_name) except KeyError: print('Error: backend "%s" not found.' % backend_name, file=self.stderr) return 1 if not backend.has_caps(CapAccount): print('Error: backend "%s" does not support accounts management' % backend_name, file=self.stderr) return 1 mail = self.acquire_input() if not backend.confirm_account(mail): print('Error: Unable to confirm account creation', file=self.stderr) return 1 return 0 def do_list(self, line): """ list [CAPS ..] Show backends. """ caps = line.split() for backend_name, module_name, params in sorted(self.weboob.backends_config.iter_backends()): try: module = self.weboob.modules_loader.get_or_load_module(module_name) except ModuleLoadError as e: self.logger.warning('Unable to load module %r: %s' % (module_name, e)) continue if caps and not module.has_caps(*caps): continue row = OrderedDict([('Name', backend_name), ('Module', module_name), ('Configuration', ', '.join( '%s=%s' % (key, ('*****' if key in module.config and module.config[key].masked else value)) for key, value in params.iteritems())), ]) self.format(row) def do_remove(self, backend_name): """ remove NAME Remove a backend. """ if not self.weboob.backends_config.remove_backend(backend_name): print('Backend instance "%s" does not exist' % backend_name, file=self.stderr) return 1 def _do_toggle(self, backend_name, state): try: module_name, items = self.weboob.backends_config.get_backend(backend_name) except KeyError: print('Backend instance "%s" does not exist' % backend_name, file=self.stderr) return 1 self.weboob.backends_config.edit_backend(backend_name, module_name, {'_enabled': state}) def do_enable(self, backend_name): """ enable BACKEND Enable a disabled backend """ return self._do_toggle(backend_name, "true") def do_disable(self, backend_name): """ disable BACKEND Disable a backend """ return self._do_toggle(backend_name, "false") def do_edit(self, line): """ edit BACKEND Edit a backend """ try: self.edit_backend(line) except KeyError: print('Error: backend "%s" not found' % line, file=self.stderr) return 1 def do_modules(self, line): """ modules [CAPS ...] Show available modules. """ caps = line.split() for name, info in sorted(self.weboob.repositories.get_all_modules_info(caps).iteritems()): row = OrderedDict([('Name', name), ('Capabilities', CapabilitiesWrapper(info.capabilities)), ('Description', info.description), ('Installed', info.is_installed()), ]) self.format(row) def do_info(self, line): """ info NAME Display information about a module. """ if not line: print('You must specify a module name. Hint: use the "modules" command.', file=self.stderr) return 2 minfo = self.weboob.repositories.get_module_info(line) if not minfo: print('Module "%s" does not exist.' % line, file=self.stderr) return 1 try: module = self.weboob.modules_loader.get_or_load_module(line) except ModuleLoadError: module = None self.start_format() self.format(self.create_minfo_dict(minfo, module)) def create_minfo_dict(self, minfo, module): module_info = {} module_info['name'] = minfo.name module_info['version'] = minfo.version module_info['maintainer'] = minfo.maintainer module_info['license'] = minfo.license module_info['description'] = minfo.description module_info['capabilities'] = minfo.capabilities module_info['installed'] = '%s%s' % (('yes' if module else 'no'), ' (new version available)' if self.weboob.repositories.versions.get(minfo.name) > minfo.version else '') module_info['location'] = '%s' % (minfo.url or os.path.join(minfo.path, minfo.name)) if module: module_info['config'] = {} for key, field in module.config.iteritems(): module_info['config'][key] = {'label': field.label, 'default': field.default, 'description': field.description, 'regexp': field.regexp, 'choices': field.choices, 'masked': field.masked, 'required': field.required} return module_info def do_applications(self, line): """ applications Show applications. """ applications = set() import weboob.applications for path in weboob.applications.__path__: regexp = re.compile('^%s/([\w\d_]+)$' % path) for root, dirs, files in os.walk(path): m = regexp.match(root) if m and '__init__.py' in files: applications.add(m.group(1)) print(' '.join(sorted(applications)).encode('utf-8')) def do_update(self, line): """ update Update weboob. """ self.weboob.update(ConsoleProgress(self)) weboob-1.2/weboob/applications/weboobcli/000077500000000000000000000000001303450110500205315ustar00rootroot00000000000000weboob-1.2/weboob/applications/weboobcli/__init__.py000066400000000000000000000014321303450110500226420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .weboobcli import WeboobCli __all__ = ['WeboobCli'] weboob-1.2/weboob/applications/weboobcli/weboobcli.py000066400000000000000000000034121303450110500230500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.tools.application.repl import ReplApplication __all__ = ['WeboobCli'] class WeboobCli(ReplApplication): APPNAME = 'weboob-cli' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' SYNOPSIS = 'Usage: %prog [-dqv] [-b backends] [-cnfs] capability method [arguments..]\n' SYNOPSIS += ' %prog [--help] [--version]' DESCRIPTION = "Weboob-Cli is a console application to call a specific method on backends " \ "which implement the given capability." SHORT_DESCRIPTION = "call a method on backends" DISABLE_REPL = True def load_default_backends(self): pass def main(self, argv): if len(argv) < 3: print("Syntax: %s capability method [args ..]" % argv[0], file=self.stderr) return 2 cap_s = argv[1] cmd = argv[2] args = argv[3:] self.load_backends(cap_s) for obj in self.do(cmd, *args): self.format(obj) self.flush() return 0 weboob-1.2/weboob/applications/weboobdebug/000077500000000000000000000000001303450110500210505ustar00rootroot00000000000000weboob-1.2/weboob/applications/weboobdebug/__init__.py000066400000000000000000000014411303450110500231610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .weboobdebug import WeboobDebug __all__ = ['WeboobDebug'] weboob-1.2/weboob/applications/weboobdebug/weboobdebug.py000066400000000000000000000065441303450110500237170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from optparse import OptionGroup from weboob.tools.application.base import Application class WeboobDebug(Application): APPNAME = 'weboobdebug' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Christophe Benz' DESCRIPTION = "Weboob-Debug is a console application to debug backends." SHORT_DESCRIPTION = "debug backends" def __init__(self, option_parser=None): super(WeboobDebug, self).__init__(option_parser) options = OptionGroup(self._parser, 'Weboob-Debug options') options.add_option('-B', '--bpython', action='store_true', help='Prefer bpython over ipython') self._parser.add_option_group(options) def load_default_backends(self): pass def main(self, argv): """ BACKEND Debug BACKEND. """ try: backend_name = argv[1] except IndexError: print('Usage: %s BACKEND' % argv[0], file=self.stderr) return 1 try: backend = self.weboob.load_backends(names=[backend_name])[backend_name] except KeyError: print(u'Unable to load backend "%s"' % backend_name, file=self.stderr) return 1 locs = dict(backend=backend, browser=backend.browser, application=self, weboob=self.weboob) banner = 'Weboob debug shell\nBackend "%s" loaded.\nAvailable variables:\n' % backend_name \ + '\n'.join([' %s: %s' % (k, v) for k, v in locs.iteritems()]) if self.options.bpython: funcs = [self.bpython, self.ipython, self.python] else: funcs = [self.ipython, self.bpython, self.python] for func in funcs: try: func(locs, banner) except ImportError: continue else: break def ipython(self, locs, banner): try: from IPython import embed embed(user_ns=locs, banner2=banner) except ImportError: from IPython.Shell import IPShellEmbed shell = IPShellEmbed(argv=[]) shell.set_banner(shell.IP.BANNER + '\n\n' + banner) shell(local_ns=locs, global_ns={}) def bpython(self, locs, banner): from bpython import embed embed(locs, banner=banner) def python(self, locs, banner): import code try: import readline import rlcompleter readline.set_completer(rlcompleter.Completer(locs).complete) readline.parse_and_bind("tab:complete") except ImportError: pass code.interact(banner=banner, local=locs) weboob-1.2/weboob/applications/weboobrepos/000077500000000000000000000000001303450110500211125ustar00rootroot00000000000000weboob-1.2/weboob/applications/weboobrepos/__init__.py000066400000000000000000000014401303450110500232220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .weboobrepos import WeboobRepos __all__ = ['WeboobRepos'] weboob-1.2/weboob/applications/weboobrepos/weboobrepos.py000066400000000000000000000212151303450110500240130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from datetime import datetime from time import mktime, strptime import tarfile import os import shutil import subprocess from copy import copy from contextlib import closing from weboob.core.repositories import Repository from weboob.tools.application.repl import ReplApplication from weboob.tools.misc import find_exe __all__ = ['WeboobRepos'] class WeboobRepos(ReplApplication): APPNAME = 'weboob-repos' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2012-YEAR Romain Bignon' DESCRIPTION = "Weboob-repos is a console application to manage a Weboob Repository." SHORT_DESCRIPTION = "manage a weboob repository" COMMANDS_FORMATTERS = {'backends': 'table', 'list': 'table', } DISABLE_REPL = True weboob_commands = copy(ReplApplication.weboob_commands) weboob_commands.remove('backends') def load_default_backends(self): pass def do_create(self, line): """ create NAME [PATH] Create a new repository. If PATH is missing, create repository on the current directory. """ name, path = self.parse_command_args(line, 2, 1) if not path: path = os.getcwd() else: path = os.path.realpath(path) if not os.path.exists(path): os.mkdir(path) elif not os.path.isdir(path): print(u'"%s" is not a directory' % path) return 1 r = Repository('http://') r.name = name r.maintainer = self.ask('Enter maintainer of the repository') r.save(os.path.join(path, r.INDEX)) print(u'Repository "%s" created.' % path) def do_build(self, line): """ build SOURCE REPOSITORY Build backends contained in SOURCE to REPOSITORY. Example: $ weboob-repos build $HOME/src/weboob/modules /var/www/updates.weboob.org/0.a/ """ source_path, repo_path = self.parse_command_args(line, 2, 2) index_file = os.path.join(repo_path, Repository.INDEX) r = Repository('http://') try: with open(index_file, 'r') as fp: r.parse_index(fp) except IOError as e: print('Unable to open repository: %s' % e, file=self.stderr) print('Use the "create" command before.', file=self.stderr) return 1 r.build_index(source_path, index_file) if r.signed: sigfiles = [r.KEYRING, Repository.INDEX] gpg = find_exe('gpg1') or find_exe('gpg') if not gpg: raise Exception('Unable to find the gpg executable.') krname = os.path.join(repo_path, r.KEYRING) if os.path.exists(krname): kr_mtime = int(datetime.fromtimestamp(os.path.getmtime(krname)).strftime('%Y%m%d%H%M')) if not os.path.exists(krname) or kr_mtime < r.key_update: print('Generate keyring') # Remove all existing keys if os.path.exists(krname): os.remove(krname) # Add all valid keys for keyfile in os.listdir(os.path.join(source_path, r.KEYDIR)): print('Adding key %s' % keyfile) keypath = os.path.join(source_path, r.KEYDIR, keyfile) subprocess.check_call([ gpg, '--no-options', '--quiet', '--no-default-keyring', '--keyring', os.path.realpath(krname), '--import', os.path.realpath(keypath)]) # Does not make much sense in our case if os.path.exists(krname + '~'): os.remove(krname + '~') if not os.path.exists(krname): raise Exception('No valid key file found.') kr_mtime = mktime(strptime(str(r.key_update), '%Y%m%d%H%M')) os.chmod(krname, 0o644) os.utime(krname, (kr_mtime, kr_mtime)) else: print('Keyring is up to date') for name, module in r.modules.iteritems(): tarname = os.path.join(repo_path, '%s.tar.gz' % name) if r.signed: sigfiles.append(os.path.basename(tarname)) module_path = os.path.join(source_path, name) if os.path.exists(tarname): tar_mtime = int(datetime.fromtimestamp(os.path.getmtime(tarname)).strftime('%Y%m%d%H%M')) if tar_mtime >= module.version: continue print('Create archive for %s' % name) with closing(tarfile.open(tarname, 'w:gz')) as tar: tar.add(module_path, arcname=name, exclude=self._archive_excludes) tar_mtime = mktime(strptime(str(module.version), '%Y%m%d%H%M')) os.utime(tarname, (tar_mtime, tar_mtime)) # Copy icon. icon_path = os.path.join(module_path, 'favicon.png') if os.path.exists(icon_path): shutil.copy(icon_path, os.path.join(repo_path, '%s.png' % name)) if r.signed: gpg = find_exe('gpg2') or find_exe('gpg') if not gpg: raise Exception('Unable to find the gpg executable.') # Find out which keys are allowed to sign fingerprints = [gpgline.strip(':').split(':')[-1] for gpgline in subprocess.Popen([ gpg, '--no-options', '--with-fingerprint', '--with-colons', '--list-public-keys', '--no-default-keyring', '--keyring', os.path.realpath(krname)], stdout=subprocess.PIPE).communicate()[0].splitlines() if gpgline.startswith('fpr:')] # Find out the first secret key we have that is allowed to sign secret_fingerprint = None for fingerprint in fingerprints: proc = subprocess.Popen([ gpg, '--no-options', '--list-secret-keys', fingerprint], stdout=subprocess.PIPE, stderr=subprocess.PIPE) proc.communicate() # if failed if proc.returncode: continue secret_fingerprint = fingerprint if secret_fingerprint is None: raise Exception('No suitable secret key found') # Check if all files have an up to date signature for filename in sigfiles: filepath = os.path.realpath(os.path.join(repo_path, filename)) sigpath = filepath + '.sig' file_mtime = int(os.path.getmtime(filepath)) if os.path.exists(sigpath): sig_mtime = int(os.path.getmtime(sigpath)) if not os.path.exists(sigpath) or sig_mtime < file_mtime: print('Signing %s' % filename) if os.path.exists(sigpath): os.remove(sigpath) subprocess.check_call([ gpg, '--no-options', '--quiet', '--local-user', secret_fingerprint, '--detach-sign', '--output', sigpath, '--sign', filepath]) os.utime(sigpath, (file_mtime, file_mtime)) print('Signatures are up to date') def _archive_excludes(self, filename): # Skip *.pyc files in tarballs. if filename.endswith('.pyc'): return True # Don't include *.png files in tarball if filename.endswith('.png'): return True return False weboob-1.2/weboob/applications/weboorrents/000077500000000000000000000000001303450110500211355ustar00rootroot00000000000000weboob-1.2/weboob/applications/weboorrents/__init__.py000066400000000000000000000014401303450110500232450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .weboorrents import Weboorrents __all__ = ['Weboorrents'] weboob-1.2/weboob/applications/weboorrents/weboorrents.py000066400000000000000000000150111303450110500240560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from weboob.capabilities.torrent import CapTorrent, MagnetOnly from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter from weboob.core import CallErrors from weboob.capabilities.base import NotAvailable, NotLoaded, empty __all__ = ['Weboorrents'] def sizeof_fmt(num): for x in ['bytes', 'KB', 'MB', 'GB', 'TB']: if num < 1024.0: return "%-4.1f%s" % (num, x) num /= 1024.0 class TorrentInfoFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'name', 'size', 'seeders', 'leechers', 'url', 'files', 'description') def format_obj(self, obj, alias): result = u'%s%s%s\n' % (self.BOLD, obj.name, self.NC) result += 'ID: %s\n' % obj.fullid if obj.size != NotAvailable and obj.size != NotLoaded: result += 'Size: %s\n' % sizeof_fmt(obj.size) result += 'Seeders: %s\n' % obj.seeders result += 'Leechers: %s\n' % obj.leechers result += 'URL: %s\n' % obj.url if hasattr(obj, 'magnet') and obj.magnet: result += 'Magnet URL: %s\n' % obj.magnet if obj.files: result += '\n%sFiles%s\n' % (self.BOLD, self.NC) for f in obj.files: result += ' * %s\n' % f result += '\n%sDescription%s\n' % (self.BOLD, self.NC) result += '%s' % obj.description return result class TorrentListFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name', 'size', 'seeders', 'leechers') def get_title(self, obj): return obj.name NB2COLOR = ((0, 'red', None), (1, 'blue', None), (5, 'green', None), (10, 'green', 'bold'), ) def _get_color(self, nb): if empty(nb): return self.colored('N/A', 'red') for threshold, _color, _attr in self.NB2COLOR: if nb >= threshold: color = _color attr = _attr return self.colored('%3d' % nb, color, attr) def get_description(self, obj): size = self.colored('%10s' % sizeof_fmt(obj.size), 'magenta') return '%s (Seed: %s / Leech: %s)' % (size, self._get_color(obj.seeders), self._get_color(obj.leechers)) class Weboorrents(ReplApplication): APPNAME = 'weboorrents' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to search for torrents on various trackers " \ "and download .torrent files." SHORT_DESCRIPTION = "search and download torrents" CAPS = CapTorrent EXTRA_FORMATTERS = {'torrent_list': TorrentListFormatter, 'torrent_info': TorrentInfoFormatter, } COMMANDS_FORMATTERS = {'search': 'torrent_list', 'info': 'torrent_info', } def complete_info(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_info(self, id): """ info ID Get information about a torrent. """ torrent = self.get_object(id, 'get_torrent', ('id', 'name', 'size', 'seeders', 'leechers', 'url', 'files', 'description')) if not torrent: print('Torrent not found: %s' % id, file=self.stderr) return 3 self.start_format() self.format(torrent) def complete_getfile(self, text, line, *ignored): args = line.split(' ', 2) if len(args) == 2: return self._complete_object() elif len(args) >= 3: return self.path_completer(args[2]) def do_getfile(self, line): """ getfile ID [FILENAME] Get the .torrent file. FILENAME is where to write the file. If FILENAME is '-', the file is written to stdout. """ id, dest = self.parse_command_args(line, 2, 1) torrent = self.get_object(id, 'get_torrent', ('description', 'files')) if not torrent: print('Torrent not found: %s' % id, file=self.stderr) return 3 dest = self.obj_to_filename(torrent, dest, '{id}-{name}.torrent') try: for buf in self.do('get_torrent_file', torrent.id, backends=torrent.backend): if buf: if dest == '-': print(buf) else: try: with open(dest, 'w') as f: f.write(buf) except IOError as e: print('Unable to write .torrent in "%s": %s' % (dest, e), file=self.stderr) return 1 return except CallErrors as errors: for backend, error, backtrace in errors: if isinstance(error, MagnetOnly): print(u'Error(%s): No direct URL available, ' u'please provide this magnet URL ' u'to your client:\n%s' % (backend, error.magnet), file=self.stderr) return 4 else: self.bcall_error_handler(backend, error, backtrace) print('Torrent "%s" not found' % id, file=self.stderr) return 3 @defaultcount(10) def do_search(self, pattern): """ search [PATTERN] Search torrents. """ self.change_path([u'search']) if not pattern: pattern = None self.start_format(pattern=pattern) for torrent in self.do('iter_torrents', pattern=pattern): self.cached_format(torrent) weboob-1.2/weboob/applications/wetboobs/000077500000000000000000000000001303450110500204105ustar00rootroot00000000000000weboob-1.2/weboob/applications/wetboobs/__init__.py000066400000000000000000000014271303450110500225250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .wetboobs import WetBoobs __all__ = ['WetBoobs'] weboob-1.2/weboob/applications/wetboobs/wetboobs.py000066400000000000000000000113301303450110500226040ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.weather import CapWeather from weboob.tools.application.repl import ReplApplication, defaultcount from weboob.tools.application.formatters.iformatter import IFormatter, PrettyFormatter __all__ = ['WetBoobs'] class ForecastsFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'low', 'high') temperature_display = staticmethod(lambda t: u'%s' % t.value) def format_obj(self, obj, alias): result = u'%s* %-15s%s (%s - %s)' % (self.BOLD, '%s:' % obj.date, self.NC, self.temperature_display(obj.low), self.temperature_display(obj.high)) if hasattr(obj, 'text') and obj.text: result += ' %s' % obj.text return result class CurrentFormatter(IFormatter): MANDATORY_FIELDS = ('id', 'date', 'temp') temperature_display = staticmethod(lambda t: u'%s' % t.value) def format_obj(self, obj, alias): result = u'%s%s%s: %s' % (self.BOLD, obj.date, self.NC, self.temperature_display(obj.temp)) if hasattr(obj, 'text') and obj.text: result += u' - %s' % obj.text return result class CitiesFormatter(PrettyFormatter): MANDATORY_FIELDS = ('id', 'name') def get_title(self, obj): return obj.name class WetBoobs(ReplApplication): APPNAME = 'wetboobs' VERSION = '1.2' COPYRIGHT = 'Copyright(C) 2010-YEAR Romain Bignon' DESCRIPTION = "Console application allowing to display weather and forecasts in your city." SHORT_DESCRIPTION = "display weather and forecasts" CAPS = CapWeather DEFAULT_FORMATTER = 'table' EXTRA_FORMATTERS = {'cities': CitiesFormatter, 'current': CurrentFormatter, 'forecasts': ForecastsFormatter, } COMMANDS_FORMATTERS = {'cities': 'cities', 'current': 'current', 'forecasts': 'forecasts', } def main(self, argv): self.load_config() return ReplApplication.main(self, argv) @defaultcount(10) def do_cities(self, pattern): """ cities PATTERN Search cities. """ self.change_path(['cities']) self.start_format() for city in self.do('iter_city_search', pattern, caps=CapWeather): self.cached_format(city) def complete_current(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_current(self, line): """ current CITY_ID Get current weather for specified city. Use the 'cities' command to find them. """ city, = self.parse_command_args(line, 1, 1) _id, backend_name = self.parse_id(city) tr = self.config.get('settings', 'temperature_display', default='C') if tr == 'C': self.formatter.temperature_display = lambda t: t.ascelsius() elif tr == 'F': self.formatter.temperature_display = lambda t: t.asfahrenheit() self.start_format() for current in self.do('get_current', _id, backends=backend_name, caps=CapWeather): if current: self.format(current) def complete_forecasts(self, text, line, *ignored): args = line.split(' ') if len(args) == 2: return self._complete_object() def do_forecasts(self, line): """ forecasts CITY_ID Get forecasts for specified city. Use the 'cities' command to find them. """ city, = self.parse_command_args(line, 1, 1) _id, backend_name = self.parse_id(city) tr = self.config.get('settings', 'temperature_display', default='C') if tr == 'C': self.formatter.temperature_display = lambda t: t.ascelsius() elif tr == 'F': self.formatter.temperature_display = lambda t: t.asfahrenheit() self.start_format() for forecast in self.do('iter_forecast', _id, backends=backend_name, caps=CapWeather): self.format(forecast) weboob-1.2/weboob/browser/000077500000000000000000000000001303450110500155615ustar00rootroot00000000000000weboob-1.2/weboob/browser/__init__.py000066400000000000000000000017721303450110500177010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .browsers import Browser, DomainBrowser, UrlNotAllowed, PagesBrowser, LoginBrowser, need_login, AbstractBrowser from .url import URL __all__ = ['Browser', 'DomainBrowser', 'UrlNotAllowed', 'PagesBrowser', 'URL', 'LoginBrowser', 'need_login', 'AbstractBrowser'] weboob-1.2/weboob/browser/browsers.py000066400000000000000000000734411303450110500200120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import absolute_import, print_function from functools import wraps import re import pickle import base64 import zlib try: from requests.packages import urllib3 except ImportError: import urllib3 try: from urllib.parse import urlparse, urljoin except ImportError: from urlparse import urlparse, urljoin import os import sys from copy import deepcopy import inspect try: import requests if int(requests.__version__.split('.')[0]) < 2: raise ImportError() except ImportError: raise ImportError('Please install python-requests >= 2.0') from weboob.exceptions import BrowserHTTPSDowngrade, ModuleInstallError from weboob.tools.log import getLogger from weboob.tools.ordereddict import OrderedDict from weboob.tools.json import json from .cookies import WeboobCookieJar from .exceptions import HTTPNotFound, ClientError, ServerError from .sessions import FuturesSession from .profiles import Firefox from .pages import NextPage from .url import URL class Browser(object): """ Simple browser class. Act like a browser, and don't try to do too much. """ PROFILE = Firefox() """ Default profile used by browser to navigate on websites. """ TIMEOUT = 10.0 """ Default timeout during requests. """ REFRESH_MAX = 0.0 """ When handling a Refresh header, the browsers considers it only if the sleep time in lesser than this value. """ VERIFY = True """ Check SSL certificates. """ PROXIES = None MAX_RETRIES = 2 MAX_WORKERS = 10 """ Maximum of threads for asynchronous requests. """ ALLOW_REFERRER = True """ Controls the behavior of get_referrer. """ @classmethod def asset(cls, localfile): """ Absolute file path for a module local file. """ if os.path.isabs(localfile): return localfile return os.path.join(os.path.dirname(inspect.getfile(cls)), localfile) def __init__(self, logger=None, proxy=None, responses_dirname=None): self.logger = getLogger('browser', logger) self.responses_dirname = responses_dirname self.responses_count = 1 self.PROXIES = proxy self._setup_session(self.PROFILE) self.url = None self.response = None if isinstance(self.VERIFY, basestring): self.VERIFY = self.asset(self.VERIFY) def deinit(self): self.session.close() def save_response(self, response, warning=False, **kwargs): if self.responses_dirname is None: import tempfile self.responses_dirname = tempfile.mkdtemp(prefix='weboob_session_') print('Debug data will be saved in this directory: %s' % self.responses_dirname, file=sys.stderr) elif not os.path.isdir(self.responses_dirname): os.makedirs(self.responses_dirname) import mimetypes # get the content-type, remove optionnal charset part mimetype = response.headers.get('Content-Type', '').split(';')[0] # due to http://bugs.python.org/issue1043134 if mimetype == 'text/plain': ext = '.txt' else: # try to get an extension (and avoid adding 'None') ext = mimetypes.guess_extension(mimetype, False) or '' path = re.sub(r'[^A-z0-9\.-_]+', '_', urlparse(response.url).path.rpartition('/')[2])[-10:] if path.endswith(ext): ext = '' filename = '%02d-%d%s%s%s' % \ (self.responses_count, response.status_code, '-' if path else '', path, ext) response_filepath = os.path.join(self.responses_dirname, filename) with open(response_filepath, 'w') as f: f.write(response.content) request = response.request with open(response_filepath + '-request.txt', 'w') as f: f.write('%s %s\n\n\n' % (request.method, request.url)) for key, value in request.headers.iteritems(): f.write('%s: %s\n' % (key, value)) if request.body is not None: # separate '' from None f.write('\n\n\n%s' % request.body) with open(response_filepath + '-response.txt', 'w') as f: if hasattr(response.elapsed, 'total_seconds'): f.write('Time: %3.3fs\n' % response.elapsed.total_seconds()) f.write('%s %s\n\n\n' % (response.status_code, response.reason)) for key, value in response.headers.iteritems(): f.write('%s: %s\n' % (key, value)) match_filepath = os.path.join(self.responses_dirname, 'url_response_match.txt') with open(match_filepath, 'a') as f: f.write('# %d %s %s\n' % (response.status_code, response.reason, response.headers.get('Content-Type', ''))) f.write('%s\t%s\n' % (response.url, filename)) self.responses_count += 1 msg = u'Response saved to %s' % response_filepath if warning: self.logger.warning(msg) else: self.logger.info(msg) def _create_session(self): return FuturesSession(max_workers=self.MAX_WORKERS, max_retries=self.MAX_RETRIES) def _setup_session(self, profile): """ Set up a python-requests session for our usage. """ session = self._create_session() session.proxies = self.PROXIES session.verify = not self.logger.settings['ssl_insecure'] and self.VERIFY if not session.verify: try: urllib3.disable_warnings() except AttributeError: # urllib3 is too old, warnings won't be disable pass # defines a max_retries. It's mandatory in case a server is not # handling keep alive correctly, like the proxy burp adapter_kwargs = dict(max_retries=self.MAX_RETRIES) # set connection pool size equal to MAX_WORKERS if needed if self.MAX_WORKERS > requests.adapters.DEFAULT_POOLSIZE: adapter_kwargs.update(pool_connections=self.MAX_WORKERS, pool_maxsize=self.MAX_WORKERS) session.mount('https://', requests.adapters.HTTPAdapter(**adapter_kwargs)) session.mount('http://', requests.adapters.HTTPAdapter(**adapter_kwargs)) if self.TIMEOUT: session.timeout = self.TIMEOUT ## weboob only can provide proxy and HTTP auth options session.trust_env = False profile.setup_session(session) if self.responses_dirname is not None: session.hooks['response'].append(self.save_response) self.session = session session.cookies = WeboobCookieJar() def set_profile(self, profile): profile.setup_session(self.session) def location(self, url, **kwargs): """ Like :meth:`open` but also changes the current URL and response. This is the most common method to request web pages. Other than that, has the exact same behavior of open(). """ assert not kwargs.get('is_async'), "Please use open() instead of location() to make asynchronous requests." response = self.open(url, **kwargs) self.response = response self.url = self.response.url return response def open(self, url, referrer=None, allow_redirects=True, stream=None, timeout=None, verify=None, cert=None, proxies=None, data_encoding=None, is_async=False, callback=lambda response: response, **kwargs): """ Make an HTTP request like a browser does: * follow redirects (unless disabled) * provide referrers (unless disabled) Unless a `method` is explicitly provided, it makes a GET request, or a POST if data is not None, An empty `data` (not None, like '' or {}) *will* make a POST. It is a wrapper around session.request(). All session.request() options are available. You should use location() or open() and not session.request(), since it has some interesting additions, which are easily individually disabled through the arguments. Call this instead of location() if you do not want to "visit" the URL (for instance, you are downloading a file). When `is_async` is True, open() returns a Future object (see concurrent.futures for more details), which can be evaluated with its result() method. If any exception is raised while processing request, it is caught and re-raised when calling result(). For example: >>> Browser().open('http://google.com', is_async=True).result().text # doctest: +SKIP :param url: URL :type url: str :param data: POST data :type url: str or dict or None :param referrer: Force referrer. False to disable sending it, None for guessing :type referrer: str or False or None :param is_async: Process request in a non-blocking way :type is_async: bool :param callback: Callback to be called when request has finished, with response as its first and only argument :type callback: function :rtype: :class:`requests.Response` """ if 'async' in kwargs: import warnings warnings.warn('Please use is_async instead of async.', DeprecationWarning) is_async = kwargs['async'] del kwargs['async'] req = self.build_request(url, referrer, data_encoding=data_encoding, **kwargs) preq = self.prepare_request(req) if hasattr(preq, '_cookies'): # The _cookies attribute is not present in requests < 2.2. As in # previous version it doesn't calls extract_cookies_to_jar(), it is # not a problem as we keep our own cookiejar instance. preq._cookies = WeboobCookieJar.from_cookiejar(preq._cookies) if proxies is None: proxies = self.PROXIES if verify is None: verify = not self.logger.settings['ssl_insecure'] and self.VERIFY if timeout is None: timeout = self.TIMEOUT # We define an inner_callback here in order to execute the same code # regardless of is_async param. def inner_callback(future, response): if allow_redirects: response = self.handle_refresh(response) self.raise_for_status(response) return callback(response) # call python-requests response = self.session.send(preq, allow_redirects=allow_redirects, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, callback=inner_callback, is_async=is_async) return response def async_open(self, url, **kwargs): """ Shortcut to open(url, is_async=True). """ if 'async' in kwargs: del kwargs['async'] if 'is_async' in kwargs: del kwargs['is_async'] return self.open(url, is_async=True, **kwargs) def raise_for_status(self, response): """ Like Response.raise_for_status but will use other classes if needed. """ http_error_msg = None if 400 <= response.status_code < 500: http_error_msg = '%s Client Error: %s' % (response.status_code, response.reason) cls = ClientError if response.status_code == 404: cls = HTTPNotFound elif 500 <= response.status_code < 600: http_error_msg = '%s Server Error: %s' % (response.status_code, response.reason) cls = ServerError if http_error_msg: raise cls(http_error_msg, response=response) # in case we did not catch something that should be response.raise_for_status() def build_request(self, url, referrer=None, data_encoding=None, **kwargs): """ Does the same job as open(), but returns a Request without submitting it. This allows further customization to the Request. """ if isinstance(url, requests.Request): req = url url = req.url else: req = requests.Request(url=url, **kwargs) # guess method if req.method is None: if req.data: req.method = 'POST' else: req.method = 'GET' # convert unicode strings to proper encoding if isinstance(req.data, unicode) and data_encoding: req.data = req.data.encode(data_encoding) if isinstance(req.data, dict) and data_encoding: req.data = OrderedDict([(k, v.encode(data_encoding) if isinstance(v, unicode) else v) for k, v in req.data.iteritems()]) if referrer is None: referrer = self.get_referrer(self.url, url) if referrer: # Yes, it is a misspelling. req.headers.setdefault('Referer', referrer) return req def prepare_request(self, req): """ Get a prepared request from a Request object. This method aims to be overloaded by children classes. """ return self.session.prepare_request(req) REFRESH_RE = re.compile(r"^(?P[\d\.]+)(; url=[\"']?(?P.*?)[\"']?)?$", re.IGNORECASE) def handle_refresh(self, response): """ Called by open, to handle Refresh HTTP header. It only redirect to the refresh URL if the sleep time is inferior to REFRESH_MAX. """ if 'Refresh' not in response.headers: return response m = self.REFRESH_RE.match(response.headers['Refresh']) if m: # XXX perhaps we should not redirect if the refresh url is equal to the current url. url = m.groupdict().get('url', None) or response.request.url sleep = float(m.groupdict()['sleep']) if sleep <= self.REFRESH_MAX: self.logger.debug('Refresh to %s' % url) return self.open(url) else: self.logger.debug('Do not refresh to %s because %s > REFRESH_MAX(%s)' % (url, sleep, self.REFRESH_MAX)) return response self.logger.warning('Unable to handle refresh "%s"' % response.headers['Refresh']) return response def get_referrer(self, oldurl, newurl): """ Get the referrer to send when doing a request. If we should not send a referrer, it will return None. Reference: https://en.wikipedia.org/wiki/HTTP_referer The behavior can be controlled through the ALLOW_REFERRER attribute. True always allows the referers to be sent, False never, and None only if it is within the same domain. :param oldurl: Current absolute URL :type oldurl: str or None :param newurl: Target absolute URL :type newurl: str :rtype: str or None """ if self.ALLOW_REFERRER is False: return if oldurl is None: return old = urlparse(oldurl) new = urlparse(newurl) # Do not leak secure URLs to insecure URLs if old.scheme == 'https' and new.scheme != 'https': return # Reloading the page. Usually no referrer. if oldurl == newurl: return # Domain-based privacy if self.ALLOW_REFERRER is None and old.netloc != new.netloc: return return oldurl class UrlNotAllowed(Exception): """ Raises by :class:`DomainBrowser` when `RESTRICT_URL` is set and trying to go on an url not matching `BASEURL`. """ class DomainBrowser(Browser): """ A browser that handles relative URLs and can have a base URL (usually a domain). For instance self.location('/hello') will get http://weboob.org/hello if BASEURL is 'http://weboob.org/'. """ BASEURL = None """ Base URL, e.g. 'http://weboob.org/' or 'https://weboob.org/' See absurl(). """ RESTRICT_URL = False """ URLs allowed to load. This can be used to force SSL (if the BASEURL is SSL) or any other leakage. Set to True to allow only URLs starting by the BASEURL. Set it to a list of allowed URLs if you have multiple allowed URLs. More complex behavior is possible by overloading url_allowed() """ def __init__(self, baseurl=None, *args, **kwargs): super(DomainBrowser, self).__init__(*args, **kwargs) if baseurl is not None: self.BASEURL = baseurl def url_allowed(self, url): """ Checks if we are allowed to visit an URL. See RESTRICT_URL. :param url: Absolute URL :type url: str :rtype: bool """ if self.BASEURL is None or self.RESTRICT_URL is False: return True if self.RESTRICT_URL is True: return url.startswith(self.BASEURL) for restrict_url in self.RESTRICT_URL: if url.startswith(restrict_url): return True return False def absurl(self, uri, base=None): """ Get the absolute URL, relative to a base URL. If base is None, it will try to use the current URL. If there is no current URL, it will try to use BASEURL. If base is False, it will always try to use the current URL. If base is True, it will always try to use BASEURL. :param uri: URI to make absolute. It can be already absolute. :type uri: str :param base: Base absolute URL. :type base: str or None or False or True :rtype: str """ if not base: base = self.url if base is None or base is True: base = self.BASEURL return urljoin(base, uri) def open(self, req, *args, **kwargs): """ Like :meth:`Browser.open` but handles urls without domains, using the :attr:`BASEURL` attribute. """ uri = req.url if isinstance(req, requests.Request) else req url = self.absurl(uri) if not self.url_allowed(url): raise UrlNotAllowed(url) if isinstance(req, requests.Request): req.url = url else: req = url return super(DomainBrowser, self).open(req, *args, **kwargs) def go_home(self): """ Go to the "home" page, usually the BASEURL. """ return self.location(self.BASEURL or self.absurl('/')) class _PagesBrowserMeta(type): """ Private meta-class used to keep order of URLs instances of PagesBrowser. """ def __new__(mcs, name, bases, attrs): urls = [(url_name, attrs.pop(url_name)) for url_name, obj in attrs.items() if isinstance(obj, URL)] urls.sort(key=lambda x: x[1]._creation_counter) new_class = super(_PagesBrowserMeta, mcs).__new__(mcs, name, bases, attrs) if new_class._urls is None: new_class._urls = OrderedDict() else: new_class._urls = deepcopy(new_class._urls) new_class._urls.update(urls) return new_class class PagesBrowser(DomainBrowser): r""" A browser which works pages and keep state of navigation. To use it, you have to derive it and to create URL objects as class attributes. When open() or location() are called, if the url matches one of URL objects, it returns a Page object. In case of location(), it stores it in self.page. Example: >>> from .pages import HTMLPage >>> class HomePage(HTMLPage): ... pass ... >>> class ListPage(HTMLPage): ... pass ... >>> class MyBrowser(PagesBrowser): ... BASEURL = 'http://example.org' ... home = URL('/(index\.html)?', HomePage) ... list = URL('/list\.html', ListPage) ... You can then use URL instances to go on pages. """ _urls = None __metaclass__ = _PagesBrowserMeta def __getattr__(self, name): if self._urls is not None and name in self._urls: return self._urls[name] else: raise AttributeError("'%s' object has no attribute '%s'" % ( self.__class__.__name__, name)) def __init__(self, *args, **kwargs): super(PagesBrowser, self).__init__(*args, **kwargs) self.page = None self._urls = deepcopy(self._urls) for url in self._urls.itervalues(): url.browser = self def open(self, *args, **kwargs): """ Same method than :meth:`weboob.browser.browsers.DomainBrowser.open`, but the response contains an attribute `page` if the url matches any :class:`URL` object. """ callback = kwargs.pop('callback', lambda response: response) # Have to define a callback to seamlessly process synchronous and # asynchronous requests, see :meth:`Browser.open` and its `is_async` # and `callback` params. def internal_callback(response): # Try to handle the response page with an URL instance. response.page = None for url in self._urls.itervalues(): page = url.handle(response) if page is not None: self.logger.debug('Handle %s with %s' % (response.url, page.__class__.__name__)) response.page = page break if response.page is None: regexp = r'^(?P\w+)://.*' proto_response = re.match(regexp, response.url) if proto_response: proto_response = proto_response.group('proto') proto_base = re.match(regexp, self.BASEURL).group('proto') if proto_base == 'https' and proto_response != 'https': raise BrowserHTTPSDowngrade() self.logger.debug('Unable to handle %s' % response.url) return callback(response) return super(PagesBrowser, self).open(callback=internal_callback, *args, **kwargs) def location(self, *args, **kwargs): """ Same method than :meth:`weboob.browser.browsers.Browser.location`, but if the url matches any :class:`URL` object, an attribute `page` is added to response, and the attribute :attr:`PagesBrowser.page` is set. """ if self.page is not None: # Call leave hook. self.page.on_leave() response = self.open(*args, **kwargs) self.response = response self.page = response.page self.url = response.url if self.page is not None: # Call load hook. self.page.on_load() # Returns self.response in case on_load recalls location() return self.response def pagination(self, func, *args, **kwargs): r""" This helper function can be used to handle pagination pages easily. When the called function raises an exception :class:`NextPage`, it goes on the wanted page and recall the function. :class:`NextPage` constructor can take an url or a Request object. >>> from .pages import HTMLPage >>> class Page(HTMLPage): ... def iter_values(self): ... for el in self.doc.xpath('//li'): ... yield el.text ... for next in self.doc.xpath('//a'): ... raise NextPage(next.attrib['href']) ... >>> class Browser(PagesBrowser): ... BASEURL = 'https://people.symlink.me' ... list = URL('/~rom1/projects/weboob/list-(?P\d+).html', Page) ... >>> b = Browser() >>> b.list.go(pagenum=1) # doctest: +ELLIPSIS >>> list(b.pagination(lambda: b.page.iter_values())) ['One', 'Two', 'Three', 'Four'] """ while True: try: for r in func(*args, **kwargs): yield r except NextPage as e: self.location(e.request) else: return def need_login(func): """ Decorator used to require to be logged to access to this function. """ @wraps(func) def inner(browser, *args, **kwargs): if browser.page is None or not browser.page.logged: browser.do_login() return func(browser, *args, **kwargs) return inner class LoginBrowser(PagesBrowser): """ A browser which supports login. """ def __init__(self, username, password, *args, **kwargs): super(LoginBrowser, self).__init__(*args, **kwargs) self.username = username self.password = password def do_login(self): """ Abstract method to implement to login on website. It is call when a login is needed. """ raise NotImplementedError() def do_logout(self): """ Logout from website. By default, simply clears the cookies. """ self.session.cookies.clear() class StatesMixin(object): """ Mixin to store states of browser. """ __states__ = [] """ Saved state variables. """ def load_state(self, state): if 'cookies' in state: try: self.session.cookies = pickle.loads(zlib.decompress(base64.b64decode(state['cookies']))) except (TypeError, zlib.error, EOFError, ValueError): self.logger.error('Unable to reload cookies from storage') else: self.logger.info('Reloaded cookies from storage') for attrname in self.__states__: if attrname in state: setattr(self, attrname, state[attrname]) if 'url' in state: try: self.location(state['url']) except (requests.exceptions.HTTPError, requests.exceptions.TooManyRedirects): pass def dump_state(self): state = {} if self.page: state['url'] = self.page.url state['cookies'] = base64.b64encode(zlib.compress(pickle.dumps(self.session.cookies, -1))) for attrname in self.__states__: state[attrname] = getattr(self, attrname) self.logger.info('Stored cookies into storage') return state class APIBrowser(DomainBrowser): """ A browser for API websites. """ def build_request(self, *args, **kwargs): if 'data' in kwargs: kwargs['data'] = json.dumps(kwargs['data']) if 'headers' not in kwargs: kwargs['headers'] = {} kwargs['headers']['Content-Type'] = 'application/json' return super(APIBrowser, self).build_request(*args, **kwargs) def open(self, *args, **kwargs): """ Do a JSON request. The "Content-Type" header is always set to "application/json". :param data: if specified, format as JSON and send as request body :type data: :class:`dict` :param headers: if specified, add these headers to the request :type headers: :class:`dict` """ return super(APIBrowser, self).open(*args, **kwargs) def request(self, *args, **kwargs): """ Do a JSON request and parse the response. :returns: a dict containing the parsed JSON server response :rtype: :class:`dict` """ return self.open(*args, **kwargs).json() class AbstractBrowserMissingParentError(Exception): pass class AbstractBrowser(Browser): """ AbstractBrowser allow inheritance of a browser defined in another module. Websites can share many pages and code base. This class allow to load a browser provided by another module and to build our own browser on top of it (like standard python inheritance. Weboob will install and download the PARENT module for you. PARENT is a mandatory attribute, it's the name of the module providing the parent Browser PARENT_ATTR is an optionnal attribute used when the parent module does not have only one browser defined as BROWSER class attribute: you can customized the path of the object to load. Note that you must pass a valid weboob instance as first argument of the constructor. """ PARENT = None PARENT_ATTR = None def __new__(cls, weboob, *args, **kwargs): if cls.PARENT is None: raise AbstractBrowserMissingParentError("PARENT is not defined for browser %s" % cls) try: module = weboob.load_or_install_module(cls.PARENT) except ModuleInstallError as err: raise ModuleInstallError('This module depends on %s module but %s\'s installation failed with: %s' % (cls.PARENT, cls.PARENT, err)) if cls.PARENT_ATTR is None: parent = module.klass.BROWSER else: parent = reduce(getattr, cls.PARENT_ATTR.split('.'), module) if parent is None: raise AbstractBrowserMissingParentError("Failed to load parent class") return type(cls.__name__, (parent,), dict(cls.__dict__))(*args, **kwargs) weboob-1.2/weboob/browser/cache.py000066400000000000000000000066371303450110500172120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2016 weboob project # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['CacheMixin'] class CacheEntry(object): def __init__(self, response): self.response = response self.etag = response.headers.get('ETag') self.last_modified = response.headers.get('Last-Modified') def has_cache_key(self): return (self.etag or self.last_modified) def update_request(self, request): if self.last_modified: request.headers['If-Modified-Since'] = self.last_modified if self.etag: request.headers['If-None-Match'] = self.etag class CacheMixin(object): """Mixin to inherit in a Browser""" def __init__(self, *args, **kwargs): super(CacheMixin, self).__init__(*args, **kwargs) self.cache = {} """Cache store object To limit the size of the cache, a :class:`weboob.tools.lrudict.LimitedLRUDict` instance can be used. """ self.is_updatable = True """Whether the cache is updatable If `False`, once a request has been successfully executed, its response will always be returned. If `True`, the `ETag` and `Last-Modified` of the response will be stored along with the cache. When the request is re-executed, instead of simply returning the previous response, the server is queried to check if a newer version of the page exists. If a newer page exists, it is returned instead and overwrites the obsolete page in the cache. """ def make_cache_key(self, request): """Make a key for the cache corresponding to the request.""" body = getattr(request, 'body', None) headers = tuple(request.headers.values()) return (request.method, request.url, body, headers) def open_with_cache(self, url, **kwargs): """Perform a request using the cache if possible.""" request = self.build_request(url, **kwargs) key = self.make_cache_key(request) if key in self.cache: if not self.is_updatable: self.logger.debug('cache HIT for %r', request.url) return self.cache[key].response else: self.cache[key].update_request(request) response = super(CacheMixin, self).open(request, **kwargs) if response.status_code == 304: self.logger.debug('cache HIT for %r', request.url) return self.cache[key].response elif response.status_code == 200: entry = CacheEntry(response) if entry.has_cache_key(): self.logger.debug('storing %r response in cache', request.url) self.cache[key] = entry self.logger.debug('cache MISS for %r', request.url) return response weboob-1.2/weboob/browser/cookies.py000066400000000000000000000050351303450110500175720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import requests.cookies try: import cookielib except ImportError: import http.cookiejar as cookielib __all__ = ['WeboobCookieJar'] class WeboobCookieJar(requests.cookies.RequestsCookieJar): @classmethod def from_cookiejar(klass, cj): """ Create a WeboobCookieJar from another CookieJar instance. """ return requests.cookies.merge_cookies(klass(), cj) def export(self, filename): """ Export all cookies to a file, regardless of expiration, etc. """ cj = requests.cookies.merge_cookies(cookielib.LWPCookieJar(), self) cj.save(filename, ignore_discard=True, ignore_expires=True) def _cookies_from_attrs_set(self, attrs_set, request): for tup in self._normalized_cookie_tuples(attrs_set): cookie = self._cookie_from_cookie_tuple(tup, request) if cookie: yield cookie def make_cookies(self, response, request): """Return sequence of Cookie objects extracted from response object.""" # get cookie-attributes for RFC 2965 and Netscape protocols headers = response.info() rfc2965_hdrs = headers.getheaders("Set-Cookie2") ns_hdrs = headers.getheaders("Set-Cookie") rfc2965 = self._policy.rfc2965 netscape = self._policy.netscape if netscape: for cookie in self._cookies_from_attrs_set(cookielib.parse_ns_headers(ns_hdrs), request): self._process_rfc2109_cookies([cookie]) yield cookie if rfc2965: for cookie in self._cookies_from_attrs_set(cookielib.split_header_words(rfc2965_hdrs), request): yield cookie def copy(self): """Return an object copy of the cookie jar.""" new_cj = type(self)() new_cj.update(self) return new_cj weboob-1.2/weboob/browser/elements.py000066400000000000000000000244251303450110500177560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import sys from copy import deepcopy from weboob.tools.log import getLogger, DEBUG_FILTERS from weboob.tools.ordereddict import OrderedDict from weboob.browser.pages import NextPage from .filters.standard import _Filter, CleanText from .filters.html import AttributeNotFound, XPathNotFound __all__ = ['DataError', 'AbstractElement', 'ListElement', 'ItemElement', 'TableElement', 'SkipItem'] class DataError(Exception): """ Returned data from pages are incoherent. """ def method(klass): """ Class-decorator to call it as a method. """ def inner(self, *args, **kwargs): return klass(self)(*args, **kwargs) return inner class AbstractElement(object): _creation_counter = 0 condition = None def __init__(self, page, parent=None, el=None): self.page = page self.parent = parent if el is not None: self.el = el elif parent is not None: self.el = parent.el else: self.el = page.doc if parent is not None: self.env = deepcopy(parent.env) else: self.env = deepcopy(page.params) # Used by debug self._random_id = AbstractElement._creation_counter AbstractElement._creation_counter += 1 self.loaders = {} def use_selector(self, func, key=None): if isinstance(func, _Filter): func._obj = self func._key = key value = func(self) elif isinstance(func, type) and issubclass(func, ItemElement): value = func(self.page, self, self.el)() elif callable(func): value = func() else: value = deepcopy(func) return value def parse(self, obj): pass def cssselect(self, *args, **kwargs): return self.el.cssselect(*args, **kwargs) def xpath(self, *args, **kwargs): return self.el.xpath(*args, **kwargs) def handle_loaders(self): for attrname in dir(self): m = re.match('load_(.*)', attrname) if not m: continue name = m.group(1) if name in self.loaders: continue loader = getattr(self, attrname) self.loaders[name] = self.use_selector(loader, key=attrname) class ListElement(AbstractElement): item_xpath = None flush_at_end = False ignore_duplicate = False def __init__(self, *args, **kwargs): super(ListElement, self).__init__(*args, **kwargs) self.logger = getLogger(self.__class__.__name__.lower()) self.objects = OrderedDict() def __call__(self, *args, **kwargs): for key, value in kwargs.iteritems(): self.env[key] = value return self.__iter__() def find_elements(self): """ Get the nodes that will have to be processed. This method can be overridden if xpath filters are not sufficient. """ if self.item_xpath is not None: for el in self.el.xpath(self.item_xpath): yield el else: yield self.el def __iter__(self): if self.condition is not None and not self.condition(): return self.parse(self.el) items = [] for el in self.find_elements(): for attrname in dir(self): attr = getattr(self, attrname) if isinstance(attr, type) and issubclass(attr, AbstractElement) and attr != type(self): item = attr(self.page, self, el) if item.condition is not None and not item.condition(): continue item.handle_loaders() items.append(item) for item in items: for obj in item: obj = self.store(obj) if obj and not self.flush_at_end: yield obj if self.flush_at_end: for obj in self.flush(): yield obj self.check_next_page() def flush(self): for obj in self.objects.itervalues(): yield obj def check_next_page(self): if not hasattr(self, 'next_page'): return next_page = getattr(self, 'next_page') try: value = self.use_selector(next_page) except (AttributeNotFound, XPathNotFound): return if value is None: return raise NextPage(value) def store(self, obj): if obj.id: if obj.id in self.objects: if self.ignore_duplicate: self.logger.warning('There are two objects with the same ID! %s' % obj.id) return else: raise DataError('There are two objects with the same ID! %s' % obj.id) self.objects[obj.id] = obj return obj class SkipItem(Exception): """ Raise this exception in an :class:`ItemElement` subclass to skip an item. """ class _ItemElementMeta(type): """ Private meta-class used to keep order of obj_* attributes in :class:`ItemElement`. """ def __new__(mcs, name, bases, attrs): _attrs = [] for base in bases: if hasattr(base, '_attrs'): _attrs += base._attrs filters = [(re.sub('^obj_', '', attr_name), attrs[attr_name]) for attr_name, obj in attrs.items() if attr_name.startswith('obj_')] # constants first, then filters, then methods filters.sort(key=lambda x: x[1]._creation_counter if hasattr(x[1], '_creation_counter') else (sys.maxsize if callable(x[1]) else 0)) new_class = super(_ItemElementMeta, mcs).__new__(mcs, name, bases, attrs) new_class._attrs = _attrs + [f[0] for f in filters] return new_class class ItemElement(AbstractElement): __metaclass__ = _ItemElementMeta _attrs = None _loaders = None klass = None validate = None class Index(object): pass def __init__(self, *args, **kwargs): super(ItemElement, self).__init__(*args, **kwargs) self.logger = getLogger(self.__class__.__name__.lower()) self.obj = None def build_object(self): if self.klass is None: return return self.klass() def __call__(self, obj=None): if obj is not None: self.obj = obj for obj in self: return obj def __iter__(self): if self.condition is not None and not self.condition(): return try: if self.obj is None: self.obj = self.build_object() self.parse(self.el) self.handle_loaders() for attr in self._attrs: self.handle_attr(attr, getattr(self, 'obj_%s' % attr)) except SkipItem: return if self.validate is not None and not self.validate(self.obj): return yield self.obj def handle_attr(self, key, func): try: value = self.use_selector(func, key=key) except Exception as e: # Help debugging as tracebacks do not give us the key self.logger.warning('Attribute %s raises %s' % (key, repr(e))) raise logger = getLogger('b2filters') logger.log(DEBUG_FILTERS, "%s.%s = %r" % (self._random_id, key, value)) setattr(self.obj, key, value) class TableElement(ListElement): head_xpath = None cleaner = CleanText def __init__(self, *args, **kwargs): super(TableElement, self).__init__(*args, **kwargs) self._cols = {} columns = {} for attrname in dir(self): m = re.match('col_(.*)', attrname) if m: cols = getattr(self, attrname) if not isinstance(cols, (list,tuple)): cols = [cols] columns[m.group(1)] = [s.lower() if isinstance(s, (str, unicode)) else s for s in cols] colnum = 0 for el in self.el.xpath(self.head_xpath): title = self.cleaner.clean(el) for name, titles in columns.iteritems(): if name in self._cols: continue if title.lower() in [s for s in titles if isinstance(s, (str, unicode))] or \ any(map(lambda x: x.match(title), [s for s in titles if isinstance(s, type(re.compile('')))])): self._cols[name] = colnum try: colnum += int(el.attrib.get('colspan', 1)) except (ValueError, AttributeError): colnum += 1 def get_colnum(self, name): return self._cols.get(name, None) class DictElement(ListElement): def find_elements(self): if self.item_xpath is None: selector = [] elif isinstance(self.item_xpath, basestring): selector = self.item_xpath.split('/') else: selector = self.item_xpath _subdict = False for el in selector: if isinstance(self.el, list): if _subdict: subdicts = [] for i in range (0, len(self.el)): _el = int(el) if isinstance(self.el[i], list) else el subdicts += self.el[i][_el] self.el = subdicts _subdict = False continue if el == '*': _subdict = True continue elif el.isdigit(): el = int(el) if int(el) < len(self.el) else 0 self.el = self.el[el] for el in self.el: yield el weboob-1.2/weboob/browser/exceptions.py000066400000000000000000000020011303450110500203050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from requests.exceptions import HTTPError from weboob.exceptions import BrowserHTTPError, BrowserHTTPNotFound class HTTPNotFound(HTTPError, BrowserHTTPNotFound): pass class ClientError(HTTPError, BrowserHTTPError): pass class ServerError(HTTPError, BrowserHTTPError): pass weboob-1.2/weboob/browser/filters/000077500000000000000000000000001303450110500172315ustar00rootroot00000000000000weboob-1.2/weboob/browser/filters/__init__.py000066400000000000000000000013311303450110500213400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . weboob-1.2/weboob/browser/filters/html.py000066400000000000000000000045451303450110500205570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import lxml.html as html from .standard import _Selector, _NO_DEFAULT, Filter, FilterError from weboob.tools.html import html2text __all__ = ['CSS', 'XPath', 'XPathNotFound', 'AttributeNotFound', 'Attr', 'Link', 'CleanHTML'] class XPathNotFound(FilterError): pass class AttributeNotFound(FilterError): pass class CSS(_Selector): @classmethod def select(cls, selector, item, obj=None, key=None): return item.cssselect(selector) class XPath(_Selector): pass class Attr(Filter): def __init__(self, selector, attr, default=_NO_DEFAULT): super(Attr, self).__init__(selector, default=default) self.attr = attr def filter(self, el): try: return u'%s' % el[0].attrib[self.attr] except IndexError: return self.default_or_raise(XPathNotFound('Unable to find tag %s' % self.selector)) except KeyError: return self.default_or_raise(AttributeNotFound('Element %s does not have attribute %s' % (el[0], self.attr))) class Link(Attr): """ Get the link uri of an element. If the tag is not found, an exception IndexError is raised. """ def __init__(self, selector=None, default=_NO_DEFAULT): super(Link, self).__init__(selector, 'href', default=default) class CleanHTML(Filter): def filter(self, txt): if isinstance(txt, (tuple, list)): return u' '.join([self.clean(item) for item in txt]) return self.clean(txt) @classmethod def clean(cls, txt): if not isinstance(txt, basestring): txt = html.tostring(txt, encoding=unicode) return html2text(txt) weboob-1.2/weboob/browser/filters/javascript.py000066400000000000000000000136361303450110500217620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Simon Murail # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from ast import literal_eval from weboob.browser.filters.standard import Filter, Regexp, RegexpError from weboob.exceptions import ParseError __all__ = ['JSPayload', 'JSValue', 'JSVar'] def _quoted(q): return r'(?: >>> JSPayload.filter('''someString = "An example comment: /* example */"; ... ... // The comment around this code has been commented out. ... // /* ... some_code(); ... // */''') 'someString = "An example comment: /* example */";\n\nsome_code();\n' """ _single_line_comment = '[ \t\v\f]*//.*\r?(?:\n|$)' _multi_line_comment = '/\*(?:.|[\r\n])*?\*/' _splitter = re.compile('(?:(%s|%s)|%s|%s)' % (_quoted('"'), _quoted("'"), _single_line_comment, _multi_line_comment)) @classmethod def filter(cls, value): return ''.join(filter(bool, cls._splitter.split(value))) class JSValue(Regexp): r""" Get one or many JavaScript literals. It only understands literal values, but should parse them well. Values are converted in python values, quotes and slashes in strings are stripped. >>> JSValue().filter('boringVar = "boring string"') u'boring string' >>> JSValue().filter('somecode(); doConfuse(0xdead, cat);') 57005 >>> JSValue(need_type=int, nth=2).filter('fazboo("3", "5", 7, "9");') 7 >>> JSValue(nth='*').filter('foo([1, 2, 3], "blah", 5.0, true, null]);') [1, 2, 3, u'blah', 5.0, True, None] """ pattern = r"""(?x) (?:(?P(?:[-+]\s*)? # float ? (?:(?:\d+\.\d*|\d*\.\d+)(?:[eE]\d+)? |\d+[eE]\d+)) |(?P(?:[-+]\s*)?(?:0[bB][01]+ # int ? |0[oO][0-7]+ |0[xX][0-9a-fA-F]+ |\d+)) |(?:(?:(?:new\s+)?String\()?(?P(?:%s|%s))) # str ? |(?Ptrue|false) # bool ? |(?Pnull)) # None ? """ % (_quoted('"'), _quoted("'")) def to_python(self, m): "Convert MatchObject to python value" values = m.groupdict() for t, v in values.iteritems(): if v is not None: break if self.need_type and t != self.need_type: raise ParseError('Value with type %s not found' % self.need_type) if t in ('int', 'float'): return literal_eval(v) if t == 'str': return literal_eval(v).decode('utf-8') if t == 'bool': return v == 'true' if t == 'None': return if self.default: return self.default raise ParseError('Unable to parse %r value' % m.group(0)) def __init__(self, selector=None, need_type=None, **kwargs): assert 'pattern' not in kwargs and 'flags' not in kwargs, \ "It would be meaningless to define a pattern and/or flags, use Regexp" assert 'template' not in kwargs, "Can't use a template, use Regexp if you have to" self.need_type = need_type.__name__ if type(need_type) == type else need_type super(JSValue, self).__init__(selector, pattern=self.pattern, template=self.to_python, **kwargs) class JSVar(JSValue): r""" Get assigned value of a variable, either as an initialisation value, either as an assignement. One can use Regexp's nth parameter to be more specific. See JSValue for more details about parsed values. >>> JSVar(var='test').filter("var test = .1;\nsomecode()") 0.1 >>> JSVar(var='test').filter("test = 666;\nsomecode()") 666 >>> JSVar(var='test').filter("test = 'Some \\'string\\' value, isn\\'t it ?';\nsomecode()") u"Some 'string' value, isn't it ?" >>> JSVar(var='test').filter('test = "Some \\"string\\" value";\nsomecode()') u'Some "string" value' >>> JSVar(var='test').filter("var test = false;\nsomecode()") False >>> JSVar(var='test', nth=1).filter("var test = false; test = true;\nsomecode()") True """ pattern_template = r"""(?x) (?:var\s+)? # optional var keyword \b%s # var name \s*=\s* # equal sign """ + JSValue.pattern def __init__(self, selector=None, var=None, need_type=None, **kwargs): assert var is not None, 'Please give a var parameter' self.var = var self.pattern = self.pattern_template % re.escape(var) super(JSVar, self).__init__(selector, need_type=need_type, **kwargs) def filter(self, txt): try: return super(JSVar, self).filter(txt) except RegexpError: raise ParseError('Variable %r not found' % self.var) weboob-1.2/weboob/browser/filters/json.py000066400000000000000000000045611303450110500205620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014-2015 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .standard import _Filter, _NO_DEFAULT, Filter, ParseError __all__ = ['Dict'] class NotFound(object): def __repr__(self): return 'NOT_FOUND' _NOT_FOUND = NotFound() class _DictMeta(type): def __getitem__(cls, name): return cls(name) class Dict(Filter): __metaclass__ = _DictMeta def __init__(self, selector=None, default=_NO_DEFAULT): super(Dict, self).__init__(self, default=default) if selector is None: self.selector = [] elif isinstance(selector, basestring): self.selector = selector.split('/') elif callable(selector): self.selector = [selector] else: self.selector = selector def __getitem__(self, name): self.selector.append(name) return self def filter(self, elements): if elements is not _NOT_FOUND: return elements else: return self.default_or_raise(ParseError('Element %r not found' % self.selector)) @classmethod def select(cls, selector, item, obj=None, key=None): if isinstance(item, (dict, list)): content = item else: content = item.el for el in selector: if isinstance(content, list): el = int(el) elif isinstance(el, _Filter): el._key = key el._obj = obj el = el(item) elif callable(el): el = el(item) try: content = content[el] except (KeyError, IndexError, TypeError): return _NOT_FOUND return content weboob-1.2/weboob/browser/filters/standard.py000066400000000000000000000634271303450110500214170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import absolute_import import datetime import re import unicodedata from decimal import Decimal, InvalidOperation from itertools import islice from collections import Iterator from dateutil.parser import parse as parse_date from weboob.capabilities.base import empty from weboob.capabilities.base import Currency as BaseCurrency from weboob.tools.compat import basestring from weboob.exceptions import ParseError from weboob.browser.url import URL from weboob.tools.log import getLogger, DEBUG_FILTERS class NoDefault(object): def __repr__(self): return 'NO_DEFAULT' _NO_DEFAULT = NoDefault() __all__ = ['FilterError', 'ColumnNotFound', 'RegexpError', 'ItemNotFound', 'Filter', 'Base', 'Env', 'TableCell', 'RawText', 'CleanText', 'Lower', 'Upper', 'Capitalize', 'CleanDecimal', 'Field', 'Regexp', 'Map', 'DateTime', 'Date', 'Time', 'DateGuesser', 'Duration', 'MultiFilter', 'CombineDate', 'Format', 'Join', 'Type', 'Eval', 'BrowserURL', 'Async', 'AsyncLoad'] class FilterError(ParseError): pass class ColumnNotFound(FilterError): pass class RegexpError(FilterError): pass class ItemNotFound(FilterError): pass class _Filter(object): _creation_counter = 0 def __init__(self, default=_NO_DEFAULT): self._key = None self._obj = None self.default = default self._creation_counter = _Filter._creation_counter _Filter._creation_counter += 1 def __or__(self, o): self.default = o return self def __and__(self, o): if isinstance(o, type) and issubclass(o, _Filter): o = o() o.selector = self return o def default_or_raise(self, exception): if self.default is not _NO_DEFAULT: return self.default else: raise exception def __str__(self): return self.__class__.__name__ def debug(*args): """ A decorator function to provide some debug information in Filters. It prints by default the name of the Filter and the input value. """ def wraper(function): def print_debug(self, value): logger = getLogger('b2filters') result = '' outputvalue = value if isinstance(value, list): from lxml import etree outputvalue = '' first = True for element in value: if first: first = False else: outputvalue += ', ' if isinstance(element, etree.ElementBase): outputvalue += "%s" % etree.tostring(element, encoding=unicode) else: outputvalue += "%r" % element if self._obj is not None: result += "%s" % self._obj._random_id if self._key is not None: result += ".%s" % self._key name = str(self) result += " %s(%r" % (name, outputvalue) for arg in self.__dict__: if arg.startswith('_') or arg == u"selector": continue if arg == u'default' and getattr(self, arg) == _NO_DEFAULT: continue result += ", %s=%r" % (arg, getattr(self, arg)) result += u')' logger.log(DEBUG_FILTERS, result) res = function(self, value) return res return print_debug return wraper class Filter(_Filter): """ Class used to filter on a HTML element given as call parameter to return matching elements. Filters can be chained, so the parameter supplied to constructor can be either a xpath selector string, or an other filter called before. >>> from lxml.html import etree >>> f = CleanDecimal(CleanText('//p'), replace_dots=True) >>> f(etree.fromstring('

blah: 229,90

')) Decimal('229.90') """ def __init__(self, selector=None, default=_NO_DEFAULT): super(Filter, self).__init__(default=default) self.selector = selector @classmethod def select(cls, selector, item, obj=None, key=None): if isinstance(selector, basestring): return item.xpath(selector) elif isinstance(selector, _Filter): selector._key = key selector._obj = obj return selector(item) elif callable(selector): return selector(item) else: return selector def __call__(self, item): return self.filter(self.select(self.selector, item, key=self._key, obj=self._obj)) @debug() def filter(self, value): """ This method have to be overrided by children classes. """ raise NotImplementedError() class _Selector(Filter): def filter(self, elements): if elements is not None: return elements else: return self.default_or_raise(ParseError('Element %r not found' % self.selector)) class AsyncLoad(Filter): def __call__(self, item): link = self.select(self.selector, item, key=self._key, obj=self._obj) return item.page.browser.async_open(link) if link else None class Async(_Filter): def __init__(self, name, selector=None): super(Async, self).__init__() self.selector = selector self.name = name def __and__(self, o): if isinstance(o, type) and issubclass(o, _Filter): o = o() self.selector = o return self def __call__(self, item): if item.loaders[self.name] is None: return None return self.selector(self.loaded_page(item).doc) def loaded_page(self, item): result = item.loaders[self.name].result() assert result.page is not None, 'The loaded url %s hasn\'t been matched by an URL object' % result.url return result.page class Base(Filter): """ Change the base element used in filters. >>> Base(Env('header'), CleanText('./h1')) # doctest: +SKIP """ def __call__(self, item): base = self.select(self.base, item, obj=self._obj, key=self._key) return self.selector(base) def __init__(self, base, selector=None, default=_NO_DEFAULT): super(Base, self).__init__(selector, default) self.base = base class Decode(Filter): """ Filter that aims to decode urlencoded strings >>> Decode(Env('_id')) # doctest: +ELLIPSIS >>> from .html import Link >>> Decode(Link('./a')) # doctest: +ELLIPSIS """ def __call__(self, item): self.encoding = item.page.ENCODING if item.page.ENCODING else 'utf-8' return self.filter(self.select(self.selector, item, key=self._key, obj=self._obj)) @debug() def filter(self, txt): from urllib import unquote try: txt = unquote(txt.encode('ascii')).decode(self.encoding) except (UnicodeDecodeError, UnicodeEncodeError): pass return txt class Env(_Filter): """ Filter to get environment value of the item. It is used for example to get page parameters, or when there is a parse() method on ItemElement. """ def __init__(self, name, default=_NO_DEFAULT): super(Env, self).__init__(default) self.name = name def __call__(self, item): try: return item.env[self.name] except KeyError: return self.default_or_raise(ParseError('Environment variable %s not found' % self.name)) class TableCell(_Filter): """ Used with TableElement, it get the cell value from its name. For example: >>> from weboob.capabilities.bank import Transaction >>> from weboob.browser.elements import TableElement, ItemElement >>> class table(TableElement): ... head_xpath = '//table/thead/th' ... item_xpath = '//table/tbody/tr' ... col_date = u'Date' ... col_label = [u'Name', u'Label'] ... class item(ItemElement): ... klass = Transaction ... obj_date = Date(TableCell('date')) ... obj_label = CleanText(TableCell('label')) ... """ def __init__(self, *names, **kwargs): support_th = kwargs.pop('support_th', False) super(TableCell, self).__init__(**kwargs) self.names = names if support_th: self.td = '(./th | ./td)[%s]' else: self.td = './td[%s]' def __call__(self, item): for name in self.names: idx = item.parent.get_colnum(name) if idx is not None: return item.xpath(self.td % (idx + 1)) return self.default_or_raise(ColumnNotFound('Unable to find column %s' % ' or '.join(self.names))) class RawText(Filter): def __init__(self, selector=None, children=False, default=_NO_DEFAULT): super(RawText, self).__init__(selector, default=default) self.children = children @debug() def filter(self, el): if isinstance(el, (tuple, list)): return u' '.join([self.filter(e) for e in el]) if self.children: text = el.text_content() else: text = el.text if text is None: result = self.default else: result = unicode(text) return result class CleanText(Filter): """ Get a cleaned text from an element. It first replaces all tabs and multiple spaces (including newlines if ``newlines`` is True) to one space and strips the result string. The result is coerced into unicode, and optionally normalized according to the ``normalize`` argument. Then it replaces all symbols given in the ``symbols`` argument. >>> CleanText().filter('coucou ') u'coucou' >>> CleanText().filter(u'coucou\xa0coucou') u'coucou coucou' >>> CleanText(newlines=True).filter(u'coucou\\r\\n coucou ') u'coucou coucou' >>> CleanText(newlines=False).filter(u'coucou\\r\\n coucou ') u'coucou\\ncoucou' """ def __init__(self, selector=None, symbols='', replace=[], children=True, newlines=True, normalize='NFC', **kwargs): super(CleanText, self).__init__(selector, **kwargs) self.symbols = symbols self.toreplace = replace self.children = children self.newlines = newlines self.normalize = normalize @debug() def filter(self, txt): if isinstance(txt, (tuple, list)): txt = u' '.join([self.clean(item, children=self.children) for item in txt]) txt = self.clean(txt, self.children, self.newlines, self.normalize) txt = self.remove(txt, self.symbols) txt = self.replace(txt, self.toreplace) # ensure it didn't become str by mistake return unicode(txt) @classmethod def clean(cls, txt, children=True, newlines=True, normalize='NFC'): if not isinstance(txt, basestring): if children: txt = [t.strip() for t in txt.itertext()] else: txt = [txt.text.strip()] txt = u' '.join(txt) # 'foo bar' if newlines: txt = re.compile(u'\s+', flags=re.UNICODE).sub(u' ', txt) # 'foo bar' else: # normalize newlines and clean what is inside txt = '\n'.join([cls.clean(l) for l in txt.splitlines()]) txt = txt.strip() # lxml under Python 2 returns str instead of unicode if it is pure ASCII txt = unicode(txt) # normalize to a standard Unicode form if normalize: txt = unicodedata.normalize(normalize, txt) return txt @classmethod def remove(cls, txt, symbols): for symbol in symbols: txt = txt.replace(symbol, '') return txt.strip() @classmethod def replace(cls, txt, replace): for before, after in replace: txt = txt.replace(before, after) return txt class Lower(CleanText): @debug() def filter(self, txt): txt = super(Lower, self).filter(txt) return txt.lower() class Upper(CleanText): @debug() def filter(self, txt): txt = super(Upper, self).filter(txt) return txt.upper() class Capitalize(CleanText): @debug() def filter(self, txt): txt = super(Capitalize, self).filter(txt) return txt.title() class Currency(CleanText): @debug() def filter(self, txt): txt = super(Currency, self).filter(txt) return BaseCurrency.get_currency(txt) class CleanDecimal(CleanText): """ Get a cleaned Decimal value from an element. replace_dots is False by default. A dot is interpreted as a decimal separator. If replace_dots is set to True, we remove all the dots. The ',' is used as decimal separator (often useful for French values) If replace_dots is a tuple, the first element will be used as the thousands separator, and the second as the decimal separator. See http://en.wikipedia.org/wiki/Thousands_separator#Examples_of_use For example, for the UK style (as in 1,234,567.89): >>> CleanDecimal('./td[1]', replace_dots=(',', '.')) # doctest: +SKIP """ def __init__(self, selector=None, replace_dots=False, sign=None, default=_NO_DEFAULT): super(CleanDecimal, self).__init__(selector, default=default) self.replace_dots = replace_dots self.sign = sign @debug() def filter(self, text): if type(text) in (float, int, long): text = str(text) if empty(text): return self.default_or_raise(ParseError('Unable to parse %r' % text)) original_text = text = super(CleanDecimal, self).filter(text) if self.replace_dots: if type(self.replace_dots) is tuple: thousands_sep, decimal_sep = self.replace_dots else: thousands_sep, decimal_sep = '.', ',' text = text.replace(thousands_sep, '').replace(decimal_sep, '.') try: v = Decimal(re.sub(r'[^\d\-\.]', '', text)) if self.sign: v *= self.sign(original_text) return v except InvalidOperation as e: return self.default_or_raise(e) class Slugify(Filter): @debug() def filter(self, label): label = re.sub(r'[^A-Za-z0-9]', ' ', label.lower()).strip() label = re.sub(r'\s+', '-', label) return label class Type(Filter): """ Get a cleaned value of any type from an element text. The type_func can be any callable (class, function, etc.). By default an empty string will not be parsed but it can be changed by specifying minlen=False. Otherwise, a minimal length can be specified. >>> Type(CleanText('./td[1]'), type=int) # doctest: +SKIP >>> Type(type=int).filter('42') 42 >>> Type(type=int, default='NaN').filter('') 'NaN' >>> Type(type=str, minlen=False, default='a').filter('') '' >>> Type(type=str, minlen=0, default='a').filter('') 'a' """ def __init__(self, selector=None, type=None, minlen=0, default=_NO_DEFAULT): super(Type, self).__init__(selector, default=default) self.type_func = type self.minlen = minlen @debug() def filter(self, txt): if empty(txt): return self.default_or_raise(ParseError('Unable to parse %r' % txt)) if self.minlen is not False and len(txt) <= self.minlen: return self.default_or_raise(ParseError('Unable to parse %r' % txt)) try: return self.type_func(txt) except ValueError as e: return self.default_or_raise(ParseError('Unable to parse %r: %s' % (txt, e))) class Field(_Filter): """ Get the attribute of object. """ def __init__(self, name): super(Field, self).__init__() self.name = name def __call__(self, item): return item.use_selector(getattr(item, 'obj_%s' % self.name), key=self._key) # Based on nth from https://docs.python.org/2/library/itertools.html def nth(iterable, n, default=None): "Returns the nth item or a default value, n can be negative, or '*' for all" if n == '*': return iterable if n < 0: iterable = reversed(list(iterable)) n = abs(n) - 1 return next(islice(iterable, n, None), default) def ordinal(n): "To have some readable debug information: '*' => all, 0 => 1st, 1 => 2nd..." if n == '*': return 'all' i = abs(n) n = n - 1 if n < 0 else n + 1 return str(n) + ('th' if i > 2 else ['st', 'nd', 'rd'][i]) class Regexp(Filter): r""" Apply a regex. >>> from lxml.html import etree >>> doc = etree.fromstring('

Date: 13/08/1988

') >>> Regexp(CleanText('//p'), r'Date: (\d+)/(\d+)/(\d+)', '\\3-\\2-\\1')(doc) u'1988-08-13' >>> (Regexp(CleanText('//body'), r'(\d+)', nth=1))(doc) u'08' >>> (Regexp(CleanText('//body'), r'(\d+)', nth=-1))(doc) u'1988' >>> (Regexp(CleanText('//body'), r'(\d+)', template='[\\1]', nth='*'))(doc) [u'[13]', u'[08]', u'[1988]'] """ def __init__(self, selector=None, pattern=None, template=None, nth=0, flags=0, default=_NO_DEFAULT): super(Regexp, self).__init__(selector, default=default) assert pattern is not None self.pattern = pattern self._regex = re.compile(pattern, flags) self.template = template self.nth = nth def expand(self, m): if self.template is None: return next(g for g in m.groups() if g is not None) return self.template(m) if callable(self.template) else m.expand(self.template) @debug() def filter(self, txt): if isinstance(txt, (tuple, list)): txt = u' '.join([t.strip() for t in txt.itertext()]) m = self._regex.search(txt) if self.nth == 0 else \ nth(self._regex.finditer(txt), self.nth) if not m: msg = 'Unable to find %s %s in %r' % (ordinal(self.nth), self.pattern, txt) return self.default_or_raise(RegexpError(msg)) if isinstance(m, Iterator): return map(self.expand, m) return self.expand(m) class Map(Filter): def __init__(self, selector, map_dict, default=_NO_DEFAULT): super(Map, self).__init__(selector, default=default) self.map_dict = map_dict @debug() def filter(self, txt): try: return self.map_dict[txt] except KeyError: return self.default_or_raise(ItemNotFound('Unable to handle %r on %r' % (txt, self.map_dict))) class DateTime(Filter): def __init__(self, selector=None, default=_NO_DEFAULT, dayfirst=False, translations=None, parse_func=parse_date, fuzzy=False): super(DateTime, self).__init__(selector, default=default) self.dayfirst = dayfirst self.translations = translations self.parse_func = parse_func self.fuzzy = fuzzy @debug() def filter(self, txt): if empty(txt) or txt == '': return self.default_or_raise(ParseError('Unable to parse %r' % txt)) try: if self.translations: for search, repl in self.translations: txt = search.sub(repl, txt) return self.parse_func(txt, dayfirst=self.dayfirst, fuzzy=self.fuzzy) except (ValueError, TypeError) as e: return self.default_or_raise(ParseError('Unable to parse %r: %s' % (txt, e))) class Date(DateTime): def __init__(self, selector=None, default=_NO_DEFAULT, dayfirst=False, translations=None, parse_func=parse_date, fuzzy=False): super(Date, self).__init__(selector, default=default, dayfirst=dayfirst, translations=translations, parse_func=parse_func, fuzzy=fuzzy) @debug() def filter(self, txt): datetime = super(Date, self).filter(txt) if hasattr(datetime, 'date'): return datetime.date() else: return datetime class DateGuesser(Filter): def __init__(self, selector, date_guesser, **kwargs): super(DateGuesser, self).__init__(selector) self.date_guesser = date_guesser self.kwargs = kwargs def __call__(self, item): values = self.select(self.selector, item, obj=self._obj, key=self._key) date_guesser = self.date_guesser # In case Env() is used to kive date_guesser. if isinstance(date_guesser, _Filter): date_guesser = self.select(date_guesser, item, obj=self._obj, key=self._key) if isinstance(values, basestring): values = re.split('[/-]', values) if len(values) == 2: day, month = map(int, values) else: raise ParseError('Unable to take (day, month) tuple from %r' % values) return date_guesser.guess_date(day, month, **self.kwargs) class Time(Filter): klass = datetime.time _regexp = re.compile(r'(?P\d+)[:h]?(?P\d+)([:m](?P\d+))?') kwargs = {'hour': 'hh', 'minute': 'mm', 'second': 'ss'} def __init__(self, selector=None, default=_NO_DEFAULT): super(Time, self).__init__(selector, default=default) @debug() def filter(self, txt): m = self._regexp.search(txt) if m: kwargs = {} for key, index in self.kwargs.iteritems(): kwargs[key] = int(m.groupdict()[index] or 0) return self.klass(**kwargs) return self.default_or_raise(ParseError('Unable to find time in %r' % txt)) class Duration(Time): klass = datetime.timedelta _regexp = re.compile(r'((?P\d+)[:;])?(?P\d+)[;:](?P\d+)') kwargs = {'hours': 'hh', 'minutes': 'mm', 'seconds': 'ss'} class MultiFilter(Filter): def __init__(self, *args, **kwargs): default = kwargs.pop('default', _NO_DEFAULT) super(MultiFilter, self).__init__(args, default) def __call__(self, item): values = [self.select(selector, item, obj=self._obj, key=self._key) for selector in self.selector] return self.filter(tuple(values)) def filter(self, values): raise NotImplementedError() class CombineDate(MultiFilter): def __init__(self, date, time): super(CombineDate, self).__init__(date, time) @debug() def filter(self, values): return datetime.datetime.combine(values[0], values[1]) class Format(MultiFilter): def __init__(self, fmt, *args): super(Format, self).__init__(*args) self.fmt = fmt @debug() def filter(self, values): return self.fmt % values class BrowserURL(MultiFilter): def __init__(self, url_name, **kwargs): super(BrowserURL, self).__init__(*kwargs.values()) self.url_name = url_name self.keys = kwargs.keys() def __call__(self, item): values = super(BrowserURL, self).__call__(item) url = getattr(item.page.browser, self.url_name) assert isinstance(url, URL), "%s.%s must be an URL object" % (type(item.page.browser).__name__, self.url_name) return url.build(**dict(zip(self.keys, values))) @debug() def filter(self, values): return values class Join(Filter): def __init__(self, pattern, selector=None, textCleaner=CleanText, newline=False, addBefore='', addAfter=''): super(Join, self).__init__(selector) self.pattern = pattern self.textCleaner = textCleaner self.newline = newline self.addBefore = addBefore self.addAfter = addAfter @debug() def filter(self, el): items = [self.textCleaner.clean(e) for e in el] items = [item for item in items if item] if self.newline: items = ['%s\r\n' % item for item in items] result = self.pattern.join(items) if self.addBefore: result = '%s%s' % (self.addBefore, result) if self.addAfter: result = '%s%s' % (result, self.addAfter) return result class Eval(MultiFilter): """ Evaluate a function with given 'deferred' arguments. >>> F = Field; Eval(lambda a, b, c: a * b + c, F('foo'), F('bar'), F('baz')) # doctest: +SKIP >>> Eval(lambda x, y: x * y + 1).filter([3, 7]) 22 """ def __init__(self, func, *args): super(Eval, self).__init__(*args) self.func = func @debug() def filter(self, values): return self.func(*values) def test_CleanText(): # This test works poorly under a doctest, or would be hard to read assert CleanText().filter(u' coucou  \n\théhé') == u'coucou héhé' assert CleanText().filter('coucou\xa0coucou') == CleanText().filter(u'coucou\xa0coucou') == u'coucou coucou' # Unicode normalization assert CleanText().filter(u'Éçã') == u'Éçã' assert CleanText(normalize='NFKC').filter(u'…') == u'...' assert CleanText().filter(u'…') == u'…' # Diacritical mark (dakuten) assert CleanText().filter(u'\u3053\u3099') == u'\u3054' assert CleanText(normalize='NFD').filter(u'\u3053\u3099') == u'\u3053\u3099' assert CleanText(normalize='NFD').filter(u'\u3054') == u'\u3053\u3099' assert CleanText(normalize=False).filter(u'\u3053\u3099') == u'\u3053\u3099' weboob-1.2/weboob/browser/pages.py000066400000000000000000000642441303450110500172440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import absolute_import import warnings from io import BytesIO import codecs from cgi import parse_header try: import urlparse except ImportError: import urllib.parse as urlparse import requests from weboob.exceptions import ParseError, ModuleInstallError from weboob.tools.compat import basestring from weboob.tools.log import getLogger from weboob.tools.ordereddict import OrderedDict from weboob.tools.pdf import decompress_pdf def pagination(func): r""" This helper decorator can be used to handle pagination pages easily. When the called function raises an exception :class:`NextPage`, it goes on the wanted page and recall the function. :class:`NextPage` constructor can take an url or a Request object. >>> class Page(HTMLPage): ... @pagination ... def iter_values(self): ... for el in self.doc.xpath('//li'): ... yield el.text ... for next in self.doc.xpath('//a'): ... raise NextPage(next.attrib['href']) ... >>> from .browsers import PagesBrowser >>> from .url import URL >>> class Browser(PagesBrowser): ... BASEURL = 'https://people.symlink.me' ... list = URL('/~rom1/projects/weboob/list-(?P\d+).html', Page) ... >>> b = Browser() >>> b.list.go(pagenum=1) # doctest: +ELLIPSIS >>> list(b.page.iter_values()) ['One', 'Two', 'Three', 'Four'] """ def inner(page, *args, **kwargs): while True: try: for r in func(page, *args, **kwargs): yield r except NextPage as e: result = page.browser.location(e.request) page = result.page else: return return inner class NextPage(Exception): """ Exception used for example in a Page to tell PagesBrowser.pagination to go on the next page. See :meth:`PagesBrowser.pagination` or decorator :func:`pagination`. """ def __init__(self, request): super(NextPage, self).__init__() self.request = request class Page(object): """ Represents a page. Encoding can be forced by setting the :attr:`ENCODING` class-wide attribute, or by passing an `encoding` keyword argument, which overrides :attr:`ENCODING`. Finally, it can be manually changed by assigning a new value to :attr:`encoding` instance attribute. A unicode version of the response content is accessible in :attr:`text`, decoded with specified :attr:`encoding`. :param browser: browser used to go on the page :type browser: :class:`weboob.browser.browsers.Browser` :param response: response object :type response: :class:`Response` :param params: optional dictionary containing parameters given to the page (see :class:`weboob.browser.url.URL`) :type params: :class:`dict` :param encoding: optional parameter to force the encoding of the page, overrides :attr:`ENCODING` :type encoding: :class:`basestring` """ ENCODING = None """ Force a page encoding. It is recommended to use None for autodetection. """ logged = False """ If True, the page is in a restricted area of the website. Useful with :class:`LoginBrowser` and the :func:`need_login` decorator. """ def __init__(self, browser, response, params=None, encoding=None): self.browser = browser self.logger = getLogger(self.__class__.__name__.lower(), browser.logger) self.response = response self.url = self.response.url self.params = params # Setup encoding and build document self.forced_encoding = encoding or self.ENCODING if self.forced_encoding: self.response.encoding = self.forced_encoding self.doc = self.build_doc(self.data) # Last chance to change encoding, according to :meth:`detect_encoding`, # which can be used to detect a document-level encoding declaration if not self.forced_encoding: encoding = self.detect_encoding() if encoding and encoding != self.encoding: self.response.encoding = encoding self.doc = self.build_doc(self.data) # Encoding issues are delegated to Response instance, implemented by # requests module. @property def encoding(self): return self.response.encoding @encoding.setter def encoding(self, value): self.forced_encoding = True self.response.encoding = value @property def content(self): """ Raw content from response. """ return self.response.content @property def text(self): """ Content of the response, in unicode, decoded with :attr:`encoding`. """ return self.response.text @property def data(self): """ Data passed to :meth:`build_doc`. """ return self.content def on_load(self): """ Event called when browser loads this page. """ def on_leave(self): """ Event called when browser leaves this page. """ def build_doc(self, content): """ Abstract method to be implemented by subclasses to build structured data (HTML, Json, CSV...) from :attr:`data` property. It also can be overriden in modules pages to preprocess or postprocess data. It must return an object -- that will be assigned to :attr:`doc`. """ raise NotImplementedError() def detect_encoding(self): """ Override this method to implement detection of document-level encoding declaration, if any (eg. html5's ). """ return None def absurl(self, url): """ Get an absolute URL from an a partial URL, relative to the Page URL """ return urlparse.urljoin(self.url, url) class FormNotFound(Exception): """ Raised when :meth:`HTMLPage.get_form` can't find a form. """ class FormSubmitWarning(UserWarning): """ A form has more than one submit element selected, and will likely generate an invalid request. """ class Form(OrderedDict): """ Represents a form of an HTML page. It is used as a dict with pre-filled values from HTML. You can set new values as strings by setting an item value. It is recommended to not use this class by yourself, but call :meth:`HTMLPage.get_form`. :param page: the page where the form is located :type page: :class:`Page` :param el: the form element on the page :param submit_el: allows you to only consider one submit button (which is what browsers do). If set to None, it takes all of them, and if set to False, it takes none. """ def __init__(self, page, el, submit_el=None): super(Form, self).__init__() self.page = page self.el = el self.submit_el = submit_el self.method = el.attrib.get('method', 'GET') self.url = el.attrib.get('action', page.url) self.name = el.attrib.get('name', '') self.req = None submits = 0 # Find all elements of the form that will be useful to create the request for inp in el.xpath('.//input | .//select | .//textarea'): # Step 1: Ignore some elements try: name = inp.attrib['name'] except KeyError: continue # Ignore checkboxes and radios that are not selected # as they are just not present in the request instead of being empty # values. try: if inp.attrib['type'] in ('checkbox', 'radio') and 'checked' not in inp.attrib: continue except KeyError: pass # Either filter the submit buttons, or count how many we have found try: if inp.attrib['type'] == 'submit': # If we chose a submit button, ignore all others if self.submit_el is not None and inp is not self.submit_el: continue else: # Register that we have found a submit button, and that it will # be used submits += 1 except KeyError: pass # Step 2: Extract the key-value pair from the remaining elements if inp.tag == 'select': options = inp.xpath('.//option[@selected]') if len(options) == 0: options = inp.xpath('.//option') if len(options) == 0: value = u'' else: value = options[0].attrib.get('value', options[0].text or u'') else: value = inp.attrib.get('value', inp.text or u'') # TODO check if value already exists, emit warning self[name] = value # Sanity checks if submits > 1: warnings.warn('Form has more than one submit input, you should chose the correct one', FormSubmitWarning, stacklevel=3) if self.submit_el is not None and self.submit_el is not False and submits == 0: warnings.warn('Form had a submit element provided, but it was not found', FormSubmitWarning, stacklevel=3) @property def request(self): """ Get the Request object from the form. """ if self.req is None: if self.method.lower() == 'get': self.req = requests.Request(self.method, self.url, params=self) else: self.req = requests.Request(self.method, self.url, data=self) self.req.headers.setdefault('Referer', self.page.url) return self.req def submit(self, **kwargs): """ Submit the form and tell browser to be located to the new page. """ kwargs.setdefault('data_encoding', self.page.encoding) return self.page.browser.location(self.request, **kwargs) class CsvPage(Page): """ Page which parses CSV files. """ DIALECT = 'excel' """ Dialect given to the :mod:`csv` module. """ FMTPARAMS = {} """ Parameters given to the :mod:`csv` module. """ ENCODING = 'utf-8' """ Encoding of the file. """ NEWLINES_HACK = True """ Convert all strange newlines to unix ones. """ HEADER = None """ If not None, will consider the line represented by this index as a header. This means the rows will be also available as dictionaries. """ def build_doc(self, content): # We may need to temporarily convert content to utf-8 because csv # does not support Unicode. encoding = self.encoding if encoding == 'utf-16le': # If there is a BOM, decode('utf-16') will get rid of it content = content.decode('utf-16').encode('utf-8') encoding = 'utf-8' if self.NEWLINES_HACK: content = content.replace('\r\n', '\n').replace('\r', '\n') return self.parse(BytesIO(content), encoding) def parse(self, data, encoding=None): """ Method called by the constructor of :class:`CsvPage` to parse the document. :param data: file stream :type data: :class:`BytesIO` :param encoding: if given, use it to decode cell strings :type encoding: :class:`str` """ import csv reader = csv.reader(data, dialect=self.DIALECT, **self.FMTPARAMS) header = None drows = [] rows = [] for i, row in enumerate(reader): if self.HEADER and i+1 < self.HEADER: continue row = map(unicode.strip, self.decode_row(row, encoding)) if header is None and self.HEADER: header = row else: rows.append(row) if header: drow = {} for i, cell in enumerate(row): drow[header[i]] = cell drows.append(drow) return drows if header is not None else rows def decode_row(self, row, encoding): """ Method called by :meth:`CsvPage.parse` to decode a row using the given encoding. """ if encoding: return [unicode(cell, encoding) for cell in row] else: return row class JsonPage(Page): """ Json Page. """ @property def data(self): return self.response.text def get(self, path): node = self.doc for name in filter(None, path.strip('.').split('.')): node = node.get(name) if node is None: break return node def path(self, path, context=None): from weboob.tools.json import mini_jsonpath return mini_jsonpath(context or self.doc, path) def build_doc(self, text): from weboob.tools.json import json return json.loads(text) class XLSPage(Page): """ XLS Page. """ HEADER = None """ If not None, will consider the line represented by this index as a header. """ SHEET_INDEX = 0 """ Specify the index of the worksheet to use. """ def build_doc(self, content): return self.parse(content) def parse(self, data): """ Method called by the constructor of :class:`XLSPage` to parse the document. """ import xlrd wb = xlrd.open_workbook(file_contents=data) sh = wb.sheet_by_index(self.SHEET_INDEX) header = None drows = [] rows = [] for i in range(sh.nrows - 1): if self.HEADER and i + 1 < self.HEADER: continue row = sh.row_values(i) if header is None and self.HEADER: header = map(lambda s: s.replace('/', ''), row) else: rows.append(row) if header: drow = {} for i, cell in enumerate(sh.row_values(i)): drow[header[i]] = cell drows.append(drow) return drows if header is not None else rows class XMLPage(Page): """ XML Page. """ def detect_encoding(self): import re m = re.search('<\?xml version="1.0" encoding="(.*)"\?>', self.data) if m: return m.group(1) def build_doc(self, content): import lxml.etree as etree parser = etree.XMLParser(encoding=self.encoding) return etree.parse(BytesIO(content), parser) class RawPage(Page): """ Raw page where the "doc" attribute is the content string. """ def build_doc(self, content): return content class HTMLPage(Page): """ HTML page. :param browser: browser used to go on the page :type browser: :class:`weboob.browser.browsers.Browser` :param response: response object :type response: :class:`Response` :param params: optional dictionary containing parameters given to the page (see :class:`weboob.browser.url.URL`) :type params: :class:`dict` :param encoding: optional parameter to force the encoding of the page :type encoding: :class:`basestring` """ FORM_CLASS = Form """ The class to instanciate when using :meth:`HTMLPage.get_form`. Default to :class:`Form`. """ REFRESH_MAX = None """ When handling a "Refresh" meta header, the page considers it only if the sleep time in lesser than this value. Default value is None, means refreshes aren't handled. """ def __init__(self, *args, **kwargs): import lxml.html as html ns = html.etree.FunctionNamespace(None) self.define_xpath_functions(ns) super(HTMLPage, self).__init__(*args, **kwargs) def on_load(self): # Default on_load handle "Refresh" meta tag. self.handle_refresh() def handle_refresh(self): if self.REFRESH_MAX is None: return for refresh in self.doc.xpath('//head/meta[@http-equiv="Refresh"]'): m = self.browser.REFRESH_RE.match(refresh.get('content', '')) if not m: continue url = urlparse.urljoin(self.url, m.groupdict().get('url', None)) sleep = float(m.groupdict()['sleep']) if sleep <= self.REFRESH_MAX: self.logger.info('Redirecting to %s', url) self.browser.location(url) break else: self.logger.debug('Do not refresh to %s because %s > REFRESH_MAX(%s)' % (url, sleep, self.REFRESH_MAX)) def define_xpath_functions(self, ns): """ Define XPath functions on the given lxml function namespace. This method is called in constructor of :class:`HTMLPage` and can be overloaded by children classes to add extra functions. """ ns['lower-case'] = lambda context, args: ' '.join([s.lower() for s in args]) ns['replace'] = lambda context, args, old, new: ' '.join([s.replace(old, new) for s in args]) def has_class(context, *classes): """ This lxml extension allows to select by CSS class more easily >>> ns = html.etree.FunctionNamespace(None) >>> ns['has-class'] = has_class >>> root = html.etree.fromstring(''' ...
... I ... LOVE ... CSS ... ... ''') >>> len(root.xpath('//b[has-class("text")]')) 3 >>> len(root.xpath('//b[has-class("one")]')) 1 >>> len(root.xpath('//b[has-class("text", "first")]')) 1 >>> len(root.xpath('//b[not(has-class("first"))]')) 2 >>> len(root.xpath('//b[has-class("not-exists")]')) 0 """ expressions = ' and '.join(["contains(concat(' ', normalize-space(@class), ' '), ' {0} ')".format(c) for c in classes]) xpath = 'self::*[@class and {0}]'.format(expressions) return bool(context.context_node.xpath(xpath)) def starts_with(context, text, prefix): if not isinstance(text, list): text = [text] return any(t.startswith(prefix) for t in text) def ends_with(context, text, suffix): if not isinstance(text, list): text = [text] return any(t.endswith(suffix) for t in text) ns['has-class'] = has_class ns['starts-with'] = starts_with ns['ends-with'] = ends_with def build_doc(self, content): """ Method to build the lxml document from response and given encoding. """ encoding = self.encoding if encoding == 'latin-1': encoding = 'latin1' if encoding: encoding = encoding.replace('ISO8859_', 'ISO8859-') import lxml.html as html parser = html.HTMLParser(encoding=encoding) return html.parse(BytesIO(content), parser) def detect_encoding(self): """ Look for encoding in the document "http-equiv" and "charset" meta nodes. """ encoding = self.encoding for content in self.doc.xpath('//head/meta[lower-case(@http-equiv)="content-type"]/@content'): # meta http-equiv=content-type content=... _, params = parse_header(content) if 'charset' in params: encoding = params['charset'].strip("'\"") for charset in self.doc.xpath('//head/meta[@charset]/@charset'): # meta charset=... encoding = charset.lower() if encoding == 'iso-8859-1' or not encoding: encoding = 'windows-1252' try: codecs.lookup(encoding) except LookupError: encoding = 'windows-1252' return encoding def get_form(self, xpath='//form', name=None, id=None, nr=None, submit=None): """ Get a :class:`Form` object from a selector. The form will be analyzed and its parameters extracted. In the case there is more than one "submit" input, only one of them should be chosen to generate the request. :param xpath: xpath string to select forms :type xpath: :class:`str` :param name: if supplied, select a form with the given name :type name: :class:`str` :param nr: if supplied, take the n+1 th selected form :type nr: :class:`int` :param submit: if supplied, xpath string to select the submit \ element from the form :type submit: :class:`str` :rtype: :class:`Form` :raises: :class:`FormNotFound` if no form is found """ i = 0 for el in self.doc.xpath(xpath): if name is not None and el.attrib.get('name', '') != name: continue if id is not None and el.attrib.get('id', '') != id: continue if nr is not None and i != nr: i += 1 continue if isinstance(submit, basestring): submit_el = el.xpath(submit)[0] else: submit_el = submit return self.FORM_CLASS(self, el, submit_el) raise FormNotFound() class PartialHTMLPage(HTMLPage): def build_doc(self, content): import lxml.etree try: return super(PartialHTMLPage, self).build_doc(content) except lxml.etree.XMLSyntaxError: content = '%s' % content return super(PartialHTMLPage, self).build_doc(content) class GWTPage(Page): """ GWT page where the "doc" attribute is a list More info about GWT protcol here : https://goo.gl/GP5dv9 """ def build_doc(self, content): """ Reponse starts with "//" followed by "OK" or "EX". 2 last elements in list are protocol and flag. We need to read the list in reversed order. """ assert content[2:4] == "OK" doc, array = [], [] from ast import literal_eval for el in reversed(literal_eval(content[4:])[:-2]): # If we find an array, args after are indices or date if not array and isinstance(el, list): array = el elif array and isinstance(el, int) and len(array) >= el >= 1: doc.append(array[el - 1]) elif array and isinstance(el, basestring): doc.append(self.get_date(el)) return doc def get_date(self, data): """ Get date from string """ base = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_$" timestamp = sum(base.index(data[el]) * (len(base) ** (len(data) - el - 1)) for el in range(len(data))) from datetime import datetime return datetime.fromtimestamp(int(str(timestamp)[:10])).strftime('%d/%m/%Y') def get_elements(self, type="String"): """ Get elements of specified type """ strings = [] for i, el in enumerate(self.doc): if i > 0 and ".%s" % type in self.doc[i - 1]: strings.append(el) return [string for string in strings if "java." not in string] class PDFPage(Page): """ Parse a PDF and write raw data in the "doc" attribute as a string. """ def build_doc(self, content): try: doc = decompress_pdf(content) except OSError as e: raise ParseError(u'Make sure mupdf-tools is installed (%s)' % e) return doc class LoggedPage(object): """ A page that only logged users can reach. If we did not get a redirection for this page, we are sure that the login is still active. Do not use this class for page with mixed content (logged/anonymous) or for pages with a login form. """ logged = True class ChecksumPage(object): """ Compute a checksum of raw content before parsing it. """ import hashlib hashfunc = hashlib.md5 checksum = None def build_doc(self, content): self.checksum = self.hashfunc(content).hexdigest() return super(ChecksumPage, self).build_doc(content) class AbstractPageError(Exception): pass class AbstractPage(Page): PARENT = None PARENT_URL = None BROWSER_ATTR = None def __new__(cls, browser, *args, **kwargs): weboob = getattr(browser, 'weboob', None) if not weboob: raise AbstractPageError("weboob is not defined in %s" % browser) if cls.PARENT is None: raise AbstractPageError("PARENT is not defined for page %s" % cls.__name__) if cls.PARENT_URL is None: raise AbstractPageError("PARENT_URL is not defined for page %s" % cls.__name__) try: parent_module = weboob.load_or_install_module(cls.PARENT) except ModuleInstallError as err: raise ModuleInstallError('This module depends on %s module but %s\'s installation failed with: %s' % (cls.PARENT, cls.PARENT, err)) if cls.BROWSER_ATTR is None: parent_browser = parent_module.klass.BROWSER else: parent_browser = reduce(getattr, cls.BROWSER_ATTR.split('.'), parent_module) parent = parent_browser._urls.get(cls.PARENT_URL, None) if parent is None: raise AbstractPageError("cls.PARENT_URL is not defined in %s" % browser) cls.__bases__ = (parent.klass,) return cls.__new__(cls, browser, *args, **kwargs) weboob-1.2/weboob/browser/profiles.py000066400000000000000000000117261303450110500177650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2014 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . class Profile(object): """ A profile represents the way Browser should act. Usually it is to mimic a real browser. """ def setup_session(self, session): """ Change default headers, set up hooks, etc. Warning: Do not enable lzma, bzip or bzip2, sdch encodings as python-requests does not support it yet. Supported as of 2.2: gzip, deflate, compress. In doubt, do not change the default Accept-Encoding header of python-requests. """ raise NotImplementedError() class Weboob(Profile): """ It's us! Recommended for Weboob-friendly websites only. """ def __init__(self, version): self.version = version def setup_session(self, session): session.headers['User-Agent'] = 'weboob/%s' % self.version class Firefox(Profile): """ Try to mimic a specific version of Firefox. Ideally, it should follow the current ESR Firefox: https://www.mozilla.org/en-US/firefox/organizations/all.html Do not change the Firefox version without checking the Gecko one! """ def setup_session(self, session): """ Set up headers for a standard Firefox request (except for DNT which isn't on by default but is a good idea). The goal is to be unidentifiable. """ # Replace all base requests headers # https://developer.mozilla.org/en/Gecko_user_agent_string_reference # https://bugzilla.mozilla.org/show_bug.cgi?id=572650 session.headers = { 'Accept-Language': 'en-us,en;q=0.5', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0', 'DNT': '1'} class GoogleBot(Profile): """ Try to mimic Googlebot. Keep in mind there are ways to authenticate real Googlebot IPs. You will most likely want to set ALLOW_REFERRER to False. """ def setup_session(self, session): """ Set up headers for a standard Firefox request (except for DNT which isn't on by default but is a good idea). The goal is to be unidentifiable. """ # Replace all base requests headers # http://googlewebmastercentral.blogspot.com/2008/03/first-date-with-googlebot-headers-and.html # Cached versions of: # http://request.urih.com/ # http://xhaus.com/headers session.headers = { 'Accept-Encoding': 'gzip,deflate', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'From': 'googlebot(at)googlebot.com', 'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'} class Wget(Profile): """ Common alternative user agent. Some websites will give you a version with less JavaScript. Some others could ban you (after all, wget is not a real browser). """ def __init__(self, version='1.11.4'): self.version = version def setup_session(self, session): # Don't remove base headers, if websites want to block fake browsers, # they will probably block any wget user agent anyway. session.headers.update({ 'Accept': '*/*', 'User-Agent': 'Wget/%s' % self.version}) class Android(Profile): """ An android profile for mobile websites """ def setup_session(self, session): """ Set up user agent. """ session.headers.update({ 'User-Agent': 'Mozilla/5.0 (Linux; U; Android 4.0.3; fr-fr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}) class IPhone(Profile): """ An iphone profile for mobile websites and some API websites """ def __init__(self, application): self.application = application def setup_session(self, session): session.headers["Accept-Language"] = "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5" session.headers["Accept"] = "*/*" session.headers["User-Agent"] = "%s (iPhone; iOS 7.1; Scale/2.00)" % self.application session.headers["Accept-Encoding"] = "gzip, deflate" weboob-1.2/weboob/browser/sessions.py000066400000000000000000000135531303450110500200100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright(C) 2014 Simon Murail # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . # Inspired by: https://github.com/ross/requests-futures/blob/master/requests_futures/sessions.py # XXX Licence issues? try: from concurrent.futures import ThreadPoolExecutor except ImportError: ThreadPoolExecutor = None from requests import Session from requests.adapters import DEFAULT_POOLSIZE, HTTPAdapter from requests.compat import cookielib, OrderedDict from requests.cookies import cookiejar_from_dict, RequestsCookieJar from requests.models import PreparedRequest from requests.sessions import merge_setting from requests.structures import CaseInsensitiveDict from requests.utils import get_netrc_auth def merge_hooks(request_hooks, session_hooks, dict_class=OrderedDict): """ Properly merges both requests and session hooks. This is necessary because when request_hooks == {'response': []}, the merge breaks Session hooks entirely. Backport from request so we can use it in wheezy """ if session_hooks is None or session_hooks.get('response') == []: return request_hooks if request_hooks is None or request_hooks.get('response') == []: return session_hooks ret = {} for (k, v) in request_hooks.items(): if v is not None: ret[k] = set(v).union(session_hooks.get(k, [])) return ret class WeboobSession(Session): def prepare_request(self, request): """Constructs a :class:`PreparedRequest ` for transmission and returns it. The :class:`PreparedRequest` has settings merged from the :class:`Request ` instance and those of the :class:`Session`. :param request: :class:`Request` instance to prepare with this session's settings. """ cookies = request.cookies or {} # Bootstrap CookieJar. if not isinstance(cookies, cookielib.CookieJar): cookies = cookiejar_from_dict(cookies) # Merge with session cookies merged_cookies = RequestsCookieJar() merged_cookies.update(self.cookies) merged_cookies.update(cookies) # Set environment's basic authentication if not explicitly set. auth = request.auth if self.trust_env and not auth and not self.auth: auth = get_netrc_auth(request.url) p = PreparedRequest() p.prepare( method=request.method.upper(), url=request.url, files=request.files, data=request.data, headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict), params=merge_setting(request.params, self.params), auth=merge_setting(auth, self.auth), cookies=merged_cookies, hooks=merge_hooks(request.hooks, self.hooks), ) return p class FuturesSession(WeboobSession): def __init__(self, executor=None, max_workers=2, max_retries=2, *args, **kwargs): """Creates a FuturesSession Notes ~~~~~ * ProcessPoolExecutor is not supported b/c Response objects are not picklable. * If you provide both `executor` and `max_workers`, the latter is ignored and provided executor is used as is. """ super(FuturesSession, self).__init__(*args, **kwargs) if executor is None and ThreadPoolExecutor is not None: executor = ThreadPoolExecutor(max_workers=max_workers) # set connection pool size equal to max_workers if needed if max_workers > DEFAULT_POOLSIZE: adapter_kwargs = dict(pool_connections=max_workers, pool_maxsize=max_workers, max_retries=max_retries) self.mount('https://', HTTPAdapter(**adapter_kwargs)) self.mount('http://', HTTPAdapter(**adapter_kwargs)) self.executor = executor def send(self, *args, **kwargs): """Maintains the existing api for :meth:`Session.send` Used by :meth:`request` and thus all of the higher level methods If the `is_async` param is True, the request is processed in a thread. Otherwise, the request is processed as usual, in a blocking way. In all cases, it will call the `callback` parameter and return its result when the request has been processed. """ if 'async' in kwargs: import warnings warnings.warn('Please use is_async instead of async.', DeprecationWarning) kwargs['is_async'] = kwargs['async'] del kwargs['async'] sup = super(FuturesSession, self).send callback = kwargs.pop('callback', lambda future, response: response) is_async = kwargs.pop('is_async', False) def func(*args, **kwargs): resp = sup(*args, **kwargs) return callback(self, resp) if is_async: if not self.executor: raise ImportError('Please install python-concurrent.futures') return self.executor.submit(func, *args, **kwargs) return func(*args, **kwargs) def close(self): super(FuturesSession, self).close() if self.executor: self.executor.shutdown() weboob-1.2/weboob/browser/tests/000077500000000000000000000000001303450110500167235ustar00rootroot00000000000000weboob-1.2/weboob/browser/tests/__init__.py000066400000000000000000000000001303450110500210220ustar00rootroot00000000000000weboob-1.2/weboob/browser/tests/filters.py000066400000000000000000000045311303450110500207500ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Matthieu Weber # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from unittest import TestCase from lxml.html import fromstring from weboob.browser.filters.standard import RawText class RawTextTest(TestCase): # Original RawText behaviour: # - the content of

is empty, we return the default value def test_first_node_is_element(self): e = fromstring('

') self.assertEqual("foo", RawText('//p', default="foo")(e)) # - the content of

starts with text, we retrieve only that text def test_first_node_is_text(self): e = fromstring('

blah: 229,90 EUR

') self.assertEqual("blah: ", RawText('//p', default="foo")(e)) # - the content of

starts with a sub-element, we retrieve the default value def test_first_node_has_no_recursion(self): e = fromstring('

229,90 EUR

') self.assertEqual("foo", RawText('//p', default="foo")(e)) # Recursive RawText behaviour # - the content of

starts with text, we retrieve all text, also the text from sub-elements def test_first_node_is_text_recursive(self): e = fromstring('

blah: 229,90 EUR

') self.assertEqual("blah: 229,90 EUR", RawText('//p', default="foo", children=True)(e)) # - the content of

starts with a sub-element, we retrieve all text, also the text from sub-elements def test_first_node_is_element_recursive(self): e = fromstring('

229,90 EUR

') self.assertEqual("229,90 EUR", RawText('//p', default="foo", children=True)(e)) weboob-1.2/weboob/browser/tests/form.py000066400000000000000000000107661303450110500202520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Julia Leven # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import collections import warnings from unittest import TestCase import lxml.html from weboob.browser import URL from weboob.browser.pages import Form, FormSubmitWarning # Mock that allows to represent a Page class MyMockPage(): url = URL("http://httpbin.org") # Class that tests different methods from the class URL class FormTest(TestCase): # Initialization of the objects needed by the tests def setUp(self): self.page = MyMockPage() self.el = lxml.html.fromstring( """
""") self.elMoreSubmit = lxml.html.fromstring( """
""") # Checks that the dictionary is correctly initialised def test_init_nominal_case(self): form = Form(self.page, self.el, None) self.assertDictEqual(form, collections.OrderedDict([ ('nom', 'Dupont'), ('prenom', ''), ('mySelect', 'item2'), ('mySelectNotSelected', 'item1'), ('submitForm', u'')])) # Checks that submit fields are not added to the dictionary when the # attribute submit_el is set to False def test_no_submit(self): formNoSubmit = Form(self.page, self.el, False) self.assertDictEqual(formNoSubmit, collections.OrderedDict([ ('nom', 'Dupont'), ('prenom', ''), ('mySelect', 'item2'), ('mySelectNotSelected', 'item1')])) # Checks that the right warning is issued when there are several submit # fields def test_warning_more_submit(self): with warnings.catch_warnings(record=True) as w: warnings.simplefilter('always') Form(self.page, self.elMoreSubmit) warningMsg = "Form has more than one submit input, you" + \ " should chose the correct one" assert len(w) == 1 assert issubclass(w[-1].category, FormSubmitWarning) assert warningMsg in str(w[-1].message) # Checks that a warning is raised when the submit passed as a parameter # does not exist in the form def test_warning_submit_not_find(self): with warnings.catch_warnings(record=True) as w: warnings.simplefilter('always') Form(self.page, self.el, lxml.html.fromstring( "")) warningMsg = "Form had a submit element provided, but" + \ " it was not found" assert len(w) == 1 assert issubclass(w[-1].category, FormSubmitWarning) assert warningMsg in str(w[-1].message) weboob-1.2/weboob/browser/tests/url.py000066400000000000000000000127101303450110500201000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Julia Leven # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from unittest import TestCase from weboob.browser import PagesBrowser, URL from weboob.browser.pages import Page from weboob.browser.url import UrlNotResolvable class MyMockBrowserWithoutBrowser(): BASEURL = "http://weboob.org" url = URL("http://test.org") # Mock that allows to represent a Page class MyMockPage(Page): pass # Mock that allows to represent a Browser class MyMockBrowser(PagesBrowser): BASEURL = "http://weboob.org" # URL used by method match urlNotRegex = URL("http://test.org", "http://test2.org") urlRegex = URL("http://test.org", "http://weboob2.org") urlRegWithoutHttp = URL("news") urlNotRegWithoutHttp = URL("youtube") # URL used by method build urlValue = URL("http://test.com/(?P\d+)") urlParams = URL("http://test.com\?id=(?P\d+)&name=(?P.+)") urlSameParams = URL("http://test.com/(?P\d+)", "http://test.com\?id=(?P\d+)&name=(?P.+)") # URL used by method is_here urlIsHere = URL('http://weboob.org/(?P)', MyMockPage) urlIsHereDifKlass = URL('http://free.fr', MyMockPage) # Class that tests different methods from the class URL class URLTest(TestCase): # Initialization of the objects needed by the tests def setUp(self): self.myBrowser = MyMockBrowser() self.myBrowserWithoutBrowser = MyMockBrowserWithoutBrowser() # Check that an assert is sent if both base and browser are none def test_match_base_none_browser_none(self): self.assertRaises(AssertionError, self.myBrowserWithoutBrowser.url.match, "http://weboob.org") # Check that no assert is raised when browser is none and a base is indeed # instanciated when given as a parameter def test_match_base_not_none_browser_none(self): try: self.myBrowserWithoutBrowser.url.match("http://weboob.org/news", "http://weboob.org") except AssertionError: self.fail("Method match returns an AssertionError while" + " base parameter is not none!") # Check that none is returned when none of the defined urls is a regex for # the given url def test_match_url_pasregex_baseurl(self): res = self.myBrowser.urlNotRegex.match("http://weboob.org/news") self.assertIsNone(res) # Check that true is returned when one of the defined urls is a regex # for the given url def test_match_url_regex_baseurl(self): res = self.myBrowser.urlRegex.match("http://weboob2.org/news") self.assertTrue(res) # Successful test with relatives url def test_match_url_without_http(self): res = self.myBrowser.urlRegWithoutHttp.match("http://weboob.org/news") self.assertTrue(res) # Unsuccessful test with relatives url def test_match_url_without_http_fail(self): browser = self.myBrowser res = browser.urlNotRegWithoutHttp.match("http://weboob.org/news") self.assertIsNone(res) # Checks that build returns the right url when it needs to add # the value of a parameter def test_build_nominal_case(self): res = self.myBrowser.urlValue.build(id=2) self.assertEquals(res, "http://test.com/2") # Checks that build returns the right url when it needs to add # identifiers and values of some parameters def test_build_urlParams_OK(self): res = self.myBrowser.urlParams.build(id=2, name="weboob") self.assertEquals(res, "http://test.com?id=2&name=weboob") # Checks that build returns the right url when it needs to add # identifiers and values of some parameters. # The same parameters can be in multiple patterns. def test_build_urlSameParams_OK(self): res = self.myBrowser.urlSameParams.build(id=2, name="weboob") self.assertEquals(res, "http://test.com?id=2&name=weboob") # Checks that an exception is raised when a parameter is missing # (here, the parameter name) def test_build_urlParams_KO_missedparams(self): self.assertRaises(UrlNotResolvable, self.myBrowser.urlParams.build, id=2) # Checks that an exception is raised when there is an extra parameter # added to the build function (here, the parameter title) def test_build_urlParams_KO_moreparams(self): self.assertRaises(UrlNotResolvable, self.myBrowser.urlParams.build, id=2, name="weboob", title="test") # Check that an assert is sent if both klass is none def test_ishere_klass_none(self): self.assertRaisesRegexp(AssertionError, "You can use this method" + " only if there is a Page class handler.", self.myBrowser.urlRegex.is_here, id=2) weboob-1.2/weboob/browser/url.py000066400000000000000000000156171303450110500167470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . try: from urllib.parse import unquote except ImportError: from urllib import unquote import re import requests from weboob.tools.regex_helper import normalize class UrlNotResolvable(Exception): """ Raised when trying to locate on an URL instance which url pattern is not resolvable as a real url. """ class URL(object): """ A description of an URL on the PagesBrowser website. It takes one or several regexps to match urls, and an optional Page class which is instancied by PagesBrowser.open if the page matches a regex. """ _creation_counter = 0 def __init__(self, *args): self.urls = [] self.klass = None self.browser = None for arg in args: if isinstance(arg, basestring): self.urls.append(arg) if isinstance(arg, type): self.klass = arg self._creation_counter = URL._creation_counter URL._creation_counter += 1 def is_here(self, **kwargs): """ Returns True if the current page of browser matches this URL. If arguments are provided, and only then, they are checked against the arguments that were used to build the current page URL. """ assert self.klass is not None, "You can use this method only if there is a Page class handler." if len(kwargs): params = self.match(self.build(**kwargs)).groupdict() else: params = None # XXX use unquote on current params values because if there are spaces # or special characters in them, it is encoded only in but not in kwargs. return self.browser.page and isinstance(self.browser.page, self.klass) \ and (params is None or params == dict([(k,unquote(v)) for k,v in self.browser.page.params.iteritems()])) def stay_or_go(self, **kwargs): """ Request to go on this url only if we aren't already here. Arguments are optional parameters for url. >>> url = URL('http://exawple.org/(?P).html') >>> url.stay_or_go(pagename='index') """ if self.is_here(**kwargs): return self.browser.page return self.go(**kwargs) def go(self, params=None, data=None, method=None, headers=None, **kwargs): """ Request to go on this url. Arguments are optional parameters for url. >>> url = URL('http://exawple.org/(?P).html') >>> url.stay_or_go(pagename='index') """ r = self.browser.location(self.build(**kwargs), params=params, data=data, method=method, headers=headers or {}) return r.page or r def open(self, params=None, data=None, **kwargs): """ Request to open on this url. Arguments are optional parameters for url. :param data: POST data :type url: str or dict or None >>> url = URL('http://exawple.org/(?P).html') >>> url.open(pagename='index') """ r = self.browser.open(self.build(**kwargs), params=params, data=data) return r.page or r def build(self, **kwargs): """ Build an url with the given arguments from URL's regexps. :param param: Query string parameters :rtype: :class:`str` :raises: :class:`UrlNotResolvable` if unable to resolve a correct url with the given arguments. """ browser = kwargs.pop('browser', self.browser) params = kwargs.pop('params', None) patterns = [] for url in self.urls: patterns += normalize(url) for pattern, _ in patterns: url = pattern # only use full-name substitutions, to allow % in URLs args = kwargs.copy() for key in args.keys(): # need to use keys() because of pop() search = '%%(%s)s' % key if search in pattern: url = url.replace(search, unicode(args.pop(key))) # if there are named substitutions left, ignore pattern if re.search('%\([A-z_]+\)s', url): continue # if not all args were used if len(args): continue url = browser.absurl(url, base=True) if params: p = requests.models.PreparedRequest() p.prepare_url(url, params) url = p.url return url raise UrlNotResolvable('Unable to resolve URL with %r. Available are %s' % (kwargs, ', '.join([pattern for pattern, _ in patterns]))) def match(self, url, base=None): """ Check if the given url match this object. """ if base is None: assert self.browser is not None base = self.browser.BASEURL for regex in self.urls: if not re.match(r'^[\w\?]+://.*', regex): regex = re.escape(base).rstrip('/') + '/' + regex.lstrip('/') m = re.match(regex, url) if m: return m def handle(self, response): """ Handle a HTTP response to get an instance of the klass if it matches. """ if self.klass is None: return if response.request.method == 'HEAD': return m = self.match(response.url) if m: page = self.klass(self.browser, response, m.groupdict()) if hasattr(page, 'is_here'): if callable(page.is_here): if page.is_here(): return page else: assert isinstance(page.is_here, basestring) if page.doc.xpath(page.is_here): return page else: return page def id2url(self, func): r""" Helper decorator to get an URL if the given first parameter is an ID. """ def inner(browser, id_or_url, *args, **kwargs): if re.match('^https?://.*', id_or_url): if not self.match(id_or_url, browser.BASEURL): return else: id_or_url = self.build(id=id_or_url, browser=browser) return func(browser, id_or_url, *args, **kwargs) return inner weboob-1.2/weboob/capabilities/000077500000000000000000000000001303450110500165275ustar00rootroot00000000000000weboob-1.2/weboob/capabilities/__init__.py000066400000000000000000000002701303450110500206370ustar00rootroot00000000000000# -*- coding: utf-8 -*- from .base import UserError, NotLoaded, NotAvailable, BaseObject, Capability __all__ = ['UserError', 'NotLoaded', 'NotAvailable', 'BaseObject', 'Capability'] weboob-1.2/weboob/capabilities/account.py000066400000000000000000000064411303450110500205420ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, IntField, Field, UserError __all__ = ['AccountRegisterError', 'Account', 'StatusField', 'CapAccount'] class AccountRegisterError(UserError): """ Raised when there is an error during registration. """ class Account(BaseObject): """ Describe an account and its properties. """ login = StringField('Login') password = StringField('Password') properties = Field('List of key/value properties', dict) def __init__(self, id=None, url=None): BaseObject.__init__(self, id, url) class StatusField(BaseObject): """ Field of an account staeobjectus. """ FIELD_TEXT = 0x001 # the value is a long text FIELD_HTML = 0x002 # the value is HTML formated key = StringField('Key') label = StringField('Label') value = StringField('Value') flags = IntField('Flags') def __init__(self, key, label, value, flags=0, url=None): super(StatusField, self).__init__(key, url) self.key = key self.label = label self.value = value self.flags = flags class CapAccount(Capability): """ Capability for websites when you can create and manage accounts. :var ACCOUNT_REGISTER_PROPERTIES: This class constant may be a list of :class:`weboob.tools.value.Value` objects. If the value remains None, weboob considers that :func:`register_account` isn't supported. """ ACCOUNT_REGISTER_PROPERTIES = None @staticmethod def register_account(account): """ Register an account on website This is a static method, it would be called even if the backend is instancied. :param account: describe the account to create :type account: :class:`Account` :raises: :class:`AccountRegisterError` """ raise NotImplementedError() def confirm_account(self, mail): """ From an email go to the confirm link. """ raise NotImplementedError() def get_account(self): """ Get the current account. """ raise NotImplementedError() def update_account(self, account): """ Update the current account. """ raise NotImplementedError() def get_account_status(self): """ Get status of the current account. :returns: a list of fields """ raise NotImplementedError() weboob-1.2/weboob/capabilities/audio.py000066400000000000000000000110471303450110500202050ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from datetime import timedelta from .image import Thumbnail from .base import Field, StringField, IntField, BaseObject from .file import CapFile, BaseFile __all__ = ['BaseAudio', 'CapAudio'] def decode_id(decode_id): def wrapper(func): def inner(self, *args, **kwargs): arg = unicode(args[0]) _id = decode_id(arg) if _id is None: return None new_args = [_id] new_args.extend(args[1:]) return func(self, *new_args, **kwargs) return inner return wrapper class Album(BaseObject): """ Represent an album """ title = StringField('album name') author = StringField('artist name') year = IntField('release year') thumbnail = Field('Image associated to the album', Thumbnail) tracks_list = Field('list of tracks', list) @classmethod def decode_id(cls, _id): if _id: m = re.match('^(album)\.(.*)', _id) if m: return m.group(2) return _id class Playlist(BaseObject): """ Represent a playlist """ title = StringField('playlist name') tracks_list = Field('list of tracks', list) @classmethod def decode_id(cls, _id): if _id: m = re.match('^(playlist)\.(.*)', _id) if m: return m.group(2) return _id class BaseAudio(BaseFile): """ Represent an audio file """ duration = Field('file duration', int, long, timedelta) bitrate = Field('file bit rate in Kbps', int) format = StringField('file format') thumbnail = Field('Image associated to the file', Thumbnail) @classmethod def decode_id(cls, _id): if _id: m = re.match('^(audio)\.(.*)', _id) if m: return m.group(2) return _id class CapAudio(CapFile): """ Audio file provider """ @classmethod def get_object_method(cls, _id): m = re.match('^(\w+)\.(.*)', _id) if m: if m.group(1) == 'album': return 'get_album' elif m.group(1) == 'playlist': return 'get_playlist' else: return 'get_audio' def search_audio(self, pattern, sortby=CapFile.SEARCH_RELEVANCE): """ search for a audio file :param pattern: pattern to search on :type pattern: str :param sortby: sort by ...(use SEARCH_* constants) :rtype: iter[:class:`BaseAudio`] """ return self.search_file(pattern, sortby) def search_album(self, pattern, sortby=CapFile.SEARCH_RELEVANCE): """ search for an album :param pattern: pattern to search on :type pattern: str :rtype: iter[:class:`Album`] """ raise NotImplementedError() def search_playlist(self, pattern, sortby=CapFile.SEARCH_RELEVANCE): """ search for an album :param pattern: pattern to search on :type pattern: str :rtype: iter[:class:`Playlist`] """ raise NotImplementedError() @decode_id(BaseAudio.decode_id) def get_audio(self, _id): """ Get an audio file from an ID. :param id: audio file ID :type id: str :rtype: :class:`BaseAudio`] """ return self.get_file(_id) @decode_id(Playlist.decode_id) def get_playlist(self, _id): """ Get a playlist from an ID. :param id: playlist ID :type id: str :rtype: :class:`Playlist`] """ raise NotImplementedError() @decode_id(Album.decode_id) def get_album(self, _id): """ Get an album from an ID. :param id: album ID :type id: str :rtype: :class:`Album`] """ raise NotImplementedError() weboob-1.2/weboob/capabilities/audiostream.py000066400000000000000000000035521303450110500214230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.capabilities.streaminfo import StreamInfo from .base import Field from .file import CapFile from .audio import CapAudio, BaseAudio __all__ = ['BaseAudioStream', 'CapAudioStream'] class BaseAudioStream(BaseAudio): """ Audio stream object """ current = Field('Information related to current broadcast', StreamInfo) def __unicode__(self): return u'%s (%s)' % (self.title, self.url) def __repr__(self): return '%r (%r)' % (self.title, self.url) class CapAudioStream(CapAudio): """ Audio streams provider """ def search_audiostreams(self, pattern, sortby=CapFile.SEARCH_RELEVANCE): """ Search an audio stream :param pattern: pattern to search :type pattern: str :param sortby: sort by ... (use SEARCH_* constants) :rtype: iter[:class:`BaseAudioStream`] """ return self.search_audio(pattern, sortby) def get_audiostream(self, _id): """ Get an audio stream :param _id: Audiostream ID :type id: str :rtype: :class:`BaseAudioStream` """ return self.get_audio(_id) weboob-1.2/weboob/capabilities/bank.py000066400000000000000000000326561303450110500200300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import date, datetime from binascii import crc32 import re from weboob.capabilities.base import empty from weboob.exceptions import BrowserQuestion from .base import BaseObject, Field, StringField, DecimalField, IntField, \ UserError, Currency, NotAvailable from .date import DateField from .collection import CapCollection __all__ = ['AccountNotFound', 'TransferError', 'TransferStep', 'Recipient', 'Account', 'Transaction', 'Investment', 'Transfer', 'CapBank', 'RecipientNotFound'] class AccountNotFound(UserError): """ Raised when an account is not found. """ def __init__(self, msg='Account not found'): UserError.__init__(self, msg) class RecipientNotFound(UserError): """ Raised when a recipient is not found. """ def __init__(self, msg='Recipient not found'): UserError.__init__(self, msg) class TransferError(UserError): """ A transfer has failed. """ class BaseAccount(BaseObject, Currency): """ Generic class aiming to be parent of :class:`Recipient` and :class:`Account`. """ label = StringField('Pretty label') currency = StringField('Currency', default=None) iban = StringField('International Bank Account Number') bank_name = StringField('Bank Name') def __init__(self, id='0', url=None): BaseObject.__init__(self, id, url) @property def currency_text(self): return Currency.currency2txt(self.currency) @property def ban(self): """ Bank Account Number part of IBAN""" if not self.iban: return NotAvailable return self.iban[4:] class Recipient(BaseAccount): """ Recipient of a transfer. """ enabled_at = DateField('Date of availability') category = StringField('Recipient category') class Account(BaseAccount): """ Bank account. """ TYPE_UNKNOWN = 0 TYPE_CHECKING = 1 "Transaction, everyday transactions" TYPE_SAVINGS = 2 "Savings/Deposit, can be used for every banking" TYPE_DEPOSIT = 3 "Term of Fixed Deposit, has time/amount constraints" TYPE_LOAN = 4 "Loan account" TYPE_MARKET = 5 "Stock market or other variable investments" TYPE_JOINT = 6 "Joint account" TYPE_CARD = 7 "Card account" TYPE_LIFE_INSURANCE = 8 "Life insurances" TYPE_PEE = 9 "Employee savings PEE" TYPE_PERCO = 10 "Employee savings PERCO" TYPE_ARTICLE_83 = 11 "Article 83" TYPE_RSP = 12 "Employee savings RSP" TYPE_PEA = 13 "Share savings" TYPE_CAPITALISATION = 14 "Life Insurance capitalisation" TYPE_PERP = 15 "Retirement savings" TYPE_MADELIN = 16 "Complementary retirement savings" type = IntField('Type of account', default=TYPE_UNKNOWN) balance = DecimalField('Balance on this bank account') coming = DecimalField('Sum of coming movements') # card attributes paydate = DateField('For credit cards. When next payment is due.') paymin = DecimalField('For credit cards. Minimal payment due.') cardlimit = DecimalField('For credit cards. Credit limit.') number = StringField('Shown by the bank to identify your account ie XXXXX7489') # market and lifeinssurance accounts valuation_diff = DecimalField('+/- values total') def __repr__(self): return "" % (self.id, self.label) class Transaction(BaseObject): """ Bank transaction. """ TYPE_UNKNOWN = 0 TYPE_TRANSFER = 1 TYPE_ORDER = 2 TYPE_CHECK = 3 TYPE_DEPOSIT = 4 TYPE_PAYBACK = 5 TYPE_WITHDRAWAL = 6 TYPE_CARD = 7 TYPE_LOAN_PAYMENT = 8 TYPE_BANK = 9 TYPE_CASH_DEPOSIT = 10 TYPE_CARD_SUMMARY = 11 TYPE_DEFERRED_CARD = 12 date = DateField('Debit date on the bank statement') rdate = DateField('Real date, when the payment has been made; usually extracted from the label or from credit card info') vdate = DateField('Value date, or accounting date; usually for professional accounts') type = IntField('Type of transaction, use TYPE_* constants', default=TYPE_UNKNOWN) raw = StringField('Raw label of the transaction') category = StringField('Category of the transaction') label = StringField('Pretty label') amount = DecimalField('Amount of the transaction') card = StringField('Card number (if any)') commission = DecimalField('Commission part on the transaction (in account currency)') # International original_amount = DecimalField('Original amount (in another currency)') original_currency = StringField('Currency of the original amount') country = StringField('Country of transaction') # Financial arbitrations investments = Field('List of investments related to the transaction', list, default=[]) def __repr__(self): return "" % (self.date, self.label, self.amount) def unique_id(self, seen=None, account_id=None): """ Get an unique ID for the transaction based on date, amount and raw. :param seen: if given, the method uses this dictionary as a cache to prevent several transactions with the same values to have the same unique ID. :type seen: :class:`dict` :param account_id: if given, add the account ID in data used to create the unique ID. Can be useful if you want your ID to be unique across several accounts. :type account_id: :class:`str` :returns: an unique ID encoded in 8 length hexadecimal string (for example ``'a64e1bc9'``) :rtype: :class:`str` """ crc = crc32(str(self.date)) crc = crc32(str(self.amount), crc) if not empty(self.raw): label = self.raw else: label = self.label crc = crc32(re.sub('[ ]+', ' ', label.encode("utf-8")), crc) if account_id is not None: crc = crc32(str(account_id), crc) if seen is not None: while crc in seen: crc = crc32("*", crc) seen.add(crc) return "%08x" % (crc & 0xffffffff) class Investment(BaseObject): """ Investment in a financial market. """ label = StringField('Label of stocks') code = StringField('Identifier of the stock (ISIN code)') description = StringField('Short description of the stock') quantity = DecimalField('Quantity of stocks') unitprice = DecimalField('Buy price of one stock') unitvalue = DecimalField('Current value of one stock') valuation = DecimalField('Total current valuation of the Investment') vdate = DateField('Value date of the valuation amount') diff = DecimalField('Difference between the buy cost and the current valuation') diff_percent = DecimalField('Difference in percentage between the buy cost and the current valuation') portfolio_share = DecimalField('Percentage of the current amount relative to the total') # International original_currency = StringField('Currency of the original amount') original_valuation = DecimalField('Original valuation (in another currency)') original_unitvalue = DecimalField('Original unitvalue (in another currency)') original_unitprice = DecimalField('Original unitprice (in another currency)') original_diff = DecimalField('Original diff (in another currency)') class TransferStep(BrowserQuestion): def __init__(self, transfer, *values): super(TransferStep, self).__init__(*values) self.transfer = transfer class Transfer(BaseObject, Currency): """ Transfer from an account to a recipient. """ amount = DecimalField('Amount to transfer') currency = StringField('Currency', default=None) fees = DecimalField('Fees', default=None) exec_date = Field('Date of transfer', date, datetime) account_id = StringField('ID of origin account') account_iban = StringField('International Bank Account Number') account_label = StringField('Label of origin account') account_balance = DecimalField('Balance of origin account before transfer') recipient_id = StringField('ID of recipient account') recipient_iban = StringField('International Bank Account Number') recipient_label = StringField('Label of recipient account') label = StringField('Reason') class CapBank(CapCollection): """ Capability of bank websites to see accounts and transactions. """ def iter_resources(self, objs, split_path): """ Iter resources. Default implementation of this method is to return on top-level all accounts (by calling :func:`iter_accounts`). :param objs: type of objects to get :type objs: tuple[:class:`BaseObject`] :param split_path: path to discover :type split_path: :class:`list` :rtype: iter[:class:`BaseObject`] """ if Account in objs: self._restrict_level(split_path) return self.iter_accounts() def iter_accounts(self): """ Iter accounts. :rtype: iter[:class:`Account`] """ raise NotImplementedError() def get_account(self, id): """ Get an account from its ID. :param id: ID of the account :type id: :class:`str` :rtype: :class:`Account` :raises: :class:`AccountNotFound` """ raise NotImplementedError() def iter_history(self, account): """ Iter history of transactions on a specific account. :param account: account to get history :type account: :class:`Account` :rtype: iter[:class:`Transaction`] :raises: :class:`AccountNotFound` """ raise NotImplementedError() def iter_coming(self, account): """ Iter coming transactions on a specific account. :param account: account to get coming transactions :type account: :class:`Account` :rtype: iter[:class:`Transaction`] :raises: :class:`AccountNotFound` """ raise NotImplementedError() def iter_investment(self, account): """ Iter investment of a market account :param account: account to get investments :type account: :class:`Account` :rtype: iter[:class:`Investment`] :raises: :class:`AccountNotFound` """ raise NotImplementedError() class CapCgp(CapBank): """ Capability of cgp website to see accounts and transactions. """ class CapBankTransfer(CapBank): def iter_transfer_recipients(self, account): """ Iter recipients availables for a transfer from a specific account. :param account: account which initiate the transfer :type account: :class:`Account` :rtype: iter[:class:`Recipient`] :raises: :class:`AccountNotFound` """ raise NotImplementedError() def init_transfer(self, transfer, **params): """ Initiate a transfer. :param :class:`Transfer` :rtype: :class:`Transfer` :raises: :class:`TransferError` """ raise NotImplementedError() def execute_transfer(self, transfer, **params): """ Execute a transfer. :param :class:`Transfer` :rtype: :class:`Transfer` :raises: :class:`TransferError` """ raise NotImplementedError() def transfer(self, transfer, **params): """ Do a transfer from an account to a recipient. :param :class:`Transfer` :rtype: :class:`Transfer` :raises: :class:`TransferError` """ t = self.init_transfer(transfer, **params) for key, value in t.iter_fields(): if hasattr(transfer, key) and key != 'id': transfer_val = getattr(transfer, key) try: if hasattr(self, 'transfer_check_%s' % key): assert getattr(self, 'transfer_check_%s' % key)(transfer_val, value) else: assert transfer_val == value or empty(transfer_val) except AssertionError: raise TransferError('%s changed during transfer processing (from %s to %s)' % (key, transfer_val, value)) return self.execute_transfer(t, **params) weboob-1.2/weboob/capabilities/base.py000066400000000000000000000410721303450110500200170ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz, Romain Bignon, Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import warnings import re from decimal import Decimal from copy import deepcopy, copy from weboob.tools.compat import unicode, long from weboob.tools.misc import to_unicode from weboob.tools.ordereddict import OrderedDict __all__ = ['UserError', 'FieldNotFound', 'NotAvailable', 'NotLoaded', 'Capability', 'Field', 'IntField', 'DecimalField', 'FloatField', 'StringField', 'BytesField', 'BoolField', 'empty', 'BaseObject'] def enum(**enums): _values = enums.values() _keys = enums.keys() _items = enums.items() _types = list((type(value) for value in enums.values())) _index = dict((value if not isinstance(value, dict) else value.itervalues().next(), i) for i, value in enumerate(enums.values())) enums['keys'] = _keys enums['values'] = _values enums['items'] = _items enums['index'] = _index enums['types'] = _types return type('Enum', (), enums) def empty(value): """ Checks if a value is empty (None, NotLoaded or NotAvailable). :rtype: :class:`bool` """ for cls in (None, NotLoaded, NotAvailable): if value is cls: return True return False def find_object(mylist, error=None, **kwargs): """ Very simple tools to return an object with the matching parameters in kwargs. """ for a in mylist: found = True for key, value in kwargs.iteritems(): if getattr(a, key) != value: found = False break if found: return a if error is not None: raise error() return None class UserError(Exception): """ Exception containing an error message for user. """ class FieldNotFound(Exception): """ A field isn't found. :param obj: object :type obj: :class:`BaseObject` :param field: field not found :type field: :class:`Field` """ def __init__(self, obj, field): Exception.__init__(self, u'Field "%s" not found for object %s' % (field, obj)) class ConversionWarning(UserWarning): """ A field's type was changed when setting it. Ideally, the module should use the right type before setting it. """ pass class AttributeCreationWarning(UserWarning): """ A non-field attribute has been created with a name not prefixed with a _. """ class NotAvailableType(object): """ NotAvailable is a constant to use on non available fields. """ def __str__(self): return unicode(self).encode('utf-8') def __copy__(self): return self def __deepcopy__(self, memo): return self def __repr__(self): return 'NotAvailable' def __unicode__(self): return u'Not available' def __nonzero__(self): return False NotAvailable = NotAvailableType() class NotLoadedType(object): """ NotLoaded is a constant to use on not loaded fields. When you use :func:`weboob.tools.backend.Module.fillobj` on a object based on :class:`BaseObject`, it will request all fields with this value. """ def __str__(self): return unicode(self).encode('utf-8') def __copy__(self): return self def __deepcopy__(self, memo): return self def __repr__(self): return 'NotLoaded' def __unicode__(self): return u'Not loaded' def __nonzero__(self): return False NotLoaded = NotLoadedType() class Capability(object): """ This is the base class for all capabilities. A capability may define abstract methods (which raise :class:`NotImplementedError`) with an explicit docstring to tell backends how to implement them. Also, it may define some *objects*, using :class:`BaseObject`. """ class Field(object): """ Field of a :class:`BaseObject` class. :param doc: docstring of the field :type doc: :class:`str` :param args: list of types accepted :param default: default value of this field. If not specified, :class:`NotLoaded` is used. """ _creation_counter = 0 def __init__(self, doc, *args, **kwargs): self.types = () self.value = kwargs.get('default', NotLoaded) self.doc = doc for arg in args: if isinstance(arg, type) or isinstance(arg, str): self.types += (arg,) else: raise TypeError('Arguments must be types or strings of type name') self._creation_counter = Field._creation_counter Field._creation_counter += 1 def convert(self, value): """ Convert value to the wanted one. """ return value class IntField(Field): """ A field which accepts only :class:`int` and :class:`long` types. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, int, long, **kwargs) def convert(self, value): return int(value) class BoolField(Field): """ A field which accepts only :class:`bool` type. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, bool, **kwargs) def convert(self, value): return bool(value) class DecimalField(Field): """ A field which accepts only :class:`decimal` type. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, Decimal, **kwargs) def convert(self, value): if isinstance(value, Decimal): return value return Decimal(value) class FloatField(Field): """ A field which accepts only :class:`float` type. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, float, **kwargs) def convert(self, value): return float(value) class StringField(Field): """ A field which accepts only :class:`unicode` strings. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, unicode, **kwargs) def convert(self, value): return to_unicode(value) class BytesField(Field): """ A field which accepts only :class:`str` strings. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, str, **kwargs) def convert(self, value): if isinstance(value, unicode): value = value.encode('utf-8') return str(value) class _BaseObjectMeta(type): def __new__(cls, name, bases, attrs): fields = [(field_name, attrs.pop(field_name)) for field_name, obj in attrs.items() if isinstance(obj, Field)] fields.sort(key=lambda x: x[1]._creation_counter) new_class = super(_BaseObjectMeta, cls).__new__(cls, name, bases, attrs) if new_class._fields is None: new_class._fields = OrderedDict() else: new_class._fields = deepcopy(new_class._fields) new_class._fields.update(fields) if new_class.__doc__ is None: new_class.__doc__ = '' for name, field in fields: doc = '(%s) %s' % (', '.join([':class:`%s`' % v.__name__ if isinstance(v, type) else v for v in field.types]), field.doc) if field.value is not NotLoaded: doc += ' (default: %s)' % field.value new_class.__doc__ += '\n:var %s: %s' % (name, doc) return new_class class BaseObject(object): """ This is the base class for a capability object. A capability interface may specify to return several kind of objects, to formalise retrieved information from websites. As python is a flexible language where variables are not typed, we use a system to force backends to set wanted values on all fields. To do that, we use the :class:`Field` class and all derived ones. For example:: class Transfer(BaseObject): " Transfer from an account to a recipient. " amount = DecimalField('Amount to transfer') date = Field('Date of transfer', basestring, date, datetime) origin = Field('Origin of transfer', int, long, basestring) recipient = Field('Recipient', int, long, basestring) The docstring is mandatory. """ __metaclass__ = _BaseObjectMeta id = None backend = None url = StringField('url') _fields = None def __init__(self, id=u'', url=NotLoaded, backend=None): self.id = to_unicode(id) self.backend = backend self._fields = deepcopy(self._fields) self.__setattr__('url', url) @property def fullid(self): """ Full ID of the object, in form '**ID@backend**'. """ return '%s@%s' % (self.id, self.backend) def __iscomplete__(self): """ Return True if the object is completed. It is usefull when the object is a field of an other object which is going to be filled. The default behavior is to iter on fields (with iter_fields) and if a field is NotLoaded, return False. """ for key, value in self.iter_fields(): if value is NotLoaded: return False return True def copy(self): obj = copy(self) obj._fields = copy(self._fields) return obj def __deepcopy__(self, memo): return self.copy() def set_empty_fields(self, value, excepts=()): """ Set the same value on all empty fields. :param value: value to set on all empty fields :param excepts: if specified, do not change fields listed """ for key, old_value in self.iter_fields(): if empty(old_value) and key not in excepts: setattr(self, key, value) def iter_fields(self): """ Iterate on the fields keys and values. Can be overloaded to iterate on other things. :rtype: iter[(key, value)] """ if hasattr(self, 'id') and self.id is not None: yield 'id', self.id for name, field in self._fields.iteritems(): yield name, field.value def __eq__(self, obj): if isinstance(obj, BaseObject): return self.backend == obj.backend and self.id == obj.id else: return False def __getattr__(self, name): if self._fields is not None and name in self._fields: return self._fields[name].value else: raise AttributeError("'%s' object has no attribute '%s'" % ( self.__class__.__name__, name)) def __setattr__(self, name, value): try: attr = (self._fields or {})[name] except KeyError: if name not in dir(self) and not name.startswith('_'): warnings.warn('Creating a non-field attribute %s. Please prefix it with _' % name, AttributeCreationWarning, stacklevel=2) object.__setattr__(self, name, value) else: if not empty(value): try: # Try to convert value to the wanted one. nvalue = attr.convert(value) # If the value was converted if nvalue is not value: warnings.warn('Value %s was converted from %s to %s' % (name, type(value), type(nvalue)), ConversionWarning, stacklevel=2) value = nvalue except Exception: # error during conversion, it will probably not # match the wanted following types, so we'll # raise ValueError. pass from collections import deque actual_types = () for v in attr.types: if isinstance(v, str): # the following is a (almost) copy/paste from # https://stackoverflow.com/questions/11775460/lexical-cast-from-string-to-type q = deque([object]) while q: t = q.popleft() if t.__name__ == v: actual_types += (t,) else: try: # keep looking! q.extend(t.__subclasses__()) except TypeError: # type.__subclasses__ needs an argument for # whatever reason. if t is type: continue else: raise else: actual_types += (v,) if not isinstance(value, actual_types) and not empty(value): raise ValueError( 'Value for "%s" needs to be of type %r, not %r' % ( name, actual_types, type(value))) attr.value = value def __delattr__(self, name): try: self._fields.pop(name) except KeyError: object.__delattr__(self, name) def to_dict(self): def iter_decorate(d): for key, value in d: if key == 'id' and self.backend is not None: value = self.fullid yield key, value fields_iterator = self.iter_fields() return OrderedDict(iter_decorate(fields_iterator)) @classmethod def from_dict(cls, values, backend=None): self = cls() for attr in values: setattr(self, attr, values[attr]) return self class Currency(object): CURRENCIES = {u'EUR': (u'€', u'EURO'), u'CHF': (u'CHF',), u'USD': (u'$',), u'GBP': (u'£',), u'LBP': (u'ل.ل',), u'AED': (u'AED',), u'XOF': (u'XOF',), u'RUB': (u'руб',), u'SGD': (u'SGD',), u'BRL': (u'R$',), u'MXN': (u'$',), u'JPY': (u'¥',), u'TRY': (u'₺', 'TRY'), u'RON': (u'lei',), u'COP': (u'$',), u'NOK': (u'kr',), u'CNY': (u'¥',), u'RSD': (u'din',), u'ZAR': (u'rand',), u'MYR': (u'RM',), u'HUF': (u'Ft',), u'HKD': (u'HK$',), u'QAR': (u'QR',), u'MAD': (u'MAD',), u'ARS': (u'ARS',), u'AUD': (u'AUD',), u'CAD': (u'CAD',), u'NZD': (u'NZD',), u'BHD': (u'BHD',), u'SEK': (u'SEK',), u'DKK': (u'DKK',), u'LUF': (u'F', u'fr.', u'LUF',), u'KZT': (u'KZT',), u'PLN': (u'PLN',), u'ILS': (u'ILS',), u'THB': (u'THB',), } EXTRACTOR = re.compile(r'[\d\s,\.\-]', re.UNICODE) @classmethod def get_currency(klass, text): u""" >>> Currency.get_currency(u'42') None >>> Currency.get_currency(u'42 €') u'EUR' >>> Currency.get_currency(u'$42') u'USD' >>> Currency.get_currency(u'42.000,00€') u'EUR' >>> Currency.get_currency(u'$42 USD') u'USD' >>> Currency.get_currency(u'%42 USD') u'USD' >>> Currency.get_currency(u'US1D') None """ curtexts = klass.EXTRACTOR.sub(' ', text.upper()).split() for curtext in curtexts: for currency, symbols in klass.CURRENCIES.iteritems(): if curtext in currency: return currency for symbol in symbols: if curtext in symbol or symbol in curtext: return currency return None @classmethod def currency2txt(klass, currency): _currency = klass.CURRENCIES.get(currency, (u'',)) return _currency[0] weboob-1.2/weboob/capabilities/bill.py000066400000000000000000000136151303450110500200310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import BaseObject, StringField, DecimalField, BoolField, UserError, Currency from .date import DateField from .collection import CapCollection __all__ = ['SubscriptionNotFound', 'DocumentNotFound', 'Detail', 'Document', 'Bill', 'Subscription', 'CapDocument'] class SubscriptionNotFound(UserError): """ Raised when a subscription is not found. """ def __init__(self, msg='Subscription not found'): UserError.__init__(self, msg) class DocumentNotFound(UserError): """ Raised when a document is not found. """ def __init__(self, msg='Document not found'): UserError.__init__(self, msg) class Detail(BaseObject, Currency): """ Detail of a subscription """ label = StringField('label of the detail line') infos = StringField('information') datetime = DateField('date information') price = DecimalField('Total price, taxes included') vat = DecimalField('Value added Tax') currency = StringField('Currency', default=None) quantity = DecimalField('Number of units consumed') unit = StringField('Unit of the consumption') class Document(BaseObject): """ Document. """ date = DateField('The day the document has been sent to the subscriber') format = StringField('file format of the document') label = StringField('label of document') type = StringField('type of document') class Bill(Document, Currency): """ Bill. """ price = DecimalField('Price to pay') currency = StringField('Currency', default=None) vat = DecimalField('VAT included in the price') duedate = DateField('The day the bill must be paid') startdate = DateField('The first day the bill applies to') finishdate = DateField('The last day the bill applies to') income = BoolField('Boolean to set if bill is income or invoice', default=False) class Subscription(BaseObject): """ Subscription to a service. """ label = StringField('label of subscription') subscriber = StringField('whe has subscribed') validity = DateField('End validity date of the subscription') renewdate = DateField('Reset date of consumption') class CapDocument(CapCollection): def iter_resources(self, objs, split_path): """ Iter resources. Will return :func:`iter_subscriptions`. """ if Subscription in objs: self._restrict_level(split_path) return self.iter_subscription() def iter_subscription(self): """ Iter subscriptions. :rtype: iter[:class:`Subscription`] """ raise NotImplementedError() def get_subscription(self, _id): """ Get a subscription. :param _id: ID of subscription :rtype: :class:`Subscription` :raises: :class:`SubscriptionNotFound` """ raise NotImplementedError() def iter_documents_history(self, subscription): """ Iter history of a subscription. :param subscription: subscription to get history :type subscription: :class:`Subscription` :rtype: iter[:class:`Detail`] """ raise NotImplementedError() def iter_bills_history(self, subscription): """ Iter history of a subscription. :param subscription: subscription to get history :type subscription: :class:`Subscription` :rtype: iter[:class:`Detail`] """ return self.iter_documents_history(subscription) def get_document(self, id): """ Get a document. :param id: ID of document :rtype: :class:`Document` :raises: :class:`DocumentNotFound` """ raise NotImplementedError() def download_document(self, id): """ Download a document. :param id: ID of document :rtype: str :raises: :class:`DocumentNotFound` """ raise NotImplementedError() def iter_documents(self, subscription): """ Iter documents. :param subscription: subscription to get documents :type subscription: :class:`Subscription` :rtype: iter[:class:`Document`] """ raise NotImplementedError() def iter_bills(self, subscription): """ Iter bills. :param subscription: subscription to get bills :type subscription: :class:`Subscription` :rtype: iter[:class:`Bill`] """ documents = self.iter_documents(subscription) return filter(lambda doc: doc.type == "bill", documents) def get_details(self, subscription): """ Get details of a subscription. :param subscription: subscription to get details :type subscription: :class:`Subscription` :rtype: iter[:class:`Detail`] """ raise NotImplementedError() def get_balance(self, subscription): """ Get the balance of a subscription. :param subscription: subscription to get balance :type subscription: :class:`Subscription` :rtype: class:`Detail` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/bugtracker.py000066400000000000000000000207071303450110500212400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField,\ IntField, UserError from .date import DateField, DeltaField __all__ = ['IssueError', 'Project', 'User', 'Version', 'Status', 'Attachment', 'Change', 'Update', 'Issue', 'Query', 'CapBugTracker'] class IssueError(UserError): """ Raised when there is an error with an issue. """ class Project(BaseObject): """ Represents a project. """ name = StringField('Name of the project') members = Field('Members of projects', list) versions = Field('List of versions available for this project', list) trackers = Field('All trackers', list) categories = Field('All categories', list) statuses = Field('Available statuses for issues', list) priorities = Field('Available priorities for issues', list) def __init__(self, id, name, url=None): BaseObject.__init__(self, id, url) self.name = unicode(name) def __repr__(self): return '' % self.name def find_user(self, id, name): """ Find a user from its ID. If not found, create a :class:`User` with the specified name. :param id: ID of user :type id: str :param name: Name of user :type name: str :rtype: :class:`User` """ for user in self.members: if user.id == id: return user if name is None: return None return User(id, name) def find_version(self, id, name): """ Find a version from an ID. If not found, create a :class:`Version` with the specified name. :param id: ID of version :type id: str :param name: Name of version :type name: str :rtype: :class:`Version` """ for version in self.versions: if version.id == id: return version if name is None: return None return Version(id, name) def find_status(self, name): """ Find a status from a name. :param name: Name of status :type name: str :rtype: :class:`Status` """ for status in self.statuses: if status.name == name: return status if name is None: return None return None class User(BaseObject): """ User. """ name = StringField('Name of user') def __init__(self, id, name, url=None): BaseObject.__init__(self, id, url) self.name = unicode(name) def __repr__(self): return '' % self.name class Version(BaseObject): """ Version of a project. """ name = StringField('Name of version') def __init__(self, id, name, url=None): BaseObject.__init__(self, id, url) self.name = unicode(name) def __repr__(self): return '' % self.name class Status(BaseObject): """ Status of an issue. **VALUE_** constants are the primary status types. """ (VALUE_NEW, VALUE_PROGRESS, VALUE_RESOLVED, VALUE_REJECTED) = range(4) name = StringField('Name of status') value = IntField('Value of status (constants VALUE_*)') def __init__(self, id, name, value, url=None): BaseObject.__init__(self, id, url) self.name = unicode(name) self.value = value def __repr__(self): return '' % self.name class Attachment(BaseObject): """ Attachment of an issue. """ filename = StringField('Filename') def __repr__(self): return '' % self.filename class Change(BaseObject): """ A change of an update. """ field = StringField('What field has been changed') last = StringField('Last value of field') new = StringField('New value of field') class Update(BaseObject): """ Represents an update of an issue. """ author = Field('Author of update', User) date = DateField('Date of update') hours = DeltaField('Time activity') message = StringField('Log message') attachments = Field('Files attached to update', list, tuple) changes = Field('List of changes', list, tuple) def __repr__(self): return '' % self.id class Issue(BaseObject): """ Represents an issue. """ project = Field('Project of this issue', Project) title = StringField('Title of issue') body = StringField('Text of issue') creation = DateField('Date when this issue has been created') updated = DateField('Date when this issue has been updated for the last time') start = DateField('Date when this issue starts') due = DateField('Date when this issue is due for') attachments = Field('List of attached files', list, tuple) history = Field('History of updates', list, tuple) author = Field('Author of this issue', User) assignee = Field('User assigned to this issue', User) tracker = StringField('Name of the tracker') category = StringField('Name of the category') version = Field('Target version of this issue', Version) status = Field('Status of this issue', Status) fields = Field('Custom fields (key,value)', dict) priority = StringField('Priority of the issue') #XXX class Query(BaseObject): """ Query to find an issue. """ project = StringField('Filter on projects') title = StringField('Filter on titles') author = StringField('Filter on authors') assignee = StringField('Filter on assignees') version = StringField('Filter on versions') category = StringField('Filter on categories') status = StringField('Filter on statuses') def __init__(self, id='', url=None): BaseObject.__init__(self, id, url) class CapBugTracker(Capability): """ Bug trackers websites. """ def iter_issues(self, query): """ Iter issues with optionnal patterns. :param query: query :type query: :class:`Query` :rtype: iter[:class:`Issue`] """ raise NotImplementedError() def get_issue(self, id): """ Get an issue from its ID. :param id: ID of issue :rtype: :class:`Issue` """ raise NotImplementedError() def create_issue(self, project): """ Create an empty issue on the given project. :param project: project :type project: :class:`Project` :returns: the created issue :rtype: :class:`Issue` """ raise NotImplementedError() def post_issue(self, issue): """ Post an issue to create or update it. :param issue: issue to create or update :type issue: :class:`Issue` """ raise NotImplementedError() def update_issue(self, issue, update): """ Add an update to an issue. :param issue: issue or id of issue :type issue: :class:`Issue` :param update: an Update object :type update: :class:`Update` """ raise NotImplementedError() def remove_issue(self, issue): """ Remove an issue. :param issue: issue :type issue: :class:`Issue` """ raise NotImplementedError() def iter_projects(self): """ Iter projects. :rtype: iter[:class:`Project`] """ raise NotImplementedError() def get_project(self, id): """ Get a project from its ID. :rtype: :class:`Project` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/calendar.py000066400000000000000000000142351303450110500206570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import BaseObject, StringField, IntField, FloatField, Field, enum from .collection import CapCollection, CollectionNotFound, Collection from .date import DateField from datetime import time, datetime from weboob.tools.date import parse_date __all__ = ['BaseCalendarEvent', 'CapCalendarEvent'] CATEGORIES = enum(CONCERT=u'Concert', CINE=u'Cinema', THEATRE=u'Theatre', TELE=u'Television', CONF=u'Conference', AUTRE=u'Autre', EXPO=u'Exposition', SPECTACLE=u'Spectacle', FEST=u'Festival', SPORT=u'Sport') #the following elements deal with ICalendar stantdards #see http://fr.wikipedia.org/wiki/ICalendar#Ev.C3.A9nements_.28VEVENT.29 TRANSP = enum(OPAQUE=u'OPAQUE', TRANSPARENT=u'TRANSPARENT') STATUS = enum(TENTATIVE=u'TENTATIVE', CONFIRMED=u'CONFIRMED', CANCELLED=u'CANCELLED') TICKET = enum(AVAILABLE=u'Available', NOTAVAILABLE=u'Not available', CLOSED='Closed') class BaseCalendarEvent(BaseObject): """ Represents a calendar event """ start_date = DateField('Start date of the event') end_date = DateField('End date of the event') timezone = StringField('Timezone of the event in order to convert to utc time', default='Etc/UCT') summary = StringField('Title of the event') city = StringField('Name of the city in witch event will take place') location = StringField('Location of the event') category = Field('Category of the event', *CATEGORIES.types) description = StringField('Description of the event') price = FloatField('Price of the event') booked_entries = IntField('Entry number') max_entries = IntField('Max entry number') event_planner = StringField('Name of the event planner') #the following elements deal with ICalendar stantdards #see http://fr.wikipedia.org/wiki/ICalendar#Ev.C3.A9nements_.28VEVENT.29 sequence = IntField('Number of updates, the first is number 1') # (TENTATIVE, CONFIRMED, CANCELLED) status = Field('Status of theevent', *STATUS.types) # (OPAQUE, TRANSPARENT) transp = Field('Describes if event is available', *TRANSP.types) # (AVAILABLE, NOTAVAILABLE, CLOSED) ticket = Field('Describes if tickets are available', *TICKET.types) @classmethod def id2url(cls, _id): """Overloaded in child classes provided by backends.""" raise NotImplementedError() @property def page_url(self): """ Get page URL of the announce. """ return self.id2url(self.id) class Query(BaseObject): """ Query to find events """ start_date = DateField('Start date of the event') end_date = DateField('End date of the event') city = StringField('Name of the city in witch event will take place') categories = Field('List of categories of the event', list, tuple, default=CATEGORIES.values) ticket = Field('List of status of the tickets sale', list, tuple, default=TICKET.values) summary = StringField('Title of the event') class CapCalendarEvent(CapCollection): """ Capability of calendar event type sites """ ASSOCIATED_CATEGORIES = 'ALL' def has_matching_categories(self, query): if self.ASSOCIATED_CATEGORIES == 'ALL': return True for category in query.categories: if category in self.ASSOCIATED_CATEGORIES: return True return False def search_events(self, query): """ Search event :param query: search query :type query: :class:`Query` :rtype: iter[:class:`BaseCalendarEvent`] """ raise NotImplementedError() def list_events(self, date_from, date_to=None): """ list coming event. :param date_from: date of beguinning of the events list :type date_from: date :param date_to: date of ending of the events list :type date_to: date :rtype: iter[:class:`BaseCalendarEvent`] """ raise NotImplementedError() def get_event(self, _id): """ Get an event from an ID. :param _id: id of the event :type _id: str :rtype: :class:`BaseCalendarEvent` or None is fot found. """ raise NotImplementedError() def attends_event(self, event, is_attending=True): """ Attends or not to an event :param event : the event :type event : BaseCalendarEvent :param is_attending : is attending to the event or not :type is_attending : bool """ raise NotImplementedError() def iter_resources(self, objs, split_path): """ Iter events by category """ if len(split_path) == 0 and self.ASSOCIATED_CATEGORIES != 'ALL': for category in self.ASSOCIATED_CATEGORIES: collection = Collection([category], category) yield collection elif len(split_path) == 1 and split_path[0] in self.ASSOCIATED_CATEGORIES: query = Query() query.categories = split_path query.start_date = datetime.combine(parse_date('today'), time.min) query.end_date = parse_date('') query.city = u'' for event in self.search_events(query): yield event def validate_collection(self, objs, collection): """ Validate Collection """ if collection.path_level == 0: return if collection.path_level == 1 and collection.split_path[0] in CATEGORIES.values: return raise CollectionNotFound(collection.split_path) weboob-1.2/weboob/capabilities/chat.py000066400000000000000000000044331303450110500200240ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from .base import Capability, BaseObject, StringField, UserError from .date import DateField __all__ = ['ChatException', 'ChatMessage', 'CapChat'] class ChatException(UserError): """ Exception raised when there is a problem with the chat. """ class ChatMessage(BaseObject): """ Message on the chat. """ id_from = StringField('ID of sender') id_to = StringField('ID of recipient') message = StringField('Content of message') date = DateField('Date when the message has been sent') def __init__(self, id_from, id_to, message, date=None, url=None): BaseObject.__init__(self, '%s.%s' % (id_from, id_to), url) self.id_from = id_from self.id_to = id_to self.message = message self.date = date if self.date is None: self.date = datetime.datetime.utcnow() class CapChat(Capability): """ Websites with a chat system. """ def iter_chat_messages(self, _id=None): """ Iter messages. :param _id: optional parameter to only get messages from a given contact. :type _id: str :rtype: iter[:class:`ChatMessage`] """ raise NotImplementedError() def send_chat_message(self, _id, message): """ Send a message to a contact. :param _id: ID of recipient :type _id: str :param message: message to send :type message: str :raises: :class:`ChatException` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/cinema.py000066400000000000000000000142401303450110500203360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, IntField, Field from .date import DateField __all__ = ['Movie', 'Person', 'CapCinema'] class Movie(BaseObject): """ Movie object. """ original_title = StringField('Original title of the movie') other_titles = Field('Titles in other countries', list) release_date = DateField('Release date of the movie') all_release_dates= StringField('Release dates list of the movie') duration = IntField('Duration of the movie in minutes') short_description= StringField('Short description of the movie') genres = Field('Genres of the movie', list) pitch = StringField('Short story description of the movie') country = StringField('Origin country of the movie') note = StringField('Notation of the movie') roles = Field('Lists of Persons related to the movie indexed by roles', dict) thumbnail_url = StringField('Url of movie thumbnail') def __init__(self, id, original_title, url=None): BaseObject.__init__(self, id, url) self.original_title = original_title def get_roles_by_person_name(self,name): for role in self.roles.keys(): if name.lower() in [person[1].lower() for person in self.roles[role]]: return role return None def get_roles_by_person_id(self,id): result = [] for role in self.roles.keys(): if id in [person[0] for person in self.roles[role]]: result.append(role) return result class Person(BaseObject): """ Person object. """ name = StringField('Star name of a person') real_name = StringField('Real name of a person') birth_date = DateField('Birth date of a person') death_date = DateField('Death date of a person') birth_place = StringField('City and country of birth of a person') gender = StringField('Gender of a person') nationality = StringField('Nationality of a person') short_biography = StringField('Short biography of a person') biography = StringField('Full biography of a person') short_description= StringField('Short description of a person') roles = Field('Lists of movies related to the person indexed by roles',dict) thumbnail_url = StringField('Url of person thumbnail') def __init__(self, id, name, url=None): BaseObject.__init__(self, id, url) self.name = name def get_roles_by_movie_title(self,title): for role in self.roles.keys(): for mt in [movie[1] for movie in self.roles[role]]: # title we have is included ? if title.lower() in mt.lower(): return role return None def get_roles_by_movie_id(self,id): result = [] for role in self.roles.keys(): if id in [movie[0] for movie in self.roles[role]]: result.append(role) return result class CapCinema(Capability): """ Cinema databases. """ def iter_movies(self, pattern): """ Search movies and iterate on results. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Movies`] """ raise NotImplementedError() def get_movie(self, _id): """ Get a movie object from an ID. :param _id: ID of movie :type _id: str :rtype: :class:`Movie` """ raise NotImplementedError() def get_movie_releases(self, _id, country=None): """ Get a list of a movie releases from an ID. :param _id: ID of movie :type _id: str :rtype: :class:`String` """ raise NotImplementedError() def iter_movie_persons(self, _id, role=None): """ Get the list of persons who are related to a movie. :param _id: ID of movie :type _id: str :rtype: iter[:class:`Person`] """ raise NotImplementedError() def iter_persons(self, pattern): """ Search persons and iterate on results. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`persons`] """ raise NotImplementedError() def get_person(self, _id): """ Get a person object from an ID. :param _id: ID of person :type _id: str :rtype: :class:`Person` """ raise NotImplementedError() def iter_person_movies(self, _id, role=None): """ Get the list of movies related to a person. :param _id: ID of person :type _id: str :rtype: iter[:class:`Movie`] """ raise NotImplementedError() def iter_person_movies_ids(self, _id): """ Get the list of movie ids related to a person. :param _id: ID of person :type _id: str :rtype: iter[str] """ raise NotImplementedError() def iter_movie_persons_ids(self, _id): """ Get the list of person ids related to a movie. :param _id: ID of movie :type _id: str :rtype: iter[str] """ raise NotImplementedError() def get_person_biography(self, id): """ Get the person full biography. :param _id: ID of person :type _id: str :rtype: str """ raise NotImplementedError() weboob-1.2/weboob/capabilities/collection.py000066400000000000000000000132031303450110500212330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Nicolas Duhamel, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.ordereddict import OrderedDict from .base import Capability, BaseObject, UserError, StringField, Field __all__ = ['CapCollection', 'BaseCollection', 'Collection', 'CollectionNotFound'] class CollectionNotFound(UserError): def __init__(self, split_path=None): if split_path is not None: msg = 'Collection not found: %s' % '/'.join(split_path) else: msg = 'Collection not found' UserError.__init__(self, msg) class BaseCollection(BaseObject): """ Inherit from this if you want to create an object that is *also* a Collection. However, this probably will not work properly for now. """ def __init__(self, split_path, id=None, url=None): BaseObject.__init__(self, id, url) self.split_path = split_path @property def basename(self): return self.split_path[-1] if self.path_level else None @property def parent_path(self): return self.split_path[0:-1] if self.path_level else None @property def path_level(self): return len(self.split_path) def to_dict(self): def iter_decorate(d): for key, value in d: if key == 'id' and self.backend is not None: value = u'%s@%s' % (self.basename, self.backend) yield key, value if key == 'split_path': yield key, '/'.join(value) fields_iterator = self.iter_fields() return OrderedDict(iter_decorate(fields_iterator)) class Collection(BaseCollection): """ A Collection is a "fake" object returned in results, which shows you can get more results if you go into its path. It is a dumb object, it must not contain callbacks to a backend. Do not inherit from this class if you want to make a regular BaseObject a Collection, use BaseCollection instead. """ title = StringField('Collection title') split_path = Field('Full collection path', list) def __init__(self, split_path=None, title=None, id=None, url=None): self.title = title BaseCollection.__init__(self, split_path, id, url) def __unicode__(self): if self.title and self.basename: return u'%s (%s)' % (self.basename, self.title) elif self.basename: return u'%s' % self.basename else: return u'Unknown collection' class CapCollection(Capability): def iter_resources_flat(self, objs, split_path, clean_only=False): """ Call iter_resources() to fetch all resources in the tree. If clean_only is True, do not explore paths, only remove them. split_path is used to set the starting path. """ for resource in self.iter_resources(objs, split_path): if isinstance(resource, Collection): if not clean_only: for res in self.iter_resources_flat(objs, resource.split_path): yield res else: yield resource def iter_resources(self, objs, split_path): """ split_path is a list, either empty (root path) or with one or many components. """ raise NotImplementedError() def get_collection(self, objs, split_path): """ Get a collection for a given split path. If the path is invalid (i.e. can't be handled by this module), it should return None. """ collection = Collection(split_path, None) return self.validate_collection(objs, collection) or collection def validate_collection(self, objs, collection): """ Tests if a collection is valid. For compatibility reasons, and to provide a default way, it checks if the collection has at least one object in it. However, it is not very efficient or exact, and you are encouraged to override this method. You can replace the collection object entirely by returning a new one. """ # Root if collection.path_level == 0: return try: i = self.iter_resources(objs, collection.split_path) i.next() except StopIteration: raise CollectionNotFound(collection.split_path) def _restrict_level(self, split_path, lmax=0): if len(split_path) > lmax: raise CollectionNotFound(split_path) def test(): c = Collection([]) assert c.basename is None assert c.parent_path is None assert c.path_level == 0 c = Collection([u'lol']) assert c.basename == u'lol' assert c.parent_path == [] assert c.path_level == 1 c = Collection([u'lol', u'cat']) assert c.basename == u'cat' assert c.parent_path == [u'lol'] assert c.path_level == 2 c = Collection([u'w', u'e', u'e', u'b', u'o', u'o', u'b']) assert c.basename == u'b' assert c.parent_path == [u'w', u'e', u'e', u'b', u'o', u'o'] assert c.path_level == 7 weboob-1.2/weboob/capabilities/contact.py000066400000000000000000000165331303450110500205440ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField, BytesField, IntField, \ UserError from weboob.tools.ordereddict import OrderedDict __all__ = ['ProfileNode', 'ContactPhoto', 'Contact', 'QueryError', 'Query', 'CapContact'] class ProfileNode(object): """ Node of a :class:`Contact` profile. """ HEAD = 0x01 SECTION = 0x02 def __init__(self, name, label, value, sufix=None, flags=0): self.name = name self.label = label self.value = value self.sufix = sufix self.flags = flags def __getitem__(self, key): return self.value[key] class ContactPhoto(BaseObject): """ Photo of a contact. """ name = StringField('Name of the photo') data = BytesField('Data of photo') thumbnail_url = StringField('Direct URL to thumbnail') thumbnail_data = BytesField('Data of thumbnail') hidden = Field('True if the photo is hidden on website', bool) def __init__(self, name, url=None): BaseObject.__init__(self, name, url) self.name = name def __iscomplete__(self): return (self.data and (not self.thumbnail_url or self.thumbnail_data)) def __str__(self): return self.url def __repr__(self): return '' % (self.id, len(self.data) if self.data else 0, len(self.thumbnail_data) if self.thumbnail_data else 0) class BaseContact(BaseObject): """ This is the blase class for a contact. """ name = StringField('Name of contact') class Advisor(BaseContact): """ An advisor. """ email = StringField('Mail of advisor') phone = StringField('Phone number of advisor') mobile = StringField('Mobile number of advisor') fax = StringField('Fax number of advisor') agency = StringField('Name of agency') address = StringField('Address of agency') role = StringField('Role of advisor', default="bank") class Contact(BaseContact): """ A contact. """ STATUS_ONLINE = 0x001 STATUS_AWAY = 0x002 STATUS_OFFLINE = 0x004 STATUS_ALL = 0xfff status = IntField('Status of contact (STATUS_* constants)') status_msg = StringField('Message of status') summary = StringField('Description of contact') photos = Field('List of photos', dict, default=OrderedDict()) profile = Field('Contact profile', dict, default=OrderedDict()) def __init__(self, id, name, status, url=None): BaseObject.__init__(self, id, url) self.name = name self.status = status def set_photo(self, name, **kwargs): """ Set photo of contact. :param name: name of photo :type name: str :param kwargs: See :class:`ContactPhoto` to know what other parameters you can use """ if name not in self.photos: self.photos[name] = ContactPhoto(name) photo = self.photos[name] for key, value in kwargs.iteritems(): setattr(photo, key, value) def get_text(self): def print_node(node, level=1): result = u'' if node.flags & node.SECTION: result += u'\t' * level + node.label + '\n' for sub in node.value.itervalues(): result += print_node(sub, level + 1) else: if isinstance(node.value, (tuple, list)): value = ', '.join(unicode(v) for v in node.value) elif isinstance(node.value, float): value = '%.2f' % node.value else: value = node.value result += u'\t' * level + u'%-20s %s\n' % (node.label + ':', value) return result result = u'Nickname: %s\n' % self.name if self.status & Contact.STATUS_ONLINE: s = 'online' elif self.status & Contact.STATUS_OFFLINE: s = 'offline' elif self.status & Contact.STATUS_AWAY: s = 'away' else: s = 'unknown' result += u'Status: %s (%s)\n' % (s, self.status_msg) result += u'URL: %s\n' % self.url result += u'Photos:\n' for name, photo in self.photos.iteritems(): result += u'\t%s%s\n' % (photo, ' (hidden)' if photo.hidden else '') result += u'\nProfile:\n' for head in self.profile.itervalues(): result += print_node(head) result += u'Description:\n' for s in self.summary.split('\n'): result += u'\t%s\n' % s return result class QueryError(UserError): """ Raised when unable to send a query to a contact. """ class Query(BaseObject): """ Query to send to a contact. """ message = StringField('Message received') def __init__(self, id, message, url=None): BaseObject.__init__(self, id, url) self.message = message class CapContact(Capability): def iter_contacts(self, status=Contact.STATUS_ALL, ids=None): """ Iter contacts :param status: get only contacts with the specified status :type status: Contact.STATUS_* :param ids: if set, get the specified contacts :type ids: list[str] :rtype: iter[:class:`Contact`] """ raise NotImplementedError() def get_contact(self, id): """ Get a contact from his id. The default implementation only calls iter_contacts() with the proper values, but it might be overloaded by backends. :param id: the ID requested :type id: str :rtype: :class:`Contact` or None if not found """ l = self.iter_contacts(ids=[id]) try: return l[0] except IndexError: return None def send_query(self, id): """ Send a query to a contact :param id: the ID of contact :type id: str :rtype: :class:`Query` :raises: :class:`QueryError` """ raise NotImplementedError() def get_notes(self, id): """ Get personal notes about a contact :param id: the ID of the contact :type id: str :rtype: unicode """ raise NotImplementedError() def save_notes(self, id, notes): """ Set personal notes about a contact :param id: the ID of the contact :type id: str :returns: the unicode object to save as notes """ raise NotImplementedError() weboob-1.2/weboob/capabilities/content.py000066400000000000000000000052251303450110500205570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, Field from .date import DateField __all__ = ['Content', 'Revision', 'CapContent'] class Content(BaseObject): """ Content object. """ title = StringField('Title of content') author = StringField('Original author of content') content = StringField('Body') revision = StringField('ID of revision') class Revision(BaseObject): """ Revision of a change on a content. """ author = StringField('Author of revision') comment = StringField('Comment log about revision') timestamp = DateField('Date of revision') minor = Field('Is this change minor?', bool) class CapContent(Capability): def get_content(self, id, revision=None): """ Get a content from an ID. :param id: ID of content :type id: str :param revision: if given, get the content at this revision :type revision: :class:`Revision` :rtype: :class:`Content` """ raise NotImplementedError() def iter_revisions(self, id): """ Iter revisions of a content. :param id: id of content :type id: str :rtype: iter[:class:`Revision`] """ raise NotImplementedError() def push_content(self, content, message=None, minor=False): """ Push a new revision of a content. :param content: object to push :type content: :class:`Content` :param message: log message to associate to new revision :type message: str :param minor: this is a minor revision :type minor: bool """ raise NotImplementedError() def get_content_preview(self, content): """ Get a HTML preview of a content. :param content: content object :type content: :class:`Content` :rtype: str """ raise NotImplementedError() weboob-1.2/weboob/capabilities/date.py000066400000000000000000000040211303450110500200130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz, Romain Bignon, Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from weboob.tools.date import new_date, new_datetime from weboob.capabilities.base import Field __all__ = ['DateField', 'TimeField', 'DeltaField'] class DateField(Field): """ A field which accepts only :class:`datetime.date` and :class:`datetime.datetime` types. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, datetime.date, datetime.datetime, **kwargs) def __setattr__(self, name, value): if name == 'value': # Force use of our date and datetime types, to fix bugs in python2 # with strftime on year<1900. if type(value) is datetime.datetime: value = new_datetime(value) if type(value) is datetime.date: value = new_date(value) return object.__setattr__(self, name, value) class TimeField(Field): """ A field which accepts only :class:`datetime.time` and :class:`datetime.time` types. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, datetime.time, datetime.datetime, **kwargs) class DeltaField(Field): """ A field which accepts only :class:`datetime.timedelta` type. """ def __init__(self, doc, **kwargs): Field.__init__(self, doc, datetime.timedelta, **kwargs) weboob-1.2/weboob/capabilities/dating.py000066400000000000000000000074641303450110500203620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField, UserError from .date import DateField from .contact import Contact __all__ = ['OptimizationNotFound', 'Optimization', 'Event', 'CapDating'] class OptimizationNotFound(UserError): """ Raised when an optimization is not found. """ class Optimization(BaseObject): """ Optimization. :var CONFIG: Configuration of optim can be made by :class:`weboob.tools.value.Value` objects in this dict. """ CONFIG = {} def start(self): """ Start optimization. """ raise NotImplementedError() def stop(self): """ Stop optimization. """ raise NotImplementedError() def is_running(self): """ Know if the optimization is currently running. :rtype: bool """ raise NotImplementedError() def get_config(self): """ Get config of this optimization. :rtype: dict """ return None def set_config(self, params): """ Set config of this optimization. :param params: parameters :type params: dict """ raise NotImplementedError() class Event(BaseObject): """ A dating event (for example a visite, a query received, etc.) """ date = DateField('Date of event') contact = Field('Contact related to this event', Contact) type = StringField('Type of event') message = StringField('Message of the event') class CapDating(Capability): """ Capability for dating websites. """ def init_optimizations(self): """ Initialization of optimizations. """ raise NotImplementedError() def add_optimization(self, name, optim): """ Add an optimization. :param name: name of optimization :type name: str :param optim: optimization :type optim: :class:`Optimization` """ optim.id = name setattr(self, 'OPTIM_%s' % name, optim) def iter_optimizations(self): """ Iter optimizations. :rtype: iter[:class:`Optimization`] """ for attr_name in dir(self): if not attr_name.startswith('OPTIM_'): continue attr = getattr(self, attr_name) if attr is None: continue yield attr def get_optimization(self, optim): """ Get an optimization from a name. :param optim: name of optimization :type optim: str :rtype: :class:`Optimization` """ optim = optim.upper() if not hasattr(self, 'OPTIM_%s' % optim): raise OptimizationNotFound() return getattr(self, 'OPTIM_%s' % optim) def iter_events(self): """ Iter events. :rtype: iter[:class:`Event`] """ raise NotImplementedError() def iter_new_contacts(self): """ Iter new contacts. :rtype: iter[:class:`Contact`] """ raise NotImplementedError() weboob-1.2/weboob/capabilities/file.py000066400000000000000000000057471303450110500200350ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, NotAvailable, Field, StringField, enum from .date import DateField __all__ = ['BaseFile', 'CapFile'] LICENSES = enum( OTHER=u'Other license', PD=u'Public Domain', COPYRIGHT=u'All rights reserved', CCBY=u'Creative Commons BY', CCBYSA=u'Creative Commons BY-SA', CCBYNC=u'Creative Commons BY-NC', CCBYND=u'Creative Commons BY-ND', CCBYNCSA=u'Creative Commons BY-NC-SA', CCBYNCND=u'Creative Commons BY-NC-ND', GFDL=u'GNU Free Documentation License') class BaseFile(BaseObject): """ Represent a file. """ title = StringField('File title') ext = StringField('File extension') author = StringField('File author') description = StringField('File description') date = DateField('File publication date') size = Field('File size in bytes',int,long, default=NotAvailable) rating = Field('Rating', int, long, float, default=NotAvailable) rating_max = Field('Maximum rating', int, long, float, default=NotAvailable) license = StringField('License name') def __str__(self): return self.url or '' def __repr__(self): return '<%s title=%r url=%r>' % (type(self).__name__, self.title, self.url) @classmethod def id2url(cls, _id): """ Overloaded in child classes provided by backends. """ raise NotImplementedError() @property def page_url(self): """ Get file page URL """ return self.id2url(self.id) class CapFile(Capability): """ Provide file download """ (SEARCH_RELEVANCE, SEARCH_RATING, SEARCH_VIEWS, SEARCH_DATE) = range(4) def search_file(self, pattern, sortby=SEARCH_RELEVANCE): """ :param pattern: pattern to search on :type pattern: str :param sortby: sort by ... (user SEARCH_* constants) :rtype: iter[:class:`BaseFile`] """ raise NotImplementedError() def get_file(self, _id): """ Get a file from an ID :param _id: the file id. I can be a numeric ID, or a page url :type _id: str :rtype: :class:`BaseFile` or None if not found. """ raise NotImplementedError() weboob-1.2/weboob/capabilities/gallery.py000066400000000000000000000100351303450110500205370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Christophe Benz, Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.image import BaseImage as CIBaseImage, Thumbnail from .base import Capability, BaseObject, NotLoaded, Field, StringField, \ IntField, FloatField from .date import DateField __all__ = ['BaseGallery', 'BaseImage', 'CapGallery'] class BaseGallery(BaseObject): """ Represents a gallery. This object has to be inherited to specify how to calculate the URL of the gallery from its ID. """ title = StringField('Title of gallery') description = StringField('Description of gallery') cardinality = IntField('Cardinality of gallery') date = DateField('Date of gallery') rating = FloatField('Rating of this gallery') rating_max = FloatField('Max rating available') thumbnail = Field('Thumbnail', Thumbnail) def __init__(self, _id, title=NotLoaded, url=NotLoaded, cardinality=NotLoaded, date=NotLoaded, rating=NotLoaded, rating_max=NotLoaded, thumbnail=NotLoaded, thumbnail_url=None, nsfw=False): BaseObject.__init__(self, unicode(_id), url) self.title = title self.date = date self.rating = rating self.rating_max = rating_max self.thumbnail = thumbnail @classmethod def id2url(cls, _id): """Overloaded in child classes provided by backends.""" raise NotImplementedError() @property def page_url(self): """ Get URL to page of this gallery. """ return self.id2url(self.id) def iter_image(self): """ Iter images. """ raise NotImplementedError() class BaseImage(CIBaseImage): """ Base class for images. """ index = IntField('Usually page number') gallery = Field('Reference to the Gallery object', BaseGallery) def __init__(self, _id, index=None, thumbnail=NotLoaded, url=NotLoaded, ext=NotLoaded, gallery=None): BaseObject.__init__(self, unicode(_id), url) self.index = index self.thumbnail = thumbnail self.ext = ext self.gallery = gallery def __str__(self): return self.url def __repr__(self): return '' % self.url def __iscomplete__(self): return self.data is not NotLoaded class CapGallery(Capability): """ This capability represents the ability for a website backend to provide videos. """ (SEARCH_RELEVANCE, SEARCH_RATING, SEARCH_VIEWS, SEARCH_DATE) = range(4) def search_galleries(self, pattern, sortby=SEARCH_RELEVANCE): """ Iter results of a search on a pattern. :param pattern: pattern to search on :type pattern: str :param sortby: sort by... :type sortby: SEARCH_* :rtype: :class:`BaseGallery` """ raise NotImplementedError() def get_gallery(self, _id): """ Get gallery from an ID. :param _id: the gallery id. It can be a numeric ID, or a page url, or so. :type _id: str :rtype: :class:`Gallery` """ raise NotImplementedError() def iter_gallery_images(self, gallery): """ Iterate images from a Gallery. :type gallery: BaseGallery :rtype: iter(BaseImage) """ raise NotImplementedError() weboob-1.2/weboob/capabilities/gauge.py000066400000000000000000000064151303450110500201770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Florent Fourcot # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, FloatField, Field, UserError, empty from .date import DateField __all__ = ['Gauge', 'GaugeSensor', 'GaugeMeasure', 'CapGauge', 'SensorNotFound'] class SensorNotFound(UserError): """ Not found a sensor """ class Gauge(BaseObject): """ Gauge class. """ name = StringField('Name of gauge') city = StringField('City of the gauge') object = StringField('What is evaluate') # For example, name of a river sensors = Field('List of sensors on the gauge', list) class GaugeMeasure(BaseObject): """ Measure of a gauge sensor. """ level = FloatField('Level of measure') date = DateField('Date of measure') alarm = StringField('Alarm level') def __repr__(self): if empty(self.level): return "" % self.level else: return "" % (self.level, self.alarm, self.date) class GaugeSensor(BaseObject): """ GaugeSensor class. """ name = StringField('Name of the sensor') unit = StringField('Unit of values') forecast = StringField('Forecast') address = StringField('Address') lastvalue = Field('Last value', GaugeMeasure) history = Field('Value history', list) # lastvalue not included gaugeid = StringField('Id of the gauge') def __repr__(self): return "" % (self.id, self.name) class CapGauge(Capability): def iter_gauges(self, pattern=None): """ Iter gauges. :param pattern: if specified, used to search gauges. :type pattern: str :rtype: iter[:class:`Gauge`] """ raise NotImplementedError() def iter_sensors(self, id, pattern=None): """ Iter instrument of a gauge. :param: ID of the gauge :param pattern: if specified, used to search sensors. :type pattern: str :rtype: iter[:class:`GaugeSensor`] """ raise NotImplementedError() def iter_gauge_history(self, id): """ Get history of a gauge sensor. :param id: ID of the gauge sensor :type id: str :rtype: iter[:class:`GaugeMeasure`] """ raise NotImplementedError() def get_last_measure(self, id): """ Get last measures of a censor. :param id: ID of the censor. :type id: str :rtype: :class:`GaugeMeasure` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/geolocip.py000066400000000000000000000032641303450110500207070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, FloatField __all__ = ['IpLocation', 'CapGeolocIp'] class IpLocation(BaseObject): """ Represents the location of an IP address. """ city = StringField('City') region = StringField('Region') zipcode = StringField('Zip code') country = StringField('Country') lt = FloatField('Latitude') lg = FloatField('Longitude') osmlink = StringField('Link to OpenStreetMap location page') host = StringField('Hostname') tld = StringField('Top Level Domain') isp = StringField('Internet Service Provider') class CapGeolocIp(Capability): """ Access information about IP addresses database. """ def get_location(self, ipaddr): """ Get location of an IP address. :param ipaddr: IP address :type ipaddr: str :rtype: :class:`IpLocation` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/housing.py000066400000000000000000000101231303450110500205520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, IntField, DecimalField, \ StringField, BytesField, enum, UserError, FloatField from .date import DateField __all__ = ['HousingPhoto', 'Housing', 'Query', 'City', 'CapHousing'] class TypeNotSupported(UserError): """ Raised when query type is not supported """ def __init__(self, msg='This type of house is not supported by this module'): UserError.__init__(self, msg) class HousingPhoto(BaseObject): """ Photo of a housing. """ data = BytesField('Data of photo') def __init__(self, url): BaseObject.__init__(self, url.split('/')[-1], url) def __iscomplete__(self): return self.data def __str__(self): return self.url def __repr__(self): return '' % (self.id, len(self.data) if self.data else 0, self.url) class Housing(BaseObject): """ Content of a housing. """ title = StringField('Title of housing') area = DecimalField('Area of housing, in m2') cost = DecimalField('Cost of housing') price_per_meter = FloatField('Price per meter ratio') currency = StringField('Currency of cost') date = DateField('Date when the housing has been published') location = StringField('Location of housing') station = StringField('What metro/bus station next to housing') text = StringField('Text of the housing') phone = StringField('Phone number to contact') photos = Field('List of photos', list) details = Field('Key/values of details', dict) class Query(BaseObject): """ Query to find housings. """ TYPE_RENT = 0 TYPE_SALE = 1 TYPE_SHARING = 2 HOUSE_TYPES = enum(APART=u'Apartment', HOUSE=u'House', PARKING=u'Parking', LAND=u'Land', OTHER=u'Other', UNKNOWN=u'Unknown') type = IntField('Type of housing to find (TYPE_* constants)') cities = Field('List of cities to search in', list, tuple) area_min = IntField('Minimal area (in m2)') area_max = IntField('Maximal area (in m2)') cost_min = IntField('Minimal cost') cost_max = IntField('Maximal cost') nb_rooms = IntField('Number of rooms') house_types = Field('List of house types', list, tuple, default=HOUSE_TYPES.values) class City(BaseObject): """ City. """ name = StringField('Name of city') class CapHousing(Capability): """ Capability of websites to search housings. """ def search_housings(self, query): """ Search housings. :param query: search query :type query: :class:`Query` :rtype: iter[:class:`Housing`] """ raise NotImplementedError() def get_housing(self, housing): """ Get an housing from an ID. :param housing: ID of the housing :type housing: str :rtype: :class:`Housing` or None if not found. """ raise NotImplementedError() def search_city(self, pattern): """ Search a city from a pattern. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`City`] """ raise NotImplementedError() weboob-1.2/weboob/capabilities/image.py000066400000000000000000000055741303450110500201760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Christophe Benz, Noé Rubinstein # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.ordereddict import OrderedDict from .base import NotLoaded, Field, BytesField from .file import CapFile, BaseFile __all__ = ['BaseImage', 'Thumbnail', 'CapImage'] class _BaseImage(BaseFile): """ Fake class to allow the inclusion of a BaseImage property within the real BaseImage class """ pass class Thumbnail(_BaseImage): """ Thumbnail of an image. """ data = BytesField('Data') def __init__(self, url): super(Thumbnail, self).__init__(url) self.url = url.replace(u' ', u'%20') def __str__(self): return self.url def __repr__(self): return '' % self.url class BaseImage(_BaseImage): """ Represents an image file. """ nsfw = Field('Is this Not Safe For Work', bool, default=False) thumbnail = Field('Thumbnail of the image', Thumbnail) data = BytesField('Data of image') def __iscomplete__(self): return self.data is not NotLoaded def to_dict(self): def iter_decorate(d): for key, value in d: if key == 'data': continue if key == 'id' and self.backend is not None: value = self.fullid yield key, value fields_iterator = self.iter_fields() return OrderedDict(iter_decorate(fields_iterator)) class CapImage(CapFile): """ Image file provider """ def search_image(self, pattern, sortby=CapFile.SEARCH_RELEVANCE, nsfw=False): """ search for an image file :param pattern: pattern to search on :type pattern: str :param sortby: sort by ...(use SEARCH_* constants) :param nsfw: include non-suitable for work images if True :type nsfw: bool :rtype: iter[:class:`BaseImage`] """ return self.search_file(pattern, sortby) def get_image(self, _id): """ Get an image file from an ID. :param id: image file ID :type id: str :rtype: :class:`BaseImage`] """ return self.get_file(_id) weboob-1.2/weboob/capabilities/job.py000066400000000000000000000066161303450110500176640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import BaseObject, Capability, StringField from .date import DateField __all__ = ['BaseJobAdvert', 'CapJob'] class BaseJobAdvert(BaseObject): """ Represents a job announce. """ publication_date = DateField('Date when the announce has been published') society_name = StringField('Name of the society taht published the announce') place = StringField('Place where the job take place') job_name = StringField('Name of the job') title = StringField('Title of the announce') contract_type = StringField('Type of the contrat : CDI, CDD') pay = StringField('Amount of the salary') description = StringField('Description of the job') formation = StringField('Required formation') experience = StringField('Required experience') def __unicode__(self): message = u'\r\n-- Advert --\r\n' message += u'id : %s\r\n' % self.id message += u'url : %s\r\n' % self.url message += u'publication_date : %s\r\n' % self.publication_date message += u'society_name : %s\r\n' % self.society_name message += u'place : %s\r\n' % self.place message += u'job_name : %s\r\n' % self.job_name message += u'title : %s\r\n' % self.title message += u'contract_type : %s\r\n' % self.contract_type message += u'pay : %s\r\n' % self.pay message += u'description : %s\r\n' % self.description message += u'formation : %s\r\n' % self.formation message += u'experience : %s\r\n' % self.experience return message @classmethod def id2url(cls, _id): """Overloaded in child classes provided by backends.""" raise NotImplementedError() @property def page_url(self): """ Get page URL of the announce. """ return self.id2url(self.id) class CapJob(Capability): """ Capability of job annouce websites. """ def search_job(self, pattern=None): """ Iter results of a search on a pattern. :param pattern: pattern to search on :type pattern: str :rtype: iter[:class:`BaseJobAdvert`] """ raise NotImplementedError() def advanced_search_job(self): """ Iter results of an advanced search :rtype: iter[:class:`BaseJobAdvert`] """ def get_job_advert(self, _id, advert=None): """ Get an announce from an ID. :param _id: id of the advert :type _id: str :param advert: the advert :type advert: BaseJobAdvert :rtype: :class:`BaseJobAdvert` or None if not found. """ raise NotImplementedError() weboob-1.2/weboob/capabilities/library.py000066400000000000000000000042511303450110500205470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Jeremy Monnet # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .collection import CapCollection from .base import BaseObject, Field, StringField from .date import DateField __all__ = ['Book', 'Renew', 'CapBook'] class Book(BaseObject): """ Describes a book. """ name = StringField('Name of the book') author = StringField('Author of the book') location = StringField('Location') date = DateField('The due date') late = Field('Are you late?', bool) class Renew(BaseObject): """ A renew message. """ message = StringField('Message') class CapBook(CapCollection): """ Library websites. """ def iter_resources(self, objs, split_path): """ Iter resources. It retuns :func:`iter_books`. """ if Book in objs: self._restrict_level(split_path) return self.iter_books() def iter_books(self): """ Iter books. :rtype: iter[:class:`Book`] """ raise NotImplementedError() def get_book(self, _id): """ Get a book from an ID. :param _id: ID of the book :type _id: str :rtype: :class:`Book` """ raise NotImplementedError() def get_booked(self, _id): raise NotImplementedError() def renew_book(self, _id): raise NotImplementedError() def get_rented(self, _id): raise NotImplementedError() def search_books(self, _string): raise NotImplementedError() weboob-1.2/weboob/capabilities/lyrics.py000066400000000000000000000032371303450110500204130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField __all__ = ['SongLyrics', 'CapLyrics'] class SongLyrics(BaseObject): """ Song lyrics object. """ title = StringField('Title of the song') artist = StringField('Artist of the song') content = StringField('Lyrics of the song') class CapLyrics(Capability): """ Lyrics websites. """ def iter_lyrics(self, criteria, pattern): """ Search lyrics by artist or by song and iterate on results. :param criteria: 'artist' or 'song' :type criteria: str :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`SongLyrics`] """ raise NotImplementedError() def get_lyrics(self, _id): """ Get a lyrics object from an ID. :param _id: ID of lyrics :type _id: str :rtype: :class:`SongLyrics` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/messages.py000066400000000000000000000142351303450110500207150ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2015 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime import time from .base import Capability, BaseObject, NotLoaded, Field, StringField, \ IntField, UserError from .date import DateField __all__ = ['Thread', 'Message', 'CapMessages', 'CantSendMessage', 'CapMessagesPost'] class Message(BaseObject): """ Represents a message read or to send. """ IS_HTML = 0x001 "The content is HTML formatted" IS_UNREAD = 0x002 "The message is unread" IS_RECEIVED = 0x004 "The receiver has read this message" IS_NOT_RECEIVED = 0x008 "The receiver hass not read this message" thread = Field('Reference to the thread', 'Thread') title = StringField('Title of message') sender = StringField('Author of this message') receivers = Field('Receivers of the message', list) date = DateField('Date when the message has been sent') content = StringField('Body of message') signature = StringField('Optional signature') parent = Field('Parent message', 'Message') children = Field('Children fields', list) flags = IntField('Flags (IS_* constants)', default=0) def __init__(self, thread=NotLoaded, id=NotLoaded, title=NotLoaded, sender=NotLoaded, receivers=NotLoaded, date=None, parent=NotLoaded, content=NotLoaded, signature=NotLoaded, children=NotLoaded, flags=0, url=None): super(Message, self).__init__(id, url) self.thread = thread self.title = title self.sender = sender self.receivers = receivers self.content = content self.signature = signature self.children = children self.flags = flags if date is None: date = datetime.datetime.utcnow() self.date = date if isinstance(parent, Message): self.parent = parent else: self.parent = NotLoaded self._parent_id = parent @property def date_int(self): """ Date of message as an integer. """ return int(time.strftime('%Y%m%d%H%M%S', self.date.timetuple())) @property def full_id(self): """ Full ID of message (in form '**THREAD_ID.MESSAGE_ID**') """ return '%s.%s' % (self.thread.id, self.id) @property def full_parent_id(self): """ Get the full ID of the parent message (in form '**THREAD_ID.MESSAGE_ID**'). """ if self.parent: return self.parent.full_id elif self._parent_id is None: return '' elif self._parent_id is NotLoaded: return NotLoaded else: return '%s.%s' % (self.thread.id, self._parent_id) def __eq__(self, msg): if not isinstance(msg, Message): return False if self.thread: return unicode(self.thread.id) == unicode(msg.thread.id) and \ unicode(self.id) == unicode(msg.id) else: return unicode(self.id) == unicode(msg.id) def __repr__(self): return '' % ( self.full_id, self.title, self.date, self.sender) class Thread(BaseObject): """ Thread containing messages. """ IS_THREADS = 0x001 IS_DISCUSSION = 0x002 root = Field('Root message', Message) title = StringField('Title of thread') date = DateField('Date of thread') flags = IntField('Flags (IS_* constants)', default=IS_THREADS) def iter_all_messages(self): """ Iter all messages of the thread. :rtype: iter[:class:`Message`] """ if self.root: yield self.root for m in self._iter_all_messages(self.root): yield m def _iter_all_messages(self, message): if message.children: for child in message.children: yield child for m in self._iter_all_messages(child): yield m class CapMessages(Capability): """ Capability to read messages. """ def iter_threads(self): """ Iterates on threads, from newers to olders. :rtype: iter[:class:`Thread`] """ raise NotImplementedError() def get_thread(self, id): """ Get a specific thread. :rtype: :class:`Thread` """ raise NotImplementedError() def iter_unread_messages(self): """ Iterates on messages which hasn't been marked as read. :rtype: iter[:class:`Message`] """ raise NotImplementedError() def set_message_read(self, message): """ Set a message as read. :param message: message read (or ID) :type message: :class:`Message` or str """ raise NotImplementedError() class CantSendMessage(UserError): """ Raised when a message can't be send. """ class CapMessagesPost(Capability): """ This capability allow user to send a message. """ def post_message(self, message): """ Post a message. :param message: message to send :type message: :class:`Message` :raises: :class:`CantSendMessage` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/parcel.py000066400000000000000000000036621303450110500203560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField, UserError from .date import DateField class Event(BaseObject): date = DateField('Date') activity = StringField('Activity') location = StringField('Location') def __repr__(self): return '' % (self.date, self.activity, self.location) class Parcel(BaseObject): STATUS_UNKNOWN = 0 STATUS_PLANNED = 1 STATUS_IN_TRANSIT = 2 STATUS_ARRIVED = 3 arrival = DateField('Scheduled arrival date') status = Field('Status of parcel', int, default=STATUS_UNKNOWN) info = StringField('Information about parcel status') history = Field('History', list) class CapParcel(Capability): def get_parcel_tracking(self, id): """ Get information abouut a parcel. :param id: ID of the parcel :type id: :class:`str` :rtype: :class:`Parcel` :raises: :class:`ParcelNotFound` """ raise NotImplementedError() class ParcelNotFound(UserError): """ Raised when a parcell is not found. It can be an user error, or an expired parcel """ def __init__(self, msg='Account not found'): UserError.__init__(self, msg) weboob-1.2/weboob/capabilities/paste.py000066400000000000000000000071351303450110500202230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, NotLoaded, Field, StringField, UserError __all__ = ['PasteNotFound', 'BasePaste', 'CapPaste'] class PasteNotFound(UserError): """ Raised when a paste is not found. """ def __init__(self): return super(PasteNotFound, self).__init__("Paste not found") class BasePaste(BaseObject): """ Represents a pasted text. """ title = StringField('Title of paste') language = StringField('Language of the paste') contents = StringField('Content of the paste') public = Field('Is this paste public?', bool) def __init__(self, _id, title=NotLoaded, language=NotLoaded, contents=NotLoaded, public=NotLoaded, url=None): BaseObject.__init__(self, unicode(_id), url) self.title = title self.language = language self.contents = contents self.public = public @classmethod def id2url(cls, _id): """Overloaded in child classes provided by backends.""" raise NotImplementedError() @property def page_url(self): """ Get URL to page of this paste. """ return self.id2url(self.id) class CapPaste(Capability): """ This capability represents the ability for a website backend to store plain text. """ def new_paste(self, *args, **kwargs): """ Get a new paste object for posting it with the backend. The parameters should be passed to the object init. :rtype: :class:`BasePaste` """ raise NotImplementedError() def can_post(self, contents, title=None, public=None, max_age=None): """ Checks if the paste can be pasted by this backend. Some properties are considered required (public/private, max_age) while others are just bonuses (language). contents: Can be used to check encodability, maximum length, etc. title: Can be used to check length, allowed characters. Should not be required. public: True must be public, False must be private, None do not care. max_age: Maximum time to live in seconds. A score of 0 means the backend is not suitable. A score of 1 means the backend is suitable. Higher scores means it is more suitable than others with a lower score. :rtype: int :returns: score """ raise NotImplementedError() def get_paste(self, url): """ Get a Paste from an ID or URL. :param _id: the paste id. It can be an ID or a page URL. :type _id: str :rtype: :class:`BasePaste` :raises: :class:`PasteNotFound` """ raise NotImplementedError() def post_paste(self, paste, max_age=None): """ Post a paste. :param paste: a Paste object :type paste: :class:`BasePaste` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/pricecomparison.py000066400000000000000000000051221303450110500222760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, DecimalField, \ StringField, UserError from .date import DateField __all__ = ['Shop', 'Price', 'Product', 'CapPriceComparison'] class PriceNotFound(UserError): """ Raised when a price is not found """ def __init__(self, msg='Price not found'): UserError.__init__(self, msg) class Product(BaseObject): """ A product. """ name = StringField('Name of product') class Shop(BaseObject): """ A shop where the price is. """ name = StringField('Name of shop') location = StringField('Location of the shop') info = StringField('Information about the shop') class Price(BaseObject): """ Price. """ date = DateField('Date when this price has been published') cost = DecimalField('Cost of the product in this shop') currency = StringField('Currency of the price') message = StringField('Message related to this price') shop = Field('Shop information', Shop) product = Field('Product', Product) class CapPriceComparison(Capability): """ Capability for price comparison websites. """ def search_products(self, pattern=None): """ Search products from a pattern. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Product`] """ raise NotImplementedError() def iter_prices(self, products): """ Iter prices for a product. :param product: product to search :type product: :class:`Product` :rtype: iter[:class:`Price`] """ raise NotImplementedError() def get_price(self, id): """ Get a price from an ID :param id: ID of price :type id: str :rtype: :class:`Price` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/radio.py000066400000000000000000000032751303450110500202060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField from weboob.tools.capabilities.streaminfo import StreamInfo __all__ = ['Radio', 'CapRadio'] class Radio(BaseObject): """ Radio object. """ title = StringField('Title of radio') description = StringField('Description of radio') current = Field('Current emission', StreamInfo) streams = Field('List of streams', list) class CapRadio(Capability): """ Capability of radio websites. """ def iter_radios_search(self, pattern): """ Search a radio. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Radio`] """ raise NotImplementedError() def get_radio(self, id): """ Get a radio from an ID. :param id: ID of radio :type id: str :rtype: :class:`Radio` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/recipe.py000066400000000000000000000151231303450110500203520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, IntField, Field, empty import lxml.etree as ET import base64 import re import urllib __all__ = ['Recipe', 'CapRecipe'] class Comment(): def __init__(self, author=None, rate=None, text=None): self.author = author self.rate = rate self.text = text def __str__(self): result = '' if self.author: result += 'author: %s, ' % self.author if self.rate: result += 'note: %s, ' % self.rate if self.text: result += 'comment: %s' % self.text return result class Recipe(BaseObject): """ Recipe object. """ title = StringField('Title of the recipe') author = StringField('Author name of the recipe') thumbnail_url = StringField('Direct url to recipe thumbnail') picture_url = StringField('Direct url to recipe picture') short_description = StringField('Short description of a recipe') nb_person = Field('The recipe was made for this amount of persons', list) preparation_time = IntField('Preparation time of the recipe in minutes') cooking_time = IntField('Cooking time of the recipe in minutes') ingredients = Field('Ingredient list necessary for the recipe', list) instructions = StringField('Instruction step list of the recipe') comments = Field('User comments about the recipe', list) def __init__(self, id='', title=u'', url=None): BaseObject.__init__(self, id, url) self.title = title def toKrecipesXml(self, author=None): """ Export recipe to KRecipes XML string """ sauthor = u'' if not empty(self.author): sauthor += '%s@' % self.author if author is None: sauthor += 'Cookboob' else: sauthor += author header = u'\n' initial_xml = '''\ ''' doc = ET.fromstring(initial_xml) recipe = doc.find('krecipes-recipe') desc = ET.SubElement(recipe, 'krecipes-description') title = ET.SubElement(desc, 'title') title.text = self.title authors = ET.SubElement(desc, 'author') authors.text = sauthor eyield = ET.SubElement(desc, 'yield') if not empty(self.nb_person): amount = ET.SubElement(eyield, 'amount') if len(self.nb_person) == 1: amount.text = '%s' % self.nb_person[0] else: mini = ET.SubElement(amount, 'min') mini.text = u'%s' % self.nb_person[0] maxi = ET.SubElement(amount, 'max') maxi.text = u'%s' % self.nb_person[1] etype = ET.SubElement(eyield, 'type') etype.text = 'persons' if not empty(self.preparation_time): preptime = ET.SubElement(desc, 'preparation-time') preptime.text = '%02d:%02d' % (self.preparation_time / 60, self.preparation_time % 60) if not empty(self.picture_url) and self.picture_url != '': data = urllib.urlopen(self.picture_url).read() datab64 = base64.encodestring(data)[:-1] pictures = ET.SubElement(desc, 'pictures') pic = ET.SubElement(pictures, 'pic', {'format': 'JPEG', 'id': '1'}) pic.text = ET.CDATA(datab64) if not empty(self.ingredients): ings = ET.SubElement(recipe, 'krecipes-ingredients') pat = re.compile('^[0-9,.]*') for i in self.ingredients: sname = u'%s' % i samount = '' sunit = '' first_nums = pat.match(i).group() if first_nums != '': samount = first_nums sname = i.lstrip('0123456789 ') ing = ET.SubElement(ings, 'ingredient') am = ET.SubElement(ing, 'amount') am.text = samount unit = ET.SubElement(ing, 'unit') unit.text = sunit name = ET.SubElement(ing, 'name') name.text = sname if not empty(self.instructions): instructions = ET.SubElement(recipe, 'krecipes-instructions') instructions.text = self.instructions if not empty(self.comments): ratings = ET.SubElement(recipe, 'krecipes-ratings') for c in self.comments: rating = ET.SubElement(ratings, 'rating') if c.author: rater = ET.SubElement(rating, 'rater') rater.text = c.author if c.text: com = ET.SubElement(rating, 'comment') com.text = c.text crits = ET.SubElement(rating, 'criterion') if c.rate: crit = ET.SubElement(crits, 'criteria') critname = ET.SubElement(crit, 'name') critname.text = 'Overall' critstars = ET.SubElement(crit, 'stars') critstars.text = c.rate.split('/')[0] return header + ET.tostring(doc, encoding='UTF-8', pretty_print=True).decode('utf-8') class CapRecipe(Capability): """ Recipe providers. """ def iter_recipes(self, pattern): """ Search recipes and iterate on results. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Recipe`] """ raise NotImplementedError() def get_recipe(self, _id): """ Get a recipe object from an ID. :param _id: ID of recipe :type _id: str :rtype: :class:`Recipe` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/shop.py000066400000000000000000000075201303450110500200560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import BaseObject, StringField, DecimalField, UserError from .date import DateField from .collection import CapCollection __all__ = ['OrderNotFound', 'Order', 'Payment', 'Item', 'CapShop'] class OrderNotFound(UserError): """ Raised when an order is not found. """ def __init__(self, msg='Order not found'): UserError.__init__(self, msg) class Order(BaseObject): """ Purchase order. """ date = DateField('Date when the order was placed') shipping = DecimalField('Shipping price') discount = DecimalField('Discounts') tax = DecimalField('Tax') total = DecimalField('Total') def __repr__(self): return "" % (self.id, self.date) class Payment(BaseObject): """ Payment for an order. """ date = DateField('The date when payment was applied') method = StringField('Payment method; e.g. "VISA 1234"') amount = DecimalField('Payment amount') def __repr__(self): return "" % \ (self.date, self.method, self.amount) class Item(BaseObject): """ Purchased item within an order. """ label = StringField('Item label') price = DecimalField('Item price') def __repr__(self): return "" % (self.label, self.price) class CapShop(CapCollection): """ Capability of online shops to see orders history. """ def iter_resources(self, objs, split_path): """ Iter resources. Default implementation of this method is to return on top-level all orders (by calling :func:`iter_accounts`). :param objs: type of objects to get :type objs: tuple[:class:`BaseObject`] :param split_path: path to discover :type split_path: :class:`list` :rtype: iter[:class:`BaseObject`] """ if Order in objs: self._restrict_level(split_path) return self.iter_orders() def get_currency(self): """ Get the currency this shop uses. :rtype: :class:`str` """ raise NotImplementedError() def iter_orders(self): """ Iter history of orders. :rtype: iter[:class:`Order`] """ raise NotImplementedError() def get_order(self, id): """ Get an order from its ID. :param id: ID of the order :type id: :class:`str` :rtype: :class:`Order` :raises: :class:`OrderNotFound` """ raise NotImplementedError() def iter_payments(self, order): """ Iter payments of a specific order. :param order: order to get payments :type order: :class:`Order` :rtype: iter[:class:`Payment`] :raises: :class:`OrderNotFound` """ raise NotImplementedError() def iter_items(self, order): """ Iter items of a specific order. :param order: order to get items :type order: :class:`Order` :rtype: iter[:class:`Item`] :raises: :class:`OrderNotFound` """ raise NotImplementedError() weboob-1.2/weboob/capabilities/subtitle.py000066400000000000000000000043511303450110500207370ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Julien Veyssier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, IntField, UserError __all__ = ['Subtitle', 'CapSubtitle'] class LanguageNotSupported(UserError): """ Raised when the language is not supported """ def __init__(self, msg='language is not supported'): UserError.__init__(self, msg) class Subtitle(BaseObject): """ Subtitle object. """ name = StringField('Name of subtitle') ext = StringField('Extension of file') nb_cd = IntField('Number of cd or files') language = StringField('Language of the subtitle') description=StringField('Description of corresponding video') def __init__(self, id, name, url=None): BaseObject.__init__(self, id, url) self.name = name class CapSubtitle(Capability): """ Subtitle providers. """ def iter_subtitles(self, pattern): """ Search subtitles and iterate on results. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Subtitle`] """ raise NotImplementedError() def get_subtitle(self, _id): """ Get a subtitle object from an ID. :param _id: ID of subtitle :type _id: str :rtype: :class:`Subtitle` """ raise NotImplementedError() def get_subtitle_file(self, _id): """ Get the content of the subtitle file. :param _id: ID of subtitle :type _id: str :rtype: str """ raise NotImplementedError() weboob-1.2/weboob/capabilities/torrent.py000066400000000000000000000046771303450110500206140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, Field, StringField, FloatField, \ IntField, UserError from .date import DateField __all__ = ['MagnetOnly', 'Torrent', 'CapTorrent'] class MagnetOnly(UserError): """ Raised when trying to get URL to torrent but only magnet is available. """ def __init__(self, magnet): self.magnet = magnet UserError.__init__(self, 'Only magnet URL is available') class Torrent(BaseObject): """ Torrent object. """ name = StringField('Name of torrent') size = FloatField('Size of torrent') date = DateField('Date when torrent has been published') magnet = StringField('URI of magnet') seeders = IntField('Number of seeders') leechers = IntField('Number of leechers') files = Field('Files in torrent', list) description = StringField('Description of torrent') filename = StringField('Name of .torrent file') class CapTorrent(Capability): """ Torrent trackers. """ def iter_torrents(self, pattern): """ Search torrents and iterate on results. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`Torrent`] """ raise NotImplementedError() def get_torrent(self, _id): """ Get a torrent object from an ID. :param _id: ID of torrent :type _id: str :rtype: :class:`Torrent` """ raise NotImplementedError() def get_torrent_file(self, _id): """ Get the content of the .torrent file. :param _id: ID of torrent :type _id: str :rtype: str """ raise NotImplementedError() weboob-1.2/weboob/capabilities/translate.py000066400000000000000000000037351303450110500211060ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Lucien Loiseau # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .base import Capability, BaseObject, StringField, UserError __all__ = ['TranslationFail', 'LanguageNotSupported', 'CapTranslate'] class LanguageNotSupported(UserError): """ Raised when the language is not supported """ def __init__(self, msg='language is not supported'): UserError.__init__(self, msg) class TranslationFail(UserError): """ Raised when no translation matches the given request """ def __init__(self, msg='No Translation Available'): UserError.__init__(self, msg) class Translation(BaseObject): """ Translation. """ lang_src = StringField('Source language') lang_dst = StringField('Destination language') text = StringField('Translation') class CapTranslate(Capability): """ Capability of online translation website to translate word or sentence """ def translate(self, source_language, destination_language, request): """ Perfom a translation. :param source_language: language in which the request is written :param destination_language: language to translate the request into :param request: the sentence to be translated :rtype: Translation """ raise NotImplementedError() weboob-1.2/weboob/capabilities/travel.py000066400000000000000000000103311303450110500203740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime from .base import Capability, BaseObject, StringField, DecimalField, UserError from .date import TimeField, DeltaField, DateField __all__ = ['Station', 'Departure', 'RoadStep', 'RoadmapError', 'RoadmapFilters', 'CapTravel'] class Station(BaseObject): """ Describes a station. """ name = StringField('Name of station') def __init__(self, id=None, name=None, url=None): BaseObject.__init__(self, id, url) self.name = name def __repr__(self): return "" % (self.id, self.name) class Departure(BaseObject): """ Describes a departure. """ type = StringField('Type of train') time = TimeField('Departure time') departure_station = StringField('Departure station') arrival_station = StringField('Destination of the train') arrival_time = TimeField('Arrival time') late = TimeField('Optional late', default=datetime.time()) information = StringField('Informations') plateform = StringField('Where the train will leave') price = DecimalField('Price of ticket') currency = StringField('Currency', default=None) def __init__(self, id=None, _type=None, _time=None, url=None): BaseObject.__init__(self, id, url) self.type = _type self.time = _time def __repr__(self): return "" % ( self.id, self.type, self.time.strftime('%H:%M'), self.departure_station, self.arrival_station) class RoadStep(BaseObject): """ A step on a roadmap. """ line = StringField('When line') start_time = TimeField('Start of step') end_time = TimeField('End of step') departure = StringField('Departure station') arrival = StringField('Arrival station') duration = DeltaField('Duration of this step') class RoadmapError(UserError): """ Raised when the roadmap is unable to be calculated. """ class RoadmapFilters(BaseObject): """ Filters to get a roadmap. """ departure_time = DateField('Wanted departure time') arrival_time = DateField('Wanted arrival time') def __init__(self, id='', url=None): BaseObject.__init__(self, id, url) class CapTravel(Capability): """ Travel websites. """ def iter_station_search(self, pattern): """ Iterates on search results of stations. :param pattern: the search pattern :type pattern: str :rtype: iter[:class:`Station`] """ raise NotImplementedError() def iter_station_departures(self, station_id, arrival_id=None, date=None): """ Iterate on departures. :param station_id: the station ID :type station_id: str :param arrival_id: optionnal arrival station ID :type arrival_id: str :param date: optional date :type date: datetime.datetime :rtype: iter[:class:`Departure`] """ raise NotImplementedError() def iter_roadmap(self, departure, arrival, filters): """ Get a roadmap. :param departure: name of departure station :type departure: str :param arrival: name of arrival station :type arrival: str :param filters: filters on search :type filters: :class:`RoadmapFilters` :rtype: iter[:class:`RoadStep`] """ raise NotImplementedError() weboob-1.2/weboob/capabilities/video.py000066400000000000000000000036031303450110500202110ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon, Christophe Benz # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import timedelta from .base import Field from .image import CapImage, BaseImage __all__ = ['BaseVideo', 'CapVideo'] class BaseVideo(BaseImage): """ Represents a video. This object has to be inherited to specify how to calculate the URL of the video from its ID. """ duration = Field('file duration', int, long, timedelta) class CapVideo(CapImage): """ Video file provider. """ def search_videos(self, pattern, sortby=CapImage.SEARCH_RELEVANCE, nsfw=False): """ search for a video file :param pattern: pattern to search on :type pattern: str :param sortby: sort by... (use SEARCH_* constants) :param nsfw: include non-suitable for work videos if True :type nsfw: bool :rtype: iter[:class:`BaseVideo`] """ return self.search_image(pattern, sortby, nsfw) def get_video(self, _id): """ Get a video file from an ID. :param _id: video file ID :type _id: str :rtype: :class:`BaseVideo` or None is fot found. """ return self.get_image(_id) weboob-1.2/weboob/capabilities/weather.py000066400000000000000000000101031303450110500205330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from datetime import datetime, date from .base import Capability, BaseObject, Field, FloatField, \ StringField, UserError, NotLoaded from .date import DateField __all__ = ['Forecast', 'Current', 'City', 'CityNotFound', 'Temperature', 'CapWeather'] class Temperature(BaseObject): value = FloatField('Temperature value') unit = StringField('Input unit') def __init__(self, value=NotLoaded, unit = u'', url=None): BaseObject.__init__(self, unicode(value), url) self.value = value if unit not in [u'C', u'F']: unit = u'' self.unit = unit def asfahrenheit(self): if not self.unit: return u'%s' % int(round(self.value)) elif self.unit == 'F': return u'%s °F' % int(round(self.value)) else: return u'%s °F' % int(round((self.value * 9.0 / 5.0) + 32)) def ascelsius(self): if not self.unit: return u'%s' % int(round(self.value)) elif self.unit == 'C': return u'%s °C' % int(round(self.value)) else: return u'%s °C' % int(round((self.value - 32.0) * 5.0 / 9.0)) def __repr__(self): if self.value is not None and self.unit: return '%r %r' % (self.value, self.unit) return '' class Forecast(BaseObject): """ Weather forecast. """ date = Field('Date for the forecast', datetime, date, basestring) low = Field('Low temperature', Temperature) high = Field('High temperature', Temperature) text = StringField('Comment on forecast') def __init__(self, date=NotLoaded, low=None, high=None, text=None, unit=None, url=None): BaseObject.__init__(self, unicode(date), url) self.date = date self.low = Temperature(low, unit) self.high = Temperature(high, unit) self.text = text class Current(BaseObject): """ Current weather. """ date = DateField('Date of measure') text = StringField('Comment about current weather') temp = Field('Current temperature', Temperature) def __init__(self, date=NotLoaded, temp=None, text=None, unit=None, url=None): BaseObject.__init__(self, unicode(date), url) self.date = date self.text = text self.temp = Temperature(temp, unit) class City(BaseObject): """ City where to find weather. """ name = StringField('Name of city') def __init__(self, id='', name=None, url=None): BaseObject.__init__(self, id, url) self.name = name class CityNotFound(UserError): """ Raised when a city is not found. """ class CapWeather(Capability): """ Capability for weather websites. """ def iter_city_search(self, pattern): """ Look for a city. :param pattern: pattern to search :type pattern: str :rtype: iter[:class:`City`] """ raise NotImplementedError() def get_current(self, city_id): """ Get current weather. :param city_id: ID of the city :rtype: :class:`Current` """ raise NotImplementedError() def iter_forecast(self, city_id): """ Iter forecasts of a city. :param city_id: ID of the city :rtype: iter[:class:`Forecast`] """ raise NotImplementedError() weboob-1.2/weboob/core/000077500000000000000000000000001303450110500150265ustar00rootroot00000000000000weboob-1.2/weboob/core/__init__.py000066400000000000000000000015231303450110500171400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .bcall import CallErrors from .ouiboube import Weboob, WebNip __all__ = ['CallErrors', 'Weboob', 'WebNip'] weboob-1.2/weboob/core/backendscfg.py000066400000000000000000000135431303450110500176400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import stat import os import sys try: from ConfigParser import RawConfigParser, DuplicateSectionError except ImportError: from configparser import RawConfigParser, DuplicateSectionError from logging import warning __all__ = ['BackendsConfig', 'BackendAlreadyExists'] class BackendAlreadyExists(Exception): pass class BackendsConfig(object): """ Config of backends. A backend is an instance of a module with a config. A module can thus have multiple instances. """ class WrongPermissions(Exception): pass def __init__(self, confpath): self.confpath = confpath try: mode = os.stat(confpath).st_mode except OSError: if not os.path.isdir(os.path.dirname(confpath)): os.makedirs(os.path.dirname(confpath)) if sys.platform == 'win32': fptr = open(confpath, 'w') fptr.close() else: try: fd = os.open(confpath, os.O_WRONLY | os.O_CREAT, 0o600) os.close(fd) except OSError: fptr = open(confpath, 'w') fptr.close() os.chmod(confpath, 0o600) else: if sys.platform != 'win32': if mode & stat.S_IRGRP or mode & stat.S_IROTH: raise self.WrongPermissions( u'Weboob will not start as long as config file %s is readable by group or other users.' % confpath) def iter_backends(self): """ Iterate on backends. :returns: each tuple contains the backend name, module name and module options :rtype: :class:`tuple` """ config = RawConfigParser() config.read(self.confpath) changed = False for backend_name in config.sections(): params = dict(config.items(backend_name)) try: module_name = params.pop('_module') except KeyError: try: module_name = params.pop('_backend') config.set(backend_name, '_module', module_name) config.remove_option(backend_name, '_backend') changed = True except KeyError: warning('Missing field "_module" for configured backend "%s"', backend_name) continue yield backend_name, module_name, params if changed: with open(self.confpath, 'wb') as f: config.write(f) def backend_exists(self, name): """ Return True if the backend exists in config. """ config = RawConfigParser() config.read(self.confpath) return name in config.sections() def add_backend(self, backend_name, module_name, params, edit=False): """ Add a backend to config. :param backend_name: name of the backend in config :param module_name: name of the Python submodule to run :param params: params to pass to the module :type params: :class:`dict` """ if not backend_name: raise ValueError(u'Please give a name to the configured backend.') config = RawConfigParser() config.read(self.confpath) if not edit: try: config.add_section(backend_name) except DuplicateSectionError: raise BackendAlreadyExists(backend_name) config.set(backend_name, '_module', module_name) for key, value in params.iteritems(): if isinstance(value, unicode): value = value.encode('utf-8') config.set(backend_name, key, value) with open(self.confpath, 'wb') as f: config.write(f) def edit_backend(self, backend_name, module_name, params): """Edit a backend from config.""" return self.add_backend(backend_name, module_name, params, True) def get_backend(self, backend_name): """ Get options of backend. :returns: a tuple with the module name and the module options dict :rtype: tuple """ config = RawConfigParser() config.read(self.confpath) if not config.has_section(backend_name): raise KeyError(u'Configured backend "%s" not found' % backend_name) items = dict(config.items(backend_name)) try: module_name = items.pop('_module') except KeyError: try: module_name = items.pop('_backend') self.edit_backend(backend_name, module_name, items) except KeyError: warning('Missing field "_module" for configured backend "%s"', backend_name) raise KeyError(u'Configured backend "%s" not found' % backend_name) return module_name, items def remove_backend(self, backend_name): """Remove a backend from config.""" config = RawConfigParser() config.read(self.confpath) if not config.remove_section(backend_name): return False with open(self.confpath, 'w') as f: config.write(f) return True weboob-1.2/weboob/core/bcall.py000066400000000000000000000140351303450110500164600ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from copy import copy from threading import Thread, Event try: import Queue except ImportError: import queue as Queue from weboob.capabilities.base import BaseObject from weboob.tools.misc import get_backtrace from weboob.tools.log import getLogger __all__ = ['BackendsCall', 'CallErrors'] class CallErrors(Exception): def __init__(self, errors): msg = 'Errors during backend calls:\n' + \ '\n'.join(['Module(%r): %r\n%r\n' % (backend, error, backtrace) for backend, error, backtrace in errors]) Exception.__init__(self, msg) self.errors = copy(errors) def __iter__(self): return self.errors.__iter__() class BackendsCall(object): def __init__(self, backends, function, *args, **kwargs): """ :param backends: List of backends to call :type backends: list[:class:`Module`] :param function: backends' method name, or callable object. :type function: :class:`str` or :class:`callable` """ self.logger = getLogger('bcall') self.responses = Queue.Queue() self.errors = [] self.tasks = Queue.Queue() self.stop_event = Event() self.threads = [] for backend in backends: t = Thread(target=self.backend_process, args=(function, args, kwargs)) t.start() self.threads.append(t) self.tasks.put(backend) def store_result(self, backend, result): """Store the result when a backend task finished.""" if result is None: return if isinstance(result, BaseObject): result.backend = backend.name self.responses.put(result) def backend_process(self, function, args, kwargs): """ Internal method to run a method of a backend. As this method may be blocking, it should be run on its own thread. """ backend = self.tasks.get() with backend: try: # Call method on backend try: self.logger.debug('%s: Calling function %s', backend, function) if callable(function): result = function(backend, *args, **kwargs) else: result = getattr(backend, function)(*args, **kwargs) except Exception as error: self.logger.debug('%s: Called function %s raised an error: %r', backend, function, error) self.errors.append((backend, error, get_backtrace(error))) else: self.logger.debug('%s: Called function %s returned: %r', backend, function, result) if hasattr(result, '__iter__') and not isinstance(result, basestring): # Loop on iterator try: for subresult in result: self.store_result(backend, subresult) if self.stop_event.is_set(): break except Exception as error: self.errors.append((backend, error, get_backtrace(error))) else: self.store_result(backend, result) finally: self.tasks.task_done() def _callback_thread_run(self, callback, errback, finishback): while not self.stop_event.is_set() and (self.tasks.unfinished_tasks or not self.responses.empty()): try: response = self.responses.get(timeout=0.1) except Queue.Empty: continue else: if callback: callback(response) # Raise errors while errback and self.errors: errback(*self.errors.pop(0)) if finishback: finishback() def callback_thread(self, callback, errback=None, finishback=None): """ Call this method to create a thread which will callback a specified function everytimes a new result comes. When the process is over, the function will be called with both arguments set to None. The functions prototypes: def callback(result) def errback(backend, error, backtrace) def finishback() """ thread = Thread(target=self._callback_thread_run, args=(callback, errback, finishback)) thread.start() return thread def wait(self): """Wait until all tasks are finished.""" for thread in self.threads: thread.join() if self.errors: raise CallErrors(self.errors) def stop(self, wait=False): """ Stop all tasks. :param wait: If True, wait until all tasks stopped. :type wait: bool """ self.stop_event.set() if wait: self.wait() def __iter__(self): try: while not self.stop_event.is_set() and (self.tasks.unfinished_tasks or not self.responses.empty()): try: yield self.responses.get(timeout=0.1) except Queue.Empty: continue except: self.stop() raise if self.errors: raise CallErrors(self.errors) weboob-1.2/weboob/core/modules.py000066400000000000000000000143341303450110500170550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os import imp import logging from weboob.tools.backend import Module from weboob.tools.log import getLogger from weboob.exceptions import ModuleLoadError __all__ = ['LoadedModule', 'ModulesLoader', 'RepositoryModulesLoader'] class LoadedModule(object): def __init__(self, package): self.logger = getLogger('backend') self.package = package self.klass = None for attrname in dir(self.package): attr = getattr(self.package, attrname) if isinstance(attr, type) and issubclass(attr, Module) and attr != Module: self.klass = attr if not self.klass: raise ImportError('%s is not a backend (no Module class found)' % package) @property def name(self): return self.klass.NAME @property def maintainer(self): return u'%s <%s>' % (self.klass.MAINTAINER, self.klass.EMAIL) @property def version(self): return self.klass.VERSION @property def description(self): return self.klass.DESCRIPTION @property def license(self): return self.klass.LICENSE @property def config(self): return self.klass.CONFIG @property def website(self): if self.klass.BROWSER and hasattr(self.klass.BROWSER, 'BASEURL') and self.klass.BROWSER.BASEURL: return self.klass.BROWSER.BASEURL if self.klass.BROWSER and hasattr(self.klass.BROWSER, 'DOMAIN') and self.klass.BROWSER.DOMAIN: return '%s://%s' % (self.klass.BROWSER.PROTOCOL, self.klass.BROWSER.DOMAIN) else: return None @property def icon(self): return self.klass.ICON def iter_caps(self): return self.klass.iter_caps() def has_caps(self, *caps): """Return True if module implements at least one of the caps.""" for c in caps: if (isinstance(c, basestring) and c in [cap.__name__ for cap in self.iter_caps()]) or \ (type(c) == type and issubclass(self.klass, c)): return True return False def create_instance(self, weboob, backend_name, config, storage, nofail=False): backend_instance = self.klass(weboob, backend_name, config, storage, self.logger, nofail) self.logger.debug(u'Created backend "%s" for module "%s"' % (backend_name, self.name)) return backend_instance class ModulesLoader(object): """ Load modules. """ def __init__(self, path, version=None): self.version = version self.path = path self.loaded = {} self.logger = getLogger('modules') def get_or_load_module(self, module_name): """ Can raise a ModuleLoadError exception. """ if module_name not in self.loaded: self.load_module(module_name) return self.loaded[module_name] def iter_existing_module_names(self): for name in os.listdir(self.path): try: if '__init__.py' in os.listdir(os.path.join(self.path, name)): yield name except OSError: # if path/name is not a directory continue def module_exists(self, name): for existing_module_name in self.iter_existing_module_names(): if existing_module_name == name: return True return False def load_all(self): for existing_module_name in self.iter_existing_module_names(): try: self.load_module(existing_module_name) except ModuleLoadError as e: self.logger.warning(e) def load_module(self, module_name): if module_name in self.loaded: self.logger.debug('Module "%s" is already loaded from %s' % (module_name, self.loaded[module_name].package.__path__[0])) return path = self.get_module_path(module_name) try: fp, pathname, description = imp.find_module(module_name, [path]) try: module = LoadedModule(imp.load_module(module_name, fp, pathname, description)) finally: if fp: fp.close() except Exception as e: if logging.root.level <= logging.DEBUG: self.logger.exception(e) raise ModuleLoadError(module_name, e) if module.version != self.version: raise ModuleLoadError(module_name, "Module requires Weboob %s, but you use Weboob %s. Hint: use 'weboob-config update'" % (module.version, self.version)) self.loaded[module_name] = module self.logger.debug('Loaded module "%s" from %s' % (module_name, module.package.__path__[0])) def get_module_path(self, module_name): return self.path class RepositoryModulesLoader(ModulesLoader): """ Load modules from repositories. """ def __init__(self, repositories): super(RepositoryModulesLoader, self).__init__(repositories.modules_dir, repositories.version) self.repositories = repositories def iter_existing_module_names(self): for name in self.repositories.get_all_modules_info().iterkeys(): yield name def get_module_path(self, module_name): minfo = self.repositories.get_module_info(module_name) if minfo is None: raise ModuleLoadError(module_name, 'No such module %s' % module_name) if minfo.path is None: raise ModuleLoadError(module_name, 'Module %s is not installed' % module_name) return minfo.path weboob-1.2/weboob/core/ouiboube.py000066400000000000000000000440571303450110500172230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os from weboob.core.bcall import BackendsCall from weboob.core.modules import ModulesLoader, RepositoryModulesLoader from weboob.core.backendscfg import BackendsConfig from weboob.core.requests import RequestsManager from weboob.core.repositories import Repositories, PrintProgress from weboob.core.scheduler import Scheduler from weboob.tools.backend import Module from weboob.tools.config.iconfig import ConfigError from weboob.tools.log import getLogger from weboob.exceptions import ModuleLoadError __all__ = ['WebNip', 'Weboob'] class VersionsMismatchError(ConfigError): pass class WebNip(object): """ Weboob in Non Integrated Programs It provides methods to build backends or call methods on all loaded backends. :param modules_path: path to directory containing modules. :type modules_path: :class:`basestring` :param storage: provide a storage where backends can save data :type storage: :class:`weboob.tools.storage.IStorage` :param scheduler: what scheduler to use; default is :class:`weboob.core.scheduler.Scheduler` :type scheduler: :class:`weboob.core.scheduler.IScheduler` """ VERSION = '1.2' def __init__(self, modules_path=None, storage=None, scheduler=None): self.logger = getLogger('weboob') self.backend_instances = {} self.requests = RequestsManager() if modules_path is None: import pkg_resources modules_path = pkg_resources.resource_filename('weboob_modules', '') if modules_path: self.modules_loader = ModulesLoader(modules_path, self.VERSION) if scheduler is None: scheduler = Scheduler() self.scheduler = scheduler self.storage = storage def __deinit__(self): self.deinit() def deinit(self): """ Call this method when you stop using Weboob, to properly unload all correctly. """ self.unload_backends() def build_backend(self, module_name, params=None, storage=None, name=None, nofail=False): """ Create a backend. It does not load it into the Weboob object, so you are responsible for deinitialization and calls. :param module_name: name of module :param params: parameters to give to backend :type params: :class:`dict` :param storage: storage to use :type storage: :class:`weboob.tools.storage.IStorage` :param name: name of backend :type name: :class:`basestring` :rtype: :class:`weboob.tools.backend.Module` :param nofail: if true, this call can't fail :type nofail: :class:`bool` """ module = self.modules_loader.get_or_load_module(module_name) backend_instance = module.create_instance(self, name or module_name, params or {}, storage, nofail) return backend_instance class LoadError(Exception): """ Raised when a backend is unabled to load. :param backend_name: name of backend we can't load :param exception: exception object """ def __init__(self, backend_name, exception): Exception.__init__(self, unicode(exception)) self.backend_name = backend_name def load_backend(self, module_name, name, params=None, storage=None): """ Load a backend. :param module_name: name of module to load :type module_name: :class:`basestring`: :param name: name of instance :type name: :class:`basestring` :param params: parameters to give to backend :type params: :class:`dict` :param storage: storage to use :type storage: :class:`weboob.tools.storage.IStorage` :rtype: :class:`weboob.tools.backend.Module` """ if name is None: name = module_name if name in self.backend_instances: raise self.LoadError(name, 'A loaded backend already named "%s"' % name) backend = self.build_backend(module_name, params, storage, name) self.backend_instances[name] = backend return backend def unload_backends(self, names=None): """ Unload backends. :param names: if specified, only unload that backends :type names: :class:`list` """ unloaded = {} if isinstance(names, basestring): names = [names] elif names is None: names = self.backend_instances.keys() for name in names: backend = self.backend_instances.pop(name) with backend: backend.deinit() unloaded[backend.name] = backend return unloaded def __getitem__(self, name): """ Alias for :func:`WebNip.get_backend`. """ return self.get_backend(name) def get_backend(self, name, **kwargs): """ Get a backend from its name. :param name: name of backend to get :type name: str :param default: if specified, get this value when the backend is not found :type default: whatever you want :raises: :class:`KeyError` if not found. """ try: return self.backend_instances[name] except KeyError: if 'default' in kwargs: return kwargs['default'] else: raise def count_backends(self): """ Get number of loaded backends. """ return len(self.backend_instances) def iter_backends(self, caps=None, module=None): """ Iter on each backends. Note: each backend is locked when it is returned. :param caps: optional list of capabilities to select backends :type caps: tuple[:class:`weboob.capabilities.base.Capability`] :param module: optional name of module :type module: :class:`basestring` :rtype: iter[:class:`weboob.tools.backend.Module`] """ for _, backend in sorted(self.backend_instances.iteritems()): if (caps is None or backend.has_caps(caps)) and \ (module is None or backend.NAME == module): with backend: yield backend def __getattr__(self, name): def caller(*args, **kwargs): return self.do(name, *args, **kwargs) return caller def do(self, function, *args, **kwargs): r""" Do calls on loaded backends with specified arguments, in separated threads. This function has two modes: - If *function* is a string, it calls the method with this name on each backends with the specified arguments; - If *function* is a callable, it calls it in a separated thread with the locked backend instance at first arguments, and \*args and \*\*kwargs. :param function: backend's method name, or a callable object :type function: :class:`str` :param backends: list of backends to iterate on :type backends: list[:class:`str`] :param caps: iterate on backends which implement this caps :type caps: list[:class:`weboob.capabilities.base.Capability`] :rtype: A :class:`weboob.core.bcall.BackendsCall` object (iterable) """ backends = self.backend_instances.values() _backends = kwargs.pop('backends', None) if _backends is not None: if isinstance(_backends, Module): backends = [_backends] elif isinstance(_backends, basestring): if len(_backends) > 0: try: backends = [self.backend_instances[_backends]] except (ValueError, KeyError): backends = [] elif isinstance(_backends, (list, tuple, set)): backends = [] for backend in _backends: if isinstance(backend, basestring): try: backends.append(self.backend_instances[backend]) except (ValueError, KeyError): pass else: backends.append(backend) else: self.logger.warning(u'The "backends" value isn\'t supported: %r', _backends) if 'caps' in kwargs: caps = kwargs.pop('caps') backends = [backend for backend in backends if backend.has_caps(caps)] # The return value MUST BE the BackendsCall instance. Please never iterate # here on this object, because caller might want to use other methods, like # wait() on callback_thread(). # Thanks a lot. return BackendsCall(backends, function, *args, **kwargs) def schedule(self, interval, function, *args): """ Schedule an event. :param interval: delay before calling the function :type interval: int :param function: function to call :type function: callabale :param args: arguments to give to function :returns: an event identificator """ return self.scheduler.schedule(interval, function, *args) def repeat(self, interval, function, *args): """ Repeat a call to a function :param interval: interval between two calls :type interval: int :param function: function to call :type function: callable :param args: arguments to give to function :returns: an event identificator """ return self.scheduler.repeat(interval, function, *args) def cancel(self, ev): """ Cancel an event :param ev: the event identificator """ return self.scheduler.cancel(ev) def want_stop(self): """ Plan to stop the scheduler. """ return self.scheduler.want_stop() def loop(self): """ Run the scheduler loop """ return self.scheduler.run() def load_or_install_module(self, module_name): """ Load a backend, but can't install it """ return self.modules_loader.get_or_load_module(module_name) class Weboob(WebNip): """ The main class of Weboob, used to manage backends, modules repositories and call methods on all loaded backends. :param workdir: optional parameter to set path of the working directory :type workdir: str :param datadir: optional parameter to set path of the data directory :type datadir: str :param backends_filename: name of the *backends* file, where configuration of backends is stored :type backends_filename: str :param storage: provide a storage where backends can save data :type storage: :class:`weboob.tools.storage.IStorage` """ BACKENDS_FILENAME = 'backends' def __init__(self, workdir=None, datadir=None, backends_filename=None, scheduler=None, storage=None): super(Weboob, self).__init__(modules_path=False, scheduler=scheduler, storage=storage) # Create WORKDIR if workdir is None: if 'WEBOOB_WORKDIR' in os.environ: workdir = os.environ['WEBOOB_WORKDIR'] else: workdir = os.path.join(os.environ.get('XDG_CONFIG_HOME', os.path.join(os.path.expanduser('~'), '.config')), 'weboob') self.workdir = os.path.realpath(workdir) self._create_dir(workdir) # Create DATADIR if datadir is None: if 'WEBOOB_DATADIR' in os.environ: datadir = os.environ['WEBOOB_DATADIR'] elif 'WEBOOB_WORKDIR' in os.environ: datadir = os.environ['WEBOOB_WORKDIR'] else: datadir = os.path.join(os.environ.get('XDG_DATA_HOME', os.path.join(os.path.expanduser('~'), '.local', 'share')), 'weboob') _datadir = os.path.realpath(datadir) self._create_dir(_datadir) # Modules management self.repositories = Repositories(workdir, _datadir, self.VERSION) self.modules_loader = RepositoryModulesLoader(self.repositories) # Backend instances config if not backends_filename: backends_filename = os.environ.get('WEBOOB_BACKENDS', os.path.join(self.workdir, self.BACKENDS_FILENAME)) elif not backends_filename.startswith('/'): backends_filename = os.path.join(self.workdir, backends_filename) self.backends_config = BackendsConfig(backends_filename) def _create_dir(self, name): if not os.path.exists(name): os.makedirs(name) elif not os.path.isdir(name): self.logger.error(u'"%s" is not a directory', name) def update(self, progress=PrintProgress()): """ Update modules from repositories. """ self.repositories.update(progress) modules_to_check = set([module_name for _, module_name, _ in self.backends_config.iter_backends()]) for module_name in modules_to_check: minfo = self.repositories.get_module_info(module_name) if minfo and not minfo.is_installed(): self.repositories.install(minfo, progress) def build_backend(self, module_name, params=None, storage=None, name=None, nofail=False): """ Create a single backend which is not listed in configuration. :param module_name: name of module :param params: parameters to give to backend :type params: :class:`dict` :param storage: storage to use :type storage: :class:`weboob.tools.storage.IStorage` :param name: name of backend :type name: :class:`basestring` :rtype: :class:`weboob.tools.backend.Module` :param nofail: if true, this call can't fail :type nofail: :class:`bool` """ minfo = self.repositories.get_module_info(module_name) if minfo is None: raise ModuleLoadError(module_name, 'Module does not exist.') if not minfo.is_installed(): self.repositories.install(minfo) return super(Weboob, self).build_backend(module_name, params, storage, name, nofail) def load_backends(self, caps=None, names=None, modules=None, exclude=None, storage=None, errors=None): """ Load backends listed in config file. :param caps: load backends which implement all of specified caps :type caps: tuple[:class:`weboob.capabilities.base.Capability`] :param names: load backends in list :type names: tuple[:class:`str`] :param modules: load backends which module is in list :type modules: tuple[:class:`str`] :param exclude: do not load backends in list :type exclude: tuple[:class:`str`] :param storage: use this storage if specified :type storage: :class:`weboob.tools.storage.IStorage` :param errors: if specified, store every errors in this list :type errors: list[:class:`LoadError`] :returns: loaded backends :rtype: dict[:class:`str`, :class:`weboob.tools.backend.Module`] """ loaded = {} if storage is None: storage = self.storage if not self.repositories.check_repositories(): self.logger.error(u'Repositories are not consistent with the sources.list') raise VersionsMismatchError(u'Versions mismatch, please run "weboob-config update"') for backend_name, module_name, params in self.backends_config.iter_backends(): if '_enabled' in params and not params['_enabled'].lower() in ('1', 'y', 'true', 'on', 'yes') or \ names is not None and backend_name not in names or \ modules is not None and module_name not in modules or \ exclude is not None and backend_name in exclude: continue minfo = self.repositories.get_module_info(module_name) if minfo is None: self.logger.warning(u'Backend "%s" is referenced in %s but was not found. ' u'Perhaps a missing repository or a removed module?', module_name, self.backends_config.confpath) continue if caps is not None and not minfo.has_caps(caps): continue if not minfo.is_installed(): self.repositories.install(minfo) module = None try: module = self.modules_loader.get_or_load_module(module_name) except ModuleLoadError as e: self.logger.error(u'Unable to load module "%s": %s', module_name, e) continue if backend_name in self.backend_instances: self.logger.warning(u'Oops, the backend "%s" is already loaded. Unload it before reloading...', backend_name) self.unload_backends(backend_name) try: backend_instance = module.create_instance(self, backend_name, params, storage) except Module.ConfigError as e: if errors is not None: errors.append(self.LoadError(backend_name, e)) else: self.backend_instances[backend_name] = loaded[backend_name] = backend_instance return loaded def load_or_install_module(self, module_name): """ Load a backend, and install it if not done before """ try: return self.modules_loader.get_or_load_module(module_name) except ModuleLoadError: self.repositories.install(module_name) return self.modules_loader.get_or_load_module(module_name) weboob-1.2/weboob/core/repositories.py000066400000000000000000000706001303450110500201320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import imp import posixpath import shutil import re import sys import os import subprocess import hashlib from datetime import datetime from contextlib import closing from compileall import compile_dir from io import BytesIO from weboob.exceptions import BrowserHTTPError, BrowserHTTPNotFound, ModuleInstallError from .modules import LoadedModule from weboob.tools.log import getLogger from weboob.tools.misc import get_backtrace, to_unicode, find_exe try: from ConfigParser import RawConfigParser, DEFAULTSECT except ImportError: from configparser import RawConfigParser, DEFAULTSECT class ModuleInfo(object): """ Information about a module available on a repository. """ def __init__(self, name): self.name = name # path to the local directory containing this module. self.path = None self.url = None self.repo_url = None self.version = 0 self.capabilities = () self.description = u'' self.maintainer = u'' self.license = u'' self.icon = u'' self.urls = u'' def load(self, items): self.version = int(items['version']) self.capabilities = items['capabilities'].split() self.description = to_unicode(items['description']) self.maintainer = to_unicode(items['maintainer']) self.license = to_unicode(items['license']) self.icon = items['icon'].strip() or None self.urls = items['urls'] def has_caps(self, *caps): """Return True if module implements at least one of the caps.""" if len(caps) == 1 and isinstance(caps[0], (list, tuple)): caps = caps[0] for c in caps: if type(c) == type: c = c.__name__ if c in self.capabilities: return True return False def is_installed(self): return self.path is not None def is_local(self): return self.url is None def dump(self): return (('version', self.version), ('capabilities', ' '.join(self.capabilities)), ('description', self.description), ('maintainer', self.maintainer), ('license', self.license), ('icon', self.icon or ''), ('urls', self.urls), ) class RepositoryUnavailable(Exception): """ Repository in not available. """ class Repository(object): """ Represents a repository. """ INDEX = 'modules.list' KEYDIR = '.keys' KEYRING = 'trusted.gpg' def __init__(self, url): self.url = url self.name = u'' self.update = 0 self.maintainer = u'' self.local = None self.signed = False self.key_update = 0 self.logger = getLogger('repository') self.modules = {} if self.url.startswith('file://'): self.local = True elif re.match('https?://.*', self.url): self.local = False else: # This is probably a file in ~/.weboob/repositories/, we # don't know if this is a local or a remote repository. with open(self.url, 'r') as fp: self.parse_index(fp) def __repr__(self): return '' % self.name def localurl2path(self): """ Get a local path of a file:// URL. """ assert self.local is True if self.url.startswith('file://'): return self.url[len('file://'):] return self.url def retrieve_index(self, browser, repo_path): """ Retrieve the index file of this repository. It can use network if this is a remote repository. :param repo_path: path to save the downloaded index file. :type repo_path: str """ if self.local: # Repository is local, open the file. filename = os.path.join(self.localurl2path(), self.INDEX) try: fp = open(filename, 'r') except IOError as e: # This local repository doesn't contain a built modules.list index. self.name = Repositories.url2filename(self.url) self.build_index(self.localurl2path(), filename) fp = open(filename, 'r') else: # This is a remote repository, download file try: fp = BytesIO(browser.open(posixpath.join(self.url, self.INDEX)).content) except BrowserHTTPError as e: raise RepositoryUnavailable(unicode(e)) self.parse_index(fp) if self.local: # Always rebuild index of a local repository. self.build_index(self.localurl2path(), filename) # Save the repository index in ~/.weboob/repositories/ self.save(repo_path, private=True) def retrieve_keyring(self, browser, keyring_path, progress): # ignore local if self.local: return keyring = Keyring(keyring_path) # prevent previously signed repos from going unsigned if not self.signed and keyring.exists(): raise RepositoryUnavailable('Previously signed repository can not go unsigned') if not self.signed: return if not keyring.exists() or self.key_update > keyring.version: # This is a remote repository, download file try: keyring_data = browser.open(posixpath.join(self.url, self.KEYRING)).content sig_data = browser.open(posixpath.join(self.url, self.KEYRING + '.sig')).content except BrowserHTTPError as e: raise RepositoryUnavailable(unicode(e)) if keyring.exists(): if not keyring.is_valid(keyring_data, sig_data): raise InvalidSignature('the keyring itself') progress.progress(0.0, 'The keyring was updated (and validated by the previous one).') elif not progress.prompt('The repository %s isn\'t trusted yet.\nFingerprint of keyring is %s\nAre you sure you want to continue?' % (self.url, hashlib.sha1(keyring_data).hexdigest())): raise RepositoryUnavailable('Repository not trusted') keyring.save(keyring_data, self.key_update) progress.progress(0.0, str(keyring)) def parse_index(self, fp): """ Parse index of a repository :param fp: file descriptor to read :type fp: buffer """ config = RawConfigParser() config.readfp(fp) # Read default parameters items = dict(config.items(DEFAULTSECT)) try: self.name = items['name'] self.update = int(items['update']) self.maintainer = items['maintainer'] self.signed = bool(int(items.get('signed', '0'))) self.key_update = int(items.get('key_update', '0')) except KeyError as e: raise RepositoryUnavailable('Missing global parameters in repository: %s' % e) except ValueError as e: raise RepositoryUnavailable('Incorrect value in repository parameters: %s' % e) if len(self.name) == 0: raise RepositoryUnavailable('Name is empty') if 'url' in items: self.url = items['url'] self.local = self.url.startswith('file://') elif self.local is None: raise RepositoryUnavailable('Missing "url" key in settings') # Load modules self.modules.clear() for section in config.sections(): module = ModuleInfo(section) module.load(dict(config.items(section))) if not self.local: module.url = posixpath.join(self.url, '%s.tar.gz' % module.name) module.repo_url = self.url module.signed = self.signed self.modules[section] = module def build_index(self, path, filename): """ Rebuild index of modules of repository. :param path: path of the repository :type path: str :param filename: file to save index :type filename: str """ print('Rebuild index') self.modules.clear() if os.path.isdir(os.path.join(path, self.KEYDIR)): self.signed = True self.key_update = self.get_tree_mtime(os.path.join(path, self.KEYDIR), True) else: self.signed = False self.key_update = 0 for name in sorted(os.listdir(path)): module_path = os.path.join(path, name) if not os.path.isdir(module_path) or '.' in name or name == self.KEYDIR: continue try: fp, pathname, description = imp.find_module(name, [path]) try: module = LoadedModule(imp.load_module(name, fp, pathname, description)) finally: if fp: fp.close() except Exception as e: print('Unable to build module %s: [%s] %s' % (name, type(e).__name__, e), file=sys.stderr) self.logger.debug(get_backtrace(e)) else: m = ModuleInfo(module.name) m.version = self.get_tree_mtime(module_path) m.capabilities = list(set([c.__name__ for c in module.iter_caps()])) m.description = module.description m.maintainer = module.maintainer m.license = module.license m.icon = module.icon or '' self.modules[module.name] = m self.update = int(datetime.now().strftime('%Y%m%d%H%M')) self.save(filename) @staticmethod def get_tree_mtime(path, include_root=False): mtime = 0 if include_root: mtime = int(datetime.fromtimestamp(os.path.getmtime(path)).strftime('%Y%m%d%H%M')) for root, dirs, files in os.walk(path): for f in files: if f.endswith('.pyc'): continue m = int(datetime.fromtimestamp(os.path.getmtime(os.path.join(root, f))).strftime('%Y%m%d%H%M')) mtime = max(mtime, m) return mtime def save(self, filename, private=False): """ Save repository into a file (modules.list for example). :param filename: path to file to save repository. :type filename: str :param private: if enabled, save URL of repository. :type private: bool """ config = RawConfigParser() config.set(DEFAULTSECT, 'name', self.name) config.set(DEFAULTSECT, 'update', self.update) config.set(DEFAULTSECT, 'maintainer', self.maintainer) config.set(DEFAULTSECT, 'signed', int(self.signed)) config.set(DEFAULTSECT, 'key_update', self.key_update) if private: config.set(DEFAULTSECT, 'url', self.url) for module in self.modules.itervalues(): config.add_section(module.name) for key, value in module.dump(): config.set(module.name, key, to_unicode(value).encode('utf-8')) with open(filename, 'wb') as f: config.write(f) class Versions(object): VERSIONS_LIST = 'versions.list' def __init__(self, path): self.path = path self.versions = {} try: with open(os.path.join(self.path, self.VERSIONS_LIST), 'r') as fp: config = RawConfigParser() config.readfp(fp) # Read default parameters for key, value in config.items(DEFAULTSECT): self.versions[key] = int(value) except IOError: pass def get(self, name): return self.versions.get(name, None) def set(self, name, version): self.versions[name] = int(version) self.save() def save(self): config = RawConfigParser() for name, version in self.versions.iteritems(): config.set(DEFAULTSECT, name, version) with open(os.path.join(self.path, self.VERSIONS_LIST), 'wb') as fp: config.write(fp) class IProgress(object): def progress(self, percent, message): raise NotImplementedError() def error(self, message): raise NotImplementedError() def prompt(self, message): raise NotImplementedError() def __repr__(self): return '<%s>' % self.__class__.__name__ class PrintProgress(IProgress): def progress(self, percent, message): print('=== [%3.0f%%] %s' % (percent*100, message)) def error(self, message): print('ERROR: %s' % message, file=sys.stderr) def prompt(self, message): print('%s (Y/n): *** ASSUMING YES ***' % message) return True DEFAULT_SOURCES_LIST = \ """# List of Weboob repositories # # The entries below override the entries above (with # backends of the same name). http://updates.weboob.org/%(version)s/main/ # DEVELOPMENT # If you want to hack on Weboob modules, you may add a # reference to sources, for example: #file:///home/rom1/src/weboob/modules/ """ class Repositories(object): SOURCES_LIST = 'sources.list' MODULES_DIR = 'modules' REPOS_DIR = 'repositories' KEYRINGS_DIR = 'keyrings' ICONS_DIR = 'icons' SHARE_DIRS = [MODULES_DIR, REPOS_DIR, KEYRINGS_DIR, ICONS_DIR] def __init__(self, workdir, datadir, version): self.logger = getLogger('repositories') self.version = version self.browser = None self.workdir = workdir self.datadir = datadir self.sources_list = os.path.join(self.workdir, self.SOURCES_LIST) self.modules_dir = os.path.join(self.datadir, self.MODULES_DIR, self.version) self.repos_dir = os.path.join(self.datadir, self.REPOS_DIR) self.keyrings_dir = os.path.join(self.datadir, self.KEYRINGS_DIR) self.icons_dir = os.path.join(self.datadir, self.ICONS_DIR) self.create_dir(self.datadir) self.create_dir(self.modules_dir) self.create_dir(self.repos_dir) self.create_dir(self.keyrings_dir) self.create_dir(self.icons_dir) self.versions = Versions(self.modules_dir) self.repositories = [] if not os.path.exists(self.sources_list): with open(self.sources_list, 'w') as f: f.write(DEFAULT_SOURCES_LIST) self.update() else: self.load() def load_browser(self): from weboob.browser.browsers import Browser from weboob.browser.profiles import Weboob as WeboobProfile class WeboobBrowser(Browser): PROFILE = WeboobProfile(self.version) if self.browser is None: self.browser = WeboobBrowser() def create_dir(self, name): if not os.path.exists(name): os.makedirs(name) elif not os.path.isdir(name): self.logger.error(u'"%s" is not a directory' % name) def _extend_module_info(self, repo, info): if repo.local: info.path = repo.localurl2path() elif self.versions.get(info.name) is not None: info.path = self.modules_dir return info def get_all_modules_info(self, caps=None): """ Get all ModuleInfo instances available. :param caps: filter on capabilities: :type caps: list[str] :rtype: dict[:class:`ModuleInfo`] """ modules = {} for repos in reversed(self.repositories): for name, info in repos.modules.iteritems(): if name not in modules and (not caps or info.has_caps(caps)): modules[name] = self._extend_module_info(repos, info) return modules def get_module_info(self, name): """ Get ModuleInfo object of a module. It tries all repositories from last to first, and set the 'path' attribute of ModuleInfo if it is installed. """ for repos in reversed(self.repositories): if name in repos.modules: m = repos.modules[name] self._extend_module_info(repos, m) return m return None def load(self): """ Load repositories from ~/.local/share/weboob/repositories/. """ self.repositories = [] for name in sorted(os.listdir(self.repos_dir)): path = os.path.join(self.repos_dir, name) try: repository = Repository(path) self.repositories.append(repository) except RepositoryUnavailable as e: print('Unable to load repository %s (%s), try to update repositories.' % (name, e), file=sys.stderr) def get_module_icon_path(self, module): return os.path.join(self.icons_dir, '%s.png' % module.name) def retrieve_icon(self, module): """ Retrieve the icon of a module and save it in ~/.local/share/weboob/icons/. """ self.load_browser() if not isinstance(module, ModuleInfo): module = self.get_module_info(module) dest_path = self.get_module_icon_path(module) icon_url = module.icon if not icon_url: if module.is_local(): icon_path = os.path.join(module.path, module.name, 'favicon.png') if module.path and os.path.exists(icon_path): shutil.copy(icon_path, dest_path) return else: icon_url = module.url.replace('.tar.gz', '.png') try: icon = self.browser.open(icon_url) except BrowserHTTPNotFound: pass # no icon, no problem else: with open(dest_path, 'wb') as fp: fp.write(icon.content) def _parse_source_list(self): l = [] with open(self.sources_list, 'r') as f: for line in f: line = line.strip() % {'version': self.version} m = re.match('(file|https?)://.*', line) if m: l.append(line) return l def update_repositories(self, progress=PrintProgress()): self.load_browser() """ Update list of repositories by downloading them and put them in ~/.local/share/weboob/repositories/. :param progress: observer object. :type progress: :class:`IProgress` """ self.repositories = [] for name in os.listdir(self.repos_dir): os.remove(os.path.join(self.repos_dir, name)) gpg_found = Keyring.find_gpg() or Keyring.find_gpgv() for line in self._parse_source_list(): progress.progress(0.0, 'Getting %s' % line) repository = Repository(line) filename = self.url2filename(repository.url) prio_filename = '%02d-%s' % (len(self.repositories), filename) repo_path = os.path.join(self.repos_dir, prio_filename) keyring_path = os.path.join(self.keyrings_dir, filename) try: repository.retrieve_index(self.browser, repo_path) if gpg_found: repository.retrieve_keyring(self.browser, keyring_path, progress) else: progress.error('Cannot find gpg or gpgv to check for repository authenticity.\n' 'You should install GPG for better security.') except RepositoryUnavailable as e: progress.error('Unable to load repository: %s' % e) else: self.repositories.append(repository) def check_repositories(self): """ Check if sources.list is consistent with repositories """ l = [] for line in self._parse_source_list(): repository = Repository(line) filename = self.url2filename(repository.url) prio_filename = '%02d-%s' % (len(l), filename) repo_path = os.path.join(self.repos_dir, prio_filename) if not os.path.isfile(repo_path): return False l.append(repository) return True def update(self, progress=PrintProgress()): """ Update repositories and install new packages versions. :param progress: observer object. :type progress: :class:`IProgress` """ self.update_repositories(progress) to_update = [] for name, info in self.get_all_modules_info().iteritems(): if not info.is_local() and info.is_installed(): if self.versions.get(name) != info.version: to_update.append(info) if len(to_update) == 0: progress.progress(1.0, 'All modules are up-to-date.') return class InstallProgress(PrintProgress): def __init__(self, n): self.n = n def progress(self, percent, message): progress.progress(float(self.n)/len(to_update) + 1.0/len(to_update)*percent, message) for n, info in enumerate(to_update): inst_progress = InstallProgress(n) try: self.install(info, inst_progress) except ModuleInstallError as e: inst_progress.progress(1.0, unicode(e)) def install(self, module, progress=PrintProgress()): """ Install a module. :param module: module to install :type module: :class:`str` or :class:`ModuleInfo` :param progress: observer object :type progress: :class:`IProgress` """ import tarfile self.load_browser() if isinstance(module, ModuleInfo): info = module elif isinstance(module, basestring): progress.progress(0.0, 'Looking for module %s' % module) info = self.get_module_info(module) if not info: raise ModuleInstallError('Module "%s" does not exist' % module) else: raise ValueError('"module" parameter might be a ModuleInfo object or a string, not %r' % module) module = info if module.is_local(): raise ModuleInstallError('%s is available on local.' % module.name) module_dir = os.path.join(self.modules_dir, module.name) installed = self.versions.get(module.name) if installed is None or not os.path.exists(module_dir): progress.progress(0.3, 'Module %s is not installed yet' % module.name) elif module.version > installed: progress.progress(0.3, 'A new version of %s is available' % module.name) else: raise ModuleInstallError('The latest version of %s is already installed' % module.name) progress.progress(0.2, 'Downloading module...') try: tardata = self.browser.open(module.url).content except BrowserHTTPError as e: raise ModuleInstallError('Unable to fetch module: %s' % e) # Check signature if module.signed and (Keyring.find_gpg() or Keyring.find_gpgv()): progress.progress(0.5, 'Checking module authenticity...') sig_data = self.browser.open(posixpath.join(module.url + '.sig')).content keyring_path = os.path.join(self.keyrings_dir, self.url2filename(module.repo_url)) keyring = Keyring(keyring_path) if not keyring.exists(): raise ModuleInstallError('No keyring found, please update repos.') if not keyring.is_valid(tardata, sig_data): raise ModuleInstallError('Invalid signature for %s.' % module.name) # Extract module from tarball. if os.path.isdir(module_dir): shutil.rmtree(module_dir) progress.progress(0.7, 'Setting up module...') with closing(tarfile.open('', 'r:gz', BytesIO(tardata))) as tar: tar.extractall(self.modules_dir) if not os.path.isdir(module_dir): raise ModuleInstallError('The archive for %s looks invalid.' % module.name) # Precompile compile_dir(module_dir, quiet=True) self.versions.set(module.name, module.version) progress.progress(0.9, 'Downloading icon...') self.retrieve_icon(module) progress.progress(1.0, 'Module %s has been installed!' % module.name) @staticmethod def url2filename(url): """ Get a safe file name for an URL. All non-alphanumeric characters are replaced by _. """ return ''.join([l if l.isalnum() else '_' for l in url]) class InvalidSignature(Exception): def __init__(self, filename): self.filename = filename Exception.__init__(self, 'Invalid signature for %s' % filename) class Keyring(object): EXTENSION = '.gpg' def __init__(self, path): self.path = path + self.EXTENSION self.vpath = path + '.version' self.version = 0 if self.exists(): with open(self.vpath, 'r') as f: self.version = int(f.read().strip()) else: if os.path.exists(self.path): os.remove(self.path) if os.path.exists(self.vpath): os.remove(self.vpath) def exists(self): if not os.path.exists(self.vpath): return False if os.path.exists(self.path): # Check the file is not empty. # This is because there was a bug creating empty keyring files. with open(self.path, 'r') as fp: if len(fp.read().strip()): return True return False def save(self, keyring_data, version): with open(self.path, 'wb') as fp: fp.write(keyring_data) self.version = version with open(self.vpath, 'wb') as fp: fp.write(str(version)) @staticmethod def find_gpgv(): return find_exe('gpgv2') or find_exe('gpgv') @staticmethod def find_gpg(): return find_exe('gpg2') or find_exe('gpg') def is_valid(self, data, sigdata): """ Check if the data is signed by an accepted key. data and sigdata should be strings. """ gpg = self.find_gpg() gpgv = self.find_gpgv() if gpg: from tempfile import mkdtemp gpg_homedir = mkdtemp(prefix='weboob_gpg_') verify_command = [gpg, '--verify', '--no-options', '--no-default-keyring', '--quiet', '--homedir', gpg_homedir] elif gpgv: verify_command = [gpgv] from tempfile import NamedTemporaryFile with NamedTemporaryFile(suffix='.sig', delete=False) as sigfile: temp_filename = sigfile.name return_code = None out = '' err = '' try: sigfile.write(sigdata) sigfile.flush() # very important sigfile.close() assert isinstance(data, basestring) # Yes, all of it is necessary proc = subprocess.Popen(verify_command + [ '--status-fd', '1', '--keyring', os.path.realpath(self.path), os.path.realpath(sigfile.name), '-'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = proc.communicate(data) return_code = proc.returncode finally: os.unlink(temp_filename) if gpg: shutil.rmtree(gpg_homedir) if return_code or 'GOODSIG' not in out or 'VALIDSIG' not in out: print(out, err, file=sys.stderr) return False return True def __str__(self): if self.exists(): with open(self.path, 'r') as f: h = hashlib.sha1(f.read()).hexdigest() return 'Keyring version %s, checksum %s' % (self.version, h) return 'NO KEYRING' weboob-1.2/weboob/core/requests.py000066400000000000000000000022541303450110500172560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from threading import RLock from collections import defaultdict __all__ = ['RequestsManager'] class RequestsManager(object): def __init__(self): self.callbacks = defaultdict(lambda: lambda *args, **kwargs: None) self.lock = RLock() def request(self, name, *args, **kwargs): with self.lock: return self.callbacks[name](*args, **kwargs) def register(self, name, callback): self.callbacks[name] = callback weboob-1.2/weboob/core/scheduler.py000066400000000000000000000122041303450110500173550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from threading import Event, RLock try: from threading import _Timer as Timer except ImportError: from threading import Timer from weboob.tools.log import getLogger from weboob.tools.misc import get_backtrace __all__ = ['Scheduler'] class IScheduler(object): """Interface of a scheduler.""" def schedule(self, interval, function, *args): """ Schedule an event. :param interval: delay before calling the function :type interval: int :param function: function to call :type function: callabale :param args: arguments to give to function :returns: an event identificator """ raise NotImplementedError() def repeat(self, interval, function, *args): """ Repeat a call to a function :param interval: interval between two calls :type interval: int :param function: function to call :type function: callable :param args: arguments to give to function :returns: an event identificator """ raise NotImplementedError() def cancel(self, ev): """ Cancel an event :param ev: the event identificator """ raise NotImplementedError() def run(self): """ Run the scheduler loop """ raise NotImplementedError() def want_stop(self): """ Plan to stop the scheduler. """ raise NotImplementedError() class RepeatedTimer(Timer): def run(self): while not self.finished.isSet(): try: self.function(*self.args, **self.kwargs) except Exception: # do not stop timer because of an exception print(get_backtrace()) self.finished.wait(self.interval) self.finished.set() class Scheduler(IScheduler): """Scheduler using Python's :mod:`threading`.""" def __init__(self): self.logger = getLogger('scheduler') self.mutex = RLock() self.stop_event = Event() self.count = 0 self.queue = {} def schedule(self, interval, function, *args): return self._schedule(Timer, interval, self._schedule_callback, function, *args) def repeat(self, interval, function, *args): return self._schedule(RepeatedTimer, interval, self._repeat_callback, function, *args) def _schedule(self, klass, interval, meta_func, function, *args): if self.stop_event.isSet(): return with self.mutex: self.count += 1 self.logger.debug('function "%s" will be called in %s seconds' % (function.__name__, interval)) timer = klass(interval, meta_func, (self.count, interval, function, args)) self.queue[self.count] = timer timer.start() return self.count def _schedule_callback(self, count, interval, function, args): with self.mutex: self.queue.pop(count) return function(*args) def _repeat_callback(self, count, interval, function, args): function(*args) with self.mutex: try: e = self.queue[count] except KeyError: return else: self.logger.debug('function "%s" will be called in %s seconds' % (function.__name__, e.interval)) def cancel(self, ev): with self.mutex: try: e = self.queue.pop(ev) except KeyError: return False e.cancel() self.logger.debug('scheduled function "%s" is canceled' % e.function.__name__) return True def _wait_to_stop(self): self.want_stop() with self.mutex: for e in self.queue.itervalues(): e.cancel() e.join() self.queue = {} def run(self): try: while True: self.stop_event.wait(0.1) except KeyboardInterrupt: self._wait_to_stop() raise else: self._wait_to_stop() return True def want_stop(self): self.stop_event.set() with self.mutex: for t in self.queue.itervalues(): t.cancel() # Contrary to _wait_to_stop(), don't call t.join # because want_stop() have to be non-blocking. self.queue = {} weboob-1.2/weboob/deprecated/000077500000000000000000000000001303450110500161765ustar00rootroot00000000000000weboob-1.2/weboob/deprecated/__init__.py000066400000000000000000000000001303450110500202750ustar00rootroot00000000000000weboob-1.2/weboob/deprecated/browser/000077500000000000000000000000001303450110500176615ustar00rootroot00000000000000weboob-1.2/weboob/deprecated/browser/__init__.py000066400000000000000000000027441303450110500220010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser.browser import BrowserIncorrectPassword, BrowserBanned, \ BrowserUnavailable, BrowserRetry, \ BrowserHTTPNotFound, BrowserHTTPError, \ Page, Browser, BrokenPageError, \ StandardBrowser, BrowserPasswordExpired, \ BrowserForbidden, StateBrowser __all__ = ['BrowserIncorrectPassword', 'BrowserPasswordExpired', 'BrowserBanned', 'BrowserUnavailable', 'BrowserRetry', 'BrowserHTTPNotFound', 'BrowserHTTPError', 'Page', 'Browser', 'BrokenPageError', 'StandardBrowser', 'BrowserForbidden', 'StateBrowser'] weboob-1.2/weboob/deprecated/browser/browser.py000066400000000000000000000663231303450110500217300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import sys if sys.version_info >= (3,0): raise ImportError("This module isn't compatible with python3") from copy import copy from httplib import BadStatusLine try: import mechanize except ImportError: raise ImportError('Please install python-mechanize') import pickle import base64 import zlib import os import re from threading import RLock import ssl import httplib import socket import hashlib import time import urllib import urllib2 import mimetypes import logging from contextlib import closing from gzip import GzipFile import warnings from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword, BrowserPasswordExpired, BrowserForbidden, BrowserBanned, BrowserHTTPNotFound, BrowserHTTPError, FormFieldConversionWarning, BrowserSSLError from weboob.tools.decorators import retry from weboob.tools.log import getLogger from weboob.deprecated.mech import ClientForm ControlNotFoundError = ClientForm.ControlNotFoundError from weboob.deprecated.browser.parsers import get_parser __all__ = ['BrowserIncorrectPassword', 'BrowserForbidden', 'BrowserBanned', 'BrowserUnavailable', 'BrowserRetry', 'BrowserPasswordExpired', 'BrowserHTTPNotFound', 'BrowserHTTPError', 'BrokenPageError', 'Page', 'StandardBrowser', 'Browser', 'StateBrowser'] class BrowserRetry(Exception): pass class NoHistory(object): """ We don't want to fill memory with history """ def __init__(self): pass def add(self, request, response): pass def back(self, n, _response): pass def clear(self): pass def close(self): pass class BrokenPageError(Exception): pass class Page(object): """ Base page """ ENCODING = None def __init__(self, browser, document, url='', groups=None, group_dict=None, logger=None): self.browser = browser self.parser = browser.parser self.document = document self.url = url self.groups = groups self.group_dict = group_dict self.logger = getLogger('page', logger) def on_loaded(self): """ Called when the page is loaded. """ pass def check_location(func): def inner(self, *args, **kwargs): if args and isinstance(args[0], basestring): url = args[0] if url.startswith('/') and hasattr(self, 'DOMAIN') and (not self.request or self.request.host != self.DOMAIN): url = '%s://%s%s' % (self.PROTOCOL, self.DOMAIN, url) url = re.sub('(.*)#.*', r'\1', url) if isinstance(url, unicode): url = url.encode('utf-8') args = (url,) + args[1:] return func(self, *args, **kwargs) return inner class StandardBrowser(mechanize.Browser): """ Standard Browser. :param firefox_cookies: path to cookies sqlite file :type firefox_cookies: str :param parser: parser to use on HTML files :type parser: :class:`weboob.deprecated.browser.parsers.iparser.IParser` :param history: history manager; default value is an object which does not keep history :type history: object :param proxy: proxy URL to use :type proxy: str :param factory: mechanize factory. None to use Mechanize's default :type factory: object """ # ------ Class attributes -------------------------------------- ENCODING = 'utf-8' USER_AGENTS = { 'desktop_firefox': 'Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0', 'android': 'Mozilla/5.0 (Linux; U; Android 2.1; en-us; Nexus One Build/ERD62) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Mobile Safari/530.17', 'microb': 'Mozilla/5.0 (X11; U; Linux armv7l; fr-FR; rv:1.9.2.3pre) Gecko/20100723 Firefox/3.5 Maemo Browser 1.7.4.8 RX-51 N900', 'wget': 'Wget/1.11.4', } USER_AGENT = USER_AGENTS['desktop_firefox'] DEBUG_HTTP = False DEBUG_MECHANIZE = False DEFAULT_TIMEOUT = 15 INSECURE = False # if True, do not validate SSL logger = None # ------ Browser methods --------------------------------------- # I'm not a robot, so disable the check of permissions in robots.txt. default_features = copy(mechanize.Browser.default_features) default_features.remove('_robots') default_features.remove('_refresh') def __init__(self, firefox_cookies=None, parser=None, history=NoHistory(), proxy=None, logger=None, factory=None, responses_dirname=None): mechanize.Browser.__init__(self, history=history, factory=factory) self.logger = getLogger('browser', logger) self.responses_dirname = responses_dirname self.save_responses = responses_dirname is not None self.responses_count = 0 self.addheaders = [ ['User-agent', self.USER_AGENT] ] # Use a proxy self.proxy = proxy if proxy is not None: self.set_proxies(proxy) # Share cookies with firefox if firefox_cookies: # Try to load cookies try: from .firefox_cookies import FirefoxCookieJar self._cookie = FirefoxCookieJar(self.DOMAIN, firefox_cookies) self._cookie.load() self.set_cookiejar(self._cookie) except ImportError as e: logging.warning("Unable to store Firefox cookies: %s", e) self._cookie = None else: self._cookie = None if parser is None: parser = get_parser()() elif isinstance(parser, (tuple,list,basestring)): parser = get_parser(parser)() self.parser = parser self.lock = RLock() if self.DEBUG_HTTP: # display messages from httplib self.set_debug_http(True) if logging.root.level <= logging.DEBUG: # Enable log messages from mechanize.Browser self.set_debug_redirects(True) mech_logger = logging.getLogger("mechanize") mech_logger.setLevel(logging.INFO) def __enter__(self): self.lock.acquire() def __exit__(self, t, v, tb): self.lock.release() def _openurl(self, *args, **kwargs): return mechanize.Browser.open(self, *args, **kwargs) @check_location @retry(BrowserHTTPError, tries=3) def openurl(self, *args, **kwargs): """ Open an URL but do not create a Page object. """ if_fail = kwargs.pop('if_fail', 'raise') self.logger.debug('Opening URL "%s", %s' % (args, kwargs)) kwargs['timeout'] = kwargs.get('timeout', self.DEFAULT_TIMEOUT) try: return self._openurl(*args, **kwargs) except (mechanize.BrowserStateError, mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError, BadStatusLine, ssl.SSLError) as e: if isinstance(e, mechanize.BrowserStateError) and hasattr(self, 'home'): self.home() return self._openurl(*args, **kwargs) elif if_fail == 'raise': raise self.get_exception(e)('%s (url="%s")' % (e, args and args[0] or 'None')) else: return None except BrowserRetry as e: return self._openurl(*args, **kwargs) def get_exception(self, e): if isinstance(e, urllib2.HTTPError) and hasattr(e, 'getcode'): if e.getcode() in (404, 403): return BrowserHTTPNotFound if e.getcode() == 401: return BrowserIncorrectPassword elif isinstance(e, mechanize.BrowserStateError): return BrowserHTTPNotFound return BrowserHTTPError def readurl(self, url, *args, **kwargs): """ Download URL data specifying what to do on failure (nothing by default). """ if 'if_fail' not in kwargs: kwargs['if_fail'] = None result = self.openurl(url, *args, **kwargs) if result: if self.save_responses: self.save_response(result) return result.read() else: return None def save_response(self, result, warning=False): """ Save a stream to a temporary file, and log its name. The stream is rewinded after saving. """ if self.responses_dirname is None: import tempfile self.responses_dirname = tempfile.mkdtemp(prefix='weboob_session_') print('Debug data will be saved in this directory: %s' % self.responses_dirname, file=sys.stderr) elif not os.path.isdir(self.responses_dirname): os.makedirs(self.responses_dirname) # get the content-type, remove optionnal charset part mimetype = result.info().get('Content-Type', '').split(';')[0] # due to http://bugs.python.org/issue1043134 if mimetype == 'text/plain': ext = '.txt' else: # try to get an extension (and avoid adding 'None') ext = mimetypes.guess_extension(mimetype, False) or '' response_filepath = os.path.join(self.responses_dirname, unicode(self.responses_count)+ext) with open(response_filepath, 'w') as f: f.write(result.read()) result.seek(0) match_filepath = os.path.join(self.responses_dirname, 'url_response_match.txt') with open(match_filepath, 'a') as f: f.write('%s\t%s\n' % (result.geturl(), os.path.basename(response_filepath))) self.responses_count += 1 msg = u'Response saved to %s' % response_filepath if warning: self.logger.warning(msg) else: self.logger.info(msg) def get_document(self, result, parser=None, encoding=None): """ Get a parsed document from a stream. :param result: HTML page stream :type result: stream """ if parser is None: parser = self.parser elif isinstance(parser, (basestring, list, tuple)): parser = get_parser(parser)() if encoding is None: encoding = self.ENCODING return parser.parse(result, encoding) def location(self, *args, **kwargs): """ Go on an URL and get the related document. """ return self.get_document(self.openurl(*args, **kwargs)) @staticmethod def buildurl(base, *args, **kwargs): """ Build an URL and escape arguments. You can give a serie of tuples in args (and the order is keept), or a dict in kwargs (but the order is lost). Example: >>> StandardBrowser.buildurl('/blah.php', ('a', '&'), ('b', '=')) '/blah.php?a=%26&b=%3D' >>> StandardBrowser.buildurl('/blah.php', a='&', b='=') '/blah.php?b=%3D&a=%26' """ if not args: args = kwargs if not args: return base else: return '%s?%s' % (base, urllib.urlencode(args)) def str(self, s): if isinstance(s, unicode): s = s.encode('iso-8859-15', 'replace') return s def set_field(self, args, label, field=None, value=None, is_list=False): """ Set a value to a form field. :param args: arguments where to look for value :type args: dict :param label: label in args :type label: str :param field: field name. If None, use label instead :type field: str :param value: value to give on field :type value: str :param is_list: the field is a list :type is_list: bool """ try: if not field: field = label if args.get(label, None) is not None: if not value: if is_list: if isinstance(is_list, (list, tuple)): try: value = [self.str(is_list.index(args[label]))] except ValueError as e: if args[label]: print('[%s] %s: %s' % (label, args[label], e), file=sys.stderr) return else: value = [self.str(args[label])] else: value = self.str(args[label]) self[field] = value except ControlNotFoundError: return def lowsslcheck(self, domain, hsh): if self.INSECURE or (self.logger is not None and self.logger.settings['ssl_insecure']) or self.proxy is not None: return certhash = self._certhash(domain) if self.logger: self.logger.debug('Found %s as certificate hash' % certhash) if isinstance(hsh, basestring): hsh = [hsh] if certhash not in hsh: raise BrowserSSLError() def _certhash(self, domain, port=443): for proto in HTTPSConnection2._PROTOCOLS: try: certs = ssl.get_server_certificate((domain, port), ssl_version=proto) except ssl.SSLError as e: continue else: return hashlib.sha256(certs).hexdigest() raise e def __setitem__(self, key, value): if isinstance(value, unicode): value = value.encode(self.ENCODING or 'utf-8') warnings.warn('Implicit conversion of form field %r from unicode to str' % key, FormFieldConversionWarning, stacklevel=2) if self.form is None: raise AttributeError('Please select a form before setting values to fields') return self.form.__setitem__(key, value) class Browser(StandardBrowser): """ Base browser class to navigate on a website. :param username: username on website :type username: str :param password: password on website. If it is None, Browser will not try to login :type password: str :param firefox_cookies: path to cookies sqlite file :type firefox_cookies: str :param parser: parser to use on HTML files :type parser: :class:`weboob.deprecated.browser.parsers.iparser.IParser` :param history: history manager; default value is an object which does not keep history :type history: object :param proxy: proxy URL to use :type proxy: dictionnary :param logger: logger to use for logging :type logger: :class:`logging.Logger` :param factory: mechanize factory. None to use Mechanize's default :type factory: object :param get_home: try to get the homepage. :type get_homme: bool :param responses_dirname: directory to store responses :type responses_dirname: str """ # ------ Class attributes -------------------------------------- DOMAIN = None PROTOCOL = 'http' PAGES = {} # SHA-256 hash of server certificate. If set, it will automatically check it, # and raise a SSLError exception if it doesn't match. CERTHASH = None # ------ Abstract methods -------------------------------------- def home(self): """ Go to the home page. """ if self.DOMAIN is not None: self.location('%s://%s/' % (self.PROTOCOL, self.DOMAIN)) def login(self): """ Login to the website. This function is called when is_logged() returns False and the password attribute is not None. """ raise NotImplementedError() def is_logged(self): """ Return True if we are logged on website. When Browser tries to access to a page, if this method returns False, it calls login(). It is never called if the password attribute is None. """ raise NotImplementedError() # ------ Browser methods --------------------------------------- def __init__(self, username=None, password=None, firefox_cookies=None, parser=None, history=NoHistory(), proxy=None, logger=None, factory=None, get_home=True, responses_dirname=None): StandardBrowser.__init__(self, firefox_cookies, parser, history, proxy, logger, factory, responses_dirname) self.page = None self.last_update = 0.0 self.username = username self.password = password if self.CERTHASH is not None and self.DOMAIN is not None: self.lowsslcheck(self.DOMAIN, self.CERTHASH) if self.password and get_home: try: self.home() # Do not abort the build of browser when the website is down. except BrowserUnavailable: pass def submit(self, *args, **kwargs): """ Submit the selected form. """ no_login = kwargs.pop('nologin', kwargs.pop('no_login', False)) try: self._change_location(mechanize.Browser.submit(self, *args, **kwargs), no_login=no_login) except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError, BadStatusLine, ssl.SSLError) as e: self.page = None raise self.get_exception(e)(e) except (mechanize.BrowserStateError, BrowserRetry) as e: raise BrowserUnavailable(e) def is_on_page(self, pageCls): """ Check the current page. :param pageCls: class of the page to check :type pageCls: :class:`Page` :rtype: bool """ return isinstance(self.page, pageCls) def absurl(self, rel): """ Get an absolute URL from a relative one. """ if rel is None: return None if not rel.startswith('/'): rel = '/' + rel return '%s://%s%s' % (self.PROTOCOL, self.DOMAIN, rel) def follow_link(self, *args, **kwargs): """ Follow a link on the page. """ try: self._change_location(mechanize.Browser.follow_link(self, *args, **kwargs)) except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError, BadStatusLine, ssl.SSLError) as e: self.page = None raise self.get_exception(e)('%s (url="%s")' % (e, args and args[0] or 'None')) except (mechanize.BrowserStateError, BrowserRetry) as e: self.home() raise BrowserUnavailable(e) def _openurl(self, *args, **kwargs): return mechanize.Browser.open_novisit(self, *args, **kwargs) @check_location @retry(BrowserHTTPError, tries=3) def location(self, *args, **kwargs): """ Change location of browser on an URL. When the page is loaded, it looks up PAGES to find a regexp which matches, and create the object. Then, the 'on_loaded' method of this object is called. If a password is set, and is_logged() returns False, it tries to login with login() and reload the page. """ keep_args = copy(args) keep_kwargs = kwargs.copy() no_login = kwargs.pop('no_login', kwargs.pop('nologin', False)) kwargs['timeout'] = kwargs.get('timeout', self.DEFAULT_TIMEOUT) try: self._change_location(mechanize.Browser.open(self, *args, **kwargs), no_login=no_login) except BrowserRetry: if not self.page or not args or self.page.url != args[0]: keep_kwargs['no_login'] = True self.location(*keep_args, **keep_kwargs) except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError, BadStatusLine, ssl.SSLError) as e: self.page = None raise self.get_exception(e)('%s (url="%s")' % (e, args and args[0] or 'None')) except mechanize.BrowserStateError: self.home() self.location(*keep_args, **keep_kwargs) # DO NOT ENABLE THIS FUCKING PEACE OF CODE EVEN IF IT WOULD BE BETTER # TO SANITARIZE FUCKING HTML. #def _set_response(self, response, *args, **kwargs): # import time # if response and hasattr(response, 'set_data'): # print time.time() # r = response.read() # start = 0 # end = 0 # new = '' # lowr = r.lower() # start = lowr[end:].find('= end: # start_stop = start + lowr[start:].find('>') + 1 # new += r[end:start_stop] # end = start + lowr[start:].find('') # new += r[start_stop:end].replace('<', '<').replace('>', '>') # start = end + lowr[end:].find(' 3600: raise DNSTimeoutException() return res except (KeyError, DNSTimeoutException): res = socket.getaddrinfoold(*args) cacheDNS[args] = res, time.time() return res socket.getaddrinfoold = socket.getaddrinfo socket.getaddrinfo = my_getaddrinfo class HTTPSConnection2(httplib.HTTPSConnection): _HOSTS = {} _PROTOCOLS = [getattr(ssl, pn) for pn in ['PROTOCOL_TLSv1_2', 'PROTOCOL_TLSv1_1', 'PROTOCOL_TLSv1', 'PROTOCOL_SSLv23'] if hasattr(ssl, pn)] def _my_create_connection(self): sock = socket.create_connection((self.host, self.port), self.timeout) if self._tunnel_host: self._tunnel() return sock def _get_protocols(self): return self._HOSTS.get('%s:%s' % (self.host, self.port), self._PROTOCOLS) def connect(self): for proto in self._get_protocols(): sock = self._my_create_connection() try: self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, ssl_version=proto) self._HOSTS['%s:%s' % (self.host, self.port)] = [proto] return except ssl.SSLError as e: sock.close() raise e httplib.HTTPSConnection = HTTPSConnection2 weboob-1.2/weboob/deprecated/browser/decorators.py000066400000000000000000000047731303450110500224130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['check_url', 'id2url'] from urlparse import urlsplit import re class check_url(object): """ Checks if the first argument matches the given regular expression (given as str, without the ^$ delimiters which are automatically added). If not, this decorator will return None instead of calling the function. """ def __init__(self, regexp): self.regexp = re.compile('^%s$' % regexp) def __call__(self, func): def wrapper(funcself, *args, **kwargs): if self.regexp.match(args[0]): return func(funcself, *args, **kwargs) return None return wrapper def id2url(id2url): """ If the first argument is not an URL, this decorator will try to convert it to one, by calling the id2url function. If id2url returns None (because the id is invalid), the decorated function will not be called and None will be returned. If the DOMAIN attribute of the method's class is not empty, it will also check it. If it does not match, the decorated function will not be called and None will be returned. """ def wrapper(func): def inner(self, *args, **kwargs): arg = unicode(args[0]) if arg.startswith('http://') or arg.startswith('https://'): domain = urlsplit(arg).netloc if not self.DOMAIN or self.DOMAIN == domain or domain.endswith('.'+self.DOMAIN): url = arg else: return None else: url = id2url(arg) if url is None: return None new_args = [url] new_args.extend(args[1:]) return func(self, *new_args, **kwargs) return inner return wrapper weboob-1.2/weboob/deprecated/browser/firefox_cookies.py000066400000000000000000000074171303450110500234220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function try: import sqlite3 as sqlite except ImportError as e: from pysqlite2 import dbapi2 as sqlite from mechanize import CookieJar, Cookie __all__ = ['FirefoxCookieJar'] class FirefoxCookieJar(CookieJar): def __init__(self, domain, sqlite_file=None, policy=None): CookieJar.__init__(self, policy) self.domain = domain self.sqlite_file = sqlite_file def __connect(self): try: db = sqlite.connect(database=self.sqlite_file, timeout=10.0) except sqlite.OperationalError as err: print('Unable to open %s database: %s' % (self.sqlite_file, err)) return None return db def load(self): db = self.__connect() if not db: return cookies = db.execute("""SELECT host, path, name, value, expiry, lastAccessed, isSecure FROM moz_cookies WHERE host LIKE '%%%s%%'""" % self.domain) for entry in cookies: domain = entry[0] initial_dot = domain.startswith(".") domain_specified = initial_dot path = entry[1] name = entry[2] value = entry[3] expires = entry[4] secure = entry[6] discard = False c = Cookie(0, name, value, None, False, domain, domain_specified, initial_dot, path, False, secure, expires, discard, None, None, {}) #if not ignore_discard and c.discard: # continue #if not ignore_expires and c.is_expired(now): # continue self.set_cookie(c) def save(self): db = self.__connect() if not db: return db.execute("DELETE FROM moz_cookies WHERE host LIKE '%%%s%%'" % self.domain) for cookie in self: if cookie.secure: secure = 1 else: secure = 0 if cookie.expires is not None: expires = cookie.expires else: expires = 0 if cookie.value is None: # cookies.txt regards 'Set-Cookie: foo' as a cookie # with no name, whereas cookielib regards it as a # cookie with no value. name = "" value = cookie.name else: name = cookie.name value = cookie.value # XXX ugly hack to keep this cookie if name == 'PHPSESSID': expires = 1854242393 db.execute("""INSERT INTO moz_cookies (host, path, name, value, expiry, isSecure) VALUES (?, ?, ?, ?, ?, ?)""", (cookie.domain, cookie.path, name, value, int(expires), int(secure))) db.commit() weboob-1.2/weboob/deprecated/browser/parsers/000077500000000000000000000000001303450110500213405ustar00rootroot00000000000000weboob-1.2/weboob/deprecated/browser/parsers/__init__.py000066400000000000000000000043251303450110500234550ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import logging __all__ = ['get_parser', 'NoParserFound'] class NoParserFound(Exception): pass def load_lxml(): from .lxmlparser import LxmlHtmlParser return LxmlHtmlParser def load_lxmlsoup(): from .lxmlsoupparser import LxmlSoupParser return LxmlSoupParser def load_xml(): from .lxmlparser import LxmlXmlParser return LxmlXmlParser def load_json(): # This parser doesn't read HTML, don't include it in the # preference_order default value below. from .jsonparser import JsonParser return JsonParser def load_csv(): # This parser doesn't read HTML, don't include it in the # preference_order default value below. from .csvparser import CsvParser return CsvParser def load_raw(): # This parser doesn't read HTML, don't include it in the # preference_order default value below. from .iparser import RawParser return RawParser def get_parser(preference_order=('lxml', 'lxmlsoup')): """ Get a parser from a preference order list. Return a parser implementing IParser. """ if not isinstance(preference_order, (tuple, list)): preference_order = [preference_order] for kind in preference_order: if not 'load_%s' % kind in globals(): continue try: return globals()['load_%s' % kind]() except ImportError: logging.debug('%s is not installed.' % kind) raise NoParserFound("No parser found (%s)" % ','.join(preference_order)) weboob-1.2/weboob/deprecated/browser/parsers/csvparser.py000066400000000000000000000051441303450110500237260ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import csv from weboob.tools.log import getLogger from .iparser import IParser class Csv(object): """ CSV parser result. header contains the first row if it is a header rows contains the raw rows drows contains the rows with cells indexed by header title """ def __init__(self): self.header = None self.rows = [] self.drows = [] class CsvParser(IParser): """ CSV Parser. Since CSV files are not normalized, this parser is intended to be derived. """ DIALECT = 'excel' FMTPARAMS = {} """ If True, will consider the first line as a header. This means the rows will be also available as dictionnaries. """ HEADER = False def parse(self, data, encoding=None): reader = csv.reader(data, dialect=self.DIALECT, **self.FMTPARAMS) c = Csv() try: for row in reader: row = self.decode_row(row, encoding) if c.header is None and self.HEADER: c.header = row else: c.rows.append(row) if c.header: drow = {} for i, cell in enumerate(row): drow[c.header[i]] = cell c.drows.append(drow) except csv.Error as error: # If there are errors in CSV, for example the file is truncated, do # not crash as there already are lines parsed. logger = getLogger('csv') logger.warning('Error during parse of CSV: %s', error) return c def decode_row(self, row, encoding): if encoding: return [unicode(cell, encoding) for cell in row] else: return row def tostring(self, element): if not isinstance(element, basestring): return unicode(element) return element weboob-1.2/weboob/deprecated/browser/parsers/iparser.py000066400000000000000000000032561303450110500233650ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re __all__ = ['IParser', 'RawParser'] class IParser(object): def parse(self, data, encoding=None): """ Parse a HTML document with a specific encoding to get a tree. @param data [str] HTML document @param encoding [str] encoding to use @return an object with the structured document """ raise NotImplementedError() def tostring(self, elem): """ Get HTML string from an element. """ raise NotImplementedError() def tocleanstring(self, elem): """ Get a clean string from an element. """ return self.strip(self.tostring(elem)) def strip(self, data): """ Strip a HTML string. """ p = re.compile(r'<.*?>') return p.sub(' ', data).strip() class RawParser(IParser): def parse(self, data, encoding=None): return data.read() def tostring(self, elem): return elem weboob-1.2/weboob/deprecated/browser/parsers/jsonparser.py000066400000000000000000000020261303450110500241000ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.json import json from .iparser import IParser __all__ = ['JsonParser'] class JsonParser(IParser): """ Json parser. """ def parse(self, data, encoding=None): return json.load(data, encoding=encoding) def tostring(self, element): return json.dumps(element) weboob-1.2/weboob/deprecated/browser/parsers/lxmlparser.py000066400000000000000000000103041303450110500241010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import lxml.html as html import lxml.etree as etree from .iparser import IParser from ..browser import BrokenPageError __all__ = ['LxmlHtmlParser', 'LxmlXmlParser'] class LxmlParser(IParser): """ Parser using lxml. Note that it is not available on every systems. """ def get_parser(encoding=None): pass def parse(self, data, encoding=None): if encoding is None: parser = None else: parser = self.get_parser(encoding=encoding) return self.module.parse(data, parser) def tostring(self, element): return self.module.tostring(element, encoding=unicode) def tocleanstring(self, element): txt = [txt.strip() for txt in element.itertext()] txt = u' '.join(txt) # 'foo bar' txt = re.sub('\s+', ' ', txt) # 'foo bar' return txt.strip() def strip(self, s): doc = self.module.fromstring(s) # parse html/xml string return self.tocleanstring(doc) @classmethod def select(cls, element, selector, nb=None, method='cssselect', **kwargs): """ Select one or many elements from an element, using lxml cssselect by default. Raises :class:`weboob.deprecated.browser.browser.BrokenPageError` if not found. :param element: element on which to apply selector :type element: object :param selector: CSS or XPath expression :type selector: str :param method: (cssselect|xpath) :type method: str :param nb: number of elements expected to be found. Use None for undefined number, and 'many' for 1 to infinite :type nb: :class:`int` or :class:`str` :rtype: Element """ if method == 'cssselect': results = element.cssselect(selector, **kwargs) elif method == 'xpath': results = element.xpath(selector, **kwargs) else: raise NotImplementedError('Only the cssselect and xpath methods are supported') if nb is None: return results elif isinstance(nb, basestring) and nb == 'many': if results is None or len(results) == 0: raise BrokenPageError('Element not found with selector "%s"' % selector) elif len(results) == 1: raise BrokenPageError('Only one element found with selector "%s"' % selector) else: return results elif isinstance(nb, int) and nb > 0: if results is None: raise BrokenPageError('Element not found with selector "%s"' % selector) elif len(results) < nb: raise BrokenPageError('Not enough elements found (%d expected) with selector "%s"' % (nb, selector)) else: return results[0] if nb == 1 else results else: raise Exception('Unhandled value for kwarg "nb": %s' % nb) class LxmlHtmlParser(LxmlParser): """ Parser using lxml. Note that it is not available on every systems. """ def __init__(self, *args, **kwargs): self.module = html def get_parser(self, encoding=None): return html.HTMLParser(encoding=encoding) class LxmlXmlParser(LxmlParser): """ Parser using lxml. Note that it is not available on every systems. """ def __init__(self, *args, **kwargs): self.module = etree def get_parser(self, encoding=None): return etree.XMLParser(encoding=encoding, strip_cdata=False) weboob-1.2/weboob/deprecated/browser/parsers/lxmlsoupparser.py000066400000000000000000000022171303450110500250140ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import lxml.html import lxml.html.soupparser from .iparser import IParser __all__ = ['LxmlSoupParser'] class LxmlSoupParser(IParser): """ Parser using lxml elementsoup. Note that it is not available on every systems. """ def parse(self, data, encoding=None): return lxml.html.soupparser.parse(data) def tostring(self, element): return lxml.html.tostring(element, encoding=unicode) weboob-1.2/weboob/deprecated/mech.py000066400000000000000000000015711303450110500174700ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['ClientForm', 'mechanize'] import mechanize if hasattr(mechanize, "FormParser"): ClientForm = mechanize else: import ClientForm weboob-1.2/weboob/exceptions.py000066400000000000000000000035041303450110500166330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . class BrowserIncorrectPassword(Exception): pass class BrowserForbidden(Exception): pass class BrowserBanned(BrowserIncorrectPassword): pass class BrowserPasswordExpired(BrowserIncorrectPassword): pass class BrowserUnavailable(Exception): pass class BrowserQuestion(BrowserIncorrectPassword): """ When raised by a browser, """ def __init__(self, *fields): self.fields = fields class BrowserHTTPNotFound(BrowserUnavailable): pass class BrowserHTTPError(BrowserUnavailable): pass class BrowserHTTPSDowngrade(BrowserUnavailable): pass class BrowserSSLError(BrowserUnavailable): pass class ParseError(Exception): pass class FormFieldConversionWarning(UserWarning): """ A value has been set to a form's field and has been implicitly converted. """ class NoAccountsException(Exception): pass class ModuleInstallError(Exception): pass class ModuleLoadError(Exception): def __init__(self, module_name, msg): Exception.__init__(self, msg) self.module = module_name class ActionNeeded(Exception): pass weboob-1.2/weboob/tools/000077500000000000000000000000001303450110500152365ustar00rootroot00000000000000weboob-1.2/weboob/tools/__init__.py000066400000000000000000000000001303450110500173350ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/000077500000000000000000000000001303450110500175415ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/__init__.py000066400000000000000000000000001303450110500216400ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/base.py000066400000000000000000000407501303450110500210330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon, Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import logging import optparse from optparse import OptionGroup, OptionParser from datetime import datetime import os import sys import warnings from weboob.capabilities.base import ConversionWarning, BaseObject from weboob.core import Weboob, CallErrors from weboob.core.backendscfg import BackendsConfig from weboob.tools.config.iconfig import ConfigError from weboob.exceptions import FormFieldConversionWarning from weboob.tools.log import createColoredFormatter, getLogger, DEBUG_FILTERS, settings as log_settings from weboob.tools.misc import to_unicode, guess_encoding from .results import ResultsConditionError __all__ = ['Application'] class MoreResultsAvailable(Exception): pass class ApplicationStorage(object): def __init__(self, name, storage): self.name = name self.storage = storage def set(self, *args): if self.storage: return self.storage.set('applications', self.name, *args) def delete(self, *args): if self.storage: return self.storage.delete('applications', self.name, *args) def get(self, *args, **kwargs): if self.storage: return self.storage.get('applications', self.name, *args, **kwargs) else: return kwargs.get('default', None) def load(self, default): if self.storage: return self.storage.load('applications', self.name, default) def save(self): if self.storage: return self.storage.save('applications', self.name) class Application(object): """ Base application. This class can be herited to have some common code within weboob applications. """ # ------ Class attributes -------------------------------------- # Application name APPNAME = '' # Configuration and work directory (if None, use the Weboob instance one) CONFDIR = None # Default configuration dict (can only contain key/values) CONFIG = {} # Default storage tree STORAGE = {} # Synopsis SYNOPSIS = 'Usage: %prog [-h] [-dqv] [-b backends] ...\n' SYNOPSIS += ' %prog [--help] [--version]' # Description DESCRIPTION = None # Version VERSION = None # Copyright COPYRIGHT = None # Verbosity of DEBUG DEBUG_FILTER = 2 stdin = sys.stdin stdout = sys.stdout stderr = sys.stderr # ------ Abstract methods -------------------------------------- def create_weboob(self): return Weboob() def _get_completions(self): """ Overload this method in subclasses if you want to enrich shell completion. @return a set object """ return set() def _handle_options(self): """ Overload this method in application type subclass if you want to handle options defined in subclass constructor. """ pass def add_application_options(self, group): """ Overload this method if your application needs extra options. These options will be displayed in an option group. """ pass def handle_application_options(self): """ Overload this method in your application if you want to handle options defined in add_application_options. """ pass # ------ Application methods ------------------------------- def __init__(self, option_parser=None): self.encoding = self.guess_encoding() self.logger = getLogger(self.APPNAME) self.weboob = self.create_weboob() if self.CONFDIR is None: self.CONFDIR = self.weboob.workdir self.config = None self.options = None self.condition = None self.storage = None if option_parser is None: self._parser = OptionParser(self.SYNOPSIS, version=self._get_optparse_version()) else: self._parser = option_parser if self.DESCRIPTION: self._parser.description = self.DESCRIPTION app_options = OptionGroup(self._parser, '%s Options' % self.APPNAME.capitalize()) self.add_application_options(app_options) if len(app_options.option_list) > 0: self._parser.add_option_group(app_options) self._parser.add_option('-b', '--backends', help='what backend(s) to enable (comma separated)') self._parser.add_option('-e', '--exclude-backends', help='what backend(s) to exclude (comma separated)') self._parser.add_option('-I', '--insecure', action='store_true', help='do not validate SSL') logging_options = OptionGroup(self._parser, 'Logging Options') logging_options.add_option('-d', '--debug', action='count', help='display debug messages. Set up it twice to more verbosity') logging_options.add_option('-q', '--quiet', action='store_true', help='display only error messages') logging_options.add_option('-v', '--verbose', action='store_true', help='display info messages') logging_options.add_option('--logging-file', action='store', type='string', dest='logging_file', help='file to save logs') logging_options.add_option('-a', '--save-responses', action='store_true', help='save every response') self._parser.add_option_group(logging_options) self._parser.add_option('--shell-completion', action='store_true', help=optparse.SUPPRESS_HELP) self._is_default_count = True def guess_encoding(self, stdio=None): return guess_encoding(stdio or self.stdout) def deinit(self): self.weboob.want_stop() self.weboob.deinit() def create_storage(self, path=None, klass=None, localonly=False): """ Create a storage object. :param path: An optional specific path :type path: :class:`str` :param klass: What class to instance :type klass: :class:`weboob.tools.storage.IStorage` :param localonly: If True, do not set it on the :class:`Weboob` object. :type localonly: :class:`bool` :rtype: :class:`weboob.tools.storage.IStorage` """ if klass is None: from weboob.tools.storage import StandardStorage klass = StandardStorage if path is None: path = os.path.join(self.CONFDIR, self.APPNAME + '.storage') elif os.path.sep not in path: path = os.path.join(self.CONFDIR, path) storage = klass(path) self.storage = ApplicationStorage(self.APPNAME, storage) self.storage.load(self.STORAGE) if not localonly: self.weboob.storage = storage return storage def load_config(self, path=None, klass=None): """ Load a configuration file and get his object. :param path: An optional specific path :type path: :class:`str` :param klass: What class to instance :type klass: :class:`weboob.tools.config.iconfig.IConfig` :rtype: :class:`weboob.tools.config.iconfig.IConfig` """ if klass is None: from weboob.tools.config.iniconfig import INIConfig klass = INIConfig if path is None: path = os.path.join(self.CONFDIR, self.APPNAME) elif os.path.sep not in path: path = os.path.join(self.CONFDIR, path) self.config = klass(path) self.config.load(self.CONFIG) def main(self, argv): """ Main method Called by run """ raise NotImplementedError() def load_backends(self, caps=None, names=None, exclude=None, *args, **kwargs): if names is None and self.options.backends: names = self.options.backends.split(',') if exclude is None and self.options.exclude_backends: exclude = self.options.exclude_backends.split(',') loaded = self.weboob.load_backends(caps, names, exclude=exclude, *args, **kwargs) if not loaded: logging.info(u'No backend loaded') return loaded def _get_optparse_version(self): version = None if self.VERSION: if self.COPYRIGHT: copyright = self.COPYRIGHT.replace('YEAR', '%d' % datetime.today().year) version = '%s v%s %s' % (self.APPNAME, self.VERSION, copyright) else: version = '%s v%s' % (self.APPNAME, self.VERSION) return version def _do_complete_obj(self, backend, fields, obj): if not obj: return obj if not isinstance(obj, BaseObject): return obj obj.backend = backend.name if fields is None or len(fields) > 0: backend.fillobj(obj, fields) return obj def _do_complete_iter(self, backend, count, fields, res): modif = 0 for i, sub in enumerate(res): sub = self._do_complete_obj(backend, fields, sub) if self.condition and self.condition.limit and \ self.condition.limit == i: return if self.condition and not self.condition.is_valid(sub): modif += 1 else: if count and i - modif == count: if self._is_default_count: raise MoreResultsAvailable() else: return yield sub def _do_complete(self, backend, count, selected_fields, function, *args, **kwargs): assert count is None or count > 0 if callable(function): res = function(backend, *args, **kwargs) else: res = getattr(backend, function)(*args, **kwargs) if hasattr(res, '__iter__'): return self._do_complete_iter(backend, count, selected_fields, res) else: return self._do_complete_obj(backend, selected_fields, res) def bcall_error_handler(self, backend, error, backtrace): """ Handler for an exception inside the CallErrors exception. This method can be overrided to support more exceptions types. """ # Ignore this error. if isinstance(error, MoreResultsAvailable): return False print(u'Error(%s): %s' % (backend.name, error), file=self.stderr) if logging.root.level <= logging.DEBUG: print(backtrace, file=self.stderr) else: return True def bcall_errors_handler(self, errors, debugmsg='Use --debug option to print backtraces', ignore=()): """ Handler for the CallErrors exception. It calls `bcall_error_handler` for each error. :param errors: Object containing errors from backends :type errors: :class:`weboob.core.bcall.CallErrors` :param debugmsg: Default message asking to enable the debug mode :type debugmsg: :class:`basestring` :param ignore: Exceptions to ignore :type ignore: tuple[:class:`Exception`] """ ask_debug_mode = False for backend, error, backtrace in errors.errors: if isinstance(error, ignore): continue elif self.bcall_error_handler(backend, error, backtrace): ask_debug_mode = True if ask_debug_mode: print(debugmsg, file=self.stderr) def parse_args(self, args): self.options, args = self._parser.parse_args(args) if self.options.shell_completion: items = set() for option in self._parser.option_list: if option.help is not optparse.SUPPRESS_HELP: items.update(str(option).split('/')) items.update(self._get_completions()) print(' '.join(items)) sys.exit(0) if self.options.debug >= self.DEBUG_FILTER: level = DEBUG_FILTERS elif self.options.debug or self.options.save_responses: level = logging.DEBUG elif self.options.verbose: level = logging.INFO elif self.options.quiet: level = logging.ERROR else: level = logging.WARNING if self.options.insecure: log_settings['ssl_insecure'] = True # this only matters to developers if not self.options.debug and not self.options.save_responses: warnings.simplefilter('ignore', category=ConversionWarning) warnings.simplefilter('ignore', category=FormFieldConversionWarning) handlers = [] if self.options.save_responses: import tempfile responses_dirname = tempfile.mkdtemp(prefix='weboob_session_') print('Debug data will be saved in this directory: %s' % responses_dirname, file=self.stderr) log_settings['responses_dirname'] = responses_dirname handlers.append(self.create_logging_file_handler(os.path.join(responses_dirname, 'debug.log'))) # file logger if self.options.logging_file: handlers.append(self.create_logging_file_handler(self.options.logging_file)) else: handlers.append(self.create_default_logger()) self.setup_logging(level, handlers) self._handle_options() self.handle_application_options() return args @classmethod def create_default_logger(cls): # stderr logger format = '%(asctime)s:%(levelname)s:%(name)s:' + cls.VERSION +\ ':%(filename)s:%(lineno)d:%(funcName)s %(message)s' handler = logging.StreamHandler(cls.stderr) handler.setFormatter(createColoredFormatter(cls.stderr, format)) return handler @classmethod def setup_logging(cls, level, handlers): logging.root.handlers = [] logging.root.setLevel(level) for handler in handlers: logging.root.addHandler(handler) def create_logging_file_handler(self, filename): try: stream = open(filename, 'w') except IOError as e: self.logger.error('Unable to create the logging file: %s' % e) sys.exit(1) else: format = '%(asctime)s:%(levelname)s:%(name)s:' + self.VERSION +\ ':%(filename)s:%(lineno)d:%(funcName)s %(message)s' handler = logging.StreamHandler(stream) handler.setFormatter(logging.Formatter(format)) return handler @classmethod def run(cls, args=None): """ This static method can be called to run the application. It creates the application object, handles options, setups logging, calls the main() method, and catches common exceptions. You can't do anything after this call, as it *always* finishes with a call to sys.exit(). For example: >>> from weboob.application.myapplication import MyApplication >>> MyApplication.run() """ cls.setup_logging(logging.INFO, [cls.create_default_logger()]) if args is None: args = [(cls.stdin.encoding and isinstance(arg, bytes) and arg.decode(cls.stdin.encoding) or to_unicode(arg)) for arg in sys.argv] try: app = cls() except BackendsConfig.WrongPermissions as e: print(e, file=cls.stderr) sys.exit(1) try: try: args = app.parse_args(args) sys.exit(app.main(args)) except KeyboardInterrupt: print('Program killed by SIGINT', file=cls.stderr) sys.exit(0) except EOFError: sys.exit(0) except ConfigError as e: print('Configuration error: %s' % e, file=cls.stderr) sys.exit(1) except CallErrors as e: try: app.bcall_errors_handler(e) except KeyboardInterrupt: pass sys.exit(1) except ResultsConditionError as e: print('%s' % e, file=cls.stderr) sys.exit(1) finally: app.deinit() weboob-1.2/weboob/tools/application/console.py000066400000000000000000000625631303450110500215710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Christophe Benz, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function from copy import copy import getpass import logging import subprocess import sys import os from weboob.capabilities import UserError from weboob.capabilities.account import CapAccount, Account, AccountRegisterError from weboob.core.backendscfg import BackendAlreadyExists from weboob.core.repositories import IProgress from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword, BrowserForbidden, \ BrowserSSLError, BrowserQuestion, BrowserHTTPSDowngrade, \ ModuleInstallError, ModuleLoadError, NoAccountsException, \ ActionNeeded from weboob.tools.value import Value, ValueBool, ValueFloat, ValueInt, ValueBackendPassword from weboob.tools.misc import to_unicode from weboob.tools.compat import check_output from weboob.tools.ordereddict import OrderedDict from .base import Application, MoreResultsAvailable __all__ = ['ConsoleApplication', 'BackendNotGiven'] class BackendNotGiven(Exception): def __init__(self, id, backends): self.id = id self.backends = sorted(backends) Exception.__init__(self, 'Please specify a backend to use for this argument (%s@backend_name). ' 'Availables: %s.' % (id, ', '.join(name for name, backend in backends))) class BackendNotFound(Exception): pass class ConsoleProgress(IProgress): def __init__(self, app): self.app = app def progress(self, percent, message): self.app.stdout.write('=== [%3.0f%%] %s\n' % (percent*100, message)) def error(self, message): self.app.stderr.write('ERROR: %s\n' % message) def prompt(self, message): return self.app.ask(message, default=True) class ConsoleApplication(Application): """ Base application class for CLI applications. """ CAPS = None # shell escape strings if sys.platform == 'win32' \ or not sys.stdout.isatty() \ or os.getenv('ANSI_COLORS_DISABLED') is not None: #workaround to disable bold BOLD = '' NC = '' # no color else: BOLD = '' NC = '' # no color def __init__(self, option_parser=None): Application.__init__(self, option_parser) self.weboob.requests.register('login', self.login_cb) self.enabled_backends = set() def login_cb(self, backend_name, value): return self.ask('[%s] %s' % (backend_name, value.label), masked=True, default='', regexp=value.regexp) def unload_backends(self, *args, **kwargs): unloaded = self.weboob.unload_backends(*args, **kwargs) for backend in unloaded.itervalues(): try: self.enabled_backends.remove(backend) except KeyError: pass return unloaded def is_module_loadable(self, info): return self.CAPS is None or info.has_caps(self.CAPS) def load_backends(self, *args, **kwargs): if 'errors' in kwargs: errors = kwargs['errors'] else: kwargs['errors'] = errors = [] ret = super(ConsoleApplication, self).load_backends(*args, **kwargs) for err in errors: print(u'Error(%s): %s' % (err.backend_name, err), file=self.stderr) if self.ask('Do you want to reconfigure this backend?', default=True): self.edit_backend(err.backend_name) self.load_backends(names=[err.backend_name]) for name, backend in ret.iteritems(): self.enabled_backends.add(backend) self.check_loaded_backends() return ret def check_loaded_backends(self, default_config=None): while len(self.enabled_backends) == 0: print('Warning: there is currently no configured backend for %s' % self.APPNAME) if not self.stdout.isatty() or not self.ask('Do you want to configure backends?', default=True): return False self.prompt_create_backends(default_config) return True def prompt_create_backends(self, default_config=None): r = '' while r != 'q': modules = [] print('\nAvailable modules:') for name, info in sorted(self.weboob.repositories.get_all_modules_info().iteritems()): if not self.is_module_loadable(info): continue modules.append(name) loaded = ' ' for bi in self.weboob.iter_backends(): if bi.NAME == name: if loaded == ' ': loaded = 'X' elif loaded == 'X': loaded = 2 else: loaded += 1 print('%s%d)%s [%s] %s%-15s%s %s' % (self.BOLD, len(modules), self.NC, loaded, self.BOLD, name, self.NC, info.description.encode(self.encoding))) print('%sa) --all--%s install all backends' % (self.BOLD, self.NC)) print('%sq)%s --stop--\n' % (self.BOLD, self.NC)) r = self.ask('Select a backend to create (q to stop)', regexp='^(\d+|q|a)$') if str(r).isdigit(): i = int(r) - 1 if i < 0 or i >= len(modules): print('Error: %s is not a valid choice' % r, file=self.stderr) continue name = modules[i] try: inst = self.add_backend(name, name, default_config) if inst: self.load_backends(names=[inst]) except (KeyboardInterrupt, EOFError): print('\nAborted.') elif r == 'a': try: for name in modules: if name in [b.NAME for b in self.weboob.iter_backends()]: continue inst = self.add_backend(name, name, default_config) if inst: self.load_backends(names=[inst]) except (KeyboardInterrupt, EOFError): print('\nAborted.') else: break print('Right right!') def _handle_options(self): self.load_default_backends() def load_default_backends(self): """ By default loads all backends. Applications can overload this method to restrict backends loaded. """ if len(self.STORAGE) > 0: self.load_backends(self.CAPS, storage=self.create_storage()) else: self.load_backends(self.CAPS) @classmethod def run(klass, args=None): try: super(ConsoleApplication, klass).run(args) except BackendNotFound as e: print('Error: Backend "%s" not found.' % e) sys.exit(1) def do(self, function, *args, **kwargs): if 'backends' not in kwargs: kwargs['backends'] = self.enabled_backends return self.weboob.do(function, *args, **kwargs) def parse_id(self, _id, unique_backend=False): try: _id, backend_name = _id.rsplit('@', 1) except ValueError: backend_name = None backends = [(b.name, b) for b in self.enabled_backends] if unique_backend and not backend_name: if len(backends) == 1: backend_name = backends[0][0] else: raise BackendNotGiven(_id, backends) if backend_name is not None and backend_name not in dict(backends): # Is the backend a short version of a real one? found = False for key in dict(backends): if backend_name in key: # two choices, ambiguous command if found: raise BackendNotFound(backend_name) else: found = True _back = key if found: return _id, _back raise BackendNotFound(backend_name) return _id, backend_name # user interaction related methods def register_backend(self, name, ask_add=True): try: backend = self.weboob.modules_loader.get_or_load_module(name) except ModuleLoadError as e: backend = None if not backend: print('Backend "%s" does not exist.' % name, file=self.stderr) return 1 if not backend.has_caps(CapAccount) or backend.klass.ACCOUNT_REGISTER_PROPERTIES is None: print('You can\'t register a new account with %s' % name, file=self.stderr) return 1 account = Account() account.properties = {} if backend.website: website = 'on website %s' % backend.website else: website = 'with backend %s' % backend.name while True: asked_config = False for key, prop in backend.klass.ACCOUNT_REGISTER_PROPERTIES.iteritems(): if not asked_config: asked_config = True print('Configuration of new account %s' % website) print('-----------------------------%s' % ('-' * len(website))) p = copy(prop) p.set(self.ask(prop, default=account.properties[key].get() if (key in account.properties) else prop.default)) account.properties[key] = p if asked_config: print('-----------------------------%s' % ('-' * len(website))) try: backend.klass.register_account(account) except AccountRegisterError as e: print(u'%s' % e) if self.ask('Do you want to try again?', default=True): continue else: return None else: break backend_config = {} for key, value in account.properties.iteritems(): if key in backend.config: backend_config[key] = value.get() if ask_add and self.ask('Do you want to add the new register account?', default=True): return self.add_backend(name, name, backend_config, ask_register=False) return backend_config def install_module(self, name): try: self.weboob.repositories.install(name, ConsoleProgress(self)) except ModuleInstallError as e: print('Unable to install module "%s": %s' % (name, e), file=self.stderr) return False print('') return True def edit_backend(self, name, params=None): return self.add_backend(name, name, params, True) def add_backend(self, module_name, backend_name, params=None, edit=False, ask_register=True): if params is None: params = {} module = None config = None try: if not edit: minfo = self.weboob.repositories.get_module_info(module_name) if minfo is None: raise ModuleLoadError(module_name, 'Module does not exist') if not minfo.is_installed(): print('Module "%s" is available but not installed.' % minfo.name) self.install_module(minfo) module = self.weboob.modules_loader.get_or_load_module(module_name) config = module.config else: module_name, items = self.weboob.backends_config.get_backend(backend_name) module = self.weboob.modules_loader.get_or_load_module(module_name) items.update(params) params = items config = module.config.load(self.weboob, module_name, backend_name, params, nofail=True) except ModuleLoadError as e: print('Unable to load module "%s": %s' % (module_name, e), file=self.stderr) return 1 # ask for params non-specified on command-line arguments asked_config = False for key, value in config.iteritems(): if not asked_config: asked_config = True print('') print('Configuration of backend %s' % module.name) print('-------------------------%s' % ('-' * len(module.name))) if key not in params or edit: params[key] = self.ask(value, default=params[key] if (key in params) else value.default) else: print(u'[%s] %s: %s' % (key, value.description, '(masked)' if value.masked else to_unicode(params[key]))) if asked_config: print('-------------------------%s' % ('-' * len(module.name))) i = 2 while not edit and self.weboob.backends_config.backend_exists(backend_name): if not self.ask('Backend "%s" already exists. Add a new one for module %s?' % (backend_name, module.name), default=False): return 1 backend_name = backend_name.rstrip('0123456789') while self.weboob.backends_config.backend_exists('%s%s' % (backend_name, i)): i += 1 backend_name = self.ask('Please give new instance name', default='%s%s' % (backend_name, i), regexp=r'^[\w\-_]+$') try: config = config.load(self.weboob, module.name, backend_name, params, nofail=True) for key, value in params.iteritems(): if key not in config: continue config[key].set(value) config.save(edit=edit) print('Backend "%s" successfully %s.' % (backend_name, 'edited' if edit else 'added')) return backend_name except BackendAlreadyExists: print('Backend "%s" already exists.' % backend_name, file=self.stderr) return 1 def ask(self, question, default=None, masked=None, regexp=None, choices=None, tiny=None): """ Ask a question to user. @param question text displayed (str) @param default optional default value (str) @param masked if True, do not show typed text (bool) @param regexp text must match this regexp (str) @param choices choices to do (list) @param tiny ask for the (small) value of the choice (bool) @return entered text by user (str) """ if isinstance(question, Value): v = copy(question) if default is not None: v.default = to_unicode(default) if isinstance(default, str) else default if masked is not None: v.masked = masked if regexp is not None: v.regexp = regexp if choices is not None: v.choices = choices if tiny is not None: v.tiny = tiny else: if isinstance(default, bool): klass = ValueBool elif isinstance(default, float): klass = ValueFloat elif isinstance(default, (int,long)): klass = ValueInt else: klass = Value v = klass(label=question, default=default, masked=masked, regexp=regexp, choices=choices, tiny=tiny) question = v.label if v.id: question = u'[%s] %s' % (v.id, question) if isinstance(v, ValueBackendPassword): print(question.encode(self.encoding) + ':') question = v.label choices = OrderedDict() choices['c'] = 'Run an external tool during backend load' if not v.noprompt: choices['p'] = 'Prompt value when needed (do not store it)' choices['s'] = 'Store value in config' if v.is_command(v.default): d = 'c' elif v.default == '' and not v.noprompt: d = 'p' else: d = 's' r = self.ask('*** How do you want to store it?', choices=choices, tiny=True, default=d) if r == 'p': return '' if r == 'c': print('Enter the shell command that will print the required value on the standard output') if v.is_command(v.default): print(': %s' % v.default[1:-1]) else: d = None while True: cmd = self.ask('') try: check_output(cmd, shell=True) except subprocess.CalledProcessError as e: print('%s' % e) else: return '`%s`' % cmd aliases = {} if isinstance(v, ValueBool): question = u'%s (%s/%s)' % (question, 'Y' if v.default else 'y', 'n' if v.default else 'N') elif v.choices: if v.tiny is None: v.tiny = True for key in v.choices.iterkeys(): if len(key) > 5 or ' ' in key: v.tiny = False break if v.tiny: question = u'%s (%s)' % (question, '/'.join((s.upper() if s == v.default else s) for s in v.choices.iterkeys())) for s in v.choices.iterkeys(): if s == v.default: aliases[s.upper()] = s for key, value in v.choices.iteritems(): print(' %s%s%s: %s' % (self.BOLD, key, self.NC, value)) else: for n, (key, value) in enumerate(v.choices.iteritems()): print(' %s%2d)%s %s' % (self.BOLD, n + 1, self.NC, value.encode(self.encoding))) aliases[str(n + 1)] = key question = u'%s (choose in list)' % question if v.masked: question = u'%s (hidden input)' % question if not isinstance(v, ValueBool) and not v.tiny and v.default not in (None, ''): question = u'%s [%s]' % (question, '*******' if v.masked else v.default) question += ': ' while True: if v.masked: if sys.platform == 'win32': line = getpass.getpass(str(question)) else: line = getpass.getpass(question.encode(self.encoding)) else: self.stdout.write(question.encode(self.encoding)) self.stdout.flush() line = self.stdin.readline() if len(line) == 0: raise EOFError() else: line = line.rstrip('\r\n') if not line and v.default is not None: line = v.default if isinstance(line, str): line = line.decode('utf-8') if line in aliases: line = aliases[line] try: v.set(line) except ValueError as e: print(u'Error: %s' % e, file=self.stderr) else: break v.noprompt = True return v.get() def print(self, txt): print(txt.encode(self.encoding, "replace")) def acquire_input(self, content=None, editor_params=None): editor = os.getenv('EDITOR', 'vi') if self.stdin.isatty() and editor: from tempfile import NamedTemporaryFile with NamedTemporaryFile() as f: filename = f.name if content is not None: if isinstance(content, unicode): content = content.encode(self.encoding) f.write(content) f.flush() try: params = editor_params[os.path.basename(editor)] except (KeyError,TypeError): params = '' os.system("%s %s %s" % (editor, params, filename)) f.seek(0) text = f.read() else: if self.stdin.isatty(): print('Reading content from stdin... Type ctrl-D ' 'from an empty line to stop.') text = self.stdin.read() return text.decode(self.encoding) def bcall_error_handler(self, backend, error, backtrace): """ Handler for an exception inside the CallErrors exception. This method can be overrided to support more exceptions types. """ if isinstance(error, BrowserQuestion): for field in error.fields: v = self.ask(field) if v: backend.config[field.id].set(v) elif isinstance(error, BrowserIncorrectPassword): msg = unicode(error) if not msg: msg = 'invalid login/password.' print('Error(%s): %s' % (backend.name, msg), file=self.stderr) if self.ask('Do you want to reconfigure this backend?', default=True): self.unload_backends(names=[backend.name]) self.edit_backend(backend.name) self.load_backends(names=[backend.name]) elif isinstance(error, BrowserSSLError): print(u'FATAL(%s): ' % backend.name + self.BOLD + '/!\ SERVER CERTIFICATE IS INVALID /!\\' + self.NC, file=self.stderr) elif isinstance(error, BrowserHTTPSDowngrade): print(u'FATAL(%s): ' % backend.name + 'Downgrade from HTTPS to HTTP') elif isinstance(error, BrowserForbidden): msg = unicode(error) print(u'Error(%s): %s' % (backend.name, msg or 'Forbidden'), file=self.stderr) elif isinstance(error, BrowserUnavailable): msg = unicode(error) print(u'Error(%s): %s' % (backend.name, msg or 'Website is unavailable.'), file=self.stderr) elif isinstance(error, ActionNeeded): msg = unicode(error) print(u'Error(%s): Action needed on website: %s' % (backend.name, msg), file=self.stderr) elif isinstance(error, NotImplementedError): print(u'Error(%s): this feature is not supported yet by this backend.' % backend.name, file=self.stderr) print(u' %s To help the maintainer of this backend implement this feature,' % (' ' * len(backend.name)), file=self.stderr) print(u' %s please contact us on the project mailing list' % (' ' * len(backend.name)), file=self.stderr) elif isinstance(error, UserError): print(u'Error(%s): %s' % (backend.name, to_unicode(error)), file=self.stderr) elif isinstance(error, MoreResultsAvailable): print(u'Hint: There are more results for backend %s' % (backend.name), file=self.stderr) elif isinstance(error, NoAccountsException): print(u'Error(%s): %s' % (backend.name, to_unicode(error) or 'No account on this backend'), file=self.stderr) else: print(u'Bug(%s): %s' % (backend.name, to_unicode(error)), file=self.stderr) minfo = self.weboob.repositories.get_module_info(backend.NAME) if minfo and not minfo.is_local(): self.weboob.repositories.update_repositories(ConsoleProgress(self)) # minfo of the new available module minfo = self.weboob.repositories.get_module_info(backend.NAME) if minfo and minfo.version > self.weboob.repositories.versions.get(minfo.name) and \ self.ask('A new version of %s is available. Do you want to install it?' % minfo.name, default=True) and \ self.install_module(minfo): print('New version of module %s has been installed. Retry to call the command.' % minfo.name) return if logging.root.level <= logging.DEBUG: print(backtrace, file=self.stderr) else: return True def bcall_errors_handler(self, errors, debugmsg='Use --debug option to print backtraces', ignore=()): """ Handler for the CallErrors exception. """ ask_debug_mode = False more_results = set() for backend, error, backtrace in errors.errors: if isinstance(error, MoreResultsAvailable): more_results.add(backend.name) elif isinstance(error, ignore): continue elif self.bcall_error_handler(backend, error, backtrace): ask_debug_mode = True if ask_debug_mode: print(debugmsg, file=self.stderr) elif len(more_results) > 0: print('Hint: There are more results available for %s (use option -n or count command)' % (', '.join(more_results)), file=self.stderr) weboob-1.2/weboob/tools/application/formatters/000077500000000000000000000000001303450110500217275ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/formatters/__init__.py000066400000000000000000000000001303450110500240260ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/formatters/csv.py000066400000000000000000000031701303450110500230750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import absolute_import import csv from .iformatter import IFormatter __all__ = ['CSVFormatter'] class CSVFormatter(IFormatter): def __init__(self, field_separator=";"): super(CSVFormatter, self).__init__() self.started = False self.field_separator = field_separator def flush(self): self.started = False def format_dict(self, item): if not isinstance(self.outfile, basestring): return self.write_dict(item, self.outfile) with open(self.outfile, "a+") as fp: return self.write_dict(item, fp) def write_dict(self, item, fp): writer = csv.writer(fp, delimiter=self.field_separator) if not self.started: writer.writerow([unicode(v).encode('utf-8') for v in item.keys()]) self.started = True writer.writerow([unicode(v).encode('utf-8') for v in item.itervalues()]) weboob-1.2/weboob/tools/application/formatters/iformatter.py000066400000000000000000000231711303450110500244610ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Christophe Benz, Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os import sys import subprocess try: from termcolor import colored except ImportError: def colored(s, color=None, on_color=None, attrs=None): if os.getenv('ANSI_COLORS_DISABLED') is None \ and attrs is not None and 'bold' in attrs: return '%s%s%s' % (IFormatter.BOLD, s, IFormatter.NC) else: return s try: import tty import termios except ImportError: PROMPT = '--Press return to continue--' def readch(): return sys.stdin.readline() else: PROMPT = '--Press a key to continue--' def readch(): fd = sys.stdin.fileno() old_settings = termios.tcgetattr(fd) tty.setcbreak(fd) try: c = sys.stdin.read(1) # XXX do not read magic number if c == '\x03': raise KeyboardInterrupt() return c finally: termios.tcsetattr(fd, termios.TCSADRAIN, old_settings) from weboob.capabilities.base import BaseObject from weboob.tools.application.console import ConsoleApplication from weboob.tools.ordereddict import OrderedDict from weboob.tools.misc import guess_encoding __all__ = ['IFormatter', 'MandatoryFieldsNotFound'] class MandatoryFieldsNotFound(Exception): def __init__(self, missing_fields): Exception.__init__(self, u'Mandatory fields not found: %s.' % ', '.join(missing_fields)) class IFormatter(object): # Tuple of fields mandatory to not crash MANDATORY_FIELDS = None # Tuple of displayed field. Set to None if all available fields are # displayed DISPLAYED_FIELDS = None BOLD = ConsoleApplication.BOLD NC = ConsoleApplication.NC def colored(self, string, color, attrs=None, on_color=None): if self.outfile != sys.stdout or not self.outfile.isatty(): return string if isinstance(attrs, basestring): attrs = [attrs] return colored(string, color, on_color=on_color, attrs=attrs) def __init__(self, display_keys=True, display_header=True, outfile=sys.stdout): self.display_keys = display_keys self.display_header = display_header self.interactive = False self.print_lines = 0 self.termrows = 0 self.termcols = None self.outfile = outfile # XXX if stdin is not a tty, it seems that the command fails. if sys.stdout.isatty() and sys.stdin.isatty(): if sys.platform == 'win32': from ctypes import windll, create_string_buffer h = windll.kernel32.GetStdHandle(-12) csbi = create_string_buffer(22) res = windll.kernel32.GetConsoleScreenBufferInfo(h, csbi) if res: import struct (bufx, bufy, curx, cury, wattr, left, top, right, bottom, maxx, maxy) = struct.unpack("hhhhHhhhhhh", csbi.raw) self.termrows = right - left + 1 self.termcols = bottom - top + 1 else: self.termrows = 80 # can't determine actual size - return default values self.termcols = 80 else: self.termrows = int( subprocess.Popen('stty size', shell=True, stdout=subprocess.PIPE).communicate()[0].split()[0] ) self.termcols = int( subprocess.Popen('stty size', shell=True, stdout=subprocess.PIPE).communicate()[0].split()[1] ) def output(self, formatted): if self.outfile != sys.stdout: with open(self.outfile, "a+") as outfile: outfile.write(formatted.encode(guess_encoding(outfile), 'replace') + os.linesep) else: for line in formatted.split('\n'): if self.termrows and (self.print_lines + 1) >= self.termrows: self.outfile.write(PROMPT) self.outfile.flush() readch() self.outfile.write('\b \b' * len(PROMPT)) self.print_lines = 0 plen = len(line.replace(self.BOLD, '').replace(self.NC, '')) if isinstance(line, unicode): line = line.encode(guess_encoding(self.outfile), 'replace') print(line) if self.termcols: self.print_lines += int(plen/self.termcols) + 1 else: self.print_lines += 1 def start_format(self, **kwargs): pass def flush(self): pass def format(self, obj, selected_fields=None, alias=None): """ Format an object to be human-readable. An object has fields which can be selected. :param obj: object to format :type obj: BaseObject or dict :param selected_fields: fields to display. If None, all fields are selected :type selected_fields: tuple :param alias: an alias to use instead of the object's ID :type alias: unicode """ if isinstance(obj, BaseObject): if selected_fields: # can be an empty list (nothing to do), or None (return all fields) obj = obj.copy() for name, value in obj.iter_fields(): if name not in selected_fields: delattr(obj, name) if self.MANDATORY_FIELDS: missing_fields = set(self.MANDATORY_FIELDS) - set([name for name, value in obj.iter_fields()]) if missing_fields: raise MandatoryFieldsNotFound(missing_fields) formatted = self.format_obj(obj, alias) else: try: obj = OrderedDict(obj) except ValueError: raise TypeError('Please give a BaseObject or a dict') if selected_fields: obj = obj.copy() for name, value in obj.iteritems(): if name not in selected_fields: obj.pop(name) if self.MANDATORY_FIELDS: missing_fields = set(self.MANDATORY_FIELDS) - set(obj.iterkeys()) if missing_fields: raise MandatoryFieldsNotFound(missing_fields) formatted = self.format_dict(obj) if formatted: self.output(formatted) return formatted def format_obj(self, obj, alias=None): """ Format an object to be human-readable. Called by format(). This method has to be overridden in child classes. :param obj: object to format :type obj: BaseObject :rtype: str """ return self.format_dict(obj.to_dict()) def format_dict(self, obj): """ Format a dict to be human-readable. :param obj: dict to format :type obj: dict :rtype: str """ return NotImplementedError() def format_collection(self, collection, only): """ Format a collection to be human-readable. :param collection: collection to format :type collection: BaseCollection :rtype: str """ if only is False or collection.basename in only: if collection.basename and collection.title: self.output(u'%s~ (%s) %s (%s)%s' % (self.BOLD, collection.basename, collection.title, collection.backend, self.NC)) else: self.output(u'%s~ (%s) (%s)%s' % (self.BOLD, collection.basename, collection.backend, self.NC)) class PrettyFormatter(IFormatter): def format_obj(self, obj, alias): title = self.get_title(obj) desc = self.get_description(obj) if alias is not None: result = u'%s %s %s (%s)' % (self.colored('%2s' % alias, 'red', 'bold'), self.colored(u'—', 'cyan', 'bold'), self.colored(title, 'yellow', 'bold'), self.colored(obj.backend, 'blue', 'bold')) else: result = u'%s %s %s' % (self.colored(obj.fullid, 'red', 'bold'), self.colored(u'—', 'cyan', 'bold'), self.colored(title, 'yellow', 'bold')) if desc is not None: result += u'%s\t%s' % (os.linesep, self.colored(desc, 'white')) return result def get_title(self, obj): raise NotImplementedError() def get_description(self, obj): return None def formatter_test_output(Formatter, obj): """ Formats an object and returns output as a string. For test purposes only. """ from tempfile import mkstemp from os import remove _, name = mkstemp() fmt = Formatter() fmt.outfile = name fmt.format(obj) fmt.flush() with open(name) as f: res = f.read() remove(name) return res weboob-1.2/weboob/tools/application/formatters/json.py000066400000000000000000000043211303450110500232520ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013-2014 Julien Hebert, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import NotAvailable, NotLoaded from weboob.tools.json import json from .iformatter import IFormatter __all__ = ['JsonFormatter', 'JsonLineFormatter'] class Encoder(json.JSONEncoder): "generic weboob object encoder" def default(self, obj): try: return json.JSONEncoder.default(self, obj) except TypeError: if obj is NotAvailable or obj is NotLoaded: return None try: dct = obj.to_dict() except AttributeError: return str(obj) return dct class JsonFormatter(IFormatter): """ Formats the whole list as a single JSON list object. """ def __init__(self): IFormatter.__init__(self) self.queue = [] def flush(self): self.output(json.dumps(self.queue, cls=Encoder)) def format_dict(self, item): self.queue.append(item) def format_collection(self, collection, only): self.queue.append(collection.to_dict()) class JsonLineFormatter(IFormatter): """ Formats the list as received, with a JSON object per line. The advantage is that it can be streamed. """ def format_dict(self, item): self.output(json.dumps(item, cls=Encoder)) def test(): from .iformatter import formatter_test_output as fmt assert fmt(JsonFormatter, {'foo': 'bar'}) == '[{"foo": "bar"}]\n' assert fmt(JsonLineFormatter, {'foo': 'bar'}) == '{"foo": "bar"}\n' weboob-1.2/weboob/tools/application/formatters/load.py000066400000000000000000000053021303450110500232200ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['FormattersLoader', 'FormatterLoadError'] class FormatterLoadError(Exception): pass class FormattersLoader(object): BUILTINS = ['htmltable', 'multiline', 'simple', 'table', 'csv', 'webkit', 'json', 'json_line'] def __init__(self): self.formatters = {} def register_formatter(self, name, klass): self.formatters[name] = klass def get_available_formatters(self): l = set(self.formatters.iterkeys()) l = l.union(self.BUILTINS) l = sorted(l) return l def build_formatter(self, name): if name not in self.formatters: try: self.formatters[name] = self.load_builtin_formatter(name) except ImportError as e: FormattersLoader.BUILTINS.remove(name) raise FormatterLoadError('Unable to load formatter "%s": %s' % (name, e)) return self.formatters[name]() def load_builtin_formatter(self, name): if name not in self.BUILTINS: raise FormatterLoadError('Formatter "%s" does not exist' % name) if name == 'htmltable': from .table import HTMLTableFormatter return HTMLTableFormatter elif name == 'table': from .table import TableFormatter return TableFormatter elif name == 'simple': from .simple import SimpleFormatter return SimpleFormatter elif name == 'multiline': from .multiline import MultilineFormatter return MultilineFormatter elif name == 'webkit': from .webkit import WebkitGtkFormatter return WebkitGtkFormatter elif name == 'csv': from .csv import CSVFormatter return CSVFormatter elif name == 'json': from .json import JsonFormatter return JsonFormatter elif name == 'json_line': from .json import JsonLineFormatter return JsonLineFormatter weboob-1.2/weboob/tools/application/formatters/multiline.py000066400000000000000000000027061303450110500243100ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import NotLoaded, NotAvailable from .iformatter import IFormatter __all__ = ['MultilineFormatter'] class MultilineFormatter(IFormatter): def __init__(self, key_value_separator=u': ', after_item=u'\n'): IFormatter.__init__(self) self.key_value_separator = key_value_separator self.after_item = after_item def flush(self): pass def format_dict(self, item): result = u'\n'.join(u'%s%s' % ( (u'%s%s' % (k, self.key_value_separator) if self.display_keys else ''), v) for k, v in item.iteritems() if (v is not NotLoaded and v is not NotAvailable)) if len(item) > 1: result += self.after_item return result weboob-1.2/weboob/tools/application/formatters/simple.py000066400000000000000000000023631303450110500235760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .iformatter import IFormatter __all__ = ['SimpleFormatter'] class SimpleFormatter(IFormatter): def __init__(self, field_separator=u'\t', key_value_separator=u'='): IFormatter.__init__(self) self.field_separator = field_separator self.key_value_separator = key_value_separator def format_dict(self, item): return self.field_separator.join(u'%s%s' % ( (u'%s%s' % (k, self.key_value_separator) if self.display_keys else ''), v) for k, v in item.iteritems()) weboob-1.2/weboob/tools/application/formatters/table.py000066400000000000000000000067701303450110500234020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from prettytable import PrettyTable from weboob.capabilities.base import empty from weboob.tools.misc import guess_encoding from .iformatter import IFormatter __all__ = ['TableFormatter', 'HTMLTableFormatter'] class TableFormatter(IFormatter): HTML = False def __init__(self): IFormatter.__init__(self) self.queue = [] self.keys = None self.header = None def flush(self): s = self.get_formatted_table() if s is not None: self.output(s.encode(guess_encoding(self.outfile), 'replace')) def get_formatted_table(self): if len(self.queue) == 0: return queue = [() for i in xrange(len(self.queue))] column_headers = [] # Do not display columns when all values are NotLoaded or NotAvailable maxrow = 0 for i in xrange(len(self.keys)): available = False for line in self.queue: if len(line)> i and not empty(line[i]): maxrow += 1 available = True break if available: column_headers.append(self.keys[i].capitalize().replace('_', ' ')) for j in xrange(len(self.queue)): if len(self.queue[j]) > i: queue[j] += (self.queue[j][i],) s = '' if self.display_header and self.header: if self.HTML: s += '

%s

' % self.header else: s += self.header s += "\n" table = PrettyTable(list(column_headers)) for column_header in column_headers: # API changed in python-prettytable. The try/except is a bad hack to support both versions # Note: two versions are not exactly the same... # (first one: header in center. Second one: left align for header too) try: table.set_field_align(column_header, 'l') except: table.align[column_header] = 'l' for line in queue: for _ in range(maxrow - len(line)): line += ('',) table.add_row(line) if self.HTML: s += table.get_html_string() else: s += table.get_string() self.queue = [] return s def format_dict(self, item): if self.keys is None: self.keys = item.keys() self.queue.append(item.values()) def set_header(self, string): self.header = string class HTMLTableFormatter(TableFormatter): HTML = True def test(): from .iformatter import formatter_test_output as fmt assert fmt(TableFormatter, {'foo': 'bar'}) == \ '+-----+\n' \ '| Foo |\n' \ '+-----+\n' \ '| bar |\n' \ '+-----+\n' weboob-1.2/weboob/tools/application/formatters/webkit/000077500000000000000000000000001303450110500232145ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/formatters/webkit/__init__.py000066400000000000000000000014561303450110500253330ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from .webkitgtk import WebkitGtkFormatter __all__ = ['WebkitGtkFormatter'] weboob-1.2/weboob/tools/application/formatters/webkit/webkitgtk.py000066400000000000000000000051331303450110500255630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os import gtk import webkit from weboob.tools.application.javascript import get_javascript from ..table import HTMLTableFormatter __all__ = ['WebkitGtkFormatter'] class WebBrowser(gtk.Window): def __init__(self): gtk.Window.__init__(self) self.connect('destroy', gtk.main_quit) self.set_default_size(800, 600) self.web_view = webkit.WebView() sw = gtk.ScrolledWindow() sw.add(self.web_view) self.add(sw) self.show_all() class WebkitGtkFormatter(HTMLTableFormatter): def flush(self): table_string = self.get_formatted_table() js_filepaths = [] js_filepaths.append(get_javascript('jquery')) js_filepaths.append(get_javascript('tablesorter')) scripts = ['' % js_filepath for js_filepath in js_filepaths] html_string_params = dict(table=table_string) if scripts: html_string_params['scripts'] = ''.join(scripts) html_string = """ %(scripts)s %(table)s """ % html_string_params web_browser = WebBrowser() web_browser.web_view.load_html_string(html_string, 'file://%s' % os.path.abspath(os.getcwd())) gtk.main() weboob-1.2/weboob/tools/application/javascript.py000066400000000000000000000034441303450110500222660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os __all__ = ['get_javascript'] def get_javascript(name, load_order=('local', 'web'), minified=True): if name == 'jquery': for src in load_order: if src == 'local': # try Debian paths if minified: filepath = '/usr/share/javascript/jquery/jquery.min.js' else: filepath = '/usr/share/javascript/jquery/jquery.js' if os.path.exists(filepath): return filepath elif src == 'web': # return Google-hosted URLs if minified: return 'http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js' else: return 'http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.js' elif name == 'tablesorter': if 'web' in load_order: if minified: return 'http://tablesorter.com/jquery.tablesorter.min.js' else: return 'http://tablesorter.com/jquery.tablesorter.js' return None weboob-1.2/weboob/tools/application/media_player.py000066400000000000000000000156031303450110500225530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz, Romain Bignon, John Obbele # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import os from subprocess import PIPE, Popen import cookielib import urllib2 from weboob.tools.log import getLogger __all__ = ['InvalidMediaPlayer', 'MediaPlayer', 'MediaPlayerNotFound'] PLAYERS = ( ('mpv', '-'), ('mplayer2', '-'), ('mplayer', '-'), ('vlc', '-'), ('parole', 'fd://0'), ('totem', 'fd://0'), ('xine', 'stdin:/'), ) class MediaPlayerNotFound(Exception): def __init__(self): Exception.__init__(self, u'No media player found on this system. Please install one of them: %s.' % ', '.join(player[0] for player in PLAYERS)) class InvalidMediaPlayer(Exception): def __init__(self, player_name): Exception.__init__(self, u'Invalid media player: %s. Valid media players: %s.' % ( player_name, ', '.join(player[0] for player in PLAYERS))) class MediaPlayer(object): """ Black magic invoking a media player to this world. Presently, due to strong disturbances in the holidays of the ether world, the media player used is chosen from a static list of programs. See PLAYERS for more information. """ def __init__(self, logger=None): self.logger = getLogger('mediaplayer', logger) def guess_player_name(self): for player_name in [player[0] for player in PLAYERS]: if self._find_in_path(os.environ['PATH'], player_name): return player_name return None def play(self, media, player_name=None, player_args=None): """ Play a media object, using programs from the PLAYERS list. This function dispatch calls to either _play_default or _play_rtmp for special rtmp streams using SWF verification. """ player_names = [player[0] for player in PLAYERS] if not player_name: self.logger.debug(u'No media player given. Using the first available from: %s.' % ', '.join(player_names)) player_name = self.guess_player_name() if player_name is None: raise MediaPlayerNotFound() if media.url.startswith('rtmp'): self._play_rtmp(media, player_name, args=player_args) else: self._play_default(media, player_name, args=player_args) def _play_default(self, media, player_name, args=None): """ Play media.url with the media player. """ # if flag play_proxy... if hasattr(media, '_play_proxy') and media._play_proxy is True: # use urllib2 to handle redirect and cookies self._play_proxy(media, player_name, args) return None args = player_name.split(' ') player_name = args[0] args.append(media.url) print('Invoking "%s".' % (' '.join(args))) os.spawnlp(os.P_WAIT, player_name, *args) def _play_proxy(self, media, player_name, args): """ Load data with python urllib2 and pipe data to a media player. We need this function for url that use redirection and cookies. This function is used if the non-standard, non-API compliant '_play_proxy' attribute of the 'media' object is defined and is True. """ if args is None: for (binary, stdin_args) in PLAYERS: if binary == player_name: args = stdin_args assert args is not None print(':: Play_proxy streaming from %s' % media.url) print(':: to %s %s' % (player_name, args)) print(player_name + ' ' + args) proc = Popen(player_name + ' ' + args, stdin=PIPE, shell=True) # Handle cookies (and redirection 302...) cj = cookielib.CookieJar() url_opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) url_handler = url_opener.open(media.url) file_size = int(url_handler.info().getheaders("Content-Length")[0]) file_size_dl = 0 block_sz = 8192 while file_size_dl < file_size: _buffer = url_handler.read(block_sz) if not buffer: break file_size_dl += len(_buffer) try: proc.stdin.write(_buffer) except: print("play_proxy broken pipe. Can't write anymore.") break def _play_rtmp(self, media, player_name, args): """ Download data with rtmpdump and pipe them to a media player. You need a working version of rtmpdump installed and the SWF object url in order to comply with SWF verification requests from the server. The last one is retrieved from the non-standard non-API compliant 'swf_player' attribute of the 'media' object. """ if not self._find_in_path(os.environ['PATH'], 'rtmpdump'): self.logger.warning('"rtmpdump" binary not found') return self._play_default(media, player_name) media_url = media.url try: player_url = media.swf_player if media.swf_player: rtmp = 'rtmpdump -r %s --swfVfy %s' % (media_url, player_url) else: rtmp = 'rtmpdump -r %s' % media_url except AttributeError: self.logger.warning('Your media object does not have a "swf_player" attribute. SWF verification will be ' 'disabled and may prevent correct media playback.') return self._play_default(media, player_name) rtmp += ' --quiet' if args is None: for (binary, stdin_args) in PLAYERS: if binary == player_name: args = stdin_args assert args is not None player_name = player_name.split(' ') args = args.split(' ') print(':: Streaming from %s' % media_url) print(':: to %s %s' % (player_name, args)) print(':: %s' % rtmp) p1 = Popen(rtmp.split(), stdout=PIPE) Popen(player_name + args, stdin=p1.stdout, stderr=PIPE) def _find_in_path(self, path, filename): for i in path.split(':'): if os.path.exists('/'.join([i, filename])): return True return False weboob-1.2/weboob/tools/application/qt5/000077500000000000000000000000001303450110500202525ustar00rootroot00000000000000weboob-1.2/weboob/tools/application/qt5/Makefile000066400000000000000000000002641303450110500217140ustar00rootroot00000000000000UI_FILES = $(wildcard *.ui) UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py) PYUIC = pyuic5 all: $(UI_PY_FILES) %_ui.py: %.ui $(PYUIC) -o $@ $^ clean: rm -f *.pyc rm -f $(UI_PY_FILES) weboob-1.2/weboob/tools/application/qt5/__init__.py000066400000000000000000000003011303450110500223550ustar00rootroot00000000000000from .qt import QtApplication, QtMainWindow, QtDo, HTMLDelegate from .backendcfg import BackendCfg __all__ = ['QtApplication', 'QtMainWindow', 'QtDo', 'HTMLDelegate', 'BackendCfg'] weboob-1.2/weboob/tools/application/qt5/backendcfg.py000066400000000000000000000503321303450110500226760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtWidgets import QDialog, QTreeWidgetItem, QLabel, QFormLayout, \ QMessageBox, QHeaderView, \ QListWidgetItem, QVBoxLayout, \ QDialogButtonBox, QProgressDialog from PyQt5.QtGui import QTextDocument, QPixmap, QImage, QIcon from PyQt5.QtCore import Qt, QVariant, QUrl, QThread from PyQt5.QtCore import pyqtSignal as Signal, pyqtSlot as Slot import re import os from logging import warning from weboob.core.repositories import IProgress from weboob.core.backendscfg import BackendAlreadyExists from weboob.capabilities.account import CapAccount, Account, AccountRegisterError from weboob.exceptions import ModuleInstallError, ModuleLoadError from .backendcfg_ui import Ui_BackendCfg from .reposdlg_ui import Ui_RepositoriesDlg from weboob.tools.ordereddict import OrderedDict from weboob.tools.misc import to_unicode from .qt import QtValue class RepositoriesDialog(QDialog): def __init__(self, filename, parent=None): QDialog.__init__(self, parent) self.filename = filename self.ui = Ui_RepositoriesDlg() self.ui.setupUi(self) self.ui.buttonBox.accepted.connect(self.save) with open(self.filename, 'r') as fp: self.ui.reposEdit.setPlainText(fp.read()) @Slot() def save(self): with open(self.filename, 'w') as fp: fp.write(self.ui.reposEdit.toPlainText()) self.accept() class IconFetcher(QThread): retrieved = Signal() def __init__(self, weboob, item, minfo): QThread.__init__(self) self.weboob = weboob self.items = [item] self.minfo = minfo def run(self): self.weboob.repositories.retrieve_icon(self.minfo) self.retrieved.emit() class ProgressDialog(IProgress, QProgressDialog): def __init__(self, *args, **kwargs): QProgressDialog.__init__(self, *args, **kwargs) def progress(self, percent, message): self.setValue(int(percent * 100)) self.setLabelText(message) def error(self, message): QMessageBox.critical(self, self.tr('Error'), '%s' % message, QMessageBox.Ok) def prompt(self, message): reply = QMessageBox.question(self, '', unicode(message), QMessageBox.Yes|QMessageBox.No) return reply == QMessageBox.Yes class BackendCfg(QDialog): def __init__(self, weboob, caps=None, parent=None): QDialog.__init__(self, parent) self.ui = Ui_BackendCfg() self.ui.setupUi(self) self.ui.backendsList.sortByColumn(0, Qt.AscendingOrder) self.to_unload = set() self.to_load = set() self.weboob = weboob self.caps = caps self.config_widgets = {} # This attribute is set when itemChanged it called, because when # a backend is enabled/disabled, we don't want to display its config # frame, and the itemClicked event is always emit just after a # itemChanged event. # is_enabling is a counter to prevent race conditions. self.is_enabling = 0 self.ui.backendsList.header().setSectionResizeMode(QHeaderView.ResizeToContents) self.ui.configFrame.hide() self.icon_cache = {} self.icon_threads = {} self.loadModules() self.loadBackendsList() self.ui.updateButton.clicked.connect(self.updateModules) self.ui.repositoriesButton.clicked.connect(self.editRepositories) self.ui.backendsList.itemClicked.connect(self.backendClicked) self.ui.backendsList.itemChanged.connect(self.backendEnabled) self.ui.modulesList.itemSelectionChanged.connect(self.moduleSelectionChanged) self.ui.proxyBox.toggled.connect(self.proxyEditEnabled) self.ui.addButton.clicked.connect(self.addEvent) self.ui.removeButton.clicked.connect(self.removeEvent) self.ui.registerButton.clicked.connect(self.registerEvent) self.ui.configButtonBox.accepted.connect(self.acceptBackend) self.ui.configButtonBox.rejected.connect(self.rejectBackend) def get_icon_cache(self, path): if path not in self.icon_cache: img = QImage(path) self.icon_cache[path] = QIcon(QPixmap.fromImage(img)) return self.icon_cache[path] def set_icon(self, item, minfo): icon_path = self.weboob.repositories.get_module_icon_path(minfo) icon = self.icon_cache.get(icon_path, None) if icon is None and not os.path.exists(icon_path): if minfo.name in self.icon_threads: self.icon_threads[minfo.name].items.append(item) else: thread = IconFetcher(self.weboob, item, minfo) thread.retrieved.connect(self._set_icon_slot) self.icon_threads[minfo.name] = thread thread.start() return self._set_icon([item], minfo) @Slot() def _set_icon_slot(self): thread = self.sender() self._set_icon(thread.items, thread.minfo) def _set_icon(self, items, minfo): icon_path = self.weboob.repositories.get_module_icon_path(minfo) icon = self.get_icon_cache(icon_path) if icon is None: return for item in items: try: item.setIcon(icon) except TypeError: item.setIcon(0, icon) self.icon_threads.pop(minfo.name, None) @Slot() def updateModules(self): self.ui.configFrame.hide() pd = ProgressDialog('Update of modules', "Cancel", 0, 100, self) pd.setWindowModality(Qt.WindowModal) try: self.weboob.repositories.update(pd) except ModuleInstallError as err: QMessageBox.critical(self, self.tr('Update error'), self.tr('Unable to update modules: %s' % (err)), QMessageBox.Ok) pd.setValue(100) self.loadModules() QMessageBox.information(self, self.tr('Update of modules'), self.tr('Modules updated!'), QMessageBox.Ok) @Slot() def editRepositories(self): if RepositoriesDialog(self.weboob.repositories.sources_list).exec_(): self.updateModules() def loadModules(self): self.ui.modulesList.clear() for name, module in sorted(self.weboob.repositories.get_all_modules_info(self.caps).iteritems()): item = QListWidgetItem(name.capitalize()) self.set_icon(item, module) self.ui.modulesList.addItem(item) def askInstallModule(self, minfo): reply = QMessageBox.question(self, self.tr('Install a module'), self.tr("Module %s is not installed. Do you want to install it?") % minfo.name, QMessageBox.Yes|QMessageBox.No) if reply != QMessageBox.Yes: return False return self.installModule(minfo) def installModule(self, minfo): pd = ProgressDialog('Installation of %s' % minfo.name, "Cancel", 0, 100, self) pd.setWindowModality(Qt.WindowModal) try: self.weboob.repositories.install(minfo, pd) except ModuleInstallError as err: QMessageBox.critical(self, self.tr('Install error'), self.tr('Unable to install module %s: %s' % (minfo.name, err)), QMessageBox.Ok) pd.setValue(100) return True def loadBackendsList(self): self.ui.backendsList.clear() for backend_name, module_name, params in self.weboob.backends_config.iter_backends(): info = self.weboob.repositories.get_module_info(module_name) if not info or (self.caps and not info.has_caps(self.caps)): continue item = QTreeWidgetItem(None, [backend_name, module_name]) item.setCheckState(0, Qt.Checked if params.get('_enabled', '1').lower() in ('1', 'y', 'true', 'on', 'yes') else Qt.Unchecked) self.set_icon(item, info) self.ui.backendsList.addTopLevelItem(item) @Slot(QTreeWidgetItem, int) def backendEnabled(self, item, col): self.is_enabling += 1 backend_name = item.text(0) module_name = item.text(1) if item.checkState(0) == Qt.Checked: self.to_load.add(backend_name) enabled = 'true' else: self.to_unload.add(backend_name) try: self.to_load.remove(backend_name) except KeyError: pass enabled = 'false' self.weboob.backends_config.edit_backend(backend_name, module_name, {'_enabled': enabled}) @Slot(QTreeWidgetItem, int) def backendClicked(self, item, col): if self.is_enabling: self.is_enabling -= 1 return backend_name = item.text(0) self.editBackend(backend_name) @Slot() def addEvent(self): self.editBackend() @Slot() def removeEvent(self): item = self.ui.backendsList.currentItem() if not item: return backend_name = item.text(0) reply = QMessageBox.question(self, self.tr('Remove a backend'), self.tr("Are you sure you want to remove the backend '%s'?") % backend_name, QMessageBox.Yes|QMessageBox.No) if reply != QMessageBox.Yes: return self.weboob.backends_config.remove_backend(backend_name) self.to_unload.add(backend_name) try: self.to_load.remove(backend_name) except KeyError: pass self.ui.configFrame.hide() self.loadBackendsList() def editBackend(self, backend_name=None): self.ui.registerButton.hide() self.ui.configFrame.show() if backend_name is not None: module_name, params = self.weboob.backends_config.get_backend(backend_name) items = self.ui.modulesList.findItems(module_name, Qt.MatchFixedString) if not items: warning('Backend not found') else: self.ui.modulesList.setCurrentItem(items[0]) self.ui.modulesList.setEnabled(False) self.ui.nameEdit.setText(backend_name) self.ui.nameEdit.setEnabled(False) if '_proxy' in params: self.ui.proxyBox.setChecked(True) self.ui.proxyEdit.setText(params.pop('_proxy')) else: self.ui.proxyBox.setChecked(False) self.ui.proxyEdit.clear() params.pop('_enabled', None) info = self.weboob.repositories.get_module_info(module_name) if info and (info.is_installed() or self.installModule(info)): module = self.weboob.modules_loader.get_or_load_module(module_name) for key, value in module.config.load(self.weboob, module_name, backend_name, params, nofail=True).iteritems(): try: l, widget = self.config_widgets[key] except KeyError: warning('Key "%s" is not found' % key) else: # Do not prompt user for value (for example a password if it is empty). value.noprompt = True widget.set_value(value) return self.ui.nameEdit.clear() self.ui.nameEdit.setEnabled(True) self.ui.proxyBox.setChecked(False) self.ui.proxyEdit.clear() self.ui.modulesList.setEnabled(True) self.ui.modulesList.setCurrentRow(-1) @Slot() def moduleSelectionChanged(self): for key, (label, value) in self.config_widgets.iteritems(): label.hide() value.hide() self.ui.configLayout.removeWidget(label) self.ui.configLayout.removeWidget(value) label.deleteLater() value.deleteLater() self.config_widgets = {} self.ui.moduleInfo.clear() selection = self.ui.modulesList.selectedItems() if not selection: return minfo = self.weboob.repositories.get_module_info(selection[0].text().lower()) if not minfo: warning('Module not found') return if not minfo.is_installed() and not self.installModule(minfo): self.editBackend(None) return module = self.weboob.modules_loader.get_or_load_module(minfo.name) icon_path = os.path.join(self.weboob.repositories.icons_dir, '%s.png' % minfo.name) img = QImage(icon_path) self.ui.moduleInfo.document().addResource(QTextDocument.ImageResource, QUrl('mydata://logo.png'), QVariant(img)) if module.name not in [n for n, ign, ign2 in self.weboob.backends_config.iter_backends()]: self.ui.nameEdit.setText(module.name) else: self.ui.nameEdit.setText('') self.ui.moduleInfo.setText(to_unicode(self.tr( u'

%s Module %s

' 'Version: %s
' 'Maintainer: %s
' 'License: %s
' '%s' 'Description: %s
' 'Capabilities: %s
')) % ('', module.name.capitalize(), module.version, to_unicode(module.maintainer).replace(u'&', u'&').replace(u'<', u'<').replace(u'>', u'>'), module.license, (self.tr('Website: %s
') % module.website) if module.website else '', module.description, ', '.join(sorted(cap.__name__.replace('Cap', '') for cap in module.iter_caps())))) if module.has_caps(CapAccount) and self.ui.nameEdit.isEnabled() and \ module.klass.ACCOUNT_REGISTER_PROPERTIES is not None: self.ui.registerButton.show() else: self.ui.registerButton.hide() for key, field in module.config.iteritems(): label = QLabel(u'%s:' % field.label) qvalue = QtValue(field) self.ui.configLayout.addRow(label, qvalue) self.config_widgets[key] = (label, qvalue) @Slot(bool) def proxyEditEnabled(self, state): self.ui.proxyEdit.setEnabled(state) @Slot() def acceptBackend(self): backend_name = self.ui.nameEdit.text() selection = self.ui.modulesList.selectedItems() if not selection: QMessageBox.critical(self, self.tr('Unable to add a backend'), self.tr('Please select a module')) return try: module = self.weboob.modules_loader.get_or_load_module(selection[0].text().lower()) except ModuleLoadError: module = None if not module: QMessageBox.critical(self, self.tr('Unable to add a backend'), self.tr('The selected module does not exist.')) return params = {} if not backend_name: QMessageBox.critical(self, self.tr('Missing field'), self.tr('Please specify a backend name')) return if self.ui.nameEdit.isEnabled(): if not re.match(r'^[\w\-_]+$', backend_name): QMessageBox.critical(self, self.tr('Invalid value'), self.tr('The backend name can only contain letters and digits')) return if self.weboob.backends_config.backend_exists(backend_name): QMessageBox.critical(self, self.tr('Unable to create backend'), self.tr('Unable to create backend "%s": it already exists') % backend_name) return if self.ui.proxyBox.isChecked(): params['_proxy'] = self.ui.proxyEdit.text() if not params['_proxy']: QMessageBox.critical(self, self.tr('Missing field'), self.tr('Please specify a proxy URL')) return config = module.config.load(self.weboob, module.name, backend_name, {}, nofail=True) for key, field in config.iteritems(): label, qtvalue = self.config_widgets[key] try: value = qtvalue.get_value() except ValueError as e: QMessageBox.critical(self, self.tr('Invalid value'), self.tr('Invalid value for field "%s":

%s') % (field.label, e)) return field.set(value.get()) try: config.save(edit=not self.ui.nameEdit.isEnabled(), params=params) except BackendAlreadyExists: QMessageBox.critical(self, self.tr('Unable to create backend'), self.tr('Unable to create backend "%s": it already exists') % backend_name) return self.to_load.add(backend_name) self.ui.configFrame.hide() self.loadBackendsList() @Slot() def rejectBackend(self): self.ui.configFrame.hide() @Slot() def registerEvent(self): selection = self.ui.modulesList.selectedItems() if not selection: return try: module = self.weboob.modules_loader.get_or_load_module(selection[0].text().lower()) except ModuleLoadError: module = None if not module: return dialog = QDialog(self) vbox = QVBoxLayout(dialog) if module.website: website = 'on the website %s' % module.website else: website = 'with the module %s' % module.name vbox.addWidget(QLabel('To create an account %s, please provide this information:' % website)) formlayout = QFormLayout() props_widgets = OrderedDict() for key, prop in module.klass.ACCOUNT_REGISTER_PROPERTIES.iteritems(): widget = QtValue(prop) formlayout.addRow(QLabel(u'%s:' % prop.label), widget) props_widgets[prop.id] = widget vbox.addLayout(formlayout) buttonBox = QDialogButtonBox(dialog) buttonBox.setStandardButtons(QDialogButtonBox.Ok|QDialogButtonBox.Cancel) buttonBox.accepted.connect(dialog.accept) buttonBox.rejected.connect(dialog.reject) vbox.addWidget(buttonBox) end = False while not end: end = True if dialog.exec_(): account = Account() account.properties = {} for key, widget in props_widgets.iteritems(): try: v = widget.get_value() except ValueError as e: QMessageBox.critical(self, self.tr('Invalid value'), self.tr('Invalid value for field "%s":

%s') % (key, e)) end = False break else: account.properties[key] = v if end: try: module.klass.register_account(account) except AccountRegisterError as e: QMessageBox.critical(self, self.tr('Error during register'), self.tr('Unable to register account %s:

%s') % (website, e)) end = False else: for key, value in account.properties.iteritems(): if key in self.config_widgets: self.config_widgets[key][1].set_value(value) def run(self): self.exec_() ret = (len(self.to_load) > 0 or len(self.to_unload) > 0) self.weboob.unload_backends(self.to_unload) self.weboob.load_backends(names=self.to_load) return ret weboob-1.2/weboob/tools/application/qt5/backendcfg.ui000066400000000000000000000227641303450110500226730ustar00rootroot00000000000000 BackendCfg 0 0 622 516 Backends configuration 6 4 QFrame::StyledPanel QFrame::Raised 4 Update modules Repositories Qt::Horizontal 40 20 Qt::Vertical QAbstractItemView::NoEditTriggers 24 24 false false false true true false false false true Name Module QFrame::StyledPanel QFrame::Raised Add Remove Qt::Vertical 20 40 QFrame::StyledPanel QFrame::Raised Available modules: 0 0 24 24 1 true 1 0 QFrame::NoFrame QFrame::Plain 0 0 true QFormLayout::ExpandingFieldsGrow Proxy: false Register an account... Name: Qt::Horizontal QDialogButtonBox::Cancel|QDialogButtonBox::Ok QDialogButtonBox::Close backendsList addButton removeButton modulesList moduleInfo nameEdit proxyBox proxyEdit registerButton configButtonBox buttonBox buttonBox clicked(QAbstractButton*) BackendCfg accept() 312 591 312 306 weboob-1.2/weboob/tools/application/qt5/models.py000066400000000000000000000056201303450110500221120ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2016 weboob project # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from PyQt5.QtCore import Qt from PyQt5.QtGui import QIcon, QImage, QPixmap, QPixmapCache, \ QStandardItemModel, QStandardItem __all__ = ['BackendListModel'] class BackendListModel(QStandardItemModel): """Model for displaying a backends list with icons""" RoleBackendName = Qt.UserRole RoleCapability = Qt.UserRole + 1 def __init__(self, weboob, *args, **kwargs): super(BackendListModel, self).__init__(*args, **kwargs) self.weboob = weboob def addBackends(self, cap=None, entry_all=True, entry_title=False): """ Populate the model by adding backends. Appends backends to the model, without clearing previous entries. For each entry in the model, the cap name is stored under role RoleBackendName and the capability object under role RoleCapability. :param cap: capabilities to add (None to add all loaded caps) :param entry_all: if True, add a "All backends" entry :param entry_title: if True, add a disabled entry with the cap name """ if entry_title: if cap: capname = cap.__name__ else: capname = '(All capabilities)' item = QStandardItem(capname) item.setEnabled(False) self.appendRow(item) first = True for backend in self.weboob.iter_backends(caps=cap): if first and entry_all: item = QStandardItem('(All backends)') item.setData('', Qt.UserRole) item.setData(cap, Qt.UserRole + 1) self.appendRow(item) first = False item = QStandardItem(backend.name) item.setData(backend.name, Qt.UserRole) item.setData(cap, Qt.UserRole + 1) minfo = self.weboob.repositories.get_module_info(backend.NAME) icon_path = self.weboob.repositories.get_module_icon_path(minfo) if icon_path: pixmap = QPixmapCache.find(icon_path) if not pixmap: pixmap = QPixmap(QImage(icon_path)) item.setIcon(QIcon(pixmap)) self.appendRow(item) weboob-1.2/weboob/tools/application/qt5/qt.py000066400000000000000000000330351303450110500212540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import sys import logging import re import gc from threading import Event from copy import copy from PyQt5.QtCore import QTimer, QObject, QSize, QVariant, QMutex, Qt from PyQt5.QtCore import pyqtSignal as Signal, pyqtSlot as Slot from PyQt5.QtWidgets import QApplication, QCheckBox, QComboBox, QInputDialog, \ QLineEdit, QMainWindow, QMessageBox, QSpinBox, \ QStyle, QStyledItemDelegate, QStyleOptionViewItem from PyQt5.QtGui import QTextDocument, QAbstractTextDocumentLayout, QPalette from weboob.core.ouiboube import Weboob, VersionsMismatchError from weboob.core.scheduler import IScheduler from weboob.tools.config.iconfig import ConfigError from weboob.exceptions import BrowserUnavailable, BrowserIncorrectPassword, BrowserForbidden, ModuleInstallError from weboob.tools.value import ValueInt, ValueBool, ValueBackendPassword from weboob.tools.misc import to_unicode from weboob.capabilities import UserError from ..base import Application, MoreResultsAvailable __all__ = ['QtApplication', 'QtMainWindow', 'QtDo', 'HTMLDelegate'] class QtScheduler(QObject, IScheduler): def __init__(self, app): QObject.__init__(self, parent=app) self.app = app self.params = {} def schedule(self, interval, function, *args): timer = QTimer() timer.setInterval(interval * 1000) timer.setSingleShot(True) self.params[timer] = (None, function, args) timer.timeout.connect(self.timeout) timer.start() def repeat(self, interval, function, *args): timer = QTimer() timer.setSingleShot(False) self.params[timer] = (interval, function, args) timer.start(0) timer.timeout.connect(self.timeout, Qt.QueuedConnection) @Slot() def timeout(self): timer = self.sender() interval, function, args = self.params[timer] function(*args) if interval is None: self.timers.pop(timer) else: timer.setInterval(interval * 1000) def want_stop(self): self.app.quit() def run(self): return self.app.exec_() class QCallbacksManager(QObject): class Request(object): def __init__(self): self.event = Event() self.answer = None def __call__(self): raise NotImplementedError() class LoginRequest(Request): def __init__(self, backend_name, value): QCallbacksManager.Request.__init__(self) self.backend_name = backend_name self.value = value def __call__(self): password, ok = QInputDialog.getText(None, '%s request' % self.value.label, 'Please enter %s for %s' % (self.value.label, self.backend_name), QLineEdit.Password) return password new_request = Signal() def __init__(self, weboob, parent=None): QObject.__init__(self, parent) self.weboob = weboob self.weboob.requests.register('login', self.callback(self.LoginRequest)) self.mutex = QMutex() self.requests = [] self.new_request.connect(self.do_request) def callback(self, klass): def cb(*args, **kwargs): return self.add_request(klass(*args, **kwargs)) return cb @Slot() def do_request(self): self.mutex.lock() request = self.requests.pop() request.answer = request() request.event.set() self.mutex.unlock() def add_request(self, request): self.mutex.lock() self.requests.append(request) self.mutex.unlock() self.new_request.emit() request.event.wait() return request.answer class QtApplication(QApplication, Application): def __init__(self): super(QtApplication, self).__init__(sys.argv) self.setApplicationName(self.APPNAME) self.cbmanager = QCallbacksManager(self.weboob, self) def create_weboob(self): return Weboob(scheduler=QtScheduler(self)) def load_backends(self, *args, **kwargs): while True: try: return Application.load_backends(self, *args, **kwargs) except VersionsMismatchError as e: msg = 'Versions of modules mismatch with version of weboob.' except ConfigError as e: msg = unicode(e) res = QMessageBox.question(None, 'Configuration error', u'%s\n\nDo you want to update repositories?' % msg, QMessageBox.Yes|QMessageBox.No) if res == QMessageBox.No: raise e # Do not import it globally, it causes circular imports from .backendcfg import ProgressDialog pd = ProgressDialog('Update of repositories', "Cancel", 0, 100) pd.setWindowModality(Qt.WindowModal) try: self.weboob.update(pd) except ModuleInstallError as err: QMessageBox.critical(None, self.tr('Update error'), self.tr('Unable to update repositories: %s' % err), QMessageBox.Ok) pd.setValue(100) QMessageBox.information(None, self.tr('Update of repositories'), self.tr('Repositories updated!'), QMessageBox.Ok) def deinit(self): super(QtApplication, self).deinit() gc.collect() class QtMainWindow(QMainWindow): def __init__(self, parent=None): QMainWindow.__init__(self, parent) class QtDo(QObject): gotResponse = Signal(object) gotError = Signal(object, object, object) finished = Signal() def __init__(self, weboob, cb, eb=None, fb=None): QObject.__init__(self) if not eb: eb = self.default_eb self.weboob = weboob self.process = None self.cb = cb self.eb = eb self.fb = fb self.gotResponse.connect(self.local_cb) self.gotError.connect(self.local_eb) self.finished.connect(self.local_fb) def do(self, *args, **kwargs): self.process = self.weboob.do(*args, **kwargs) self.process.callback_thread(self.thread_cb, self.thread_eb, self.thread_fb) def stop(self, wait=False): if self.process is not None: self.process.stop(wait) @Slot(object, object, object) def default_eb(self, backend, error, backtrace): if isinstance(error, MoreResultsAvailable): # This is not an error, ignore. return msg = unicode(error) if isinstance(error, BrowserIncorrectPassword): if not msg: msg = 'Invalid login/password.' elif isinstance(error, BrowserUnavailable): if not msg: msg = 'Website is unavailable.' elif isinstance(error, BrowserForbidden): if not msg: msg = 'This action is forbidden.' elif isinstance(error, NotImplementedError): msg = u'This feature is not supported by this backend.\n\n' \ u'To help the maintainer of this backend implement this feature, please contact: %s <%s>' % (backend.MAINTAINER, backend.EMAIL) elif isinstance(error, UserError): if not msg: msg = type(error).__name__ elif logging.root.level <= logging.DEBUG: msg += u'
' ul_opened = False for line in backtrace.split('\n'): m = re.match(' File (.*)', line) if m: if not ul_opened: msg += u'
    ' ul_opened = True else: msg += u'' msg += u'
  • %s' % m.group(1) else: msg += u'
    %s' % to_unicode(line) if ul_opened: msg += u'
' print(error, file=sys.stderr) print(backtrace, file=sys.stderr) QMessageBox.critical(None, self.tr('Error with backend %s') % backend.name, msg, QMessageBox.Ok) @Slot(object) def local_cb(self, data): if self.cb: self.cb(data) @Slot(object, object, object) def local_eb(self, backend, error, backtrace): if self.eb: self.eb(backend, error, backtrace) @Slot() def local_fb(self): if self.fb: self.fb() self.gotResponse.disconnect(self.local_cb) self.gotError.disconnect(self.local_eb) self.finished.disconnect(self.local_fb) self.process = None def thread_cb(self, data): self.gotResponse.emit(data) def thread_eb(self, backend, error, backtrace): self.gotError.emit(backend, error, backtrace) def thread_fb(self): self.finished.emit() class HTMLDelegate(QStyledItemDelegate): def paint(self, painter, option, index): option = QStyleOptionViewItem(option) # copy option self.initStyleOption(option, index) style = option.widget.style() if option.widget else QApplication.style() doc = QTextDocument() doc.setHtml(option.text) # painting item without text option.text = "" style.drawControl(QStyle.CE_ItemViewItem, option, painter) ctx = QAbstractTextDocumentLayout.PaintContext() # Hilight text if item is selected if option.state & QStyle.State_Selected: ctx.palette.setColor(QPalette.Text, option.palette.color(QPalette.Active, QPalette.HighlightedText)) textRect = style.subElementRect(QStyle.SE_ItemViewItemText, option) painter.save() painter.translate(textRect.topLeft()) painter.setClipRect(textRect.translated(-textRect.topLeft())) doc.documentLayout().draw(painter, ctx) painter.restore() def sizeHint(self, option, index): self.initStyleOption(option, index) doc = QTextDocument() doc.setHtml(option.text) doc.setTextWidth(option.rect.width()) return QSize(doc.idealWidth(), max(doc.size().height(), option.decorationSize.height())) class _QtValueStr(QLineEdit): def __init__(self, value): QLineEdit.__init__(self) self._value = value if value.default: self.setText(unicode(value.default)) if value.masked: self.setEchoMode(self.Password) def set_value(self, value): self._value = value self.setText(self._value.get()) def get_value(self): self._value.set(self.text()) return self._value class _QtValueBackendPassword(_QtValueStr): def get_value(self): self._value._domain = None return _QtValueStr.get_value(self) class _QtValueBool(QCheckBox): def __init__(self, value): QCheckBox.__init__(self) self._value = value if value.default: self.setChecked(True) def set_value(self, value): self._value = value self.setChecked(self._value.get()) def get_value(self): self._value.set(self.isChecked()) return self._value class _QtValueInt(QSpinBox): def __init__(self, value): QSpinBox.__init__(self) self._value = value if value.default: self.setValue(int(value.default)) def set_value(self, value): self._value = value self.setValue(self._value.get()) def get_value(self): self._value.set(self.getValue()) return self._value class _QtValueChoices(QComboBox): def __init__(self, value): QComboBox.__init__(self) self._value = value for k, l in value.choices.iteritems(): self.addItem(l, QVariant(k)) if value.default == k: self.setCurrentIndex(self.count()-1) def set_value(self, value): self._value = value for i in xrange(self.count()): if self.itemData(i) == self._value.get(): self.setCurrentIndex(i) return def get_value(self): self._value.set(self.itemData(self.currentIndex())) return self._value def QtValue(value): if isinstance(value, ValueBool): klass = _QtValueBool elif isinstance(value, ValueInt): klass = _QtValueInt elif isinstance(value, ValueBackendPassword): klass = _QtValueBackendPassword elif value.choices is not None: klass = _QtValueChoices else: klass = _QtValueStr return klass(copy(value)) # if the default excepthook is used, PyQt 5.5 *aborts* the app when an unhandled exception occurs # see http://pyqt.sourceforge.net/Docs/PyQt5/incompatibilities.html # as this behaviour is questionable, we restore the old one if sys.excepthook is sys.__excepthook__: sys.excepthook = lambda *args: sys.__excepthook__(*args) weboob-1.2/weboob/tools/application/qt5/reposdlg.ui000066400000000000000000000022351303450110500224320ustar00rootroot00000000000000 RepositoriesDlg 0 0 400 300 Repositories Qt::Horizontal QDialogButtonBox::Cancel|QDialogButtonBox::Ok buttonBox rejected() RepositoriesDlg reject() 316 260 286 274 weboob-1.2/weboob/tools/application/qt5/search_history.py000066400000000000000000000041721303450110500236560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 weboob project # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import codecs import os from PyQt5.QtWidgets import QCompleter from PyQt5.QtCore import QStringListModel __all__ = ['HistoryCompleter'] class HistoryCompleter(QCompleter): def __init__(self, hist_path, *args, **kwargs): super(HistoryCompleter, self).__init__(*args, **kwargs) self.setModel(QStringListModel()) self.max_history = 50 self.search_history = [] self.hist_path = hist_path def addString(self, s): if not s: return if len(self.search_history) > self.max_history: self.search_history.pop(0) if s not in self.search_history: self.search_history.append(s) self.updateCompletion() def updateCompletion(self): self.model().setStringList(self.search_history) def load(self): """ Return search string history list loaded from history file """ self.search_history = [] if os.path.exists(self.hist_path): with codecs.open(self.hist_path, 'r', 'utf-8') as f: conf_hist = f.read().strip() if conf_hist: self.search_history = conf_hist.split('\n') self.updateCompletion() def save(self): """ Save search history in history file. """ if len(self.search_history) > 0: with codecs.open(self.hist_path, 'w', 'utf-8') as f: f.write('\n'.join(self.search_history)) weboob-1.2/weboob/tools/application/repl.py000066400000000000000000001342331303450110500210630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Christophe Benz, Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import atexit from cmd import Cmd import logging import re from optparse import OptionGroup, OptionParser, IndentedHelpFormatter from datetime import datetime import os from weboob.capabilities.base import FieldNotFound, BaseObject, UserError from weboob.core import CallErrors from weboob.tools.application.formatters.iformatter import MandatoryFieldsNotFound from weboob.tools.misc import to_unicode from weboob.tools.path import WorkingPath from weboob.tools.ordereddict import OrderedDict from weboob.capabilities.collection import Collection, BaseCollection, CapCollection, CollectionNotFound from .console import BackendNotGiven, ConsoleApplication from .formatters.load import FormattersLoader, FormatterLoadError from .results import ResultsCondition, ResultsConditionError __all__ = ['NotEnoughArguments', 'ReplApplication'] class NotEnoughArguments(Exception): pass class ReplOptionParser(OptionParser): def format_option_help(self, formatter=None): if not formatter: formatter = self.formatter return '%s\n%s' % (formatter.format_commands(self.commands), OptionParser.format_option_help(self, formatter)) class ReplOptionFormatter(IndentedHelpFormatter): def format_commands(self, commands): s = u'' for section, cmds in commands.iteritems(): if len(cmds) == 0: continue if len(s) > 0: s += '\n' s += '%s Commands:\n' % section for c in cmds: c = c.split('\n')[0] s += ' %s\n' % c return s def defaultcount(default_count=10): def deco(f): def inner(self, *args, **kwargs): oldvalue = self.options.count if self._is_default_count: self.options.count = default_count try: return f(self, *args, **kwargs) finally: self.options.count = oldvalue inner.__doc__ = f.__doc__ assert inner.__doc__ is not None, "A command must have a docstring" inner.__doc__ += '\nDefault is limited to %s results.' % default_count return inner return deco class ReplApplication(Cmd, ConsoleApplication): """ Base application class for Repl applications. """ SYNOPSIS = 'Usage: %prog [-dqv] [-b backends] [-cnfs] [command [arguments..]]\n' SYNOPSIS += ' %prog [--help] [--version]' DISABLE_REPL = False EXTRA_FORMATTERS = {} DEFAULT_FORMATTER = 'multiline' COMMANDS_FORMATTERS = {} # Objects to allow in do_ls / do_cd COLLECTION_OBJECTS = tuple() weboob_commands = set(['backends', 'condition', 'count', 'formatter', 'logging', 'select', 'quit', 'ls', 'cd']) hidden_commands = set(['EOF']) def __init__(self): Cmd.__init__(self) ConsoleApplication.__init__(self, ReplOptionParser(self.SYNOPSIS, version=self._get_optparse_version())) copyright = self.COPYRIGHT.replace('YEAR', '%d' % datetime.today().year).encode(self.encoding) self.intro = '\n'.join(('Welcome to %s%s%s v%s' % (self.BOLD, self.APPNAME, self.NC, self.VERSION), '', copyright, 'This program is free software: you can redistribute it and/or modify', 'it under the terms of the GNU Affero General Public License as published by', 'the Free Software Foundation, either version 3 of the License, or', '(at your option) any later version.', '', 'Type "help" to display available commands.', '', )) self.formatters_loader = FormattersLoader() for key, klass in self.EXTRA_FORMATTERS.iteritems(): self.formatters_loader.register_formatter(key, klass) self.formatter = None self.commands_formatters = self.COMMANDS_FORMATTERS.copy() commands_help = self.get_commands_doc() self._parser.commands = commands_help self._parser.formatter = ReplOptionFormatter() results_options = OptionGroup(self._parser, 'Results Options') results_options.add_option('-c', '--condition', help='filter result items to display given a boolean expression. See CONDITION section for the syntax') results_options.add_option('-n', '--count', type='int', help='limit number of results (from each backends)') results_options.add_option('-s', '--select', help='select result item keys to display (comma separated)') self._parser.add_option_group(results_options) formatting_options = OptionGroup(self._parser, 'Formatting Options') available_formatters = self.formatters_loader.get_available_formatters() formatting_options.add_option('-f', '--formatter', choices=available_formatters, help='select output formatter (%s)' % u', '.join(available_formatters)) formatting_options.add_option('--no-header', dest='no_header', action='store_true', help='do not display header') formatting_options.add_option('--no-keys', dest='no_keys', action='store_true', help='do not display item keys') formatting_options.add_option('-O', '--outfile', dest='outfile', help='file to export result') self._parser.add_option_group(formatting_options) self._interactive = False self.working_path = WorkingPath() self._change_prompt() @property def interactive(self): return self._interactive def _change_prompt(self): self.objects = [] self.collections = [] # XXX can't use bold prompt because: # 1. it causes problems when trying to get history (lines don't start # at the right place). # 2. when typing a line longer than term width, cursor goes at start # of the same line instead of new line. #self.prompt = self.BOLD + '%s> ' % self.APPNAME + self.NC if len(self.working_path.get()): wp_enc = unicode(self.working_path).encode(self.encoding) self.prompt = '%s:%s> ' % (self.APPNAME, wp_enc) else: self.prompt = '%s> ' % (self.APPNAME) def change_path(self, split_path): self.working_path.location(split_path) self._change_prompt() def add_object(self, obj): self.objects.append(obj) def _complete_object(self): return [obj.fullid for obj in self.objects] def parse_id(self, id, unique_backend=False): if self.interactive: try: obj = self.objects[int(id) - 1] except (IndexError, ValueError): # Try to find a shortcut in the cache for obj in self.objects: if id in obj.id: id = obj.fullid break else: if isinstance(obj, BaseObject): id = obj.fullid try: return ConsoleApplication.parse_id(self, id, unique_backend) except BackendNotGiven as e: backend_name = None while not backend_name: print('This command works with an unique backend. Availables:') for index, (name, backend) in enumerate(e.backends): print('%s%d)%s %s%-15s%s %s' % (self.BOLD, index + 1, self.NC, self.BOLD, name, self.NC, backend.DESCRIPTION)) i = self.ask('Select a backend to proceed with "%s"' % id) if not i.isdigit(): if i not in dict(e.backends): print('Error: %s is not a valid backend' % i, file=self.stderr) continue backend_name = i else: i = int(i) if i < 0 or i > len(e.backends): print('Error: %s is not a valid choice' % i, file=self.stderr) continue backend_name = e.backends[i-1][0] return id, backend_name def get_object(self, _id, method, fields=None, caps=None): if self.interactive: try: obj = self.objects[int(_id) - 1] except (IndexError, ValueError): pass else: try: backend = self.weboob.get_backend(obj.backend) actual_method = getattr(backend, method, None) if actual_method is None: return None else: if callable(actual_method): return backend.fillobj(obj, fields) else: return None except UserError as e: self.bcall_error_handler(backend, e, '') _id, backend_name = self.parse_id(_id) kargs = {} if caps is not None: kargs = {'caps': caps} backend_names = (backend_name,) if backend_name is not None else self.enabled_backends # if backend's service returns several objects, try to find the one # with wanted ID. If not found, get the last not None object. obj = None # remove backends that do not have the required method new_backend_names = [] for backend in backend_names: if isinstance(backend, (str, unicode)): actual_backend = self.weboob.get_backend(backend) else: actual_backend = backend if getattr(actual_backend, method, None) is not None: new_backend_names.append(backend) backend_names = tuple(new_backend_names) try: for objiter in self.do(method, _id, backends=backend_names, fields=fields, **kargs): if objiter: obj = objiter if objiter.id == _id: return obj except CallErrors as e: if obj is not None: self.bcall_errors_handler(e) else: raise return obj def get_object_list(self, method=None, *args, **kwargs): # return cache if not empty if len(self.objects) > 0: return self.objects elif method is not None: kwargs['backends'] = self.enabled_backends for _object in self.do(method, *args, **kwargs): self.add_object(_object) return self.objects # XXX: what can we do without method? return tuple() def unload_backends(self, *args, **kwargs): self.objects = [] self.collections = [] return ConsoleApplication.unload_backends(self, *args, **kwargs) def load_backends(self, *args, **kwargs): self.objects = [] self.collections = [] return ConsoleApplication.load_backends(self, *args, **kwargs) def main(self, argv): cmd_args = argv[1:] if cmd_args: cmd_line = u' '.join(cmd_args) cmds = cmd_line.split(';') for cmd in cmds: ret = self.onecmd(cmd) if ret: return ret elif self.DISABLE_REPL: self._parser.print_help() self._parser.exit() else: try: import readline except ImportError: pass else: # Remove '-' from delims readline.set_completer_delims(readline.get_completer_delims().replace('-', '')) history_filepath = os.path.join(self.weboob.workdir, '%s_history' % self.APPNAME) try: readline.read_history_file(history_filepath) except IOError: pass def savehist(): readline.write_history_file(history_filepath) atexit.register(savehist) self.intro += '\nLoaded backends: %s\n' % ', '.join(sorted(backend.name for backend in self.weboob.iter_backends())) self._interactive = True self.cmdloop() def do(self, function, *args, **kwargs): """ Call Weboob.do(), passing count and selected fields given by user. """ backends = kwargs.pop('backends', None) if backends is None: kwargs['backends'] = [] for backend in self.enabled_backends: actual_function = getattr(backend, function, None) if isinstance(function, basestring) else function if callable(actual_function): kwargs['backends'].append(backend) else: kwargs['backends'] = backends fields = kwargs.pop('fields', self.selected_fields) if not fields and fields != []: fields = self.selected_fields fields = self.parse_fields(fields) if fields and self.formatter.MANDATORY_FIELDS is not None: missing_fields = set(self.formatter.MANDATORY_FIELDS) - set(fields) # If a mandatory field is not selected, do not use the customized formatter if missing_fields: print('Warning: you do not select enough mandatory fields for the formatter. Fallback to another. Hint: use option -f', file=self.stderr) self.formatter = self.formatters_loader.build_formatter(ReplApplication.DEFAULT_FORMATTER) if self.formatter.DISPLAYED_FIELDS is not None: if fields is None: missing_fields = True else: missing_fields = set(fields) - set(self.formatter.DISPLAYED_FIELDS + self.formatter.MANDATORY_FIELDS) # If a selected field is not displayed, do not use the customized formatter if missing_fields: print('Warning: some selected fields will not be displayed by the formatter. Fallback to another. Hint: use option -f', file=self.stderr) self.formatter = self.formatters_loader.build_formatter(ReplApplication.DEFAULT_FORMATTER) return self.weboob.do(self._do_complete, self.options.count, fields, function, *args, **kwargs) # -- command tools ------------ def parse_command_args(self, line, nb, req_n=None): if line.strip() == '': # because ''.split() = [''] args = [] else: args = line.strip().split(' ', nb - 1) if req_n is not None and (len(args) < req_n): raise NotEnoughArguments('Command needs %d arguments' % req_n) if len(args) < nb: args += tuple(None for i in xrange(nb - len(args))) return args # -- cmd.Cmd methods --------- def postcmd(self, stop, line): """ This REPL method is overrided to return None instead of integers to prevent stopping cmdloop(). """ if not isinstance(stop, bool): stop = None return stop def parseline(self, line): """ This REPL method is overrided to search "short" alias of commands """ cmd, arg, ignored = Cmd.parseline(self, line) if cmd is not None: names = set(name for name in self.get_names() if name.startswith('do_')) if 'do_' + cmd not in names: long = set(name for name in names if name.startswith('do_' + cmd)) # if more than one result, ambiguous command, do nothing (error will display suggestions) if len(long) == 1: cmd = long.pop()[3:] return cmd, arg, ignored def onecmd(self, line): """ This REPL method is overrided to catch some particular exceptions. """ line = to_unicode(line) cmd, arg, ignored = self.parseline(line) # Set the right formatter for the command. try: formatter_name = self.commands_formatters[cmd] except KeyError: formatter_name = self.DEFAULT_FORMATTER self.set_formatter(formatter_name) try: try: return super(ReplApplication, self).onecmd(line) except CallErrors as e: self.bcall_errors_handler(e) except BackendNotGiven as e: print('Error: %s' % str(e), file=self.stderr) except NotEnoughArguments as e: print('Error: not enough arguments. %s' % str(e), file=self.stderr) except (KeyboardInterrupt, EOFError): # ^C during a command process doesn't exit application. print('\nAborted.') finally: self.flush() def emptyline(self): """ By default, an emptyline repeats the previous command. Overriding this function disables this behaviour. """ pass def default(self, line): print('Unknown command: "%s"' % line, file=self.stderr) cmd, arg, ignore = Cmd.parseline(self, line) if cmd is not None: names = set(name[3:] for name in self.get_names() if name.startswith('do_' + cmd)) if len(names) > 0: print('Do you mean: %s?' % ', '.join(names), file=self.stderr) return 2 def completenames(self, text, *ignored): return [name for name in Cmd.completenames(self, text, *ignored) if name not in self.hidden_commands] def path_completer(self, arg): dirname = os.path.dirname(arg) try: children = os.listdir(dirname or '.') except OSError: return () l = [] for child in children: path = os.path.join(dirname, child) if os.path.isdir(path): child += '/' l.append(child) return l def complete(self, text, state): """ Override of the Cmd.complete() method to: * add a space at end of proposals * display only proposals for words which match the text already written by user. """ super(ReplApplication, self).complete(text, state) # When state = 0, Cmd.complete() set the 'completion_matches' attribute by # calling the completion function. Then, for other states, it only tries to # get the right item in list. # So that's the good place to rework the choices. if state == 0: self.completion_matches = [choice for choice in self.completion_matches if choice.startswith(text)] try: match = self.completion_matches[state] except IndexError: return None else: if match[-1] != '/': return '%s ' % match return match # -- errors management ------------- def bcall_error_handler(self, backend, error, backtrace): """ Handler for an exception inside the CallErrors exception. This method can be overrided to support more exceptions types. """ return super(ReplApplication, self).bcall_error_handler(backend, error, backtrace) def bcall_errors_handler(self, errors, ignore=()): if self.interactive: ConsoleApplication.bcall_errors_handler(self, errors, 'Use "logging debug" option to print backtraces.', ignore) else: ConsoleApplication.bcall_errors_handler(self, errors, ignore=ignore) # -- options related methods ------------- def _handle_options(self): if self.options.formatter: self.commands_formatters = {} self.DEFAULT_FORMATTER = self.options.formatter self.set_formatter(self.DEFAULT_FORMATTER) if self.options.select: self.selected_fields = self.options.select.split(',') else: self.selected_fields = ['$direct'] if self.options.count is not None: self._is_default_count = False if self.options.count <= 0: # infinite search self.options.count = None if self.options.condition: self.condition = ResultsCondition(self.options.condition) else: self.condition = None return super(ReplApplication, self)._handle_options() def get_command_help(self, command, short=False): try: func = getattr(self, 'do_' + command) except AttributeError: return None doc = func.__doc__ assert doc is not None, "A command must have a docstring" lines = [line.strip() for line in doc.strip().split('\n')] if not lines[0].startswith(command): lines = [command, ''] + lines if short: return lines[0] return '\n'.join(lines) def get_commands_doc(self): names = set(name for name in self.get_names() if name.startswith('do_')) appname = self.APPNAME.capitalize() d = OrderedDict(((appname, []), ('Weboob', []))) for name in sorted(names): cmd = name[3:] if cmd in self.hidden_commands.union(self.weboob_commands).union(['help']): continue d[appname].append(self.get_command_help(cmd)) if not self.DISABLE_REPL: for cmd in self.weboob_commands: d['Weboob'].append(self.get_command_help(cmd)) return d # -- default REPL commands --------- def do_quit(self, arg): """ Quit the application. """ return True def do_EOF(self, arg): """ Quit the command line interpreter when ^D is pressed. """ # print empty line for the next shell prompt to appear on the first column of the terminal print() return self.do_quit(arg) def do_help(self, arg=None): """ help [COMMAND] List commands, or get information about a command. """ if arg: cmd_names = set(name[3:] for name in self.get_names() if name.startswith('do_')) if arg in cmd_names: command_help = self.get_command_help(arg) if command_help is None: logging.warning(u'Command "%s" is undocumented' % arg) else: lines = command_help.split('\n') lines[0] = '%s%s%s' % (self.BOLD, lines[0], self.NC) self.stdout.write('%s\n' % '\n'.join(lines)) else: print('Unknown command: "%s"' % arg, file=self.stderr) else: cmds = self._parser.formatter.format_commands(self._parser.commands) self.stdout.write('%s\n' % cmds) self.stdout.write('Type "help " for more info about a command.\n') return 2 def complete_backends(self, text, line, begidx, endidx): choices = [] commands = ['enable', 'disable', 'only', 'list', 'add', 'register', 'edit', 'remove', 'list-modules'] available_backends_names = set(backend.name for backend in self.weboob.iter_backends()) enabled_backends_names = set(backend.name for backend in self.enabled_backends) args = line.split(' ') if len(args) == 2: choices = commands elif len(args) >= 3: if args[1] == 'enable': choices = sorted(available_backends_names - enabled_backends_names) elif args[1] == 'only': choices = sorted(available_backends_names) elif args[1] == 'disable': choices = sorted(enabled_backends_names) elif args[1] in ('add', 'register') and len(args) == 3: for name, module in sorted(self.weboob.repositories.get_all_modules_info(self.CAPS).iteritems()): choices.append(name) elif args[1] == 'edit': choices = sorted(available_backends_names) elif args[1] == 'remove': choices = sorted(available_backends_names) return choices def do_backends(self, line): """ backends [ACTION] [BACKEND_NAME]... Select used backends. ACTION is one of the following (default: list): * enable enable given backends * disable disable given backends * only enable given backends and disable the others * list list backends * add add a backend * register register a new account on a website * edit edit a backend * remove remove a backend * list-modules list modules """ line = line.strip() if line: args = line.split() else: args = ['list'] action = args[0] given_backend_names = args[1:] for backend_name in given_backend_names: if action in ('add', 'register'): minfo = self.weboob.repositories.get_module_info(backend_name) if minfo is None: print('Module "%s" does not exist.' % backend_name, file=self.stderr) return 1 else: if not minfo.has_caps(self.CAPS): print('Module "%s" is not supported by this application => skipping.' % backend_name, file=self.stderr) return 1 else: if backend_name not in [backend.name for backend in self.weboob.iter_backends()]: print('Backend "%s" does not exist => skipping.' % backend_name, file=self.stderr) return 1 if action in ('enable', 'disable', 'only', 'add', 'register', 'edit', 'remove'): if not given_backend_names: print('Please give at least a backend name.', file=self.stderr) return 2 given_backends = set(backend for backend in self.weboob.iter_backends() if backend.name in given_backend_names) if action == 'enable': for backend in given_backends: self.enabled_backends.add(backend) elif action == 'disable': for backend in given_backends: try: self.enabled_backends.remove(backend) except KeyError: print('%s is not enabled' % backend.name, file=self.stderr) elif action == 'only': self.enabled_backends = set() for backend in given_backends: self.enabled_backends.add(backend) elif action == 'list': enabled_backends_names = set(backend.name for backend in self.enabled_backends) disabled_backends_names = set(backend.name for backend in self.weboob.iter_backends()) - enabled_backends_names print('Enabled: %s' % ', '.join(enabled_backends_names)) if len(disabled_backends_names) > 0: print('Disabled: %s' % ', '.join(disabled_backends_names)) elif action == 'add': for name in given_backend_names: instname = self.add_backend(name, name) if instname: self.load_backends(names=[instname]) elif action == 'register': for name in given_backend_names: instname = self.register_backend(name) if isinstance(instname, basestring): self.load_backends(names=[instname]) elif action == 'edit': for backend in given_backends: enabled = backend in self.enabled_backends self.unload_backends(names=[backend.name]) self.edit_backend(backend.name) for newb in self.load_backends(names=[backend.name]).itervalues(): if not enabled: self.enabled_backends.remove(newb) elif action == 'remove': for backend in given_backends: self.weboob.backends_config.remove_backend(backend.name) self.unload_backends(backend.name) elif action == 'list-modules': modules = [] print('Modules list:') for name, info in sorted(self.weboob.repositories.get_all_modules_info().iteritems()): if not self.is_module_loadable(info): continue modules.append(name) loaded = ' ' for bi in self.weboob.iter_backends(): if bi.NAME == name: if loaded == ' ': loaded = 'X' elif loaded == 'X': loaded = 2 else: loaded += 1 print('[%s] %s%-15s%s %s' % (loaded, self.BOLD, name, self.NC, info.description)) else: print('Unknown action: "%s"' % action, file=self.stderr) return 1 if len(self.enabled_backends) == 0: print('Warning: no more backends are loaded. %s is probably unusable.' % self.APPNAME.capitalize(), file=self.stderr) def complete_logging(self, text, line, begidx, endidx): levels = ('debug', 'info', 'warning', 'error', 'quiet', 'default') args = line.split(' ') if len(args) == 2: return levels return () def do_logging(self, line): """ logging [LEVEL] Set logging level. Availables: debug, info, warning, error. * quiet is an alias for error * default is an alias for warning """ args = self.parse_command_args(line, 1, 0) levels = (('debug', logging.DEBUG), ('info', logging.INFO), ('warning', logging.WARNING), ('error', logging.ERROR), ('quiet', logging.ERROR), ('default', logging.WARNING) ) if not args[0]: current = None for label, level in levels: if logging.root.level == level: current = label break print('Current level: %s' % current) return levels = dict(levels) try: level = levels[args[0]] except KeyError: print('Level "%s" does not exist.' % args[0], file=self.stderr) print('Availables: %s' % ' '.join(levels.iterkeys()), file=self.stderr) return 2 else: logging.root.setLevel(level) for handler in logging.root.handlers: handler.setLevel(level) def do_condition(self, line): """ condition [EXPRESSION | off] If an argument is given, set the condition expression used to filter the results. See CONDITION section for more details and the expression. If the "off" value is given, conditional filtering is disabled. If no argument is given, print the current condition expression. """ line = line.strip() if line: if line == 'off': self.condition = None else: try: self.condition = ResultsCondition(line) except ResultsConditionError as e: print('%s' % e, file=self.stderr) return 2 else: if self.condition is None: print('No condition is set.') else: print(str(self.condition)) def do_count(self, line): """ count [NUMBER | off] If an argument is given, set the maximum number of results fetched. NUMBER must be at least 1. "off" value disables counting, and allows infinite searches. If no argument is given, print the current count value. """ line = line.strip() if line: if line == 'off': self.options.count = None self._is_default_count = False else: try: count = int(line) except ValueError: print('Could not interpret "%s" as a number.' % line, file=self.stderr) return 2 else: if count > 0: self.options.count = count self._is_default_count = False else: self.options.count = None self._is_default_count = False else: if self.options.count is None: print('Counting disabled.') else: print(self.options.count) def complete_formatter(self, text, line, *ignored): formatters = self.formatters_loader.get_available_formatters() commands = ['list', 'option'] + formatters options = ['header', 'keys'] option_values = ['on', 'off'] args = line.split(' ') if len(args) == 2: return commands if args[1] == 'option': if len(args) == 3: return options if len(args) == 4: return option_values elif args[1] in formatters: return list(set(name[3:] for name in self.get_names() if name.startswith('do_'))) def do_formatter(self, line): """ formatter [list | FORMATTER [COMMAND] | option OPTION_NAME [on | off]] If a FORMATTER is given, set the formatter to use. You can add a COMMAND to apply the formatter change only to a given command. If the argument is "list", print the available formatters. If the argument is "option", set the formatter options. Valid options are: header, keys. If on/off value is given, set the value of the option. If not, print the current value for the option. If no argument is given, print the current formatter. """ args = line.strip().split() if args: if args[0] == 'list': print(', '.join(self.formatters_loader.get_available_formatters())) elif args[0] == 'option': if len(args) > 1: if len(args) == 2: if args[1] == 'header': print('off' if self.options.no_header else 'on') elif args[1] == 'keys': print('off' if self.options.no_keys else 'on') else: if args[2] not in ('on', 'off'): print('Invalid value "%s". Please use "on" or "off" values.' % args[2], file=self.stderr) return 2 else: if args[1] == 'header': self.options.no_header = True if args[2] == 'off' else False elif args[1] == 'keys': self.options.no_keys = True if args[2] == 'off' else False else: print('Don\'t know which option to set. Available options: header, keys.', file=self.stderr) return 2 else: if args[0] in self.formatters_loader.get_available_formatters(): if len(args) > 1: self.commands_formatters[args[1]] = self.set_formatter(args[0]) else: self.commands_formatters = {} self.DEFAULT_FORMATTER = self.set_formatter(args[0]) else: print('Formatter "%s" is not available.\n' 'Available formatters: %s.' % (args[0], ', '.join(self.formatters_loader.get_available_formatters())), file=self.stderr) return 1 else: print('Default formatter: %s' % self.DEFAULT_FORMATTER) for key, klass in self.commands_formatters.iteritems(): print('Command "%s": %s' % (key, klass)) def do_select(self, line): """ select [FIELD_NAME]... | "$direct" | "$full" If an argument is given, set the selected fields. $direct selects all fields loaded in one http request. $full selects all fields using as much http requests as necessary. If no argument is given, print the currently selected fields. """ line = line.strip() if line: split = line.split() self.selected_fields = split else: print(' '.join(self.selected_fields)) # First sort in alphabetical of backend # Second, sort with ID def comp_object(self, obj1, obj2): if obj1.backend == obj2.backend: if obj1.id == obj2.id: return 0 elif obj1.id > obj2.id: return 1 else: return -1 elif obj1.backend > obj2.backend: return 1 return -1 @defaultcount(40) def do_ls(self, line): """ ls [-d] [-U] [PATH] List objects in current path. If an argument is given, list the specified path. Use -U option to not sort results. It allows you to use a "fast path" to return results as soon as possible. Use -d option to display information about a collection (and to not display the content of it). It has the same behavior than the well known UNIX "ls" command. """ # TODO: real parsing of options path = line.strip() only = False sort = True if '-U' in line.strip().partition(' '): path = line.strip().partition(' ')[-1] sort = False if '-d' in line.strip().partition(' '): path = None only = line.strip().partition(' ')[-1] if path: for _path in path.split('/'): # We have an argument, let's ch to the directory before the ls self.working_path.cd1(_path) objects = [] collections = [] self.objects = [] self.start_format() for res in self._fetch_objects(objs=self.COLLECTION_OBJECTS): if isinstance(res, Collection): collections.append(res) if sort is False: self.formatter.format_collection(res, only) else: if sort: objects.append(res) else: self._format_obj(res, only) if sort: objects.sort(cmp=self.comp_object) collections = self._merge_collections_with_same_path(collections) collections.sort(cmp=self.comp_object) for collection in collections: self.formatter.format_collection(collection, only) for obj in objects: self._format_obj(obj, only) if path: for _path in path.split('/'): # Let's go back to the parent directory self.working_path.up() else: # Save collections only if we listed the current path. self.collections = collections def _find_collection(self, collection, collections): for col in collections: if col.split_path == collection.split_path: return col return None def _merge_collections_with_same_path(self, collections): to_return = [] for collection in collections: col = self._find_collection(collection, to_return) if col: col.backend += " %s" % collection.backend else: to_return.append(collection) return to_return def _format_obj(self, obj, only): if only is False or not hasattr(obj, 'id') or obj.id in only: self.cached_format(obj) def do_cd(self, line): """ cd [PATH] Follow a path. ".." is a special case and goes up one directory. "" is a special case and goes home. """ if not len(line.strip()): self.working_path.home() elif line.strip() == '..': self.working_path.up() else: self.working_path.cd1(line) collections = [] try: for res in self.do('get_collection', objs=self.COLLECTION_OBJECTS, split_path=self.working_path.get(), caps=CapCollection): if res: collections.append(res) except CallErrors as errors: self.bcall_errors_handler(errors, CollectionNotFound) if len(collections): # update the path from the collection if possible if len(collections) == 1: self.working_path.split_path = collections[0].split_path else: print(u"Path: %s not found" % unicode(self.working_path), file=self.stderr) self.working_path.restore() return 1 self._change_prompt() def _fetch_objects(self, objs): split_path = self.working_path.get() try: for res in self.do('iter_resources', objs=objs, split_path=split_path, caps=CapCollection): yield res except CallErrors as errors: self.bcall_errors_handler(errors, CollectionNotFound) def all_collections(self): """ Get all objects that are collections: regular objects and fake dumb objects. """ obj_collections = [obj for obj in self.objects if isinstance(obj, BaseCollection)] return obj_collections + self.collections def obj_to_filename(self, obj, dest=None, default=None): """ This method can be used to get a filename from an object, using a mask filled by information of this object. All patterns are braces-enclosed, and are name of available fields in the object. :param obj: object :type obj: BaseObject :param dest: dest given by user (default None) :type dest: str :param default: default file mask (if not given, this is '{id}-{title}.{ext}') :type default: str :rtype: str """ if default is None: default = '{id}-{title}.{ext}' if dest is None: dest = '.' if os.path.isdir(dest): dest = os.path.join(dest, default) def repl(m): field = m.group(1) if hasattr(obj, field): return re.sub('[?:/]', '-', '%s' % getattr(obj, field)) else: return m.group(0) return re.sub(r'\{(.+?)\}', repl, dest) # for cd & ls def complete_path(self, text, line, begidx, endidx): directories = set() if len(self.working_path.get()): directories.add('..') mline = line.partition(' ')[2] offs = len(mline) - len(text) # refresh only if needed if len(self.objects) == 0 and len(self.collections) == 0: try: self.objects, self.collections = self._fetch_objects(objs=self.COLLECTION_OBJECTS) except CallErrors as errors: self.bcall_errors_handler(errors, CollectionNotFound) collections = self.all_collections() for collection in collections: directories.add(collection.basename.encode(self.encoding)) return [s[offs:] for s in directories if s.startswith(mline)] def complete_ls(self, text, line, begidx, endidx): return self.complete_path(text, line, begidx, endidx) def complete_cd(self, text, line, begidx, endidx): return self.complete_path(text, line, begidx, endidx) # -- formatting related methods ------------- def set_formatter(self, name): """ Set the current formatter from name. It returns the name of the formatter which has been really set. """ try: self.formatter = self.formatters_loader.build_formatter(name) except FormatterLoadError as e: print('%s' % e, file=self.stderr) if self.DEFAULT_FORMATTER == name: self.DEFAULT_FORMATTER = ReplApplication.DEFAULT_FORMATTER print('Falling back to "%s".' % (self.DEFAULT_FORMATTER), file=self.stderr) self.formatter = self.formatters_loader.build_formatter(self.DEFAULT_FORMATTER) name = self.DEFAULT_FORMATTER if self.options.no_header: self.formatter.display_header = False if self.options.no_keys: self.formatter.display_keys = False if self.options.outfile: self.formatter.outfile = self.options.outfile if self.interactive: self.formatter.interactive = True return name def set_formatter_header(self, string): pass def start_format(self, **kwargs): self.formatter.start_format(**kwargs) def cached_format(self, obj): self.add_object(obj) alias = None if self.interactive: alias = '%s' % len(self.objects) self.format(obj, alias=alias) def parse_fields(self, fields): if '$direct' in fields: return [] if '$full' in fields: return None return fields def format(self, result, alias=None): fields = self.parse_fields(self.selected_fields) try: self.formatter.format(obj=result, selected_fields=fields, alias=alias) except FieldNotFound as e: print(e, file=self.stderr) except MandatoryFieldsNotFound as e: print('%s Hint: select missing fields or use another formatter (ex: multiline).' % e, file=self.stderr) def flush(self): self.formatter.flush() weboob-1.2/weboob/tools/application/results.py000066400000000000000000000136601303450110500216220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities import UserError __all__ = ['ResultsCondition', 'ResultsConditionError'] class IResultsCondition(object): def is_valid(self, obj): raise NotImplementedError() class ResultsConditionError(UserError): pass class Condition(object): def __init__(self, left, op, right): self.left = left # Field of the object to test self.op = op self.right = right def is_egal(left, right): return left == right def is_notegal(left, right): return left != right def is_sup(left, right): return left < right def is_inf(left, right): return left > right def is_in(left, right): return left in right functions = {'!=': is_notegal, '=': is_egal, '>': is_sup, '<': is_inf, '|': is_in} class ResultsCondition(IResultsCondition): condition_str = None # Supported operators # =, !=, <, > for float/int/decimal # =, != for strings # We build a list of list. Return true if each conditions of one list is TRUE def __init__(self, condition_str): self.limit = None or_list = [] _condition_str = condition_str.split(' LIMIT ') if len(_condition_str) == 2: try: self.limit = int(_condition_str[1]) except ValueError: raise ResultsConditionError(u'Syntax error in the condition expression, please check documentation') condition_str= _condition_str[0] for _or in condition_str.split(' OR '): and_list = [] for _and in _or.split(' AND '): operator = None for op in ['!=', '=', '>', '<', '|']: if op in _and: operator = op break if operator is None: raise ResultsConditionError(u'Could not find a valid operator in sub-expression "%s". Protect the complete condition expression with quotes, or read the documentation in the man manual.' % _and) try: l, r = _and.split(operator) except ValueError: raise ResultsConditionError(u'Syntax error in the condition expression, please check documentation') and_list.append(Condition(l.strip(), operator, r.strip())) or_list.append(and_list) self.condition = or_list self.condition_str = condition_str def is_valid(self, obj): import weboob.tools.date as date_utils import re from datetime import date, datetime, timedelta d = obj.to_dict() # We evaluate all member of a list at each iteration. for _or in self.condition: myeval = True for condition in _or: if condition.left in d: # in the case of id, test id@backend and id if condition.left == 'id': tocompare = condition.right evalfullid = functions[condition.op](tocompare, d['id']) evalid = functions[condition.op](tocompare, obj.id) myeval = evalfullid or evalid else: # We have to change the type of v, always gived as string by application typed = type(d[condition.left]) try: if isinstance(d[condition.left], date_utils.date): tocompare = date(*[int(x) for x in condition.right.split('-')]) elif isinstance(d[condition.left], date_utils.datetime): splitted_datetime = condition.right.split(' ') tocompare = datetime(*([int(x) for x in splitted_datetime[0].split('-')] + [int(x) for x in splitted_datetime[1].split(':')])) elif isinstance(d[condition.left], timedelta): time_dict = re.match('^\s*((?P\d+)\s*h)?\s*((?P\d+)\s*m)?\s*((?P\d+)\s*s)?\s*$', condition.right).groupdict() tocompare = timedelta(seconds=int(time_dict['seconds'] or "0"), minutes=int(time_dict['minutes'] or "0"), hours=int(time_dict['hours'] or "0")) else: tocompare = typed(condition.right) myeval = functions[condition.op](tocompare, d[condition.left]) except: myeval = False else: raise ResultsConditionError(u'Field "%s" is not valid.' % condition.left) # Do not try all AND conditions if one is false if not myeval: break # Return True at the first OR valid condition if myeval: return True # If we are here, all OR conditions are False return False def __str__(self): return unicode(self).encode('utf-8') def __unicode__(self): return self.condition_str weboob-1.2/weboob/tools/backend.py000066400000000000000000000367201303450110500172070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import os from threading import RLock from copy import copy from weboob.capabilities.base import BaseObject, FieldNotFound, \ Capability, NotLoaded, NotAvailable from weboob.tools.misc import iter_fields from weboob.tools.log import getLogger from weboob.tools.value import ValuesDict from weboob.exceptions import ModuleInstallError __all__ = ['BackendStorage', 'BackendConfig', 'Module'] class BackendStorage(object): """ This is an abstract layer to store data in storages (:mod:`weboob.tools.storage`) easily. It is instancied automatically in constructor of :class:`Module`, in the :attr:`Module.storage` attribute. :param name: name of backend :param storage: storage object :type storage: :class:`weboob.tools.storage.IStorage` """ def __init__(self, name, storage): self.name = name self.storage = storage def set(self, *args): """ Set value in the storage. Example: >>> from weboob.tools.storage import StandardStorage >>> backend = BackendStorage('blah', StandardStorage('/tmp/cfg')) >>> backend.storage.set('config', 'nb_of_threads', 10) >>> :param args: the path where to store value """ if self.storage: return self.storage.set('backends', self.name, *args) def delete(self, *args): """ Delete a value from the storage. :param args: path to delete. """ if self.storage: return self.storage.delete('backends', self.name, *args) def get(self, *args, **kwargs): """ Get a value or a dict of values in storage. Example: >>> from weboob.tools.storage import StandardStorage >>> backend = BackendStorage('blah', StandardStorage('/tmp/cfg')) >>> backend.storage.get('config', 'nb_of_threads') 10 >>> backend.storage.get('config', 'unexistant', 'path', default='lol') 'lol' >>> backend.storage.get('config') {'nb_of_threads': 10, 'other_things': 'blah'} :param args: path to get :param default: if specified, default value when path is not found """ if self.storage: return self.storage.get('backends', self.name, *args, **kwargs) else: return kwargs.get('default', None) def load(self, default): """ Load storage. It is made automatically when your backend is created, and use the ``STORAGE`` class attribute as default. :param default: this is the default tree if storage is empty :type default: :class:`dict` """ if self.storage: return self.storage.load('backends', self.name, default) def save(self): """ Save storage. """ if self.storage: return self.storage.save('backends', self.name) class BackendConfig(ValuesDict): """ Configuration of a backend. This class is firstly instanced as a :class:`weboob.tools.value.ValuesDict`, containing some :class:`weboob.tools.value.Value` (and derivated) objects. Then, using the :func:`load` method will load configuration from file and create a copy of the :class:`BackendConfig` object with the loaded values. """ modname = None instname = None weboob = None def load(self, weboob, modname, instname, config, nofail=False): """ Load configuration from dict to create an instance. :param weboob: weboob object :type weboob: :class:`weboob.core.ouiboube.Weboob` :param modname: name of the module :type modname: :class:`str` :param instname: name of this backend :type instname: :class:`str` :param params: parameters to load :type params: :class:`dict` :param nofail: if true, this call can't fail :type nofail: :class:`bool` :rtype: :class:`BackendConfig` """ cfg = BackendConfig() cfg.modname = modname cfg.instname = instname cfg.weboob = weboob for name, field in self.iteritems(): value = config.get(name, None) if value is None: if not nofail and field.required: raise Module.ConfigError('Backend(%s): Configuration error: Missing parameter "%s" (%s)' % (cfg.instname, name, field.description)) value = field.default field = copy(field) try: field.load(cfg.instname, value, cfg.weboob.requests) except ValueError as v: if not nofail: raise Module.ConfigError( 'Backend(%s): Configuration error for field "%s": %s' % (cfg.instname, name, v)) cfg[name] = field return cfg def dump(self): """ Dump config in a dictionary. :rtype: :class:`dict` """ settings = {} for name, value in self.iteritems(): settings[name] = value.dump() return settings def save(self, edit=True, params=None): """ Save backend config. :param edit: if true, it changes config of an existing backend :type edit: :class:`bool` :param params: if specified, params to merge with the ones of the current object :type params: :class:`dict` """ assert self.modname is not None assert self.instname is not None assert self.weboob is not None dump = self.dump() if params is not None: dump.update(params) self.weboob.backends_config.add_backend(self.instname, self.modname, dump, edit) class Module(object): """ Base class for modules. You may derivate it, and also all capabilities you want to implement. :param weboob: weboob instance :type weboob: :class:`weboob.core.ouiboube.Weboob` :param name: name of backend :type name: :class:`str` :param config: configuration of backend :type config: :class:`dict` :param storage: storage object :type storage: :class:`weboob.tools.storage.IStorage` :param logger: logger :type logger: :class:`logging.Logger` """ # Module name. NAME = None # Name of the maintainer of this module. MAINTAINER = u'' # Email address of the maintainer. EMAIL = '' # Version of module (for information only). VERSION = '' # Description DESCRIPTION = '' # License of this module. LICENSE = '' # Configuration required for backends. # Values must be weboob.tools.value.Value objects. CONFIG = BackendConfig() # Storage STORAGE = {} # Browser class BROWSER = None # URL to an optional icon. # If you want to create your own icon, create a 'favicon.ico' ico in # the module's directory, and keep the ICON value to None. ICON = None # Supported objects to fill # The key is the class and the value the method to call to fill # Method prototype: method(object, fields) # When the method is called, fields are only the one which are # NOT yet filled. OBJECTS = {} class ConfigError(Exception): """ Raised when the config can't be loaded. """ def __enter__(self): self.lock.acquire() def __exit__(self, t, v, tb): self.lock.release() def __repr__(self): return "" % self.name def __init__(self, weboob, name, config=None, storage=None, logger=None, nofail=False): self.logger = getLogger(name, parent=logger) self.weboob = weboob self.name = name self.lock = RLock() if config is None: config = {} # Private fields (which start with '_') self._private_config = dict((key, value) for key, value in config.iteritems() if key.startswith('_')) # Load configuration of backend. self.config = self.CONFIG.load(weboob, self.NAME, self.name, config, nofail) self.storage = BackendStorage(self.name, storage) self.storage.load(self.STORAGE) def deinit(self): """ This abstract method is called when the backend is unloaded. """ if self._browser is None: return if hasattr(self.browser, 'dump_state'): self.storage.set('browser_state', self.browser.dump_state()) self.storage.save() if hasattr(self.browser, 'deinit'): self.browser.deinit() _browser = None @property def browser(self): """ Attribute 'browser'. The browser is created at the first call of this attribute, to avoid useless pages access. Note that the :func:`create_default_browser` method is called to create it. """ if self._browser is None: self._browser = self.create_default_browser() return self._browser def create_default_browser(self): """ Method to overload to build the default browser in attribute 'browser'. """ return self.create_browser() def create_browser(self, *args, **kwargs): """ Build a browser from the BROWSER class attribute and the given arguments. :param klass: optional parameter to give another browser class to instanciate :type klass: :class:`weboob.browser.browsers.Browser` """ klass = kwargs.pop('klass', self.BROWSER) if not klass: return None tmpproxy = None tmpproxys = None if '_proxy' in self._private_config: tmpproxy = self._private_config['_proxy'] elif 'http_proxy' in os.environ: tmpproxy = os.environ['http_proxy'] elif 'HTTP_PROXY' in os.environ: tmpproxy = os.environ['HTTP_PROXY'] if '_proxy_ssl' in self._private_config: tmpproxys = self._private_config['_proxy_ssl'] elif 'https_proxy' in os.environ: tmpproxys = os.environ['https_proxy'] elif 'HTTPS_PROXY' in os.environ: tmpproxys = os.environ['HTTPS_PROXY'] if any((tmpproxy, tmpproxys)): kwargs['proxy'] = {} if tmpproxy is not None: kwargs['proxy']['http'] = tmpproxy if tmpproxys is not None: kwargs['proxy']['https'] = tmpproxys kwargs['logger'] = self.logger if self.logger.settings['responses_dirname']: kwargs.setdefault('responses_dirname', os.path.join(self.logger.settings['responses_dirname'], self._private_config.get('_debug_dir', self.name))) elif os.path.isabs(self._private_config.get('_debug_dir', '')): kwargs.setdefault('responses_dirname', self._private_config['_debug_dir']) browser = klass(*args, **kwargs) if hasattr(browser, 'load_state'): browser.load_state(self.storage.get('browser_state', default={})) return browser @classmethod def iter_caps(klass): """ Iter capabilities implemented by this backend. :rtype: iter[:class:`weboob.capabilities.base.Capability`] """ def iter_caps(cls): for base in cls.__bases__: if issubclass(base, Capability) and base != Capability: yield base for cap in iter_caps(base): yield cap return iter_caps(klass) def has_caps(self, *caps): """ Check if this backend implements at least one of these capabilities. """ for c in caps: if (isinstance(c, basestring) and c in [cap.__name__ for cap in self.iter_caps()]) or \ isinstance(self, c): return True return False def fillobj(self, obj, fields=None): """ Fill an object with the wanted fields. :param fields: what fields to fill; if None, all fields are filled :type fields: :class:`list` """ if obj is None: return obj def not_loaded(v): return (v is NotLoaded or isinstance(v, BaseObject) and not v.__iscomplete__()) if isinstance(fields, basestring): fields = (fields,) missing_fields = [] if fields is None: # Select all fields if isinstance(obj, BaseObject): fields = [item[0] for item in obj.iter_fields()] else: fields = [item[0] for item in iter_fields(obj)] for field in fields: if not hasattr(obj, field): raise FieldNotFound(obj, field) value = getattr(obj, field) missing = False if hasattr(value, '__iter__'): for v in (value.itervalues() if isinstance(value, dict) else value): if not_loaded(v): missing = True break elif not_loaded(value): missing = True if missing: missing_fields.append(field) if not missing_fields: return obj for key, value in self.OBJECTS.iteritems(): if isinstance(obj, key): self.logger.debug(u'Fill %r with fields: %s' % (obj, missing_fields)) return value(self, obj, missing_fields) or obj # Object is not supported by backend. Do not notice it to avoid flooding user. # That's not so bad. for field in missing_fields: setattr(obj, field, NotAvailable) return obj class AbstractModuleMissingParentError(Exception): pass class AbstractModule(Module): """ Abstract module allow inheritance between modules. Sometimes, several websites are based on the same website engine. This module allow to simplify code with a fake inheritance: weboob will install (if needed) and load a PARENT module and build our AbstractModule on top of this class. PARENT is a mandatory attribute of any AbstractModule Note that you must pass a valid weboob instance as first argument of the constructor. """ PARENT = None def __new__(cls, weboob, name, config=None, storage=None, logger=None, nofail=False): if cls.PARENT is None: raise AbstractModuleMissingParentError("PARENT is not defined for module %s" % cls.__name__) try: parent = weboob.load_or_install_module(cls.PARENT).klass except ModuleInstallError as err: raise ModuleInstallError('The module %s depends on %s module but %s\'s installation failed with: %s' % (name, cls.PARENT, cls.PARENT, err)) cls.__bases__ = tuple([parent] + list(cls.iter_caps())) return cls.__new__(cls, weboob, name, config, storage, logger, nofail) weboob-1.2/weboob/tools/capabilities/000077500000000000000000000000001303450110500176675ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/__init__.py000066400000000000000000000000001303450110500217660ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/audio/000077500000000000000000000000001303450110500207705ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/audio/__init__.py000077500000000000000000000000001303450110500230720ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/audio/audio.py000077500000000000000000000025211303450110500224460ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.filters.standard import Format class AlbumIdFilter(Format): """ Filter that help to fill Albums id field """ def __init__(self, *args): super(AlbumIdFilter, self).__init__(u'album.%s', *args) class PlaylistIdFilter(Format): """ Filter that help to fill Albums id field """ def __init__(self, *args): super(PlaylistIdFilter, self).__init__(u'playlist.%s', *args) class BaseAudioIdFilter(Format): """ Filter that help to fill Albums id field """ def __init__(self, *args): super(BaseAudioIdFilter, self).__init__(u'audio.%s', *args) weboob-1.2/weboob/tools/capabilities/bank/000077500000000000000000000000001303450110500206025ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/bank/__init__.py000066400000000000000000000000001303450110500227010ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/bank/iban.py000066400000000000000000000062311303450110500220670ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2016 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re _country2length = dict( AL=28, AD=24, AT=20, AZ=28, BE=16, BH=22, BA=20, BR=29, BG=22, CR=21, HR=21, CY=28, CZ=24, DK=18, DO=28, EE=20, FO=18, FI=18, FR=27, GE=22, DE=22, GI=23, GR=27, GL=18, GT=28, HU=28, IS=26, IE=22, IL=23, IT=27, KZ=20, KW=30, LV=21, LB=28, LI=21, LT=20, LU=20, MK=19, MT=31, MR=27, MU=30, MC=27, MD=24, ME=22, NL=18, NO=15, PK=24, PS=29, PL=28, PT=25, RO=24, SM=27, SA=24, RS=22, SK=24, SI=19, ES=24, SE=24, CH=21, TN=24, TR=26, AE=23, GB=22, VG=24 ) def clean(iban): return iban.replace(' ','').replace('\t', '') def is_iban_valid(iban): # Ensure upper alphanumeric input. iban = clean(iban) if not re.match(r'^[\dA-Z]+$', iban): return False # Validate country code against expected length. if iban[:2] not in _country2length: return False if len(iban) != _country2length[iban[:2]]: return False digits = iban2numeric(iban) return digits % 97 == 1 def iban2numeric(iban): # Shift and convert. iban = iban[4:] + iban[:4] # BASE 36: 0..9,A..Z -> 0..35 digits = int(''.join(str(int(ch, 36)) for ch in iban)) return digits def find_iban_checksum(iban): iban = iban[:2] + '00' + iban[4:] digits = str(iban2numeric(iban)) checksum = 0 for char in digits: checksum *= 10 checksum += int(char) checksum %= 97 return 98-checksum def rebuild_iban(iban): return unicode(iban[:2] + ('%02d' % find_iban_checksum(iban)) + iban[4:]) def rib2iban(rib): return rebuild_iban('FR00' + rib) def find_rib_checksum(bank, counter, account): letters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' digits = '12345678912345678923456789' account = ''.join([(char if char.isdigit() else digits[letters.find(char.upper())]) for char in account]) rest = (89*int(bank) + 15*int(counter) + 3*int(account)) % 97 return 97 - rest def is_rib_valid(rib): if len(rib) != 23: return False return find_rib_checksum(rib[:5], rib[5:10], rib[10:21]) == int(rib[21:23]) def rebuild_rib(rib): rib = clean(rib) assert len(rib) >= 21 key = find_rib_checksum(rib[:5], rib[5:10], rib[10:21]) return unicode(rib[:21] + ('%02d' % key)) def test(): assert rebuild_iban('FR0013048379405300290000355') == "FR7613048379405300290000355" assert rebuild_iban('GB87BARC20658244971655') == "GB87BARC20658244971655" assert rebuild_rib('30003021990005077567600') == "30003021990005077567667" weboob-1.2/weboob/tools/capabilities/bank/transactions.py000066400000000000000000000315651303450110500236760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2012 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from decimal import Decimal, InvalidOperation import datetime import re from weboob.capabilities.bank import Transaction, Account from weboob.capabilities import NotAvailable, NotLoaded from weboob.tools.misc import to_unicode from weboob.tools.log import getLogger from weboob.exceptions import ParseError from weboob.browser.elements import TableElement, ItemElement from weboob.browser.filters.standard import Filter, CleanText, CleanDecimal, TableCell __all__ = ['FrenchTransaction', 'AmericanTransaction'] class classproperty(object): def __init__(self, f): self.f = f def __get__(self, obj, owner): return self.f(owner) class FrenchTransaction(Transaction): """ Transaction with some helpers for french bank websites. """ PATTERNS = [] def __init__(self, id='', *args, **kwargs): Transaction.__init__(self, id, *args, **kwargs) self._logger = getLogger('FrenchTransaction') @classmethod def clean_amount(klass, text): """ Clean a string containing an amount. """ text = text.replace('.','').replace(',','.') return re.sub(u'[^\d\-\.]', '', text) def set_amount(self, credit='', debit=''): """ Set an amount value from a string. Can take two strings if there are both credit and debit columns. """ credit = self.clean_amount(credit) debit = self.clean_amount(debit) if len(debit) > 0: self.amount = - abs(Decimal(debit)) elif len(credit) > 0: self.amount = Decimal(credit) else: self.amount = Decimal('0') def parse_date(self, date): if date is None: return NotAvailable if not isinstance(date, (datetime.date, datetime.datetime)): if date.isdigit() and len(date) == 8: date = datetime.date(int(date[4:8]), int(date[2:4]), int(date[0:2])) elif '/' in date: date = datetime.date(*reversed(map(int, date.split('/')))) if not isinstance(date, (datetime.date, datetime.datetime)): self._logger.warning('Unable to parse date %r' % date) date = NotAvailable elif date.year < 100: date = date.replace(year=2000 + date.year) return date def parse(self, date, raw, vdate=None): """ Parse date and raw strings to create datetime.date objects, determine the type of transaction, and create a simplified label When calling this method, you should have defined patterns (in the PATTERN class attribute) with a list containing tuples of regexp and the associated type, for example:: PATTERNS = [(re.compile(r'^VIR(EMENT)? (?P.*)'), FrenchTransaction.TYPE_TRANSFER), (re.compile(r'^PRLV (?P.*)'), FrenchTransaction.TYPE_ORDER), (re.compile(r'^(?P.*) CARTE \d+ PAIEMENT CB (?P
\d{2})(?P\d{2}) ?(.*)$'), FrenchTransaction.TYPE_CARD) ] In regexps, you can define this patterns: * text: part of label to store in simplified label * category: part of label representing the category * yy, mm, dd, HH, MM: date and time parts """ self.date = self.parse_date(date) self.vdate = self.parse_date(vdate) self.rdate = self.date self.raw = to_unicode(raw.replace(u'\n', u' ').strip()) self.category = NotAvailable if ' ' in self.raw: self.category, _, self.label = [part.strip() for part in self.raw.partition(' ')] else: self.label = self.raw for pattern, _type in self.PATTERNS: m = pattern.match(self.raw) if m: args = m.groupdict() def inargs(key): """ inner function to check if a key is in args, and is not None. """ return args.get(key, None) is not None self.type = _type if inargs('text'): self.label = args['text'].strip() if inargs('category'): self.category = args['category'].strip() # Set date from information in raw label. if inargs('dd') and inargs('mm'): dd = int(args['dd']) mm = int(args['mm']) if inargs('yy'): yy = int(args['yy']) else: d = self.date try: d = d.replace(month=mm, day=dd) except ValueError: d = d.replace(year=d.year-1, month=mm, day=dd) yy = d.year if d > self.date: yy -= 1 if yy < 100: yy += 2000 try: if inargs('HH') and inargs('MM'): self.rdate = datetime.datetime(yy, mm, dd, int(args['HH']), int(args['MM'])) else: self.rdate = datetime.date(yy, mm, dd) except ValueError as e: self._logger.warning('Unable to date in label %r: %s' % (self.raw, e)) return @classproperty def TransactionElement(k): class _TransactionElement(ItemElement): klass = k obj_date = klass.Date(TableCell('date')) obj_vdate = klass.Date(TableCell('vdate', 'date')) obj_raw = klass.Raw(TableCell('raw')) obj_amount = klass.Amount(TableCell('credit'), TableCell('debit', default='')) return _TransactionElement @classproperty def TransactionsElement(klass): class _TransactionsElement(TableElement): col_date = [u'Date'] col_vdate = [u'Valeur'] col_raw = [u'Opération', u'Libellé', u'Intitulé opération'] col_credit = [u'Crédit', u'Montant'] col_debit = [u'Débit'] item = klass.TransactionElement return _TransactionsElement class Date(CleanText): def __call__(self, item): date = super(FrenchTransaction.Date, self).__call__(item) return date def filter(self, date): date = super(FrenchTransaction.Date, self).filter(date) if date is None: return NotAvailable if not isinstance(date, (datetime.date, datetime.datetime)): if date.isdigit() and len(date) == 8: date = datetime.date(int(date[4:8]), int(date[2:4]), int(date[0:2])) elif '/' in date: date = datetime.date(*reversed(map(int, date.split('/')))) if not isinstance(date, (datetime.date, datetime.datetime)): date = NotAvailable elif date.year < 100: date = date.replace(year=2000 + date.year) return date @classmethod def Raw(klass, *args, **kwargs): patterns = klass.PATTERNS class Filter(CleanText): def __call__(self, item): raw = super(Filter, self).__call__(item) if item.obj.rdate is NotLoaded: item.obj.rdate = item.obj.date item.obj.category = NotAvailable if ' ' in raw: item.obj.category, useless, item.obj.label = [part.strip() for part in raw.partition(' ')] else: item.obj.label = raw for pattern, _type in patterns: m = pattern.match(raw) if m: args = m.groupdict() def inargs(key): """ inner function to check if a key is in args, and is not None. """ return args.get(key, None) is not None item.obj.type = _type if inargs('text'): item.obj.label = args['text'].strip() if inargs('category'): item.obj.category = args['category'].strip() # Set date from information in raw label. if inargs('dd') and inargs('mm'): dd = int(args['dd']) if args['dd'] != '00' else 1 mm = int(args['mm']) if inargs('yy'): yy = int(args['yy']) else: d = item.obj.date try: d = d.replace(month=mm, day=dd) except ValueError: d = d.replace(year=d.year-1, month=mm, day=dd) yy = d.year if d > item.obj.date: yy -= 1 if yy < 100: yy += 2000 try: if inargs('HH') and inargs('MM'): item.obj.rdate = datetime.datetime(yy, mm, dd, int(args['HH']), int(args['MM'])) else: item.obj.rdate = datetime.date(yy, mm, dd) except ValueError as e: raise ParseError('Unable to parse date in label %r: %s' % (raw, e)) break return raw def filter(self, text): text = super(Filter, self).filter(text) return to_unicode(text.replace(u'\n', u' ').strip()) return Filter(*args, **kwargs) class Currency(CleanText): def filter(self, text): text = super(FrenchTransaction.Currency, self).filter(text) return Account.get_currency(text) class Amount(Filter): def __init__(self, credit, debit=None, replace_dots=True): self.credit_selector = credit self.debit_selector = debit self.replace_dots = replace_dots def __call__(self, item): if self.debit_selector: try: return - abs(CleanDecimal(self.debit_selector, replace_dots=self.replace_dots)(item)) except InvalidOperation: pass if self.credit_selector: try: return CleanDecimal(self.credit_selector, replace_dots=self.replace_dots)(item) except InvalidOperation: pass return Decimal('0') class AmericanTransaction(Transaction): """ Transaction with some helpers for american bank websites. """ @classmethod def clean_amount(klass, text): """ Clean a string containing an amount. """ # Convert "American" UUU.CC format to "French" UUU,CC format if re.search(r'\d\.\d\d(?: [A-Z]+)?$', text): text = text.replace(',', ' ').replace('.', ',') return FrenchTransaction.clean_amount(text) @classmethod def decimal_amount(klass, text): """ Convert a string containing an amount to Decimal. """ amnt = AmericanTransaction.clean_amount(text) return Decimal(amnt) if amnt else Decimal('0') def test(): clean_amount = AmericanTransaction.clean_amount assert clean_amount('42') == '42' assert clean_amount('42,12') == '42.12' assert clean_amount('42.12') == '42.12' assert clean_amount('$42.12 USD') == '42.12' assert clean_amount('$12.442,12 USD') == '12442.12' assert clean_amount('$12,442.12 USD') == '12442.12' decimal_amount = AmericanTransaction.decimal_amount assert decimal_amount('$12,442.12 USD') == Decimal('12442.12') assert decimal_amount('') == Decimal('0') weboob-1.2/weboob/tools/capabilities/gallery/000077500000000000000000000000001303450110500213265ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/gallery/__init__.py000066400000000000000000000000001303450110500234250ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/gallery/genericcomicreader.py000066400000000000000000000067331303450110500255230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re from weboob.capabilities.gallery import CapGallery, BaseGallery, BaseImage from weboob.tools.backend import Module from weboob.deprecated.browser import Browser, Page __all__ = ['GenericComicReaderModule'] class DisplayPage(Page): def get_page(self, gallery): src = self.document.xpath(self.browser.params['img_src_xpath'])[0] return BaseImage(src, gallery=gallery, url=src) def page_list(self): return self.document.xpath(self.browser.params['page_list_xpath']) class GenericComicReaderBrowser(Browser): def __init__(self, browser_params, *args, **kwargs): self.params = browser_params Browser.__init__(self, *args, **kwargs) def iter_gallery_images(self, gallery): self.location(gallery.url) assert self.is_on_page(DisplayPage) for p in self.page.page_list(): if 'page_to_location' in self.params: self.location(self.params['page_to_location'] % p) else: self.location(p) assert self.is_on_page(DisplayPage) yield self.page.get_page(gallery) def fill_image(self, image, fields): if 'data' in fields: image.data = self.readurl(image.url) class GenericComicReaderModule(Module, CapGallery): NAME = 'genericcomicreader' MAINTAINER = u'Noé Rubinstein' EMAIL = 'noe.rubinstein@gmail.com' VERSION = '1.2' DESCRIPTION = 'Generic comic reader backend; subclasses implement specific sites' LICENSE = 'AGPLv3+' BROWSER = GenericComicReaderBrowser BROWSER_PARAMS = {} ID_REGEXP = None URL_REGEXP = None ID_TO_URL = None PAGES = {} def create_default_browser(self): b = self.create_browser(self.BROWSER_PARAMS) b.PAGES = self.PAGES try: b.DOMAIN = self.DOMAIN except AttributeError: pass return b def iter_gallery_images(self, gallery): with self.browser: return self.browser.iter_gallery_images(gallery) def get_gallery(self, _id): match = re.match(r'^%s$' % self.URL_REGEXP, _id) if match: _id = match.group(1) else: match = re.match(r'^%s$' % self.ID_REGEXP, _id) if match: _id = match.group(0) else: return None gallery = BaseGallery(_id, url=(self.ID_TO_URL % _id)) with self.browser: return gallery def fill_gallery(self, gallery, fields): gallery.title = gallery.id def fill_image(self, image, fields): with self.browser: self.browser.fill_image(image, fields) OBJECTS = { BaseGallery: fill_gallery, BaseImage: fill_image} weboob-1.2/weboob/tools/capabilities/gallery/genericcomicreadertest.py000066400000000000000000000020231303450110500264070ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Noé Rubinstein # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.tools.test import BackendTest class GenericComicReaderTest(BackendTest): def _test_download(self, _id): g = self.backend.get_gallery(_id) it = self.backend.iter_gallery_images(g) it.next() img = it.next() self.backend.fillobj(img, ('url', 'data')) weboob-1.2/weboob/tools/capabilities/housing/000077500000000000000000000000001303450110500213435ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/housing/__init__.py000077500000000000000000000000001303450110500234450ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/housing/housing.py000077500000000000000000000024141303450110500233750ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2009-2015 Bezleputh # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.browser.filters.standard import _Filter, Field, debug from weboob.capabilities.base import empty from decimal import Decimal class PricePerMeterFilter(_Filter): """ Filter that help to fill PricePerMeter field """ def __init__(self): super(PricePerMeterFilter, self).__init__() @debug() def __call__(self, item): cost = Field('cost')(item) area = Field('area')(item) if not (empty(cost) or empty(area)): return Decimal(cost or 0) / Decimal(area or 1) return Decimal(0) weboob-1.2/weboob/tools/capabilities/messages/000077500000000000000000000000001303450110500214765ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/messages/GenericModule.py000066400000000000000000000077311303450110500246020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import time from weboob.capabilities.messages import CapMessages, Message, Thread from weboob.capabilities.base import find_object from weboob.tools.backend import Module from weboob.tools.newsfeed import Newsfeed class GenericNewspaperModule(Module, CapMessages): """ GenericNewspaperModule class """ MAINTAINER = u'Julien Hebert' EMAIL = 'juke@free.fr' VERSION = '1.2' LICENSE = 'AGPLv3+' STORAGE = {'seen': {}} RSS_FEED = None RSSID = None URL2ID = None RSSSIZE = 0 def get_thread(self, _id): if isinstance(_id, Thread): thread = _id id = thread.id else: thread = find_object(self.iter_threads(), id=_id) id = _id with self.browser: content = self.browser.get_content(id) if content is None: return None if not thread: thread = Thread(id) flags = Message.IS_HTML if thread.id not in self.storage.get('seen', default={}): flags |= Message.IS_UNREAD thread.title = content.title if not thread.date: thread.date = content.date thread.root = Message( thread=thread, id=0, title=content.title, sender=content.author, receivers=None, date=thread.date, parent=None, content=content.body, signature= u'URL \n' % content.url, flags=flags, children=[]) return thread def iter_threads(self): for article in Newsfeed(self.RSS_FEED, GenericNewspaperModule.RSSID).iter_entries(): thread = Thread(article.id) thread.title = article.title thread.date = article.datetime yield(thread) def fill_thread(self, thread, fields): "fill the thread" t = self.get_thread(thread) return t or thread def iter_unread_messages(self): for thread in self.iter_threads(): if thread.id in self.storage.get('seen', default={}): continue self.fill_thread(thread, 'root') for msg in thread.iter_all_messages(): yield msg def set_message_read(self, message): self.storage.set( 'seen', message.thread.id, 'comments', self.storage.get( 'seen', message.thread.id, 'comments', default=[]) + [message.id]) if self.URL2ID and self.RSSSIZE != 0: url2id = self.URL2ID lastpurge = self.storage.get('lastpurge', default=0) l = [] if time.time() - lastpurge > 7200: self.storage.set('lastpurge', time.time()) for id in self.storage.get('seen', default={}): l.append((int(url2id(id)), id)) l.sort() l.reverse() tosave = [v[1] for v in l[0:self.RSSSIZE + 10]] toremove = set([v for v in self.storage.get('seen', default={})]).difference(tosave) for id in toremove: self.storage.delete('seen', id) self.storage.save() OBJECTS = {Thread: fill_thread} weboob-1.2/weboob/tools/capabilities/messages/__init__.py000066400000000000000000000000001303450110500235750ustar00rootroot00000000000000weboob-1.2/weboob/tools/capabilities/messages/genericArticle.py000066400000000000000000000116061303450110500247740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Julien Hebert # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.deprecated.browser import Page from weboob.deprecated.browser import BrokenPageError from lxml.etree import Comment def try_remove(parser, base_element, selector): try: base_element.remove(parser.select(base_element, selector, 1)) except (BrokenPageError, ValueError): pass def try_drop_tree(parser, base_element, selector): for el in parser.select(base_element, selector): el.drop_tree() def remove_from_selector_list(parser, base_element, selector_list): for selector in selector_list: base_element.remove(parser.select(base_element, selector, 1)) def try_remove_from_selector_list(parser, base_element, selector_list): for selector in selector_list: try_remove(parser, base_element, selector) def drop_comments(base_element): for comment in base_element.getiterator(Comment): comment.drop_tree() # Replace relative url in link and image with a complete url # Arguments: the html element to clean, and the domain name (with http:// prefix) def clean_relativ_urls(base_element, domain): for a in base_element.findall('.//a'): if "href" in a.attrib: if a.attrib["href"] and a.attrib["href"][0:7] != "http://" and a.attrib["href"][0:7] != "https://": a.attrib["href"] = domain + a.attrib["href"] for img in base_element.findall('.//img'): if img.attrib["src"][0:7] != "http://" and img.attrib["src"][0:7] != "https://": img.attrib["src"] = domain + img.attrib["src"] class NoAuthorElement(BrokenPageError): pass class NoBodyElement(BrokenPageError): pass class NoTitleException(BrokenPageError): pass class NoneMainDiv(AttributeError): pass class Article(object): author = u'' title = u'' def __init__(self, browser, _id): self.browser = browser self.id = _id self.body = u'' self.url = u'' self.date = None class GenericNewsPage(Page): __element_body = NotImplementedError __article = Article element_title_selector = NotImplementedError main_div = NotImplementedError element_body_selector = NotImplementedError element_author_selector = NotImplementedError def get_body(self): return self.parser.tostring(self.get_element_body()) def get_author(self): try: return u'%s' % self.get_element_author().text_content().strip() except (NoAuthorElement, NoneMainDiv): #TODO: Mettre un warning return self.__article.author def get_title(self): try: return u'%s' % self.parser.select( self.main_div, self.element_title_selector, 1).text_content().strip() except AttributeError: if self.main_div is None: #TODO: Mettre un warning return self.__article.title else: raise except BrokenPageError: if self.element_title_selector == 'h1': raise NoTitleException("no title on %s" % (self.browser)) self.element_title_selector = "h1" return self.get_title() def get_element_body(self): try: return self.parser.select(self.main_div, self.element_body_selector, 1) except BrokenPageError: raise NoBodyElement("no body on %s" % (self.browser)) except AttributeError: if self.main_div is None: raise NoneMainDiv("main_div is none on %s" % (self.browser)) else: raise def get_element_author(self): try: return self.parser.select(self.main_div, self.element_author_selector, 1) except BrokenPageError: raise NoAuthorElement() except AttributeError: if self.main_div is None: raise NoneMainDiv("main_div is none on %s" % (self.browser)) else: raise def get_article(self, _id): __article = Article(self.browser, _id) __article.author = self.get_author() __article.title = self.get_title() __article.url = self.url __article.body = self.get_body() return __article weboob-1.2/weboob/tools/capabilities/paste.py000066400000000000000000000073421303450110500213630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.paste import CapPaste import binascii class BasePasteModule(CapPaste): EXPIRATIONS = {} """ List of expirations and their corresponding remote codes (any type can be used). The expirations, i.e. the keys, are integers representing the duration in seconds. There also can be one False key, for the "forever" expiration. """ def get_closest_expiration(self, max_age): """ Get the expiration closest (and less or equal to) max_age (int, in seconds). max_age set to False means we want it to never expire. @return int or False if found, else None """ # "forever" if max_age is False and False in self.EXPIRATIONS: return max_age # get timed expirations, longest first expirations = sorted([e for e in self.EXPIRATIONS if e is not False], reverse=True) # find the first expiration that is below or equal to the maximum wanted age for e in expirations: if max_age is False or max_age >= e: return e def image_mime(data_base64, supported_formats=('gif', 'jpeg', 'png')): """ Return the MIME type of an image or None. :param data_base64: data to detect, base64 encoded :type data_base64: str :param supported_formats: restrict list of formats to test """ try: beginning = data_base64[:24].decode('base64') except binascii.Error: return None if 'gif' in supported_formats and 'GIF8' in beginning: return 'image/gif' elif 'jpeg' in supported_formats and 'JFIF' in beginning: return 'image/jpeg' elif 'png' in supported_formats and '\x89PNG' in beginning: return 'image/png' elif 'xcf' in supported_formats and 'gimp xcf' in beginning: return 'image/x-xcf' elif 'pdf' in supported_formats and '%PDF' in beginning: return 'application/pdf' elif 'tiff' in supported_formats and ('II\x00\x2a' in beginning or 'MM\x2a\x00' in beginning): return 'image/tiff' def test(): class MockPasteModule(BasePasteModule): def __init__(self, expirations): self.EXPIRATIONS = expirations # all expirations are too high assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(1) is None # we found a suitable lower or equal expiration assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(84) is 42 assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(False) is False assert MockPasteModule({1337: '', 42: ''}).get_closest_expiration(False) is 1337 assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(1336) is 42 assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(1337) is 1337 assert MockPasteModule({1337: '', 42: '', False: ''}).get_closest_expiration(1338) is 1337 # this format should work, though of doubtful usage assert MockPasteModule([1337, 42, False]).get_closest_expiration(84) is 42 weboob-1.2/weboob/tools/capabilities/streaminfo.py000066400000000000000000000023561303450110500224160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2013 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from weboob.capabilities.base import BaseObject, StringField __all__ = ['StreamInfo'] class StreamInfo(BaseObject): """ Stream related information. """ who = StringField('Who is currently on air') what = StringField('What is currently on air') def __iscomplete__(self): # This volatile information may be reloaded everytimes. return False def __unicode__(self): if self.who: return u'%s - %s' % (self.who, self.what) else: return self.what weboob-1.2/weboob/tools/captcha/000077500000000000000000000000001303450110500166415ustar00rootroot00000000000000weboob-1.2/weboob/tools/captcha/__init__.py000066400000000000000000000000001303450110500207400ustar00rootroot00000000000000weboob-1.2/weboob/tools/captcha/virtkeyboard.py000066400000000000000000000213451303450110500217250ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2011 Pierre Mazière # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import hashlib import tempfile try: from PIL import Image except ImportError: raise ImportError('Please install python-imaging') class VirtKeyboardError(Exception): pass class VirtKeyboard(object): """ Handle a virtual keyboard. :attribute margin: Margin used by :meth:`get_symbol_coords` to reduce size of each "key" of the virtual keyboard. This attribute is always converted to a 4-tuple, and has the same semantic as the CSS ``margin`` property (top, right, bottom, right), in pixels. :type margin: int or float or (2|3|4)-tuple """ margin = None codesep = '' """Output separator between code strings. See :func:`get_string_code`. """ def __init__(self, file=None, coords=None, color=None, convert=None): # file: virtual keyboard image # coords: dictionary : # color: color of the symbols in the image # depending on the image, it can be a single value or a tuple # convert: if not None, convert image to this target type (for example 'RGB') if file is not None: assert color, 'No color provided !' self.load_image(file, color, convert) if type(self.margin) in (int, float): self.margin = (self.margin,) * 4 elif self.margin is not None: if len(self.margin) == 2: self.margin = self.margin + self.margin elif len(self.margin) == 3: self.margin = self.margin + (self.margin[1],) assert len(self.margin) == 4 if coords is not None: self.load_symbols(coords) def load_image(self, file, color, convert=None): self.image = Image.open(file) if convert is not None: self.image = self.image.convert(convert) self.bands = self.image.getbands() if isinstance(color, int) and not isinstance(self.bands, str) and len(self.bands) != 1: raise VirtKeyboardError("Color requires %i component but only 1 is provided" % len(self.bands)) if not isinstance(color, int) and len(color) != len(self.bands): raise VirtKeyboardError("Color requires %i components but %i are provided" % (len(self.bands), len(color))) self.color = color self.width, self.height = self.image.size self.pixar = self.image.load() def load_symbols(self, coords): self.coords = {} self.md5 = {} for i in coords: coord = self.get_symbol_coords(coords[i]) if coord == (-1, -1, -1, -1): continue self.coords[i] = coord self.md5[i] = self.checksum(self.coords[i]) def check_color(self, pixel): return pixel == self.color def get_symbol_coords(self, coords): """Return narrow coordinates around symbol.""" (x1, y1, x2, y2) = coords if self.margin: top, right, bottom, left = self.margin x1, y1, x2, y2 = x1 + left, y1 + top, x2 - right, y2 - bottom newY1 = -1 newY2 = -1 for y in range(y1, min(y2 + 1, self.height)): empty_line = True for x in range(x1, min(x2 + 1, self.width)): if self.check_color(self.pixar[x, y]): empty_line = False if newY1 < 0: newY1 = y break if newY1 >= 0 and not empty_line: newY2 = y newX1 = -1 newX2 = -1 for x in range(x1, min(x2 + 1, self.width)): empty_column = True for y in range(y1, min(y2 + 1, self.height)): if self.check_color(self.pixar[x, y]): empty_column = False if newX1 < 0: newX1 = x break if newX1 >= 0 and not empty_column: newX2 = x return (newX1, newY1, newX2, newY2) def checksum(self, coords): (x1, y1, x2, y2) = coords s = '' for y in range(y1, min(y2 + 1, self.height)): for x in range(x1, min(x2 + 1, self.width)): if self.check_color(self.pixar[x, y]): s += "." else: s += " " return hashlib.md5(s).hexdigest() def get_symbol_code(self, md5sum_list): if isinstance(md5sum_list, basestring): md5sum_list = [md5sum_list] for md5sum in md5sum_list: for i in self.md5: if md5sum == self.md5[i]: return i raise VirtKeyboardError('Symbol not found for hash "%s".' % md5sum) def get_string_code(self, string): return self.codesep.join(self.get_symbol_code(self.symbols[c]) for c in string) def check_symbols(self, symbols, dirname): # symbols: dictionary : for s in symbols: try: self.get_symbol_code(symbols[s]) except VirtKeyboardError: if dirname is None: dirname = tempfile.mkdtemp(prefix='weboob_session_') self.generate_MD5(dirname) raise VirtKeyboardError("Symbol '%s' not found; all symbol hashes are available in %s" % (s, dirname)) def generate_MD5(self, dir): for i in self.coords: width = self.coords[i][2] - self.coords[i][0] + 1 height = self.coords[i][3] - self.coords[i][1] + 1 img = Image.new(''.join(self.bands), (width, height)) matrix = img.load() for y in range(height): for x in range(width): matrix[x, y] = self.pixar[self.coords[i][0] + x, self.coords[i][1] + y] img.save(dir + "/" + self.md5[i] + ".png") self.image.save(dir + "/image.png") class MappedVirtKeyboard(VirtKeyboard): def __init__(self, file, document, img_element, color, map_attr="onclick", convert=None): map_id = img_element.attrib.get("usemap")[1:] map = document.find('//map[@id="%s"]' % map_id) if map is None: map = document.find('//map[@name="%s"]' % map_id) coords = {} for area in map.getiterator("area"): code = area.attrib.get(map_attr) area_coords = [] for coord in area.attrib.get("coords").split(' ')[0].split(','): area_coords.append(int(coord)) coords[code] = tuple(area_coords) super(MappedVirtKeyboard, self).__init__(file, coords, color, convert) class GridVirtKeyboard(VirtKeyboard): """ Make a virtual keyboard where "keys" are distributed on a grid. For example: https://www.esgbl.com/part/fr/idehom.html Parameters: :param symbols: Sequence of symbols, ordered in the grid from left to right and up to down :type symbols: iterable :param cols: Column count of the grid :type cols: int :param rows: Row count of the grid :type rows: int :param image: File-like object to be used as data source :type image: file :param color: Color of the meaningful pixels :type color: 3-tuple :param convert: Mode to which convert color of pixels, see :meth:`Image.Image.convert` for more information Attributes: :attribute symbols: Association table between symbols and md5s :type symbols: dict """ symbols = {} def __init__(self, symbols, cols, rows, image, color, convert=None): self.load_image(image, color, convert) tileW = float(self.width) / cols tileH = float(self.height) / rows positions = ((s, i * tileW % self.width, i / cols * tileH) for i, s in enumerate(symbols)) coords = dict((s, tuple(map(int, (x, y, x + tileW, y + tileH)))) for (s, x, y) in positions) super(GridVirtKeyboard, self).__init__() self.load_symbols(coords) weboob-1.2/weboob/tools/compat.py000066400000000000000000000032671303450110500171030ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['unicode', 'long', 'basestring', 'check_output'] try: unicode = unicode except NameError: unicode = str try: long = long except NameError: long = int try: basestring = basestring except NameError: basestring = str try: from subprocess import check_output except ImportError: import subprocess def check_output(*popenargs, **kwargs): r"""Run command with arguments and return its output as a byte string. Backported from Python 2.7 as it's implemented as pure python on stdlib. """ process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs) output, unused_err = process.communicate() retcode = process.poll() if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] error = subprocess.CalledProcessError(retcode, cmd) error.output = output raise error return output weboob-1.2/weboob/tools/config/000077500000000000000000000000001303450110500165035ustar00rootroot00000000000000weboob-1.2/weboob/tools/config/__init__.py000066400000000000000000000000001303450110500206020ustar00rootroot00000000000000weboob-1.2/weboob/tools/config/iconfig.py000066400000000000000000000036231303450110500204770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . class ConfigError(Exception): pass class IConfig(object): """ Interface for config storage. Config stores keys and values. Each key is a path of components, allowing to group multiple options. """ def load(self, default={}): """ Load config. :param default: default values for the config :type default: dict[:class:`str`] """ raise NotImplementedError() def save(self): """Save config.""" raise NotImplementedError() def set(self, *args): """ Set a config value. :param args: all args except the last arg are the path of the option key. :type args: str or object """ raise NotImplementedError() def delete(self, *args): """ Delete an option from config. :param args: path to the option key. :type args: str """ raise NotImplementedError() def get(self, *args, **kwargs): """ Get the value of an option. :param args: path of the option key. :param default: if specified, default value when path is not found """ raise NotImplementedError() weboob-1.2/weboob/tools/config/iniconfig.py000066400000000000000000000077301303450110500210310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from ConfigParser import RawConfigParser, DEFAULTSECT from decimal import Decimal import logging import os from weboob.tools.ordereddict import OrderedDict from .iconfig import IConfig __all__ = ['INIConfig'] class INIConfig(IConfig): ROOTSECT = 'ROOT' def __init__(self, path): self.path = path self.values = OrderedDict() self.config = RawConfigParser() def load(self, default={}): self.values = OrderedDict(default) if os.path.exists(self.path): logging.debug(u'Loading application configuration file: %s.' % self.path) self.config.read(self.path) for section in self.config.sections(): args = section.split(':') if args[0] == self.ROOTSECT: args.pop(0) for key, value in self.config.items(section): self.set(*(args + [key, value])) # retro compatibility if len(self.config.sections()) == 0: first = True for key, value in self.config.items(DEFAULTSECT): if first: logging.warning('The configuration file "%s" uses an old-style' % self.path) logging.warning('Please rename the %s section to %s' % (DEFAULTSECT, self.ROOTSECT)) first = False self.set(key, value) logging.debug(u'Application configuration file loaded: %s.' % self.path) else: self.save() logging.debug(u'Application configuration file created with default values: %s. ' 'Please customize it.' % self.path) return self.values def save(self): def save_section(values, root_section=self.ROOTSECT): for k, v in values.iteritems(): if isinstance(v, (int, Decimal, float, basestring)): if not self.config.has_section(root_section): self.config.add_section(root_section) self.config.set(root_section, k, unicode(v)) elif isinstance(v, dict): new_section = ':'.join((root_section, k)) if (root_section != self.ROOTSECT or k == self.ROOTSECT) else k if not self.config.has_section(new_section): self.config.add_section(new_section) save_section(v, new_section) save_section(self.values) with open(self.path, 'w') as f: self.config.write(f) def get(self, *args, **kwargs): default = None if 'default' in kwargs: default = kwargs['default'] v = self.values for k in args[:-1]: if k in v: v = v[k] else: return default try: return v[args[-1]] except KeyError: return default def set(self, *args): v = self.values for k in args[:-2]: if k not in v: v[k] = OrderedDict() v = v[k] v[args[-2]] = args[-1] def delete(self, *args): v = self.values for k in args[:-1]: if k not in v: return v = v[k] v.pop(args[-1], None) weboob-1.2/weboob/tools/config/yamlconfig.py000066400000000000000000000066161303450110500212160ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import logging import os import tempfile import weboob.tools.date import yaml from .iconfig import ConfigError, IConfig try: from yaml import CLoader as Loader from yaml import CDumper as Dumper except ImportError: from yaml import Loader from yaml import Dumper __all__ = ['YamlConfig'] class WeboobDumper(Dumper): pass WeboobDumper.add_representer(weboob.tools.date.date, WeboobDumper.represent_date) WeboobDumper.add_representer(weboob.tools.date.datetime, WeboobDumper.represent_datetime) class YamlConfig(IConfig): def __init__(self, path): self.path = path self.values = {} def load(self, default={}): self.values = default.copy() logging.debug(u'Loading application configuration file: %s.' % self.path) try: with open(self.path, 'r') as f: self.values = yaml.load(f, Loader=Loader) logging.debug(u'Application configuration file loaded: %s.' % self.path) except IOError: self.save() logging.debug(u'Application configuration file created with default values: %s. Please customize it.' % self.path) if self.values is None: self.values = {} def save(self): # write in a temporary file to avoid corruption problems with tempfile.NamedTemporaryFile(dir=os.path.dirname(self.path), delete=False) as f: yaml.dump(self.values, f, Dumper=WeboobDumper, default_flow_style=False) if os.path.isfile(self.path): os.remove(self.path) os.rename(f.name, self.path) def get(self, *args, **kwargs): v = self.values for a in args[:-1]: try: v = v[a] except KeyError: if 'default' in kwargs: v[a] = {} v = v[a] else: raise ConfigError() except TypeError: raise ConfigError() try: v = v[args[-1]] except KeyError: v = kwargs.get('default') return v def set(self, *args): v = self.values for a in args[:-2]: try: v = v[a] except KeyError: v[a] = {} v = v[a] except TypeError: raise ConfigError() v[args[-2]] = args[-1] def delete(self, *args): v = self.values for a in args[:-1]: try: v = v[a] except KeyError: return except TypeError: raise ConfigError() v.pop(args[-1], None) weboob-1.2/weboob/tools/date.py000066400000000000000000000353121303450110500165310ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2013 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import dateutil.parser from datetime import date as real_date, datetime as real_datetime, timedelta import time import re try: from dateutil import tz except ImportError: raise ImportError('Please install python-dateutil') __all__ = ['local2utc', 'utc2local', 'LinearDateGuesser', 'date', 'datetime', 'new_date', 'new_datetime', 'closest_date'] def local2utc(dateobj): dateobj = dateobj.replace(tzinfo=tz.tzlocal()) dateobj = dateobj.astimezone(tz.tzutc()) return dateobj def utc2local(dateobj): dateobj = dateobj.replace(tzinfo=tz.tzutc()) dateobj = dateobj.astimezone(tz.tzlocal()) return dateobj class date(real_date): def strftime(self, fmt): return strftime(self, fmt) @classmethod def from_date(cls, d): return cls(d.year, d.month, d.day) class datetime(real_datetime): def strftime(self, fmt): return strftime(self, fmt) def combine(self, date, time): return datetime(date.year, date.month, date.day, time.hour, time.minute, time.microsecond, time.tzinfo) def date(self): return date(self.year, self.month, self.day) @classmethod def from_datetime(cls, dt): return cls(dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second, dt.microsecond, dt.tzinfo) def new_date(d): """ Generate a safe date from a datetime.date object """ return date(d.year, d.month, d.day) def new_datetime(d): """ Generate a safe datetime from a datetime.date or datetime.datetime object """ kw = [d.year, d.month, d.day] if isinstance(d, real_datetime): kw.extend([d.hour, d.minute, d.second, d.microsecond, d.tzinfo]) return datetime(*kw) # No support for strftime's "%s" or "%y". # Allowed if there's an even number of "%"s because they are escaped. _illegal_formatting = re.compile(r"((^|[^%])(%%)*%[sy])") def _findall(text, substr): # Also finds overlaps sites = [] i = 0 while True: j = text.find(substr, i) if j == -1: break sites.append(j) i = j+1 return sites def strftime(dt, fmt): if dt.year >= 1900: return super(type(dt), dt).strftime(fmt) illegal_formatting = _illegal_formatting.search(fmt) if illegal_formatting: raise TypeError("strftime of dates before 1900 does not handle" + illegal_formatting.group(0)) year = dt.year # For every non-leap year century, advance by # 6 years to get into the 28-year repeat cycle delta = 2000 - year off = 6*(delta // 100 + delta // 400) year = year + off # Move to around the year 2000 year = year + ((2000 - year)//28)*28 timetuple = dt.timetuple() s1 = time.strftime(fmt, (year,) + timetuple[1:]) sites1 = _findall(s1, str(year)) s2 = time.strftime(fmt, (year+28,) + timetuple[1:]) sites2 = _findall(s2, str(year+28)) sites = [] for site in sites1: if site in sites2: sites.append(site) s = s1 syear = "%4d" % (dt.year,) for site in sites: s = s[:site] + syear + s[site+4:] return s class LinearDateGuesser(object): """ The aim of this class is to guess the exact date object from a day and a month, but not a year. It works with a start date (default is today), and all dates must be sorted from recent to older. """ def __init__(self, current_date=None, date_max_bump=timedelta(31)): self.date_max_bump = date_max_bump if current_date is None: current_date = date.today() self.current_date = current_date def try_assigning_year(self, day, month, start_year, max_year): """ Tries to create a date object with day, month and start_year and returns it. If it fails due to the year not matching the day+month combination (i.e. due to a ValueError -- TypeError and OverflowError are not handled), the previous or next years are tried until max_year is reached. In case initialization still fails with max_year, this function raises a ValueError. """ while True: try: return date(start_year, month, day) except ValueError as e: if start_year == max_year: raise e start_year += cmp(max_year, start_year) def set_current_date(self, current_date): self.current_date = current_date def guess_date(self, day, month, change_current_date=True): """ Returns a date object built from a given day/month pair. """ today = self.current_date # The website only provides dates using the 'DD/MM' string, so we have to # determine the most possible year by ourselves. This implies tracking # the current date. # However, we may also encounter "bumps" in the dates, e.g. "12/11, # 10/11, 10/11, 12/11, 09/11", so we have to be, well, quite tolerant, # by accepting dates in the near future (say, 7 days) of the current # date. (Please, kill me...) # We first try to keep the current year naively_parsed_date = self.try_assigning_year(day, month, today.year, today.year - 5) if (naively_parsed_date.year != today.year): # we most likely hit a 29/02 leading to a change of year if change_current_date: self.set_current_date(naively_parsed_date) return naively_parsed_date if (naively_parsed_date > today + self.date_max_bump): # if the date ends up too far in the future, consider it actually # belongs to the previous year parsed_date = date(today.year - 1, month, day) if change_current_date: self.set_current_date(parsed_date) elif (naively_parsed_date > today and naively_parsed_date <= today + self.date_max_bump): # if the date is in the near future, consider it is a bump parsed_date = naively_parsed_date # do not keep it as current date though else: # if the date is in the past, as expected, simply keep it parsed_date = naively_parsed_date # and make it the new current date if change_current_date: self.set_current_date(parsed_date) return parsed_date class ChaoticDateGuesser(LinearDateGuesser): """ This class aim to find the guess the date when you know the day and month and the minimum year """ def __init__(self, min_date, current_date=None, date_max_bump=timedelta(31)): if min_date is None: raise ValueError("min_date is not set") self.min_date = min_date super(ChaoticDateGuesser, self).__init__(current_date, date_max_bump) def guess_date(self, day, month): """Returns a possible date between min_date and current_date""" parsed_date = super(ChaoticDateGuesser, self).guess_date(day, month, False) if parsed_date >= self.min_date: return parsed_date else: raise ValueError("%s is inferior to min_date %s" % (parsed_date, self.min_date)) DATE_TRANSLATE_FR = [(re.compile(ur'janvier', re.I), u'january'), (re.compile(ur'février', re.I), u'february'), (re.compile(ur'mars', re.I), u'march'), (re.compile(ur'avril', re.I), u'april'), (re.compile(ur'mai', re.I), u'may'), (re.compile(ur'juin', re.I), u'june'), (re.compile(ur'juillet', re.I), u'july'), (re.compile(ur'août?', re.I), u'august'), (re.compile(ur'septembre', re.I), u'september'), (re.compile(ur'octobre', re.I), u'october'), (re.compile(ur'novembre', re.I), u'november'), (re.compile(ur'décembre', re.I), u'december'), (re.compile(ur'jan\.', re.I), u'january'), (re.compile(ur'janv\.', re.I), u'january'), (re.compile(ur'\bjan\b', re.I), u'january'), (re.compile(ur'fév\.', re.I), u'february'), (re.compile(ur'févr\.', re.I), u'february'), (re.compile(ur'\bfév\b', re.I), u'february'), (re.compile(ur'avr\.', re.I), u'april'), (re.compile(ur'\bavr\b', re.I), u'april'), (re.compile(ur'juil\.', re.I), u'july'), (re.compile(ur'juill\.', re.I), u'july'), (re.compile(ur'\bjuil\b', re.I), u'july'), (re.compile(ur'sep\.', re.I), u'september'), (re.compile(ur'sept\.', re.I), u'september'), (re.compile(ur'\bsep\b', re.I), u'september'), (re.compile(ur'oct\.', re.I), u'october'), (re.compile(ur'\boct\b', re.I), u'october'), (re.compile(ur'nov\.', re.I), u'november'), (re.compile(ur'\bnov\b', re.I), u'november'), (re.compile(ur'déc\.', re.I), u'december'), (re.compile(ur'\bdéc\b', re.I), u'december'), (re.compile(ur'lundi', re.I), u'monday'), (re.compile(ur'mardi', re.I), u'tuesday'), (re.compile(ur'mercredi', re.I), u'wednesday'), (re.compile(ur'jeudi', re.I), u'thursday'), (re.compile(ur'vendredi', re.I), u'friday'), (re.compile(ur'samedi', re.I), u'saturday'), (re.compile(ur'dimanche', re.I), u'sunday')] DATE_TRANSLATE_IT = [(re.compile(ur'gennaio', re.I), u'january'), (re.compile(u'febbraio', re.I), u'february'), (re.compile(ur'marzo', re.I), u'march'), (re.compile(ur'aprile', re.I), u'april'), (re.compile(ur'maggio', re.I), u'may'), (re.compile(ur'giugno', re.I), u'june'), (re.compile(ur'luglio', re.I), u'july'), (re.compile(ur'agosto', re.I), u'august'), (re.compile(ur'settembre', re.I), u'september'), (re.compile(ur'ottobre', re.I), u'october'), (re.compile(ur'novembre', re.I), u'november'), (re.compile(ur'dicembre', re.I), u'december'), (re.compile(ur'luned[iì]', re.I), u'monday'), (re.compile(ur'marted[iì]', re.I), u'tuesday'), (re.compile(ur'mercoled[iì]', re.I), u'wednesday'), (re.compile(ur'gioved[iì]', re.I), u'thursday'), (re.compile(ur'venerd[iì]', re.I), u'friday'), (re.compile(ur'sabato', re.I), u'saturday'), (re.compile(ur'domenica', re.I), u'sunday')] def parse_french_date(date, **kwargs): for fr, en in DATE_TRANSLATE_FR: date = fr.sub(en, date) if 'dayfirst' not in kwargs: kwargs['dayfirst'] = True return dateutil.parser.parse(date, **kwargs) WEEK = {'MONDAY': 0, 'TUESDAY': 1, 'WEDNESDAY': 2, 'THURSDAY': 3, 'FRIDAY': 4, 'SATURDAY': 5, 'SUNDAY': 6, 'LUNDI': 0, 'MARDI': 1, 'MERCREDI': 2, 'JEUDI': 3, 'VENDREDI': 4, 'SAMEDI': 5, 'DIMANCHE': 6, } def get_date_from_day(day): today = date.today() today_day_number = today.weekday() requested_day_number = WEEK[day.upper()] if today_day_number < requested_day_number: day_to_go = requested_day_number - today_day_number else: day_to_go = 7 - today_day_number + requested_day_number requested_date = today + timedelta(day_to_go) return date(requested_date.year, requested_date.month, requested_date.day) def parse_date(string): matches = re.search('\s*([012]?[0-9]|3[01])\s*/\s*(0?[1-9]|1[012])\s*/?(\d{2}|\d{4})?$', string) if matches: year = matches.group(3) if not year: year = date.today().year elif len(year) == 2: year = 2000 + int(year) return date(int(year), int(matches.group(2)), int(matches.group(1))) elif string.upper() in WEEK.keys(): return get_date_from_day(string) elif string.upper() == "TODAY": return date.today() def closest_date(date, date_from, date_to): """ Adjusts year so that the date is closest to the given range. Transactions dates in a statement usually contain only day and month. Statement dates range have a year though. Merge them all together to get a full transaction date. """ # If the date is within given range, we're done. if date_from <= date <= date_to: return date dates = [real_datetime(year, date.month, date.day) for year in xrange(date_from.year, date_to.year+1)] # Ideally, pick the date within given range. for d in dates: if date_from <= d <= date_to: return d # Otherwise, return the most recent date in the past. return min(dates, key=lambda d: abs(d-date_from)) def test(): dt = real_datetime range1 = [dt(2012,12,20), dt(2013,1,10)] assert closest_date(dt(2012,12,15), *range1) == dt(2012,12,15) assert closest_date(dt(2000,12,15), *range1) == dt(2012,12,15) assert closest_date(dt(2020,12,15), *range1) == dt(2012,12,15) assert closest_date(dt(2013,1,15), *range1) == dt(2013,1,15) assert closest_date(dt(2000,1,15), *range1) == dt(2013,1,15) assert closest_date(dt(2020,1,15), *range1) == dt(2013,1,15) assert closest_date(dt(2013,1,1), *range1) == dt(2013,1,1) assert closest_date(dt(2000,1,1), *range1) == dt(2013,1,1) assert closest_date(dt(2020,1,1), *range1) == dt(2013,1,1) range2 = [dt(2012,12,20), dt(2014,1,10)] assert closest_date(dt(2012,12,15), *range2) == dt(2013,12,15) assert closest_date(dt(2014,1,15), *range2) == dt(2013,1,15) weboob-1.2/weboob/tools/decorators.py000066400000000000000000000037331303450110500177630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import logging import time __all__ = ['retry'] def retry(ExceptionToCheck, tries=4, delay=3, backoff=2): """ Retry decorator from http://www.saltycrane.com/blog/2009/11/trying-out-retry-decorator-python/ original from http://wiki.python.org/moin/PythonDecoratorLibrary#Retry """ def deco_retry(f): def f_retry(*args, **kwargs): mtries, mdelay = tries, delay mtries = kwargs.pop('_tries', mtries) mdelay = kwargs.pop('_delay', mdelay) try_one_last_time = True while mtries > 1: try: return f(*args, **kwargs) try_one_last_time = False break except ExceptionToCheck as e: try: logging.debug(u'%s, Retrying in %d seconds...' % (e, mdelay)) except UnicodeDecodeError: logging.debug(u'%s, Retrying in %d seconds...' % (repr(e), mdelay)) time.sleep(mdelay) mtries -= 1 mdelay *= backoff if try_one_last_time: return f(*args, **kwargs) return return f_retry # true decorator return deco_retry weboob-1.2/weboob/tools/html.py000066400000000000000000000030541303450110500165560ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import warnings __all__ = ['html2text'] try: from html2text import HTML2Text def html2text(html): h = HTML2Text() h.unicode_snob = True h.skip_internal_links = True h.inline_links = False h.links_each_paragraph = True return unicode(h.handle(html)) except ImportError: # Older versions of html2text do not have a class, so we have # to configure the module globally. try: import html2text as h2t h2t.UNICODE_SNOB = 1 h2t.SKIP_INTERNAL_LINKS = True h2t.INLINE_LINKS = False h2t.LINKS_EACH_PARAGRAPH = True html2text = h2t.html2text except ImportError: def html2text(html): warnings.warn('python-html2text is not present. HTML pages are not converted into text.', stacklevel=2) return html weboob-1.2/weboob/tools/js.py000066400000000000000000000032131303450110500162230ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2015 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . __all__ = ['Javascript'] from weboob.tools.log import getLogger class Javascript(object): HEADER = """ function btoa(str) { var buffer ; if (str instanceof Buffer) { buffer = str; } else { buffer = new Buffer(str.toString(), 'binary'); } return buffer.toString('base64'); } function atob(str) { return new Buffer(str, 'base64').toString('binary'); } var document = {}; """ def __init__(self, script, logger=None): try: import execjs except ImportError: raise ImportError('Please install PyExecJS') self.runner = execjs.get() self.logger = getLogger('js', logger) self.ctx = self.runner.compile(self.HEADER + script) def call(self, *args, **kwargs): retval = self.ctx.call(*args, **kwargs) self.logger.debug('Calling %s%s = %s', args[0], args[1:], retval) return retval weboob-1.2/weboob/tools/json.py000066400000000000000000000061471303450110500165710ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012 Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . # because we don't want to import this file by "import json" from __future__ import absolute_import from decimal import Decimal from datetime import datetime, date, time, timedelta __all__ = ['json', 'mini_jsonpath'] try: # try simplejson first because it is faster import simplejson as json except ImportError: # Python 2.6+ has a module similar to simplejson import json from weboob.capabilities.base import BaseObject, NotAvailable, NotLoaded def mini_jsonpath(node, path): """ Evaluates a dot separated path against JSON data. Path can contains star wilcards. Always returns a generator. Relates to http://goessner.net/articles/JsonPath/ but in a really basic and simpler form. >>> list(mini_jsonpath({"x": 95, "y": 77, "z": 68}, 'y')) [77] >>> list(mini_jsonpath({"x": {"y": {"z": "nested"}}}, 'x.y.z')) ['nested'] >>> list(mini_jsonpath('{"data": [{"x": "foo", "y": 13}, {"x": "bar", "y": 42}, {"x": "baz", "y": 128}]}', 'data.*.y')) [13, 42, 128] """ def iterkeys(i): return range(len(i)) if type(i) is list else i.iterkeys() def cut(s): p = s.split('.', 1) if s else [None] return p + [None] if len(p) == 1 else p if isinstance(node, basestring): node = json.loads(node) queue = [(node, cut(path))] while queue: node, (name, rest) = queue.pop(0) if name is None: yield node continue elif name == '*': keys = iterkeys(node) elif type(node) not in (dict, list) or name not in node: continue else: keys = [int(name) if type(node) is list else name] for k in keys: queue.append((node[k], cut(rest))) class WeboobEncoder(json.JSONEncoder): """JSON encoder class for weboob objects (and Decimal and dates) >>> json.dumps(object, cls=WeboobEncoder) '{"id": "1234@backend", "url": null}' """ def default(self, o): if o is NotAvailable: return None elif o is NotLoaded: return None elif isinstance(o, BaseObject): return o.to_dict() elif isinstance(o, Decimal): return str(o) elif isinstance(o, (datetime, date, time)): return o.isoformat() elif isinstance(o, timedelta): return o.total_seconds() return super(WeboobEncoder, self).default(o) weboob-1.2/weboob/tools/log.py000066400000000000000000000043261303450110500163760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import print_function import sys from collections import defaultdict from logging import addLevelName, Formatter, getLogger as _getLogger __all__ = ['getLogger', 'createColoredFormatter', 'settings'] RESET_SEQ = "\033[0m" COLOR_SEQ = "%s%%s" + RESET_SEQ COLORS = { 'DEBUG': COLOR_SEQ % "\033[0;36m", 'INFO': COLOR_SEQ % "\033[32m", 'WARNING': COLOR_SEQ % "\033[1;33m", 'ERROR': COLOR_SEQ % "\033[1;31m", 'CRITICAL': COLOR_SEQ % ("\033[1;33m\033[1;41m"), 'DEBUG_FILTERS': COLOR_SEQ % "\033[0;35m", } DEBUG_FILTERS = 8 addLevelName(DEBUG_FILTERS, 'DEBUG_FILTERS') # Global settings f logger. settings = defaultdict(lambda: None) def getLogger(name, parent=None): if parent: name = parent.name + '.' + name logger = _getLogger(name) logger.settings = settings return logger class ColoredFormatter(Formatter): """ Class written by airmind: http://stackoverflow.com/questions/384076/how-can-i-make-the-python-logging-output-to-be-colored """ def format(self, record): levelname = record.levelname msg = Formatter.format(self, record) if levelname in COLORS: msg = COLORS[levelname] % msg return msg def createColoredFormatter(stream, format): if (sys.platform != 'win32') and stream.isatty(): return ColoredFormatter(format) else: return Formatter(format) if __name__ == '__main__': for levelname, cs in COLORS.items(): print(cs % levelname, end=' ') weboob-1.2/weboob/tools/lrudict.py000066400000000000000000000027151303450110500172630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2012-2016 weboob project # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from collections import OrderedDict __all__ = ['LimitedLRUDict', 'LRUDict'] class LRUDict(OrderedDict): """dict to store items in the order the keys were last added/fetched.""" def __setitem__(self, key, value): if key in self: del self[key] super(LRUDict, self).__setitem__(key, value) def __getitem__(self, key): value = super(LRUDict, self).__getitem__(key) self[key] = value return value class LimitedLRUDict(LRUDict): """dict to store only the N most recent items.""" max_entries = 100 def __setitem__(self, key, value): super(LimitedLRUDict, self).__setitem__(key, value) if len(self) > self.max_entries: self.popitem(last=False) weboob-1.2/weboob/tools/misc.py000066400000000000000000000124101303450110500165410ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2014 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from time import time, sleep import locale import os import sys import traceback import types # keep compatibility from .compat import unicode __all__ = ['get_backtrace', 'get_bytes_size', 'iter_fields', 'to_unicode', 'limit', 'find_exe'] def get_backtrace(empty="Empty backtrace."): """ Try to get backtrace as string. Returns "Error while trying to get backtrace" on failure. """ try: info = sys.exc_info() trace = traceback.format_exception(*info) sys.exc_clear() if trace[0] != "None\n": return "".join(trace) except: return "Error while trying to get backtrace" return empty def get_bytes_size(size, unit_name): r"""Converts a unit and a number into a number of bytes. >>> get_bytes_size(2, 'KB') 2048.0 """ unit_data = { 'bytes': 1, 'KB': 1024, 'KiB': 1024, 'MB': 1024 * 1024, 'MiB': 1024 * 1024, 'GB': 1024 * 1024 * 1024, 'GiB': 1024 * 1024 * 1024, 'TB': 1024 * 1024 * 1024 * 1024, 'TiB': 1024 * 1024 * 1024 * 1024, } return float(size * unit_data.get(unit_name, 1)) def iter_fields(obj): for attribute_name in dir(obj): if attribute_name.startswith('_'): continue attribute = getattr(obj, attribute_name) if not isinstance(attribute, types.MethodType): yield attribute_name, attribute def to_unicode(text): r""" >>> to_unicode('ascii') u'ascii' >>> to_unicode(u'utf\xe9'.encode('UTF-8')) u'utf\xe9' >>> to_unicode(u'unicode') u'unicode' """ if isinstance(text, unicode): return text if not isinstance(text, str): try: text = str(text) except UnicodeError: return unicode(text) try: return unicode(text, 'utf-8') except UnicodeError: try: return unicode(text, 'iso-8859-15') except UnicodeError: return unicode(text, 'windows-1252', 'replace') def guess_encoding(stdio): try: encoding = stdio.encoding or locale.getpreferredencoding() except AttributeError: encoding = None # ASCII or ANSII is most likely a user mistake if not encoding or encoding.lower() == 'ascii' or encoding.lower().startswith('ansi'): encoding = 'UTF-8' return encoding def limit(iterator, lim): """Iterate on the lim first elements of iterator.""" count = 0 iterator = iter(iterator) while count < lim: yield iterator.next() count += 1 def ratelimit(group, delay): """ Simple rate limiting. Waits if the last call of lastlimit with this group name was less than delay seconds ago. The rate limiting is global, shared between any instance of the application and any call to this function sharing the same group name. The same group name should not be used with different delays. This function is intended to be called just before the code that should be rate-limited. This function is not thread-safe. For reasonably non-critical rate limiting (like accessing a website), it should be sufficient nevertheless. @param group [string] rate limiting group name, alphanumeric @param delay [int] delay in seconds between each call """ from tempfile import gettempdir path = os.path.join(gettempdir(), 'weboob_ratelimit.%s' % group) while True: try: offset = time() - os.stat(path).st_mtime except OSError: with open(path, 'w'): pass offset = 0 if delay < offset: break sleep(delay - offset) os.utime(path, None) def find_exe(basename): """ Find the path to an executable by its base name (such as 'gpg'). The executable can be overriden using an environment variable in the form `NAME_EXECUTABLE` where NAME is the specified base name in upper case. If the environment variable is not provided, the PATH will be searched both without and with a ".exe" suffix for Windows compatibility. If the executable can not be found, None is returned. """ env_exe = os.getenv('%s_EXECUTABLE' % basename.upper()) if env_exe and os.path.exists(env_exe) and os.access(env_exe, os.X_OK): return env_exe paths = os.getenv('PATH', os.defpath).split(os.pathsep) for path in paths: for ex in (basename, basename + '.exe'): fpath = os.path.join(path, ex) if os.path.exists(fpath) and os.access(fpath, os.X_OK): return fpath weboob-1.2/weboob/tools/newsfeed.py000066400000000000000000000055731303450110500174220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Clément Schreiner # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import datetime try: import feedparser except ImportError: raise ImportError('Please install python-feedparser') if '5.1' > feedparser.__version__ >= '5.0': # feedparser 5.0.x replaces this regexp on sgmllib # and mechanize < 0.2 fails with malformed pages. import sgmllib import re sgmllib.endbracket = re.compile('[<>]') __all__ = ['Entry', 'Newsfeed'] class Entry(object): def __init__(self, entry, rssid_func=None): if hasattr(entry, 'id'): self.id = entry.id else: self.id = None if "link" in entry: self.link = entry["link"] if not self.id: self.id = entry["link"] else: self.link = None if "title" in entry: self.title = entry["title"] else: self.title = None if "author" in entry: self.author = entry["author"] else: self.author = None if "updated_parsed" in entry: self.datetime = datetime.datetime(*entry['updated_parsed'][:7]) elif "published_parsed" in entry: self.datetime = datetime.datetime(*entry['published_parsed'][:7]) else: self.datetime = None if "summary" in entry: self.summary = entry["summary"] else: self.summary = None self.content = [] if "content" in entry: for i in entry["content"]: self.content.append(i.value) elif self.summary: self.content.append(self.summary) if "wfw_commentrss" in entry: self.rsscomment = entry["wfw_commentrss"] else: self.rsscomment = None if rssid_func: self.id = rssid_func(self) class Newsfeed(object): def __init__(self, url, rssid_func=None): self.feed = feedparser.parse(url) self.rssid_func = rssid_func def iter_entries(self): for entry in self.feed['entries']: yield Entry(entry, self.rssid_func) def get_entry(self, id): for entry in self.iter_entries(): if entry.id == id: return entry weboob-1.2/weboob/tools/ordereddict.py000066400000000000000000000116031303450110500201010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from __future__ import absolute_import __all__ = ['OrderedDict'] try: from collections import OrderedDict except ImportError: try: from simplejson import OrderedDict except ImportError: try: from ordereddict import OrderedDict except ImportError: # {{{ http://code.activestate.com/recipes/576693/ (r6) from UserDict import DictMixin class OrderedDict(dict, DictMixin): def __init__(self, *args, **kwds): if len(args) > 1: raise TypeError('expected at most 1 arguments, got %d' % len(args)) try: self.__end except AttributeError: self.clear() self.update(*args, **kwds) def clear(self): self.__end = end = [] end += [None, end, end] # sentinel node for doubly linked list self.__map = {} # key --> [key, prev, next] dict.clear(self) def __setitem__(self, key, value): if key not in self: end = self.__end curr = end[1] curr[2] = end[1] = self.__map[key] = [key, curr, end] dict.__setitem__(self, key, value) def __delitem__(self, key): dict.__delitem__(self, key) key, prev, next = self.__map.pop(key) prev[2] = next next[1] = prev def __iter__(self): end = self.__end curr = end[2] while curr is not end: yield curr[0] curr = curr[2] def __reversed__(self): end = self.__end curr = end[1] while curr is not end: yield curr[0] curr = curr[1] def popitem(self, last=True): if not self: raise KeyError('dictionary is empty') if last: key = reversed(self).next() else: key = iter(self).next() value = self.pop(key) return key, value def __reduce__(self): items = [[k, self[k]] for k in self] tmp = self.__map, self.__end del self.__map, self.__end inst_dict = vars(self).copy() self.__map, self.__end = tmp if inst_dict: return (self.__class__, (items,), inst_dict) return self.__class__, (items,) def keys(self): return list(self) setdefault = DictMixin.setdefault update = DictMixin.update pop = DictMixin.pop values = DictMixin.values items = DictMixin.items iterkeys = DictMixin.iterkeys itervalues = DictMixin.itervalues iteritems = DictMixin.iteritems def __repr__(self): if not self: return '%s()' % (self.__class__.__name__,) return '%s(%r)' % (self.__class__.__name__, self.items()) def copy(self): return self.__class__(self) @classmethod def fromkeys(cls, iterable, value=None): d = cls() for key in iterable: d[key] = value return d def __eq__(self, other): if isinstance(other, OrderedDict): return len(self) == len(other) and self.items() == other.items() return dict.__eq__(self, other) def __ne__(self, other): return not self == other # end of http://code.activestate.com/recipes/576693/ }}} weboob-1.2/weboob/tools/path.py000066400000000000000000000051321303450110500165450ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2012 Nicolas Duhamel, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from copy import copy from posixpath import sep, join class WorkingPath(object): def __init__(self): self.split_path = [] self.previous = copy(self.split_path) def cd1(self, user_input): """ Append *one* level to the current path. This means that separators (/) will get escaped. """ split_path = self.get() split_path.append(user_input) self.location(split_path) def location(self, split_path): """ Go to a new path, and store the previous path. """ self.previous = self.get() self.split_path = split_path def restore(self): """ Go to the previous path """ self.split_path, self.previous = self.previous, self.split_path def home(self): """ Go to the root """ self.location([]) def up(self): """ Go up one directory """ self.location(self.split_path[:-1]) def get(self): """ Get the current working path """ return copy(self.split_path) def __unicode__(self): return join(sep, *[s.replace(u'/', u'\/') for s in self.split_path]) def test(): wp = WorkingPath() assert wp.get() == [] assert unicode(wp) == u'/' wp.cd1(u'lol') assert wp.get() == [u'lol'] assert unicode(wp) == u'/lol' wp.cd1(u'cat') assert wp.get() == [u'lol', u'cat'] assert unicode(wp) == u'/lol/cat' wp.restore() assert unicode(wp) == u'/lol' wp.home() assert wp.get() == [] assert unicode(wp) == u'/' wp.up() assert wp.get() == [] assert unicode(wp) == u'/' wp.location(['aa / aa', 'bbbb']) assert unicode(wp) == u'/aa \/ aa/bbbb' wp.up() assert unicode(wp) == u'/aa \/ aa' wp.cd1(u'héhé/hé') assert unicode(wp) == u'/aa \/ aa/héhé\/hé' weboob-1.2/weboob/tools/pdf.py000066400000000000000000000203341303450110500163630ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from bisect import bisect_left from cStringIO import StringIO from collections import namedtuple import os import subprocess from tempfile import mkstemp __all__ = ['decompress_pdf', 'get_pdf_rows'] def decompress_pdf(inpdf): """ Takes PDF file contents as a string and returns decompressed version of the file contents, suitable for text parsing. External dependencies: MuPDF (http://www.mupdf.com). """ inh, inname = mkstemp(suffix='.pdf') outh, outname = mkstemp(suffix='.pdf') os.write(inh, inpdf) os.close(inh) os.close(outh) subprocess.call(['mutool', 'clean', '-d', inname, outname]) with open(outname) as f: outpdf = f.read() os.remove(inname) os.remove(outname) return outpdf # fuzzy floats to smooth comparisons because lines are actually rects # and seemingly-contiguous lines are actually not contiguous class ApproxFloat(float): APPROX_THRESHOLD = 2 @classmethod def _almost_eq(cls, a, b): return abs(a - b) < cls.APPROX_THRESHOLD def __eq__(self, other): return self._almost_eq(self, other) def __ne__(self, other): return not self == other def __lt__(self, other): return self - other < 0 and self != other def __le__(self, other): return self - other <= 0 or self == other def __gt__(self, other): return not self <= other def __ge__(self, other): return not self < other Rect = namedtuple('Rect', ('x0', 'y0', 'x1', 'y1')) TextRect = namedtuple('TextRect', ('x0', 'y0', 'x1', 'y1', 'text')) def lt_to_coords(obj, ltpage): # in a pdf, 'y' coords are bottom-to-top return Rect( ApproxFloat(min(obj.x0, obj.x1)), ApproxFloat(min(ltpage.y1 - obj.y0, ltpage.y1 - obj.y1)), ApproxFloat(max(obj.x0, obj.x1)), ApproxFloat(max(ltpage.y1 - obj.y0, ltpage.y1 - obj.y1)) ) def lttext_to_multilines(obj, ltpage): # text lines within 'obj' are probably the same height x0 = min(obj.x0, obj.x1) y0 = min(ltpage.y1 - obj.y0, ltpage.y1 - obj.y1) x1 = max(obj.x0, obj.x1) y1 = max(ltpage.y1 - obj.y0, ltpage.y1 - obj.y1) lines = obj.get_text().rstrip('\n').split('\n') h = (y1 - y0) / len(lines) for n, line in enumerate(lines): yield TextRect(x0, y0 + n * h, x1, y0 + n * h + h, line) def is_rect_contained_in(inner, outer): return (outer.x0 <= inner.x0 <= inner.x1 <= outer.x1 and outer.y0 <= inner.y0 <= inner.y1 <= outer.y1) ANGLE_VERTICAL = 0 ANGLE_HORIZONTAL = 1 ANGLE_OTHER = 2 def angle(r): if r.x0 == r.x1: return ANGLE_VERTICAL elif r.y0 == r.y1: return ANGLE_HORIZONTAL return ANGLE_OTHER def build_boxes(objs): # TODO find rects that are not drawn by 4 lines top-right-bottom-left boxes = [] objs = [obj for obj in objs if angle(obj) in (ANGLE_HORIZONTAL, ANGLE_VERTICAL)] i = 0 while i + 3 < len(objs): angles = tuple(map(angle, objs[i:i+4])) if angles != (ANGLE_HORIZONTAL, ANGLE_VERTICAL, ANGLE_HORIZONTAL, ANGLE_VERTICAL): i += 1 continue if not (objs[i].x1 == objs[i+1].x1 and objs[i].y0 == objs[i+1].y0 and objs[i+1].x1 == objs[i+2].x1 and objs[i+1].y1 == objs[i+2].y1 and objs[i+2].x0 == objs[i+3].x0 and objs[i+2].y1 == objs[i+3].y1 and objs[i+3].x0 == objs[i].x0 and objs[i].y0 == objs[i].y0): i += 1 continue boxes.append(Rect(objs[i].x0, objs[i].y0, objs[i+1].x0, objs[i+1].y1)) i += 4 return boxes class OrderedMap(object): # keys are ordered by total ordering on key objects def __init__(self): self.data = [] def __setitem__(self, k, v): pos = bisect_left(self.data, (k,)) try: if self.data[pos][0] == k: self.data[pos] = (k, v) return except IndexError: pass self.data.insert(pos, (k, v)) def __getitem__(self, k): pos = bisect_left(self.data, (k,)) if pos >= len(self.data): raise KeyError() elif self.data[pos][0] == k: return self.data[pos][1] else: raise KeyError() def __delitem__(self, k): pos = bisect_left(self.data, (k,)) if pos >= len(self.data): raise KeyError() elif self.data[pos][0] == k: del self.data[pos] else: raise IndexError() def setdefault(self, k, v): try: return self[k] except KeyError: self[k] = v return v def __iter__(self): return iter(t[1] for t in self.data) def build_rows(boxes): rows = OrderedMap() for box in boxes: row = rows.setdefault((box.y0, box.y1), []) row.append(box) for row in rows: row.sort(key=lambda box: box.x0) return rows def arrange_texts_in_rows(trects, rows): trows = {} for trect in trects: for nrow, row in enumerate(rows): for ncell, cell in enumerate(row): if is_rect_contained_in(trect, cell): if nrow not in trows: trows[nrow] = [[] for _ in row] trows[nrow][ncell].append(trect.text) return trows def get_pdf_rows(data): """ Takes PDF file content as string and yield table row data for each page. A dict is yielded for each page. Each dict contains rows as values. Each row is a list of cells. Each cell is a list of strings present in the cell. Note that the rows may belong to different tables. There are no logic tables in PDF format, so this parses PDF drawing instructions and tries to find rectangles and arrange them in rows, then arrange text in the rectangles. External dependencies: PDFMiner (http://www.unixuser.org/~euske/python/pdfminer/index.html). """ try: from pdfminer.pdfparser import PDFParser, PDFSyntaxError except ImportError: raise ImportError('Please install python-pdfminer') try: from pdfminer.pdfdocument import PDFDocument from pdfminer.pdfpage import PDFPage newapi = True except ImportError: from pdfminer.pdfparser import PDFDocument newapi = False from pdfminer.converter import PDFPageAggregator from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter from pdfminer.layout import LAParams, LTRect, LTTextBox, LTTextLine parser = PDFParser(StringIO(data)) try: if newapi: doc = PDFDocument(parser) else: doc = PDFDocument() parser.set_document(doc) doc.set_parser(parser) except PDFSyntaxError: return rsrcmgr = PDFResourceManager() device = PDFPageAggregator(rsrcmgr, laparams=LAParams()) interpreter = PDFPageInterpreter(rsrcmgr, device) if newapi: pages = PDFPage.get_pages(StringIO(data), check_extractable=True) else: doc.initialize() pages = doc.get_pages() for npage, page in enumerate(pages): interpreter.process_page(page) page_layout = device.get_result() texts = sum([list(lttext_to_multilines(obj, page_layout)) for obj in page_layout._objs if isinstance(obj, (LTTextBox, LTTextLine))], []) lines = [lt_to_coords(obj, page_layout) for obj in page_layout._objs if isinstance(obj, LTRect)] boxes = build_boxes(lines) rows = build_rows(boxes) textrows = arrange_texts_in_rows(texts, rows) yield textrows device.close() weboob-1.2/weboob/tools/property.py000066400000000000000000000033421303450110500174760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # For Python 2.5-, this will enable the similar property mechanism as in # Python 2.6+/3.0+. The code is based on # http://bruynooghe.blogspot.com/2008/04/xsetter-syntax-in-python-25.html # Copyright(C) 2010-2011 Christophe Benz # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import sys import __builtin__ class property(property): def __init__(self, fget, *args, **kwargs): self.__doc__ = fget.__doc__ super(property, self).__init__(fget, *args, **kwargs) def setter(self, fset): cls_ns = sys._getframe(1).f_locals for k, v in cls_ns.iteritems(): if v == self: propname = k break cls_ns[propname] = property(self.fget, fset, self.fdel, self.__doc__) return cls_ns[propname] def deleter(self, fdel): cls_ns = sys._getframe(1).f_locals for k, v in cls_ns.iteritems(): if v == self: propname = k break cls_ns[propname] = property(self.fget, self.fset, fdel, self.__doc__) return cls_ns[propname] __builtin__.property = property weboob-1.2/weboob/tools/regex_helper.py000066400000000000000000000336051303450110500202700ustar00rootroot00000000000000# Copyright (c) Django Software Foundation and individual contributors. # All rights reserved. # # Redistribution and use in source and binary forms, with or without modification, # are permitted provided that the following conditions are met: # # 1. Redistributions of source code must retain the above copyright notice, # this list of conditions and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # 3. Neither the name of Django nor the names of its contributors may be used # to endorse or promote products derived from this software without # specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR # ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES # (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; # LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON # ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS # SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. """ Functions for reversing a regular expression (used in reverse URL resolving). Used internally by Django and not intended for external use. This is not, and is not intended to be, a complete reg-exp decompiler. It should be good enough for a large class of URLS, however. """ # Mapping of an escape character to a representative of that class. So, e.g., # "\w" is replaced by "x" in a reverse URL. A value of None means to ignore # this sequence. Any missing key is mapped to itself. ESCAPE_MAPPINGS = { "A": None, "b": None, "B": None, "d": u"0", "D": u"x", "s": u" ", "S": u"x", "w": u"x", "W": u"!", "Z": None, } class Choice(list): """ Used to represent multiple possibilities at this point in a pattern string. We use a distinguished type, rather than a list, so that the usage in the code is clear. """ class Group(list): """ Used to represent a capturing group in the pattern string. """ class NonCapture(list): """ Used to represent a non-capturing group in the pattern string. """ def normalize(pattern): """ Given a reg-exp pattern, normalizes it to a list of forms that suffice for reverse matching. This does the following: (1) For any repeating sections, keeps the minimum number of occurrences permitted (this means zero for optional groups). (2) If an optional group includes parameters, include one occurrence of that group (along with the zero occurrence case from step (1)). (3) Select the first (essentially an arbitrary) element from any character class. Select an arbitrary character for any unordered class (e.g. '.' or '\w') in the pattern. (4) Ignore comments and any of the reg-exp flags that won't change what we construct ("iLmsu"). "(?x)" is an error, however. (5) Raise an error on all other non-capturing (?...) forms (e.g. look-ahead and look-behind matches) and any disjunctive ('|') constructs. Django's URLs for forward resolving are either all positional arguments or all keyword arguments. That is assumed here, as well. Although reverse resolving can be done using positional args when keyword args are specified, the two cannot be mixed in the same reverse() call. """ # Do a linear scan to work out the special features of this pattern. The # idea is that we scan once here and collect all the information we need to # make future decisions. result = [] non_capturing_groups = [] consume_next = True pattern_iter = next_char(iter(pattern)) num_args = 0 # A "while" loop is used here because later on we need to be able to peek # at the next character and possibly go around without consuming another # one at the top of the loop. try: ch, escaped = pattern_iter.next() except StopIteration: return zip([u''], [[]]) try: while True: if escaped: result.append(ch) elif ch == '.': # Replace "any character" with an arbitrary representative. result.append(u".") elif ch == '|': # FIXME: One day we'll should do this, but not in 1.0. raise NotImplementedError() elif ch == "^": pass elif ch == '$': break elif ch == ')': # This can only be the end of a non-capturing group, since all # other unescaped parentheses are handled by the grouping # section later (and the full group is handled there). # # We regroup everything inside the capturing group so that it # can be quantified, if necessary. start = non_capturing_groups.pop() inner = NonCapture(result[start:]) result = result[:start] + [inner] elif ch == '[': # Replace ranges with the first character in the range. ch, escaped = pattern_iter.next() result.append(ch) ch, escaped = pattern_iter.next() while escaped or ch != ']': ch, escaped = pattern_iter.next() elif ch == '(': # Some kind of group. ch, escaped = pattern_iter.next() if ch != '?' or escaped: # A positional group name = "_%d" % num_args num_args += 1 result.append(Group(((u"%%(%s)s" % name), name))) walk_to_end(ch, pattern_iter) else: ch, escaped = pattern_iter.next() if ch in "iLmsu#": # All of these are ignorable. Walk to the end of the # group. walk_to_end(ch, pattern_iter) elif ch == ':': # Non-capturing group non_capturing_groups.append(len(result)) elif ch != 'P': # Anything else, other than a named group, is something # we cannot reverse. raise ValueError("Non-reversible reg-exp portion: '(?%s'" % ch) else: ch, escaped = pattern_iter.next() if ch not in ('<', '='): raise ValueError("Non-reversible reg-exp portion: '(?P%s'" % ch) # We are in a named capturing group. Extra the name and # then skip to the end. if ch == '<': terminal_char = '>' # We are in a named backreference. else: terminal_char = ')' name = [] ch, escaped = pattern_iter.next() while ch != terminal_char: name.append(ch) ch, escaped = pattern_iter.next() param = ''.join(name) # Named backreferences have already consumed the # parenthesis. if terminal_char != ')': result.append(Group(((u"%%(%s)s" % param), param))) walk_to_end(ch, pattern_iter) else: result.append(Group(((u"%%(%s)s" % param), None))) elif ch in "*?+{": # Quanitifers affect the previous item in the result list. count, ch = get_quantifier(ch, pattern_iter) if ch: # We had to look ahead, but it wasn't need to compute the # quanitifer, so use this character next time around the # main loop. consume_next = False if count == 0: if contains(result[-1], Group): # If we are quantifying a capturing group (or # something containing such a group) and the minimum is # zero, we must also handle the case of one occurrence # being present. All the quantifiers (except {0,0}, # which we conveniently ignore) that have a 0 minimum # also allow a single occurrence. result[-1] = Choice([None, result[-1]]) else: result.pop() elif count > 1: result.extend([result[-1]] * (count - 1)) else: # Anything else is a literal. result.append(ch) if consume_next: ch, escaped = pattern_iter.next() else: consume_next = True except StopIteration: pass except NotImplementedError: # A case of using the disjunctive form. No results for you! return zip([u''], [[]]) return zip(*flatten_result(result)) def next_char(input_iter): """ An iterator that yields the next character from "pattern_iter", respecting escape sequences. An escaped character is replaced by a representative of its class (e.g. \w -> "x"). If the escaped character is one that is skipped, it is not returned (the next character is returned instead). Yields the next character, along with a boolean indicating whether it is a raw (unescaped) character or not. """ for ch in input_iter: if ch != '\\': yield ch, False continue ch = input_iter.next() representative = ESCAPE_MAPPINGS.get(ch, ch) if representative is None: continue yield representative, True def walk_to_end(ch, input_iter): """ The iterator is currently inside a capturing group. We want to walk to the close of this group, skipping over any nested groups and handling escaped parentheses correctly. """ if ch == '(': nesting = 1 else: nesting = 0 for ch, escaped in input_iter: if escaped: continue elif ch == '(': nesting += 1 elif ch == ')': if not nesting: return nesting -= 1 def get_quantifier(ch, input_iter): """ Parse a quantifier from the input, where "ch" is the first character in the quantifier. Returns the minimum number of occurences permitted by the quantifier and either None or the next character from the input_iter if the next character is not part of the quantifier. """ if ch in '*?+': try: ch2, escaped = input_iter.next() except StopIteration: ch2 = None if ch2 == '?': ch2 = None if ch == '+': return 1, ch2 return 0, ch2 quant = [] while ch != '}': ch, escaped = input_iter.next() quant.append(ch) quant = quant[:-1] values = ''.join(quant).split(',') # Consume the trailing '?', if necessary. try: ch, escaped = input_iter.next() except StopIteration: ch = None if ch == '?': ch = None return int(values[0]), ch def contains(source, inst): """ Returns True if the "source" contains an instance of "inst". False, otherwise. """ if isinstance(source, inst): return True if isinstance(source, NonCapture): for elt in source: if contains(elt, inst): return True return False def flatten_result(source): """ Turns the given source sequence into a list of reg-exp possibilities and their arguments. Returns a list of strings and a list of argument lists. Each of the two lists will be of the same length. """ if source is None: return [u''], [[]] if isinstance(source, Group): if source[1] is None: params = [] else: params = [source[1]] return [source[0]], [params] result = [u''] result_args = [[]] pos = last = 0 for pos, elt in enumerate(source): if isinstance(elt, basestring): continue piece = u''.join(source[last:pos]) if isinstance(elt, Group): piece += elt[0] param = elt[1] else: param = None last = pos + 1 for i in range(len(result)): result[i] += piece if param: result_args[i].append(param) if isinstance(elt, (Choice, NonCapture)): if isinstance(elt, NonCapture): elt = [elt] inner_result, inner_args = [], [] for item in elt: res, args = flatten_result(item) inner_result.extend(res) inner_args.extend(args) new_result = [] new_args = [] for item, args in zip(result, result_args): for i_item, i_args in zip(inner_result, inner_args): new_result.append(item + i_item) new_args.append(args[:] + i_args) result = new_result result_args = new_args if pos >= last: piece = u''.join(source[last:]) for i in range(len(result)): result[i] += piece return result, result_args weboob-1.2/weboob/tools/storage.py000066400000000000000000000043761303450110500172660ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . from copy import deepcopy from .config.yamlconfig import YamlConfig class IStorage(object): def load(self, what, name, default={}): """ Load data from storage. """ raise NotImplementedError() def save(self, what, name): """ Write changes in storage on the disk. """ raise NotImplementedError() def set(self, what, name, *args): """ Set data in a path. """ raise NotImplementedError() def delete(self, what, name, *args): """ Delete a value or a path. """ raise NotImplementedError() def get(self, what, name, *args, **kwargs): """ Get a value or a path. """ raise NotImplementedError() class StandardStorage(IStorage): def __init__(self, path): self.config = YamlConfig(path) self.config.load() def load(self, what, name, default={}): d = {} if what not in self.config.values: self.config.values[what] = {} else: d = self.config.values[what].get(name, {}) self.config.values[what][name] = deepcopy(default) self.config.values[what][name].update(d) def save(self, what, name): self.config.save() def set(self, what, name, *args): self.config.set(what, name, *args) def delete(self, what, name, *args): self.config.delete(what, name, *args) def get(self, what, name, *args, **kwargs): return self.config.get(what, name, *args, **kwargs) weboob-1.2/weboob/tools/test.py000066400000000000000000000075711303450110500166010ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon, Laurent Bachelier # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import sys from functools import wraps from unittest import TestCase from weboob.core import Weboob # This is what nose does for Python 2.6 and lower compatibility # We do the same so nose becomes optional try: from unittest.case import SkipTest except: from nose.plugins.skip import SkipTest __all__ = ['BackendTest', 'SkipTest', 'skip_without_config'] class BackendTest(TestCase): MODULE = None def __init__(self, *args, **kwargs): TestCase.__init__(self, *args, **kwargs) self.backends = {} self.backend_instance = None self.backend = None self.weboob = Weboob() # Skip tests when passwords are missing self.weboob.requests.register('login', self.login_cb) if self.weboob.load_backends(modules=[self.MODULE]): # provide the tests with all available backends self.backends = self.weboob.backend_instances def login_cb(self, backend_name, value): raise SkipTest('missing config \'%s\' is required for this test' % value.label) def run(self, result): """ Call the parent run() for each backend instance. Skip the test if we have no backends. """ # This is a hack to fix an issue with nosetests running # with many tests. The default is 1000. sys.setrecursionlimit(10000) try: if not len(self.backends): self.backend = self.weboob.build_backend(self.MODULE, nofail=True) TestCase.run(self, result) else: # Run for all backend for backend_instance in self.backends.keys(): print(backend_instance) self.backend = self.backends[backend_instance] TestCase.run(self, result) finally: self.weboob.deinit() def shortDescription(self): """ Generate a description with the backend instance name. """ # do not use TestCase.shortDescription as it returns None return '%s [%s]' % (str(self), self.backend_instance) def is_backend_configured(self): """ Check if the backend is in the user configuration file """ return self.weboob.backends_config.backend_exists(self.backend.config.instname) def skip_without_config(*keys): """Decorator to skip a test if backend config is missing :param keys: if any of these keys is missing in backend config, skip test. Can be empty. """ for key in keys: if callable(key): raise TypeError('skip_without_config() must be called with arguments') def decorator(func): @wraps(func) def wrapper(self, *args, **kwargs): config = self.backend.config if not self.is_backend_configured(): raise SkipTest('a backend must be declared in configuration for this test') for key in keys: if not config[key].get(): raise SkipTest('config key %r is required for this test' % key) return func(self, *args, **kwargs) return wrapper return decorator weboob-1.2/weboob/tools/tokenizer.py000066400000000000000000000063341303450110500176300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2014 Oleg Plakhotniuk # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re __all__ = ['ReTokenizer'] class ReTokenizer(object): """ Simple regex-based tokenizer (AKA lexer or lexical analyser). Useful for PDF statements parsing. 1. There's a lexing table consisting of type-regex tuples. 2. Lexer splits text into chunks using the separator character. 3. Text chunk is sequentially matched against regexes and first successful match defines the type of the token. Check out test() function below for examples. """ def __init__(self, text, sep, lex): self._lex = lex self._tok = [ReToken(lex, chunk) for chunk in text.split(sep)] def tok(self, index): if 0 <= index < len(self._tok): return self._tok[index] else: return ReToken(self._lex, eof=True) def simple_read(self, token_type, pos, transform=lambda v: v): t = self.tok(pos) is_type = getattr(t, 'is_%s' % token_type)() return (pos+1, transform(t.value())) if is_type else (pos, None) class ReToken(object): def __init__(self, lex, chunk=None, eof=False): self._lex = lex self._eof = eof self._value = None self._type = None if chunk is not None: for type_, regex in self._lex: m = re.match(regex, chunk, flags=re.UNICODE) if m: self._type = type_ if len(m.groups()) == 1: self._value = m.groups()[0] elif m.groups(): self._value = m.groups() else: self._value = m.group(0) break def is_eof(self): return self._eof def value(self): return self._value def __getattr__(self, name): if name.startswith('is_'): return lambda: self._type == name[3:] raise AttributeError() def test(): t = ReTokenizer('foo bar baz', ' ', [('f', r'^f'), ('b', r'^b')]) assert t.tok(0).is_f() assert t.tok(1).is_b() assert t.tok(2).is_b() assert t.tok(-1).is_eof() assert t.tok(3).is_eof() assert not t.tok(-1).is_f() assert not t.tok(0).is_b() assert not t.tok(0).is_eof() t = ReTokenizer('nogroup onegroup multigroup', ' ', [ ('ng', r'^n.*$'), ('og', r'^one(g.*)$'), ('mg', r'^(m.*)(g.*)$')]) assert t.tok(-1).value() is None assert t.tok(0).value() == 'nogroup' assert t.tok(1).value() == 'group' assert t.tok(2).value() == ('multi', 'group') weboob-1.2/weboob/tools/value.py000066400000000000000000000215001303450110500167220ustar00rootroot00000000000000# -*- coding: utf-8 -*- # Copyright(C) 2010-2011 Romain Bignon # # This file is part of weboob. # # weboob is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # weboob is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Affero General Public License for more details. # # You should have received a copy of the GNU Affero General Public License # along with weboob. If not, see . import re import subprocess from .compat import check_output from .misc import to_unicode from .ordereddict import OrderedDict __all__ = ['ValuesDict', 'Value', 'ValueBackendPassword', 'ValueInt', 'ValueFloat', 'ValueBool'] class ValuesDict(OrderedDict): """ Ordered dictionarry which can take values in constructor. >>> ValuesDict(Value('a', label='Test'), ValueInt('b', label='Test2')) """ def __init__(self, *values): OrderedDict.__init__(self) for v in values: self[v.id] = v class Value(object): """ Value. :param label: human readable description of a value :type label: str :param required: if ``True``, the backend can't load if the key isn't found in its configuration :type required: bool :param default: an optional default value, used when the key is not in config. If there is no default value and the key is not found in configuration, the **required** parameter is implicitly set :param masked: if ``True``, the value is masked. It is useful for applications to know if this key is a password :type masked: bool :param regexp: if specified, on load the specified value is checked against this regexp, and an error is raised if it doesn't match :type regexp: str :param choices: if this parameter is set, the value must be in the list :param tiny: the value of choices can be entered by an user (as they are small) :type choices: (list,dict) """ def __init__(self, *args, **kwargs): if len(args) > 0: self.id = args[0] else: self.id = '' self.label = kwargs.get('label', kwargs.get('description', None)) self.description = kwargs.get('description', kwargs.get('label', None)) self.default = kwargs.get('default', None) if isinstance(self.default, str): self.default = to_unicode(self.default) self.regexp = kwargs.get('regexp', None) self.choices = kwargs.get('choices', None) if isinstance(self.choices, (list, tuple)): self.choices = OrderedDict(((v, v) for v in self.choices)) self.tiny = kwargs.get('tiny', None) self.masked = kwargs.get('masked', False) self.required = kwargs.get('required', self.default is None) self._value = kwargs.get('value', None) def show_value(self, v): if self.masked: return len(unicode(v)) * '*' else: return v def check_valid(self, v): """ Check if the given value is valid. :raises: ValueError """ if self.default is not None and v == self.default: return if v == '' and self.default != '' and (self.choices is None or v not in self.choices): raise ValueError('Value can\'t be empty') if self.regexp is not None and not re.match(self.regexp, unicode(v)): raise ValueError('Value "%s" does not match regexp "%s"' % (self.show_value(v), self.regexp)) if self.choices is not None and v not in self.choices: raise ValueError('Value "%s" is not in list: %s' % ( self.show_value(v), ', '.join(unicode(s) for s in self.choices))) def load(self, domain, v, requests): """ Load value. :param domain: what is the domain of this value :type domain: str :param v: value to load :param requests: list of weboob requests :type requests: weboob.core.requests.Requests """ return self.set(v) def set(self, v): """ Set a value. """ if isinstance(v, str): v = to_unicode(v) self.check_valid(v) self._value = v def dump(self): """ Dump value to be stored. """ return self.get() def get(self): """ Get the value. """ return self._value def is_command(self, v): """ Test if a value begin with ` and end with ` (`command` is used to call external programms) """ return isinstance(v, basestring) and v.startswith(u'`') and v.endswith(u'`') class ValueBackendPassword(Value): _domain = None _requests = None _stored = True def __init__(self, *args, **kwargs): kwargs['masked'] = kwargs.pop('masked', True) self.noprompt = kwargs.pop('noprompt', False) Value.__init__(self, *args, **kwargs) self.default = kwargs.get('default', '') def load(self, domain, password, requests): if self.is_command(password): cmd = password[1:-1] try: password = check_output(cmd, shell=True) except subprocess.CalledProcessError as e: raise ValueError(u'The call to the external tool failed: %s' % e) else: password = password.partition('\n')[0].strip('\r\n\t') self.check_valid(password) self._domain = domain self._value = to_unicode(password) self._requests = requests def check_valid(self, passwd): if passwd == '': # always allow empty passwords return True return Value.check_valid(self, passwd) def set(self, passwd): if self.is_command(passwd): self._value = passwd return self.check_valid(passwd) if passwd is None: # no change return self._value = '' if passwd == '': return if self._domain is None: self._value = to_unicode(passwd) return try: raise ImportError('Keyrings are disabled (see #706)') import keyring keyring.set_password(self._domain, self.id, passwd) except Exception: self._value = to_unicode(passwd) else: self._value = '' def dump(self): if self._stored: return self._value else: return '' def get(self): if self._value != '' or self._domain is None: return self._value try: raise ImportError('Keyrings are disabled (see #706)') import keyring except ImportError: passwd = None else: passwd = keyring.get_password(self._domain, self.id) if passwd is not None: # Password has been read in the keyring. return to_unicode(passwd) # Prompt user to enter password by hand. if not self.noprompt and self._requests: self._value = self._requests.request('login', self._domain, self) if self._value is None: self._value = '' else: self._value = to_unicode(self._value) self._stored = False return self._value class ValueInt(Value): def __init__(self, *args, **kwargs): kwargs['regexp'] = '^\d+$' Value.__init__(self, *args, **kwargs) self.default = kwargs.get('default', 0) def get(self): return int(self._value) class ValueFloat(Value): def __init__(self, *args, **kwargs): kwargs['regexp'] = '^[\d\.]+$' Value.__init__(self, *args, **kwargs) self.default = kwargs.get('default', 0.0) def check_valid(self, v): try: float(v) except ValueError: raise ValueError('Value "%s" is not a float value' % self.show_value(v)) def get(self): return float(self._value) class ValueBool(Value): def __init__(self, *args, **kwargs): kwargs['choices'] = {'y': 'True', 'n': 'False'} Value.__init__(self, *args, **kwargs) self.default = kwargs.get('default', False) def check_valid(self, v): if not isinstance(v, bool) and \ unicode(v).lower() not in ('y', 'yes', '1', 'true', 'on', 'n', 'no', '0', 'false', 'off'): raise ValueError('Value "%s" is not a boolean (y/n)' % self.show_value(v)) def get(self): return (isinstance(self._value, bool) and self._value) or \ unicode(self._value).lower() in ('y', 'yes', '1', 'true', 'on')