pax_global_header00006660000000000000000000000064151014174260014513gustar00rootroot0000000000000052 comment=65ba2c1c3a348fc54d5aa37e6c178bb181242bbe noss-1.10/000077500000000000000000000000001510141742600124165ustar00rootroot00000000000000noss-1.10/Changes000066400000000000000000000143021510141742600137110ustar00rootroot00000000000000Revision history for noss (WWW::Noss) 1.10 Oct 16 2025 - noss now attempts to resolve relative URLs ('/foo' and '//foo'). - Trim whitespace for %e feed descriptions. - noss no longer ignores Atom feed entries with no summary/content. - Fix " contains no posts" error message. - Require List::Util 1.45 for uniq(). - Document --autoclean in usage message. 1.09 Sep 16 2025 - Add support for MRSS (MediaRSS) feeds, like those used by YouTube. - Added ability to specify a negative post ID in open, read, cat, and post commands. This allows you to look for a post relative to the end of the feed (-1 corresponds to the last post, -2 to the 2nd to last, etc). - 'list --sort ...' now does alphabetical sorting case-insensitively. - Add --hard option to update and reload commands: Hard reload a feed by deleting it prior to reloading it into noss's database. - For posts that have no title but have a description, noss generates a title from the post's description. If a post has no description, its link is used instead. - Properly read HTML titles in Atom feeds. - Post and feed titles now have whitespace trimmed and truncated. - Fix bug where updating an existing post would replace its contents with its category JSON. - Fix bug introduced in 1.05 that messed up how noss processed Atom HTML post entries. - Fix feeds not being updated with the update command if their cached feed file is deleted. - Consider it an error if a feed was fetched without error but the respective feed file does not exist. - Fix lynx tests on Windows. - Fix formatting code error message. 1.08 Aug 30 2025 - noss now has its own mascot and logo. - Use date of the latest post as the updated date for an Atom feed if it doesn't have one itself. - Fixed conditional feed updates. - help command no longer requires a feeds file. - help and import commands no longer attempt to connect to noss's post database. - Updated curl error code list. 1.07 Aug 16 2025 - Add help command: Print documentation for a specific command. - Change formatting of usage message. - Change manual formatting. - New behavior for the open command: When not given a post ID to open, open's the feed's homepage URL instead. - Fix documentation errors. 1.06 Jul 25 2025 - Drastically speed up list command. - Drastically speed up update and reload commands. - The mark command accurately prints the number of posts that were actually changed. - Change database cache size to 10M, which should be speed up any command that involves interacting with the database. - Speed up start-up time. 1.05 Jul 17 2025 - Remove dependency on DateTime, DateTime::Format::Mail, and DateTime::Format::RFC3339 modules. This should give noss some minor performance improvements and drastically simplify its dependency tree. - Add dependency to Time::Piece. - Improved reading Atom post content elements. - export no longer exports shell:// and file:// feeds by default. This can be re-enabled with the --export-special option. - Added --list-limit option to list command: Limit the number of posts that are returned by the list command. - The feed and post commands now use the %z code for their updated fields by default. - Empty Atom author names/emails are now ignored. Existing databases will have to be reloaded to reflect this change. - More informative error messages. - Fix grammar in some messages. - Fix documentation errors. 1.04 Jul 04 2025 - Skip lynx tests if lynx is not installed. - nossui: Clean up temporary files that were forgotten in some menus. 1.03 Jun 27 2025 - Add cat command: Basically just an alias to `read --stdout`. - Add --html option read command: Read the unformatted HTML contents of a post. - Add '%z' formatting code for feed and post formatting, and the --time-format option: Print post/feed dates using the strftime format specified by --time-format. - nossui: Add 'Read' button to posts selection menu. - nossui: Allow updating/reloading by group or feed. - If a post does not have a specific author, use the feed's author if present. Existing databases will have to be reloaded to reflect this change. - Don't leak open directory handles in clean command. - No longer maintain META files manually. 1.02 Jun 24 2025 - Add nossui: dialog-based frontend to noss. - Fix documentation errors. 1.01 Jun 23 2025 - Include read status of post in default post command output. - feeds command now accepts group names as argument. - feeds will no longer print the same feed more than once. - Fix updating by group. - Fix reloading by group. - Fix group listing in list command. - Fix feeds not being included in the default update when they are a part of a group. - Fix list when no valid groups are given. 1.00 Jun 22 2025 - Add --read-format option for the read command: Specify the format to use for the HTML contents of a post to format when reading. - Add --list-format option for the list command: Specify the format to use for posts in the list command. - Add --post-format option for the post command: Specify the format to use for printing post information. - Add --feeds-option option for the feeds command: Specify the format to use for printing feed information. - Add --autoclean option: Automatically run the clean command after performing an operation. - Add --width option for the read command: Specify the line width to use for formatted text. - Changed default content formatting for the read command. - list command now prints 'r' for read posts. - The clean command now VACUUMs databases, which dramatically increases the amount of space shaved off. - noss no longer aborts when it fails to read a feed file in the update command, it will just throw a warning and continue processing subsequent feeds. - Have --sort take priority over the "sort" configuration field. - strftime is now used for formatting timestamps, which adds support for non-English locales. - Local time zone is now used for timestamps. - Fix typos in documentation. - Fix POD errors in developer documentation. 0.01 Jun 20 2025 - Initial release. noss-1.10/LICENSE000066400000000000000000001045151510141742600134310ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . noss-1.10/MANIFEST000066400000000000000000000016301510141742600135470ustar00rootroot00000000000000bin/noss bin/nossui Changes doc/nossui.pod img/noss-logo.png img/noss-the-mouse.png lib/WWW/Noss.pm lib/WWW/Noss/BaseConfig.pm lib/WWW/Noss/Curl.pm lib/WWW/Noss/DB.pm lib/WWW/Noss/Dir.pm lib/WWW/Noss/FeedConfig.pm lib/WWW/Noss/FeedReader.pm lib/WWW/Noss/FeedReader/Atom.pm lib/WWW/Noss/FeedReader/MediaRSS.pm lib/WWW/Noss/FeedReader/RSS.pm lib/WWW/Noss/GroupConfig.pm lib/WWW/Noss/Home.pm lib/WWW/Noss/Lynx.pm lib/WWW/Noss/OPML.pm lib/WWW/Noss/TextToHtml.pm lib/WWW/Noss/Timestamp.pm LICENSE Makefile.PL MANIFEST This list of files README.md t/01_init.t t/05_text2html.t t/10_home.t t/15_feed_config.t t/20_dir.t t/25_lynx.t t/27_times.t t/30_atom.t t/35_rss.t t/40_read_feed.t t/45_opml.t t/50_db.t t/data/atom.xml t/data/dir/.hidden.txt t/data/dir/a.txt t/data/dir/b.txt t/data/dir/c.txt t/data/lynx.html t/data/mail-dates t/data/mrss-atom.xml t/data/mrss-rss.xml t/data/opml.xml t/data/rfc3339-dates t/data/rss.xml noss-1.10/MANIFEST.SKIP000066400000000000000000000030761510141742600143220ustar00rootroot00000000000000 #!start included /usr/share/perl/5.36/ExtUtils/MANIFEST.SKIP # Avoid version control files. \bRCS\b \bCVS\b \bSCCS\b ,v$ \B\.svn\b \B\.git\b ^\.github\b \B\.gitignore\b \b_darcs\b \B\.cvsignore$ # Avoid VMS specific MakeMaker generated files \bDescrip.MMS$ \bDESCRIP.MMS$ \bdescrip.mms$ # Avoid Makemaker generated and utility files. \bMANIFEST\.bak \bMakefile$ \bblib/ \bMakeMaker-\d \bpm_to_blib\.ts$ \bpm_to_blib$ \bblibdirs\.ts$ # 6.18 through 6.25 generated this \b_eumm/ # 7.05_05 and above # Avoid Module::Build generated and utility files. \bBuild$ \b_build/ \bBuild.bat$ \bBuild.COM$ \bBUILD.COM$ \bbuild.com$ # and Module::Build::Tiny generated files \b_build_params$ # Avoid temp and backup files. ~$ \.old$ \#$ \b\.# \.bak$ \.tmp$ \.bak\b \.# \.rej$ \..*\.sw.?$ # Avoid OS-specific files/dirs # Mac OSX metadata \B\.DS_Store # Mac OSX SMB mount metadata files \B\._ # Placeholder files created when iCloud will "optimize Mac storage" \.i[cC]loud$ # Avoid Devel::Cover and Devel::CoverX::Covered files. \bcover_db\b \bcovered\b # Avoid prove files \B\.prove$ # Avoid MYMETA files ^MYMETA\. # Temp files for new META ^META_new\.(?:json|yml) # Avoid travis-ci.org file ^\.travis\.yml # Avoid AppVeyor file ^\.?appveyor.yml #!end included /usr/share/perl/5.36/ExtUtils/MANIFEST.SKIP # Avoid configuration metadata file ^MYMETA\. # Avoid Module::Build generated and utility files. \bBuild$ \bBuild.bat$ \b_build \bBuild.COM$ \bBUILD.COM$ \bbuild.com$ ^MANIFEST\.SKIP # Avoid archives of this distribution \bWWW-Noss[\d\.\_]+ ^TODO$ ^ignore ^contrib noss-1.10/Makefile.PL000066400000000000000000000062101510141742600143670ustar00rootroot00000000000000use 5.016; use strict; use warnings; use ExtUtils::MakeMaker; use DBD::SQLite; my %WriteMakefileArgs = ( NAME => 'WWW::Noss', AUTHOR => q{Samuel Young }, VERSION_FROM => 'lib/WWW/Noss.pm', ABSTRACT_FROM => 'lib/WWW/Noss.pm', LICENSE => 'gpl_3', MIN_PERL_VERSION => '5.016', CONFIGURE_REQUIRES => { 'DBD::SQLite' => '0', 'ExtUtils::MakeMaker' => '0', }, TEST_REQUIRES => { 'Test::More' => '0', 'File::Spec' => '0', # for UNLINK 'File::Temp' => '0.10', }, PREREQ_PM => { 'Cwd' => '0', 'DBD::SQLite' => '0', # for 'AutoInactiveDestroy' 'DBI' => '1.614', 'Getopt::Long' => '0', 'File::Basename' => '0', 'File::Copy' => '0', 'File::Spec' => '0', # for UNLINK 'File::Temp' => '0.10', 'JSON' => '0', # for any, none, all 'List::Util' => '1.45', # for retrieving data structures from children 'Parallel::ForkManager' => '0.7.6', 'Pod::Usage' => '0', 'POSIX' => '0', 'Time::Piece' => '0', # for load_xml() 'XML::LibXML' => '1.70', }, EXE_FILES => [ 'bin/noss', 'bin/nossui', ], MAN1PODS => { 'bin/noss' => 'blib/man1/noss.1', 'doc/nossui.pod' => 'blib/man1/nossui.1', }, META_MERGE => { 'meta-spec' => { version => 2 }, resources => { repository => { url => 'https://codeberg.org/1-1sam/noss.git', web => 'https://codeberg.org/1-1sam/noss', type => 'git', }, bugtracker => { web => 'https://codeberg.org/1-1sam/noss/issues', }, }, }, dist => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', }, clean => { FILES => 'WWW-Noss-*' }, ); # Compatibility with old versions of ExtUtils::MakeMaker unless (eval { ExtUtils::MakeMaker->VERSION('6.64'); 1 }) { my $test_requires = delete $WriteMakefileArgs{TEST_REQUIRES} || {}; @{$WriteMakefileArgs{PREREQ_PM}}{keys %$test_requires} = values %$test_requires; } unless (eval { ExtUtils::MakeMaker->VERSION('6.55_03'); 1 }) { my $build_requires = delete $WriteMakefileArgs{BUILD_REQUIRES} || {}; @{$WriteMakefileArgs{PREREQ_PM}}{keys %$build_requires} = values %$build_requires; } delete $WriteMakefileArgs{CONFIGURE_REQUIRES} unless eval { ExtUtils::MakeMaker->VERSION('6.52'); 1 }; delete $WriteMakefileArgs{MIN_PERL_VERSION} unless eval { ExtUtils::MakeMaker->VERSION('6.48'); 1 }; delete $WriteMakefileArgs{LICENSE} unless eval { ExtUtils::MakeMaker->VERSION('6.31'); 1 }; delete $WriteMakefileArgs{META_MERGE} unless eval { ExtUtils::MakeMaker->VERSION('6.46'); 1 }; # for RETURNING clause # NOTE: noss no longer actually uses the RETURNING clause, but we'll continue # to stick with 3.35.0 as the minimum supported version anyway, for the sake of # consistency. unless ($DBD::SQLite::sqlite_version_number >= 3035000) { die "noss requires at least sqlite version 3.35.0\n"; } WriteMakefile(%WriteMakefileArgs); noss-1.10/README.md000066400000000000000000000045461510141742600137060ustar00rootroot00000000000000# noss ![noss](img/noss-logo.png) **noss** is a command-line RSS/Atom feed reader and aggregator written in Perl. ## Installing **noss** has been packaged for the following repositories: * CPAN: * Alpine Linux: * Slackware Linux (SBo): If you cannot install **noss** through any of the above repositories, you can try building **noss** from source by following the instructions provided in the [Building](#Building) section below. ## Building **noss** should be able to run on most Unix-like and Windows systems. **noss** depends on the following: * `perl` (>= `5.16`) * `DBI` * `DBD::SQLite` * `JSON` * `Parallel::ForkManager` * `XML::LibXML` * `curl` (>= `7.68.0`) * `lynx` * `sqlite3` (>= `3.35.0`) **noss** optionally depends on the following: * `JSON::XS`: For faster JSON processing. * `dialog`: For the `nossui` script. Once the aforementioned dependencies are installed, **noss** can be built and installed via the following commands: ```bash perl Makefile.PL make make test make install ``` Please consult the documentation for the `ExtUtils::MakeMaker` Perl module for information on configuring the build process. ## Usage The documentation for the usage of **noss** can be found in its manual. ```bash man noss ``` It should contain all the information you need to get started using **noss**. **noss** also comes with **nossui**, a `dialog`-based TUI frontend to **noss** written in POSIX `sh`. Most of the regular functionality of **noss** should also be available through **nossui**. ## Thanks A huge thanks to the [newsraft](https://codeberg.org/newsraft/newsraft) and [newsboat](https://newsboat.org/) projects, which inspired the creation of this program and are also where I got many ideas for this project from. ## Author This program was written by Samuel Young, *\*. This project's source can be found on its [Codeberg page](https://codeberg.org/1-1sam/noss). Comments and pull requests are welcome! ## Copyright Copyright (C) 2025 Samuel Young This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. noss-1.10/bin/000077500000000000000000000000001510141742600131665ustar00rootroot00000000000000noss-1.10/bin/noss000066400000000000000000000737131510141742600141060ustar00rootroot00000000000000#!/usr/bin/perl use 5.016; use strict; use warnings; use WWW::Noss; my $noss = WWW::Noss->init(@ARGV); $noss->run; =head1 NAME noss - RSS/Atom feed reader and aggregator =head1 SYNOPSIS B command syntax: noss [options] [arguments] Commands: update Fetch and reload feeds reload Reload cached feeds read Read post in pager open Open feed or post URL in browser cat Print post to stdout list List and filter posts unread List unread posts mark Mark posts as read or unread post View post information feeds List feeds groups List feed groups clean Clean up obsolete files and data export Export feed list as OPML import Import feed list from OPML help View documentation for a noss command Global options: -c|--config= Specify path to configuration file -D|--data= Specify path to data directory -f|--feeds= Specify path to feeds file -z|--time-format= Specify default strftime format for time strings -a|--autoclean[=1|0] Toggle autoclean -h|--help Print usage message -v|--version Print version and copyright information =head1 DESCRIPTION B is a command-line program for aggregating and reading RSS/Atom feeds. B can fetch feed updates, list posts, read posts from the command-line, open posts in a browser, and much more. B works by storing feed and post information in a local database, and the user interacts with the database by supplying commands to B. B requires a feed list in order for it to function. A feed list is supplied to B via a feeds file, a file that contains a list of feeds and any special configuration to use for those feeds. The format of this file is documented in the subequent L section of this manual. Once a list of feeds is supplied to B through the feeds file, the feeds can then be fetched and loaded into its database via the C command. In order for B to read posts, it must be able to uniquely identify and distinguish posts from each other. Posts in B are identified via two pieces of information: the name of the feed they're in and their unique ID within that feed. When reading a post, like with either the C or C command, you would typically supply the command with the feed name and post ID to refer to that specific post. # Read post #1 in the PlanetDebian feed noss read PlanetDebian 1 To determine the feed and ID of a post, the C command can be used to query the post database for a list of posts that conform to whatever search parameters are provided (like title containing a string, content matching a regex, being tagged with a specific tag, etc.). The C command can also be used to query more detailed post information than the C command provides for a specific post. Consult the documentation for the C and C commands in the L section of this manual. So basically, the typical usage of B would involve managing your feed list from the feeds file, using the C command to fetch feed updates, using the C command to see what posts are available, and then using C or C commands for actually reading posts. # Add your feeds vim ~/.config/noss/noss.feeds # Fetch and update feed updates noss update # Look to see what posts you have noss list # ...or only check for unread ones noss unread # And then actually reading a post noss read whatever 12 The L section of this manual documents the complete list of commands and any options that are available to them. B can also have its behavior modified through a configuration file, whose format is documented in the L section of this manual. =head2 Feeds File As previously mentioned, B reads its feed list from a feeds file. The feeds file is a JSON file that contains a list of feeds, feed groups, and feed configurations for B to use. B uses a special JSON format that supports comments and trailing commas, so that it can be better suited as a configuration file format. Comments start with a hash (C<#>) sign, are terminated by the end of a line, and can be placed anywhere where JSON allows whitespace. B will look for a feeds file in the following locations: =over 4 =item Path specified by the C<-f>|C<--feeds> option =item F> =item F/noss/noss.feeds> =item F/noss.feeds> =item F<~/.config/noss/noss.feeds> =item F<~/.config/noss.feeds> =item F<~/.noss.feeds> =back The feeds file can have three different fields: C, C, and C. The only required field is the C field. =head3 feeds The C field is a key-value map of feed names and their configuration. Feed names can only contain alphanumeric/underscore characters, and must not be the name of another feed or group. The feed can either be mapped to a URL string of the feed it represents or a key-value map of configuration parameters for the feed. The former has the benefit of better readability but lacks the ability to have its configuration fine-tuned. The latter can have fine-tuned configuration but may look more cluttered than the former. { "feeds": { # Just the URL, no specific configuration "feed1": "https://phonysite.com/feed1.rss", # Key-value map allows for fine-tuned configuration "feed2": { "feed": "https://phonysite.com/feed2.rss", "limit": 100, "autoread": false, }, }, # yes, you can have trailing commas :-) # yes, you can have comments, too! } When using the latter-style key-value map syntax for a feed, the only required field is the C field, which must store the feed's URL. Additional fields, which are listed below, can be set to provide the previously mentioned fine-tuned configuration. =over 4 =item "B": I Specify a limit to the number of posts a feed can contain. If a feed exceeds the limit, B will discard older posts to fit into that limit. By default, there is no limit to the number of posts a feed can contain. =item "B": I Specify whether to respect the C and C fields in RSS feeds, which tells feed aggregators to skip updating feeds on certain hours or days. This is enabled by default, and is recommended to not be disabled. =item "B": "I" | [ "I", ... ] =item "B": "I" | [ "I", ... ] Specify a target string or list of target strings a post title must include or not include in order to not be filtered out by B during aggregation. A I string can either be a literal string that a post title must include/exclude, or a Perl regex (signified by a leading and trailing forward slash character) that a title must match/not match. =item "B": "I" | [ "I", ... ] =item "B": "I" | [ "I", ... ] Similar to the C and C parameters, except for a post's content instead of its title. =item "B": "I" | [ "I", ... ] =item "B": "I" | [ "I", ... ] Specify a tag or list of tags that a post must be tagged with or without in order to not be filtered out by B during aggregation. =item "B": I Specify whether new posts from this feed should automatically be marked as C or not. Defaults to C. =item "B": I Specify whether this feed should be included in a default update or not (C when ran with no arguments). Defaults to C. =item "B": I Specify whether this feed should be omitted from the C command's default listing (the list shown when ran with no arguments). Defaults to C. =back A feed's specific configuration takes priority over the configuration it may inherit from the C field or any groups it may be a part of. B uses L for fetching feeds, so generally any URL that is acceptable for L will be acceptable for B. There are two unique types of URLs that B will process in a special way. =over 4 =item file://I A URL that starts with the C protocol will be interpreted as a local file which B will copy over to its feed cache and load instead of fetching remotely via L. # Use the contents of "/home/sam/feeds/feed.rss" as a feed "file_feed": "file:///home/sam/feeds/feed.rss" =item shell://I A URL that starts with the C protocol will be interpreted as a shell command which B will execute and capture the output of, then use that output as the contents of the feed. # Use the output of the "cat feed.rss" command as a feed "shell_feed": "shell://cat feed.rss" =back =head3 groups C is a key-value map of group names and their configuration. Feed groups in B serve two purposes: 1) to provide a way to logically group multiple feeds together, and 2) provide a way to "share" a common configuration between multiple different feeds. Just like with feeds, group names can only contain alphanumeric/underscore characters and cannot be the same name as any other group or feed. A group can either be defined by a list of feed names with no special configuration, purely acting as a logical collection of feeds, or a key-value map of feed configuration parameters with a feed list stored in the C field of the map. The same options listed above for a feed's configuration in the C field can be used for a group's configuration. { "groups": { # purely logical feed grouping, no special configuration "group1": [ "f1", "f2", "f3" ], # each feed will inherit group2's configuration "group2": { "feeds": [ "f2", "f4", "f6" ], "limit": 100, "autoread": false, }, }, "feeds": { ... }, } Feeds are also allowed to be a part of multiple different groups. If a feed is a part of multiple different groups that try to configure the same parameter, B will try to resolve the conflict by picking the most "non-default" option available. Group configurations take priority over the C field but are overridden by a feed's own configuration. =head3 default The C field is a special feed group that every feed is a part of. Any configuration in the C feed will used by every feed and group (unless their configuration overrides the default's). The same configuration options available to a feed and group are available to C. { # All feeds have a limit of 100 posts and must contain 'foo' in their # titles. "default": { "limit": 100, "include_title": [ "/foo/" ], }, .. } =head1 COMMANDS =head2 update noss update [feed|group] ... Fetch and reload feeds. Whenever a new feed is added to B's feed list, this command can be used to load it into B's database. If no specific feeds or groups are specified, all feeds are updated. B has the following options: =over 4 =item B<--new-only> Only update feeds that are not present in the database. =item B<--non-defaults> When running B with no arguments, update all feeds, even those that are configured to not be updated by default. =item B<--downloads>=I Specify number of parallel downloads to perform. Default amount is C<10>. Can also be configured via the C configuration field. =item B<--unconditional> Unconditionally download feeds, even if they have not been modified since the last update. This option should be used with caution, as performing unconditional downloads is wasteful on feed servers and considered bad practice. =item B<--hard> Force B to do a hard update on the updated feeds. A hard update means that B will delete all of a feed's previous data prior to updating, meaning data like the read/unread status of a post are lost. This option implies C<--unconditional>. =item B<--limit-rate>=I Specify the maximum transfer rate for downloads. Speeds are measured in bytes/second, unless a suffix is appended. C/C, C/C, and C/C correspond to kilobytes, megabytes, and gigabytes respectively. Can also be configured via the C configuration field. =item B<--user-agent>=I Specify the string to use as the user-agent string. Can also be configured via the C configuration field. =item B<--timeout>=I Specify the maximum number of seconds a transfer is allowed to take. Can also be configured via the C configuration field. =item B<--proxy>=[I://]I[:I] Specify the proxy server to use. If a protocol is not specified, defaults as a C proxy. If no port number is specified, defaults to C<1000>. Can also be configured via the C configuration field. =item B<--proxy-user>=I:I Specify the username and password to use for proxy authentication. Can also be configured via the C configuration field. =back =head2 reload noss reload [feed|group] ... Reload cached feeds. Useful for when you make changes to a feed's configuration and want B to recognize the changes. If no feeds are specified, reloads all cached feeds. B has the following options: =over 4 =item B<--hard> Forces B to do a hard reload on the updated feeds. A hard reload means that B will delete a feed's data prior to performing the reload, resulting in data like posts' read/unread statuses are lost. =back =head2 read noss read [post] Read the specified post's contents from the specified feed via a pager. If no specific post is provided, the latest unread post in the feed is selected. A negative post ID can be specified to look for a post relative to end of the feed instead of the beginning. B has the following options: =over 4 =item B<--pager>=I Use the specified command as the pager for reading the post's contents. Defaults to L on Unix systems and L on Windows systems. Can also be configured via the C environment variable and C configuration field. =item B<--no-mark> Do not mark post as read. =item B<--stdout> Instead of reading post via a pager, write the post's formatted contents to F. =item B<--read-format>=I Specify the format for the post's HTML contents that B will format when reading. I is a string that consists of formatting codes which will be substituted by B for whatever information the code represents. A formatting code starts with a percentage sign, followed by an optional integar specifying the pad width, and a character signifying what the code is actually formatting. If the pad width is negative, the text will be left-justified. See also the C configuration option. The format string can use the following formatting codes: =over 2 =item %% A percentage sign. =item %f The name of the feed the post is in. =item %i The ID of the post. =item %t The title of the post. =item %u The URL of the post. =item %a The name of the author of the post. =item %c The list of categories the post is tagged under. =item %s A single character signifying the post's read status (C for read, C for unread). =item %S The post's read status, C or C. =item %P The HTML contents of the post. =item %C The date of the post in the preferred date/time representation of the current locale. =item %d The post's day of the month as a decimal number (range 01 to 31). =item %w The post's abbreviated day of the week according to the current locale. =item %W The post's full day of the week according to the current locale. =item %m The post's abbreviated month name according to the current locale. =item %M The post's full month name according to the current locale. =item %n THe post's month as a decimal number (range 01 to 12). =item %y The post's year in two-digit form (range 00 to 99). =item %Y The post's full year. =item %z The post's time formatted according to the C<--time-format> option or C configuration field. =back =item B<--width>=I Specify the line width to use for the formatted text output. Defaults to C<80>. See also the C configuration field. =item B<--html> Instead of reading the formatted contents of a post, read its unformatted HTML contents. =back =head2 cat noss cat [post] Prints the specified post to F. This command is basically just an alias to C. If no post is specified, prints the latest unread post. This command has the same options as the C command. =head2 open noss open noss open Open the URL of the specified feed or post in a web browser. Like with the C and C commands, a negative post ID can be specified to look for a post relative to the end of a feed. C has the following options: =over 4 =item B<--browser>=I Use the specified command as the browser to use for opening the URL. Defaults to L. Can also be configured via the C environment variable and C configuration field. =item B<--no-mark> Do not mark post as read. =back =head2 list noss list [feed|group] ... List posts from specified feeds. When listing posts, B displays the post's read status, the name of the feed the post is a part of, the post's ID, and the title of the post. Each post is listed on a single line, which should make it easy to process in a pipeline. If no feeds or groups are specified, all feeds are searched. B has the following options: =over 4 =item B<--title>=I Only list posts containing the specified target string in their title. If the string starts and ends with a forward slash (C), B will interpret it as a Perl regex and search for posts whose titles match the given regex. =item B<--tag>=I Only list posts tagged with the specified tag. This option can be used multiple times to specify multiple different tags. =item B<--status>=I Only list posts marked with the specified status. Valid values are C and C. =item B<--content>=I Only list posts containing the specified target string in their HTML content. If the string starts and ends with a forward slash (C), B will interpret it as a Perl regex and search for posts that match the given regex. This option can be used multiple times to specify multiple different targets. =item B<--sort>=I Sort post list by the given method. The following are valid methods: =over 2 =item date Sort posts by date (default). =item feed Sort posts by feed. =item title Sort posts by title. =back This option can also be configured via the C configuration field. =item B<--reverse> Print the post list in reverse order. =item B<--list-limit>=I Only list the first I posts. When used with the C<--reverse> option, shows the last I posts. If I is less than or equal to C<0>, there is no limit. By default, there is no limit. This option can also be configured via the C configuration field. =item B<--hidden> Show hidden feeds when using C with no arguments. =item B<--list-format>=I Print post information using the specified format. This option works just like the C<--read-format> option in the C command. Consult its documentation for how formatting works and list of valid formatting codes. =back =head2 unread noss unread [feed|group] ... Similar to the list command, except only shows unread posts. This command has the same options as the list command. =head2 mark noss mark [posts] ... noss mark --all Mark the given posts in the specified feeds as C or C. I is a list of post IDs to mark as read. If a post argument is a range (C-I>), then each post in that range is marked. I can only be specified when marking posts in individual feeds, not when marking groups of feeds. This command has the following options: =over 4 =item B<--all> Mark all posts in every feed as read or unread. =back =head2 post noss post View the post information for the specified post. This command has the following options: =over 4 =item B<--post-format>=I Specify the format to use for printing post information. Consult the documentation for the read command's C<--read-format> option for information on how formatting works and a list of valid formatting codes. =back =head2 feeds noss feeds [feed|group] ... View feed information for specified feeds and/or groups. If no feeds are specified, prints information for all feeds. This command has the following options: =over 4 =item B<--brief> Only print feed names, not additional feed info. =item B<--feeds-format>=I Specify the format to use for printing feed information. Consult the documentation for the read command's C<--read-format> option for information on how format strings are processed. The following format codes are available: =over 2 =item %% A percentage sign. =item %f The feed's name. =item %l The feed's link. =item %t The feed's title =item %u The feed's homepage. =item %e The feed's description. =item %a The feed's author. =item %c The feed's tags. =item %p The number of posts in the feed. =item %r The number of read posts in the feed. =item %U The number of unread posts in the feed. =item %C =item %d =item %w =item %W =item %m =item %M =item %n =item %y =item %Y =item %z The same as their post formatting counterparts. =back =back =head2 groups noss groups [groups] ... View group information. If no groups are specified, prints information for all groups. This command has the following options: =over 4 =item B<--brief> Only print group names, not additional group info. =back =head2 clean noss clean Cleans up obsolete cache files and database data. This command has no unique options. =head2 export noss export [opml] Export feed list as an OPML file for transferring feed data to another feed reader. If no output file is specified, the OPML will be written directly to F. This command has the following options: =over 4 =item B<--no-groups> Do not try to generate feed group structures in the exported OPML. =item B<--export-special> By default, B does not export feeds that use special B-specific URLs like C or C feeds. This option disables that behavior. =back =head2 import noss import [json] Import feed list from an OPML file. Will write the feed list as a JSON file which can be used by B. Writes the feed file to I if specified, otherwise writes it directly to F. This command has the following options: =over 4 =item B<--no-groups> Do no try to import feed groups from the OPML. =back =head2 help noss help [command] Print documentation for the specified command. If no command is specified, the B manual will be printed instead. This command has no unique options. =head1 GLOBAL OPTIONS These options can be used with any command. =over 4 =item B<-c>|B<--config>=I Specify path to the B configuration file. =item B<-D>|B<--data>=I Specify path to the B data directory. The data directory is the directory where B stores data like its feed database and feed cache. =item B<-f>|B<--feeds>=I Specify path to the B feeds file. =item B<-A>|B<--autoclean>[=I<0>|I<1>] Toggle whether B should automatically run the C command after any operation. C<0> disables autoclean, C<1> enables it. Giving no argument is the equivalent to enabling autoclean. This command helps with automatically freeing up unused space at the cost of a slight performance penalty. By default, autoclean is disabled. This option can also be configured via the C configuration field. =item B<-z>|B<--time-format>=I Specify the time format to use for the C<%z> formatting code. L is used to perform the time formatting, so consult its manual for writing a format string. See also the C configuration field. C<%z> is the default formatting code used by the C and C commands, so this option will also affect their output. If this option is not set, C<%z> will default to the C<%c> L formatting code. =item B<-h>|B<--help> Print B's usage message and exit. =item B<-v>|B<--version> Print B's version and copyright info, then exit. =back =head1 CONFIGURATION B can have its behavior configured by writing it a configuration file. A B configuration file is a JSON file that supports the same extensions as the feeds file (comments and trailing commas). B will look for a configuration file in the following locations: =over 4 =item Path specified by the C<-c>|C<--config> option =item F> =item F/noss/noss.conf> =item F/noss.conf> =item F<~/.config/noss/noss.conf> =item F<~/.config/noss.conf> =item F<~/.noss.conf> =back A B configuration file can contain the following fields: =over 4 =item "B": "I" Path to feeds file. See also the C<-f>|C<--feeds> option. =item "B": "I" Path to use for the data directory. See also the C<-D>|C<--data> option. =item "B": I Number of parallel downloads to perform when fetching feeds for the C command. See also the C command's C<--downloads> option. =item "B": "I" The maximum transfer rate for downloads. I follows the same format as the speed given to the C<--limit-rate> option. =item "B": "I" The string to use as user-agent string for downloads. See also the C command's C<--user-agent> option. =item "B": I The maximum number of seconds a transfer is allowed to take. See also the C command's C<--timeout> option. =item "B": "[I]I[:I]" The proxy server to use for performing downloads. The proxy host string follows the same format as one used in the C command's C<--proxy> option. =item "B": "I:I" The username and password to use for proxy authentication. Follows the same format as the one used in the C command's C<--proxy-user> option. =item "B": "I" The command to use for reading posts via the C command. See also the C command's C<--pager> option. =item "B": "I" The command to use for opening URLs. =item "B": "I" How you would like the C command to sort posts. Valid methods are C, C, and C. See also the C<list> command's C<--sort> option. =item "B<list_limit>": I<limit> Limit the number of posts that are listed with the C<list> command. If I<limit> is less than or equal to C<0>, there is no limit. See also the C<list> command's C<--list-limit> option. =item "B<line_width>": I<width> The line width to use for the formatted text output of the C<read> command. See also the C<read> command's C<--width> option. =item "B<read_format>": "I<fmt>" The format to use for the HTML contents of a post that B<noss> will format for the C<read> command. See the documentation for the C<read> command's C<--read-format> option for more information on the details of how text formatting works. =item "B<list_format>": "I<fmt>" The format to use for posts in the C<list> command. See also the C<list> command's C<--list-format> option. =item "B<post_format>": "I<fmt>" The format to use for the post information printed in the C<post> command. See also the C<post> command's C<--post-format> option. =item "B<feeds_format>": "I<fmt>" The format to use for feed information printed in the C<feeds> command. See also the C<feeds> command's C<--feeds-format> option. =item "B<autoclean>": I<bool> Boolean determining whether B<noss> should automatically run the C<clean> command after performing any operation. See also the C<--autoclean> option. =item "B<time_format>": "I<fmt>" The L<strftime(3)> format string to use for formatting C<%z> times. See also the C<--time-format> option. =back =head1 NOSSUI For Unix-like systems, B<noss> also comes with a script called L<nossui(1)>, which is a L<dialog(1)>-based frontend that provides a terminal user interface for B<noss>. Not all of the functionality of B<noss> is available through L<nossui(1)>, so it shouldn't be used as a complete replacement for B<noss>, but it should be suitable for most of B<noss>'s routine usage. =head1 ENVIRONMENT =over 4 =item NOSS_DATA Directory for B<noss> to store program data in. =item NOSS_CONFIG Path to B<noss>'s configuration file. =item NOSS_FEEDS Path to B<noss>'s feeds file. =item XDG_DATA_HOME Directory for B<noss> to store its data directory in, if no data directory path is configured otherwise. =item XDG_CONFIG_HOME Directory where B<noss> looks for configuration in by default. =item PAGER Default pager to use for the C<read> command. =item BROWSER Default browser to use for opening URLs. =back =head1 CAVEATS Updating a feed too frequently can be wasteful on a server's resources, and in extreme cases could result in them banning you from accessing their feed. The acceptable frequency of feed updates varies from server to server, but a good rule of thumb is once per hour at most. You should also refrain from using the C<--unconditional> option without a good reason. =head1 AUTHOR Written by Samuel Young, E<lt>samyoung12788@gmail.comE<gt>. This project's source can be found on its L<Codeberg page|https://codeberg.org/1-1sam/noss.git>. Comments and pull requests are welcome! =head1 COPYRIGHT Copyright (C) 2025 Samuel Young This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. =head1 SEE ALSO L<nossui(1)>, L<curl(1)>, L<less(1)>, L<lynx(1)>, L<more(1)>, L<sqlite3(1)>, L<strftime(3)> =cut # vim: expandtab shiftwidth=4 �����������������������������������������������������noss-1.10/bin/nossui��������������������������������������������������������������������������������0000664�0000000�0000000�00000040527�15101417426�0014441�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh PRGNAM=nossui NOSS_CONFIG= NOSS_FEEDS= NOSS_DATA= export DIALOG_CANCEL=1 export DIALOG_ERROR=-1 export DIALOG_ESC=255 export DIALOG_EXTRA=3 export DIALOG_HELP=2 export DIALOG_ITEM_HELP=2 export DIALOG_TIMEOUT=5 export DIALOG_OK=0 export DIALOGOPTS="--backtitle $PRGNAM" SORT_OPT='--reverse --sort date' # Disable globbing set -f print_help() { cat <<HERE Usage: $PRGNAM [options] ... Options: -c <file> Specify path to configuration file -D <dir> Specify path to data directory -f <file> Specify path to feeds file -h Print this usage message -v Print nossui version/copyright info HERE } print_version() { noss -v | sed -e 's/noss/nossui/' } die() { printf "%s\n" "$1" 1>&2 exit 1 } wnoss() { if [ -n "$NOSS_CONFIG" ] then set -- '--config' "$NOSS_CONFIG" "$@" fi if [ -n "$NOSS_FEEDS" ] then set -- '--feeds' "$NOSS_FEEDS" "$@" fi if [ -n "$NOSS_DATA" ] then set -- '--data' "$NOSS_DATA" "$@" fi noss --autoclean 0 "$@" } dialog_read_post() { wnoss read $(printf "%s" "$1" | sed -e 's/:/ /') } dialog_open_post() { wnoss open $(printf "%s" "$1" | sed -e 's/:/ /') } dialog_update() { wnoss update "$@" 2>&1 | dialog \ --title 'Updating' \ --programbox 'Updating...' -1 -1 } dialog_reload() { wnoss reload "$@" 2>&1 | dialog \ --title 'Reloading' \ --programbox 'Reloading...' -1 -1 } dialog_mark_read() { wnoss mark read "$@" if [ "$?" = '0' ] then dialog --msgbox "$* has been marked as read." 0 0 else dialog --msgbox "Could not mark $* as read." 0 0 fi } dialog_mark_unread() { wnoss mark unread "$@" if [ "$?" = '0' ] then dialog --msgbox "$* has been marked as unread." 0 0 else dialog --msgbox "Could not mark $* as unread." 0 0 fi } dialog_info_post() { dialog \ --no-collapse \ --msgbox "$(wnoss post $(printf "%s" "$1" | sed -e 's/:/ /'))" 0 0 } dialog_select_post() { dialog_select_post_tmpsel="$(mktemp)" while true; do dialog \ --title "$1" \ --erase-on-exit \ --menu 'Please select an action for this post.' 0 0 0 \ 'Read' 'Read post in pager' \ 'Open' 'Open post in browser' \ 'Mark Read' 'Mark post as read' \ 'Mark Unread' 'Mark post as unread' \ 'Info' 'View more post information' \ 2> "$dialog_select_post_tmpsel" if [ "$?" != "$DIALOG_OK" ] then break fi case "$(cat "$dialog_select_post_tmpsel")" in 'Read') dialog_read_post "$1" ;; 'Open') dialog_open_post "$1" ;; 'Mark Read') dialog_mark_read $(printf "%s" "$1" | sed -e 's/:/ /') ;; 'Mark Unread') dialog_mark_unread $(printf "%s" "$1" | sed -e 's/:/ /') ;; 'Info') dialog_info_post "$1" ;; esac done rm -f "$dialog_select_post_tmpsel" } dialog_select_sort() { dialog_select_sort_tmpsel="$(mktemp)" dialog \ --menu "How would you like $PRGNAM to sort posts?" 0 0 0 \ 'Date' 'Sort posts by date (default)' \ 'Feed' 'Sort posts by their feed alphabetically' \ 'Title' 'Sort posts by their titles alphabetically' \ 2> "$dialog_select_sort_tmpsel" if [ "$?" != "$DIALOG_OK" ] then rm -f "$dialog_select_sort_tmpsel" return fi case "$(cat "$dialog_select_sort_tmpsel")" in 'Date') SORT_OPT='--reverse --sort date' ;; 'Feed') SORT_OPT='--sort feed' ;; 'Title') SORT_OPT='--sort title' ;; esac rm -f "$dialog_select_sort_tmpsel" } dialog_posts() { dialog_posts_tmpposts="$(mktemp)" dialog_posts_tmpsel="$(mktemp)" remember_post= while true; do wnoss list "$@" \ --list-format "$(printf "%%f:%%i\n(%%s) %%t")" \ --list-limit 0 \ $SORT_OPT | \ perl -plE '$_ = "\"" . s/\"/\\"/gr . "\""' \ > "$dialog_posts_tmpposts" if [ ! -s "$dialog_posts_tmpposts" ] then dialog --msgbox 'No posts were found' 0 0 rm -f "$dialog_posts_tmpposts" "$dialog_posts_tmpsel" return fi dialog \ --erase-on-exit \ --ok-label 'Read' \ --extra-button --extra-label 'Actions' \ --cancel-label 'Sort' \ --help-button --help-label 'Cancel' \ $remember_post \ --menu 'Please select a post to view' 0 0 0 \ --file "$dialog_posts_tmpposts" \ 2> "$dialog_posts_tmpsel" case "$?" in "$DIALOG_OK") remember_post="--default-item $(cat "$dialog_posts_tmpsel")" dialog_read_post "$(cat "$dialog_posts_tmpsel")" ;; "$DIALOG_EXTRA") remember_post="--default-item $(cat "$dialog_posts_tmpsel")" dialog_select_post "$(cat "$dialog_posts_tmpsel")" ;; "$DIALOG_CANCEL") remember_post= dialog_select_sort ;; *) break ;; esac done rm -f "$dialog_posts_tmpsel" "$dialog_posts_tmpposts" } dialog_search() { search_title= search_content= search_feedgroup= search_tags= search_status= search_hidden='0' dialog_search_tmpform="$(mktemp)" while true; do dialog \ --ok-label 'Search' \ --form 'Please enter your search parameters.' 0 0 0 \ 'Title' 1 0 "$search_title" 1 12 18 255 \ 'Content' 2 0 "$search_content" 2 12 18 255 \ 'Feed/Group' 3 0 "$search_feedgroup" 3 12 18 255 \ 'Tags' 4 0 "$search_tags" 4 12 18 255 \ 'Status' 5 0 "$search_status" 5 12 18 255 \ 'Hidden' 6 0 "$search_hidden" 6 12 18 1 \ 2> "$dialog_search_tmpform" if [ "$?" != "$DIALOG_OK" ] then break fi set -- while IFS= read -r line; do set -- "$@" "$line" done < "$dialog_search_tmpform" search_title="$1" search_content="$2" search_feedgroup="$3" search_tags="$4" search_status="$5" search_hidden="$6" set -- if [ -n "$search_title" ] then set -- "$@" '--title' "$search_title" fi if [ -n "$search_content" ] then set -- "$@" '--content' "$search_content" fi if [ -n "$search_tags" ] then for tag in $search_tags; do set -- "$@" '--tag' "$tag" done fi if [ "$search_status" = 'read' ] then set -- "$@" '--status' 'read' elif [ "$search_status" = 'unread' ] then set -- "$@" '--status' 'unread' elif [ -n "$search_status" ] then dialog --msgbox "'status' must either be 'read' or 'unread'." 0 0 continue fi if [ "$search_hidden" = '1' ] then set -- "$@" '--hidden' elif [ -n "$search_hidden" ] && [ "$search_hidden" != '0' ] then dialog --msgbox "'hidden' must either be '1' or '0'" 0 0 continue fi if [ -n "$search_feedgroup" ] then for feedgroup in $search_feedgroup; do set -- "$@" "$feedgroup" done fi dialog_posts "$@" done rm -f "$dialog_search_tmpform" } dialog_info_feed() { dialog \ --no-collapse \ --msgbox "$(wnoss feeds "$1")" 0 0 } dialog_open_feed() { dialog_open_feed_err="$(mktemp)" wnoss open "$1" 2> "$dialog_open_feed_err" if [ "$?" != '0' ] then dialog \ --title "Failed to open $1" \ --msgbox "$(cat "$dialog_open_feed_err")" 0 0 fi rm -f "$dialog_open_feed_err" } dialog_select_feed() { dialog_select_feed_tmpsel="$(mktemp)" while true; do dialog \ --title "$1" \ --ok-label 'Select' \ --menu 'Please select an action to perform.' 0 0 0 \ 'Posts' "View all posts in $1" \ 'Open' "Open $1's homepage in a browser" \ 'Mark Read' "Mark all posts in $1 as read" \ 'Mark Unread' "Mark all posts in $1 as unread" \ 'Update' "Update $1" \ 'Reload' "Reload $1" \ 'Info' "View $1 feed information" \ 2> "$dialog_select_feed_tmpsel" if [ "$?" != '0' ] then break fi case "$(cat "$dialog_select_feed_tmpsel")" in 'Posts') dialog_posts "$1" ;; 'Open') dialog_open_feed "$1" ;; 'Mark Read') dialog_mark_read "$1" ;; 'Mark Unread') dialog_mark_unread "$1" ;; 'Update') dialog --yesno "Would you like to update $1?" 0 0 if [ "$?" != '0' ] then continue fi dialog_update "$1" ;; 'Reload') dialog --yesno "Would you like to reload $1?" 0 0 if [ "$?" != '0' ] then continue fi dialog_reload "$1" ;; 'Info') dialog_info_feed "$1" ;; esac done rm -f "$dialog_select_feed_tmpsel" } dialog_feeds() { dialog_feeds_tmpsel="$(mktemp)" dialog_feeds_tmpfeeds="$(mktemp)" while true; do wnoss feeds "$@" \ --feeds-format "$(printf "%%f\n%%t")" | \ perl -plE '$_ = "\"" . s/\"/\\"/gr . "\""' \ > "$dialog_feeds_tmpfeeds" if [ ! -s "$dialog_feeds_tmpfeeds" ] then dialog --msgbox 'No feeds were found' 0 0 break fi dialog \ --ok-label 'Posts' \ --extra-button --extra-label 'Actions' \ --menu 'Please select a feed to view' 0 0 0 \ --file "$dialog_feeds_tmpfeeds" \ 2> "$dialog_feeds_tmpsel" case "$?" in "$DIALOG_OK") dialog_posts "$(cat "$dialog_feeds_tmpsel")" ;; "$DIALOG_EXTRA") dialog_select_feed "$(cat "$dialog_feeds_tmpsel")" ;; *) break ;; esac done rm -f "$dialog_feeds_tmpsel" "$dialog_feeds_tmpfeeds" } dialog_select_group() { dialog_select_group_tmpsel="$(mktemp)" while true; do dialog \ --ok-label 'Select' \ --menu 'Please select an action to perform for this group.' 0 0 0 \ 'Posts' 'View list of posts from this group' \ 'Feeds' 'View list of feeds in this group' \ 'Mark Read' 'Mark all posts in this group as read' \ 'Mark Unread' 'Mark all posts in this group as unread' \ 'Update' 'Update all feeds in this group' \ 'Reload' 'Reload all feeds in this group' \ 2> "$dialog_select_group_tmpsel" if [ "$?" != "$DIALOG_OK" ] then break fi case "$(cat "$dialog_select_group_tmpsel")" in 'Posts') dialog_posts "$1" ;; 'Feeds') dialog_feeds "$1" ;; 'Mark Read') dialog_mark_read "$1" ;; 'Mark Unread') dialog_mark_unread "$1" ;; 'Update') dialog --yesno "Would you like to update the feeds in $1?" 0 0 if [ "$?" != '0' ] then continue fi dialog_update "$1" ;; 'Reload') dialog --yesno "Would you like to reload the feeds in $1?" 0 0 if [ "$?" != '0' ] then continue fi dialog_reload "$1" ;; esac done rm -f "$dialog_select_group_tmpsel" } dialog_groups() { dialog_groups_tmpsel="$(mktemp)" dialog_groups_tmpgroups="$(mktemp)" while true; do wnoss groups --brief > "$dialog_groups_tmpgroups" if [ ! -s "$dialog_groups_tmpgroups" ] then dialog --msgbox 'No feed groups were found' 0 0 break fi dialog \ --ok-label 'Posts' \ --extra-button --extra-label 'Actions' \ --no-items \ --menu 'Please select a group to view.' 0 0 0 \ --file "$dialog_groups_tmpgroups" \ 2> "$dialog_groups_tmpsel" case "$?" in "$DIALOG_OK") dialog_posts "$(cat "$dialog_groups_tmpsel")" ;; "$DIALOG_EXTRA") dialog_select_group "$(cat "$dialog_groups_tmpsel")" ;; *) break ;; esac done rm -f "$dialog_groups_tmpsel" "$dialog_groups_tmpgroups" } dialog_clean() { dialog --infobox 'Cleaning up...' 0 0 wnoss clean dialog --msgbox 'Finished cleanup.' 0 0 } dialog_main() { dialog_main_tmpsel="$(mktemp)" while true; do dialog \ --erase-on-exit \ --menu 'Please select a menu item.' 0 0 0 \ 'Posts' 'View all posts' \ 'Unread' 'View unread posts' \ 'Search' 'Search for posts that match given parameters' \ 'Feeds' 'Browse your feeds' \ 'Groups' 'Browse your feed groups' \ 'Update' 'Fetch and load feed updates' \ 'Reload' 'Reload feed cache' \ 'Clean' 'Clean up feed cache and database' \ 2> "$dialog_main_tmpsel" if [ "$?" != "$DIALOG_OK" ] then rm -f "$dialog_main_tmpsel" exit 0 fi case "$(cat "$dialog_main_tmpsel")" in 'Posts') dialog_posts ;; 'Unread') dialog_posts --status 'unread' ;; 'Search') dialog_search ;; 'Feeds') dialog_feeds ;; 'Groups') dialog_groups ;; 'Update') dialog --yesno "Would you like to update all of your feeds?" 0 0 if [ "$?" != '0' ] then continue fi dialog_update ;; 'Reload') dialog --yesno "Would you like to reload all of your feeds?" 0 0 if [ "$?" != '0' ] then continue fi dialog_reload ;; 'Clean') dialog_clean ;; esac done rm -f "$dialog_main_tmpsel" } main() { if [ ! -x "$(command -v noss)" ] then die "noss is not installed or is not in PATH" fi if [ ! -x "$(command -v "dialog")" ] then die "dialog is not installed or is not in PATH" fi while getopts 'c:D:f:hv' opt do case "$opt" in 'c') NOSS_CONFIG="$OPTARG" ;; 'D') NOSS_DATA="$OPTARG" ;; 'f') NOSS_FEEDS="$OPTARG" ;; 'h') print_help exit 0 ;; 'v') print_version exit 0 ;; '?') print_help 1>&2 exit 1 ;; esac done if [ -n "$NOSS_CONFIG" ] && [ ! -f "$NOSS_CONFIG" ] then die "$NOSS_CONFIG does not exist" fi if [ -n "$NOSS_FEEDS" ] && [ ! -f "$NOSS_FEEDS" ] then die "$NOSS_FEEDS does not exist" fi dialog_main } main "$@" # vim: expandtab shiftwidth=4 �������������������������������������������������������������������������������������������������������������������������������������������������������������������������noss-1.10/contrib/����������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15101417426�0014056�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������noss-1.10/contrib/MAYBE_TODO.md���������������������������������������������������������������������0000664�0000000�0000000�00000005566�15101417426�0016076�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# TODOs(?) Some ideas for potential features that I have been reluctant to add. ## Command Aliases Allow the user to specify custom noss commands in their `noss.conf`. ```json "alias": { "cat": "noss read --stdout $1" } ``` How this feature would be implemented is that the command's alias would be treated as a shell command with `$@` set to the list of arguments given to noss. This would mean that `$1`, `$2`, `$3`, etc. would be set to their respective positional arguments and `$@` would be the entire argument list. What's keeping me from implementing this feature is that handling arguments that look options (start with `-` or `--`) would be difficult as `GetOptions` would slurp those arguments up and not allow them to be available to the shell command. ## Color code format sequences Allow the user to color the output of some commands via color codes. ```json "list_format": "<1>%s <2>%-15f <3>%3i <4>%t" ``` This feature would be implemented by modifying the `_fmt()` subroutine to accept a hash ref of color codes. Color codes that appear in the format string would be replaced by their corresponding ANSI escape sequence returned by `Term::ANSIColor`. What's keeping me from implementing this feature is that colorful output can already be achieved by embedding ANSI color sequences inside of the format strings themselves. ## "open" feed setting Setting for feeds/groups that allow the user to specify a custom command to use for opening that specific feed or group which would override the global open setting. ```json "WatameCh": { "feed": "http://www.youtube.com/feeds/videos.xml?channel_id=UCqm3BQLlJfvkTsX_hvm0UmA", "open": "yt-dlp", } ``` The idea for this feature came when I was experimenting with subscribing to YouTube channel feeds, and I thought it would be cool if I could use the `open` command to open YouTube posts via yt-dlp. The main issue keeping me from implementing this feature is how niche this feature would be. I feel like implementing <##Command Aliases> would be the better, more general solution. For example, instead of defining a specific open command for a YouTube feed, you could define a `yt-dlp` alias which opens a URL with yt-dlp. ## "list\_unread\_fmt" setting Specify a format string to use for specifically listing unread posts in the `list` command. The idea for this feature came when I was experimenting with embedding ANSI color codes in the "list\_format" setting to give the `list` command colorful output. I thought it would be cool if I could give unread posts a unique color to distinguish them from read posts, which could be achieved by giving them a separate format string. The issue keeping me from implementing this feature is how useful it would be outside my very specific use-case. I'd like to refrain from adding features that only exist to accomplish an extremely niche goal, as there is usually a better, more general solution available. ������������������������������������������������������������������������������������������������������������������������������������������noss-1.10/contrib/atom-reminders.pl�����������������������������������������������������������������0000664�0000000�0000000�00000003614�15101417426�0017345�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl use 5.016; use strict; use warnings; use List::Util qw(any); use POSIX qw(strftime); use constant { SUN => 0, MON => 1, TUE => 2, WED => 3, THU => 4, FRI => 5, SAT => 6, }; my $DAY = 60 * 60 * 24; my $ATOM_TIME_FMT = '%Y-%m-%dT%H:%M:%SZ'; my $HOSTNAME = qx`hostname -f` =~ s/\n$//r; my $WHOAMI = qx`whoami` =~ s/\n$//r; my $TITLE = "$WHOAMI\'s system reminders"; my $SUBTITLE = "Atom feed for various system reminders."; my $SELF_LINK = "file://$0"; my $LINK = $HOSTNAME; my $ID = "$HOSTNAME:reminders"; my $AUTHOR = $WHOAMI; my $now = time; my $now_fmt = strftime($ATOM_TIME_FMT, localtime $now); my @reminders = ( { title => sub { "... - $_[0]->{ date }" }, id => sub { "$ID/...:$_[0]->{ date }" }, summary => sub { "... - $_[0]->{ date }" }, days => [ ... ], date => undef, }, ); sub prev_day { my ($days) = @_; my $day = $now; until (any { $_ == (localtime($day))[6] } @$days) { $day -= $DAY; } return $day; } print <<"HERE"; <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> <title>$TITLE $SUBTITLE $ID $now_fmt $AUTHOR HERE for my $r (@reminders) { my $prev = prev_day($r->{ days }); $r->{ date } = strftime($ATOM_TIME_FMT, 0, 0, 0, (localtime($prev))[3 .. 8]); $r->{ title } = $r->{ title }($r); $r->{ id } = $r->{ id }($r); $r->{ summary } = $r->{ summary }($r); print <<"HERE"; $r->{ title } $r->{ id } $r->{ date } $r->{ summary } HERE } print <<"HERE"; HERE 1; # vim: expandtab shiftwidth=4 noss-1.10/contrib/html2rss.pl000066400000000000000000001331551510141742600162010ustar00rootroot00000000000000#!/usr/bin/perl use 5.016; use strict; use warnings; use Digest; use Encode qw(encode_utf8); use File::Temp qw(tempfile); use Getopt::Long; use List::Util qw(max min uniq); use POSIX qw(strftime); use Time::Piece; use Time::Seconds; use XML::LibXML; use constant { SELECTOR_STRING => 0, SELECTOR_SINGLE_TEXT => 1, SELECTOR_MULTI_TEXT => 2, SELECTOR_SINGLE_HTML => 3, SELECTOR_MULTI_HTML => 4, }; my $USAGE = <<"HERE"; Usage: html2rss.pl [options] config [location] Options: -a Generate atom feed -r Generate RSS 2.0 feed -o Output feed to -h Print this usage message HERE my $NOW = localtime; # TODO: If title is SELECTOR_STRING, make it unique somehow? # TODO: Allow combining selector statements? like: # SlackBuilds.org: [.//div[@class="whatever"]] my $DEFAULT_AGENT = "html2rss.pl ($^O; perl $^V)"; my %DAY_MAP = ( 0 => 'Sunday', 1 => 'Monday', 2 => 'Tuesday', 3 => 'Wednesday', 4 => 'Thursday', 5 => 'Friday', 6 => 'Saturday', ); my %DAY_STRINGS = ( '0' => 0, '7' => 0, 'sun' => 0, 'sunday' => 0, '1' => 1, 'mon' => 1, 'monday' => 1, '2' => 2, 'tue' => 2, 'tuesday' => 2, '3' => 3, 'wed' => 3, 'wednesday' => 3, '4' => 4, 'thu' => 4, 'thursday' => 4, '5' => 5, 'fri' => 5, 'friday' => 5, '6' => 6, 'sat' => 6, 'saturday' => 6, ); my %FEED_FORMATS = map { $_ => 1 } qw( atom rss ); sub read_config { my ($file) = @_; my $config = {}; open my $fh, '<', $file or die "Failed to open $file for reading: $!\n"; my $section = undef; my $ln = 0; while (my $l = readline $fh) { $ln++; chomp $l; if ($l =~ /^#/ or $l =~ /^\s*$/) { next; } if ($l =~ /^\[(\w+)\]$/) { $section = $1; $config->{ $section } //= {}; next; } if (not defined $section) { die "$file $ln: key-value pair not under section\n"; } $l =~ s/^\s+|\s+$//g; my ($key, $value) = split /\s*=\s*/, $l, 2; if ($key !~ /^\w+$/) { die "$file $ln: '$key' key contains invalid characters\n"; } $config->{ $section }{ $key } = $value; } close $fh; return $config; } sub curl_to { my ($url, $to, %param) = @_; my $agent = $param{ user_agent }; my @args; if (defined $agent) { push @args, '-A', $agent; } push @args, '-s', '--show-error'; push @args, '-o', $to; push @args, $url; system 'curl', @args; if ($? == -1 or $? >> 8 != 0) { die "Failed to curl '$url' to $to\n"; } return 1; } sub id_selector { my ($str) = @_; if ($str =~ /^\[\[(.*)\]\]$/) { return ($1, SELECTOR_MULTI_TEXT); } elsif ($str =~ /^<<(.*)>>$/) { return ($1, SELECTOR_MULTI_HTML); } elsif ($str =~ /^\[(.*)\]$/) { return ($1, SELECTOR_SINGLE_TEXT); } elsif ($str =~ /^<(.*)>$/) { return ($1, SELECTOR_SINGLE_HTML); } else { return ($str, SELECTOR_STRING); } } sub selector_str_eq { my ($selector, $string) = @_; return 0 if $selector->[1] != SELECTOR_STRING; return $selector->[0] eq $string; } sub truncate_week { my ($time) = @_; $time = $time->truncate(to => 'day'); while ($time->wday != 1) { $time -= ONE_DAY; } return $time; } sub format_atom_time { my ($time) = @_; return strftime('%Y-%m-%dT%H:%M:%SZ', gmtime $time); } sub format_rss_time { my ($time) = @_; return strftime('%a, %d %b %Y %H:%M:%S -0000', gmtime $time); } sub generate_item_id { my ($item, $salt) = @_; my $sha = Digest->new('SHA-256'); if (defined $salt) { $sha->add(encode_utf8($salt)); } if (not defined $item->{ title }) { die '$item missing title'; } if (not defined $item->{ link }) { die '$item missing link'; } $sha->add(encode_utf8($item->{ title })); $sha->add(encode_utf8($item->{ link })); return $sha->hexdigest; } sub generate_feed_id { my ($feed, $salt) = @_; my $sha = Digest->new('SHA-256'); if (defined $salt) { $sha->add(encode_utf8($salt)); } if (not defined $feed->{ title }) { die '$feed missing title'; } if (not defined $feed->{ link }) { die '$feed missing link'; } $sha->add(encode_utf8($feed->{ title })); $sha->add(encode_utf8($feed->{ link })); return $sha->hexdigest; } sub generate_atom_item { my ($item) = @_; if (ref $item ne 'HASH') { die '$item is not a hash ref'; } my $node = XML::LibXML::Element->new('entry'); if (not defined $item->{ title }) { die "item missing title\n"; } my $title = $node->appendChild( XML::LibXML::Element->new('title') ); $title->appendChild( XML::LibXML::Text->new($item->{ title }) ); if (not defined $item->{ link }) { die "item missing link\n"; } my $link = $node->appendChild( XML::LibXML::Element->new('link') ); $link->setAttribute('href', $item->{ link }); my $id = $node->appendChild( XML::LibXML::Element->new('id') ); $id->appendChild( XML::LibXML::Text->new(generate_item_id($item)) ); if (defined $item->{ summary }) { my $summary = $node->appendChild( XML::LibXML::Element->new('summary') ); $summary->setAttribute('type', $item->{ summary_html } ? 'html' : 'text'); $summary->appendChild( XML::LibXML::Text->new($item->{ summary }) ); } else { my $summary = $node->appendChild( XML::LibXML::Element->new('summary') ); $summary->setAttribute('type', 'text'); $summary->appendChild( XML::LibXML::Text->new('') ); } if (defined $item->{ author }) { my $author = $node->appendChild( XML::LibXML::Element->new('author') ); my $name = $author->appendChild( XML::LibXML::Element->new('name') ); $name->appendChild( XML::LibXML::Text->new($item->{ author }) ); } # TODO: Should this be mandatory? if (defined $item->{ updated } or defined $item->{ published }) { my $updated = $node->appendChild( XML::LibXML::Element->new('updated') ); $updated->appendChild( XML::LibXML::Text->new( format_atom_time($item->{ updated } // $item->{ published }) ) ); } if (defined $item->{ categories }) { for my $c (@{ $item->{ categories } }) { my $cat = $node->appendChild( XML::LibXML::Element->new('category') ); $cat->setAttribute('term', $c); } } if (defined $item->{ rights }) { my $rights = $node->appendChild( XML::LibXML::Element->new('rights') ); $rights->appendChild( XML::LibXML::Text->new($item->{ rights }) ); } return $node; } sub generate_rss_item { my ($item) = @_; my $node = XML::LibXML::Element->new('item'); if (not defined $item->{ title }) { die "item missing title\n"; } if (not defined $item->{ link }) { die "item missing link\n"; } my $title = $node->appendChild( XML::LibXML::Element->new('title') ); $title->appendChild( XML::LibXML::Text->new($item->{ title }) ); my $link = $node->appendChild( XML::LibXML::Element->new('link') ); $link->appendChild( XML::LibXML::Text->new($item->{ link }) ); my $id = $node->appendChild( XML::LibXML::Element->new('guid') ); $id->appendChild( XML::LibXML::Text->new(generate_item_id($item)) ); if (defined $item->{ published } or defined $item->{ updated }) { my $pub = $node->appendChild( XML::LibXML::Element->new('pubDate') ); $pub->appendChild( XML::LibXML::Text->new( format_rss_time($item->{ published } // $item->{ updated }) ) ); } if (defined $item->{ author }) { my $author = $node->appendChild( XML::LibXML::Element->new('author') ); $author->appendChild( XML::LibXML::Text->new($item->{ author }) ); } if (defined $item->{ summary }) { my $desc = $node->appendChild( XML::LibXML::Element->new('description') ); $desc->appendChild( XML::LibXML::Text->new($item->{ summary }) ); } if (defined $item->{ categories }) { for my $c (@{ $item->{ categories } }) { my $cat = $node->appendChild( XML::LibXML::Element->new('category') ); $cat->appendChild( XML::LibXML::Text->new($c) ); } } return $node; } sub generate_atom_feed { my ($feed) = @_; my $doc = XML::LibXML::Document->new(); my $node = XML::LibXML::Element->new('feed'); $node->setAttribute('xmlns', 'http://www.w3.org/2005/Atom'); $doc->setDocumentElement($node); if (not defined $feed->{ title }) { die "feed is missing title\n"; } if (not defined $feed->{ link }) { die "feed is missing link\n"; } my $title = $node->appendChild( XML::LibXML::Element->new('title') ); $title->appendChild( XML::LibXML::Text->new($feed->{ title }) ); my $link = $node->appendChild( XML::LibXML::Element->new('link') ); $link->setAttribute('href', $feed->{ link }); my $id = $node->appendChild( XML::LibXML::Element->new('id') ); $id->appendChild( XML::LibXML::Text->new(generate_feed_id($feed)) ); # TODO: Updated/published should be mandatory my $updated = $node->appendChild( XML::LibXML::Element->new('updated') ); $updated->appendChild( XML::LibXML::Text->new( format_atom_time($feed->{ updated } // $feed->{ published }) ) ); if (defined $feed->{ description }) { my $desc = $node->appendChild( XML::LibXML::Element->new('subtitle') ); $desc->appendChild( XML::LibXML::Text->new($feed->{ description }) ); } if (defined $feed->{ rights }) { my $rights = $node->appendChild( XML::LibXML::Element->new('rights') ); $rights->appendChild( XML::LibXML::Text->new($feed->{ rights }) ); } if (defined $feed->{ categories }) { for my $c (@{ $feed->{ categories } }) { my $cat = $node->appendChild( XML::LibXML::Element->new('category') ); $cat->setAttribute('term', $c); } } if (defined $feed->{ generator }) { my $gen = $node->appendChild( XML::LibXML::Element->new('generator') ); $gen->appendChild( XML::LibXML::Text->new($feed->{ generator }) ); } if (defined $feed->{ items }) { for my $i (@{ $feed->{ items } }) { my $item = generate_atom_item($i); $node->appendChild($item); } } return $doc; } sub generate_rss_feed { my ($feed) = @_; my $doc = XML::LibXML::Document->new; my $node = XML::LibXML::Element->new('rss'); $node->setAttribute('version', '2.0'); $doc->setDocumentElement($node); my $channel = $node->appendChild( XML::LibXML::Element->new('channel') ); if (not defined $feed->{ title }) { die "feed is missing title\n"; } if (not defined $feed->{ link }) { die "feed is missing link\n"; } my $title = $channel->appendChild( XML::LibXML::Element->new('title') ); $title->appendChild( XML::LibXML::Text->new($feed->{ title }) ); my $link = $channel->appendChild( XML::LibXML::Element->new('link') ); $link->appendChild( XML::LibXML::Text->new($feed->{ link }) ); my $desc = $channel->appendChild( XML::LibXML::Element->new('description') ); $desc->appendChild( XML::LibXML::Text->new($feed->{ description } // '') ); if (defined $feed->{ language }) { my $lang = $channel->appendChild( XML::LibXML::Element->new('language') ); $lang->appendChild( XML::LibXML::Text->new($feed->{ language }) ); } if (defined $feed->{ rights }) { my $rights = $channel->appendChild( XML::LibXML::Element->new('copyright') ); $rights->appendChild( XML::LibXML::Text->new($feed->{ rights }) ); } if (defined $feed->{ published }) { my $pub = $channel->appendChild( XML::LibXML::Element->new('pubDate') ); $pub->appendChild( XML::LibXML::Text->new(format_rss_time($feed->{ published })) ); } if (defined $feed->{ updated }) { my $updated = $channel->appendChild( XML::LibXML::Element->new('lastBuildDate') ); $updated->appendChild( XML::LibXML::Text->new(format_rss_time($feed->{ updated })) ); } if (defined $feed->{ categories }) { for my $c (@{ $feed->{ categories } }) { my $cat = $channel->appendChild( XML::LibXML::Element->new('category') ); $cat->appendChild( XML::LibXML::Text->new($c) ); } } if (defined $feed->{ generator }) { my $gen = $channel->appendChild( XML::LibXML::Element->new('generator') ); $gen->appendChild( XML::LibXML::Text->new($feed->{ generator }) ); } if (defined $feed->{ ttl }) { if ($feed->{ ttl } !~ /^\d+$/) { die "channel ttl is not an integar\n"; } my $ttl = $channel->appendChild( XML::LibXML::Element->new('ttl') ); $ttl->appendChild( XML::LibXML::Text->new($feed->{ ttl }) ); } if (defined $feed->{ skip_hours }) { my $skip = $channel->appendChild( XML::LibXML::Element->new('skipHours') ); for my $s (uniq sort { $a <=> $b } @{ $feed->{ skip_hours } }) { if ($s < 0 or $s > 23) { die "skip_hours contains invalid value ($s)\n"; } if ($s !~ /^\d+$/) { die "skip_hours contains invalid value ($s)\n"; } my $hour = $skip->appendChild( XML::LibXML::Element->new('hour') ); $hour->appendChild( XML::LibXML::Text->new($s) ); } } if (defined $feed->{ skip_days }) { my $skip = $channel->appendChild( XML::LibXML::Element->new('skipDays') ); for my $s (uniq sort { $a <=> $b } @{ $feed->{ skip_days } }) { if (not exists $DAY_MAP{ $s }) { die "skip_days contains invalid value ($s)\n"; } my $hour = $skip->appendChild( XML::LibXML::Element->new('day') ); $hour->appendChild( XML::LibXML::Text->new($DAY_MAP{ $s }) ); } } if (defined $feed->{ items }) { for my $i (@{ $feed->{ items } }) { my $item = generate_rss_item($i); $channel->appendChild($item); } } return $doc; } sub process_config { my ($file) = @_; my $config = { Feed => { Title => undef, Link => undef, Description => undef, Language => undef, Rights => undef, Published => undef, Updated => undef, Categories => undef, Generator => undef, TTL => undef, SkipHours => undef, SkipDays => undef, Format => undef, PublishedFmt => undef, UpdatedFmt => undef, }, Item => { Select => undef, Title => undef, Link => undef, Published => undef, Updated => undef, Author => undef, Content => undef, Categories => undef, Rights => undef, PublishedFmt => undef, UpdatedFmt => undef, }, }; my $hash = read_config($file); if (not exists $hash->{ Feed }) { die "$file missing [Feed] section\n"; } if (not exists $hash->{ Item }) { die "$file missing [Item] section\n"; } # TODO: Are there sane defaults we can default to for these? if (not exists $hash->{ Feed }{ Title }) { die "$file missing [Feed].Title configuration\n"; } if (not exists $hash->{ Feed }{ Link }) { die "$file missing [Feed].Link configuration\n"; } if (not exists $hash->{ Item }{ Select }) { die "$file missing [Item].Select configuration\n"; } if (not exists $hash->{ Item }{ Title }) { die "$file missing [Item].Title configuration\n"; } if (not exists $hash->{ Item }{ Link }) { die "$file missing [Item].Link configuration\n"; } $config->{ Feed }{ Title } = [ id_selector($hash->{ Feed }{ Title }) ]; $config->{ Feed }{ Link } = (id_selector($hash->{ Feed }{ Link }))[0]; $config->{ Item }{ Select } = (id_selector($hash->{ Item }{ Select }))[0]; $config->{ Item }{ Title } = [ id_selector($hash->{ Item }{ Title }) ]; $config->{ Item }{ Link } = [ id_selector($hash->{ Item }{ Link }) ]; if (exists $hash->{ Feed }{ Description }) { $config->{ Feed }{ Description } = [ id_selector($hash->{ Feed }{ Description }) ]; } if (exists $hash->{ Feed }{ Language }) { $config->{ Feed }{ Language } = [ id_selector($hash->{ Feed }{ Language }) ]; } if (exists $hash->{ Feed }{ Rights }) { $config->{ Feed }{ Rights } = [ id_selector($hash->{ Feed }{ Rights }) ]; } if (exists $hash->{ Feed }{ Published }) { $config->{ Feed }{ Published } = [ id_selector($hash->{ Feed }{ Published }) ]; } if (exists $hash->{ Feed }{ Updated }) { $config->{ Feed }{ Updated } = [ id_selector($hash->{ Feed }{ Updated }) ]; } if (exists $hash->{ Feed }{ Categories }) { $config->{ Feed }{ Categories } = [ id_selector($hash->{ Feed }{ Categories }) ]; } if (exists $hash->{ Feed }{ Generator }) { $config->{ Feed }{ Generator } = [ id_selector($hash->{ Feed }{ Generator }) ]; } if (exists $hash->{ Feed }{ TTL }) { if ($hash->{ Feed }{ TTL } !~ /^\d+$/) { die "TTL must be an integar\n"; } $config->{ Feed }{ TTL } = $hash->{ Feed }{ TTL }; } if (exists $hash->{ Feed }{ SkipHours }) { my @vals = split /\s*,\s*/, $hash->{ Feed }{ SkipHours }; for my $v (@vals) { if ($v !~ /^\d+$/ or ($v < 0 or $v > 23)) { die "SkipHours can only contain integars between 0 and 23\n"; } push @{ $config->{ Feed }{ SkipHours } }, $v; } } if (exists $hash->{ Feed }{ SkipDays }) { my @vals = split /\s*,\s*/, $hash->{ Feed }{ SkipDays }; for my $v (@vals) { my $vv = lc $v; if (not exists $DAY_STRINGS{ $vv }) { die "SkipHours contains invalid value ($v)\n"; } push @{ $config->{ Feed }{ SkipDays } }, $DAY_STRINGS{ $vv }; } } if (exists $hash->{ Feed }{ Format }) { $config->{ Feed }{ Format } = lc $hash->{ Feed }{ Format }; if (not $FEED_FORMATS{ $config->{ Feed }{ Format } }) { die "'$hash->{ Feed }{ Format }' is not a valid feed format\n"; } } else { $config->{ Feed }{ Format } = 'atom'; } if (exists $hash->{ Feed }{ PublishedFmt }) { $config->{ Feed }{ PublishedFmt } = $hash->{ Feed }{ PublishedFmt }; } if (exists $hash->{ Feed }{ UpdatedFmt }) { $config->{ Feed }{ UpdatedFmt } = $hash->{ Feed }{ UpdatedFmt }; } if (exists $hash->{ Item }{ Published }) { $config->{ Item }{ Published } = [ id_selector($hash->{ Item }{ Published }) ]; } if (exists $hash->{ Item }{ Updated }) { $config->{ Item }{ Updated } = [ id_selector($hash->{ Item }{ Updated }) ]; } if (exists $hash->{ Item }{ Author }) { $config->{ Item }{ Author } = [ id_selector($hash->{ Item }{ Author }) ]; } if (exists $hash->{ Item }{ Content }) { $config->{ Item }{ Content } = [ id_selector($hash->{ Item }{ Content }) ]; } if (exists $hash->{ Item }{ Categories }) { $config->{ Item }{ Categories } = [ id_selector($hash->{ Item }{ Categories }) ]; } if (exists $hash->{ Item }{ Rights }) { $config->{ Item }{ Rights } = [ id_selector($hash->{ Item }{ Rights }) ]; } if (exists $hash->{ Item }{ PublishedFmt }) { $config->{ Item }{ PublishedFmt } = $hash->{ Item }{ PublishedFmt }; } if (exists $hash->{ Item }{ UpdatedFmt }) { $config->{ Item }{ UpdatedFmt } = $hash->{ Item }{ UpdatedFmt }; } if (defined $config->{ Feed }{ Published }) { if ( $config->{ Feed }{ Published }[0] eq 'updated' and $config->{ Feed }{ Published }[1] == SELECTOR_STRING ) { if (not defined $config->{ Feed }{ Updated }) { die "published set to 'updated', but 'updated' not defined\n"; } if ( $config->{ Feed }{ Updated }[0] eq 'published' and $config->{ Feed }{ Published }[1] == SELECTOR_STRING ) { die "published cannot be set to 'updated' if updated is set to 'published'\n"; } } if ( $config->{ Feed }{ Published }[0] eq 'published' and $config->{ Feed }{ Published }[1] == SELECTOR_STRING ) { die "published cannot be set to 'published'\n"; } } if (defined $config->{ Feed }{ Updated }) { if ( $config->{ Feed }{ Updated }[0] eq 'updated' and $config->{ Feed }{ Updated }[1] == SELECTOR_STRING ) { die "updated cannot be set to 'updated'\n"; } } return $config; } sub xpath_select { my ($node, $selector) = @_; if ($selector->[1] == SELECTOR_STRING) { return $selector->[0]; } elsif ($selector->[1] == SELECTOR_SINGLE_TEXT) { my ($n) = $node->findnodes($selector->[0]); if (not defined $n) { die "Nothing matches '$selector->[0]'\n"; } if ($n->isa('XML::LibXML::Attr')) { return $n->value // ''; } return $n->textContent; } elsif ($selector->[1] == SELECTOR_MULTI_TEXT) { my @found = $node->findnodes($selector->[0]); if (!@found) { die "Nothing matches '$selector->[0]'\n"; } my $text = ''; for my $n (@found) { if ($n->isa('XML::LibXML::Attr')) { $text .= $n->value // ''; } else { $text .= $n->textContent; } } return $text; } elsif ($selector->[1] == SELECTOR_SINGLE_HTML) { my ($n) = $node->findnodes($selector->[0]); if (not defined $n) { die "Nothing matches '$selector->[0]'\n"; } if ($n->isa('XML::LibXML::Attr')) { die "Cannot select the HTML of an attribute node\n"; } return $n->toString; } elsif ($selector->[1] == SELECTOR_MULTI_HTML) { my @found = $node->findnodes($selector->[1]); if (!@found) { die "Nothing matches '$selector->[0]'\n"; } my $text = ''; for my $n (@found) { if ($n->isa('XML::LibXML::Attr')) { die "Cannot select the HTML of an attribute node\n"; } $text .= $n->toString; } return $text; } die "Invalid selector type"; } sub xpath_multi_select { my ($dom, $selector) = @_; if ($selector->[1] == SELECTOR_STRING) { return ($selector->[0]); } elsif ($selector->[1] == SELECTOR_SINGLE_TEXT or $selector->[1] == SELECTOR_MULTI_TEXT) { my @found = $dom->findnodes($selector->[0]); return map { $_->isa('XML::LibXML::Attr') ? $_->value // '' : $_->textContent } @found; } elsif ($selector->[1] == SELECTOR_SINGLE_HTML or $selector->[1] == SELECTOR_MULTI_HTML) { my @found = $dom->findnodes($selector->[1]); @found = grep { !$_->isa('XML::LibXML::Attr') } @found; return map { $_->toString } @found; } die "Invalid selector type"; } sub html2feed { my ($html, $config) = @_; my $feed = { title => undef, link => undef, updated => undef, published => undef, description => undef, language => undef, rights => undef, categories => undef, generator => undef, ttl => undef, skip_hours => undef, skip_days => undef, items => [], }; my $dom = XML::LibXML->load_html( location => $html, recover => 2, ); $feed->{ title } = xpath_select($dom, $config->{ Feed }{ Title }); $feed->{ link } = $config->{ Feed }{ Link }; if (defined $config->{ Feed }{ Published }) { if ($config->{ Feed }{ Published }[1] == SELECTOR_STRING) { my $p = $config->{ Feed }{ Published }[0]; if ($p eq 'now') { $feed->{ published } = $NOW->epoch; } elsif ($p eq 'today') { $feed->{ published } = $NOW->truncate(to => 'day')->epoch; } elsif ($p eq 'week') { $feed->{ published } = truncate_week($NOW)->epoch; } elsif ($p eq 'month') { $feed->{ published } = $NOW->truncate(to => 'month')->epoch; } elsif ($p eq 'year') { $feed->{ published } = $NOW->truncate(to => 'year')->epoch; } elsif ($p eq 'updated') { if (not defined $config->{ Feed }{ Updated }) { die "Cannot set published to 'updated': [Feed].Updated not configured\n"; } # Set this later... } elsif ($p eq 'published') { die "Cannot set published to 'published'\n"; } elsif (defined $config->{ Feed }{ PublishedFmt }) { my $t = Time::Piece->strptime($p, $config->{ Feed }{ PublishedFmt }); $feed->{ published } = $t->epoch; } else { die "Cannot set [Feed].Published to a string without setting PublishedFmt\n"; } } else { if (not defined $config->{ Feed }{ PublishedFmt }) { die "Cannot determine publish time; [Feed].PublishedFmt not configured\n"; } my $sel = xpath_select($dom, $config->{ Feed }{ Published }); my $t = Time::Piece->strptime($sel, $config->{ Feed }{ PublishedFmt }); $feed->{ published } = $t->epoch; } } if (defined $config->{ Feed }{ Updated }) { if ($config->{ Feed }{ Updated }[1] == SELECTOR_STRING) { my $p = $config->{ Feed }{ Updated }[0]; if ($p eq 'now') { $feed->{ updated } = $NOW->epoch; } elsif ($p eq 'today') { $feed->{ updated } = $NOW->truncate(to => 'day')->epoch; } elsif ($p eq 'week') { $feed->{ updated } = truncate_week($NOW)->epoch; } elsif ($p eq 'month') { $feed->{ updated } = $NOW->truncate(to => 'month')->epoch; } elsif ($p eq 'year') { $feed->{ updated } = $NOW->truncate(to => 'year')->epoch; } elsif ($p eq 'updated') { die "Cannot set updated to 'updated'\n"; } elsif ($p eq 'published') { if (not defined $feed->{ published }) { die "Cannot set updated to 'published': published is not set\n"; } $feed->{ updated } = $feed->{ published }; } elsif (defined $config->{ Feed }{ UpdatedFmt }) { my $t = Time::Piece->strptime($p, $config->{ Feed }{ UpdatedFmt }); $feed->{ updated } = $t->epoch; } else { die "Cannot set [Feed].Updated to a string without setting UpdatedFmt\n"; } } else { if (not defined $config->{ Feed }{ UpdatedFmt }) { die "Cannot determine publish time; [Feed].UpdatedFmt not configured\n"; } my $sel = xpath_select($dom, $config->{ Feed }{ Updated }); my $t = Time::Piece->strptime($sel, $config->{ Feed }{ UpdatedFmt }); $feed->{ updated } = $t->epoch; } } if ( defined $config->{ Feed }{ Published } and $config->{ Feed }{ Published }[1] == SELECTOR_STRING and $config->{ Feed }{ Published }[0] eq 'updated' ) { $feed->{ published } = $feed->{ updated }; } if (defined $config->{ Feed }{ Description }) { $feed->{ description } = xpath_select($dom, $config->{ Feed }{ Description }); } if (defined $config->{ Feed }{ Language }) { $feed->{ language } = xpath_select($dom, $config->{ Feed }{ Language }); } if (defined $config->{ Feed }{ Rights }) { $feed->{ Rights } = xpath_select($dom, $config->{ Feed }{ Rights }); } if (defined $config->{ Feed }{ Categories }) { if ($config->{ Feed }{ Categories }[1] == SELECTOR_STRING) { my @cats = split /\s*,\s*/, $config->{ Feed }{ Categories }[0]; $feed->{ categories } = \@cats; } else { $feed->{ categories } = [ xpath_multi_select($dom, $config->{ Feed }{ Categories }) ]; } } if (defined $config->{ Feed }{ Generator }) { $feed->{ generator } = xpath_select($dom, $config->{ Feed }{ Generator }); } if (defined $config->{ Feed }{ TTL }) { $feed->{ ttl } = $config->{ Feed }{ TTL }; } if (defined $config->{ Feed }{ SkipHours }) { $feed->{ skip_hours } = $config->{ Feed }{ SkipHours }; } if (defined $config->{ Feed }{ SkipDays }) { $feed->{ skip_days } = $config->{ Feed }{ SkipDays }; } my @found = $dom->findnodes($config->{ Item }{ Select }); for my $n (@found) { my $item = { title => undef, link => undef, published => undef, updated => undef, author => undef, summary => undef, summary_html => 0, categories => undef, rights => undef, }; $item->{ title } = xpath_select($n, $config->{ Item }{ Title }); $item->{ link } = xpath_select($n, $config->{ Item }{ Link }); if (defined $config->{ Item }{ Published }) { if ($config->{ Item }{ Published }[1] == SELECTOR_STRING) { my $p = $config->{ Item }{ Published }[0]; if ($p eq 'now') { $item->{ published } = $NOW->epoch; } elsif ($p eq 'today') { $item->{ published } = $NOW->truncate(to => 'day')->epoch; } elsif ($p eq 'week') { $item->{ published } = truncate_week($NOW)->epoch; } elsif ($p eq 'month') { $item->{ published } = $NOW->truncate(to => 'month')->epoch; } elsif ($p eq 'year') { $item->{ published } = $NOW->truncate(to => 'year')->epoch; } elsif ($p eq 'updated') { if (not defined $config->{ Item }{ Updated }) { die "Cannot set published to 'updated': [Item].Updated not set\n"; } # Set this later... } elsif ($p eq 'published') { die "Cannot set published to 'published'\n"; } elsif (defined $config->{ Item }{ PublishedFmt }) { my $t = Time::Piece->strptime($p, $config->{ Item }{ PublishedFmt }); $item->{ published } = $t->epoch; } else { die "Cannot set [Item].Published to string without setting PublishedFmt\n"; } } else { if (not defined $config->{ Item }{ PublishedFmt }) { die "Cannot determine publish time; [Item].PublishedFmt not configured\n"; } my $sel = xpath_select($dom, $config->{ Item }{ Published }); my $t = Time::Piece->strptime($sel, $config->{ Item }{ PublishedFmt }); $item->{ published } = $t->epoch; } } if (defined $config->{ Item }{ Updated }) { if ($config->{ Item }{ Updated }[1] == SELECTOR_STRING) { my $p = $config->{ Item }{ Updated }[0]; if ($p eq 'now') { $item->{ updated } = $NOW->epoch; } elsif ($p eq 'today') { $item->{ updated } = $NOW->truncate(to => 'day')->epoch; } elsif ($p eq 'week') { $item->{ updated } = truncate_week($NOW)->epoch; } elsif ($p eq 'month') { $item->{ updated } = $NOW->truncate(to => 'month')->epoch; } elsif ($p eq 'year') { $item->{ updated } = $NOW->truncate(to => 'year')->epoch; } elsif ($p eq 'updated') { die "Cannot set updated to 'updated'\n"; } elsif ($p eq 'published') { if (not defined $item->{ published }) { die "Cannot set updated to 'published': [Item].published is not set\n"; } $item->{ updated } = $item->{ published }; } elsif (defined $config->{ Item }{ UpdatedFmt }) { my $t = Time::Piece->strptime($p, $config->{ Item }{ UpdatedFmt }); $item->{ updated } = $t->epoch; } else { die "Cannot set [Item].Updated to string without setting UpdatedFmt\n"; } } else { if (not defined $config->{ Item }{ UpdatedFmt }) { die "Cannot determine publish time; [Item].UpdatedFmt not configured\n"; } my $sel = xpath_select($dom, $config->{ Item }{ Updated }); my $t = Time::Piece->strptime($sel, $config->{ Item }{ UpdatedFmt }); $item->{ updated } = $t->epoch; } } if ( defined $config->{ Item }{ Published } and $config->{ Item }{ Published }[1] == SELECTOR_STRING and $config->{ Item }{ Published }[0] eq 'updated' ) { $item->{ published } = $item->{ updated }; } if (defined $config->{ Item }{ Author }) { $item->{ author } = xpath_select($n, $config->{ Item }{ Author }); } if (defined $config->{ Item }{ Content }) { $item->{ summary } = xpath_select($n, $config->{ Item }{ Content }); if ( $config->{ Item }{ Content }->[1] == SELECTOR_SINGLE_HTML or $config->{ Item }{ Content } == SELECTOR_MULTI_HTML ) { $item->{ summary_html } = 1; } } if (defined $config->{ Item }{ Categories }) { if ($config->{ Item }{ Categories }[1] == SELECTOR_STRING) { my @cats = split /\s*,\s*/, $config->{ Item }{ Categories }[0]; $item->{ categories } = \@cats; } else { $item->{ categories } = [ xpath_multi_select($n, $config->{ Item }{ Categories }) ]; } } if (defined $config->{ Item }{ Rights }) { $item->{ rights } = xpath_select($n, $config->{ Item }{ Rights }); } push @{ $feed->{ items } }, $item; } my @times = grep { defined } map { @{ $_ }{ qw(published updated) } } @{ $feed->{ items } }; if (not defined $feed->{ published }) { $feed->{ published } = min @times; } if (not defined $feed->{ updated }) { $feed->{ updated } = max @times; } return $feed; } GetOptions( 'a' => \my $force_atom, 'r' => \my $force_rss, 'o=s' => \my $output, 'h' => sub { print $USAGE; exit 0 }, ) or die $USAGE; my ($config_file, $html) = @ARGV; if (not defined $config_file) { die $USAGE; } my $config = process_config($config_file); if (not defined $html) { $html = $config->{ Feed }{ Link }; if (not defined $html) { die "html2rss.pl requires an HTML file/page supplied to it either via a command-line argument or the [Feed].Link configuration field\n"; } } my $file; if ($html =~ /^\w+:\/\//) { my $tmp = do { my ($h, $p) = tempfile(UNLINK => 1); close $h; $p; }; curl_to($html, $tmp, user_agent => $DEFAULT_AGENT); $file = $tmp; } else { $file = $html; } my $feed = html2feed($file, $config); my $feed_xml; if ($force_atom) { $feed_xml = generate_atom_feed($feed); } elsif ($force_rss) { $feed_xml = generate_rss_feed($feed); } elsif (defined $config->{ Feed }{ Format }) { if ($config->{ Feed }{ Format } eq 'atom') { $feed_xml = generate_atom_feed($feed); } elsif ($config->{ Feed }{ Format } eq 'rss') { $feed_xml = generate_rss_feed($feed); } else { die "'$config->{ Feed }{ Format }' is not a valid feed format\n"; } } else { $feed_xml = generate_atom_feed($feed); } if (not defined $output) { binmode *STDOUT; $feed_xml->toFH(*STDOUT, 2); } else { $feed_xml->toFile($output, 2); say "Wrote feed to $output"; } =head1 NAME html2rss.pl - Convert HTML pages to RSS feeds =head1 USAGE html2rss.pl [options] config [location] =head1 DESCRIPTION B is a Perl script that converts HTML pages to RSS feeds based on parameters read from a given feed configuration file. B takes a configuration file as input, whose format is described in the subsequent section. B can also optionally take either a path to an HTML file or a URL to manually specify the HTML file/page to convert. If no location is given, B will convert the URL/path set as the feed link in the configuration file. =head1 CONFIGURATION B processes HTML pages based on parameters it reads from configuration files. B uses a configuration format similar to the INI file format. A configuration file consists of sections marked with their name enclosed in square brackets. # Section named "Feed" [Feed] Sections contain lists of configuration options which are lines of key-value pairs that are seperated by an equals sign. [Feed] Title = Yadda yadda... Link = https://phony.com/ Description = [/html/body/div[1]/p] Lines starting with a hash are comments and will be ignored by B. B requires two sections: C<[Feed]> and C<[Item]>. =head2 Selectors Many configuration options can take an XPath selector statement as a value. A selector statement will select elements from the HTML's DOM tree and use the captured text as the value for the configuration field. Selector statements are enclosed in either square or angle brackets. A single pair of brackets will capture only the first selected element. A double pair of brackets will capture the contents of all matching elements. =over 4 =item [I] =item [[I]] Capture the text contents of the selected elements. =item > =item <>> Capture the serialized HTML of the selected elements. =back =head2 [Feed] options =over 4 =item Title = I | I The string or element selector for the feed's title. B. =item Link = I The HTML page's URL. B. B. =item Description = I | I The string or element selector for the feed's description. =item Language = I | I The string or element selector for the feed's language. =item Rights = I | I The string or element selector for the feed's rights disclaimer. =item Published = I