Minion-7.05/000755 000765 000024 00000000000 13142141670 012635 5ustar00sristaff000000 000000 Minion-7.05/Changes000644 000765 000024 00000035426 13142105353 014140 0ustar00sristaff000000 000000 7.05 2017-08-07 - Improved foreground method in Minion to rethrow job exceptions. 7.04 2017-08-01 - Added foreground method to Minion. - Added id option to dequeue methods in Minion::Worker and Minion::Backend::Pg. - Added attempts option to retry and retry_job methods in Minion::Job and Minion::Backend::Pg. - Added -f option to job command. 7.03 2017-07-06 - Updated Mojo::Pg requirement to 4.0. - Improved Minion::Backend::Pg to support sharing the database connection cache with existing Mojo::Pg objects. 7.02 2017-07-05 - Improved performance of worker command when processing jobs that are finished very quickly. 7.01 2017-06-25 - Added note methods to Minion::Job and Minion::Backend::Pg. - Added notes option to enqueue methods in Minion and Minion::Backend::Pg. - Added notes field to info method in Minion::Job and job_info method in Minion::Backend::Pg. - Improved performance of stats and lock methods in Minion::Backend::Pg with a new index and other optimizations. (depesz) - Improved benchmark script to be more consistent. (depesz) 7.0 2017-06-18 - Added support for rate limiting and unique jobs. - Added lock and unlock methods to Minion and Minion::Backend::Pg. - Improved performance of Minion::Backend::Pg significantly with a new index and other optimizations. 6.06 2017-06-03 - Added an example application to demonstrate how to integrate background jobs into well-structured Mojolicious applications. 6.05 2017-04-03 - Added support for sharing worker status information. - Improved retry methods to allow for active jobs to be retried as well. - Improved job command to show timestamps in RFC 3339 (1994-11-06T08:49:37Z) format. 6.04 2017-03-18 - Added -f option to worker command. - Removed -r option from job command, so you have to write --remove from now on, which should prevent accidental mixups with the -R option in the future. 6.03 2017-03-14 - Fixed serious performance problems in Minion::Backend::Pg. 6.02 2017-01-02 - Updated Mojo::Pg requirement to 2.18. 6.01 2017-01-01 - Updated Mojo::Pg requirement to 2.33. - Improved performance of Minion::Backend::Pg with a new index. 6.0 2016-09-17 - Removed TTIN, TTOU and USR1 signals from worker command. - Changed return value of start method in Minion::Job. - Added support for worker remote control commands. - Added commands attribute to Minion::Worker. - Added add_command and process_commands methods to Minion::Worker. - Added pid and stop methods to Minion::Job. - Added broadcast and receive methods to Minion::Backend::Pg. - Added -b option to job command. - Improved worker command with support for jobs and stop remote control commands. 5.09 2016-08-31 - Added EXPERIMENTAL enqueued_jobs field to stats methods in Minion and Minion::Backend::Pg. - Improved Minion::Backend::Pg performance slightly with a new index. 5.08 2016-05-20 - Improved repair methods not to remove finished jobs with unresolved dependencies. 5.07 2016-05-17 - Added support for job dependencies. (jberger, sri) - Added parents option to enqueue methods in Minion and Minion::Backend::Pg. (jberger, sri) - Added children and parents fields to info method in Minion::Job and job_info method in Minion::Backend::Pg. - Added -P option to job command. - Improved stats methods to include jobs with unresolved dependencies in delayed_jobs count. 5.06 2016-05-05 - Improved worker command to support the TTIN, TTOU and USR1 signals. - Improved Minion::Backend::Pg to handle delayed and retried jobs more efficiently. 5.05 2016-04-20 - Added queue option to list_jobs method in Minion::Backend::Pg. - Improved performance of stats method in Minion::Backend::Pg slightly. 5.04 2016-04-19 - Added EXPERIMENTAL delayed_jobs field to stats methods in Minion and Minion::Backend::Pg. - Updated Mojo::Pg requirement to 2.18. - Improved job command to show more detailed information for jobs and workers. 5.03 2016-04-10 - Added enqueue event to Minion. (jberger) 5.02 2016-03-23 - Fixed copyright notice. 5.01 2016-02-24 - Fixed worker command to repair in regular intervals. 5.0 2016-02-17 - Minion::Backend::Pg now requires PostgreSQL 9.5. - Added start event to Minion::Job. - Added -R option to worker command. - Reduced default missing_after value to 30 minutes. - Reduced default remove_after value to 2 days. - Improved Minion::Backend::Pg performance significantly with a new index and PostgreSQL 9.5 features. - Improved Minion::Job to capture more exceptions. - Improved worker command to support the QUIT signal. - Improved worker command to repair in less regular intervals. 4.06 2016-02-06 - Improved performance of Minion::Backend::Pg slightly. 4.05 2016-02-05 - Improved Minion::Backend::Pg to check the PostgreSQL version. 4.04 2016-01-23 - Updated Minion::Backend::Pg to use new Mojo::Pg features. 4.03 2016-01-17 - Removed an unused index from Minion::Backend::Pg. - Fixed a bug where the worker command would always watch the default queue. (avkhozov) 4.02 2016-01-03 - Updated links to Mojolicious website. 4.01 2015-11-12 - Improved retry methods to allow options to be changed for already inactive jobs. 4.0 2015-11-09 - Removed attempts attribute from Minion::Job. - Improved Minion::Backend::Pg to preserve more information across retries for debugging. - Fixed bug where jobs could not be retried automatically if a worker went away. 3.03 2015-11-08 - Added queues option to perform_jobs method in Minion. 3.02 2015-10-31 - Fixed portability issues in tests. 3.01 2015-10-30 - Added support for retrying failed jobs automatically. - Added backoff attribute to Minion. - Added attempts attribute to Minion::Job. - Added attempts option to enqueue methods in Minion and Minion::Backend::Pg. - Added -A option to job command. 3.0 2015-10-30 - Removed Minion::Backend::File, because DBM::Deep quickly becomes unusably slow, you can use the CPAN module Minion::Backend::SQLite instead. 2.05 2015-10-15 - Fixed bug where jobs could sometimes not be finished correctly by the worker command. 2.04 2015-10-14 - Fixed portability issue in worker command. 2.03 2015-10-09 - Improved commands to show all options that can affect their behavior. 2.02 2015-10-08 - Improved job command to show the queue in job lists. 2.01 2015-10-02 - Fixed Windows portability issues in tests. 2.0 2015-10-01 - Removed -t option from worker command. - Added support for multiple named queues. - Added retries attribute to Minion::Job. - Added retries argument to fail_job, finish_job and retry_job methods in Minion::Backend::File and Minion::Backend::Pg. - Added queue option to enqueue method in Minion. - Added queue option to enqueue and retry_job methods in Minion::Backend::File and Minion::Backend::Pg. - Added queues option to dequeue methods in Minion::Worker, Minion::Backend::File and Minion::Backend::Pg. - Added -q option to job and worker commands. - Improved worker command to be more resilient to time jumps. - Fixed a race condition in Minion::Backend::File and Minion::Backend::Pg where a retried job did not have to be dequeued again before it could be finished. 1.19 2015-09-28 - Added support for retrying jobs with a new priority. - Added priority option to retry method in Minion::Job. - Added priority option to retry_job methods in Minion::Backend::File and Minion::Backend::Pg. 1.18 2015-08-30 - Fixed Makefile.PL to be compliant with version 2 of the CPAN distribution metadata specification. 1.17 2015-08-29 - Fixed bug in worker command where new jobs would still be dequeued after receiving an INT/TERM signal. 1.16 2015-08-28 - Improved worker command to detect workers without heartbeat a little faster. 1.15 2015-05-15 - Added support for retrying jobs with a delay. (kwa) - Added delay option to retry method in Minion::Job. (kwa) - Added delay option to retry_job methods in Minion::Backend::File and Minion::Backend::Pg. (kwa) 1.14 2015-04-21 - Improved performance of Minion::Backend::Pg with a new index. (avkhozov) 1.13 2015-03-25 - Improved Minion::Backend::Pg to reset the job queue a little faster. 1.12 2015-03-17 - Improved portability of some tests. 1.11 2015-03-10 - Fixed tests to work without Mojo::Pg. 1.10 2015-03-09 - Added support for performing multiple jobs concurrently with a single worker. (bpmedley, sri) - Added is_finished and start methods to Minion::Job. (bpmedley, sri) - Added -j option to worker command. (bpmedley, sri) - Fixed concurrency bugs in Minion::Backend::File. - Fixed bug in job command where timing information was not displayed correctly. 1.09 2015-03-02 - Added support for monitoring workers with heartbeats instead of signals. - Added missing_after attribute to Minion. - Added -I option to worker command. - Fixed bug where workers were considered active even if they had no active jobs assigned. 1.08 2015-02-20 - Updated for Mojolicious 5.81. 1.07 2015-02-12 - Updated Minion::Backend::Pg for Mojo::Pg 1.08. 1.06 2015-01-26 - Improved commands to use less punctuation. 1.05 2015-01-05 - Improved repair methods in Minion::Backend::File and Minion::Backend::Pg to mention the current process in results of failed jobs. 1.04 2015-01-03 - Improved Minion::Backend::Pg to use new JSON features of Mojo::Pg. 1.03 2014-12-19 - Added -t option to worker command. 1.02 2014-11-22 - Renamed -L and -T options to -l and -S. - Improved job command table formatting. 1.01 2014-11-20 - Improved documentation. 1.0 2014-11-19 - Removed experimental status from distribution. 0.45 2014-11-18 - Improved dequeue performance in Minion::Backend::Pg significantly. (bpmedley) 0.44 2014-11-17 - Fixed bug where jobs could not be dequeued inside a running event loop. 0.43 2014-11-17 - Fixed bug where advisory locks would run out of shared memory. 0.42 2014-11-16 - Improved Minion::Backend::Pg performance with advisory locks and notifications. (bpmedley, sri) 0.41 2014-11-15 - Improved Minion::Backend::Pg performance. 0.40 2014-11-11 - Added PostgreSQL support with Mojo::Pg. (bpmedley, sri) - Added support for job results. - Added Minion::Backend::Pg. (bpmedley, sri) 0.39 2014-10-05 - Added DBM::Deep support to Minion::Backend::File. - Renamed -S option to -o. 0.38 2014-10-04 - Removed support for non-blocking enqueue. - Removed Minion::Backend::Mango. 0.37 2014-09-22 - Fixed packaging bug. 0.36 2014-09-21 - Updated Makefile.PL for version 2 of the CPAN distribution metadata specification. 0.35 2014-09-06 - Improved Minion::Backend::File to write less often. 0.34 2014-08-26 - Improved job command to show higher precision times. 0.33 2014-07-10 - Replaced state argument of list_jobs methods in Minion::Backend::File and Minion::Backend::Mango with more versatile options argument. - Added -t option to job command. 0.32 2014-07-10 - Added state argument to list_jobs methods in Minion::Backend::File and Minion::Backend::Mango. - Added -T option to job command. 0.31 2014-07-09 - Reduced CPU usage of Minion::Backend::Mango when waiting for new jobs. 0.30 2014-07-08 - Added timeout argument to dequeue methods in Minion::Backend::File, Minion::Backend::Mango and Minion::Worker. 0.29 2014-07-07 - Renamed restart_job method to retry_job in Minion::Backend, Minion::Backend::File and Minion::Backend::Mango. - Renamed restart method to retry in Minion::Job. - Improved worker command to repair in regular intervals. 0.28 2014-06-28 - Added spawn event to Minion::Job. 0.27 2014-06-21 - Replaced delayed option with delay. 0.26 2014-06-18 - Renamed clean_up_after attribute in Minion to remove_after. 0.25 2014-06-17 - Removed auto_perform attribute from Minion. - Added perform_jobs method to Minion. - Fixed multiple Windows bugs. 0.24 2014-06-16 - Improved Minion::Job to reset Mojo::IOLoop. - Fixed Windows bugs in tests. 0.23 2014-06-15 - Fixed Minion::Backend::File Windows support. 0.22 2014-06-15 - Reduced default clean_up_after value to 10 days. 0.21 2014-06-14 - Added clean_up_after attribute to Minion. - Improved performance of repair methods. 0.20 2014-06-13 - Added module Minion::Backend::File. - Improved Minion::Backend to provide a generic repair method. 0.15 2014-06-04 - Fixed a few small bugs in Minion::Backend::Mango. 0.14 2014-06-04 - Fixed Minion::Backend::Mango to work with strings in addition to object ids. 0.13 2014-06-03 - Added list_workers methods to Minion::Backend and Minion::Backend::Mango. 0.12 2014-06-03 - Fixed enqueue to use the correct time format. 0.11 2014-06-03 - Changed a few return values in Minion::Backend::Mango. 0.10 2014-06-02 - Removed created, delayed, error, finished, priority, restarted, restarts, started and state methods from Minion::Job. - Removed started method from Minion::Worker. - Added support for pluggable backends. - Added modules Minion::Backend and Minion::Backend::Mango. - Added backend attribute to Minion. - Added reset method to Minion. - Added info methods to Minion::Job and Minion::Worker. - Added -L and -S options to job command. 0.09 2014-04-05 - Added worker event to Minion. - Added dequeue event to Minion::Worker. - Added failed and finished events to Minion::Job. - Added restarted method to Minion::Job. - Changed return values of fail, finish and restart methods in Minion::Job. 0.08 2014-04-04 - Added support for non-blocking enqueue. 0.07 2014-04-03 - Improved performance by reusing Mango connections. 0.06 2014-04-03 - Added delayed and priority methods to Minion::Job. - Renamed after option to delayed. - Improved job command to use a human readable time format and allow new jobs to be enqueued. 0.05 2014-04-03 - Improved job command to stream job lists and show more information. 0.04 2014-04-02 - Removed all_jobs and one_job methods from Minion::Job. - Removed repair method from Minion::Worker; - Added module Minion::Command::minion::job. - Added auto_perform attribute to Minion. - Added repair method to Minion. - Added created, error, finished, remove, restart, restarts and started methods to Minion::Job. - Added started method to Minion::Worker. 0.03 2014-03-30 - Removed doc and worker attributes from Minion::Job. - Added args, id and minion attributes to Minion::Job. - Added id attribute to Minion::Worker. - Added job method to Minion. - Added state method to Minion::Job. 0.02 2014-03-28 - Added support for delayed jobs. - Added stats method to Minion. - Added app method to Minion::Job. - Reduced Perl requirement to 5.10.1. 0.01 2014-03-27 - First release. Minion-7.05/examples/000755 000765 000024 00000000000 13142141670 014453 5ustar00sristaff000000 000000 Minion-7.05/lib/000755 000765 000024 00000000000 13142141670 013403 5ustar00sristaff000000 000000 Minion-7.05/LICENSE000644 000765 000024 00000021413 13123461250 013641 0ustar00sristaff000000 000000 The Artistic License 2.0 Copyright (c) 2000-2006, The Perl Foundation. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble This license establishes the terms under which a given free software Package may be copied, modified, distributed, and/or redistributed. The intent is that the Copyright Holder maintains some artistic control over the development of that Package while still keeping the Package available as open source and free software. You are always permitted to make arrangements wholly outside of this license directly with the Copyright Holder of a given Package. If the terms of this license do not permit the full use that you propose to make of the Package, you should contact the Copyright Holder and seek a different licensing arrangement. Definitions "Copyright Holder" means the individual(s) or organization(s) named in the copyright notice for the entire Package. "Contributor" means any party that has contributed code or other material to the Package, in accordance with the Copyright Holder's procedures. "You" and "your" means any person who would like to copy, distribute, or modify the Package. "Package" means the collection of files distributed by the Copyright Holder, and derivatives of that collection and/or of those files. A given Package may consist of either the Standard Version, or a Modified Version. "Distribute" means providing a copy of the Package or making it accessible to anyone else, or in the case of a company or organization, to others outside of your company or organization. "Distributor Fee" means any fee that you charge for Distributing this Package or providing support for this Package to another party. It does not mean licensing fees. "Standard Version" refers to the Package if it has not been modified, or has been modified only in ways explicitly requested by the Copyright Holder. "Modified Version" means the Package, if it has been changed, and such changes were not explicitly requested by the Copyright Holder. "Original License" means this Artistic License as Distributed with the Standard Version of the Package, in its current version or as it may be modified by The Perl Foundation in the future. "Source" form means the source code, documentation source, and configuration files for the Package. "Compiled" form means the compiled bytecode, object code, binary, or any other form resulting from mechanical transformation or translation of the Source form. Permission for Use and Modification Without Distribution (1) You are permitted to use the Standard Version and create and use Modified Versions for any purpose without restriction, provided that you do not Distribute the Modified Version. Permissions for Redistribution of the Standard Version (2) You may Distribute verbatim copies of the Source form of the Standard Version of this Package in any medium without restriction, either gratis or for a Distributor Fee, provided that you duplicate all of the original copyright notices and associated disclaimers. At your discretion, such verbatim copies may or may not include a Compiled form of the Package. (3) You may apply any bug fixes, portability changes, and other modifications made available from the Copyright Holder. The resulting Package will still be considered the Standard Version, and as such will be subject to the Original License. Distribution of Modified Versions of the Package as Source (4) You may Distribute your Modified Version as Source (either gratis or for a Distributor Fee, and with or without a Compiled form of the Modified Version) provided that you clearly document how it differs from the Standard Version, including, but not limited to, documenting any non-standard features, executables, or modules, and provided that you do at least ONE of the following: (a) make the Modified Version available to the Copyright Holder of the Standard Version, under the Original License, so that the Copyright Holder may include your modifications in the Standard Version. (b) ensure that installation of your Modified Version does not prevent the user installing or running the Standard Version. In addition, the Modified Version must bear a name that is different from the name of the Standard Version. (c) allow anyone who receives a copy of the Modified Version to make the Source form of the Modified Version available to others under (i) the Original License or (ii) a license that permits the licensee to freely copy, modify and redistribute the Modified Version using the same licensing terms that apply to the copy that the licensee received, and requires that the Source form of the Modified Version, and of any works derived from it, be made freely available in that license fees are prohibited but Distributor Fees are allowed. Distribution of Compiled Forms of the Standard Version or Modified Versions without the Source (5) You may Distribute Compiled forms of the Standard Version without the Source, provided that you include complete instructions on how to get the Source of the Standard Version. Such instructions must be valid at the time of your distribution. If these instructions, at any time while you are carrying out such distribution, become invalid, you must provide new instructions on demand or cease further distribution. If you provide valid instructions or cease distribution within thirty days after you become aware that the instructions are invalid, then you do not forfeit any of your rights under this license. (6) You may Distribute a Modified Version in Compiled form without the Source, provided that you comply with Section 4 with respect to the Source of the Modified Version. Aggregating or Linking the Package (7) You may aggregate the Package (either the Standard Version or Modified Version) with other packages and Distribute the resulting aggregation provided that you do not charge a licensing fee for the Package. Distributor Fees are permitted, and licensing fees for other components in the aggregation are permitted. The terms of this license apply to the use and Distribution of the Standard or Modified Versions as included in the aggregation. (8) You are permitted to link Modified and Standard Versions with other works, to embed the Package in a larger work of your own, or to build stand-alone binary or bytecode versions of applications that include the Package, and Distribute the result without restriction, provided the result does not expose a direct interface to the Package. Items That are Not Considered Part of a Modified Version (9) Works (including, but not limited to, modules and scripts) that merely extend or make use of the Package, do not, by themselves, cause the Package to be a Modified Version. In addition, such works are not considered parts of the Package itself, and are not subject to the terms of this license. General Provisions (10) Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license. (11) If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license. (12) This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder. (13) This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed. (14) Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Minion-7.05/Makefile.PL000644 000765 000024 00000001733 13123461250 014611 0ustar00sristaff000000 000000 use 5.010001; use strict; use warnings; use ExtUtils::MakeMaker; WriteMakefile( NAME => 'Minion', VERSION_FROM => 'lib/Minion.pm', ABSTRACT => 'Job queue', AUTHOR => 'Sebastian Riedel ', LICENSE => 'artistic_2', META_MERGE => { dynamic_config => 0, 'meta-spec' => {version => 2}, no_index => {directory => ['t']}, prereqs => {runtime => {requires => {perl => '5.010001'}}}, resources => { bugtracker => {web => 'https://github.com/kraih/minion/issues'}, homepage => 'http://mojolicious.org', license => ['http://www.opensource.org/licenses/artistic-license-2.0'], repository => { type => 'git', url => 'https://github.com/kraih/minion.git', web => 'https://github.com/kraih/minion', }, x_IRC => 'irc://irc.perl.org/#mojo' }, }, PREREQ_PM => {Mojolicious => '7.29'}, test => {TESTS => 't/*.t t/*/*.t'} ); Minion-7.05/MANIFEST000644 000765 000024 00000001736 13142141670 013775 0ustar00sristaff000000 000000 Changes examples/linkcheck/lib/LinkCheck.pm examples/linkcheck/lib/LinkCheck/Controller/Links.pm examples/linkcheck/lib/LinkCheck/Task/CheckLinks.pm examples/linkcheck/linkcheck.conf examples/linkcheck/script/linkcheck examples/linkcheck/t/linkcheck.t examples/linkcheck/templates/layouts/linkcheck.html.ep examples/linkcheck/templates/links/index.html.ep examples/linkcheck/templates/links/result.html.ep examples/minion_bench.pl lib/Minion.pm lib/Minion/Backend.pm lib/Minion/Backend/Pg.pm lib/Minion/Command/minion.pm lib/Minion/Command/minion/job.pm lib/Minion/Command/minion/worker.pm lib/Minion/Job.pm lib/Minion/Worker.pm lib/Mojolicious/Plugin/Minion.pm LICENSE Makefile.PL MANIFEST This list of files MANIFEST.bak MANIFEST.SKIP README.md t/backend.t t/commands.t t/pg.t t/pg_lite_app.t t/pod.t t/pod_coverage.t META.yml Module YAML meta-data (added by MakeMaker) META.json Module JSON meta-data (added by MakeMaker) Minion-7.05/MANIFEST.bak000644 000765 000024 00000000654 13123461250 014525 0ustar00sristaff000000 000000 Changes CONTRIBUTING.md lib/Minion.pm lib/Minion/Backend.pm lib/Minion/Backend/File.pm lib/Minion/Backend/Pg.pm lib/Minion/Command/minion.pm lib/Minion/Command/minion/job.pm lib/Minion/Command/minion/worker.pm lib/Minion/Job.pm lib/Minion/Worker.pm lib/Mojolicious/Plugin/Minion.pm LICENSE Makefile.PL MANIFEST This list of files MANIFEST.SKIP README.md t/commands.t t/file.t t/file_lite_app.t t/pg.t t/pod.t t/pod_coverage.t Minion-7.05/MANIFEST.SKIP000644 000765 000024 00000000132 13136606205 014532 0ustar00sristaff000000 000000 ^\.(?!perltidyrc) .*\.old$ \.tar\.gz$ ^Makefile$ ^MYMETA\. ^blib ^pm_to_blib \B\.DS_Store Minion-7.05/META.json000644 000765 000024 00000002615 13142141670 014262 0ustar00sristaff000000 000000 { "abstract" : "Job queue", "author" : [ "Sebastian Riedel " ], "dynamic_config" : 0, "generated_by" : "ExtUtils::MakeMaker version 7.3, CPAN::Meta::Converter version 2.150010", "license" : [ "artistic_2" ], "meta-spec" : { "url" : "http://search.cpan.org/perldoc?CPAN::Meta::Spec", "version" : 2 }, "name" : "Minion", "no_index" : { "directory" : [ "t", "inc", "t" ] }, "prereqs" : { "build" : { "requires" : { "ExtUtils::MakeMaker" : "0" } }, "configure" : { "requires" : { "ExtUtils::MakeMaker" : "0" } }, "runtime" : { "requires" : { "Mojolicious" : "7.29", "perl" : "5.010001" } } }, "release_status" : "stable", "resources" : { "bugtracker" : { "web" : "https://github.com/kraih/minion/issues" }, "homepage" : "http://mojolicious.org", "license" : [ "http://www.opensource.org/licenses/artistic-license-2.0" ], "repository" : { "type" : "git", "url" : "https://github.com/kraih/minion.git", "web" : "https://github.com/kraih/minion" }, "x_IRC" : "irc://irc.perl.org/#mojo" }, "version" : "7.05", "x_serialization_backend" : "JSON::PP version 2.94" } Minion-7.05/META.yml000644 000765 000024 00000001456 13142141670 014114 0ustar00sristaff000000 000000 --- abstract: 'Job queue' author: - 'Sebastian Riedel ' build_requires: ExtUtils::MakeMaker: '0' configure_requires: ExtUtils::MakeMaker: '0' dynamic_config: 0 generated_by: 'ExtUtils::MakeMaker version 7.3, CPAN::Meta::Converter version 2.150010' license: artistic_2 meta-spec: url: http://module-build.sourceforge.net/META-spec-v1.4.html version: '1.4' name: Minion no_index: directory: - t - inc - t requires: Mojolicious: '7.29' perl: '5.010001' resources: IRC: irc://irc.perl.org/#mojo bugtracker: https://github.com/kraih/minion/issues homepage: http://mojolicious.org license: http://www.opensource.org/licenses/artistic-license-2.0 repository: https://github.com/kraih/minion.git version: '7.05' x_serialization_backend: 'CPAN::Meta::YAML version 0.018' Minion-7.05/README.md000644 000765 000024 00000003435 13123776076 014136 0ustar00sristaff000000 000000 # Minion [![Build Status](https://travis-ci.org/kraih/minion.svg?branch=master)](https://travis-ci.org/kraih/minion) A job queue for the [Mojolicious](http://mojolicious.org) real-time web framework, with support for multiple named queues, priorities, delayed jobs, job progress, job results, retries with backoff, rate limiting, unique jobs, statistics, distributed workers, parallel processing, autoscaling, resource leak protection and multiple backends (such as [PostgreSQL](http://www.postgresql.org)). Job queues allow you to process time and/or computationally intensive tasks in background processes, outside of the request/response lifecycle. Among those tasks you'll commonly find image resizing, spam filtering, HTTP downloads, building tarballs, warming caches and basically everything else you can imagine that's not super fast. ```perl use Mojolicious::Lite; use 5.20.0; use experimental 'signatures'; plugin Minion => {Pg => 'postgresql://postgres@/test'}; # Slow task app->minion->add_task(slow_log => sub ($job, $msg) { sleep 5; $job->app->log->debug(qq{Received message "$msg"}); }); # Perform job in a background worker process get '/log' => sub ($c) { $c->minion->enqueue(slow_log => [$c->param('msg') // 'no message']); $c->render(text => 'Your message will be logged soon.'); }; app->start; ``` Just start one or more background worker processes in addition to your web server. $ ./myapp.pl minion worker ## Installation All you need is a one-liner, it takes less than a minute. $ curl -L https://cpanmin.us | perl - -M https://cpan.metacpan.org -n Minion We recommend the use of a [Perlbrew](http://perlbrew.pl) environment. ## Want to know more? Take a look at our excellent [documentation](http://mojolicious.org/perldoc/Minion)! Minion-7.05/t/000755 000765 000024 00000000000 13142141670 013100 5ustar00sristaff000000 000000 Minion-7.05/t/backend.t000644 000765 000024 00000004445 13123755650 014673 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Test::More; use Minion::Backend; # Abstract methods eval { Minion::Backend->broadcast }; like $@, qr/Method "broadcast" not implemented by subclass/, 'right error'; eval { Minion::Backend->dequeue }; like $@, qr/Method "dequeue" not implemented by subclass/, 'right error'; eval { Minion::Backend->enqueue }; like $@, qr/Method "enqueue" not implemented by subclass/, 'right error'; eval { Minion::Backend->fail_job }; like $@, qr/Method "fail_job" not implemented by subclass/, 'right error'; eval { Minion::Backend->finish_job }; like $@, qr/Method "finish_job" not implemented by subclass/, 'right error'; eval { Minion::Backend->job_info }; like $@, qr/Method "job_info" not implemented by subclass/, 'right error'; eval { Minion::Backend->list_jobs }; like $@, qr/Method "list_jobs" not implemented by subclass/, 'right error'; eval { Minion::Backend->list_workers }; like $@, qr/Method "list_workers" not implemented by subclass/, 'right error'; eval { Minion::Backend->lock }; like $@, qr/Method "lock" not implemented by subclass/, 'right error'; eval { Minion::Backend->note }; like $@, qr/Method "note" not implemented by subclass/, 'right error'; eval { Minion::Backend->receive }; like $@, qr/Method "receive" not implemented by subclass/, 'right error'; eval { Minion::Backend->register_worker }; like $@, qr/Method "register_worker" not implemented by subclass/, 'right error'; eval { Minion::Backend->remove_job }; like $@, qr/Method "remove_job" not implemented by subclass/, 'right error'; eval { Minion::Backend->repair }; like $@, qr/Method "repair" not implemented by subclass/, 'right error'; eval { Minion::Backend->reset }; like $@, qr/Method "reset" not implemented by subclass/, 'right error'; eval { Minion::Backend->retry_job }; like $@, qr/Method "retry_job" not implemented by subclass/, 'right error'; eval { Minion::Backend->stats }; like $@, qr/Method "stats" not implemented by subclass/, 'right error'; eval { Minion::Backend->unlock }; like $@, qr/Method "unlock" not implemented by subclass/, 'right error'; eval { Minion::Backend->unregister_worker }; like $@, qr/Method "unregister_worker" not implemented by subclass/, 'right error'; eval { Minion::Backend->worker_info }; like $@, qr/Method "worker_info" not implemented by subclass/, 'right error'; done_testing(); Minion-7.05/t/commands.t000644 000765 000024 00000001254 13123461250 015066 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Test::More; # minion require Minion::Command::minion; my $minion = Minion::Command::minion->new; ok $minion->description, 'has a description'; like $minion->message, qr/minion/, 'has a message'; like $minion->hint, qr/help/, 'has a hint'; # job require Minion::Command::minion::job; my $job = Minion::Command::minion::job->new; ok $job->description, 'has a description'; like $job->usage, qr/job/, 'has usage information'; # worker require Minion::Command::minion::worker; my $worker = Minion::Command::minion::worker->new; ok $worker->description, 'has a description'; like $worker->usage, qr/worker/, 'has usage information'; done_testing(); Minion-7.05/t/pg.t000644 000765 000024 00000075654 13142127710 013713 0ustar00sristaff000000 000000 use Mojo::Base -strict; BEGIN { $ENV{MOJO_REACTOR} = 'Mojo::Reactor::Poll' } use Test::More; plan skip_all => 'set TEST_ONLINE to enable this test' unless $ENV{TEST_ONLINE}; use Minion; use Mojo::IOLoop; use Sys::Hostname 'hostname'; use Time::HiRes qw(time usleep); # Isolate tests require Mojo::Pg; my $pg = Mojo::Pg->new($ENV{TEST_ONLINE}); $pg->db->query('drop schema if exists minion_test cascade'); $pg->db->query('create schema minion_test'); my $minion = Minion->new(Pg => $ENV{TEST_ONLINE}); $minion->backend->pg->search_path(['minion_test']); # Nothing to repair my $worker = $minion->repair->worker; isa_ok $worker->minion->app, 'Mojolicious', 'has default application'; # Migrate up and down is $minion->backend->pg->migrations->active, 17, 'active version is 17'; is $minion->backend->pg->migrations->migrate(0)->active, 0, 'active version is 0'; is $minion->backend->pg->migrations->migrate->active, 17, 'active version is 17'; # Register and unregister $worker->register; like $worker->info->{started}, qr/^[\d.]+$/, 'has timestamp'; my $notified = $worker->info->{notified}; like $notified, qr/^[\d.]+$/, 'has timestamp'; my $id = $worker->id; is $worker->register->id, $id, 'same id'; usleep 50000; ok $worker->register->info->{notified} > $notified, 'new timestamp'; is $worker->unregister->info, undef, 'no information'; my $host = hostname; is $worker->register->info->{host}, $host, 'right host'; is $worker->info->{pid}, $$, 'right pid'; is $worker->unregister->info, undef, 'no information'; # Repair missing worker $minion->add_task(test => sub { }); my $worker2 = $minion->worker->register; isnt $worker2->id, $worker->id, 'new id'; $id = $minion->enqueue('test'); my $job = $worker2->dequeue(0); is $job->id, $id, 'right id'; is $worker2->info->{jobs}[0], $job->id, 'right id'; $id = $worker2->id; undef $worker2; is $job->info->{state}, 'active', 'job is still active'; ok !!$minion->backend->worker_info($id), 'is registered'; $minion->backend->pg->db->query( "update minion_workers set notified = now() - interval '1 second' * ? where id = ?", $minion->missing_after + 1, $id ); $minion->repair; ok !$minion->backend->worker_info($id), 'not registered'; like $job->info->{finished}, qr/^[\d.]+$/, 'has finished timestamp'; is $job->info->{state}, 'failed', 'job is no longer active'; is $job->info->{result}, 'Worker went away', 'right result'; # Repair abandoned job $worker->register; $id = $minion->enqueue('test'); $job = $worker->dequeue(0); is $job->id, $id, 'right id'; $worker->unregister; $minion->repair; is $job->info->{state}, 'failed', 'job is no longer active'; is $job->info->{result}, 'Worker went away', 'right result'; # Repair old jobs $worker->register; $id = $minion->enqueue('test'); my $id2 = $minion->enqueue('test'); my $id3 = $minion->enqueue('test'); $worker->dequeue(0)->perform for 1 .. 3; my $finished = $minion->backend->pg->db->query( 'select extract(epoch from finished) as finished from minion_jobs where id = ?', $id2 )->hash->{finished}; $minion->backend->pg->db->query( 'update minion_jobs set finished = to_timestamp(?) where id = ?', $finished - ($minion->remove_after + 1), $id2); $finished = $minion->backend->pg->db->query( 'select extract(epoch from finished) as finished from minion_jobs where id = ?', $id3 )->hash->{finished}; $minion->backend->pg->db->query( 'update minion_jobs set finished = to_timestamp(?) where id = ?', $finished - ($minion->remove_after + 1), $id3); $worker->unregister; $minion->repair; ok $minion->job($id), 'job has not been cleaned up'; ok !$minion->job($id2), 'job has been cleaned up'; ok !$minion->job($id3), 'job has been cleaned up'; # List workers $worker = $minion->worker->register; $worker2 = $minion->worker->status({whatever => 'works!'})->register; my $batch = $minion->backend->list_workers(0, 10); ok $batch->[0]{id}, 'has id'; is $batch->[0]{host}, $host, 'right host'; is $batch->[0]{pid}, $$, 'right pid'; like $batch->[0]{started}, qr/^[\d.]+$/, 'has timestamp'; is $batch->[1]{host}, $host, 'right host'; is $batch->[1]{pid}, $$, 'right pid'; ok !$batch->[2], 'no more results'; $batch = $minion->backend->list_workers(0, 1); is $batch->[0]{id}, $worker2->id, 'right id'; is_deeply $batch->[0]{status}, {whatever => 'works!'}, 'right status'; ok !$batch->[1], 'no more results'; $worker2->status({whatever => 'works too!'})->register; $batch = $minion->backend->list_workers(0, 1); is_deeply $batch->[0]{status}, {whatever => 'works too!'}, 'right status'; $batch = $minion->backend->list_workers(1, 1); is $batch->[0]{id}, $worker->id, 'right id'; ok !$batch->[1], 'no more results'; $worker->unregister; $worker2->unregister; # Exclusive lock ok $minion->lock('foo', 3600), 'locked'; ok !$minion->lock('foo', 3600), 'not locked again'; ok $minion->unlock('foo'), 'unlocked'; ok !$minion->unlock('foo'), 'not unlocked again'; ok $minion->lock('foo', -3600), 'locked'; ok $minion->lock('foo', 3600), 'locked again'; ok !$minion->lock('foo', -3600), 'not locked again'; ok !$minion->lock('foo', 3600), 'not locked again'; ok $minion->unlock('foo'), 'unlocked'; ok !$minion->unlock('foo'), 'not unlocked again'; ok $minion->lock('yada', 3600, {limit => 1}), 'locked'; ok !$minion->lock('yada', 3600, {limit => 1}), 'not locked again'; # Shared lock ok $minion->lock('bar', 3600, {limit => 3}), 'locked'; ok $minion->lock('bar', 3600, {limit => 3}), 'locked again'; ok $minion->lock('bar', -3600, {limit => 3}), 'locked again'; ok $minion->lock('bar', 3600, {limit => 3}), 'locked again'; ok !$minion->lock('bar', 3600, {limit => 2}), 'not locked again'; ok $minion->lock('baz', 3600, {limit => 3}), 'locked'; ok $minion->unlock('bar'), 'unlocked'; ok $minion->lock('bar', 3600, {limit => 3}), 'locked again'; ok $minion->unlock('bar'), 'unlocked again'; ok $minion->unlock('bar'), 'unlocked again'; ok $minion->unlock('bar'), 'unlocked again'; ok !$minion->unlock('bar'), 'not unlocked again'; ok $minion->unlock('baz'), 'unlocked'; ok !$minion->unlock('baz'), 'not unlocked again'; # Reset $minion->reset->repair; ok !$minion->backend->pg->db->query( 'select count(id) as count from minion_jobs')->hash->{count}, 'no jobs'; ok !$minion->backend->pg->db->query( 'select count(id) as count from minion_locks')->hash->{count}, 'no locks'; ok !$minion->backend->pg->db->query( 'select count(id) as count from minion_workers')->hash->{count}, 'no workers'; # Stats $minion->add_task( add => sub { my ($job, $first, $second) = @_; $job->finish({added => $first + $second}); } ); $minion->add_task(fail => sub { die "Intentional failure!\n" }); my $stats = $minion->stats; is $stats->{active_workers}, 0, 'no active workers'; is $stats->{inactive_workers}, 0, 'no inactive workers'; is $stats->{enqueued_jobs}, 0, 'no enqueued jobs'; is $stats->{active_jobs}, 0, 'no active jobs'; is $stats->{failed_jobs}, 0, 'no failed jobs'; is $stats->{finished_jobs}, 0, 'no finished jobs'; is $stats->{inactive_jobs}, 0, 'no inactive jobs'; is $stats->{delayed_jobs}, 0, 'no delayed jobs'; $worker = $minion->worker->register; is $minion->stats->{inactive_workers}, 1, 'one inactive worker'; $minion->enqueue('fail'); is $minion->stats->{enqueued_jobs}, 1, 'one enqueued job'; $minion->enqueue('fail'); is $minion->stats->{enqueued_jobs}, 2, 'two enqueued jobs'; is $minion->stats->{inactive_jobs}, 2, 'two inactive jobs'; $job = $worker->dequeue(0); $stats = $minion->stats; is $stats->{active_workers}, 1, 'one active worker'; is $stats->{active_jobs}, 1, 'one active job'; is $stats->{inactive_jobs}, 1, 'one inactive job'; $minion->enqueue('fail'); my $job2 = $worker->dequeue(0); $stats = $minion->stats; is $stats->{active_workers}, 1, 'one active worker'; is $stats->{active_jobs}, 2, 'two active jobs'; is $stats->{inactive_jobs}, 1, 'one inactive job'; ok $job2->finish, 'job finished'; ok $job->finish, 'job finished'; is $minion->stats->{finished_jobs}, 2, 'two finished jobs'; $job = $worker->dequeue(0); ok $job->fail, 'job failed'; is $minion->stats->{failed_jobs}, 1, 'one failed job'; ok $job->retry, 'job retried'; is $minion->stats->{failed_jobs}, 0, 'no failed jobs'; ok $worker->dequeue(0)->finish(['works']), 'job finished'; $worker->unregister; $stats = $minion->stats; is $stats->{active_workers}, 0, 'no active workers'; is $stats->{inactive_workers}, 0, 'no inactive workers'; is $stats->{active_jobs}, 0, 'no active jobs'; is $stats->{failed_jobs}, 0, 'no failed jobs'; is $stats->{finished_jobs}, 3, 'three finished jobs'; is $stats->{inactive_jobs}, 0, 'no inactive jobs'; is $stats->{delayed_jobs}, 0, 'no delayed jobs'; # List jobs $id = $minion->enqueue('add'); $batch = $minion->backend->list_jobs(0, 10); ok $batch->[0]{id}, 'has id'; is $batch->[0]{task}, 'add', 'right task'; is $batch->[0]{state}, 'inactive', 'right state'; is $batch->[0]{retries}, 0, 'job has not been retried'; like $batch->[0]{created}, qr/^[\d.]+$/, 'has created timestamp'; is $batch->[1]{task}, 'fail', 'right task'; is_deeply $batch->[1]{args}, [], 'right arguments'; is_deeply $batch->[1]{notes}, {}, 'right metadata'; is_deeply $batch->[1]{result}, ['works'], 'right result'; is $batch->[1]{state}, 'finished', 'right state'; is $batch->[1]{priority}, 0, 'right priority'; is_deeply $batch->[1]{parents}, [], 'right parents'; is_deeply $batch->[1]{children}, [], 'right children'; is $batch->[1]{retries}, 1, 'job has been retried'; like $batch->[1]{created}, qr/^[\d.]+$/, 'has created timestamp'; like $batch->[1]{delayed}, qr/^[\d.]+$/, 'has delayed timestamp'; like $batch->[1]{finished}, qr/^[\d.]+$/, 'has finished timestamp'; like $batch->[1]{retried}, qr/^[\d.]+$/, 'has retried timestamp'; like $batch->[1]{started}, qr/^[\d.]+$/, 'has started timestamp'; is $batch->[2]{task}, 'fail', 'right task'; is $batch->[2]{state}, 'finished', 'right state'; is $batch->[2]{retries}, 0, 'job has not been retried'; is $batch->[3]{task}, 'fail', 'right task'; is $batch->[3]{state}, 'finished', 'right state'; is $batch->[3]{retries}, 0, 'job has not been retried'; ok !$batch->[4], 'no more results'; $batch = $minion->backend->list_jobs(0, 10, {state => 'inactive'}); is $batch->[0]{state}, 'inactive', 'right state'; is $batch->[0]{retries}, 0, 'job has not been retried'; ok !$batch->[1], 'no more results'; $batch = $minion->backend->list_jobs(0, 10, {task => 'add'}); is $batch->[0]{task}, 'add', 'right task'; is $batch->[0]{retries}, 0, 'job has not been retried'; ok !$batch->[1], 'no more results'; $batch = $minion->backend->list_jobs(0, 10, {queue => 'default'}); is $batch->[0]{queue}, 'default', 'right queue'; is $batch->[1]{queue}, 'default', 'right queue'; is $batch->[2]{queue}, 'default', 'right queue'; is $batch->[3]{queue}, 'default', 'right queue'; ok !$batch->[4], 'no more results'; $batch = $minion->backend->list_jobs(0, 10, {queue => 'does_not_exist'}); is_deeply $batch, [], 'no results'; $batch = $minion->backend->list_jobs(0, 1); is $batch->[0]{state}, 'inactive', 'right state'; is $batch->[0]{retries}, 0, 'job has not been retried'; ok !$batch->[1], 'no more results'; $batch = $minion->backend->list_jobs(1, 1); is $batch->[0]{state}, 'finished', 'right state'; is $batch->[0]{retries}, 1, 'job has been retried'; ok !$batch->[1], 'no more results'; ok $minion->job($id)->remove, 'job removed'; # Enqueue, dequeue and perform is $minion->job(12345), undef, 'job does not exist'; $id = $minion->enqueue(add => [2, 2]); ok $minion->job($id), 'job does exist'; my $info = $minion->job($id)->info; is_deeply $info->{args}, [2, 2], 'right arguments'; is $info->{priority}, 0, 'right priority'; is $info->{state}, 'inactive', 'right state'; $worker = $minion->worker; is $worker->dequeue(0), undef, 'not registered'; ok !$minion->job($id)->info->{started}, 'no started timestamp'; $job = $worker->register->dequeue(0); is $worker->info->{jobs}[0], $job->id, 'right job'; like $job->info->{created}, qr/^[\d.]+$/, 'has created timestamp'; like $job->info->{started}, qr/^[\d.]+$/, 'has started timestamp'; is_deeply $job->args, [2, 2], 'right arguments'; is $job->info->{state}, 'active', 'right state'; is $job->task, 'add', 'right task'; is $job->retries, 0, 'job has not been retried'; $id = $job->info->{worker}; is $minion->backend->worker_info($id)->{pid}, $$, 'right worker'; ok !$job->info->{finished}, 'no finished timestamp'; $job->perform; is $worker->info->{jobs}[0], undef, 'no jobs'; like $job->info->{finished}, qr/^[\d.]+$/, 'has finished timestamp'; is_deeply $job->info->{result}, {added => 4}, 'right result'; is $job->info->{state}, 'finished', 'right state'; $worker->unregister; $job = $minion->job($job->id); is_deeply $job->args, [2, 2], 'right arguments'; is $job->retries, 0, 'job has not been retried'; is $job->info->{state}, 'finished', 'right state'; is $job->task, 'add', 'right task'; # Retry and remove $id = $minion->enqueue(add => [5, 6]); $job = $worker->register->dequeue(0); is $job->info->{attempts}, 1, 'job will be attempted once'; is $job->info->{retries}, 0, 'job has not been retried'; is $job->id, $id, 'right id'; ok $job->finish, 'job finished'; ok !$worker->dequeue(0), 'no more jobs'; $job = $minion->job($id); ok !$job->info->{retried}, 'no retried timestamp'; ok $job->retry, 'job retried'; like $job->info->{retried}, qr/^[\d.]+$/, 'has retried timestamp'; is $job->info->{state}, 'inactive', 'right state'; is $job->info->{retries}, 1, 'job has been retried once'; $job = $worker->dequeue(0); is $job->retries, 1, 'job has been retried once'; ok $job->retry, 'job retried'; is $job->id, $id, 'right id'; is $job->info->{retries}, 2, 'job has been retried twice'; $job = $worker->dequeue(0); is $job->info->{state}, 'active', 'right state'; ok $job->finish, 'job finished'; ok $job->remove, 'job has been removed'; ok !$job->retry, 'job not retried'; is $job->info, undef, 'no information'; $id = $minion->enqueue(add => [6, 5]); $job = $minion->job($id); is $job->info->{state}, 'inactive', 'right state'; is $job->info->{retries}, 0, 'job has not been retried'; ok $job->retry, 'job retried'; is $job->info->{state}, 'inactive', 'right state'; is $job->info->{retries}, 1, 'job has been retried once'; $job = $worker->dequeue(0); is $job->id, $id, 'right id'; ok $job->fail, 'job failed'; ok $job->remove, 'job has been removed'; is $job->info, undef, 'no information'; $id = $minion->enqueue(add => [5, 5]); $job = $minion->job("$id"); ok $job->remove, 'job has been removed'; $worker->unregister; # Jobs with priority $minion->enqueue(add => [1, 2]); $id = $minion->enqueue(add => [2, 4], {priority => 1}); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{priority}, 1, 'right priority'; ok $job->finish, 'job finished'; isnt $worker->dequeue(0)->id, $id, 'different id'; $id = $minion->enqueue(add => [2, 5]); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{priority}, 0, 'right priority'; ok $job->finish, 'job finished'; ok $job->retry({priority => 100}), 'job retried with higher priority'; $job = $worker->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{retries}, 1, 'job has been retried once'; is $job->info->{priority}, 100, 'high priority'; ok $job->finish, 'job finished'; ok $job->retry({priority => 0}), 'job retried with lower priority'; $job = $worker->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{retries}, 2, 'job has been retried twice'; is $job->info->{priority}, 0, 'low priority'; ok $job->finish, 'job finished'; $worker->unregister; # Delayed jobs $id = $minion->enqueue(add => [2, 1] => {delay => 100}); is $minion->stats->{delayed_jobs}, 1, 'one delayed job'; is $worker->register->dequeue(0), undef, 'too early for job'; ok $minion->job($id)->info->{delayed} > time, 'delayed timestamp'; $minion->backend->pg->db->query( "update minion_jobs set delayed = now() - interval '1 day' where id = ?", $id); $job = $worker->dequeue(0); is $job->id, $id, 'right id'; like $job->info->{delayed}, qr/^[\d.]+$/, 'has delayed timestamp'; ok $job->finish, 'job finished'; ok $job->retry, 'job retried'; ok $minion->job($id)->info->{delayed} < time, 'no delayed timestamp'; ok $job->remove, 'job removed'; ok !$job->retry, 'job not retried'; $id = $minion->enqueue(add => [6, 9]); $job = $worker->dequeue(0); ok $job->info->{delayed} < time, 'no delayed timestamp'; ok $job->fail, 'job failed'; ok $job->retry({delay => 100}), 'job retried with delay'; is $job->info->{retries}, 1, 'job has been retried once'; ok $job->info->{delayed} > time, 'delayed timestamp'; ok $minion->job($id)->remove, 'job has been removed'; $worker->unregister; # Events my ($enqueue, $pid); my $failed = $finished = 0; $minion->once(enqueue => sub { $enqueue = pop }); $minion->once( worker => sub { my ($minion, $worker) = @_; $worker->on( dequeue => sub { my ($worker, $job) = @_; $job->on(failed => sub { $failed++ }); $job->on(finished => sub { $finished++ }); $job->on(spawn => sub { $pid = pop }); $job->on( start => sub { my $job = shift; return unless $job->task eq 'switcheroo'; $job->task('add')->args->[-1] += 1; } ); } ); } ); $worker = $minion->worker->register; $id = $minion->enqueue(add => [3, 3]); is $enqueue, $id, 'enqueue event has been emitted'; $minion->enqueue(add => [4, 3]); $job = $worker->dequeue(0); is $failed, 0, 'failed event has not been emitted'; is $finished, 0, 'finished event has not been emitted'; my $result; $job->on(finished => sub { $result = pop }); ok $job->finish('Everything is fine!'), 'job finished'; $job->perform; is $result, 'Everything is fine!', 'right result'; is $failed, 0, 'failed event has not been emitted'; is $finished, 1, 'finished event has been emitted once'; isnt $pid, $$, 'new process id'; $job = $worker->dequeue(0); my $err; $job->on(failed => sub { $err = pop }); $job->fail("test\n"); $job->fail; is $err, "test\n", 'right error'; is $failed, 1, 'failed event has been emitted once'; is $finished, 1, 'finished event has been emitted once'; $minion->add_task(switcheroo => sub { }); $minion->enqueue(switcheroo => [5, 3]); $job = $worker->dequeue(0); $job->perform; is_deeply $job->info->{result}, {added => 9}, 'right result'; $worker->unregister; # Queues $id = $minion->enqueue(add => [100, 1]); is $worker->register->dequeue(0 => {queues => ['test1']}), undef, 'wrong queue'; $job = $worker->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{queue}, 'default', 'right queue'; ok $job->finish, 'job finished'; $id = $minion->enqueue(add => [100, 3] => {queue => 'test1'}); is $worker->dequeue(0), undef, 'wrong queue'; $job = $worker->dequeue(0 => {queues => ['test1']}); is $job->id, $id, 'right id'; is $job->info->{queue}, 'test1', 'right queue'; ok $job->finish, 'job finished'; ok $job->retry({queue => 'test2'}), 'job retried'; $job = $worker->dequeue(0 => {queues => ['default', 'test2']}); is $job->id, $id, 'right id'; is $job->info->{queue}, 'test2', 'right queue'; ok $job->finish, 'job finished'; $worker->unregister; # Failed jobs $id = $minion->enqueue(add => [5, 6]); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->info->{result}, undef, 'no result'; ok $job->fail, 'job failed'; ok !$job->finish, 'job not finished'; is $job->info->{state}, 'failed', 'right state'; is $job->info->{result}, 'Unknown error', 'right result'; $id = $minion->enqueue(add => [6, 7]); $job = $worker->dequeue(0); is $job->id, $id, 'right id'; ok $job->fail('Something bad happened!'), 'job failed'; is $job->info->{state}, 'failed', 'right state'; is $job->info->{result}, 'Something bad happened!', 'right result'; $id = $minion->enqueue('fail'); $job = $worker->dequeue(0); is $job->id, $id, 'right id'; $job->perform; is $job->info->{state}, 'failed', 'right state'; is $job->info->{result}, "Intentional failure!\n", 'right result'; $worker->unregister; # Nested data structures $minion->add_task( nested => sub { my ($job, $hash, $array) = @_; $job->note(bar => {baz => [1, 2, 3]}); $job->note(baz => 'yada'); $job->finish([{23 => $hash->{first}[0]{second} x $array->[0][0]}]); } ); $minion->enqueue( 'nested', [{first => [{second => 'test'}]}, [[3]]], {notes => {foo => [4, 5, 6]}} ); $job = $worker->register->dequeue(0); $job->perform; is $job->info->{state}, 'finished', 'right state'; ok $job->note(yada => ['works']), 'added metadata'; ok !$minion->backend->note(-1, yada => ['failed']), 'not added metadata'; my $notes = { foo => [4, 5, 6], bar => {baz => [1, 2, 3]}, baz => 'yada', yada => ['works'] }; is_deeply $job->info->{notes}, $notes, 'right metadata'; is_deeply $job->info->{result}, [{23 => 'testtesttest'}], 'right structure'; $worker->unregister; # Perform job in a running event loop $id = $minion->enqueue(add => [8, 9]); Mojo::IOLoop->delay(sub { $minion->perform_jobs })->wait; is $minion->job($id)->info->{state}, 'finished', 'right state'; is_deeply $minion->job($id)->info->{result}, {added => 17}, 'right result'; # Non-zero exit status $minion->add_task(exit => sub { exit 1 }); $id = $minion->enqueue('exit'); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; $job->perform; is $job->info->{state}, 'failed', 'right state'; is $job->info->{result}, 'Non-zero exit status (1)', 'right result'; $worker->unregister; # Multiple attempts while processing is $minion->backoff->(0), 15, 'right result'; is $minion->backoff->(1), 16, 'right result'; is $minion->backoff->(2), 31, 'right result'; is $minion->backoff->(3), 96, 'right result'; is $minion->backoff->(4), 271, 'right result'; is $minion->backoff->(5), 640, 'right result'; is $minion->backoff->(25), 390640, 'right result'; $id = $minion->enqueue(exit => [] => {attempts => 2}); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->retries, 0, 'job has not been retried'; $job->perform; $info = $job->info; is $info->{attempts}, 2, 'job will be attempted twice'; is $info->{state}, 'inactive', 'right state'; is $info->{result}, 'Non-zero exit status (1)', 'right result'; ok $info->{retried} < $job->info->{delayed}, 'delayed timestamp'; $minion->backend->pg->db->query( 'update minion_jobs set delayed = now() where id = ?', $id); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->retries, 1, 'job has been retried once'; $job->perform; $info = $job->info; is $info->{attempts}, 2, 'job will be attempted twice'; is $info->{state}, 'failed', 'right state'; is $info->{result}, 'Non-zero exit status (1)', 'right result'; ok $job->retry({attempts => 3}), 'job retried'; $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; $job->perform; $info = $job->info; is $info->{attempts}, 3, 'job will be attempted three times'; is $info->{state}, 'failed', 'right state'; is $info->{result}, 'Non-zero exit status (1)', 'right result'; $worker->unregister; # Multiple attempts during maintenance $id = $minion->enqueue(exit => [] => {attempts => 2}); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->retries, 0, 'job has not been retried'; is $job->info->{attempts}, 2, 'job will be attempted twice'; is $job->info->{state}, 'active', 'right state'; $worker->unregister; $minion->repair; is $job->info->{state}, 'inactive', 'right state'; is $job->info->{result}, 'Worker went away', 'right result'; ok $job->info->{retried} < $job->info->{delayed}, 'delayed timestamp'; $minion->backend->pg->db->query( 'update minion_jobs set delayed = now() where id = ?', $id); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; is $job->retries, 1, 'job has been retried once'; $worker->unregister; $minion->repair; is $job->info->{state}, 'failed', 'right state'; is $job->info->{result}, 'Worker went away', 'right result'; # A job needs to be dequeued again after a retry $minion->add_task(restart => sub { }); $id = $minion->enqueue('restart'); $job = $worker->register->dequeue(0); is $job->id, $id, 'right id'; ok $job->finish, 'job finished'; is $job->info->{state}, 'finished', 'right state'; ok $job->retry, 'job retried'; is $job->info->{state}, 'inactive', 'right state'; $job2 = $worker->dequeue(0); is $job->info->{state}, 'active', 'right state'; ok !$job->finish, 'job not finished'; is $job->info->{state}, 'active', 'right state'; is $job2->id, $id, 'right id'; ok $job2->finish, 'job finished'; ok !$job->retry, 'job not retried'; is $job->info->{state}, 'finished', 'right state'; $worker->unregister; # Perform jobs concurrently $id = $minion->enqueue(add => [10, 11]); $id2 = $minion->enqueue(add => [12, 13]); $id3 = $minion->enqueue('test'); my $id4 = $minion->enqueue('exit'); $worker = $minion->worker->register; $job = $worker->dequeue(0); $job2 = $worker->dequeue(0); my $job3 = $worker->dequeue(0); my $job4 = $worker->dequeue(0); $job->start; $job2->start; $job3->start; $job4->start; my ($first, $second, $third, $fourth); usleep 50000 until $first ||= $job->is_finished and $second ||= $job2->is_finished and $third ||= $job3->is_finished and $fourth ||= $job4->is_finished; is $minion->job($id)->info->{state}, 'finished', 'right state'; is_deeply $minion->job($id)->info->{result}, {added => 21}, 'right result'; is $minion->job($id2)->info->{state}, 'finished', 'right state'; is_deeply $minion->job($id2)->info->{result}, {added => 25}, 'right result'; is $minion->job($id3)->info->{state}, 'finished', 'right state'; is $minion->job($id3)->info->{result}, undef, 'no result'; is $minion->job($id4)->info->{state}, 'failed', 'right state'; is $minion->job($id4)->info->{result}, 'Non-zero exit status (1)', 'right result'; $worker->unregister; # Stopping jobs $minion->add_task(long_running => sub { sleep 1000 }); $worker = $minion->worker->register; $minion->enqueue('long_running'); $job = $worker->dequeue(0); ok $job->start->pid, 'has a process id'; ok !$job->is_finished, 'job is not finished'; $job->stop; usleep 5000 until $job->is_finished; is $job->info->{state}, 'failed', 'right state'; like $job->info->{result}, qr/Non-zero exit status/, 'right result'; $worker->unregister; # Job dependencies $worker = $minion->remove_after(0)->worker->register; is $minion->repair->stats->{finished_jobs}, 0, 'no finished jobs'; $id = $minion->enqueue('test'); $id2 = $minion->enqueue('test'); $id3 = $minion->enqueue(test => [] => {parents => [$id, $id2]}); is $minion->stats->{delayed_jobs}, 1, 'one delayed job'; $job = $worker->dequeue(0); is $job->id, $id, 'right id'; is_deeply $job->info->{children}, [$id3], 'right children'; is_deeply $job->info->{parents}, [], 'right parents'; $job2 = $worker->dequeue(0); is $job2->id, $id2, 'right id'; is_deeply $job2->info->{children}, [$id3], 'right children'; is_deeply $job2->info->{parents}, [], 'right parents'; ok !$worker->dequeue(0), 'parents are not ready yet'; ok $job->finish, 'job finished'; ok !$worker->dequeue(0), 'parents are not ready yet'; ok $job2->fail, 'job failed'; ok !$worker->dequeue(0), 'parents are not ready yet'; ok $job2->retry, 'job retried'; $job2 = $worker->dequeue(0); is $job2->id, $id2, 'right id'; ok $job2->finish, 'job finished'; $job = $worker->dequeue(0); is $job->id, $id3, 'right id'; is_deeply $job->info->{children}, [], 'right children'; is_deeply $job->info->{parents}, [$id, $id2], 'right parents'; is $minion->stats->{finished_jobs}, 2, 'two finished jobs'; is $minion->repair->stats->{finished_jobs}, 2, 'two finished jobs'; ok $job->finish, 'job finished'; is $minion->stats->{finished_jobs}, 3, 'three finished jobs'; is $minion->repair->stats->{finished_jobs}, 0, 'no finished jobs'; $id = $minion->enqueue(test => [] => {parents => [-1]}); $job = $worker->dequeue(0); is $job->id, $id, 'right id'; ok $job->finish, 'job finished'; $worker->unregister; # Foreground $id = $minion->enqueue(test => [] => {attempts => 2}); $id2 = $minion->enqueue('test'); $id3 = $minion->enqueue(test => [] => {parents => [$id, $id2]}); ok !$minion->foreground($id3 + 1), 'job does not exist'; ok !$minion->foreground($id3), 'job is not ready yet'; $info = $minion->job($id)->info; is $info->{attempts}, 2, 'job will be attempted twice'; is $info->{state}, 'inactive', 'right state'; is $info->{queue}, 'default', 'right queue'; ok $minion->foreground($id), 'performed first job'; $info = $minion->job($id)->info; is $info->{attempts}, 1, 'job will be attempted once'; is $info->{retries}, 1, 'job has been retried'; is $info->{state}, 'finished', 'right state'; is $info->{queue}, 'minion_foreground', 'right queue'; ok $minion->foreground($id2), 'performed second job'; $info = $minion->job($id2)->info; is $info->{retries}, 1, 'job has been retried'; is $info->{state}, 'finished', 'right state'; is $info->{queue}, 'minion_foreground', 'right queue'; ok $minion->foreground($id3), 'performed third job'; $info = $minion->job($id3)->info; is $info->{retries}, 2, 'job has been retried twice'; is $info->{state}, 'finished', 'right state'; is $info->{queue}, 'minion_foreground', 'right queue'; $id = $minion->enqueue('fail'); eval { $minion->foreground($id) }; like $@, qr/Intentional failure!/, 'right error'; $info = $minion->job($id)->info; ok $info->{worker}, 'has worker'; ok !$minion->backend->worker_info($info->{worker}), 'not registered'; is $info->{retries}, 1, 'job has been retried'; is $info->{state}, 'failed', 'right state'; is $info->{queue}, 'minion_foreground', 'right queue'; is $info->{result}, "Intentional failure!\n", 'right result'; # Worker remote control commands $worker = $minion->worker->register->process_commands; $worker2 = $minion->worker->register; my @commands; $_->add_command(test_id => sub { push @commands, shift->id }) for $worker, $worker2; $worker->add_command(test_args => sub { shift and push @commands, [@_] }) ->register; ok $minion->backend->broadcast('test_id', [], [$worker->id]), 'sent command'; ok $minion->backend->broadcast('test_id', [], [$worker->id, $worker2->id]), 'sent command'; $worker->process_commands->register; $worker2->process_commands; is_deeply \@commands, [$worker->id, $worker->id, $worker2->id], 'right structure'; @commands = (); ok $minion->backend->broadcast('test_id'), 'sent command'; ok $minion->backend->broadcast('test_whatever'), 'sent command'; ok $minion->backend->broadcast('test_args', [23], []), 'sent command'; ok $minion->backend->broadcast('test_args', [1, [2], {3 => 'three'}], [$worker->id]), 'sent command'; $_->process_commands for $worker, $worker2; is_deeply \@commands, [$worker->id, [23], [1, [2], {3 => 'three'}], $worker2->id], 'right structure'; $_->unregister for $worker, $worker2; ok !$minion->backend->broadcast('test_id', []), 'command not sent'; # Clean up once we are done $pg->db->query('drop schema minion_test cascade'); done_testing(); Minion-7.05/t/pg_lite_app.t000644 000765 000024 00000003666 13127475654 015602 0ustar00sristaff000000 000000 use Mojo::Base -strict; BEGIN { $ENV{MOJO_REACTOR} = 'Mojo::Reactor::Poll' } use Test::More; plan skip_all => 'set TEST_ONLINE to enable this test' unless $ENV{TEST_ONLINE}; use Mojo::IOLoop; use Mojolicious::Lite; use Test::Mojo; # Missing backend eval { plugin Minion => {Something => 'fun'} }; like $@, qr/^Backend "Minion::Backend::Something" missing/, 'right error'; # Isolate tests require Mojo::Pg; my $pg = Mojo::Pg->new($ENV{TEST_ONLINE}); $pg->db->query('drop schema if exists minion_app_test cascade'); $pg->db->query('create schema minion_app_test'); plugin Minion => {Pg => $pg}; app->minion->add_task( add => sub { my ($job, $first, $second) = @_; Mojo::IOLoop->next_tick( sub { $job->finish($first + $second); Mojo::IOLoop->stop; } ); Mojo::IOLoop->start; } ); get '/add' => sub { my $c = shift; my $id = $c->minion->enqueue( add => [$c->param('first'), $c->param('second')] => {queue => 'test'}); $c->render(text => $id); }; get '/result' => sub { my $c = shift; $c->render(text => $c->minion->job($c->param('id'))->info->{result}); }; my $t = Test::Mojo->new; # Perform jobs automatically $t->get_ok('/add' => form => {first => 1, second => 2})->status_is(200); $t->app->minion->perform_jobs({queues => ['test']}); $t->get_ok('/result' => form => {id => $t->tx->res->text})->status_is(200) ->content_is('3'); $t->get_ok('/add' => form => {first => 2, second => 3})->status_is(200); my $first = $t->tx->res->text; $t->get_ok('/add' => form => {first => 4, second => 5})->status_is(200); my $second = $t->tx->res->text; Mojo::IOLoop->delay(sub { $t->app->minion->perform_jobs({queues => ['test']}) }) ->wait; $t->get_ok('/result' => form => {id => $first})->status_is(200) ->content_is('5'); $t->get_ok('/result' => form => {id => $second})->status_is(200) ->content_is('9'); # Clean up once we are done $pg->db->query('drop schema minion_app_test cascade'); done_testing(); Minion-7.05/t/pod.t000644 000765 000024 00000000400 13123461250 014037 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Test::More; plan skip_all => 'set TEST_POD to enable this test (developer only!)' unless $ENV{TEST_POD}; plan skip_all => 'Test::Pod 1.14+ required for this test!' unless eval 'use Test::Pod 1.14; 1'; all_pod_files_ok(); Minion-7.05/t/pod_coverage.t000644 000765 000024 00000000427 13123461250 015723 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Test::More; plan skip_all => 'set TEST_POD to enable this test (developer only!)' unless $ENV{TEST_POD}; plan skip_all => 'Test::Pod::Coverage 1.04+ required for this test!' unless eval 'use Test::Pod::Coverage 1.04; 1'; all_pod_coverage_ok(); Minion-7.05/lib/Minion/000755 000765 000024 00000000000 13142141670 014634 5ustar00sristaff000000 000000 Minion-7.05/lib/Minion.pm000644 000765 000024 00000036053 13142105526 015201 0ustar00sristaff000000 000000 package Minion; use Mojo::Base 'Mojo::EventEmitter'; use Carp 'croak'; use Config; use Minion::Job; use Minion::Worker; use Mojo::Loader 'load_class'; use Mojo::Server; use Scalar::Util 'weaken'; has app => sub { Mojo::Server->new->build_app('Mojo::HelloWorld') }; has 'backend'; has backoff => sub { \&_backoff }; has missing_after => 1800; has remove_after => 172800; has tasks => sub { {} }; our $VERSION = '7.05'; sub add_task { ($_[0]->tasks->{$_[1]} = $_[2]) and return $_[0] } sub enqueue { my $self = shift; my $id = $self->backend->enqueue(@_); $self->emit(enqueue => $id); return $id; } sub foreground { my ($self, $id) = @_; return undef unless my $job = $self->job($id); return undef unless $job->retry({attempts => 1, queue => 'minion_foreground'}); my $worker = $self->worker->register; $job = $worker->dequeue(0 => {id => $id, queues => ['minion_foreground']}); my $err; if ($job) { $job->finish unless defined($err = $job->_run) } $worker->unregister; return defined $err ? die $err : !!$job; } sub job { my ($self, $id) = @_; return undef unless my $job = $self->backend->job_info($id); return Minion::Job->new( args => $job->{args}, id => $job->{id}, minion => $self, retries => $job->{retries}, task => $job->{task} ); } sub lock { shift->backend->lock(@_) } sub new { my $self = shift->SUPER::new; my $class = 'Minion::Backend::' . shift; my $e = load_class $class; croak ref $e ? $e : qq{Backend "$class" missing} if $e; $self->backend($class->new(@_)); weaken $self->backend->minion($self)->{minion}; return $self; } sub perform_jobs { my ($self, $options) = @_; my $worker = $self->worker; while (my $job = $worker->register->dequeue(0, $options)) { $job->perform } $worker->unregister; } sub repair { shift->_delegate('repair') } sub reset { shift->_delegate('reset') } sub stats { shift->backend->stats } sub unlock { shift->backend->unlock(@_) } sub worker { my $self = shift; # No fork emulation support croak 'Minion workers do not support fork emulation' if $Config{d_pseudofork}; my $worker = Minion::Worker->new(minion => $self); $self->emit(worker => $worker); return $worker; } sub _backoff { (shift()**4) + 15 } sub _delegate { my ($self, $method) = @_; $self->backend->$method; return $self; } 1; =encoding utf8 =head1 NAME Minion - Job queue =head1 SYNOPSIS use Minion; # Connect to backend my $minion = Minion->new(Pg => 'postgresql://postgres@/test'); # Add tasks $minion->add_task(something_slow => sub { my ($job, @args) = @_; sleep 5; say 'This is a background worker process.'; }); # Enqueue jobs $minion->enqueue(something_slow => ['foo', 'bar']); $minion->enqueue(something_slow => [1, 2, 3] => {priority => 5}); # Perform jobs for testing $minion->enqueue(something_slow => ['foo', 'bar']); $minion->perform_jobs; # Build more sophisticated workers my $worker = $minion->repair->worker; while (int rand 2) { if (my $job = $worker->register->dequeue(5)) { $job->perform } } $worker->unregister; =head1 DESCRIPTION L is a job queue for the L real-time web framework, with support for multiple named queues, priorities, delayed jobs, job dependencies, job progress, job results, retries with backoff, rate limiting, unique jobs, statistics, distributed workers, parallel processing, autoscaling, remote control, resource leak protection and multiple backends (such as L). Job queues allow you to process time and/or computationally intensive tasks in background processes, outside of the request/response lifecycle. Among those tasks you'll commonly find image resizing, spam filtering, HTTP downloads, building tarballs, warming caches and basically everything else you can imagine that's not super fast. use Mojolicious::Lite; plugin Minion => {Pg => 'postgresql://sri:s3cret@localhost/test'}; # Slow task app->minion->add_task(poke_mojo => sub { my $job = shift; $job->app->ua->get('mojolicious.org'); $job->app->log->debug('We have poked mojolicious.org for a visitor'); }); # Perform job in a background worker process get '/' => sub { my $c = shift; $c->minion->enqueue('poke_mojo'); $c->render(text => 'We will poke mojolicious.org for you soon.'); }; app->start; Background worker processes are usually started with the command L, which becomes automatically available when an application loads the plugin L. $ ./myapp.pl minion worker Jobs can be managed right from the command line with L. $ ./myapp.pl minion job To manage background worker processes with systemd, you can use a unit configuration file like this. [Unit] Description=My Mojolicious application workers After=postgresql.service [Service] Type=simple ExecStart=/home/sri/myapp/myapp.pl minion worker -m production KillMode=process [Install] WantedBy=multi-user.target Every job can fail or succeed, but not get lost, the system is eventually consistent and will preserve job results for as long as you like, depending on L. While individual workers can fail in the middle of processing a job, the system will detect this and ensure that no job is left in an uncertain state, depending on L. =head1 GROWING And as your application grows, you can move tasks into application specific plugins. package MyApp::Task::PokeMojo; use Mojo::Base 'Mojolicious::Plugin'; sub register { my ($self, $app) = @_; $app->minion->add_task(poke_mojo => sub { my $job = shift; $job->app->ua->get('mojolicious.org'); $job->app->log->debug('We have poked mojolicious.org for a visitor'); }); } 1; Which are loaded like any other plugin from your application. # Mojolicious $app->plugin('MyApp::Task::PokeMojo'); # Mojolicious::Lite plugin 'MyApp::Task::PokeMojo'; =head1 EXAMPLES This distribution also contains a great example application you can use for inspiration. The L will show you how to integrate background jobs into well-structured L applications. =head1 EVENTS L inherits all events from L and can emit the following new ones. =head2 enqueue $minion->on(enqueue => sub { my ($minion, $id) = @_; ... }); Emitted after a job has been enqueued, in the process that enqueued it. $minion->on(enqueue => sub { my ($minion, $id) = @_; say "Job $id has been enqueued."; }); =head2 worker $minion->on(worker => sub { my ($minion, $worker) = @_; ... }); Emitted in the worker process after it has been created. $minion->on(worker => sub { my ($minion, $worker) = @_; my $id = $worker->id; say "Worker $$:$id started."; }); =head1 ATTRIBUTES L implements the following attributes. =head2 app my $app = $minion->app; $minion = $minion->app(MyApp->new); Application for job queue, defaults to a L object. =head2 backend my $backend = $minion->backend; $minion = $minion->backend(Minion::Backend::Pg->new); Backend, usually a L object. =head2 backoff my $cb = $minion->backoff; $minion = $minion->backoff(sub {...}); A callback used to calculate the delay for automatically retried jobs, defaults to C<(retries ** 4) + 15> (15, 16, 31, 96, 271, 640...), which means that roughly C<25> attempts can be made in C<21> days. $minion->backoff(sub { my $retries = shift; return ($retries ** 4) + 15 + int(rand 30); }); =head2 missing_after my $after = $minion->missing_after; $minion = $minion->missing_after(172800); Amount of time in seconds after which workers without a heartbeat will be considered missing and removed from the registry by L, defaults to C<1800> (30 minutes). =head2 remove_after my $after = $minion->remove_after; $minion = $minion->remove_after(86400); Amount of time in seconds after which jobs that have reached the state C and have no unresolved dependencies will be removed automatically by L, defaults to C<172800> (2 days). =head2 tasks my $tasks = $minion->tasks; $minion = $minion->tasks({foo => sub {...}}); Registered tasks. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 add_task $minion = $minion->add_task(foo => sub {...}); Register a task. # Job with result $minion->add_task(add => sub { my ($job, $first, $second) = @_; $job->finish($first + $second); }); my $id = $minion->enqueue(add => [1, 1]); my $result = $minion->job($id)->info->{result}; =head2 enqueue my $id = $minion->enqueue('foo'); my $id = $minion->enqueue(foo => [@args]); my $id = $minion->enqueue(foo => [@args] => {priority => 1}); Enqueue a new job with C state. Arguments get serialized by the L (often with L), so you shouldn't send objects and be careful with binary data, nested data structures with hash and array references are fine though. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted, with a delay based on L after the first attempt, defaults to C<1>. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job that gets serialized by the L (often with L), so you shouldn't send objects and be careful with binary data, nested data structures with hash and array references are fine though. =item parents parents => [$id1, $id2, $id3] One or more existing jobs this job depends on, and that need to have transitioned to the state C before it can be processed. =item priority priority => 5 Job priority, defaults to C<0>. Jobs with a higher priority get performed first. =item queue queue => 'important' Queue to put job in, defaults to C. =back =head2 foreground my $bool = $minion->foreground($id); Retry job in C queue, then perform it right away with a temporary worker in this process, very useful for debugging. =head2 job my $job = $minion->job($id); Get L object without making any changes to the actual job or return C if job does not exist. # Check job state my $state = $minion->job($id)->info->{state}; # Get job metadata my $progress = $minion->$job($id)->info->{notes}{progress}; # Get job result my $result = $minion->job($id)->info->{result}; =head2 lock my $bool = $minion->lock('foo', 3600); my $bool = $minion->lock('foo', 3600, {limit => 20}); Try to acquire a named lock that will expire automatically after the given amount of time in seconds. You can release the lock manually with L to limit concurrency, or let it expire for rate limiting. # Only one job should run at a time (unique job) $minion->add_task(do_unique_stuff => sub { my ($job, @args) = @_; return $job->finish('Previous job is still active') unless $minion->lock('fragile_backend_service', 7200); ... $minion->unlock('fragile_backend_service'); }); # Only five jobs should run at a time and we wait for our turn $minion->add_task(do_concurrent_stuff => sub { my ($job, @args) = @_; sleep 1 until $minion->lock('some_web_service', 60, {limit => 5}); ... $minion->unlock('some_web_service'); }); # Only a hundred jobs should run per hour and we try again later if necessary $minion->add_task(do_rate_limited_stuff => sub { my ($job, @args) = @_; return $job->retry({delay => 3600}) unless $minion->lock('another_web_service', 3600, {limit => 100}); ... }); These options are currently available: =over 2 =item limit limit => 20 Number of shared locks with the same name that can be active at the same time, defaults to C<1>. =back =head2 new my $minion = Minion->new(Pg => 'postgresql://postgres@/test'); my $minion = Minion->new(Pg => Mojo::Pg->new); Construct a new L object. =head2 perform_jobs $minion->perform_jobs; $minion->perform_jobs({queues => ['important']}); Perform all jobs with a temporary worker, very useful for testing. # Longer version my $worker = $minion->worker; while (my $job = $worker->register->dequeue(0)) { $job->perform } $worker->unregister; These options are currently available: =over 2 =item queues queues => ['important'] One or more queues to dequeue jobs from, defaults to C. =back =head2 repair $minion = $minion->repair; Repair worker registry and job queue if necessary. =head2 reset $minion = $minion->reset; Reset job queue. =head2 stats my $stats = $minion->stats; Get statistics for jobs and workers. # Check idle workers my $idle = $minion->stats->{inactive_workers}; These fields are currently available: =over 2 =item active_jobs active_jobs => 100 Number of jobs in C state. =item active_workers active_workers => 100 Number of workers that are currently processing a job. =item delayed_jobs delayed_jobs => 100 Number of jobs in C state that are scheduled to run at specific time in the future or have unresolved dependencies. Note that this field is EXPERIMENTAL and might change without warning! =item enqueued_jobs enqueued_jobs => 100000 Rough estimate of how many jobs have ever been enqueued. Note that this field is EXPERIMENTAL and might change without warning! =item failed_jobs failed_jobs => 100 Number of jobs in C state. =item finished_jobs finished_jobs => 100 Number of jobs in C state. =item inactive_jobs inactive_jobs => 100 Number of jobs in C state. =item inactive_workers inactive_workers => 100 Number of workers that are currently not processing a job. =back =head2 unlock my $bool = $minion->unlock('foo'); Release a named lock that has been previously acquired with L. =head2 worker my $worker = $minion->worker; Build L object. =head1 REFERENCE This is the class hierarchy of the L distribution. =over 2 =item * L =item * L =over 2 =item * L =back =item * L =item * L =item * L =item * L =item * L =item * L =back =head1 AUTHOR Sebastian Riedel, C. =head1 CREDITS In alphabetical order: =over 2 Andrey Khozov Brian Medley Hubert "depesz" Lubaczewski Joel Berger Paul Williams =back =head1 COPYRIGHT AND LICENSE Copyright (C) 2014-2017, Sebastian Riedel and others. This program is free software, you can redistribute it and/or modify it under the terms of the Artistic License version 2.0. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Mojolicious/000755 000765 000024 00000000000 13142141670 015677 5ustar00sristaff000000 000000 Minion-7.05/lib/Mojolicious/Plugin/000755 000765 000024 00000000000 13142141670 017135 5ustar00sristaff000000 000000 Minion-7.05/lib/Mojolicious/Plugin/Minion.pm000644 000765 000024 00000004032 13127476534 020740 0ustar00sristaff000000 000000 package Mojolicious::Plugin::Minion; use Mojo::Base 'Mojolicious::Plugin'; use Minion; use Scalar::Util 'weaken'; sub register { my ($self, $app, $conf) = @_; push @{$app->commands->namespaces}, 'Minion::Command'; my $minion = Minion->new(each %$conf); weaken $minion->app($app)->{app}; $app->helper(minion => sub {$minion}); } 1; =encoding utf8 =head1 NAME Mojolicious::Plugin::Minion - Minion job queue plugin =head1 SYNOPSIS # Mojolicious (choose a backend) $self->plugin(Minion => {Pg => 'postgresql://postgres@/test'}); # Mojolicious::Lite (choose a backend) plugin Minion => {Pg => 'postgresql://postgres@/test'}; # Share the database connection cache (PostgreSQL backend) helper pg => sub { state $pg = Mojo::Pg->new('postgresql://postgres@/test') }; plugin Minion => {Pg => app->pg}; # Add tasks to your application app->minion->add_task(slow_log => sub { my ($job, $msg) = @_; sleep 5; $job->app->log->debug(qq{Received message "$msg"}); }); # Start jobs from anywhere in your application $c->minion->enqueue(slow_log => ['test 123']); # Perform jobs in your tests $t->get_ok('/start_slow_log_job')->status_is(200); $t->get_ok('/start_another_job')->status_is(200); $t->app->minion->perform_jobs; =head1 DESCRIPTION L is a L plugin for the L job queue. =head1 HELPERS L implements the following helpers. =head2 minion my $minion = $app->minion; my $minion = $c->minion; Get L object for application. # Add job to the queue $c->minion->enqueue(foo => ['bar', 'baz']); # Perform jobs for testing $app->minion->perform_jobs; =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 register $plugin->register(Mojolicious->new, {Pg => 'postgresql://postgres@/test'}); Register plugin in L application. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Backend/000755 000765 000024 00000000000 13142141670 016163 5ustar00sristaff000000 000000 Minion-7.05/lib/Minion/Backend.pm000644 000765 000024 00000027740 13140046616 016535 0ustar00sristaff000000 000000 package Minion::Backend; use Mojo::Base -base; use Carp 'croak'; has 'minion'; sub broadcast { croak 'Method "broadcast" not implemented by subclass' } sub dequeue { croak 'Method "dequeue" not implemented by subclass' } sub enqueue { croak 'Method "enqueue" not implemented by subclass' } sub fail_job { croak 'Method "fail_job" not implemented by subclass' } sub finish_job { croak 'Method "finish_job" not implemented by subclass' } sub job_info { croak 'Method "job_info" not implemented by subclass' } sub list_jobs { croak 'Method "list_jobs" not implemented by subclass' } sub list_workers { croak 'Method "list_workers" not implemented by subclass' } sub lock { croak 'Method "lock" not implemented by subclass' } sub note { croak 'Method "note" not implemented by subclass' } sub receive { croak 'Method "receive" not implemented by subclass' } sub register_worker { croak 'Method "register_worker" not implemented by subclass'; } sub remove_job { croak 'Method "remove_job" not implemented by subclass' } sub repair { croak 'Method "repair" not implemented by subclass' } sub reset { croak 'Method "reset" not implemented by subclass' } sub retry_job { croak 'Method "retry_job" not implemented by subclass' } sub stats { croak 'Method "stats" not implemented by subclass' } sub unlock { croak 'Method "unlock" not implemented by subclass' } sub unregister_worker { croak 'Method "unregister_worker" not implemented by subclass'; } sub worker_info { croak 'Method "worker_info" not implemented by subclass' } 1; =encoding utf8 =head1 NAME Minion::Backend - Backend base class =head1 SYNOPSIS package Minion::Backend::MyBackend; use Mojo::Base 'Minion::Backend'; sub broadcast {...} sub dequeue {...} sub enqueue {...} sub fail_job {...} sub finish_job {...} sub job_info {...} sub list_jobs {...} sub list_workers {...} sub lock {...} sub note {...} sub receive {...} sub register_worker {...} sub remove_job {...} sub repair {...} sub reset {...} sub retry_job {...} sub stats {...} sub unlock {...} sub unregister_worker {...} sub worker_info {...} =head1 DESCRIPTION L is an abstract base class for L backends, like L. =head1 ATTRIBUTES L implements the following attributes. =head2 minion my $minion = $backend->minion; $backend = $backend->minion(Minion->new); L object this backend belongs to. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 broadcast my $bool = $backend->broadcast('some_command'); my $bool = $backend->broadcast('some_command', [@args]); my $bool = $backend->broadcast('some_command', [@args], [$id1, $id2, $id3]); Broadcast remote control command to one or more workers. =head2 dequeue my $job_info = $backend->dequeue($worker_id, 0.5); my $job_info = $backend->dequeue($worker_id, 0.5, {queues => ['important']}); Wait a given amount of time in seconds for a job, dequeue it and transition from C to C state, or return C if queues were empty. Meant to be overloaded in a subclass. These options are currently available: =over 2 =item id id => '10023' Dequeue a specific job. =item queues queues => ['important'] One or more queues to dequeue jobs from, defaults to C. =back These fields are currently available: =over 2 =item args args => ['foo', 'bar'] Job arguments. =item id id => '10023' Job ID. =item retries retries => 3 Number of times job has been retried. =item task task => 'foo' Task name. =back =head2 enqueue my $job_id = $backend->enqueue('foo'); my $job_id = $backend->enqueue(foo => [@args]); my $job_id = $backend->enqueue(foo => [@args] => {priority => 1}); Enqueue a new job with C state. Meant to be overloaded in a subclass. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted, with a delay based on L after the first attempt, defaults to C<1>. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job. =item parents parents => [$id1, $id2, $id3] One or more existing jobs this job depends on, and that need to have transitioned to the state C before it can be processed. =item priority priority => 5 Job priority, defaults to C<0>. Jobs with a higher priority get performed first. =item queue queue => 'important' Queue to put job in, defaults to C. =back =head2 fail_job my $bool = $backend->fail_job($job_id, $retries); my $bool = $backend->fail_job($job_id, $retries, 'Something went wrong!'); my $bool = $backend->fail_job( $job_id, $retries, {whatever => 'Something went wrong!'}); Transition from C to C state, and if there are attempts remaining, transition back to C with a delay based on L. Meant to be overloaded in a subclass. =head2 finish_job my $bool = $backend->finish_job($job_id, $retries); my $bool = $backend->finish_job($job_id, $retries, 'All went well!'); my $bool = $backend->finish_job( $job_id, $retries, {whatever => 'All went well!'}); Transition from C to C state. Meant to be overloaded in a subclass. =head2 job_info my $job_info = $backend->job_info($job_id); Get information about a job, or return C if job does not exist. Meant to be overloaded in a subclass. # Check job state my $state = $backend->job_info($job_id)->{state}; # Get job result my $result = $backend->job_info($job_id)->{result}; These fields are currently available: =over 2 =item args args => ['foo', 'bar'] Job arguments. =item attempts attempts => 25 Number of times performing this job will be attempted. =item children children => ['10026', '10027', '10028'] Jobs depending on this job. =item created created => 784111777 Epoch time job was created. =item delayed delayed => 784111777 Epoch time job was delayed to. =item finished finished => 784111777 Epoch time job was finished. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job. =item parents parents => ['10023', '10024', '10025'] Jobs this job depends on. =item priority priority => 3 Job priority. =item queue queue => 'important' Queue name. =item result result => 'All went well!' Job result. =item retried retried => 784111777 Epoch time job has been retried. =item retries retries => 3 Number of times job has been retried. =item started started => 784111777 Epoch time job was started. =item state state => 'inactive' Current job state, usually C, C, C or C. =item task task => 'foo' Task name. =item worker worker => '154' Id of worker that is processing the job. =back =head2 list_jobs my $batch = $backend->list_jobs($offset, $limit); my $batch = $backend->list_jobs($offset, $limit, {state => 'inactive'}); Returns the same information as L but in batches. Meant to be overloaded in a subclass. These options are currently available: =over 2 =item queue queue => 'important' List only jobs in this queue. =item state state => 'inactive' List only jobs in this state. =item task task => 'test' List only jobs for this task. =back =head2 list_workers my $batch = $backend->list_workers($offset, $limit); Returns the same information as L but in batches. Meant to be overloaded in a subclass. =head2 lock my $bool = $backend->lock('foo', 3600); my $bool = $backend->lock('foo', 3600, {limit => 20}); Try to acquire a named lock that will expire automatically after the given amount of time in seconds. These options are currently available: =over 2 =item limit limit => 20 Number of shared locks with the same name that can be active at the same time, defaults to C<1>. =back =head2 note $backend->note($job_id, foo => 'bar'); Change a metadata field for a job. =head2 receive my $commands = $backend->receive($worker_id); Receive remote control commands for worker. =head2 register_worker my $worker_id = $backend->register_worker; my $worker_id = $backend->register_worker($worker_id); my $worker_id = $backend->register_worker( $worker_id, {status => {queues => ['default', 'important']}}); Register worker or send heartbeat to show that this worker is still alive. Meant to be overloaded in a subclass. These options are currently available: =over 2 =item status status => {queues => ['default', 'important']} Hash reference with whatever status information the worker would like to share. =back =head2 remove_job my $bool = $backend->remove_job($job_id); Remove C, C or C job from queue. Meant to be overloaded in a subclass. =head2 repair $backend->repair; Repair worker registry and job queue if necessary. Meant to be overloaded in a subclass. =head2 reset $backend->reset; Reset job queue. Meant to be overloaded in a subclass. =head2 retry_job my $bool = $backend->retry_job($job_id, $retries); my $bool = $backend->retry_job($job_id, $retries, {delay => 10}); Transition job back to C state, already C jobs may also be retried to change options. Meant to be overloaded in a subclass. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item priority priority => 5 Job priority. =item queue queue => 'important' Queue to put job in. =back =head2 stats my $stats = $backend->stats; Get statistics for jobs and workers. Meant to be overloaded in a subclass. These fields are currently available: =over 2 =item active_jobs active_jobs => 100 Number of jobs in C state. =item active_workers active_workers => 100 Number of workers that are currently processing a job. =item delayed_jobs delayed_jobs => 100 Number of jobs in C state that are scheduled to run at specific time in the future or have unresolved dependencies. Note that this field is EXPERIMENTAL and might change without warning! =item failed_jobs failed_jobs => 100 Number of jobs in C state. =item finished_jobs finished_jobs => 100 Number of jobs in C state. =item inactive_jobs inactive_jobs => 100 Number of jobs in C state. =item inactive_workers inactive_workers => 100 Number of workers that are currently not processing a job. =back =head2 unlock my $bool = $backend->unlock('foo'); Release a named lock. =head2 unregister_worker $backend->unregister_worker($worker_id); Unregister worker. Meant to be overloaded in a subclass. =head2 worker_info my $worker_info = $backend->worker_info($worker_id); Get information about a worker, or return C if worker does not exist. Meant to be overloaded in a subclass. # Check worker host my $host = $backend->worker_info($worker_id)->{host}; These fields are currently available: =over 2 =item host host => 'localhost' Worker host. =item jobs jobs => ['10023', '10024', '10025', '10029'] Ids of jobs the worker is currently processing. =item notified notified => 784111777 Epoch time worker sent the last heartbeat. =item pid pid => 12345 Process id of worker. =item started started => 784111777 Epoch time worker was started. =item status status => {queues => ['default', 'important']} Hash reference with whatever status information the worker would like to share. =back =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Command/000755 000765 000024 00000000000 13142141670 016212 5ustar00sristaff000000 000000 Minion-7.05/lib/Minion/Job.pm000644 000765 000024 00000017403 13142104460 015705 0ustar00sristaff000000 000000 package Minion::Job; use Mojo::Base 'Mojo::EventEmitter'; use Mojo::IOLoop; use POSIX 'WNOHANG'; has [qw(args id minion retries task)]; sub app { shift->minion->app } sub fail { my ($self, $err) = (shift, shift // 'Unknown error'); my $ok = $self->minion->backend->fail_job($self->id, $self->retries, $err); return $ok ? !!$self->emit(failed => $err) : undef; } sub finish { my ($self, $result) = @_; my $ok = $self->minion->backend->finish_job($self->id, $self->retries, $result); return $ok ? !!$self->emit(finished => $result) : undef; } sub info { $_[0]->minion->backend->job_info($_[0]->id) } sub is_finished { my $self = shift; return undef unless waitpid($self->{pid}, WNOHANG) == $self->{pid}; $? ? $self->fail("Non-zero exit status (@{[$? >> 8]})") : $self->finish; return 1; } sub note { $_[0]->minion->backend->note($_[0]->id, @_[1, 2]) } sub perform { my $self = shift; waitpid $self->start->pid, 0; $? ? $self->fail("Non-zero exit status (@{[$? >> 8]})") : $self->finish; } sub pid { shift->{pid} } sub remove { $_[0]->minion->backend->remove_job($_[0]->id) } sub retry { my $self = shift; return $self->minion->backend->retry_job($self->id, $self->retries, @_); } sub start { my $self = shift; # Parent die "Can't fork: $!" unless defined(my $pid = fork); return $self->emit(spawn => $pid) if $self->{pid} = $pid; # Child $self->_run; POSIX::_exit(0); } sub stop { kill 'KILL', shift->{pid} } sub _run { my $self = shift; return undef if eval { # Reset event loop Mojo::IOLoop->reset; $self->minion->tasks->{$self->emit('start')->task}->($self, @{$self->args}); 1; }; $self->fail(my $err = $@); return $err; } 1; =encoding utf8 =head1 NAME Minion::Job - Minion job =head1 SYNOPSIS use Minion::Job; my $job = Minion::Job->new(id => $id, minion => $minion, task => 'foo'); =head1 DESCRIPTION L is a container for L jobs. =head1 EVENTS L inherits all events from L and can emit the following new ones. =head2 failed $job->on(failed => sub { my ($job, $err) = @_; ... }); Emitted in the worker process managing this job or the process performing it, after it has transitioned to the C state. $job->on(failed => sub { my ($job, $err) = @_; say "Something went wrong: $err"; }); =head2 finished $job->on(finished => sub { my ($job, $result) = @_; ... }); Emitted in the worker process managing this job or the process performing it, after it has transitioned to the C state. $job->on(finished => sub { my ($job, $result) = @_; my $id = $job->id; say "Job $id is finished."; }); =head2 spawn $job->on(spawn => sub { my ($job, $pid) = @_; ... }); Emitted in the worker process managing this job, after a new process has been spawned for processing. $job->on(spawn => sub { my ($job, $pid) = @_; my $id = $job->id; say "Job $id running in process $pid"; }); =head2 start $job->on(start => sub { my $job = shift; ... }); Emitted in the process performing this job, after it has been spawned. $job->on(start => sub { my $job = shift; $0 = $job->id; }); =head1 ATTRIBUTES L implements the following attributes. =head2 args my $args = $job->args; $job = $job->args([]); Arguments passed to task. =head2 id my $id = $job->id; $job = $job->id($id); Job id. =head2 minion my $minion = $job->minion; $job = $job->minion(Minion->new); L object this job belongs to. =head2 retries my $retries = $job->retries; $job = $job->retries(5); Number of times job has been retried. =head2 task my $task = $job->task; $job = $job->task('foo'); Task name. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 app my $app = $job->app; Get application from L. # Longer version my $app = $job->minion->app; =head2 fail my $bool = $job->fail; my $bool = $job->fail('Something went wrong!'); my $bool = $job->fail({whatever => 'Something went wrong!'}); Transition from C to C state, and if there are attempts remaining, transition back to C with a delay based on L. =head2 finish my $bool = $job->finish; my $bool = $job->finish('All went well!'); my $bool = $job->finish({whatever => 'All went well!'}); Transition from C to C state. =head2 info my $info = $job->info; Get job information. # Check job state my $state = $job->info->{state}; # Get job metadata my $progress = $job->info->{notes}{progress}; # Get job result my $result = $job->info->{result}; These fields are currently available: =over 2 =item args args => ['foo', 'bar'] Job arguments. =item attempts attempts => 25 Number of times performing this job will be attempted. =item children children => ['10026', '10027', '10028'] Jobs depending on this job. =item created created => 784111777 Epoch time job was created. =item delayed delayed => 784111777 Epoch time job was delayed to. =item finished finished => 784111777 Epoch time job was finished. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job. =item parents parents => ['10023', '10024', '10025'] Jobs this job depends on. =item priority priority => 3 Job priority. =item queue queue => 'important' Queue name. =item result result => 'All went well!' Job result. =item retried retried => 784111777 Epoch time job has been retried. =item retries retries => 3 Number of times job has been retried. =item started started => 784111777 Epoch time job was started. =item state state => 'inactive' Current job state, usually C, C, C or C. =item task task => 'foo' Task name. =item worker worker => '154' Id of worker that is processing the job. =back =head2 is_finished my $bool = $job->is_finished; Check if job performed with L is finished. =head2 note my $bool = $job->note(foo => 'bar'); Change a metadata field for this job. The new value will get serialized by L (often with L), so you shouldn't send objects and be careful with binary data, nested data structures with hash and array references are fine though. # Share progress information $job->note(progress => 95); # Share stats $job->note(stats => {utime => '0.012628', stime => '0.002429'}); =head2 perform $job->perform; Perform job in new process and wait for it to finish. =head2 pid my $pid = $job->pid; Process id of the process spawned by L if available. =head2 remove my $bool = $job->remove; Remove C, C or C job from queue. =head2 retry my $bool = $job->retry; my $bool = $job->retry({delay => 10}); Transition job back to C state, already C jobs may also be retried to change options. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item priority priority => 5 Job priority. =item queue queue => 'important' Queue to put job in. =back =head2 start $job = $job->start; Perform job in new process, but do not wait for it to finish. # Perform two jobs concurrently $job1->start; $job2->start; my ($first, $second); sleep 1 until $first ||= $job1->is_finished and $second ||= $job2->is_finished; =head2 stop $job->stop; Stop job performed with L immediately. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Worker.pm000644 000765 000024 00000010427 13137736206 016460 0ustar00sristaff000000 000000 package Minion::Worker; use Mojo::Base 'Mojo::EventEmitter'; has [qw(commands status)] => sub { {} }; has [qw(id minion)]; sub add_command { $_[0]->commands->{$_[1]} = $_[2] and return $_[0] } sub dequeue { my ($self, $wait, $options) = @_; # Worker not registered return undef unless my $id = $self->id; my $minion = $self->minion; return undef unless my $job = $minion->backend->dequeue($id, $wait, $options); $job = Minion::Job->new( args => $job->{args}, id => $job->{id}, minion => $minion, retries => $job->{retries}, task => $job->{task} ); $self->emit(dequeue => $job); return $job; } sub info { $_[0]->minion->backend->worker_info($_[0]->id) } sub process_commands { my $self = shift; for my $command (@{$self->minion->backend->receive($self->id)}) { next unless my $cb = $self->commands->{shift @$command}; $self->$cb(@$command); } return $self; } sub register { my $self = shift; my $status = {status => $self->status}; return $self->id($self->minion->backend->register_worker($self->id, $status)); } sub unregister { my $self = shift; $self->minion->backend->unregister_worker(delete $self->{id}); return $self; } 1; =encoding utf8 =head1 NAME Minion::Worker - Minion worker =head1 SYNOPSIS use Minion::Worker; my $worker = Minion::Worker->new(minion => $minion); =head1 DESCRIPTION L performs jobs for L. =head1 EVENTS L inherits all events from L and can emit the following new ones. =head2 dequeue $worker->on(dequeue => sub { my ($worker, $job) = @_; ... }); Emitted in the worker process after a job has been dequeued. $worker->on(dequeue => sub { my ($worker, $job) = @_; my $id = $job->id; say "Job $id has been dequeued."; }); =head1 ATTRIBUTES L implements the following attributes. =head2 commands my $commands = $worker->commands; $worker = $worker->commands({jobs => sub {...}}); Registered worker remote control commands. =head2 id my $id = $worker->id; $worker = $worker->id($id); Worker id. =head2 minion my $minion = $worker->minion; $worker = $worker->minion(Minion->new); L object this worker belongs to. =head2 status my $status = $worker->status; $worker = $worker->status({queues => ['default', 'important']); Status information to share every time L is called. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 add_command $worker = $worker->add_command(jobs => sub {...}); Register a worker remote control command. $worker->add_command(foo => sub { my ($worker, @args) = @_; ... }); =head2 dequeue my $job = $worker->dequeue(0.5); my $job = $worker->dequeue(0.5 => {queues => ['important']}); Wait a given amount of time in seconds for a job, dequeue L object and transition from C to C state, or return C if queues were empty. These options are currently available: =over 2 =item id id => '10023' Dequeue a specific job. =item queues queues => ['important'] One or more queues to dequeue jobs from, defaults to C. =back =head2 info my $info = $worker->info; Get worker information. # Check worker host my $host = $worker->info->{host}; These fields are currently available: =over 2 =item host host => 'localhost' Worker host. =item jobs jobs => ['10023', '10024', '10025', '10029'] Ids of jobs the worker is currently processing. =item notified notified => 784111777 Epoch time worker sent the last heartbeat. =item pid pid => 12345 Process id of worker. =item started started => 784111777 Epoch time worker was started. =item status status => {queues => ['default', 'important']} Hash reference with whatever status information the worker would like to share. =back =head2 process_commands $worker = $worker->process_commands; Process worker remote control commands. =head2 register $worker = $worker->register; Register worker or send heartbeat to show that this worker is still alive. =head2 unregister $worker = $worker->unregister; Unregister worker. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Command/minion/000755 000765 000024 00000000000 13142141670 017503 5ustar00sristaff000000 000000 Minion-7.05/lib/Minion/Command/minion.pm000644 000765 000024 00000003313 13123461250 020037 0ustar00sristaff000000 000000 package Minion::Command::minion; use Mojo::Base 'Mojolicious::Commands'; has description => 'Minion job queue'; has hint => < sub { shift->extract_usage . "\nCommands:\n" }; has namespaces => sub { ['Minion::Command::minion'] }; sub help { shift->run(@_) } 1; =encoding utf8 =head1 NAME Minion::Command::minion - Minion command =head1 SYNOPSIS Usage: APPLICATION minion COMMAND [OPTIONS] =head1 DESCRIPTION L lists available L commands. =head1 ATTRIBUTES L inherits all attributes from L and implements the following new ones. =head2 description my $description = $minion->description; $minion = $minion->description('Foo'); Short description of this command, used for the command list. =head2 hint my $hint = $minion->hint; $minion = $minion->hint('Foo'); Short hint shown after listing available L commands. =head2 message my $msg = $minion->message; $minion = $minion->message('Bar'); Short usage message shown before listing available L commands. =head2 namespaces my $namespaces = $minion->namespaces; $minion = $minion->namespaces(['MyApp::Command::minion']); Namespaces to search for available L commands, defaults to L. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 help $minion->help('app'); Print usage information for L command. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Command/minion/job.pm000644 000765 000024 00000014261 13140041301 020603 0ustar00sristaff000000 000000 package Minion::Command::minion::job; use Mojo::Base 'Mojolicious::Command'; use Mojo::Date; use Mojo::JSON 'decode_json'; use Mojo::Util qw(dumper getopt tablify); has description => 'Manage Minion jobs'; has usage => sub { shift->extract_usage }; sub run { my ($self, @args) = @_; my ($args, $options) = ([], {}); getopt \@args, 'A|attempts=i' => \$options->{attempts}, 'a|args=s' => sub { $args = decode_json($_[1]) }, 'b|broadcast=s' => (\my $command), 'd|delay=i' => \$options->{delay}, 'e|enqueue=s' => \my $enqueue, 'f|foreground' => \my $foreground, 'l|limit=i' => \(my $limit = 100), 'o|offset=i' => \(my $offset = 0), 'P|parent=s' => ($options->{parents} = []), 'p|priority=i' => \$options->{priority}, 'q|queue=s' => \$options->{queue}, 'R|retry' => \my $retry, 'remove' => \my $remove, 'S|state=s' => \$options->{state}, 's|stats' => \my $stats, 't|task=s' => \$options->{task}, 'w|workers' => \my $workers; # Worker remote control command my $minion = $self->app->minion; return $minion->backend->broadcast($command, $args, \@args) if $command; # Enqueue return say $minion->enqueue($enqueue, $args, $options) if $enqueue; # Show stats return $self->_stats if $stats; # List jobs/workers my $id = @args ? shift @args : undef; return $id ? $self->_worker($id) : $self->_list_workers($offset, $limit) if $workers; return $self->_list_jobs($offset, $limit, $options) unless defined $id; die "Job does not exist.\n" unless my $job = $minion->job($id); # Remove job return $job->remove || die "Job is active.\n" if $remove; # Retry job return $job->retry($options) || die "Job is active.\n" if $retry; # Perform job in foreground return $minion->foreground($id) || die "Job is not ready.\n" if $foreground; # Job info print dumper _datetime($job->info); } sub _datetime { my $hash = shift; $hash->{$_} and $hash->{$_} = Mojo::Date->new($hash->{$_})->to_datetime for qw(created delayed finished notified retried started); return $hash; } sub _list_jobs { my $jobs = shift->app->minion->backend->list_jobs(@_); print tablify [map { [@$_{qw(id state queue task)}] } @$jobs]; } sub _list_workers { my $workers = shift->app->minion->backend->list_workers(@_); my @workers = map { [$_->{id}, $_->{host} . ':' . $_->{pid}] } @$workers; print tablify \@workers; } sub _stats { print dumper shift->app->minion->stats } sub _worker { die "Worker does not exist.\n" unless my $worker = shift->app->minion->backend->worker_info(@_); print dumper _datetime($worker); } 1; =encoding utf8 =head1 NAME Minion::Command::minion::job - Minion job command =head1 SYNOPSIS Usage: APPLICATION minion job [OPTIONS] [IDS] ./myapp.pl minion job ./myapp.pl minion job 10023 ./myapp.pl minion job -w ./myapp.pl minion job -w 23 ./myapp.pl minion job -s ./myapp.pl minion job -f 10023 ./myapp.pl minion job -q important -t foo -S inactive ./myapp.pl minion job -e foo -a '[23, "bar"]' ./myapp.pl minion job -e foo -P 10023 -P 10024 -p 5 -q important ./myapp.pl minion job -R -d 10 10023 ./myapp.pl minion job --remove 10023 ./myapp.pl minion job -b jobs -a '[12]' ./myapp.pl minion job -b jobs -a '[12]' 23 24 25 Options: -A, --attempts Number of times performing this new job will be attempted, defaults to 1 -a, --args Arguments for new job or worker remote control command in JSON format -b, --broadcast Broadcast remote control command to one or more workers -d, --delay Delay new job for this many seconds -e, --enqueue New job to be enqueued -f, --foreground Retry job in "minion_foreground" queue and perform it right away in the foreground (very useful for debugging) -h, --help Show this summary of available options --home Path to home directory of your application, defaults to the value of MOJO_HOME or auto-detection -l, --limit Number of jobs/workers to show when listing them, defaults to 100 -m, --mode Operating mode for your application, defaults to the value of MOJO_MODE/PLACK_ENV or "development" -o, --offset Number of jobs/workers to skip when listing them, defaults to 0 -P, --parent One or more jobs the new job depends on -p, --priority Priority of new job, defaults to 0 -q, --queue Queue to put new job in, defaults to "default", or list only jobs in this queue -R, --retry Retry job --remove Remove job -S, --state List only jobs in this state -s, --stats Show queue statistics -t, --task List only jobs for this task -w, --workers List workers instead of jobs, or show information for a specific worker =head1 DESCRIPTION L manages the L job queue. =head1 ATTRIBUTES L inherits all attributes from L and implements the following new ones. =head2 description my $description = $job->description; $job = $job->description('Foo'); Short description of this command, used for the command list. =head2 usage my $usage = $job->usage; $job = $job->usage('Foo'); Usage information for this command, used for the help screen. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 run $job->run(@ARGV); Run this command. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Command/minion/worker.pm000644 000765 000024 00000015750 13136453465 021375 0ustar00sristaff000000 000000 package Minion::Command::minion::worker; use Mojo::Base 'Mojolicious::Command'; use Mojo::Util qw(getopt steady_time); has description => 'Start Minion worker'; has usage => sub { shift->extract_usage }; sub run { my ($self, @args) = @_; my $app = $self->app; my $worker = $self->{worker} = $app->minion->worker; my $status = $worker->status; $status->{performed} //= 0; getopt \@args, 'C|command-interval=i' => \($status->{command_interval} //= 10), 'f|fast-start' => \my $fast, 'I|heartbeat-interval=i' => \($status->{heartbeat_interval} //= 300), 'j|jobs=i' => \($status->{jobs} //= 4), 'q|queue=s' => ($status->{queues} //= []), 'R|repair-interval=i' => \($status->{repair_interval} //= 21600); @{$status->{queues}} = ('default') unless @{$status->{queues}}; $status->{repair_interval} -= int rand $status->{repair_interval} / 2; $self->{last_repair} = $fast ? steady_time : 0; local $SIG{CHLD} = sub { }; local $SIG{INT} = local $SIG{TERM} = sub { $self->{finished}++ }; local $SIG{QUIT} = sub { ++$self->{finished} and kill 'KILL', keys %{$self->{jobs}} }; # Remote control commands need to validate arguments carefully $worker->add_command( jobs => sub { $status->{jobs} = $_[1] if ($_[1] // '') =~ /^\d+$/ }); $worker->add_command( stop => sub { $self->{jobs}{$_[1]}->stop if $self->{jobs}{$_[1] // ''} }); # Log fatal errors my $log = $app->log; $log->info("Worker $$ started"); eval { $self->_work until $self->{finished} && !keys %{$self->{jobs}}; 1 } or $log->fatal("Worker error: $@"); $worker->unregister; $log->info("Worker $$ stopped"); } sub _work { my $self = shift; # Send heartbeats in regular intervals my $worker = $self->{worker}; my $status = $worker->status; $self->{last_heartbeat} ||= 0; $worker->register and $self->{last_heartbeat} = steady_time if ($self->{last_heartbeat} + $status->{heartbeat_interval}) < steady_time; # Process worker remote control commands in regular intervals $self->{last_command} ||= 0; $worker->process_commands and $self->{last_command} = steady_time if ($self->{last_command} + $status->{command_interval}) < steady_time; # Repair in regular intervals (randomize to avoid congestion) my $app = $self->app; my $log = $app->log; if (($self->{last_repair} + $status->{repair_interval}) < steady_time) { $log->debug('Checking worker registry and job queue'); $app->minion->repair; $self->{last_repair} = steady_time; } # Check if jobs are finished my $jobs = $self->{jobs} ||= {}; $jobs->{$_}->is_finished and ++$status->{performed} and delete $jobs->{$_} for keys %$jobs; # Wait if job limit has been reached or worker is stopping if (($status->{jobs} <= keys %$jobs) || $self->{finished}) { sleep 1 } # Try to get more jobs elsif (my $job = $worker->dequeue(5 => {queues => $status->{queues}})) { $jobs->{my $id = $job->id} = $job->start; my ($pid, $task) = ($job->pid, $job->task); $log->debug(qq{Process $pid is performing job "$id" with task "$task"}); } } 1; =encoding utf8 =head1 NAME Minion::Command::minion::worker - Minion worker command =head1 SYNOPSIS Usage: APPLICATION minion worker [OPTIONS] ./myapp.pl minion worker ./myapp.pl minion worker -f ./myapp.pl minion worker -m production -I 15 -C 5 -R 3600 -j 10 ./myapp.pl minion worker -q important -q default Options: -C, --command-interval Worker remote control command interval, defaults to 10 -f, --fast-start Start processing jobs as fast as possible and skip repairing on startup -h, --help Show this summary of available options --home Path to home directory of your application, defaults to the value of MOJO_HOME or auto-detection -I, --heartbeat-interval Heartbeat interval, defaults to 300 -j, --jobs Maximum number of jobs to perform parallel in forked worker processes, defaults to 4 -m, --mode Operating mode for your application, defaults to the value of MOJO_MODE/PLACK_ENV or "development" -q, --queue One or more queues to get jobs from, defaults to "default" -R, --repair-interval Repair interval, up to half of this value can be subtracted randomly to make sure not all workers repair at the same time, defaults to 21600 (6 hours) =head1 DESCRIPTION L starts a L worker. You can have as many workers as you like. =head1 SIGNALS The L process can be controlled at runtime with the following signals. =head2 INT, TERM Stop gracefully after finishing the current jobs. =head2 QUIT Stop immediately without finishing the current jobs. =head1 REMOTE CONTROL COMMANDS The L process can be controlled at runtime through L, from anywhere in the network, by broadcasting the following remote control commands. =head2 jobs $ ./myapp.pl minion job -b jobs -a '[10]' $ ./myapp.pl minion job -b jobs -a '[10]' 23 Instruct one or more workers to change the number of jobs to perform concurrently. Setting this value to C<0> will effectively pause the worker. That means all current jobs will be finished, but no new ones accepted, until the number is increased again. =head2 stop $ ./myapp.pl minion job -b stop -a '[10025]' $ ./myapp.pl minion job -b stop -a '[10025]' 23 Instruct one or more workers to stop a job that is currently being performed immediately. This command will be ignored by workers that do not have a job matching the id. That means it is safe to broadcast this command to all workers. =head1 ATTRIBUTES L inherits all attributes from L and implements the following new ones. =head2 description my $description = $worker->description; $worker = $worker->description('Foo'); Short description of this command, used for the command list. =head2 usage my $usage = $worker->usage; $worker = $worker->usage('Foo'); Usage information for this command, used for the help screen. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 run $worker->run(@ARGV); Run this command. =head1 SEE ALSO L, L, L. =cut Minion-7.05/lib/Minion/Backend/Pg.pm000644 000765 000024 00000054751 13140046564 017107 0ustar00sristaff000000 000000 package Minion::Backend::Pg; use Mojo::Base 'Minion::Backend'; use Carp 'croak'; use Mojo::IOLoop; use Mojo::Pg 4.0; use Sys::Hostname 'hostname'; has 'pg'; sub broadcast { my ($self, $command, $args, $ids) = (shift, shift, shift || [], shift || []); return !!$self->pg->db->query( q{update minion_workers set inbox = inbox || $1::jsonb where (id = any ($2) or $2 = '{}')}, {json => [[$command, @$args]]}, $ids )->rows; } sub dequeue { my ($self, $id, $wait, $options) = @_; if ((my $job = $self->_try($id, $options))) { return $job } return undef if Mojo::IOLoop->is_running; my $db = $self->pg->db; $db->listen('minion.job')->on(notification => sub { Mojo::IOLoop->stop }); my $timer = Mojo::IOLoop->timer($wait => sub { Mojo::IOLoop->stop }); Mojo::IOLoop->start; $db->unlisten('*') and Mojo::IOLoop->remove($timer); undef $db; return $self->_try($id, $options); } sub enqueue { my ($self, $task, $args, $options) = (shift, shift, shift || [], shift || {}); my $db = $self->pg->db; return $db->query( "insert into minion_jobs (args, attempts, delayed, notes, parents, priority, queue, task) values (?, ?, (now() + (interval '1 second' * ?)), ?, ?, ?, ?, ?) returning id", {json => $args}, $options->{attempts} // 1, $options->{delay} // 0, {json => $options->{notes} || {}}, $options->{parents} || [], $options->{priority} // 0, $options->{queue} // 'default', $task )->hash->{id}; } sub fail_job { shift->_update(1, @_) } sub finish_job { shift->_update(0, @_) } sub job_info { shift->pg->db->query( 'select id, args, attempts, array(select id from minion_jobs where parents @> ARRAY[j.id]) as children, extract(epoch from created) as created, extract(epoch from delayed) as delayed, extract(epoch from finished) as finished, notes, parents, priority, queue, result, extract(epoch from retried) as retried, retries, extract(epoch from started) as started, state, task, worker from minion_jobs as j where id = ?', shift )->expand->hash; } sub list_jobs { my ($self, $offset, $limit, $options) = @_; return $self->pg->db->query( 'select id from minion_jobs where (queue = $1 or $1 is null) and (state = $2 or $2 is null) and (task = $3 or $3 is null) order by id desc limit $4 offset $5', @$options{qw(queue state task)}, $limit, $offset )->arrays->map(sub { $self->job_info($_->[0]) })->to_array; } sub list_workers { my ($self, $offset, $limit) = @_; my $sql = 'select id from minion_workers order by id desc limit ? offset ?'; return $self->pg->db->query($sql, $limit, $offset) ->arrays->map(sub { $self->worker_info($_->[0]) })->to_array; } sub lock { my ($self, $name, $duration, $options) = (shift, shift, shift, shift // {}); return !!$self->pg->db->query('select * from minion_lock(?, ?, ?)', $name, $duration, $options->{limit} || 1)->array->[0]; } sub new { my $self = shift->SUPER::new(pg => Mojo::Pg->new(@_)); my $db = Mojo::Pg->new(@_)->db; croak 'PostgreSQL 9.5 or later is required' if $db->dbh->{pg_server_version} < 90500; $db->disconnect; $self->pg->auto_migrate(1)->migrations->name('minion')->from_data; return $self; } sub note { my ($self, $id, $key, $value) = @_; return !!$self->pg->db->query( 'update minion_jobs set notes = jsonb_set(notes, ?, ?, true) where id = ?', [$key], {json => $value}, $id )->rows; } sub receive { my $array = shift->pg->db->query( "update minion_workers as new set inbox = '[]' from (select id, inbox from minion_workers where id = ? for update) as old where new.id = old.id and old.inbox != '[]' returning old.inbox", shift )->expand->array; return $array ? $array->[0] : []; } sub register_worker { my ($self, $id, $options) = (shift, shift, shift || {}); return $self->pg->db->query( q{insert into minion_workers (id, host, pid, status) values (coalesce($1, nextval('minion_workers_id_seq')), $2, $3, $4) on conflict(id) do update set notified = now(), status = $4 returning id}, $id, $self->{host} //= hostname, $$, {json => $options->{status} // {}} )->hash->{id}; } sub remove_job { !!shift->pg->db->query( "delete from minion_jobs where id = ? and state in ('inactive', 'failed', 'finished') returning 1", shift )->rows; } sub repair { my $self = shift; # Workers without heartbeat my $db = $self->pg->db; my $minion = $self->minion; $db->query( "delete from minion_workers where notified < now() - interval '1 second' * ?", $minion->missing_after ); # Jobs with missing worker (can be retried) my $fail = $db->query( "select id, retries from minion_jobs as j where state = 'active' and not exists (select 1 from minion_workers where id = j.worker)" )->hashes; $fail->each(sub { $self->fail_job(@$_{qw(id retries)}, 'Worker went away') }); # Old jobs with no unresolved dependencies $db->query( "delete from minion_jobs as j where finished <= now() - interval '1 second' * ? and not exists ( select 1 from minion_jobs where parents @> ARRAY[j.id] and state != 'finished' ) and state = 'finished'", $minion->remove_after ); } sub reset { shift->pg->db->query( 'truncate minion_jobs, minion_locks, minion_workers restart identity'); } sub retry_job { my ($self, $id, $retries, $options) = (shift, shift, shift, shift || {}); return !!$self->pg->db->query( "update minion_jobs set attempts = coalesce(?, attempts), delayed = (now() + (interval '1 second' * ?)), priority = coalesce(?, priority), queue = coalesce(?, queue), retried = now(), retries = retries + 1, state = 'inactive' where id = ? and retries = ? returning 1", $options->{attempts}, $options->{delay} // 0, @$options{qw(priority queue)}, $id, $retries )->rows; } sub stats { my $self = shift; my $stats = $self->pg->db->query( "select count(*) filter (where state = 'inactive') as inactive_jobs, count(*) filter (where state = 'active') as active_jobs, count(*) filter (where state = 'failed') as failed_jobs, count(*) filter (where state = 'finished') as finished_jobs, count(*) filter (where state = 'inactive' and (delayed > now() or parents != '{}')) as delayed_jobs, count(distinct worker) filter (where state = 'active') as active_workers, (select case when is_called then last_value else 0 end from minion_jobs_id_seq) as enqueued_jobs, (select count(*) from minion_workers) as inactive_workers from minion_jobs" )->hash; $stats->{inactive_workers} -= $stats->{active_workers}; return $stats; } sub unlock { !!shift->pg->db->query( 'delete from minion_locks where id = ( select id from minion_locks where expires > now() and name = ? order by expires limit 1 ) returning 1', shift )->rows; } sub unregister_worker { shift->pg->db->query('delete from minion_workers where id = ?', shift); } sub worker_info { shift->pg->db->query( "select id, extract(epoch from notified) as notified, array( select id from minion_jobs where state = 'active' and worker = minion_workers.id ) as jobs, host, pid, status, extract(epoch from started) as started from minion_workers where id = ?", shift )->expand->hash; } sub _try { my ($self, $id, $options) = @_; return $self->pg->db->query( "update minion_jobs set started = now(), state = 'active', worker = ? where id = ( select id from minion_jobs as j where delayed <= now() and id = coalesce(?, id) and (parents = '{}' or not exists ( select 1 from minion_jobs where id = any (j.parents) and state in ('inactive', 'active', 'failed') )) and queue = any (?) and state = 'inactive' and task = any (?) order by priority desc, id limit 1 for update skip locked ) returning id, args, retries, task", $id, $options->{id}, $options->{queues} || ['default'], [keys %{$self->minion->tasks}] )->expand->hash; } sub _update { my ($self, $fail, $id, $retries, $result) = @_; return undef unless my $row = $self->pg->db->query( "update minion_jobs set finished = now(), result = ?, state = ? where id = ? and retries = ? and state = 'active' returning attempts", {json => $result}, $fail ? 'failed' : 'finished', $id, $retries )->array; return 1 if !$fail || (my $attempts = $row->[0]) == 1; return 1 if $retries >= ($attempts - 1); my $delay = $self->minion->backoff->($retries); return $self->retry_job($id, $retries, {delay => $delay}); } 1; =encoding utf8 =head1 NAME Minion::Backend::Pg - PostgreSQL backend =head1 SYNOPSIS use Minion::Backend::Pg; my $backend = Minion::Backend::Pg->new('postgresql://postgres@/test'); =head1 DESCRIPTION L is a backend for L based on L. All necessary tables will be created automatically with a set of migrations named C. Note that this backend uses many bleeding edge features, so only the latest, stable version of PostgreSQL is fully supported. =head1 ATTRIBUTES L inherits all attributes from L and implements the following new ones. =head2 pg my $pg = $backend->pg; $backend = $backend->pg(Mojo::Pg->new); L object used to store all data. =head1 METHODS L inherits all methods from L and implements the following new ones. =head2 broadcast my $bool = $backend->broadcast('some_command'); my $bool = $backend->broadcast('some_command', [@args]); my $bool = $backend->broadcast('some_command', [@args], [$id1, $id2, $id3]); Broadcast remote control command to one or more workers. =head2 dequeue my $job_info = $backend->dequeue($worker_id, 0.5); my $job_info = $backend->dequeue($worker_id, 0.5, {queues => ['important']}); Wait a given amount of time in seconds for a job, dequeue it and transition from C to C state, or return C if queues were empty. These options are currently available: =over 2 =item id id => '10023' Dequeue a specific job. =item queues queues => ['important'] One or more queues to dequeue jobs from, defaults to C. =back These fields are currently available: =over 2 =item args args => ['foo', 'bar'] Job arguments. =item id id => '10023' Job ID. =item retries retries => 3 Number of times job has been retried. =item task task => 'foo' Task name. =back =head2 enqueue my $job_id = $backend->enqueue('foo'); my $job_id = $backend->enqueue(foo => [@args]); my $job_id = $backend->enqueue(foo => [@args] => {priority => 1}); Enqueue a new job with C state. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted, with a delay based on L after the first attempt, defaults to C<1>. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job. =item parents parents => [$id1, $id2, $id3] One or more existing jobs this job depends on, and that need to have transitioned to the state C before it can be processed. =item priority priority => 5 Job priority, defaults to C<0>. Jobs with a higher priority get performed first. =item queue queue => 'important' Queue to put job in, defaults to C. =back =head2 fail_job my $bool = $backend->fail_job($job_id, $retries); my $bool = $backend->fail_job($job_id, $retries, 'Something went wrong!'); my $bool = $backend->fail_job( $job_id, $retries, {whatever => 'Something went wrong!'}); Transition from C to C state, and if there are attempts remaining, transition back to C with a delay based on L. =head2 finish_job my $bool = $backend->finish_job($job_id, $retries); my $bool = $backend->finish_job($job_id, $retries, 'All went well!'); my $bool = $backend->finish_job( $job_id, $retries, {whatever => 'All went well!'}); Transition from C to C state. =head2 job_info my $job_info = $backend->job_info($job_id); Get information about a job, or return C if job does not exist. # Check job state my $state = $backend->job_info($job_id)->{state}; # Get job result my $result = $backend->job_info($job_id)->{result}; These fields are currently available: =over 2 =item args args => ['foo', 'bar'] Job arguments. =item attempts attempts => 25 Number of times performing this job will be attempted. =item children children => ['10026', '10027', '10028'] Jobs depending on this job. =item created created => 784111777 Epoch time job was created. =item delayed delayed => 784111777 Epoch time job was delayed to. =item finished finished => 784111777 Epoch time job was finished. =item notes notes => {foo => 'bar', baz => [1, 2, 3]} Hash reference with arbitrary metadata for this job. =item parents parents => ['10023', '10024', '10025'] Jobs this job depends on. =item priority priority => 3 Job priority. =item queue queue => 'important' Queue name. =item result result => 'All went well!' Job result. =item retried retried => 784111777 Epoch time job has been retried. =item retries retries => 3 Number of times job has been retried. =item started started => 784111777 Epoch time job was started. =item state state => 'inactive' Current job state, usually C, C, C or C. =item task task => 'foo' Task name. =item worker worker => '154' Id of worker that is processing the job. =back =head2 list_jobs my $batch = $backend->list_jobs($offset, $limit); my $batch = $backend->list_jobs($offset, $limit, {state => 'inactive'}); Returns the same information as L but in batches. These options are currently available: =over 2 =item queue queue => 'important' List only jobs in this queue. =item state state => 'inactive' List only jobs in this state. =item task task => 'test' List only jobs for this task. =back =head2 list_workers my $batch = $backend->list_workers($offset, $limit); Returns the same information as L but in batches. =head2 lock my $bool = $backend->lock('foo', 3600); my $bool = $backend->lock('foo', 3600, {limit => 20}); Try to acquire a named lock that will expire automatically after the given amount of time in seconds. These options are currently available: =over 2 =item limit limit => 20 Number of shared locks with the same name that can be active at the same time, defaults to C<1>. =back =head2 new my $backend = Minion::Backend::Pg->new('postgresql://postgres@/test'); my $backend = Minion::Backend::Pg->new(Mojo::Pg->new); Construct a new L object. =head2 note my $bool = $backend->note($job_id, foo => 'bar'); Change a metadata field for a job. =head2 receive my $commands = $backend->receive($worker_id); Receive remote control commands for worker. =head2 register_worker my $worker_id = $backend->register_worker; my $worker_id = $backend->register_worker($worker_id); my $worker_id = $backend->register_worker( $worker_id, {status => {queues => ['default', 'important']}}); Register worker or send heartbeat to show that this worker is still alive. These options are currently available: =over 2 =item status status => {queues => ['default', 'important']} Hash reference with whatever status information the worker would like to share. =back =head2 remove_job my $bool = $backend->remove_job($job_id); Remove C, C or C job from queue. =head2 repair $backend->repair; Repair worker registry and job queue if necessary. =head2 reset $backend->reset; Reset job queue. =head2 retry_job my $bool = $backend->retry_job($job_id, $retries); my $bool = $backend->retry_job($job_id, $retries, {delay => 10}); Transition job back to C state, already C jobs may also be retried to change options. These options are currently available: =over 2 =item attempts attempts => 25 Number of times performing this job will be attempted. =item delay delay => 10 Delay job for this many seconds (from now), defaults to C<0>. =item priority priority => 5 Job priority. =item queue queue => 'important' Queue to put job in. =back =head2 stats my $stats = $backend->stats; Get statistics for jobs and workers. These fields are currently available: =over 2 =item active_jobs active_jobs => 100 Number of jobs in C state. =item active_workers active_workers => 100 Number of workers that are currently processing a job. =item delayed_jobs delayed_jobs => 100 Number of jobs in C state that are scheduled to run at specific time in the future or have unresolved dependencies. Note that this field is EXPERIMENTAL and might change without warning! =item enqueued_jobs enqueued_jobs => 100000 Rough estimate of how many jobs have ever been enqueued. Note that this field is EXPERIMENTAL and might change without warning! =item failed_jobs failed_jobs => 100 Number of jobs in C state. =item finished_jobs finished_jobs => 100 Number of jobs in C state. =item inactive_jobs inactive_jobs => 100 Number of jobs in C state. =item inactive_workers inactive_workers => 100 Number of workers that are currently not processing a job. =back =head2 unlock my $bool = $backend->unlock('foo'); Release a named lock. =head2 unregister_worker $backend->unregister_worker($worker_id); Unregister worker. =head2 worker_info my $worker_info = $backend->worker_info($worker_id); Get information about a worker, or return C if worker does not exist. # Check worker host my $host = $backend->worker_info($worker_id)->{host}; These fields are currently available: =over 2 =item host host => 'localhost' Worker host. =item jobs jobs => ['10023', '10024', '10025', '10029'] Ids of jobs the worker is currently processing. =item notified notified => 784111777 Epoch time worker sent the last heartbeat. =item pid pid => 12345 Process id of worker. =item started started => 784111777 Epoch time worker was started. =item status status => {queues => ['default', 'important']} Hash reference with whatever status information the worker would like to share. =back =head1 SEE ALSO L, L, L. =cut __DATA__ @@ minion -- 1 up create table if not exists minion_jobs ( id bigserial not null primary key, args json not null, created timestamp with time zone not null, delayed timestamp with time zone not null, finished timestamp with time zone, priority int not null, result json, retried timestamp with time zone, retries int not null, started timestamp with time zone, state text not null, task text not null, worker bigint ); create table if not exists minion_workers ( id bigserial not null primary key, host text not null, pid int not null, started timestamp with time zone not null ); -- 1 down drop table if exists minion_jobs; drop table if exists minion_workers; -- 2 up alter table minion_jobs alter column created set default now(); alter table minion_jobs alter column state set default 'inactive'; alter table minion_jobs alter column retries set default 0; alter table minion_workers add column notified timestamp with time zone not null default now(); alter table minion_workers alter column started set default now(); -- 4 up alter table minion_jobs add column queue text not null default 'default'; -- 5 up alter table minion_jobs add column attempts int not null default 1; -- 7 up create type minion_state as enum ('inactive', 'active', 'failed', 'finished'); alter table minion_jobs alter column state set default 'inactive'::minion_state; alter table minion_jobs alter column state type minion_state using state::minion_state; alter table minion_jobs alter column args type jsonb using args::jsonb; alter table minion_jobs alter column result type jsonb using result::jsonb; -- 7 down alter table minion_jobs alter column state type text using state; alter table minion_jobs alter column state set default 'inactive'; drop type if exists minion_state; -- 8 up alter table minion_jobs add constraint args check(jsonb_typeof(args) = 'array'); -- 9 up create or replace function minion_jobs_notify_workers() returns trigger as $$ begin if new.delayed <= now() then notify "minion.job"; end if; return null; end; $$ language plpgsql; set client_min_messages to warning; drop trigger if exists minion_jobs_insert_trigger on minion_jobs; drop trigger if exists minion_jobs_notify_workers_trigger on minion_jobs; set client_min_messages to notice; create trigger minion_jobs_notify_workers_trigger after insert or update of retries on minion_jobs for each row execute procedure minion_jobs_notify_workers(); -- 9 down drop trigger if exists minion_jobs_notify_workers_trigger on minion_jobs; drop function if exists minion_jobs_notify_workers(); -- 10 up alter table minion_jobs add column parents bigint[] not null default '{}'; -- 11 up create index on minion_jobs (state, priority desc, id); -- 12 up alter table minion_workers add column inbox jsonb check(jsonb_typeof(inbox) = 'array') not null default '[]'; -- 15 up alter table minion_workers add column status jsonb check(jsonb_typeof(status) = 'object') not null default '{}'; -- 16 up create index on minion_jobs using gin (parents); create table if not exists minion_locks ( id bigserial not null primary key, name text not null, expires timestamp with time zone not null ); create function minion_lock(text, int, int) returns bool as $$ declare new_expires timestamp with time zone = now() + (interval '1 second' * $2); begin delete from minion_locks where expires < now(); lock table minion_locks in exclusive mode; if (select count(*) >= $3 from minion_locks where name = $1) then return false; end if; if new_expires > now() then insert into minion_locks (name, expires) values ($1, new_expires); end if; return true; end; $$ language plpgsql; -- 16 down drop function if exists minion_lock(text, int, int); drop table if exists minion_locks; -- 17 up alter table minion_jobs add column notes jsonb check(jsonb_typeof(notes) = 'object') not null default '{}'; alter table minion_locks set unlogged; create index on minion_locks (name, expires); Minion-7.05/examples/linkcheck/000755 000765 000024 00000000000 13142141670 016406 5ustar00sristaff000000 000000 Minion-7.05/examples/minion_bench.pl000644 000765 000024 00000005744 13125754212 017455 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Minion; use Time::HiRes 'time'; my $ENQUEUE = 10000; my $DEQUEUE = 1000; my $REPETITIONS = 2; my $WORKERS = 4; my $INFO = 100; my $STATS = 100; my $REPAIR = 100; my $LOCK = 1000; my $UNLOCK = 1000; # A benchmark script for comparing backends and evaluating the performance # impact of proposed changes my $minion = Minion->new(Pg => 'postgresql://tester:testing@/test'); $minion->add_task(foo => sub { }); $minion->add_task(bar => sub { }); $minion->reset; # Enqueue say "Clean start with $ENQUEUE jobs"; my @parents = map { $minion->enqueue('foo') } 1 .. 5; my $before = time; $minion->enqueue($_ % 2 ? 'foo' : 'bar' => [] => {parents => \@parents}) for 1 .. $ENQUEUE; my $elapsed = time - $before; my $avg = sprintf '%.3f', $ENQUEUE / $elapsed; say "Enqueued $ENQUEUE jobs in $elapsed seconds ($avg/s)"; $minion->backend->pg->db->query('analyze minion_jobs'); # Dequeue sub dequeue { my @pids; for (1 .. $WORKERS) { die "Couldn't fork: $!" unless defined(my $pid = fork); unless ($pid) { my $worker = $minion->repair->worker->register; say "$$ will finish $DEQUEUE jobs"; my $before = time; $worker->dequeue(0.5)->finish for 1 .. $DEQUEUE; my $elapsed = time - $before; my $avg = sprintf '%.3f', $DEQUEUE / $elapsed; say "$$ finished $DEQUEUE jobs in $elapsed seconds ($avg/s)"; $worker->unregister; exit; } push @pids, $pid; } say "$$ has started $WORKERS workers"; my $before = time; waitpid $_, 0 for @pids; my $elapsed = time - $before; my $avg = sprintf '%.3f', ($DEQUEUE * $WORKERS) / $elapsed; say "$WORKERS workers finished $DEQUEUE jobs each in $elapsed seconds ($avg/s)"; } dequeue() for 1 .. $REPETITIONS; # Job info say "Requesting job info $INFO times"; $before = time; my $backend = $minion->backend; $backend->job_info($_) for 1 .. $INFO; $elapsed = time - $before; $avg = sprintf '%.3f', $INFO / $elapsed; say "Received job info $INFO times in $elapsed seconds ($avg/s)"; # Stats say "Requesting stats $STATS times"; $before = time; $minion->stats for 1 .. $STATS; $elapsed = time - $before; $avg = sprintf '%.3f', $STATS / $elapsed; say "Received stats $STATS times in $elapsed seconds ($avg/s)"; # Repair say "Repairing $REPAIR times"; $before = time; $minion->repair for 1 .. $REPAIR; $elapsed = time - $before; $avg = sprintf '%.3f', $REPAIR / $elapsed; say "Repaired $REPAIR times in $elapsed seconds ($avg/s)"; # Lock say "Acquiring locks $LOCK times"; $before = time; $minion->lock($_ % 2 ? 'foo' : 'bar', 3600, {limit => int($LOCK / 2)}) for 1 .. $LOCK; $elapsed = time - $before; $avg = sprintf '%.3f', $LOCK / $elapsed; say "Acquired locks $LOCK times in $elapsed seconds ($avg/s)"; # Unlock say "Releasing locks $UNLOCK times"; $before = time; $minion->unlock($_ % 2 ? 'foo' : 'bar') for 1 .. $UNLOCK; $elapsed = time - $before; $avg = sprintf '%.3f', $UNLOCK / $elapsed; say "Releasing locks $UNLOCK times in $elapsed seconds ($avg/s)"; Minion-7.05/examples/linkcheck/lib/000755 000765 000024 00000000000 13142141670 017154 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/linkcheck.conf000644 000765 000024 00000000116 13123461250 021204 0ustar00sristaff000000 000000 { pg => 'postgresql://tester:testing@/test', secrets => ['s3cret'] } Minion-7.05/examples/linkcheck/script/000755 000765 000024 00000000000 13142141670 017712 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/t/000755 000765 000024 00000000000 13142141670 016651 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/templates/000755 000765 000024 00000000000 13142141670 020404 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/templates/layouts/000755 000765 000024 00000000000 13142141670 022104 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/templates/links/000755 000765 000024 00000000000 13142141670 021524 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/templates/links/index.html.ep000644 000765 000024 00000000233 13123461250 024120 0ustar00sristaff000000 000000 % layout 'linkcheck', title => 'Check links'; %= form_for 'check' => begin %= url_field url => 'http://mojolicious.org' %= submit_button 'Check' % end Minion-7.05/examples/linkcheck/templates/links/result.html.ep000644 000765 000024 00000000717 13123461250 024336 0ustar00sristaff000000 000000 % layout 'linkcheck', title => 'Result'; % if (ref $result eq 'ARRAY') { % for my $status (@$result) { % }
URL Status
<%= $status->[0] %> <%= $status->[1] %>
% } % elsif (!$result) { % content_for header => begin % end

Waiting for result...

% } % else { %= $result % } Minion-7.05/examples/linkcheck/templates/layouts/linkcheck.html.ep000644 000765 000024 00000000577 13123461250 025337 0ustar00sristaff000000 000000 <%= title %> %= content_for 'header'

<%= link_to 'Link Check' => 'index' %>

%= content Minion-7.05/examples/linkcheck/t/linkcheck.t000644 000765 000024 00000002675 13123461250 021001 0ustar00sristaff000000 000000 use Mojo::Base -strict; use Test::More; # This test requires a PostgreSQL connection string for an existing database # # TEST_ONLINE=postgres://tester:testing@/test script/linkcheck test # plan skip_all => 'set TEST_ONLINE to enable this test' unless $ENV{TEST_ONLINE}; use Mojo::Pg; use Mojo::URL; use Test::Mojo; # Isolate tests my $url = Mojo::URL->new($ENV{TEST_ONLINE})->query([search_path => 'linkcheck_test']); my $pg = Mojo::Pg->new($url); $pg->db->query('drop schema if exists linkcheck_test cascade'); $pg->db->query('create schema linkcheck_test'); # Override configuration for testing my $t = Test::Mojo->new(LinkCheck => {pg => $url, secrets => ['test_s3cret']}); $t->ua->max_redirects(10); # Enqueue a background job $t->get_ok('/')->status_is(200)->text_is('title' => 'Check links') ->element_exists('form input[type=url]'); $t->post_ok('/links' => form => {url => 'http://mojolicious.org'}) ->status_is(200)->text_is('title' => 'Result') ->text_is('p' => 'Waiting for result...')->element_exists_not('table'); # Perform the background job $t->get_ok('/links/1')->status_is(200)->text_is('title' => 'Result') ->text_is('p' => 'Waiting for result...')->element_exists_not('table'); $t->app->minion->perform_jobs; $t->get_ok('/links/1')->status_is(200)->text_is('title' => 'Result') ->element_exists_not('p')->element_exists('table'); # Clean up once we are done $pg->db->query('drop schema linkcheck_test cascade'); done_testing(); Minion-7.05/examples/linkcheck/script/linkcheck000755 000765 000024 00000000352 13123461250 021571 0ustar00sristaff000000 000000 #!/usr/bin/env perl use strict; use warnings; use FindBin; BEGIN { unshift @INC, "$FindBin::Bin/../lib" } # Start command line interface for application require Mojolicious::Commands; Mojolicious::Commands->start_app('LinkCheck'); Minion-7.05/examples/linkcheck/lib/LinkCheck/000755 000765 000024 00000000000 13142141670 021007 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/lib/LinkCheck.pm000644 000765 000024 00000001250 13123461250 021341 0ustar00sristaff000000 000000 package LinkCheck; use Mojo::Base 'Mojolicious'; sub startup { my $self = shift; # Configuration my $config = $self->plugin(Config => {file => 'linkcheck.conf'}); $self->secrets($config->{secrets}); # Job queue (requires a background worker process) # # script/linkcheck minion worker # $self->plugin(Minion => {Pg => $config->{pg}}); $self->plugin('LinkCheck::Task::CheckLinks'); # Controller my $r = $self->routes; $r->get('/' => sub { shift->redirect_to('index') }); $r->get('/links')->to('links#index')->name('index'); $r->post('/links')->to('links#check')->name('check'); $r->get('/links/:id')->to('links#result')->name('result'); } 1; Minion-7.05/examples/linkcheck/lib/LinkCheck/Controller/000755 000765 000024 00000000000 13142141670 023132 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/lib/LinkCheck/Task/000755 000765 000024 00000000000 13142141670 021711 5ustar00sristaff000000 000000 Minion-7.05/examples/linkcheck/lib/LinkCheck/Task/CheckLinks.pm000644 000765 000024 00000001136 13123461250 024264 0ustar00sristaff000000 000000 package LinkCheck::Task::CheckLinks; use Mojo::Base 'Mojolicious::Plugin'; use Mojo::URL; sub register { my ($self, $app) = @_; $app->minion->add_task(check_links => \&_check_links); } sub _check_links { my ($job, $url) = @_; my @results; my $ua = $job->app->ua; my $res = $ua->get($url)->result; push @results, [$url, $res->code]; for my $link ($res->dom->find('a[href]')->map(attr => 'href')->each) { my $abs = Mojo::URL->new($link)->to_abs(Mojo::URL->new($url)); $res = $ua->head($abs)->result; push @results, [$link, $res->code]; } $job->finish(\@results); } 1; Minion-7.05/examples/linkcheck/lib/LinkCheck/Controller/Links.pm000644 000765 000024 00000001102 13123461250 024540 0ustar00sristaff000000 000000 package LinkCheck::Controller::Links; use Mojo::Base 'Mojolicious::Controller'; sub check { my $self = shift; my $validation = $self->validation; $validation->required('url'); return $self->render(action => 'index') if $validation->has_error; my $id = $self->minion->enqueue(check_links => [$validation->param('url')]); $self->redirect_to('result', id => $id); } sub index { } sub result { my $self = shift; return $self->reply->not_found unless my $job = $self->minion->job($self->param('id')); $self->render(result => $job->info->{result}); } 1;