mail_room-0.9.1/0000755000004100000410000000000013113762314013530 5ustar www-datawww-datamail_room-0.9.1/Rakefile0000644000004100000410000000016513113762314015177 0ustar www-datawww-datarequire "bundler/gem_tasks" require 'rspec/core/rake_task' RSpec::Core::RakeTask.new(:spec) task :default => :spec mail_room-0.9.1/bin/0000755000004100000410000000000013113762314014300 5ustar www-datawww-datamail_room-0.9.1/bin/mail_room0000755000004100000410000000011013113762314016174 0ustar www-datawww-data#!/usr/bin/env ruby require 'mail_room' MailRoom::CLI.new(ARGV).start mail_room-0.9.1/Gemfile0000644000004100000410000000013613113762314015023 0ustar www-datawww-datasource 'https://rubygems.org' # Specify your gem's dependencies in mail_room.gemspec gemspec mail_room-0.9.1/.ruby-version0000644000004100000410000000000513113762314016170 0ustar www-datawww-data2.2.2mail_room-0.9.1/CODE_OF_CONDUCT.md0000644000004100000410000000445413113762314016336 0ustar www-datawww-data# Contributor Code of Conduct As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality. Examples of unacceptable behavior by participants include: * The use of sexualized language or imagery * Personal attacks * Trolling or insulting/derogatory comments * Public or private harassment * Publishing other's private information, such as physical or electronic addresses, without explicit permission * Other unethical or unprofessional conduct Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. By adopting this Code of Conduct, project maintainers commit themselves to fairly and consistently applying these principles to every aspect of managing this project. Project maintainers who do not follow or enforce the Code of Conduct may be permanently removed from the project team. This code of conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting a project maintainer at [INSERT EMAIL ADDRESS]. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. Maintainers are obligated to maintain confidentiality with regard to the reporter of an incident. This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.3.0, available at [http://contributor-covenant.org/version/1/3/0/](http://contributor-covenant.org/version/1/3/0/) mail_room-0.9.1/LICENSE.txt0000644000004100000410000000205313113762314015353 0ustar www-datawww-dataCopyright (c) 2013 Tony Pitale MIT License Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.mail_room-0.9.1/spec/0000755000004100000410000000000013113762314014462 5ustar www-datawww-datamail_room-0.9.1/spec/spec_helper.rb0000644000004100000410000000110713113762314017277 0ustar www-datawww-datarequire 'simplecov' SimpleCov.start require 'bundler/setup' require 'rspec' require 'mocha/api' require 'bourne' require 'fakeredis/rspec' require File.expand_path('../../lib/mail_room', __FILE__) RSpec.configure do |config| config.mock_with :mocha config.run_all_when_everything_filtered = true config.filter_run :focus # Run specs in random order to surface order dependencies. If you find an # order dependency and want to debug it, you can fix the order by providing # the seed, which is printed after each run. # --seed 1234 config.order = 'random' end mail_room-0.9.1/spec/lib/0000755000004100000410000000000013113762314015230 5ustar www-datawww-datamail_room-0.9.1/spec/lib/connection_spec.rb0000644000004100000410000000313213113762314020725 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::Connection do let(:imap) {stub} let(:mailbox) {MailRoom::Mailbox.new(delete_after_delivery: true)} before :each do MailRoom::IMAP.stubs(:new).returns(imap) end context "with imap set up" do let(:connection) {MailRoom::Connection.new(mailbox)} before :each do imap.stubs(:starttls) imap.stubs(:login) imap.stubs(:select) end it "is logged in" do expect(connection.logged_in?).to eq(true) end it "is not idling" do expect(connection.idling?).to eq(false) end it "is not disconnected" do imap.stubs(:disconnected?).returns(false) expect(connection.disconnected?).to eq(false) end it "is ready to idle" do expect(connection.ready_to_idle?).to eq(true) end it "waits for a message to process" do new_message = 'a message' new_message.stubs(:seqno).returns(8) connection.on_new_message do |message| expect(message).to eq(new_message) true end mailbox.stubs(:deliver?).returns(true) imap.stubs(:idle) imap.stubs(:uid_search).returns([]).then.returns([1]) imap.stubs(:uid_fetch).returns([new_message]) imap.stubs(:store) connection.wait expect(imap).to have_received(:idle) expect(imap).to have_received(:uid_search).with(mailbox.search_command).twice expect(imap).to have_received(:uid_fetch).with([1], "RFC822") expect(mailbox).to have_received(:deliver?).with(1) expect(imap).to have_received(:store).with(8, "+FLAGS", [Net::IMAP::DELETED]) end end end mail_room-0.9.1/spec/lib/cli_spec.rb0000644000004100000410000000233313113762314017337 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::CLI do let(:config_path) {File.expand_path('../fixtures/test_config.yml', File.dirname(__FILE__))} let!(:configuration) {MailRoom::Configuration.new({:config_path => config_path})} let(:coordinator) {stub(:run => true, :quit => true)} describe '.new' do let(:args) {["-c", "a path"]} before :each do MailRoom::Configuration.stubs(:new).returns(configuration) MailRoom::Coordinator.stubs(:new).returns(coordinator) end it 'parses arguments into configuration' do expect(MailRoom::CLI.new(args).configuration).to eq(configuration) expect(MailRoom::Configuration).to have_received(:new).with({:config_path => 'a path'}) end it 'creates a new coordinator with configuration' do expect(MailRoom::CLI.new(args).coordinator).to eq(coordinator) expect(MailRoom::Coordinator).to have_received(:new).with(configuration.mailboxes) end end describe '#start' do let(:cli) {MailRoom::CLI.new([])} before :each do cli.configuration = configuration cli.coordinator = coordinator end it 'starts running the coordinator' do cli.start expect(coordinator).to have_received(:run) end end end mail_room-0.9.1/spec/lib/mailbox_spec.rb0000644000004100000410000000623313113762314020226 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::Mailbox do describe "#deliver" do context "with arbitration_method of noop" do it 'arbitrates with a Noop instance' do mailbox = MailRoom::Mailbox.new({:arbitration_method => 'noop'}) noop = stub(:deliver?) MailRoom::Arbitration['noop'].stubs(:new => noop) uid = 123 mailbox.deliver?(uid) expect(noop).to have_received(:deliver?).with(uid) end end context "with arbitration_method of redis" do it 'arbitrates with a Redis instance' do mailbox = MailRoom::Mailbox.new({:arbitration_method => 'redis'}) redis = stub(:deliver?) MailRoom::Arbitration['redis'].stubs(:new => redis) uid = 123 mailbox.deliver?(uid) expect(redis).to have_received(:deliver?).with(uid) end end context "with delivery_method of noop" do it 'delivers with a Noop instance' do mailbox = MailRoom::Mailbox.new({:delivery_method => 'noop'}) noop = stub(:deliver) MailRoom::Delivery['noop'].stubs(:new => noop) mailbox.deliver(stub(:attr => {'RFC822' => 'a message'})) expect(noop).to have_received(:deliver).with('a message') end end context "with delivery_method of logger" do it 'delivers with a Logger instance' do mailbox = MailRoom::Mailbox.new({:delivery_method => 'logger'}) logger = stub(:deliver) MailRoom::Delivery['logger'].stubs(:new => logger) mailbox.deliver(stub(:attr => {'RFC822' => 'a message'})) expect(logger).to have_received(:deliver).with('a message') end end context "with delivery_method of postback" do it 'delivers with a Postback instance' do mailbox = MailRoom::Mailbox.new({:delivery_method => 'postback'}) postback = stub(:deliver) MailRoom::Delivery['postback'].stubs(:new => postback) mailbox.deliver(stub(:attr => {'RFC822' => 'a message'})) expect(postback).to have_received(:deliver).with('a message') end end context "with delivery_method of letter_opener" do it 'delivers with a LetterOpener instance' do mailbox = MailRoom::Mailbox.new({:delivery_method => 'letter_opener'}) letter_opener = stub(:deliver) MailRoom::Delivery['letter_opener'].stubs(:new => letter_opener) mailbox.deliver(stub(:attr => {'RFC822' => 'a message'})) expect(letter_opener).to have_received(:deliver).with('a message') end end context "without an RFC822 attribute" do it "doesn't deliver the message" do mailbox = MailRoom::Mailbox.new({:delivery_method => 'noop'}) noop = stub(:deliver) MailRoom::Delivery['noop'].stubs(:new => noop) mailbox.deliver(stub(:attr => {'FLAGS' => [:Seen, :Recent]})) expect(noop).to have_received(:deliver).never end end context "with ssl options hash" do it 'replaces verify mode with constant' do mailbox = MailRoom::Mailbox.new({:ssl => {:verify_mode => :none}}) expect(mailbox.ssl_options).to eq({:verify_mode => OpenSSL::SSL::VERIFY_NONE}) end end end end mail_room-0.9.1/spec/lib/mailbox_watcher_spec.rb0000644000004100000410000000325213113762314021741 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::MailboxWatcher do let(:mailbox) {MailRoom::Mailbox.new} describe '#running?' do it 'is false by default' do watcher = MailRoom::MailboxWatcher.new(mailbox) expect(watcher.running?).to eq(false) end end describe '#run' do let(:imap) {stub(:login => true, :select => true)} let(:watcher) {MailRoom::MailboxWatcher.new(mailbox)} before :each do Net::IMAP.stubs(:new).returns(imap) # prevent connection end it 'loops over wait while running' do connection = MailRoom::Connection.new(mailbox) connection.stubs(:on_new_message) connection.stubs(:wait) MailRoom::Connection.stubs(:new).returns(connection) watcher.stubs(:running?).returns(true).then.returns(false) watcher.run watcher.watching_thread.join # wait for finishing run expect(watcher).to have_received(:running?).times(2) expect(connection).to have_received(:wait).once expect(connection).to have_received(:on_new_message).once end end describe '#quit' do let(:imap) {stub(:login => true, :select => true)} let(:watcher) {MailRoom::MailboxWatcher.new(mailbox)} before :each do Net::IMAP.stubs(:new).returns(imap) # prevent connection end it 'closes and waits for the connection' do connection = MailRoom::Connection.new(mailbox) connection.stubs(:wait) connection.stubs(:quit) MailRoom::Connection.stubs(:new).returns(connection) watcher.run expect(watcher.running?).to eq(true) watcher.quit expect(connection).to have_received(:quit) expect(watcher.running?).to eq(false) end end end mail_room-0.9.1/spec/lib/arbitration/0000755000004100000410000000000013113762314017546 5ustar www-datawww-datamail_room-0.9.1/spec/lib/arbitration/redis_spec.rb0000644000004100000410000000632413113762314022220 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/arbitration/redis' describe MailRoom::Arbitration::Redis do let(:mailbox) { MailRoom::Mailbox.new( arbitration_options: { namespace: "mail_room" } ) } let(:options) { described_class::Options.new(mailbox) } subject { described_class.new(options) } # Private, but we don't care. let(:client) { subject.send(:client) } describe '#deliver?' do context "when called the first time" do it "returns true" do expect(subject.deliver?(123)).to be_truthy end it "increments the delivered flag" do subject.deliver?(123) expect(client.get("delivered:123")).to eq("1") end it "sets an expiration on the delivered flag" do subject.deliver?(123) expect(client.ttl("delivered:123")).to be > 0 end end context "when called the second time" do before do subject.deliver?(123) end it "returns false" do expect(subject.deliver?(123)).to be_falsey end it "increments the delivered flag" do subject.deliver?(123) expect(client.get("delivered:123")).to eq("2") end end context "when called for another uid" do before do subject.deliver?(123) end it "returns true" do expect(subject.deliver?(234)).to be_truthy end end end context 'redis client connection params' do context 'when only url is present' do let(:redis_url) { "redis://redis.example.com:8888" } let(:mailbox) { MailRoom::Mailbox.new( arbitration_options: { redis_url: redis_url } ) } it 'client has same specified url' do subject.deliver?(123) expect(client.options[:url]).to eq redis_url end it 'client is a instance of Redis class' do expect(client).to be_a Redis end end context 'when namespace is present' do let(:namespace) { 'mail_room' } let(:mailbox) { MailRoom::Mailbox.new( arbitration_options: { namespace: namespace } ) } it 'client has same specified namespace' do expect(client.namespace).to eq(namespace) end it 'client is a instance of RedisNamespace class' do expect(client).to be_a ::Redis::Namespace end end context 'when sentinel is present' do let(:redis_url) { 'redis://:mypassword@sentinel-master:6379' } let(:sentinels) { [{ host: '10.0.0.1', port: '26379' }] } let(:mailbox) { MailRoom::Mailbox.new( arbitration_options: { redis_url: redis_url, sentinels: sentinels } ) } before { ::Redis::Client::Connector::Sentinel.any_instance.stubs(:resolve).returns(sentinels) } it 'client has same specified sentinel params' do expect(client.client.instance_variable_get(:@connector)).to be_a Redis::Client::Connector::Sentinel expect(client.client.options[:host]).to eq('sentinel-master') expect(client.client.options[:password]).to eq('mypassword') expect(client.client.options[:sentinels]).to eq(sentinels) end end end end mail_room-0.9.1/spec/lib/configuration_spec.rb0000644000004100000410000000135013113762314021435 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::Configuration do let(:config_path) {File.expand_path('../fixtures/test_config.yml', File.dirname(__FILE__))} describe 'set_mailboxes' do it 'parses yaml into mailbox objects' do MailRoom::Mailbox.stubs(:new).returns('mailbox1', 'mailbox2') configuration = MailRoom::Configuration.new(:config_path => config_path) expect(configuration.mailboxes).to eq(['mailbox1', 'mailbox2']) end it 'sets mailboxes to an empty set when config_path is missing' do MailRoom::Mailbox.stubs(:new) configuration = MailRoom::Configuration.new expect(configuration.mailboxes).to eq([]) expect(MailRoom::Mailbox).to have_received(:new).never end end end mail_room-0.9.1/spec/lib/coordinator_spec.rb0000644000004100000410000000363713113762314021123 0ustar www-datawww-datarequire 'spec_helper' describe MailRoom::Coordinator do describe '#initialize' do it 'builds a watcher for each mailbox' do MailRoom::MailboxWatcher.stubs(:new).returns('watcher1', 'watcher2') coordinator = MailRoom::Coordinator.new(['mailbox1', 'mailbox2']) expect(coordinator.watchers).to eq(['watcher1', 'watcher2']) expect(MailRoom::MailboxWatcher).to have_received(:new).with('mailbox1') expect(MailRoom::MailboxWatcher).to have_received(:new).with('mailbox2') end it 'makes no watchers when mailboxes is empty' do coordinator = MailRoom::Coordinator.new([]) expect(coordinator.watchers).to eq([]) end end describe '#run' do it 'runs each watcher' do watcher = stub watcher.stubs(:run) watcher.stubs(:quit) MailRoom::MailboxWatcher.stubs(:new).returns(watcher) coordinator = MailRoom::Coordinator.new(['mailbox1']) coordinator.stubs(:sleep_while_running) coordinator.run expect(watcher).to have_received(:run) expect(watcher).to have_received(:quit) end it 'should go to sleep after running watchers' do coordinator = MailRoom::Coordinator.new([]) coordinator.stubs(:running=) coordinator.stubs(:running?).returns(false) coordinator.run expect(coordinator).to have_received(:running=).with(true) expect(coordinator).to have_received(:running?) end it 'should set attribute running to true' do coordinator = MailRoom::Coordinator.new([]) coordinator.stubs(:sleep_while_running) coordinator.run expect(coordinator.running).to eq(true) end end describe '#quit' do it 'quits each watcher' do watcher = stub(:quit) MailRoom::MailboxWatcher.stubs(:new).returns(watcher) coordinator = MailRoom::Coordinator.new(['mailbox1']) coordinator.quit expect(watcher).to have_received(:quit) end end end mail_room-0.9.1/spec/lib/delivery/0000755000004100000410000000000013113762314017053 5ustar www-datawww-datamail_room-0.9.1/spec/lib/delivery/que_spec.rb0000644000004100000410000000214313113762314021204 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/delivery/que' describe MailRoom::Delivery::Que do describe '#deliver' do let(:mailbox) {MailRoom::Mailbox.new({ delivery_options: { database: 'delivery_test', username: 'postgres', password: '', queue: 'default', priority: 5, job_class: 'ParseMailJob' } })} let(:connection) {stub} let(:options) {MailRoom::Delivery::Que::Options.new(mailbox)} it 'stores the message in que_jobs table' do PG.stubs(:connect).returns(connection) connection.stubs(:exec) MailRoom::Delivery::Que.new(options).deliver('email') expect(PG).to have_received(:connect).with({ host: 'localhost', port: 5432, dbname: 'delivery_test', user: 'postgres', password: '' }) expect(connection).to have_received(:exec).with( "INSERT INTO que_jobs (priority, job_class, queue, args) VALUES ($1, $2, $3, $4)", [ 5, 'ParseMailJob', 'default', JSON.dump(['email']) ] ) end end end mail_room-0.9.1/spec/lib/delivery/logger_spec.rb0000644000004100000410000000227213113762314021674 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/delivery/logger' describe MailRoom::Delivery::Logger do describe '#initialize' do context "without a log path" do let(:mailbox) {MailRoom::Mailbox.new} it 'creates a new ruby logger' do ::Logger.stubs(:new) MailRoom::Delivery::Logger.new(mailbox) expect(::Logger).to have_received(:new).with(STDOUT) end end context "with a log path" do let(:mailbox) {MailRoom::Mailbox.new(:log_path => '/var/log/mail-room.log')} it 'creates a new file to append to' do ::Logger.stubs(:new) file = stub(:sync=) ::File.stubs(:open).returns(file) MailRoom::Delivery::Logger.new(mailbox) expect(File).to have_received(:open).with('/var/log/mail-room.log', 'a') expect(::Logger).to have_received(:new).with(file) end end end describe '#deliver' do let(:mailbox) {MailRoom::Mailbox.new} it 'writes the message to info' do logger = stub(:info) ::Logger.stubs(:new).returns(logger) MailRoom::Delivery::Logger.new(mailbox).deliver('a message') expect(logger).to have_received(:info).with('a message') end end end mail_room-0.9.1/spec/lib/delivery/sidekiq_spec.rb0000644000004100000410000000427113113762314022047 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/delivery/sidekiq' describe MailRoom::Delivery::Sidekiq do subject { described_class.new(options) } let(:redis) { subject.send(:client) } let(:options) { MailRoom::Delivery::Sidekiq::Options.new(mailbox) } describe '#options' do let(:redis_url) { 'redis://redis.example.com' } context 'when only redis_url is specified' do let(:mailbox) { MailRoom::Mailbox.new( delivery_method: :sidekiq, delivery_options: { redis_url: redis_url } ) } it 'client has same specified redis_url' do expect(redis.options[:url]).to eq(redis_url) end it 'client is a instance of RedisNamespace class' do expect(redis).to be_a ::Redis end end context 'when namespace is specified' do let(:namespace) { 'sidekiq_mailman' } let(:mailbox) { MailRoom::Mailbox.new( delivery_method: :sidekiq, delivery_options: { redis_url: redis_url, namespace: namespace } ) } it 'client has same specified namespace' do expect(redis.namespace).to eq(namespace) end it 'client is a instance of RedisNamespace class' do expect(redis).to be_a ::Redis::Namespace end end context 'when sentinel is specified' do let(:redis_url) { 'redis://:mypassword@sentinel-master:6379' } let(:sentinels) { [{ host: '10.0.0.1', port: '26379' }] } let(:mailbox) { MailRoom::Mailbox.new( delivery_method: :sidekiq, delivery_options: { redis_url: redis_url, sentinels: sentinels } ) } before { ::Redis::Client::Connector::Sentinel.any_instance.stubs(:resolve).returns(sentinels) } it 'client has same specified sentinel params' do expect(redis.client.instance_variable_get(:@connector)).to be_a Redis::Client::Connector::Sentinel expect(redis.client.options[:host]).to eq('sentinel-master') expect(redis.client.options[:password]).to eq('mypassword') expect(redis.client.options[:sentinels]).to eq(sentinels) end end end end mail_room-0.9.1/spec/lib/delivery/letter_opener_spec.rb0000644000004100000410000000163713113762314023270 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/delivery/letter_opener' describe MailRoom::Delivery::LetterOpener do describe '#deliver' do let(:mailbox) {MailRoom::Mailbox.new(:location => '/tmp/somewhere')} let(:delivery_method) {stub(:deliver!)} let(:mail) {stub} before :each do Mail.stubs(:read_from_string).returns(mail) ::LetterOpener::DeliveryMethod.stubs(:new).returns(delivery_method) MailRoom::Delivery::LetterOpener.new(mailbox).deliver('a message') end it 'creates a new LetterOpener::DeliveryMethod' do expect(::LetterOpener::DeliveryMethod).to have_received(:new).with(:location => '/tmp/somewhere') end it 'parses the message string with Mail' do expect(::Mail).to have_received(:read_from_string).with('a message') end it 'delivers the mail message' do expect(delivery_method).to have_received(:deliver!).with(mail) end end endmail_room-0.9.1/spec/lib/delivery/postback_spec.rb0000644000004100000410000000156613113762314022230 0ustar www-datawww-datarequire 'spec_helper' require 'mail_room/delivery/postback' describe MailRoom::Delivery::Postback do describe '#deliver' do let(:mailbox) {MailRoom::Mailbox.new({ :delivery_url => 'http://localhost/inbox', :delivery_token => 'abcdefg' })} it 'posts the message with faraday' do connection = stub request = stub Faraday.stubs(:new).returns(connection) connection.stubs(:token_auth) connection.stubs(:post).yields(request) request.stubs(:url) request.stubs(:body=) MailRoom::Delivery::Postback.new(mailbox).deliver('a message') expect(connection).to have_received(:token_auth).with('abcdefg') expect(connection).to have_received(:post) expect(request).to have_received(:url).with('http://localhost/inbox') expect(request).to have_received(:body=).with('a message') end end end mail_room-0.9.1/spec/fixtures/0000755000004100000410000000000013113762314016333 5ustar www-datawww-datamail_room-0.9.1/spec/fixtures/test_config.yml0000644000004100000410000000051613113762314021364 0ustar www-datawww-data--- :mailboxes: - :email: "user1@gmail.com" :password: "password" :name: "inbox" :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" - :email: "user2@gmail.com" :password: "password" :name: "inbox" :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" mail_room-0.9.1/.travis.yml0000644000004100000410000000014713113762314015643 0ustar www-datawww-datalanguage: ruby rvm: - 2.0.0 - 2.1.5 - 2.2.4 - 2.3.0 script: bundle exec rspec spec sudo: false mail_room-0.9.1/lib/0000755000004100000410000000000013113762314014276 5ustar www-datawww-datamail_room-0.9.1/lib/mail_room.rb0000644000004100000410000000050313113762314016577 0ustar www-datawww-datarequire 'net/imap' require 'optparse' require 'yaml' module MailRoom end require "mail_room/version" require "mail_room/backports/imap" require "mail_room/configuration" require "mail_room/mailbox" require "mail_room/mailbox_watcher" require "mail_room/connection" require "mail_room/coordinator" require "mail_room/cli" mail_room-0.9.1/lib/mail_room/0000755000004100000410000000000013113762314016254 5ustar www-datawww-datamail_room-0.9.1/lib/mail_room/configuration.rb0000644000004100000410000000167013113762314021454 0ustar www-datawww-datarequire "erb" module MailRoom # Wraps configuration for a set of individual mailboxes with global config # @author Tony Pitale class Configuration attr_accessor :mailboxes, :log_path, :quiet # Initialize a new configuration of mailboxes def initialize(options={}) self.mailboxes = [] self.quiet = options.fetch(:quiet, false) if options.has_key?(:config_path) begin erb = ERB.new(File.read(options[:config_path])) erb.filename = options[:config_path] config_file = YAML.load(erb.result) set_mailboxes(config_file[:mailboxes]) rescue => e raise e unless quiet end end end # Builds individual mailboxes from YAML configuration # # @param mailboxes_config def set_mailboxes(mailboxes_config) mailboxes_config.each do |attributes| self.mailboxes << Mailbox.new(attributes) end end end end mail_room-0.9.1/lib/mail_room/mailbox_watcher.rb0000644000004100000410000000214613113762314021754 0ustar www-datawww-datamodule MailRoom # TODO: split up between processing and idling? # Watch a Mailbox # @author Tony Pitale class MailboxWatcher attr_accessor :watching_thread # Watch a new mailbox # @param mailbox [MailRoom::Mailbox] the mailbox to watch def initialize(mailbox) @mailbox = mailbox @running = false @connection = nil end # are we running? # @return [Boolean] def running? @running end # run the mailbox watcher def run @running = true connection.on_new_message do |message| @mailbox.deliver(message) end self.watching_thread = Thread.start do while(running?) do connection.wait end end watching_thread.abort_on_exception = true end # stop running, cleanup connection def quit @running = false if @connection @connection.quit @connection = nil end if self.watching_thread self.watching_thread.join end end private def connection @connection ||= Connection.new(@mailbox) end end end mail_room-0.9.1/lib/mail_room/arbitration.rb0000644000004100000410000000037713113762314021126 0ustar www-datawww-datamodule MailRoom module Arbitration def [](name) require_relative("./arbitration/#{name}") case name when "redis" Arbitration::Redis else Arbitration::Noop end end module_function :[] end end mail_room-0.9.1/lib/mail_room/connection.rb0000644000004100000410000000762713113762314020754 0ustar www-datawww-datamodule MailRoom class Connection def initialize(mailbox) @mailbox = mailbox # log in and set the mailbox reset setup end def on_new_message(&block) @new_message_handler = block end # is the connection logged in? # @return [Boolean] def logged_in? @logged_in end # is the connection blocked idling? # @return [Boolean] def idling? @idling end # is the imap connection closed? # @return [Boolean] def disconnected? @imap.disconnected? end # is the connection ready to idle? # @return [Boolean] def ready_to_idle? logged_in? && !idling? end def quit stop_idling reset end def wait begin # in case we missed any between idles process_mailbox idle process_mailbox rescue Net::IMAP::Error, IOError reset setup end end private def reset @imap = nil @logged_in = false @idling = false end def setup start_tls log_in set_mailbox end # build a net/imap connection to google imap def imap @imap ||= MailRoom::IMAP.new(@mailbox.host, :port => @mailbox.port, :ssl => @mailbox.ssl_options) end # start a TLS session def start_tls imap.starttls if @mailbox.start_tls end # send the imap login command to google def log_in imap.login(@mailbox.email, @mailbox.password) @logged_in = true end # select the mailbox name we want to use def set_mailbox imap.select(@mailbox.name) if logged_in? end # is the response for a new message? # @param response [Net::IMAP::TaggedResponse] the imap response from idle # @return [Boolean] def message_exists?(response) response.respond_to?(:name) && response.name == 'EXISTS' end # @private def idle_handler lambda {|response| imap.idle_done if message_exists?(response)} end # maintain an imap idle connection def idle return unless ready_to_idle? @idling = true imap.idle(@mailbox.idle_timeout, &idle_handler) ensure @idling = false end # trigger the idle to finish and wait for the thread to finish def stop_idling return unless idling? imap.idle_done # idling_thread.join # self.idling_thread = nil end def process_mailbox return unless @new_message_handler msgs = new_messages msgs. map(&@new_message_handler). # deliver each new message, collect success zip(msgs). # include messages with success select(&:first).map(&:last). # filter failed deliveries, collect message each {|message| scrub(message)} # scrub delivered messages end def scrub(message) if @mailbox.delete_after_delivery imap.store(message.seqno, "+FLAGS", [Net::IMAP::DELETED]) end end # @private # fetch all messages for the new message ids def new_messages # Both of these calls may results in # imap raising an EOFError, we handle # this exception in the watcher messages_for_ids(new_message_ids) end # TODO: label messages? # @imap.store(id, "+X-GM-LABELS", [label]) # @private # search for all new (unseen) message ids # @return [Array] message ids def new_message_ids # uid_search still leaves messages UNSEEN imap.uid_search(@mailbox.search_command).select { |uid| @mailbox.deliver?(uid) } end # @private # fetch the email for all given ids in RFC822 format # @param ids [Array] list of message ids # @return [Array] the net/imap messages for the given ids def messages_for_ids(uids) return [] if uids.empty? # uid_fetch marks as SEEN, will not be re-fetched for UNSEEN imap.uid_fetch(uids, "RFC822") end end end mail_room-0.9.1/lib/mail_room/mailbox.rb0000644000004100000410000000656613113762314020251 0ustar www-datawww-datarequire "mail_room/delivery" require "mail_room/arbitration" module MailRoom # Mailbox Configuration fields MAILBOX_FIELDS = [ :email, :password, :host, :port, :ssl, :start_tls, :idle_timeout, :search_command, :name, :delete_after_delivery, :delivery_method, # :noop, :logger, :postback, :letter_opener :log_path, # for logger :delivery_url, # for postback :delivery_token, # for postback :location, # for letter_opener :delivery_options, :arbitration_method, :arbitration_options ] IdleTimeoutTooLarge = Class.new(RuntimeError) # Holds configuration for each of the email accounts we wish to monitor # and deliver email to when new emails arrive over imap Mailbox = Struct.new(*MAILBOX_FIELDS) do # Keep it to 29 minutes or less # The IMAP serve will close the connection after 30 minutes of inactivity # (which sending IDLE and then nothing technically is), so we re-idle every # 29 minutes, as suggested by the spec: https://tools.ietf.org/html/rfc2177 IMAP_IDLE_TIMEOUT = 29 * 60 # 29 minutes in in seconds # Default attributes for the mailbox configuration DEFAULTS = { :search_command => 'UNSEEN', :delivery_method => 'postback', :host => 'imap.gmail.com', :port => 993, :ssl => true, :start_tls => false, :idle_timeout => IMAP_IDLE_TIMEOUT, :delete_after_delivery => false, :delivery_options => {}, :arbitration_method => 'noop', :arbitration_options => {} } # Store the configuration and require the appropriate delivery method # @param attributes [Hash] configuration options def initialize(attributes={}) super(*DEFAULTS.merge(attributes).values_at(*members)) validate! end def delivery_klass Delivery[delivery_method] end def arbitration_klass Arbitration[arbitration_method] end def delivery @delivery ||= delivery_klass.new(parsed_delivery_options) end def arbitrator @arbitrator ||= arbitration_klass.new(parsed_arbitration_options) end def deliver?(uid) arbitrator.deliver?(uid) end # deliver the imap email message # @param message [Net::IMAP::FetchData] def deliver(message) body = message.attr['RFC822'] return true unless body delivery.deliver(body) end # true, false, or ssl options hash def ssl_options replace_verify_mode(ssl) end def validate! if self[:idle_timeout] > IMAP_IDLE_TIMEOUT raise IdleTimeoutTooLarge.new("Please use an idle timeout smaller than #{29*60} to prevent IMAP server disconnects") end end private def parsed_arbitration_options arbitration_klass::Options.new(self) end def parsed_delivery_options delivery_klass::Options.new(self) end def replace_verify_mode(options) return options unless options.is_a?(Hash) return options unless options.has_key?(:verify_mode) options[:verify_mode] = case options[:verify_mode] when :none, 'none' OpenSSL::SSL::VERIFY_NONE when :peer, 'peer' OpenSSL::SSL::VERIFY_PEER when :client_once, 'client_once' OpenSSL::SSL::VERIFY_CLIENT_ONCE when :fail_if_no_peer_cert, 'fail_if_no_peer_cert' OpenSSL::SSL::VERIFY_FAIL_IF_NO_PEER_CERT end options end end end mail_room-0.9.1/lib/mail_room/cli.rb0000644000004100000410000000261713113762314017356 0ustar www-datawww-datamodule MailRoom # The CLI parses ARGV into configuration to start the coordinator with. # @author Tony Pitale class CLI attr_accessor :configuration, :coordinator # Initialize a new CLI instance to handle option parsing from arguments # into configuration to start the coordinator running on all mailboxes # # @param args [Array] `ARGV` passed from `bin/mail_room` def initialize(args) options = {} OptionParser.new do |parser| parser.banner = [ "Usage: #{@name} [-c config_file]\n", " #{@name} --help\n" ].compact.join parser.on('-c', '--config FILE') do |path| options[:config_path] = path end parser.on('-q', '--quiet') do options[:quiet] = true end # parser.on("-l", "--log FILE") do |path| # options[:log_path] = path # end parser.on_tail("-?", "--help", "Display this usage information.") do puts "#{parser}\n" exit end end.parse!(args) self.configuration = Configuration.new(options) self.coordinator = Coordinator.new(configuration.mailboxes) end # Start the coordinator running, sets up signal traps def start Signal.trap(:INT) do coordinator.running = false end Signal.trap(:TERM) do exit end coordinator.run end end end mail_room-0.9.1/lib/mail_room/version.rb0000644000004100000410000000011413113762314020262 0ustar www-datawww-datamodule MailRoom # Current version of MailRoom gem VERSION = "0.9.1" end mail_room-0.9.1/lib/mail_room/arbitration/0000755000004100000410000000000013113762314020572 5ustar www-datawww-datamail_room-0.9.1/lib/mail_room/arbitration/redis.rb0000644000004100000410000000341113113762314022224 0ustar www-datawww-datarequire "redis" module MailRoom module Arbitration class Redis Options = Struct.new(:redis_url, :namespace, :sentinels) do def initialize(mailbox) redis_url = mailbox.arbitration_options[:redis_url] || "redis://localhost:6379" namespace = mailbox.arbitration_options[:namespace] sentinels = mailbox.arbitration_options[:sentinels] super(redis_url, namespace, sentinels) end end # Expire after 10 minutes so Redis doesn't get filled up with outdated data. EXPIRATION = 600 attr_accessor :options def initialize(options) @options = options end def deliver?(uid) key = "delivered:#{uid}" incr = nil client.multi do |c| # At this point, `incr` is a future, which will get its value after # the MULTI command returns. incr = c.incr(key) c.expire(key, EXPIRATION) end # If INCR returns 1, that means the key didn't exist before, which means # we are the first mail_room to try to deliver this message, so we get to. # If we get any other value, another mail_room already (tried to) deliver # the message, so we don't have to anymore. incr.value == 1 end private def client @client ||= begin sentinels = options.sentinels redis_options = { url: options.redis_url } redis_options[:sentinels] = sentinels if sentinels redis = ::Redis.new(redis_options) namespace = options.namespace if namespace require 'redis/namespace' ::Redis::Namespace.new(namespace, redis: redis) else redis end end end end end end mail_room-0.9.1/lib/mail_room/arbitration/noop.rb0000644000004100000410000000036613113762314022077 0ustar www-datawww-datamodule MailRoom module Arbitration class Noop Options = Class.new do def initialize(*) super() end end def initialize(*) end def deliver?(*) true end end end end mail_room-0.9.1/lib/mail_room/backports/0000755000004100000410000000000013113762314020244 5ustar www-datawww-datamail_room-0.9.1/lib/mail_room/backports/imap.rb0000644000004100000410000000166313113762314021525 0ustar www-datawww-datamodule MailRoom class IMAP < Net::IMAP # Backported 2.3.0 version of net/imap idle command to support timeout def idle(timeout = nil, &response_handler) raise LocalJumpError, "no block given" unless response_handler response = nil synchronize do tag = Thread.current[:net_imap_tag] = generate_tag put_string("#{tag} IDLE#{CRLF}") begin add_response_handler(response_handler) @idle_done_cond = new_cond @idle_done_cond.wait(timeout) @idle_done_cond = nil if @receiver_thread_terminating raise Net::IMAP::Error, "connection closed" end ensure unless @receiver_thread_terminating remove_response_handler(response_handler) put_string("DONE#{CRLF}") response = get_tagged_response(tag, "IDLE") end end end return response end end end mail_room-0.9.1/lib/mail_room/delivery.rb0000644000004100000410000000066613113762314020434 0ustar www-datawww-datamodule MailRoom module Delivery def [](name) require_relative("./delivery/#{name}") case name when "postback" Delivery::Postback when "logger" Delivery::Logger when "letter_opener" Delivery::LetterOpener when "sidekiq" Delivery::Sidekiq when "que" Delivery::Que else Delivery::Noop end end module_function :[] end end mail_room-0.9.1/lib/mail_room/delivery/0000755000004100000410000000000013113762314020077 5ustar www-datawww-datamail_room-0.9.1/lib/mail_room/delivery/sidekiq.rb0000644000004100000410000000474113113762314022063 0ustar www-datawww-datarequire "redis" require "securerandom" require "json" require "charlock_holmes" module MailRoom module Delivery # Sidekiq Delivery method # @author Douwe Maan class Sidekiq Options = Struct.new(:redis_url, :namespace, :sentinels, :queue, :worker) do def initialize(mailbox) redis_url = mailbox.delivery_options[:redis_url] || "redis://localhost:6379" namespace = mailbox.delivery_options[:namespace] sentinels = mailbox.delivery_options[:sentinels] queue = mailbox.delivery_options[:queue] || "default" worker = mailbox.delivery_options[:worker] super(redis_url, namespace, sentinels, queue, worker) end end attr_accessor :options # Build a new delivery, hold the mailbox configuration # @param [MailRoom::Delivery::Sidekiq::Options] def initialize(options) @options = options end # deliver the message by pushing it onto the configured Sidekiq queue # @param message [String] the email message as a string, RFC822 format def deliver(message) item = item_for(message) client.lpush("queue:#{options.queue}", JSON.generate(item)) true end private def client @client ||= begin sentinels = options.sentinels redis_options = { url: options.redis_url } redis_options[:sentinels] = sentinels if sentinels redis = ::Redis.new(redis_options) namespace = options.namespace if namespace require 'redis/namespace' Redis::Namespace.new(namespace, redis: redis) else redis end end end def item_for(message) { 'class' => options.worker, 'args' => [utf8_encode_message(message)], 'queue' => options.queue, 'jid' => SecureRandom.hex(12), 'retry' => false, 'enqueued_at' => Time.now.to_f } end def utf8_encode_message(message) message = message.dup message.force_encoding("UTF-8") return message if message.valid_encoding? detection = CharlockHolmes::EncodingDetector.detect(message) return message unless detection && detection[:encoding] # Convert non-UTF-8 body UTF-8 so it can be dumped as JSON. CharlockHolmes::Converter.convert(message, detection[:encoding], 'UTF-8') end end end end mail_room-0.9.1/lib/mail_room/delivery/letter_opener.rb0000644000004100000410000000164313113762314023277 0ustar www-datawww-datarequire 'erb' require 'mail' require 'letter_opener' module MailRoom module Delivery # LetterOpener Delivery method # @author Tony Pitale class LetterOpener Options = Struct.new(:location) do def initialize(mailbox) location = mailbox.location || mailbox.delivery_options[:location] super(location) end end # Build a new delivery, hold the delivery options # @param [MailRoom::Delivery::LetterOpener::Options] def initialize(delivery_options) @delivery_options = delivery_options end # Trigger `LetterOpener` to deliver our message # @param message [String] the email message as a string, RFC822 format def deliver(message) method = ::LetterOpener::DeliveryMethod.new(:location => @delivery_options.location) method.deliver!(Mail.read_from_string(message)) true end end end end mail_room-0.9.1/lib/mail_room/delivery/logger.rb0000644000004100000410000000165413113762314021711 0ustar www-datawww-datarequire 'logger' module MailRoom module Delivery # File/STDOUT Logger Delivery method # @author Tony Pitale class Logger Options = Struct.new(:log_path) do def initialize(mailbox) log_path = mailbox.log_path || mailbox.delivery_options[:log_path] super(log_path) end end # Build a new delivery, hold the delivery options # open a file or stdout for IO depending on the options # @param [MailRoom::Delivery::Logger::Options] def initialize(delivery_options) io = File.open(delivery_options.log_path, 'a') if delivery_options.log_path io ||= STDOUT io.sync = true @logger = ::Logger.new(io) end # Write the message to our logger # @param message [String] the email message as a string, RFC822 format def deliver(message) @logger.info message true end end end end mail_room-0.9.1/lib/mail_room/delivery/postback.rb0000644000004100000410000000233313113762314022233 0ustar www-datawww-datarequire 'faraday' module MailRoom module Delivery # Postback Delivery method # @author Tony Pitale class Postback Options = Struct.new(:delivery_url, :delivery_token) do def initialize(mailbox) delivery_url = mailbox.delivery_url || mailbox.delivery_options[:delivery_url] delivery_token = mailbox.delivery_token || mailbox.delivery_options[:delivery_token] super(delivery_url, delivery_token) end end # Build a new delivery, hold the delivery options # @param [MailRoom::Delivery::Postback::Options] def initialize(delivery_options) @delivery_options = delivery_options end # deliver the message using Faraday to the configured delivery_options url # @param message [String] the email message as a string, RFC822 format def deliver(message) connection = Faraday.new connection.token_auth @delivery_options.delivery_token connection.post do |request| request.url @delivery_options.delivery_url request.body = message # request.options[:timeout] = 3 # request.headers['Content-Type'] = 'text/plain' end true end end end end mail_room-0.9.1/lib/mail_room/delivery/que.rb0000644000004100000410000000351313113762314021220 0ustar www-datawww-datarequire 'pg' require 'json' module MailRoom module Delivery # Que Delivery method # @author Tony Pitale class Que Options = Struct.new(:host, :port, :database, :username, :password, :queue, :priority, :job_class) do def initialize(mailbox) host = mailbox.delivery_options[:host] || "localhost" port = mailbox.delivery_options[:port] || 5432 database = mailbox.delivery_options[:database] username = mailbox.delivery_options[:username] password = mailbox.delivery_options[:password] queue = mailbox.delivery_options[:queue] || '' priority = mailbox.delivery_options[:priority] || 100 # lowest priority for Que job_class = mailbox.delivery_options[:job_class] super(host, port, database, username, password, queue, priority, job_class) end end attr_reader :options # Build a new delivery, hold the mailbox configuration # @param [MailRoom::Delivery::Que::Options] def initialize(options) @options = options end # deliver the message by pushing it onto the configured Sidekiq queue # @param message [String] the email message as a string, RFC822 format def deliver(message) queue_job(message) end private def connection PG.connect(connection_options) end def connection_options { host: options.host, port: options.port, dbname: options.database, user: options.username, password: options.password } end def queue_job(*args) sql = "INSERT INTO que_jobs (priority, job_class, queue, args) VALUES ($1, $2, $3, $4)" connection.exec(sql, [options.priority, options.job_class, options.queue, JSON.dump(args)]) end end end end mail_room-0.9.1/lib/mail_room/delivery/noop.rb0000644000004100000410000000057013113762314021401 0ustar www-datawww-datamodule MailRoom module Delivery # Noop Delivery method # @author Tony Pitale class Noop Options = Class.new do def initialize(*) super() end end # build a new delivery, do nothing def initialize(*) end # accept the delivery, do nothing def deliver(*) true end end end end mail_room-0.9.1/lib/mail_room/coordinator.rb0000644000004100000410000000162513113762314021130 0ustar www-datawww-datamodule MailRoom # Coordinate the mailbox watchers # @author Tony Pitale class Coordinator attr_accessor :watchers, :running # build watchers for a set of mailboxes # @params mailboxes [Array] mailboxes to be watched def initialize(mailboxes) self.watchers = [] mailboxes.each {|box| self.watchers << MailboxWatcher.new(box)} end alias :running? :running # start each of the watchers to running def run watchers.each(&:run) self.running = true sleep_while_running ensure quit end # quit each of the watchers when we're done running def quit watchers.each(&:quit) end private # @private def sleep_while_running # do we need to sweep for dead watchers? # or do we let the mailbox rebuild connections while(running?) do; sleep 1; end end end end mail_room-0.9.1/.gitlab-ci.yml0000644000004100000410000000136613113762314016172 0ustar www-datawww-data# Cache gems in between builds .test-template: &test cache: paths: - vendor/ruby script: - bundle exec rspec spec before_script: - apt update && apt install -y libicu-dev - ruby -v # Print out ruby version for debugging # Uncomment next line if your rails app needs a JS runtime: # - apt-get update -q && apt-get install nodejs -yqq - gem install bundler --no-ri --no-rdoc # Bundler is not installed with the image - bundle install -j $(nproc) --path vendor # Install dependencies into ./vendor/ruby rspec-2.0: image: "ruby:2.0" <<: *test rspec-2.1: image: "ruby:2.1" <<: *test rspec-2.2: image: "ruby:2.2" <<: *test rspec-2.3: image: "ruby:2.3" <<: *test mail_room-0.9.1/.gitignore0000644000004100000410000000023513113762314015520 0ustar www-datawww-data*.gem *.rbc .bundle .config .yardoc Gemfile.lock InstalledFiles _yardoc coverage doc/ lib/bundler/man pkg rdoc spec/reports test/tmp test/version_tmp tmp logmail_room-0.9.1/mail_room.gemspec0000644000004100000410000000256113113762314017057 0ustar www-datawww-data# -*- encoding: utf-8 -*- lib = File.expand_path('../lib', __FILE__) $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) require 'mail_room/version' Gem::Specification.new do |gem| gem.name = "mail_room" gem.version = MailRoom::VERSION gem.authors = ["Tony Pitale"] gem.email = ["tpitale@gmail.com"] gem.description = %q{mail_room will proxy email (gmail) from IMAP to a delivery method} gem.summary = %q{mail_room will proxy email (gmail) from IMAP to a callback URL, logger, or letter_opener} gem.homepage = "http://github.com/tpitale/mail_room" gem.files = `git ls-files`.split($/) gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) } gem.test_files = gem.files.grep(%r{^(test|spec|features)/}) gem.require_paths = ["lib"] gem.add_development_dependency "rake" gem.add_development_dependency "rspec" gem.add_development_dependency "mocha" gem.add_development_dependency "bourne" gem.add_development_dependency "simplecov" gem.add_development_dependency "fakeredis" # for testing delivery methods gem.add_development_dependency "faraday" gem.add_development_dependency "mail" gem.add_development_dependency "letter_opener" gem.add_development_dependency "redis-namespace" gem.add_development_dependency "pg" gem.add_development_dependency "charlock_holmes" end mail_room-0.9.1/CHANGELOG.md0000644000004100000410000000424313113762314015344 0ustar www-datawww-data## mail_room 0.9.1 ## * __FILE__ support in yml ERb config - PR#80 *Gabriel Mazetto <@brodock>* ## mail_room 0.9.0 ## * Redis Sentinel configuration support - PR#79 *Gabriel Mazetto <@brodock>* ## mail_room 0.8.1 ## * Check watching thread exists before joining - PR#78 *Michal Galet <@galet>* ## mail_room 0.8.0 ## * Rework the mailbox watcher and handler into a new Connection class to abstract away IMAP handling details *Tony Pitale <@tpitale>* ## mail_room 0.7.0 ## * Backports idle timeout from ruby 2.3.0 * Sets default to 29 minutes to prevent IMAP disconnects * Validates that the timeout does not exceed 29 minutes *Tony Pitale <@tpitale>* ## mail_room 0.6.1 ## * ERB parsing of configuration yml file to enable using ENV variables *Douwe Maan <@DouweM>* ## mail_room 0.6.0 ## * Add redis Arbitration to reduce multiple deliveries of the same message when running multiple MailRoom instances on the same inbox *Douwe Maan <@DouweM>* ## mail_room 0.5.2 ## * Fix Sidekiq delivery method for non-UTF8 email *Douwe Maan <@DouweM>* * Add StartTLS session support *Tony Pitale <@tpitale>* ## mail_room 0.5.1 ## * Re-idle after 29 minutes to maintain IDLE connection *Douwe Maan <@DouweM>* ## mail_room 0.5.0 ## * Que delivery method *Tony Pitale <@tpitale>* ## mail_room 0.4.2 ## * rescue from all IOErrors, not just EOFError *Douwe Maan <@DouweM>* ## mail_room 0.4.1 ## * Fix redis default host/port configuration * Mailbox does not attempt delivery without a message *Douwe Maan <@DouweM>* ## mail_room 0.4.0 ## * Sidekiq delivery method * Option to delete messages after delivered *Douwe Maan <@DouweM>* * -q/--quiet do not raise errors on missing configuration * prefetch mail messages before idling * delivery-method-specific delivery options configuration *Tony Pitale <@tpitale>* ## mail_room 0.3.1 ## * Rescue from EOFError and re-setup mailroom *Tony Pitale <@tpitale>* ## mail_room 0.3.0 ## * Reconnect and idle if disconnected during an existing idle. * Set idling thread to abort on exception so any unhandled exceptions will stop mail_room running. *Tony Pitale <@tpitale>* mail_room-0.9.1/README.md0000644000004100000410000002475313113762314015022 0ustar www-datawww-data# mail_room # mail_room is a configuration based process that will idle on IMAP connections and execute a delivery method when a new message is received. Examples of delivery methods include: * POST to a delivery URL (Postback) * Queue a job to Sidekiq or Que for later processing (Sidekiq or Que) * Log the message or open with LetterOpener (Logger or LetterOpener) [![Build Status](https://travis-ci.org/tpitale/mail_room.png?branch=master)](https://travis-ci.org/tpitale/mail_room) [![Code Climate](https://codeclimate.com/github/tpitale/mail_room/badges/gpa.svg)](https://codeclimate.com/github/tpitale/mail_room) ## Installation ## Add this line to your application's Gemfile: gem 'mail_room' And then execute: $ bundle Or install it yourself as: $ gem install mail_room You will also need to install `faraday` or `letter_opener` if you use the `postback` or `letter_opener` delivery methods, respectively. ## Usage ## mail_room -c /path/to/config.yml **Note:** To ignore missing config file or missing `mailboxes` key, use `-q` or `--quiet` ## Configuration ## ```yaml --- :mailboxes: - :email: "user1@gmail.com" :password: "password" :name: "inbox" :search_command: 'NEW' :delivery_options: :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" - :email: "user2@gmail.com" :password: "password" :name: "inbox" :delivery_method: postback :delivery_options: :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" - :email: "user3@gmail.com" :password: "password" :name: "inbox" :delivery_method: logger :delivery_options: :log_path: "/var/log/user3-email.log" - :email: "user4@gmail.com" :password: "password" :name: "inbox" :delivery_method: letter_opener :delete_after_delivery: true :delivery_options: :location: "/tmp/user4-email" - :email: "user5@gmail.com" :password: "password" :name: "inbox" :delivery_method: sidekiq :delivery_options: :redis_url: redis://localhost:6379 :worker: EmailReceiverWorker - :email: "user6@gmail.com" :password: "password" :name: "inbox" :delivery_method: sidekiq :delivery_options: # When pointing to sentinel, follow this sintax for redis URLs: # redis://:@/ :redis_url: redis://:password@my-redis-sentinel/ :sentinels: - :host: 127.0.0.1 :port: 26379 :worker: EmailReceiverWorker ``` ## delivery_method ## ### postback ### Requires `faraday` gem be installed. *NOTE:* If you're using Ruby `>= 2.0`, you'll need to use Faraday from `>= 0.8.9`. Versions before this seem to have some weird behavior with `mail_room`. The default delivery method, requires `delivery_url` and `delivery_token` in configuration. As the postback is essentially using your app as if it were an API endpoint, you may need to disable forgery protection as you would with a JSON API. In our case, the postback is plaintext, but the protection will still need to be disabled. ### sidekiq ### Deliver the message by pushing it onto the configured Sidekiq queue to be handled by a custom worker. Requires `redis` gem to be installed. Configured with `:delivery_method: sidekiq`. Delivery options: - **redis_url**: The Redis server to connect with. Use the same Redis URL that's used to configure Sidekiq. Required, defaults to `redis://localhost:6379`. - **sentinels**: A list of sentinels servers used to provide HA to Redis. (see [Sentinel Support](#sentinel-support)) Optional. - **namespace**: The Redis namespace Sidekiq works under. Use the same Redis namespace that's used to configure Sidekiq. Optional. - **queue**: The Sidekiq queue the job is pushed onto. Make sure Sidekiq actually reads off this queue. Required, defaults to `default`. - **worker**: The worker class that will handle the message. Required. An example worker implementation looks like this: ```ruby class EmailReceiverWorker include Sidekiq::Worker def perform(message) mail = Mail::Message.new(message) puts "New mail from #{mail.from.first}: #{mail.subject}" end end ``` ### que ### Deliver the message by pushing it onto the configured Que queue to be handled by a custom worker. Requires `pg` gem to be installed. Configured with `:delivery_method: que`. Delivery options: - **host**: The postgresql server host to connect with. Use the database you use with Que. Required, defaults to `localhost`. - **port**: The postgresql server port to connect with. Use the database you use with Que. Required, defaults to `5432`. - **database**: The postgresql database to use. Use the database you use with Que. Required. - **queue**: The Que queue the job is pushed onto. Make sure Que actually reads off this queue. Required, defaults to `default`. - **job_class**: The worker class that will handle the message. Required. - **priority**: The priority you want this job to run at. Required, defaults to `100`, lowest Que default priority. An example worker implementation looks like this: ```ruby class EmailReceiverJob < Que::Job def run(message) mail = Mail::Message.new(message) puts "New mail from #{mail.from.first}: #{mail.subject}" end end ``` ### logger ### Configured with `:delivery_method: logger`. If `:log_path:` is not provided, defaults to `STDOUT` ### noop ### Configured with `:delivery_method: noop`. Does nothing, like it says. ### letter_opener ### Requires `letter_opener` gem be installed. Configured with `:delivery_method: letter_opener`. Uses Ryan Bates' excellent [letter_opener](https://github.com/ryanb/letter_opener) gem. ## Receiving `postback` in Rails ## If you have a controller that you're sending to, with forgery protection disabled, you can get the raw string of the email using `request.body.read`. I would recommend having the `mail` gem bundled and parse the email using `Mail.read_from_string(request.body.read)`. ## idle_timeout ## By default, the IDLE command will wait for 29 minutes (in order to keep the server connection happy). If you'd prefer not to wait that long, you can pass `imap_timeout` in seconds for your mailbox configuration. ## Search Command ## This setting allows configuration of the IMAP search command sent to the server. This still defaults 'UNSEEN'. You may find that 'NEW' works better for you. ## IMAP Server Configuration ## You can set per-mailbox configuration for the IMAP server's `host` (default: 'imap.gmail.com'), `port` (default: 993), `ssl` (default: true), and `start_tls` (default: false). If you want to set additional options for IMAP SSL you can pass a YAML hash to match [SSLContext#set_params](http://docs.ruby-lang.org/en/2.2.0/OpenSSL/SSL/SSLContext.html#method-i-set_params). If you set `verify_mode` to `:none` it'll replace with the appropriate constant. If you're seeing the error `Please log in via your web browser: https://support.google.com/mail/accounts/answer/78754 (Failure)`, you need to configure your Gmail account to allow less secure apps to access it: https://support.google.com/accounts/answer/6010255. ## Running in Production ## I suggest running with either upstart or init.d. Check out this wiki page for some example scripts for both: https://github.com/tpitale/mail_room/wiki/Init-Scripts-for-Running-mail_room ## Arbitration ## When running multiple instances of MailRoom against a single mailbox, to try to prevent delivery of the same message multiple times, we can configure Arbitration using Redis. ```yaml :mailboxes: - :email: "user1@gmail.com" :password: "password" :name: "inbox" :delivery_method: postback :delivery_options: :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" :arbitration_method: redis :arbitration_options: # The Redis server to connect with. Defaults to redis://localhost:6379. :redis_url: redis://redis.example.com:6379 # The Redis namespace to house the Redis keys under. Optional. :namespace: mail_room - :email: "user2@gmail.com" :password: "password" :name: "inbox" :delivery_method: postback :delivery_options: :delivery_url: "http://localhost:3000/inbox" :delivery_token: "abcdefg" :arbitration_method: redis :arbitration_options: # When pointing to sentinel, follow this sintax for redis URLs: # redis://:@/ :redis_url: redis://:password@my-redis-sentinel/ :sentinels: - :host: 127.0.0.1 :port: 26379 # The Redis namespace to house the Redis keys under. Optional. :namespace: mail_room ``` **Note:** This will likely never be a _perfect_ system for preventing multiple deliveries of the same message, so I would advise checking the unique `message_id` if you are running in this situation. **Note:** There are other scenarios for preventing duplication of messages at scale that _may_ be more appropriate in your particular setup. One such example is using multiple inboxes in reply-by-email situations. Another is to use labels and configure a different `SEARCH` command for each instance of MailRoom. ## Sentinel Support Redis Sentinel provides high availability for Redis. Please read their [documentation](http://redis.io/topics/sentinel) first, before enabling it with mail_room. To connect to a Sentinel, you need to setup authentication to both sentinels and redis daemons first, and make sure both are binding to a reachable IP address. In mail_room, when you are connecting to a Sentinel, you have to inform the `master-name` and the `password` through `redis_url` param, following this syntax: ``` redis://:@/ ``` You also have to inform at least one pair of `host` and `port` for a sentinel in your cluster. To have a minimum reliable setup, you need at least `3` sentinel nodes and `3` redis servers (1 master, 2 slaves). ## Contributing ## 1. Fork it 2. Create your feature branch (`git checkout -b my-new-feature`) 3. Commit your changes (`git commit -am 'Add some feature'`) 4. Push to the branch (`git push origin my-new-feature`) 5. Create new Pull Request 6. If accepted, ask for commit rights ## TODO ## 1. specs, this is just a (working) proof of concept √ 2. finish code for POSTing to callback with auth √ 3. accept mailbox configuration for one account directly on the commandline; or ask for it 4. add example rails endpoint, with auth examples 5. add example configs for upstart/init.d √ 6. log to stdout √ 7. add a development mode that opens in letter_opener by ryanb √