pax_global_header00006660000000000000000000000064122200651300014501gustar00rootroot0000000000000052 comment=a7d6340bf98536e77b098bc8d32c3bc9a5d52ed0 unicorn-worker-killer-0.4.2/000077500000000000000000000000001222006513000157505ustar00rootroot00000000000000unicorn-worker-killer-0.4.2/AUTHORS000066400000000000000000000001751222006513000170230ustar00rootroot00000000000000Kazuki Ohta Sadayuki FURUHASHI Jonathan Clem unicorn-worker-killer-0.4.2/ChangeLog000066400000000000000000000021641222006513000175250ustar00rootroot00000000000000Release 0.4.1 - 2013/03/12 * Kill attempts happen once per request, not in a loop Release 0.4.0 - 2013/03/12 * Send signals from inside a Thread (required for QUIT signal) * Fix broken if/else in kill_self Release 0.3.6 - 2013/03/09 * do not require configuration * fix missing randomize() in MaxRequests killer Release 0.3.5 - 2013/03/08 * configurable max QUIT, TERM, and sleep interval * fix warning message of signal handling Release 0.3.4 - 2013/03/08 * try SIGQUIT first, rather than SIGTERM Release 0.3.3 - 2013/02/28 * fix for ruby 1.8 Release 0.3.2 - 2013/02/03 * add an option to disable the check for both MaxRequests and MaxMemory Release 0.3.1 - 2013/01/13 * fix for ruby 1.9.2 Release 0.3.0 - 2012/12/31 * fix for 'preload_app true' Release 0.2.1 - 2012/12/10 * fix critical namespace error, introduced at the previous version Release 0.2.0 - 2012/11/22 * change namespace to Unicorn::WorkerKiller from UnicornWorkerKiller Release 0.1.2 - 2012/11/17 * change unicorn version dependency to '~> 4' Release 0.1.1 - 2012/11/17 * fix homepage Release 0.1 - 2012/11/17 * first release unicorn-worker-killer-0.4.2/Gemfile000066400000000000000000000000471222006513000172440ustar00rootroot00000000000000source 'https://rubygems.org' gemspec unicorn-worker-killer-0.4.2/LICENSE000066400000000000000000000055451222006513000167660ustar00rootroot00000000000000Unicorn is copyrighted free software by all contributors, see logs in revision control for names and email addresses of all of them. You can redistribute it and/or modify it under either the terms of the GNU General Public License (GPL) as published by the Free Software Foundation (FSF), version {3.0}[http://www.gnu.org/licenses/gpl-3.0.txt] or version {2.0}[http://www.gnu.org/licenses/gpl-2.0.txt] or the Ruby-specific license terms (see below). The unicorn project leader (Eric Wong) reserves the right to add future versions of the GPL (and no other licenses) as published by the FSF to the licensing terms. === Ruby-specific terms (if you're not using the GPLv2 or GPLv3) 1. You may make and give away verbatim copies of the source form of the software without restriction, provided that you duplicate all of the original copyright notices and associated disclaimers. 2. You may modify your copy of the software in any way, provided that you do at least ONE of the following: a) place your modifications in the Public Domain or otherwise make them Freely Available, such as by posting said modifications to Usenet or an equivalent medium, or by allowing the author to include your modifications in the software. b) use the modified software only within your corporation or organization. c) rename any non-standard executables so the names do not conflict with standard executables, which must also be provided. d) make other distribution arrangements with the author. 3. You may distribute the software in object code or executable form, provided that you do at least ONE of the following: a) distribute the executables and library files of the software, together with instructions (in the manual page or equivalent) on where to get the original distribution. b) accompany the distribution with the machine-readable source of the software. c) give non-standard executables non-standard names, with instructions on where to get the original software distribution. d) make other distribution arrangements with the author. 4. You may modify and include the part of the software into any other software (possibly commercial). But some files in the distribution are not written by the author, so that they are not under this terms. 5. The scripts and library files supplied as input to or produced as output from the software do not automatically fall under the copyright of the software, but belong to whomever generated them, and may be sold commercially, and may be aggregated with this software. 6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. unicorn-worker-killer-0.4.2/README.md000066400000000000000000000052031222006513000172270ustar00rootroot00000000000000# unicorn-worker-killer [Unicorn](http://unicorn.bogomips.org/) is widely used HTTP-server for Rack applications. One thing we thought Unicorn missed, is killing the Unicorn workers based on the number of requests and consumed memories. `unicorn-worker-killer` gem provides automatic restart of Unicorn workers based on 1) max number of requests, and 2) process memory size (RSS), without affecting any requests. This will greatly improves site's stability by avoiding unexpected memory exhaustion at the application nodes. # Install No external process like `god` is required. Just install one gem: `unicorn-worker-killer`. gem 'unicorn-worker-killer' # Usage Add these lines to your `config.ru`. (These lines should be added above the `require ::File.expand_path('../config/environment', __FILE__)` line. # Unicorn self-process killer require 'unicorn/worker_killer' # Max requests per worker use Unicorn::WorkerKiller::MaxRequests, 3072, 4096 # Max memory size (RSS) per worker use Unicorn::WorkerKiller::Oom, (192*(1024**2)), (256*(1024**2)) This gem provides two modules. ### Unicorn::WorkerKiller::MaxRequests(max_requests_min=3072, max_requests_max=4096, verbose=false) This module automatically restarts the Unicorn workers, based on the number of requests which worker processed. `max_requests_min` and `max_requests_max` specify the min and max of maximum requests per worker. The actual limit is decided by rand() between `max_requests_min` and `max_requests_max` per worker, to prevent all workers to be dead at the same time. Once the number exceeds the limit, that worker is automatically restarted. If `verbose` is set to true, then after every request, your log will show the requests left before restart. This logging is done at the `info` level. ### Unicorn::WorkerKiller::Oom(memory_limit_min=(1024**3), memory_limit_max=(2*(1024**3)), check_cycle = 16, verbose = false) This module automatically restarts the Unicorn workers, based on its memory size. `memory_limit_min` and `memory_limit_max` specify the min and max of maximum memory per worker. The actual limit is decided by rand() between `memory_limit_min` and `memory_limit_max` per worker, to prevent all workers to be dead at the same time. Once the memory size exceeds `memory_size`, that worker is automatically restarted. The memory size check is done in every `check_cycle` requests. If `verbose` is set to true, then every memory size check will be shown in your logs. This logging is done at the `info` level. # Special Thanks - [@hotchpotch](http://github.com/hotchpotch/) for the [original idea](https://gist.github.com/hotchpotch/1258681) unicorn-worker-killer-0.4.2/Rakefile000066400000000000000000000001461222006513000174160ustar00rootroot00000000000000require 'bundler' Bundler::GemHelper.install_tasks require 'rake/testtask' task :default => [:build] unicorn-worker-killer-0.4.2/VERSION000066400000000000000000000000061222006513000170140ustar00rootroot000000000000000.4.2 unicorn-worker-killer-0.4.2/lib/000077500000000000000000000000001222006513000165165ustar00rootroot00000000000000unicorn-worker-killer-0.4.2/lib/unicorn/000077500000000000000000000000001222006513000201735ustar00rootroot00000000000000unicorn-worker-killer-0.4.2/lib/unicorn/configuration.rb000066400000000000000000000003411222006513000233650ustar00rootroot00000000000000module Unicorn::WorkerKiller class Configuration attr_accessor :max_quit, :max_term, :sleep_interval def initialize self.max_quit = 10 self.max_term = 15 self.sleep_interval = 1 end end end unicorn-worker-killer-0.4.2/lib/unicorn/worker_killer.rb000066400000000000000000000126161222006513000234010ustar00rootroot00000000000000require 'unicorn/configuration' module Unicorn::WorkerKiller class << self attr_accessor :configuration end # Kill the current process by telling it to send signals to itself. If # the process isn't killed after `configuration.max_quit` QUIT signals, # send TERM signals until `configuration.max_term`. Finally, send a KILL # signal. A single signal is sent per request. # @see http://unicorn.bogomips.org/SIGNALS.html def self.kill_self(logger, start_time) alive_sec = (Time.now - start_time).round worker_pid = Process.pid @@kill_attempts ||= 0 @@kill_attempts += 1 sig = :QUIT sig = :TERM if @@kill_attempts > configuration.max_quit sig = :KILL if @@kill_attempts > configuration.max_term logger.warn "#{self} send SIG#{sig} (pid: #{worker_pid}) alive: #{alive_sec} sec (trial #{@@kill_attempts})" Process.kill sig, worker_pid end module Oom # Killing the process must be occurred at the outside of the request. We're # using similar techniques used by OobGC, to ensure actual killing doesn't # affect the request. # # @see https://github.com/defunkt/unicorn/blob/master/lib/unicorn/oob_gc.rb#L40 def self.new(app, memory_limit_min = (1024**3), memory_limit_max = (2*(1024**3)), check_cycle = 16, verbose = false) ObjectSpace.each_object(Unicorn::HttpServer) do |s| s.extend(self) s.instance_variable_set(:@_worker_memory_limit_min, memory_limit_min) s.instance_variable_set(:@_worker_memory_limit_max, memory_limit_max) s.instance_variable_set(:@_worker_check_cycle, check_cycle) s.instance_variable_set(:@_worker_check_count, 0) s.instance_variable_set(:@_verbose, verbose) end app # pretend to be Rack middleware since it was in the past end def randomize(integer) RUBY_VERSION > "1.9" ? Random.rand(integer) : rand(integer) end def process_client(client) super(client) # Unicorn::HttpServer#process_client return if @_worker_memory_limit_min == 0 && @_worker_memory_limit_max == 0 @_worker_process_start ||= Time.now @_worker_memory_limit ||= @_worker_memory_limit_min + randomize(@_worker_memory_limit_max - @_worker_memory_limit_min + 1) @_worker_check_count += 1 if @_worker_check_count % @_worker_check_cycle == 0 rss = _worker_rss() logger.info "#{self}: worker (pid: #{Process.pid}) using #{rss} bytes." if @_verbose if rss > @_worker_memory_limit logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds memory limit (#{rss} bytes > #{@_worker_memory_limit} bytes)" Unicorn::WorkerKiller.kill_self(logger, @_worker_process_start) end @_worker_check_count = 0 end end private def _worker_rss proc_status = "/proc/#{Process.pid}/status" if File.exists? proc_status open(proc_status).each_line { |l| if l.include? 'VmRSS' ls = l.split if ls.length == 3 value = ls[1].to_i unit = ls[2] case unit.downcase when 'kb' return value*(1024**1) when 'mb' return value*(1024**2) when 'gb' return value*(1024**3) end end end } end # Forking the child process sometimes fails under low memory condition. # It would be ideal for not forking the process to get RSS. For Linux, # this module reads '/proc//status' to get RSS, but not for other # environments (e.g. MacOS and Windows). kb = `ps -o rss= -p #{Process.pid}`.to_i return kb * 1024 end end module MaxRequests # Killing the process must be occurred at the outside of the request. We're # using similar techniques used by OobGC, to ensure actual killing doesn't # affect the request. # # @see https://github.com/defunkt/unicorn/blob/master/lib/unicorn/oob_gc.rb#L40 def self.new(app, max_requests_min = 3072, max_requests_max = 4096, verbose = false) ObjectSpace.each_object(Unicorn::HttpServer) do |s| s.extend(self) s.instance_variable_set(:@_worker_max_requests_min, max_requests_min) s.instance_variable_set(:@_worker_max_requests_max, max_requests_max) s.instance_variable_set(:@_verbose, verbose) end app # pretend to be Rack middleware since it was in the past end def randomize(integer) RUBY_VERSION > "1.9" ? Random.rand(integer) : rand(integer) end def process_client(client) super(client) # Unicorn::HttpServer#process_client return if @_worker_max_requests_min == 0 && @_worker_max_requests_max == 0 @_worker_process_start ||= Time.now @_worker_cur_requests ||= @_worker_max_requests_min + randomize(@_worker_max_requests_max - @_worker_max_requests_min + 1) @_worker_max_requests ||= @_worker_cur_requests logger.info "#{self}: worker (pid: #{Process.pid}) has #{@_worker_cur_requests} left before being killed" if @_verbose if (@_worker_cur_requests -= 1) <= 0 logger.warn "#{self}: worker (pid: #{Process.pid}) exceeds max number of requests (limit: #{@_worker_max_requests})" Unicorn::WorkerKiller.kill_self(logger, @_worker_process_start) end end end def self.configure self.configuration ||= Configuration.new yield(configuration) if block_given? end self.configure end unicorn-worker-killer-0.4.2/unicorn-worker-killer.gemspec000066400000000000000000000016321222006513000235630ustar00rootroot00000000000000# encoding: utf-8 $:.push File.expand_path('../lib', __FILE__) Gem::Specification.new do |gem| gem.name = "unicorn-worker-killer" gem.description = "Kill unicorn workers by memory and request counts" gem.homepage = "https://github.com/kzk/unicorn-worker-killer" gem.summary = gem.description gem.version = File.read("VERSION").strip gem.authors = ["Kazuki Ohta", "Sadayuki Furuhashi", "Jonathan Clem"] gem.email = ["kazuki.ohta@gmail.com", "frsyuki@gmail.com", "jonathan@jclem.net"] gem.has_rdoc = false #gem.platform = Gem::Platform::RUBY gem.files = `git ls-files`.split("\n") gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n") gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) } gem.require_paths = ['lib'] gem.add_dependency "unicorn", "~> 4" gem.add_development_dependency "rake", ">= 0.9.2" end