lockfile-2.1.3/0000755000004100000410000000000012311354417013336 5ustar www-datawww-datalockfile-2.1.3/bin/0000755000004100000410000000000012311354417014106 5ustar www-datawww-datalockfile-2.1.3/bin/rlock0000755000004100000410000002347612311354417015162 0ustar www-datawww-data#!/usr/bin/env ruby # # built-in # require 'optparse' require 'logger' # # http://raa.ruby-lang.org/project/lockfile/ # require 'lockfile' class Main #--{{{ VERSION = Lockfile::VERSION USAGE = #--{{{ <<-usage NAME rlock v#{ VERSION } SYNOPSIS rlock [options]+ lockfile [program [args]+ | -- program options+ [args]+] DESCRIPTTION rlock creates NFS safe lockfiles. it can optionally run a program while holding the lock, ensuring lockfile removal on program exit. if a program is specified to be run rlock will spawn a background thread to kept the lockfile 'fresh' by touching it at a regular interval. in this way a lease is maintained on the lockfile and other processes attempting to obtain the lock can determine that it is in use. see the '--refresh' option for how to control the touch interval. any other process trying to obtain a lock will automatically remove a stale lockfile; a stale lockfile is one that is older than a certain age. this age be controled via the '--max_age' option. ENVIRONMENT LOCKFILE_DEBUG=1 causes internal actions of the library to be shown on STDERR DIAGNOSTICS rlock attempts to exit with the status of 'program' except where it cannot due to exceptional conditions. in addition the message 'RLOCK SUBCOMMAND FAILURE' will be printed on STDERR if 'program' exits with non-zero status. success => $? == 0 failure => $? != 0 AUTHOR ara.t.howard@gmail.com BUGS 1 < bugno && bugno < 42 OPTIONS usage #--}}} EXAMPLES = #--{{{ <<-examples EXAMPLES 0) simple usage - create lockfile in an atomic fashion (obtain a lock) ~ > rlock lockfile 1) safe usage - create a lockfile, execute a command, and remove lockfile ~ > rlock lockfile ls lockfile 2) same as above, but logging verbose messages ~ > rlock -v4 lockfile ls lockfile 3) same as above, but logging verbose messages and showing actions internal to lockfile library ~ > rlock -v4 -d lockfile ls lockfile 4) same as above ~ > LOCKFILE_DEBUG=1 rlock -v4 lockfile ls lockfile 5) same as above ~ > export LOCKFILE_DEBUG=1 ~ > rlock -v4 lockfile ls lockfile 6) you need to tell the option parser to stop parsing rlock options if you intend to pass options to 'program' ~ > rlock -v4 -d lockfile -- ls -ltar lockfile without the '--' rlock would consume the '-ltar' option as one of it's own. 7) lock lockfile and exec 'program' - remove the lockfile if it is older than 4242 seconds ~ > rlock --max_age=4242 lockfile program 8) lock lockfile and exec 'program' - remove the lockfile if it is older than 4242 seconds, set the refresh rate to be 8 seconds. ~ > rlock --max_age=4242 --refresh=8 lockfile program 9) same as above, but fail if lockfile cannot be obtained within 1 minute ~ > rlock --max_age=4242 --refresh=8 --timeout=60 lockfile program 10) lockfile creation involves making some temporary files. normally these are cleaned up unless rlock is killed with 'kill -9'. these temp files are normally 'sweeped' - searched for and removed - unless the '--dont_sweep' option is given. note that sweeping can remove ONLY old temp files created by the same host since there is otherwise no way to tell if the offending process is still running. lock lockfile and run program - do not do any sweeping ~ > rlock --dont_sweep lockfile program examples #--}}} EXIT_SUCCESS = 0 EXIT_FAILURE = 1 attr :argv attr :op attr :logger attr :config def initialize argv = ARGV #--{{{ @argv = mcp argv parse_opts if @opt_version puts Main::VERSION exit EXIT_SUCCESS end if @opt_help usage exit EXIT_SUCCESS end parse_argv run #--}}} end def run #--{{{ init_logging debug{ "lockpath <#{ @lockpath }>" } opts = {} options = %w(retries max_age sleep_inc min_sleep max_sleep suspend timeout refresh poll_retries poll_max_sleep) options.each do |opt| #if((val = eval("opt_#{ opt }"))) if(send("opt_#{ opt }?")) val = send "opt_#{ opt }" begin val = (opts[opt] = String === val ? Integer(val) : val) logger.debug{ "<#{ opt }> <#{ val.inspect }>" } rescue logger.fatal{ "illegal value <#{ val.inspect }> for opt <#{ opt }>" } exit EXIT_FAILURE end end end opts['debug'] = true if opt_debug begin case @argv.size when 0 opts['dont_clean'] = true logger.debug{ "opts <#{ opts.inspect }>" } logger.debug{ "aquiring lock <#{ @lockpath }>..." } # # simple usage - just create the lockfile with opts # lockfile = ::Lockfile.new @lockpath, opts lockfile.lock logger.debug{ "aquired lock <#{ @lockpath }>" } else logger.debug{ "opts <#{ opts.inspect }>" } logger.debug{ "aquiring lock <#{ @lockpath }>..." } # # block usage - create the lockfile with opts, run block, rm lockfile # status = 1 lockfile = ::Lockfile.new @lockpath, opts lockfile.lock do logger.debug{ "aquired lock <#{ @lockpath }>" } logger.debug{ "cmd <#{ @argv.join ' ' }>" } v = nil begin v = $VERBOSE $VERBOSE = nil STDOUT.flush STDERR.flush fork{ exec(*@argv) } pid, status = Process::wait2 ensure $VERBOSE = v end logger.debug{ "status <#{ $? }>" } end status = status.exitstatus STDERR.puts "RLOCK SUBCOMMAND FAILURE" unless status == 0 exit status end rescue => e logger.fatal{ e } exit EXIT_FAILURE end exit EXIT_SUCCESS #--}}} end def parse_opts #--{{{ @op = OptionParser::new @op.banner = '' define_options @op.parse! argv #--}}} end def parse_argv #--{{{ usage and exit EXIT_FAILURE if @argv.empty? @lockpath = @argv.shift #--}}} end def define_options #--{{{ options = [ ['--retries=n','-r', "default(#{ Lockfile.retries.inspect }) - (nil => forever)"], ['--max_age=n','-a', "default(#{ Lockfile.max_age.inspect })"], ['--sleep_inc=n','-s', "default(#{ Lockfile.sleep_inc.inspect })"], ['--max_sleep=n','-p', "default(#{ Lockfile.max_sleep.inspect })"], ['--min_sleep=n','-P', "default(#{ Lockfile.min_sleep.inspect })"], ['--suspend=n','-u', "default(#{ Lockfile.suspend.inspect })"], ['--timeout=n','-t', "default(#{ Lockfile.timeout.inspect }) - (nil => never)"], ['--refresh=n','-f', "default(#{ Lockfile.refresh.inspect })"], ['--debug','-d', "default(#{ Lockfile.debug.inspect })"], ['--poll_retries=n','-R', "default(#{ Lockfile.poll_retries.inspect })"], ['--poll_max_sleep=n','-S', "default(#{ Lockfile.poll_max_sleep.inspect })"], ['--dont_sweep','-w', "default(#{ Lockfile.dont_sweep.inspect })"], ['--version'], ['--verbosity=0-4|debug|info|warn|error|fatal','-v'], ['--log=path','-l'], ['--log_age=log_age'], ['--log_size=log_size'], ['--help','-h'], ] options.each do |option| opt = option.first.gsub(%r/(?:--)|(?:=.*$)/o,'').strip get, set = opt_attr opt value4 = lambda do |v| case v when NilClass, %r/^t|true$/i true when %r/^f|false$/i false when %r/^nil|nul|null$/i nil else v end end @op.def_option(*option) do |v| send set, value4[v] end end #--}}} end %w(debug info warn error fatal).each do |m| eval "def #{ m }(*args,&block);@logger.#{ m }(*args,&block);end" end def init_logging #--{{{ if @opt_log_age @opt_log_age = @opt_log_age.to_i if @opt_log_age =~ /\d/ end if @opt_log_size @opt_log_size = @opt_log_size.to_i if @opt_log_size =~ /\d/ end $logger = @logger = Logger::new(@opt_log || STDERR, @opt_log_age, @opt_log_size) level = nil @opt_verbosity ||= 'info' @opt_verbosity = case @opt_verbosity when /^\s*(?:4|d|debug)\s*$/io level = 'Logging::DEBUG' 4 when /^\s*(?:3|i|info)\s*$/io level = 'Logging::INFO' 3 when /^\s*(?:2|w|warn)\s*$/io level = 'Logging::WARN' 2 when /^\s*(?:1|e|error)\s*$/io level = 'Logging::ERROR' 1 when /^\s*(?:0|f|fatal)\s*$/io level = 'Logging::FATAL' 0 else abort "illegal verbosity setting <#{ @opt_verbosity }>" end @logger.level = 2 - ((@opt_verbosity % 5) - 2) #--}}} end def usage io = STDOUT #--{{{ io << USAGE io << "\n" io << @op io << "\n" io << EXAMPLES if defined? EXAMPLES self #--}}} end def opt_attr opt #--{{{ query = "opt_#{ opt }?" get = "opt_#{ opt }" set = "#{ get }=" code = <<-code class << self def #{ query }; defined? @#{ get }; end def #{ get }; defined?(@#{ get }) ? @#{ get } : nil; end def #{ set } value; @#{ get } = value; end end code instance_eval code [get, set] #--}}} end def mcp obj #--{{{ Marshal::load(Marshal::dump(obj)) #--}}} end #--}}} end Main::new lockfile-2.1.3/doc/0000755000004100000410000000000012311354417014103 5ustar www-datawww-datalockfile-2.1.3/doc/rlock.help0000644000004100000410000000521212311354417016067 0ustar www-datawww-data NAME rlock v1.3.0 SYNOPSIS rlock [options]+ file.lock [program [-- [options]+] [args]+] DESCRIPTTION rlock creates NFS resistent lockfiles ENVIRONMENT LOCKFILE_DEBUG=1 will show internal actions of the library DIAGNOSTICS success => $? == 0 failure => $? != 0 AUTHOR ara.t.howard@noaa.gov BUGS > 1 OPTIONS -r, --retries=n default(nil) - (nil => forever) -a, --max_age=n default(1024) -s, --sleep_inc=n default(2) -p, --max_sleep=n default(32) -P, --min_sleep=n default(2) -u, --suspend=n default(64) -t, --timeout=n default(nil) - (nil => never) -f, --refresh=n default(8) -d, --debug default(false) -R, --poll_retries=n default(16) -S, --poll_max_sleep=n default(0.08) -w, --dont_sweep default(false) -v=0-4|debug|info|warn|error|fatal --verbosity -l, --log=path --log_age=log_age --log_size=log_size -h, --help EXAMPLES 0) simple usage - just create a file.lock in an atomic fashion ~ > rlock file.lock 1) safe usage - create a file.lock, execute a command, and remove file.lock ~ > rlock file.lock ls file.lock 2) same as above, but logging verbose messages ~ > rlock -v4 file.lock ls file.lock 3) same as above, but logging verbose messages and showing actions internal to lockfile library ~ > rlock -v4 -d file.lock ls file.lock 4) same as above ~ > LOCKFILE_DEBUG=1 rlock -v4 file.lock ls file.lock 5) same as above ~ > export LOCKFILE_DEBUG=1 ~ > rlock -v4 file.lock ls file.lock 6) note that you need to tell the option parser to stop parsing rlock options if you intend to pass options to 'program' ~ > rlock -v4 -d file.lock -- ls -ltar file.lock without the '--' rlock would consume the '-ltar' options, parsing it as the logfile name 'tar' 7) lock file.lock and exec 'program' - remove the file.lock if it is older than 4242 seconds ~ > rlock --max_age=4242 file.lock program 8) lock file.lock and exec 'program' - remove the file.lock if it is older than 4242 seconds, also spawn a background thread which will refresh file.lock every 8 seonds will 'program' is executing ~ > rlock --max_age=4242 --refresh=8 file.lock program 9) same as above, but fail if file.lock cannot be obtained within 1 minute ~ > rlock --max_age=4242 --refresh=8 --timeout=60 file.lock program lockfile-2.1.3/README0000644000004100000410000001727112311354417014226 0ustar www-datawww-dataURLS http://rubyforge.org/projects/codeforpeople/ http://codeforpeople.com/lib/ruby/ SYNOPSIS lib/lockfile.rb : a ruby library for creating perfect and NFS safe lockfiles bin/rlock : ruby command line tool which uses this library to create lockfiles and to run arbitrary commands while holding them. for example rlock lockfile -- cp -r huge/ huge.bak/ run 'rlock -h' for more info INSTALL sudo ruby install.rb BASIC ALGORITHIM * create a globally uniq filename in the same filesystem as the desired lockfile - this can be nfs mounted * link(2) this file to the desired lockfile, ignore all errors * stat the uniq filename and desired lockfile to determine is they are the same, use only stat.rdev and stat.ino - ignore stat.nlink as NFS can cause this report incorrect values * iff same, you have lock. either return or run optional code block with optional refresher thread keeping lockfile fresh during execution of code block, ensuring that the lockfile is removed.. * iff not same try again a few times in rapid succession (poll), then, iff this fails, sleep using incremental backoff time. optionally remove lockfile if it is older than a certain time, timeout if more than a certain amount of time has passed attempting to lock file. BASIC USAGE 1) lockfile = Lockfile.new 'file.lock' begin lockfile.lock p 42 ensure lockfile.unlock end 2) require 'pstore' # which is NOT nfs safe on it's own opts = { # the keys can be symbols or strings :retries => nil, # we will try forever to aquire the lock :sleep_inc => 2, # we will sleep 2 seconds longer than the # previous sleep after each retry, cycling from # min_sleep upto max_sleep downto min_sleep upto # max_sleep, etc., etc. :min_sleep => 2, # we will never sleep less than 2 seconds :max_sleep => 32, # we will never sleep longer than 32 seconds :max_age => 3600, # we will blow away any files found to be older # than this (lockfile.thief? #=> true) :suspend => 1800, # iff we steal the lock from someone else - wait # this long to give them a chance to realize it :refresh => 8, # we will spawn a bg thread that touches file # every 8 sec. this thread also causes a # StolenLockError to be thrown if the lock # disappears from under us - note that the # 'detection' rate is limited to the refresh # interval - this is a race condition :timeout => nil, # we will wait forever :poll_retries => 16, # the initial attempt to grab a lock is done in a # polling fashion, this number controls how many # times this is done - the total polling attempts # are considered ONE actual attempt (see retries # above) :poll_max_sleep => 0.08, # when polling a very brief sleep is issued # between attempts, this is the upper limit of # that sleep timeout :dont_clean => false, # normally a finalizer is defined to clean up # after lockfiles, settin this to true prevents this :dont_sweep => false, # normally locking causes a sweep to be made. a # sweep removes any old tmp files created by # processes of this host only which are no # longer alive :debug => true, # trace execution step on stdout } pstore = PStore.new 'file.db' lockfile = Lockfile.new 'file.db.lock', opts lockfile.lock do pstore.transaction do pstore[:last_update_time] = Time.now end end 3) same as 1 above - Lockfile.new takes a block and ensures unlock is called Lockfile.new('file.lock') do p 42 end 4) watch locking algorithim in action (debugging only) Lockfile.debug = true Lockfile.new('file.lock') do p 42 end you can also set debugging via the ENV var LOCKFILE_DEBUG, eg. ~ > LOCKFILE_DEBUG=true rlock lockfile 5) simplified interface : no lockfile object required Lockfile('lock', :retries => 0) do puts 'only one instance running!' end SAMPLES * see samples/a.rb * see samples/nfsstore.rb * see samples/lock.sh * see bin/rlock AUTHOR Ara T. Howard EMAIL Ara.T.Howard@noaa.gov BUGS bugno > 1 && bugno < 42 HISTORY 2.0.0: - lock fires up a refresher thread when called without a block 1.4.3: - fixed a small non-critical bug in the require gaurd 1.4.2: - upped defaults for max_age to 3600 and suspend to 1800. - tweaked a few things to play nice with rubygems 1.4.1: - Mike Kasick reported a bug whereby false/nil values for options were ignored. patched to address this bug - added Lockfile method for high level interface sans lockfile object - updated rlock program to allow nil/true/false values passed on command line. eg rlock --max_age=nil lockfile -- date --iso-8601=seconds 1.4.0: - gem'd up - added Lockfile::create method which atomically creates a file and opens it: Lockfile::create("atomic_even_on_nfs", "r+") do |f| f.puts 42 end arguments are the same as those for File::open. note that there is no way to accomplish this otherwise since File::O_EXCL fails silently on nfs, flock does not work on nfs, and posixlock (see raa) can only lock a file after it is open so the open itself is still a race. 1.3.0: - added sweep functionality. hosts can clean up any tmp files left around by other processes that may have been killed using -9 to prevent proper clean up. it is only possible to clean up after processes on the same host as there is no other way to determine if the process that created the file is alive. in summary - hosts clean up after other processes on that same host if needed. - added attempt/try_again methods 1.2.0: - fixed bug where stale locks not removed when retries == 0 1.1.0 - unfortunately i've forgotten 1.0.1: - fixed bugette in sleep cycle where repeated locks with same lockfile would not reset the cycle at the start of each lock 1.0.0: - allow rertries to be nil, meaning to try forever - default timeout is now nil - never timeout - default refresh is now 8 - fixed bug where refresher thread was not actually touching lockfile! (ouch) - added cycle method to timeouts 1-2-3-2-1-2-3-1... pattern is constructed using min_sleep, sleep_inc, max_sleep 0.3.0: - removed use of yaml in favour of hand parsing the lockfile contents, the file as so small it just wasn't worth and i have had one core dump when yaml failed to parse a (corrupt) file 0.2.0: - added an initial polling style attempt to grab lock before entering normal sleep/retry loop. this has really helped performance when lock is under heavy contention: i see almost no sleeping done by any of in the interested processes 0.1.0: - added ability of Lockfile.new to accept a block 0.0.0: - initial version lockfile-2.1.3/lib/0000755000004100000410000000000012311354417014104 5ustar www-datawww-datalockfile-2.1.3/lib/lockfile.rb0000644000004100000410000003762112311354417016232 0ustar www-datawww-dataunless(defined?($__lockfile__) or defined?(Lockfile)) require 'socket' require 'timeout' require 'fileutils' class Lockfile VERSION = '2.1.3' def Lockfile.version() Lockfile::VERSION end def version() Lockfile::VERSION end def Lockfile.description 'a ruby library for creating perfect and NFS safe lockfiles' end class LockError < StandardError; end class StolenLockError < LockError; end class StackingLockError < LockError; end class StatLockError < LockError; end class MaxTriesLockError < LockError; end class TimeoutLockError < LockError; end class NFSLockError < LockError; end class UnLockError < LockError; end class SleepCycle < Array attr_reader :min attr_reader :max attr_reader :range attr_reader :inc def initialize(min, max, inc) @min, @max, @inc = Float(min), Float(max), Float(inc) @range = @max - @min raise RangeError, "max(#{ @max }) <= min(#{ @min })" if @max <= @min raise RangeError, "inc(#{ @inc }) > range(#{ @range })" if @inc > @range raise RangeError, "inc(#{ @inc }) <= 0" if @inc <= 0 raise RangeError, "range(#{ @range }) <= 0" if @range <= 0 s = @min push(s) and s += @inc while(s <= @max) self[-1] = @max if self[-1] < @max reset end def next ret = self[@idx] @idx = ((@idx + 1) % self.size) ret end def reset @idx = 0 end end HOSTNAME = Socket.gethostname DEFAULT_RETRIES = nil # maximum number of attempts DEFAULT_TIMEOUT = nil # the longest we will try DEFAULT_MAX_AGE = 3600 # lockfiles older than this are stale DEFAULT_SLEEP_INC = 2 # sleep cycle is this much longer each time DEFAULT_MIN_SLEEP = 2 # shortest sleep time DEFAULT_MAX_SLEEP = 32 # longest sleep time DEFAULT_SUSPEND = 1800 # iff we steal a lock wait this long before we go on DEFAULT_REFRESH = 8 # how often we touch/validate the lock DEFAULT_DONT_CLEAN = false # iff we leave lock files lying around DEFAULT_POLL_RETRIES = 16 # this many polls makes one 'try' DEFAULT_POLL_MAX_SLEEP = 0.08 # the longest we'll sleep between polls DEFAULT_DONT_SWEEP = false # if we cleanup after other process on our host DEFAULT_DONT_USE_LOCK_ID = false # if we dump lock info into lockfile DEFAULT_DEBUG = ENV['LOCKFILE_DEBUG'] || false class << Lockfile attr_accessor :retries attr_accessor :max_age attr_accessor :sleep_inc attr_accessor :min_sleep attr_accessor :max_sleep attr_accessor :suspend attr_accessor :timeout attr_accessor :refresh attr_accessor :debug attr_accessor :dont_clean attr_accessor :poll_retries attr_accessor :poll_max_sleep attr_accessor :dont_sweep attr_accessor :dont_use_lock_id def init @retries = DEFAULT_RETRIES @max_age = DEFAULT_MAX_AGE @sleep_inc = DEFAULT_SLEEP_INC @min_sleep = DEFAULT_MIN_SLEEP @max_sleep = DEFAULT_MAX_SLEEP @suspend = DEFAULT_SUSPEND @timeout = DEFAULT_TIMEOUT @refresh = DEFAULT_REFRESH @dont_clean = DEFAULT_DONT_CLEAN @poll_retries = DEFAULT_POLL_RETRIES @poll_max_sleep = DEFAULT_POLL_MAX_SLEEP @dont_sweep = DEFAULT_DONT_SWEEP @dont_use_lock_id = DEFAULT_DONT_USE_LOCK_ID @debug = DEFAULT_DEBUG STDOUT.sync = true if @debug STDERR.sync = true if @debug end end Lockfile.init attr_reader :klass attr_reader :path attr_reader :opts attr_reader :locked attr_reader :thief attr_reader :refresher attr_reader :dirname attr_reader :basename attr_reader :clean attr_reader :retries attr_reader :max_age attr_reader :sleep_inc attr_reader :min_sleep attr_reader :max_sleep attr_reader :suspend attr_reader :refresh attr_reader :timeout attr_reader :dont_clean attr_reader :poll_retries attr_reader :poll_max_sleep attr_reader :dont_sweep attr_reader :dont_use_lock_id attr_accessor :debug alias thief? thief alias locked? locked alias debug? debug def Lockfile.create(path, *a, &b) opts = { 'retries' => 0, 'min_sleep' => 0, 'max_sleep' => 1, 'sleep_inc' => 1, 'max_age' => nil, 'suspend' => 0, 'refresh' => nil, 'timeout' => nil, 'poll_retries' => 0, 'dont_clean' => true, 'dont_sweep' => false, 'dont_use_lock_id' => true, } begin new(path, opts).lock rescue LockError raise Errno::EEXIST, path end open(path, *a, &b) end def initialize(path, opts = {}, &block) @klass = self.class @path = path @opts = opts @retries = getopt 'retries' , @klass.retries @max_age = getopt 'max_age' , @klass.max_age @sleep_inc = getopt 'sleep_inc' , @klass.sleep_inc @min_sleep = getopt 'min_sleep' , @klass.min_sleep @max_sleep = getopt 'max_sleep' , @klass.max_sleep @suspend = getopt 'suspend' , @klass.suspend @timeout = getopt 'timeout' , @klass.timeout @refresh = getopt 'refresh' , @klass.refresh @dont_clean = getopt 'dont_clean' , @klass.dont_clean @poll_retries = getopt 'poll_retries' , @klass.poll_retries @poll_max_sleep = getopt 'poll_max_sleep' , @klass.poll_max_sleep @dont_sweep = getopt 'dont_sweep' , @klass.dont_sweep @dont_use_lock_id = getopt 'dont_use_lock_id' , @klass.dont_use_lock_id @debug = getopt 'debug' , @klass.debug @sleep_cycle = SleepCycle.new @min_sleep, @max_sleep, @sleep_inc @clean = @dont_clean ? nil : lambda{ File.unlink @path rescue nil } @dirname = File.dirname @path @basename = File.basename @path @thief = false @locked = false @refrsher = nil lock(&block) if block end ## # Executes the given block after acquiring the lock and # ensures that the lock is relinquished afterwards. # def synchronize raise ArgumentError, 'block must be given' unless block_given? begin lock yield ensure unlock end end def lock raise StackingLockError, "<#{ @path }> is locked!" if @locked sweep unless @dont_sweep ret = nil attempt do begin @sleep_cycle.reset create_tmplock do |f| begin Timeout.timeout(@timeout) do tmp_path = f.path tmp_stat = f.lstat n_retries = 0 trace{ "attempting to lock <#{ @path }>..." } begin i = 0 begin trace{ "polling attempt <#{ i }>..." } begin File.link tmp_path, @path rescue Errno::ENOENT try_again! end lock_stat = File.lstat @path raise StatLockError, "stat's do not agree" unless tmp_stat.rdev == lock_stat.rdev and tmp_stat.ino == lock_stat.ino trace{ "aquired lock <#{ @path }>" } @locked = true rescue => e i += 1 unless i >= @poll_retries t = [rand(@poll_max_sleep), @poll_max_sleep].min trace{ "poll sleep <#{ t }>..." } sleep t retry end raise end rescue => e n_retries += 1 trace{ "n_retries <#{ n_retries }>" } case validlock? when true raise MaxTriesLockError, "surpased retries <#{ @retries }>" if @retries and n_retries >= @retries trace{ "found valid lock" } sleeptime = @sleep_cycle.next trace{ "sleep <#{ sleeptime }>..." } sleep sleeptime when false trace{ "found invalid lock and removing" } begin File.unlink @path @thief = true warn "<#{ @path }> stolen by <#{ Process.pid }> at <#{ timestamp }>" trace{ "i am a thief!" } rescue Errno::ENOENT end trace{ "suspending <#{ @suspend }>" } sleep @suspend when nil raise MaxTriesLockError, "surpased retries <#{ @retries }>" if @retries and n_retries >= @retries end retry end # begin end # timeout rescue Timeout::Error raise TimeoutLockError, "surpassed timeout <#{ @timeout }>" end # begin end # create_tmplock if block_given? stolen = false @refresher = (@refresh ? new_refresher : nil) begin begin ret = yield @path rescue StolenLockError stolen = true raise end ensure begin @refresher.kill if @refresher and @refresher.status @refresher = nil ensure unlock unless stolen end end else @refresher = (@refresh ? new_refresher : nil) ObjectSpace.define_finalizer self, @clean if @clean ret = self end rescue Errno::ESTALE, Errno::EIO => e raise(NFSLockError, errmsg(e)) end end return ret end def sweep begin glob = File.join(@dirname, ".*lck") paths = Dir[glob] paths.each do |path| begin basename = File.basename path pat = %r/^\s*\.([^_]+)_([^_]+)/o if pat.match(basename) host, pid = $1, $2 else next end host.gsub!(%r/^\.+|\.+$/,'') quad = host.split %r/\./ host = quad.first pat = %r/^\s*#{ host }/i if pat.match(HOSTNAME) and %r/^\s*\d+\s*$/.match(pid) unless alive?(pid) trace{ "process <#{ pid }> on <#{ host }> is no longer alive" } trace{ "sweeping <#{ path }>" } FileUtils.rm_f path else trace{ "process <#{ pid }> on <#{ host }> is still alive" } trace{ "ignoring <#{ path }>" } end else trace{ "ignoring <#{ path }> generated by <#{ host }>" } end rescue next end end rescue => e warn(errmsg(e)) end end def alive? pid pid = Integer("#{ pid }") begin Process.kill 0, pid true rescue Errno::ESRCH false end end def unlock raise UnLockError, "<#{ @path }> is not locked!" unless @locked @refresher.kill if @refresher and @refresher.status @refresher = nil begin File.unlink @path rescue Errno::ENOENT raise StolenLockError, @path ensure @thief = false @locked = false ObjectSpace.undefine_finalizer self if @clean end end def new_refresher Thread.new(Thread.current, @path, @refresh, @dont_use_lock_id) do |thread, path, refresh, dont_use_lock_id| loop do begin touch path trace{"touched <#{ path }> @ <#{ Time.now.to_f }>"} unless dont_use_lock_id loaded = load_lock_id(IO.read(path)) trace{"loaded <\n#{ loaded.inspect }\n>"} raise unless loaded == @lock_id end sleep refresh rescue Exception => e trace{errmsg e} thread.raise StolenLockError Thread.exit end end end end def validlock? if @max_age uncache @path rescue nil begin return((Time.now - File.stat(@path).mtime) < @max_age) rescue Errno::ENOENT return nil end else exist = File.exist?(@path) return(exist ? true : nil) end end def uncache file refresh = nil begin is_a_file = File === file path = (is_a_file ? file.path : file.to_s) stat = (is_a_file ? file.stat : File.stat(file.to_s)) refresh = tmpnam(File.dirname(path)) File.link path, refresh File.chmod stat.mode, path File.utime stat.atime, stat.mtime, path ensure begin File.unlink refresh if refresh rescue Errno::ENOENT end end end def create_tmplock tmplock = tmpnam @dirname begin create(tmplock) do |f| unless dont_use_lock_id @lock_id = gen_lock_id dumped = dump_lock_id trace{"lock_id <\n#{ @lock_id.inspect }\n>"} f.write dumped f.flush end yield f end ensure begin; File.unlink tmplock; rescue Errno::ENOENT; end if tmplock end end def gen_lock_id Hash[ 'host' => "#{ HOSTNAME }", 'pid' => "#{ Process.pid }", 'ppid' => "#{ Process.ppid }", 'time' => timestamp, ] end def timestamp time = Time.now usec = time.usec.to_s usec << '0' while usec.size < 6 "#{ time.strftime('%Y-%m-%d %H:%M:%S') }.#{ usec }" end def dump_lock_id(lock_id = @lock_id) "host: %s\npid: %s\nppid: %s\ntime: %s\n" % lock_id.values_at('host','pid','ppid','time') end def load_lock_id(buf) lock_id = {} kv = %r/([^:]+):(.*)/o buf.each_line do |line| m = kv.match line k, v = m[1], m[2] next unless m and k and v lock_id[k.strip] = v.strip end lock_id end def tmpnam(dir, seed = File.basename($0)) pid = Process.pid time = Time.now sec = time.to_i usec = time.usec "%s%s.%s_%d_%s_%d_%d_%d.lck" % [dir, File::SEPARATOR, HOSTNAME, pid, seed, sec, usec, rand(sec)] end def create(path) umask = nil f = nil begin umask = File.umask 022 f = open path, File::WRONLY|File::CREAT|File::EXCL, 0644 ensure File.umask umask if umask end return(block_given? ? begin; yield f; ensure; f.close; end : f) end def touch(path) FileUtils.touch path end def getopt(key, default = nil) [ key, key.to_s, key.to_s.intern ].each do |k| return @opts[k] if @opts.has_key?(k) end return default end def to_str @path end alias to_s to_str def trace(s = nil) STDERR.puts((s ? s : yield)) if @debug end def errmsg(e) "%s (%s)\n%s\n" % [e.class, e.message, e.backtrace.join("\n")] end def attempt ret = nil loop{ break unless catch('attempt'){ ret = yield } == 'try_again' } ret end def try_again! throw 'attempt', 'try_again' end alias again! try_again! def give_up! throw 'attempt', 'give_up' end end def Lockfile(path, *a, &b) Lockfile.new(path, *a, &b) end $__lockfile__ = __FILE__ end lockfile-2.1.3/readme.erb0000644000004100000410000001714212311354417015272 0ustar www-datawww-dataURLS http://rubyforge.org/projects/codeforpeople/ http://codeforpeople.com/lib/ruby/ SYNOPSIS lib/lockfile.rb : a ruby library for creating NFS safe lockfiles bin/rlock : ruby command line tool which uses this library to create lockfiles and to run arbitrary commands while holding them. for example rlock lockfile -- cp -r huge/ huge.bak/ run 'rlock -h' for more info INSTALL sudo ruby install.rb BASIC ALGORITHIM * create a globally uniq filename in the same filesystem as the desired lockfile - this can be nfs mounted * link(2) this file to the desired lockfile, ignore all errors * stat the uniq filename and desired lockfile to determine is they are the same, use only stat.rdev and stat.ino - ignore stat.nlink as NFS can cause this report incorrect values * iff same, you have lock. either return or run optional code block with optional refresher thread keeping lockfile fresh during execution of code block, ensuring that the lockfile is removed.. * iff not same try again a few times in rapid succession (poll), then, iff this fails, sleep using incremental backoff time. optionally remove lockfile if it is older than a certain time, timeout if more than a certain amount of time has passed attempting to lock file. BASIC USAGE 1) lockfile = Lockfile.new 'file.lock' begin lockfile.lock p 42 ensure lockfile.unlock end 2) require 'pstore' # which is NOT nfs safe on it's own opts = { # the keys can be symbols or strings :retries => nil, # we will try forever to aquire the lock :sleep_inc => 2, # we will sleep 2 seconds longer than the # previous sleep after each retry, cycling from # min_sleep upto max_sleep downto min_sleep upto # max_sleep, etc., etc. :min_sleep => 2, # we will never sleep less than 2 seconds :max_sleep => 32, # we will never sleep longer than 32 seconds :max_age => 3600, # we will blow away any files found to be older # than this (lockfile.thief? #=> true) :suspend => 1800, # iff we steal the lock from someone else - wait # this long to give them a chance to realize it :refresh => 8, # we will spawn a bg thread that touches file # every 8 sec. this thread also causes a # StolenLockError to be thrown if the lock # disappears from under us - note that the # 'detection' rate is limited to the refresh # interval - this is a race condition :timeout => nil, # we will wait forever :poll_retries => 16, # the initial attempt to grab a lock is done in a # polling fashion, this number controls how many # times this is done - the total polling attempts # are considered ONE actual attempt (see retries # above) :poll_max_sleep => 0.08, # when polling a very brief sleep is issued # between attempts, this is the upper limit of # that sleep timeout :dont_clean => false, # normally a finalizer is defined to clean up # after lockfiles, settin this to true prevents this :dont_sweep => false, # normally locking causes a sweep to be made. a # sweep removes any old tmp files created by # processes of this host only which are no # longer alive :debug => true, # trace execution step on stdout } pstore = PStore.new 'file.db' lockfile = Lockfile.new 'file.db.lock', opts lockfile.lock do pstore.transaction do pstore[:last_update_time] = Time.now end end 3) same as 1 above - Lockfile.new takes a block and ensures unlock is called Lockfile.new('file.lock') do p 42 end 4) watch locking algorithim in action (debugging only) Lockfile.debug = true Lockfile.new('file.lock') do p 42 end you can also set debugging via the ENV var LOCKFILE_DEBUG, eg. ~ > LOCKFILE_DEBUG=true rlock lockfile 5) simplified interface : no lockfile object required Lockfile('lock', :retries => 0) do puts 'only one instance running!' end SAMPLES * see samples/a.rb * see samples/nfsstore.rb * see samples/lock.sh * see bin/rlock AUTHOR Ara T. Howard EMAIL Ara.T.Howard@gmail.com BUGS bugno > 1 && bugno < 42 HISTORY 1.4.3: - fixed a small non-critical bug in the require gaurd 1.4.2: - upped defaults for max_age to 3600 and suspend to 1800. - tweaked a few things to play nice with rubygems 1.4.1: - Mike Kasick reported a bug whereby false/nil values for options were ignored. patched to address this bug - added Lockfile method for high level interface sans lockfile object - updated rlock program to allow nil/true/false values passed on command line. eg rlock --max_age=nil lockfile -- date --iso-8601=seconds 1.4.0: - gem'd up - added Lockfile::create method which atomically creates a file and opens it: Lockfile::create("atomic_even_on_nfs", "r+") do |f| f.puts 42 end arguments are the same as those for File::open. note that there is no way to accomplish this otherwise since File::O_EXCL fails silently on nfs, flock does not work on nfs, and posixlock (see raa) can only lock a file after it is open so the open itself is still a race. 1.3.0: - added sweep functionality. hosts can clean up any tmp files left around by other processes that may have been killed using -9 to prevent proper clean up. it is only possible to clean up after processes on the same host as there is no other way to determine if the process that created the file is alive. in summary - hosts clean up after other processes on that same host if needed. - added attempt/try_again methods 1.2.0: - fixed bug where stale locks not removed when retries == 0 1.1.0 - unfortunately i've forgotten 1.0.1: - fixed bugette in sleep cycle where repeated locks with same lockfile would not reset the cycle at the start of each lock 1.0.0: - allow rertries to be nil, meaning to try forever - default timeout is now nil - never timeout - default refresh is now 8 - fixed bug where refresher thread was not actually touching lockfile! (ouch) - added cycle method to timeouts 1-2-3-2-1-2-3-1... pattern is constructed using min_sleep, sleep_inc, max_sleep 0.3.0: - removed use of yaml in favour of hand parsing the lockfile contents, the file as so small it just wasn't worth and i have had one core dump when yaml failed to parse a (corrupt) file 0.2.0: - added an initial polling style attempt to grab lock before entering normal sleep/retry loop. this has really helped performance when lock is under heavy contention: i see almost no sleeping done by any of in the interested processes 0.1.0: - added ability of Lockfile.new to accept a block 0.0.0: - initial version lockfile-2.1.3/rakefile0000644000004100000410000002364012311354417015050 0ustar www-datawww-dataThis.rubyforge_project = 'codeforpeople' This.author = "Ara T. Howard" This.email = "ara.t.howard@gmail.com" This.homepage = "https://github.com/ahoward/#{ This.lib }" task :license do open('LICENSE', 'w'){|fd| fd.puts "Ruby"} end task :default do puts((Rake::Task.tasks.map{|task| task.name.gsub(/::/,':')} - ['default']).sort) end task :test do run_tests! end namespace :test do task(:unit){ run_tests!(:unit) } task(:functional){ run_tests!(:functional) } task(:integration){ run_tests!(:integration) } end def run_tests!(which = nil) which ||= '**' test_dir = File.join(This.dir, "test") test_glob ||= File.join(test_dir, "#{ which }/**_test.rb") test_rbs = Dir.glob(test_glob).sort div = ('=' * 119) line = ('-' * 119) test_rbs.each_with_index do |test_rb, index| testno = index + 1 command = "#{ This.ruby } -w -I ./lib -I ./test/lib #{ test_rb }" puts say(div, :color => :cyan, :bold => true) say("@#{ testno } => ", :bold => true, :method => :print) say(command, :color => :cyan, :bold => true) say(line, :color => :cyan, :bold => true) system(command) say(line, :color => :cyan, :bold => true) status = $?.exitstatus if status.zero? say("@#{ testno } <= ", :bold => true, :color => :white, :method => :print) say("SUCCESS", :color => :green, :bold => true) else say("@#{ testno } <= ", :bold => true, :color => :white, :method => :print) say("FAILURE", :color => :red, :bold => true) end say(line, :color => :cyan, :bold => true) exit(status) unless status.zero? end end task :gemspec do ignore_extensions = ['git', 'svn', 'tmp', /sw./, 'bak', 'gem'] ignore_directories = ['pkg'] ignore_files = ['test/log'] shiteless = lambda do |list| list.delete_if do |entry| next unless test(?e, entry) extension = File.basename(entry).split(%r/[.]/).last ignore_extensions.any?{|ext| ext === extension} end list.delete_if do |entry| next unless test(?d, entry) dirname = File.expand_path(entry) ignore_directories.any?{|dir| File.expand_path(dir) == dirname} end list.delete_if do |entry| next unless test(?f, entry) filename = File.expand_path(entry) ignore_files.any?{|file| File.expand_path(file) == filename} end end lib = This.lib object = This.object version = This.version files = shiteless[Dir::glob("**/**")] executables = shiteless[Dir::glob("bin/*")].map{|exe| File.basename(exe)} #has_rdoc = true #File.exist?('doc') test_files = "test/#{ lib }.rb" if File.file?("test/#{ lib }.rb") summary = object.respond_to?(:summary) ? object.summary : "summary: #{ lib } kicks the ass" description = object.respond_to?(:description) ? object.description : "description: #{ lib } kicks the ass" license = object.respond_to?(:license) ? object.license : "Ruby" if This.extensions.nil? This.extensions = [] extensions = This.extensions %w( Makefile configure extconf.rb ).each do |ext| extensions << ext if File.exists?(ext) end end extensions = [extensions].flatten.compact if This.dependencies.nil? dependencies = [] else case This.dependencies when Hash dependencies = This.dependencies.values when Array dependencies = This.dependencies end end template = if test(?e, 'gemspec.erb') Template{ IO.read('gemspec.erb') } else Template { <<-__ ## <%= lib %>.gemspec # Gem::Specification::new do |spec| spec.name = <%= lib.inspect %> spec.version = <%= version.inspect %> spec.platform = Gem::Platform::RUBY spec.summary = <%= lib.inspect %> spec.description = <%= description.inspect %> spec.license = <%= license.inspect %> spec.files =\n<%= files.sort.pretty_inspect %> spec.executables = <%= executables.inspect %> spec.require_path = "lib" spec.test_files = <%= test_files.inspect %> <% dependencies.each do |lib_version| %> spec.add_dependency(*<%= Array(lib_version).flatten.inspect %>) <% end %> spec.extensions.push(*<%= extensions.inspect %>) spec.rubyforge_project = <%= This.rubyforge_project.inspect %> spec.author = <%= This.author.inspect %> spec.email = <%= This.email.inspect %> spec.homepage = <%= This.homepage.inspect %> end __ } end Fu.mkdir_p(This.pkgdir) gemspec = "#{ lib }.gemspec" open(gemspec, "w"){|fd| fd.puts(template)} This.gemspec = gemspec end task :gem => [:clean, :gemspec] do Fu.mkdir_p(This.pkgdir) before = Dir['*.gem'] cmd = "gem build #{ This.gemspec }" `#{ cmd }` after = Dir['*.gem'] gem = ((after - before).first || after.first) or abort('no gem!') Fu.mv(gem, This.pkgdir) This.gem = File.join(This.pkgdir, File.basename(gem)) end task :readme do samples = '' prompt = '~ > ' lib = This.lib version = This.version Dir['sample*/*'].sort.each do |sample| samples << "\n" << " <========< #{ sample } >========>" << "\n\n" cmd = "cat #{ sample }" samples << Util.indent(prompt + cmd, 2) << "\n\n" samples << Util.indent(`#{ cmd }`, 4) << "\n" cmd = "ruby #{ sample }" samples << Util.indent(prompt + cmd, 2) << "\n\n" cmd = "ruby -e'STDOUT.sync=true; exec %(ruby -I ./lib #{ sample })'" samples << Util.indent(`#{ cmd } 2>&1`, 4) << "\n" end template = if test(?e, 'README.erb') Template{ IO.read('README.erb') } else Template { <<-__ NAME #{ lib } DESCRIPTION INSTALL gem install #{ lib } SAMPLES #{ samples } __ } end open("README", "w"){|fd| fd.puts template} end task :clean do Dir[File.join(This.pkgdir, '**/**')].each{|entry| Fu.rm_rf(entry)} end task :release => [:clean, :gemspec, :gem] do gems = Dir[File.join(This.pkgdir, '*.gem')].flatten raise "which one? : #{ gems.inspect }" if gems.size > 1 raise "no gems?" if gems.size < 1 cmd = "gem push #{ This.gem }" puts cmd puts system(cmd) abort("cmd(#{ cmd }) failed with (#{ $?.inspect })") unless $?.exitstatus.zero? cmd = "rubyforge login && rubyforge add_release #{ This.rubyforge_project } #{ This.lib } #{ This.version } #{ This.gem }" puts cmd puts system(cmd) abort("cmd(#{ cmd }) failed with (#{ $?.inspect })") unless $?.exitstatus.zero? end BEGIN { # support for this rakefile # $VERBOSE = nil require 'ostruct' require 'erb' require 'fileutils' require 'rbconfig' require 'pp' # fu shortcut # Fu = FileUtils # cache a bunch of stuff about this rakefile/environment # This = OpenStruct.new This.file = File.expand_path(__FILE__) This.dir = File.dirname(This.file) This.pkgdir = File.join(This.dir, 'pkg') # grok lib # lib = ENV['LIB'] unless lib lib = File.basename(Dir.pwd).sub(/[-].*$/, '') end This.lib = lib # grok version # version = ENV['VERSION'] unless version require "./lib/#{ This.lib }" This.name = lib.capitalize This.object = eval(This.name) version = This.object.send(:version) end This.version = version # see if dependencies are export by the module # if This.object.respond_to?(:dependencies) This.dependencies = This.object.dependencies end # we need to know the name of the lib an it's version # abort('no lib') unless This.lib abort('no version') unless This.version # discover full path to this ruby executable # c = Config::CONFIG bindir = c["bindir"] || c['BINDIR'] ruby_install_name = c['ruby_install_name'] || c['RUBY_INSTALL_NAME'] || 'ruby' ruby_ext = c['EXEEXT'] || '' ruby = File.join(bindir, (ruby_install_name + ruby_ext)) This.ruby = ruby # some utils # module Util def indent(s, n = 2) s = unindent(s) ws = ' ' * n s.gsub(%r/^/, ws) end def unindent(s) indent = nil s.each_line do |line| next if line =~ %r/^\s*$/ indent = line[%r/^\s*/] and break end indent ? s.gsub(%r/^#{ indent }/, "") : s end extend self end # template support # class Template def initialize(&block) @block = block @template = block.call.to_s end def expand(b=nil) ERB.new(Util.unindent(@template)).result((b||@block).binding) end alias_method 'to_s', 'expand' end def Template(*args, &block) Template.new(*args, &block) end # colored console output support # This.ansi = { :clear => "\e[0m", :reset => "\e[0m", :erase_line => "\e[K", :erase_char => "\e[P", :bold => "\e[1m", :dark => "\e[2m", :underline => "\e[4m", :underscore => "\e[4m", :blink => "\e[5m", :reverse => "\e[7m", :concealed => "\e[8m", :black => "\e[30m", :red => "\e[31m", :green => "\e[32m", :yellow => "\e[33m", :blue => "\e[34m", :magenta => "\e[35m", :cyan => "\e[36m", :white => "\e[37m", :on_black => "\e[40m", :on_red => "\e[41m", :on_green => "\e[42m", :on_yellow => "\e[43m", :on_blue => "\e[44m", :on_magenta => "\e[45m", :on_cyan => "\e[46m", :on_white => "\e[47m" } def say(phrase, *args) options = args.last.is_a?(Hash) ? args.pop : {} options[:color] = args.shift.to_s.to_sym unless args.empty? keys = options.keys keys.each{|key| options[key.to_s.to_sym] = options.delete(key)} color = options[:color] bold = options.has_key?(:bold) parts = [phrase] parts.unshift(This.ansi[color]) if color parts.unshift(This.ansi[:bold]) if bold parts.push(This.ansi[:clear]) if parts.size > 1 method = options[:method] || :puts Kernel.send(method, parts.join) end # always run out of the project dir # Dir.chdir(This.dir) } lockfile-2.1.3/samples/0000755000004100000410000000000012311354417015002 5ustar www-datawww-datalockfile-2.1.3/samples/lock.sh0000755000004100000410000000066012311354417016273 0ustar www-datawww-data#!/bin/sh export RUBYLIB="../lib:./lib" export PATH="../bin:./bin:$PATH" #export LOCKFILE_DEBUG=1 (r=`ruby -e 'p(rand(2))'`; sleep $r; rlock ./lockfile 'sleep 1; printf "\n$$ aquired lock @ `date`\n\n"') & (r=`ruby -e 'p(rand(2))'`; sleep $r; rlock ./lockfile 'sleep 1; printf "\n$$ aquired lock @ `date`\n\n"') & (r=`ruby -e 'p(rand(2))'`; sleep $r; rlock ./lockfile 'sleep 1; printf "\n$$ aquired lock @ `date`\n\n"') & wait lockfile-2.1.3/samples/out0000644000004100000410000000361012311354417015534 0ustar www-datawww-datalock_id < {"pid"=>"15681", "time"=>"2004-11-16 17:08:42.931418", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15674"} > attempting to lock <./lockfile>... polling attempt <0>... aquired lock <./lockfile> touched <./lockfile> @ <1100650122.93628> loaded < {"pid"=>"15681", "time"=>"2004-11-16 17:08:42.931418", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15674"} > 15682 aquired lock @ Tue Nov 16 17:08:43 MST 2004 lock_id < {"pid"=>"15685", "time"=>"2004-11-16 17:08:43.962744", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15676"} > attempting to lock <./lockfile>... polling attempt <0>... poll sleep <0.08>... lock_id < {"pid"=>"15684", "time"=>"2004-11-16 17:08:43.964636", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15672"} > attempting to lock <./lockfile>... polling attempt <0>... poll sleep <0.08>... polling attempt <1>...polling attempt <1>... aquired lock <./lockfile> poll sleep <0.08>... touched <./lockfile> @ <1100650124.04751> loaded < {"pid"=>"15685", "time"=>"2004-11-16 17:08:43.962744", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15676"} > polling attempt <2>... poll sleep <0.08>... polling attempt <3>... poll sleep <0.08>... polling attempt <4>... poll sleep <0.08>... polling attempt <5>... poll sleep <0.08>... polling attempt <6>... poll sleep <0.08>... polling attempt <7>... poll sleep <0.08>... polling attempt <8>... poll sleep <0.08>... polling attempt <9>... poll sleep <0.08>... polling attempt <10>... poll sleep <0.08>... polling attempt <11>... poll sleep <0.08>... polling attempt <12>... poll sleep <0.08>... polling attempt <13>... poll sleep <0.08>... 15690 aquired lock @ Tue Nov 16 17:08:45 MST 2004 polling attempt <14>... poll sleep <0.08>... polling attempt <15>... aquired lock <./lockfile> touched <./lockfile> @ <1100650125.16655> loaded < {"pid"=>"15684", "time"=>"2004-11-16 17:08:43.964636", "host"=>"jib.ngdc.noaa.gov", "ppid"=>"15672"} > 15694 aquired lock @ Tue Nov 16 17:08:46 MST 2004 lockfile-2.1.3/samples/a.rb0000755000004100000410000000552112311354417015555 0ustar www-datawww-data#!/usr/bin/env ruby $:.unshift '../lib' # # puts this script in an nfs located directory and run from a couple of nodes at # once. the list should appear ordered from either host - note that times may # not be ordered depending on the system clocks # # # builtin # require 'socket' require 'pstore' # # do what we can to invalidate any nfs caching # def timestamp time = Time.now #{{{ usec = "#{ time.usec }" usec << ('0' * (6 - usec.size)) if usec.size < 6 time.strftime('%Y-%m-%d %H:%M:%S.') << usec #}}} end def hostname #{{{ @__hostname__ ||= Socket::gethostname #}}} end def tmpnam dir = Dir.tmpdir, seed = File.basename($0) #{{{ pid = Process.pid path = "%s_%s_%s_%s_%d" % [hostname, seed, pid, timestamp.gsub(/\s+/o,'_'), rand(101010)] File.join(dir, path) #}}} end def uncache file #{{{ refresh = nil begin is_a_file = File === file path = (is_a_file ? file.path : file.to_s) stat = (is_a_file ? file.stat : File.stat(file.to_s)) refresh = tmpnam(File.dirname(path)) File.link path, refresh rescue File.symlink path, refresh File.chmod stat.mode, path File.utime stat.atime, stat.mtime, path ensure begin File.unlink refresh if refresh rescue Errno::ENOENT end end #}}} end # # raa - http://raa.ruby-lang.org/project/lockfile/ # require 'lockfile' pstore = PStore.new 'test.db' timeout = 60 max_age = 8 refresh = 2 debug = false lockfile = Lockfile.new 'test.lock', :timeout => timeout, :max_age => max_age, :refresh => refresh, :debug => debug # # flock throws ENOLCK on nfs file systems in newer linux kernels # plus we want to show that lockfile alone can do the locking # class File def flock(*args,&block);true;end end # # if locking does not work this loop will blow up (Marshal load error) or appear # un-ordered. actually it will eventually blow up due to nfs caching - but that # is not the fault of the lockfile class! for the most part it is a simply demo # of locking. the file will never become corrupt, it just will be unreadable at # times due to kernel caching. # loop do lockfile.lock do uncache pstore.path pstore.transaction do # # get/update list # pstore[:list] = [] unless pstore.root? :list list = pstore[:list] tuple = [list.size, hostname, Time.now.to_f] list << tuple # # show last 16 elements # puts '---' list[-([list.size, 16].min)..-1].each{|tuple| p tuple} puts '---' # # keep it a reasonable size # list.shift while list.size > 1024 # # write back updates # pstore[:list] = list end end sleep 1 end lockfile-2.1.3/samples/lock0000644000004100000410000000011712311354417015654 0ustar www-datawww-datahost: jib.ngdc.noaa.gov pid: 18176 ppid: 5245 time: 2004-11-18 14:40:43.218731 lockfile-2.1.3/samples/nfsstore.rb0000644000004100000410000001031612311354417017173 0ustar www-datawww-data# # How to use: # # db = NFSStore.new("/tmp/foo") # db.transaction do # p db.roots # ary = db["root"] = [1,2,3,4] # ary[0] = [1,1.5] # end # db.transaction do # p db["root"] # end require "ftools" require "digest/md5" require "socket" require 'lockfile' class NFSStore HOSTNAME = Socket::gethostname class Error < StandardError end def initialize(file) dir = File::dirname(file) unless File::directory? dir raise NFSStore::Error, format("directory %s does not exist", dir) end if File::exist? file and not File::readable? file raise NFSStore::Error, format("file %s not readable", file) end @transaction = false @filename = file @lockfile = Lockfile.new "#{ @filename }.lock",:max_age=>64,:refresh=>8 @abort = false end def in_transaction raise NFSStore::Error, "not in transaction" unless @transaction end private :in_transaction def [](name) in_transaction @table[name] end def fetch(name, default=NFSStore::Error) unless @table.key? name if default==NFSStore::Error raise NFSStore::Error, format("undefined root name `%s'", name) else default end end self[name] end def []=(name, value) in_transaction @table[name] = value end def delete(name) in_transaction @table.delete name end def roots in_transaction @table.keys end def root?(name) in_transaction @table.key? name end def path @filename end def commit in_transaction @abort = false throw :pstore_abort_transaction end def abort in_transaction @abort = true throw :pstore_abort_transaction end # do what we can to invalidate any nfs caching def uncache file begin stat = file.stat path = file.path refresh = "%s.%s.%s.%s" % [path, HOSTNAME, $$, Time.now.to_i % 1024] File.link path, refresh file.chmod stat.mode File.utime stat.atime, stat.mtime, path rescue Exception => e warn e ensure begin File.unlink refresh if File.exist? refresh rescue Exception => e warn e end end end def transaction(read_only=false) raise NFSStore::Error, "nested transaction" if @transaction file = nil value = nil @lockfile.lock do begin @transaction = true backup = @filename+"~" begin file = File::open(@filename, read_only ? "rb" : "rb+") orig = true rescue Errno::ENOENT raise if read_only file = File::open(@filename, "wb+") end #file.flock(read_only ? File::LOCK_SH : File::LOCK_EX) uncache file file.rewind if read_only @table = Marshal::load(file) elsif orig and (content = file.read) != "" @table = Marshal::load(content) size = content.size md5 = Digest::MD5.digest(content) content = nil # unreference huge data else @table = {} end begin catch(:pstore_abort_transaction) do value = yield(self) end rescue Exception @abort = true raise ensure if !read_only and !@abort file.rewind content = Marshal::dump(@table) if !md5 || size != content.size || md5 != Digest::MD5.digest(content) File::copy @filename, backup begin file.write(content) file.truncate(file.pos) content = nil # unreference huge data rescue File::rename backup, @filename if File::exist?(backup) raise end end end if @abort and !orig File.unlink(@filename) end @abort = false end ensure @table = nil @transaction = false file.close if file end end value end end if __FILE__ == $0 db = NFSStore.new("/tmp/foo") db.transaction do p db.roots ary = db["root"] = [1,2,3,4] ary[1] = [1,1.5] end 1000.times do db.transaction do db["root"][0] += 1 p db["root"][0] end end db.transaction(true) do p db["root"] end end lockfile-2.1.3/samples/lockfile0000644000004100000410000000011712311354417016514 0ustar www-datawww-datahost: jib.ngdc.noaa.gov pid: 18193 ppid: 5245 time: 2004-11-18 14:41:59.320552 lockfile-2.1.3/metadata.yml0000644000004100000410000000221012311354417015634 0ustar www-datawww-data--- !ruby/object:Gem::Specification name: lockfile version: !ruby/object:Gem::Version version: 2.1.3 platform: ruby authors: - Ara T. Howard autorequire: bindir: bin cert_chain: [] date: 2014-03-04 00:00:00.000000000 Z dependencies: [] description: a ruby library for creating perfect and NFS safe lockfiles email: ara.t.howard@gmail.com executables: - rlock extensions: [] extra_rdoc_files: [] files: - README - bin/rlock - doc/rlock.help - lib/lockfile.rb - lockfile.gemspec - rakefile - readme.erb - samples/a.rb - samples/lock - samples/lock.sh - samples/lockfile - samples/nfsstore.rb - samples/out homepage: https://github.com/ahoward/lockfile licenses: - Ruby metadata: {} post_install_message: rdoc_options: [] require_paths: - lib required_ruby_version: !ruby/object:Gem::Requirement requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' required_rubygems_version: !ruby/object:Gem::Requirement requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' requirements: [] rubyforge_project: codeforpeople rubygems_version: 2.0.3 signing_key: specification_version: 4 summary: lockfile test_files: [] lockfile-2.1.3/lockfile.gemspec0000644000004100000410000000153012311354417016472 0ustar www-datawww-data## lockfile.gemspec # Gem::Specification::new do |spec| spec.name = "lockfile" spec.version = "2.1.3" spec.platform = Gem::Platform::RUBY spec.summary = "lockfile" spec.description = "a ruby library for creating perfect and NFS safe lockfiles" spec.license = "Ruby" spec.files = ["README", "bin", "bin/rlock", "doc", "doc/rlock.help", "lib", "lib/lockfile.rb", "lockfile.gemspec", "rakefile", "readme.erb", "samples", "samples/a.rb", "samples/lock", "samples/lock.sh", "samples/lockfile", "samples/nfsstore.rb", "samples/out"] spec.executables = ["rlock"] spec.require_path = "lib" spec.test_files = nil spec.extensions.push(*[]) spec.rubyforge_project = "codeforpeople" spec.author = "Ara T. Howard" spec.email = "ara.t.howard@gmail.com" spec.homepage = "https://github.com/ahoward/lockfile" end lockfile-2.1.3/checksums.yaml.gz0000444000004100000410000000064212311354417016626 0ustar www-datawww-data@rSj0y ,l_ >6Fy8C ;IPu4oFνCo(ny }KYѣ.VK& 3 yNIz<Ϗ],6Em=Sd}! L :5a<6~ @g Lhg4&r\)}@ MtF# ts1҉kM_doVIO&]$+ѡcKi6X섺HSz̠o-Ө +&«:€u/^G=.[