AMQP.start(:port => 5671,
:ssl => {
:cert_chain_file => certificate_chain_file_path,
:private_key_file => client_private_key_file_path
}) do |connection|
puts "Connected, authenticated. TLS seems to work."
connection.disconnect { puts "Now closing the connection…"; EventMachine.stop }
end
Note that TLS connection may take a bit of time to establish (up to several seconds in some cases). To verify that
broker connection actually uses TLS, refer to RabbitMQ log file:
=INFO REPORT==== 28-Jun-2011::08:41:24 === accepted TCP connection on 0.0.0.0:5671 from 127.0.0.1:53444 =INFO REPORT==== 28-Jun-2011::08:41:24 === starting TCP connection <0.9904.0> from 127.0.0.1:53444 =INFO REPORT==== 28-Jun-2011::08:41:24 === upgraded TCP connection <0.9904.0> to SSLh2. Example code TLS example (as well as sample certificates you can use to get started with) can be found in the "amqp gem git repository":https://github.com/ruby-amqp/amqp/tree/master/examples h2. Authors This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee. h2. Tell us what you think! Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp: what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is key to making documentation better. If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation amqp-1.8.0/docs/VendorSpecificExtensions.textile 0000644 0000041 0000041 00000017312 13321132064 022020 0 ustar www-data www-data # @title Ruby AMQP gem: Using vendor-specific AMQP extensions h1. Vendor-specific AMQP extensions support in amqp gem h2. This Documentation Has Moved to rubyamqp.info amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info. h2. RabbitMQ extensions h2. Supported extensions AMQP gem supports many RabbitMQ extensions to AMQP 0.9.1: * "Publisher confirmations":http://www.rabbitmq.com/extensions.html#confirms (confirm.* class) * "Negative acknowledgements":http://www.rabbitmq.com/extensions.html#negative-acknowledgements (basic.nack) * "Alternate Exchanges":http://www.rabbitmq.com/extensions.html#alternate-exchange * "Per-queue Message Time-to-Live":http://www.rabbitmq.com/extensions.html#queue-ttl * "Queue Leases":http://www.rabbitmq.com/extensions.html#queue-leases * "Sender-selected Distribution":http://www.rabbitmq.com/extensions.html#sender-selected-distribution * "Validated user_id":http://www.rabbitmq.com/extensions.html#validated-user-id h2. Enabling RabbitMQ extensions If you are using RabbitMQ as AMQP broker and want to use these extensions, simply replace
require "amqp"with
require "amqp" require "amqp/extensions/rabbitmq"h2. Per-queue Message Time-to-Live Per-queue Message Time-to-Live (TTL) is a RabbitMQ extension to AMQP 0.9.1 that lets developers control for how long a message published to a queue can live before it is discarded. A message that has been in the queue for longer than the configured TTL is said to be dead. Dead messages will not be delivered to consumers and cannot be fetched using *basic.get* operation ({AMQP::Queue#pop}). Message TTL is specified using *x-message-ttl* argument on declaration. With amqp gem, you pass it to {AMQP::Queue#initialize} or {AMQP::Channel#queue}:
# 1000 milliseconds
channel.queue("", :arguments => { "x-message-ttl" => 1000 })
When a published messages is routed to multiple queues, each of the queues gets a _copy of the message_. If then the message dies in
one of the queues, it has no effect on copies of the message in other queues.
h3. Example
The example below sets message TTL for a new server-named queue to be 1000 milliseconds. It then publishes several messages that
are routed to the queue and tries to fetch messages using *basic.get* AMQP method ({AMQP::Queue#pop} after 0.7 and 1.5 seconds:
h3. Learn More
See also rabbitmq.com section on "Per-queue Message TTL":http://www.rabbitmq.com/extensions.html#queue-ttl
h2. Publisher Confirms (Publisher Acknowledgements)
In some situations not a single message can be lost. The only reliable way of doing so is using confirmations.
"Publisher Confirms AMQP extension":http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/ was designed to solve the reliable publishing problem.
Publisher confirms are similar to message acknowledgements documented in the {file:docs/Queues.textile Working With Queues} guide but involve publisher and AMQP broker
instead of consumer and AMQP broker.
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/006_amqp_091_message_acknowledgements.png!
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/007_rabbitmq_publisher_confirms.png!
h3. Public API
To use publisher confirmations, first put channel into confirmation mode using {AMQP::Channel#confirm_select}:
channel.confirm_select
From this moment on, every message published on this channel will cause channel's _publisher index_ (message counter) to be incremented. It is possible to access
using {AMQP::Channel#publisher_index} method. To check whether channel is in the confirmation mode, use {AMQP::Channel#uses_publisher_confirmations?} predicate.
To handle AMQP broker acknowledgements, define a handler using {AMQP::Channel#on_ack}, for example:
channel.on_ack do |basic_ack|
puts "Received basic_ack: multiple = #{basic_ack.multiple}, delivery_tag = #{basic_ack.delivery_tag}"
end
Delivery tag will indicate number of confirmed messages. If *multiple* attribute is true, the confirmation is for all messages up to the number
delivery tag indicates. In other words, AMQP broker may confirm just one message or a batch of them.
h3. Example
h3. Learn More
See also rabbitmq.com section on "Confirms aka Publisher Acknowledgements":http://www.rabbitmq.com/extensions.html#confirms
h2. basic.nack
The AMQP specification defines the basic.reject method that allows clients to reject individual, delivered messages, instructing the broker
to either discard them or requeue them. Unfortunately, basic.reject provides no support for negatively acknowledging messages in bulk.
To solve this, RabbitMQ supports the basic.nack method that provides all the functionality of basic.reject whilst also
allowing for bulk processing of messages.
h3. Public API
When RabbitMQ extensions are loaded, {AMQP::Channel#reject} method is overriden via mixin to take one additional argument: multi (defaults to false).
When it is given and is true, amqp gem will use basic.nack AMQP method instead of basic.reject, to reject multiple messages at once.
Otherwise, basic.reject is used as usual.
h3. Learn More
See also rabbitmq.com section on "Confirms aka Publisher Acknowledgements":http://www.rabbitmq.com/extensions.html#negative-acknowledgements
h2. Alternate Exchanges
Alternate Exchanges is a RabbitMQ extension to AMQP 0.9.1 that lets developers define "fallback" exchanges where unroutable messages will be sent.
h3. Public API
To specify exchange A as alternate exchange to exchange B, specify 'alternate-exchange' argument on declaration of B:
exchange1 = channel.fanout("ops.fallback", :auto_delete => true)
exchange2 = channel.fanout("events.collector", :auto_delete => true, :arguments => { "alternate-exchange" => "ops.fallback" })
h3. Example
h3. Learn More
See also rabbitmq.com section on "Alternate Exchanges":http://www.rabbitmq.com/extensions.html#alternate-exchange
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/RabbitMQVersions.textile 0000644 0000041 0000041 00000007742 13321132064 020235 0 ustar www-data www-data # @title Ruby AMQP gem: RabbitMQ versions compatibility
h1. Ruby amqp gem and RabbitMQ versions compatibility
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide covers compatibility of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp with various versions of "RabbitMQ":http://rabbitmq.com messaging broker.
h2. Covered versions
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and later.
h2. RabbitMQ Version Requirement
amqp gem before 0.8.0 (0.6.x, 0.7.x) series implemented (most of) AMQP 0.8 specification. amqp gem 0.8.0 implements AMQP 0.9.1 and thus
*requires RabbitMQ version 2.0 or later*. See {file:docs/RabbitMQVersions.textile RabbitMQ versions} for more information about
RabbitMQ versions support and how obtain up-todate packages for your operating system.
amqp gem 0.8.0 and later versions implement AMQP 0.9.1 and thus *requires RabbitMQ version 2.0 or later*
h2. Using recent versions on Debian and Ubuntu
Ubuntu (even 10.10) and Debian both "ship with an old RabbitMQ version":http://packages.ubuntu.com/maverick/rabbitmq-server,
that only supports AMQP protocol 0.8. Ruby amqp gem 0.8.0 and later *will not work with RabbitMQ versions before 2.0.0*.
We strongly recommend that you use "RabbitMQ apt repository":http://www.rabbitmq.com/debian.html#apt that has recent versions of RabbitMQ.
h2. OpsCode Chef & Puppet
h3. Chef cookbook for RabbitMQ
There is a "Chef cookbook for RabbitMQ":https://github.com/opscode/cookbooks/tree/master/rabbitmq that installs recent versions from
the rabbitmq.com apt repository. It also has LWPRs (providers) for managing users and vhosts.
h3. RabbitMQ Puppet module
There is a "RabbitMQ Puppet module":https://github.com/puppetlabs/puppetlabs-rabbitmq by the Puppet Labs team. It uses .deb packages from
Debian testing and unstable repositories. Note that it has two dependencies:
* "puppet-stdlib":https://github.com/puppetlabs/puppetlabs-stdlib
* "puppet-apt":https://github.com/puppetlabs/puppet-apt
h2. TLS (SSL) support
Note that "before 1.7.0, RabbitMQ did not support TLS":http://www.rabbitmq.com/ssl.html. In order to have TLS
support, RabbitMQ 1.7.0 requires
* Erlang/OTP R13B or later
* Erlang SSL 3.10 or later
and recommends using Erlang R141B that ships with Erlang SSL 4.0.1. Learn more in our {file:docs/ConnectionEncryptionWithTLS.textile Using TLS (SSL)} guide.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/DocumentationGuidesIndex.textile 0000644 0000041 0000041 00000017363 13321132064 022005 0 ustar www-data www-data # @title Ruby AMQP gem documentation guides
h1. Ruby AMQP gem documentation guides
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. Which versions of the amqp gem do the guides cover?
These guides cover v0.8.0.RC14 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
h2. Documentation structure and how to read these guides
We recommend that you read the documentation guides in the order that they are listed. The *Getting Started* guide is written as a tutorial that
describes several typical applications and their problem scope, then provides full code examples and finally breaks them down
into smaller pieces that are explained in detail.
Once you are finished with the tutorial, reading the other guides in sequence will gradually explain all of the AMQP 0.9.1 features that are relevant to
application developers, including application design concerns and common practices. At present, some guides are yet to be written
but *most of the content is concentrated in just 3-4 guides* that are about 80% finished and provide plenty of examples.
Here is a summary of guides and their content:
{
:host => "127.0.0.1",
:port => 5672,
:user => "guest",
:pass => "guest",
:vhost => "/",
:ssl => false,
:frame_max => 131072
}
h3. Using connection strings
It is convenient to be able to specify the AMQP connection
parameters as a URI string, and various "amqp" URI schemes
exist. Unfortunately, there is no standard for these URIs, so
while the schemes share the same basic idea, they differ in some
details. This implementation aims to encourage URIs that work
as widely as possible.
Here are some examples:
* amqp://dev.rabbitmq.com
* amqp://dev.rabbitmq.com:5672
* amqp://guest:guest@dev.rabbitmq.com:5672
* amqp://hedgehog:t0ps3kr3t@hub.megacorp.internal/production
* amqps://hub.megacorp.internal/%2Fvault
The URI scheme should be "amqp", or "amqps" if SSL is required.
The host, port, username and password are represented in the
authority component of the URI in the same way as in http URIs.
The vhost is obtained from the first segment of the path, with the
leading slash removed. The path should contain only a single segment
(i.e, the only slash in it should be the leading one). If the vhost
is to include slashes or other reserved URI characters, these should
be percent-escaped.
Here are some examples that demonstrate how
{AMQP::Client.parse_connection_uri} parses out the vhost from
connection URIs:
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com") # => vhost is nil, so default ("/") will be used
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/") # => vhost is an empty string
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/%2Fvault") # => vhost is "/vault"
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/production") # => vhost is "production"
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/a.b.c") # => vhost is "a.b.c"
AMQP::Client.parse_connection_uri("amqp://dev.rabbitmq.com/foo/bar") # => ArgumentError
h2. Starting the event loop and connecting in standalone applications
h3. EventMachine event loop
The amqp gem uses "EventMachine":http://rubyeventmachine.com under the hood and needs an EventMachine
event loop to be running in order to connect to an AMQP broker or to send any data. This means that
before connecting to an AMQP broker, we need to _start the EventMachine reactor_ (get the event loop
going). Here is how to do it:
require "amqp"
EventMachine.run do
# ...
end
"EventMachine.run":http://eventmachine.rubyforge.org/EventMachine.html#M000461 will block the current thread until the event loop is stopped.
Standalone applications often can afford to start the event loop on the main thread. If you have no experience with threading, this is a
recommended way to proceed.
h3. Using AMQP.connect with a block
Once the event loop is running, the {AMQP.connect} method will attempt to connect to the broker. It can be used in two ways. Here is the
first one:
require "amqp"
EventMachine.run do
# using AMQP.connect with a block
AMQP.connect(:host => "localhost") do |client|
# connection is open and ready to be used
end
end
{AMQP.connect} takes a block that will be executed as soon as the AMQP connection is open. In order for a connection to be opened a TCP connection has to be set up,
authentication has to succeed, and the broker and client need to complete negotiation of connection parameters like max frame size.
h3. Using AMQP.connect without a callback
An alternative way of connecting is this:
require "amqp"
EventMachine.run do
# using AMQP.connect with a block
client = AMQP.connect(:host => "hub.megacorp.internal", :username => "hedgehog", :password => "t0ps3kr3t")
# connection is not yet open, however, amqp gem will delay channel
# operations until after the connection is open. Bear in mind that
# amqp gem cannot solve every possible race condition so be careful
end
If you do not need to assign the returned value to a variable, then the "block version" is recommended because it eliminates issues that may
arise from attempts to use a connection object that is not fully opened yet. For example, handling of authentication failures is simpler
with the block version, as we will see in the following sections.
h3. Using AMQP.start
EventMachine.run and {AMQP.connect} with a block is such a common combination that the amqp gem provides a shortcut:
require "amqp"
AMQP.start("amqp://dev.rabbitmq.com:5672") do |client|
# connection is open and ready to be used
end
As these examples demonstrate, {AMQP.connect} and {AMQP.start} accept either a Hash of connection options or a connection URI string.
See the reference documentation for each method to learn all of the options that they accept and what the default values are.
h3. On Thread#sleep use
When not passing a block to {AMQP.connect}, it is tempting to "give the connection some time to become established" by using Thread#sleep. Unless you are
running the event loop in a separate thread, please do not do this. Thread#sleep blocks the current thread so that if the event loop is running in the current thread,
blocking the thread _will also block the event loop_. *When the event loop is blocked, no data is sent or received, so the connection does not proceed.*
h3. Detecting TCP connection failures
When applications connect to the broker, they need to handle connection failures. Networks are not 100% reliable and even with modern system configuration tools,
like "Chef":http://http://www.opscode.com/chef or "Puppet":http://http://www.puppetlabs.com, misconfigurations can happen. Also, the broker might be down for some reason. Ideally, error detection should happen as early as possible. There are two ways of detecting
TCP connection failure, the first one is to catch an exception:
#!/usr/bin/env ruby
# encoding: utf-8
require "rubygems"
require "amqp"
puts "=> TCP connection failure handling with a rescue statement"
puts
connection_settings = {
:port => 9689,
:vhost => "/amq_client_testbed",
:user => "amq_client_gem",
:password => "amq_client_gem_password",
:timeout => 0.3
}
begin
AMQP.start(connection_settings) do |connection, open_ok|
raise "This should not be reachable"
end
rescue AMQP::TCPConnectionFailed => e
puts "Caught AMQP::TCPConnectionFailed => TCP connection failed, as expected."
end
{AMQP.connect} (and {AMQP.start}) will raise {AMQP::TCPConnectionFailed} if the connection fails. Code that catches the error can write to a log
about the issue or use retry to execute the begin block one more time. Because initial connection failures are due to misconfiguration or network outage, reconnection
to the same endpoint (hostname, port, vhost combination) will result in the same error over and over again. TBD: failover, connection to the cluster.
An alternative way of handling connection failure is with an _errback_ (a callback for a specific kind of error):
#!/usr/bin/env ruby
# encoding: utf-8
require "rubygems"
require "amqp"
puts "=> TCP connection failure handling with a callback"
puts
handler = Proc.new { |settings| puts "Failed to connect, as expected"; EM.stop }
connection_settings = {
:port => 9689,
:vhost => "/amq_client_testbed",
:user => "amq_client_gem",
:password => "amq_client_gem_password",
:timeout => 0.3,
:on_tcp_connection_failure => handler
}
AMQP.start(connection_settings) do |connection, open_ok|
raise "This should not be reachable"
end
the ":on_tcp_connection_failure" option accepts any object that responds to #call.
If you connect to the broker from code in a class (as opposed to top-level scope in a script), Object#method can be used to pass an object method as a handler
instead of a Proc.
TBD: provide an example
h3. Detecting authentication failures
A connection may also fail due to authentication failure. Handling authentication failure is very similar to handling an initial TCP
connection failure:
#!/usr/bin/env ruby
# encoding: utf-8
require "rubygems"
require "amqp"
puts "=> Authentication failure handling with a callback"
puts
handler = Proc.new { |settings| puts "Failed to connect, as expected"; EM.stop }
connection_settings = {
:port => 5672,
:vhost => "/amq_client_testbed",
:user => "amq_client_gem",
:password => "amq_client_gem_password_that_is_incorrect #{Time.now.to_i}",
:timeout => 0.3,
:on_tcp_connection_failure => handler,
:on_possible_authentication_failure => Proc.new { |settings|
puts "Authentication failed, as expected, settings are: #{settings.inspect}"
EM.stop
}
}
AMQP.start(connection_settings) do |connection, open_ok|
raise "This should not be reachable"
end
default handler raises {AMQP::PossibleAuthenticationFailureError}:
#!/usr/bin/env ruby
# encoding: utf-8
require "rubygems"
require "amqp"
puts "=> Authentication failure handling with a rescue block"
puts
handler = Proc.new { |settings| puts "Failed to connect, as expected"; EM.stop }
connection_settings = {
:port => 5672,
:vhost => "/amq_client_testbed",
:user => "amq_client_gem",
:password => "amq_client_gem_password_that_is_incorrect #{Time.now.to_i}",
:timeout => 0.3,
:on_tcp_connection_failure => handler
}
begin
AMQP.start(connection_settings) do |connection, open_ok|
raise "This should not be reachable"
end
rescue AMQP::PossibleAuthenticationFailureError => afe
puts "Authentication failed, as expected, caught #{afe.inspect}"
EventMachine.stop if EventMachine.reactor_running?
end
In case you are wondering why the callback name has "possible" in it, {http://bit.ly/amqp091spec AMQP 0.9.1 spec} requires broker implementations to
simply close the TCP connection without sending any more data when an exception, such as authentication failure, occurs before the AMQP connection
is open. In practice, however, when a broker closes a TCP connection after a successful TCP connection has been established but before an AMQP connection is open,
it means that authentication has failed.
h2. Starting the event loop and connecting in Web applications (Ruby on Rails, Sinatra, Merb, Rack)
Web applications are different from standalone applications in that the main thread is occupied by a Web/application server like Unicorn
or Thin, so you need to start the EventMachine reactor before you attempt to use {AMQP.connect}.
In a Ruby on Rails application, probably the best place for this is in the initializer (like config/initializers/amqp.rb). For Merb applications it is config/init.rb.
For Sinatra and pure Rack applications, place it next to the other configuration code.
Next, we are going to discuss issues specific to particular Web servers.
h3. Using Ruby amqp gem with Unicorn
h4. Unicorn is a pre-forking server
"Unicorn":http://unicorn.bogomips.org is a pre-forking server. That means it forks worker processes that serve HTTP requests. The "fork(2)":http://en.wikipedia.org/wiki/Fork_(operating_system) system call
has several gotchas associated with it, two of which affect EventMachine and the "Ruby amqp gem":http://github.com/ruby-amqp/amqp:
* Unintentional file descriptor sharing
* The fact that a "forked child process only inherits one thread":http://bit.ly/fork-and-threads and therefore the EventMachine thread is not inherited
To avoid both problems, start the EventMachine reactor and AMQP connection *after* the master process forks workers. The master Unicorn process never serves HTTP requests and usually
does not need to hold an AMQP connection. Next, let us see how to spin up the EventMachine reactor and connect to the broker after Unicorn forks a worker.
h4. Starting the EventMachine reactor and connecting to the broker after Unicorn forks worker processes
Unicorn lets you specify a configuration file to use. In that file you define a callback that Unicorn runs after it forks worker process(es):
ENV["FORKING"] = "true"
listen 3000
worker_processes 1
timeout 30
preload_app true
after_fork do |server, worker|
require "amqp"
# the following is *required* for Rails + "preload_app true",
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
t = Thread.new { AMQP.start }
sleep(1.0)
EventMachine.next_tick do
AMQP.channel ||= AMQP::Channel.new(AMQP.connection)
AMQP.channel.queue("amqpgem.examples.rails23.warmup", :durable => true)
3.times do |i|
puts "[after_fork/amqp] Publishing a warmup message ##{i}"
AMQP.channel.default_exchange.publish("A warmup message #{i} from #{Time.now.strftime('%H:%M:%S %m/%b/%Y')}", :routing_key => "amqpgem.examples.rails23.warmup")
end
end
end
In the example above we start the EventMachine reactor in a separate thread, block the current thread for 1 second to let the event loop spin up and then
connect to the AMQP broker on the next event loop tick. Publishing several warmup messages on boot is a good idea because it
allows the early detection of issues that forking may cause.
Note that a configuration file can easily be used in development environments because, other than the fact that Unicorn runs in the foreground,
it gives you exactly the same application boot behavior as in QA and production environments.
An "example Ruby on Rails application that uses the Ruby amqp gem and Unicorn":http://bit.ly/ruby-amqp-gem-example-with-ruby-on-rails-and-unicorn is available on GitHub.
h3. Using the Ruby amqp gem with Passenger
"Phusion Passenger":http://www.modrails.com is also a pre-forking server, and just as with Unicorn, the EventMachine reactor and AMQP connection should be started *after* it forks worker
processes. The Passenger documentation has "a section":http://bit.ly/passenger-forking-gotchas that explains how to avoid problems related to the behavior of the fork(2) system
call, namely:
* Unintentional file descriptor sharing
* The fact that a "forked child process only inherits one thread":http://bit.ly/fork-and-threads and therefore the EventMachine thread is not inherited
h4. Using an event handler to spawn one amqp connection per worker
Passenger provides a hook that you should use for spawning AMQP connections:
if defined?(PhusionPassenger) # otherwise it breaks rake commands if you put this in an initializer
PhusionPassenger.on_event(:starting_worker_process) do |forked|
if forked
# We're in a smart spawning mode
# Now is a good place to connect to the broker
end
end
end
Basically, the recommended default smart spawn mode works exactly the same as in Unicorn (with all of the same common pitfalls). An "example application":http://bit.ly/ruby-amqp-gem-example-with-ruby-on-rails-and-passenger is available on github.
h3. Using the Ruby amqp gem with Thin and Goliath
h4. Thin and Goliath start the EventMachine reactor for you, but there is a little nuance
If you use "Thin":http://code.macournoyer.com/thin/ or "Goliath":https://github.com/postrank-labs/goliath/, you are all set because those two servers use EventMachine under the hood.
There is no need to start the EventMachine reactor. However, depending on the application server, its version, the version of the framework and Rack middleware being used,
EventMachine reactor start may be slightly delayed. To overcome this potential difficulty, use EventMachine.next_tick to delay connection until after the reactor is actually running:
EventMachine.next_tick { AMQP.connect(...) }
So, in case the EventMachine reactor is not yet running on server/application boot, the connection will not fail but will instead wait for the reactor to start.
Thin and Goliath are not pre-forking servers so there is no need to re-establish the connection as you do with Unicorn and Passenger.
h2. If it just does not work: troubleshooting
If you have read this guide and your issue is still unresolved, check our {file:docs/Troubleshooting.textile Troubleshooting guide} before asking on the mailing list.
h2. What to read next
* {file:docs/Queues.textile Working With Queues}. This guide focuses on features consumer applications use heavily.
* {file:docs/Exchanges.textile Working With Exchanges}. This guide focuses on features producer applications use heavily.
* {file:docs/ErrorHandling.textile Error handling}
* {file:docs/ConnectionEncryptionWithTLS.textile Using TLS (SSL)} (if you want to use an SSL encrypted connection to the broker)
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
key to making the documentation better.
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/ErrorHandling.textile 0000644 0000041 0000041 00000060525 13321132064 017577 0 ustar www-data www-data # @title Ruby AMQP gem: Error handling and recovery
h1. Error handling and recovery
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
Development of a robust application, be it message publisher or message consumer, involves dealing with multiple kinds of failures: protocol exceptions, network failures, broker failures and so on. Correct error handling and recovery is not easy. This guide explains how amqp gem helps you in dealing with issues like
* Initial broker connection failures
* Network connection interruption
* AMQP connection-level exceptions
* AMQP channel-level exceptions
* Broker failure
* TLS (SSL) related issues
as well as
* How to recover after a network failure
* What is automatic recovery mode, when you should and should not use it
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Covered versions
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0.RC14 and later.
h2. Code examples
There are several {https://github.com/ruby-amqp/amqp/tree/master/examples/error_handling examples} in the git repository dedicated to the topic of error handling and recovery. Feel free to contribute new examples.
h3. Initial broker connection failures
When applications connect to the broker, they need to handle connection failures. Networks are not 100% reliable, even with modern system configuration tools like Chef or Puppet misconfigurations happen and broker might be down, too. Error detection should happen as early as possible. There are two ways of detecting TCP connection failure, the first one is to catch an exception:
begin
AMQP.start(connection_settings) do |connection, open_ok|
raise "This should not be reachable"
end
rescue AMQP::TCPConnectionFailed => e
puts "Caught AMQP::TCPConnectionFailed => TCP connection failed, as expected."
end
Full example:
{AMQP.connect} (and {AMQP.start}) will raise {AMQP::TCPConnectionFailed} if connection fails. Code that catches it can write to log about the issue or use retry to execute begin block one more time. Because initial connection failures are due to misconfiguration or network outage, reconnection to the same endpoint (hostname, port, vhost combination) will result in the same issue over and over. TBD: failover, connection to the cluster.
Alternative way of handling connection failure is with an errback (a callback for specific kind of error):
handler = Proc.new { |settings| puts "Failed to connect, as expected"; EventMachine.stop }
connection_settings = {
:port => 9689,
:vhost => "/amq_client_testbed",
:user => "amq_client_gem",
:password => "amq_client_gem_password",
:timeout => 0.3,
:on_tcp_connection_failure => handler
}
Full example:
:on_tcp_connection_failure option accepts any object that responds to #call.
If you connect to the broker from a code in a class (as opposed to top-level scope in a script), Object#method can be used to pass object method as a handler instead of a Proc.
TBD: provide an example
h3. Authentication failures
Another reason why connection may fail is authentication failure. Handling authentication failure is very similar to handling initial TCP connection failure:
h4. Default handler
default handler raises {AMQP::PossibleAuthenticationFailureError}:
In case you wonder why callback name has "possible" in it: {http://bit.ly/amqp091spec AMQP 0.9.1 spec} requires broker implementations to simply close TCP connection without sending any more data when an exception (such as authentication failure) occurs before AMQP connection is open. In practice, however, when broker closes TCP connection between successful TCP connection and before AMQP connection is open, it means that authentication has failed.
h2. Handling network connection interruptions
Network connectivity issues are sad fact of life in modern software systems. Event small products and projects these days consist of multiple applications, often running on more than one machine. Ruby amqp gem detects TCP connection failures and lets you handle them by defining a callback using {AMQP::Session#on_tcp_connection_loss}. That callback will be run when TCP connection fails, and will be passed two parameters: connection object and settings of the last successful connection.
connection.on_tcp_connection_loss do |connection, settings|
# reconnect in 10 seconds, without enforcement
connection.reconnect(false, 10)
end
Sometimes it is necessary for other entities in an application to react to network failures. amqp gem 0.8.0 and later provides a number event handlers to make this task easier for developers. This set of features is know as the "shutdown protocol" (the word "protocol" here means "API interface" or "behavior", not network protocol).
{AMQP::Session}, {AMQP::Channel}, {AMQP::Exchange}, {AMQP::Queue} and {AMQP::Consumer all implement shutdown protocol and thus error handling API is consistent for all classes, with {AMQP::Session} and #{AMQP::Channel} having a few methods other entities do not have.
The Shutdown protocol revolves around two events:
* Network connection fails
* Broker closes AMQP connection (or channel)
In this section, we concentrate only on the former. When network connection fails, the underlying networking library detects it and runs a piece of code on {AMQP::Session} to handle it. That, in turn, propagates this event to channels, channels propagate it to exchanges and queues, queues propagate it to their consumers (if any). Each of these entities in the object graph can react to network interruption by executing application-defined callbacks.
h3. Shutdown Protocol methods on AMQP::Session
* {AMQP::Session#on_tcp_connection_loss}
* {AMQP::Session#on_connection_interruption}
The difference between these methods is that {AMQP::Session#on_tcp_connection_loss} is used to define a callback that will be executed *once* when TCP connection fails. It is possible that reconnection attempts will not succeed immediately, so there will be subsequent failures. To react to those, {AMQP::Session#on_connection_interruption} method is used.
First argument that both of these methods yield to the handler your application defines is the connection itself. This is done to make sure you can register Ruby objects as handlers, and they do not have to keep any state around (for example, connection instances):
connection.on_connection_interruption do |conn|
puts "Connection detected connection interruption"
end
# or
class ConnectionInterruptionHandler
#
# API
#
def handle(connection)
# handling logic
end
end
handler = ConnectionInterruptionHandler.new
connection.on_connection_interruption(&handler.method(:handle))
Note that {AMQP::Session#on_connection_interruption} callback is called *before* this event is propagated to channels, queues and so on.
Different applications handle connection failures differently. It is very common to use {AMQP::Session#reconnect} method to schedule a reconnection to the same host, or use {AMQP::Session#reconnect_to} to connect to a different one.
For some applications it is OK to simply exit and wait to be restarted at a later point in time, for example, by a process monitoring system like Nagios or Monit.
h3. Shutdown Protocol methods on AMQP::Channel
{AMQP::Channel} provides only one method: {AMQP::Channel#on_connection_interruption}, that registers a callback similar to the one seen in the previous section:
channel.on_connection_interruption do |ch|
puts "Channel #{ch.id} detected connection interruption"
end
Note that {AMQP::Channel#on_connection_interruption} callback is called *after* this event is propagated to exchanges, queues and so on. Right after that channel state is reset, except for error handling/recovery-related callbacks.
Many applications do not need per-channel network failure handling.
h3. Shutdown Protocol methods on AMQP::Exchange
{AMQP::Exchange} provides only one method: {AMQP::Exchange#on_connection_interruption}, that registers a callback similar to
the one seen in the previous section:
exchange.on_connection_interruption do |ex|
puts "Exchange #{ex.name} detected connection interruption"
end
Many applications do not need per-exchange network failure handling.
h3. Shutdown Protocol methods on AMQP::Queue
{AMQP::Queue} provides only one method: {AMQP::Queue#on_connection_interruption}, that registers a callback similar to the one seen in the previous section:
queue.on_connection_interruption do |q|
puts "Queue #{q.name} detected connection interruption"
end
Note that {AMQP::Queue#on_connection_interruption} callback is called *after* this event is propagated to consumers.
Many applications do not need per-queue network failure handling.
h3. Shutdown Protocol methods on AMQP::Consumer
{AMQP::Consumer} provides only one method: {AMQP::Consumer#on_connection_interruption}, that registers a callback similar to the one seen in the previous section:
consumer.on_connection_interruption do |c|
puts "Consumer with consumer tag #{c.consumer_tag} detected connection interruption"
end
Many applications do not need per-consumer network failure handling.
h2. Recovering from network connection failures
Detecting network connections is nearly useless if AMQP-based application cannot recover from them. Recovery is the hard part
in "error handling and recovery". Fortunately, recovery process for many applications follows one simple scheme that amqp
gem can perform automatically for you.
Recovery process, both manual and automatic, always begins with re-opening AMQP connection and then all the channels on that connection.
h3. Manual recovery
Similarly to the Shutdown Protocol, amqp gem entities implement Recovery Protocol. Recovery Protocol consists of 3 methods connections, channels, queues, consumers and exchanges implement:
* {AMQP::Session#before_recovery}
* {AMQP::Session#auto_recover}
* {AMQP::Session#after_recovery}
{AMQP::Session#before_recovery} lets application developers register a callback that will be executed *after TCP connection is re-established but before AMQP connection is reopened*. {AMQP::Session#after_recovery} is similar except that the callback is run *after AMQP connection is reopened*.
{AMQP::Channel}, {AMQP::Queue}, {AMQP::Consumer} and {AMQP::Exchange} methods behavior is identical.
Recovery process for AMQP applications usually involves the following steps:
# Re-open AMQP connection.
# Once connection is open again, re-open all AMQP channels on that connection.
# For each channel, re-declare all exchanges
# For each channel, re-declare all queues
# Once queue is declared, for each queue, re-register all bindings
# Once queue is declared, for each queue, re-register all consumers
h3. Automatic recovery
Many applications use the same recovery strategy, that consists of the following steps:
* Re-open channels
* For each channel, re-declare exchanges
* For each channel, re-declare queues
* For each queue, recover all bindings
* For each queue, recover all consumers
amqp gem provides a feature known as "automatic recovery" that is *opt-in* (not opt-out, not used by default) and lets application developers get aforementioned recovery strategy by setting one additional attribute on {AMQP::Channel} instance:
ch = AMQP::Channel.new(connection)
ch.auto_recovery = true
A more verbose way to do the same thing:
ch = AMQP::Channel.new(connection, AMQP::Channel.next_channel_id, :auto_recovery => true)
Note that if you do not want to pass any options, 2nd argument can be left out as well, then it will default to {AMQP::Channel.next_channel_id}.
To find out whether channel uses automatic recovery mode, use {AMQP::Channel#auto_recovering?}.
Auto recovery mode can be turned on and off any number of times during channel life cycle, although very small percentage of applications really does this. Typically you decide what channels should be using automatic recovery at application design stage.
Full example (run it, then shut down AMQP broker running on localhost, then bring it back up and use management tools such as `rabbitmqctl`
to see that queues & bindings & consumer have all recovered):
Server-named queues, when recovered automatically, will get *new server-generated names* to guarantee there are no name collisions.
When in doubt, try using automatic recovery first. If it is not sufficient for you application, switch to manual recovery using events and callbacks introduced in the "Manual recovery" section.
h2. Detecting broker failures
AMQP applications see broker failure as TCP connection loss. There is no reliable way to know whether there is a network split or network peer is down.
h2. AMQP connection-level exceptions
h3. Handling connection-level exceptions
Connection-level exceptions are rare and may indicate a serious issue with client library or in-flight data corruption. They mandate that connection cannot be used any more and must be closed. In any case, your application should be prepared to handle this kind of errors. To define a handler, use {AMQP::Session#on_error} method that takes a callback and yields two arguments to it when connection-level exception happens:
connection.on_error do |conn, connection_close|
puts "Handling a connection-level exception."
puts
puts "AMQP class id : #{connection_close.class_id}"
puts "AMQP method id: #{connection_close.method_id}"
puts "Status code : #{connection_close.reply_code}"
puts "Error message : #{connection_close.reply_text}"
end
Status codes are similar to those of HTTP. For the full list of error codes and their meaning, consult {http://www.rabbitmq.com/amqp-0-9-1-reference.html#constants AMQP 0.9.1 constants reference}.
Only one connection-level exception handler can be defined per connection instance (the one added last replaces previously added ones).
Full example:
h2. Handling graceful broker shutdown
When AMQP broker is shut down, it properly closes connection first. To do so, it uses *connection.close* AMQP method. AMQP clients then need to check if the reply code is equal to 320 (CONNECTION_FORCED) to distinguish graceful shutdown. With RabbitMQ, when broker is stopped using
rabbitmqctl stopreply_text will be set to
CONNECTION_FORCED - broker forced connection closure with reason 'shutdown'Each application choose how to handle graceful broker shutdowns individually, so *amqp gem's automatic reconnection does not cover graceful broker shutdowns*. Applications that want to reconnect when broker is stopped can use {AMQP::Session#periodically_reconnect} like so:
connection.on_error do |conn, connection_close|
puts "[connection.close] Reply code = #{connection_close.reply_code}, reply text = #{connection_close.reply_text}"
if connection_close.reply_code == 320
puts "[connection.close] Setting up a periodic reconnection timer..."
# every 30 seconds
conn.periodically_reconnect(30)
end
end
Once AMQP connection is re-opened, channels in automatic recovery mode will recover just like they do after network outages.
h2. Integrating channel-level exceptions handling with object-oriented Ruby code
Error handling can be easily integrated into object-oriented Ruby code (in fact, this is highly encouraged). A common technique is to combine {http://rubydoc.info/stdlib/core/1.8.7/Object:method Object#method} and {http://rubydoc.info/stdlib/core/1.8.7/Method:to_proc Method#to_proc} and use object methods as error handlers:
class ConnectionManager
#
# API
#
def connect(*args, &block)
@connection = AMQP.connect(*args, &block)
# combines Object#method and Method#to_proc to use object
# method as a callback
@connection.on_error(&method(:on_error))
end # connect(*args, &block)
def on_error(connection, connection_close)
puts "Handling a connection-level exception."
puts
puts "AMQP class id : #{connection_close.class_id}"
puts "AMQP method id: #{connection_close.method_id}"
puts "Status code : #{connection_close.reply_code}"
puts "Error message : #{connection_close.reply_text}"
end # on_error(connection, connection_close)
end
Full example that uses objects:
TBD
h2. AMQP channel-level exceptions
h3. Handling channel-level exceptions
Channel-level exceptions are more common than connection-level ones. They are handled in a similar manner, by defining a callback with {AMQP::Channel#on_error} method that takes a callback and yields two arguments to it when channel-level exception happens:
channel.on_error do |ch, channel_close|
puts "Handling a channel-level exception."
puts
puts "AMQP class id : #{channel_close.class_id}"
puts "AMQP method id: #{channel_close.method_id}"
puts "Status code : #{channel_close.reply_code}"
puts "Error message : #{channel_close.reply_text}"
end
Status codes are similar to those of HTTP. For the full list of error codes and their meaning, consult {http://www.rabbitmq.com/amqp-0-9-1-reference.html#constants AMQP 0.9.1 constants reference}.
Only one channel-level exception handler can be defined per channel instance (the one added last replaces previously added ones).
Full example:
h3. Integrating channel-level exceptions handling with object-oriented Ruby code
Error handling can be easily integrated into object-oriented Ruby code (in fact, this is highly encouraged). A common technique is to combine {http://rubydoc.info/stdlib/core/1.8.7/Object:method Object#method} and {http://rubydoc.info/stdlib/core/1.8.7/Method:to_proc Method#to_proc} and use object methods as error handlers. For example of this, see section on connection-level exceptions above.
Because channel-level exceptions may be raised because of multiple unrelated reasons and often indicate misconfigurations, how they are handled is very specific to particular applications. A common strategy is to log an error and then open and use another channel.
h3. Common channel-level exceptions and what they mean
A few channel-level exceptions are common and deserve more attention.
h4. 406 Precondition Failed
EventedSpec::AMQPSpec module into your context and add #done
calls to your examples:
h3. Testing in the Asynchronous Environment
Since we are using callback mechanisms in order to provide asynchronicity, we have to deal with situation when we expect a response,
and response never comes. Usual solution includes setting a timeout which makes the given tests fail if they aren't finished in a timely
manner. When #done is called, your tests confirm successful ending of specs. Try removing done from the above
example and see what happens. (spoiler: EventedSpec::SpecHelper::SpecTimeoutExceededError: Example timed out)
h3. The #done method
The *#done* method call is a hint for evented-spec to consider the example finished. If this method is not called, example will be forcefully
terminated after a certain period of time or "time out". This means there are two approaches to testing of asynchronous code:
* Have timeout value high enough for all operations to finish (for example, expected number of messages is received).
* Call #done when some condition holds true (for example, message with a specific property or payload is received).
The latter approach is recommended because it makes tests less dependent on machine-specific throughput or timing: it is very
common for continuous integration environments to use virtual machines that are significantly less powerful than machines developers
use, so timeouts have to be carefully adjusted to work in both settings.
h3. Default Connection Options and Timeout
It is sometimes desirable to use custom connection settings for your test environment as well as the default timeout value used. evented-spec lets you do
it:
Available options are passed to {AMQP.connect} so it is possible to specify host, port, vhost, username and password your test suite needs.
h3. Lifecycle Callbacks
evented-spec provides various callbacks similar to rspec's before(:each) / after(:each). They are called amqp_before and
amqp_after and happen right after connection is established or before connection is closed. It is a good place to put your channel initialization routines.
h3. Full Example
Now that you're filled on theory part, it's time to do something with all this knowledge. Below goes a slightly modified version of one of the integration specs from AMQP
suite. It sets up default topic exchange and publishes various messages about sports events:
Couple of things to notice: #done is invoked using an optional callback and optional delay, also instance variables behavior in hooks is the same as in "normal" rspec
hooks.
h3. Using #delayed
AMQP gem uses "EventMachine":http://eventmachine.rubyforge.org/ under hood. If you don't know about eventmachine, you can read more about it on the official site.
What's important for us is that you *cannot use sleep for delays*. Why? Because all the specs code is processed directly in the "reactor":http://en.wikipedia.org/wiki/Reactor_pattern thread, if you
sleep in that thread, reactor cannot send frames. What you need to use instead is #delayed method. It takes delay time in seconds and callback
which it launches once that time passes. Basic usage is either sleep replacement or ensuring certain order of execution (though, the latter should not bother
you too much). You can also use it to cleanup your environment after tests if any is needed.
In the following example, we declare two channels, then declare the same queue twice with the same name but different options (which raises a channel-level exception in AMQP):
If you draw a timeline, various events happen at 0.0s, then at 0.1s, then at 0.3s and eventually at 0.4s.
h3. Design For Testability
As *Integration With Objects* section of the {file:docs/GettingStarted.textile Getting Started with Ruby amqp gem and RabbitMQ} demonstrates, good object-oriented design
often makes it possible to test AMQP consumers in isolation without connecting to the broker or even starting EventMachine even loop. All the "Design for testability"
practices apply fully to AMQP application testing.
h3. Real worldExamples
Please refer to the "amqp gem test suite":https://github.com/ruby-amqp/amqp/tree/master/spec to see evented-spec in action.
h3. How evented-spec Works
When you include EventedSpec::AMQPSpec module, #it calls are wrapped in EventMachine.start + AMQP.connect
calls, so you can start writing your examples as if you're connected. Please note that you still need to open your own channel(s).
h2. What to read next
There is a lot more to evented-spec than described in this guide. "evented-spec documentation":http://rdoc.info/github/ruby-amqp/evented-spec/master
covers that gem in more detail gem. For more code examples, see "amqp Ruby gem test suite":https://github.com/ruby-amqp/amqp/tree/master/spec.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/Bindings.textile 0000644 0000041 0000041 00000015276 13321132064 016601 0 ustar www-data www-data # @title Ruby AMQP gem: Bindings
h1. Working With Bindings
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide covers bindings in AMQP 0.9.1, what they are, what role do they play and how to accomplish typical operations using
Ruby amqp gem.
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Covered versions
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and later.
h2. Bindings in AMQP 0.9.1
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q,
Q has to _be bound_ to E. Bindings may have an optional _routing key_ attribute used by some exchange types. The purpose of the routing key is to
selectively match only specific (matching) messages published to an exchange to the bound queue. In other words, the routing key acts like a filter.
Learn more about how bindings fit into the AMQP Model in the {file:docs/AMQP091ModelExplained.textile AMQP 0.9.1 Model Explained} documentation guide.
h2. What Are AMQP 0.9.1 Bindings
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q,
Q has to _be bound_ to E. Bindings may have an optional _routing key_ attribute used by some exchange types. The purpose of the routing key is to
selectively match only specific (matching) messages published to an exchange to the bound queue. In other words, the routing key acts like a filter.
To draw an analogy:
* Queue is like your destination in New York city
* Exchange is like JFK airport
* Bindings are routes from JFK to your destination. There can be none or more than one way to reach it
Some exchange types use routing key while some others do not (and route messages unconditionally or based on message metadata).
If AMQP message cannot be routed to any queue (for example, because there are no bindings for the exchange it was published to), it is either
dropped or returned to the publisher, depending on message attributes the publisher has set.
If application wants to connect a queue to an exchange, it needs to _bind_ them. The opposite operation is called _unbinding_.
h2. Binding queues to exchanges
In order to receive messages, a queue needs to be bound to at least one exchange. Most of the time binding is explcit
(done by applications). To bind a queue to an exchange, use {AMQP::Queue#bind} where the argument passed can be
either an {AMQP::Exchange} instance or a string.
queue.bind(exchange) do |bind_ok|
puts "Just bound #{queue.name} to #{exchange.name}"
end
Full example:
The same example using a string without callback:
queue.bind("amq.fanout")
Full example:
h2. Unbinding queues from exchanges
To unbind a queue from an exchange use {AMQP::Queue#unbind}:
queue.unbind(exchange)
Full example:
Trying to unbind a queue from an exchange that the queue was never bound to will result in a
channel-level exception.
h2. Bindings, Routing and Returned Messages
h3. How AMQP 0.9.1 Brokers Route Messages
After AMQP message reaches AMQP broker and before it reaches a consumer, several things happen:
* AMQP broker needs to find one or more queues the message needs to be routed to, depending on exchange
* AMQP broker puts a copy of the message into each of those queues or decides to return the messages to the publisher
* AMQP broker pushes messages to consumers on those queues or waits for applications to fetch them on demand
A more in-depth description is this:
* AMQP broker needs to consult bindings list for the exchange the message was published to find one or more queues the message needs to be routed to (step 1)
* If there are no suitable queues found during step 1 and the message was published as mandatory, it is returned to the publisher (step 1b)
* If there are suitable queues, a _copy_ of the message is placed into each one (step 2)
* If the message was published as mandatory, but there are no active consumers for it, it is returned to the publisher (step 2b)
* If there are active consumers on those queues and basic.qos setting permits, message is pushed to those consumers (step 3)
* If there are no active consumers and the message is *not* published as mandatory, it will be left in the queue
The takeaway is that messages may or may not be routed and it is important for applications to handle unroutable messages.
h3. Handling of Unroutable Messages
Unroutable messages are either dropped or returned back to producers. Vendor-specific extensions can provide additional ways of handing of unroutable
messages: for example, RabbitMQ's "Alternate Exchanges extension":http://www.rabbitmq.com/extensions.html#alternate-exchange makes it possible to route unroutable messages to another exchange.
amqp gem support for it is documented in the {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}.
RabbitMQ 2.6 will introduce a new feature callled dead letter queue where unroutable messages will be put instead of dropping them.
amqp gem provides a way to handle returned messages with the {AMQP::Exchange#on_return} method:
exchange.on_return do |basic_return, metadata, payload|
puts "#{payload} was returned! reply_code = #{basic_return.reply_code}, reply_text = #{basic_return.reply_text}"
end
{file:docs/Exchanges.textile Working With Exchanges} documentation guide provides more information on the subject, including full code examples.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/Troubleshooting.textile 0000644 0000041 0000041 00000016033 13321132064 020223 0 ustar www-data www-data # @title Ruby AMQP gem: Troubleshooting and debugging AMQP applications
h1. Troubleshooting and debugging AMQP applications
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide describes tools and strategies that help in troubleshooting and debugging applications that use AMQP in general and "Ruby AMQP gem":http://github.com/ruby-amqp/amqp
in particular.
h2. Covered versions
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.8.0 and later.
h2. First steps
Whenever something doesn't work, check the following things before asking on the mailing list:
* AMQP broker log.
* List of users in a particular vhost you are trying to connect.
* Network connectivity. We know, it's obvious, yet even experienced developers and devops engineers struggle with network access misconfigurations every once in a while.
* If EventMachine is started in a separate thread, make sure that isn't dead. If it is, this usually means there was an exception that caused it to terminate,
or environment it is running in "mixes threads and fork(2) system call":http://bit.ly/fork-and-threads.
h2. Inspecting AMQP broker log file
In this section we will cover typical problems that can be tracked down by reading AMQP broker log. We will use RabbitMQ as an example, however, different AMQP brokers
often log most of the same issues.
RabbitMQ logs abrupt TCP connection failures, timeouts, protocol version mismatches and so on.
If you are running RabbitMQ, log locations for various operating systems and distributions is documented in the "RabbitMQ installation guide":http://www.rabbitmq.com/install.html
On Mac OS X, RabbitMQ installed via Homebrew logs to $HOMEBREW_HOME/var/log/rabbitmq/rabbit@$HOSTNAME.log. For example, if you have Homebrew installed at /usr/local and
your hostname is giove, log will be at /usr/local/var/log/rabbitmq/rabbit@giove.log.
Here is what authentication failure looks like in RabbitMQ log:
=ERROR REPORT==== 17-May-2011::17:37:58 ===
exception on TCP connection <0.4770.0> from 127.0.0.1:46551
{channel0_error,starting,
{amqp_error,access_refused,
"AMQPLAIN login refused: user 'pipeline_agent' - invalid credentials",
'connection.start_ok'}}
This means that connection attempt with username pipeline_agent failed because credentials were invalid. If you are seeing this message, make sure username,
password *and vhost* are correct.
The following entry:
=ERROR REPORT==== 17-May-2011::17:26:28 ===
exception on TCP connection <0.4201.62> from 10.8.0.30:57990
{bad_header,<<65,77,81,80,0,0,9,1>>}
Means that client supports AMQP 0.9.1 but broker doesn't (RabbitMQ versions pre-2.0 only support AMQP 0.8, for example). If you are using amqp gem 0.8 or later
and seeing this entry in your broker log, you are connecting to AMQP broker that is too old to support this AMQP version. In case of RabbitMQ, make sure you run
version 2.0 or later.
TBD
h2. Handling channel-level exceptions
A broad range of problems result in AMQP channel exceptions: an indication by the broker that there was an issue application needs to be aware of.
Channel-level exceptions are typically not fatal and can be recovered from. Some examples are:
* Exchange is re-declared with attributes different from the original declaration. For example, a non-durable exchange is being re-declared as durable.
* Queue is re-declared with attributes different from the original declaration. For example, an autodeletable queue is being re-declared as non-autodeletable.
* Queue is bound to an exchange that does not exist.
and so on. When troubleshooting AMQP applications, it is recommended that you detect and handle channel-level exceptions on all channels your
application may use. For that, use {AMQP::Channel#on_error} method as demonstrated below:
events_channel.on_error do |ch, channel_close|
puts "Channel-level exception on the events channel: #{channel_close.reply_text}"
end
commands_channel.on_error do |ch, channel_close|
puts "Channel-level exception on the commands channel: #{channel_close.reply_text}"
end
Defining channel-level exception handlers will reveal many issues that it might take more time to detect using other troubleshooting techniques.
h2. Testing network connection with AMQP broker using Telnet
One simple way to check network connection between a particular network node and a node running AMQP broker is to use `telnet`:
telnet [host or ip] 5672then enter any random string of text and hit Enter. AMQP broker should immediately close down the connection. Here is an example session:
telnet localhost 5672 Connected to localhost. Escape character is '^]'. adjasd AMQP Connection closed by foreign host.If instead Telnet exits after printing
telnet: connect to address [host or ip]: Connection refused telnet: Unable to connect to remote hostthen connection from the machine you are running Telnet tests on and AMQP broker fails. This can be due to many different reasons, but it is a good idea to check these two things first: * Firewall configuration for port 5672 * DNS setup (if hostname is used) h2. Broker Startup Issues h3. Missing erlang-os-mon on Debian and Ubuntu The following error on RabbitMQ startup on Debian or Ubuntu
ERROR: failed to load application os_mon: {"no such file or directory","os_mon.app"}
suggests that *erlang-os-mon* package is not installed.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/08Migration.textile 0000644 0000041 0000041 00000026324 13321132064 017141 0 ustar www-data www-data # @title Ruby AMQP gem: Migrating to version 0.8.0 and later
h1. amqp gem 0.8 migration guide
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide explains how (and why) applications that use amqp gem versions 0.6.x and 0.7.x should migrate
to 0.8.0.RCs and future versions. It also outlines deprecated features, when and why they will be removed.
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Covered versions
This guide covers "Ruby amqp gem":http://github.com/ruby-amqp/amqp v0.6.x and v0.7.x.
h2. Brief history of amqp gem
amqp gem was born in early July 2008. Since then, it has seen 7 releases, used by all kinds of companies, have been
key component to systems that process more than 100 gigabytes of data per day and, despite not being a "1.0 library",
proven to be very usable.
For the most part of 2008 and 2009 the gem did not receive much of attention. In 2010, some other AMQP clients (for example,
RabbitMQ Java & .NET clients) switched to AMQP 0.9.1 specification. By the fall of 2010 it became clear that amqp gem needs
a new team of maintainers, deep refactoring, new solid test suite and extensive documentation guides. Long story short,
by December 2010, the "new maintainers group":https://github.com/ruby-amqp was formed and now maintains number of other libraries
in the Ruby AMQP ecosystem, most notably the "Bunny gem":github.com/ruby-amqp/bunny.
Over first few months of 2011, amqp gem undergone a number of maintenance releases (known as 0.7.x series), a ground up rewrite of
the test suite, major documentation update as well as a major refactoring. It was upgraded to AMQP 0.9.1 specification and
based on a new, much more efficient AMQP serialization library, "amq-protocol":https://github.com/ruby-amqp/amq-protocol.
As part of this transition, amqp gem maintainers team pursued a number of goals:
* Implement AMQP 0.9.1 spec plus all the RabbitMQ extensions
* End "300 of 1 patch forks" situation that had gotten especially bad in early 2010
* Produce a solid end-to-end test suite that imitates real asynchronous applications
* Resolve many API usability issues: inconsistent behavior, weird class and method naming, heavy reliance on implicit connection objects and so on
* Produce documentation guides that other AMQP client users will be envious of
* Define a stable public API
* Improve memory efficiency and performance
* Resolve many limitations of the original API and implementation design
* Drop all the "experimental-stuff-that-was-never-finished"
In the end, several projects like "evented-spec":https://github.com/ruby-amqp/evented-spec were born and became part of the amqp gem
family of libraries. "300 one patch forks" hell is a thing of the past. Even initial pure Ruby implementation of the new serialization
library is "several times as memory efficient":http://www.rabbitmq.com/blog/2011/03/01/ruby-amqp-benchmarks/. "Documentation guides":http://bit.ly/amqp-gem-docs
response if very positive. amqp gem 0.8.0.RCs implement all but 1 RabbitMQ extensions as well as AMQP 0.9.1 spec features original gem
never implemented fully. Many large users of the gem upgraded to 0.8.0.RCs. The future for amqp gem and its spin-offs is bright.
h2. amqp gem 0.8.0 backwards compatibility policy
amqp gem 0.8.0 is *not 100% backwards compatible*. That said, for most applications, upgrade path will be easy.
Over 90% of the deprecated API classes & methods are still in place and will only be dropped in the 0.9.0 release.
Plenty of incompatibilities between 0.6.x and 0.7.x series were ironed out in 0.8.0.RCs. Finally, this guide will explain
what has changed, why, why it is the right thing to do and how you should adapt your application code.
h2. Why developers should upgrade to 0.8.0
* Great documentation
* amqp gem 0.8.0 RCs and later versions are actively maintained
* Largest library users either already switched to amqp gem 0.8.0 RCs or are switching in the near future (2011)
* AMQP 0.9.1 spec implementation: many other popular clients, for example the RabbitMQ Java client, has dropped support for AMQP 0.8 specification
* RabbitMQ extensions
* Several AMQP brokers, including RabbitMQ (by far the most popular from the maintainers team observation), are planning to drop AMQP 0.8 spec support
* Applications that do not use depricated API features & behaviors will have very seamless upgrade path to the amqp gem 0.9.0 and beyond
h2. AMQP protocol version change
amqp gem before 0.8.0 (0.6.x, 0.7.x) series implemented (most of) AMQP 0.8 specification. amqp gem 0.8.0 implements AMQP 0.9.1 and thus
*requires RabbitMQ version 2.0 or later*. See {file:docs/RabbitMQVersions.textile RabbitMQ versions} for more information about
RabbitMQ versions support and how obtain up-todate packages for your operating system.
amqp gem 0.8.0 and later versions implement AMQP 0.9.1 and thus *requires RabbitMQ version 2.0 or later*
h2. Follow established AMQP terminology
h3. require "mq" is deprecated
Instead of the following:
require "amqp"
require "mq"
or
require "mq"
switch to
require "amqp"
mq.rb will be removed before 1.0 release.
h3. MQ class is deprecated
Please use {AMQP::Channel} instead. MQ class & its methods are implemented in amqp gem 0.8.x for backwards compatibility,
but only for transition period (that will end after 0.9 release).
Why is it deprecated? Because it was a poor name choice all along. Both mq.rb and MQ class step away from AMQP terminology and
make 8 out of 10 engineers think it has something to do with AMQP queues (in fact, MQ should have been called Channel all along).
No other AMQP client library we know of invents its own terminology when it comes to AMQP entities, and amqp gem shouldn't, too.
MQ class and class methods that use implicit connection (MQ.queue and so on) will be removed before 1.0 release.
h2. MQ class is now AMQP::Channel
MQ class was renamed to AMQP::Channel to follow established {file:docs/AMQP091ModelExplained.textile AMQP 0.9.1 terminology}. Implicit per-thread
channels are deprecated so code like this:
# amqp gem 0.6.x code style: connection is implicit, channel is implicit. Error handling & recovery are thus
# not possible.
# Deprecated, do not use.
MQ.queue("search.indexing")
# same for exchanges. Deprecated, do not use.
MQ.direct("services.imaging")
MQ.fanout("services.broadcast")
MQ.topic("services.weather_updates")
# connection object lets you define error handlers
connection = AMQP.connect(:vhost => "myapp/production", :host => "192.168.0.18")
# channek object lets you define error handlers and use multiple channels per application,
# for example, one per thread
channel = AMQP::Channel.new(connection)
# AMQP::Channel#queue has unchanged otherwise
channel.queue("search.indexing")
# exchanges examples
channel.direct("services.imaging")
channel.fanout("services.broadcast")
channel.topic("services.weather_updates")
MQ.queue, MQ.direct, MQ.fanout and MQ.topic methods will be removed before 1.0 release. Use
instance methods on AMQP::Channel (AMQP::Channel#queue, AMQP::Channel#direct, AMQP::Channel#default_exchange
and so on) instead.
h2. MQ::Queue is now AMQP::Queue
MQ::Queue is now AMQP::Queue. All the methods from 0.6.x series are still available.
MQ::Queue alias will be removed before 1.0 release. Please switch to AMQP::Queue.
h2. MQ::Exchange is now AMQP::Exchange
MQ::Exchange is now AMQP::Exchange. All the methods from 0.6.x series are still available.
MQ::Exchange alias will be removed before 1.0 release. Please switch to AMQP::Exchange.
h2. MQ::Header is now AMQP::Header
MQ::Header is now AMQP::Header. If you code has any type checks or case matches on
MQ::Header, it needs to change to AMQP::Header.
h2. AMQP.error is deprecated
Catch-all solutions for error handling are very difficult to use. Automatic recovery and fine-grained
event handling is also not possible. amqp gem 0.8.0.RC14 and later includes fine-grained {file:docs/ErrorHandling.textile Error Handling and Recovery API}
that is both significantly easier to use in real-world cases but also makes automatic recovery mode possible.
AMQP.error method will be removed before 1.0 release.
h2. MQ::RPC is deprecated
It was an experiment that was never finished. Some API design choices are very opinionated as well. amqp gem should not ship with
half-baked poorly designed or overly opinionated solutions.
MQ::RPC class will be removed before 1.0 release.
h2. MQ::Logger is removed
It was an experiment that was never finished. Some API design choices are very opinionated as well. amqp gem should not ship with
half-baked poorly designed or overly opinionated solutions.
MQ::Logger class was removed in the the amqp gem 0.8.0 development cycle.
h2. AMQP::Buffer is removed
AMQP::Buffer was AMQP 0.8 protocol implementation detail. With the new "amq-protocol gem":http://rubygems.org/gems/amq-protocol, it is no
longer relevant.
AMQP::Buffer class was removed in the the amqp gem 0.8.0 development cycle.
h2. AMQP::Frame is removed
AMQP::Frame was AMQP 0.8 protocol implementation detail. With the new "amq-protocol gem":http://rubygems.org/gems/amq-protocol, it is no
longer relevant.
AMQP::Frame class was removed in the the amqp gem 0.8.0 development cycle.
h2. AMQP::Server is removed
AMQP::Server was (an unfinished) toy implementation of AMQP 0.8 broker in Ruby on top of EventMachine. We believe that Ruby is not an optimal
choice for AMQP broker implementations. We also never heard from anyone using it.
AMQP::Server class was removed in the the amqp gem 0.8.0 development cycle.
h2. AMQP::Protocol::* classes are removed
AMQP::Protocol::* classes were AMQP 0.8 protocol implementation detail. With the new "amq-protocol gem":http://rubygems.org/gems/amq-protocol, they are no
longer relevant.
AMQP::Protocol::* classes were removed in the the amqp gem 0.8.0 development cycle.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/GettingStarted.textile 0000644 0000041 0000041 00000057223 13321132064 017772 0 ustar www-data www-data # @title Ruby amqp gem: Getting Started with AMQP and Ruby
h1. Getting started with the Ruby amqp gem
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide is a quick tutorial that helps you to get started with v0.9.1 of the AMQP specification in general and the "Ruby amqp gem":http://github.com/ruby-amqp/amqp in particular.
It should take about 20 minutes to read and study the provided code examples. This guide covers:
* Installing RabbitMQ, a mature popular server implementation of the AMQP protocol.
* Installing the amqp gem via "Rubygems":http://rubygems.org and "Bundler":http://gembundler.com.
* Running a "Hello, world" messaging example that is a simple demonstration of 1:1 communication.
* Creating a "Twitter-like" publish/subscribe example with 1 publisher and 4 subscribers that demonstrates 1:n communication.
* Creating a topic routing example with 2 publishers and 8 subscribers showcasing n:m communication when subscribers only receive messages that they are interested in.
* Learning how the amqp gem can be integrated with Ruby objects in a way that makes unit testing easy.
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Which versions of the amqp gem does this guide cover?
This guide covers v0.8.0 and later of the "Ruby amqp gem":https://github.com/ruby-amqp/amqp.
h2. Installing RabbitMQ
The "RabbitMQ site":http://rabbitmq.com has a good "installation guide":http://www.rabbitmq.com/install.html that addresses many operating systems.
On Mac OS X, the fastest way to install RabbitMQ is with "Homebrew":http://mxcl.github.com/homebrew/:
brew install rabbitmq
then run it:
rabbitmq-server
On Debian and Ubuntu, you can either "download the RabbitMQ .deb package":http://www.rabbitmq.com/server.html and install it with
"dpkg":http://www.debian.org/doc/FAQ/ch-pkgtools.en.html or make use of the "apt repository":http://www.rabbitmq.com/debian.html#apt that the RabbitMQ team provides.
For RPM-based distributions like RedHat or CentOS, the RabbitMQ team provides an "RPM package":http://www.rabbitmq.com/install.html#rpm.
The RabbitMQ package that ships with recent Ubuntu versions (for example, 10.10) is outdated and *will not work with v0.8.0 and later of the amqp gem* (we need at least RabbitMQ v2.0 for use with this guide).
h2. Installing the Ruby amqp gem
h3. Make sure that you have Ruby and "Rubygems":http://docs.rubygems.org/read/chapter/3 installed
This guide assumes that you have installed one of the following supported Ruby implementations:
* Ruby v1.8.7 [except for 1.8.7-p248 and -p249 that have "a bug that severely affects amqp gem":http://bit.ly/iONBmH]
* Ruby v1.9.2
* Ruby v1.9.3
* JRuby (we recommend v1.6)
* Rubinius v1.2 or higher
* Ruby Enterprise Edition
h3. You can use Rubygems to install the amqp gem
h4. On Microsoft Windows 7:
gem install eventmachine --pre gem install amqph4. On other OSes or JRuby:
gem install amqph3. You can also use Bundler to install the gem
source :rubygems
gem "amqp", "~> 0.9.0" # optionally: :git => "git://github.com/ruby-amqp/amqp.git", :branch => "0.9.x-stable"
h3. Verifying your installation
Let us verify your installation with this quick irb session:
irb -rubygems
:001 > require "amqp"
=> true
:002 > AMQP::VERSION
=> "0.9.0"
h2. "Hello, world" example
Let us begin with the classic "Hello, world" example. First, here is the code:
(if the example above isn't displayed, see this "gist":https://gist.github.com/998690)
This example demonstrates a very common communication scenario: *application A* wants to publish a message that will end up in
a queue that *application B* listens on. In this case, the queue name is "amqpgem.examples.hello". Let us go through the code
step by step:
require "rubygems"
require "amqp"
is the simplest way to load the amqp gem if you have installed it with RubyGems. The following piece of code
EventMachine.run do
# ...
end
runs what is called the "EventMachine":http://rubyeventmachine.com reactor. We will not go into what the term 'reactor' means here, but suffice it to say
that the amqp gem is asynchronous and is based on an asynchronous network I/O library called _EventMachine_.
The next line
connection = AMQP.connect(:host => '127.0.0.1')
connects to the server running on localhost, with the default port (5672), username (guest), password (guest) and virtual host ('/').
The next line
channel = AMQP::Channel.new(connection)
opens a new _channel_. AMQP is a multi-channeled protocol that uses channels to multiplex a TCP connection.
Channels are opened on a connection, therefore the AMQP::Channel constructor takes a connection object as a parameter.
This line
queue = channel.queue("amqpgem.examples.helloworld", :auto_delete => true)
declares a _queue_ on the channel that we have just opened. Consumer applications get messages from queues.
We declared this queue with the "auto-delete" parameter. Basically, this means that the queue will be deleted when there are no more processes
consuming messages from it.
The next line
exchange = channel.direct("")
instantiates an _exchange_. Exchanges receive messages that are sent by producers. Exchanges route messages to queues
according to rules called _bindings_. In this particular example, there are no explicitly defined bindings.
The exchange that we defined is known as the _default exchange_ and it has implied bindings to all queues. Before we get
into that, let us see how we define a handler for incoming messages
queue.subscribe do |payload|
puts "Received a message: #{payload}. Disconnecting..."
connection.close { EventMachine.stop }
end
{AMQP::Queue#subscribe} takes a block that will be called every time a message arrives. {AMQP::Session#close} closes the
AMQP connection and runs a callback that stops the EventMachine reactor.
Finally, we publish our message
exchange.publish "Hello, world!", :routing_key => queue.name
Routing key is one of the _message attributes_. The default exchange will route the message to a queue that has the same name
as the message's routing key. This is how our message ends up in the "amqpgem.examples.helloworld" queue.
This first example can be modified to use the method chaining technique:
This diagram demonstrates the "Hello, world" example data flow:
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/001_hello_world_example_routing.png!
For the sake of simplicity, both the message producer (App I) and the consumer (App II) are running in the same Ruby process.
Now let us move on to a little bit more sophisticated example.
h2. Blabbr: one-to-many publish/subscribe (pubsub) example
The previous example demonstrated how a connection to a broker is made and how to do 1:1 communication
using the default exchange. Now let us take a look at another common scenario: broadcast, or multiple consumers
and one producer.
A very well-known broadcast example is Twitter: every time a person tweets, followers receive a notification.
Blabbr, our imaginary information network, models this scenario: every network member has a separate
queue and publishes blabs to a separate exchange. 3 Blabbr members, Joe, Aaron and Bob, follow the official NBA
account on Blabbr to get updates about what is happening in the world of basketball. Here is the code:
The first line has a few differences from the "Hello, world" example above:
* We use {AMQP.start} instead of {AMQP.connect}
* Instead of return values, we pass a block to the connection method and it yields a connection
object back as soon as the connection is established.
* Instead of passing connection parameters as a hash, we use a URI string.
{AMQP.start} is just a convenient way to do
EventMachine.run do
AMQP.connect(options) do |connection|
# ...
end
end
The {AMQP.start} call blocks the current thread which means that its use is limited to scripts and small command
line applications. Blabbr is just that.
{AMQP.connect}, when invoked with a block, will yield a connection object as soon as the AMQP connection
is open. Finally, connection parameters may be supplied as a Hash or as a connection string. The {AMQP.connect}
method documentation contains all of the details.
In this example, opening a channel is no different to opening a channel in the previous example,
however, the exchange is declared differently
exchange = channel.fanout("nba.scores")
The exchange that we declare above using {AMQP::Channel#fanout} is a _fanout exchange_. A fanout exchange delivers messages to all of the queues that
are bound to it: exactly what we want in the case of Blabbr.
This piece of code
channel.queue("joe", :auto_delete => true).bind(exchange).subscribe do |payload|
puts "#{payload} => joe"
end
is similar to the subscription code that we used for message delivery previously, but what does that {AMQP::Queue#bind}
method do? It sets up a binding between the queue and the exchange that you pass to it. We need to do this
to make sure that our fanout exchange routes messages to the queues of any subscribed followers.
exchange.publish("BOS 101, NYK 89").publish("ORL 85, ALT 88")
demonstrates {AMQP::Exchange#publish} call chaining. Blabbr members use a fanout exchange
for publishing, so there is no need to specify a message routing key because every queue that is bound to the exchange will get its own copy of all messages, regardless of the queue name and routing key used.
A diagram for Blabbr looks like this:
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/002_blabbr_example_routing.png!
Next we use EventMachine's {http://eventmachine.rubyforge.org/EventMachine.html#M000466 add_timer} method to
run a piece of code in 2 seconds from now:
EventMachine.add_timer(2) do
exchange.delete
connection.close { EventMachine.stop }
end
The code that we want to run deletes the exchange that we declared earlier using {AMQP::Exchange#delete} and closes the AMQP
connection with {AMQP::Session#close}. Finally, we stop the EventMachine event loop and exit.
Blabbr is pretty unlikely to secure hundreds of millions of dollars in funding, but it does a pretty good job of
demonstrating how one can use AMQP fanout exchanges to do broadcasting.
h2. Weathr: many-to-many topic routing example
So far, we have seen point-to-point communication and broadcasting. Those two communication styles are possible with many protocols, for instance,
HTTP handles these scenarios just fine. You may ask "what differentiates AMQP?" Well, next we are going to introduce you to _topic
exchanges_ and routing with patterns, one of the features that makes AMQP very powerful.
Our third example involves weather condition updates. What makes it different from the previous two examples is that
not all of the consumers are interested in all of the messages. People who live in Portland usually do not care about
the weather in Hong Kong (unless they are visiting soon). They are much more interested in
weather conditions around Portland, possibly all of Oregon and sometimes a few neighbouring states.
Our example features multiple consumer applications monitoring updates for different regions. Some are
interested in updates for a specific city, others for a specific state and so on, all the way up to continents.
Updates may overlap so that an update for San Diego, CA appears as an update for California, but also should show up
on the North America updates list.
Here is the code:
The first line that is different from the Blabbr example is
exchange = channel.topic("pub/sub", :auto_delete => true)
We use a topic exchange here. Topic exchanges are used for "multicast":http://en.wikipedia.org/wiki/Multicast messaging
where consumers indicate which topics they are interested in (think of it as subscribing to a feed for an individual tag
in your favourite blog as opposed to the full feed). Routing with a topic exchange is done by specifying a _routing pattern_ on binding, for example:
channel.queue("americas.south").bind(exchange, :routing_key => "americas.south.#").subscribe do |headers, payload|
puts "An update for South America: #{payload}, routing key is #{headers.routing_key}"
end
Here we bind a queue with the name of "americas.south" to the topic exchange declared earlier using the {AMQP::Queue#bind} method.
This means that only messages with a routing key matching "americas.south.#" will be routed to that queue. A routing pattern consists of several words
separated by dots, in a similar way to URI path segments joined by slashes. Here are a few examples:
* asia.southeast.thailand.bangkok
* sports.basketball
* usa.nasdaq.aapl
* tasks.search.indexing.accounts
Now let us take a look at a few routing keys that match the "americas.south.#" pattern:
* americas.south
* americas.south.*brazil*
* americas.south.*brazil.saopaolo*
* americas.south.*chile.santiago*
In other words, the "#" part of the pattern matches 0 or more words.
For a pattern like "americas.south.*", some matching routing keys would be:
* americas.south.*brazil*
* americas.south.*chile*
* americas.south.*peru*
but not
* americas.south
* americas.south.chile.santiago
so "*" only matches a single word. The AMQP v0.9.1 specification says that topic segments (words) may contain the letters A-Z and a-z
and digits 0-9.
A (very simplistic) diagram to demonstrate topic exchange in action:
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/003_weathr_example_routing.png!
One more thing that is different from previous examples is that the block we pass to {AMQP::Queue#subscribe} now takes two arguments:
a _header_ and a _body_ (often called the _payload_). Long story short, the header parameter lets you access metadata associated with the message. Some
examples of message metadata attributes are:
* message content type
* message content encoding
* message priority
* message expiration time
* message identifier
* reply to (specifies which message this is a reply to)
* application id (identifier of the application that produced the message)
and so on.
As the following binding demonstrates, "#" and "*" can also appear at the beginning of routing patterns:
channel.queue("us.tx.austin").bind(exchange, :routing_key => "#.tx.austin").subscribe do |headers, payload|
puts "An update for Austin, TX: #{payload}, routing key is #{headers.routing_key}"
end
For this example the publishing of messages is no different from that of previous examples. If we were to run the program,
a message published with a routing key of "americas.north.us.ca.berkeley" would be routed to 2 queues: "us.california" and the
_server-named queue_ that we declared by passing a blank string as the name:
channel.queue("", :exclusive => true) do |queue|
queue.bind(exchange, :routing_key => "americas.north.#").subscribe do |headers, payload|
puts "An update for North America: #{payload}, routing key is #{headers.routing_key}"
end
end
The name of the server-named queue is generated by the broker and sent back to the client with a queue declaration confirmation.
Because the queue name is not known before the reply arrives, we pass {AMQP::Channel#queue} a callback and it yields us back
a queue object once confirmation has arrived.
h3. Avoid race conditions
A word of warning: you may find examples on the Web of {AMQP::Channel#queue} usage that do not use
callbacks. We *recommend that you use a callback for server-named queues*, otherwise your code may be subject
to "race conditions":http://en.wikipedia.org/wiki/Race_condition. Even though the amqp gem tries to be reasonably smart and protect you from most common problems
(for example, binding operations will be delayed until after queue name is received from the broker), there
is no way it can do so for every case. The primary reason for supporting {AMQP::Channel#queue} usage without a callback for server-named queues is
backwards compatibility with earlier versions.
h2. Integration with objects
Since Ruby is a genuine object-oriented language, it is important to demonstrate how the Ruby amqp gem can be integrated
into rich object-oriented code.
The {AMQP::Queue#subscribe} callback does not have to be a block. It can be any Ruby object that responds to `call`.
A common technique is to combine {http://rubydoc.info/stdlib/core/1.8.7/Object:method Object#method} and {http://rubydoc.info/stdlib/core/1.8.7/Method:to_proc Method#to_proc}
and use object methods as message handlers.
An example to demonstrate this technique:
class Consumer
#
# API
#
def handle_message(metadata, payload)
puts "Received a message: #{payload}, content_type = #{metadata.content_type}"
end # handle_message(metadata, payload)
end
class Worker
#
# API
#
def initialize(channel, queue_name = AMQ::Protocol::EMPTY_STRING, consumer = Consumer.new)
@queue_name = queue_name
@channel = channel
@channel.on_error(&method(:handle_channel_exception))
@consumer = consumer
end # initialize
def start
@queue = @channel.queue(@queue_name, :exclusive => true)
@queue.subscribe(&@consumer.method(:handle_message))
end # start
#
# Implementation
#
def handle_channel_exception(channel, channel_close)
puts "Oops... a channel-level exception: code = #{channel_close.reply_code}, message = #{channel_close.reply_text}"
end # handle_channel_exception(channel, channel_close)
end
The "Hello, world" example can be ported to use this technique:
The most important line in this example is
@queue.subscribe(&@consumer.method(:handle_message))
Ampersand (&) preceding an object is equivalent to calling the #to_proc method on it. We obtain a Consumer#handle_message method reference
with
@consumer.method(:handle_message)
and then the ampersand calls #to_proc on it. {AMQP::Queue#subscribe} then will be using this Proc instance to handle incoming messages.
Note that the `Consumer` class above can be easily tested in isolation, without spinning up any AMQP connections.
Here is one example using "RSpec":http://relishapp.com/rspec
require "ostruct"
require "json"
# RSpec example
describe Consumer do
describe "when a new message arrives" do
subject { described_class.new }
let(:metadata) do
o = OpenStruct.new
o.content_type = "application/json"
o
end
let(:payload) { JSON.encode({ :command => "reload_config" }) }
it "does some useful work" do
# check preconditions here if necessary
subject.handle_message(metadata, payload)
# add your code expectations here
end
end
end
h2. Wrapping up
This is the end of the tutorial. Congratulations! You have learned quite a bit about both AMQP v0.9.1 and the amqp gem. This is only
the tip of the iceberg. AMQP has many more features built into the protocol:
* Reliable delivery of messages
* Message confirmations (a way to tell broker that a message was or was not processed successfully)
* Message redelivery when consumer applications fail or crash
* Load balancing of messages between multiple consumers
* Message metadata attributes
and so on. Other guides explain these features in depth, as well as use cases for them. To stay up to date with amqp gem
development, "follow @rubyamqp on Twitter":http://twitter.com/rubyamqp and "join our mailing list":http://groups.google.com/group/ruby-amqp.
h2. What to read next
Documentation is organized as a number of {file:docs/DocumentationGuidesIndex.textile documentation guides}, covering all kinds of
topics from {file:docs/Exchanges.textile use cases for various exchange types} to {file:docs/ErrorHandling.textile error handling} and
{file:docs/VendorSpecificExchanges.textile Broker-specific AMQP 0.9.1 extensions}.
We recommend that you read the following guides next, if possible, in this order:
* {file:docs/AMQP091ModelExplained.textile AMQP 0.9.1 Model Explained}. A simple 2 page long introduction to the AMQP Model concepts and features. Understanding the AMQP Model
will make a lot of other documentation, both for the Ruby amqp gem and RabbitMQ itself, easier to follow. With this guide, you don't have to waste hours of time reading the whole specification.
* {file:docs/ConnectingToTheBroker.textile Connection to the broker}. This guide explains how to connect to an AMQP broker and how to integrate the amqp gem into standalone and Web applications.
* {file:docs/Queues.textile Working With Queues}. This guide focuses on features that consumer applications use heavily.
* {file:docs/Exchanges.textile Working With Exchanges}. This guide focuses on features that producer applications use heavily.
* {file:docs/PatternsAndUseCases.textile Patterns & Use Cases}. This guide focuses implementation of "common messaging patterns":http://www.eaipatterns.com/ using AMQP Model features as building blocks.
* {file:docs/ErrorHandling.textile Error Handling & Recovery}. This guide explains how to handle protocol errors, network failures and other things that may go wrong in real world projects.
If you are migrating your application from earlier versions of the amqp gem (0.6.x and 0.7.x), to 0.8.x and later, there is the
{file:docs/08Migration.textile amqp gem 0.8 migration guide}.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
key to making the documentation better.
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/AMQP091ModelExplained.textile 0000644 0000041 0000041 00000050620 13321132064 020637 0 ustar www-data www-data # @title Ruby amqp gem: AMQP 0.9.1 Model Explained
h1. AMQP 0.9.1 Model Explained
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide explains the AMQP 0.9.1 Model used by RabbitMQ. Understanding the AMQP Model will make a lot of other documentation, both for the Ruby amqp gem and
RabbitMQ itself, easier to follow. This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h3. What this guide covers
This guide covers:
* High-level overview of the AMQP 0.9.1 Model
* Key differences of the AMQP model from some other messaging models
* What exchanges are
* What queues are
* What bindings are
* How AMQP protocol is structured and what AMQP methods are
* What attributes AMQP 0.9.1 messages have
* What message acknowledgements are
* What negative message acknowledgements are
* and a lot of other things
h2. High-level overview of AMQP 0.9.1 and the AMQP Model
h3. What is AMQP
AMQP (Advanced Message Queuing Protocol) is a networking protocol which enables conforming client applications to communicate with conforming
messaging middleware brokers.
h3. Why AMQP was created
Messaging solutions have been around since the 1970s with a view to solving the problem of integrating incompatible products from diverse vendors. Without the use of messaging middleware, the integration of heterogenous systems has proved to be very expensive and complex. However, messaging solutions, such as IBM Websphere MQ and Tibco Enterprise Message Service, are also very costly and tend to be exclusively employed by large companies (who can afford them), especially those in the financial services industry.
There is also a problem with interoperability between messaging solutions. Vendors have created their own proprietary messaging protocols which do not interoperate with others, therefore resulting in 'vendor lock-in'.
AMQP has multiple design goals but two of the most important are:
* To produce an open standard for messaging middleware
* To enable interoperability between various technologies and platforms
There is a lot of software running on many operating systems built with multiple programming languages running on
various hardware architectures and virtual machines. AMQP not only makes it possible for these disparate systems to
communicate with one another, but also enables different products that implement AMQP to exchange information.
h3. Brokers and their role
Messaging brokers receive messages from _producers_ (applications that publish them) and route them to _consumers_ (applications
that process them).
If you imagine the human body, then brokers would be equivalent to centers of the nervous system and applications would be more like limbs.
h3. AMQP 0.9.1 Model in brief
The AMQP 0.9.1 Model has the following view of the world: messages are published by producers to _exchanges_, often compared to post offices or mailboxes. Exchanges then
distribute message copies to _queues_ using rules called _bindings_. Then AMQP brokers either push messages to _consumers_ subscribed to queues, or consumers
fetch/pull messages from queues on demand.
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/001_hello_world_example_routing.png!
When publishing a message, producers may specify various _message attributes_ (message metadata). Some of this metadata may be used by the broker, however, the rest of it
is completely opaque to the broker and is only used by applications that receive the message.
Networks are unreliable and applications may fail to process messages, therefore the AMQP Model has a notion of
_message acknowledgements_: when a message is pushed down to a consumer, the consumer _notifies the broker_, either automatically or as soon as the application
developer chooses to do so. When message acknowledgements are in use, a broker will only completely remove a message from a queue when it receives a notification for that message (or group of messages).
In certain situations, for example, when a message cannot be routed, messages may be _returned_ to producers, dropped, or, if the broker implements an extension,
placed into a so-called "dead letter queue". Producers choose how to handle situations like this by publishing messages using certain parameters.
Queues, exchanges and bindings are commonly referred to as _AMQP entities_.
h3. AMQP is a Programmable Protocol
AMQP 0.9.1 is a programmable protocol in the sense that AMQP entities and routing schemes are defined by applications themselves, not a broker administrator. Accordingly, provision is made for protocol
operations that declare queues and exchanges, define bindings between them, subscribe to queues and so on.
This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice, definition conflicts
are rare and often indicate misconfigurations. This can be very useful as it is a good thing if misconfigurations are caught early.
Applications declare the AMQP entities that they need, define necessary routing schemes and may choose to delete AMQP entities when they are no longer used.
h2. AMQP Exchanges and Exchange Types
_Exchanges_ are AMQP entities where messages are sent. Exchanges then take a message and route it into one or more (or no) queues. The routing algorithm used
depends on _exchange type_ and rules called _bindings_. AMQP 0.9.1 brokers typically provide 4 exchange types out of the box:
* Direct exchange (typically used for for 1-to-1 communication or unicasting)
* Fanout exchange (1-to-n communication or broadcasting)
* Topic exchange (1-to-n or n-to-m communication, multicasting)
* Headers exchange (message metadata-based routing)
but it is possible to extend AMQP 0.9.1 brokers with custom exchange types, for example:
* x-random exchange (randomly chooses a queue to route incoming messages to)
* x-recent-history (a fanout exchange that also keeps N recent messages in memory)
* regular expressions based variations of headers exchange
and so on.
Besides the type, exchanges have a number of attributes, most important of which are:
* Name
* Can be durable (information about them is persisted to disk and thus survives broker restarts) or non-durable (information is only kept in RAM)
* Can have metadata associated with them on declaration
h2. AMQP Queues
Queues in the AMQP Model are very similar to queues in other message and "task queueing" systems: they store messages that are consumed by
applications. Like AMQP exchanges, an AMQP queue has a name and a durability property but also
* Can be exclusive (used by only one connection)
* Can be automatically deleted when last consumer unsubscribes
* Can have metadata associated with them on declaration (some brokers use it to implement features like message TTL)
h2. AMQP Bindings
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q,
Q has to _be bound_ to E. Bindings may have an optional _routing key_ attribute used by some exchange types. The purpose of the routing key is to
selectively match only specific (matching) messages published to an exchange to the bound queue. In other words, the routing key acts like a filter.
To draw an analogy:
* Queue is like your destination in New York city
* Exchange is like JFK airport
* Bindings are routes from JFK to your destination. There can be none or more than one way to reach it
Having this layer of indirection enables routing scenarios that are impossible of very hard to implement using publishing directly to queues and
also eliminates certain amount of duplicated work application developers have to do.
If AMQP message cannot be routed to any queue (for example, because there are no bindings for the exchange it was published to), it is either
dropped or returned to the publisher, depending on message attributes the publisher has set.
h2. AMQP Message Consumers
Storing messages in queues is useless unless applications can _consume_ them. In the AMQP 0.9.1 Model, there are two ways for applications to do this:
* Have messages pushed to them ("push API")
* Fetch messages as needed ("pull API")
With the "push API", applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they _register a consumer_
or, simply put, _subscribe to a queue_. It is possible to have more than one consumer per queue or to register an _exclusive consumer_ (excludes all other consumers from
the queue while it is consuming).
Each consumer (subscription) has an identifier called a _consumer tag_. It can be used to unsubscribe from messages. Consumer tags are just strings.
h2. AMQP Message Attributes and Payload
Messages in the AMQP Model have _attributes_. Some attributes are so common that the AMQP v0.9.1 specification defines them and application developers do not have to think
about the exact attribute name. Some examples are
* Content type
* Content encoding
* Routing key
* Delivery mode (persistent or not)
* Message priority
* Message publishing timestamp
* Expiration period
* Producer application id
Some attributes are used by AMQP brokers, but most are open to interpretation by applications that receive them. Some attributes are optional and known
as _headers_. They are similar to X-Headers in HTTP. Message attributes are set when a message is published.
AMQP messages also have a _payload_ (the data that they carry). Brokers treat this data as opaque (it is neither modified nor used by them). It is possible for messages to contain only attributes
and no payload. It is common to use serialization formats like JSON, Thrift, Protocol Buffers and MessagePack
to serialize structured data
in order to publish it as AMQP message payload.
h2. AMQP Message Acknowledgements
Since networks are unreliable and applications fail, it is often necessary to have some kind of "processing acknowledgement". Sometimes it is only
necessary to acknowledge the fact that a message has been received. Sometimes acknowledgements mean that a message was validated and processed by a consumer,
for example, verified as having mandatory data and persisted to a data store or indexed.
This situation is very common, so AMQP 0.9.1 has a built-in feature called _message acknowledgements_ (sometimes referred to as _acks_) that consumers
use to confirm message delivery and/or processing. If an application crashes (AMQP broker notices this when connection is closed), if an acknowledgement for a
message was expected but not received by the AMQP broker, the message is re-queued (and possibly immediately delivered to another consumer, if any exists).
Having acknowledgements built into the protocol helps developers to build more robust software.
h2. AMQP 0.9.1 Methods
AMQP 0.9.1 is structured as a number of _methods_. Methods are operations (like HTTP methods) and have nothing in common with methods in object-oriented programming
languages. AMQP methods are grouped into _classes_. Classes are just logical groupings of AMQP methods. The "AMQP 0.9.1 reference":http://www.rabbitmq.com/amqp-0-9-1-reference.html can be found on
the RabbitMQ website.
Let us take a look at the _exchange.*_ class, a group of methods related to operations on exchanges. It includes the following operations:
* exchange.declare
* exchange.declare-ok
* exchange.delete
* exchange.delete-ok
(note that the RabbitMQ site reference also includes RabbitMQ-specific extensions to the exchange.* class that we will not discuss in this guide).
The operations above form logical pairs: *exchange.declare* and *exchange.declare-ok*,
*exchange.delete* and *exchange.delete-ok*. These operations are "requests" (sent by clients) and "responses" (sent by
brokers in response to the aforementioned "requests").
As an example, the client asks the broker to declare a new exchange using the *exchange.declare* method:
!https://img.skitch.com/20110720-c4qjdhmdrih9bn56npqnic4die.jpg!
As shown on the diagram above, *exchange.declare* carries several _parameters_. They enable the client to specify exchange name,
type, durability flag and so on.
If the operation succeeds, the broker responds with the *exchange.declare-ok* method:
!https://img.skitch.com/20110720-m4ptjbnex2sa52g6wdwj3e9ahm.jpg!
*exchange.declare-ok* does not carry any parameters except for the channel number (channels will be described later in this guide).
The sequence of events is very similar for another method pair, *queue.declare* and *queue.declare-ok*:
!https://img.skitch.com/20110720-tmxswrie71ubb5m5nh8n17idhk.jpg!
!https://img.skitch.com/20110720-g67urxg75c71qtwhwsjs684323.jpg!
Not all AMQP methods have counterparts. Some (*basic.publish* being the most widely used one) do not have corresponding "response" methods
and some others (*basic.get*, for example) have more than one possible "response".
h2. AMQP Connections
AMQP connections are typically long-lived. AMQP is an application level protocol that uses TCP for reliable delivery. AMQP connections use
authentication and can be protected using TLS (SSL). When an application no longer needs to be connected to an AMQP broker, it should gracefully
close the AMQP connection instead of abruptly closing the underlying TCP connection.
h2. AMQP Channels
Some applications need multiple connections to an AMQP broker. However, it is undesirable to keep many TCP connections open at the same time
because doing so consumes system resources and makes it more difficult to configure firewalls. AMQP 0.9.1 connections are multiplexed with
_channels_ that can be thought of as "lightweight connections that share a single TCP connection".
For applications that use multiple threads/processes/etc for processing, it is very common to open a new channel per thread (process, etc.)
and *not share* channels between them.
Communication on a particular channel is completely separate from communication on another channel, therefore every AMQP method also carries a channel number that clients
use to figure out which channel the method is for (and thus, which event handler needs to be invoked, for example).
h2. AMQP Virtual Hosts (vhosts)
To make it possible for a single broker to host multiple isolated "environments" (groups of users, exchanges, queues and so on), AMQP includes the concept
of _virtual hosts_ (vhosts). They are similar to virtual hosts used by many popular Web servers and provide completely isolated environments
in which AMQP entities live. AMQP clients specify what vhosts they want to use during AMQP connection negotiation.
An AMQP 0.9.1 vhost name can be any non-blank string. Some most common use cases for vhosts are
* To separate AMQP entities used by different groups of applications
* To separate multiple installations/environments (e.g. production, staging) of one or more applications
* To implement a multi-tenant environment
h2. AMQP is Extensible
AMQP 0.9.1 has several extension points:
* Custom exchange types let developers implement routing schemes that exchange types provided out-of-the-box do not cover well, for example, geodata-based routing.
* Declaration of exchanges and queues can include additional attributes that the broker can use. For example, per-queue message TTL in RabbitMQ is implemented this way.
* Broker-specific extensions to the protocol. See, for example, "extensions RabbitMQ implements":http://www.rabbitmq.com/extensions.html.
* New AMQP 0.9.1 method classes can be introduced.
* Brokers can be extended with additional plugins, for example, RabbitMQ management frontend and HTTP API are implemented as a plugin.
These features make the AMQP 0.9.1 Model even more flexible and applicable to a very broad range of problems.
h2. Key differences from some other messaging models
One key difference to understand about the AMQP 0.9.1 Model is that *messages are not sent to queues. They are sent to exchanges that route them to
queues according to rules called "bindings"*. This means that routing is primarily handled by AMQP brokers and not applications themselves.
TBD
h2. AMQP 0.9.1 clients ecosystem
h3. Overview
There are many AMQP 0.9.1 clients for many popular programming languages and platforms. Some of them follow AMQP terminology closely
and only provide implementation of AMQP methods. Some others have additional features, convenience methods and abstractions. Some of the
clients are asynchronous (non-blocking), some are synchronous (blocking), some support both models. Some clients support vendor-specific
extensions (for example, RabbitMQ-specific extensions).
Because one of the main AMQP goals is interoperability, it is a good idea for developers to understand protocol operations
and not limit themselves to terminology of a particular client library. This way communicating with developers using different libraries will
be significantly easier.
h2. Wrapping up
This is the end of the AMQP 0.9.1 Model tutorial. Congratulations! Armed with this knowledge, you will find it easier to follow the rest of
the amqp gem documentation as well as the rabbitmq.com documentation and the "rabbitmq-discuss mailing list":https://groups.google.com/forum/#!forum/rabbitmq-users
To stay up to date with amqp gem development, "follow @rubyamqp on Twitter":http://twitter.com/rubyamqp and "join our mailing list":http://groups.google.com/group/ruby-amqp.
h2. What to read next
Documentation is organized as a number of {file:docs/DocumentationGuidesIndex.textile documentation guides}, covering all kinds of
topics from {file:docs/Exchanges.textile use cases for various exchange types} to {file:docs/ErrorHandling.textile error handling} and
{file:docs/VendorSpecificExchanges.textile Broker-specific AMQP 0.9.1 extensions}.
We recommend that you read the following guides next, if possible, in this order:
* {file:docs/ConnectingToTheBroker.textile Connection to the broker}. This guide explains how to connect to an AMQP broker and how to integrate the amqp gem into standalone and Web applications.
* {file:docs/Queues.textile Working With Queues}. This guide focuses on features that consumer applications use heavily.
* {file:docs/Exchanges.textile Working With Exchanges}. This guide focuses on features that producer applications use heavily.
* {file:docs/PatternsAndUseCases.textile Patterns & Use Cases}. This guide focuses implementation of "common messaging patterns":http://www.eaipatterns.com/ using AMQP Model features as building blocks.
* {file:docs/ErrorHandling.textile Error Handling & Recovery}. This guide explains how to handle protocol errors, network failures and other things that may go wrong in real world projects.
If you are migrating your application from earlier versions of the amqp gem (0.6.x and 0.7.x), to 0.8.x and later, there is the
{file:docs/08Migration.textile amqp gem 0.8 migration guide}.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
key to making the documentation better.
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michaelklishin@me.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/Exchanges.textile 0000644 0000041 0000041 00000126724 13321132064 016752 0 ustar www-data www-data # @title Ruby AMQP gem: Working with exchanges
h1. Working with exchanges
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide covers the use of exchanges according to the AMQP v0.9.1 specification including message publishing,
common usage scenarios and how to accomplish typical operations using the Ruby amqp gem. This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Which versions of the amqp gem does this guide cover?
This guide covers v0.8.0 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
h2. Exchanges in AMQP v0.9.1 - overview
h3. What are AMQP exchanges?
An _exchange_ accepts messages from a producer application and routes them to message queues. They can be thought of
as the "mailboxes" of the AMQP world. Unlike some other messaging middleware products and protocols, in AMQP,
messages are *not* published directly to queues. Messages are published to exchanges that route them to queue(s)
using pre-arranged criteria called _bindings_.
There are multiple exchange types in the AMQP v0.9.1 specification, each with its own routing semantics. Custom
exchange types can be created to deal with sophisticated routing scenarios (e.g. routing based on geolocation data
or edge cases) or just for convenience.
h3. Concept of bindings
A _binding_ is an association between a queue and an exchange. A queue must be bound to at least one exchange in
order to receive messages from publishers. Learn more about bindings in the {file:docs/Bindings.textile Bindings guide}.
h3. Exchange attributes
Exchanges have several attributes associated with them:
* Name
* Type (direct, fanout, topic, headers or some custom type)
* Durability
* Whether the exchange is auto-deleted when no longer used
* Other metadata (sometimes known as _X-arguments_)
h2. Exchange types
There are 4 built-in exchange types in AMQP v0.9.1:
* Direct
* Fanout
* Topic
* Headers
As stated previously, each exchange type has its own routing semantics and new exchange types can be added by
extending brokers with plugins. Custom exchange types begin with "x-", much like custom HTTP headers, e.g.
"x-recent-history exchange":https://github.com/videlalvaro/rabbitmq-recent-history-exchange or
"x-random exchange":https://github.com/jbrisbin/random-exchange.
h2. Message attributes
Before we start looking at various exchange types and their routing semantics, we need to introduce message
attributes. Every AMQP message has a number of *attributes*. Some attributes are important and used very often,
others are rarely used. AMQP message attributes are metadata and are similar in purpose to HTTP request and
response headers.
Every AMQP v0.9.1 message has an attribute called *routing key*. The routing key is an "address" that the exchange
may use to decide how to route the message . This is similar to, but more generic than, a URL in HTTP. Most exchange
types use the routing key to implement routing logic, but some ignore it and use other criteria (e.g. message
content).
h2. Fanout exchanges
h3. How fanout exchanges route messages
A fanout exchange routes messages to all of the queues that are bound to it and the routing key is ignored. If N
queues are bound to a fanout exchange, when a new message is published to that exchange a *copy of the message* is
delivered to all N queues. Fanout exchanges are ideal for the
"broadcast routing":http://en.wikipedia.org/wiki/Broadcasting_%28computing%29 of messages.
Graphically this can be represented as:
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/004_fanout_exchange.png!
h3. Declaring a fanout exchange
There are two ways to declare a fanout exchange:
* By instantiating an {AMQP::Exchange} and specifying the type as ":fanout"
* By using the {AMQP::Channel#fanout} method
Here are two examples to demonstrate:
exchange = AMQP::Exchange.new(channel, :fanout, "nodes.metadata")
exchange = channel.fanout("nodes.metadata")
Both methods asynchronously declare a queue. Because the declaration necessitates a network round-trip, publishing
operations on {AMQP::Exchange} instances are delayed until the broker reply (`exchange.declare-ok`) is received.
Also, both methods let you pass a block to run a piece of code when the broker responds with an `exchange.declare-ok`
(meaning that the exchange has been successfully declared).
channel.fanout("nodes.metadata") do |exchange|
# exchange is declared and ready to be used.
end
h3. Fanout routing example
To demonstrate fanout routing behavior we can declare 10 server-named exclusive queues, bind them all to one
fanout exchange and then publish a message to the exchange:
exchange = channel.topic("amqpgem.examples.routing.fanout_routing", :auto_delete => true)
10.times do
q = channel.queue("", :exclusive => true, :auto_delete => true).bind(exchange)
q.subscribe do |payload|
puts "Queue #{q.name} received #{payload}"
end
end
# Publish some test data after all queues are declared and bound
EventMachine.add_timer(1.2) { exchange.publish "Hello, fanout exchanges world!" }
When run, this example produces the following output:
Queue amq.gen-0p/BjxGNCue42RcJhpUrdg== received Hello, fanout exchanges world! Queue amq.gen-3GXULvZuYh1KsOD83yvlNg== received Hello, fanout exchanges world! Queue amq.gen-4EcyydTfoZzXjNSSLsh09Q== received Hello, fanout exchanges world! Queue amq.gen-B1isyTpR5svB6ClQ2TQEBQ== received Hello, fanout exchanges world! Queue amq.gen-FwLLioB7Mk4LGA4yJ1Mo7A== received Hello, fanout exchanges world! Queue amq.gen-OtBQokiA/DmNkB5bPzaRig== received Hello, fanout exchanges world! Queue amq.gen-RYHQUrj3yihb0DRF7KVpRg== received Hello, fanout exchanges world! Queue amq.gen-SZJ40mGwbhdcbOGeHMhUkg== received Hello, fanout exchanges world! Queue amq.gen-sDeVZg9Vx1knq+n9EMi8tA== received Hello, fanout exchanges world! Queue amq.gen-uWOuVaosW4bWAHqKG6pZVw== received Hello, fanout exchanges world!Each of the queues bound to the exchange receives a *copy* of the message. Full example: h3. Fanout use cases Because a fanout exchange delivers a copy of a message to every queue bound to it, its use cases are quite similar: * Massively multiplayer online (MMO) games can use it for leaderboard updates or other global events * Sport news sites can use fanout exchanges for distributing score updates to mobile clients in near real-time * Distributed systems can broadcast various state and configuration updates * Group chats can distribute messages between participants using a fanout exchange (although AMQP does not have a built-in concept of presence, so "XMPP":http://xmpp.org may be a better choice) h3. Pre-declared fanout exchanges AMQP v0.9.1 brokers must implement a fanout exchange type and pre-declare one instance with the name of "amq.fanout". Applications can rely on that exchange always being available to them. Each vhost has a separate instance of that exchange, it is *not shared across vhosts* for obvious reasons. h2. Direct exchanges h3. How direct exchanges route messages A direct exchange delivers messages to queues based on a *message routing key*, an attribute that every AMQP v0.9.1 message contains. Here is how it works: * A queue binds to the exchange with a routing key K * When a new message with routing key R arrives at the direct exchange, the exchange routes it to the queue if K = R A direct exchange is ideal for the "unicast routing":http://en.wikipedia.org/wiki/Unicast of messages (although they can be used for "multicast routing":http://en.wikipedia.org/wiki/Multicast as well). Here is a graphical representation: !https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/005_direct_exchange.png! h3. Declaring a direct exchange There are two ways to declare a direct exchange: * By instantiating a {AMQP::Exchange} and specifying its type as ":direct" * By using the {AMQP::Channel#direct} method Here are two examples to demonstrate:
exchange = AMQP::Exchange.new(channel, :direct, "nodes.metadata")
exchange = channel.direct("nodes.metadata")
Both methods asynchronously declare an exchange named "nodes.metadata". Because the declaration necessitates a network round trip, publishing
operations on {AMQP::Exchange} instances are delayed until a broker reply (`exchange.declare-ok`) is received.
Also, both methods let you pass a block to run a piece of code when the broker responds with `exchange.declare-ok`
(meaning that the exchange has been successfully declared).
channel.direct("pages.content.extraction") do |exchange|
# exchange is declared and ready to be used.
end
h3. Direct routing example
Since direct exchanges use the *message routing key* for routing, message producers need to specify it:
exchange.publish("Hello, direct exchanges world!", :routing_key => "amqpgem.examples.queues.shared")
The routing key will then be compared for equality with routing keys on bindings, and consumers that subscribed with
the same routing key each get a copy of the message:
Full example:
h3. Direct exchanges and load balancing of messages
Direct exchanges are often used to distribute tasks between multiple workers (instances of the same application) in
a round robin manner. When doing so, it is important to understand that, in AMQP v0.9.1, *messages are load balanced
between consumers and not between queues*.
The Ruby amqp gem historically has a limitation that only one consumer (message handler) is allowed per {AMQP::Queue}
instance, however, this limitation will be addressed in the future. With the amqp gem v0.8.x, if you want to load
balance messages between multiple consumers in the same application/OS process, then you need to use a separate
channel for each of the consumers.
The {file:docs/Queues.textile Working With Queues} and {file:docs/PatternsAndUseCases.textile Patterns and Use Cases}
guides provide more information on this subject.
h3. Pre-declared direct exchanges
AMQP v0.9.1 brokers must implement a direct exchange type and pre-declare two instances:
* *amq.direct*
* *""* exchange known as *default exchange* (unnamed, referred to as an empty string by many clients including amqp Ruby gem)
Applications can rely on those exchanges always being available to them. Each vhost has separate instances of those
exchanges, they are *not shared across vhosts* for obvious reasons.
h3. Default exchange
The default exchange is a direct exchange with no name (the amqp gem refers to it using an empty string) pre-declared
by the broker. It has one special property that makes it very useful for simple applications, namely that *every
queue is automatically bound to it with a routing key which is the same as the queue name*.
For example, when you declare a queue with the name of "search.indexing.online", the AMQP broker will bind it to the
default exchange using "search.indexing.online" as the routing key. Therefore a message published to the default
exchange with routing key = "search.indexing.online" will be routed to the queue "search.indexing.online". In other
words, the default exchange makes it *seem like it is possible to deliver messages directly to queues*, even though
that is not technically what is happening.
The amqp gem offers two ways of obtaining a reference to the default exchange:
* Using the {AMQP::Channel#default_exchange} method
* Using the {AMQP::Channel#direct} method with an empty string as the exchange name
{AMQP::Exchange#initialize} can also be used, but requires more coding effort and it offers no benefits over instance
methods on {AMQP::Channel} in this particular case.
Some examples of usage:
exchange = AMQP::Exchange.new(channel, :direct, "")
exchange = channel.default_exchange
exchange = channel.direct("")
The default exchange is used by the "Hello, World" example:
Additionally, the routing example above can be rewritten to use the default exchange:
h3. Direct exchange use cases
Direct exchanges can be used in a wide variety of cases:
* Direct (near real-time) messages to individual players in an MMO game
* Delivering notifications to specific geographic locations (for example, points of sale)
* Distributing tasks between multiple instances of the same application all having the same function, for example, image processors
* Passing data between workflow steps, each having an identifier (also consider using headers exchange)
* Delivering notifications to individual software services in the network
h2. Topic exchanges
h3. How topic exchanges route messages
Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern
that was used to bind a queue to an exchange. The topic exchange type is often used to implement various
"publish/subscribe pattern":http://en.wikipedia.org/wiki/Publish/subscribe variations.
Topic exchanges are commonly used for the "multicast routing":http://en.wikipedia.org/wiki/Multicast of messages.
!http://upload.wikimedia.org/wikipedia/commons/thumb/3/30/Multicast.svg/500px-Multicast.svg.png!
Topic exchanges can be used for "broadcast routing":http://en.wikipedia.org/wiki/Broadcasting_%28computing%29, but
fanout exchanges are usually more efficient for this use case.
h3. Topic exchange routing example
Two classic examples of topic-based routing are stock price updates and location-specific data (for instance, weather
broadcasts). Consumers indicate which topics they are interested in (think of it like subscribing to a feed for an
individual tag of your favourite blog as opposed to the full feed). The routing is enabled by specifying a _routing
pattern_ to the {AMQP::Queue#bind} method, for example:
channel.queue("americas.south").bind(exchange, :routing_key => "americas.south.#").subscribe do |headers, payload|
puts "An update for South America: #{payload}, routing key is #{headers.routing_key}"
end
In the example above we bind a queue with the name of "americas.south" to the topic exchange declared earlier using
the {AMQP::Queue#bind} method. This means that only messages with a routing key matching "americas.south.#" will be
routed to the "americas.south" queue.
A routing pattern consists of several words separated by dots, in a similar way to URI path segments being joined by
slash. A few of examples:
* asia.southeast.thailand.bangkok
* sports.basketball
* usa.nasdaq.aapl
* tasks.search.indexing.accounts
The following routing keys match the "americas.south.#" pattern:
* americas.south
* americas.south.*brazil*
* americas.south.*brazil.saopaolo*
* americas.south.*chile.santiago*
In other words, the "#" part of the pattern matches 0 or more words.
For the pattern "americas.south.*", some matching routing keys are:
* americas.south.*brazil*
* americas.south.*chile*
* americas.south.*peru*
but not
* americas.south
* americas.south.chile.santiago
As you can see, the "*" part of the pattern matches 1 word only.
Full example:
h3. Topic exchange use cases
Topic exchanges have a very broad set of use cases. Whenever a problem involves multiple consumers/applications that
selectively choose which type of messages they want to receive, the use of topic exchanges should be considered.
To name a few examples:
* Distributing data relevant to specific geographic location, for example, points of sale
* Background task processing done by multiple workers, each capable of handling specific set of tasks
* Stocks price updates (and updates on other kinds of financial data)
* News updates that involve categorization or tagging (for example, only for a particular sport or team)
* Orchestration of services of different kinds in the cloud
* Distributed architecture/OS-specific software builds or packaging where each builder can handle only one architecture or OS
h2. Declaring/Instantiating exchanges
With the Ruby amqp gem, exchanges can be declared in two ways:
* By using the {AMQP::Exchange#initialize} method that takes an optional callback
* By using a number of convenience methods on {AMQP::Channel} instances:
** {AMQP::Channel#direct}
** {AMQP::Channel#default_exchange}
** {AMQP::Channel#topic}
** {AMQP::Channel#fanout}
** {AMQP::Channel#headers}
The previous sections on specific exchange types (direct, fanout, headers, etc.) provide plenty of examples of how
these methods can be used.
h2. Publishing messages
To publish a message to an AMQP exchange, use {AMQP::Exchange#publish}:
exchange.publish("Some payload")
{AMQP::Exchange#publish} can accept any object that responds to the `to_s` method, not just string instances:
The message payload is completely opaque to the library and is not modified in any way.
h3. Data serialization
You are encouraged to take care of data serialization before publishing (i.e. by using JSON, Thrift, Protocol Buffers
or some other serialization library). Note that because AMQP is a binary protocol, text formats like JSON largely
lose their advantage of being easy to inspect as data travels across the network, so consider using
"BSON":http://bsonspec.org instead.
A few popular options for data serialization are:
* JSON: "json gem":https://rubygems.org/gems/json (part of standard Ruby library on Ruby 1.9) or "yajl-ruby":https://rubygems.org/gems/yajl-ruby (Ruby bindings to YAJL)
* BSON: "bson gem":https://rubygems.org/gems/bson for JRuby (implemented as a Java extension) and "bson_ext":https://rubygems.org/bson_ext (C extension) for C-based Rubies
* "Message Pack":http://msgpack.org has Ruby bindings but currently does not provide Java implementation for JRuby
* XML: "Nokogiri":https://nokogiri.org is a swiss army knife for XML processing with Ruby, built on top of libxml2
* Protocol Buffers: "beefcake":https://github.com/bmizerany/beefcake
* Thrift: "thrift-client":https://github.com/fauna/thrift_client
h3. Message metadata
AMQP messages have various metadata attributes that can be set when a message is published. Some of the attributes are
well-known and mentioned in the AMQP v0.9.1 specification, others are specific to a particular application.
Well-known attributes are listed here as options that {AMQP::Exchange#publish} takes:
* :routing_key
* :persistent
* :immediate
* :mandatory
* :content_type
* :content_encoding
* :priority
* :message_id
* :correlation_id
* :reply_to
* :type
* :user_id
* :app_id
* :timestamp
* :expiration
All other attributes can be added to a _headers table_ (in Ruby parlance, headers hash) that {AMQP::Exchange#publish}
accepts as the ":headers" argument.
An example to show how message metadata attributes are passed to {AMQP::Exchange#publish}:
exchange.publish(payload, :persistent => true, :type => "reports.done") do
# ...
end
A common expectation of the code above is that it is run after the message "has been sent", or even "has been delivered". Unfortunately, neither of
these expectations can be met by the Ruby amqp gem alone. Message publishing happens in several steps:
* {AMQP::Exchange#publish} takes a message and various metadata attributes
* {AMQP::Exchange#publish} internally calls #to_s on the message argument to get message payload
* Resulting payload is staged for writing
* On the next event loop tick, data is transferred to the OS kernel using one of the underlying system calls ("epoll":http://en.wikipedia.org/wiki/Epoll, "kqueue":http://en.wikipedia.org/wiki/Kqueue and so on)
or NIO channels (in the case of JRuby)
* OS kernel buffers data before sending it
* Network driver may also employ buffering
As you can see, "when data is sent" is a complicated issue and while methods to flush buffers certainly exist on various platforms, doing so in
a cross-platform way that *includes the JVM* (that EventMachine also runs on) is non-trivial. In addition, even flushing buffers does not guarantee that the
data was received by the broker because it might have crashed while data was travelling down the wire. The only way to reliably know whether data was
received by the broker or a peer application is to use message acknowledgements. This is how TCP works and this approach is proven to work at the
enormous scale of the modern Internet. AMQP (the protocol) fully embraces this fact and the amqp gem follows.
Given all of this, you may ask "when does the {AMQP::Exchange#publish} callback fire?" The answer is on the next event loop tick. By then the data is pushed down to
the OS kernel. As far as the Ruby library is concerned, it is reasonably safe behavior.
The AMQP::Exchange#publish callback is fired on the next event loop tick. Data is staged for delivery immediately. Applications MUST NOT assume that
by the time the callback has fired, the data is guaranteed to leave the local machine networking stack, reach the AMQP broker or any peer applications that the
message needs to be routed to.
In cases when you cannot afford to lose a single message, AMQP v0.9.1 applications can use one (or a combination of) the following protocol features:
* Publisher confirms (a RabbitMQ-specific extension to AMQP v0.9.1)
* Publishing messages as immediate and/or mandatory
* Transactions (these introduce noticeable overhead and have a relatively narrow set of use cases)
A more detailed overview of the pros and cons of each option can be found in a "blog post that introduces Publisher Confirms extension":http://bit.ly/rabbitmq-publisher-confirms by the RabbitMQ team.
The next sections of this guide will describe how the features above can be used with the Ruby amqp gem.
h3. Publishing messages as immediate
When publishing messages, it is possible to use the ":immediate" option to publish a message as "immediate". When an immediate message cannot be
delivered to any consumer (meaning that one or more queues to which the message was routed have no active consumers), then the message is returned to the
producer.
An example of {AMQP::Exchange#publish} being used to publish an immediate message:
exchange.publish("Message ##{i}", :immediate => true)
The following code example demonstrates a message that is published as immediate but cannot be immediately consumed (no consumers) and thus is returned
back to the producer:
h3. Publishing messages as mandatory
When publishing messages, it is possible to use the ":mandatory" option to publish a message as "mandatory". When a mandatory message cannot be
*routed* to any queue (for example, there are no bindings or none of the bindings match), the message is returned to the producer.
The following code example demonstrates a message that is published as mandatory but cannot be routed (no bindings) and thus is returned
back to the producer:
h3. Returned messages
When a message is returned, the application that produced it can handle that message in different ways:
* Store it for later redelivery in a persistent store
* Publish it to a different destination
* Log the event and discard the message
Returned messages contain information about the exchange they were published to. For convenience, the amqp gem associates returned message callbacks with {AMQP::Exchange}
instances. To handle returned messages, use {AMQP::Exchange#on_return}:
exchange.on_return do |basic_return, metadata, payload|
puts "#{payload} was returned! reply_code = #{basic_return.reply_code}, reply_text = #{basic_return.reply_text}"
end
A returned message handler has access to AMQP method (basic.return) information, message metadata and payload. The metadata and message body are returned without modifications so that the application can store the message for later redelivery.
h3. Publishing persistent messages
Messages potentially spend some time in the queues to which they were routed before they are consumed. During this period of time, the broker may crash or
experience a restart. To survive it, messages must be persisted to disk. This has a negative effect on performance, especially with network attached
storage like NAS devices and Amazon EBS. AMQP v0.9.1 lets applications trade off performance for durability, or vice versa, on a message-by-message basis.
To publish a persistent message, use the ":persistent" option that {AMQP::Exchange#publish} accepts:
exchange.publish(payload, :persistent => true)
Note that in order to survive a broker crash, both the message and the queue that it was routed to must be persistent/durable.
{file:docs/Durability.textile Durability and Message Persistence} provides more information on the subject.
h3. Publishing In Multi-threaded Environments
When using amqp gem in multi-threaded environments, the rule of thumb is: avoid sharing {AMQP::Channel} instances
across threads.
Starting with 0.8.0.RC14, {AMQP::Exchange#publish} synchronizes data delivery on the channel object associated
with exchange. This protects application developers from the most common problems related to publishing messages on a
shared channel from multiple threads, however, by no means protects from every possible concurrency hazard.
When using amqp gem in multi-threaded environments, the rule of thumb is: avoid sharing {AMQP::Channel} instances
across threads.
h3. Sending one-off messages
The following example publishes a message and *safely* closes the AMQP connection afterwards by passing a block
to {AMQP::Exchange#publish}:
h2. Headers exchanges
Now that message attributes and publishing have been introduced, it is time to take a look at one more core exchange type in AMQP v0.9.1. It is called the
_headers_ exchange type and is quite powerful.
h3. How headers exchanges route messages
h4. An example problem definition
The best way to explain headers-based routing is with an example. Imagine a distributed "continuous integration":http://martinfowler.com/articles/continuousIntegration.html system
that distributes builds across multiple machines with different hardware architectures (x86, IA-64, AMD64, ARM family and so on) and operating systems.
It strives to provide a way for a community to contribute machines to run tests on and a nice build matrix like "the one WebKit uses":http://build.webkit.org/waterfall?category=core.
One key problem such systems face is build distribution. It would be nice if a messaging broker could figure out
which machine has which OS, architecture or combination of the two and route build request messages accordingly.
A headers exchange is designed to help in situations like this by routing on multiple attributes that are more easily expressed as
message metadata attributes (headers) rather than a routing key string.
h4. Routing on multiple message attributes
Headers exchanges route messages based on message header matching. Headers exchanges ignore the routing key attribute. Instead, the attributes used for routing
are taken from the "headers" attribute. When a queue is bound to a headers exchange, the ":arguments" attribute is used to define matching rules:
# when binding to a headers exchange, :arguments parameter is used to specify matching rules
@channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :os => 'linux' })
When matching on one header, a message is considered matching if the value of the header equals the value specified upon binding. Using the example above,
some messages that match would be:
exchange.publish "For linux/IA64", :headers => { :arch => "IA64", :os => 'linux' }
exchange.publish "For linux/x86", :headers => { :arch => "x86", :os => 'linux' }
exchange.publish "For any linux", :headers => { :os => 'linux' }
The following example demonstrates matching on integer values:
# consumer part
@channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :cores => 8 })
# ...
# producer part
exchange.publish "For ocotocore", :headers => { :cores => 8 }
Matching on hashes (in AMQP v0.9.1 parlance - _attribute tables_) is also supported:
# consumer part
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { :package => { :name => 'riak', :version => '0.14.2' } })
# ...
# producer part
exchange.publish "For nodes with Riak 0.14.2", :headers => { :package => { :name => 'riak', :version => '0.14.2' } }
h4. Matching all vs matching one
It is possible to bind a queue to a headers exchange using more than one header for matching. In this case, the broker needs one more piece of
information from the application developer, namely, should it consider messages with any of the headers matching, or all of them? This is what the "x-match"
binding argument is for:
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { 'x-match' => 'all', :arch => "ia64", :os => 'linux' })
In the example above, only messages that have an "arch" header value equal to "ia64" and an "os" header value equal to "linux" will
be considered matching.
channel.queue("", :auto_delete => true).bind(exchange, :arguments => { 'x-match' => 'any', :os => 'macosx', :cores => 8 })
When the "x-match" argument is set to "any", just one matching header value is sufficient. So in the example above, any message with a
"cores" header value equal to 8 will be considered matching.
h4. More examples
TBD
h3. Declaring a headers exchange
There are two ways to declare a headers exchange:
* By instantiating {AMQP::Exchange} and specifying type as ":headers"
* By using the {AMQP::Channel#headers} method
Here are two examples to demonstrate:
exchange = AMQP::Exchange.new(channel, :headers, "builds")
exchange = channel.headers("builds")
Both methods asynchronously declare a queue. Because declaration necessitates a network round trip, publishing operations on {AMQP::Exchange} instances are
delayed until the broker reply (`exchange.declare-ok`) is received.
Both methods let you pass a block to run a piece of code when the broker responds with `exchange.declare-ok` (meaning that the exchange has been successfully declared).
channel.headers("builds") do |exchange|
# exchange is declared and ready to be used.
end
h3. Headers exchange routing example
When there is just one queue bound to a headers exchange, messages are routed to it if one or all headers of the message match those specified
upon binding. Whether it is "any header" or "all of them" depends on the "x-match" header value.
In the case of multiple queues, a headers exchange will deliver a copy of a message to each queue, just like direct exchanges do. Distribution rules between consumers
on a particular queue are the same as for a direct exchange.
Full example:
h3. Headers exchange use cases
Headers exchanges can be looked upon as "direct exchanges on steroids". Because they route based on header values, they can be used as
direct exchanges where the routing key does not have to be a string; it could be an integer or a hash (dictionary) for example.
Some specific use cases:
* Transfer of work between stages in a multi-step workflow ("routing slip pattern":http://eaipatterns.com/RoutingTable.html)
* Distributed build/continuous integration systems can distribute builds based on multiple parameters (OS, CPU architecture, availability
of a particular package).
h3. Pre-declared headers exchanges
AMQP v0.9.1 brokers should (as defined by "IETF RFC 2119":http://www.ietf.org/rfc/rfc2119.txt) implement a headers exchange type
and pre-declare one instance with the name of "amq.match". RabbitMQ also pre-declares one instance with the name of "amq.headers".
Applications can rely on that exchange always being available to them. Each vhost has a separate instance of those exchanges and
they are *not shared across vhosts* for obvious reasons.
h2. Custom exchange types
h3. x-random
The "x-random AMQP exchange type":https://github.com/jbrisbin/random-exchange is a custom exchange type developed as a
RabbitMQ plugin by Jon Brisbin. To quote from the project README:
bq. It is basically a direct exchange, with the exception that, instead of each consumer bound to that exchange with the
same routing key getting a copy of the message, the exchange type randomly selects a queue to route to.
This plugin is licensed under "Mozilla Public License 1.1":http://www.mozilla.org/MPL/MPL-1.1.html, same as RabbitMQ.
h3. x-recent-history
The "x-recent-history AMQP exchange type":https://github.com/videlalvaro/rabbitmq-recent-history-exchange is a customer exchange type
implemented as a RabbitMQ plugin by Alvaro Videla, one of the authors of "RabbitMQ in action":http://bit.ly/rabbitmq.
This plugin is licensed under the "MIT license":https://github.com/videlalvaro/rabbitmq-recent-history-exchange/blob/master/LICENSE.md.
h2. Using the Publisher Confirms extension to AMQP v0.9.1
Please refer to {file:docs/VendorSpecificExtensions.textile Vendor-specific extensions to AMQP 0.9.1 spec}
h3. Message acknowledgements and their relationship to transactions and publisher confirms
Consumer applications (applications that receive and process messages) may occasionally fail to process individual messages, or might just
crash. Additionally, network issues might be experienced. This raises a question - "when should the AMQP broker remove messages from queues?" This
topic is covered in depth in the {file:docs/Queues.textile Working With Queues} guide, including prefetching and examples.
In this guide, we will only mention how message acknowledgements are related to AMQP transactions and the Publisher Confirms extension.
Let us consider a publisher application (P) that communications with a consumer (C) using AMQP v0.9.1. Their communication can be graphically
represented like this:
----- ----- ----- | | S1 | | S2 | | | P | ====> | B | ====> | C | | | | | | | ----- ----- -----We have two network segments, S1 and S2. Each of them may fail. P is concerned with making sure that messages cross S1, while the broker (B) and C are concerned with ensuring that messages cross S2 and are only removed from the queue when they are processed successfully. Message acknowledgements cover reliable delivery over S2 as well as successful processing. For S1, P has to use transactions (a heavyweight solution) or the more lightweight Publisher Confirms, a RabbitMQ-specific extension. h2. Using AMQP transactions TBD h2. Binding queues to exchanges Queues are bound to exchanges using the {AMQP::Queue#bind} method. This topic is described in detail in the {file:docs/Queues.textile Working with queues} documentation guide. h2. Unbinding queues from exchanges Queues are unbound from exchanges using the {AMQP::Queue#unbind} method. This topic is described in detail in the {file:docs/Queues.textile Working with queues} documentation guide. h2. Deleting exchange h3. Explicitly deleting an exchange Exchanges are deleted using the {AMQP::Exchange#delete} method:
exchange.delete
{AMQP::Exchange#delete} takes an optional callback that is run when a `exchange.delete-ok` reply arrives from the broker.
exchange.delete do |delete_ok|
# by now exchange is guaranteed to be deleted
end
h3. Auto-deleted exchanges
Exchanges can be *auto-deleted*. To declare an exchange as auto-deleted, use the ":auto_delete" option on declaration:
exchange = AMQP::Exchange.new(channel, :direct, "nodes.metadata", :auto_delete => true)
exchange = channel.direct("nodes.metadata", :auto_delete => true)
Full example:
TBD: explain when exchange is considered to be "no longer in use"
h2. Objects as message producers.
Since Ruby is a genuine object-oriented language, it is important to demonstrate how the Ruby amqp gem can be integrated
into rich object-oriented code. This part of the guide focuses on exchanges and the problems/solutions concerning producer applications
(applications that primarily generate and publish messages, as opposed to consumers that receive and process them).
Full example:
TBD
h2. Exchange durability vs Message durability
See {file:docs/Durability.textile Durability guide}
h2. Error handling and recovery
See {file:docs/ErrorHandling.textile Error handling and recovery guide}
h2. Vendor-specific extensions related to exchanges
See {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}
h2. What to read next
Documentation is organized as several {file:docs/DocumentationGuidesIndex.textile documentation guides} that cover all kinds of
topics. Guides related to this one are
* {file:docs/Durability.textile Durability and message persistence}
* {file:docs/Bindings.textile Bindings}
* {file:docs/PatternsAndUseCases.textile Patterns and Use Cases}
* {file:docs/Queues.textile Working With Queues}
* {file:docs/ErrorHandling.textile Error handling and recovery}
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
key to making the documentation better.
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/RunningTests.textile 0000644 0000041 0000041 00000007527 13321132064 017507 0 ustar www-data www-data # @title Ruby AMQP gem: Running tests
h1. Running amqp gem test suite
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide is for people who want to contribute to amqp gem development. It has no relevant information for those
who want to build applications using the library.
This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Covered versions
Not applicable.
h2. Canonical repository
Since late 2010, the canonical amqp gem repository is at "github.com/ruby-amqp/amqp":https://github.com/ruby-amqp/amqp and *not tmm1/amqp*.
h2. Initial setup
amqp gem currently uses recent releases of RabbitMQ for automated tests. Broker installation is described in the {file:docs/GettingStarted.textile Getting started} guide
and on "rabbitmq.com":http://www.rabbitmq.com/install.html. Once the broker is running, the only other step is to run
./bin/set_test_suite_realms_up.sh
gem install bundler
bundle install
bundle exec rspec -c ./spec
rake spec:ci
require "amqp/extensions/rabbitmq"
and then define a callback for publisher confirms using {AMQP::Channel#confirm}:
# enable publisher acknowledgements for this channel
channel.confirm_select
# define a callback that will be executed when message is acknowledged
channel.on_ack do |basic_ack|
puts "Received an acknowledgement: delivery_tag = #{basic_ack.delivery_tag}, multiple = #{basic_ack.multiple}"
end
# define a callback that will be executed when message is rejected using basic.nack (a RabbitMQ-specific extension)
channel.on_nack do |basic_nack|
puts "Received a nack: delivery_tag = #{basic_nack.delivery_tag}, multiple = #{basic_nack.multiple}"
end
Note that the same callback is used for all messages published via all exchanges on the given channel.
h3. Clustering
To achieve degree of durability critical applications need, it's necessary but not enough to use durable queues,
exchanges and persistent messages. You need to use a cluster of brokers because otherwise, a single hardware problem
may bring broker down completely.
See {file:docs/Clustering.textile Clustering guide} for in-depth discussion of this topic.
h2. Tell us what you think!
Please take a moment and tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp:
what was unclear? what wasn't covered? maybe you don't like guide style or grammar and spelling are incorrect? Readers feedback is
key to making documentation better.
If mailing list communication is not an option for you for some reason, you can "contact guides author directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/Queues.textile 0000644 0000041 0000041 00000126406 13321132064 016311 0 ustar www-data www-data # @title Ruby AMQP gem: Working with queues
h1. Working with queues
h2. This Documentation Has Moved to rubyamqp.info
amqp gem documentation guides are now hosted on "rubyamqp.info":http://rubyamqp.info.
h2. About this guide
This guide covers everything related to queues in the AMQP v0.9.1 specification, common usage scenarios and how to accomplish typical operations using the
amqp gem. This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available "on Github":https://github.com/ruby-amqp/amqp/tree/master/docs.
h2. Which versions of the amqp gem does this guide cover?
This guide covers v0.8.0 and later of the "Ruby amqp gem":http://github.com/ruby-amqp/amqp.
h2. Queues in AMQP v0.9.1 - overview
h3. What are AMQP queues?
_Queues_ store and forward messages to consumers. They are similar to mailboxes in SMTP.
Messages flow from producing applications to {file:docs/Exchanges.textile exchanges} that route them
to queues and finally queues deliver the messages to consumer applications (or consumer applications fetch messages as needed).
Note that unlike some other messaging protocols/systems, messages are not delivered directly
to queues. They are delivered to exchanges that route messages to queues using rules
known as _bindings_.
AMQP is a programmable protocol, so queues and bindings alike are declared by applications.
h3. Concept of bindings
A _binding_ is an association between a queue and an exchange. Queues must be bound to at least one exchange in order to receive messages from publishers.
Learn more about bindings in the {file:docs/Bindings.textile Bindings guide}.
h3. Queue attributes
Queues have several attributes associated with them:
* Name
* Exclusivity
* Durability
* Whether the queue is auto-deleted when no longer used
* Other metadata (sometimes called _X-arguments_)
These attributes define how queues can be used, what their life-cycle is like and other aspects of queue
behavior.
The amqp gem represents queues as instances of {AMQP::Queue}.
h2. Queue names and declaring queues
Every AMQP queue has a name that identifies it. Queue names often contain several segments separated by a dot ".", in a similar fashion to URI
path segments being separated by a slash "/", although almost any string can represent a segment (with some limitations - see below).
Before a queue can be used, it has to be *declared*. Declaring a queue will cause it to be created if it does not already exist. The declaration will have no effect if the queue does already exist
and its attributes are the *same as those in the declaration*. When the existing queue attributes are not the same as those in the declaration a channel-level exception is raised. This case is explained later in this
guide.
h3. Explicitly named queues
Applications may pick queue names or ask the broker to generate a name for them.
To declare a queue with a particular name, for example, "images.resize", pass it to the Queue class constructor:
queue = AMQP::Queue.new(channel, "images.resize", :auto_delete => true)
Full example:
h3. Server-named queues
To ask an AMQP broker to generate a unique queue name for you, pass an *empty string* as the queue name argument:
AMQP::Queue.new(channel, "", :auto_delete => true) do |queue, declare_ok|
puts "#{queue.name} is ready to go. AMQP method: #{declare_ok.inspect}"
end
Full example:
The amqp gem allows server-named queues to be declared without callbacks:
queue = AMQP::Queue.new(channel, "", :auto_delete => true)
In this case, as soon as the AMQP broker reply (`queue.declare-ok` AMQP method) arrives, the queue object name will
be assigned to the value that the broker generated. Many AMQP operations require a queue name, so before an
{AMQP::Queue} instance receives its name, those operations are delayed. This example demonstrates this:
queue = channel.queue("")
queue.bind("builds").subscribe do |metadata, payload|
# message handling implementation...
end
In this example, binding will be performed as soon as the queue has received its name generated by the broker.
If a particular piece of code relies on the queue name being available immediately a callback should be used.
h3. Reserved queue name prefix
Queue names starting with "amq." are reserved for internal use by the broker. Attempts to declare a queue with a name
that violates this rule will result in a channel-level exception with reply code 403 (ACCESS_REFUSED) and a reply
message similar to this:
ACCESS_REFUSED - queue name 'amq.queue' contains reserved prefix 'amq.*'h3. Queue re-declaration with different attributes When queue declaration attributes are different from those that the queue already has, a channel-level exception with code 406 (PRECONDITION_FAILED) will be raised. The reply text will be similar to this:
PRECONDITION_FAILED - parameters for queue 'amqpgem.examples.channel_exception' in vhost '/' not equivalenth2. Queue life-cycle patterns According to the AMQP v0.9.1 specification, there are two common message queue life-cycle patterns: * Durable message queues that are shared by many consumers and have an independent existence: i.e. they will continue to exist and collect messages whether or not there are consumers to receive them. * Temporary message queues that are private to one consumer and are tied to that consumer. When the consumer disconnects, the message queue is deleted. There are some variations of these, such as shared message queues that are deleted when the last of many consumers disconnects. Let us examine the example of a well-known service like an event collector (event logger). A logger is usually up and running regardless of the existence of services that want to log anything at a particular point in time. Other applications know which queues to use in order to communicate with the logger and can rely on those queues being available and able to survive broker restarts. In this case, explicitly named durable queues are optimal and the coupling that is created between applications is not an issue. Another example of a well-known long-lived service is a distributed metadata/directory/locking server like "Apache Zookeeper":http://zookeeper.apache.org, "Google's Chubby":http://labs.google.com/papers/chubby.html or DNS. Services like this benefit from using well-known, not server-generated, queue names and so do any other applications that use them. A different sort of scenario is in "a cloud setting" when some kind of worker/instance might start and stop at any time so that other applications cannot rely on it being available. In this case, it is possible to use well-known queue names, but a much better solution is to use server-generated, short-lived queues that are bound to topic or fanout exchanges in order to receive relevant messages. Imagine a service that processes an endless stream of events - Twitter is one example. When traffic increases, development operations may start additional application instances in the cloud to handle the load. Those new instances want to subscribe to receive messages to process, but the rest of the system does not know anything about them and cannot rely on them being online or try to address them directly. The new instances process events from a shared stream and are the same as their peers. In a case like this, there is no reason for message consumers not to use queue names generated by the broker. In general, use of explicitly named or server-named queues depends on the messaging pattern that your application needs. {http://www.eaipatterns.com/ Enterprise Integration Patterns} discusses many messaging patterns in depth and the RabbitMQ FAQ also has a section on {http://www.rabbitmq.com/faq.html#scenarios use cases}. h2. Declaring a durable shared queue To declare a durable shared queue, you pass a queue name that is a non-blank string and use the ":durable" option:
queue = AMQP::Queue.new(channel, "images.resize", :durable => true)
Full example:
the same example rewritten to use {AMQP::Channel#queue}:
channel.queue("images.resize", :durable => true) do |queue, declare_ok|
puts "#{queue.name} is ready to go."
end
h2. Declaring a temporary exclusive queue
To declare a server-named, exclusive, auto-deleted queue, pass "" (empty string) as the queue name and
use the ":exclusive" and ":auto_delete" options:
AMQP::Queue.new(channel, "", :auto_delete => true, :exclusive => true) do |queue, declare_ok|
puts "#{queue.name} is ready to go."
end
Full example:
The same example can be rewritten to use {AMQP::Channel#queue}:
channel.queue("", :auto_delete => true, :exclusive => true) do |queue, declare_ok|
puts "#{queue.name} is ready to go."
end
Full example:
Exclusive queues may only be accessed by the current connection and are deleted when that connection closes.
The declaration of an exclusive queue by other connections is not allowed and will result in a channel-level exception
with the code 405 (RESOURCE_LOCKED) and a reply message similar to
RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'amqpgem.examples.queue' in vhost '/'The following example demonstrates this: h2. Binding queues to exchanges In order to receive messages, a queue needs to be bound to at least one exchange. Most of the time binding is explcit (done by applications). To bind a queue to an exchange, use {AMQP::Queue#bind} where the argument passed can be either an {AMQP::Exchange} instance or a string.
queue.bind(exchange) do |bind_ok|
puts "Just bound #{queue.name} to #{exchange.name}"
end
Full example:
The same example using a string without callback:
queue.bind("amq.fanout")
Full example:
h2. Subscribing to receive messages ("push API")
To set up a queue subscription to enable an application to receive messages as they arrive in a queue, one uses the
{AMQP::Queue#subscribe} method. Then when a message arrives, the message header (metadata) and body (payload) are
passed to the handler:
queue.subscribe do |metadata, payload|
puts "Received a message: #{payload.inspect}."
end
Full example:
Subscriptions for message delivery are usually referred to as _consumers_ in the AMQP v0.9.1 specification, client
library documentation and books. Consumers last as long as the channel that they were declared on, or until the
client cancels them (unsubscribes).
Consumers are identified by consumer tags. If you need to obtain the consumer tag of a subscribed
queue then use {AMQP::Queue#consumer_tag}.
h3. Accessing message metadata
The `header` object in the example above provides access to message metadata and delivery information:
* Message content type
* Message content encoding
* Message routing key
* Message delivery mode (persistent or not)
* Consumer tag this delivery is for
* Delivery tag
* Message priority
* Whether or not message is redelivered
* Producer application id
and so on. An example to demonstrate how to access some of those attributes:
# producer
exchange.publish("Hello, world!",
:app_id => "amqpgem.example",
:priority => 8,
:type => "kinda.checkin",
# headers table keys can be anything
:headers => {
:coordinates => {
:latitude => 59.35,
:longitude => 18.066667
},
:participants => 11,
:venue => "Stockholm"
},
:timestamp => Time.now.to_i)
# consumer
queue.subscribe do |metadata, payload|
puts "metadata.routing_key : #{metadata.routing_key}"
puts "metadata.content_type: #{metadata.content_type}"
puts "metadata.priority : #{metadata.priority}"
puts "metadata.headers : #{metadata.headers.inspect}"
puts "metadata.timestamp : #{metadata.timestamp.inspect}"
puts "metadata.type : #{metadata.type}"
puts "metadata.delivery_tag: #{metadata.delivery_tag}"
puts "metadata.redelivered : #{metadata.redelivered?}"
puts "metadata.app_id : #{metadata.app_id}"
puts "metadata.exchange : #{metadata.exchange}"
puts
puts "Received a message: #{payload}."
end
Full example:
h3. Exclusive consumers
Consumers can request exclusive access to the queue (meaning only this consumer can access the queue). This is useful
when you want a long-lived shared queue to be temporarily accessible by just one application (or thread, or process).
If the application employing the exclusive consumer crashes or loses the TCP connection to the broker, then the
channel is closed and the exclusive consumer is cancelled.
To exclusively receive messages from the queue, pass the ":exclusive" option to {AMQP::Queue#subscribe}:
queue.subscribe(:exclusive => true) do |metadata, payload|
# message handling logic...
end
TBD: describe what happens when exclusivity property is violated and how to handle it.
h3. Using multiple consumers per queue
Historically, amqp gem versions before v0.8.0.RC14 (current master branch in the repository) have had a "one consumer
per Queue instance" limitation. Previously, to work around this problem, application developers had to open multiple
channels and work with multiple queue instances on different channels. This is not very convenient and is surprising
for developers familiar with AMQP clients for other languages.
With more and more Ruby implementations dropping the "GIL":http://en.wikipedia.org/wiki/Global_Interpreter_Lock,
load balancing between multiple consumers in the same queue in the same OS process has become more and more common.
In certain cases, even applications that do not need any concurrency benefit from having multiple consumers on the
same queue in the same process.
Starting from amqp gem v0.8.0.RC14, it is possible to add any number of consumers by instantiating {AMQP::Consumer} directly:
# non-exclusive consumer, consumer tag is generated
consumer1 = AMQP::Consumer.new(channel, queue)
# non-exclusive consumer, consumer tag is explicitly given
consumer2 = AMQP::Consumer.new(channel, queue, "#{queue.name}-consumer-#{rand}-#{Time.now}")
# exclusive consumer, consumer tag is generated
consumer3 = AMQP::Consumer.new(channel, queue, nil, true)
Instantiated consumers do not begin consuming messages immediately. This is because in certain cases, it is useful to
add a consumer but make it active at a later time. To consume messages, use the {AMQP::Consumer#consume} method in
combination with {AMQP::Consumer#on_delivery}:
consumer1.consume.on_delivery do |metadata, payload|
@consumer1_mailbox << payload
end
{AMQP::Consumer#on_delivery} takes a block that is used exactly like the block passed to {AMQP::Queue#subscribe}.
In fact, {AMQP::Queue#subscribe} uses {AMQP::Consumer} under the hood, adding a _default consumer_ to the queue.
Default consumers do not have any special properties, they just provide a convenient way for application developers
to register multiple consumers and a means of preserving backwards compatibility. Application developers are
always free to use AMQP::Consumer instances directly, or intermix them with AMQP::Queue#subscribe.
Most of the public API methods on {AMQP::Consumer} return self, so it is possible to use method chaining extensively.
An example from "amqp gem spec suite":https://github.com/ruby-amqp/amqp/tree/master/spec:
consumer1 = AMQP::Consumer.new(@channel, @queue).consume.on_delivery { |metadata, payload| mailbox1 << payload }
consumer2 = AMQP::Consumer.new(@channel, @queue).consume.on_delivery { |metadata, payload| mailbox2 << payload }
To cancel a particular consumer, use {AMQP::Consumer#cancel} method. To cancel a default queue consumer, use {AMQP::Queue#unsubscribe}.
h3. Message acknowledgements
Consumer applications - applications that receive and process messages - may occasionally fail to process individual
messages, or will just crash. There is also the possibility of network issues causing problems. This raises a
question - "When should the AMQP broker remove messages from queues?" The AMQP v0.9.1 specification proposes
two choices:
* After broker sends a message to an application (using either basic.deliver or basic.get-ok methods).
* After the application sends back an acknowledgement (using basic.ack AMQP method).
The former choice is called the *automatic acknowledgement model*, while the latter is called the *explicit
acknowledgement model*. With the explicit model, the application chooses when it is time to send an acknowledgement.
It can be right after receiving a message, or after persisting it to a data store before processing, or after fully
processing the message (for example, successfully fetching a Web page, processing and storing it into some persistent
data store).
!https://github.com/ruby-amqp/amqp/raw/master/docs/diagrams/006_amqp_091_message_acknowledgements.png!
If a consumer dies without sending an acknowledgement, the AMQP broker will redeliver it to another consumer, or, if
none are available at the time, the broker will wait until at least one consumer is registered for the same queue
before attempting redelivery.
The acknowledgement model is chosen when a new consumer is registered for a queue. By default,
{AMQP::Queue#subscribe} will use the *automatic* model. To switch to the *explicit* model, the ":ack" option should
be used:
queue.subscribe(:ack => true) do |metadata, payload|
# message handling logic...
end
To demonstrate how redelivery works, let us have a look at the following code example:
So what is going on here? This example uses 3 AMQP connections to imitate 3 applications, 1 producer and two
consumers. Each AMQP connection opens a single channel:
# open three connections to imitate three apps
connection1 = AMQP.connect
connection2 = AMQP.connect
connection3 = AMQP.connect
channel_exception_handler = Proc.new { |ch, channel_close| EventMachine.stop; raise "channel error: #{channel_close.reply_text}" }
# open two channels
channel1 = AMQP::Channel.new(connection1)
channel1.on_error(&channel_exception_handler)
# ...
channel2 = AMQP::Channel.new(connection2)
channel2.on_error(&channel_exception_handler)
# ...
# app 3 will just publish messages
channel3 = AMQP::Channel.new(connection3)
channel3.on_error(&channel_exception_handler)
The consumers share a queue and the producer publishes messages to the queue periodically using an `amq.direct`
exchange. Both "applications" subscribe to receive messages using the explicit acknowledgement model. The AMQP broker
by default will send each message to the next consumer in sequence (this kind of load balancing is known as
*round-robin*). This means that some messages will be delivered to consumer #1 and some to consumer #2.
exchange = channel3.direct("amq.direct")
# ...
queue1 = channel1.queue("amqpgem.examples.acknowledgements.explicit", :auto_delete => false)
# purge the queue so that we do not get any redeliveries from previous runs
queue1.purge
queue1.bind(exchange).subscribe(:ack => true) do |metadata, payload|
# do some work
sleep(0.2)
# acknowledge some messages, they will be removed from the queue
if rand > 0.5
# FYI: there is a shortcut, metadata.ack
channel1.acknowledge(metadata.delivery_tag, false)
puts "[consumer1] Got message ##{metadata.headers['i']}, ack-ed"
else
# odd messages are not ack-ed and will remain in the queue for redelivery
# when app #1 connection is closed (either properly or due to a crash)
puts "[consumer1] Got message ##{metadata.headers['i']}, SKIPPED"
end
end
queue2 = channel2.queue!("amqpgem.examples.acknowledgements.explicit", :auto_delete => false)
queue2.subscribe(:ack => true) do |metadata, payload|
metadata.ack
# app 2 always acks messages
puts "[consumer2] Received #{payload}, redelivered = #{metadata.redelivered}, ack-ed"
end
To demonstrate message redelivery we make consumer #1 randomly select which messages to acknowledge. After 4 seconds
we disconnect it (to imitate a crash). When that happens, the AMQP broker redelivers unacknowledged messages to
consumer #2 which acknowledges them unconditionally. After 10 seconds, this example closes all outstanding
connections and exits.
An extract of output produced by this example:
=> Subscribing for messages using explicit acknowledgements model [consumer2] Received Message #0, redelivered = false, ack-ed [consumer1] Got message #1, SKIPPED [consumer1] Got message #2, SKIPPED [consumer1] Got message #3, ack-ed [consumer2] Received Message #4, redelivered = false, ack-ed [consumer1] Got message #5, SKIPPED [consumer2] Received Message #6, redelivered = false, ack-ed [consumer2] Received Message #7, redelivered = false, ack-ed [consumer2] Received Message #8, redelivered = false, ack-ed [consumer2] Received Message #9, redelivered = false, ack-ed [consumer2] Received Message #10, redelivered = false, ack-ed [consumer2] Received Message #11, redelivered = false, ack-ed ----- Connection 1 is now closed (we pretend that it has crashed) ----- [consumer2] Received Message #5, redelivered = true, ack-ed [consumer2] Received Message #1, redelivered = true, ack-ed [consumer2] Received Message #2, redelivered = true, ack-ed [consumer2] Received Message #12, redelivered = false, ack-ed [consumer2] Received Message #13, redelivered = false, ack-ed [consumer2] Received Message #14, redelivered = false, ack-ed [consumer2] Received Message #15, redelivered = false, ack-ed [consumer2] Received Message #16, redelivered = false, ack-ed [consumer2] Received Message #17, redelivered = false, ack-ed [consumer2] Received Message #18, redelivered = false, ack-ed [consumer2] Received Message #19, redelivered = false, ack-ed [consumer2] Received Message #20, redelivered = false, ack-ed [consumer2] Received Message #21, redelivered = false, ack-ed [consumer2] Received Message #22, redelivered = false, ack-ed [consumer2] Received Message #23, redelivered = false, ack-ed [consumer2] Received Message #24, redelivered = false, ack-ed [consumer2] Received Message #25, redelivered = false, ack-ed [consumer2] Received Message #26, redelivered = false, ack-ed [consumer2] Received Message #27, redelivered = false, ack-ed [consumer2] Received Message #28, redelivered = false, ack-ed [consumer2] Received Message #29, redelivered = false, ack-ed [consumer2] Received Message #30, redelivered = false, ack-ed [consumer2] Received Message #31, redelivered = false, ack-ed [consumer2] Received Message #32, redelivered = false, ack-ed [consumer2] Received Message #33, redelivered = false, ack-ed [consumer2] Received Message #34, redelivered = false, ack-ed [consumer2] Received Message #35, redelivered = false, ack-edAs we can see, consumer #1 did not acknowledge 3 messages (labelled 1, 2 and 5):
[consumer1] Got message #1, SKIPPED [consumer1] Got message #2, SKIPPED ... [consumer1] Got message #5, SKIPPEDand then, once consumer #1 had "crashed", those messages were immediately redelivered to the consumer #2:
Connection 1 is now closed (we pretend that it has crashed) [consumer2] Received Message #5, redelivered = true, ack-ed [consumer2] Received Message #1, redelivered = true, ack-ed [consumer2] Received Message #2, redelivered = true, ack-edTo acknowledge a message use {AMQP::Channel#acknowledge}:
channel1.acknowledge(metadata.delivery_tag, false)
{AMQP::Channel#acknowledge} takes two arguments: message *delivery tag* and a flag that indicates whether or not we
want to acknowledge multiple messages at once. Delivery tag is simply a channel-specific increasing number that
the server uses to identify deliveries.
When acknowledging multiple messages at once, the delivery tag is treated as "up to and including". For example, if
delivery tag = 5 that would mean "acknowledge messages 1, 2, 3, 4 and 5".
As a shortcut, it is possible to acknowledge messages using the {AMQP::Header#ack} method:
queue2.subscribe(:ack => true) do |metadata, payload|
metadata.ack
end
Acknowledgements are channel-specific. Applications must not receive messages on one channel and acknowledge them on
another.
A message MUST not be acknowledged more than once. Doing so will result in a channel-level exception
(PRECONDITION_FAILED) with an error message like this: «PRECONDITION_FAILED - unknown delivery tag»
h3. Rejecting messages
When a consumer application receives a message, processing of that message may or may not succeed. An application can
indicate to the broker that message processing has failed (or cannot be accomplished at the time) by rejecting a
message. When rejecting a message, an application can ask the broker to discard or requeue it.
To reject a message use the {AMQP::Channel#reject} method:
queue.bind(exchange).subscribe do |metadata, payload|
# reject but do not requeue (simply discard)
channel.reject(metadata.delivery_tag)
end
in the example above, messages are rejected without requeueing (broker will simply discard them). To requeue a
rejected message, use the second argument that {AMQP::Channel#reject} takes:
queue.bind(exchange).subscribe do |metadata, payload|
# reject and requeue
channel.reject(metadata.delivery_tag, true)
end
When there is only one consumer on a queue, make sure you do not create infinite message delivery loops by rejecting
and requeueing a message from the same consumer over and over again.
Another way to reject a message is by using {AMQP::Header#reject}:
queue.bind(exchange).subscribe do |metadata, payload|
# reject but do not requeue (simply discard)
metadata.reject
end
queue.bind(exchange).subscribe do |metadata, payload|
# reject and requeue
metadata.reject(:requeue => true)
end
h3. Negative acknowledgements
Messages are rejected with the `basic.reject` AMQP method. There is one limitation that `basic.reject` has: there
is no way to reject multiple messages, as you can do with acknowledgements. However, if you are using
"RabbitMQ":http://rabbitmq.com, then there is a solution. RabbitMQ provides an AMQP v0.9.1 extension known as
"negative acknowledgements":http://www.rabbitmq.com/extensions.html#negative-acknowledgements (nacks) and
the amqp gem supports this extension. For more information, please refer to the
{file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}.
h3. QoS - Prefetching messages
For cases when multiple consumers share a queue, it is useful to be able to specify how many messages each consumer
can be sent at once before sending the next acknowledgement. This can be used as a simple load balancing technique or
to improve throughput if messages tend to be published in batches. For example, if a producing application sends
messages every minute because of the nature of the work it is doing.
Imagine a website that takes data from social media sources like Twitter or Facebook during the Champions League
final (or the Superbowl), and then calculates how many tweets mention a particular team during the last minute.
The site could be structured as 3 applications:
* A crawler that uses streaming APIs to fetch tweets/statuses, normalizes them and sends them in JSON for processing by other applications ("app A").
* A calculator that detects what team is mentioned in a message, updates statistics and pushes an update to the Web UI once a minute ("app B").
* A Web UI that fans visit to see the stats ("app C").
In this imaginary example, the "tweets per second" rate will vary, but to improve the throughput of the system and
to decrease the maximum number of messages that the AMQP broker has to hold in memory at once, applications can be
designed in such a way that application "app B", the "calculator", receives 5000 messages and then acknowledges them
all at once. The broker will not send message 5001 unless it receives an acknowledgement.
In AMQP parlance this is know as *QoS* or *message prefetching*. Prefetching is configured on a per-channel
(typically) or per-connection (rarely used) basis. To configure prefetching per channel, use
the {AMQP::Channel#prefetch} method. Let us return to the example we used in the "Message acknowledgements" section:
# app #1 will be given up to 3 messages at a time. If it does not
# send an ack after receiving the messages, then the messages will
# be routed to app #2.
channel1.prefetch(3)
# app #2 processes messages one-by-one and has to send an ack after receiving each message
channel2.prefetch(1)
In that example, one consumer prefetches 3 messages and another consumer prefetches just 1. If we take a look at the output that the example produces, we will see that `consumer1` fetched 4 messages and acknowledged 1. After that,
all subsequent messages were delivered to `consumer2`:
[consumer2] Received Message #0, redelivered = false, ack-ed [consumer1] Got message #1, SKIPPED [consumer1] Got message #2, SKIPPED [consumer1] Got message #3, ack-ed [consumer2] Received Message #4, redelivered = false, ack-ed [consumer1] Got message #5, SKIPPED --- by now consumer 1 has received 3 messages it did not acknowledge. With prefetch = 3, AMQP broker will not send it any more messages until consumer 1 sends an ack --- [consumer2] Received Message #6, redelivered = false, ack-ed [consumer2] Received Message #7, redelivered = false, ack-ed [consumer2] Received Message #8, redelivered = false, ack-ed [consumer2] Received Message #9, redelivered = false, ack-ed [consumer2] Received Message #10, redelivered = false, ack-ed [consumer2] Received Message #11, redelivered = false, ack-edThe prefetching setting is ignored for consumers that do not use explicit acknowledgements. h2. How message acknowledgements relate to transactions and Publisher Confirms In cases where you cannot afford to lose a single message, AMQP v0.9.1 applications can use one or a combination of the following protocol features: * Publisher confirms (a RabbitMQ-specific extension to AMQP v0.9.1) * Publishing messages as immediate * Transactions (noticeable overhead) This topic is covered in depth in the {file:docs/Exchanges.textile Working With Exchanges} guide. In this guide, we will only mention how message acknowledgements are related to AMQP transactions and the Publisher Confirms extension. Let us consider a publisher application (P) that communications with a consumer (C) using AMQP v0.9.1. Their communication can be graphically represented like this:
----- ----- ----- | | S1 | | S2 | | | P | ====> | B | ====> | C | | | | | | | ----- ----- -----We have two network segments, S1 and S2. Each of them may fail. P is concerned with making sure that messages cross S1, while broker (B) and C are concerned with ensuring that messages cross S2 and are only removed from the queue when they are processed successfully. Message acknowledgements cover reliable delivery over S2 as well as successful processing. For S1, P has to use transactions (a heavyweight solution) or the more lightweight Publisher Confirms RabbitMQ extension. h2. Fetching messages when needed ("pull API") The AMQP v0.9.1 specification also provides a way for applications to fetch (pull) messages from the queue only when necessary. For that, use {AMQP::Queue#pop}:
queue.pop do |metadata, payload|
if payload
puts "Fetched a message: #{payload.inspect}, content_type: #{metadata.content_type}. Shutting down..."
else
puts "No messages in the queue"
end
end
Full example:
If the queue is empty, then the `payload` argument will be nil, otherwise arguments are identical to those of
the {AMQP::Queue#subscribe} callback.
h2. Unsubscribing from messages
Sometimes it is necessary to unsubscribe from messages without deleting a queue. To do that, use
the {AMQP::Queue#unsubscribe} method:
queue.unsubscribe
By default {AMQP::Queue#unsubscribe} uses the ":noack" option to inform the broker that there is no need to send
a confirmation. In other words, it does not expect you to pass in a callback, because the consumer tag on the queue
instance and the registered callback for messages are cleared immediately.
If an application needs to execute a piece of code after the broker response arrives, {AMQP::Queue#unsubscribe} takes
an optional callback:
queue.unsubscribe do |unbind_ok|
# server response arrived, handle it if necessary...
end
Full example:
In AMQP parlance, unsubscribing from messages is often referred to as "cancelling a consumer". Once a consumer is
cancelled, messages will no longer be delivered to it, however, due to the asynchronous nature of the protocol,
it is possible for "in flight" messages to be received after this call completes.
Fetching messages with {AMQP::Queue#pop} is still possible even after a consumer is cancelled.
h2. Unbinding queues from exchanges
To unbind a queue from an exchange use {AMQP::Queue#unbind}:
queue.unbind(exchange)
Full example:
Note that trying to unbind a queue from an exchange that the queue was never bound to will result in a
channel-level exception.
h2. Querying the number of messages in a queue
It is possible to query the number of messages sitting in the queue by declaring the queue with the ":passive"
attribute set. The response (`queue.declare-ok` AMQP method) will include the number of messages along with other
attributes. However, the amqp gem provides a convenience method, {AMQP::Queue#status}:
queue.status do |number_of_messages, number_of_consumers|
puts
puts "# of messages in the queue #{queue.name} = #{number_of_messages}"
puts
end
Full example:
h2. Querying the number of consumers on a queue
It is possible to query the number of consumers on a queue by declaring the queue with the ":passive" attribute set.
The response (`queue.declare-ok` AMQP method) will include the number of consumers along with other attributes.
However, the amqp gem provides a convenience method, {AMQP::Queue#status}:
queue.status do |number_of_messages, number_of_consumers|
puts
puts "# of consumers on the queue #{queue.name} = #{number_of_consumers}"
puts
end
Full example:
h2. Purging queues
It is possible to purge a queue (remove all of the messages from it) using {AMQP::Queue#purge}:
queue.purge
This method takes an optional callback. However, remember that this operation is performed asynchronously.
To run a piece of code when the AMQP broker confirms that a queue has been purged, use a callback that
{AMQP::Queue#purge} takes:
queue.purge do |_|
puts "Purged #{queue.name}"
end
Full example:
Note that this example purges a newly declared queue with a unique server-generated name. When a queue is declared,
it is empty, so for server-named queues, there is no need to purge them before they are used.
h2. Deleting queues
To delete a queue, use {AMQP::Queue#delete}. When a queue is deleted, all of the messages in it are deleted as well.
queue.delete
This method takes an optional callback. However, remember that this operation is performed asynchronously.
To run a piece of code when the AMQP broker confirms that a queue has been deleted, use a callback that
{AMQP::Queue#delete} takes:
queue.delete do |_|
puts "Deleted #{queue.name}"
end
Full example:
h2. Objects as message consumers and unit testing consumers in isolation
Since Ruby is a genuine object-oriented language, it is important to demonstrate how the Ruby amqp gem can be
integrated into rich object-oriented code. This part of the guide focuses on queues and the problems/solutions
concerning consumer applications (applications that primarily receive and process messages, as opposed to producers
that publish them).
An {AMQP::Queue#subscribe} callback does not have to be a block. It can be any Ruby object that responds to the
`call` method. A common technique is to combine {http://rubydoc.info/stdlib/core/1.8.7/Object:method Object#method}
and {http://rubydoc.info/stdlib/core/1.8.7/Method:to_proc Method#to_proc} and use object methods as message handlers.
An example to demonstrate this technique:
class Consumer
#
# API
#
def initialize(channel, queue_name = AMQ::Protocol::EMPTY_STRING)
@queue_name = queue_name
@channel = channel
# Consumer#handle_channel_exception will handle channel
# exceptions. Keep in mind that you can only register one error handler,
# so the last one registered "wins".
@channel.on_error(&method(:handle_channel_exception))
end # initialize
def start
@queue = @channel.queue(@queue_name, :exclusive => true)
# #handle_message method will be handling messages routed to @queue
@queue.subscribe(&method(:handle_message))
end # start
#
# Implementation
#
def handle_message(metadata, payload)
puts "Received a message: #{payload}, content_type = #{metadata.content_type}"
end # handle_message(metadata, payload)
def handle_channel_exception(channel, channel_close)
puts "Oops... a channel-level exception: code = #{channel_close.reply_code}, message = #{channel_close.reply_text}"
end # handle_channel_exception(channel, channel_close)
end
Full example:
In this example, `Consumer` instances have to be instantiated with an {AMQP::Channel} instance. If the message
handling was done by an aggregated object, it would completely separate the handling logic and would be make it
easy to unit test in isolation:
class Consumer
#
# API
#
def handle_message(metadata, payload)
puts "Received a message: #{payload}, content_type = #{metadata.content_type}"
end # handle_message(metadata, payload)
end
class Worker
#
# API
#
def initialize(channel, queue_name = AMQ::Protocol::EMPTY_STRING, consumer = Consumer.new)
@queue_name = queue_name
@channel = channel
@channel.on_error(&method(:handle_channel_exception))
@consumer = consumer
end # initialize
def start
@queue = @channel.queue(@queue_name, :exclusive => true)
@queue.subscribe(&@consumer.method(:handle_message))
end # start
#
# Implementation
#
def handle_channel_exception(channel, channel_close)
puts "Oops... a channel-level exception: code = #{channel_close.reply_code}, message = #{channel_close.reply_text}"
end # handle_channel_exception(channel, channel_close)
end
Full example:
Note that the `Consumer` class demonstrated above can be easily tested in isolation without spinning up any AMQP
connections:
require "ostruct"
require "json"
# RSpec example
describe Consumer do
describe "when a new message arrives" do
subject { described_class.new }
let(:metadata) do
o = OpenStruct.new
o.content_type = "application/json"
o
end
let(:payload) { JSON.encode({ :command => "reload_config" }) }
it "does some useful work" do
# check preconditions here if necessary
subject.handle_message(metadata, payload)
# add your code expectations here
end
end
end
TBD
h2. Queue durability vs message durability
See {file:docs/Durability.textile Durability guide}
h2. Error handling and recovery
See {file:docs/ErrorHandling.textile Error handling and recovery guide}
h2. Vendor-specific extensions related to queues
See {file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}
h2. What to read next
The documentation is organized as several {file:docs/DocumentationGuidesIndex.textile documentation guides},
covering all kinds of topics. Guides related to this one are:
* {file:docs/Exchanges.textile Working With Exchanges}
* {file:docs/Bindings.textile Bindings}
* {file:docs/ErrorHandling.textile Error handling and recovery}
RabbitMQ implements a number of extensions to AMQP v0.9.1 functionality that are covered in the
{file:docs/VendorSpecificExtensions.textile Vendor-specific Extensions guide}. At least one extension,
per-queue messages time-to-live (TTL), is related to this guide and can be used with the amqp gem v0.8.0 and later.
h2. Authors
This guide was written by "Michael Klishin":http://twitter.com/michaelklishin and edited by "Chris Duncan":https://twitter.com/celldee.
h2. Tell us what you think!
Please take a moment to tell us what you think about this guide "on Twitter":http://twitter.com/rubyamqp or the "Ruby AMQP mailing list":http://groups.google.com/group/ruby-amqp.
Let us know what was unclear or what has not been covered. Maybe you do not like the guide style or grammar or discover spelling mistakes. Reader feedback is
key to making the documentation better.
If, for some reason, you cannot use the communication channels mentioned above, you can "contact the author of the guides directly":mailto:michael@novemberain.com?subject=amqp%20gem%20documentation
amqp-1.8.0/docs/diagrams/ 0000755 0000041 0000041 00000000000 13321132064 015220 5 ustar www-data www-data amqp-1.8.0/docs/diagrams/007_rabbitmq_publisher_confirms.png 0000644 0000041 0000041 00000270525 13321132064 024105 0 ustar www-data www-data PNG
IHDR s2 pHYs IDATx O<$ P\b-_i)V
) )jqjT8@ƿBxQG0}γ {%LE:3<2KwKOr}ǯntA903駟>L;Ǘ셷] ׄ%~_Pʸql:aСag_dMf :"B |/K,y(sKX, i1X'A*O 74*y&~Xц>ڪ[/˦O^ʸi\GBb3 ! k ίM~ӵ} d`żsȐ㮳SJV'pxM&6Wq4\ijqe?^ʋW"Ie<.Dr}kתy6__} daZd@"ߝ [ڍ
Un4]؉*O>n
U)q u>?n U@6^~U_]~_ɼydÆ
ow @1я~T=P3f\veҧO +`ܾ}UB3
'@IC ]U B0 7
3g\yRVV&C9 ^|MYv<2zh뮓^ǼPC(n&"C4{mD;(4+++0m6⋥X.9 =A D`ӦMrUWW_"ر) qeDd@ 1qb@ar *{rGI'$=pP ެ}g峟\tE7sw%z-N7}$ 0h,^ ֣Ѻ`ؽ{]@رC&M$={']@ @A ;
ǝ:u :أӺ[ @L 1iTZbwe/0} Z =>=p@mwѥK{|ZS!
4
L7
V@$ T(s jXrj,#/L ;,όgJ7mR _tmV
YL7:>bldgαҵԤwD
7:PvuWeEAPІpat#y1? yN=8cxʢ\`_Ⴜx㍹*o:,NM58(q\pKzxÝvQVնLx_|,M
< 672X{ӥ^Li2]ab2}ꩧÜrxW~^Jo0F~NDrڂIk#ud)ǝ?#56~2- frw,ɿ^2wDX~ _Eu} ef\vNS݃\: C:ogU${n~71* X8{FB
SO=%/BRC"y8 9 I8f
Q ' pWmiN5pes jn te#hHPd/+ƭ74AxS:4 v&:7|Ю?>Ǟ\/` CutáNn* .Tá>8{gyHL_졒H;$hq:._dq
+Ŵ>@]#oݜo vrцTz=x? N?PgZ/ vKSkù \ʾjAAqee ~P}N~KZu~:zs®v Db渮PI;rB\
!so|
{ G)H$ %?G q~ ^x~Ö[nYx`?~i 98SpO2|pՉe]&C ;*~'4b%Fdiڇ}l0!ELrr3LrYN:O6 GR- ( '4riu,Qk+q₦fd=wۮt|mFŞҬE39]v[c<8&o@?ew3Cc˵W]+LXڵo'oV ɜ9sq J]K˶-e3s rӷH-)oek1g P - *TrB
ay+#ɐ1EZZó/ .9kl[^\^WrQ}dJeVnL}idW#<~Gjا$tRNژ]unR [ߵ;*ꘝwβM/W8?L}BءszcB:o3$: ޚ*Sń_1IU>c}Sѭe0I~t38IWXٱ 7 P.:ctB;ro+wG l"= dAx"ܫԃKggV-v֖\5kUunr[!l w/,qwQzmj2_oޏep&@&r;yn vy. Ⱦ;~| 8- NOY1/3R_g s$V ;wh*w%{J.%V^ +u%w5 4o|->p^۪]⸰^@ 2l-ۦU#-7mRO_s5AQ8]Kmg?H-q=|6 (pG lj"hhG}.% CnX
=h.QWcrp2i; 'x"\}Z ˃#?ҶjڨnupötW^iUu
x =ߡ3R:μwPm1=`rMgx]6+ع}악z] p^
O?r|*3ou|vS~UW9ia<Ġ~ ɭxvU)x Z,KljҐߵc3b\U4+Y mͫ]gfi07j%<迅荷Ɔ O/cFde?Yrϱ\ OŊ½ޫb3?^\|S8eey?J~dܸqqٳg%J99#e˖rK>}c {
I ӽEGvȑrGk/'|rr,Ա%l Z06
)y<%R1g+"kb ɴHQ=ӹ4Se&N#spD ݖV)GC JTrt2g[yVnqQe!NM( (Vfј̢L!KuuN2dfC裏փ]3F^u٤7XW1zkҥkժjzg2˲`p1 ge3[͡d{ xr0 J%RXJdL^ *K+ ] vTN6 l|v|} TCɘ