Managing external daemons/services

uWSGI can easily monitor external processes, allowing you to increase reliability and usability of your multi-tier apps. For example you can manage services like Memcached, Redis, Celery, Ruby delayed_job or even dedicated PostgreSQL instances.

Kinds of services

Currently uWSGI supports 3 categories of processes:

  • --attach-daemon – directly attached non daemonized processes

  • --smart-attach-daemon – pidfile governed (both foreground and daemonized)

  • --smart-attach-daemon2 – pidfile governed with daemonization management

The first category allows you to directly attach processes to the uWSGI master. When the master dies or is reloaded these processes are destroyed. This is the best choice for services that must be flushed whenever the app is restarted.

Pidfile governed processes can survive death or reload of the master so long as their pidfiles are available and the pid contained therein matches a running pid. This is the best choice for processes requiring longer persistence, and for which a brutal kill could mean loss of data such as a database.

The last category is a superset of the second one. If your process does not support daemonization or writing to pidfile, you can let the master do the management. Very few daemons/applications require this feature, but it could be useful for tiny prototype applications or simply poorly designed ones.

Since uWSGI 2.0 a fourth option, --attach-daemon2 has been added for advanced configurations (see below).

Examples

Managing a memcached instance in ‘dumb’ mode. Whenever uWSGI is stopped or reloaded, memcached is destroyed.

[uwsgi]
master = true
socket = :3031
attach-daemon = memcached -p 11311 -u roberto

Managing a memcached instance in ‘smart’ mode. Memcached survives uWSGI stop and reload.

[uwsgi]
master = true
socket = :3031
smart-attach-daemon = /tmp/memcached.pid memcached -p 11311 -d -P /tmp/memcached.pid -u roberto

Managing 2 mongodb instances in smart mode:

[uwsgi]
master = true
socket = :3031
smart-attach-daemon = /tmp/mongo1.pid mongod --pidfilepath /tmp/mongo1.pid --dbpath foo1 --port 50001
smart-attach-daemon = /tmp/mongo2.pid mongod --pidfilepath /tmp/mongo2.pid --dbpath foo2 --port 50002

Managing PostgreSQL dedicated-instance (cluster in /db/foobar1):

[uwsgi]
master = true
socket = :3031
smart-attach-daemon = /db/foobar1/postmaster.pid /usr/lib/postgresql/9.1/bin/postgres -D /db/foobar1

Managing celery:

[uwsgi]
master = true
socket = :3031
smart-attach-daemon = /tmp/celery.pid celery -A tasks worker --pidfile=/tmp/celery.pid

Managing delayed_job:

[uwsgi]
master = true
socket = :3031
env = RAILS_ENV=production
rbrequire = bundler/setup
rack = config.ru
chdir = /var/apps/foobar
smart-attach-daemon = %(chdir)/tmp/pids/delayed_job.pid %(chdir)/script/delayed_job start

Managing dropbear:

[uwsgi]
namespace = /ns/001/:testns
namespace-keep-mount = /dev/pts
socket = :3031
exec-as-root = chown -R www-data /etc/dropbear
attach-daemon = /usr/sbin/dropbear -j -k -p 1022 -E -F -I 300

When using the namespace option you can attach a dropbear daemon to allow direct access to the system inside the specified namespace. This requires the /dev/pts filesystem to be mounted inside the namespace, and the user your workers will be running as have access to the /etc/dropbear directory inside the namespace.

Legion support

Starting with uWSGI 1.9.9 it’s possible to use the The uWSGI Legion subsystem subsystem for daemon management. Legion daemons will be executed only on the legion lord node, so there will always be a single daemon instance running in each legion. Once the lord dies a daemon will be spawned on another node. To add a legion daemon use –legion-attach-daemon, –legion-smart-attach-daemon and –legion-smart-attach-daemon2 options, they have the same syntax as normal daemon options. The difference is the need to add legion name as first argument.

Example:

Managing celery beat:

[uwsgi]
master = true
socket = :3031
legion-mcast = mylegion 225.1.1.1:9191 90 bf-cbc:mysecret
legion-smart-attach-daemon = mylegion /tmp/celery-beat.pid celery beat --pidfile=/tmp/celery-beat.pid

--attach-daemon2

This option has been added in uWSGI 2.0 and allows advanced configurations. It is a keyval option, and it accepts the following keys:

  • command/cmd/exec: the command line to execute

  • freq: maximum attempts before considering a daemon “broken”

  • pidfile: the pidfile to check (enable smart mode)

  • control: if set, the daemon becomes a ‘control’ one: if it dies the whole uWSGI instance dies

  • daemonize/daemon: daemonize the process (enable smart2 mode)

  • touch semicolon separated list of files to check: whenever they are ‘touched’, the daemon is restarted

  • stopsignal/stop_signal: the signal number to send to the daemon when uWSGI is stopped

  • reloadsignal/reload_signal: the signal to send to the daemon when uWSGI is reloaded

  • stdin: if set the file descriptor zero is not remapped to /dev/null

  • uid: drop privileges to the specified uid (requires master running as root)

  • gid: drop privileges to the specified gid (requires master running as root)

  • ns_pid: spawn the process in a new pid namespace (requires master running as root, Linux only)

  • chdir: chdir() to the specified directory before running the command (added in uWSGI 2.0.6)

Example:

[uwsgi]
attach-daemon2 = cmd=my_daemon.sh,pidfile=/tmp/my.pid,uid=33,gid=33,stopsignal=3