So I often find myself setting up a server running with celery and I want to use supervisord to make everything work nice.
Well the cool thing is supervisord is pretty damn awesome. The other thing is I'm forgetful as all get out and I always forget what I did on the previous set up and when it comes time to set up the new one I can't remember what to do. SO that's where this post comes in. I'm writing down what I do so that I can remember how to set up celery again in the future.
First thing first, install rabbitmq and supervisor.
Next need to create /etc/supervisord.conf:
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
chmod=0777 ; sockef file mode (default 0700)
chown=nick:nick ; socket file uid:gid owner
[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[include]
files = supervisord/*.conf
What this is doing is setting up the config file that supervisor will use. Then I start supervisor by sudo supervisor -c /etc/supervisord.conf
Now to check that supervisor is running use supervisorctrl status it will probably be empty or return nothing. Don't worry.
I usually include my celeryd.conf files in a directory in the root of my project, and I will symlink those files from the project to /etc/supervisor.
Now what goes in the celeryd.conf file? mine usually look something like this...
[program:celery-weekly-report]
environment = PYTHONPATH="/home/nick/code/python/southpark/lib/python2.6:/home/nick/code/python/southpark/src:/home/nick/code/python/southpark/src/leadville:/home/nick/code/python/southpark/src/southpark:/home/nick/code/python/southpark/lib/python2.6/site-packages:$PYTHONPATH",DJANGO_SETTINGS_MODULE="southpark.settings"
command=/home/nick/code/python/southpark/src/southpark/southpark/manage.py celeryd -v 2 -B -s celery -E -l INFO
user=nobody
numprocs=1
stdout_logfile=/var/log/celery/southpark.log
stderr_logfile=/var/log/celery/southpark.err
autostart=true
autorestart=true
The PYTHONPATH section is setting up the python path to mimic that of my virtualenv. So when supervisor goes to execute the command it will be executing it within the confines of my virtualenv.
The command section is the actual command that supervisor executes to run celery. Make sure your manage.py is executable.
Now everything is all setup all you gotta do is restart supervisor. supervisorctl shutdown and start it again with `sudo supervisord -c /etc/supervisord.conf``
Now check that your celery process is running with supervisorctl status.
tada it should be working.