Init Scripts for Web Apps on Linux, and Why You Should Be Using Them

No, not a script that makes you talk like Ali G

I have seen too many servers running production applications, where if the server has to be rebooted, someone has to log into it and restart the app. Everyone knows of these magical arcane things called 'init scripts' which start various services when a server boots up, but for some reason they are often not used by developers - perhaps due to fear of complexity. While they were fairly complex a while back, things have gotten much easier with the advent of newer technologies on the scene.

Although it could get very complex in the past to write these, it is not actually that complex these days, as we have modern 'init daemons' to help us. An 'init daemon' is what controls how the system starts up and shuts down, and init scripts are the scripts we write to tell it how to start up or shut down our apps.

Upstart (Older Ubuntus, Amazon Linux, Old Red Hat Enterprise Linux)

'Upstart' is an init daemon written and popularized by Ubuntu. Newer Ubuntu versions have replaced it with Systemd, but for compatibility's sake, let you use Upstart scripts anyway.

Here is a sample script that starts a Rails app called 'app' from the directory '/srv/app'. It starts it as the 'appuser' user, sets some environment variables, tells it to handle up to 4 requests concurrently, and does some super-cool restart-if-abnormally-terminated magic. Note: the puma web server is NOT told to start in the background with the '--daemon' flag - Upstart will take care of that.

# /etc/init/app.conf
description "My App Server"

start on runlevel [2345]
stop on runlevel [016]

setuid appuser
chdir /srv/app

# restarts service if it abnormally terminates...
respawn
# ...but quit trying if it fails 5 times in 60 seconds
respawn limit 5 60

env RAILS_ENV=production
env SECRET_KEY_BASE=foobar

exec bin/bundle exec puma -w4 -e production --preload -b tcp://localhost:3000/

The app can be controlled with:

sudo service app start/stop/restart/status

Annoyingly, Amazon Linux, as used on AWS, wishes to stay compatible with the ancient Red Hat Linux 6, and has an ancient version of Upstart that does not support the 'setuid' stanza. Thus we are forced to use 'su' to do the user changing for us, like so:

#/etc/init/app.conf
description "My App Server" 

start on runlevel [2345]
stop on runlevel [016] 

respawn
respawn limit 5 60

env RAILS_ENV=production
env SECRET_KEY_BASE=foobar

exec su -s /bin/bash -c 'cd /srv/app && bin/bundle exec puma -w4 -e production --preload -b tcp://localhost:3000/' appuser

In this case, the app is controlled with:

sudo initctl start/stop/restart/status app

Find out more about Upstart at http://upstart.ubuntu.com/cookbook/

Systemd (Almost Every Current Linux Distribution)

'Systemd' is an init daemon that has slowly replaced Upstart. It is more than an init daemon actually - but let's ignore that for now, as we do not want to get into a flame war (https://en.wikipedia.org/wiki/Systemd#History_and_controversy).

Again, here is a sample script that starts a Rails app called 'app' from the directory '/srv/app'. It starts it as the 'appuser' user, sets some environment variables, tells it to handle up to 4 requests concurrently, and does it's own restart-if-abnormally-terminated magic. Note: as above, the puma web server is NOT told to start in the background with the '--daemon' flag - Systemd will take care of that.

# /etc/systemd/system/app.service
[Unit]
Description=App
# Run after these
After=syslog.target
After=network.target

[Service]
Type=simple
User=appuser
Group=appuser
WorkingDirectory=/srv/app
Environment="RAILS_ENV=production" "SECRET_KEY_BASE=foobar"
# Always restart on abnormal termination
Restart=always
# Note that first argument must be an absolute path, rest are arguments to it
ExecStart=/srv/app/bin/bundle exec puma -w4 -e production --preload -b tcp://localhost:3000/
# Startup/shutdown grace period
TimeoutSec=60

[Install]
# Run before this
WantedBy=multi-user.target

The app can be controlled with:

sudo systemctl start/stop/restart/status app

The status sub-command is particularly cool, showing the app's status, standard output and process tree. Passing it the '-l' flag adds even more information to the output.

However, to actually make it start on boot, you need to 'enable' the service with:

sudo systemctl enable app

More at:

https://www.freedesktop.org/software/systemd/man/systemd.unit.html

https://www.freedesktop.org/software/systemd/man/systemd.exec.html

https://www.freedesktop.org/software/systemd/man/systemd.service.html

A Note on Restarting Apps on Deployment

Since these are system services, and you are not doing a deploy as root (I hope!), issuing a restart during a deploy will require you to use 'sudo'. You could always just give the user complete sudo access (as is the default if you are using EC2 instances, and using the default user), or you could limit the user to only being able to run certain commands as root using sudo - this is the topic for another blog post.

Conclusion

There you have it. It really is simple, once you know how, to create an init script for whatever flavour of Linux you are running on. No more having to log into the server to restart the application just because writing init scripts is too much hassle.

Asfand Qazi has been a Ruby on Rails developer for many years, but had always been given responsibility for sysadmin and automation tasks, as his colleagues thought he had some knowledge on the subject. So he started The DevOps Doctors, through which he can offer his services to anyone who needs his help.