Ruby Rose: deploying moebooru for development

Ruby Rose

This time I’m going to show how to deploy moebooru on good ol debian. Most of the parts are valid for any other linux distros too. So, moebooru is a RoR+postgreSQL server application and that means that we need both of these installed on the server beforehand. But to do that, we must prepare the server. In this guide I assume that you are running the commands as root if not stated otherwise.

Part 0: Pre-requirements

In most (if not all) of the manuals curl is used to get files from web. As RVM installer depends on curl internally and not only for downloading itself, we’ll need to install it, updating the existing packages at the same time. It may also be a good practice to install sudo because in older versions rvm required it even if run from the root user.

apt-get update
apt-get upgrade
apt-get install curl sudo

Now we’re ready to go to next step and deal with ruby.

Part 1: Ruby

Ok, we’re ready to install ruby, but first we need to install rvm that will help us to manage ruby versions dynamically. It is important because in most of distros (minus arch, slackware and gentoo) it is either quite hard to manage ruby versions (RHEL-based) or outright impossible (Debian-based). Firstly we have to add a mpapis public key:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

We had to do it because rvm have to run checks during installation and it won’t install if the kay is not imported. We now may proceed with the installation itself.

curl -sSL https://get.rvm.io | bash -s stable

This may take time, but be patient because it is one of the most important steps. It is very hard to clean the mess after the broken rvm installation - trust me. When it finishes, you will have to add users that need to have access to ruby to rvm group.

usermod -g rvm user
usermod -g rvm root

We are not finished yet, but we have to relogin for the changes in user groups to apply. After successful relogin execute source /etc/profile.d/rvm.sh to make rvm usable. Now we should list versions of ruby available to be installed.

rvm list known

Right now version 2.2.0 is the latest more or less stable one so I’m going to install it, but you will have to choose it yourselves.

rvm install 2.2.0

RVM will try to download and install the precompiled version, but if it fails, it will automatically try to compile the correct version of ruby for your system. This will look like:

Unpacking bin-ruby-2.2.0.tar.bz2 failed.
Mounting remote ruby failed with status 6, trying to compile.
Checking requirements for debian.
Requirements installation successful.
Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.2.0, this may take a while depending on your cpu(s)...
ruby-2.2.0 - #downloading ruby-2.2.0, this may take a while depending on your connection...
ruby-2.2.0 - #extracting ruby-2.2.0 to /usr/local/rvm/src/ruby-2.2.0....
ruby-2.2.0 - #applying patch /usr/local/rvm/patches/ruby/2.2.0/fix_installing_bundled_gems.patch.
ruby-2.2.0 - #configuring.........................................................
ruby-2.2.0 - #post-configuration..
ruby-2.2.0 - #compiling.........................................................

If the server you have started the proccess on is not very fast (4+ cores, 8gb ram), this may take a lot of time so I would advice getting yourself a cup or two of coffee while you wait. When it finishes compilation, it will install ruby and proceed with gems installation.

ruby-2.2.0 - #installing............................
ruby-2.2.0 - #making binaries executable..
ruby-2.2.0 - #downloading rubygems-2.4.6
ruby-2.2.0 - #extracting rubygems-2.4.6....
ruby-2.2.0 - #removing old rubygems.........
ruby-2.2.0 - #installing rubygems-2.4.6.....................
ruby-2.2.0 - #gemset created /usr/local/rvm/gems/ruby-2.2.0@global
ruby-2.2.0 - #importing gemset /usr/local/rvm/gemsets/global.gems...........................................................
ruby-2.2.0 - #generating global wrappers........
ruby-2.2.0 - #gemset created /usr/local/rvm/gems/ruby-2.2.0
ruby-2.2.0 - #importing gemsetfile  /usr/local/rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.2.0 - #generating default wrappers........
ruby-2.2.0 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.2.0 - #complete
Ruby was built without documentation, to build it run: rvm docs generate-ri

Yay. It’s installed! Now just execute rvm use 2.2.0 (or another version of your choice) and we are ready proceed with Postgres and miscellaneous other tools (i.e., nginx and Node.JS) that will make our life easier.

Part 2: PostgreSQL

In comparison with the pain of installing Ruby, Postgres installation will be a breeze. All we have to do is import the repository key, add repository and apt-get postgres.

curl -sL https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
echo "deb http://apt.postgresql.org/pub/repos/apt/ wheezy-pgdg main" > /etc/apt/sources.list.d/postgres.list
echo "deb-src http://apt.postgresql.org/pub/repos/apt/ wheezy-pgdg main" >> /etc/apt/sources.list.d/postgres.list
apt-get update
apt-get install postgresql-9.4 postgresql-contrib-9.4

“And that’s it?” - you ask - “nope, not quite.” We still have to configure Postgres to be able to handle moebooru.

su postgres
psql

You will be dropped into postgres root console. Just in case we better reset the whole database encoding because sometimes encoding is set improperly in the linux distro and this will result in numerous errors and will ultimately fail moebooru installation.

UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';
DROP DATABASE template1;
CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE';
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';
\c template1
CREATE extension test_parser;
VACUUM FREEZE;

After this done we can add a database for moebooru with corresponding login credentials.

CREATE user moe WITH password '%password%' CREATEDB;
\q

And don’t forget to change %password% to the password you wish to use. You may also change the username, but I don’t think it is all that necessary.

Part 3: Miscellaneous software

Okay, moebooru has some optional dependencies. These are a javascript processing engine and an external web-server. There are various JS processor gems (ruby packages), but they do not always work with moebooru and thus I recommend installing Node.JS. Aside from ruby, it is quite a good language for developing dynamic web applications and building ReSTful APIs.

curl -sL https://deb.nodesource.com/setup | bash -
apt-get install nodejs

Now let’s talk about nginx. “Why should we even bother?” you ask and will be wrong. Ruby may be a good and rich language, but it is EXTREMELY resources hungry. And that’s the reason we need nginx. It will serve static files, taking load off of Ruby. It may also be a good practice to use varnish between nginx and moebooru, but making a proper config for it is a hard task so I will leave it for later. We found out why we need nginx - now let’s install it:

echo "deb http://nginx.org/packages/mainline/debian/ wheezy nginx" > /etc/apt/sources.list.d/nginx.list
echo "deb-src http://nginx.org/packages/mainline/debian/ wheezy nginx" >> /etc/apt/sources.list.d/nginx.list
curl -sL http://nginx.org/keys/nginx_signing.key | apt-key add -
apt-get update
apt-get install nginx
service nginx stop

So we have finished dealing with dependencies - let’s proceed with the installation of moebooru itself!

Part 4: moebooru

We have finished with the dependencies on the previous step, but there are still some packages that should be installed for moebooru to work properly.

apt-get install build-essential libxml2-dev libxslt1-dev libpq-dev git jhead  libgd2-noxpm libgd2-noxpm-dev imagemagick

Let me explain:

  • build-essential - a meta-package that installs various tools and libraries necessary for program (mostly C++ and Perl) compilation and interpretation.
  • libxml2 - library that is required for XML support. It is necessary for moebooru’s external API to work.
  • libxslt1 - extension of libxml that allows better xml parsing and converting.
  • libpq-dev - library that is necessary to build postgres connection module for ruby.
  • git - version control system - we will use it for moebooru installation and updating.
  • jhead - image processing tool that extracts EXIF information from image files.
  • libgd2-noxpm - an image processing library.
  • libgs2-noxpm-dev - an extension for compiling GD2 support for applications.
  • imagemagick - collection of tools for image processing

This will also install a whole lot of dependencies. After that run gem install bundler to make one last step of preparation. Now let’s switch to the non-root user that we added to the rvm group earlier and continue with the installation.

su user
cd ~
git clone  https://github.com/moebooru/moebooru.git
cd moebooru/
cp config/database.yml.example config/database.yml
cp config/local_config.rb.example config/local_config.rb

Now edit config/database.yml accordingly to what you executed in part 2 and config/local_config.rb as you wish. config/local_config.rb is well commented so I don’t see a point in explaining it. All that’s left is to create some directories necessary to run moebooru and kick the installer.

mkdir public/data
mkdir -p public/data/{avatars,frame,frame-preview,image,inline,jpeg,preview,sample,search}
bundle install
bundle exec rake db:create
bundle exec rake db:reset
bundle exec rake db:migrate
bundle exec rake i18n:js:export
bundle exec rake assets:precompile

I would not provide example output of these commands because it is huge (more than 2 1200x1920 screens on 8px font). Here’s the paste with it, though.

Now edit /etc/nginx/nginx.conf to your liking, but minding the following parameters:

client_max_body_size 200m;
sendfile        on;
keepalive_timeout  65;
gzip  on;

We’re almost there. Let’s create moebooru config for nginx. In this installation nginx configs are at the directory /etc/nginx/conf.d/, but in your case it might be different.

server {
    root    /home/twkr/moebooru/public;
    listen  80;
    server_name moebooru.lcl;
    location / { try_files /cache/$uri /cache/$uri.html $uri @moe; }
    location @moe {
            expires off;
            proxy_pass http://127.0.0.1:8080;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_redirect off;
            proxy_set_header Host $host:$server_port;
            proxy_set_header X-Forwarded-Proto $scheme;
            }
    }

You can leave it as it is, only changing the directory with the moebooru installation. Now last two steps:

bundle exec unicorn -D
service nginx start

Moebooru running

Aaaaahhh… Feels good, right?

Hope you liked this as much as I did. I did not cover init script here, but it’s relatively easy to write it so I leave it to your imagination to come up with it. Enjoy and have fun!

twkr out.