charltones.com

Radio Pi

2019-01-17T21:05:00+00:00

I wanted to set up a radio alarm in my son’s room but didn’t have a clock radio. That’s no issue, clearly this is a software problem!

I thought I might be able to hack something together with the raspberry pi, and indeed I was right.

Setup

The only thing you need to install on the Pi is mplayer, which you can get by:

sudo apt-get install mplayer

I found the URL to play Radio 1 from here: http://www.suppertime.co.uk/blogmywiki/2015/04/updated-list-of-bbc-network-radio-urls/

Script

I then created this script:

#!/bin/sh

/usr/bin/mplayer -msglevel all=-1 http://bbcmedia.ic.llnwd.net/stream/bbcmedia_radio1_mf_p &
TASK_PID=$!
sleep 2400
kill $TASK_PID

This will start mplayer playing Radio 1 and then kill it after 40 minutes (2400 seconds). I saved this in /usr/bin/radio1.sh, making sure to:

sudo chmod +x /usr/bin/radio1.sh

Timing

The last part is to make sure it plays at 7am every day. This is where cron comes in. This will let you run commands on a schedule. If you run crontab -eit will open an editor to let you enter the rather arcane cron syntax. I entered:

0 7 * * * /usr/bin/radio1.sh

which means 7am every day.

Job done, just in time for bed.

PS

I originally used the timeout command to run the radio for 40 minutes. This hung mplayer, but only when running in a script. I’ve shaved enough yaks in my time to know a knotty problem when I see it, so I cheated and did something else instead.

Making an electronic drum kit

2019-01-14T00:35:00+00:00

We decided for Christmas 2018 to make presents. Of course this meant I had to escalate things and I spent loads of time on this project. It was great fun though and I learned a lot doing it. Plus I only burned one finger with the soldering iron.

My son is learning drums so I thought I would have a go at making an electronic drum kit. The basic idea is to have a set of pads that you can hit with real drum sticks but which make no noise themselves. The pads measure how hard you hit them and connect to a box to turn the ‘I’ve been hit’ signals into MIDI messages. The MIDI messages are sent to a cheap drum machine I found on eBay. The drum machine makes drum noises into headphones, so we can all be in the house without being deafened by drums. The bit in the middle that turns drum hits into MIDI messages is actually a real thing that musicians use that you can buy, if you have the money, like this: https://www.gear4music.com/Drums-and-Percussion/DDrum-DDTI-Drum-Trigger-Interface/2MB9.

I thought I would post showing what I’ve built and how I put it together, along with the very helpful blog posts that showed the way.

Research and planning

This tutorial from Sparkfun shows the basic principal: https://www.sparkfun.com/tutorials/330. It uses an Arduino to measure the signal from the Piezo. I was originally going to make a bare bones electronics bundle and leave it at that, but no, I had to go further.

I came across this post: Spooky Arduino Projects which really shows the way. By sandwiching the sensor between two mouse mat circles you can create a tough mat that you can hit with drum sticks. You can also make the Arduino emit MIDI really easily, since the Arduino can send serial signals from the digital output pins.

This post: https://www.instructables.com/id/MIDI-Arduino-Drums/ was really helpful in designing the kit and building the pads. It also had links to some really good Arduino sketches (software for Arduino is called a sketch).

I was originally thinking of using a Raspberry Pi to make the drum sounds but luckily I decided that was going to be too much effort - I bought a cheap drum machine off eBay instead. Vintage 1992, the Yamaha RY-10: http://www.muzines.co.uk/articles/yamaha-ry10-drum-machine/2345.

Let’s make this!

First I had to make the pads. To start I cut circles from mouse mats.

Then, after soldering the piezo sensor to a shielded cable, I taped the sensor to the middle of an old CD.

I used superglue to sandwich the cd between two of the mouse mat disks.

Finally I soldered a 1/4” jack onto the end of the shielded cable. This is actually a fairly standard musical instrument connection.

Prototype

Before going further I wanted to test the concept worked so I built a prototype using the todbot.com blog earlier.

In the picture you can see the first pad that I made, with the 1/4” jack plugged into a jack socket. The socket is wired to a circuit on the breadboard and ultimately connected to the analog input pins of the Arduino. I’ve also connected a 5-pin DIN socket to the serial output of the Arduino to create a MIDI port. I’ve plugged a MIDI cable into this and the other end plugged into my Alesis MIDI keyboard. The keyboard is playing the part of the drum machine in my prototype.

I loaded the Arduino with this sketch https://todbot.com/arduino/sketches/midi_drum_kit/midi_drum_kit.pde - though you have to adjust the pin numbers in the sketch to match which pins you have connected to. Also note the sketch uses MIDI channel 1, whereas drums are normally on channel 10. After fiddling with these values a bit, it started working! I hit the pad, drum sounds came from my keyboard. It was going to work!

Productizing

Now I knew the idea would work, I started to plan how the finished product might look. The Arduino I was using had 6 analog inputs, so I could connect at most 6 drum pads. The little circuit on the breadboard is to protect the Arduino from the voltage that the piezo sensor generates. So for 6 pads I would need to repeat the circuit 6 times and have 6 jack sockets. That’s a fair bit of wiring as I quickly discovered. I didn’t think the breadboard would be robust enough so I ordered some veroboard and soldered 6 copies of the protection circuit to it like this. Yes, I know my soldering isn’t going to win any awards, but I hadn’t done it before.

Pedals

Now I could make 6 pads, which would already have been a fairly decent drum collection. But everytime I showed this to someone they asked me where would the pedals for bass drum and hi-hat come into it. I thought about wiring a piezo to a real drum pedal, but real drum pedals seemed to be really expensive. Then I would still need to rig up something for the pedal to hit. This sounded like too much work to get done. I already had a sustain pedal for my keyboard which I wasn’t using, so I wondered if I could use that. Turns out that electronic keyboard sustain pedals tend to all work in the same way - they’re effectively a big switch that ends in the ubiquitous 1/4” jack. If I wired these to the digital inputs of the Arduino then I could get pedal input. The only drawback is that these are not velocity sensitive, but I thought that was a good compromise for a pedal. I ordered another cheap sustain pedal. Now I could have 8 velocity sensitive pads and two pedals. By sheer luck I had enough for a complete drum kit: bass drum (pedal), snare, high tom, low tom, floor tom, crash cymbal, ride cymbal and hi-hat (pedal).

More circuits

To wire in the pedals I added some more circuitry. I wanted to at least try and protect in case someone plugged a piezo-based pad into the pedal ports. I also wanted an LED to show when the circuit was sending a drum hit, just because I like having flashing lights on things. Cue me soldering a second piece of veroboard. This time I managed to burn my finger on the soldering iron. Nothing like the smell of burning flesh to make you wonder, “why am I doing this again?”.

The box

I realised that I couldn’t hand over a wired together bunch of circuits and expect it to last in front of a kid wielding drum sticks. I had to put all the electronics into a box. I thought the box should have a power light and an on / off switch too. Here I learned that a combination of my amateur soldering, solid core wires and poor planning meant there was no way the whole thing would squeeze into a tiny box. I had to go for a large box but found it still wouldn’t fit. So after that I had to go larger still. Hence the rather large dimensions of this beast.

To complete the box I ordered a power socket, power switch and two led holders. Putting this together involved a fair bit of Dremel action.

Coding

Up until this point the Arduino code from the todbot blog I was using was a bit unreliable. It uses delay() to sleep for short periods which might just about work for one drum pad but doesn’t work for lots being hit at the same time. Luckily the code from the Instructables post is better designed. It doesn’t have any delay() code and just continuously loops reading the state of the pads. I was able to adapt this code by adjusting all the pin numbers, matching the notes to be played to the RY-10 drum machine, and also to add code for the drum pedals.

//Xylophone
//Adapted for an ArduinoMega 
//from Jenna deBoisblanc and Spiekenzie Labs initial code

//*******************************************************************************************************************
// User settable variables
//*******************************************************************************************************************

int pinRead;
// On the Yamaha RY10 this amounts to:
// Pads 1-6 play: SD1 (C), SD2 (D), TOM1 (E), TOM2 (F), RIDE (K), CRASH (L)
// Pedals 1-2 play: BD1 (A), HH CLOSE1 (H)
byte PadNote[8] = {
  38,40,50,47,51,49,35,42};         // MIDI notes from 0 to 127 (Mid C = 60)
int PadCutOff[6] = 
{
  70,70,70,70,70,70};           // Minimum Analog value to cause a drum hit
int MaxPlayTime[8] = {
  90,90,90,90,90,90,200,200};               // Cycles before a 2nd hit is allowed
#define  midichannel 9;                              // MIDI channel from 0 to 15 (+1 in "real world")
boolean VelocityFlag  = true;                           // Velocity ON (true) or OFF (false)

//*******************************************************************************************************************
// Internal Use Variables
//*******************************************************************************************************************
boolean activePad[8] = {
  0,0,0,0,0,0,0,0};                   // Array of flags of pad currently playing
int PinPlayTime[8] = {
  0,0,0,0,0,0,0,0};                     // Counter since pad started to play
byte status1;

int pin = 0;     
int hitavg = 0;

// define the pins we use
#define switchAPin 7
#define switchBPin 6
#define ledNotePin     4  // for midi out status
#define ledPowerPin     5  // to show we're on

//*******************************************************************************************************************
// Setup
//*******************************************************************************************************************
void setup() 
{
  pinMode(switchAPin, INPUT);
  pinMode(switchBPin, INPUT);
  digitalWrite(switchAPin, HIGH);  // turn on internal pullup
  digitalWrite(switchBPin, HIGH);  // turn on internal pullup

  pinMode(ledNotePin, OUTPUT);
  pinMode(ledPowerPin, OUTPUT);
  digitalWrite(ledPowerPin,HIGH);  // indicate we're sending MIDI data

  Serial.begin(31250);                                  // SET HAIRLESS TO THE SAME BAUD RATE IN THE SETTINGS

}
//*******************************************************************************************************************
// Main Program
//*******************************************************************************************************************
void loop() 
{
  // deal with the analog pads
  for(int pin=0; pin < 6; pin++)                          //
  {
    hitavg = analogRead(pin);  
    // read the input pin

    if((hitavg > PadCutOff[pin]))
    {
      if((activePad[pin] == false))
      {
        if(VelocityFlag == true)
        {
          hitavg = (hitavg / 8) -1 ;               // Upper range
        }
        else
        {
          hitavg = 127;
        }
        MIDI_TX(144,PadNote[pin],hitavg); //note on

        PinPlayTime[pin] = 0;
        activePad[pin] = true;
      }
      else
      {
        PinPlayTime[pin] = PinPlayTime[pin] + 1;
      }
    }
    else if((activePad[pin] == true))
    {
      PinPlayTime[pin] = PinPlayTime[pin] + 1;
      if(PinPlayTime[pin] > MaxPlayTime[pin])
      {
        activePad[pin] = false;
        MIDI_TX(144,PadNote[pin],0); 
      }
    }
  }
  // deal with the switches
  handle_switch_pin(switchAPin);
  handle_switch_pin(switchBPin); 
  boolean active = false;
  for(int pin=0; pin < 8; pin++)
  {
    if (activePad[pin] == true) {
      active = true;
    }
  }
  if (active == true) {
    digitalWrite(ledNotePin,HIGH);  // indicate we're sending MIDI data
  } else {
    digitalWrite(ledNotePin,LOW);
  }
}

void handle_switch_pin(int pin) 
{
  int currentSwitchState = digitalRead(pin);
  if (!activePad[pin] && currentSwitchState == LOW) {
    activePad[pin] = true;
    PinPlayTime[pin] = 0;
    MIDI_TX(144,PadNote[pin],127); //note on
  } else if (activePad[pin]) {
    PinPlayTime[pin] = PinPlayTime[pin] + 1;
    if(PinPlayTime[pin] > MaxPlayTime[pin]) {
      MIDI_TX(144,PadNote[pin],0); //note off
      if (currentSwitchState == HIGH) {
        activePad[pin] = false;      
      }
    }
  }
}

//*******************************************************************************************************************
// Transmit MIDI Message
//*******************************************************************************************************************
void MIDI_TX(byte MESSAGE, byte PITCH, byte VELOCITY) 
{
  status1 = MESSAGE + midichannel;
  Serial.write(status1);
  Serial.write(PITCH);
  Serial.write(VELOCITY);
}

End result

I surprised myself in the end and was really pleased how well this turned out. I think my son liked it too. I may let him play on it occasionally :)

Ode to discrete maths

2019-01-13T19:35:26+00:00

The Birrow Manchkins came snorting up the Quangley Way,
drooping their fleeb grotchits on the swailing clay.
The Turlingdrones vooped and quelched all night,
and the Gloorfelts opened their jaws to bite.

But Quazrak stood and fired his twangle,
his uluzong, his trop and his electric mangle.
There was thunder, lightning and squelchy bangs.
as the twangle implusticated into a million fangs.

The uluzong virried, quooped and obfamlicated,
and the army of Manchkins were instantaneously decapitated.
The trop spleeved obfuciously on the Turlingdrones,
and mutated them all into malfunctioning telephones.

The mangle volluped into a thousand queeds,
which flimped and whined at atrocious speeds.
The queeds flew into the Gloorfelts’ jaws,
and cut them up with a thousand power saws.

Then Quazrak looked down upon the Quangley Way,
at all the bodies and slime that he had managed to slay.
He had got his revenge for all the things that his foe did,
but then he stepped on a vop and immediately imploded.

Totally Dockerized, Part 3: Tuning and fixing

2015-09-11T16:59:13+01:00

I mentioned in my last Dockerized post that I would follow up with how I tuned the setup to work more reliably. Here are some of the problems I encountered and what I did to fix them:

MariaDB exit

One of the first problems I had was that the MariaDB container would just stop after a short period. This led to the wordpress site stopping working a lot, which was annoying. From the logs, it looked like the database was complaining about a lack of memory. I first tried switching to mysql but that didn’t seem to make any difference so I logged into the docker host to have a look:

$ docker-machine ssh <your environment name>

Once on the machine I used top to see where all the memory was going. Bear in mind I’m running a lot of containers on a micro instance, so there is only 1GB to go around. To my surprise mariadb wasn’t the memory hog, just the victim of another memory hog – Apache. The wordpress official Docker container comes with two flavours – a default one that uses Apache, and a second one that uses PHP FPM but which doesn’t have a web server. My solution was to move to the second flavour and add a new container running nginx. I found a gist which contained the necessary nginx foo and put it in github here: https://github.com/charltones/docker-nginx-fpm

Once I added this to my docker compose file, everything started up using a lot less memory and stayed running.

Could not find bridge docker0

A couple of times, after running docker-compose up and docker-compose-down a lot, I got this error:

adding interface veth1287a59 to bridge docker0 failed: could not find bridge docker0: no such network interface

The short answer to this is that it means docker is broken and needs to be restarted:

$ docker-machine ssh <your environment>

<docker host>$ sudo service docker restart

After this docker becomes happy again. I guess this is one of those hints that docker isn’t quite ready to be used in commercial production systems.

Docker stats

In the course of investigating low memory issues, I found a more accurate way of seeing exactly how much memory each container was using:

$ docker stats <list of container names>

Provides this kind of info:

CONTAINER                         CPU %               MEM USAGE/LIMIT     MEM %               NET I/O
docker_charltones_1               0.00%               2.048 MB/1.041 GB   0.20%               739.3 MB/776.5 MB
docker_charltonesbackup_1         0.00%               3.011 MB/1.041 GB   0.29%               5.624 MB/64.09 kB
docker_charltonesbackupnotify_1   0.00%               1.802 MB/1.041 GB   0.17%               26.38 kB/738 B
docker_charltonesdb_1             0.03%               102.1 MB/1.041 GB   9.81%               44.04 MB/858.8 MB
docker_charltoneswp_1             0.00%               113.5 MB/1.041 GB   10.91%              879.1 MB/773.5 MB
docker_dashboard_1                0.00%               26.54 MB/1.041 GB   2.55%               47.63 kB/224.7 kB
docker_fudgetiger_1               0.01%               26.48 MB/1.041 GB   2.54%               314.2 kB/23.54 MB
docker_hostmybusiness_1           0.01%               26.61 MB/1.041 GB   2.56%               44.24 kB/34.87 kB
docker_infonimbus_1               0.00%               24.94 MB/1.041 GB   2.40%               24.7 kB/738 B
docker_inkymum_1                  0.01%               25.9 MB/1.041 GB    2.49%               140.8 kB/3.94 MB
docker_mobilemechanic_1           0.00%               27.86 MB/1.041 GB   2.68%               365 kB/3.361 MB
docker_proxy_1                    0.09%               7.733 MB/1.041 GB   0.74%               820.1 MB/814.2 MB
docker_s3charltones_1             0.00%               11.68 MB/1.041 GB   1.12%               49.09 MB/41.75 MB
docker_s3dashboard_1              0.00%               2.63 MB/1.041 GB    0.25%               1.379 MB/467.5 kB
docker_s3fudgetiger_1             0.00%               2.236 MB/1.041 GB   0.21%               5.553 MB/492.5 kB
docker_s3hostmybusiness_1         0.00%               2.175 MB/1.041 GB   0.21%               2.582 MB/471.9 kB
docker_s3infonimbus_1             0.00%               2.114 MB/1.041 GB   0.20%               1.138 MB/464.8 kB
docker_s3inkymum_1                0.00%               2.122 MB/1.041 GB   0.20%               7.004 MB/506.9 kB
docker_s3mobilemechanic_1         0.00%               2.093 MB/1.041 GB   0.20%               10.83 MB/539.6 kB
docker_s3shonarain_1              0.00%               2.834 MB/1.041 GB   0.27%               124.8 MB/1.701 MB
docker_shonarain_1                0.00%               25.97 MB/1.041 GB   2.50%               217.3 kB/3.545 MB

This really lets you home in on problem areas – particularly if you are trying to optimise for a tight memory environment. In doing this I found that mariadb was using more memory than anything else. After some research I found that I could drop the innodb_buffer_pool_size from 256MB to 64MB and save a lot of runtime memory. I made a custom Dockerfile and posted it here: https://github.com/charltones/docker-mariadb-lomem

After running with this, memory usage was much more healthy.

Where’s that email?

My WordPress container wouldn’t send emails. This is a common problem and it can be quite tricky to find the root cause because there are so many different levels on which it can go wrong. At the most basic level, at least when using ssmtp (http://linux.die.net/man/8/ssmtp) like I am, you need to make sure your SMTP server settings are correct. Then, you need to make sure that you can send emails from the command line:

$ docker exec -it <container name> bash
<container>$ mail -s "Test Subject" user@example.com < /dev/null

If you receive the email, then the next stage is to make sure that php can send emails:

$ docker exec -it <container name> bash
<container>$ php -a
> mail ('you@example.com', "Test PHP mail", "Test mail from PHP");
> exit

This is where it went wrong for me. I got the brilliant error message:

sh: 1: -t: not found

Luckily I’d come across this before. There is a great blog post that dissects exactly what causes this http://axiac.ro/blog/2013/06/sh-1-t-not-found/

The solution was to add a php.ini in the right place which specified the sendmail command line. I fixed it in my customised wordpress dockerfile here: https://github.com/charltones/wordpress/commit/d898b007282c101321da205d303617e280def2c1

Restart!

Occasionally one of the docker containers will exit. In my experience there is little in the logs to explain why this happens. One thing that can help is to specify a restart policy in the compose file. This is as simple as adding a:

restart: always

line to every container inside your docker-compose.yml

Wipe, Rinse, Repeat

In applying these fixes, it meant cleaning everything and restarting many times. If I had to restore data manually each time, I would never have bothered. This really showed the benefits of a fully bootstrapped, wipe-clean start. Every time I wipe and restart – all the data is restored to the latest automatically. This is the best way to start, not by adding it later.

Squared circles

2015-08-15T16:55:38+01:00

My father-in-law sent me some fantastic geometrical designs that he had drawn and inked, which inspired me to try and recreate them using Python. Here’s my attempt at squaring the circle on to infinity using IPython notebook:

And here is another one – I couldn’t quite master the original drawing, but this is my best attempt:

Why can’t I run rsync on my Synology?

2015-08-06T16:47:45+01:00

I use a Synology NAS at work, and I also happen to have one at home. They’re a great bit of kit but I ran into a quirk yesterday that I thought I would share.

We run a backup job in Jenkins to copy some files regularly onto the NAS. It runs a simple command, along the lines of:

ssh login@nas.local rsync remote_files local_folder

Nothing too strange about that, only for some reason it stopped working a few days ago giving an error:

Permission denied, please try again.

Now, that’s not normally a very challenging problem, but it was odd that it was working and then stopped. It turns out that this part of the mystery was the easy bit. On the NAS itself I found this:

synology$ cat /var/log/synoupdate.log
2015/07/28 00:15:20 start critical update to buildnumber: 5592 original smallfixnumbre: 0 new_smallfixnumber: 1 build date: 2015/07/09

So the Synology box had done a critical update just before the backups stopped working! But what was actually going wrong? After some investigation I found the permission error wasn’t related to the rsync binary, nor to the files I was trying to copy. In fact I narrowed it down to this rather odd occurrence:

$ ssh login@nas.local "rsync"
Permission denied, please try again.
$ ssh login@nas.local " rsync"
rsync version 3.0.9 protocol version 30
etc.

So any characters in front of the rsync command (in this case a single space) allowed it to work. W. T. F. Fortunately at this point I managed to find a post on the Synology forums that revealed everything: http://forum.synology.com/enu/viewtopic.php?f=160&t=92627

I discovered a new setting in DSM 5.1 that fixes this problem. From Main Menu->Backup & Replication, click on “Backup Services”. There is a setting “Enable network backup services”. Click on this and now rsync works like it always has.

After checking that setting, everything works fine! So what seems to be happening is that the Synology ssh service has a check in place for the first 5 characters of the command you are running. If it starts “rsync” and you have not checked the magic checkbox it will not allow the command to run. It is a pretty lame check, since even a space, or providing the full path or a symlink to another name will defeat it. When the NAS did its auto update the setting must have become unchecked. Hopefully this post will save others from the long debugging and Googling session it took me to solve!

Totally Dockerized, Part 2: Docker Compose

2015-07-27T13:48:48+01:00

I wanted to use Docker to host 8 different websites. Some of them use a little bit of PHP, and one of them (this site) uses WordPress and requires a database. I wanted all the websites to bootstrap from S3 so that I could smash the setup and rebuild pretty much instantly, no longer worrying about the sites getting hacked or broken. I wanted the static sites to update periodically from S3, so that I could edit them by editing the files on S3 (S3 is the data master). I wanted the WordPress site to back up regularly to S3 (WP is the data master).

I realised to do all this would require many different containers, and I didn’t want to start them by hand. I did that with my last attempt and although it worked it was a bit of a hack. I also wanted the entire solution to be containerized. The last attempt used Haproxy running on the host to send traffic to each site – this time I wanted that to happen in the container.

A colleague pointed me to Docker Compose which is described as:

Compose is a tool for defining and running multi-container applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

That is exactly what I needed! I’ve noticed a few Docker containers on Github now include Compose recipes (like this one) so this could be the way to go. I’ll describe how I set up and used Compose to spin up all my sites.

Installing

You can install Compose using pip as follows:

pip install -U docker-compose

but a word of caution here – pip will downgrade any dependencies necessary to install this, and in my case it downgraded the requests library which broke another tool I was running. Of course, the answer is to use virtualenv which is what I ended up doing.

Working with Compose

Compose is a very simple tool to get started with. The process is:

Define a Dockerfile or set of Dockerfiles for your application
Create a docker-compose.yml file which is a description of the services that make up your application and how they connect
Bring up the whole thing by typing docker-compose up

There are some useful tips for running Compose in production here, which I’d recommend reading. In particular you should note the advice on having multiple compose files:

docker-compose-dev.yml
docker-compose-prod.yml

and choosing which one to run using an environment variable:

$ COMPOSE_FILE=docker-compose-prod.yml
$ docker-compose up <span class="hljs-operator">-d</span>

My Compose file

The Compose file ended up being quite big, so I’ll explain it one section at a time. The whole thing needs to be in one plain text file though. The file is written using yaml and is split into a number of sections, one for each service that you are starting. Each service normally corresponds to one container, and the section effectively captures what you would normally have passed on the command line to Docker.

Proxy

proxy:
build: nginx-proxy
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro

This section uses Jason Wilder’s cool automatic reverse proxy for Nginx to set up a top level reverse proxy to all the other sites I’m hosting. This container hooks into Docker’s event stream to pick up when other containers are started and stopped and automatically creates Nginx config to route traffic to them. The source is here and a write up here. The ports section just says that the container will expose port 80 to the outside and pass on requests to port 80 on each child container.

S3

s3fudgetiger:
 build: docker-s3-volume
 command: /data s3://fudgetiger.com/www
 environment:
 AWS_DEFAULT_REGION: eu-west-1
s3inkymum:
 build: docker-s3-volume
 command: /data s3://inkymum.com/www
 environment:
 AWS_DEFAULT_REGION: eu-west-1
...
one of these sections for each site
...
s3charltones:
 build: docker-s3-volume
 command: /data s3://charltones.com
 environment:
 AWS_DEFAULT_REGION: eu-west-1

This section has one service per website that uses docker-s3-volume (source here). This container creates a Docker data volume (basically a folder that can be shared with other containers) which is backed up periodically to an S3 bucket. The data volume appears in /data. When the container is started it will do a restore, and it will do a backup whenever it receives a USR1 signal. More on that later.

I created one bucket per site and copied the root web folder to them. The environment section contains environment variables to pass to the container. In this case I needed to pass variables to allow the AWS command line tools to work since these are used to sync with S3. In my development environment, I needed to also specify AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. In production I granted a role to the machine running Docker so that it could access the required buckets.

I only made a minor change to the original container recipe – I changed it to do a restore every 30 minutes. This means the /data folder will be updated with any changes made to S3 every half hour. This is how I can edit the contents of the sites.

PHP / static sites

fudgetiger:
 build: nginxphp
 volumes_from:
 - s3fudgetiger
 environment:
 VIRTUAL_HOST: www.fudgetiger.com,fudgetiger.com
inkymum:
 build: nginxphp
 volumes_from:
 - s3inkymum
 environment:
 VIRTUAL_HOST: www.inkymum.com,inkymum.com
...
one of these sections per static site
...
mobilemechanic:
 build: nginxphp
 hostname: mobilemechanic-hampshire
 domainname: co.uk
 volumes_from:
 - s3mobilemechanic
 environment:
 VIRTUAL_HOST: www.mobilemechanic-hampshire.co.uk,mobilemechanic-hampshire.co.uk
 SMTP_SERVER: ********
 SMTP_USERNAME: ********
 SMTP_PASSWORD: ********

These sites are all based off this container. This is a standard recipe that uses Nginx and PHP to create a web site. The volumes_from section links each site to the /data volume that is copied from S3 every 30 minutes. The environment section is necessary for the top level proxy container to know what vhost to put in the top level nginx config.

I made one major change to the recipe. A couple of sites need to send emails. I altered the recipe to install ssmtp. This is a very simple MTA which is useful for cases where you need a service to be able to send emails but don’t want to have to set up EXIM or Postfix. I also had to change the recipe to point to /data as the web root.

WordPress

charltones:
 build: wordpress/apache
 hostname: charltones
 domainname: com
 links:
 - charltonesdb:mysql
 environment:
 VIRTUAL_HOST: www.charltones.com,charltones.com
 WORDPRESS_DB_USER: ********
 WORDPRESS_DB_PASSWORD: ********
 WORDPRESS_DB_NAME: ********
 SMTP_SERVER: ********
 SMTP_USERNAME: ****@********
 SMTP_PASSWORD: ********

charltonesdb:
 image: mariadb
 environment:
 MYSQL_ROOT_PASSWORD: ********
 MYSQL_DATABASE: ********
 MYSQL_USER: ********
 MYSQL_PASSWORD: ********

This section defines the site you’re currently reading. It uses the offical Docker WordPress image from here with only one modification by me to send emails as in the nginxphp recipe.

The yaml above will start two containers. One running apache and php to host WordPress, the other running MariaDB for storage. To link the two together, the first one uses a links section. This uses Docker’s linking facility to hook the two containers together without the need to specify ports.

WordPress Backup

charltonesbackup:
 build: wordpress-backup
 links:
 - charltonesdb:mysql
 volumes_from:
 - s3charltones
 - charltones

charltonesbackupnotify:
 build: docker-inotify-signal-container
 links:
 - s3charltones:monitor
 volumes:
 - /var/run/docker.sock:/var/run/docker.sock
 volumes_from:
 - s3charltones
 command: SIGUSR1 /data

The last section sets up two containers to help automatically back up WordPress. The source for wordpress-backup came from here. The source for the notify container came from here.

The wordpress-backup container sets up a cron to back up WordPress daily. It uses links to connect to both the database and the php containers, then creates an archive for each. It normally puts these on its own /backups volume, but I modified it to copy the archives to the s3 volume instead. I also added the ability to restore the latest archive, which is what it does when the container starts for the first time. This allows me to completely bootstrap this site from fresh when needed.

The first container makes backups, but these will just be stored inside my docker container. What is also needed is a trigger to copy the backups back to S3 when they are added. This is what the inotify-signal container does. It watches the S3 container, and when the /data folder changes it triggers a sync back to S3. The triggering is done by sending a USR1 signal to the S3 container. This is trapped by the main script which copies back to S3. I had to modify this container to be a bit more compose friendly.

That’s it! {.}

I’ll post next time details of some of the modifications I made to get these containers working how I wanted. I will also describe some of the challenges I had in getting it all working in production and how I overcame them.

Totally Dockerized, Part 1: Docker Machine

2015-07-19T13:43:20+01:00

Docker Machine is a great tool that is part of Docker itself, which is used for creating hosts to run Docker containers on. It can do this locally in a virtual machine, or in the Cloud using various providers. It is a Vagrant for Docker, if you like. Note it is only in Beta and not recommended for production use yet.

I decided that this would be a great replacement for Vagrant and the AWS provider that I was using. I’m working on Linux, and although I can run Docker natively and use my own machine as the host, I wanted to test what I was doing in a virtual machine so that it was clean and closer to the final hosting environment in AWS. I have tried setting up Docker Machine under windows. That was a trial and still isn’t working. When I get that going I will blog about it separately.

Before you start you will need VirtualBox installed, and will probably need to stick with a 4.x version since 5.0 only just came out. I followed the instructions on this link and was able to do:

$ docker-machine create --driver virtualbox dev

This creates a local virtual machine running under Virtualbox and gives it the name ‘dev’. The machine is running boot2docker which is a tiny Linux that runs just from RAM, but has everything in it you need to run Docker. If you need to log into the machine, you can do this:

$ docker-machine ssh dev

But most of the time you wouldn’t need to do this. You can set up your environment so the docker command line operates on this docker virtual machine instead of your own machine. To see how, type this:

(docker)rob@kirk ~/projects/hosting/docker $ docker-machine env dev
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/home/rob/.docker/machine/machines/dev"
export DOCKER_MACHINE_NAME="dev"
# Run this command to configure your shell: 
# eval "$(docker-machine env dev)"

The important one is the DOCKER_HOST variable – this tells docker where to connect to launch and manage containers. So by typing this:

$ eval "$(docker-machine env dev)"

whenever you run docker now, it is pointing to your virtual machine:

(docker)rob@kirk ~/projects/hosting/docker $ docker run -d -p 8000:80 nginx
5f9a0faa00123ccd166daac1454241a01946e13449daa2ca8cd8acd021b500f7
(docker)rob@kirk ~/projects/hosting/docker $ docker ps 
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5f9a0faa0012 nginx "nginx -g 'daemon of 5 seconds ago Up 6 seconds 443/tcp, 0.0.0.0:8000->80/tcp jolly_swartz

So now I’m running a container running nginx on a virtual machine running on my laptop. How do I test that?

(docker)rob@kirk ~/projects/hosting/docker $ docker-machine ip dev
192.168.99.100

So if I connect to http://192.168.99.100:8000 in my browser, then I see:

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

In the next post I will introduce Docker Compose, which is another tool from Docker which will let me orchestrate the creation of sets of related containers. For other posts in the series, go to the introduction.

Totally Dockerized, Introduction

2015-07-19T13:39:57+01:00

In this series of posts I’m going to show how I set up a much more slick Docker hosting environment and migrated several websites to it. I’ll cover how I used Docker Compose and Docker Machine, along with some really handy Docker containers from Github. I’ll also describe a lot of the gotchas and quirks I encountered along the way.

After my last post, I had a long todo list of things to change with the hosting of the websites I’m using. I was using Vagrant with the AWS provider to run an EC2 instance to host my Docker containers. Each container was a separate website and I ran haproxy on the host machine to direct traffic to the right site. I also hacked together a setup using Dropbox so different people could update the content of their own sites. It all worked, it just didn’t seem entirely… dockerish. I wanted a setup where I could recreate the whole thing by running one command, and not rely on hand editing or configuring anything.

Bye bye Cpanel, hello Docker and AWS

2015-03-08T12:35:17+00:00

I’ve had a bunch of small websites, including this one, running on a variety of different cpanel (http://cpanel.net/) hosting accounts for a number of years. I’ve never been particularly happy with cpanel but since it seemed like the de-facto way of doing web hosting unless you run your own dedicated machine I haven’t really touched it. I always thought cpanel just had to much… stuff in it. Too many ways in, too many features, too much clutter.

Last year my reseller account was compromised a couple of times which I discovered was ultimately down to vulnerabilities in WHMCS (http://www.whmcs.com/) the reseller management front end that came with my account. Now, this was surely my fault for not keeping up to date with the patches, but for me this was enough of a nudge to clean everything up. There were just too many layers of PHP stuff for my liking: WHMCS, WHM, cpanel (a master one and then one for each client site). All of these exposed FTP, Webdev, web, management portals and backups and none of it over SSL. None of my customers actually wanted cpanel – they just wanted a place to put HTML files so they would appear on the web.

I decided I would consolidate all of my sites in one place. Since I use AWS for work and I’m interested in learning about Docker, I decided to combine everything and run the whole lot as containers on a single AWS instance (since I’m trying to keep everything in the free tier). I wanted to keep everything as maintenance free as possible, so I tried to automate from the beginning. The first thing to automate was the AWS machine creation. I decided to try and use Vagrant for this since I’d been playing with Vagrant while trying to get the Docker setup working on a local machine. I used the Vagrant AWS provider from here: https://github.com/mitchellh/vagrant-aws

Here’s my Vagrantfile:

Vagrant.configure(2) do |config|

  # This configuration is for our EC2 instance
  config.vm.provider :aws do |aws, override|
    aws.access_key_id = "<aws access key>"
    aws.secret_access_key = "<aws secret access key>"
    # ubuntu AMI - 64 bit trusty, eu-west-1, hvm
    aws.ami = "ami-73f97204"
    aws.keypair_name = "<aws keypair>"
    aws.security_groups = ["<aws security group"]
    aws.region = "eu-west-1 or other region"
    aws.instance_type = "t2.micro"

    override.vm.box = "dummy"
    override.vm.box_url = "https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box"
    override.ssh.username = "ubuntu"
    override.ssh.private_key_path = "path to private key.pem"
  end 

  # Install latest docker
  config.vm.provision "docker"
  # rsync our content
  config.vm.synced_folder "sync", "/vagrant"
  # bootstrap haproxy and all the containers
  config.vm.provision :shell, :path => "bootstrap.sh"
end

When I do “vagrant up” this does the following:

Create a new micro instance in the specified region using the specified AMI and set the security group
Install the latest version of Docker on it (this uses the Vagrant Docker provisioner)
Set up a folder of local files to be rsync’ed to the new machine
Run a bootstrapping script to do the rest of the setup

I only encountered one problem with this: the rsync happens later, so my bootstrap (which needs some files from the rsync) initially fails. I’m still working on that bit.

The bootstrapping script does the following:

#!/bin/sh

# install any extra packages
apt-get update
apt-get install haproxy

# copy our config changes over - mostly to get haproxy logging to rsyslog
cp -r /vagrant/config/etc /

# restart changed services
service rsyslog restart
service haproxy start

# build our nginx / docker basic container
docker build -t nginxphp /vagrant/nginxphp
# and the wordpress one
docker build -t wp /vagrant/docker-wordpress-nginx

# launch the docker containers
docker run -p <docker_host_port1>:80 --name dashboard -v /vagrant/<rsynced folder name>/www:/var/www:rw -d nginxphp /sbin/my_init
docker run -p <docker_host_port2>:80 --name charltones -v /vagrant/<rsynced folder name>/www:/var/www:rw -d wp

# and many more of these, one for each site

This script sets up haproxy, which I run on the docker host to forward web traffic to the correct container. It then builds a couple of containers. I use two – one for wordpress and one generic nginx/php one for mostly static sites. It then launches each container. The static files come from the rsynced folder and the port is exposed to haproxy – each one on a different port. Whenever I edit any site content, I just type “vagrant rsync” and the new content is automatically rsync’ed to AWS.

I would like to take this setup further – here are some next steps I want to look at:

Completely automate the bootstrapping so no intervention is required. This means fixing the bootstrap script so it runs after the initial rsync.

Automate the wordpress bootstrapping so that it restores content from a backup. This would mean if the wordpress install was broken I could just ‘vagrant up’ again and have all the content back.

Backups, probably to S3, so I can put everything back easily.

Investigate Docker Compose (http://docs.docker.com/compose/) the replacement for Fig in order to automate setting up complex Docker environments.

Find a nicer way for other people to work on the sites. Vagrant rsync only works for me if I’m doing all the editing. I’ve hacked up something using dropbox so that a few others can edit the sites but I’d like to come up with a better way of syncing the content.