Raspberry Pi+Minibian: Adjusting the Locale and Time Zone

I really have enjoyed playing with the Raspberry Pi, but many of my projects don't require a GUI (for example OpenVPN and web servers). For projects like these, I usually install Minibian which is a great, small version of Raspbian with all the GUI goodness removed. It is a perfect starting point for my headless projects, except that out of the box, it comes setup with locale and timezones configured for Great Britian. This causes some strange characters to come up when typing on the keyboard, and the clock doesn't reflect my local time zone.

Fortunately, these issues are easy to address, and I will show you the steps I take to configure my system for my locale and time zone (us-EN & America/Los_Angeles). Just use your locale and time zone in the steps below to configure your system as desired.

1. Install Minibian on your SD card following the provided instructions. Insert the card into your Raspberry Pi, power it up, and log in. (The default username/password is root/raspberry). You may also want to change the name of your system from the default "raspberrypi" by editing the /etc/hostname file.
2. Change the keyboard locale. Run the following command:

   dpkg-reconfigure keyboard-configuration

   then
   • Set your "Keyboard model" to "Generic 105-key (Intl) PC".
   • Set the "Country of origin for the keyboard" and "Keyboard layout" both to "English (US).
   • Next for the "Key to function as AltGr" choose "The default for the keyboard layout".
   • For "Compose key" I chose "No compose key".
   • Finally, choose "No" for "Use Control+Alt+Backspace to terminate the X server".
   Once the command completes, reboot your Raspberry Pi to pickup the new keyboard layout.
3. Now change the system local. Run the command

   dpkg-reconfigure locales

   and de-select "en_GB.UTF-8 UTF-8" and select "en_US.UTF-8 UTF-8", then set the default locale to "en_US.UTF-8" on the next screen.
4. Edit the /etc/default/locale file and change/add the following lines:

   LANG=en_US.UTF-8
   LC_ALL=en_US.UTF-8
   LANGUAGE=en_US.UTF-8

   then log out and back in again to pick up the new locale.
5. Finally, run

   dpkg-reconfigure tzdata

   and select "America", then "Los_Angeles" to set the local time zone for your system.
6. Now that your system understands you a little better, it's probably a good time to update your system using

   apt-get update
   apt-get upgrade

   then reboot your system again.

Optional Tweaks

If you'd like to make your system a little more secure, you can enable sudo and disable the root login with the following steps:

1. Install sudo:

   apt-get install sudo

2. Create your new user. For example, to add the user "bob" run the command

   adduser bob

   and follow the prompts.
3. Add your new user to the sudoer's list. Run visudo then add the following line (changing the username to the user you just added) after the root ALL=(ALL:ALL) ALL line:

   bob ALL=(ALL:ALL) ALL

4. Now disable the root user with the command

   passwd -l root

   (note that is a lower-case L, not a one)
5. Now log out of your root session and log in as your new user.

Finally, it's a good idea to change the SSH server keys because the ones that come with the image file make it easy to impersonate your system (since everybody has access to those keys):

sudo rm /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh-server

Now you should have a headless Raspberry Pi system that you can use for various server projects that is customized for your locale and timezone! If you have any questions or comments on this procedure, please feel free to leave a comment.

Read More......

Time-Lapse Video Capture From Network Cameras (Linux)

I have several network cameras watching the outside of my home, monitored by ZoneMinder. I have it set up so that when there is motion detected, it will record for several seconds and send me an email with stills of the incident. While this is nice and gives me a little peace-of-mind, I've always thought about having it record continuously. While it is easy enough to do in ZoneMinder, I didn't really want to use up that much storage recording video and then have to scroll through it to find anything interesting.

The other day I saw a blog post where someone was using a Raspberry Pi and a webcam to do some time-lapse photography, and that sparked an idea that seemed easy enough to do in an afternoon – I could come up with a Python script to grab images from the network cameras at fixed intervals, and write them to a video file in order to generate a time-lapse video!

The first step was to figure out how to build a video file a frame at a time using Python. I had played with the motion-jpeg (mjpeg) format in the past, which pretty much consists of jpeg images streamed one after the other in a file (sometimes with a boundary record between them). I discovered that I could simply capture and append jpeg images to a file and get a video file that could be read by a few video players and converters. Best of all, I could use a simple avconv (formerly ffmpeg) command to convert the mjpeg files to mp4, which is smaller and viewable by almost any player.

Next, I wanted to be able to time-stamp each image so that I could tell when the video was created. For this I stumbled across the Python Imaging Library (PIL) which supports several image formats, including jpeg. Using it, I was able to select a font and write a time-stamp on each image as it was captured before adding it to the mjpeg video file. If it isn't already installed on your system, you can install it using

sudo apt-get install python-imaging

for Debian-based systems or by using the appropriate package manager for your distro.

With all the pieces in place, I developed a little Python script that periodically grabs images from several network cameras and builds a separate mjpeg file for each of them:

talicam.py:

#!/usr/bin/python

# Number of seconds between frames:
LAPSE_TIME = 30

# Name of truetype font file to use for timestamps (should be a monospace font!)
FONT_FILENAME = "UbuntuMono-B.ttf"

# Format of timestamp on each frame
TIMESTAMP_FORMAT = "%Y-%m-%d %H:%M:%S"

# Command to batch convert mjpeg to mp4 files:
#  for f in *.mjpeg; do echo $f ; avconv -r 30000/1001 -i "$f" "${f%mjpeg}mp4" 2>/dev/null ; done

import urllib
import sys, time, datetime
import StringIO
import Image, ImageDraw, ImageFont

class Camera:
    def __init__(self, name, url, filename):
        self.name = name
        self.url = url
        self.filename = filename
        
    def CaptureImage(self):
        camera = urllib.urlopen(self.url)
        image_buffer = StringIO.StringIO()
        image_buffer.write(camera.read())
        image_buffer.seek(0)
        image = Image.open(image_buffer)
        camera.close()
        return image
        
    def TimestampImage(self, image):
        draw_buffer = ImageDraw.Draw(image)
        font = ImageFont.truetype(FONT_FILENAME, 16)
        timestamp = datetime.datetime.now()
        stamptext = "{0} - {1}".format(timestamp.strftime(TIMESTAMP_FORMAT), self.name)
        draw_buffer.text((5, 5), stamptext, font=font)

    def SaveImage(self, image):
        with open(self.filename, "a+b") as video_file:
            image.save(video_file, "JPEG")
            video_file.flush()

    def Update(self):
        image = self.CaptureImage()
        self.TimestampImage(image)
        self.SaveImage(image)
        print("Captured image from {0} camera to {1}".format(self.name, self.filename))


if __name__ == "__main__":
    cameras = []
    cameras.append(Camera("porch", "http://username:password@10.17.42.172/SnapshotJPEG?Resolution=640x480&Quality=Clarity", "cam1.mjpeg"))
    cameras.append(Camera("driveway", "http://username:password@10.17.42.174/SnapshotJPEG?Resolution=640x480&Quality=Clarity", "cam2.mjpeg"))
    cameras.append(Camera("backyard", "http://username:password@10.17.42.173/SnapshotJPEG?Resolution=640x480&Quality=Clarity", "cam3.mjpeg"))
    cameras.append(Camera("sideyard", "http://10.17.42.176/image/jpeg.cgi", "cam4.mjpeg"))
    cameras.append(Camera("stairway", "http://10.17.42.175/image/jpeg.cgi", "cam5.mjpeg"))
    
    print("Capturing images from {0} cameras every {1} seconds...".format(len(cameras), LAPSE_TIME))
    
    try:
        while (True):
            for camera in cameras:
                camera.Update()
                
            time.sleep(LAPSE_TIME)
            
    except KeyboardInterrupt:
        print("\nExit requested, terminating normally")
        sys.exit(0)

Notice the URLs supplied in the Camera constructors. These are specific to each brand of camera, but you can usually find the format with a little Googling. In my program above, the first three cameras are Panasonic BL-C101A network cameras, the last two are a D-Link DCS-930L and a TrendNet TV-IP551W which both have very similar software and URLs.

The font file referenced above needs to be located in the same directory as the Python script, and for best results should be a mono-space font. I just grabbed the Ubuntu Monospace Bold TrueType font file for use here, but you could use anything you like.

You will probably want to launch this as a background task so that it can run for extended periods of time. I have it running on the same server that runs my ZoneMinder setup, so it can run 24-7 collecting time-lapse video. I also wrote a quick little script file that iterates the mjpeg files it finds and converts them to mp4 for easier viewing and archiving:

mjpeg2mp4:

#!/bin/bash

echo "Removing old files..."
rm -fv *.mp4

echo "Converting files to mp4..."
for f in *.mjpeg ; do
    t=${f%mjpeg}mp4
    echo "  Converting $f to $t"
    avconv -r 30000/1001 -i "$f" -q 5 "$t" 2>/dev/null
done

echo "Done!"

I had a lot of fun learning a few new tricks while working on this, and hopefully you can use it as a starting point for your own time-lapse adventure. If you find this post useful, or have questions about how it works, please leave a comment below.

Read More......

Dependency Injection Using Nested Classes

In my day job (lately it seems to be a 24 hour day) I do most of my work in C#, and all our new projects are implemented using Test Driven Development. Like most people doing TDD, we need to inject mock/stub/dummy objects into our objects under test. The traditional way to accomplish this is either via Constructor Injection or Setter Injection.

I have never been a fan of using setters to inject test objects, because it clutters the code with properties and methods that are only used for testing and are never used by production code. At first, I relied on constructor injection, but would sometimes run into cases where I was passing in half a dozen or more objects into the constructor. Many of these objects were simply helper classes that bundled up some functionality needed by the object under test only, and would never normally be used outside of that class.

Over the last year we have evolved another way of injecting mock objects into our test objects – Nested Class Injection (or as we call it, “TestHook Injection”). In C#, a nested class has access to it's containing class's private members, so we can use methods on the nested class to reach into the private details of the outer class. Here is a simple example:

    public class ObjectUnderTest {

        private IHelperObject _helper;

        public ObjectUnderTest() {
            _helper = new ProductionHelper();
        }

        public bool MethodToTest(int testNumber) {
            return _helper.HelperMethod(testNumber);
        }

        public abstract class TestHook {

            public static void InjectObjectHelper(ObjectUnderTest objectUnderTest, IHelperObject helper ) {
                objectUnderTest._helper = helper;
            }

        }
    }

Now all the test related code is contained within the abstract TestHook class. This class acts like the diagnostic connector on your car, allowing access to internal data and manufacturer's info without exposing it to the user. In fact, you can even use conditional compilation to exclude the TestHook class in the release build to remove the “connector” if code size or security is an issue.

Now when you are ready to test your objects, you can do something like:

        [Test]
        public void Test_MethodToTest() {
            ObjectUnderTest objectUnderTest = new ObjectUnderTest();

            IHelperObject mockHelperObject = new MockHelperObject();
            ObjectUnderTest.TestHook.InjectObjectHelper(objectUnderTest, mockHelperObject);

            objectUnderTest.MethodToTest(12);

            // Test expectations here...
        }

Note that I do not normally create my own mocks as shown above, I typically use Rhino Mocks to create mock objects, and Ninject as my IoC container. The contrived examples above are just to show the basic technique of using a nested class to cleanly access the internals of an object for testing purposes.

Now injecting objects like this is really only useful when those objects are only going to be used internally by the class under test. If you need to pass in an instance of an object that is shared between multiple objects, traditional constructor or setter injection is fine, since that is actually part of the object's contract with the world.

We have been using this technique for several months now, and it has resulted in cleaner production code, and more streamlined testing. Our test setup methods are simpler now, because we only need to inject mocks in the tests that need them instead of in the common test setup method.

I figured I should write this up since I have found the technique very useful, and a quick Google search didn't turn up anything useful. I expect this can be used in any language that supports nested classes where the nested class has access to the outer class's private member variables. Let me know in the comments if you find this technique helpful, or have any improvements. Thanks for stopping by!

Read More......

std::string and sprintf

Amongst the many projects I am currently juggling, one of them involves developing some C++ code for an embedded Linux project. I've just hit an instance where I'd like to be able to use “printf” style formatting using std::string instances, but it can be pretty clumsy – I need to allocate buffer space for sprintf, invoke the function, and package up the result into another string.

I did a quick search on the Internet and found many others looking for the same thing: “Is there a way I can use sprintf in c++ using std:strings?” And invariably the answers were either

• “No”


• “Use std::ostringstream and c++'s built-in manipulators”


• “Use boost::format”

And they are all right!

It turns out that you can't safely use the *printf style variable argument lists with C++, as only POD (Plain Old Data) types can be passed in the list. Attempting to pass most other types will result in a segmentation fault at run time. The official C++ way is to use ostringstream and the usual stream manipulators to format parameters, but for me, that make's writing tests more difficult because I need to “build” the expected streams exactly the same way in my test code. The final method could work for me, except that this is an embedded project and I am trying to keep the number of libraries referenced to a minimum, since I need to build and include them on my target Linux system.

So my solution was to accept the limitations of the printf function and write a wrapper function that hides the buffer setup and string creation:

#include <string>
#include <stdarg.h>
#include <stdio.h>

string FormatString(const string& format, ...) {
    char buffer[1024];

    va_list arglist;
    va_start(arglist, format);
    int length = vsnprintf(buffer, sizeof(buffer), format.c_str(), arglist);
    va_end(arglist);

    return string(buffer, length);
}

I've tucked this away in a nested namespace so I can invoke it easily when needed. Now when I need formatted text I can just supply a c string or std::string format and my parameters, and get back a nice std::string instance in return. But there's a catch! I can't pass in a std::string as a parameter because the old-school c variable argument list can't handle objects. If I try to do that, I will get a “warning: cannot pass objects of non-POD type 'struct std::string' through '...'; call will abort at runtime” warning and a segmentation fault at runtime. The work around (at least for std::string objects) is to invoke the c_str() method on the string parameter instance. So for example

    string argument = "a string instance";
    string result = FormatString("I can format a %s parameter!", argument.c_str());

allows me to supply std::string arguments as well. The c_str() function returns a pointer to a temporary buffer containing a c style string version of the string object's value.

So now I have my template style formatting and I can move on with my project. This project is my first foray into any substantial C++ programming, so if you seen any glaring problems or know a better solution, or if this little tip helped you too, please let me know by leaving a comment.

Read More......