Monday, November 14, 2011

Source Control with Mercurial and BitBucket

The benefits of source control have been covered in numerous places around the Internet. The reference for Mercurial is available here. We'll simply describe how we manage the process.

BitBucket

Create a BitBucket account. This will provide you with unlimited storage for your source code.

Create a base folder for your projects.

$HOME/hgdev/org.bitbucket/${username}

The reasoning for the path segments of this layout are:

  • hgdev enables us to segregate source control systems {e.g. svndev, gitdev}
  • org.bitbucket enables us to segregate hosting providers
  • ${username} is your BitBucket username so you can also pull easily from others' repositories

We'll refer to this folder as your BitBucket folder (the BitBucket folder under your username).

New Repositories

To work with a new repository:

  1. Log onto BitBucket and create the project (e.g. myproject) there.
  2. cd into your BitBucket folder.
  3. Clone your newly created project
    hg clone https://${username}@bitbucket.org/${username}/myproject


Bash Development Tools

We introduce our organization of custom bash scripts for development.

Script Types

Bash is essential for automating tasks in Ubuntu. Generally, you'll work with three kinds of bash scripts:

  • Distribution scripts—these come with your Linux distribution
  • Deployed scripts—these come with deb packages that you install
  • Personal scripts—these are fully under your control and live in your home folder

You'll need a folder where you store your personal bash scripts. As with any code, this should be placed under source control.

Please see our short guide on source control and ensure your environment is set up appropriately.

Personal Bash Repository

In the interest of consistency, let's choose mysh as the project name for personal scripts. Create your project on your BitBucket account. Then check out your new project into

$HOME/hgdev/org.bitbucket/${user}/mysh

Now modify your $HOME/.bashrc to include this folder on your PATH

export PATH=$HOME/hgdev/org.bitbucket/${user}/mysh:$PATH

You'll need to reload your .bashrc for this to take effect. You can just open a new terminal.

Let's create a sample program hello.sh to test our configuration.

#!/usr/bin/env bash
echo "hello"

This should be in your mysh folder. Make this script executable

chmod +x hello.sh

Let's change folders to our home.

cd
hello.sh

This should execute your program.

Eclipse Setup for Bash

ShellEd is a good plugin for Eclipse for editing bash scripts. However it can be tricky to install. We assume Eclipse Indigo Service release 1.

Prerequisites

ShellEd requires Linux Man Tools.

Add Update Site

Go to Help -> Install New Software... ->

Linux Tools
http://download.eclipse.org/technology/linuxtools/update

Install the Linux Man Tools. Then install ShellEd

ShellEd
https://downloads.sourceforge.net/project/shelled/shelled/ShellEd%202.0.1/update


Using Eclipse for Python on Ubuntu 10.04 LTS

We use Eclipse for Python, Java and C++ development. Eclipse is a fairly large, complicated beast. To stay true to the Unix Philosophy, we use command line tools wherever possible. Also, to reduce potential conflicts between tools and plug-ins, we break up the workspaces so that each development environment has a dedicated workspace.

Download Eclipse

Go to the Eclipse website and download the Eclipse IDE for Java EE Developers. This version comes with a collection of plug-ins that support database access, XML editing, and various other common tasks that you'll need.

Once downloaded, create a folder in your home directory called local/eclipse. Unzip the package you downloaded into this folder.

% cd $HOME/local/eclipse
% tar zvxf $HOME/Downloads/eclipse-jee-indigo-SR1-linux-gtk-x86_64.tar.gz

This unpacks a folder called eclipse. Rename it to eclipse-indigo.

% mv eclipse eclipse-indigo
% ln -s eclipse-indigo current

You will now have a folder layout that looks like the following:

$HOME/local
        /eclipse/eclipse-indigo
        /eclipse/current -> eclipse-indigo

$HOME/workspaces
        /python
        /java
        /cpp

We've gone to this trouble to make it easier to write scripts that involve Eclipse—even if we upgrade, we only need to point to $HOME/local/eclipse/current. As we mentioned earlier, Eclipse is a complicated application that often requires environmental modifications and parameters tweaks. Rather than typing these out each time, we'll capture our requirements in a bash script.

Eclipse bash Launch Script

Please see the Bash Development Tools article.

Create a script called py-eclipse.sh in mysh.


You can now launch Eclipse with the appropriate parameters and environmental variables

py-eclipse.sh &

When you start your Eclipse session, be sure to set the appropriate path to your workspace. In this case, it's $HOME/workspaces/python

PyDev

PyDev is a useful extension for Eclipse that provides support for code completion and debugging. Once you've launched your Python workspace, install the Pydev extension.

First Project

Let's create our first project. Do not create projects in your workspace! The workspace contains settings unrelated to your project—it's best to keep them separate.

Create a repository for your Python project in BitBucket and check it out into your BitBucket folder. For example, pywork.

  • From Eclipse, create a new PyDev project. Give it the same name as your BitBucket repository (pywork).
  • Uncheck Use default and select the root of pywork.
  • Ensure that Add project directory to PYTHONPATH? is selected.

You'll now have a new PyDev project that you can also add to your shell PYTHONPATH.



Wednesday, November 9, 2011

Bind9 and DHCP3 on Ubuntu 10.04 LTS

We continue documentation of a full Linux server stack. You should consider testing this setup using an experimental network on VirtualBox.

Blackbox Bind9 Package

Make sure you've configured your /etc/apt/sources.list appropriately.

The Blackbox Bind9 package configures a bind9 server for Ubuntu 10.04 LTS. Combined with DHCP3, you'll be able to join machines to your network and have them resolve each other as well as external addresses.

The source code for this package is available on-line.

Begin with

$ sudo apt-get install bbbind9

You'll be asked a series of questions. bbbind9 will attempt to guess reasonable defaults for your settings. You can accept them by pressing [Enter]:

Enter the base domain that your DNS server will service.

base domain name { home.lan }: 

Enter the netmask for the network your server will manage (generally it's a /24)

static interface netmask { 255.255.255.0 }:

Your DNS server should have a static ip address. All your other machines can have DHCP assigned addresses.

static ip assigned to bind9 server { 10.0.2.15 }: 

These are the DNS servers you hit for machines outside your domain.

external dns forwarders { 8.8.8.8;8.8.4.4; }: 

Network interface to which service binds.

interface to which static ip is assigned { eth0 }: 

Fully qualified name of this DNS server.

fully qualified name for dns host { stemcell.home.lan }: 

Email address of administrator. Note that this convention does not allow the @ symbol. hostmaster.home.lan should be read as hostmaster@home.lan This should be a mail alias, or a mailbox, where the person(s) maintaining DNS should read mail frequently. Any mail regarding the domain will be sent to the address listed here. The name need not be hostmaster, it can be your normal e-mail address, but the e-mail address hostmaster is often expected to work as well.

email address for hostmaster (no @ allowed, use .) { hostmaster.home.lan }: 

Gateway for the interface that has a static ip address.

static interface gateway { 10.0.2.2 }: 

Subnet for which this DNS server provides naming services.

ip zone for dns domain { 10.0.2 }: 

To confirm everything is working properly, perform the following:

nslookup ibm.com

You should receive the results of a query against your new host:

Server:     192.168.1.254
Address:    192.168.1.254#53

Non-authoritative answer:
Name:   ibm.com
Address: 129.42.38.1

Blackbox DHCP3 Package

The Dynamic Host Configuration Protocol enables machines on your network to be automatically configured with IP addresses, DNS, and other features. The Blackbox package is configured to be installed on the same host as bbbind9.

Subnet for which the DHCP3 server will be handing out leases.

subnet of dhcp zone { 192.168.1.0 }: 

DNS domain to be assigned to client.

base domain name { home.lan }: 

DNS server that clients use.

ip address for nameserver to grant to dhcp clients { 192.168.1.100 }: 

Fully qualified name of DNS server (this host).

fully qualified name for dns host { stemcell.home.lan }: 

IP gateway assigned to clients.

gateway that dhcp clients will use { 192.168.1.1 }: 

Pool of addresses handed out to clients. Enter ranges as a tuple ().

available dhcp pool { ('192.168.1.50', '192.168.1.250') }: 

Try adding a client to your network and confirm that its properties now point to your new Bind9 server and its address is from your DHCP3 address pool.



Tuesday, November 8, 2011

Experimental Networks with pfSense

Virtual Networks

In addition to providing an easy way to test operating systems, VirtualBox also provides a way to test networks.

Setup

VirtualBox

Turn off VirtualBox's internal DHCP server (via Preferences).

  1. Create a new guest on VirtualBox as a 64-bit FreeBSD instance. pfSense has very light resource requirements.
  2. Assign 128Mb of RAM and 8Gb of storage.
  3. Create two network interfaces. Both should be PCnet ll (Am79C970A)—FreeBSD has native support for these cards. One should be internal (intnet) and the other should be either NAT or Bridged.

Install pfSense

  1. Download pfSense. I used the Chicago mirror.
  2. Install using Quick Install.
  3. Assign one interface le0 to WAN and the second, le1, to LAN.
  4. Then point your browser (from a machine on your internal network) to https://ipaddress

You may want to change the subnet for your experimental network. I use 172.16.50.1 to avoid colliding with 192.168.x.x and 10.0.0.0 networks that are more common.

pfSense also includes a DHCP server by default. If you are testing DHCP services on your internal network, be sure to turn it off. It's listed under Services.

Finally, if you are running DHCP on your lan (host machine network), you may want to set up a DHCP lease reservation for the WAN interface to ensure you always get the same IP address for your experimental network.

Set Static IP on Ubuntu

If you are going to run DHCP on your experimental network, you'll need to assign a static ip to your DHCP server. Edit /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 192.168.1.100
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.1

Note that your values may differ. Then restart networking

$ sudo /etc/init.d/networking restart

Set Full Hostname on Ubuntu

Edit /etc/hosts to provide your machine with a fully qualified domain name:

127.0.0.1   stemsrv01.lab.lan stemsrv01 localhost
127.0.1.1   stemsrv01.localdomain   stemsrv01

Note that order matters. See the following post.

Port Forwarding

You may want to access the machines on your experimental network via SSH. To enable this, you'll need to turn on port forwarding on pfSense. But before you do, make sure you uncheck Block private networks on your WAN interface settings.

Then go to Firewall -> NAT and create a new port forwarding rule (in this case, forward WAN port 2222 to 22 on an internal host).