Open Sourcery

Preserve formatting when pasting in vi/vim

dave — Thu, 10 Jul 2008 16:34:53 +0000

When copy and pasting into vi or vim, you can often end up with badly formatted text. This is often referred to as “the staircase effect”. It’s easily prevented by entering :set paste before you paste, when you have finished turn it off again with :set nopaste.

Preserve formatting when pasting in vi/vim

Handling temporary files in BASH shell scripts

dave — Thu, 05 Jun 2008 13:41:59 +0000

Temporary files left over from shell scripts clutter up your /tmp directory and may result in information leakage. Below are a pair of functions we use to gracefully handle the creation and removal of temporary files in shell scripts.

The first function is used to create a temporary file:

function os_mktemp { [[ ! $1 ]] && echo “os_mktemp: required a handle name” && exit 99 let OS_TEMPFILEHANDLE++; OS_TEMPFILE[$OS_TEMPFILEHANDLE]=`mktemp /tmp/ostmp.XXXXXXXX` eval F_$1=${OS_TEMPFILE[$OS_TEMPFILEHANDLE]} }

It requires a single parameter, which is used to create a variable name containing the path to the temporary file. For example. os_mktemp FTPOUTPUT will return a variable $F_FTPOUTPUT.

The array OS_TEMPFILE is an array holding the names of all the temporary file names, this is used by the cleanup function to remove the temporary files.

os_mktemp OUTPUT; this results in a temporary file with a random name being created and the name being stored in the variable $F_OUTPUT.

The second function is used to remove all temporary files.

function os_cleanup { for FILE in ${OS_TEMPFILE[@]}; do [[ -e "$FILE" ]] && rm “$FILE” || echo “os_cleanup: couldn’t remove temporary file $FILE.” done }

To ensure the os_cleanup code is executed whenever the shell script closes, we use the BASH trap command.

trap os_cleanup INT TERM EXIT

Handling temporary files in BASH shell scripts

Automatically backing up files before making changes

dave — Sun, 01 Jun 2008 08:40:01 +0000

It’s best practice (and common sense) to make a backup of a file before you edit it. Unfortunately it’s easy to forget to do this. We use this simple script below to make a time/date stamped copy of a file before launching the editor (in this case vim). We create it as /usr/local/bin/bvi.

#!/bin/bash [[ -r $1 ]] && cp $1{,.`date +%Y%m%d-%H%M`} || echo “$1 is a new file”
vim $1

We then add the following two aliases to our ~/.bashrc file to make sure it’s run automatically when we call vim.

alias vi=/usr/local/bin/bvi alias vim=/usr/local/bin/bvi

This isn’t a replacement for good version control of important files, but it’s a good safety net. It’s also worth noting that this can leave a lot of old copies of files laying about, so it’s work cleaning out old copies every now and again.

Automatically backing up files before making changes

Check, repair and optimize all MySQL databases

dave — Wed, 21 May 2008 19:49:14 +0000

The following command will check, repair and optimize all databases on your MySQL server.

mysqlcheck -u root -p --auto-repair --check --optimize --all-databases

It is the equivalent of calling CHECK TABLE, REPAIR TABLE, ANALYZE TABLE and OPTIMZE TABLE for each table in each database on your server.

We tend to run this command directly after our scheduled backups.

Check, repair and optimize all MySQL databases

BASH quick tip: Change to last directory

dave — Fri, 16 May 2008 19:55:21 +0000

You can change to the previous working directory in BASH by using the command:

cd -

BASH quick tip: Change to last directory

Using mod_rewrite to prevent hotlinking

dave — Mon, 12 May 2008 07:22:05 +0000

Hotlinking is the practice of linking to an object (generally an image file) from one site (the donor site) onto another (the linking site). We’re frequently asked by our hosting clients if there is a way to stop people doing this as it’s consuming their bandwidth or server resources. This is easily done with the mod_rewrite rules below.

RewriteEngine on RewriteCond %{HTTP_REFERER} !^$ RewriteCond %{HTTP_REFERER} !^http://(www\.)?opensourcery.co.uk/.*$ [NC] RewriteRule \.(gif|png|jpe?g|$ - [F]

This will deny requests for any .gif, .png, .jpg or .jpeg file unless the referrer is either www.opensourcery.co.uk or opensourcery.co.uk.

You can substitute the last line for the one below if you wish to redirect people to an alternative image rather than simply denying the requests.

#RewriteRule \.(gif|png|jpe?g|$ img/dontsteal.gif [L,NC]

It’s worth remembering that the referrer header is optional so although almost every modern browser sends this with each request it’s possible that some won’t do meaning these rewrite rules may result in broken images for some users of your site.

Using mod_rewrite to prevent hotlinking

Installing VMware Tools on Debian Etch

dave — Sat, 26 Apr 2008 11:04:10 +0000

VMware don’t provide a packaged version of VMware tools for Debian, but they do provide the source and a script to build and install it. Installation is started in the usual way, but right clicking the virtual machine in Virtual Centre and choosing “Install/Upgrade VMware Tools”. This will mount a CD image containing the required files.

Then, as root, install the kernel headers and tools required to build VMware tools:

apt-get install autoconf automake binutils cpp gcc linux-headers-$(uname -r) make psmisc

Next, mount the installation media and copy the file to /tmp. Obviously the build number of the archive may differ on your system.

mount /dev/cdrom /mnt cp /mnt/VMwareTools-3.0.2-55869.tar.gz /tmp umount /mnt

Untar the archive and start the installer. You can accept all the defaults here.

cd /tmp tar zxf VMwareTools-3.0.2-55869.tar.gz cd vmware-tools-distrib ./vmware-install.pl

Next, start the configuration program.

vmware-config-tools.pl

Accept all the defaults except for the question: What is the location of the directory of C header files that match your runing kernel? The files will be in /usr/src/linux-headers-version/include - on our system this was /usr/src/linux-headers-2.6.18-6/include.

Finally, remove the existing NIC driver and replace it with the VMware supplied driver. For obvious reasons, this needs to be done from the console.

/etc/init.d/networking stop rmmod pcnet32 rmmod vmxnet modprobe vmxnet /etc/init.d/networking start

Installing VMware Tools on Debian Etch

Scheduled maintenance with mod_rewrite

dave — Sun, 13 Apr 2008 10:45:38 +0000

A public sector customer recently had a requirement to redirect visitors to one of their applications to a holding page overnight while the company who support the application carried out some maintenance work. The work was being carried out between 1am and 7am to minimize disruption to the members of public who use the application.

Keen to help, but also to get a good nights sleep we looked to mod_rewrite to provide a solution (we manage the front end servers, which run Apache).

We added the following lines to our httpd.conf to redirect any traffic to the site to a holding page at /static/offline.php from 00:55 until 07:30 to a holding page:

RewriteEngine on RewriteCond %{TIME} >20080407055000 RewriteCond %{TIME} <20080407073000 RewriteRule !^/static/offline.php$ /static/offline.php [PT,NS]
Alias /static "/var/www/www.example.com/static"

As the application sits on a Tomcat instance behind mod_proxy, we also added the following line to prevent mod_proxy from handling requests for the /static directory.

ProxyPass /static !

We created the offline.php file, including with the line below to ensure that search engines didn’t index the holding page.

header("HTTP/1.1 503 Service Temporarily Unavailable")

a

Scheduled maintenance with mod_rewrite

Parsing parameters in a BASH shell script

dave — Mon, 07 Apr 2008 13:19:57 +0000

This is a simple alternative to using getopts to parse parameters in a BASH shell script which makes use of the powerful parameter substitution functions in BASH. It should be sufficient for most scripts:

until [[ ! "$*" ]]; do if [[ ${1:0:2} = '--' ]]; then PAIR=${1:2} PARAMETER=`echo ${PAIR%=*} | tr [:lower:] [:upper:]` eval P_$PARAMETER=${PAIR##*=} fi shift done

The script processes parameters in the format --name=value or --flag.

So, executing: ./example.sh --number=123 --show

Will result in the variable $P_NUMBER being set to “123″ and the variable $P_SHOW evaluating to true as it is set, albeit to a empty value.

Parsing parameters in a BASH shell script

New website

dave — Sun, 06 Apr 2008 21:03:59 +0000

Our new website is now online.

As you would expect, the site is created wholly using Open Source software. It has been created using the excellent WordPress software and is
hosted on our own LAMP (Linux/Apache/MySQL/PHP) servers.

New website