bash completion for unison

just add the following to you ~/.bashrc or to /etc/bash_completion.d/unison:

_unison() { COMPREPLY=( $(compgen -W "$(find ~/.unison -name '*.prf' -printf '%P ')" -- "${COMP_WORDS[COMP_CWORD]}" ) ); }
complete -F _unison unison

http://www.cis.upenn.edu/~bcpierce/unison/

C# Conditional Execution

[ConditionalAttribute("DEBUG")] for methods. All calls to that method are only executed in Debug builds!

man wrapper: man in own window

Alt-F2, man emacs.

well, this does not work. not automatically.

i created a wrapper-script in ~/bin (which is first in my path variable) with the following content:

#!/bin/bash
# mru, jun 2010

# if ! [ -t 0 ]; then
if [ "$DISPLAY" ]; then
 exec gnome-terminal --hide-menubar --title "man $@" -x /usr/bin/man "$@"
else 
 exec /usr/bin/man "$@"
fi

to make it work.

one could use if ! [ -t 0 ]; then to get a normal in-terminal man if in terminal and a seperate gnome-terminal if in X11.

latexmk super powers

latexmk is a handy tool to build latex documents. these days i investigated in the advanced usage of latexmk.

latexmk can use configuration files, latexmkrc in the current directory is used as a default, then ~/.latexmkrc.

with latexmk -pvc latexmk watches a .tex file and builds it whenever its content or a dependency of it changes. -pvc also opens a viewer for the file and updates it, so it really supports the workflow when editing latex documents.

a sample latexmkrc file might look like that:

$pdf_previewer = 'start evince';
@default_files = ('thesis');
$print_type = 'pdf';
$pdf_mode = 1;
$pdflatex = 'pdflatex -interaction=nonstopmode -halt-on-error';

update: $pdflatex = 'pdflatex -interaction=nonstopmode -halt-on-error'; makes pdflatex stop on the first error, not asking for any user input.

watch-files

just like continuous-make, but executes arbitrary commands when a file changes.

#! /bin/bash

# mru, feb 2010
# if a file changes, execute command

# example:
#
# watch-file document.tex lit.bib latexmk

# 

function error() {
 echo "error: $@"
 echo "${0##*/} {files-to-watch} 'command-upon-modification'"
 exit 1
}

if [ ${#} -lt 2 ]; then
 error "not enough arguments"
fi


while [ $# -gt 1 ]; do
 FILES=(${FILES[*]} "$1")
 shift
done

CMD=$*

for i in "${FILES[@]}"; do
 if [ -z "$i" ] || ! [ -f "$i" ]; then
  error "no such file: $i"
 fi
done

echo "${0##*/}: onchange ("
for i in "${FILES[@]}"; do
 echo " * $i"
done
echo ") { ${CMD} }"

ftime() { stat -c %y "${FILES[@]}"; }

D=$(ftime)
while true; do
 sleep 1
 DN=$(ftime)
 if [ "$D" != "$DN" ]; then
  $CMD
  D="$DN"
 fi
done

continuous make

build-on-save is easy. i put this in my .bash_aliases and use it everywhere!

continuous-make() {
 while true; do
  if ! make "$@" -q; then
   make "$@"
  fi
  sleep 1
 done
}

you can use it like that:

continuous-make doc

and it will call make doc whenever a dependency of doc is changed.

mydoc: documentation management

i have a lot of pdf files lying around. some of these files are datasheets, some manuals. i repeatedly access them, and because usually i can't find them fast enough, i search the internet and download it the thousandth time. but not any longer! mydoc behaves like texdoc, but for my own documents.

it accepts arguments that are used as conjuncted filters.

if more files match the filters, a selection dialog is displayed

it detects if its called from console or from gui (Alt-F2, e.g.).

#! /bin/bash

# search $MYDOCPATH for pdf documents
# commandline arguments are used as conjucted -iname's
#
# mru, may 2010


# configure where to search for the pdf's
MYDOCPATH=~/dev/01atmel/terrarium/docs/:~/dev/01atmel/stepper/docs/:~/books/




set -f  # no globbing, free the star!


# detect if run in console
# the variable TERM is used to automagically call the right function
if [ -t 0 ]; then
 TERM='cli'
else
 TERM='x11'
fi

error_cli() { echo "$*"; }
error_x11() { zenity --warning --text "$*" --title "${0##*/}"; }
info_cli()  { echo "$*"; }
info_x11()  { zenity --info --text "$*"  --title "${0##*/}"; }

usage()     { info_$TERM "usage: ${0##*/} {filter-string}"; } 
open()      { gnome-open "$@"; }

get_choice_cli() {
 PS3=$'more possible matches; select the file to open or change filter: '
 select c in $@ ; do  
  echo $c
  break
 done
}

get_choice_x11() {
 zenity --list \
  --width=800 \
  --height=450 \
  --text "select file" \
  --title "${0##*/}: filter not unique" \
  --column "filename" \
  "$@"
}

if [ -z "$MYDOCPATH" ]; then
 error_$TERM "\$MYDOCPATH not set"
 exit 1
fi


# create the `find` query, search for --help
while (( $# > 0 )); do
 if [ "$1" = "--help" ]; then
  usage
  exit 1
 fi

 QUERY="$QUERY -iwholename *$1*"
 shift
done


# call `find` for all paths in $MYDOCPATH and store the result in
# RESULTS

RESULTS=$(IFS=$': '; for i in $MYDOCPATH; do
 find $i -iname "*pdf" $QUERY
done)

# well, nothing found, RESULTS is empty
if [ -z "$RESULTS" ]; then
 info_$TERM "nothing found; stopping"
 exit 1
fi

# on unique match, open pdf
if (( $(wc -l <<< "$RESULTS") == 1 )); then
 open "$RESULTS"

# multiple matches, open selection
else
 IFS=$'\n'
 c=$(get_choice_$TERM $RESULTS)
 [ "$c" ] && open "$c"
fi

prettyprint bibtex .bib files

some sed code to optimize .bib code

invoke like this: sed -f pretty-bib.sed literature.bib

# pretty-printer for .bib files
# mru, may 2010


# canonical comments
s/^%[ \t]*/% /

# canoical spaces around assignments
s/^[^%][ \t]*\([a-zA-Z0-9]\+\)[ \t]*=[ \t]*/  \1 = /

# double-braces for title and notes
s/title = \({[^{].*}\)/title = {\1}/
s/note = \({[^{].*}\)/note = {\1}/

# remove trailing whitespace
s/[ \t]*$//;

# delete multiple empty lines
/./,/^$/!d

file modification time

To get the access|modification|change time of a file, one could use stat to get the data. For example:

$ stat -c %y ~/.bash_aliases 2010-02-20 17:12:31.161203715 +0100

Of course, this can be easily cutted to get only the date:

$ stat -c %y ~/.bash_aliases | cut -d' ' -f1 2010-02-20

make arch ftw!

I really like using Makefiles to package my projects into .tar.gz files. And I really like those .tar.gz where all the content is contained in a single top-level directory, named exactly as the archive file, so it's easy to delete all files easily when extracted so a crowded directory.

This can be easily achieved using something like this:

TITLE = music-quality-$(shell date +%Y%m%d)

arch:
 tar cvzf $(TITLE).tar.gz \
  --transform 's#^#$(TITLE)/#' \
  --exclude '*~' \
  --exclude plot* \
  --exclude feature_pres \
  src/*.m

.PHONY: arch

multiline sed

Given you have a PostScript file that contains blocks like the following, but with different commands repeatedly:

gsave
/Courier findfont 10 scalefont ISOEncode setfont
0.000 0.000 0.000 setrgbcolor AdjustColor
202 2.8421709430404e-14 [
[(0.4)]
] 12 -0.5 1 0 false DrawText

or

gsave
/Helvetica findfont 10 scalefont ISOEncode setfont
0.000 0.000 0.000 setrgbcolor AdjustColor
10 240 [
[(File: file1.wav   Page: 1 of 3   Printed: Fri Feb 19 18:32:26)]
] 14 -0 0 0 false DrawText
grestore

and you want to remove those blocks that match '(File' in the fifth line of input, you could try this: sed -e '/gsave/ { N; N; N; N; /(File/ { N; N; d; } }'

server backup script

My Server creates daily backups incremental backups to monthly full backups. It will keep the last three daily backups, as well as the last three monthly backups.

#!/bin/bash

# mru, feb 2010

# make backups.
#


DATE_D=$(date "+%Y%m%d")
DATE_M=$(date "+%Y%m")

BACKUPS=/var/backups
TARGET_D=${BACKUPS}/daily_backup_${DATE_D}.tar.gz
TARGET_M=${BACKUPS}/monthly_backup_${DATE_M}.tar.gz
RECEIVER=mru
TAROPTS=" --preserve-permission --label=$DATE_D"

FILES='
root
home
etc
boot
var/trac
var/www
var/mail
var/repositories
'

EXCLUDE_CLAUSE="
--exclude *~
--exclude ${BACKUPS}
--exclude root/.cpan
--exclude root/caldav
--exclude root/tmp
"

trap '{
 echo \"Failed. Stopping.\";
 exit 1;
 if [ -f "$TARGET" ]; then
   rm $TARGET;
 fi
}' ERR

# remove old files

OLD_D=$(find $BACKUPS -name daily_backup_????????.tar.gz | sort -n | head -n -3)
OLD_M=$(find $BACKUPS -name monthly_backup_??????.tar.gz | sort -n | head -n -3)
OLD=${OLD_M}${OLD_D}
echo "$OLD" | xargs --no-run-if-empty rm

# make monthly spinoff

if [ -f $TARGET_M ]; then
    TAROPTS+=" --newer $TARGET_M"
    TARGET=$TARGET_D
else
    TARGET=$TARGET_M
fi


# create archive
# chdir to / to get the right globbing
( cd / && tar czf "$TARGET" $TAROPTS $EXCLUDE_CLAUSE $FILES )


chmod og-r "$TARGET"

# send report

mail -s "Backup Message for ${HOSTNAME} $(date)" ${RECEIVER} <<EOF
Backup created.

Filename: $TARGET
size:     $(ls -lah $TARGET)

$(if [ -n "$OLD" ]; then echo "==== removed these files: $OLD"; fi)

==== archive content $TARGET:
$(tar tzf $TARGET)

==== folder content $BACKUPS:
$(find ${BACKUPS})
EOF

server status email

My server sends me an email about it's status every day:

Key Fingerprint to Wikimedia Commons Image

It's been some time, since my last serious python code, so excuse the mess ;) This will take any key fingerprint, for e.g. from ssh, in a hexadicaml format and show an image from commons.wikimedia.org which represents the key. The script will hash the key and does some sort of table lookup. I still need to implement some double hashing, when the matching image has a too high time offset, to eliminate some collisions. The current implementation also can't really deal with deleted images, tho I came up with some workarounds to that already.

#!/usr/bin/python

# convert ssh key figerprint to wikimedia commons image

import os
import sys
import commands

from datetime import datetime
from datetime import timedelta

import re

from xml.sax import saxutils
from xml.sax import make_parser
from xml.sax.handler import feature_namespaces


def convertMonth(ds):
    """ Replaces all written out months in a string with a xx number."""
    # don't depend on locals
    ds = ds.replace("January", "01") 
    ds = ds.replace("February", "02") 
    ds = ds.replace("March", "03") 
    ds = ds.replace("April", "04") 
    ds = ds.replace("May", "05") 
    ds = ds.replace("June", "06") 
    ds = ds.replace("July", "07") 
    ds = ds.replace("August", "08") 
    ds = ds.replace("September", "09") 
    ds = ds.replace("October", "10") 
    ds = ds.replace("November", "11") 
    ds = ds.replace("December", "12") 
    return ds



class ImageParser(saxutils.DefaultHandler):

    def __init__(self):
       self.inThumb = False
       self.dateElement = False
       self.thumbs = []

    def startElement(self, name, attrs):

        if self.dateElement:
            self.dateElement = False

        if name == 'div':
            if attrs.get('class', None) == 'thumb':
                self.inThumb = True

        if self.inThumb:
            if name == 'img':
                self.thumbs.append([attrs.get('src', None)])
            if name == 'i':
                self.dateElement = True 
                self.inThumb = False

    def characters(self, content):
        if self.dateElement:
            date = datetime.strptime(convertMonth(content), "%H:%M, %d %m %Y")
            self.thumbs[len(self.thumbs)-1].append(date)

    def endDocument(self):
        self.thumbList = []
        for l in self.thumbs:
            if l[0].find('http') != -1:
                if len(l) == 2:
                    self.thumbList.append(l)


class HashFunction:

    def __init__(self, startDate, duration, ticSize):
        """ Provides a Hash function, which takes any integer
        and returns a date within a given interval.

        Keyword Arguments:
        startDate -- the maximum date in the time interval, a datetime object
        duration  -- the positiv length of the timer interval, a deltatime object
        ticSize   -- the tic size in seconds is the unit in which the interval is divided
        """
        self.startDate = startDate
        self.duration = duration
        self.ticSize = ticSize

        seconds = duration.days*24*60*60 + duration.seconds
        self.numHashes = seconds / ticSize

    def h(self, x):
        """ Converts an Integer to a date in the configured period,
        based on a hashfunction.
        """
        h1 = x%self.numHashes
        second = h1 * self.ticSize
        return self.startDate - timedelta(0, second)
        
    def hFormat(self, x):
        """ Converts an Integer to a date in the configured period,
        based on a hashfunction. The date is formated string.
        """
        dt = self.h(x)
        return dt.strftime("%Y%m%d%H%M%S")

        
class KeyImage:

    def __init__(self):
        self.hf = HashFunction(startDate, duration, resolution)

    def __init__(self, startDate=datetime(2010, 2, 1, 12, 00, 00), duration=timedelta(days=365*4), resolution=60):
        self.hf = HashFunction(startDate, duration, resolution)

    def getImage(self, x, size=-1):
        """ Converts an Integer value to an image url from wikimedia commons."""
        # hasing 
        imageDate = self.hf.hFormat(x)

        # download html file
        url = "'http://commons.wikimedia.org/w/index.php?title=Special:NewFiles&from=" + imageDate + "'"
        htmlFilename = "/tmp/commons_thumbs.html"
        commands.getstatusoutput("wget -E " + url + " -O " + htmlFilename)

        # parse xml file
        parser = make_parser()                  
        ip = ImageParser()
        parser.setFeature(feature_namespaces, 0)
        parser.setContentHandler(ip)
        file = open(htmlFilename)
        parser.parse(file)

        # remove temporary html file
        commands.getstatusoutput("rm " + htmlFilename)

        imageUrl = ip.thumbList[0][0]
        if (size > -1):
            imageUrl = re.sub("[0-9]{2,3}px", str(size) + "px", imageUrl)

        return imageUrl
        

def main():
    skey = sys.argv[1].replace(".","").replace(":","")
    try:
        key = int(skey, 16)
    except:
        sys.stderr.write("[ERROR] Invalid Key")
        sys.exit(1)

    thumbSize = -1
    if len(sys.argv) >= 3:
        try:
            thumbSize = int(sys.argv[2])
        except:
            sys.stderr.write("[ERROR] Invalid Size")
            sys.exit(1)

    ki = KeyImage()
    print ki.getImage(key, thumbSize)

main()

Here is a short wrapper script, which downloads the image and displays it with feh.

#!/bin/bash

IMAGEURL=`key_to_thumb.py $1 $2|tail -1`
echo $IMAGEURL
TMPFILE="/tmp/kthumb.img"
wget "$IMAGEURL" -O$TMPFILE &> /dev/null
feh $TMPFILE
rm $TMPFILE

Tor Network Status

This will pull a csv file with data from all Tor nodes running in the network and print basic bandwith statistics. I wrote this script to estimate, if adding more non-exit nodes to the tor network could speed it up, or if the exit nodes bandwith is the bottle neck anyway.

#!/bin/bash

wget http://torstatus.kgprog.com/query_export.php/Tor_query_EXPORT.csv -P/tmp &> /dev/null

BANDWITH=`cat /tmp/Tor_query_EXPORT.csv |sed 1d|cut -d, -f3`
EXIT=`cat /tmp/Tor_query_EXPORT.csv |sed 1d|cut -d, -f10`
rm /tmp/Tor_query_EXPORT.csv

NUMNODES=0
NUMEXITS=0

BEXIT=0
BNOEXIT=0
BTOTAL=0

while read -u3 bandw; read -u4 exit 
do 
    BTOTAL=$(($BTOTAL+$bandw))
    NUMNODES=$(($NUMNODES+1))

    if [ $exit = '1' ]
    then
        BEXIT=$(($BEXIT+$bandw)) 
        NUMEXITS=$(($NUMEXITS+1))
    else
        BNOEXIT=$(($BNOEXIT+$bandw)) 
    fi
done 3<<<"$BANDWITH" 4<<<"$EXIT"

echo "Active Tor Routers:" $NUMNODES
echo "Exit Nodes" $NUMEXITS "("$(($(($NUMEXITS*100))/$NUMNODES))"%)"
echo "Bandwith Exits: " $BEXIT "kB/s" "("$(($BEXIT/1024))" MB/s)"
echo "Bandwith rest:  " $BNOEXIT "kB/s" "("$(($BNOEXIT/1024))" MB/s)"
echo "Bandwith total: " $BTOTAL   "kB/s" "("$(($BTOTAL/1024))" MB/s)"

Multiple VPN Tunnels, Gentoo Style

Let tunname be the specific VPN tunnel to be started, using /etc/openvpn/tunname.conf as the config file. Then the required initscripts are created and started as follows:

# creating 
ln -s /etc/init.d/openvpn /etc/init.d/tunname

# starting 
/etc/init.d/tunname start

However, since starting the default initscript (/etc/init.d/openvpn) will cause a failure if no file named openvpn.conf is present, I'd strongly recommend to delete it from the default runlevel ;)

This is caused by the init script itself, as it determines the config solely by its name. Therefore the default initscript (/etc/ini.d/openvpn) will look for the aforementioned config file (/etc/openvpn/openvpn.conf) and report an error if it couldn't be found.

To start the tunnel on boot, the usual rc-update call will do the trick.

Cheers!

favorite man pages

I wondered what my favorite man pages would be. A little awk'ing shows it:

26 man bash
9 man find
8 man sed
8 man mplayer
8 man itoa
8 man dot
8 man cpio
7 man sleep
6 man read
6 man maxima
5 man xterm
5 man wget
5 man bc
5 man awk
4 man transcode
4 man sprintf
4 man script
4 man scp
4 man pushd
4 man popd
4 man ncftp
4 man latexmk
4 man gimp
4 man basename
3 man sudoers
3 man grep
3 man gcal
3 man cut
2 man wc
2 man units
2 man tee
2 man shutdown
2 man sh
2 man set
2 man pdfcrop
2 man paste
2 man netstat
2 man join
2 man cat
2 man cal
1 man wodim
1 man watch
1 man uname
1 man trap
1 man tar
1 man tail
1 man select
1 man quote
1 man profile
1 man patch
1 man nc
1 man mplayer 
1 man lsof
1 man ln
1 man hexdump
1 man head
1 man git-tag
1 man git-gui 
1 man git-commit
1 man git
1 man getopt
1 man fdevopen
1 man emacs
1 man display
1 man dijkstra
1 man diff
1 man convert
1 man col
1 man cmp
1 man ImageMagick

This is the output of:

awk '/^man / { cmds[$i] ++ } END { for (i in cmds) print cmds[i] " " i }' .bash_history | sort -nr

My bash history is configured as:

export HISTCONTROL=erasedups
export HISTSIZE=10000
shopt -s histappend

It would probably look a lot different if erasedups would not be set.

Unison

Unison is still maybe the only free and useable unidictional synchronisation tool out there, tho itself far from perfect (mind-boggling stupid sometimes). Here's a little script, which makes usage more practical. The basic setup is, that the user stores all his synced data in ~/data and will syncronize to a central node with unison over ssh. The remote server stores it's synced data on an encrypted partition, which is only unlocked while syncronization is in progress. Note that one node doesn't have to sync to the entire share. You can specify a subset in the dirlist (and also matching templates, so new directories will be added automatically).

#!/bin/bash

#configuration
host="remote.host.org"
localdir="/home/user/data/"
remotedir="/srv/sync/data/"

#objects to sync
subdirs="foodir
bardir
buz/bum
`cd $localdir && ls -d moo/bar/be*`
"
#subdirs="`cd $localdir && ls -d *`"


read -s -p "Passowrd: "
pass=$REPLY
if ! ssh "$host" sudo mount_sync.sh -m <<EOF  ###!!!!! cms error with brackets
$pass
EOF
then
    echo "ABORTING: Remote mount failed." 
#exit 1
fi

for dir in $subdirs
do
    unison-2.13.16 -rsync -perms -1 -path $dir "$localdir" ssh://"$host"/"$remotedir"
    echo "$localdir"$dir ssh://"$host"/"$remotedir"$dir
done

if ! ssh "$host" sudo mount_sync.sh -u
then
    echo "WARNING: Closing crypted partition failed!!!"
    exit 1
fi

On the server side, you need a script to open the partition and mount it (mount_sync.sh), something like this will do.

#!/bin/bash

# configution
encdev="/dev/sdb1"
mountdir="/srv/sync/"


if [ "$1" = "-m" ]
then
    if [ ! -b $encdev ]
    then
        echo "ABORTING: Can't find block device" $encdev
        exit 1
    fi

    if ! cryptsetup isLuks $encdev
    then
        echo "ABORTING: Device doesn't include a lux partition."
        exit 1
    fi

    if ! cryptsetup luksOpen $encdev sync_part0
    then
        echo "ABORTING: Failed to open ecrypted device."
        exit 1
    fi

    if ! mount /dev/mapper/sync_part0 $mountdir
    then
        echo "ABORTING: Mount failed."
        exit 1
    fi
fi

if [ "$1" = "-u" ]
then
    if ! umount $mountdir
    then
        echo "ABORTING: Failed to unmount."
        exit 1
    fi

    if ! cryptsetup luksClose sync_part0
    then
        echo "ABORTING: Failed to close encrypted disc"
        exit 1
    fi
fi

And a sudoers config to allow this script for your login user.

loginuser   ALL=NOPASSWD: /bin/mount_sync.sh

gentoo: list installed packages

This commandline lists the installed packages (with version) and prefixes every entry with an '=':

find /var/db/pkg -maxdepth 2 -mindepth 2 | sed -e 's/^\/var\/db\/pkg\//=/'

The output looks like that:

=sys-devel/automake-1.9.6-r2
=sys-devel/autoconf-2.63-r1
=sys-devel/icecream-0.9.2
=sys-devel/automake-wrapper-3-r1
=sys-devel/bison-2.3

• Gentoo 10.0 Desktop

Different Color Schemes per Terminal in VIM

I use fluxbox as an window manager and use different terminal settings, looks and fonts for different working situations. Something I found pretty useful is to also adapt parts of my vim config and colorscheme to these settings. So I got different shortcuts for terminals with for e.g. big dark on white and one with small white on black letters. Based on those I set different vim color schemes. Nothing fancy, but still,... Just export an environment variable before you execute the terminal and check in vimrc. For e.g. in a fluxbox keys file:

Mod4 g :ExecCommand export VIMCLS=dark && gnome-terminal

And in a vimrc

if "$VIMCLS" == "dark"
    set t_Co=256
    colorscheme Mustang_Vim_Colorscheme_by_hcalves
endif

Booting on x86

Just the absolute basic stuff you need to boot on any x86 hardware (or in this case in qemu). For more info see osdev wiki. You will need the following files:

• linker.ld -- to configure your ld
• loader.s -- loader
• kernel.c -- kmain
• grub/stage1
• grub/stage2

Copy the linker.ld and loader.s files from the osdev wiki page. A simple kernel.c, which would write to the framebuffer could look like this:

#define CBLACK 0x00
#define CBLUE  0x01
#define CGREEN 0x02
#define LIGHT(x)  ((x)+8)

void clear_screen() {
    unsigned char *vr = (unsigned char *) 0xb8000;
    for (int i=0; i<4000; i+=2) {
        *(vr + i) = 0;
    }
}

void print(char *text, int x, int y, unsigned char fcolour, unsigned char bcolour) {
    unsigned char *vr = (unsigned char *) 0xb8000;
    vr = vr + (y*160) + (x*2);
    for (int i=0; *text; i+=2,text++) {
        *(vr + i) = *text;
        *(vr + i+1) = (bcolour << 4) | (fcolour & 0x0F);
    }
}

void kmain( void* mbd, unsigned int magic ) {
    clear_screen();
    print("Kernel is running now,...", 7, 3, LIGHT(CGREEN), CBLACK);
}

Now you need grub stage1 and stage2, you can grab these from most linux distros in /boot/grub/. To get this running now, here's a little shell script.

#!/bin/bash

############# CONFIG ###############
IMAGENAME="myos.img"

#the grup bins
STAGE1="grub/stage1"
STAGE2="grub/stage2"

#kernel starting block
KBLOCK=220
###################################

function ceil() {
        par=$1
        c=${par/.*}
        echo $(($c+1))
}


#build
nasm -f elf -o loader.o loader.s
gcc -std=c99 -c kernel.c -o kernel.o -Wall -nostdlib -nostartfiles -nodefaultlibs

#link
ld -T linker.ld -o kernel.bin loader.o kernel.o

#create pad
read STAGE1_SIZE varfoo<<<`du -bs $STAGE1`
read STAGE2_SIZE varfoo<<<`du -bs $STAGE2`
GRUB_SIZE=$(($STAGE1_SIZE+$STAGE2_SIZE))
BKB_SIZE=$(($KBLOCK*512))
PAD_SIZE=$(($BKB_SIZE-$GRUB_SIZE))

dd if=/dev/zero of=pad bs=$PAD_SIZE count=1 &> /dev/null

#make image
cat grub/stage1 grub/stage2 pad kernel.bin > $IMAGENAME

#print grub commands
read KERNEL_SIZE varfoo<<<`du -bs kernel.bin`
KERNEL_BLOCKS=`echo "scale=2; "$KERNEL_SIZE"/512"|bc`
KERNEL_BLOCKS=`ceil $KERNEL_BLOCKS`
echo ---------------------------
echo Now enter on GRUB cmd line:
echo kernel $KBLOCK+$KERNEL_BLOCKS
echo boot
echo ---------------------------

#run qemu
qemu  -boot a -fda $IMAGENAME

get ip of interface

The IP of an interface? Nothing easier than that:

ifconfig wlan1 | sed -n -e 's/.*inet addr:\([0-9]\+.[0-9]\+.[0-9]\+.[0-9]\+\).*/\1/p;d'

get default interface

If one want's to know which interfaces is connected to the default gateway, one might find out:

route -n | sed -n -e 's/^0.0.0.0\(.*[[:blank:]]\)\+\([a-z0-9]\+\)/\2/p' 

or

route -n | grep -E "^0.0.0.0" | sed -e 's/[[:blank:]]\+/ /g' | cut -d' ' -f8

Bluesocket Login

Many WLAN hotspots or networks will redirect any http request, after you connected, to some sort of login page or legal disclaimer. Here is a short script which will authenticate to bluesocket, a "widely-used proprietary wlan managment solution".

#!/bin/bash

IP=`ifconfig wlan0 |grep "inet addr:"|cut -d: -f2|cut -d\  -f1`
USER="username"
PASS="password"

curl -d require_terms_g="1" -d bs_name=${USER} -d bs_password=${PASS} \
-d _FORM_SUBMIT=1 -d which_form=reg -d source=${IP} https://loginpage.foo/login.pl &> /dev/null

You may not require the variable require_terms_g.

Get IP

A simple script, which parses the content from, whatismyip.com and prints out the IP address. Is compatible with busybox (sh, nc, tail), to be used on embedded systems.

#!/bin/sh

get_ip_() {
    busybox nc www.whatismyip.com 80 << EOF
GET /automation/n09230945.asp HTTP/1.0
User-Agent: netcat
Accept: */*
Host: www.whatismyip.com
Connection: close

  
EOF
}
get_ip() {
    get_ip_ |tail -1
}

get_ip
echo

posting tips

Since this blog engine is really stupid and bad, a few special settings had to be set:

• only html editor available
• no automatic <br> - use <p> instead!
• when posting code, use the <pre class="prettyprint"> tag. The tag can also contain the language, class="prettyprint lang-bash" for example.

  <p><pre class="prettyprint lang-bash">
  #! /bin/bash
  do_something_here
  </pre>
  

• if the code contains symbols that would be interpeted as HTML, (<, >, &,...) use CDATA around the code:

  <p><pre class="prettyprint lang-bash"><![CDATA[
  #! /bin/bash
  do_something_here <<EOF
  something
  EOF
  ]]></pre>
  

• for keyboard inputs, use the <kbd> tag.
• the "Configuration" block at the end of some posts is written like that:

  <p><div class="configbox">
  <ul>
  <li> Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
  <li> GNU bash, version 4.0.35(2)-release (i686-pc-linux-gnu)
  <li> lircd 0.8.5
  </ul>
  </div>
  

hf!

Random mac per day and device

This bash script lets you select a random mac address per day and device. I found useful in many networks, which require authentication and lock a session to one mac address.

#!/bin/bash

#------------------------------------------------#
# Aurel, 20 May 2008                             #
#------------------------------------------------#
# This script chooses a random mac but only one  #
# per day.                                       #
# Usage: changemac                   #
#------------------------------------------------#
# Copyrigth: public domain                       #
#------------------------------------------------#

# configuration
logfile="/var/log/todays_mac_"

device=$1

if [ ! -f $logfile$device ]
then
    touch $logfile$device
    echo none > $logfile$device
fi

if read -r line
then
    mac_day=`echo "$line"|cut -d\  -f1`
    mac_address=`echo "$line"|cut -d\  -f2`
    today=`date "+%j"`

    if [ "$mac_day" = "$today" ]
    then
        macchanger -m  $mac_address $device
    else
        macchanger -r $device
        cmac=`macchanger -s $device|cut -d\  -f3`
        echo `date "+%j"` $cmac > $logfile$device
    fi
fi < $logfile$device

tarjans algorithm to find SCC's in digraphs

An implementation of Tarjan's algorithm in python:

#! /usr/bin/python

# mru, 2010

# tarjan's algorithm to find strongly connected components, as
# presented in "Algorithms in C++" by Sedgewick.

# example graph:
nodes = range(10)
graph = [ 
    (0, 1), (0, 6), (0, 8), 
    (1, 1), (1, 7), (1, 2),
    (2, 7), (2, 3),
    (3, 4), (3, 7),
    (4, 5),
    (5, 6), 
    (6, 4),
    (7, 6), (7, 3),
    (8, 9), 
    (9, 8), (9, 5)]

def scr(w):

    global cnt, low, id, pre, scnt, S

    low[w] = cnt
    pre[w] = cnt
    min = cnt
    cnt = cnt + 1

    S.append(w)
    
    for t in [ t for (s,t) in graph if s == w ]:
        if pre[t] == -1:
            scr(t)
        if low[t] < min: 
            min = low[t]

    if min < low[w]:
        low[w] = min
        return

    t = S.pop()
    id[t] = scnt
    low[t] = len(nodes)

    while t != w:
        t = S.pop()
        id[t] = scnt
        low[t] = len(nodes)
    
    scnt = scnt + 1

S = []
cnt = 0
scnt = 0
pre = [ -1 ] * len(nodes)
low = [ 0 ] * len(nodes)
id = [ 0 ] * len(nodes)

for i in nodes:
    if pre[i] == -1:
        scr(i)

for index in range(max(id)+1):
    print "nodes in scc ", index, ": ", [ j for (i,j) in map(None, id, nodes) if i == index ]

The output looks like this:

nodes in scc  0 :  [4, 5, 6]
nodes in scc  1 :  [3, 7]
nodes in scc  2 :  [2]
nodes in scc  3 :  [1]
nodes in scc  4 :  [8, 9]
nodes in scc  5 :  [0]

The example graph in the code corresponds to this graph:

The algorithm is taken from the very good book "Algorithms in C++" by Sedgewick, therefore the implementation style is not very python-styled.

poweroff with lirc

I keep trying to use my remote control to do things, but i always forget where i put it.

Anyways, i did this little scriptie to power off my computer when i press a remote-control button several times within a specified time period.

#! /bin/bash

# use with irexec
# if a button was pressed N times with less the DT seconds between 
# the presses, EXEC will be executed

# mru, dez 2009


SAVE="$HOME/.lirc-poweroff"
UPTIME="$(cut -d' ' -f1 /proc/uptime)"
EXEC="sudo shutdown -h +5"

DT=1
N=2

touch "$SAVE"

while read LAST; do

 RES=$(bc <<< "scale=10; ($UPTIME-$LAST) > $DT")

 if [ "$RES" == "1" ]; then
  : > "$SAVE"
  break
 elif (( "$N" == 0 )); then
  $EXEC
  : > "$SAVE"
  exit 0
 fi

 N=$(( $N - 1 ))

done < "$SAVE"
  
echo "$UPTIME" >> "$SAVE"

The script creates a file $SAVE and stores the time when the button was pressed last time. If a subsequent press is close (in respect to $DT), the entry will be appended to the file. If the file reaches a linecount of $N, $EXEC will be executed.

Here is the entry in my ~/.lircrc:

begin
  button = KEY_BACK
  prog = irexec
  config = ~/bin/lirc-poweroff.sh
  repeat = 0
end

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU bash, version 4.0.35(2)-release (i686-pc-linux-gnu)
• lircd 0.8.5

jpg to avi

Wouldn't it be nice if one could easily convert a bunch of jpegs to a .avi movie? Have a look at that:

#! /usr/bin/make -f                                                                                                                         

# mru, 2009-04-05                                                                                                                           

# make [TARGET=output.avi] [PATTERN='*.jpg'] [SOUND=] [FPS=20]                                                                              

# you have to define TARGET and PATTERN on cli                                                                                              
# SOUND is optional; not defined -> not used                                                                                                
#                                                                                                                                           
# FPS..     frames per second  [ default = 20 ]                                                                                             
# TARGET..  output file name for avi [ output.avi ]                                                                                         
# PATTERN.. wildcard for all jpegs to use [ *.jpg ]                                                                                         
# SOUND..   filename of .wav file as soundtrack  [ optional ]                                                                               

# make play [TARGET=output.avi]                                                                                                             
# make clean                                                                                                                                
# make resize [RESOLUTION=1024x768] [-j 4]                                                                                                  


# requires MJPEG tools                                                                                                                      


FPS = 20
PATTERN = *.jpg
TARGET = output.avi
RESOLUTION = 1024x768

all: play
    ls -lah $(TARGET)

# creates a string with newline between every element of wordlist on stdin                                                                  
MAKENEWLINE = tr " " "\n"
FILELIST = $(wildcard $(PATTERN))

%.avi: $(FILELIST)
    @tr ' ' '\n' <<< "$?" | jpeg2yuv -v 0 -I p -f $(FPS) | yuv2lav -f a -b 10000 -o silent.avi
ifdef SOUND
    @lavaddwav silent.avi $(SOUND) $@ 
    @rm silent.avi
else
    @mv silent.avi $@
endif


play: $(TARGET)
    @mplayer $(TARGET)

clean:
    @rm -f $(TARGET)

resize: $(FILELIST:.jpg=.res)

# i love makefile parallelism, try -j 4 !!!                                                                                                 
%.res: %.jpg
    convert $< -resize $(RESOLUTION) $<

.PHONY: all play clean resize

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU bash, version 4.0.35(2)-release (i686-pc-linux-gnu)
• GNU Make 3.81

makeclean: clean typical temporary files

I often want to clean my home directory, just to get rid of all the temporary files i left during my escapades.

This script does it nicely, informative and interactive:

#! /bin/bash                                                                                                                                

# mru, 2009-something                                                                                                                       

# typical clean situation... erase temp, bak - files, recursively from                                                                      
# the current directory.                                                                                                                    
#                                                                                                                                           
# which files to delete is configured in the variable FILTER:                                                                               

FILTER=("*.aux" "*~" "*.bak")



# i can't use the globbing, turn it off                                                                                                     
set -f

join_cond() {
    echo -name "${FILTER[0]}";
    for i in $(seq 1 $((${#FILTER[@]} - 1))); do
        echo -or -name "${FILTER[i]}"
    done
}

FILES=$(find . "(" $(join_cond) ")" -print )

if [ -z "$FILES" ]; then
    echo "No files to clean found. Stopping."
    exit 0
fi

echo "Considering these files:"
echo

while read filename; do
    echo " - $filename"
done <<< "$FILES"

read -p "really delete [y/N]?"
if [ "$REPLY" == "y" ]; then
    while read filename; do
        rm -v "$filename"
    done <<< "$FILES"
    echo "Done"
else
    echo "Cancelled"
fi

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU bash, version 4.0.35(2)-release (i686-pc-linux-gnu)

sed on multiple files

To replace strings in multiple files, i use this commandline:

sed -e $EXP -i $FILEGLOB

For example:

sed -e 's/old_identifier/new_identifier/g' -i *.c

It will execute the expression on every file, performing inplace changes (= -i). An optional backup argument could be supplied to the -i argument to create backup files first.

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU sed version 4.2

poweroff after mplayer

For those who use the non-gui version of mplayer and like to junk movies before bed-time it might be useful to shutdown the computer when the movie is done. I do this with this little script:

#! /bin/bash                                                                    

while pidof mplayer; do
        sleep 1
done

sudo poweroff

As soon as mplayer quits, the computer will shut down.

Clearly, the user must be allowed to sudo the poweroff command.

Another way to do it would be something like this:

watch "pidof mplayer || (poweroff; kill -15 $$)"

But who would take the easy way?

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU bash, version 4.0.35(2)-release (i686-pc-linux-gnu)
• MPlayer SVN-r29796-4.3.4 (C) 2000-2009 MPlayer Team

removing ansi escape codes from typescript

I like using the script tool, included in the util-linux-ng project, http://www.kernel.org/pub/linux/utils/util-linux-ng/.

But all the escape codes make the files unreadable in less, for example.

This small sed script removes some escape codes. I did not search for a spec of the codes, so i don't know what all could get wrong:

#! /bin/sed -f

# removes escape sequences from "script" typescripts
# read about "util-linux[-ng]" for more info
 
# consider using -i to filter "in-place"
# `man script` for more information...
 
# mru, dez. 2009

s/\r//g
s/^[\]0;//g
s/^G//g
s/^[\[\(\([[:digit:]]*;\|?\)\?[[:digit:]]*\)\?[Kmh]//g

Something is missing: the output will be wrong for some inputs. The codes for Ctrl-C, Ctrl-W, ... should be interpreted to reconstruct the real output as seen in the shell. Well, nothing is perfect. I think the tool col (also util-linux-ng would do the job. man col would give the details.

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU sed version 4.2

shared emacs config

One might want to share an emacs config across machines. An easy way to make conditional configuration would be to move machine specific configuration to another file, like .emacs-local. This solution is not optimal for people like me that synchronize configs a lot. A nicer solution follows. Emacs reads the hostname and executes a block of code if an entry for that name exists:

(defun load-local-conf ()
  ;;  (interactive)
  (let ((host (system-name)))
    (cond ((string= host "mru-laptop")
           (set-default-font "-unknown-DejaVu Sans Mono-normal-normal-normal-*-16-*-*-*-m-0-iso10646-1"))
          ((string= host "knuth.olymp")
           (print "hello from knuth")))))

(load-local-conf)

Also, one could test for filenames with the hostname, lets say machine knuth saves its Emacs config in .emacs-knuth.

(defun load-local-conf ()
  (interactive)
  (let ((filename (concat "~/.emacs-" (system-name))))
    (if (file-readable-p filename)
        (load filename)
      nil)))

• Linux knuth 2.6.31-gentoo-r6 #1 SMP Thu Nov 26 18:01:11 CET 2009 i686 Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
• GNU Emacs 23.1.1