Linux, Unix, /etc

Danger Will Robinson! You are now entering a condescending Unix user zone!
Sponsored links (requires javascript):

Scripts: Networking

ip: get current ip address

ippp0 is my Net device; adjust to suit. Obviously, I'm generally only interested in this ip, because it's the only one that changes. This simple script is useful in getting that IP address for other scripts that need to know it e.g. those that update my dynamic DNS. 

#!/bin/sh
ifconfig ippp0 |
sed -n 's/.*inet addr:\([0-9.]*\).*/\1/p'

domain.pl: regexp to match a domain name

Tried this in grep, sed and awk, but perl is the only one that works in full generality. The awk version came second, but would only match a domain name if that was all there was on the input line. 

 
#!/usr/bin/perl -n
while (m/(([-a-zA-Z0-9]+\.[-a-zA-Z0-9]+)(\.[-a-zA-Z0-9]+)*)/g) {
    print "$1\n"; 
}

ftpfile: simple non-interactive ftp

Modified version of the original from Unix Power Tools. 

 
#! /bin/sh
# ftpfile: get a file ($2) from an ftp site ($1), or interpret a url 
# to do the same
PATH=$PATH:/usr/local/bin

progname=`basename $0`
url=0
case $# in
    0) 1>&2 echo "$progname: usage $progname ftpsite file or $progname url"; exit 1;;
esac

url=`echo $1|sed 's/\(ftp:\/\/\).*/\1/'`
case $url in
    "ftp://") isurl=1 ;;
    "*") 1>&2 echo "$progname: usage $progname ftpsite file"; exit 1;;
esac
if [ "$isurl" -gt 0 ]
then
    SOURCE=`echo $1|sed 's/ftp:\/\/\([^/]*\)\/.*/\1/'`
    FILE=`echo $1|sed 's/ftp:\/\/\([^/]*\)\/\(.*\)/\2/'`
else
    SOURCE=$1
    FILE=$2
fi

case $USER in
    "") USER='paul' ;;
esac

BFILE=`basename $FILE`
if [ -f /tmp/$FILE ]
then
    op='reget'
else
    op='get'
fi

case $SOURCE in
    ftp.demon.co.uk) ftpuser='hostname'; ftppassword='password' ;;
    *) ftpuser='anonymous'; ftppassword="$USER@" ;;
esac

ftp -n $SOURCE <<EndFTP
user $ftpuser $ftppassword
binary
$op $FILE /tmp/$BFILE
EndFTP

dilbert: get the latest Dilbert cartoon

The only complexity here is introduced by United Media not giving a direct link to the image file, in a sinister attempt to stop leeches like myself downloading it. Note how much simpler the doonesbury script is. 

 
#!/bin/sh
export http_proxy=http://localhost:80/
dilberthome=$HOME/library/700/790/humour/dilbert
public=/usr/local/etc/httpd/htdocs/
cd /tmp
rm index.html
set `date '+%d %m %y'`
day=$1
month=$2
year=$3
# first get the main page — this gives us the URL for the picture
if lynx -source http://www.unitedmedia.com/comics/dilbert/index.html > index.html
then
	:
else
	exit 1
fi
# look for something like dilbert98081090304.gif
picture=`egrep 'dilbert[0-9]*\.gif|dilbert[0-9]*\.[jg][ip][fg]' index.html | head -1 | sed 's/.*\(dilbert[0-9]*\.[jg][ip][gf]\).*/\1/'`
if lynx -dump http://www.unitedmedia.com/comics/dilbert/archive/images/$picture > $picture
then
	cp $picture $public/dailydilbert
	chmod 644 $public/dailydilbert
	echo y | mv $picture $dilberthome
	ln -s -f $dilberthome/$picture $HOME/dailydilbert
	ln -s -f $dilberthome/$picture $dilberthome/`todaysdate`
	exit 0
else
	exit 2
fi
exit 0

doonesbury: get the daily comic

#!/bin/sh
#doonesbury: get the daily comic
cd $HOME
# url is of the form http://images.ucomics.com/comics/db/2002/db021026.gif
date=$(date +'%y%m%d')
year=$(date +'%Y')
if wget -q http://images.ucomics.com/comics/db/"$year"/db"$date".gif 
then
    mv db"$date".gif dailydoonesbury
    echo 'Doonesbury is here!' | wall
fi

get-howtos: retrieve a set of the latest Linux HOWTO docs

I don't do this anymore, so not sure if this still works. But the basic idea should be obvious. 

 
#!/bin/sh

dir=/home/paul/library/computing/os/unix/flavours/linux/LDP/howto/
tmpdir=/tmp
url1=http://metalab.unc.edu/pub/Linux/docs/HOWTO
url2=http://metalab.unc.edu/pub/Linux/docs/HOWTO/mini
file1=Linux-HOWTOs.tar.gz
file2=Linux-mini-HOWTOs.tar.gz

cd $tmpdir
lynx -dump $url1/$file1 > $file1
lynx -dump $url2/$file2 > $file2
cd $dir
for i in $file1 $file2
do
    gzip -dc $tmpdir/$i | tar xf -
done

chkpop: Cheap and cheerful way to check a POP3 mailbox

nc, Note: "netcat", is a very useful little program; make it part of your toolbox. 

#!/bin/sh
#chkpop: check for mail in a pop3 mailbox
case $1 in
"")
    1>&2 echo "chkpop: chkpop <server>"
    exit 2
    ;;
esac
line=$(awk '/'$1'/ {print $2, $4, $6}' < $HOME/.netrc)
if [ "$line" = "" ]
then
    exit 0
fi
set x $(echo $line)
server=$2
login=$3
password=$4
(echo user $login;echo pass $password;echo quit) |
nc $server 110 |
tee /tmp/chkpop.server_responses |
awk '
	/\+OK .*([0-9][0-9]*).*messages.*/{
	if (match($0,/[1-9]/))
		{print "you have mail on '$server'"; exit 0}
	else
		{exit 1}
	}
'
exit $?

urlminder: notify via e-mail when a url has changed

This simple system has a number of components. First, the inevitable shell script! 

#!/bin/sh 
#urlminder: a generalised version of leacom, stcpairings etc
#limited to notification of change
#urls are stored in a file, one url per line, along with who to tell
#caveat: lynx returns 0 exit status even if it can't find the page!
cd /tmp || exit 1
list=/usr/local/etc/urlminder_list
for item in `sed -e '/^#/d' -e 's!http://!!' < $list`
do
	url=$(echo $item | awk 'BEGIN{FS=":"}{print $1}')
	tell=$(echo $item | awk 'BEGIN{FS=":"}{print $2}')
	file=$(echo "$url" | sed 's!/!.!g')
	if [ ! -f "$file".old ]
	then
		lynx -dump $url > "$file"
	else
		lynx -dump $url > "$file"
# note: unfortunately, lynx returns 0 error status even if it couldn't
# retrieve the file.  checking to see if the file was actually retrieved,
# or if "$file" actually contains an error message, is more trouble than 
# it's worth.
#		if [ $? = 0 ]
#		then
			diff "$file" "$file".old > /dev/null 2>&1 
			case $? in
				0) ;;
				1) mail -s "urlminder report: http://$url has changed" $tell << EOM
urlminder report: http://$url has changed. 
EOM
				;;
				2) ;;
			esac
#		fi
	fi
	cp "$file" "$file".old
done

The list of urls to check is kept in /usr/local/etc/urlminder_list, and looks like this: 

#format of two fields seperated by colon — url:mailto
http://example.com/index.html:paul

The url list in maintained by sending mail to a special account, urlminder. Mail to urlminder, which is an alias for my main account, is handled by a procmail recipe: 

:0
* ^To: .*urlminder.*
{
	:0
	* ^Subject: add
	| grep '^http:' >> /usr/local/etc/urlminder_list
}
# add deletion here at some stage

watch.weblogs

Keep an eye on your favourite weblogs. 

#!/bin/sh
#watch.weblogs: give a report on weblogs that have changed since the
#script last ran

# persistent
datafile=/tmp/watch.weblogs.data
# non-persistent
newdatafile=/tmp/watch.weblogs.newdata
listfile=/tmp/watch.weblogs.list.$$
tempfile=/tmp/changed.$$
oldtempfile=/tmp/oldchanged.$$

trap 'rm -f $newdatafile $listfile $tempfile $oldtempfile; exit 0' 0 1 2 15

# list of weblogs that have changed recently from weblogs.com
lynx -dump http://www.weblogs.com/weblogUpdates/changes.xml |
# get just the stuff we want from the list
sed -n 's/.*url="\(.*\)" when="\(.*\)".*/\1 \2/p' > $tempfile

# now get the list of weblogs I'm interested in
# the list is a lynx bookmark file, so needs a bit of massaging
grep http $HOME/.weblogs_worth_watching_bookmarks.html |
grep -v META | 
sed -f ~/scripts/strip_url.sed | 
sed -e 's?.*http://\([^/]*\)\(/.*\)?\1 \2?' > $listfile

# and work through the list, checking each one
exec < $listfile
while read line
do
    host=$(echo $line|awk '{print $1}')
    port=80
    url=$(echo $line|awk '{print $2}')
    case $host in
	# Bloody blogspot!  no e-tag, and last-modified is *wrong*!
	*blogspot.com) modtime="" ;;
	*) modtime=$(get_http_header.last-modified "$host" "$port" "$url") ;;
    esac
    if [ "$modtime" = "" ]
    then
	case $host in
	    # Bloody blogspot!  no e-tag, and last-modified is *wrong*!
	    *blogspot.com) etag="" ;;
	    *) etag=$(get_http_header.etag "$host" "$port" "$url") ;;
	esac
	if [ "$etag" = "" ]
	then
	    # no useful header; we may try checking on www.weblogs.com...
	    # don't affix a terminating / to the hostname if it's not
	    # followed by a path
	    if [ "$url" = "/" ]
	    then
		url=""
	    fi
	    newrec=$(2>/dev/null grep "$host$url" $tempfile)
	    newwhen=$(echo $newrec | awk '{print $2}')
	    if [ "$newrec" != "" ]
	    then
		prevrec=$(2>/dev/null grep "$host$url" $datafile)
		oldwhen=$(echo $prevrec | awk '{print $2}')
		# no previous record?  it's changed, then
		if [ "$prevrec" = "" ]
		then
		    echo "$newrec"
		# previous record, hasn't changed? do nothing
		elif [ $newwhen -ge $oldwhen ]
		then
		    :
		# previous record, has changed?
		elif [ $newwhen -lt $oldwhen ]
		then
		    echo "$newrec"
		fi
		echo "$newrec" >> $newdatafile
	    fi
	    else
	    # we have an etag
	    prevrec=$(2>/dev/null grep "$host$url" $datafile)
	    # no previous record? do nothing (it'll be added later)
	    if [ "$prevrec" = "" ]
	    then
		:
	    # previous record, hasn't changed? do nothing
	    elif [ "$prevrec" = "$host$url: $etag" ]
	    then
		:
	    # previous record, has changed?
	    elif [ "$prevrec" != "$host$url: $etag" ]
	    then
		echo "$host$url has changed"
	    fi
	    # build a new data file
	    echo "$host$url: $etag" >> $newdatafile
	fi
    else
	# we have a last-modified header
	prevrec=$(2>/dev/null grep "$host$url" $datafile)
	# no previous record? do nothing (it'll be added later)
	if [ "$prevrec" = "" ]
	then
	    :
	# previous record, hasn't changed? do nothing
	elif [ "$prevrec" = "$host$url: $modtime" ]
	then
	    :
	# previous record, has changed?
	elif [ "$prevrec" != "$host$url: $modtime" ]
	then
	    echo "$host$url has changed"
	fi
	# build a new data file
	echo "$host$url: $modtime" >> $newdatafile
    fi
done
cp $tempfile $oldtempfile
mv $newdatafile $datafile

get_http_header.last-modified

#!/bin/sh
#get_http_header.last-modified: get Last-Modified header of a webpage
host=${1?need hostname}
port=${2:-80}
url=${3:-/}
2>/dev/null lynx -dump -head -connect_timeout=30 http://$host$url | 
grep '^Last-Modified'

Why can't I use plain nc in place of lynx here, you're wondering, like so — echo "HEAD http://$host$url HTTP/1.0\n\n" | nc $host $port ? Well, that works with some httpd servers, but for some unaccountable reason, with blogspot it doesn't. lynx does return the right values for a blogspot weblog, so lynx it is.

get_http_header.etag

#!/bin/sh
#conditional_get.sh: check last modifed time of a webpage
host=${1?need hostname}
port=${2:-80}
url=${3:-/}
2>/dev/null lynx -dump -head -connect_timeout=30 http://$host$url | 
grep '^ETag'

msgid2gg

A simple enough idea; can be handy if you read USENET. 

#!/bin/sh
#msgid2gg: transform a usenet msg-id into a url accessing the archived 
#message on google groups
case $1 in
    "") exit 1 ;;
    *) ;;
esac
baseurl='http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&selm='
msgurl=$(echo $1 | sed -e 's/^<\(.*\)>$/\1/' -e 's/@/%40/')
echo $baseurl$msgurl

amped.watch

Keep me up to date on what's playing on my favourite Internet radio station. The idea is simple and can obviously be generalised to watch any frequently-changed webpage and extract the data you're interested in. 

#!/bin/sh
url=http://www.ampedout.net/playlist.php
wait=120
while :
do
    lynx -dump -nolist -connect_timeout=30 $url |
	sed -n '/Recent Playlist/{n;p;q;}' 
    sleep $wait
done

radio

Unfortunately, not all broadcasters over the Internet follow the sensible course of providing an mpg stream. Some prefer bloody Realplayer(TM, copyright, etc, etc). 

#!/bin/sh
#radio: listen to an internet radio station
export PATH=$PATH:$HOME/bin/RealPlayer8/
player='realplay'
case $1 in
    bbc1) url=http://www.bbc.co.uk/radio1/realaudio/media/r1live.ram ;;
    bbc2) echo "fuck OFF!!"; exit 2 ;;
    bbc3) url=http://www.bbc.co.uk/radio3/ram/r3g2.ram ;;
    bbc4) url=http://www.bbc.co.uk/radio4/realplayer/media/fmg2.ram ;;
    rte1) url=http://www.rte.ie/rams/radio1new.ram ;;
    rte2) url=http://www.rte.ie/rams/radio2new.ram ;;
    mocheol) url=http://www.rte.ie/rams/radio/mocheolthu.ram ;;
    late) url=http://www.rte.ie/rams/radio/thelatesession.ram ;;
    gothic) url=http://gothic.viirex.com/cgi-bin/countdown/countdown.cgi?cgi-bin/rsong.cgi ;;
    ipm) url=http://www.ipmradio.com/current.ram ;;
    3wk) url=http://www.3wk.com/3wknetsc.ram ;;
    woxy) url=http://www.woxy.com/stream/48k.ram ;;
    ggrs) url=http://www.ggrs.freeserve.co.uk/TurnOnTuneInListenOut.ram ;;
    diu) url=http://live.str3am.com:2200 ;;
    amped) url=http://war.str3am.com:7090 ;;
    kunm) url=http://www.kunm.org/listen/kunmlive.ram ;;
    ssat) url=http://real.sri.ch/ramgen/encoder/ssatclass.rm ;;
    sr2) url=http://www.sr-online.de/streaming/sr2.ram ;;
    rsi2) url=http://real.xobix.ch/live/rsi2.ram ;;
    rnd) url=http://www.radionotredame.com/sons/live.ram ;;
    ebm) url=http://www.vtuner.com/vtunerweb/ram/10840.ram ;;
    2sr2) url=http://www.sr-online.de/streaming/sr2.ram ;;
    *) url=$1
esac
2>/dev/null $player $url

checkip

More fun with lynx's -cmd_log /-cmd_script feature. Here's checkip, a script to find out the rough geographical location of an IP address. It's not perfect, since just because an IP address range is allocated to a company in the US, doesn't mean everyone using that network is actually in the States e.g. in the case of multi-national companies; but still it's handy enough. 

`
echo "<html>
<head>
</head>
<body>
<form method=post action="http://www.google-watch.org/cgi-bin/nbbc.cgi">
<input name="Na" type="text" size="45" maxlength="70" value=$1>
<input type="submit" value="Search">
</form>
</body>
</html>" |
lynx -stdin -cmd_script=$HOME/scripts/checkip.lynx
'

hansehack

The name speaks for itself, I think! 

#!/bin/sh
#hansehack: keep the mailbox open so sendmail can send mail via the relay
echo "user USERNAME%hanse.net\npass PASSWORD\nquit\n" |
nc webmail.hansenet.de 110

[back to Scripts index] [back to Linux, Unix, etc] [Main Site] [Weblog]

Contents licensed under the GPL