Linux, Unix, /etc

Danger Will Robinson! You are now entering a condescending Unix user zone!

Getting the Most From Your Shell

Preface

What is the most-used program on every Unix system, by every Unix user? It's the shell. The shell is ubiquitous. Yet, people often learn the few commands they absolutely must: just what they need to, say, read mail, start their editor, and print documents, and leave it at that. But the shell can do far, far more than start applications. This article will show you how to start getting more from your shell.

Introduction

While the rest of the world points and clicks in a scary little world of icons, all alike, we in the world of Unix get to use a good old-fashioned CLI, or Command Line Interface. One reason why the command line has remained so pervasive in Unix environments is that the implementation, the Unix shell in its various incarnations, is very good. It allows the user to use the tools provided to build new tools. I'll focus on three areas: editing the command line, using alias and functions as shortcuts, and a little bit about shell programming.

What Shell

Of course, there is more than one shell. To keeps things simple, I'll talk about what I know. That means shells in the Bourne shell tradition, and particularly three: /bin/sh, /bin/ksh, and /bin/bash. I'll try to focus on features that all three share, though inevitably, particularly for interactive use, I'll have to discuss features that the older shell doesn't have, notably job control and command-line editing. I won't cover the C shell. I've never used it, it's not well-regarded for anything but interactive use (and one of the purposes of this article is to blur the line between interactive and batch).

Command Line Editing

The first way to make the shell easier to use is to set up command-line editing. Two types are commonly-available in shells derived from the Korn shell. Those using the vi commands, and those using emacs edting keystrokes. They are very different, and users who swear by one will take pains to avoid the other. In both ksh and bash, to activate vi command-line editing, type

set -o vi

Then you can use more or less the same set of command keys that you have available in vi. In both ksh and bash, to activate emacs command-line editing, type

set -o emacs

Then you can use a subset of emacs command keys, and treat the command line as a sort of one-line emacs buffer.

Shell Options

Setting which type of command line editing you prefer is just one instance of setting shell options, which control may aspects of your shell's behaviour. You set shell options using set -o. Options can be unset using set +o.

The man page for your shell will list all the options; here, I'll focus on some that I think you'll find useful.

notify

Print job notification messages asynchronously, instead of just before the prompt. Both ksh and bash support this option.

noclobber

In both ksh and bash, this prevents > redirection from overwriting existing files. With this set, >| must be used to force an overwrite, else the shell will tell you "cannot create [file]: File exists".

bgnice

Background jobs are run with lower priority. This is a ksh option, and is not in bash.

ignoreeof

The shell will not exit on end-of-file (^D), "exit" must be typed. Actually, to avoid infinite loops, pdksh *will* exit if eof is read 13 times in a row. Bash is more sophisticated. To quote the man page:

"If set, the value is the number of consecutive EOF characters typed before bash exits. If the variable exists but does not have a numeric value, or has no value, the default value is 10. If it does not exist, EOF signifies the end of input to the shell."

nohup

Do not kill running background jobs when a login shell exists. This is another ksh option that isn't in bash.

Aliases

How do we use the shell? In day to day use, our most common use is in typing in commands, one after the other:

$ vi article
...
$ cat todo
...
$ date

and so on and so forth. Some commands are simple, other commands are quite complex. Where the command takes some typing effort, it is worthwhile to make a short cut. The easiest way to do this is to use what we call an "alias". Making an alias is straighforward. At the simplest level, to make our short cut, it is as though we simply replaced one command by another. A simple example from my own system is

alias pg=less

This saves me two keystrokes every time I want to examine a file using the "less" pager. Trivial? Maybe. But it's something I use many times each day.

An alias is a way of giving a nickname to a command or series of commands. The lefthand token is expanded to the righthand value by the shell, just the same as shell wildcards are expanded. So, for example, if I define the alias

alias rm=rm -i'
'

then when I type

$ rm foobar

the shell actually runs

$ rm -i foobar

which prompts me before doing anything rash like, oh, removing a file.

Aliases are also handy as a way of running a short series of shell comamnds that might otherwise have to be put in a shell script. If they are short, consider implementing such things as an alias rather than an script, saving load time. Aliases are "inherited". That is to say, in:

alias vi='vi -F'
alias vp="vi $HOME/.profile"

The 'vi' in 'vp' will be expaned to 'vi -F'. A useful way of seeing how much work I have to do is setup by the following:

alias la='ls -tr1 $HOME/articles/wip'

Another use of aliases is to save one remembering arcane syntax. why bother with "chmod +x" or "chmod 0755" or whatever, when a two-character alias does what you want i.e. makes a file exectuble?

alias cx="chmod +x $*"

Sometimes, one grows used to certain comamnds. On some flavours of Unix, for example, the pager program is neither more nor less, but pg. Why bother getting used to typing a new command. Simply use an alias:

alias pg='less'

Something I find useful is to have my browser point at a default location if I don't give a URL on the command line.

alias lynx="lynx -cache 100 ${1:-$HOME/homepage/index.html}"

Functions

The shell is a programming langauge, remember, so it's only natural that it should have functions. The syntax for functions is simple. A function defined in the shell looks not dissimilar to the way it would be defined in the C programming language: the function name, followed by a pair of parentheses, then the body of commands that make up the function enclosed in braces.

Functions are a natural progression from aliases, and avoid the overhead of a shell script at the cost of a bigger environment. Which is as good a place as any to mention, that this stuff doesn't come free. The more stuff you do in your init files — setting variables, defining fucntions, etc — the bigger your environment— an area of memory that each new process you start will copy — gets. For example, on my system

$ env | wc -c 3441

3,441 bytes. Quite a bit. You shouldn't notice preformance degradation on a modern machine, but this is something to be aware of. If most of your enivronment is only for interactive use, make sure that it isn't defined for non-interactive shells. You can do this by adding the lines

case $- in
        *i*) ;;
	*) return 0;;
esac

A little set of functions I find useful is this, for simplying job control.

b() { bg %$1 } f() { fg %$1 } k() { kill %${1:?must give job number} } twep() { kill -9 ${1:?need process number for termination with extreme prejudice} } ) What have we here? Well, b puts a job in the background, f brings it into the foreground again, and k kills it — all using handly job numbers rather than long-winded process numbers. Finally, twep is a bit of whimsy. Notice that arguments are passed to functions using the standard shell convention of $n, where n is 1-9, and that unlike C, the arguments aren't declared between the parenthese following the function name (in fact, those parentheses are really just a bit of syntactic sugar).

The boundary between aliases, fucntions and shell scripts is blurred. Given aliases and fucntions, making the jump to shell scripting is not difficult.

As my final example, I'll show a quite complicated function that could just as well be put in an executable file and called a shell script.

The MH mailer stores draft e-mails as files in a directory, just as it stores all other e-mails. One then uses comp -use <filename> to work on a particular draft. However, I often have no intention of sending an e-mail straight away, and would rather just fire up my editor on the message file. So, wrote a simple script, in effect adding a new command to MH. The script works as other MH commands by taking as argument the number of a mail message to edit. So now I can look at my drafts directory using the "drafts" command, the edit the message I want with, say, "vd 20". Here's the script:

vd()
{
case $1 in
    "");;
    last) vi $(ls $HOME/Mail/drafts/[1-9]*|sort -t '/' +5 -n|tail -1) ;;
    first) vi $(ls $HOME/Mail/drafts/[1-9]*|sort -t '/' +5 -n|head -1) ;;
    [0-9]*) vi $HOME/Mail/drafts/$1;;
    *) echo 'vd: usage: vd [first|last|<msg-nr.>]';;
esac
}

Here, we are in fact doing simple programming, using a case statement to decide what to do. $1, as we know, is the first argument to the function, and depending on its value — last, first, a number, or anything else — we execute the appropriate line of shell commands, and then, skipping the rest, end up at the end of the function. Those shell commands look complex, but breaking them down, they become clear. The lines for last and first are the most complicated, and since they are very similar,.we'll consider them together. First, we build a list of path names from the draft directory:

ls $HOME/Mail/drafts/[1-9]*

Then, we sort them in numerical order on the file name:

sort -t '/' +5 -n

(-t identifies the field seperator, +5 says *4th* field (we start counting with 0, -n says sort numerically). Now, we grab either the last or the first line of this list — the first or last draft e-mail, respectively.

tail -1

head -1

Now that we've got our file name, we want to run the editor on it. We do this by passing the output of our long command to the argument list of the editor command, by enclosing the whole thing in $(...). And there it is.

The other command lines are much simpler. If we're given a numerical argument, then we just build the pathname and hand it to vi directly. And if we have anything else, we don't know what to do, so we complain and don't try to do anything!

I can't hope to provide a shell programming tutorial in the space available, instead, I'll give a practical example, and hope that this inspires you to learn more.

[Box?] A Note on init Files

Of course, we don't type our aliases and fucntions in anew each time we start up the shell. An interactive shell reads a number of init files on startup, and it is here that's the best place to put our definitions.

What init file goes where depends on what shell you are using. In the beginning, there was the Bourne shell, /bin/sh. By the way, you won't find this if you go looking on your Linux box; there, /bin/sh is really /bin/bash, the GNU Bourne-Again SHell. This had just one init file, ~/.profile.

Nowadays, there are at least two profile files for all shells derived from /bin/sh: one global file in /etc, /etc/profile; and one user-specific file in the user's home directory, ~/.profile.

Then, there are shell-specific init. files.

For example, with pdksh we have; ~/.kshrc More properly, this is the default value of the environment variable ENV, which points to a file to source; we can set ENV to signify any file we like. In any case, every time a new interactive shell is started, this file will be sourced.

Similarly, bash has, in addition to the two standard files, a number of others in the user's home directory.

Bash's use of init files depends on the type of shell it is invoked as. Bash knows of three types of shell: two interactive, login, and non-login, and one non-interactive.

For login shells, these rules apply: if /etc/profile exists, it is sourced. Then, bash looks for several files in the user's home directory, and sources the first one it finds. Firstly, it looks for ~/.bash_profile If this file does not exist, it looks for ~/.bash_login, and sources that instead. Finally, if neither are found, bash will try to source ~/.profile. On exit, if ~/.bash_logout exists, it is sourced.

For non-login interactive shells, if ~/.bashrc exists, it will be sourced. And that's it!

For non-interactive shells, if the environment variable ENV is set, the file it points to is sourced, as if the command

if [ "$ENV" ]; 
then 
. $ENV; 
fi

had been executed. However PATH is *not* used to search for the pathname given in ENV.

users of ksh can get a similar functionality to the bash .bash_logout file by using the following trick. Put this line into .profile:

trap '. $HOME/.sh_logout; exit' 0

Now, when the shell exits, it will source the .sh_logout file. The same trick can also be used for bash when running as a non-login shell. Put the line into .bashrc.

Note that commands in these init files are "sourced"; that is, commands affect the current evironment (normally, executing a command doesn't affect the current i.e. parent environment — this is why shell scripts to cd to a directory don't affect the current directory of your login shell).

Resources

For pdksh, see

For bash, see

There are plenty of books on using the shell. The classic chapters in The Unix Programming Environment are aging but evergreen. Another great resource that every shell user should have is Unix Power Tools from O'Reilly, now in its second edition.

Paul Dunne 1999

[back to Linux, Unix, /etc]