Donation?

Harley Hahn
Home Page

Send a Message
to Harley


A Personal Note
from Harley Hahn

Unix Book
Home Page

SEARCH

List of Chapters

Table of Contents

List of Figures

Chapters...
   1   2   3
   4   5   6
   7   8   9
  10  11  12
  13  14  15
  16  17  18
  19  20  21
  22  23  24
  25  26

Glossary

Appendixes...
  A  B  C
  D  E  F
  G  H

Command
Summary...

• Alphabetical
• By category

Unix-Linux
Timeline

Internet
Resources

Errors and
Corrections

Endorsements


INSTRUCTOR
AND STUDENT
MATERIAL...

Home Page
& Overview

Exercises
& Answers

The Unix Model
Curriculum &
Course Outlines

PowerPoint Files
for Teachers

Chapter 13...

Using the Shell: Commands and Customization

In Chapter 11, we discussed the shell in general. In Chapter 12 we built on that foundation and discussed interactive shells, processes, environment variables, shell variables, and shell options. In this chapter, we will cover the rest of the fundamental concepts.

The time you spend reading this chapter will repay you well for two reasons. First, the ideas we are going to cover are crucial to using the shell well. Second, as you begin to integrate your knowledge of the shell, you will find that the skills you are about to learn will save you a lot of time in your day-to-day work.

Jump to top of page

Metacharacters

Using your keyboard, you can type letters, numbers, and a variety of other characters, such as punctuation and arithmetic symbols. In addition, you can use the <Space>, <Tab> and <Return> keys to generate the space, tab and newline characters (see Chapter 7).

Collectively, we refer to the letters and numbers as the ALPHANUMERIC characters. Using alphanumeric characters is straightforward: you type what you want, and what you see is what you get. However, when you use the shell, many of the other symbols have special meanings. We call such characters METACHARACTERS, and you need to be aware of how they work.

For example, as we discussed in Chapter 10, you can enter multiple commands on the same line by separating them with a semicolon. Because it has a special meaning to the shell, the semicolon is a metacharacter.

 

In a more abstract way, when you press <Space>, <Tab> or <Return>, you also are using metacharacters. The space and tab characters are used as whitespace to separate the various parts of a command (Chapter 10), while the newline character is used to mark the end of a line (Chapter 7).

If you try to memorize the purpose of all the metacharacters right now, it will be difficult because what you are memorizing won't make sense. Since each metacharacter is used to implement a particular service offered by the shell, it makes sense to learn about a metacharacter as you learn about the service it provides.

For example, I explained earlier that the $ (dollar) character is used when you want to refer to the value of a variable (such as $TERM). As such, $ is a metacharacter. But how much sense would it have made to you if, when I told you that, you didn't yet understand about variables?

I am going to give you a list of all the metacharacters, but I don't expect you to understand them all now. You will learn about them later, one by one. For now, I just want to make sure that you recognize them when you see them, so you don't accidentally use them the wrong way.

For example, if you were to enter the following command, the shell would see the semicolon and interpret what you typed as two commands:

echo The search path is $PATH; the shell is $SHELL.

However, if you recognize that the semicolon is a metacharacter, you can tell the shell to leave it alone:

echo "The search path is $PATH; the shell is $SHELL."

When we protect metacharacters in this way, we say that we "quote" them.

We'll talk about quoting in a moment. Before we do, however, I'd like to introduce you to the full cast of metacharacters. To start, take a look at Figure 13-1. This shows all the non-alphanumeric characters that we use with Unix, not all of which are metacharacters. Then look at Figure 13-2, where you will see the metacharacters along with a short description of how they are used. For reference, I have included the chapter number in which we discuss each particular metacharacter.

Within Figure 13-2, you will notice that several of the metacharacters are used for filename expansion, which is also known as "globbing". Globbing has a whole set of rules for using metacharacters, and we will discuss it in detail in Chapter 24.

Figure 13-1: Non-alphanumeric characters used with Unix

Here are all the non-alphanumeric characters we use with Unix. As part of the Unix tradition, most of the characters have nicknames that we use when we talk. For example, the exclamation mark is called a "bang"; $TERM is pronounced "dollar term"; and so on. If you want to sound like a Unix pro, use the nicknames.

Character English Name Unix Nickname
&ampersand
'apostrophequote, single quote
*asteriskstar
@at signat
`backquotebacktick
\backslash
{ }brace bracketsbraces, curly brackets
^circumflexcarat
:colon
,comma
$dollar signdollar
<Return>enter, returnnewline
=equal signequals
!exclamation markbang
>greater-than signgreater-than
-hyphen, minus signdash, minus (see Hint)
<less-than signless-than
#number signhash, pound (see Hint)
( )parentheses
%percent signpercent
.perioddot
+plus signplus
?question mark
"quotation markdouble quote
;semicolon
/slashforward slash
<Space>space
[ ]square bracketsbrackets
<Tab>tab
~tilde
_underscore
|vertical barpipe
Character English Name Unix Nickname

— hint —

Two of the non-alphanumeric characters in Figure 13-1 have different names depending on how long ago you started to use Unix. This is because traditions changed in the mid-1990s as Linux began to be popular.

Hyphen(-): Young people call the hyphen "dash"; old people call it "minus".

Number sign(#): Young people call the number sign "hash"; old people call it "pound".

If you are a student, this is important when you are talking to professors who are over 30. Remember to say "minus" and "pound", and you will make an old person comfortable.

Figure 13-2: Metacharacters used with the shell

Many of the non-alphanumeric characters have a special meaning within the shell. These are the metacharacters. This table shows all the metacharacters, along with their Unix nicknames. Eventually, you will learn the exact rules for using each of the metacharacters. Note that braces are used in two different ways.

For reference, I have specified the chapter in which each particular metacharacter is discussed.

Character Chapter Name Purpose
{ }24bracesbrace expansion: generate a pattern of characters
|15pipecommand line: create a pipeline
<15less-thancommand line: redirect input
>15greater-thancommand line: redirect output
( )15parenthesescommand line: run commands in a subshell
#14hash, poundcommand line: start of comment, ignore rest of line
;10semicoloncommand line: separates multiple commands
`13backquotecommand line: command substitution
~24tildefilename expansion: insert name of home directory
?24question markfilename expansion: match any single character
[ ]24bracketsfilename expansion: match from a set of characters
*24starfilename expansion: match zero or more characters
!13banghistory list: event marker
&26ampersandjob control: run command in background
\13backslashquoting: escape the next character
'13quote, single quotequoting: suppress all substitutions
"13double quotequoting: suppress most substitutions
{ }13bracesvariables: delimit a variable name
$13dollarvariables: substitute the value of a variable
<Return>7newlinewhitespace: mark the end of a line
<Tab>10tabwhitespace: separate words within the command line
<Space>10spacewhitespace: separate words within the command line
Character Chapter Name Purpose

Jump to top of page

Quoting and Escaping

From time to time, you will want to use a metacharacter literally, with no special meaning. For example, you may want to use a semicolon as a semicolon, not as a command separator. Or you may want to use the | (vertical bar) but not have it act as a pipe(*). In such cases, you must tell the shell to interpret the character literally. When you do so, we say that you QUOTE the character.

* Footnote

As Freud once said, "Sometimes a vertical bar is just a vertical bar."

There are three ways to quote characters: using a backslash, using a pair of single quotes, or using a pair of double quotes.

The most straightforward way to quote a metacharacter is to put a backslash (\) in front of it. This tells the shell to ignore any special meaning the character might have. For example:

echo It is warm and sunny\; come over and visit

In this example, we put a backslash in front of the ; (semicolon) character. If we hadn't, the shell would have interpreted the ; as a metacharacter, which would have led it to assume that you meant to type two separate commands, echo and come. Of course, this would produce an error.

— hint —

When you want to quote a single character, be careful to use the backslash (\), not a regular slash (/).

The regular slash has a completely different purpose. It is used within pathnames (see Chapter 23).

When you use a backslash to quote a single character, we call the backslash an "escape character". This is an important concept, so I want to spend a moment discussing it.

You may remember that, when we talked about runlevels in Chapter 6, we discussed the idea of a mode. Specifically, when a computer system, a program, or a device can be in one of several states, we use the term MODE to refer to a particular state. In Chapter 6, for example, I explained that your Unix or Linux system could boot in single-user mode or multiuser mode. Similarly, when we discuss the vi text editor in Chapter 22, you will see that, at any time, vi, is either in input mode or command mode.

When a program is in a specific mode and we do something to change to another mode, we say that we ESCAPE from one mode to another. When we change modes by pressing a particular key, we call it an ESCAPE CHARACTER(*). When you are typing a command for the shell, the backslash is an escape character, because it tells the shell to change from one mode (pay attention to metacharacters) to another mode (don't pay attention to metacharacters). I want you to remember the idea of an escape character, because you will encounter it again and again, especially if you are a programmer.

* Footnote

Now you know why there is an <Esc> or <Escape> key on your keyboard. It was designed to be used by programs to change from one mode to another.

Within Unix, the word "escape" is used in two ways. Most commonly, we talk about escaping from one mode to another. For example, when you use the vi text editor, you press the <Esc> key to escape from insert mode to command mode.

With the shell, we use the word "escape" a bit differently, as a synonym for quote. For example, consider this example:

echo It is warm and sunny\; come over and visit

We can say we are using a backslash to escape the semicolon. This is the same as saying we are using a backslash to quote the semicolon(*).

* Footnote

To be precise, when we talk about escaping from one mode to another, we are using "escape" as an intransitive verb, that is, a verb that does not take an object. When we talk about escaping (quoting) a character, we are using "escape" as a transitive verb, a verb that takes an object.

Unless you went to school before the Beatles were popular, your English teacher may have forgotten to teach you grammar. If so, you won't know the difference between transitive and intransitive verbs, so here is another example to pique your interest.

When you say, "Feel the fabric and tell me if it is soft," you are using a transitive verb. The verb is "feel"; the object is "fabric".

When you say, "Do you feel lucky?", you are using an intransitive verb. The verb is "feel", and there is no object. (To be precise, "feel" is a copula verb, and "lucky" is a predicate adjective, acting as a subjective completion.)

Although you might find it hard to believe, stuff like this is important, especially if you want to be a writer.

If we had just the backslash to escape metacharacters, it would be enough, as we can use the backslash more than once in the same line. Consider the following command:

echo It is warm (and sunny); come over & visit

This command won't work properly, because of all the metacharacters: (, ), ; and &. To make the command work, we need to escape all four characters:

echo It is warm \(and sunny\)\; come over \& visit

We now have a valid command, one that will work just fine. However, it is much too hard to read. As an alternative, the shell allows us to use single quotes to quote a string of characters. Thus, instead of the example above, we can use:

echo 'It is warm (and sunny); come over & visit'

In this case, we have quoted everything between the single quotes. Of course, this includes all the characters, not just the metacharacters, but it doesn't hurt to quote alphanumeric characters. (Stop a moment, and think about why this should be the case.)

Thus, we have (so far) two ways to quote metacharacters: we can use a backslash to quote single characters or single quotes to quote a string of characters. If the need arises, you can combine both types of quoting in the same command:

echo It is warm '(and sunny);' come over \& visit

Most of the time, the backslash and single quotes will be all that you need. However, there will be situations when it will be more convenient to use a third type of quoting using double quotes. Here is an example.

From time to time, you may want to use the $ character within a quoted string, usually to refer to the value of a variable. For example, the following command displays your userid and terminal type within angled brackets:

echo My userid is <$USER>; my terminal is <TERM>

In this form, the command doesn't work, because the metacharacters <, ; and > have a special meaning. (The $ is okay; we want it to be a metacharacter.) The solution is to quote only those metacharacters we want to be taken literally:

echo My userid is \<$USER\>\; my terminal is \<$TERM\>

This works, but it is much too complicated. We could, of course, use single quotes instead of backslashes:

echo 'My userid is <$USER>; my terminal is <$TERM>'

This is easier to read, but it quotes all the metacharacters, including the $. This means that we will literally see $USER and $TERM, rather than the values of the variables. For situations like this, we use double quotes because all the $ metacharacters retain their special meaning. For example:

echo "My userid is <$USER>; my terminal is <$TERM>"

Because we used double quotes, the , ; and > characters are all quoted, but the $ is not. (Try it for yourself.)

Aside from $, double quotes also preserve the meaning of two other metacharacters, \ (backslash) and ` (backquote). We'll talk about the backquote later. For now, all you have to know is that it has a special meaning that is preserved within double quotes, but not within single quotes.

To summarize:

• Use a backslash to quote a single character. (When you do this, we say that you escape that character.)

• Use single quotes to quote a string of characters.

• Use double quotes to quote a string of characters, but to keep the special meaning of $ (dollar), ` (backquote) or \ (backslash).

Jump to top of page

Strong and Weak Quotes

From the previous discussion, you can see that single quotes are more powerful than double quotes. For this reason, we sometimes refer to single quotes as STRONG QUOTES and double quotes as WEAK QUOTES. Here is an easy way to remember the names: single quotes are so strong, they only need a single symbol; double quotes are weaker, so they need a double symbol.

Actually, the backslash is the strongest quote of all (although we don't give it a special name). A backslash will quote anything, so if your single quotes ever fail you, try a backslash. For example, one day you may have to escape a single quote:

echo Don\'t let gravity get you down

The backslash is so powerful, it can even quote a newline character. (Take a moment to think about that.)

Let's say you type \<Return> at the end of a line. This generates \ followed by newline (see Chapter 7). The cursor will move to the next line but, since the newline character has lost its special meaning, it will not signal the end of a line. This means that whatever you type will be a continuation of the previous line.

Try this example, pressing <Return> at the end of each line:

echo This is a very, very long \
line that goes on and on for a \
very, very long time.

If you do it just right, you will type one long command, and you will see one long line of output.

Unlike the backslash, single and double quotes will not quote a newline. This being the case, what do you think happens if you press <Return> within a string that is quoted by a single or double quote? For example, say that you type the following:

echo 'This line ends without a second quote

When you press <Return> at the end of the line, the newline is not quoted, so it retains its special meaning: that is, it marks the end of the line. However, there is a problem because the single quote is unmatched.

If you are using the C-Shell or the Tcsh, you will get an error message telling you there is an unmatched ' character.

If you are using Bash or the Korn shell, the shell will simply wait for you to enter more input hoping that, eventually, you will type the second quote. (Bourne shells are more optimistic than C-Shells.) Once you type the second quote the shell will put everything together into one very long command with a newline in the middle.

As an exercise, type the following two lines, and see what your shell does:

echo 'This line ends without a matching quote
and here is the missing quote'

Can you explain what happened and why?

Jump to top of page

Commands That Are Built into the Shell: type

When you enter a command, the shell breaks the command line into parts that it analyzes. We say that the shell PARSES the command. The first part of each command is the name; the other parts are options or arguments (see Chapter 10).

After parsing the command, the shell decides what to do with it. There are two possibilities. Some commands are internal to the shell, which means the shell interprets them directly. These are the INTERNAL COMMANDS, often called BUILTIN COMMANDS or, more simply, BUILTINS. All the other commands are EXTERNAL COMMANDS, separate programs that must be run on their own.

When you enter a builtin command, the shell simply runs the command itself, within its own process. When you enter an external command, the shell searches for the appropriate program and then runs it in a separate process. As an analogy, let's say you call the customer service line for a large company. If the person who takes your call can answer your question, he does so himself (an internal command). Otherwise, he transfers you to the appropriate person (an external command).(*)

* Footnote

To be a bit more precise, here is how an external command is handled. The shell forks to create a child process and then waits. The child process execs to run the actual program. When the program is done, the child process dies. Control then returns to the parent, which causes the child to vanish. The parent then displays a shell prompt, inviting you to enter another command. (For more details, see the discussion about processes in Chapter 12.)

There are two ways to find out if a command is built into the shell. First, you can try to display a man page for the command. External commands have their own man page; builtins do not. Builtin commands are documented either within the man page for the shell, or on a special man page for all the builtin commands.

A faster way to check if a command is a builtin is to use the type command. The syntax is:

type command...

For instance:

type date time set

The exact output depends on which shell you are using. For example, here is what I saw when I used this command with Bash:

date is /bin/date
time is a shell keyword
set is a shell builtin

Here is what I saw with the Korn shell, Tcsh and C-Shell:

date is a tracked alias for /bin/date
time is a reserved shell keyword
set is a special builtin

Although the output differs a bit, the results are the same: date is an external command; the others are builtins.

At this point, it is not important to understand what is meant by a "tracked alias". It is a technical distinction you can ignore. Similarly, we do not need to distinguish between builtins and keywords: they are both built into the shell. (Keywords are special internal commands used for writing shell scripts.) The important thing is to realize that date is an external command residing in its own file (/bin/date on one system, /usr/bin/date on the other). This will make sense after we discuss pathnames in Chapter 24.

Unix and Linux systems come with literally hundreds of external commands, but how many builtins are there? That depends on the shell you are using. As an interesting reference, Figure 13-3 shows the number of builtin commands for the shells we discussed in Chapter 11.

Figure 13-3: Number of builtin commands for various shells

There are hundreds of different Unix commands, most of which are external; that is, they exist as separate programs. Each shell, however, has a certain number of internal commands, the builtins, that it can run directly. The table in this figure shows the number of builtin commands, including keywords, for each shell. As an interesting comparison, I have also included the old version of the Bourne shell as used by the early Unix users at Bell Labs.

Shell Builtin Commands
Bourne Shell18
Korn Shell47
C-Shell55
Bash69
Tcsh87
FreeBSD Shell97
Zsh129

Jump to top of page

Learning About Builtin Commands

Part of the Unix tradition is that when someone creates a tool, he should document that tool for other users. Specifically, it is expected that, when a programmer writes a new command, he will furnish a man page for that command. Because the format of the online manual is well-established (see Chapter 9), it is not hard for a programmer to create a man page once the programming is done. In fact, virtually all Unix programs are distributed with a man page that acts as the official documentation.

This system works fine when it comes to external commands. Because each external command is a program in its own right, it comes with its own man page. But what about the builtin commands? As we discussed, builtins are not separate programs; they are part of the shell. Since there are so many builtin commands (see Figure 13-3), it is unrealistic to expect the programmers who work on the shell to create a separate man page for every builtin.

Instead, all the builtin commands are documented within the man page for the shell. For example, the Korn shell builtins are documented in the Korn shell man page. Thus, for information about the builtins for a particular shell, you need to look at the appropriate man page. You can use one of the following commands:

man bash
man ksh
man tcsh
man csh

Bear in mind, however, that man pages for shells are quite long, and you may have to search a bit to find what you want.

Some Unix/Linux systems have a separate man page for builtin commands. To see if this is the case on your system, you can use the apropos command (see Chapter 9):

apropos builtin

If your system has such a page, that is the place to look for a quick list of all the builtins. For Linux and FreeBSD, you can use:

man builtin

For Solaris, use:

man shell_builtins

Linux also has a help command you can use to display information from the builtin man page in several ways. The syntax is:

help [-s] [command...]

where command is the name of a command.

To start, you can display a one-line summary of all the builtin commands by entering help by itself. If the output is too long, you can send it to less (Chapter 21) to display the information one screenful at a time:

help
help | less

This is the command to use when you want to display a compact list of all the builtins, for example, when you are looking for a particular command.

You can also use help to display information about one or more specific commands, for example:

help set
help pwd history kill
help help

(As you can see, help itself is a builtin command.)

Finally, if you only want to take a look at the syntax for a command, you can use the -s (syntax) option:

help -s help
help -s pwd history kill

— hint —

When you write a shell script, you use special builtin commands — for, if, while, and so on — to control the flow of your script. These commands are sometimes called KEYWORDS.

As you are working on a Bash script, the fastest way to check the syntax of a keyword is by using the help command. For example, to check syntax for all the Bash keywords, use:

help -s case for if select while until

If you need more information, leave out the -s option:

help case for if select while until | less

Notice that I have used less to make sure the output doesn't scroll off the screen.

Jump to top of page

External Commands and the Search Path

If a command is not built into the shell — and most commands are not — the shell must find the appropriate program to execute. For example, when you enter the date command, the shell must find the date program and run it for you. Thus, date is an example of an external command.

How does the shell know where to look for external commands? It checks the PATH environment variable (see Chapter 12). As with all variables, PATH contains a string of characters, in this case a series of directory names, which we call the SEARCH PATH.

We won't discuss directories in detail until Chapter 24. For now, all I want you to know is that programs are stored in files, and every file resides in a directory(*). The search path is the list of directories that contain the programs for all the external commands. Thus, one of the directories in the search path will contain the file that holds the date program.

* Footnote

To relate this to your experience, you can think of a Unix directory as being similar to a Windows or Macintosh folder. There are, however, subtle, but important, differences.

If you would like to see your search path, just display the value of the PATH variable:

echo $PATH

Here is some typical output:

/bin:/usr/bin:/usr/ucb:/usr/local/bin:/home/harley/bin

In this case, the search path consists of five directories:

/bin
/usr/bin
/usr/ucb
/usr/local/bin
/home/harley/bin

Your search path may be a bit different from this example but, for the most part, Unix systems tend to use standard names for the directories that hold external commands. For example, every Unix system I have ever seen has had a /bin and a /usr/bin directory, and many have /usr/ucb.

The names will make more sense after we discuss directories in Chapter 24. For now, I'll just mention the name bin is used to indicate that a directory holds programs.

In our example, the first three directories — /bin, /usr/bin and /usr/ucb — hold programs that are used by all the users on the system. The first two directories are found on all Unix systems and are set up automatically when Unix is installed. The /usr/ucb directory is found on some systems. Its job is to hold programs derived from Berkeley Unix (see Chapter 2). (The name ucb is an abbreviation for University of California at Berkeley.)

The next two directories are for customization: /usr/local/bin is set up by the system manager to hold programs that he or she has installed specifically for local use; /home/harley/bin refers to a directory named bin within the home directory of userid harley. You can make such a directory for yourself, and use it to hold your own programs.

When the shell needs to find an external command, it checks each directory in the search path in the order they are specified. In our example, the shell would start by looking in /bin. If it couldn't find what it wanted, it would then look in /usr/bin, and so on down the line.

When the shell finds the external command, it stops the search and executes the program. If none of the directories contains the command, the shell will give up and display an error message. For example, if you enter the command weedly, you will see a message similar to:

weedly: command not found

Jump to top of page

Modifying Your Search Path

On most systems, you don't have to define the search path yourself, because the PATH variable is set for you. However, in certain circumstances, which we will discuss in a moment, you may want to modify the search path, so I'm going to show you how to do so. The basic idea is to put the command that modifies the PATH variable in an initialization file that is executed automatically whenever you log in. (We'll talk about initialization files in Chapter 14.)

To start, let's talk about how to set the PATH variable to a particular value. We'll deal with the Bourne shell family separately from the C-Shell family, because the commands are a bit different.

With the Bourne shell family (Bash, Korn shell), you set PATH by using the export command (Chapter 12). Using export makes PATH an environment variable, which means it is available to the shell and all subsequent processes. Here is a typical command that will do the job:

export PATH="/bin:/usr/bin:/usr/ucb:/usr/local/bin"

The command itself is straightforward: it sets the value of PATH to a character string consisting of a list of several directory names. As you can see, the names are separated by colons, and there are no spaces on either side of the equal sign.

To set the value of PATH yourself, you put this command (or one like it) in your "login file", an initialization file that is executed automatically each time you log in. To make a change to the search path, you simply modify the login file. (All of this is explained in Chapter 14.)

With the C-Shell family (C-Shell, Tcsh), we use a somewhat different command, because we set the path shell variable rather than the PATH environment variable:

set path=(/bin /usr/bin /usr/ucb /usr/local/bin)

As you may remember from Chapter 12, whenever you change path, the shell resets PATH automatically. Thus, this command results in the same setting for PATH as did the earlier Bourne shell command.

Notice, however, the difference in syntax. In the Bourne shell command, we set the value of an environment variable (PATH) to a long string of characters. In the C-Shell command, we set the value of a shell variable (path) to a list of names. In C-Shell syntax, a list consists of one or more elements, separated by spaces and enclosed in parentheses.

A moment ago, I mentioned that the Bourne shell command to set PATH would go in your login file. The C-Shell command to set path goes in your "environment file", a different initialization file that is executed automatically every time a new shell starts. (Again, this is all explained in Chapter 14, where you will find examples.)

On most systems, the command to define the PATH variable is already set up for you, so you don't have to use commands like the ones we have been discussing. However, you may want to modify the default search path for your own use. For example, if you write your own shell scripts and programs, which you keep in your own personal bin directory ($HOME/bin), you will want to add the name of this directory to your search path.

Here are two commands to show you how to do it. The first command is for the Bourne shell family; the second is for the C-Shell family.

export PATH="$PATH:$HOME/bin"
set path = ($path $HOME/bin)

Take note of the syntax. It means "Change the value of the search path to be the old value followed by $HOME/bin." In other words, we append the name $HOME/bin to the end of the existing search path.

Alternatively, you may want to insert the new directory name at the beginning of the search path. The commands are:

export PATH="$HOME/bin:$PATH"
set path = ($HOME/bin $path)

As these examples show, it is possible to modify a variable based on its current value. This is an important idea, so don't just memorize the pattern. Take a moment to make sure you understand exactly what is happening.

Now that you know how to add your own directory to a search path, the question arises: should you put a directory of your own at the beginning or at the end of the list? It all depends on what you want.

If you put your personal directory at the end of the search path, the shell will check your directory last. If you put your directory at the beginning of the search path, the shell will check it first. For example, say that you write a program named date, which you put in $HOME/bin. You now enter:

date

If you put your directory at the beginning of the search path, the shell will run your date program, not the standard version. In this way, you can effectively replace any external command with a program of your own. Alternatively, if you put your directory at the end of the search path, the shell will run the standard date program, not your version. This keeps you from inadvertently replacing a program with a file of the same name. It is up to you to choose what works best for you.

As a programmer, there is one more directory name you may also wish to add to your search path. If you specify a dot (.) character, it adds your "working directory" to the search path. (Your working directory is the one in which you are currently working. We'll talk about it in Chapter 24.)

Here is an example that will help you clarify the concept. The following commands add both $HOME/bin and your working directory to the end of the current path (in that order). The first command is for the Bourne shell family; the second is for the C-Shell family:

export PATH="$PATH:$HOME/bin:."
set path = ($path $HOME/bin .)

This tells the shell that — when it looks for a program — the last directory to check is the one in which you are currently working.

A detailed discussion of search paths is beyond the scope of this book (and not all that necessary). Normally, you can just accept the search path that is set up for you by default, possibly with minor changes.

— hint —

Unless you are an expert, play it safe by putting your personal directories at the end of the search path.

Jump to top of page

How a Hacker Can Use the Search Path

It's fine to put the working directory (.) in your own search path, but never do so for the superuser (root) or for any other userid with special privileges. Doing so can create a security hazard. For example...

You are a system administrator and, for convenience, you have put the working directory at the beginning of the root search path. One of your users — who is really a hacker(*) — asks you for help, so you log in as root and change to the user's home directory (a common occurrence). You then enter the ls command to list the contents of the user's directory (also a common occurrence).

* Footnote

Of course, I am referring to a bad hacker — that is, a cracker — not a good hacker. For a discussion of good and bad hackers, see Chapter 4.

What you don't know is that the hacker has created an alternate version of ls, which he has placed in his home directory. The spurious ls acts like the real thing but — when run by root — it has the side effect of creating a secret file with special privileges. Later, the hacker can use that file, called a BACK DOOR, to gain superuser access for himself.

Here is how it happens: The moment you change to the user's home directory, it becomes your working directory. When you enter the ls command, the shell looks in the working directory, finds the hacker's version of ls, and runs it. The next thing you know, the hacker has taken over your system, and your life is exposed as a total sham.

Actually, this is an old hacker's trick: finding a way to run a doctored program as root to create a back door that can be used later. The moral of the story? Think carefully before you modify the search path for any userid that is used for system administration.

— hint —

Make sure that the search paths used by all the system administration userids (including root) do not contain the working directory, or any other directory to which users might have access.

Jump to top of page

The Shell Prompt

As you know, the shell displays a prompt whenever it is ready for you to enter a command. Should you so desire, it is possible to change this prompt. In fact, there is wide latitude in how prompts can be displayed, and some people have developed elaborate prompts that display colors, as well as various types of information. Let's start simple, and then move on to more complex customizations.

Originally, all shells had a two-character prompt: a single character followed by a space. The Bourne shell used a $ (dollar) character; the C-Shell used a % (percent) character. Today, the tradition is maintained. Thus, if you use a member of the Bourne shell family (Bash, Korn shell), the simplest shell prompt you will see is:

$ date

I have typed the date command after the prompt so you can see the space that follows the $. The space is part of the prompt.

If you use the C-Shell or the Tcsh, the simplest shell prompt looks like this:

% date

Although tradition dictates that the % character be used for the C-Shell family, many Tcsh users use a > (greater-than) character instead as a reminder that they are using an extended C-Shell:

> date

The final tradition concerns the superuser. When you are logged in as root, your prompt will always be a # (hash) character, regardless of which shell you are using. The intention is that you should always remember you are superuser, so you can be extra careful:

# date

Before we move on, take a look at Figure 13-4 in which the basic prompts are summarized. There are only a few conventions, and I want you to memorize what they mean so, whenever you see a shell prompt, you can answer two questions instantly: (1) Are you logged in as superuser? (2) If not, what type of shell are you using?

Figure 13-4: Standard shell prompts

By convention, the standard shell prompt consists of a single character, followed by a space. The Bourne shell family uses a $ (dollar) character; the C-Shell family uses a % (percent) character. The only exception is that the Tcsh sometimes uses a > (greater-than) character. When you are logged in as superuser, you will see a # (hash) character, regardless of what shell you are using.

Shell Prompt Character
Bash$
Korn Shell$
C-Shell%
Tcsh% or >
superuser#

Jump to top of page

Modifying the Shell Prompt

As I explained in the last section, it is possible to modify your shell prompt by changing the value of a variable. With the Bourne shell family, you change an environment variable named PS1(*). With the C-Shell family, you change a shell variable named prompt.

* Footnote

The name PS1 means "prompt for the shell, number 1". There are three other such variables — PS2, PS3 and PS4 — but you won't ever need to change them, so don't worry about them. If you are curious about the details, see the man page for your shell.

Let's start with a simple example. Here is a command, suitable for the Bourne shell family, that sets the prompt to a $ (dollar) character followed by a space:

export PS1="$ "

Similarly, here is a command, suitable for the C-Shell family, that uses the standard % (percent) character:

set prompt = "% "

If you are a Tcsh user, you should use the customary > (greater-than) character instead:

set prompt = "> "

Before we move on, I want to make sure you understand these commands by reviewing the basic concepts we covered in Chapter 12.

There are two types of variables, global and local: the global variables are called "environment variables". The local variables are called "shell variables".

All Bourne shells store the value of their prompt in an environment variable named PS1. To change the value of an environment variable, you use export. Hence, the export command above. Please pay attention to the syntax. In particular, you must not put a space before or after the = (equal sign) character.

All C-Shells store the value of their prompt in a shell variable named prompt. Within a C-Shell, you use set to modify a shell variable. Hence, the two set commands.

At this point, you might be wondering, is it significant that the Bourne shells use an environment (global) variable to hold the prompt, while the C-Shells use a shell (local) variable? In general, it's not all that important, as long as you make sure to use the appropriate command (export or set) if you want to change your shell prompt. The distinction is important, however, when we talk about initialization files, which help us set the prompt automatically each time we log in. We'll get to that in Chapter 14.

To continue, so far we have made only simple changes to the shell prompt. However, by manipulating the appropriate variable, you can set your shell prompt to be whatever you want. For example, you might set the prompt to display a cute message:

export PS1="Enter a command please, Harley$ "
set prompt = "Enter a command please Harley% "

Actually, cute shell prompts lose their appeal fast. Here is something more useful: a prompt that shows you the name of the shell as a reminder. (Choose the command for your particular shell.)

export PS1="bash$ "
export PS1="ksh$ "
set prompt = "csh% "
set prompt = "tcsh> "

Here are the prompts that are generated by these commands:

bash$
ksh$
csh%
tcsh>

This type of prompt is particularly handy for the superuser. For example, say that your root userid uses Bash as a default shell. If you set the prompt as follows:

export PS1="bash# "

The prompt will be:

bash#

The # will remind you that you are superuser, and the name will remind you which shell you are using.

Aside from using words and characters, there are three other ways to enhance your shell prompt. You can:

• Insert the value of a variable into the prompt.

• Use an escape character to make use of a variety of special codes.

• Insert the results of a command into the prompt. (This is called command substitution.)

Each of these techniques is important in its own right, and has a usefulness that goes far beyond modifying a shell prompt. For this reason, I am going to discuss each idea separately so you can understand the general principles.

Jump to top of page

Using the Value of a Variable

As we discussed in Chapter 12, to use the value of a variable, you type a $ (dollar) character followed by the name of the variable enclosed in brace brackets. For example:

echo "My userid is ${USER}."

For convenience, you can omit the braces if they are not necessary to separate a variable name from other characters. For example:

echo "My userid is $USER."

— hint —

When you use the value of a variable, it is a good habit to use brace brackets, even when it is not necessary.

Doing so enhances the readability of your commands, especially within shell scripts. Moreover, because the braces insulate your variables, they help avoid mysterious syntax problems that might otherwise baffle you.

Using the value of a variable within your shell prompt is straightforward. For example, to insert your userid into the prompt, you can use:

export PS1="${USER}$ "
set prompt = "${USER}% "

(The first command is for a Bourne shell; the second is for a C-Shell.)

If your userid were harley (which would be way cool), these commands would generate the following prompts:

harley$
harley%

Which environment variables might you want to use in a shell prompt? You can find a list of the most important environment variables in Figure 12-2 in Chapter 12. In principle, you can use any variables you want. However, most of them are not suitable for a shell prompt. To help you narrow down your choices, Figure 13-5 shows the variables I think are most useful or interesting for shell prompts. To experiment, just use one of the following commands, whichever works with your shell, substituting a variable from Figure 13-5.

export PS1="${VARIABLE}$ "
set prompt = "${VARIABLE}% "

Figure 13-5: Environment variables that are useful within a shell prompt

One way to enhance your shell prompt is to include the value of a variable that is either useful or interesting. Figure 12-2 in Chapter 12 contains a list of all the important environmental variables. This table shows you the ones that are suitable for a shell prompt.

The leftmost column shows which shells support each variable: B = Bash; K = Korn shell; C = C-Shell; T = Tcsh. A dot indicates that a shell does not support that option.

Shells Variable Meaning
B K C THOMEyour home directory
• • • THOSTname of your computer
B • • •HOSTNAMEname of your computer
B • • THOSTTYPEtype of host computer
B K C TLOGNAMEcurrent userid
B K C TPWDyour working [current] directory
B K • •RANDOMrandom number between 0 and 32,767
B K • •SECONDStime (in seconds) since current shell started
B K C TSHELLpathname of your login shell
B K C TUSERcurrent userid

Most people like to use either LOGNAME, PWD, SHELL or USER. However, to me, the most interesting variables to see over and over are RANDOM and SECONDS. However, they are available only with Bash and the Korn shell. If you want to experiment, here are the commands:

export PS1='Your lucky number is ${RANDOM} $ '
export PS1='Working time: ${SECONDS} seconds $ '

Jump to top of page

Which Quotes to Use When Quoting Variables

Take a careful look at two examples from the previous section:

export PS1='Your lucky number is ${RANDOM} $ '
export PS1="${USER}$ "

Did you notice that one command uses single quotes and the other uses double quotes? This difference illustrates a subtle, but important, point I want you to understand, especially if you want to write shell scripts.

The reason we use two different types of quotes is that, of the two variables in question, one of them changes and the other doesn't. To be precise, the value of RANDOM is a random number that is different every time you look at it. The value of USER is your userid, which is always the same.

We quote ${USER} with double quotes to allow the $ character to be interpreted as a metacharacter. This means that the value of USER is fixed at the moment the command is processed, which is fine because the value of USER never changes.

We quote ${RANDOM} with single quotes, which enables us to preserve the meaning of the $ character for later use. This technique ensures that the value of RANDOM is not evaluated until the actual prompt is created. In this way, when it comes time to display a new prompt, the shell uses whatever value RANDOM happens to have at that moment.

At this point, it may help to recall our discussion about strong and weak quoting from earlier in the chapter. To summarize:

• Single quotes ('...'), also known as strong quotes, quote everything. Within single quotes, no characters have special meanings.

• Double quotes ("..."), also known as weak quotes, quote everything except the three metacharacters $ (dollar), ` (backquote) and \ (backslash). Within double quotes, these three characters retain their special meaning.

Thus, when you use '${VARIABLE}' within a command, all the characters are taken literally, and the meaning of $ is preserved to be used later. When you use "${VARIABLE}", the $ is interpreted as a metacharacter and the entire expression is replaced at that moment with the value of VARIABLE.

So when you need to quote a variable, just ask yourself: "Will the value of the variable change before I use it?" If the answer is yes, use strong quotes (that is, single quotes) to keep the $ characters from being interpreted until you need them. Otherwise, use weak quotes (double quotes) to allow the $ characters to be interpreted right away.

Jump to top of page

Special Codes That Use an Escape Character

So far, I have explained that your shell prompt can contain any characters you want, as well as the value of one or more variables. In this section, we'll discuss how to enhance your prompt by using special codes.

Of the four shells we have been discussing, only Bash and the Tcsh allow you to use such codes. The codes allow you to insert various types of information into your prompt: the name of your working directory, your userid, the hostname of your computer, and so on. If you want to spend time being creative, you can even incorporate colors, underlining and boldface, although most people don't bother.

For reference, Figure 13-6 shows the most useful codes. The full list is documented on the man page for your particular shell. If you are very ambitious and you have extra time on your hands, you may want to learn how to use colors and other such effects in your prompt. If so, you'll find the help you need by searching the Web. (Hint: Using such codes is complicated, so don't worry if you don't understand it right away.)

Figure 13-6: Special codes, commands, and variables to use within shell prompts

Bash and the Tcsh allow you to use special codes to insert information into your shell prompt. The table shows the most useful codes. For a full list, see the Bash or Tcsh man page. A dot indicates that a shell does not support a particular function.

The Korn shell and C-Shell do not support such codes. However, for completeness, the table shows several ways in which a few of the codes can be simulated by using variables.

Meaning Bash Tcsh Korn Shell C-Shell
working directory: ~ notation\w%~
working directory: basename only\W
working directory: full pathname%/$PWD
hostname of your computer\h%m`hostname` `hostname`
current userid\u%n$LOGNAME $LOGNAME
name of the shell\s`basename $SHELL``basename $SHELL`
time: AM/PM notation\@%@
time: 24-hour notation\A%T
date\d
day of week%d
day of month%D
month%w
year%Y
history list: event number\!%!!!

You will notice that each of the Bash and Tcsh codes in Figure 13-6 consists of an escape character followed by another character. (As we discussed earlier in the chapter, an escape character tells the shell to change from one mode to another.) With Bash, the shell prompt escape character is \ (backslash); with the Tcsh, it is % (percent).

As I mentioned, only Bash and the Tcsh use special codes. However, with the other shells, there is still a need to place such information within the shell prompt. In a few cases, it can be done by using commands and variables. For completeness, this is shown in Figure 13-6. (Compare to Figure 13-5.) You will notice that some of these variables are used within an expression containing backquotes. This syntax is explained later, in the section on command substitution.

Most of the shell prompt codes are easy to understand. However I will mention that the codes for your working directory will make sense once you understand directories (Chapter 24), and the codes for history event numbers will make sense once you learn about using the history list (later in this chapter.)

To show you how it all works, here are a couple of examples. To insert your userid into a prompt, you would use \u for Bash and %n for the Tcsh. Thus, the following two Bash commands have the same effect:

export PS1="\u$ "
export PS1="${USER}$ "

If your userid were harley, your prompt would be:

harley$

Similarly, the following two Tcsh commands have the same effect:

set prompt = "%n> "
set prompt = "${USER}> "

The codes in Figure 13-6 are straightforward so, when you get time, feel free to experiment. If you like, you can use more than one code at a time. For example, to display the date and time in the Bash prompt, you use:

export PS1="\d \@$ "

In the Tcsh prompt, you use:

set prompt = "%d %w %D %@> "

Jump to top of page

Command Substitution

In this section we will talk about one of the most fascinating and powerful features offered by the shell: command substitution. COMMAND SUBSTITUTION allows you to embed one command within another. The shell first executes the embedded command and replaces it by its output. The shell then executes the overall command.

Obviously, we are dealing with a complex idea, so I'll start with a few examples. I'll then show you a practical application: how to use command substitution within a shell prompt to display useful information that would otherwise be unavailable as part of a prompt.

Let's begin with the basic syntax. You embed a command within another command by enclosing the first command in ` (backquote) characters. For example:

echo "The time and date are `date`."

In this example, the date command is enclosed by backquotes. This tells the shell to execute the overall command in two stages. First, evaluate the date command and substitute its output into the larger command. Then execute the larger command (in this case, echo).

Let's say it happens to be 10:30 am on Monday, December 21, 2008, and you are in the Pacific time zone. The output of the date command would be:

Mon Dec 21 10:30:00 PST 2008

During the first stage, the shell substitutes this value for date, changing the original command to:

echo "The time and date are Mon Dec 21 10:30:00 PST 2008."

During the second stage, the shell executes the modified echo command to produce the following output:

The time and date are Mon Dec 21 10:30:00 PST 2008.

Although I have broken down my explanation into two parts, it all happens so quickly that, to you, it will look as if the shell is displaying the final output instantly.

Here is another example. The environment variable $SHELL contains the pathname of your shell program. For example, say you are using Bash and you enter:

echo $SHELL

You will see the following output (or something similar):

/bin/bash

This means that your shell is the bash program, which resides in the /bin directory. (We will discuss directories and pathnames in Chapter 24.)

The full specification /bin/bash is called a pathname. In this case, however, we don't care about the entire pathname, just the last part (bash). To isolate the last part of any pathname, we use basename, a command whose purpose is to read a full pathname and output the last part of it. For example, the output of the following command is bash:

basename /bin/bash

More generally, to display the name of your shell program without the rest of the pathname, you can use:

basename $SHELL

Now consider how this might be used as part of the shell prompt using command substitution. Say you want to display the name of your shell as part of the prompt. All you need to do is insert the output of the basename command within the command that sets the prompt:

export PS1="`basename ${SHELL}`$ "
set prompt = "`basename ${SHELL}`% "
set prompt = "`basename ${SHELL}`> "

The first command is for Bash or the Korn shell; the second is for the C-Shell; the third is for the Tcsh.

As you can see, command substitution is used to create functionality that would otherwise not exist. For instance, in the last section, we discussed using special codes to insert information into your shell prompt. However, these codes are available only with Bash and the Tcsh. What about the other shells? The solution is to use command substitution.

For example, in Figure 13-6, you can see that the codes to insert the hostname (name of your computer) into the shell prompt are \h for Bash and %m for the Tcsh. With the other shells, we can use command substitution instead.

The basic approach is always the same. Start by asking the question: What command will do the first part of the job? Then figure out the best way to substitute the output of this command into another command that will do the second part of the job.

In this case, you would ask yourself: What command displays the name of your computer? The answer is hostname (see Chapter 8). More specifically, depending on your version of Unix or Linux, you may or may not need the -s option. See which variation works best on your system:

hostname
hostname -s

Now all you have to do is substitute the output of hostname into the command to set the prompt. (Leave out the -s if you don't need it.)

export PS1="`hostname -s`$ "
set prompt = "`hostname -s`% "

The first command will work with a Bourne shell (Bash, Korn shell); the second one will work with a C-Shell (C-Shell, Tcsh).

One last example. In the same way that Bash and the Tcsh have codes to display your hostname within a shell prompt, they also have codes to display your userid (\u and %n respectively). However, as you know, there are no codes for the other shells.

One solution is to use the $USER variable within the shell prompt, as we did earlier in the chapter. Alternatively, you can use command substitution with the whoami command (Chapter 8):

export PS1="`whoami`$ "
set prompt = "`whoami`% "

To conclude this section, let me remind you of something we discussed earlier. When you use single quotes (strong quotes), nothing between the two quotes retains a special meaning. When you use double quotes (weak quotes), three metacharacters do retain their special meaning: $ (dollar), ` (backquote) and \ (backslash). Now you understand why the backquote is included in this list.

— hint —

The backquote character is used only for command substitution. Be careful not to confuse it with the single quote or double quote. In spite of its name and appearance, the backquote has nothing to do with quoting. This is probably why many Unix people use the name "backtick" instead of "backquote".

Jump to top of page

Typing Commands and Making Changes

Once you have used Unix for a while, you will know how frustrating it is to have to type an entire command over because you need to make a tiny change. This is especially bothersome if you are prone to making spelling mistakes (as I am). As a convenience, the shell has several features that make it easy to enter commands: the history list, command line editing, autocompletion, and aliasing. We'll cover these features one at a time throughout the rest of the chapter. The details vary from one shell to another so, when you need help, remember that the man page for your particular shell is always the definitive reference.

As long as there have been shells, people have been arguing about which ones are best. In my opinion, you can talk all you want about this feature or that, but the shells that make it easy to type (and retype) commands get my vote. As a general rule, the best features are available only in Bash and the Tcsh, which is why knowledgeable users prefer to use one of these two shells.

To start, let me remind you of the ideas we discussed in Chapter 7, with respect to correcting mistakes and editing the command line. We will then move on to new material.

• To erase the last character you typed, you would press <Backspace>. This will send the erase signal. (With some computers, such as a Macintosh, you would use <Delete> instead of <Backspace>.)

• To erase the last word you typed, you would press ^W (<Ctrl-W>) to send the werase signal.

• To erase the entire line, you would press ^X or ^U (depending on your system) to send the kill signal.

• To display all the key mappings on your system, you would use the stty command.

With most (but not all) shells, you can also use the <Left> and <Right> cursor control keys to move around the command line. For example, let's say you mean to enter the date command but, instead, you type:

dakte

Your cursor is at the end of the line. Just press <Left> twice to move two positions to the left. Then press <Backspace> to erase the k. You can now press <Return> to enter the command. Note that you can press <Return> from anywhere in the line. You do not need to be at the end of the line.

In addition to changing the current line, you can press <Up>, to recall a previous command, which you can then modify and reenter. You can press <Up> more than once to find old commands, and you can press <Down> to go back to more recent commands.

You can use <Left>, <Right>, <Up> and <Down> in this way with Bash and the Tcsh, but not with the Korn shell and C-Shell. With Bash, you get a bonus: not only can you use <Backspace> to erase a character to the left, you can use <Delete> to erase a character to the right. This is a big deal once you get used to it. (Note: If <Delete> doesn't work on your system, you can use ^D instead.)

— hint —

Using <Left> and <Right> to move within the command line and <Up> and <Down> to recall previous commands is so handy that I urge you to practice until you find yourself using these keys without thinking about it.

When it comes to entering commands, your motto should be: Reuse; don't retype.

Jump to top of page

The History List: fc, history

As you enter commands, the shell saves them in what is called your HISTORY LIST. You can access the history list in a variety of ways to recall previous commands, which you can then modify and reenter. For example, when you press <Up> and <Down> to recall commands, you are actually moving backwards and forwards through the history list.

<Up> and <Down>, however, allow you to see only one command at a time. A more powerful feature enables you to display all or part of the history list and then select a particular command. How this works depends on your shell. As a general rule, the Bourne shell family (Bash, Korn shell) uses the fc command, and the C-Shell (C-Shell, Tcsh) family uses the history and ! commands. Most people find the C-Shell system easier and, for this reason, Bash allows you to use either system. Here are the details.

Within the history list, each command is called an EVENT, and each event is given an internal number called an EVENT NUMBER. The power of the history list is that it is easy to recall a command based on its event number. For example, you might tell the shell to recall command #24.

With the Bourne shell family, you display the history list by using the fc command with the -l (list) option. (I'll explain the name fc in a moment).

fc -l

With the C-Shell family, you use the history command:

history

The output of these commands consists of one event per line, prefaced by its event number. The event numbers are not part of the commands; they are displayed only for your convenience. Here is an example:

20 cp tempfile backup
21 ls
22 who
23 datq
24 date
25 vi tempfile
26 history

If your history list is so long that it scrolls off the screen, use the less command:

history | less

Notice that every command you enter is added to the history list, including commands with mistakes, as well as the history or fc commands themselves.

You can recall and execute a specific command by referencing its event number. With a Bourne shell, you type fc with the -s (substitute) option, followed by the number. For example, to re-execute command number 24, use:

fc -s 24

With a C-Shell, it is even easier. Just type a ! (bang) character followed by the number. Note that you do not use a space after the !:

!24

A special case occurs when you want to repeat the very last command you entered. With a Bourne shell, you re-execute the previous command by using fc -s without a number:

fc -s

With a C-Shell, you type two ! characters in a row:

!!

Both types of shells allow you to make small changes before you re-execute the command. With fc, the syntax is:

fc -s pattern=replacement number

With the C-Shell family, the syntax is:

!number:s/pattern/replacement/

In both cases, pattern and replacement refer to strings of characters, and number refers to an event number.

For example, in the previous example, event number 25 is a command that starts the vi editor with a file called tempfile:

25 vi tempfile

Say that you want to run the command again, but with a file named data instead. That is, you want to recall event number 25, change tempfile to data, and then re-execute the command. With a Bourne shell you would use:

fc -s tempfile=data 25

With a C-Shell, you would use:

!25:s/tempfile/data/

Once again, if you want to use the most recent command, the commands are simpler. For example, say that you want to run the date command, but by accident, you enter datq, which displays an error message:

$ datq
datq: Command not found.

You want to change the q to an e and re-execute the command. With fc -s, if you leave out the event number, the default is the previous command:

fc -s q=e

With a C-Shell, you use the syntax:

^pattern^replacement

For example:

^q^e

I know it looks odd, but it's quick and easy, and you will use it a lot, especially when you have a long command that needs a tiny change. For example, suppose you want to make a copy of a file named masterdata and call it backup. Using the cp command (Chapter 25) you type:

cp masterxata backup

You get an error message because, when you typed the first filename, you accidentally pressed x instead of d. To fix the mistake and re-run the command, just enter:

^x^d

— hint —

When you use Bash, you get two important advantages over the other shells.

First, with respect to the history list commands, you get the best of both worlds. You can use either the fc command, or the history/! system, whichever you like better.

If you are not sure which one to use, start with the history/! system.

Second, Bash supports an extra feature using ^R. (Think of it as the "recall" key".) Press ^R and then start typing a pattern. Bash will recall the most recent command that contains that pattern. For example, to recall the most recent ls command, press ^R and then ls.

If this is not the command you want, simply press ^R again to get the next most recent command that contains the pattern. In our example, you would press ^R again to get another ls command.

When you see the command you want, you can press <Return> to run it, or you can make a change and press <Return>.

What's in a Name?

fc


The Bourne shells (Bash, Korn shell) use the fc command to display and modify commands from the history list. fc is a powerful command with complicated syntax that can take awhile to master.

The name fc stands for "fix command". This is because, when you make a typing mistake, you can use fc to fix the command and then re-execute it.

Jump to top of page

History List: Setting the Size

The shell stores the history list in a file. This file can be saved automatically when you log out and can be restored when you log in. This is important, because it means that a record of what you do will be saved from one work session to the next.

With the Bourne shell family, the history file is saved and restored automatically. With the C-Shell family, your file will not be saved unless you set the savehist shell variable (see below).

The important thing to realize is that, when you examine your history list you are, in effect, looking back in time, possibly across separate work sessions. As with life in general, you will find that it is counterproductive to remember too much. For this reason, the shell lets you limit the size of your history list by setting a variable.

With the Bourne shell family, you set the HISTSIZE environment variable. For example, to specify that you want your history list to hold 50 commands (large enough for most people), use:

export HISTSIZE=50

With the C-Shell family, you set the history shell variable:

set history = 50

If you want to keep a longer record of your work, just set the variable to a larger number. If you don't set the size, it's okay. The shell will use a default value that will probably be fine.

As I mentioned above, if you want the C-Shell or Tcsh to save your history list when you log out, you must set the savehist shell variable. As with history, you must specify how many commands you want to save. For example, to save the last 30 commands from one work session to the next, use

set savehist = 30

— hint —

If you want to set the size of your history list, the place to put the command is in an initialization file, so that the variable will be set automatically each time you log in. We will discuss how to do this in Chapter 14.

Jump to top of page

History List Example:
Avoid Deleting the Wrong Files

As we will discuss in Chapter 25, you use the rm (remove) command to delete files. When you use rm, you can specify patterns to represent lists of files. For example, the pattern temp* stands for any filename that begins with temp followed by zero or more characters; the pattern extra? refers to any filename that starts with extra followed by a single character.

The danger with rm is that once you delete a file it is gone for good. If you discover that you have made a mistake and erased the wrong file — even the instant after you press <Return> — there is no way to recover the file. (We will now pause for a moment, to allow Macintosh users to regain their composure.)(*)

* Footnote

Believe it or not, the fact that the Unix rm command deletes files permanently is actually a good thing. Experienced Unix users rarely lose a file by accident, because they have trained themselves to think carefully before they act. Moreover, they learn to take responsibility for their actions, because they cannot depend on the operating system to compensate for their weaknesses. As you would imagine, such intelligence and self-reliance influence all aspects of life, which is why Unix people are, as a whole, so accomplished and fulfilled as human beings.

(We will now pause for a moment, once again, to allow Macintosh users to regain their composure.)

Let's say you want to delete a set of files with the names temp, temp_backup, extra1 and extra2. You are thinking about entering the command:

rm temp* extra?

However, you have forgotten that you also have an important file called temp.important. If you enter the preceding command, this file will also be deleted.

A better strategy is to first use the (ls list files) command using the patterns that you propose to use with rm:

ls temp* extra?

This will list the names of all the files that match these patterns. If this list contains a file you have forgotten, such as temp.important, you will not enter the rm command as planned. If, however, the list of files is what you expected, you can go ahead and remove the files by changing the ls to rm and re-executing the command. With a Bourne shell, you would use:

fc -s ls=rm

With a C-Shell:

^ls^rm

You may ask, why reuse the previous command? Once you have confirmed that the patterns match the files I want, why not simply type the rm command using those same patterns?

You could, but when you reuse the ls command, you are guaranteed to get exactly what you want. If you retype the patterns, you may make a typing mistake and, in spite of all your precautions, still end up deleting the wrong files. Also, in many cases, it is faster to modify the previous command than it is to type a brand new one.

If you like this idea, you can make the process even easier by using an alias. I'll show you how to do so later in the chapter.

Jump to top of page

Displaying Event Number &
Working Directory in Your Shell Prompt

Earlier in the chapter, we discussed how to put various types of information into your shell prompt: your userid, the name of your shell, and so on. In this section, I'll show you how to display two items that change from time to time: the event number and the name of your working directory. Putting these items in your shell prompt helps you keep track of what you are doing.

To display the current value of the event number, you use a special code. The codes vary from one shell to another, so I have summarized them in Figure 13-7.

Figure 13-7: Displaying the history list event number in your shell prompt

To display the current value of the history list event number in your shell prompt, use the code for your particular shell. See text for examples.

Shell Code
Bash\!
Korn Shell!
C-Shell!
Tcsh%!

Here are some examples that show how to do it for the four major shells. In the examples, I have placed the event number within square brackets(*), which looks nice when it is displayed.

export PS1="bash[\!]$ "
export PS1="ksh[!]$ "
set prompt = "csh[\!]% "
set prompt = "tcsh[%\!]> "

* Footnote

It happens that, with the C-Shell and Tcsh, a ! character followed by a right square bracket causes a problem. Thus, in these examples, I used a backslash to quote the !. The reason for this is obscure, so don't worry about it.

Let's say, for example, the current event number is 57. The four prompts as defined above would look like this:

bash[57]$
ksh[57]$
csh[57]%
tcsh[57]>

As you might expect, the event number can be combined with other codes or with variables to construct a more elaborate prompt. For example, here are several prompts that display the name of the shell, the working directory (see Chapter 24), and the event number. (For information on displaying the name of your working directory in a shell prompt, see Figure 13-6.)

For readability, I have inserted some spaces, placed the name of the working directory in parentheses, and placed the event number in square brackets.

To start, here is the prompt for Bash. We use \w to display the working directory and \! to display the event number:

export PS1="(\w) bash[\!]$ "

The same prompt for the Korn shell is a bit trickier. Because the Korn shell has no code to display the name of the working directory, we use the PWD variable. However, because PWD changes from time to time, we must use strong quotes, rather than weak quotes. (See the discussion earlier in the chapter.)

export PS1='($PWD) ksh[!]$ '

Alternatively, we could use weak quotes, as long as we make sure to use a backslash to quote the $ character:

export PS1="(\$PWD) ksh[!]$ "

Finally, here is the command to use with the Tcsh. We use %~ to display the working directory and %! to display the event number.

set prompt = "(%~) tcsh[%\!]> "

What about the C-Shell? As you can see from Figure 13-6, there is no easy way to display the name of the working directory in a C-Shell prompt, so I have no example to show you. However, there is a more complicated way to do it using what are called "aliases". We'll talk about this idea later in the chapter.

Jump to top of page

Autocompletion

One of the ways in which the shell makes it easier for you to enter commands is by helping you complete words as you type. This feature is called AUTOCOMPLETION.

For example, you are entering a command and you need to type the name of a very large file, such as:

harley-1.057-386i.rpm

If there are no other files with similar names, why should you have to type the entire name? You should be able to type just a few letters and let the shell do the rest for you.

With autocompletion, the shell carefully looks at everything you type. At any time, you can press a special key combination, and the shell will do its best to complete the current word for you. If it can't complete the word, the shell will beep. I'll give you an example just to show you the concept, and then we'll go over the details.

Let's say you have four files:

haberdashery
hardcopy
harley-1.057-i386.rpm
hedgehog

If you type harl and press the autocomplete key combination, there is no ambiguity. Since there is only one file that begins with harl, the shell will fill in the rest of the name for you.

However, what happens if you type har and then press the autocomplete key? There are two filenames that begin with har, so the shell beeps to indicate that what you have typed is ambiguous.

At this point, you have two choices. You can type a bit more and try again. Or, if you are not sure what to type, you can press a second autocomplete key combination and have the shell display a list of all possible matches.

With our example, if you type har and press the second key combination, the shell will display:

hardcopy
harley-1.057-i386.rpm

You can then type either a d or an l, and tell the shell to finish the job.

As you can see, to use the basic autocomplete facility, you only need to remember two different keys combinations. For historical reasons, these keys differ from one shell to another, so I have listed them in Figure 13-8. The first key combination tells the shell to try to complete the current word. The second key combination tells the shell to display a list of all possible completions that match what you have already typed.

Figure 13-8: Autocomplete keys

The basic autocompletion features use two different keys combinations. The first one tells the shell to try to complete the word you are currently typing. If this doesn't work, you can use the second key combination to have the shell display all possible completions that match your pattern.

Shell Complete the Word Display All Possibilities
Bash<Tab><Tab><Tab>
Korn Shell<Esc><Esc><Esc>=
C-Shell<Esc>^D
Tcsh<Tab>^D

To let you see how autocompletion works, I'll show you some examples that you can try at your own computer. Before we begin, however, I want to make sure autocompletion is turned on for your shell. For Bash, the Korn shell and the Tcsh, this is always the case. However, for the C-Shell, you need to turn on autocompletion by setting the filec variable. The command to do so is:

set filec

The best place to put this command is in an initialization file, so the variable will be set automatically each time you start a new shell. I'll show you how to do this in Chapter 14.

To return to our examples, in order to experiment with autocompletion, we will need a few files that have similar names. To create them, we will use the touch command. The files will be xaax, xabx, xacx and xccx(*). Here is the command:

touch xaax xabx xacx xccx

(We'll talk about the touch command in Chapter 25. For right now, all you need to know is this is the easiest way to create empty files.)

* Footnote

In case you are wondering, I named these files after four of my ex-girlfriends.

We will now use the ls -l command, which lists file names along with other information, to demonstrate autocompletion. To start, let me show you what happens when you complete a filename. Type the following and then stop, without pressing <Return>:

ls -l xc

Now, look at Figure 13-8 and press the "Complete Word" key combination for your particular shell. That is, with Bash or the Tcsh, press <Tab>; with the C-Shell, press <Esc>; with the Korn shell, press <Esc> twice.

Since there is no ambiguity, the shell will complete the filename for you. You will see:

ls -l xccx

You can now press <Return> to enter the command.

This type of autocompletion is called FILENAME COMPLETION. There are several other types, which we will discuss later.

Now, let's see what happens when the shell cannot complete a file name. Type the following and then stop, without pressing <Return>:

ls -l xa

Once again, press the "Complete Word" key combination for your shell (<Tab>, <Esc> or <Esc><Esc>). This time, the shell will beep, because there is no single filename that matches what you have typed. (In fact, there are three.) Type the letter b and then stop, without pressing <Return>. You will see:

ls -l xab

Now press the key combination again. This time, the shell will be able to make the completion for you, as there is only one filename that matches xab (xabx). Press <Return> to enter the command.

One final example. Type the following and then stop, without pressing <Return>:

ls -l xa

This time, look at Figure 13-8 and press the "Display Possibilities" key combination for your particular shell. That is, with Bash, press <Tab> twice; with the Korn shell, press <Esc>=; and with the C-Shell or the Tcsh, press ^D (Ctrl-D). This tells the shell to list all possible matches.

The shell will display the matches and then retype your command for you, so you can complete it. You will see:

xaax xabx xacx
ls -l xa

You can now complete the command however you want and press <Return>.

Finally, when you are finished experimenting, you need to clean up after yourself by removing the four practice files:

rm xaax xabx xacx xccx

Here is one last example. At any time, the directory in which you are currently working is called your "working directory" (see Chapter 24). From time to time, you will want to change your working directory and, to do so, you will type the cd (change directory) command, followed by the name of a directory. There will be times when you find yourself typing long directory names, especially if you are a system administrator. When this happens, you are better off using autocompletion.

For example, let's say you are using a Linux system, and you want to change to the following directory:

/usr/lib/ImageMagick-5.5.7/modules-Q16/filters

You could type cd followed by the very long directory name. However, it is a lot easier to type the minimum number of characters and use autocompletion. In this case, with Bash, you could type:

cd /us<Tab>/li<Tab>/Im<Tab>/mo<Tab>/fi<Tab><Return>

If you use a different shell, the autocomplete key would be different, but the idea is the same: Why type a long name if the shell will do it for you?

Jump to top of page

Autocompletion: Beyond the Basics

In Chapter 11, when I explained where the names C-Shell and Tcsh came from, I mentioned that the creator of the original Tcsh, Ken Greer, had been working on a PDP-10 computer using the TENEX command interpreter (similar to a shell).

TENEX used very long command names because they were easy to understand, but it was a lot of bother to type the full commands. A facility called "command completion" was used to do a lot of the work. All you had to do was type a few letters and press the <Esc> key. The command interpreter would then expand what you had typed into the full command. Greer added this feature to the new C-Shell he was writing and, when it came time to name the shell, he called it the Tcsh, the "T" referring to TENEX.

In the last section, we used autocompletion to help us type the names of files, and this is how you will use it most of the time. However, as you can see from the Tcsh story, autocompletion is an old idea. Moreover, it can be used to complete more than just filenames. In fact, modern shells offer autocompletion for a variety of different types of names.

The details vary from shell to shell, and they are complex. In fact, most shells, particularly Bash and the Zsh (mentioned in Chapter 11), give you many more autocomplete features than you would ever use in three lifetimes. Most of the time, the techniques that we discussed in the last section will be all you need. In this section, I'll explain a bit more. If you want more details, display the man page for your particular shell and search for the word "completion".

In general, there are five types of autocompletion. Not all shells support every type, although all modern shells offer filename completion, which is the most important type of autocompletion. For reference, Figure 13-9 shows which types of autocompletion you can use with the various shells.

Figure 13-9: Types of autocompletion

Autocompletion allows you to type part of a name and have the shell complete it for you. In general, there are five different types of autocompletion, each of which completes a different type of name. See text for details.

The leftmost column shows which shells support which type of autocompletion: B = Bash; K = Korn shell; C = C-Shell; T = Tcsh. A dot indicates that a shell does not support that feature.

Shells Autocompletion Completes names of...
B K C TFilename completionFiles and directories
B • • TCommand completionCommands, including pathnames
B • • TVariable completionVariables
B • C TUserid completionUserids on your system
B • • •Hostname completionComputers on your local network

We have already discussed filename completion. COMMAND COMPLETION (Bash, Tcsh only) is similar. When you are typing the beginning of a line, you can use autocompletion to help you type the name of a command. This can be an external command, a builtin command, an alias, or a function.

For example, say you are using Bash or the Tcsh, and you want to enter the whoami command to display your userid (see Chapter 8). You start by typing:

whoa

You then press <Tab> and the shell will complete the command for you.

VARIABLE COMPLETION (Bash, Tcsh) comes into play whenever you start a word with a $ character. The shell assumes you are about to type the name of a variable.

For example, you are using Bash and you want to display the value of an environment variable, but you forget its name. All you remember is that it begins with the letter H. Type the following:

echo $H<Tab><Tab>

With the Tcsh, you would type $H followed by ^D (Ctrl-D):

echo $H^D

The shell lists all the variables whose names begin with H. For example:

HISTCMD   HISTFILESIZE  HOME      HOSTTYPE
HISTFILE  HISTSIZE      HOSTNAME

You recognize the variable you want as HOSTTYPE, so you type enough of the name so that it can be recognized and (with Bash) press <Tab> to finish the job:

echo $HOSTT<Tab>

With the Tcsh, you would use:

echo $HOSTT<Esc><Esc>

USERID COMPLETION (also called USER NAME COMPLETION) is available with Bash, the C-Shell and the Tcsh. It works like variable completion, except that you must start the word with a ~ (tilde) character. This is because the syntax ~userid is an abbreviation for userid's home directory.

Finally, HOSTNAME COMPLETION, available only with Bash, will complete the names of computers on your local network. Hostname completion is used when you start a word with the @ (at sign) character. You will do this, for example, if you are typing an email address.

— hint —

Autocompletion is particularly useful when you have an idea of what you want to type, but you can't remember the full name.

For example, if you are using Bash, and you want to enter a command that begins with lp but you can't remember which one, just type lp<Tab><Tab>. (With the Tcsh, use lp^D.)

Similarly, you can list the names of all your variables by typing $ <Tab><Tab> (or $^D). Try it.

Jump to top of page

Using Autocompletion for Fun and Profit

You may remember The Fable of the Programmer and the Princess from Chapter 7. In this story, a handsome young programmer is able to rescue a beautiful princess by entering a command without pressing the <Return> key or ^M. (He uses ^J.) Here is something even cooler: how to use autocompletion to make a few bucks and (assuming you are a guy) impress women at the same time.

The next time you are at a gathering of Linux users, look around at all the geeks and find one who looks like he has a bit of money. Since this is a Linux geek, you know he will be using Bash. Offer a small bet (say, five dollars) that you can list the names of all his environment variables without pressing the <Return> key. When he takes the bet, enter:

env^M

He will now see how you tricked him, so offer to double the bet. This time, you promise not to use <Return> or ^M. When he takes the bet, enter:

env^J

You are now ready to move in for the kill. Offer to triple the bet and, this time, you promise not to use <Return>, ^M or ^J. And, to make it harder, you won't even type a command.

By now, you will have attracted the attention of a lot of other Linux geeks who will want in on the action. Make them put their cash on the table and, once you have gathered all the bets you can, type:

$<Tab><Tab>

As you scoop up the money and walk away, look back at the geeks and say, "Haven't you guys ever heard of RTFM?"

Jump to top of page

Command Line Editing: bindkey

In the past few sections, we talked about several interrelated topics: making changes as you type a command, using the history list, and using autocompletion. As you read these sections, you may have noticed two things. First, the three newer shells — Bash, Korn shell, and the Tcsh — offer significantly more features than the older C-Shell. Second, there seems to be an underlying thread tying all of this together.

This is indeed the case. The general principle at work here is called COMMAND LINE EDITING, and it is available only with the newer shells, not with the C-Shell. Command line editing is a powerful facility that allows you to use a large variety of commands to manipulate what you type on the command line, including the ability to use the history list and autocompletion.

You will remember my telling you several times that there are two main Unix text editors: vi (pronounced "vee-eye") and Emacs. Eventually, you must learn how to use at least one of the editors well. Indeed, I have devoted an entire chapter of this book to vi (Chapter 22).

Both vi and Emacs offer a large, powerful set of commands that allow you to view and modify text files. These commands are so well-designed that they are suitable for editing any type of text in any situation. In particular, the shell lets you use either the vi commands or Emacs commands (your choice) to view and modify what you type on the command line as well as your history list.

It happens that the vi commands are very different from the Emacs commands, so the shell lets you use only one set at a time. By default, the shell assumes you want to use Emacs commands. We call this EMACS MODE. However, you can change to vi if you want. If you do, we say that the shell is in vi MODE.

The way in which you change from one command editing mode to another depends on your shell. With Bash and the Korn shell, you set a shell option, either emacs or vi. (Shell options are explained in Chapter 12.) Thus, you would use one of the following commands:

set -o emacs
set -o vi

With the Tcsh, you use the bindkey command. You can do so with either the -e (Emacs) or -v (vi) option:

bindkey -e
bindkey -v

The best place to put either of these commands is in an initialization file, so the command will be executed automatically each time you log in. I will show you how to do so in Chapter 14.

When it comes to editing regular text files, vi is the best choice for beginners. This is because, while vi is difficult to learn, Emacs is very difficult to learn. So, if you are a beginner, when the time comes to learn how to edit files, I will recommend that you start with vi, not Emacs.

However, when it comes to editing the history list and command line, Emacs is actually easier to use than vi. The reason is that, most of the time, you only need to move up and down within the history list or make small changes to the command line. With Emacs, this is straightforward. The vi editor is more complicated, because it has two different modes — command mode and insert mode — and before you can use vi, you need to learn how to switch back and forth from one mode to another. (We'll go into the details in Chapter 22.) For this reason, all the shells use Emacs mode as the default.

When I taught you how to use <Up> and <Down> to move within the history list, and how to make basic changes to your command line (earlier in the chapter), I was actually showing you simple Emacs commands. Thus, you have already been using Emacs for command line editing, even though you didn't realize it at the time. In fact, if your shell had been in vi mode, you would have found that the cursor movement keys would not have worked the way you expected.

Both vi and Emacs offer a very large number of ways to manipulate your history list and command line. However, none of this helps you at all until you learn how to use one of the editors. For this reason, I'm not going to explain the details of advanced command line editing. If you are so inclined, you can experiment with the vi or Emacs commands once you learn how to use them. At that time, come back to this chapter, and read the rest of this section. (For vi, that will be after you read Chapter 22.)

To teach yourself command line editing, start by using set or bindkey to put your shell in either vi or Emacs mode. Now, imagine you are working with an invisible file that contains the history list. At any time, you can copy one line from this file to your command line, where you can modify the text as you wish. Whenever you press <Return> to run a command, the contents of the command line are added to the bottom of the invisible file (that is, to the history list).

Keeping this image in mind, it is easy to experiment. All you have to do is use the vi or Emacs commands in a way that makes sense. Start by practicing the basic maneuvers: moving through the invisible file, searching for patterns, making replacements, and so on. You will find that, once you are comfortable with vi or Emacs, command line editing is straightforward and intuitive.

If you need a reference, take a look at the man page for your particular shell and search for information about command editing. You may find the instructions a bit confusing, but be patient. Rearranging your brain cells takes time.

Jump to top of page

Aliases: alias, unalias

An ALIAS is a name that you give to a command or list of commands. You can use aliases as abbreviations, or you can use them to create customized variations of existing commands. For example, let's say you often find yourself entering the command:

ls -l temp*

If you give it an alias of lt, you can enter the command more simply by typing:

lt

To create an alias, you use the alias command. The syntax varies slightly depending on which shell you use. For the Bourne shell family (Bash, Korn shell), the syntax is:

alias [name=commands]

Be sure not to put a space on either side of the equals sign (the same as when you create a variable).

For the C-Shell family (C-Shell, Tcsh), the syntax is almost the same. The only difference is you leave out the equals sign:

alias [name commands]

In both cases, name is the name of the alias you want to create, and commands is a list of one or more commands.

An an example, let's create the alias I mentioned above. The first command is for a Bourne shell; the second command (which leaves out the equals sign) is for a C-Shell:

alias lt='ls -l temp*'
alias lt 'ls -l temp*'

Notice that I have enclosed the command in single quotes. This is because the command contains both spaces and a metacharacter (*). In general, strong quotes (single quotes) work better than weak quotes (double quotes), because they preserve the meaning of the metacharacters until the alias is executed.

Here is an example that creates an alias for a list of two commands. Again, the first command is for a Bourne shell; the second is for a C-Shell:

alias info='date; who'
alias info 'date; who'

Once you have created this alias, you can enter info whenever you want to run these two commands.

Here is my favorite alias. It creates an abbreviation for the alias command itself:

alias a=alias
alias a alias

Once you create this alias, you can create more by using a instead of having to type the whole word alias. For example, once you define this alias, you could enter:

a info='date; who'
a info 'date; who'

If you want to change the meaning of an alias, just redefine it. For example, if info is an alias, you can change it whenever you want simply by using another alias command:

alias info='date; uname; who'
alias info 'date; uname; who'

To check the current value of an alias, enter the alias command with a name only. For example, to display the meaning of the alias info, use:

alias info

To display all your aliases at once, enter the alias command with no arguments:

alias

To remove an alias, you use the unalias command. The syntax is:

unalias name

where name is the name of an alias. For example, to remove the alias we just defined, you would use:

unalias info

If you want to remove all your aliases at once (say, if you are experimenting), use the unalias command with either the -a option (for a Bourne shell), or with a * character (for a C-Shell):

unalias -a
unalias *

Do you remember the type command, we discussed earlier in this chapter? (You specify the name of a command, and type tells you what type of command it is.) You can use type to find out if a particular command is an alias. For example, say you define the info alias as shown above. You then enter:

type info

You will see a message similar to this one:

info is aliased to 'date; who'

As you might imagine, you are likely to develop a whole set of aliases that you use all the time. However, it is bothersome to have to retype the alias commands every time you log in. Instead, you can put all your favorite alias definitions in an initialization file, which causes them to be executed automatically whenever you start a new shell. I'll show you how to do this in Chapter 14.

Jump to top of page

Suspending an Alias Temporarily

One very common use for aliases is to make it easy to use the same options every time you run a particular command.

For example, the ls command (which we will discuss in Chapter 24) lists the contents of a directory. When you use ls by itself, you get a "short" listing; when you use ls with the -l option, you get a "long" listing.

Suppose you find that, almost all the time, you use ls with the -l option. To save yourself having to type the option every time, you define the following alias:

alias ls="ls -l"
alias ls "ls -l"

(The first definition is for the Bourne shell family; the second is for the C-Shell family.)

Now, you can simply enter the command by itself. You don't have to type the option:

ls

This will produce a long listing, just as if you entered:

ls -l

When you use such aliases, you may find that, from time to time, you want to run the original command, not the alias. For example, you may want to run ls without the -l option.

To suspend an alias temporarily — for one command only — type a \ (backslash) character at the beginning of the command name:

\ls

This tells the shell to run the actual command, not an alias. In our example, the shell will ignore the ls alias, and you will get the (default) short listing.

Jump to top of page

Alias Example:
Avoid Deleting the Wrong Files

In this section, I will show you how to combine an alias with a command recalled from the history list to produce an exceptionally handy tool.

Earlier in the chapter, we discussed an example in which we were thinking about using the rm (remove) command to delete all the files whose names match a particular pattern. The example we discussed was:

rm temp* extra?

To make sure we don't make a mistake, we should check the pattern we are using before we perform the actual deletion. We do this by using the same pattern with the ls command:

ls temp* extra?

If ls lists the files we want, we proceed with the deletion. Otherwise, we can try again with a different pattern, until we get what we want. In this way, we ensure that rm does exactly what we want it to do. This is important because once Unix deletes a file it is gone forever.

So let's say the ls command finds the files we want to delete. We could simply enter the rm command using the same pattern. However, what if we make a typing mistake? We might end up deleting a wrong file after all. A better idea is to let the shell do the work for us. To do so, we recall the ls command from the history list, change ls to rm, and then execute the modified command.

With a member of the Bourne shell family (Bash, Korn shell), we use:

fc -s ls=rm

To make this command easy to use, we define an alias named del:

alias del='fc -s ls=rm'

With a member of the C-Shell family (C-Shell, Tcsh), we would normally use:

^ls^rm

However, for technical reasons I won't go into, this won't work within an alias. Instead, we need to use the following command:

rm !ls:*

Obviously, this is tricky. (Unix is full of tricky commands.) Informally, we are asking the shell to extract the arguments from the most recent ls command, and use them to run an rm command. The effect is to run the rm command with the same arguments as the ls command.

To make this command easy to use, we define an alias. Notice that we quote the ! to preserve its meaning. (Does this make sense to you?)

alias del 'rm \!ls:*'

Once we have defined a del alias, we can use the following procedure to delete files that match a particular pattern. The nice thing is that the procedure is the same regardless of which shell we are using.

First, we enter an ls command with the pattern that describes the files you wish to delete. For example:

ls temp* extra?

If the pattern displays the names we expect, we enter:

del

That's all there is to it.

If the pattern does not work, we re-enter the ls command with a different pattern until we get what we want. Then we use del. In this way, it is impossible for us to delete the wrong files because of a mismatched pattern.

If you make a habit of using ls with a del alias in this way, I promise you that, one day, you will save yourself from a catastrophe. In fact, I have mathematical proof — using mathematical induction and hypergeometric functions —ios that this one trick alone is worth the price of this book. (Unfortunately, the explanation of the proof is beyond the scope of the book.)

Jump to top of page

Alias Example:
Reusing Commands From the History List

Earlier in the chapter, I explained that the Bourne shell family and the C-Shell family use different commands to access the history list. In particular, the Bourne shells (Bash, Korn shell) use the fc command, while the C-Shells (C-Shell, Tcsh) use the history and ! commands.

The original history list facility was written for the C-Shell. It was a breakthrough at the time and, in fact, it is still useful and easy to use. Later, the Korn shell introduced a much more powerful system using the fc command. Unfortunately, the syntax of fc was designed poorly and the details of the command itself are awkward to remember and to use. However, by using aliases, we can make fc look like the C-Shell system.

To start, we define an alias named history that uses fc with the -l (list) option to display lines from the history list:

alias history="fc -l"

To make it even easier, we can abbreviate history to h:

alias h=history

This is one of my favorite aliases, and I use it with every shell, even the C-Shell and Tcsh. After all, who wants to type the word history over and over?(*)

* Footnote

For that matter, who wants to type the word alias over and over? This is why I suggest you create an alias for the alias command itself:

alias a=alias

Next, we define an alias r (recall) to take the place of fc -s, the command that recalls and re-executes a line from the history list:

alias r="fc -s"

Now, whenever we want, it is easy to re-execute the last command we entered. We just use the r alias:

r

If we want to make a change, we simply specify an old pattern and a new pattern. For example, suppose we just typed the command:

vi tempfile

This starts vi to edit a file named tempfile. We decide to run the command again to edit a file named data. All we need to type is:

r tempfile=data

Working with a specific line in the history list is just as easy. Just specify the event number (line number). For example, let's say your history list looks like this:

20 cp tempfile backup
21 diff backup backup1
22 whoami
23 date
24 vi tempfile
25 vi data

You are wondering what time it is, so you want to re-execute the date command, event number 23:

r 23

Next, you want to re-execute command 20. However, first you want to change tempfile to data:

r 20 tempfile=data

If you specify one or more characters, the shell will re-execute the most recent command that starts with those characters. For example, to re-execute the last command that began with a di (in this case, number 21, the diff command), use:

r di

If you want to re-execute the date command, you can specify the last command that begins with d:

r d

With a little practice, such substitutions can save you a lot of time and effort.

To finish this section, let me give you some specific advice with respect to using the history, h and r aliases with your particular shell.

Bash: As I explained earlier in the chapter, Bash comes with both the fc command and the history and !! commands. However, you should create the h and r aliases for yourself:

alias h=history
alias r="fc -s"

Korn shell: The Korn shell uses fc, and it comes with the history and r aliases already defined, so you don't need to create them. For convenience, however, you should add the h alias:

alias h=history

C-Shell and Tcsh: As I explained earlier in the chapter, both these shells come with a history command as well as an easy way to modify and re-use commands from the history list. For convenience, all you need to add is the h alias:

alias h history

The beauty of these aliases is twofold: First, they make it easy to use the history list; second, they allow you to access the history list the same way regardless of which shell you are using.

Jump to top of page

Alias Example: Displaying Name of
Working Directory in Shell Prompt

The goal of this section is to solve a specific problem that pertains only to the C-Shell. However, we will be covering several concepts that are generally useful so, regardless of which shell you happen to use, I want you to read this section and think about the ideas that emerge from the discussion.

Earlier in the chapter, we talked about how to display the name of your working directory in your shell prompt. At the end of that discussion, I mentioned that the C-Shell does not have an easy way to do this. There is a complicated way to do so, however, that uses aliases, and that is what we are going to discuss here.

The discussion will involve directories, which we will cover in Chapter 24. For now, all you need to know is that directories hold files, and your "working directory" is the one in which you are currently working. You can change your working directory whenever you want and, when you do, it's handy to see the name of the new directory in your shell prompt, so you can keep track of where you are.

Displaying the name of your working directory in this way is easy with Bash (use \w), the Korn shell (use $PWD), and the Tcsh (use %~). Here are some sample commands that do the job. For readability, they display the name of the working directory in parentheses:

export PS1="(\w) bash$ "
export PS1='($PWD) ksh$ '
set prompt = "(%~) tcsh> "

The reason these commands work is that the shells automatically update the code or variable within the shell prompt whenever your working directory changes.

To be sure, the C-Shell has a PWD variable. However, if you put it in your shell prompt, you will find that the variable is not updated automatically. This is because the C-Shell is older than the other three shells, and it does not have this capability.

The approach to solving this problem is to use an alias that redefines the shell prompt every time you change your working directory. To start, we need to answer the question: Which command do we use to change our working directory? The answer is the cd (change directory) command.

We'll talk about cd in detail in Chapter 24. For now, I'll tell you that, to change to a new directory, you type cd followed by the name of the directory. For example, if you want to change to the bin directory, you would enter:

cd bin

You may remember from Chapter 12 that, at all times, the C-Shell maintains the name of your working directory in two different variables: cwd (a shell variable) and PWD (an environment variable). Whenever you use cd to change your working directory, these two variables are updated.

Thus, our plan is to create an alias to redefine cd so that it does two things: (1) Change the working directory according to whatever you specify, then (2) Use either the cwd or PWD variable to redefine the shell prompt to reflect the new working directory. The following alias does the job:

alias cd 'cd \!* && set prompt="($PWD)% "'

To understand how this works, you need to know that && separates two commands. The meaning of && is to run the first command and then, if it terminates normally, to run the second command. If the first command fails, the second command is not executed. In other words, if, for some reason, the cd command fails, there's no point in updating the prompt.

Our cd alias starts by executing the command:

cd \!*

The notation \!* refers to whatever arguments were typed on the original command line. In this way, the original arguments from the command line are passed to the cd command inside the alias. (This is a programming thing, so if this doesn't make sense to you, don't worry about it.)

If the first command terminates normally, the PWD variable will be updated by the shell automatically. We can then run the second command:

set prompt="($PWD)% "

This command changes the shell prompt to display the name of the working directory in parenthesis, followed by a % character, followed by a space. That's all there is to it.

The reason the whole thing works is that alias expansion is done before the shell parses and interprets the command line. For example, say we have defined the cd alias above and we enter:

cd bin

The first thing the shell does is expand the alias. Internally, the command line changes to:

cd bin && set prompt="($PWD)% "

Then the cd command is executed, followed by the set command.

Once you have the basic alias defined, you can make it more elaborate. For example, why not have the prompt display more than the working directory and a % character?

The following alias defines a more complex prompt in which we display the working directory in parentheses, a space, the name of the shell, the event number in square brackets, a % character, and a space:

alias cd 'cd \!* && set prompt = "($PWD) csh[\\!]% "'

A typical prompt defined in this way would look like this:

(/export/home/harley) csh[57]%

This is the type of alias you would put in an initialization file, so that your prompt will be updated for you automatically. We'll cover initialization files in Chapter 14.

One final comment. You will notice in the last example that the ! character is quoted twice (by two backslashes). The first backslash quotes the ! when it is parsed the first time, as part of the alias command. The second backslash quotes the ! when it is parsed the second time, as part of the set command.

This is a concept I want to make sure you understand: when something is being parsed more than once, you may need to quote it more than once. Please take a moment to think about this until it makes sense to you.

Jump to top of page



Exercises

Review Question #1:

What is an alphanumeric character?

What is a metacharacter?

Name three metacharacters and explain what they are used for.

Review Question #2:

Within the world of Unix, some characters have nicknames. For example, an apostrophe is usually referred to as a "quote" or a "single quote". What are the nicknames Unix people use for the following characters:

• Asterisk
• Enter/return
• Exclamation mark
• Period
• Quotation mark
• Vertical bar

Review Question #3:

What are the three different ways to quote characters? How do they differ?

Review Question #4:

What is a builtin command?

Where do you find the documentation for a builtin command?

Review Question #5:

What is the search path?

How can you display your search path?

Review Question #6:

What is the history list?

The simplest, most common use of the history list is to re-execute the previous command. How do you do this?

Using Bash or the Tcsh, how would you recall, edit, and then execute the previous command?

Review Question #7:

What is autocompletion?

How many different types of autocompletion are there?

Explain briefly what each type of autocompletion does.

Which type of autocompletion is the most important?

Applying Your Knowledge #1:

How do you modify the Bash shell prompt to display your userid, the working directory, and the current time?

How do you do the same for the Tcsh shell prompt?

Applying Your Knowledge #2:

What is command substitution?

Use command substitution to create a command that displays "These userids are logged in right now:" followed by a list of the userids.

Applying Your Knowledge #3:

Enter the command:

echo "I am a very smary person."

Using a history list tool, change this command to correct the spelling of "smart".

Applying Your Knowledge #4:

Your working directory contains the following files (only):

datanew dataold important phonenumbers platypus

Using autocompletion, what are the minimum number of characters you need to type to reference each of the files?

For Further Thought #1:

In this chapter, we have discussed several tools that help you enter commands quickly: the history list, autocompletion and aliases. These tools are complicated and take time to master. Some people can't be bothered to put in the time because, to them, it is not worth the effort. Other people embrace such tools with alacrity.

What type of person has a strong need to enter commands as quickly as possible?

For Further Thought #2:

What are the advantages of creating a great many specialized aliases? What are the disadvantages?

Jump to top of page