OReilly UNIX Power Tools 3rd Edition Oct 2002 ISBN 0596003307

  

Chapter 36. Shell Programming for the Initiated

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  36.1 Beyond the Basics

  This chapter has a bunch of tricks and techniques for programming with the Bourne shell. Some of them are documented but hard to find; others aren't documented at all. Here is a summary of this chapter's articles:

  The first group of articles is about making a file directly executable with #! on the first line. On many versions of Unix, an executable file can start with a first line like this:

  #!/path/to/interpreter

  The kernel will start the program named in that line and give it the file to read. Chris Torek's Usenet classic, xplains how started.

  #!

xplains that your "shell scripts" may not need a shell at all.

  The next bunch of articles are about processes and commands. The exec command, eplaces the shell with another process; it can also be used to change input/output redirection (see below). The : (colon) operator evaluates its arguments and returns a zero status — explains why you should care. Next are techniques for handling variables and parameters. Parameter substitution, explained in , is a compact way to test, set, and give default values for variables. You can use the parameter and Unix

  $0

  links to give the same script multiple names and make it do multiple things; see hows the easy way to get the last command- line argument has an easy way to remove all the command- line arguments.

  Four articles cover sh loops. A for loop usually reads a list of single arguments into a single shell variable shows how to make the

  for loop read from standard inputs techniques for making

  a for loop set more than one variable. The dirname and basename commands can be used to split pathnames with a loop; see . A

  

  Next is an assortment of articles about input/output. introduces open files and file descriptors — there's more to know about standard input/output/error than you might have realized! s a look at file-descriptor handling in the Bourne shell, swapping standard output and standard error.

  

  

  but that can cause some problems. shows one place scripts from stdin are useful: writing a script that creates another script as it goes. Next are two articles about miscellaneous I/O. One gotcha with the here- document operator (for redirecting input from a script file) is that the terminators are different in the Bourne and C shells; explains.

Section 36.20 shows how to turn off echoing while your script reads a "secret" answer such as a password.

  Two articles — s a quick reference to expr. .

   )

  test two things at once. Finally, s a simple technique for getting exclusive access to a file or other system resource.

  — JP

36.2 The Story of : # #!

  Once upon a time, there was the Bourne shell. Since there was only "the" shell, there was no trouble deciding how to run a script: run it with the shell. It worked, and everyone was happy. Along came progress and wrote another shell. The people thought this was good, for now they could choose their own shell. So some chose the one, and some the other, and they wrote shell scripts and were happy. But one day someone who used the "other" shell ran a script by someone who used the "other other" shell, and alas! it bombed spectacularly. The people wailed and called upon their Guru for help. "Well," said the Guru, "I see the problem. The one shell and the other are not compatible. We need to make sure that the shells know which other shell to use to run each script. And lo! the one shell has a `comment' called :, and the other a true comment called . I hereby decree that henceforth, the one shell will run

  #

  scripts that start with :, and the other those that start with ." And it was so, and

  # the people were happy.

  But progress was not finished. This time he noticed that only shells ran scripts and thought that if the kernel too could run scripts, this would be good, and the people would be happy. So he wrote more code, and now the kernel could run scripts but only if they began with the magic incantation , and if they told the

  #! kernel which shell ran the script. And it was so, and the people were confused.

  For the looked like a "comment." Though the kernel could see the and

  #! #!

  run a shell, it would not do so unless certain magic bits were set. And if the incantation were mispronounced, that too could stop the kernel, which, after all, was not omniscient. And so the people wailed, but alas! the Guru did not respond. And so it was, and still it is today. Anyway, you will get best results from a 4BSD machine by using

  #! /bin/sh

  or:

  #! /bin/csh

  as the first line of your script. is also helpful on occasion,

  #! /bin/csh -f and it's usually faster because csh won't read your .cshrc file ).

  — CT

36.3 Don't Need a Shell for Your Script? Don't Use One

  If your Unix understands files that start with:

  #!/interpreter/program

  (and nearly all of them do by now) you don't have to use those lines to start a shell, such as . If your script is just starting a program like awk,

  #!/bin/sh

  Unix can start the program directly and save execution time. This is especially useful on small or overloaded computers, or when your script has to be called over and over (such as in a loop). First, here are two scripts. Both scripts print the second word from each line of text files. One uses a shell; the other runs awk directly:

  % cat with_sh #!/bin/sh awk ' { print $2 } ' $* % cat no_sh #!/usr/bin/awk -f { print $2 } % cat afile one two three four five

  Let's run both commands and time ( them. (This is running on a very slow machine. On faster systems, this difference may be harder to measure — though the difference can still add up over time.)

  % time with_sh afile two 0.1u 0.2s 0:00 26% % time no_sh afile two 0.0u 0.1s 0:00 13%

  One of the things that's really important to understand here is that when the kernel runs the program on the interpreter line, it is given the script's filename as an argument. If the intepreter program understands a file directly, like /bin/sh does, nothing special needs to be done. But a program like awk or sed requires the -f option to read its script from a file. This leads to the seemingly odd syntax in the example above, with a call to with no following filename. The

  

awk -f

  script itself is the input file! One implication of this usage is that the interpreter program needs to understand as a comment, or the first interpreter-selection line itself will be acted upon

  #

  (and probably rejected by) the interpreter. (Fortunately, the shells, Perl, sed, and

  awk, among other programs, do recognize this comment character.)

  [One last comment: if you have GNU time or some other version that has a verbose mode, you can see that the major difference between the two invocations is in terms of the page faults each requires. On a relatively speedy Pentium

  III/450 running RedHat Linux, the version using a shell as the interpreter required more than twice the major page faults and more than three times as many minor page faults as the version calling awk directly. On a system, no matter how fast, that is using a large amount of virtual memory, these differences can be crucial. So opt for performance, and skip the shell when it's not needed. — SJC]

  —JP and SJC

  As xplains, you can use to run a script with

  #!/path/name

  the interpreter located at /path/name in the filesystem. The problem comes if a new version of the interpreter is installed somewhere else or if you run the script on another system that has a different location. It's usually not a problem for Bourne shell programmers: /bin/sh exists on every Unix-type system I've seen. But some newer shells — and interpreters like Perl — may be lurking almost anywhere (although this is becoming more and more standardized as Perl and other tools like it become part of standard Linux distributions and the like). If the interpreter isn't found, you'll probably get a cryptic message like

  , where scriptname is the name

  scriptname: Command not found of the script file.

  The env command will search your PATH ), replace itself) with the interpreter. If you want to try this, type ; env will find and run ls for you. This is pretty useless

  env ls

  when you have a shell around to interpret your commands — because the shell can do the same thing without getting env involved. But when the kernel interprets an executable file that starts with , there's no shell (yet!). That's

  #!

  where you can use env. For instance, to run your script with zsh, you could start its file with:

  #!/usr/bin/env zsh ...zsh script here...

  The kernel execs /usr/bin/env, then env finds and execs the zsh it found. Nice trick, eh? What do you think the problem is? (You have ten seconds... tick, tick, tick...) The catch is: if the env command isn't in /usr/bin on your system, this trick won't work. So it's not as portable as it might be, but it's still handy and probably still better than trying to specify the pathname of a less common interpreter like zsh.

  Running an interpreter this way can also be a security problem. Someone's PATH might be wrong; for instance, it might execute some random command named

  zsh in the user's bin directory. An intruder could change the PATH to make the script use a completely different interpreter with the same name.

  One more problem worth mentioning: you can't specify any options for the interpreter on the first line. Some shell options can be set later, as the script starts, with a command like set, shopt, and so on — check the shell's manual page.

  Finally, understand that using env like this pretty much erases any performance gains you may have achieved using the trick in the previous article.

  —JP and SJC

36.5 The exec Command

  The exec command executes a command in place of the current shell; that is, it terminates the current shell and starts a new process ( ) in its place. Historically, exec was often used to execute the last command of a shell script. This would kill the shell slightly earlier; otherwise, the shell would wait until the last command was finished. This practice saved a process and some memory.

  (Aren't you glad you're using a modern system? This sort of conservation usually isn't necessary any longer unless your system limits the number of processes each user can have.)

  exec can be used to replace one shell with another shell: % exec ksh $

  without incurring the additional overhead of having an unused shell waiting for the new shell to finish.

  exec also manipulates file descriptors ( in the Bourne shell.

  When you use exec to manage file descriptors, it does not replace the current process. For example, the following command makes the standard input of all commands come from the file formfile instead of the default place (usually, your terminal):

  exec < formfile —ML and JP

36.6 The Unappreciated Bourne Shell ":" Operator

  Some people think that the Bourne shell's : is a comment character. It isn't, really. It evaluates its arguments and returns a zero exit status ( . Here are a few places to use it:

  

   process each time around the loop (as it does when you use

  while

  ):

  true while : do commands done

  (Of course, one of the commands will probably be break, to end the loop eventually. This presumes that it is actually a savings to have the break test inside the loop body rather than at the top, but it may well be clearer under certain circumstances to do it that way.) When you want to use the else in an if ( but leave the then empty, the : makes a nice "do-nothing" place filler:

  if something then : else commands fi

  If your Bourne shell doesn't have a true comment character (but nearly all

  #

  of them do nowadays), you can use : to "fake it." It's safest to use quotes so the shell won't try to interpret characters like or in your "comment":

  > | : 'read answer and branch if < 3 or > 6'

  Finally, it's useful with parameter substitution ( like or . For instance, using this line in your

  ${var?} ${var=default}

  script will print an error and exit if either the USER or HOME variables aren't set:

  : ${USER?} ${HOME?}

  — JP

36.7 Parameter Substitution

  The Bourne shell has a handy set of operators for testing and setting shell variables. They're listed in .

  

Table 36-1. Bourne shell parameter substitution operators

Operator Explanation If var is not set or is empty, use default instead.

  ${var:-default} If var is not set or is empty, set it to default and use that value. ${var:=default}

  If var is set and is not empty, use instead. Otherwise, use ${var:+instead} nothing (null string).

  If var is set and is not empty, use its value. Otherwise, print message, if any, and exit from the shell. If message is missing,

  ${var:?message} print a default message (which depends on your shell).

  If you omit the colon (:) from the expressions in , the shell doesn't check for an empty parameter. In other words, the substitution happens whenever the parameter is set. (That's how some early Bourne shells work: they don't understand a colon in parameter substitution.) To see how parameter substitution works, here's another version of the bkedit

  script ( ):

• #!/bin/sh if cp "$1" "$1.bak" then

  ${VISUAL:-/usr/ucb/vi} "$1" exit # Use status from editor else

echo "`basename $0` quitting: can't make backup?" 1>&2

exit 1 fi

  If the VISUAL ) environment variable is set and is not empty, its value (such as /usr/local/bin/emacs) is used and the command line becomes . If VISUAL isn't set, the command

  /usr/local/bin/emacs "$1" line defaults to .

  /usr/ucb/vi "$1"

  You can use parameter substitution operators in any command line. You'll see them used with the colon (:) operator ), checking or setting default values. There's an example below. The first substitution ( ) leaves empty because the

  ${nothing=default} $nothing

  variable has been set. The second substitution sets to default

  $nothing

  because the variable has been set but is empty. The third substitution leaves set to stuff:

  $something

• nothing= something=stuff : ${nothing=default} : ${nothing:=default} : ${something:=default}

  Several Bourne-type shells have similar string editing operators, such as

  pattern

  ## . They're useful in shell programs, as well as on the

  ${var }

  command line and in shell setup files. See your shell's manual page for more details.

  — JP

36.8 Save Disk Space and Programming:

  Multiple Names for a Program

  If you're writing: several programs that do the same kinds of things, programs that use a lot of the same code (as you're writing the second, third, etc., programs, you copy a lot of lines from the first program), or a program with several options that make big changes in the way it works,

  

  through case or test commands, work in different ways. For instance, the Berkeley Unix commands ex, vi, view, edit, and others are all links to the same executable file. This takes less disk space and makes maintenance easier. It's usually sensible only when most of the code is the same in each program. If the program is full of name tests and lots of separate code, this technique may be more trouble than it's worth.

  Depending on how the script program is called, this name can be a simple relative pathname like or — it can also be an absolute

  prog ./prog pathname like xplains pathnames). /usr/joe/bin/prog

  There are a couple of ways to handle this in a shell script. If there's just one main piece of code in the script, as in the lf script, a case that tests might be best.

  $0

• The asterisk ( ) wildcard at the start of each case (see handles the different pathnames that might be used to call the script:

  case "$0" in

• name1) ...do this when called as name1... ;;
• name2) ...do this when called as name2... ;;

  ...

• *) ...print error and exit if $0 doesn't match...

  ;; esac

  You might also want to use basename ) to strip off any leading pathname and store the cleaned-up in a variable called myname. You can test

  $0

  anywhere in the script and also use it for error messages:

  $myname myname=`basename $0` ... case "$myname" in ... echo "$myname: aborting; error in xxxxxx" 1>&2 ...

  — JP

  36.9 Finding the Last Command-Line Argument

  Do you need to pick up the last parameter , from the parameter list $1 $2 ... on the command line? It looks like would do it:

  

eval \$$#

eval $ set foo bar baz $ eval echo \$$# baz

  except for a small problem with sh argument syntax:

  $ set m n o p q r s t u v w x $ echo $11

  m1

  means , not . Trying directly gives

  $11 ${1}1 ${11} ${11} bad

  . (More recent shells, such as bash, do support the

  substitution ${11}

  syntax, however, to arbitrary lengths. Our copy of bash, for example, allowed at least 10240 command line arguments to with recall of the last via

  set ). Your mileage may vary. ${10240}

  The only reliable way to get at the last parameter in the Bourne shell is to use something like this:

  for i do last="$i"; done

  The for loop assigns each parameter to the shell variable named last; after the loop ends, will have the last parameter. Also, note that you won't need

  $last this trick on all sh-like shells. The Korn shell, zsh, and bash understand .

  ${11} — CT

  36.10 How to Unset All Command-Line Parameters The shift ( command "shifts away" one command-line parameter.

  You can shift three times if there are three command-line parameters. Many shells also can take an argument, like shift 3, that tells how many times to shift; on those shells, you can shift $# ) to unset all parameters. The portable way to unset all command-line parameters is probably to set ( a single dummy parameter, then shift it away:

• set x shift

  Setting the single parameter wipes out whatever other parameters were set before.

  — JP

  36.11 Standard Input to a for Loop

  An obvious place to use a Bourne shell for loop ) is to step through a list of arguments — from the command line or a variable. But combine the loop with backquotes ( ), and the loop will step through the words on standard input.

  Here's an example:

  for x in `cat` do ...handle $x done

  Because this method splits the input into separate words, no matter how many words are on each input line, it can be more convenient than a while loop running the read command. When you use this script interactively, though, the loop won't start running until you've typed all of the input; using while read will run the loop after each line of input.

  — JP

  36.12 Making a for Loop with Multiple Variables

  The normal Bourne shell for loop ( lets you take a list of items, store the items one by one in a shell variable, and loop through a set of commands once for each item:

  for file in prog1 prog2 prog3 do ...process $file done

  I wanted a for loop that stores several different shell variables and makes one pass through the loop for each set of variables (instead of one pass for each item, as a regular for loop does). This loop does the job:

  set for bunch in "ellie file16" "donna file23" "steve file34" do # PUT FIRST WORD (USER) IN $1, SECOND (FILE) IN $2... set $bunch mail $1 < $2 done

  If you have any command-line arguments and still need them, store them in another variable before you use the set command. Or you can make the loop this way:

  for bunch in "u=ellie f=file16 s='your files'" \

"u=donna f=file23 s='a memo'" "u=steve f=file34 s=report"

do # SET $u (USER), $f (FILENAME), $s (SUBJECT): eval $bunch mail -s "$s" $u < $f done

  This script uses the shell's eval ) command to rescan the contents of the bunch variable and store it in separate variables. Notice the single quotes, as in ; this groups the words for eval. The shell removes the

  s='your files' single quotes before it stores the value into the s variable.

  — JP

36.13 Using basename and dirname

  Almost every Unix command can use relative and absolute pathnames

  ( to find a file or directory. There are times you'll need part of a pathname — the head (everything before the last slash) or the tail (the name after the last slash). The utilities basename and dirname, available on most Unix systems, handle that.

36.13.1 Introduction to basename and dirname

  The basename command strips any "path" name components from a filename, leaving you with a "pure" filename. For example:

  % basename /usr/bin/gigiplot gigiplot % basename /home/mikel/bin/bvurns.sh bvurns.sh basename can also strip a suffix from a filename. For example: % basename /home/mikel/bin/bvurns.sh .sh bvurns

  The dirname command strips the filename itself, giving you the "directory" part of the pathname:

  % dirname /usr/bin/screenblank /usr/bin % dirname local .

  If you give dirname a "pure" filename (i.e., a filename with no path, as in the second example), it tells you that the directory is . (the current directory).

  dirname and basename have a bug in some implementations. They don't recognize the second argument as a filename suffix to strip.

  Here's a good test:

  % basename 0.foo .foo If the result is , your basename implementation is good. If the answer is , the implementation is bad. If basename doesn't

  0.foo work, dirname won't, either.

36.13.2 Use with Loops

  Here's an example of basename and dirname. There's a directory tree with some very large files — over 100,000 characters. You want to find those files, run

  split ( on them, and add huge. to the start of the original filename.

  By default, split names the file chunks xaa, xab, xac, and so on; you want to use the original filename and a dot (.) instead of x:

   || exit

for path in `find /home/you -type f -size +100000c -print`

do cd `dirname $path` || exit filename=`basename $path` split $filename $filename. mv -i $filename huge.$filename done

  The find command will output pathnames like these:

  /home/you/somefile /home/you/subdir/anotherfile

  (The absolute pathnames are important here. The cd would fail on the second pass of the loop if you use relative pathnames.) In the loop, the cd command uses dirname to go to the directory where the file is. The filename variable, with the output of basename, is used several places — twice on the split command line.

  If the previous code results in the error ,

  command line too long

  replace the first lines with the two lines below. This makes a redirected-input loop:

  find /home/you -type f -size +100000c -print | while read path —JP and ML

36.14 A while Loop with Several Loop Control Commands

  I used to think that the Bourne shell's while loop ( ) looked like this, with a single command controlling the loop:

  while command do ...whatever done

  But command can actually be a list of commands. The exit status of the last command controls the loop. This is handy for prompting users and reading answers. When the user types an empty answer, the read command returns "false" and the loop ends:

  

while echo -e "Enter command or CTRL-d to quit: \c"

read command do ...process $command done

  You may need a -e option to make echo treat escaped characters like the way

  \c

  you want. In this case, the character rings the terminal bell, however your terminal interprets that (often with a flash of the screen, for instance.)

  Here's a loop that runs who and does a quick search on its output. If the grep returns nonzero status (because it doesn't find in ), the

  $who $tempfile

  loop quits — otherwise, the loop does lots of processing:

  while who > $tempfile grep "$who" $tempfile >/dev/null do ...process $tempfile... done —JP and SJC

36.15 Overview: Open Files and File Descriptors

  This introduction is general and simplified. If you're a technical person who needs a complete and exact description, read a book on Unix programming. Unix shells let you redirect the input and output of programs with operators such as and . How does that work? How can you use it better? Here's an overview.

  > |

  When the Unix kernel starts any process ) — for example, grep, ls, or a shell — it sets up several places for that process to read from and write to, as shown in .

  

Figure 36-1. Open standard I/O files with no command-line

redirection These places are called open files. The kernel gives each file a number called a

  file descriptor. But people usually use names for these places instead of the

  numbers: The standard input or stdin (File Descriptor (F.D.) number 0) is the place where the process can read text. This might be text from other programs (through a pipe, on the command line) or from your keyboard. The standard output or stdout (F.D. 1) is a place for the process to write its results.

  The standard error or stderr (F.D. 2) is where the process can send error messages.

  By default, ashows, the file that's opened for stdin, stdout, and

  

stderr is /dev/tty — a name for your terminal. This makes life easier for users —

  and programmers, too. The user doesn't have to tell a program where to read or write because the default is your terminal. A programmer doesn't have to open files to read or write from (in many cases); the programs can just read from stdin, write to stdout, and send errors to stderr.

  It gets better. When the shell starts a process (when you type a command at a prompt), you can tell the shell what file to "connect to" any of those file descriptors. For example, shows what happens when you run grep and make the shell redirect grep's standard output away from the terminal to a file named grepout.

  

Figure 36-2. Standard output redirected to a file Programs can read and write files besides the ones on stdin, stdout, and stderr. For instance, in , grep opened the file somefile itself — it didn't use any of the standard file descriptors for somefile. A Unix convention is that if you don't name any files on the command line, a program will read from its standard input. Programs that work that way are called filters.

  All shells can do basic redirection with stdin, stdout, and stderr. But as you'll see in he Bourne shell also handles file descriptors 3 through 9 (and

  

bash and the other newer shells can handle arbitrary numbers of file descriptiors,

  up to whatever happens to be set). That's useful sometimes:

  ulimit -n

  Maybe you have a few data files that you want to keep reading from or writing to. Instead of giving their names, you can use the file descriptor numbers. Once you open a file, the kernel remembers what place in the file you last read from or wrote to. Each time you use that file descriptor number while the file is open, you'll be at the same place in the file. That's especially nice when you want to read from or write to the same file with more than one program. For example, the line command on some Unix systems reads one line from a file — you can call line over and over, whenever you want to read the next line from a file. Once the file has been opened, you can remove its link (name) from the directory; the process can access the file through its descriptor without using the name. When Unix starts a new subprocess, the open file descriptors are given to that process. A subprocess can read or write from file descriptors opened by its parent process. A redirected-I/O loop, as discussed i , takes advantage of this.

  — JP 36.16 n>&m: Swap Standard Output and Standard Error

  By default, a command's standard error goes to your terminal. The standard output goes to the terminal or is redirected somewhere (to a file, down a pipe, into backquotes). Sometimes you want the opposite. For instance, you may need to send a command's standard output to the screen and grab the error messages (standard error) with backquotes. Or you might want to send a command's standard output to a file and the standard error down a pipe to an error-processing command. Here's how to do that in the Bourne shell. (The C shell can't do this, although

  tcsh can.)

  File descriptors 0, 1, and 2 are, respectively, the standard input, standard output, and standard error explains). Without redirection, they're all associated with the terminal file /dev/tty ). It's easy to redirect any descriptor to any file — if you know the filename. For instance, to redirect file descriptor 2 to errfile, type:

  $ command 2>errfile

  You know that a pipe and backquotes also redirect the standard output: $ command | ...

  $ var=` command `

  But there's no filename associated with the pipe or backquotes, so you can't use the redirection. You need to rearrange the file descriptors without knowing

  2>

  the file (or whatever) that they're associated with. Here's how. You may find it useful to run this short Perl script, which simply prints "stdout" to standard output, and "stderr" to standard error:

  #!/usr/bin/perl print STDOUT "stdout\n"; print STDERR "stderr\n";

  Let's start slowly. We will combine both standard output and standard error, sending them both as output, to be used as the input to a pipe or as the output of backquotes. The Bourne shell operator n rearranges the files and file

  >&m

  descriptors. It says, "Make file descriptor n point to the same file as file descriptor m." Let's use that operator on the previous example. We'll send standard error to the same place standard output is going: $ command 2>&1 | ...

  $ var=` command 2>&1`

  In both those examples, means "send standard error (file descriptor 2) to

  2>&1

  the same place standard output (file descriptor 1) is going." Simple, eh? You can use more than one n operator. The shell reads them left-to-right

  >&m before it executes the command.

  "Oh!" you might say. "To swap standard output and standard error — make

  stderr go down a pipe and stdout go to the screen — I could do this!" $ command 2>&1 1>&2 | ... wrong...

  Sorry, Charlie. When the shell sees , the shell first does .

  2>&1 1>&2 2>&1

  You've seen that before — it makes file descriptor 2 (stderr) go the same place as file descriptor 1 (stdout). Then the shell does . It makes stdout ( ) go

  1>&2

  1

  the same place as stderr ( )... but stderr is already going the same place as

  2 stdout, down the pipe.

  This is one place the other file descriptors, 3 through 9 (and higher in bash), come in handy. They normally aren't used. You can use one of them as a "holding place," to remember where another file descriptor "pointed." For example, one way to read the operator is "make point the same place

  3>&2

  3

  as ." After you use to grab the location of , you can make point

  2 3>&2

  2

  2 somewhere else. Then make point where used to (where points now).

  1

  2

  3 We'll take that step-by-step below. The command line you want is one of these: $ command 3>&2 2>&1 1>&3 | ...

  $ var=` command 3>&2 2>&1 1>&3`

  How does it work? eak the second command line (with the backquotes) into the same steps the shell follows as it rearranges the file descriptors. You can try these on your terminal, if you'd like. Each figure adds another n operator and shows the location of each file descriptor after

  >&m that operator.

  

Figure 36-3. File descriptors before redirection

Figure 36-4. File descriptors after 3>&2 redirection The figures use a grep command reading two files. afone is readable, and grep finds one matching line in it; the line is written to the standard output. bfoen is misspelled and so is not readable; grep writes an error message to the standard error. In each figure, you'll see the terminal output (if any) just after the variable- setting command with the backquotes. The text grabbed by the backquotes goes into the shell variable; the echo command shows that text.

  

Figure 36-5. File descriptors after 3>&2 2>&1 redirection

  Bhe redirection is correct. Standard output goes to the screen, and standard error is captured by the backquotes.

  

Figure 36-6. File descriptors after 3>&2 2>&1 1>&3 redirection Open files are automatically closed when a process exits, but it's safer to close the files yourself as soon as you're done with them. That way, if you forget and use the same descriptor later for something else (for instance, use F.D. 3 to redirect some other command, or a subprocess uses F.D. 3), you won't run into conflicts. Use m to close input file descriptor m and m to close output

  <&- >&-

  file descriptor m . If you need to, you can close standard input with and

  <&- standard output with . >&-

  — JP

  36.17 A Shell Can Read a Script from Its Standard Input, but...

  Q: What is the difference between sh < file and sh file ? A:The first way keeps the script from reading anything else from its input.

  Consider the stdin-demo script:

  while read word do echo $word | sed s/foo/bar/ done

  If run as , it will read from your terminal, replacing

  sh stdin-demo foo

  with . If run as , it will exit right away, since after reading the script, there's no input left.

  — CT

36.18 Shell Scripts On-the-Fly from Standard Input

  The shell can read commands from its standard input or from a file. To run a series of commands that can change, you may want to use one program to create the command lines automatically — and pipe that program's output to a shell, which will run those "automatic" commands.

  

  Here's an exampleou want to copy files from a subdirectory and all its subdirectories into a single directory. The filenames in the destination directory can't conflict; no two files can have the same name. An easy way to name the copies is to replace each slash ( ) in the file's relative pathname with a minus

  /

  sign ( ). For instance, the file named lib/glob/aprog.c would be copied to a file named lib-glob-aprog.c. You can use sed ( to convert the filenames and output cp commands like these:

  cp from/lib/glob/aprog.c to/lib-glob-aprog.c cp from/lib/glob/aprog.h to/lib-glob-aprog.h ...

  However, an even better solution can be developed using nawk. The following example uses find ) to make a list of pathnames, one per line, in and below the copyfrom directory. Next it runs nawk to create the destination file pathnames (like to ) and write the

  /lib-glob-aprog.c

  completed command lines to the standard output. The shell reads the command lines from its standard input, through the pipe. This example is in a script file because it's a little long to type at a prompt. But you can type commands like these at a prompt, too, if you want to:

  #!/bin/sh find copyfrom -type f -print | awk '{ out = $0 gsub("/", "-", out) sub("^copyfrom-", "copyto/", out) print "cp", $0, out }' | sh

  If you change the last line to , the shell's verbose option (

  sh -v

  will show each command line before executing it. If the last line has ,

  sh -e

  the shell will quit immediately after any command returns a nonzero exit status ( — that might happen, for instance, if the disk fills up and cp can't make the copy. Finally, you may need to use nawk rather than awk, depending on your system.

  — JP

  

36.19 Quoted hereis Document Terminators: sh

Versus csh

  When you need to quote your hereis document ( terminators, there's an annoying problem: sh and csh demand different conventions. If you are using sh, you must not quote the terminator. For example,

• #! /bin/sh cat << 'eof' Hi there. eof

  If you are using csh, however, you must quote the terminator. The following script prints three lines, not one:

• #! /bin/csh cat << \eof

  Hi. You might expect this to be the only line, but it's not.

  eof 'e'of \eof

  — CT

36.20 Turn Off echo for "Secret" Answers

  When you type your password, Unix turns off echoing so what you type won't show on the screen. You can do the same thing in shell scripts with

  stty - . echo

   stty read

• #!/bin/sh ... trap 'stty echo; exit' 0 1 2 3 15 # use the right echo for your Unix: echo "Enter code name: \c" #echo -n "Enter code name: " stty -echo read ans stty echo ...

  The response is stored in . The trap helps to make sure

  $ans

  that, if the user presses CTRL-c to abort the script, characters will be echoed again.

  — JP

36.21 Quick Reference: expr

  

expr is a very handy tool in shell programming, since it provides the ability to

  evaluate a wide range of arithmetic, logical, and relational expressions. It evaluates its arguments as expressions and prints the result.

36.21.1 Syntax

  Here's the syntax. The [brackets] mean "optional"; don't type the brackets:

  

arg1 operator arg2 operator arg3

expr [ ... ]

  Arguments and operators must be separated by spaces. In many cases, an argument is an integer, typed literally or represented by a shell variable. There are three types of operators: arithmetic, relational, and logical.

  Exit status ( values for expr are 0 if the expression evaluates

  nonzero and non-null, 1 if the expression evaluates to 0 or null, and 2 if the expression is invalid.

  Arithmetic operators

  Use these to produce mathematical expressions whose results are printed:

• Add arg2 to arg1 .
• arg2 from arg1 .

  Subtract

• Multiply the arguments.

  /

  Divide arg1 by arg2 .

  % Take the remainder when arg1 is divided by arg2 (modulus).

  Addition and subtraction are evaluated last, unless they are grouped inside

• parentheses. The symbols , , and have meaning to the shell, so they

  ( ) must be escaped (preceded by a backslash or enclosed in quotes).

  Relational operators

  Use these to compare two arguments. Arguments can also be words, in which case comparisons assume a z and A Z. If the comparison

  < <

  statement is true, expr writes 1 to standard output ( ; if false, it writes 0. The symbols and must be escaped.

  > < =

  Are the arguments equal?

  !=

  Are the arguments different?

  >

  Is arg1 greater than arg2 ?

  >=

  Is arg1 greater than or equal to arg2 ?

  <

  Is arg1 less than arg2 ?

  <=

  Is arg1 less than or equal to arg2 ?

  Logical operators

  Use these to compare two arguments. Depending on the values, the result written to standard output can be arg1 (or some portion of it), arg2 , or

  0. The symbols and must be escaped.

  | & |

  Logical OR; if arg1 has a nonzero (and non-null) value, the output is arg1 ; otherwise, the output is arg2 .

  &

  Logical AND; if both arg1 and arg2 have a nonzero (and non- null) value, the output is arg1 ; otherwise, the output is 0.

  :