The Shell Fundamentals

The shell scripts can be executed in different ways. On Linux, every user needs a shell to interact with the operating system. If the bash shell is used as the default login shell for a user, then every command (or program or script) executed by the user in that session is executed by the bash shell. If the user is executing a script without specifying the shell to execute it, then bash assumes that it is a bash shell script, and the script is validated against the bash scripting language syntax. If the script does not conform to the bash scripting syntax, then error messages will be sent to the standard output. For example, if your login shell is bash shell, and you attempt to run a Korn shell script without specifying that Korn shell should execute the script, then bash shell will throw errors indicating the syntax rule violations. The same rule applies if Korn shell is your login shell and you attempt to execute a bash shell script without specifying that bash should execute the script.

For a program or script to execute, the user should have execute permission on the executable file or should belong to the group that owns the file and has execute permission on it. In addition, the shell needs to be specified where the file is located. This can be done in different ways. If the executable file name is specified with the qualifying pathname (i.e., the full directory name containing the file), the shell can execute the program. If the file is located in one of the directories specified in the PATH environment variable, the shell searches for the file in the directories in the order in which the directory names are specified in the PATH variable. The search stops at a point when the file is first located in one of the PATH directories, and the file is executed. If the file is located in the current directory (or in the parent directory of the current directory), it should be executed with commands similar to the following.

    $ ./<execfile>
    $ ../<execfile>

In the first example, the execfile is assumed to be in the current directory, and in the second example, it is assumed to be in the parent directory of the current directory. If files from a particular directory are frequently executed, then it may be a good idea to add that directory to the PATH variable. Before adding a directory to the PATH directory list, it is always good to check if it is already set in order to avoid duplicate entries. To see the directories set in the PATH variable, type the command shown below at the shell prompt.

    $ echo $PATH

The echo command throws the value stored in the environment variable (PATH variable in this case) to the standard output. To access the contents of a shell variable, the variable should be prefixed with the $ symbol. The output of the above command might look something like the one shown below.

    /usr/local/bin:/usr/bin:/usr/X11R6/bin:/sbin:/bin:/usr/sbin

The value set in the PATH command may be very long; for presentation sake only a few directories are shown here. To add a directory (say /usr/satya/bin) at the end of the PATH variable, the following command should be executed.

    $ export PATH=$PATH:/usr/satya/bin

It can be noticed that the new directory is separated from the current value of the PATH variable with the : delimiter, and the new value of the PATH variable is evaluated from the righthand side of the expression and set to the PATH variable. The export command makes the new value of the variable available in the current shell session and all the child processes that will be created by the current shell session. The export command can be separately executed after setting the variable value as shown below.

    $ PATH=$PATH:/usr/satya/bin
    $ export PATH

It should also be noted that whitespaces and tabs are not permitted while setting the PATH variable. The new directory may also be added at the beginning of the PATH variable (preceding the list of all the current directories set in the variable), as shown below.

    $ export PATH=/usr/satya/bin:$PATH

Usually there is no need to add custom directories before the system PATH, due to possible malfunctioning of the system or a risk of compromising system security. If it is an absolute requirement that your own directory should precede the system PATH, you should ensure that the specific directory should have very restricted access permissions. The value set to the environment variable in a particular shell session is available only in that session. To make the change permanent for a user, the changes must be made in the login shell script, as described in the following subsection.

At this point, you will need to understand an important feature of the shell, and that is, the precedence rule that the shell follows in order to identify which program to execute while executing a command line. There are built-in commands supported by the shell. In addition, there are commands that are individual programs (or application executables) and shell scripts. These are all executable files, and some of them might have same names as others; there will not be any name conflicts as long as they are uniquely identified by their respective inodes. Also, the bash shell supports functions within shell scripts, which might have the same names as one of the executable files on the system. This raises the question about the precedence rule that the shell should follow when it encounters two or more executable entities (commands, programs, scripts, and functions) having the same name. The precedence rule is: first functions, then built-in commands, then scripts, and finally executable application programs. However, the precedence can be overridden, as is explained later in the chapter.

The terminal-based programs usually interact with the standard input (the keyboard) for receiving the user input and the standard output (the terminal window) for displaying output to the user. Errors are displayed to the standard error stream. Although the standard input stream is identified by the file descriptor 0, the standard output stream is identified by the file descriptor 1, and the standard error stream is identified by the file descriptor 2. This can be noticed from the device descriptions in the /dev directory, which contains entries for stdin (standard input), stdout (standard output), and stderr (standard error) streams as fd/0, fd/1, and fd/2, respectively.

However, it is likely that the input might be available in a file, or the output is required to be saved to a file. This is performed by the I/O redirection mechanism. The < redirection symbol represents input redirection from a file or another input stream. Similarly, the > redirection symbol represents output redirection to a file or another stream. Using pipes, the standard output of a command can be redirected to the standard input of another command. The following examples demonstrate these techniques.

    $ ls –l | more
    $ ls –l > listing.txt

The first line shows that the output of the ls –l command is redirected to the more command; so that the output is paginated and can be scrolled using the standard more command features. The second command shows that the output is redirected to a file and hence is not displayed on the screen. Later, the file can be viewed using the more command or edited using a text editor. When the > redirection symbol is used, only the standard output is redirected to the specified file or stream. Typically, the program streams its output to the standard output (file descriptor 1) and streams errors to the standard error (file descriptor 2). In order to redirect the standard error to the same stream as the standard output, one of the following forms of the command should be executed.

    $ ls –l &> listing.txt
    $ ls –l > listing.txt 2>&1

Here the ls –l command is shown as an example; however, the same concept can be used to capture standard output and/or standard error messages to files/streams. When the > redirection symbol is used, the corresponding file or stream is opened for writing, and the existing contents of the stream are erased automatically. However, the >> redirection stream can be used to append to the stream (if it already exists) or create the stream (if it does not exist). If the >> redirection symbol is used to capture both the standard output and standard error streams, then the second form of the syntax (from the two forms shown before) should be used, and the first form does not work. Usually, applications that write message and error log files use the >> redirection symbol to append to the existing file, and the file accumulates messages over a period of time (e.g., a week or month) and then it is closed, and the next file is opened for the next collection period.

Similar to the output redirection, the input can also be redirected using command syntax similar to that shown here. In the example, it is assumed that process_listing is a program that reads from standard input and can be redirected to read from the file listing.txt.

    $ process_listing < listing.txt

So far, the discussion has focused on how to execute a program or shell script from the shell prompt. However, a program or script may be invoked from another shell script. It is also possible to introduce controlled execution of the programs, check for error codes returned by called programs, and modularize the scripts by grouping code into functions. All these topics are discussed in the subsections that follow.

The User Login Environment

As noted previously, users are associated with a default login shell, which is set in the user profile where the user login id is created. This information is stored in the /etc/passwd file and can only be updated by the super user root. If you desire to have a particular shell as the default one permanently, you can get this done by the super user. Otherwise, from one shell prompt, another shell can be invoked by just executing that program. It should also be noted that the Linux operating system has many system-level user ids created during installation automatically, and these user ids usually do not permit anyone to log in to the command-line mode. Therefore they do not have an associated login shell in the /etc/passwd file.

The shell can be invoked in one of the three modes: as an interactive login shell, an interactive non-login shell, or a noninteractive shell. When users log in to their default shells, typically the shell is invoked in the first of the three modes, and a set of shell variables and options are preset and available to the user. Some of them are predefined in the shell, and most of them are configurable in the shell login scripts. The default name of the user-specific login script for bash is either .bash_profile or .bash_login or .profile, and is located in the user’s home directory, such as /home/Michael/.bash_profile or /home/Michael/.bash_login or /home/Michael/.profile for a user id named Michael. When the user logs into the system, the shell first executes the default shell profile script /etc/profile. This is the master file (to be executed by the shell every time a user logs into the operating system) and should only be edited carefully by the system administrator. Errors in this file could potentially bar every user from logging in to the system. The /etc/profile typically contains commands to set the default shell variables, including the default PATH setting. After the standard settings are done, the script may execute additional scripts to add some application-specific directories to the PATH value. The process of making these additional settings may be different across different Linux distributions. On Red Hat Linux 8.0/9.0 and SuSE Linux 8.1/8.2, these additional settings are described in individual scripts under the /etc/profile.d directory. For example, every Linux distribution usually comes with a Java2 SDK (or at least the Java Runtime Environment), and therefore includes a script that determines the Java/JRE bin directory and adds it to the standard PATH. If it is necessary to add a new directory to the PATH variable across the system (for all the users), just create a new script with ‘.sh’ extension, place it in the /etc/profile.d directory, and change the file permissions to include execute permission for everyone. However, if a setting is exclusive to a user and not intended to be valid for all the users system-wide, then such settings should not be placed in this directory; rather they should be placed in the user’s login/session configuration script as mentioned in the description that follows.

Once the default login script is executed, the shell attempts to locate the customized (user-specific) login file such as .bash_profile, .bash_login, or .profile in the home directory of the user, in the same order. If the shell is able to locate at least one of these files, then the commands contained in that script (the first of the three files found) are executed. If none of these files is present in the user’s home directory, then the login process ends with the default login script itself.

Every time a new session of the shell is invoked in the second mode (i.e., interactive non-login mode), the shell attempts to run the session configuration script .bashrc, if it exists in the user’s home directory. The .bashrc file contains additional/updated settings of environment variables including the PATH. The purpose of this file is to provide settings that should be done when an instance of the shell is created for user interaction. If this file does not exist for a user, there is no harm, as the login script was already executed when the user logged in. The shell is invoked in the third mode when executing a script noninteractively. In this mode, shell does not use the value of the PATH environmental variable to search for the file. Therefore, it is always necessary to qualify the executable file with full pathname.

The shell maintains a history of previously used commands, which can be displayed by executing the history command, as shown in examples below.

    $ history
    $ history 25

When the history command is executed with no arguments, all the commands currently stored in the memory are displayed. If a numeric argument is provided as in the second example, it displays only the most recent requested item of history commands. The sole purpose of maintaining the history of commands is to improve user productivity by providing a way to navigate through the command list and also to move the cursor from left to right within a single command, very similar to the way a text file is navigated; in this analogy, the history file may be viewed as a text file, and each command is viewed as a line in the text file. In order to do a productive navigation of the history commands, we can choose our favorite text editor as template, so that we can use that editor’s navigation and edit commands to choose a particular command and modify it before execution. By default, the Emacs editor is enabled to edit the commands at command mode. But this can be changed to the vi editor by typing the following command at the shell prompt.

    $ set –o vi

The size of the history memory, the number of commands that should be saved for future sessions, and the name of the file that contains the saved history commands can be controlled by a set of predefined environment variables, as described in Table 4.1, along with several other shell variables. Linux application developers very commonly use the variables described in this table. As mentioned earlier, the shell variables must be accessed through the $ notation, as in $HOME to access the variable HOME. The best way to check values of these variables is to issue an echo command at the command prompt, as shown here.

    $ echo $<variable>

It should be noted that additional application-level variables could be defined and used by developers in their own shell scripts, which is the normal practice.

Command prompt variables PS1 and PS2 provide a way to customize the command prompt string. PS1 is the primary string, and PS2 is the secondary string used as the command prompt. When a shell session is initiated by invoking bash (graphically or at the command prompt), the command prompt primary string is displayed. This is somewhat similar to the C:> prompt in the command prompt window of the Windows operating system. By setting the PS1 variable to a desired setting and style, the command prompt can be customized. In order to make the new command prompt active automatically for every bash session, the setting should be made in the $HOME/.bashrc file or the $HOME/.bash_profile file, as the case may be. There are default commands that the shell interprets to evaluate the command prompt strings. The commands should be enclosed in double quotes. The following examples will make the concept clearer.

    $ export PS1="\d ==>"
    $ export PS1="\t ==>"
    $ export PS1="\u@\h \W ==>"
    $ export PS1="\s \v ==>"

These settings would give command prompts as shown here respectively.

    Sat Dec 28 ==>
    21:46:17 ==>
    satya@etslinux LinuxBook ==>
    bash 2.05b ==>

From the examples, it is clear that the \d command prints the current date in ‘Day Mon dd’ format, the \t command prints the current time in ‘HH:MM:SS’ (24 hour) format, \u command prints the current user, \h command prints the hostname, \W command prints the current base directory name (not the complete path), the \s command prints the shell, and the \v command prints the current version of bash shell. It should be noted that the commands shown here are only a subset of a complete set of commands available, and the user is encouraged to refer to other documentation and manual pages for additional commands or more details. The default value of the secondary command prompt PS2 is > and can be set to a desired value. This is used to let the user know that the command line typed at the primary command prompt is incomplete before pressing the ENTER key. At the secondary command prompt string PS2, the user can continue typing the command that was started at the primary command prompt, complete the command, and finally press the ENTER key. If the command is too long, the system keeps showing the secondary prompt string as long as the command is not complete. The bash shell automatically determines when to display the secondary command prompt based on the command-line text being entered.

Command aliasing is a technique used to give alias names to commands. This is generally useful for (and is typically used by) people who are familiar with another operating system and are trying familiarize themselves with Linux. Because Linux was very rich in command-line mode before it became a desktop operating system, and as Linux’s origin is UNIX and the majority of the commands are short-named, cryptic, and not easily remembered by a new user, providing an alias name to a command is very helpful. An alias name is meaningful to the user yet can shorten a very lengthy compound command involving several pipes and arguments. The example shown below makes this concept easily understood.

    $ alias MechPay="gen_paystub_summary.sh | grep 'Mechanic'"

In a scenario where the accounting department of a machine shop generates pay stubs for its entire staff every month, the gen_paystub_summary.sh script executes the necessary programs and finally generates the summary for the entire staff for the month. The output of the script is directed to the grep command, which extracts all those lines that represent payment to the Mechanic type staff. As shown in the example, the entire command can be aliased with the short name MechPay, so that the user does not have to remember the command. Whenever the user needs to type this entire command, it is sufficient to type the alias alone as a command. Because aliases are usually meant to be specific for a single user or a group of users, they are usually set in the user login or session configuration script and available to the user every time a shell session is initiated. Although aliases make the job very easy for users, the disadvantage of using too many aliases is that the user may develop the tendency not to master any of the commands. However, the user makes the ultimate decision of whether to attempt mastering the rich command set of the operating system or to depend on the alias names to make the day-to-day job more comfortable. The command unalias will reset an alias name to null, as shown here.

    $ unalias MechPay

Fundamentals of Shell Scripts

Earlier, we discussed the fact that files representing the shell scripts need to have execute permission set at least at one of the user categories (owner, group, or public). This can be done using the chmod command, as shown in the examples below.

    $ chmod +x <script name>
    $ chmod 755 <script name>

In the first example, the execute permission is added to every category of user. In the second example, the owner gets read, write, and execute permissions and everyone else gets read and execute permissions. More details on the file permissions may be obtained from Chapter 2 Linux for Windows Programmers.

Shell scripts are nothing more than a group of statements of different types. These can be assignment statements that assign values to variables, execution commands that perform specific tasks, flow control constructs that introduce the control logic to the script, or functions that add the necessary modularity to the script. In addition, the scripts also contain commented lines to provide script level documentation. The commented lines start with the # character. Some of these concepts were discussed earlier, and the rest of the discussion in this section is going to expand them further.

Variables are used to store values. They form an important feature of any programming language. It has been discussed earlier that the shell provides a number of built-in variables having special meaning. In addition to the concepts of built-in variables and command execution as discussed in the previous section, the bash shell provides a very powerful programming environment that includes/permits user-defined variables and functions, flow-control and looping constructs, ability to handle error conditions and process signals, and so on. Using these features, we can build very useful (and powerful) scripts that are interpreted by the bash shell. In fact, the entire Linux operating system and its processes are heavily dependent on the shell scripts, whether the bash shell, the Korn shell, or another shell. For example, the installation programs used to install most of the popular software use the shell scripts internally. Usually, the user does not need to modify or view these installation scripts; however, if a particular script fails to execute properly due to inconsistencies between the script contents and the system environment, knowing the concepts of shell scripts and the ability to understand them could help the user to debug the installation and probably fix the problem instead of waiting to receive support from the vendor. If the particular software is freeware or shareware, most likely the users would not get any free support and might have to fix the problem themselves anyway. In a way, all Linux users are expected to understand Linux shell scripts to some extent, if not to the expert level. With the advent of GUI-based IDEs, users may not have to write scripts, but they need to be able to understand the scripts written by others.

Very similar to the pre-built environment variables discussed so far, the shell permits user-defined variables to be used in simple user sessions or in scripts. Variables do not have to be declared before setting their values. If we do not set a value to any variable, the shell assumes that the variable does not exist. By default, the values stored in the variables are treated as character strings. However, when the variables are used in arithmetic expressions, and the format understood by the arithmetic expressions, then they are treated as arithmetic variables. A running shell script is identified as a process. When a process invokes another shell script, a sub-process is initiated by the main process. Variables declared in the parent process are not visible to the sub-process unless they are exported in the main process. This is the reason we execute the export command after setting a value to a variable. For example, if we set a value to a variable at the command prompt and do not export the variable, the subsequent execution of the echo command will not be able to identify that variable. This is because the shell session running the command prompt window is itself a process and initiates a sub-process when the echo command is executed. So variables set in the shell session are not visible to the commands executed in the same shell session unless they are exported using the export command.

As mentioned earlier, shell variable values are interpreted as strings, and they are accessed through the $ notation as in $VAR to access the value stored in the VAR variable. However, there are a few operators that can be used—along with the pair of curly braces {}— while accessing the shell variables, in order to facilitate additional functionality. To check if a variable VAR is defined or not, the ${VAR:+value} expression is used; if the variable defined, then the ‘value’ is returned, otherwise null is returned. Similarly when the ${VAR:-defaultvalue} expression is used, the shell checks if the VAR variable is defined. If the variable is defined, then the value of the variable is returned by the expression. If the variable is not defined, the ‘defaultvalue’ is returned by the expression but not assigned to the variable. However, when the ${VAR:=defaultvalue} expression is used, the shell returns the variable value if it is defined, otherwise sets the VAR variable with the ‘defaultvalue’. The expression ${VAR:<index>} returns the substring starting at the ‘index’ position from the beginning of the VAR variable, if it is defined. Similarly, the expression ${VAR:<index>:<len>} returns the substring starting at the ‘index’ position and of length ‘len’. In both the cases, null is returned if the VAR is not defined, and the string is considered to be zero-index based. The expression ${#VAR} returns the length of the variable content, which is the number of characters in the string value of the variable.

The expression ${VAR#pattern} checks if the evaluated pattern matches at the beginning of the variable value, then the shortest matching value is removed from the variable and the resultant string is returned. On the other hand, the expression ${VAR##pattern} returns the string remaining after removing the longest matching pattern value in the variable. In both these cases, the pattern may contain wildcard characters such as * and ?. When the * wildcard character is used, it actually implies any number of occurrences of any characters where * is positioned within the pattern. For example, the pattern di*t might evaluate to a number of possibilities including words such as ‘distinct’, ‘different’, ‘district’, ‘distract’, ‘distort’, ‘distant’, and so on. Therefore, the meaning of shortest matching and longest matching patterns is different only when the * wildcard character is used. In other cases, both the variations of the expression would give the same result. This can be demonstrated with the following example script saved as StringExpr.sh.

    # StringExpr.sh
    VAR="distinct and different values"
    echo ${VAR#di*t}
    echo ${VAR##di*t}

When the script is executed using the ./StringExpr.sh command line (assuming that the script is located in the current directory), the following output is shown.

    Inct and different values
    values

The first output line demonstrates that the shortest match to the di*t pattern from the beginning of the string is ‘dist’, which is removed from the displayed string. The second output line demonstrates that the longest match to the same di*t pattern is ‘distinct and different,’ which is removed from the displayed string.

Similar to the previous two pattern-matching expressions, the two expressions ${VAR%pattern} and ${VAR%%pattern} attempt to match the pattern at the end of the variable value. There are two more pattern-matching expressions that help in substituting the longest string matching the specified pattern with alternate values. They are ${VAR/pattern/value} and ${VAR//pattern/value}. Between these two, the first expression replaces the first occurrence of the matching pattern, and the second expression replaces all the occurrences of the matching pattern.

In addition to the different types of expressions discussed so far, it is possible to construct expressions that function like compound components. Expressions of the form ((expression)) are evaluated using the arithmetic evaluation principles and are equivalent to using the let built-in command. Variables used within this type of expression are directly accessed by their names without prefixing the $ symbol (i.e., without using the $ notation). If logical operators are used within this type of expression, the logical value (zero or non-zero) is computed and used in conditional statements. If simple arithmetic operators are used, then the computed value represents the result of arithmetic expression and can be stored in arithmetic variables and used in further arithmetic expressions.

While executing the shell scripts, any number of arguments can be passed, and within the script they are identified by their relative position in the command-line string. The general syntax and an example are provided to make the concept clearer.

    $ <script name> [argument1] [argument2] [argument3] . . . .
    $ ./Generate_Invoices.sh 2002 March NA

The first line is the general syntax indicating the arguments are optional. The second line shows an example command line executing the script Generate_Invoices.sh with three arguments 2002, March, and NA. The script may interpret these values such that the invoices need to be generated for March 2002 for the North American (NA) region. When the script is invoked with a command line such as this, the entire command line is parsed and stored by the string in a number of built-in variables. The built-in variable $0 identifies the name of the script, while the arguments are identified relative to the order in which they are passed, as $1 for the first argument, $2 for the second argument, $3 for the third argument, and so on. The variable $* stores all the command-line arguments (separated by the first character of the internal field separator, IFS variable, which is usually the whitespace) as a single string, and the variable $@ stores all the command-line arguments individually separated by whitespace. $@ notation is useful for working with the set of arguments as a list. The ‘$#’ variable specifies the number of arguments (3 in the previous example) passed in the command line. However, all the three variables $*, $@, and $# do not consider the name of the script in the context of their own interpretation.

Any programming (or scripting) language may be considered incomplete if it does not support some of the necessary concepts as discussed in the beginning of this section. User-defined functions are one of them. Using functions, one can write modular code that will be easy to understand and maintain. The general syntax of a function in bash language is very simple and is shown below.

Function <function name> {

statements

}

function <function name> () {

statements

}

<function name> () {

statements

}

The keyword function is optional. However, if it is present, the parentheses ‘()’ appearing after the function name are optional. If the keyword function is not present, then the pair of parentheses must be present. Then follows the body of the function enclosed in curly braces. The function accepts any number of arguments passed during its invocation, and there is no need to mention the arguments in the function heading line, as they are positional, just like the command-line arguments of the script. The $0 variable, which stands for the name of the script, is the same throughout the script and does not change its value even within internal functions defined in the script. However, the other positional arguments such as $1, $2, and so on change their meaning based on the context. They represent the script arguments if they are referenced outside of any of the functions and represent function arguments if they are referenced inside a function. Variables defined within the script are accessible from within the functions if the references are made after the variable is defined. Thus, a function will be able to change values of variables defined outside of its definition. Consider the following example script for a better understanding.

    # CallMe.sh script
    # function definition
    function CallMe {
        echo Today is : $TODAY
        echo Hello $1 $2
    }
    #
    TODAY=`date`
    echo Number of script arguments: "$#"
    echo Hello "$*"
    #
    # reverse the script arguments
    # while calling the CallMe function
    CallMe $2 $1

As the script shows, the script arguments are reversed and passed to the function. Execute the following command line, assuming that the script is saved as CallMe.sh in the current directory. The script is executed with the ./ command-line prefix as the script is saved in the current directory

    $ ./CallMe.sh John Smith

The output of the script, as shown below, explains some of the concepts discussed before.

    Number of script arguments: 2
    Hello John Smith
    Today is : Tue Dec 31 06:57:42 CST 2002
    Hello Smith John

The variable TODAY is defined outside the function and is still accessible within the function, whereas the function arguments are different from the script arguments. If the script arguments are not explicitly passed to the function as we did here, the function will not be able to recognize them. You may write a variation of this script to test this concept. When setting the value of the TODAY variable, a different notation is used. The date command is executed to produce the current date value, and then the value is captured into the variable on the left side of the statement (which is TODAY in this case) by surrounding the command on the right side with the reverse-quote character ` on both the sides. This is not the same as the single-quote character. It is the character that is combined with the tilde ~ symbol on the keyboard and is usually located toward the top left corner below the ESC key. The same result can be obtained by the $(command) notation. For example, both the forms $(date) and date would compute the current date and timestamp and assign this value to the variable on the left side of the assignment statement.

Built-In Shell Commands

Earlier, we learned that many Linux commands are individual programs. However, similar to the rich set of built-in variables, the shell also provides a rich set of built-in commands. As the name suggests, these are not individual programs; rather, they are built into the shell itself. Upon successful execution, the built-in commands return zero and a non-zero value if the command fails to execute. Table 4.2 displays some of the most commonly used built-in commands of the bash shell, which means that the list is not exhaustive. Some of these commands were already discussed in the previous sections, and the others are described in the table. The descriptions provided in the table represent typical use of the respective commands. However, there are more options and variations of the commands, which the reader is encouraged to explore independently.

    $ declare –i counter=10 # declares integer
         variable
    $ declare –x hstr="Hello World" #declares &
         exports
    $ declare –f # display currently defined
         functions

    $ set –o allexport # marks future variable   
                       # definitions as exported
    $ set –o vi # uses vi editor for history
         navigation
    $ set –o emacs # uses Emacs for history
         navigation
    $ set –o noexec # used in non-interactive
         shell to
                    # read command for syntax
                         checking
                    # only and not for executing.

Flow Control Constructs in the Shell Scripts

The bash language provides a rich set of flow control constructs, which form the backbone of the scripts. These are discussed in this subsection.

The ‘if/else’ construct can be used in several forms, as shown here. It should be noted that the ‘if’ construct is surrounded by the ‘if’ and ‘fi’ keywords.

if <condition> then

statements

fi

if <condition> then

statements1

else

statements2

fi

if <condition> then

statements1

elif <condition>

statements2

. . .

else

statementsN

fi

The first form of the ‘if’ construct is very simple and evaluates the condition. If the condition is true, then the statements are executed; otherwise the control transfers to the command following the ‘fi’ keyword. The second form of the construct ensures that either of the statement groups is executed depending on whether the condition is evaluated to true or false. In the third form, the ‘elif’ stands for ‘else if’ and checks another condition if the first one fails. In this form, there could be as many ‘elif’s as there are conditions to be evaluated, and the ‘else’ clause appears only at the end. In all the cases, there should be only one ‘else’ clause, even though there could be more than one ‘elif’ clause.

The conditions tested in the ‘if’ and ‘elif’ constructs are one or more statements that are treated as logical conditions. If the condition contains more than one statement, then they are linked through logical operators. For example, every Linux command that returns a zero or non-zero value can be tested in these constructs. The zero return code is considered as success (and hence true), and the non-zero return code is considered as failure (and hence false). The logical operators && and || are used to test the ‘logical and’ and ‘logical or’ conditions for two or more statements. The logical operator ! preceding a condition is used to test for negation of the condition. The logical operators <, >, =, and != may be used on string operands for testing whether the first string is less than, greater than, equal to, and not equal to, respectively. The -n operator preceding a string checks for the string to be not null or non-zero length, and the -z operator preceding a string checks for the string to be null or zero length. The logical operators -lt, -gt, -eq, -ne, -ge, and le are used on integer operands and stand for less than, greater than, equal to, not equal to, greater than or equal to, and less than or equal to, respectively.

The first type of conditions test whether a specific command was run successfully or not. Such conditions are checked without enclosing brackets. The second type of conditions that can be tested with the ‘if’ and ‘elif’ constructs are those that are enclosed in square brackets, as in [ . . . ]. When conditions are specified within the square brackets, there should be a space after the opening bracket [ and the beginning of condition statement and before the ending of condition statement and the closing bracket ].

There are special test conditions designed for testing the file attributes. These are of the form [ <option> <filename> ], where the option checks for the specific attribute or condition. The different command options used to check for the existence of the file are presented in Table 4.3.

There are logical operators that compare two files, as in <file1> <operator> <file2>, as shown in Table 4.4.

The test built-in command is used along with the ‘if’ and ‘elif’ constructs, in place of the square brackets.

Example condition statements are shown here to demonstrate some of these concepts.

    # CpFile.sh script
    OLDFILE=oldscript.sh
    NEWFILE=newscript.sh
    if cp "$OLDFILE" "$NEWFILE" 2>/dev/null; then
        echo $OLDFILE is copied to $NEWFILE
    else
        echo Failed to copy $OLDFILE to $NEWFILE
    fi


    # CheckString.sh script
    YOURNAME="Satya Sai"
    NAME1="Satya Sai"
    NAME2="John Smith"
    if [ "$YOURNAME" = "$NAME1" ]; then
        echo $YOURNAME is $NAME1
    else
        echo $YOURNAME is not $NAME1
    fi

    # CheckFile.sh script
    FILE=/home/satya/linkFile
    if [ -h "$FILE" ]; then
        echo $FILE is a symbolic link
    else
        echo $FILE is not a symbolic link
    fi

    # testCommand.sh script
    VARNAME=
    if test –z "$VARNAME"; then
        echo VARNAME is a null string
    else
        echo VARNAME is not a null string
    fi

The examples are provided only to demonstrate how to code the ‘if’ construct or to help the readers understand existing shell scripts. However, the readers are encouraged to practice writing their own examples to test many of the conditions not shown in the book.

There are two forms of the ‘for’ loop: in the first form, it is used to execute a group of statements a fixed number of times, and in this form it is very similar to the ‘for’ loop in the C/C++ language. The general syntax and an example script are provided here.

for (( (expr1); (expr2); (expr3) )) do

statements

done

    # ForLoop.sh
    for (( (i=0); (i< 4); i++ )) do
        echo Loop counter value is $i
    done

The output of this script would be as shown below.

    Loop counter value is 0
    Loop counter value is 1
    Loop counter value is 2
    Loop counter value is 3

The ‘expr1’ is an expression that is evaluated while entering the loop and may be considered as the loop initiation expression. Typically, a loop counter variable is initiated in this expression. The ‘expr2’ expression is a conditional expression and is executed at the beginning of every iteration; the loop counter variable is tested in this expression. An iteration is defined as executing the set of statements inside the loop for once. If the expression is evaluated to false or non-zero, then the next iteration is executed. The loop exits when this expression is evaluated to true or zero for the first time. Therefore, when beginning the loop, it should be evaluated to false; otherwise the loop does not execute even one iteration. The ‘expr3’ is an expression that is executed every time after executing the iteration. Therefore, this expression is used to change the value of the variable that is initiated in ‘expr1’ and being tested in ‘expr2’, so that the loop exit criteria are built within the loop description itself. It is important to note that the surrounding pairs of parentheses (( . . . )) are required for the shell to interpret the expressions as arithmetic. It is very easy to setup the ‘for’ loop with perfect exit criteria and therefore to avoid the infinite looping situation; however, the conditional expression should be set up properly.

The second kind of ‘for’ loop also executes the iteration for a fixed number of times, but the number of iterations is not controlled by a counter as has been done in the previous case. Rather, the loop is executed over a list of items, such as the list of command-line arguments, the list of directories in the $PATH variable, the list of files found within a directory, a list of values built programmatically and so on. In all these cases, the values within the list are assumed to be separated by the first character of the IFS special variable, which was discussed earlier. The general syntax and an example are provided here.

for <variable> in $<listvar>; do

statements

done

# ForLoop2.sh
FRUITS="appl orange pear banana"
for fruit in $FRUITS; do
    echo $fruit
done

The output of this script would be as shown below.

    appl
    orange
    pear
    banana

If the field separator in the list variable is not the default IFS character (usually whitespace), then the IFS can be set explicitly before the for loop as in IFS=: to separate the directories within the PATH global variable, or as in IFS=/ to separate the individual subdirectory names within a directory name, so that the shell interprets the items in the list appropriately. Similarly, the command-line arguments can be accessed through the $@ list variable, as shown in the following example.

    # ForLoop3.sh
    for arg in "$@"; do
        echo $arg
    done

The output of this script would be similar to the previous outputs, depending on the arguments provided at the command-line. However, it should be remembered that the $0 variable (the name of the script) is not an item in the $@ list.

The next type of looping construct is the ‘while’ loop, which is very similar to the one seen in C/C++ programming languages. The general syntax of the construct is provided here.

while <condition>; do

statements

done

In the ‘while’ loop, the set of statements within the loop are executed as long as the ‘condition’ is true. The condition is checked at the beginning of every iteration. Whenever the condition is evaluated to false, the ‘while’ loop immediately stops further iterations, and the control flows to the statement that follows the end of the loop. However, if it is desired to exit the loop in the middle of an iteration, then the break command should be executed to break the loop by force. Another looping construct of interest is the ‘until’ loop, which is very similar to the ‘while’ loop but only different in one aspect. The ‘until’ loop executes the iterations as long as the condition is evaluated to false and exits the loop whenever the condition becomes true. The meaning of this construct is to execute the loop until the condition is true. In all other aspects, it behaves similar to the ‘while’ loop. The general syntax is provided below.

until <condition>; do

statements

done

The ‘case’ construct may be used when it is necessary to test for multiple values of an expression and then execute different groups of statements each time when the expression is evaluated to a different value. In other words, the case statement effectively replaces a complicated if …elif … elif … type of construct. The general syntax and an example script using the ‘case’ construct are given here.

case <expr> in

( pattern1 )

statements ;;

( pattern2 )

statements ;;

. . .

. . .

esac

    # CaseConstruct.sh
    for arg in "$@"; do
        echo $arg
        case $arg in
             ( 1 ) echo One ;;
        case $arg in
             ( 2 ) echo Two ;;
        case $arg in
             ( 3 ) echo Three ;;
        esac
    done

When the script is executed with the command-line ./CaseConstruct.sh 1 2 3, the output of this script would be as shown below.

    1
    One
    2
    Two
    3
    Three

The last construct that we are going to discuss is the ‘select’ construct, which is used to construct a menu of options and to take appropriate action based on the input provided by the user. The general syntax of the construct is shown below.

select <variable> in <listvar>; do

statements

done

In this construct, the ‘listvar’ is the list of items displayed in the menu and is equivalent to the other lists that we have discussed before. The items in the list can be simple words if the default IFS character is used as a field separator. By assigning a different value to the IFS before using the ‘select’ construct, whitespace can be used to create menu options containing multiple words separated by whitespace. When executing a ‘select’ construct, the shell builds the menu options and numbers them sequentially from 1 to N, where 1 and N represent the first and last items in the list and sequence numbers between represent the corresponding menu options. Then the shell displays the menu with the sequence numbers and accepts a value from the standard input when the user chooses an option by entering the appropriate number and clicking the ENTER key. The value entered by the user is then stored in the ‘variable’ mentioned in the select clause. The value of the variable is accessible within the construct’s statements using the default $<variable> notation. Within the ‘select’ construct, the break statement should be used to appropriately come out of the construct; otherwise, the ‘select’ construct behaves like an infinite loop. The PS3 command-line prompt is used to display a message prompt for the user to understand that a choice has to be made at the prompt. The example provided here helps you to understand the concept of using this construct.

    # SelectConstruct.sh
    PS3='Please enter your choice '
    OPTIONS="Apple Orange Pear Banana"
    select choice in $OPTIONS; do
        if [ $choice ]; then
            echo Your choice of fruit is $choice
        else
            echo Your choice is not valid – exiting . . .
            break
        fi
    done

When this script is executed with the command line ‘/.SelectConstruct.sh’, the menu is displayed as shown below.

    1) Apple
    2) Orange
    3) Pear
    4) Banana
    Please enter your choice

If you enter anything other than the specified choices, then the message “Your choice is not valid – exiting . . .” and the script exits.

The discussion to this point refers to the features provided by the bash shell and is neither complete nor exhaustive. As mentioned earlier, the purpose of this discussion is to enable you to understand the power of the shell and start writing simple scripts. You are therefore encouraged to further explore the shell functionality and the rich set of commands in order to get more details.