Taming the TerminalNote: this blog post was written to accompany an audio segment on episode 449 of the NosillaCast Mac Podcast.

Given the times we live in, the word ‘environment’ probably invokes images of polar bears and melting ice, but the Al Gore definition of the word ‘environment’ is a relatively recent narrow definition of a much broader word. The first definition of the work in the OS X dictionary is:

The surroundings or conditions in which a person, animal, or plant lives or operates

In this instalment we’ll introduce a digital extension of this concept – the digital conditions within which a process exists, and specifically, in which a BASH command shell exists. Although this might sound like a simple topic, there’s actually a lot to cover, so we’ll be spreading it out over a few instalments.

The Basic Environment

Although we’ve not used the word ‘environment’ before, we have already discussed some elements that make up a process’s environment. Specifically, we know that every process has a user ID associated with it (we say that every process runs as a user), and we have come across the concept of the present working directory. Both of these elements make up part of the basic environment that every process on your computer executes within, not just command shells. The third major pillar in the basic environment are environment variables. These are name-value pairs that can be accessed by running processes.

When one process starts another process, the child process inherits a copy of the parent process’s environment. The child process runs as the same users the parent process was running as, it starts with the same present working directory, and it gets a copy of all the environment variables that existed in the parent’s environment at the moment the child was spawned. The important thing to note is that child processes do not share a single environment with their parents, they get a duplicate that they are then free to alter without affecting the parent process’s environment. When a child process changes it’s present working directory, that has no effect on the parent process’s present working directory, and similarly, when a child process changes the value stored in a given environment variable, that has no effect on the value stored in the same environment variable within the parent process’s environment.

While all processes have access to a basic environment, command shells extend this basic foundation to provide a much richer environment for their users. Until now very little that we have looked at has been shell-specific, but that changes with this instalment. Each command shell gets to create it’s own environment and to define it’s own mechanisms for interacting with it. What works in BASH will not necessarily work in KSH or ZSH etc.. In this series we’ll only be dealing with the default command shell on most modern Unix and Linux OSes (including OS X), BASH. Note that BASH is an extended versions of SH, so what works in SH works in BASH, and much, though not all, of what works in BASH also works in SH.

Environment Variables

In this instalment we’ll be focusing on Environment Variables, and specifically, how BASH interacts with them.

The command to list the names and values of all currently set environment variables is simply env (or printenv on some systems). E.g.:

bart-imac2013:~ bart$ env
TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/_8/s3xv9qg94dl9cbrqq9x3ztwm0000gn/T/
Apple_PubSub_Socket_Render=/tmp/launch-MLs1hi/Render
TERM_PROGRAM_VERSION=326
TERM_SESSION_ID=7661AF3B-0D62-435F-B880-C5428000E9D8
USER=bart
SSH_AUTH_SOCK=/tmp/launch-hwTXSO/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0:2
PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
__CHECKFIX1436934=1
PWD=/Users/bart
LANG=en_IE.UTF-8
SHLVL=1
HOME=/Users/bart
LOGNAME=bart
_=/usr/bin/env
bart-imac2013:~ bart$

env lists the environment variables one per line. On each line the name of the variable is the text before the first =, and the value is everything after it.

Some of these variables are purely informational, while other are used to affect how a process behaves.

Environment Variables & Bash Shell Variables

BASH, like every other process, has access to all the variables set within it’s environment. However, BASH extends the concept and of variables into shell variables, of which the environment variables are just a subset. Bash shell variables can be local to the shell, or can exist within the shell and the environment. We already know that env lets us see all the environment variables which exist in our shell, but there is another command to let us see all the variables in our shell both those in the environment, and the local ones, and that command is set. To see all the shell variables that exist, call set with no arguments. E.g.

bart-imac2013:~ bart$ set
Apple_PubSub_Socket_Render=/tmp/launch-MLs1hi/Render
BASH=/bin/bash
BASH_ARGC=()
BASH_ARGV=()
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="3" [1]="2" [2]="51" [3]="1" [4]="release" [5]="x86_64-apple-darwin13")
BASH_VERSION='3.2.51(1)-release'
CCATP=rocks
COLUMNS=80
DIRSTACK=()
EUID=501
GROUPS=()
HISTFILE=/Users/bart/.bash_history
HISTFILESIZE=500
HISTSIZE=500
HOME=/Users/bart
HOSTNAME=bart-imac2013.localdomain
HOSTTYPE=x86_64
IFS=$' \t\n'
LANG=en_IE.UTF-8
LINES=24
LOGNAME=bart
MACHTYPE=x86_64-apple-darwin13
MAILCHECK=60
OPTERR=1
OPTIND=1
OSTYPE=darwin13
PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
PIPESTATUS=([0]="0")
PPID=17153
PROMPT_COMMAND='update_terminal_cwd; '
PS1='\h:\W \u\$ '
PS2='> '
PS4='+ '
PWD=/Users/bart
SHELL=/bin/bash
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor
SHLVL=1
SSH_AUTH_SOCK=/tmp/launch-hwTXSO/Listeners
TERM=xterm-256color
TERM_PROGRAM=Apple_Terminal
TERM_PROGRAM_VERSION=326
TERM_SESSION_ID=41E9B4E3-BC9B-4FC0-B934-E2607FF6DC35
TMPDIR=/var/folders/_8/s3xv9qg94dl9cbrqq9x3ztwm0000gn/T/
UID=501
USER=bart
_=PATH
__CF_USER_TEXT_ENCODING=0x1F5:0:2
__CHECKFIX1436934=1
update_terminal_cwd () 
{ 
    local SEARCH=' ';
    local REPLACE='%20';
    local PWD_URL="file://$HOSTNAME${PWD//$SEARCH/$REPLACE}";
    printf '\e]7;%s\a' "$PWD_URL"
}
bart-imac2013:~ bart$

If you compare the output of env and set you’ll see that every environment variable is a shell variable, but, there are many more shell variables that there are environment variables. Remember, when a child process is created only the environment variables get copied into the child process’s environment, even if the child process is another BASH process. Shell variables are local to a single command shell, hence they are often called local variables.

Shell variables can be used when invoking shell commands. To access the content of a variable you use the $ operator. When you enter $VARIABLE_NAME in the shell it will be replaced with the value of the variable named VARIABLE_NAME. E.g. to change to the Desktop directory in your home folder you could use:

cd $HOME/Desktop

or (if you have a Mac configured in the default way)

cd /Users/$LOGNAME/Desktop

Way back in the second instalment we discussed quoting strings in the shell, and we mentioned that there was a very important difference between using double and single quotes, and that it would become important later, well, this is where that difference becomes important. If you use the $ operator within a string enclosed by double quotes the variable name will get replaced by the variable’s value, if you use it within a string contained within single quotes it will not!

This is why the following do work (this is an OS X-specific example):

cd $HOME/Library/Application\ Support
cd "$HOME/Library/Application Support"

But the following does not:

cd '$HOME/Library/Application Support'

Note that you can also inhibit the $ operator by escaping it with a \ character. Hence, the following has exact the same effect as the previous command:

cd \$HOME/Library/Application\ Support

Sometimes when we type the $ symbol we mean the $ operator, and sometimes we just mean the character $. If we mean the character, we have to inhibit the operator either by escaping it, or by using single quotes around the string containing it. When ever you find yourself typing the $ character, pause and think which you mean before hitting enter, and be sure you have it escaped or not as appropriate.

While we can list the values stored in all variables with set, it’s also helpful to know how to show the value stored in a single variable. The easiest way to do this is to make use of the initially useless-seeming command echo. All echo does is print out the argument you pass to it, so, a simple example would be:

echo 'Hellow World!'

This seems pretty dull, but, when you combine echo with the $ operator it becomes much more useful:

echo $LOGNAME

We can even get a little more creative:

echo "I am logged in as the user $LOGNAME with the home directory $HOME"

Now that we can use variables, lets look at how we create them and alter their values. You create variables simply by assigning them a value, and you alter their value by assigning them a new vale. The = operator assigns a value to a variable. In our examples we won’t use a variable set by the system, but we’ll create our own one called MY_FIRST_VAR.

Before we start, we can verify that our variable does not exist yet:

echo $MY_FIRST_VAR

Now lets create our variable by giving it a value:

MY_FIRST_VAR='Hello World!'

Now lets verify that we did indeed initialise our new variable with the value we specified:

echo $MY_FIRST_VAR

Now lets get a little more creating and change the value stored in our variable using values stored in two variables inherited from the environment:

MY_FIRST_VAR="Hi, my name is $LOGNAME and my home directory is $HOME"

Because we used double quotes, it is the value stored in the variables LOGNAME and HOME that have been stored in MY_FIRST_VAR, not the strings $LOGNAME and $HOME.

At this stage our new variable exists only as a local shell variable, it is not stored in our process’s environment:

env

The export command can be used to to ‘promote’ a variable into the Environment. Simply call the command with the name of the variable to be promoted as an argument, e.g. to push our variable to the environment use:

export MY_FIRST_VAR

We can now verify that we really have pushed our new variable to the environment:

env

Environment Variables and sub-shells – OPTIONAL

As mentioned, when one process starts another, the child process inherits a copy of the parent’s environment. If a child makes a change to an environment variable, that change is not seen by the parent. We can illustrate this easily using so-called sub-shells.

When one BASH process started another BASH process that child process is called a sub-shell. The most common way to create a sub-shell is by executing a shell script. A shell script is simply a text file that contains a list of shell commands. While we won’t be looking at shell scripting in detail until much later in this series, we’ll use some very simple shell scripts here to illustrate how child processes inherit their parent’s environment.

Lets start by creating a very simple shell script that will print the value of an environment variable:

nano ~/Documents/ttt12script1.sh

Add the following into the file and then save and exit:

  1. #!/bin/bash
  2.  
  3. echo "TTT_VAR=$TTT_VAR"

Asside: The first line of this script is the so-called “shebang line”, and it tells BASH what interpreter it should use to run the file. If we were writing a Perl script instead of a BASH script we would start our file with the line:

#!/usr/bin/perl

Before we can run our new script we need to make it executable:

chmod 755 ~/Documents/ttt12script1.sh

The environment variable TTT_VAR does not exist yet, so running our shell script now will show that:

~/Documents/ttt12script1.sh

We can now give our variable a value:

TTT_VAR='Hello World!'

And if we run our script again, we can see that it still does not print out the value because we have only created a local shell variable, not an environment variable:

~/Documents/ttt12script1.sh

Now lets push our variable to the environment and run our script again:

export TTT_VAR
~/Documents/ttt12script1.sh

To prove that the sub-shell is working on a copy of the environment variable, lets copy our first script and create a new script that alters the value of the variable:

cp ~/Documents/ttt12script1.sh ~/Documents/ttt12script2.sh
nano ~/Documents/ttt12script2.sh

Upate the new script so it contains the following code, then save and exit:

  1. #!/bin/bash
  2.  
  3. echo "Initially: TTT_VAR=$TTT_VAR"
  4. echo "Altering TTT_VAR in script"
  5. TTT_VAR='new value!'
  6. echo "Now: TTT_VAR=$TTT_VAR"

Now run the following:

echo $TTT_VAR
~/Documents/ttt12script2.sh
echo $TTT_VAR

You should get output that looks something like:

bart-imac2013:~ bart$ echo $TTT_VAR
Hello World!
bart-imac2013:~ bart$ ~/Documents/ttt12script2.sh
Initially: TTT_VAR=Hello World!
Altering TTT_VAR in script
Now: TTT_VAR=new value!
bart-imac2013:~ bart$ echo $TTT_VAR
Hello World!
bart-imac2013:~ bart$

As you can see, the sub-shell inherited the value of the environment variable TTT_VAR, but changing it in the sub-shell had no effect on the value seen in the parent shell, even though it was exported to the child shell’s environment.

You might expect that this means that you can’t use scripts to build or alter your environment, but, actually, you can. You just can’t do it by accident, you must be explicit about it, and use the source command. To see this in action run the following:

echo $TTT_VAR
source ~/Documents/ttt12script2.sh
echo $TTT_VAR

This should give you output something like:

bart-imac2013:~ bart$ echo $TTT_VAR
Hello World!
bart-imac2013:~ bart$ source ~/Documents/ttt12script2.sh
Initially: TTT_VAR=Hello World!
Altering TTT_VAR in script
Now: TTT_VAR=new value!
bart-imac2013:~ bart$ echo $TTT_VAR
new value!
bart-imac2013:~ bart$

What the source command does is to run each command in the shell script within the current shell’s environment, hence, all changes made within the script are made within the shell that executes the script. As we’ll see in a future instalment, the source command plays a pivotal role in the initialisation of every BASH shell.

Conclusions

In this introductory instalment we focused mainly on how processes inherit their environment, and on the concept of shell and environment variables, in particular how they are inherited, and how they can be accessed and altered. In the next instalment we’ll start by focusing on one of the most important environment variables of all – PATH. We’ll also go on to look at how a new BASH shell assembles it’s environment, and how to make permanent customisations to that environment, including things like customising your shell prompt, and creating command shortcuts called aliases.

Comments

One Response to “Taming the Terminal – Part 12 of n (The Environment)”

  1. Tim McCoy on December 17th, 2013 11:04 pm

    It is always interesting ( aka “amusing” ) to listen as Allison “learns” new and wonderful *n*x things. You are more patient than I.

    Did you get through your adoption of “Perl Best Practices”; the only thing I found useful was
    $entry =~ s{\A \s* | \s* \z}{}gxm;
    I was in the habit of doing two separate calls to minimize a token or string. After 25 years of perl I’m kinda “fixed” in my methods.

Leave a Reply




Before you post a comment please remember that commenting on my blog is a privilege not a right. I won't approve comments that are obscene, offensive or insulting. For more info please read this post.

Subscribe without commenting