The Apple/Mac community lost one of it’s finest podcasters today. Tim Verpoorten wasn’t the first Apple/Mac podcaster, but he was one of the very earliest generation. I think it would be fair to call him a father figure to many of us who followed. I know he was one of the podcasters who inspired me to pick up the microphone myself, and I doubt I’m alone in that.

Tim had been unwell for some time, and hung up his microphone to concentrate on his health a while ago, but we all hoped it would just be a temporary hiatus. I don’t think any of us in the community wanted to believe we’d heard the last of Tim’s distinctive and friendly voice.

Every good Apple/Mac podcast brings something unique to the table, and Tim’s Mac Review Cast brought fantastic reviews week after week after week for years and years. Tim had a knack for finding great apps, particularly free ones, and he was able to find and review them at a truly impressive rate. Most people can mange either quantity or quality, but Tim could do both at the same time. Although he reviewed many many apps, you could always tell when an app really appealed to him. Those apps were almost never large apps with lots of features, but small apps that did just one thing, but did it really well. It’s fair to say Tim had a bit of a thing for menubar apps.

Because I learned about so many great apps on the Mac Review Cast, I regularly look up at my menu bar, or into my dock, and think of Tim. One app in particular that I’ll always associate with him is the light-weight Mac-like text editor Smultron. I’d almost given up on finding an editor like this for the Mac, when I heard Tim review Smultron, and gave it a go. It was love at first sight, and that cute red strawberry icon will always bring back fond memories of Tim.

Tim was one of the founders of the Mac Round Table Podcast (MRT), and it was through that podcast that I was fortunate enough to get to ‘work’ (play more like) with Tim. One of the great things about the MRT is how different all the contributors are, and how that opens up some great conversations. We often agreed on things, but when it comes to temperament, I think myself and Tim were polar opposites – I’m know for being the cranky Irishman (sorta) who’s prone to impassioned (and hopefully entertaining) rants, while Tim was always as cool as a cucumber – I can’t remember him ever getting flapped, and I can’t remember him ever having a bad word to say about anyone. I think it’s much easier to go on a rant than it is to remain calm and collected, and I greatly admired Tim’s coolness.

I never met Tim in the real world, yet I feel I’ve lost a friend. The Mac community has certainly lost one of it’s finest ambassadors, but my thoughts are with the Verpoorten family tonight – their loss is so much greater than ours.

The photo that accompanies this post is a crop from this image by Allison Sheridan.

In the first installment we started with the 40,000ft view, looking at what command shells are, and why they’re still relevant in today’s GUI-dominated world. In the second instalment we looked at OS X’s Terminal.app, the anatomy of the Bash command prompt, and the anatomy of a Unix/Linux command. This time we’ll be looking at the anatomy of file systems in general, and the Unix/Linux file system in particular, and how it differs from the Windows/DOS file system many of us grew up using.

Physical storage media are nothing more than a massive array of virtual pigeon holes, each of which can hold a single 1 or 0. All your information is stored by grouping together a whole bunch of these pigeon holes and giving that grouping of 1s and 0s some kind of name. Humans simply could not deal with remembering that the essay they were working on is stored in sectors 4 to 1024 on cylinder 213 on the disk connected to the first SATA channel on the motherboard. We need some kind of abstraction to bring order to the chaos and to allow us to organise our data in a human-friendly way.

A good analogy would be a pre-computer office where the unit of storage was a single sheet of paper. Without some sort of logical system for organising all this paper no one would ever be able to find anything, hence, in the real world we developed ‘systems’ for ‘filing’ paper. Or, to put it another way, we invented physical filesystems, based around different ways of grouping and naming the pieces of paper. If a single document contained so much information that it ran over multiple pages, those piece of paper were physically attached to each other using a tie, a paperclip, or a staple. To be able to recognise a given document at a glance, documents were given titles. Related documents were then put together into holders that, for some reason were generally green, and those holders were then placed into cabinets with rails designed to hold the green holders in an organised way. I.e. we had filing cabinets containing folders which contained files. The exact organisation of the files and folders were up to the individual clerks who managed the data, and were dependant on the kind of data being stored. Doctors tend to store files alphabetically by surname, while libraries love the Dewey Decimal system.

When it comes to computers, the job of bringing order to the chaos falls to our operating systems. We call the many different schemes that have been devised to provide that order filesystems. Some filesystems are media dependent, while others are operating system dependent. E.g. the Joliet file system is used on CDs and DVDs regardless of OS, while FAT and NTFS are Windows filesystems, EXT is a family of Linux file systems, and HFS+ is a Mac file system.

There are an infinite number of possible ways computer scientists could have chosen to bring order to the chaos of bits on our various media, but, as is often the case, a single real-world analogy was settled on by just about all operating system authors. Whether you use Linux, Windows, or OS X, you live in a world of filesystems that contain folders (AKA directories) that contain files and folders. Each folder and file in this recursive hierarchical structure has a name, so it allows us humans to keep our digital documents organised in a way that we can get our heads around. Although all our modern filesystems have their own little quirks under the hood, they all share the same simple architecture, your data goes in files which go in folders which can go in other folders which eventually go into file systems.

You can have lots of files with the same name in this kind of file system, but, you can never have two items with the same name in the same folder. This means that each file and folder can be uniquely identified by listing all the folders you pass to get from the so-called ‘root’ of the filesystem as far as the file or folder you are describing. This is what we call the full path to a file or folder. On all modern consumer operating systems we write file paths as a list of folder and file names separated by some character, called the ‘path separator’. Where operating systems diverge is in their choice of separator, and in the rules they impose on file and folder names. DOS and Windows use \ (the backslash) as the path separator, on classic MacOS it was : (old OS X apps that use Carbon instead of Cocoa still use : when showing file paths, iTunes did this up until the recent version 11!), and on Linux/Unix(including OS X), / (the forward-slash) is used.

A single floppy disk and a single CD or DVD contain a single file system to hold all the data on a given disk, but that’s not true for hard drives, thumb drives, or networks. When formatting our hard drives or thumb drives we can choose to sub-divide a single physical device into multiple so-called partitions, each of which will then contain a single filesystem.

You’ve probably guessed by now that on our modern computers we tend to have more than one filesystem. Even if we only have one internal hard disk in our computer that has been formatted to have only a single partition, every CD, DVD, or thumb drive we own contains a filesystem, and, each network share we connect to is seen by our OS as yet another file system. In fact, we can even choose to store an entire filesystem (even an encrypted one) in a single file, e.g. DMG files, or TrueCrypt vaults.

So, all operating systems have to merge lots of file systems into a single over-arching namespace for their users. Or, put another way, even if two files have identical paths on two filesystems mounted by the OS at the same time, there has to be a way to distinguish them from each other. There are lots of different ways you could combine multiple filesystems into a single unified namespace, and this is where the DOS/Windows designers parted ways with the Unix/Linux folks. Microsoft combines multiple file systems together in a very different way to Unix/Linux/OS X.

Lets start by looking at the approach Microsoft chose to. In DOS, and later Windows, each filesystem is presented to the user as a separate island of data named with a single letter, referred to as a drive letter. THis approaches has an obvious limitation, you can only have 26 file systems in use at any one time! For historical reasons, A:\ and B:\ were reserved for floppy drives, so, the first partition on the hard drive connected to the first IDE/SATA bus on the motherboard is given the drive letter C:\, the second one D:\ and so on. When ever you plug in a USB thumb drive or a memory card from a camera it gets ‘mounted’ on the next free drive letter. Network shares also get mounted to drive letters.

Just like files and folders, filesystems themselves have names too, often referred to as Volume Names. Windows makes very little use of these volume names though, they don’t show up in file paths, but, Windows Explorer will show them in some situations to help you figure out which of your USB hard drives ended up as K:\ today.

An analogy you can use for file systems is that of a tree. The trunk of the tree is the base of the file system, each branch is a folder, and each leaf a file. Branches ‘contain’ branches and leaves, just like folders contain folders and files. If you bring that analogy to Microsoft’s way of handling filesystems, then the global namespace is not a single tree, but a small copse of between 1 and 26 trees, each a separate entity, and each named with a single letter.

If we continue this analogy, Linux/Unix doesn’t plant a little copse of separate trees DOS/Windows does, instead they construct one massive franken-tree by grafting smaller trees onto the branches of a single all-containing master tree. When Linux/Unix boots, one filesystem is considered to be the main filesystem, and used as the master file system into which other file systems get inserted as folders. In OS X parlance, we call the partition containing this master file system the System Disk. Because the system disk becomes the root of the entire filesystem it is gets assigned the shortest possible file path, /.

If your system disk’s file system contained just two folders, folder_1 and folder_2, they would get the file paths /folder_1/ and /folder_2/ in Linux/Unix/OS X. The Unix/Linux command mount can then be used to ‘graft’ filesystems into the master filesystem using any empty folder as the so-called mount point.

On Linux systems it’s common practice to keep home folders on a separate partition, and to then mount that separate partition’s file system as /home/. This means that the main filesystem has an empty folder in it called home, and that as the computer boots, the OS mounts a specified partition’s file system into that folder. A folder at the root of the that partition’s file system called just allison would then become /home/allison/.

On regular Linux/Unix distributions the file /etc/fstab (file system table) tells the OS what filesystems to mount to what mount points. A basic version of this file will be created by the installer, but in the past, when ever you added a new disk to a Linux/Unix system you had to manually edit this file. Thankfully, we now have something called automount to automatically mount any readable filesystems to a pre-defined location on the filesystem when they are connected.

The exact details will change from OS to OS, but on Ubuntu, the folder /media/ is used to hold mount points for any file system you connect to the computer. Unlike Windows, most Linux/Unix systems make use of filesystems’ volume names, and use them to give the mount points sensible names, rather than random letters. If I connect a USB drive containing a single partition with a filesystem with the volume name Allison_Pen_Drive, Ubuntu will automatically mount the filesystem on that thumb drive when you plug it in, using the mount point /media/Allison_Pen_Drive/. If that pen drive contained a single folder called myFolder containing a single file called myFile.txt, then myFile.txt would be added to the filesystem as /media/Allison_Pen_Driver/myFolder/MyFile.txt.

Having the ability to mount any filesystem as any folder within a single master filesystem allows you to easily separate different parts of your OS across different drives. This is very useful if you are a Linux/Unix sysadmin or power user, but it can really confuse regular users. Because of this, OS X took a simpler route. There is no /etc/fstab (actually, there sort of is, but it just contains some text telling you that the file does nothing and not to use it – for a LOL try typing cat /etc/fstab.hd into a terminal). The OS X installer does not allow you to split OS X over multiple partitions, everything belonging to the OS X system, including all the users home folders, are installed on a single partition, the system disk, and all other file systems, be they internal, external, network, or disk images, get automatically mounted in /Volumes/ as folders named for the file systems’ volume labels.

Going back to our imaginary thumb drive called Allison_Pen_Drive (which Ubuntu would mount as /media/Allison_Pen_Drive/), OS X will mount that as /Volumes/Allison_Pen_Drive/ when you plug it in. If you had a second partition, or a second internal drive, called, say, Fatso (a little in-joke for Allison), OS X would mount that as /Volumes/Fatso/. Likewise, if you double-clicked on a DMG file you downloaded from the net, say with the Adium installer, OS X would mount that as something like /Volumes/Adium/ until you eject the DMG. The ‘disks’ listed in the Finder side bar in the section headed Devices are just links to the contents of /Volumes/. You can see this for yourself by opening a Finder Window and either hitting the key-combo cmd+shift+g, or navigating to Go→Go To Folder ... in the menubar to bring up the Go To Folder text box, and then typing the path /Volumes and hitting return.

OS X’s greatly simplified handling of mount points definitely makes OS X less confusing, but, the simplicity comes at a price. If you DO want to do more complicated things like have your home folders on a separate partition, you are stepping outside of what Apple consider the norm, and into a world of pain. On Linux/Unix separating out home folders is trivial, on OS X it’s a mine-field!

We’ll leave it here for now, next time we’ll learn how to navigate around a Unix/Linux/OS X filesystem.

The trains consisted of the RPSI Cravens coaches, and were worked by the RPSI’s 0-6-0 steam loco No.186. The trains were a little more special because No.186′s boiler certificate will expire later this year, so this could well have been the last time she’ll be on this line for a decade or more. Once her certificate expires she’ll have to be fully overhauled, and given that the RPSI is a volunteer group, that will take quite some time. No.186 has proven herself to be a remarkably reliable engine over the last few years, and I’ll certainly miss seeing her around.

The weather forecast for the day was awful, but thankfully it turned out to be a little too pessimistic.

The first train of the day departed Dublin at 10:50am, and was scheduled to do the journey out to Maynooth in about 45 minutes. I decided to meet the train at my new favourite location, Donaghmore Cemetery, located a little east of Maynooth near Pike Bridge and milepost 13¼. The track runs on a slight embankment here, so you get a nice view of the trains with the old walled cemetery framing the left side of the shot.

My plan was to shoot video and stills in the field next to the cemetery, and to set up my little GoPro camera next to the line on a little tripod. Things didn’t quite go to plan when I foolishly started to tinker with the settings on the GoPro (over a wireless connection) a little too close to the train’s due time. The train was 5 minutes early, and I missed the shot. I did still get usable video and stills from the field, but no closeup of the train. I was also unlucky with the light, the forecast had been for wind and rain, but instead it was windy and partially overcast – the sun had been out until about 30 seconds before the train arrived, and this would be the pattern for the rest of the day.

View Large on Flickr

I wasn’t sure how long the train would stay in Maynooth before starting back towards Dublin so I didn’t want to move too far to capture the return leg. I decided to set up on Matt Goff bridge on outskirts of Leixlip village which gives a nice view of the line as it snakes down hill through the s-bends between Blakestown level crossing and the bridge.

Because there is no longer a turntable in Maynooth, No.186 would be running ‘backwards’ (tender-first) on the return leg. This makes her less photogenic, but still worth shooting IMO.

My plan was to set up my main video camera with a zoom lens to capture the train rounding the curves in the distance, and to use the GoPro to capture the wider field of view. I shot some test footage and it looked great, but, as had happened earlier, the sun went away just as the train approached, and this time the wind picked up too – adding a lot of camera shake to the video.

View Large on Flickr

The afternoon train departed Dublin at 2:15pm, and I had planned to meet it at Louisa Bridge in Leixlip, but, at the time I would have had to leave for Louisa Bridge it was raining heavily. By the time the rain stopped I only had enough time to get as far as Donaghmore comfortably. Having been disappointed to miss the closeup shot in the morning, I decided to have a second try at it, and to experiment with a different point of view for shooting my main shots as well. Once again, the sun went away about 30 seconds before the train arrived, but, I did get my closeup shot this time!

View Large on Flickr

For the final shot of the day I’d planned to move to the 13th lock near Blakestown level crossing, but once again the rain interfered with my plans. I could see a very heavy shower on the horizon, so I decided to shelter under Pike Bridge, and to try to capture the train from there, using the bridge as a nice frame for the shots. I set up my D5100 on my tripod to shoot video, turned around to set up the GoPro for a reverse view, and heard a bone-chilling clatter. The bridge had acted like a funnel, amplifying the strong gusts of wind, and blown my tripod over. The D5100 didn’t survive it’s fall. After uttering some choice expletives, I set the GoPro up on my main tripod, and captured some usable footage anyway.

View Large on Flickr

I always try to choose the location for my final shot of any steam special shoot such that it allows me to get a shot of the train heading off into the distance. The nice thing about bridges is that they have two sides, so as soon as I captured the train approaching the bridge I turned around ready to get the final ‘off into the distance’ shot. Conveniently a dog stopped just in front of me to watch the train, so despite the poor light, I really like my final shot of the day.

View Large on Flickr

I collected all my shots from the day into a set on Flickr:

And I uploaded my video to YouTube:

Last time I focused on the shell, and avoided getting in any way specific about the actual commands that we will be executing within the Bash shell. I thought it was very important to make as clear a distinction between command shells and commands as possible, so I split the two concepts into two separate segments. Having focused on command shells last time, this instalment will focus on the anatomy of a command, but will start with a quick intro to the Terminal app in OS X first.

Introducing the Terminal Window

You’ll find Terminal.app in /Applications/Utilities. Unless you’ve changed some of the default settings (or are using a very old version of OS X), you will now see a white window that is running the bash command shell that looks something like this:

Lets start at the very top of the window with its title bar. At the left of the title is a proxy icon representing the current directory for the current Bash command shell, and beside it the name of that folder. (Note that directory is the Unix/Linux/DOS word for what OS X and Windows refers to as a folder.) Like Finder windows, Terminal sessions are always “in” a particular directory/folder. After the current directory will be a dash, followed by the name of the process currently running in the Terminal session (in our case, a bash shell). The current process is followed by another dash, and then the dimensions of the window in fixed-width characters.

Within the window itself you will likely see a line of output telling you when you last logged in, and from where (if it was on this computer it will say ttys followed by some number, if it was from another computer, it will give the computer’s hostname). This will be followed on the next line by the so-called command prompt, and then the input cursor.

Lets have a closer look at the command prompt. As with almost everything in Linux/Unix, the prompt is entirely customisable, so although Bash is the default shell on lots of different operating systems, the default prompt on each of those systems can be different. Lets first look at the default Bash command prompt on OS X:

On OS X, the prompt takes the following form:

hostname:current_directory username$

First you have the hostname of the computer on which the command shell is running (defined in System Preferences → Sharing → Computer Name). This might seem superfluous, but it becomes exceptionally useful once you start using ssh to log in to other computers via the Terminal.

The hostname is followed by a : and then the command shell’s current directory (note that ~ is short-hand for “the current user’s home folder”, more on this next time).

The current directory is followed by a space, and then the Unix username of the user running the command shell (defined when you created your OS X account, defaults to your first name if available). Finally, there is a $ character (which changes to a # when you run bash as the root user). Again, this might not seem very useful at first, but there are many reason you may want to switch your command shell to run as a different user from time to time, so it is also very useful information.

As an example of how the default shells differ on different operating systems, below is an example from a RedHat-style Linux distribution (CentOS in this case):

As you can see, it contains the same information, but arranged a little differently:

[username@hostname current_directory]$

Finally, Debian-style Linux distributions like Ubuntu use a different default promote again, but also showing the same information:

username@hostname:current_directory$

Handy Tip: if you find the text in the Terminal window to small to read, you can make it bigger or smaller with cmd+ or cmd-. This will affect just your current Terminal window. You can permanently change the default by editing the default profile in Terminal → Preferences ... → Settings

The Anatomy of a Command

Now that we understand the different parts of our Terminal window, lets have a look at the structure of the actual commands we will be typing at that cursor!

I want to start by stressing that the commands executed by a command shell are not determined by the command shell, but by the operating system. Regardless of whether you use Bash on OS X, or zsh on OS X, you will have to enter OS X commands. Similarly, if you use Bash on Linux, you will have to enter Linux commands. Thankfully Linux and Unix agree almost entirely on the structure of their basic commands, so with a very few (and very annoying) exceptions, you can use the same basic commands on any Linux or Unix distribution (remember that at its heart OS X is Free BSD Unix).

Commands take the form of the command itself optionally followed by a list of arguments separated by spaces i.e.:

command argument_1 argument_2 ... argument_n

Arguments are a mechanism for passing information to a command. Most commands need at least one argument to be able to perform their task, but some don’t. Both commands and arguments are case-sensitive, so beware your capitalisation!

For example, the cd (change directory) command takes one argument (a directory path):

bart-imac:~ bart$ cd /Users/Shared/
bart-imac:Shared bart$

In this example, the command is cd, and the one argument passed is /Users/Shared/.

Some commands don’t require any arguments at all, e.g. the pwd (present working directory) command:

bart-imac:~ bart$ pwd
/Users/bart
bart-imac:~ bart$

It is up to each command to determine how it will process the arguments it is given. When the developer was creating the command he or she will have had to make decisions about what arguments are compulsory, what arguments are optional, and how to parse the list of arguments the command is given by the shell when being executed.

In theory every developer could come up with their own mad complex scheme for parsing argument lists, but in reality most developers loath re-inventing the wheel (thank goodness), so a small number of standard libraries have come into use for parsing arguments. This means that many apps use very similar argument styles.

As well as accepting simple arguments like the cd command above, many apps accept specially formatted arguments referred to as flags. Flags are usually used to specify optional extra information, with information that is required taken as simple arguments. Flags are arguments (or pairs of arguments) that start with the - symbol.

The simplest kinds of flags are those that don’t take a value, they are specified using a single argument consisting of a - sign followed by a single letter. For example, the ls (list directory) command can accept the flag -l (long form listing) as an argument. e.g.

bart-imac:Shared bart$ ls -l
total 632
drwxrwxrwx  3 root   wheel     102  5 Dec  2010 Adobe
drwxrwxrwx  3 bart   wheel     102 27 Mar  2012 Library
drwxrwxrwx@ 5 bart   wheel     170 28 Dec 21:24 SC Info
drwxr-xr-x  4 bart   wheel     136 22 Feb 21:42 cfx collagepro
bart-imac:Shared bart$

The way the standard argument processing libraries work, flags can generally be specified in an arbitrary order. The ls command also accepts the flag -a (list all), so the following are both valid and equivalent:

bart-imac:Shared bart$ ls -l -a

and

bart-imac:Shared bart$ ls -a -l

The standard libraries also allow flags that don’t specify values to be compressed into a single argument like so:

bart-imac:Shared bart$ ls -al

Sometimes flags need to accept a value, in which case the flag stretches over two arguments which have to be contiguous. For example, the ssh (secure shell) command allows the port to be used for the connection to be specified with the -p flag, and the username to connect as with the -l flag, e.g.:

bart-imac:Shared bart$ ssh bw-server.localdomain -l bart -p 443

These single-letter flags works great for simple commands that don’t have too many options, but more complex commands often support many tens of optional flags. For that reason another commonly used argument processing library came into use that accepts long-form flags that start with a -- instead of a single -. As well as allowing a command to support more flags, these longer form flags also allow values to be set within a single argument by using the = sign.

As an example, the mysql command (needs to be installed separately on OS X) allows the username and password to be used when making a database connection to be specified using long-form flags:

...$ mysql --username=bart --password=open123 example_database

Many commands support both long and short form arguments, and they can be used together, e.g.:

...$ mysql --username=bart --password=open123 example_database -v

So far we know that commands consist of a command optionally followed by a list of arguments separated by spaces, and that many Unix/Linux commands use similar schemes for processing arguments where arguments starting with - or -- are treated in a special way, and referred to as flags. That all seems very simple, but, there is one important complication that we have to address before finishing up for this segment, and that's special characters.

Within Bash (and indeed every other command shell), there are some characters that have a special meaning, so they cannot be used in commands or arguments without signifying to the command shell in some way that is should interpret these symbols as literal symbols, and not as representations of some sort of special value or function.

The most obvious example from what we have learned today is the space character, it is used as the separator between commands and the argument list that follows, and within that argument list as the separator between individual arguments. What if we want to pass some text that contains a space to a command as an argument? This happens a lot because spaces are valid characters within file and folder names on Unix and Linux, and file and folder names are often passed as arguments.

As well as the space there are other symbols that have special meanings. I won’t explain what they mean today, but I will list them:

• space
• #
• ;
• "
• '
• `
• \
• !
• $
• (
• )
• &
• <
• >
• |

You have two choices for how you deal with these special characters when you need to include them within an argument, you can escape each individual special character within the argument, or you can quote the entire argument.

Escaping is easy, you simply pre-fix the special character in question with a \. If there are only one or two special characters in an argument this is the simplest and easiest solution. But, it can become tedious if there are many such special characters.

Lets use the echo command to illustrate escaping. The echo command simply prints out the input it receives. The follow example passes the phrase Hello World! to the echo command as a single argument. Note that this phrase contains two special characters that will need to be escaped, the space and the !:

bart-imac:~ bart$ echo Hello\ World\!
Hello World!
bart-imac:~ bart$

If you don’t want to escape each special character in an argument, you can quote the argument by prepending and appending either a " or a ' symbol to it. There is a subtle difference between using ' or ".

When you quote with ' you are doing so-called full quoting, every special character can be used inside a full quote, but, it is impossible to use a ' character inside a fully quoted argument. For example:

bart-imac:~ bart$ echo '# ;"`\!$()&<>|'
# ;"`\!$()&<>|
bart-imac:~ bart$

When you quote with " on the other hand you are doing so-called partial quoting, which means you can use most special characters without escaping them, but not all. Partial quoting will become very important later when we start to use variables and things, because the biggest difference between full and partial quoting is that you can’t use variable substitution with full quoting, but you can with partial quoting (don’t worry if that makes no sense at the moment, it will later in the series).

When using partial quoting you still have to escape the following special characters:

• "
• `
• \
• $

For example:

bart-imac:~ bart$ echo "# ;\!()&<>|"
# ;\!()&<>|
bart-imac:~ bart$

and:

bart-imac:~ bart$ echo "\\ \$ \" \`"
\ $ " `
bart-imac:~ bart$

There are a few other peculiar edge cases with partial quoting – for example, you can’t end a partial quote with a !, and you can’t quote just a * on its own (there may well be more edge cases I haven’t bumped into yet).

That’s where we’ll leave it for this segment. We’ve now familiarised ourselves with the OS X Terminal window, and we’ve described the anatomy of a Unix/Linux command. In the next segment we’ll look at the Unix/Linux file system, and at some of the commands used to navigate around it.

In the last CCATP we did a very detailed segment on email security, and despite the fact that with the benefit of hind-sight I realise it was too much to do at once and should have been split into two segments, it received the strongest listener response of anything of any of my many contributions to the NosillaCast in the last 5 or more years. I hope I’m right in interpreting that as evidence that there are a lot of NosillaCast listeners who want to get a little more technical, and get their hands dirty with some good old-fashioned nerdery!

In this first segment I just want to lay a very basic foundation. I plan to take things very slowly with this series, so I’m going to start the way I mean to continue. Lets start with some history and some wider context.

Before the days of GUIs (Graphical User Interfaces), and even before the days of simple menu-driven non-graphical interfaces like the original Word Perfect on DOS, the way humans interacted with computers was through a “command shell”. Computers couldn’t (and still can’t) speak or interpret human languages properly, and humans find it very hard to speak in native computer languages like binary machine codes (though early programers did actually have to do that), so new languages were invented to help bridge the gap, and allow humans and computers to meet somewhere in the middle. The really big problem is that computers have absolutely no intelligence, so they can’t deal with ambiguity at all. Command shells use commands that look Englishy, but they have very rigid structures (or grammars if you will) that remove all ambiguity. It’s this rigid structure that allows the automated translation from shell commands to binary machine code the computer can execute.

When using any command shell, the single most important thing to remember is that computers are absolutely stupid, so they will do EXACTLY what you tell them to, no matter how silly the command you give them. If you tell a computer to delete all the files on a hard drive, it will, because, well, that’s what you asked it to do! Another important effect of the computers’ total lack of intelligence is that there is no such thing as “close enough” – if you give it a command that’s nearly valid, a computer can no more execute it than if you’d just mashed the keyboard with your face. Nearly right and absolute gibberish are just as unintelligible to a computer. You must be exact, and you must be explicit at all times.

No wonder we went on to invent the GUI, this command shell malarky sounds really complicated! There is no doubt that if the GUI hadn’t been invented the personal computer wouldn’t have taken off like it has. If it wasn’t for the GUI there’s no way there would be more computers than people in my parents home (two of them, and including iPhones and tablets, 6 computers)! Even the nerdiest of nerds use GUI operating systems most of the time, because they make a lot of things both easier and more pleasant. BUT – not everything.

We all know that a picture says a thousand words, but when you are using a GUI it’s the computer that is showing you a picture, all you get to do is make crude hand gestures at the computer, which I’d say is worth about a thousandth of a word – so, a single shell command can easily be worth a thousand clicks. This is why all desktop OSes still have command shells built in – not as their ONLY user interface like in times past, but as a window within the GUI environment that lets you communicate with the computer using the power of a command shell. Even Microsoft understands the power of the command shell, DOS may be dead, but the new Windows Power Shell is giving Windows power users a new, more modern, and more powerful command shell than ever before. Windows 8 may have removed the Start menu, but Powershell is still there! All Linux and Unix distros have command shells, and OS X gives you access to an array of different Command Shells through Terminal.app.

Just like there is no one GUI interface, there is no one command shell. Also, just like most GUIs are at least somewhat similar to each other, they all use icons for example, most command shells are also quite similar, having a command prompt that accepts commands with arguments to supply input to the commands or alter their behaviour. OS X does not ship with one command shell, it ships with SIX (sh, bash, zsh, csh, tcsh, and ksh)!

You can see the list of available shells (and set your default shell) by opening System Preferences, going to the Users & Groups pref pane, unlocking it, then right-clicking on your username in the side bar, and selecting Advanced Options ...:

The default shell on OS X is the Bourne Again Shell (bash), so that’s the shell we’ll be using for the remainder of this series. If you’ve not altered the defaults, then when ever you open a Terminal window on OS X, what you’re presented with is a bash command shell. Bash is an updated and improved version of the older Bourne Shell (sh), which was the default shell in Unix for many years. The original Bourne Shell dates all the way back to 1977, and is called after it’s creator, Stephen Bourne. The Bourne Again Shell is a ‘newer’ update to the Bourne Shell dating back to 1989, the name being a nerdy religious joke by it’s author, Brian Fox. The Bourne Again Shell was not the last in the line of shells tracing their origins to the Bourne Shell, there is also zsh which dates back to 1990, but despite being a more powerful shell, it hasn’t taken off like bash has.

So, what does a shell do? Does it just let you enter a single command and then run it? Or is there more to it? Unsurprisingly, there’s a lot more to it! The shell does it’s own pre-processing before issuing a command for you, so a lot of things that we think of as being part of how a command works, are actually features provided by the command shell. The best example is the almost ubiquitous * symbol. When you issue a command like chmod 755 *.php, the actual command is not given *.php as a single argument that it must then interpret, no, the * is interpreted and processed by the shell before being passed on to the chmod command. It’s the shell that goes and looks for all the files in the current folder that end in .php, and replaces the *.php bit of the command with a list of all the actual files that end in .php, and passes that list on to the chmod command.

As well as providing wildcard substitution (the * thing), almost all shells also provide ‘plumbing’ for routing command inputs and outputs between commands and files, the definition of variables to allow sequences of commands to be generalised in a more re-usable way, simple programming constructs to enable conditional actions, looping, and the grouping of sequences of commands into named functions, and the execution of a sequence of commands inside a file (scripting), and much more. Different shells also provide their own custom features to help make life at the command prompt easier for users – my favourite is tab-completion which is the single best thing bash has to offer over sh IMO. OS X also brings some unique features to the table, with superb integration between the GUI and the command shell through features like drag-and-drop support in Terminal.app and shell scripting support Automator. Of all the OSes I’ve used, OS X is the one that makes it the easiest to integrate command line programs into the GUI.

I’ll end today by explaining an important part of the Unix philosophy, a part that’s still very much alive and well within OS X today – Unix aims to provide a suite of many simple command line tools that each do just one thing, but do it very well – complex tasks can then be achieved by chaining these simple commands together using a powerful command shell. Each Unix/Linux command line program can be seen as a lego brick – not all that exciting on their own, but using a bunch of them, you can build fantastic things! My hope for this series is to help readers and listeners like you develop the skills to build your own fantastic things to make your computing lives easier. Ultimately the goal is to help you create more and better things by using automation to free you from as many of your repetitive tasks as possible!

I’ve blogged about the tyranny of free before, so I don’t want to focus on that today, instead I want to take a step back and talk about the importance of following the money.

When it comes to software we generally don’t tend to get locked in anymore these days. Not even Office and Photoshop lock you in now. If we are using an app that suddenly goes out of existence, or changes in some way that turns us off, we are almost always free to switch to an alternative app now. So, understanding the finances behind most of the apps we use is not that important. There is one very important caveat though – apps that act as interfaces to cloud services are very different beasts, because we are using those apps to share our data with a company or organisation.

The really important thing is understanding the financial underpinnings of the cloud services we use. Cloud services are entrusted with a lot of our information, and we can very easily find ourselves locked into them. This is why I think it’s very important to understand how a service is supposed to make money before starting to use it. In some rare cases like wikipedia, online services are run by non-profit charitable organisations, but in general, services are run by companies, and companies exist to make money. Some people like to fantasise that companies are altruistic beings that are there to serve their admirers, but those are the ravings of fanboys. All companies exist to make money. Apple, Microsoft, FaceBook, Google, Amazon, Evernote, DropBox, Yahoo, Instapaper …

While all companies exist to make money, all companies are not the same, because there are many different strategies for making money, and those different approaches affect users in very different ways. You can take it for granted that companies exist to make money, what you need to concern yourself with is how those companies make their money, because that’s what will determine the incentives acting on those companies, and affect how they see you and your data.

As strange as it sounds, the first question you have to ask about a company is whether or not they have a business model at all! You’d imagine that no company could get funding without knowing how it will turn a profit some day, but you’d be wrong. The digital equivalents of South Park’s underpants gnomes are hard at work in silicone valley! Step 1 – get lots of users, step 3 profit!

When you become dependant on a service that’s run by digital underpants gnomes you’re likely to end up suffering in one of two ways. Either, the service collapses because running losses year after year after year is actually a really bad way to run a business. Or, the service changes dramatically once it has attracted enough users to be able to sell them. Sometimes that takes the form of the entire company being sold, e.g. Instagram, and sometimes it takes the form of the company pivoting in some way to convert their users into profit themselves.

Instagram is a good recent example of the first way digital underpants gnomes can convert users into profit, and for me Twitter is the ultimate example of the second. Users, and particularly third party app developers, played a major role in making Twitter the success it is today. Many of the things we consider core features of Twitter were actually developed by third party developers, not Twitter themselves. The two most striking examples are probably the word ‘tweet’ (Twitter called them ‘twits’ originally), and the concept of @replies. Now, Twitter are pivoting to try sell their user base to advertisers. Originally, they had no customers, now advertisers are their customers. Twitter users are not Twitter’s customers, they are Twitter’s product, and the third party developers that played a large part in making Twitter are now a problem that needs to be death with, hence the crushing blows against third part app developers over the last year. As a Twitter user who has relied on a thriving third-party client ecosystem to get a good Twitter experience, I am suffering from Twitter’s pivot. Had I understood what I do now about the dangers of digital underpants gnomes I’d never have joined Twitter in the first place.

Assuming the company you’re evaluating does have a business model, the next key question is who their customers are. The primary concern of any company is to keep their customers happy and coming back for more. It’s in a company’s interest to do the things that their customers like, and it’s against a company’s interests to upset their customers.

When the users who use a service are also the customers, things are pretty simple – companies are incentivised to do right by their users. When users are not the customers though, then their interests are inevitably secondary to the interests of the customers, and when the interests of the customers are at odds with the interests of users, then the customers win and users lose out.

Because, for what ever reason, many people in our society want something for nothing, many services rely on advertising as their income stream. This sets up precisely the kind of perverse incentive I worry about. If users were customers then it would be in the company’s interest to protect their data, but since they are not, that’s not the case. But, it gets worse, because it is in the advertisers interests to get as much information as possible about the users they are paying to advertise at, so, companies are incentivised to give as much data about their users to their customers, the advertisement agencies, as they can get away with.

Ultimately, we are dealing with market forces here, and as anyone living in a capitalist society knows, market forces are very strong, so when they are working against you, that’s dangerous.

Perhaps it might be helpful to look at some real-world examples. Please bear in mind that these are my personal opinions, obviously I think I’m right, but I could of course be wrong.

1. FaceBook – it seldom gets more black and white than this IMO. FaceBook’s customers are advertisers! Recently the lines have gotten a little blurred when FaceBook started to treat some of their users like advertisers by charging to have posts ‘promoted’ (i.e shown to all your ‘friends’ and not just to a random sub-set of them), but, even still, it seems clear to me that the vast majority of FaceBook users not FaceBook’s customers, while advertisers clearly are. I think you can understand the way their terms of services have evolved over the years as the inevitable result of the perverse incentives created by having advertisers as their customers rather than their users.
2. Google are a little more complicated than FaceBook because they do a lot. They are best know as a search engine, but they are a big service provider too, with extensive cloud offerings, and they also make two operating systems, and a number of apps including a popular browser. They are also stating to directly monetize some of their web services by charging customers for their use. E.g., if you want Google Apps on your own domain, you have to pay now, the free tier has been removed. However, when it comes to making the big money, I think it’s still all about the ads.
3. Amazon – I find Amazon an interesting case – they clearly gather a lot of data about their users – they know everything you search for on their store, and everything you buy there, but they use that data differently to Google, they use it to figure out what else you might like to buy from them via their uncannily good suggestion engine. At first glance, you might think Amazon’s data gathering is no different to FaceBook’s, but I think it is, Amazon are not using our data as their product, they are using our data to sell, us, their customers, more of their products, the things they sell in their stores. I sometimes think of Amazon as the digital Wall Mart.
5. Apple are not clear-cut either. They sell hardware, they sell apps, they have a cloud service, and they sell ads. When you look at Apple’s earnings reports though, it’s clear that their primary customers are people who buy iThings, followed by people who buy media and software through Apple’s stores. iAds is really quite insignificant in the grand scheme of things. iCloud is also a very different cloud service to Google’s. There are no ads, it’s a freemium service to make iThings more attractive. I think Apple had no choice but to make a free iCloud tier because Google have a free cloud as do Microsoft. Google use their cloud as an ad platform though, while Apple don’t. On the whole, Apple primarily make their money by selling things to their users, so, Apple’s users are their main customers.
6. I find Microsoft very difficult to analyse at the moment. Historically they were very easy to understand, they made software, and they primarily sold it to corporate IT departments. You could see by their feature sets that corporate IT was who Microsoft focused on. Now It’s very hard to tell what Microsoft’s focus is. They still do the software thing, and they still sell to corporate IT, but, they are also trying to copy Google and become a search and advertisement company, and they are also starting to focus on end-users as well as corporate IT with the XBox and their new phones and tablets. The fact that ads are starting to infest XBox Live shows how complicated the incentives are. I’m not sure if even Microsoft know what their long-term strategy is, so it’s very hard to divine what incentives are driving Microsoft’s decisions, and how they will affect users going forward.
7. Evernote and DropBox both have, and have had for a long time, business models that work by selling a premium services to a small number of their users, while offering a free service to many more users to help support their ecosystem. Free users on Evernote and DropBox are not their products though, they are their market. The hope is not to sell free users’ data for profit, but to turn a few percent of those free users into paying customers. Those that never sign up are still adding value to the products because both consider easy sharing an important feature, and you can’t expect people to want to share with you if it will cost them! One of the reasons I chose to trust dropbox with my data is that I heard an interview with their CEO on a podcast where he explained their business model candidly I was impressed that even from the early days they knew where their incentives lay, and I thought they were healthy incentives.
8. Flickr, SmugMug and 500px also have a similar model to DropBox and EverNote, while Picassa by contrast has a very different model, and can be thought of as the GMail of photosharing.

The bottom line is that I’m not interested in telling people what companies or services to choose. Different people will make different decisions based on the same data because we all have different priorities. My hope is that people will stop, investigate, and think, before signing up to a service. It doesn’t matter what decision you make, as long as it’s an informed one – so follow the money before you invest your time and data into a cloud service! Oh, and watch out for those digital underpants gnomes too

]]> http://www.bartbusschots.ie/blog/?feed=rss2&p=2409 0 http://www.bartbusschots.ie/blog/?p=2406 http://www.bartbusschots.ie/blog/?p=2406#comments Thu, 17 Jan 2013 00:56:16 +0000 Bart Busschots http://www.bartbusschots.ie/blog/?p=2406 As usual, it’s taken me a while to get all my photos and videos from the Santa Special season edited and organised. Mind you, mid-January is a definite improvement over March, which is how late I was last year!

As usual, the Railway Preservation Society of Ireland (RPSI) ran two Santa Specials a day from Dublin to Maynooth to Dublin on the first three weekends of December. The trains were all scheduled to be hauled by the societies recently overhauled 2-6-0 steam loco No.461, which has been painted in the beautiful old CIE green livery. The same loco had been scheduled to run the 2011 trains too, but she was not ready on time, and all the 2011 Santa Specials from Dublin had to be hauled by diesel locos instead. Having missed the Santa Specials No.461 went on to make a decidedly unreliable return to steam over the spring and summer of 2012. Apparently No.461 took a while to bed in after her previous overhaul too, but apparently she did become very reliable over time. Hopes were high that she’d prove herself with the Santa Specials. She almost made it through the whole season, but the last two days proved too much for her, and the final three of the twelve trains had to be hauled by diesel locos.

Unfortunately my health proved to be even less reliable than No.461, and caused me to miss three out of the six days, and hence six out of the twelve trains.




Day 1 (1 December 2012)

The weather for the first day of the 2012 Santa Special season could not have been better with beautiful clear blue skies all day.

In the morning I was waiting for the outward leg of the first train at Pike Bridge just east of Maynooth, and I caught the return leg at Matt Goff Bridge just next to Intel Ireland in Leixlip. Because there is no turntable in Maynooth anymore, the return legs from Maynooth back to Dublin have to run tender first (i.e. with the loco facing backwards but at the front of the train).

In the afternoon I headed for Leixlip, meeting the outward leg at Louisa Bridge on the western edge of the village, and then headed back towards Maynooth to meet the return leg at Donaghmore Cemetery near Pike Bridge.

My favourite shot from the day was probably the one taken at Louisa Bridge which I’ve included below. The shot needs a black background to really shine, so click on the image to see it large and on black over on my Flickr stream.


(Click to Enlarge)

Days 2 – 4 (2, 8 & 9 December 2012)

On the second day I had the start of what would turn out to be a very nasty cold indeed, but I felt well enough that I would have gone out had the weather been OK, but it wasn’t. It was cold and wet all day, so I decided to play it safe and stay home here it was warm and the dry.

By the time the weekend of the third and fourth day rolled around the cold that was just starting on the previous Sunday had turned very nasty indeed. I was so miserable that I didn’t feel up to going out and shooting even though the sun shone all day.

Day 5 (15 December 2012)

Thankfully I was over my cold by the final weekend of the season, and was very keen to get out there with my camera.

On the Saturday morning I decided to try out a new location, and set myself up in a field next to the beautiful old cemetery at Donaghmore near Pike Bridge. The weather forecast had been for a beautiful sunny morning, but alas Met Eireann got it wrong, instead it was dull, overcast, and very windy. The shot I was able to get was beautiful, but very bleak. Most of my train shots have a happy colourful feel, this one is very different. Bleak though my first shot was, it did show that the location had promise, so I resolved to try again the next day.


(Click to Enlarge)

For the return leg that morning I moved the short distance to Pike bridge, where I could get a good view of the train passing under the bridge and past the small harbour on the Royal Canal located just east of the bridge which served Carton Estate, the seat of the Dukes of Leinster. I didn’t know it at the time, but this would be my last view of No.461 in 2012.


(Click to Enlarge)

That afternoon I was hoping to get my first shot of a steam train at milepost 13. This is a location that only became usable this year when Waterways Ireland cleared a lot of the trees and bushes that had blocked the view of the railway line along this part of the Royal Canal. The train was late, and as the delay got longer and longer I started to fear that No.461 had run into trouble. Sure enough, about half an hour after the train was due into Maynooth, a single headlamp appeared in the distance without any accompanying clouds of steam. Soon the distinctive sound of an Irish Rail 071 class diesel broke the silence, the train was being pulled by loco No.083. This was my first shot of a diesel loco at milepost 13, so I still got a first, just not the one I was hoping for.


(Click to Enlarge)

For the return leg I moved to Matt Goff Bridge in Leixlip again, but because of the delay in getting to Maynooth the return leg was also delayed, and the short winter day had run out of good light before the train arrived.

Day 6 (16 December 2012)

The morning of the sixth and final day of the season started off fantastically, the sun was shining in a clear blue sky, and it was even quite warm for a December day in Ireland. Having decided that my new location at Donaghmore had potential, I headed back to capture a train in the sun. I was hopeful that perhaps No.461 could have been repaired overnight, but I wasn’t surprised when another 071 class diesel loco came into view instead, No.076 this time.

For the return leg I headed to Blakestown level crossing at the 13th lock on the Royal Canal. I set up my camera along the banks of the canal just beyond the level crossing the the lock, where there is an uninterrupted view along a very picturesque section of the canal. I was able to use a hiking signpost for the Royal Canal Way to add foreground interest to the shot. Conveniently, the little man on the sign was painted on in almost the same shade of yellow as the speed limit sign next to the track and the front of the loco.


(Click to Enlarge)

By the afternoon the weather had completely turned. It was raining so heavily that I didn’t think I’d be able to get any shots at all, but just as the train was due the rain died down and the sun even came out for a few minutes. It turns out the very bad weather was a blessing in disguise, providing simply superb storm light, and even a subtle hint of a rainbow!


(Click to Enlarge)

The weather continued very showery, so I decided to play it safe for the return leg, which would be my last opportunity to photograph a special in 2012, and set up my gear under Pike Bridge. This allowed me to use the bridge both as shelter, and to frame the train as it approached. I could also get a second shot by running out the other side of the bridge to catch the back of the train as it vanished off into the distance. The first shot worked really well, and is my favourite shot from the season by far, and I quite like the other shot I got too. I’ve included both shots below.


(Click to Enlarge)


(Click to Enlarge)

I’ve gathered all my best shots from the season into a Set over on Flickr:



]]> http://www.bartbusschots.ie/blog/?feed=rss2&p=2406 0 http://www.bartbusschots.ie/blog/?p=2393 http://www.bartbusschots.ie/blog/?p=2393#comments Sat, 29 Dec 2012 17:57:41 +0000 Bart Busschots http://www.bartbusschots.ie/blog/?p=2393 Back in 2011 I wrote a blog post explaining how to create an OS X Service for stripping keywords from image files. In this post we’ll use the same technique to create a Service for stripping geotags from JPEG images.

As with the keyword stripping service, there are two prerequisites for this action, one is required, one is optional. You absolutely MUST have install EXIFTool installed, and it would be good if you also had Growl installed, but it’s not essential.

Start by opening Automator, and creating a new Service workflow.



At the top of the new service workflow, alter the settings so the services takes as it’s input Image Files and that the service is available in just the Finder.



Find the Run Shell Script action and drag and drop it into the workflow (you’ll find the action in the Utilities category, or by searching in the search bar). Change the Shell option at the top of the Run Shell Script action to /usr/bin/perl, and the Pass input option to as arguments.

To do the actual work we will add the following perl code into the action:

perl
< view plain text >
1. foreach my $image (@ARGV) {
2.     $image = quote_file_name($image);
3.     `/usr/bin/exiftool -geotag= -overwrite_original $image`;
4. }
5.  
6. # utility function to sanitise file names for use when shelling out
7. sub quote_file_name{
8.     my $file = shift;
9.     $file =~ s/([`"\\\$])/\\$1/g;
10.     return '"'.$file.'"';
11. }

Finally, it would be very helpful to have the service tell us when it is finished, so this is where Growl comes in. If you have it installed, drag and drop a Show Growl Notification into the workflow and enter a message of your choice.

Your Service should now look something like the screenshot below. You can now go ahead and save the Service and give it a sensible name like Strip Geotag.



You should now be able to use the service by right-clicking on a file in the Finder and choosing your new service from the Services section. If you have only a few services installed they will be shown at the bottom of the contextual menu, if you have more, they will show in a sub-menu titled Services.



I’ve included a zipped version of my service below. This service uses the Growl action, so if you don’t have Growl installed you’ll need to open the Service in Automator and remove the Growl action before you can use the Service.

Download

• Strip Geotag Service Version 1 (Requires Growl and EXIFTool)
]]> http://www.bartbusschots.ie/blog/?feed=rss2&p=2393 1 http://www.bartbusschots.ie/blog/?p=2386 http://www.bartbusschots.ie/blog/?p=2386#comments Sat, 22 Dec 2012 17:02:03 +0000 Bart Busschots http://www.bartbusschots.ie/blog/?p=2386 I recently gave a talk to Astro2, the Astronomy and Physics Society in NUI Maynooth. The talk focused on taking photos of the heavens without breaking the bank. If you set your expectations appropriately you can shoot the sky with literally any camera. You’ve not going turn your iPhone into the Hubble Space Telescope of course, but you can always get something nice, even if it’s “just” including the Moon and planets into your landscape shots.

The core idea is that you need to take as much control away from the camera’s computer as possible, so that you can push the device right up to it’s limits. To do that you need to understand how your camera works, so the talk starts with a primer on the theory of photography.

The talk is very heavily illustrated with example photos, and each example comes with a description of the settings used when shooting it.

I was asked in the talk to upload the slide deck, so I have (sorry it took so long).


Click to View Presentation

]]> http://www.bartbusschots.ie/blog/?feed=rss2&p=2386 1 http://www.bartbusschots.ie/blog/?p=2383 http://www.bartbusschots.ie/blog/?p=2383#comments Sat, 22 Dec 2012 16:36:37 +0000 Bart Busschots http://www.bartbusschots.ie/blog/?p=2383 This week’s Insgragram TOS kerfuffle is nothing new. Instagram is not the problem, it’s just the latest symptom of a sick business model that has been allowed to become so dominant as to be almost un-challengeable – services on the web MUST be free, so you MUST give up your privacy and/or your intellectual property rights to enable the service providers profits. If you dare stand up for privacy then you are a greedy idiot who wants something for nothing, and you need to grow up and let the companies make money.

My problem is not that companies want to make profits, it’s their instance on selling our data to do it that I have a problem with. How about this for an idea – why not let people pay for services rather than insisting we all whore out our privacy and intellectual property?

In the past, when you wanted to set up a company you started with a business plan, and then you went out looking for the money to get your business off the ground. Before you started you knew how you were going to make money. In general, the users of your service were your customers, and you made your money by selling your service to them.

The modern ‘web2.0′ approach is different – you have an idea, you set up a free service (which you bank roll yourself or with ‘Angel’ or ‘Venture’ capital), you suck in as many users as you can, and then you sell the company to someone else and make a big profit. The whole time you are losing money hand-over-fist. The more popular your free service, the more money you lose, but the more valuable your company becomes. Although it’s ostensibly the company that is being sold, what’s really up for sale is the users – we are the cash-crop of this web2.0 business model.

Even if a free startup decides not to sell, like for example Twitter, the company still has to make the switch from losing to making money, and the way that is done is almost always by exploiting the users.

In order to make your human crop as valuable as possible you need to collect as much personal information about them and their activities as you can, and you need to get them to sign over as many of their rights to you as you can. It’s this perverse incentive that drives the massive erosion of our privacy that we are seeing at the moment.

It’s these perverse incentives that the free business model creates that I have such a problem with. As many people have said, if you are not paying for a service, then you are the product. I don’t want to be anyone’s product, I want to be their customer!

It’s true that there are non-free choices for a lot of things online. This blog is not hosted on a free service like Blogger or Tumblr because I want full control over my work, and the information I share. Instead, this blog is hosted on a server I pay for, so I am the customer. My chosen photo network is Flickr, because I can pay them for a pro account and become their customer. Similarly, I have a paid Instapaper account for managing my reading lists, and I pay to use the App.net social network.

However, the tyranny of free still hangs over me. I use many free services because I feel I have to. I have a Twitter account and a G+ account, because as a podcaster you need to be on at least SOME popular social networking sites. I’ve chosen those two over the likes of FaceBook and LinkedIn because I find their terms and practices the least worrying, but I’d still love to be able to pay for those services so they could stop selling me on to third parties to make their profit. My videos are hosted on YouTube because that’s where the community is. I’m perpetually tempted to switch to Vimeo’s paid service, but that’s not where the Irish railway nerds hang out, and YouTube’s terms are not too odious.

The tyranny of free is also spreading into applications. More and more companies are offering free apps that they then fill with ads. Thankfully there are enough companies that let you pay to remove the ads that I have only one app that I use regularly that is constantly advertising at me. When choosing apps I will always choose the paid alternative over the free add-supported one if I can.

Everyone has their limits on what they will tolerate in exchange for a free service, which is why you see so many angry responses to terms of service that push the boundaries. Unfortunately, over time all our boundaries are slowly being pushed further and further out. Had FaceBook’s TOS been what it is today on their first day, people would just not have joined, but like a frog in boiling water we have all been slowly ground down and we accept more and more without a fight.

I absolutely do not like the future we are heading towards. By exploiting our collective greed and dangling free stuff in front of our faces corporations have managed to entice our society into giving up our privacy in ways even totalitarian governments have failed to do. The Stasi would have killed for Google’s database!

It just does not have to be this way!

Some people will insist on not paying for things, and many services, particularly social networks, require a critical mass of people to become useful, so free can’t go away. However, it can co-exist with paid services. The simplest way to do this is with a so-called ‘freemium’ model. Your free users get sold to advertisers, while your paying users do not, and perhaps also get a more feature-rich service. This is how both Flickr and Instapaper work, and I’m a happy user of both services.

I’ve always hoped Twitter would go the freemium route. Imagine a world where you could pay a few Euro a month for a pro Twitter account which would enable the use of 3rd party apps free of silly limitations. The reason Twitter are hell-bent on destroying the 3rd party apps that made their service a success is that they need control of the user experience so they can sell their users to advertisers. Why not let the users who want apps pay for the use of the service instead!?

Another very interesting development is App.net, this is a completely paid social network. There are no free App.net accounts, which makes it an exceptionally brave move on their part. I’m really hoping it takes off and proves that there are alternatives to free, even in the social networking space. Because App.net was a paid service from day 1 they did not anger anyone by having to transition from a free loss-maker to a paid profit maker.

I say it’s time to stand up and demand to be allowed to pay for services. Each time someone puts up the straw-man that we HAVE to give up our privacy and intellectual property because a service is free, knock it down with the simple alternative that we be allowed pay for the service.

Let us become customers, I’m fed up of being the product!

]]> http://www.bartbusschots.ie/blog/?feed=rss2&p=2383 2