This post is part 6 of 37 in the series Taming the Terminal

In the previous instalment of this series we had a look at how standard Unix File Permissions worked. We looked at how to understand the permissions on existing files and folders, but not at how to change them. We also mentioned that the standard unix file permissions are now only a sub-set of the file permissions on OS X and Linux (OS X also supports file ACLs, and Linux has SELinux as an optional extra layer of security).

In this instalment we’ll start by biting the bullet and dive into how to alter standard Unix File permissions. This could well turn out to be the most difficult segment in this entire series, regardless of how big n gets, but it is very important, so if you have trouble with it, please don’t give up. After we do all that hard work we’ll end with a simpler topic, reading OS X file ACLs, and OS X extended file attributes. We’ll only be looking at how to read these attributes though, not how to alter them.

Listen Along: Taming the Terminal Podcast Episode 6

As a reminder, last time we learned that every file and folder in a Unix/Linux file system has three pieces of metadata associated with it that control the standard unix file permissions that apply to that file or folder – files have an owner (a user), a group, and a Unix File Permission Mask associated with them, and all three if these pieces of information can be displayed with ls -l. We’ll be altering each of these three piece of metadata in this instalment.

Altering Unix File Permissions – Setting the File Ownership:

The command to change the user that owns one or more files or folders is chown (change owner). The command takes a minimum of two arguments, the username to change the ownership to, and one or more files or folders to modify. E.g.:

The command can also optionally take a -R flag to indicate that the changes should be applied ‘recursively’, that is that if the ownership of a folder is changed, the ownership of all files and folders contained within that folder should also be changed. The chown command is very picky about the placement of the flag though, it MUST come before any other arguments E.g.:

Similarly, the command to change the group that a file belongs to is chgrp (change group), it behaves in the same was as chown, and also supports the -R flag to recursively change the group. E.g.:

Finally, if you want to change both user and group ownership of files or folders at the same time the chown command provides a handy shortcut – instead of passing just a username as the first argument, you can pass a username and group name pair separated by a :, so the previous two examples can be rolled into the one example below:

Altering Unix File Permissions – Setting the Permission Mask

The command to alter the permission mask, or file mode, is chmod (change mode). In many ways it’s similar to the chown and chgrp commands, it takes the same basic form, and supports the -R flag, however, the formatting of the first argument – the permission you want to set – can be very confusing.

The command actually supports two entirely different approaches to setting the permissions. I find both of them equally obtuse, and my advice to people is to pick one and stick with it. Long ago I chose the numeric approach to setting file permissions, so that’s the approach we’ll use here.

This approach is based on treating the three permissions, read, write, and execute as a three-digit binary number, if you have read permission, the first digit is a 1, if not, it’s a 0, and the same for the write and execute permissions. So, the permissions rwx would be represented by the binary number 111, the permissions r-x by 101, and r-- by 100. Since there are three sets of rwx permissions (user, group, everyone), a full unix file permission mask is defined by three three-digit binary numbers.

Unfortunately the chmod command doesn’t take the binary numbers in binary form, it expects you to convert them to decimal first, and pass it the three sets of permissions as three digits. This sounds hard, but with a little practice it’ll soon become second-nature.

The key to reading off the permissions is this table:

Unix File Permissions Conversion Table

Rather than trying to memorise the table itself, you should try to learn the process for creating it instead. The lighter coloured cells in the centre of the table are the important ones to be able to re-create on demand. They are not random, they are a standard binary to decimal conversion table, and you should notice that the three columns have a distinct pattern – the right-most column alternates from 0 to 1 as you go down, the column second from the right has two 0s, then two 1s, then two 0s etc, and finally the third column from the right has four 0s, then four 1s. If you wanted to convert a 4 digit binary number to decimal you would add a fourth column that has 8 0s then 1s, if you wanted to convert a 5 bit binary number you’d add yet another column where it’s eight 0s then eight 1s, and so on – each column you go to the left doubles the number of 0s and 1s before repeating.

If you can re-produce this table on demand you’ll have learned two things – how to do Unix file permissions, and how to convert any arbitrary binary number to decimal (though there are better ways if the binary number is has many digits).

Even if you don’t want to learn how to create the table, you’ll probably still be fine if you remember just the most common permissions:

  • 4 = read only
  • 5 = read and execute
  • 6 = read and write
  • 7 = full access

If you run a website for example, regular files like images or HTML pages and images should have permissions 644 (rw-r--r--: you get read and write, everyone gets read), and executable files and folders should have 755 (rwxr-xr-x: you get full permission, everyone can list the folder contents and read the files within).

Lets end with a few examples. If you want to alter a file you own so that you have read, write and execute permission, but no one else can access the file in any way you would use the command:

If the file should not be executable even by you, then you would use:

Clearly this is not intuitive, and it’s understandably very confusing to most people at first. Everyone needs to go over this a few times before it sinks in, so if it doesn’t make sense straight away, you’re not alone. Do please keep at it though, this is very important stuff.

Reading OS X File ACLs

We said last time that on OS X, a + at the end of a file permission mask signifies that the file has ACLs (access control lists) associated with it. These ACLS allow more granular permissions to be applied to files on top of the standard Unix File Permissions. If either the ACLs OR the standard unix permissions deny you the access you are requesting, OS X will block you.

You can read the ACLs associated with files by adding the -le flags to the ls command. If a file in the folder you are listing the contents of has file ACLs, they will be listed underneath the file, one ACL per line, and indented relative to the files in the list. Each ACL associated with a file is numbered, and the numbering starts from 0.

The ACLs read as quite Englishy, so you should be able to figure out what they mean just by looking at them. As an example, lets have a look at the extended permissions on OS X home directories:

By default all OS X home folders are in the folder /Users, which is the folder the above commands lists the contents of. You can see here that my home folder (bart) has one or more file ACLs associated with it because it has a + at the end of the permissions mask. On the lines below you can see that there is only one ACL associated with my home folder, and that it’s numbered 0. The contents of the ACL is:

As you might expect, this means that the group everyone is denied permission to delete my home folder. Everyone includes me, so while the unix file permissions (rwxr-xr-x) give me full control over my home folder, the ACL stops me deleting it. The same is true of the standard folders within my account like Documents, Downloads, Library, Movies, Music, etc..

If you’re interested in learning to add ACLs to files or folders, you might find this link helpful: www.techrepublic.com/blog/mac/…

Reading OS X Extended File Attributes

In the last instalment we mentioned that all files in a Linux/Unix file system have metadata associated with them such as their creation date, last modified date, and their ownership and file permission information. OS X allows arbitrary extra metadata to be added to any file. This metadata can be used by applications or the OS when interacting with the file.

For example, when you give a file a colour label, that label is stored in an extended attribute. If you give a file or folder a custom Finder icon, that gets stored in an extended attribute (this is how DropBox, the app that is, makes your DropBox folder look different even though it’s a regular folder. Similarly, spotlight comments are stored in an extended attribute, and third-party tagging apps also use extended attributes to store the tags you associate with a given file (presumably OS X Mavericks will adopt the same approach for the new standard file tagging system it will introduce to OS X).

Extended attributes take the form of name-value-pairs. The name, or key, is usually quite long to prevent collisions between applications, and, like plist files, is usually named in reverse-DNS order. E.g., all extended attributes set by Apple have names that start with com.apple, which is the reverser of Apple’s domain name, apple.com. So, if I were to write and OS X app that used extended file attributes, the correct thing for me to do would be for me to pre-fix all my extended attribute names with ie.bartb, and if Allison were to do the same she should pre-fix hers with com.nosillacast. (Note that this is a great way to avoid name-space collisions, since every domain only has one owner. This approach is used in many places, including Java package naming). The values associated with the keys are stored as strings, with complex data and binary data stored as 64bit encoded (i.e. HEX) strings. This means the contents of many extended attributes is not easily human-readable.

Any file that has extended attributes will have an @ symbol appended to its unix file permission mask in the output of ls -l. To see the list of the names/keys for the extended attributes belonging to a file you can use ls -l@.

You can’t use ls to see the actual contents of the extended attributes though, only to get their names. To see the names and values of all extended attributes on one or more files use:

The nice thing about the -l flag is that if the value stored in an extended attribute looks like it’s a base 64 encoded HEX string it automatically does a conversion to ASCII for you, and displays the ASCII value next to the HEX value.

Apple uses extended attributes to track where files have been downloaded from, by what app, and if they are executable, whether or not you have dismissed the warning you get the first time you run a downloaded file. Because of this every file in your Downloads folder will contain extended attributes, so ~/Downloads is a great place to experiment with xattr.

As an example I downloaded the latest version of the XKpasswd library from my website (xkpasswd-v0.2.1.zip). I can now use xattr to see all the extended attributes OS X added to that file like so:

You can see that OS X has added three extended attributes to the file, com.apple.metadata:kMDItemDownloadedDate, com.apple.metadata:kMDItemWhereFroms and com.apple.quarantine.

All three of these attributes are base 64 encoded HEX. The HEX representation of the data looks meaningless to us humans of course, but OS X understands what it all means, and the xattr command is nice enough to display the ASCII next to the HEX for us. In the case of the download date, it’s encoded in such a way that even the ASCII representation of the data is of no use to us, but we can read the URL from the second extended attribute, and we can see that Safari didn’t just save the URL of the file (http://www.bartbusschots.ie/downloads/xkpasswd-v0.2.1.zip), but also the URL of the page we were on when we clicked to download the file (http://www.bartbusschots.ie/blog/?page_id=2137). Finally, the quarantine information is mostly meaningless to humans, except that we can clearly see that the file was downloaded by Safari.

The xattr command can also be used to add, edit, or remove extended attributes from a file, but we won’t be going into that here.

Wrapup

That’s where we’ll leave things for this instalment. Hopefully you can now read all the metadata and security permissions associated with files and folders in OS X, and you can alter the Unix file permissions on files and folders.

We’re almost covered all the basics when it comes to dealing with files in the Terminal now. We’ll finish up with files next time when we look at how to copy, move, delete, and create files from the Terminal.