Fixed width data
How many households had just 1 person? Referring to the file layout, we see
that the 106th and 107th characters of a household record indicate the number
of people in the household. We can use the
cut command to pull out just that
bit of data from each record. The argument
cut to print
the 106th through 107th characters of each line. The
head command prints just
the first few lines of a file (or its standard input).
You can give
cut a comma separated list to pull out multiple ranges. To see
the household id along with the number of occupants of the household:
-c argument is used for working with so called "fixed-width" data. Data
where the columns of a record are found at certain offset in bytes from the
beginning of a record. Fixed width data abounds on a Unix system.
writes its output in a fixed width format:
Returning to the question of how many 1 person households are there in Washington:
7,192, or about 28% of households have only one occupant.
In delimited data, elements of a record are separated by a special delimiter character. In the password file, fields are delimited by colons:
The 7th column of the password file is the user's login shell. How many people use bash as their shell?
You can give either
-f a comma separated list, so to see a few users
tcsh as their shell:
The space character is a common delimiter. Unfortunately, your shell probably
discards all extra whitespace on the command line. You can sneak a space
character past your shell by wrapping it in quotes, like:
cut -d" " -f 5.
The tab character is another common delimiter. It can be hard to spot, because
on the screen it just looks like any other white space. The
od (octal dump)
command can give you insight into the precise formatting of a file. For
instance I have a file which maps first names to genders (with 95%
probability). When casually inspected, it looks like fixed width data:
But on closer inspection there are tab characters delimiting the columns:
The first thing to do is read your system's manpage on
cut: it may already
delimit by tab by default. If not, it requires a bit of trickery to get a tab
character past your shell to the
cut command. First, many shells have a
feature called tab completion; when you hit tab they don't actually insert a
tab, instead they attempt to figure out which file, directory or command you
want and type that instead. In many shells you can overcome this special
functionality by typing a
control-v first. Whatever character you type after
control-v is literally inserted. Like a space character, you need to
protect the tab character with quotes or the shell will discard it like any
other white space separating pieces of the command line.
So to get the ratio of male first names to female first names I might run the
following commands. Between the double quotes I typed
control-v and then hit
Apparently there's much more variation in female names than male names.
If your system's
cut command delimits on tab, the above command becomes
cut -f2 gender.txt.