greynode

I was pondering the unix command line the other day while reading sed & awk, and I came to the following revelation: variables aside, the unix command line is a lot like functional programming. Using the command line, you can build a functional program by redirecting I/O between various programs, or functions. Many of the programs one uses to build one-liners are essentially pure functions. An example of a pure function on the unix command line might be the programs uniq or sort. Various other functions such as ps and wget are not so pure, as their output relies on I/O with the operating sytem or server.

All theory aside, here are a few good one-liners.

Uploading Files

tar cjvf - yourDirectory | ssh uname@host "cat > yourDirectory.tar.bz2"

This command will create an archive of “yourDirectory” and upload it to your server via SSH, all in one chained command. Quite handy for uploading local content for sharing or backup.

Edit: Apparently this command transfers large files extremely slow. Oops.

Checking FreeBSD UPDATING File Automatically

portversion | awk '/</ {print $1}' |
xargs -I '{}' awk '/AFFECTS:.*{}/ {print}' /usr/ports/UPDATING > updates.txt

Ok. So this command pertains to the FreeBSD ports system. It is not recommended that you upgrade all your ports at once. There is often information contained in the file /usr/ports/UPDATING about special instructions you may have to follow while upgrading a package. Often times you may have to recompile other packages, or add a line to a configuration file, etc. The string above will check all of the ports tht are out of date, and compare them to the UPDATING file to see if the package is contained. If it is, it writes the name of that port to updates.txt. You can then use this file to know which ports have special instructions. You should probably write an alias for this one.

Converting Line Endings

The following two commands should help with reformatting files. Often times I find myself getting web designs from someone who uses windows. Most editors on windows will save files with DOS line endings. When you open one of these files in unix (in certain editors), the line endings won’t appear properly. The following commands will circumvent this problem, in either situation.

Convert from unix (\n) to DOS (\r\n)

awk '{sub(/$/, "\r")};1' unix_endings.txt > dos_endings.txt

Convert from DOS (\r\n) to unix (\n)

awk '{sub(/\r$/, "")};1' dos_endings.txt > unix_endings.txt

And yes, I realize that I use too many commas when I write: I’m working on it.

If you install XCode for OS X 10.5, it automatically comes with a version of rails. This is (generally) not the latest version however, and you must take a few steps to update it. First you need to update rubygems with the following:

macintosh$ sudo gem install rubygems-update

macintosh$ sudo update_rubygems

After updating rubygems, you should be able to update the rest of your gems without a problem (including rails) via the following:

macintosh$ sudo gem update

After that you should be ready to build a rails app.

I just thought I’d share with you Random thoughts on scripting languages by Brian Kernighan.

After reading this, I looked up the source code to the unix utility wc. The GNU wc program was written in C, and amounted to about 800 lines with comments. A similar program written in awk is a mere 2 lines long:

{ nc += length($0) + 1; nw += NF }
END { print NR, "lines", nw, "words", nc, "characters" }

I know you are probably thinking that I just made an unfair comparison. Yes, I did.

I have recently been continuing my journey with Haskell, the purely functional programming language. I had dabbled with it a little bit over the winter, but haven’t had much practice since then. It is my hope that over the summer I will be able to get a fair amount of practice with the language.

I have settled on using the book Real World Haskell as my first stepping stone. It has gotten good reviews, and was recommended to me by one of my professors. There is a free version of it online, if you wish to check it out. It uses quite a bit of comparison with imperative languages, at least in the early chapters, so I don’t recommend it to someone who has never programmed before.

For those of you who are not familiar with functional programming, I am going to present to you a few problems and the elegant solutions provided by Haskell. The first I will show you is an implementation of quicksort.

qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++ qsort (filter (>= x) xs)

This implementation of quicksort is so clean it makes me sick. It is essentially the definition of the quicksort algorithm. Because of the facilities provided by Haskell, it translates quite nicely.

Another problem I’d like to show you involves counting. Say you had a string of words, and you would like to count the number of words that start with an upper-case letter. In an imperative language, this might involve several loops and/or the use of regular expressions. With Haskell, its a simple matter of function composition. Have a look:

capCount = length . filter (isUpper . head) . words

This might look odd if you are not familiar with functional programming or function composition, but it is a really simple function. The ‘.’ operator simply takes the output from the function on the right, and feeds it into the function on the left. The ‘words’ function breaks a string up into a list of words wherever there is a space. The ‘head’ function returns the first element of the list, ‘isUpper’ will return a boolean value saying whether or not a Character is upper-case, ‘filter’ applies a function over a list and removes items that evaluate to False, and ‘length’ returns the length of the list. As you can see, it’s much easier to describe with code than with words.

Anyway, I will be posting anything I find interesting on here. Hopefully someone other than myself will benefit from it.