I was pondering the unix command line the other day while reading sed & awk, and I came to the following revelation: variables aside, the unix command line is a lot like functional programming. Using the command line, you can build a functional program by redirecting I/O between various programs, or functions. Many of the programs one uses to build one-liners are essentially pure functions. An example of a pure function on the unix command line might be the programs uniq or sort. Various other functions such as ps and wget are not so pure, as their output relies on I/O with the operating sytem or server.
All theory aside, here are a few good one-liners.
Uploading Files
tar cjvf - yourDirectory | ssh uname@host "cat > yourDirectory.tar.bz2"
This command will create an archive of “yourDirectory” and upload it to your server via SSH, all in one chained command. Quite handy for uploading local content for sharing or backup.
Edit: Apparently this command transfers large files extremely slow. Oops.
Checking FreeBSD UPDATING File Automatically
portversion | awk '/</ {print $1}' |
xargs -I '{}' awk '/AFFECTS:.*{}/ {print}' /usr/ports/UPDATING > updates.txt
Ok. So this command pertains to the FreeBSD ports system. It is not recommended that you upgrade all your ports at once. There is often information contained in the file /usr/ports/UPDATING about special instructions you may have to follow while upgrading a package. Often times you may have to recompile other packages, or add a line to a configuration file, etc. The string above will check all of the ports tht are out of date, and compare them to the UPDATING file to see if the package is contained. If it is, it writes the name of that port to updates.txt. You can then use this file to know which ports have special instructions. You should probably write an alias for this one.
Converting Line Endings
The following two commands should help with reformatting files. Often times I find myself getting web designs from someone who uses windows. Most editors on windows will save files with DOS line endings. When you open one of these files in unix (in certain editors), the line endings won’t appear properly. The following commands will circumvent this problem, in either situation.
Convert from unix (\n) to DOS (\r\n)
awk '{sub(/$/, "\r")};1' unix_endings.txt > dos_endings.txt
Convert from DOS (\r\n) to unix (\n)
awk '{sub(/\r$/, "")};1' dos_endings.txt > unix_endings.txt
And yes, I realize that I use too many commas when I write: I’m working on it.