greynode

Thought I would share an interesting article on why there are no games on linux. Want more games on linux? Solve these problems!

Recently the rsync server at work broke down, and I needed a quick solution for incremental backup using just FTP. As it turns out, the FreeBSD version of tar is not GNU tar. So in order to use the following trick on FreeBSD, the gtar port must be installed. Once you have the software, making an increment backup with tar is easy:

$ gtar -g /backup/tar-incremental.log -cjvf /backup/backup.tar.bz2 /etc /usr/local/etc/ /usr/www

The tar-incremental.log file must be saved as it contains information for the incremental backup. Every time you run this file a new file will be created, so do not over-write your files. A more in-depth discussion can be found on the GNU tar incremental backup page.

If you run arch linux or a similar minimal distro, you may be familiar with the local dns server pdnsd, used to cache DNS queries. If you try to run pdnsd on the same computer as NetworkManager, you may run into a problem. Using default configurations, your DNS lookups end up in a vicious loop redirecting to 127.0.0.1. To circumvent this problem, I did the following.

I let NetworkManager generate my /etc/resolv.conf file. I then copied it to /etc/resolv.conf.head. Once this was done, the server section in /etc/pdnsd.conf was edited to contain the following:

server {
label= "comcast;
file = "/etc/resolv.conf.head"; # Preferably do not use /etc/resolv.conf
# proxy_only=on; # Do not query any name servers beside your ISP's.
timeout=4; # Server timeout; this may be much shorter
# that the global timeout option.
uptest=if; # Test if the network interface is active.
interface=wlan0; # The name of the interface to check.
interval=10m; # Check every 10 minutes.
purge_cache=off; # Keep stale cache entries in case the ISP's
# DNS servers go offline.
}

Notice the file variable which tells pdnsd where to find the nameservers it will query. Once this is complete, the pdnsd daemon can be restarted. Now, NetworkManager must be told to use localhost for DNS lookups. This can be done by right-clicking the nm-applet icon and configuring the wireless for “Automatic (DHCP) Address Only”, and enter 127.0.0.1 as the DNS server. Once this is complete, you can test it with the following commands:

jaigner /var/cache $ dig kernel.org | grep Query
;; Query time: 75 msec
jaigner /var/cache $ dig kernel.org | grep Query
;; Query time: 0 msec


The second time the command was called, the cached result was fetched. If your Query time is greater than 1ms, it is likely that something is misconfigured.

I was pondering the unix command line the other day while reading sed & awk, and I came to the following revelation: variables aside, the unix command line is a lot like functional programming. Using the command line, you can build a functional program by redirecting I/O between various programs, or functions. Many of the programs one uses to build one-liners are essentially pure functions. An example of a pure function on the unix command line might be the programs uniq or sort. Various other functions such as ps and wget are not so pure, as their output relies on I/O with the operating sytem or server.

All theory aside, here are a few good one-liners.

Uploading Files

tar cjvf - yourDirectory | ssh uname@host "cat > yourDirectory.tar.bz2"

This command will create an archive of “yourDirectory” and upload it to your server via SSH, all in one chained command. Quite handy for uploading local content for sharing or backup.

Edit: Apparently this command transfers large files extremely slow. Oops.

Checking FreeBSD UPDATING File Automatically

portversion | awk '/</ {print $1}' |
xargs -I '{}' awk '/AFFECTS:.*{}/ {print}' /usr/ports/UPDATING > updates.txt

Ok. So this command pertains to the FreeBSD ports system. It is not recommended that you upgrade all your ports at once. There is often information contained in the file /usr/ports/UPDATING about special instructions you may have to follow while upgrading a package. Often times you may have to recompile other packages, or add a line to a configuration file, etc. The string above will check all of the ports tht are out of date, and compare them to the UPDATING file to see if the package is contained. If it is, it writes the name of that port to updates.txt. You can then use this file to know which ports have special instructions. You should probably write an alias for this one.

Converting Line Endings

The following two commands should help with reformatting files. Often times I find myself getting web designs from someone who uses windows. Most editors on windows will save files with DOS line endings. When you open one of these files in unix (in certain editors), the line endings won’t appear properly. The following commands will circumvent this problem, in either situation.

Convert from unix (\n) to DOS (\r\n)

awk '{sub(/$/, "\r")};1' unix_endings.txt > dos_endings.txt

Convert from DOS (\r\n) to unix (\n)

awk '{sub(/\r$/, "")};1' dos_endings.txt > unix_endings.txt

And yes, I realize that I use too many commas when I write: I’m working on it.

I just thought I’d share with you Random thoughts on scripting languages by Brian Kernighan.

After reading this, I looked up the source code to the unix utility wc. The GNU wc program was written in C, and amounted to about 800 lines with comments. A similar program written in awk is a mere 2 lines long:

{ nc += length($0) + 1; nw += NF }
END { print NR, "lines", nw, "words", nc, "characters" }

I know you are probably thinking that I just made an unfair comparison. Yes, I did.