Log File UNIX Commands

One of the biggest parts of my job is managing website log files. Our business is very “marketing-centric”, so missing log file data isn’t expectable. There have been a few situations were log files have been split or broken. Below are three nice little commands which have helped tremendously.

Removes all log files with no data
find /home/httpd/logs -size 0 -type f -print0 | xargs -0 rm –f
This is a simple little command which helps keep things neat and organized. Once files rotate I’ve found a bunch of empty files on the folder. This really confuses some of the marketing people.

Converts log files extention into logical month.
for i in `dir /home/httpd/logs/*-access_log.1|sed s/\.1$//`; do mv $i.1 $i.feb; done
A little more complicated. Again, the overall goal here is to make the log files more friendly for marketing people connecting into the FTP server. I am converting the *.1 extension file created by the log rotate into *.month. I use this single line to change the extension on 300+ files. Works fantastic!

Combines log files with extentions .1.1 -> .1 and creates a new logical month.
for i in `dir /home/httpd/logs/*-access_log.1|sed s/\.1$//`; do cat $i.1.1 $i.1 >$i.mar; done
Here is the big daddy. One month because of a mistake the log files rotated mid-month. This would have been a nightmare to manually concatenate each file. This script above takes the files and combines them. Try to do this in windows!

For the most part I’ve learned to hate access logs. They are a big, always changes, difficult to backup, and from a technical perspective somewhat useless. These little few commands above however make life a little easier.