Home page Contact information List of services provided Automation details Perl, Shell, IDL code examples Professional and personal experience Weblog Charities We Support Miscellaneous Info and Links

Using log-splitter and Analog

I made a comment on Tenon's iTools mailing list about having Analog running on my system and would help out setting up someone else's system if they needed it. After a quick volley of incoming emails, I thought it might save me some time to put up the info on a web page. Please let me know if I've left out something or butchered an explanation along the way.

Mike Schienle

log-splitter

Tenon's log-splitter.pl program is nicely suited for use with Analog. Be sure to read Tenon's Readme file and configure the server.txt file, also. I have made only two necessary changes and one convenience change to log-splitter.pl. The necessary changes to log-splitter.pl are commenting out the two lines that look like this:

system "$GZIP $to_WebTen_logs/archive/$the_time\_$log_file";

so they do not compress the log file. The convenience change is just a matter of reordering the date format of the split file so it will sort the same either alphabetically or chronologically. Take your pick between easier readability and easier sorting. I changed Tenon's date format from:

$date_fmt = "%m-%d-%y";

to this:

$date_fmt = "%y-%m-%d";

Analog

Analog compiled out of the box for me. I have a binary of it available if you are not interested in compiling it. It contains the $site.cfg files for the virtual domains I run, as well as dnsfile.txt, which is the dnslookup table that analog populates if it is asked. I specified $site as a variable for a reason, which I'll get to very quickly. Having said that, feel free to grab the Analog binary distribution for MacOS X Server 1.2.

I installed the distribution as /Local/Public/analog-4.11. There are a couple path dependencies, so this may have to be the place you use if you grab the binary. Be sure to copy/link the images from analog's image directory to the images directory of your web site(s) or your graphics will be broken.

Analog can be called from the command line with many options. I use a very brief command set for my needs:

analog -G +g$site.cfg

The -G tells analog not to read the default configuration file. If $site has "ivsoftware" assigned to it, then +g$site.cfg tells analog to use the configuration file ivsoftware.cfg for its processing needs.

I use a very simple shell script to call analog for each of my virtual domains. It just loops through the specified domain names and calls analog with the above options. The script is installed in /Local/Users/Administrator/Applications and is named analogSite.sh.

#!/bin/sh
# change to the analog directory
cd /Local/Public/analog-4.11
# loop through the domains
for site in ivsoftware windastro
do
    # call analog with each domain's config file
    ./analog -G +g$site.cfg
done

ivsoftware.cfg was copied from analog.cfg which is in the normal distribution. This isn't nearly as bad as it looks. Follow along with analog.cfg to see what has been changed. Below is the meat of ivsoftware.cfg. You only need to make a few changes at the top and bottom of the config file.

# If you need a LOGFORMAT command it must go here, above the LOGFILE command.
# use any log file matching the pattern
LOGFILE /Local/Library/WebServer/WebSites/www.customvisuals.com/logs/*.log
# write the resulting output to this file
OUTFILE /Local/Library/WebServer/WebSites/www.customvisuals.com/webstats.html
HOSTNAME "Custom Visuals"
# REQINCLUDE pages
LINKINCLUDE pages
REFLINKINCLUDE *
# don't include entries from these domains
# these are places where I work, so I'm trying to ignore my accesses
ORGEXCLUDE windastro.com  
ORGEXCLUDE customvisuals.com  
ORGEXCLUDE pilot.net
# [ removed a few related commands ]
DOMEXCLUDE http://www.windastro.com/*
DOMEXCLUDE http://www.customvisuals.com/* 
UNCOMPRESS *.gz,*.Z "gzip -cd"
# [ identical to analog.cfg from here to last TYPEOUTPUTALIAS]
TYPEOUTPUTALIAS .asp     ".asp  [Active Server Pages]"

# [ items not found in analog.cfg ]
#	report flags
FULLDAILY ON
DAILY OFF
HOURLY OFF
FULLHOURLY ON
DOMAIN ON
REFERRER ON
REFSITE ON
# DNS flags
# write the DNS data to a file
DNS WRITE
# specify the DNS file
DNSFILE /Local/Public/analog-4.11/dnsfile.txt
# how long to keep a good DNS entry (4 weeks)
DNSGOODHOURS 672
# how long before we look up an unresolved DNS entry again (1 week)
DNSBADHOURS 168

Cron

The last step is to have cron automate the tasks. These are the cron entries for root on my system. Tenon specifies the top one with "root perl" before the command, but I've taken it out for some reason that I don't recall anymore. The log-splitter.pl command runs every night at 11:59 pm. The log-splitter.pl program only takes 10-15 seconds to run on my system. The analogSite.sh runs at 10 minutes past midnight.

59 23 * * * /Local/Library/WebServer/log_splitter/log_splitter.pl
10 00 * * * /Local/Users/Administrator/Applications/analogSite.sh

Home Contact Services Automation Code Background Weblog Charity Other

analog_splitterXS.shtml was last updated on 07/05/08