                      _         _     _
                     |_) _ |_  /  |_   \ |   _  _
                     | \(_)(_, \_ (_, _/ (_,(_)(_)
                     By Stefano Falsetto       ,_)

This is Rot[t]Log

archive, rotates, compresses, and mails system logs.

This is a replacement to Red Hat's logrotate. It have similar
syntax, but more powerful features to cut and store logs. 
This README is heavily bases on Red Hat's logrotate man page.
It's all written in BASH (2.x compatible).

This script require GNU fileutils, GNU grep and util-linux.

USE WITH CARE
At this moment this software is still in alpha stage, so... handle with care 
your logs...

Main configuration file	  : /etc/rottlog/rc
Other configuration files : /etc/rottlog/[daily|weekly|monthly]

rottlog first read main configuration file, then read (in order)
daily, weekly and monthly configuration files.

  ::::: RC FILE SYNTAX :::::

  This is the main configuration file. It defines default behavoir of
  Rot[t]Log. 
  Following keywords are valid:
  
  compress=<options>
  createdir
  dir_grp=<group>
  dir_own=<owner>
  dir_perm=<permissions>
  extension=<ext>
  follow_symlinks
  fromuser=<sender address>
  ifempty
  mail=<mailer prg>
  maxdepth=<depth>
  missingok
  nocompress
  nocreate
  nomail
  nomissingok
  notifempty
  packdir=<path-to-archive-dir>
  packer=<absolute path> 
  pager=<pager-program>
  remove_missing
  touser=<receiver address>
  uncompress=<options>
  unpacker=<complete path>

  compress=<options>
  options to be used with packer program to compress data
  IMPORTANT: Packer program must output data to stdout.
  (i.e. for gzip/bzip2 use compress="bzip2 -c")
  
  createdir
  Create dirs specified in storedir if needed. It's useful to use
  with meta-vars. See section USE OF META-VARIABLES for more info.
  
  dir_grp=<group>
  Default group of newly created directory. Must exists
  in /etc/group file.
  
  dir_own=<owner>
  Default owner of newly created directory. Must exists 
  in /etc/passwd file.
  
  dir_perm=<permissions>
  Default permissions of newly created directory. Only numerical
  notation is permitted.
  
  fil_grp=<group>
  Default group of newly created logfile. Must exists
  in /etc/group file.
  
  fil_own=<owner>
  Default owner of newly created logfile. Must exists 
  in /etc/passwd file.
  
  fil_perm=<permissions>
  Default permissions of newly created logfile. Only numerical
  notation is permitted.

  extension=<ext>
  Extension normally used when compressing data with packer.
  (i.e. for gzip is gz, for bzip2 is bz2, ...)
  This variable is used to expand meta-variable @COMP_EXT.

  follow_symlinks
  Follow symlinks to logfiles. Default behavoir is to not to follow
  symlinks for security reasons.
  
  fromuser=<sender address>
  Sender address used in mail messages.

  ifempty
  Rotate the log file even if it is empty.
  
  mail=<mailer prg>
  Mail program to send messages. Must be used option to read header
  recipient addresses.  
  Example:
   - for sendmail use: mail="sendmail -t"
   - for qmail use: mail="qmail-inject -h"

  maxdepth=<depth>
  Default maximum depth to search for files when used * metachar in log
  filename definition.
  
  missingok
  If the log file is missing, go on to the next one without issuing an
  error message. Lastaction (if defined) will be performed.  Postrotate
  script (if defined) will NOT be performed.

  nocompress
  Don't compress logfiles to be archived/rotated.
  
  nocreate
  Don't create a new filename. Can't be used with logpart option.
  
  nomail
  Don't mail rottlog's activity report.
  
  nomissingok
  rottlog will return an error message and stop execution 
  if the log file is missing.

  notifempty
  Do not rotate a logfile if it's empty.

  packdir=<path-to-archive-dir>
  default basedir to store rotated/archived files.
  This variable is used to expand meta-variable @DEF_PATH.

  packer=<absolute path> 
  complete path to packer program to be used to compress logs.

  pager=<pager-program>
  Path to program to use with --showlog parameter. It's used in a pipe,
  so must be a program able to read input from stdin.
  
  remove_missing
  rottlog will stop execution if a delayed file is no longer existant
  or readable.

  touser=<receiver address>
  Receiver of mail messages.
 
  uncompress=<options>
  options to be used with unpacker program to uncompress data
  IMPORTANT: Unpacker program must output data to stdout.
  (i.e. for gunzip/bunzip2 use -c)

  unpacker=<complete path>
  complete path to program to be used to uncompress logs.
  (i.e. gunzip, bunzip2...)


  ::::: DAILY|WEEKLY|MONTHLY FILE SYNTAX :::::

  Period config files are checked once per month, week and day, 
  and all logfiles are handled at the same time.
  Example: 
   - rottlog is run on 1st of a month 
     1. Logfiles in 'monthly' will all be handled on 1st of 
        each following month
     2. Logfiles in 'weekly' will all be handled on 1st, 7th, 14th,
        21th, 28th day and so on
     3. Logfiles in 'daily' will all be handled each day in the 
        month

  So if a logfile must be rotated/archived only one day of each
  month it must be inserted in *daily* config file and must be 
  used "forceday" parameter.

  Global structure of a period-config file is:

  out-of-block-definitions

  configuration-block-1-start {
    block-contents
  }

  configuration-block-2-start {
    block-contents
  }

  ....

  Comments can be inserted in rows beginning with '#'.
  Comments on same row of valid items are not allowed.
  Example:

  # This is a valid comment
  logpart "#1 day" # This is an INVALID comment

  
  There is only one out-of-block-definition:

  include <path-to-conffiles>[,<path-to-conffiles>,...]

  This option permits to include other files at the end of configuration
  file. Nested includes are not supported: only one level of inclusion
  is supported. 
  This keyword must be inserted at beginning of configuration file before
  and out of all configuration blocks. If used in other positions will
  be treated as error.
  <path-to-conffiles> must be an absolute path to a file or a group of
  files (using * wildcard). To include all files in a directory must be
  used path-to-dir/* (i.e. /etc/rottlog/conf-monthly/*)

  A configuration block for logfiles has following structure:

  <logfile1>,<logfile2>,... {
    <append-only>
    <collate> <logfile>
    <create> <mode> <owner> <group>
    <createdir> [<perms>] [<own> <grp>]
    <delaycompress>
    <dateoffset>
    <firstaction> ... <endaction>
    <forceday>
    <ifempty>
    <lastaction> ... <endaction>
    <logpart> <qdldl>
    <maxdepth> <depth>
    <missingok>
    <nocompress>
    <nocreate>
    <nomail>
    <nostoredir>
    <notifempty>
    <postrotate> ... <endscript>
    <prerotate> ... <endscript>
    <rotate> <number>
    <sharedscripts>
    <size> <num[b,k,M]>
    <storedir> <path-to-dir>
    <storefile> <filename>
    <tarcollate> <logfile>
    <touser> <email address>
  }

  Where:
  
  logfile1, logfile2, ... 
  Is a comma separated list of complete absolute path to logfiles to
  archive/rotate. From 0.10alpha+ version it's possible to use ONLY
  metachar asterisk. (i.e. /var/adm/log.* means that definitions between
  curly brackets are applied to all files in /var/adm dir beginning by
  'log.').  If logfile is a filename containing special chars like
  ']' or '{' you must protect it from expansion with DOUBLE slash
  (i.e. apache\\[1\\].log). If same logfile is used more than one time
  in same configuration file, will be used only the first.

  append-only
  To be used only on ext[23] fs. 
  Set append-only attribute to a log (More precisely: remove this
  attribute before handle log, and reset it after performed all
  operations).

  collate
  Store more than one logfile in only one file compressed.
  This option makes possible to weekly rotate a logfile (i.e. in a little
  volume) and monthly store a compressed file containing previously rotated
  files (i.e. in a bigger backup volume). 
  If used in monthly configuration file, <logfile> will be searched in weekly
  configuration file.
  If used in weekly configuration file, <logfile> will be searched in daily
  configuration file.
  Can't be used in daily configuration file.
  Logfiles referenced in collate option must use rotate option with correct
  periods:
  - collate monthly a weekly rotated file : rotate 4
  - collate weekly a daily rotated file   : rotate 7
  Example:
  Collect monthly a weekly rotated log.daemon
 
  In weekly configuration file:
  /var/adm/log.daemon {
    storedir /mnt/tiny/@YEAR/@BASENAME
    createdir 4700 stefano root
    create 600 stefano root 
    prerotate
        /sbin/killall -HUP syslogd
    endscript
    # Important!
    rotate 4
    delaycompress
  }

  In monthly configuration file:
  /var/adm/log.daemon {
    storedir /mnt/BigVolume/Collections/@YEAR/@BASENAME
    createdir
    collate
  }

  In this example in /mnt/BigVolume/Collections/2002/log.daemon will
  be stored a file called log.daemon.<number> containing log.daemon.1,
  log.daemon.2, log.daemon.3 and log.daemon.4 weekly created in
  /mnt/tiny/2002/log.daemon dir.

  This option is compatible only with following options:
    - [no]storedir
    - storefile
    - createdir
    - [no]missingok (TODO)
    - firstaction/lastaction
    - prerotate/postrotate
    - nocompress
    - touser
    - nomail
  Options ifempty/notifempty are silently ignored.

  create [[<mode>] [<owner> <group>]]
  Immediately  after  rotation (before the postrotate script is run)
  the log file is  created  (with  the same  name  as  the  log  file
  just rotated).  mode specifies the mode for the log file in  octal (the
  same  as  chmod(2)),  owner specifies the user name who will own the
  log file, and group specifies  the group  the  log file will belong to.
  If none of three parameters is specified new file will be created
  with permissions and owner defined in default variables fil_perm,
  fil_own, and fil_grp. You can specify mode only, or mode and owner,
  but you can't to specify mode and group (second parameter is always
  interpreted as owner).

  createdir [[<mode>] [<owner> <group>]] 
  Create (if needed) dir to store archived/rotated logs with specified
  permissions and owner:group.  Directory will be created with permissions
  and owner defined in default variables dir_perm, dir_own, and dir_grp if
  none of three parameters is used. You can specify mode only, or mode
  and owner, but you can't to specify owner and group (second parameter
  is always interpreted as owner).  perms specifies the mode for the
  log file in octal (the same as chmod(2)).

  IMPORTANT:
  If path to be created is deeper than one dir, only last dir will have
  owner, group and mode specified. 
  Example:
  
  storedir 2002/04/apache_logs
  createdir 4300 rottlog

  If dir 2002 (and following) are not already present on your box after
  rottlog run you will have (if rottlog is run by user stefano):
  $ ls -ld 2002 2002/04 2002/04/apache_logs
  drwxr-xr-x   2 stefano  users        1024 Apr 11 18:35 2002/
  drwxr-xr-x   2 stefano  users        1024 Apr 11 18:35 2002/04
  d-ws------   2 rottlog  users        1024 Apr 11 18:35 2002/04/apache_logs

  delaycompress
  Postpone compression of the previous log file to the next rotation
  cycle.
  It can be used when some program can not be told to close its
  logfile and thus might continue writing to the previous log file for
  some time.

  dateoffset <+/-day>
  Force time related metavariables to be calculed with specified days of offset.
  Example:
  if today is Aug 5 and is used "dateoffset -1", @DAY metavariable will be 04 
  and not 05.

  firstaction <EXIT_CODE>... endaction
  Unique line between firstaction and endaction (both of which must
  appear on lines by themselves) are executed before ANY operation. It
  will be performed during parameter parsing, before filename and metavars
  expansion.  Optional parameter EXIT_CODE is default exit code expected
  by action. If not specified will expect 0.
  If used in multiple files definition action will be run only one 
  time, before processing first file.

  forceday <day>
  Force daily rotation/archiviation to be performed only at the specified day
  of the month. 
  This keyword is only valid in daily configuration file.

  ifempty
  Rotate the log file even if it is empty.
  
  lastaction <EXIT_CODE>... endaction
  Unique line between lastaction and endaction (both of which must
  appear on lines by themselves) are executed after ALL operations. It
  will be performed just before handle next logfile.  Optional parameter
  EXIT_CODE is default exit code expected by action.  If not specified
  will expect 0.
  If used in multiple files definition action will be run only one
  time, after processed all files.

  logpart <qdldl>
  Leave a part of log to archive/rotate in new logfile.
  qdldl can be:
  <number>l
  <number>b
  "<regexp>"
  "<#1 day>"
  Where l and b means lines and bytes respectively.
  "regexp" is a regular expression used by grep to find first row to
  leave in log.
  "#1 day" is a shortcut to leave in log all events recorded from first
  day of current month.

  maxdepth <depth>
  Default maximum depth to search for files when used * metachar in log
  filename definition.

  missingok
  If the log file is missing, go on to the next one without issuing an
  error message. Default behavoir is to stop executing and return an
  error exit code.

  nocompress
  Old versions of log files are not compressed with packer defined in
  rc file.

  nocreate
  Don't create a new filename. Can't be used with logpart.

  nomail
  Don't send notification e-mail message.

  nostoredir
  Logs are rotated/archived in the same directory the log normally
  resides in.

  notifempty
  Do not rotate the log if it is empty.

  postrotate ... endscript
  Lines between postrotate and endscript (both of which must
  appear on lines by themselves) are executed after the log file is
  archived/rotated.  Can be used metavariables. See section USE OF
  META-VARIABLES for more info.

  prerotate ... endscript
  Lines between prerotate and endscript (both of which must appear
  on lines by themselves) are executed before the log file is
  archived/rotated.  Can be used metavariables. See section USE OF
  META-VARIABLES for more info.
  
  rotate <number>
  Log files are rotated <number> times before being overwrited.

  sharedscripts
  Let postrotate script to be run only one time, after handled last file in 
  a multiple file definition block.
  Example:
  /var/adm/log.* {
    sharedscripts
    postrotate
      killall -HUP syslogd
    endscript
  }
  The killall command will be run only after handled all log files beginning 
  by "log" in /var/adm dir.

  size <num[b,k,M]>
  Rotate a logfile only if it grows specified dimension.
  Parameter must be a number followed by one of following chars:
  b - means bytes
  k - means kbytes (num * 1024)
  M - means megabytes (num * 1000000)

  If none of these three chars is used, b is default.

  storedir <path-to-dir>
  If it's a relative path it will be appended to packdir defined in rc
  file, if it's an absolute path it will be used instead of packdir. Can
  be used metavariables. See section USE OF META-VARIABLES for more info.

  storefile <filename>
  Name of compressed logfile to be used (whitout any path: filename
  only) Can be used metavariables. See section USE OF META-VARIABLES
  for more info.

  tarcollate
  Same of collect, but store files in a compressed tar file. In addition this
  option is compatible with ifempty option too. If set, a void file will be
  archived in tar file. If not set it will be skipped.
  Tar archive will contain relative path if possible (if archived files are in
  subdirs of packdir defined in main rc file) or absolute path.
  
  touser <email address>
  Override global value of touser variable defined in rc file.

  ::::: USE OF META-VARIABLES :::::

  In monthly/weekly/daily logs can be used metavariables.
  Metavariables are identifiers of a runtime defined value.
  Handled metavariables:

  @DIRNAME  : Dir containing logfile to be archived/rotated
  @FILENAME : filename with numerical extension (BASENAME+NEXT_EXT)
  @BASENAME : filename without extension
  @DEF_PATH : default path to archive dir
  @COMP_EXT : extension of compressed format file (bz2, gz, ...)
  @NEXT_EXT : next available numbered extension 
  @YEAR     : actual year(*)
  @DAY      : actual day(*)
  @MONTH    : actual month(*)
  @TEMPDIR  : a temporary directory 
  @1..@n    : string corresponding to first, second and so on directory
              in path of file to archive/rotate

 (*) Can't be used in storefile if using rotate parameter too

  Parameters handling metavariables:
  storedir
  storefile
  prerotate...endscript
  postrotate...endscript

  Values of metavariables are defined while script is running, so a definition
  like this:

  /var/adm/log.* {
    storedir /var/adm/archive-log/@BASENAME
    rotate 5
    ...
  }

  Means that I wants to rotate 5 times all files in /var/adm beginning by 
  "log." and store each file in a dir named with logname.
  (i.e. /var/adm/archive-log/log.kern, /var/adm/archive-log/log.daemon, ...)

  WARNING:
  Metavariables substitution is merely textual, so in a definition like:

  /var/http/www.mysite.com/log/access.log {
    storefile @3-@13-@FILENAME
    rotate 4
    ....
  }
  Will create logfiles called somethink like:
  www.mysite.com-var3-access.log.1
  
  @13 is not defined (path in first line of block definition contains only 5 tokens)
  so @1 will be substituted and following text will not be affected.

  ::::: COMMAND LINE PARAMETERS :::::

  Valid command line parameters allowed are:
  
    --help, -h
    Display help page and exit. Does nothing else.
    
    --version, -V
    Display version, disclamier and exit. Does nothing else.
    
    --defaults
    Display defaults values and exit. Useful to know default behavoir of
    rottlog
    
    --showlog <logfile>, -s <logfile>
    Show contents of archived <logfile> with default pager. <logfile> can be
    only a filename without any path or an absolute or relative path. If
    <logfile> is a relative path to a file, will be searched in $packdir
    dir.
    
    --pager <program>
    Use <program> for paging output of showlog. Useful only with --showlog
    
    --summary <logfile>
    Display a summary of some useful information about <logfile>. Does not
    exec any rotate/archive action. It's something like a --checkrc 
    but more verbose.
    
    --checkrc <what>,<what>,...
    Check for syntax of specified configuration file(s)
    <what> must be one of: daily, weekly, monthly, all

    --force [<daily>] [<weekly>] [<monthly>]

    Force rotation/archiviation of logfiles , even  if  it
    doesn't think this is necessary.  Sometimes this is useful
    after adding new entries to some config file, or if old log files
    have been removed by hand, as the new files will be created,
    and  logging  will  continue correctly.
    If none of daily,weekly and monthly is used, will be forced
    rotation/archiviation of all entries in all config files.

