PMDF Statistics

PMDF Statistics

Version 3.1

PMDF-Stats is a program to generate usage reports for PMDF based on the information in the MAIL.LOG files. PMDF-Stats runs on OpenVMS VAX, OpenVMS Alpha and OpenVMS IA64.

2010-01-11 The information in this document is subject to change without notice and should not be construed as a commitment by anyone.

The software described in this document is provided as is. No guarantee is made to the suitability, reliability, security, usefulness, performance or good taste of this software.

Availability: PMDF-Stats may be distributed provided no fee is charged, and no alteration is made to the copyright notice in this documentation or in program sources. All rights are reserved.

Restrictions Sources are provided under the understanding that

Copyright ©2000 Tom Wade

Contents

Chapter 1
Introduction

1.1 Purpose of PMDF-Stats

 PMDF-Stats is a management tool to accompany the PMDF mailer product. It analyzes the MAIL.LOG file produced by PMDF and outputs a report of the flow of mail traffic between different domains through your mailer. It can also generate summary reports (including per mailbox statistics) of the number and volume of messages processed, as well as average message transit times.

1.2 PMDF

 PMDF is an electronic mail product for VMS, which utilizes a wide variety of network transports. PMDF is developed and maintained by Process Software. For information on this product, see http://www.process.com.

1.3 Traffic Statistics

 To configure PMDF-Stats to produce traffic statistics, you gather various domains into groups. PMDF-Stats produces traffic statistics, showing the number and volume of messages passing between each of the groups. The groups are specified in the Domain Groups file (see the chapter on Domain Group File for details of how to configure this file). PMDF-Stats will if requested produce totals of messages passing to and from each group.

1.3.1 Traffic Flow Analysis

 PMDF-Stats can generate a traffic flow matrix, showing the number and volume of messages passing into and out of each pair of groups.

1.3.2 Summary Reports

 PMDF-Stats can produce summary reports, showing the total number and volume of messages going into and out of each individual group. Optionally, it can also list active mailboxes (localparts) in each group, showing the individual totals for each user.

1.3.3 Transit Statistics

 PMDF-Stats can produce transit statistics, showing the average time difference between the time a message was enqueued and the time it was dequeued. This statistic is produced on a per group basis.

1.3.4 Fax Reports

 PMDF-Stats can analyze the S records produced by PMDF-Fax, allowing you to see the fax traffic for each group.

1.4 Processing Log Files

 PMDF-Stats may also be used to process MAIL.LOG files produced by PMDF, filtering records based on the values of the various fields. The output files produced by this are in the same format, and can subsequently be subjected to further analysis by PMDF-Stats.

1.5 Log File Format

 The format of MAIL.LOG files may be customized by the LOG_FORMAT option in the PMDF Option File. PMDF-Stats supports either of the following formats: Note that PMDF-Stats does not support FORMAT=3 (counted Ascii strings).

Chapter 2
Installation

2.1 Requirements

 PMDF-Stats 3.1 requires OpenVMS VAX 5.5-2 or later, OpenVMS Alpha 6.1 or later (tested to VMS 7.3-3) or OpenVMS IA64 8.2-1 or later. The disk utilization is about 1000 blocks including documentation and sources. Full sources are provided, along with an automated procedure to compile and relink. A FORTRAN compiler is required if you wish to recompile, but is not necessary for running PMDF-Stats. PMDF-Stats has been tested on log files produced by PMDF versions 4.0 through 6.2.

2.1.1 PMDF system

 You are not required to shut down any PMDF components while performing the installation. The installation will not interact with PMDF in any way.

2.1.2 System without PMDF

 PMDF-Stats may be installed and used on a VMS system without PMDF. PMDF-Stats can process MAIL.LOG files imported from other machines. Note that if PMDF is not present, the use of the /LOCAL qualifier requires an explicit value, as the default cannot be obtained from PMDF (see the section on Local Channel Addresses in the Domain Group File chapter.

2.2 Savesets

 PMDF-Stats comes in three savesets:

Saveset Contents
PMDF-STATS030.A Executable code
PMDF-STATS030.B Program sources
PMDF-STATS030.C Documentation
PMDF-Stats is installed using the VMSINSTAL facility. The same savesets work for both VAX and Alpha.

2.2.1 Getting the saveset from the Internet

If you get the PMDF-Stats saveset from an Internet FTP server, it will most likely be in a ZIP archive (file PMDF-STATS030.ZIP). This format is used because it reduces network traffic by compressing the file, and because the compressed file is 512 byte fixed length records, which corresponds to a BINARY file type (allows any implementation of TCP/IP to retrieve the file without having to fiddle around with file headers afterwards). To decompress the file, you need the UNZIP program (a copy is normally available on the server, and as it is an .EXE file, it can be BINARY transferred as is). The ZIP file can be transferred via non-VMS systems provided binary mode is used. Note that you should only perform the UNZIP operation on a VMS system. 


      $ UNZIP :== $SYS$DISK:[]UNZIP 
      $ UNZIP PMDF-STATS030 
      $

2.3 Invoking VMSINSTAL

Invoke the VMSINSTAL procedure from the SYSTEM account. When asked for the product name, respond 'PMDF-STATS'. The installation procedure will ask if you wish to purge previous versions of files. Answering 'YES' to this prompt will only affect previous versions of PMDF-Stats, and will not cause any components of the PMDF software to be deleted.

2.3.1 Target Directory

 The installation procedure will ask for a target directory for the files. All PMDF-Stats files will be placed in this directory, and no other files on your system will be modified. The target directory defaults to PMDF_ROOT:[STATS]) (note that the PMDF_ROOT logical must be defined, or PMDF must be started for this value to be accepted). It is recommended that PMDF-Stats be placed in a directory solely used for this product. The installation procedure will install VAX or Alpha binaries depending on the target machine's architecture. The final EXE file is linked by the installation process from .OBJ files contained in the kit.

2.3.2 Documentation

 You will be asked if you want the documentation in source form (DOCUMENT), postscript or HTML. These files may be removed after printing.

2.4 Files

 The following files are created:

File Purpose
PMDF-STATS.EXE PMDF-Stats executable file.
DOMAIN_GROUPS.TXT-TEMPLATE Sample Domain Groups file.
BUILD.COM Command procedure to compile & link.
PMDF-STATS-CLD.CLD PMDF-Stats CLD definitions for command input.
PMDF-STATS-CLD.OBJ PMDF-Stats CLD object file.
PMDF-STATS.FOR Fortran source file.
PMDF-STATS.OBJ Object file.
PMDF-ROUTINES.FOR Fortran source file.
PMDF-ROUTINES.OBJ Object file.
REPORT.FOR Fortran source file.
REPORT.OBJ Object file.
MAILBOX.FOR Fortran source file.
MAILBOX.OBJ Object file.
MESSAGEID.FOR Fortran source file.
MESSAGEID.OBJ Object file.
FAX.FOR Fortran source file.
FAX.OBJ Object file.
PMDF-STATS.SDML Source file for documentation (DOCUMENT).
PMDF-STATS.LN3 Documentation in LN03 format.
PMDF-STATS.PS Documentation in Postscript format.
REC_DEFS.INC Include file (source).
STAT_FLAGS.INC Include file (source).
CONSTANTS.INC Include file (source).
MESSAGEID.INC Include file (source).
MAILBOX.INC Include file (source).
PMDF-STATS-MSG.MSG Definitions of error messages.
PMDF-STATS-MSG.OBJ Error messages object file.
OS.MAR Source file for OS interface routines.
OS.OBJ Object file.
PMDF-STATS.SDML Documentation source.
PMDF-STATS.PS Postscript documentation.
PMDF-STATS.HTML HTML documentation.
PMDF-STATS.TXT Plain text documentation.

Only PMDF-STATS.EXE and DOMAIN_GROUPS.TXT are required to run PMDF-Stats.

Chapter 3
Domain Group File

3.1 Format of the Group File

 The Domain Group file consists of group definitions, each starting on a new line. Blank lines are ignored, as are lines starting with the comment character `!'. Lines may be continued by ending the line with a `-' character. The order of group definitions is not significant.

3.2 Definition of a group

 A group is defined within the Domain Group file as follows: 


    Short-Name (Long Name) [flags]: <domain-1>, <domain-2>, .... , <domain-n> 
e.g. 
    ACME (ACME Electronics) [mailbox]: acme.com, acme.net, acme.org

3.2.1 Short Name

The short name is a 1--6 character name which you assign to be the name of the group. It is output on the report, and can be used with the /FROM_GROUP or TO_GROUP qualifiers to filter MAIL.LOG records. It is also used to assign file names to the Site Report for this group. Group names are case insensitive, and you must not define two or more groups with the same name. Group names are limited to Alphanumeric characters.

3.2.2 Long Name

 The long name is optional, and is used in the summary and site reports. If specified, it must be enclosed within parentheses. A long name can consist of any printable characters except parentheses, angle brackets or square brackets. If omitted, the short name is used in place of the long name.

3.2.3 Flags

 Flags consist of one or more keywords separated by commas. Flag keywords are case insensitive, and must be spelt in full. All flag keywords are enclosed within square brackets. The following flag keywords are defined: The use of these keywords is described in the chapters describing the different kinds of reports.

3.2.4 Domains

 The domains are partial rightmost domains from RFC-822 style addresses (do not specify a leading dot --- e.g. `edu' rather than `.edu'). Domain names are case insensitive.

3.3 Matching an address to a group

 When an address of a sender or receiver is processed from the MAIL.LOG file, the domain part of the address is searched in the Group Database (this is loaded by PMDF-Stats from the Domain Groups file when it is activated). The match is a best rather than first match. For example, an address `fred@vax1.company.com' will match `vax1.company.com' if it is present. If not, it will match `company.com', or failing that, `com'. If the top level domain does not match, it will match a catch-all rule `*' if present. If this is not present, then the record will not be considered part of any group, and will not figure in the report.

3.4 Sample File

 The following is a sample Domain Group file, together with a table of possible addresses, and the groups to which they would be assigned (note: this file does not claim to be complete, and serves as a sample only): 


!   This company. 
 
Acme (Acme Chemicals) [mailbox]: acme.ie, acme.com, acme.net, acme.org 
 
!   Sister company 
 
Froboz (Frobozzco) [mailbox]: froboz.com, froboz.net, froboz.org 
 
!  Earn/Bitnet                                 
 
Earn: bitnet, earn 
 
!  North American customers 
 
NthAm (North America) [mailbox]: us, ca, com, net, org, edu, mil, gov 
 
!  European customers 
 
EUR (Europe) [mailbox]: at, be, ch, de, dk, ie, es, fi, fr, il, is, it, nl, - 
   no, pt, se, gr, uk, lu 
 
!   Anything else. 
 
Other: *
Sample Addresses Assigned Groups
fred@acme.ie Acme
bob@vax.froboz.com Froboz
t.wade@vms.eurokom.ie EUR
joe@node.anywhere.com NthAm
tom@sun.bob-jones.edu Other

3.4.1 Special Qualifiers

The following qualifiers may be used in place of a domain entry:

3.4.2 Ignored Group

 If you include a group with the name IGNORE, then any domains matching this will not be included in the statistics, nor will the group be included in any report.

Chapter 4
Producing a Matrix Report

4.1 Report File

 The report file is specified using the /REPORT qualifier. If the qualifier or its value is omitted, no matrix report will be generated.

Note
This is a change of behavior from previous versions. Previously if the qualifier was omitted, a report of name REPORT.TXT was generated, and the /NOREPORT qualifier was required to suppress report generation.

You may specify either one or two files with the /REPORT qualifier. If you specify two files, then the first file will contain the data on the actual number of messages, and the second will contain data on the volume of messages. If you specify one report file, then this file will contain both sets of information. To specify two files, use: 


         /Report=(FILE1.RPT,FILE2.RPT)

4.2 Report Format

There are two supported report formats, TABLE and DATA. The latter format is useful if you wish to pass the output from PMDF-Stats into a program such as a spreadsheet or graphic presentation program. You specify the format using the /FORMAT qualifier. The default is TABLE.

4.2.1 TABLE Format

 TABLE format (the default) presents the data in a human readable form, with a title, group name headings, the table matrix, and a trailer giving the overall total.

4.2.2 DATA format

 This format assumes the data is to be processed further, and therefore does not include any header or trailer information, but only the actual data. The data is laid out as follows: 


 m (1, 1), m (1, 2), ... m (1, n), 
 v (1, 1), v (1, 2), ... v (1, n), 
 ... 
 m (n, 1), m (n, 2), ... m (n, n), 
 v (n, 1), v (n, 2), ... v (n, n),
where m (i, j) is the number of messages passing from group i to group j, and v (i, j) is the volume of messages passing between those groups. If two filenames were supplied to the /REPORT qualifier, then the m records appear in the first file, and the v records in the second. The records are stored as fixed length 8 digit integers, left filled with zeroes, and separated by a comma character.

4.2.3 Title

The first line is the title specified by the /TITLE qualifier. If this qualifier is omitted, the title defaults to "PMDF Traffic Statistics". It is followed by the block size (only if the file contains volume data). The block size is set by the /BLOCK_SIZE qualifier, and defaults to 1024. You should ensure this matches the block size that PMDF used when it generated the log records. The next line indicates the date and time of the earliest and latest record which contributed to the statistics. The qualifier is ignored for DATA format reports.

4.2.4 Traffic Matrix

 The matrix is laid out with originating groups down the left side, and destination groups along the top. For a single report file, two figures are given: If two report files are requested, the first contains only number of messages, and the second contains only the volumes. For example, the entry corresponding to KOM on the left and UL on the top indicates the number and volume of messages passing from the domains in the KOM group to the domains in the UL group.

4.2.5 Formats

 Fields showing number of messages are output as 8 digit integers, and fields showing volumes are output as 8 digit floating point values, specified to two places of decimals. If a value will not fit in either of these fields, it will be converted to an 8 digit exponential format, e.g. 3.456E07, meaning 34560000.

4.3 Totals

 If you specify the /TOTAL qualifier, PMDF-Stats will add an extra column and row containing partial totals. The extra column contains totals for all messages coming from domains in the group defined on that line. The extra row contains totals for all messages going to domains in the group defined in that column. Note that the totals effectively form an extra group, thus reducing by one the number of groups you can define.

4.4 Invoking PMDF-Stats

 To invoke PMDF-Stats you must first define a foreign command to activate the image. 


     $ Pstat == "$PMDF_ROOT:[STATS]PMDF-STATS"
You then invoke PMDF-Stats using this command. 


     $ Pstat MAIL.LOG /report=JUN92.REPORT /Title="Statistics for June 1992"

Next Contents