Actsync(8)

ACTSYNC(8) ACTSYNC(8)

NAME
actsync, actsyncd - synchronize newsgroups

SYNOPSIS
actsync [-b hostid] [-d hostid] [-g max] [-i ignore_file]
[-I hostid] [-k] [-l hostid] [-m] [-n name]
[-o fmt] [-p min_%_unchg] [-q hostid] [-s size]
[-s spool_dir] [-t hostid] [-T] [-v verbose_lvl]
[-z sec] [host1] host2

actsyncd [-x] actsync.cfg [debug_level [debug_outfmt] ]

DESCRIPTION
Actsync(8) permits one to synchronize, compare or merge two active(5)
files. With this utility one may add, change or remove newsgroups on
the local news server to make it similar to the list the newsgroups
found on another system or file. The synchronization need not be
exact. Local differences in newsgroup lists may be maintained and pre-
served. Certain newsgroup errors may be detected and optionally cor-
rected.

There are several reasons to run actsync(8) (or actsyncd(8)), on a
periodic basis. Among the reasons are:

A control message to add, change or remove a newsgroup may fail to
reach your site.

Your control.ctl(5) is out of date or incomplete.

News articles for a new newsgroup arrive ahead (sometimes days
ahead) of the control message.

Control messages may be forged, thus bypassing the restrictions
found in control.ctl(5).

Your active(5) file may have been trashed.

If either host1 or host2 begin with a ``.'' or ``/'', then they
assumed to be a name of a file containing information in the active(5)
format. The getlist(1) utility may be used to obtain copy a remote
system's active file via its NNTP server, or an FTP client program can
get retrieve such a file from an FTP archive (such as
ftp://ftp.isc.org/pub/usenet/CONFIG/active; see more about this below).
Newsgroup information from a file may be treated as if it was obtained
from a host. In this man page host1 and host2 are called hosts, even
though they may be file names.

If a host argument does not begin with ``.'' or ``/'', is assumed to
be a hostname or Internet address. In this case, actsync(8) will
attempt to use the NNTP protocol to obtain a copy of the the specified
system's active file.

Regardless how the active file information is obtained, the actions of
actsync(8) remain the same.

If only one host is specified, it is assumed to be host2. If host1, is
not specified, it assumed to be the default local NNTP server as speci-
fied by the NNTPSERVER environment variable, or by the server value
found in inn.conf(5).

The newsgroup synchronization by default, involves all newsgroups found
on both hosts. One may also synchronize on a subset of newsgroups by
directing actsync(8) to ignore certain newsgroups from both systems.

The actsyncd(8) daemon provides a convenient interface to configure and
run actsync(8). If a host is not initially reachable, the daemon will
thrice retry 9 times, waiting 6 minutes before each retry. This daemon
runs in the foreground, sending output to standard output and standard
error.

If the -x flag is given to actsyncd(8), then a ctlinndxexec will be
used instead of a ctlinndreload to load the newly modified active file.

The configuration filename for the daemon is given in the actsync.cfg
argument. The actsync.cfg file understands the following options:

host=host2
ftppath=/remote/path/to/active/file
spool=<normally patharticles in inn.conf>
ignore_file=ignore_file
flags=actsyncd (8) options

The host, ignore_file and flags lines are mandatory.

The keyword must start at the beginning of the line, and there may be
no whitespace before the ``='' character. Blank lines are ignored.
Comments start with ``#'' and are ignored. All other lines may produce
undefined results.

The host config file line refers to the host2 value to sync off of.
the ftppath directive causes the machine named in the host line to
accessed as an ftp server, retrieving the file named. If the filename
ends in .gz or .Z, then it will automatically be uncompressed after
retrieval. The spool config file lines determines where top the news
spool tree is to be found. The ignore_file config file line names the
ignore file to be used by actsync(8). The flags config file line
refers to all flags that you wish to pass to actsync(8).

Note that the -i ignore_file option, the -o format option and the -S
spool_dir option should not be given in the flags= line because they
are automatically taken care of by actsyncd(8).

INN is shipped with default values of ftp.isc.org for host and
/pub/usenet/CONFIG/active for ftppath. You can read about the policies
used for maintaining that active file at
ftp://ftp.isc.org/pub/usenet/CONFIG/README. Consider sychronizing from
this file on a daily basis by using cron.

OPTIONS
The options to actsync(8) are as follows:

-b hostid
This flag causes actsync(8) to ignore newsgroups with
``bork.bork.bork'' style names. That is, newsgroups whose last
3 components are identical. For example, the following news-
groups have bork style names:

alt.helms.dork.dork.dork
alt.auto.accident.sue.sue.sue
alt.election.vote.vote.vote

The value hostid determines on which hosts this action is per-
formed:

0 neither host
1 local default server
2 remove server
12 both servers
21 both servers

The default is -b 0, no bork newsgroups are ignored.

-d hostid
This flag causes actsync(8) to ignore newsgroups that have all
numeric path components. The hostid value is interpreted the
same as in -b. For example, the following newsgroups have
numeric path components:

alt.prime.chongo.23209
391581.times.2.to_the.216193.power.-1
99.bottles.of.treacle.on.the.wall
linfield.class.envio_bio.101.d

The newsgroups directory of a newsgroups with a all numeric com-
ponent could conflict with an article from another group. For
example, the directory for the first newsgroup listed above is
the same path as article number 23209 from the newsgroup:

alt.prime.chongo

The default is -d 0, all numeric newsgroups from both hosts will
be processed.

-g max Ignore any newsgroup with more than max levels. For example, -g
6 would ignore:

alt.feinstien.votes.to.trash.freedom.of.speech
alt.senator.exon.enemy.of.the.internet
alt.crypto.export.laws.dumb.dumb.dumb

but would not ignore:

alt.feinstien.acts.like.a.republican
alt.exon.admendment
alt.crypto.export.laws

If max is 0, then the max level feature is disabled.

By default, the max level feature is disabled.

-i ignore_file
The ignore_file allows one to have a fine degree of control over
which newsgroups are ignored. It contains a set of rules that
specifies which newsgroups will be checked and which will be
ignored.

By default, these rules apply to both hosts. This can be modi-
fied by using the -I hostid flag.

By default, all newsgroups are checked. If no ignore_file if
specified, or if the ignore file contains no rule lines, all
newsgroups will be checked.

Blank lines, and text after a ``#'' are considered comments and
are ignored.

Rule lines consist of tokens separated by whitespace. Rule
lines may be one of two forms:

c newsgroup [type ...]
i newsgroup [type ...]

If the rule begins with a c then the rule requests certain news-
groups to be checked. If the rule begins with an i then the
rule requests certain newsgroups to be ignored. The newsgroup
field may be a specific newsgroup, or a wildmat(3) pattern.

If one or more types are specified, then the rule applies to the
newsgroup only if is of the specified type. Types refer to the
4th field of the active(5) file. A type may be one of:

y
n
m
j
x
=group.name

Unlike active files, the group.name may be a newsgroup name or a
wildmat(3) pattern. Also, ``='' is equivalent to ``=*''.

For given rule line may, one may not repeat a given pattern
type. For example, one may not have more than one type that
begins with ``='', per line. However, one may achieve the
effect of multiple ``='' types by using multiple rule lines for
the same group.

By default, all newsgroups are candidates to be checked. If an
ignore file is used, each newsgroup in turn is checked against
the ignore file. If multiple lines match a given newsgroup, the
last line in the ignore file is used.

For example, consider the following ignore file lines:

i *.general
c *.general m
i nsa.general

The newsgroup: ba.general would be ignored if it was not moder-
ated. The newsgroup: mod.general would be checked if it was
moderated. The newsgroup: nsa.general would be ignored even if
it was moderated.

-I hostid
This flag restricts which hosts, the ignore file applies. The
hostid value is interpreted the same as in -b.

This flag may be useful in conjunction with the -m merge flag.
For example:

actsync -i actsync.ign -I 2 -m host1 host2

will keep all newsgroups currently on host1. It will also will
only compare host1 groups with non-ignored newsgroups from
host2.

The default is -I 12, newsgroups from both hosts to be ignored
per the -I hostid flag.

-k By default, any newsgroup on host1 that is in error will be con-
sidered for removal. This causes actsync(8) simply ignore such
newsgroups. This flag, in combination with -m will prevent any
newsgroup from being scheduled for removal.

-l hostid
Flag problem newsgroups of type ``='' from host1 or host2 as
errors. The hostid value is interpreted the same as in -b.
Newsgroups of type ``='' are newsgroups active entries that have
4th field that begins with ``=''. I.e., a newsgroup that is
equivalent to another newsgroup.

A newsgroup that is equivalent to itself, or that is in a equiv-
alence chain that loops around to itself is a problem. A news-
group that is in a chain that is longer than 16 is a problem
group. A newsgroup that is equivalent to a non-existent news-
group is a problem. A newsgroup that is equivalent to a news-
group that is has a error of some kind a problem. However, a
newsgroup that is equivalent to an ignored newsgroup is not a
problem.

By default, problem newsgroups from both hosts are marked as
errors.

-m Merge newsgroups instead of sync. By default, if a newsgroup
exists on host1 but not host2, it will be scheduled to be
removed. This flag disables this process, permitting newsgroups
unique to host1 to be kept.

-n name
Newsgroups that are created, are created via the ctlinnd(8) com-
mand. By default, the creator name used is actsync. This flag
changes the creator name to name.

-o fmt
Determine the output / action format of this utility. The fmt
may one of:

a output in active(5) format,
a1 output in active(5) format,
and output host1 non-error ignored groups
ak output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for any newsgroup being created
aK output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for all newsgroups found in host2
a1k output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for any newsgroup being created,
and output host1 non-error ignored groups
a1K output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for all newsgroups found in host2,
and output host1 non-error ignored groups
ak1 same as a1k
aK1 same as a1K
c output in ctlinnd(8) format
x no output, directly exec ctlinnd(8) commands
xi no output, directly exec ctlinnd(8) commands,
in an interactive mode

The a, a1, ak, aK, a1k, a1K, ak1 and aK1 style formats allow one
to form a new active file instead of producing ctlinnd(8) com-
mands. They use hi & low values of 0000000000 and 0000000001
respectively for newsgroups that are created. The ak and aK
variants change the the hi & low (2nd & 3rd active fields). In
the case of ak, newsgroups created take their hi & low values
from host2. In the case of aK, all newsgroups found on host2
take their hi & low values from host2.

The c format produces ctlinnd(8) commands. No actions are taken
because actsync(8) simply prints ctlinnd(8) commands on standard
output. The sync (or merge if -m) with host2 may be accom-
plished by piping this output into sh(1). A paranoid person
might prefer to use x or xi in case a newsgroup name or type
contains bogus characters that might be interpreted by sh(1).
Even so, this output format is useful to let you see how host1
may be synced (or merge) with host2.

The sync (or merge if -m) may be accomplished directly by use of
the x. With this format, actsync(8) uses the execl(2) system
call to directly executes ctlinnd(8) commands. Because of the
exec, there is no risk of bogus newsgroups containing bogus
characters causing a shell to do bogus (or dangerous) things.
The output of such execs may be seen of the verbosity level is
at least 2.

The actsync(8) utility will pause for 4 seconds before each com-
mand is executed if -o x is selected. See the -z sec flag
below.

The xi format interactively prompts on standard output and reads
directives on standard input. One may pick and choose changes
using this format.

Care should be taken when producing active(5) formatted output.
One should check to be sure that actsync(8) exited with a zero
status prior to using such output. Also one should realize that
such output will not contain lines ignored by the -i ignore_file
process even if -p 100 is used.

By default, -o c is assumed.

-p min_%_unchg
By default, the actsync(8) utility has safeguards against per-
forming massive changes. If fewer than min_%_unchg percent of
the non-ignored lines from host1 remain unchanged, no actions
(output, execution, etc.) are performed and actsync(8) exits
with a non-zero exit status. The min_%_unchg may be a floating
point value such as 66.666.

A change is considered a host1 line that was found to be in
error, was removed, was added or was changed. Changing the 2nd
or 3rd active fields via -oak or -o aK are not considered
changes by -p.

To force actsync(8) to accept any amount of change, use the -p 0
option. To force actsync(8) to reject any changes, use the -p
100 option.

By default, 96% of the lines not ignored in host1 must be
unchanged. That is, by default, -p 90 is assumed.

-q hostid
By default, all newsgroup errors are reported on standard
errors. This flag quiets errors from host1 or host2. The
hostid value is interpreted the same as in -b.

-s size
If size>0, then ignore newsgroups with names longer than size,
and ignore newsgroups equivalenced to names longer than size.
Length checking is perform on both the local and remote hosts.

By default, size is 0 and thus no length checking is performed.

-S spool_dir
For each new newsgroup (i.e., selected groups found on host2
that were not found on host), the newsgroup directory under
spool_dir will be examined. If storageapi is turned on, this
should be the same name as pathoverview in inn.conf. If this
newsgroup directory exists, then the hi & low (2nd & 3rd active
fields) values of the active entry will be changed to reflect
the range articles found.

This flag is only useful with -o a, -o a1, -o ak, -o aK, -o alk,
-o alK, -o ak1 or -o aK1. The -S spool_dir will override any hi
& low (2nd & 3rd active fields) values that would normally have
been used. This is an important and very much recommended
option as it will prevent article number collisions on news-
groups that have been removed previous but still have unexpired
articles in them.

-t hostid
Ignore improper newsgroups with only a top component from host1
or host2. The hostid value is interpreted the same as in -b.
The following newsgroups are considered proper newsgroups for
top only names:

control
general
junk
test
to

For example, the following newsgroup names are improper because
they only contain a top level component:

dole_for_pres
dos
microsoft
windoes95

By default, all improper top level only newsgroups from the
remote ( -t 2 ) are ignored.

-T This flag causes host2 newsgroups from new hierarchies to be
ignored. Normally if only host2 has the newsgroup
chongo.was.here then it will be created for host1. However if
host1 does not have any 'chongo.*' newsgroups and this flag is
given, then chongo.was.here will be ignored and will not be cre-
ated on host1.

-v verbose_lvl
No default, actsync(8) is not verbose. This flag controls the
verbosity level as follows:

0 no debug or status reports (default)
1 print summary,
if work was needed or done
2 print actions, exec output & summary,
if work was needed or done
3 print actions, exec output & summary
4 full debug output

-z sec If -o x is selected, actsync(8) will pause for sec seconds
before each command is executed. This helps prevent innd(8)
from being busied-out if a large number of ctlinnd(8) commands
are needed. One can disable this sleeping by using -z 0.

By default, actsync(8) will pause for 4 seconds before each
command is executed if -o x is selected.

EXAMPLES
Determine the difference (but don't change anything) between your news-
group set and uunet's set:

actsync news.uu.net

Same as above, with full debug and progress reports:

actsync -v 4 news.uu.net

Force a site to have the same newsgroups some other site:

actsync -o x master

This may be useful to sync a slave site to its master, or to sync
internal site to a gateway.

Compare your site with uunet, disregarding local groups and certain
local differences with uunet. Produce a report if any differences were
encountered:

actsync -v 2 -i actsync.ign news.uu.net

where actsync.ign contains:

# Don't compare to.* groups as they will differ.
#
i to.*

# These are our local groups that nobody else
# (should) carry. So ignore them for the sake
# of the compare.
#
i nsa.*

# These groups are local favorites, so keep them
# even if uunet does not carry them.
#
i ca.dump.bob.dorman
i ca.keep.bob.dorman
i alt.tv.dinosaurs.barney.die.die.die
i alt.tv.dinosaurs.barney.love.love.love
i alt.sounds.* =alt.binaries.sounds.*

To interactively sync against news.uu.net, using the same ignore file:

actsync -o xi -v 2 -i actsync.ign news.uu.net

Based on newsgroups that you decided to keep, one could make changes to
the actsync.ign file:

# Don't compare to.* groups as they will differ.
#
i to.*

# These are our local groups that nobody else
# (should) carry. So ignore them for the sake
# of the compare.
#
i nsa.*

# These groups are local favorites, so keep them
# even if uunet does not carry them.
#
i ca.dump.bob.dorman
i alt.tv.dinosaurs.barney.die.die.die
i alt.sounds.* =alt.binaries.sounds.*

# Don't sync test groups, except for ones that are
# moderated or that are under the gnu hierarchy.
i *.test
c *.test m # check moderated test groups
c gnu.*.test
c gnu.test # just in case it ever exists

Automatic processing may be setup by using the following actsync.cfg
file:

# host to sync off of (host2)
host=news.uu.net

# location of the ignore file
ignore_file=<PREFIX specified with --prefix at configure>/etc/actsync.ign

# where news articles are kept
spool=<patharticles in inn.conf>

# actsync(8) flags
#
# Automatic execs, report if something was done,
# otherwise don't say anything, don't report
# uunet active file problems, just ignore
# the effect entries.
flags=-o x -v 2 -q 2

and then by running actsyncd with the path to the config file.

actsyncd <PREFIX specified with --prefix at configure>/etc/act-
sync.cfg

One may produce a trial actsyncd(8) run without changing anything on
the server by supplying the debug_level arg:

actsyncd <PREFIX specified with --prefix at configure>/etc/act-
sync.cfg 2

The debug_level causes actsyncd(8) to run actsync(8) with an -v
debug_level (overriding any -v flag on the flags line), prevents any
changes from being made to the active(5) file, writes a new active file
to standard output and writes debug messages to standard error.

If the debug_outfmt arg is also given to actsyncd(8) then the data
written to standard output will be in -o debug_outfmt instead of in -o
a1 format. The following /bin/sh command:

actsyncd <PREFIX specified with --prefix at configure>/etc/act-
sync.cfg 4 >cmd 2>dbg

Will operate in debug mode, not change the active(5) file, write
ctlinnd(8) style commands to cmd and write debug statements to dbg.

To check only the major hierarchies against news.uu,net, use the fol-
lowing actsync.ign file:

# by default, ignore everything
i *

# check the major groups
c comp.*
c gnu.*
c sci.*
c alt.*
c misc.*
c news.*
c rec.*
c soc.*
c talk.*

and running:

actsync -i actsync.ign news.uu.net

To determine the differences between your old active and your current
default server:

actsync <pathetc in inn.conf>/active.old -

To report but not fix any newsgroup problems with the current active
file:

actsync - -

To detect any newsgroup errors on your local server, and to remove any
*.bork.bork.bork style silly newsgroup names:

actsync -b 2 - -

The active file produced by:

actsync ... flags ... -o x erehwon.honey.edu

or by:

actsync ... flags ... -o c erehwon.honey.edu | sh

is effectively the same as the active file produced by:

ctlinnd pause 'running actsync'
rm -f active.new
actsync ... flags ... -o a1 erehwon.honey.edu > active.new
rm -f active.old
ln active active.old
mv active.new active
ctlinnd reload active 'running actsync'
ctlinnd go 'running actsync'

It should be noted that the above 'pause', 'actsync', 'reload' and 'go'
method is faster. However, in order to avoid article number collisions
on newsgroups that have been removed previous but still have unexpired
articles in them, it is very much recommended that the -S spool_dir be
used with any of the -oa* flags. Thus, a much better and safer version
of the above would be:

ctlinnd pause 'running actsync'
rm -f active.new
actsync ... flags ... -o a1 -S <patharticles or pathoverview in inn.conf> erehwon.honey.edu > active.new
rm -f active.old
ln active active.old
mv active.new active
ctlinnd reload active 'running actsync'
ctlinnd go 'running actsync'

The above process is similar to what actsyncd(8) does by default.

CAUTION
Careless use of this tool may result in the addition change or removal
of newsgroups that you don't want. You should avoid using the x output
format until you are sure it will do what you want.

Be aware that innd(8) servers older than version 1.5 may corrupt the
active file when multiple rmgroups are performed if the server is
paused or throttled. This is not a actsync(8) bug, it is a server bug.
Using the pause, actsync, reload and go method noted above avoids this
problem of older servers.

BUGS
If a newsgroup appears multiple times, actsync(8) will treat all copies
as errors. However, if the group is marked for removal, only one
rmgroup will be issued.

The timeout for ctlinnd(8) commands is fixed at 30 seconds when running
in ``x'' or ``xi'' output format. Perhaps the timeout value should be
controlled via a command line option?

SEE ALSO
active(5),
simpleftp(1),
mod-active(8),
ctlinnd(8),
getlist(8),
inn.conf(5).

HISTORY
Written by Landon Curt Noll <chongo@toad.com> for InterNetNews.
Updated to support ftp fetching by David Lawrence <tale@isc.org>.

ACTSYNC(8)

Man(1) output converted with man2html

list of all man pages