These step-by-step instructions detail how to replicate a Harvest broker using our mirrord/floodd replication tools. In brief, floodd measures the available network bandwidth and delay between broker replicas and creates and maintains the network topology over which replicas should be updated to use the network most efficiently. Mirrord invokes the ftp-mirror package written by Lee McLoughlin <lmjm@doc.ic.ac.uk> when it is time to grab a new version of the broker database. As you start testing your own replica, you might want to query an operating floodd and its corresponding mirrord mirrord .
Follow the steps to configure mirrord and replicate a broker. These
instructions are written assuming that you want to replicate the
www-home-pages broker and that you will run this replica from directory
/usr/local/mirrord
.
The master copy of the www-home-pages is maintained on powell.cs.colorado.edu the www-home-pages group coordinated from catarina.usc.edu
gunzip mirrord.tar.gz and tar xf mirrord.tar
This will leave you with the directory mirrord-dist.
cd mirrord-dist
configure --prefix=/usr/local/mirrord
Note: without the --prefix
argument, configure
builds mirrord for
installation in `/usr/local'. Don't use --prefix=~smith.
Use --prefix=/home/smith.
make
make install
Note: this will create and populate /usr/local/mirrord.
cd /usr/local/mirrord/lib/
cp join.conf replica.conf
Configuration file `join.conf' is a template for joining an existing replication group.
cd ~ftp; mkdir www-home-pages;
If not already enabled, you will need to enable anonymous FTP. Refer to your machine's man pages for `ftpd'.
This installation guide assumes that your FTP directory is `/usr/local/ftp' and that your FTP directory is `/usr/local/ftp/www-home-pages'.
chown www-home-pages smith
Set directory protection so that it can be written by the user account under which you are going to run mirrord. Here, we assume you will run mirrord from account `smith'.
ftp localhost; dir; quit;
Verify that your anonymous ftpd returns times in GMT. You can check this
by using anonymous ftp to connect to your server and issuing the
dir
command. The date returned for any particular file should
differ from the date returned by a cd ~ftp; ls -l
', unless your local
timezone is GMT. Triple check that the difference between the anonymous ftpd
notion of the date and the file system notion of the date differs by your
site's GMT offset.
Explanation: To keep timestamps synchronized across multiple time zones, mirror exports all dates in GMT. If one machine exports dates in its local time, your replica may cause havoc for the entire replica group. On many UNIX systems, you can specify the timezone using the TZ environment variable. If you needed to force your ftpd to use GMT times, please tell dante@usc.edu your solution.
vi /usr/local/mirrord/lib/replica.conf
Examine your `replica.conf' and substitute the CAPITALIZED-VARIABLE-NAMES with appropriate values for your installation. To replicate the www-home-pages, substitute the following names wherever they appear in replica.conf.
You should not need to change the following defaults unless some other daemon
process has already allocated particular TCP ports or you decide to relocate
the RootDirectory after your original configure --prefix=/usr/local/mirrord
:
/usr/local/mirrord/bin/setup-replica MYHOST.LOCAL.DOMAIN
Be sure to use your hostname for MYHOST.LOCAL.DOMAIN. This copies necessary files to directory /usr/local/mirrord/replica.
cd /usr/local/mirrord/replica;
cp ../lib/mirrord.conf.ex mirrord.conf;
vi mirrord.conf;
Substitute local values for the CAPITALIZED-VARIABLES. We know, the list looks formidable:
# SKIP THIS STEP IF YOUR PERL INSTALLATION IS COMPLETE
If perl has not been completely installed on your system, you might need to copy the file `ftp-mirror/socket.ph' from the full harvest distribution to `/usr/local/mirrord/lib/perl/sys/socket.ph'. The `socket.ph' file shipped is for SunOs4.1.3. Your mileage may vary on other systems and you ought to have your sysadmin install perl correctly.
cd /usr/local/mirrord/replica
/usr/local/mirrord/bin/floodd
/usr/local/mirrord/bin/mirrord -l LOG &
When setting up floodd/mirrord on a multi-homed machine, check the floodd and mirrord pages to be sure that all sites are referring to your replica by a single IP address.
If you are installing the www-home-pages broker, you may want to install a WAIS index rather than a glimpse index. We assume that you create this broker in directory /usr/local/httpd/htdocs/www-home-pages.
cd /usr/local/httpd/htdocs/www-home-pages/admin
ln -s /usr/local/ftp/www-home-pages/admin/Registry .
cd ..
ln -s /usr/local/ftp/www-home-pages/objects .
RunBroker
You will have to wait until mirrord copies your entire replica. For the www-home-page broker, this can take all night at 28,000 baud or an hour on a faster network.