Below is the bash script I use to mirror Centos repository into our local server. Due to storage limitation, we have to be selective and only copies the packages/branches that we need. Currently we only mirror Centos 5 for “os and updates” base packages limited to i386 and x86_64 architecture only.
During the initial intensive downloading when the server is replicating for the first time, I run the script below in cron for every 30 minutes. The script will check whether the previous instance is still running or dead (sometimes we got disconnected). If dead then it will restart and continue the rsync process. Once everything is in place and all needed packages have been downloaded, I modify the cron job to do the job once in every 6 hours to make sure my server is up to date.
The script below is inspired by the script posted here by jlar310
#!/bin/sh # created by Jalte @ http://www.ridinglinux.org DATE=`/bin/date +%Y-%m-%d` OUTFILE=/var/www/html/mirror/centos-mirror.log RSYNC="/bin/nice /usr/bin/rsync --verbose --progress --delete-excluded --stats --archive --partial --timeout=600" MIRROR=mirrors.kernel.org::centos VER=5 ARCHLIST="i386 x86_64" BASELIST="os updates" LOCAL=/var/www/html/mirror/centos date >> /var/log/rsynccentos.log if [ -f "/var/run/rsynccentos.pid" ]; then RUNPID=`cat /var/run/rsynccentos.pid` if ps -p $RUNPID; then echo "Mirror is already running..." echo "Mirror is already running..." >> /var/log/rsynccentos.log exit 1 else echo "Mirror pid found but process dead, cleaning up" rm -f /var/run/rsynccentos.pid echo "Mirror pid found but process dead, cleaning up" >> /var/log/rsynccentos.log fi else echo "No process Detected" echo "No process Detected" >> /var/log/rsynccentos.log fi echo $$ > /var/run/rsynccentos.pid echo -n "Rsync Started at " echo "Rsync Started " >> /var/log/rsynccentos.log date for ARCH in $ARCHLIST do for BASE in $BASELIST do DIR=$LOCAL/$VER/$BASE/$ARCH/ if [ -d $DIR ] then echo "Directory exists." else echo "Directory does not exist, and will be created." mkdir -p $DIR fi REMOTE=$MIRROR/$VER/$BASE/$ARCH/ $RSYNC $REMOTE $DIR > $OUTFILE 2>&1 done done echo "`date`" > /var/www/html/mirror/centos-last-updated.log chown -R apache:apache $LOCAL
Any comments are greatly appreciated