Debian GNU/Linux CD Images

->- You want to become a Debian CD Image mirror site.

Excellent!  You need to run a Linux or UNIX-like machine for this.

You should not use FTP or HTTP to get the images. These transfers have a high failing probability because of the enourmous size of the files. Moreover, if the images change only slightly (as is the case with subreleases), they would be downloaded entirely all over again. This puts a high stress on our computers and network connections.

For these reasons, we use the `rsync' program that is essentially doing remote binary patching. With rsync, subrelease images are updated very efficiently, as only the changed parts are transferred over the network; the unchanged parts are copied from the old image.

Rsync alone still does full downloads for `all new' releases, as those images contain virtually no unchanged parts. However, with the Pseudo-Image Kit we can create `pseudo' CD images from data that is available already on the nearest Debian FTP mirror -- maybe on your own local harddisk. Then rsync can binary patch these `pseudo-images' to the official ones. And even for most subrelease images this is found to be more efficient than patching the old image.

The Pseudo-Image Kit is designed mainly for downloading only one or a few images, like most people do. There is also the `debcdmirror' script which makes mirroring really easy because it automatically uses the Pseudo-Image Kit and rsync to keep your mirror up to date -- even to the point of automatically MD5-checksumming every downloaded image. With debcdmirror, you can also specify exactly what you want to mirror.

So we have the following possibilities for getting the images and keeping them up to date:
 

• By FTP or HTTP
This should not be done.
• By using only rsync (daily cron job)
This is preferable only on sites that don't have a nearby/fast Debian FTP mirror.
• By using the Pseudo-Image Kit manually (maybe with rsync to update subreleases)
Lots of work and lots of trouble.
• By using the debcdmirror script (which "internally" uses the Pseudo-Image Kit and rsync) (daily cron job)
This is what you should do!

The latest version of the Pseudo-Image Kit is available from http://cdimage.debian.org/~costar/pseudo-image-kit. ; the debcdmirror script is available at http://cdimage.debian.org/~costar/debcdmirror. Read the README's in both packages for extensive information about the procedures.

Note 1: For the debcdmirror script, you must choose an rsync mirror that offers an `ls-lR' file. Addresses are on http://cdimage.debian.org/rsync-mirrors.html. Official public mirrors are allowed to rsync directly from the master server, cdimage.debian.org; you probably need a password for that one, ask the maintainer.

If you don't have a local "regular" Debian FTP mirror, check the list of FTP sites.

debcdmirror requires bash version 2 or higher. If necessary you can install that in a non-default location, but then be sure to update the path to bash in the first line of the script.

Note 2: If you use rsync only, use at least the options --times --links --hard-links --block-size=8192. This will preserve modification time, symlinks and hardlinks, and use a block size of 8192 bytes (most efficient for CD images). When modification time and size are the same, rsync will just leave the file alone, so --times is really obligatory.

Note 3: The `codename_test' directory contains images that are hard-linked to the ones in the versioned (e.g. 2.1_r3) directories. The versioned directory name will change between (sub)releases, but the names in the `test' directories will stay the same. So, in the rsync-only situation, mirroring both the versioned and the appropriate `test' directories, and using the --hard-links option is the only way for rsync to update subreleases properly. [You should not include `test' directories with debcdmirror, because debcdmirror will handle things differently and more efficiently.]

Note 4: The ls-lR file describes the directory structure of the mirror it is on. DO NOT mirror the ls-lR file; either make one yourself (with the provided `Makels-lR', every hour via cron) or simply have no ls-lR file at all. [debcdmirror will create a local ls-lR automatically after mirroring.]
 

Once you have acquired the CD images you want to mirror (i386 and Source CDs are most popular), you should run the rsync server program. This will not place a heavy load on your machine, and will generate much less network traffic than an FTP/HTTP service. Instructions for setting this up are in the README of the Pseudo-Image Kit.

Of course, if you don't have problems with many gigabytes of network traffic per day, we'd very much appreciate another FTP/HTTP mirror. We'll be happy with everything you can afford.

Please notify debian-cd@lists.debian.org that you are running a mirror, so that we can update the appropriate pages. Thanks in advance!

There may also be some interesting information in the CD Image Frequently Asked Questions.