HowTO Private/Local Mirror with MRepo

HowTo Mrepo:

After going through some tools to maintain a number of mirrors ans Repositories I have finally settled for Mrepo. It has a number of advantages over the others or you can say a good tool for me with my given set of limitations.

• No connectivity for RSYNC port (i.e rsync out of question).

• All transfers through http via a proxy.

• To maintain number of mirrors, repositories and connectivity to RHN.

• Provide installation base for PXE and TFTBoot.

With these limitations and requirements when I started I had no idea of how a mirror is maintained or the process to set them up. Maximum of the HowTO's and forums always talk about rsync which is indeed a great tool but cannot be used in my case.

So here I end up with this tool Mrepo from http://dag.wieers.com/home-made/mrepo/ which gave all the features I required.

So lets start the show:

Note: Start trying this howto once you have gone through the whole document( since my organization of topics is not good enough and the other is you will need points from different areas, personally I lost a lot of time trying to figure out something already said at the end).

We need all these packages for the process.

• Running httpd deamon.

• createrepo-0.4.11-3.el5.noarch.rpm

• mrepo-0.8.6-1.el5.rf.noarch.rpm

Along with all the dependencies required.

Note: If you are going for RHN on a CentOS installation you need to install rhnlib-2.2.5-1.el5.noarch.rpm before installing the above mentioned packages.

Get ready with the config parameters required.

1. Source location where the downloaded packages are to be saved to SRC_LOC

2. Location where the repodata is to be created to WWW_LOC. This will become your httpd's DirectoryRoot.

3. If you are to communicate through a proxy get http_proxy.

4. If you are going to connect to RHN network get https_proxy.

5. All the iso files for the distributions (required if this repo is going to be used for baremetal else optional).

Once all the packages are installed.

In /etc/mrepo.conf change/add the parameters that are required.

[main]

srcdir = SRC_LOC

wwwdir = WWW_LOC

confdir = /etc/mrepo.conf.d

arch = i386

http_proxy=XXXX

https_proxy=YYYY

mailto = root@localhost

smtp-server = localhost

#If connecting to RHN use

rhnlogin = abc:xyz

Here we shall see the configurations of CentOS 5 and Rhel 5 mirrors.

CentOS 5 :

There are some template files present in “/usr/share/doc/mrepo-0.8.6/dists” you can take one related to centos5 and copy it to /etc/mrepo.conf.d/centos5.conf . You can strrip down all the unwanted section. As of mine my conf file looks like

### Centos 5

[centos5]

name = CentOS $release ($arch)

release = 5

#arch = i386 x86_64 ia64 ppc s390 s390x alpha

arch = i386

metadata = repomd repoview

### ISO images

iso = http://mirror.centos.org/pub/centos/$release/isos/$arch/CentOS-$release-$arch-bin?of4.iso

os = http://mirror.centos.org/centos/$release/os/$arch/

### Additional repositories

updates = http://mirror.centos.org/centos/$release/updates/$arch/

fasttrack = http://mirror.centos.org/centos/$release/fasttrack/$arch/

centosplus = http://mirror.centos.org/centos/$release/centosplus/$arch/

extras = http://mirror.centos.org/centos/$release/extras/$arch/

addons = http://mirror.centos.org/centos/$release/addons/$arch/

when you are going to maintain multiple versions take care that the labels that you are specifying for each do not collide. The reason is when you run mrepo it will create the folder structure like <label>-<arch> in SRC_LOC directory. So for each arch requirement a directory is created.

Now postmortem of the conf file:

name : Name of the release.

release : Version to be downloaded. In case of CentOs5 5/5.1/5.2/5.3

arch : Architecture of the files to be downloaded options i386, x86_64, ia64, ppc, s390, s390x, alpha .

Metadata : what type of repodata is to be created for. Yum uses repomd and repoview . You can get all the options from the templete files present in the dists/docs folders in /usr/share/docs/mrepoXXXX.

<channels>: All the areas that are to be mirrored.

We shall discuss about the iso channel down the line.

Patch work:

To create the rpm groups for os stream then we need to copy the comp.xml file of os stream to SRC_LOC/<centos_conf lable>-<arch>/ as os-comp.xml

Configuration for Rhel 5:

Before proceeding forward, first we need to register the system with RHN and create system id. This can be done by using “gensystemid -u abc -p xyz --release=5Server --arch=i386 /xyz “ .

You can get all the options by “gensystemid –help”.

when you do this, mrepo will do the process of registering your system with RHN. Here mrepo will register with RHN as <hostname>_mrepo . If you want to change the naming then you need to change line 113 in /usr/bin/gensystemid file.

Once done with registration create the conf file in /etc/mrepo.conf.d/ you can a get a template from “/usr/share/doc/mrepo-0.8.6/dists”.

My conf file looks like

[rhel5s]

name = Red Hat Enterprise Server $release ($arch)

release = 5.3

#arch = i386 x86_64 ia64 ppc s390 s390x

arch = i386

metadata = repomd repoview

### ISO images

iso = rhel-server-$release-$arch-disc?.iso rhel-$release-server-srpms-disc?.iso

### Additional repositories

updates = rhns:///rhel-$arch-server-5

vt = rhns:///rhel-$arch-server-$repo-5

supplementary = rhns:///rhel-$arch-server-$repo-5

fastrack = rhns:///rhel-$arch-server-$repo-5

hts = rhns:///rhel-$arch-server-$repo-5

rhn-tools = rhns:///$repo-rhel-$arch-server-5

Patch work:

we need to copy the system id file previously created from gensystemid at /xyz to SRC_LOC/<redhatconf lable>-<arch>/systemid

Setting up httpd daemon:

In “/etc/httpd/conf/httpd.conf” we need to change DocumentRoot and Directory to WWW_LOC at two different places typically 281, 306 lines in an unchanged httpd conf file.

Now everything is set start mrepo and httpd services if already started just restart them.

service httpd start

service mrepo start

now run the command mrepo -guvv to sync your private mirror with the central mirrors.This command will download all the rpms to the SRC_LOC file and triggers createrepo to create repodata at WWW_LOC location.which can be access by http call to the location.(ex: http://localhost:8080/centos5-i386/”)

Setup source location for tftpboot:

ISO's are required for this setup to work as tftpbase.

Create a directory “iso” at SRC_LOC and copy all the iso into that folder (i.e all distributions and version). Take care that the naming of the iso files are in similar to the once declared in the conf files.

Ex:

Conf:

release = 5 .2

arch = i386

iso = http://mirror.centos.org/pub/centos/$release/isos/$arch/CentOS-$release-$arch-bin?of6.iso

the actual file name is

SRC_LOC/iso/CentOS-5.2-i386-bin-4of6.iso

Setting up of the TFTServer and DHCP is not in the scope of this HOWTO You need them to be already present.

In the kickstart file just keep the URL parameter to the designated Url which you can get from the browser “http://localhost:8080/centos5-i386/”.

Once everything is done you need to run “mrepo -gvv” to build the new repodata.

LoopBack issue:

When you are going to maintain more number of versions the total number of iso's increases making you in need of more loop back interfaces. You can get over this issue by following the steps given in /usr/share/docs/mrepo/docs/loop-devices.txt.

This issue occurs when there are more than 8 iso's (I do not exactly remember).

gpg check:

To enable gpgcheck we need to get the key files of respective vendors and place them in a location which can be accessed through httpd.

Usage of the Mirror:

In the client system you need to create local mirror config file.

Example Client conf:

[base]

name=CentOS-$releasever - Base

baseurl=http://xx.xxx.xxx.xxx/centos5_2-$basearch/RPMS.os

gpgcheck=1

gpgkey=http://xx.xxx.xxx.xxx/gpgkeys/RPM-GPG-KEY-CentOS-5

#released updates

[updates]

name=CentOS-$releasever - Updates

baseurl=http://xx.xxx.xxx.xxx/centos5_2-$basearch/RPMS.updates

gpgcheck=1

gpgkey=http://xx.xxx.xxx.xxx/gpgkeys/RPM-GPG-KEY-CentOS-5

#packages used/produced in the build but not released

[addons]

name=CentOS-$releasever - Addons

baseurl=http://xx.xxx.xxx.xxx/centos5_2-$basearch/RPMS.addons

gpgcheck=1

gpgkey=http://xx.xxx.xxx.xxx/gpgkeys/RPM-GPG-KEY-CentOS-5

#additional packages that may be useful

[extras]

name=CentOS-$releasever - Extras

baseurl=http://xx.xxx.xxx.xxx/centos5_2-$basearch/RPMS.extras

gpgcheck=1

gpgkey=http://1xx.xxx.xxx.xxxgpgkeys/RPM-GPG-KEY-CentOS-5

#additional packages that extend functionality of existing packages

[centosplus]

name=CentOS-$releasever - Plus

baseurl=http://xx.xxx.xxx.xxx/centos5_2-$basearch/RPMS.centosplus

gpgcheck=1

enabled=1

gpgkey=http://xx.xxx.xxx.xxx/gpgkeys/RPM-GPG-KEY-CentOS-5