Many package formats are used on Linux. The two main ones are Debian's deb format and Red Hat's RPM format. Debian has a package called alien that is able to assimilate and transform foreign package formats, specifically RPM, so they can be installed. The alien tool can be used to exchange .rpm, .tgz, and .deb package formats.
The general methodology when building a package is to set down a build process for the source that is as close as possible to what someone would do by hand, but then to pack the files into their own directory hierarchy that contains only the files that belong in the package. The contents of the hierarchy are then packaged. A lot of the magical things happen in the build process to put files into this dummy hierarchy in such a way that they will still work when the user installs them in the / and /usr directory hierachies.
Building RPM packages from an SRPM (source RPM) package is pretty straightforward. First, of course, you must find and download the source package. These are typically stored in src.rpm files.
There are two ways of building an RPM source package. If the package is satisfactory to you and you only want to recompile it for your system (for example, because you use SUSE while the binary package was built for Red Hat), you can simply enter:
rpmbuild --rebuild package
.src.rpm
This builds a binary package from the source package and places the result in the RPMS directory discussed later.
The other way of building a source RPM package involves installing it with the ordinary rpm command. This is what you should do if you want to modify a package. When you use rpm -ivh to install a source package, the contents are distributed among different directories under /usr/src/redhat on Red Hat and Fedora systems or under something similar, such as /usr/src/RPM, in other RPM-based distributions. The directories are:
Used for the build process. If you need to troubleshoot a build process, its leftovers are here.
Where finished packages are placed. There are subdirectories for the different available architectures such as athlon, i386, i486, and so on. These are ready-to-install binary packages.
Contains untouched source files and the patches needed for them.
Contains package build recipes. These are comparatively small text files that specify the complete build process for a package.
Where finished source packages are placed. These are source packages that are ready for rebuilding into binary packages.
When you are done with a specific package, remove the build tree and accompanying files. If you build any significant number of packages, the amount of space used can grow quite a bit.
Once the package is installed, change to the SPECS directory. There you should find a file called package
.spec. This is the build recipe for the package. The contents of this will be explained in the next section. To build a package, enter:
rpmbuild --bb package
.spec
Once the package is built, the resulting files are dropped into the SRPMS and appropriate RPMS subdirectories. The exact locations are printed at the end of the build process, such as:
... Wrote: /usr/src/redhat/SRPMS/gzip-1.3.3-11.src.rpm Wrote: /usr/src/redhat/RPMS/i386/gzip-1.3.3-11.i386.rpm ...
rpmbuild
rpmbuild {-ba|-bb|-bp|-bc|-bi|-bl|-bs} [rpmbuild-options]specfile
rpmbuild {-ta|-tb|-tp|-tc|-ti|-tl|-ts} [rpmbuild-options]tarbal
rpmbuild {--rebuild|--recompile}sourcepkg
rpmbuild builds RPM binary or source packages based on .spec files, tarballs containing a .spec file, or a source RPM package. The -b? options build from .spec, and the -t files from tarfiles that have been compressed through gzip or bzip2.
Build binary and source packages (a=all).
Build binary packages.
Build source package.
For packages that understand this option, it sets the compilation target. The most likely use is to specify an optimization, such as --target i686.
Build binary packages directly from a source RPM.
You are already familiar with a typical build process from Chapter 4. The process for a particular package is laid out in a spec file in the package's source RPM package; by altering the spec file you can change subsequent builds. We'll show a typical spec file here, using the Fedora Core 1 gzip package as an example. Modifying a package is quite simple, but there is a bit of magical syntax in the spec file.
A good, if somewhat dated, introduction to the RPM format is available in the book Maximum RPM. It is still available in print and also available on the Web at http://www.rpm.org/max-rpm.
The gzip spec file begins:
1 Summary: The GNU data compression program. 2 Name: gzip 3 Version: 1.3.3 4 Release: 11 5 License: GPL 6 Group: Applications/File 7 Source: ftp://alpha.gnu.org/gnu/gzip/gzip-%{version}.tar.gz
The opening lines offer various formalities. Name, Version
, and Release
, are important; the others less so.
8 Patch0: gzip-1.3-openbsd-owl-tmp.diff 9 Patch1: gzip-1.2.4-zforce.patch 10 Patch2: gzip-1.2.4a-dirinfo.patch 11 Patch3: gzip-1.3-stderr.patch 12 Patch4: gzip-1.3.1-zgreppipe.patch 13 Patch5: gzip-1.3-rsync.patch 14 Patch6: gzip-1.3.3-window-size.patch
The preceding lines enumerate the patches in the package. They are referred to by number rather than name later in the file when the time comes to apply them.
15 URL: http://www.gzip.org/ 16 Prereq: /sbin/install-info 17 Requires: mktemp less 18 Buildroot: %{_tmppath}/gzip-%{version}-root
The URL
tag is not very important, but the Prereq
and Requires
tags are. They define the prerequisites for installing the package. Requires
is now a synonym for Prereq
. They used to be different, Prereq
specifying components that had to be installed before the package itself was installed and Requires
specifying components that were needed only for the execution of the package after installation.
20 %description 21 The gzip package contains the popular GNU gzip data compression 22 program. Gzipped files have a .gz extension. 23 24 Gzip should be installed on your Red Hat Linux system, because it is a 25 very commonly used data compression program. 26
Some descriptive paragraphs. These should try to include recommendations about when the package may be needed or useful. This section is ended by the next %
section heading, which in this spec file is:
27 %prep 28 %setup -q 29 %patch0 -p1 30 %patch1 -p1 31 #patch2 -p1 32 %patch3 -p1 33 %patch4 -p1 -b .nixi 34 %patch5 -p1 -b .rsync 35 %patch6 -p1 -b .window-size 36
These steps prepare the source for compilation. Note that patch number 2 is commented out. All the lines shown in this section are macros.
%setup -q unpacks the source. Each %patch
n
line applies the numbered patch, giving the options specified, such as -p1, to the patch command.
37 %build 38 export DEFS="-DNO_ASM" 39 export CPPFLAGS="-DHAVE_LSTAT" 40 %configure --bindir=/bin 41 42 make 43 make gzip.info 44
These are all the steps needed to build the package. Not too bad. %configure
is a macro that invokes a GNU-style configure script with all the right options for placement, optimization, and so on. The environment variables set before invoking the macro modify the behavior of the script.
45 %install 46 rm -rf ${RPM_BUILD_ROOT} 47 %makeinstall bindir=${RPM_BUILD_ROOT}/bin 48 mkdir -p ${RPM_BUILD_ROOT}%{_bindir} 49 ln -sf ../../bin/gzip ${RPM_BUILD_ROOT}%{_bindir}/gzip 50 ln -sf ../../bin/gunzip ${RPM_BUILD_ROOT}%{_bindir}/gunzip 51 52 for i in zcmp zegrep zforce zless znew gzexe zdiff zfgrep zgrep zmore ; do 53 mv ${RPM_BUILD_ROOT}/bin/$i ${RPM_BUILD_ROOT}%{_bindir}/$i 54 done 55 56 gzip -9nf ${RPM_BUILD_ROOT}%{_infodir}/gzip.info* 57 58 59 cat > ${RPM_BUILD_ROOT}%{_bindir}/zless <<EOF 60 #!/bin/sh 61 /bin/zcat "$@" | /usr/bin/less 62 EOF 63 chmod 755 ${RPM_BUILD_ROOT}%{_bindir}/zless 64 65 # we don't ship it, so let's remove it from ${RPM_BUILD_ROOT} 66 rm -f ${RPM_BUILD_ROOT}%{_infodir}/dir 67
While the build process wasn't too complex, the install process on the preceding lines is quite a mouthful. On line 46, the target directory for the installation is emptied of any old contents. Line 47 is another macro, this one calling make install
with the right options for a GNU-style Makefile and overriding the directory where the binaries should be installed. Lines 49–66 are devoted to symlinking, moving, compressing, creating the zless command, and then removing one file we don't want distributed, the dir file that should already exist on the host where the installation is taking place.
68 %clean 69 rm -rf ${RPM_BUILD_ROOT} 70
The preceding lines remove some of the leftovers after the build process.
71 %post 72 /sbin/install-info %{_infodir}/gzip.info.gz %{_infodir}/dir 73
%post
is a postinstall script used when installing the binary package. This script modifies the dir on the install subject to include the gzip info pages.
74 %preun 75 if [ $1 = 0 ]; then 76 /sbin/install-info --delete %{_infodir}/gzip.info.gz %{_infodir}/dir 77 fi 78
The %preun
, or preuninstall script is run right before uninstalling the binary package. If a package installs services, this is a good place to stop the service so that it's not left running after the software is installed.
79 %files 80 %defattr(-,root,root) 81 %doc NEWS README AUTHORS ChangeLog THANKS TODO 82 /bin/* 83 %{_bindir}/* 84 %{_mandir}/*/* 85 %{_infodir}/gzip.info* 86
The file list enumerates all the files that are to be packaged. The list should be complete and exhaustive if some file is forgotten, rpmbuild will complain. If the file is not needed, remove it in the build step. Otherwise, make sure it gets on the list. The files installed here are enumerated in lines 82 through 85. Line 80 sets the default attributes: the permissions, owner, and group of the files installed. The permissions is an octal number such as you would use with chmod. Line 81 declares which files are documentation, so that they can be specially handled when needed.
87 %changelog 88 * Tue Oct 28 2003 Jeff Johnson <[email protected]> 1.3.3-11 89 - rebuilt. 90 ... 241 242 * Tue Apr 22 1997 Marc Ewing <[email protected]> 243 - (Entry added for Marc by Erik) fixed gzexe to use /bin/gzip
The final section is simply documentation for the build itself.
After all that verbiage, it's good to know that you don't need a deep understanding of it all to modify a package. Modifying a package mostly consists of adding a patch, removing a patch, or perhaps adding or removing an option from a build or install command. How this is done should be obvious from the preceding example and comments. If you ever find old packages that you want to compile for your newer installation, they may contain things that are now obsolete, or they may fail to enumerate some file in the %files
section. In most cases, this can be dealt with by intuition, web searches, or looking through the documentation files in the RPM package itself.
Debian source packages are documented, among other places, in http://www.tldp.org/HOWTO/Debian-Binary-Package-Building-HOWTO and http://www.debian.org/doc/maint-guide.
Debian packages are usually fetched and installed by apt-get. Sources can be fetched using apt-get source
package
. The Debian repository system tells apt-get what is needed. It fetches the files and creates a source directory, named after the package and suitable to build from, with the help of dpkg-source. If you make any unfortunate changes in the source directories, they can be regenerated with dpkg-source -x package.dsc.
During the build process, the package files are generated and are available in the directory above the source directory. The build process shows this step like this:
dpkg-deb: building package 'gzip' in '../gzip_1.3.2-3woody1_i386.deb'.
These packages can be installed normally with dpkg or uploaded to a Debian repository. If you maintain more than a few Debian packages for more than a few machines, it will probably save time if you build a private package repository to use with apt-get.
Debian requires that packages are signed with gpg. Because you probably do not have the package maintainers' cryptography keys, you need your own. If you do not already have one, you can easily create one, as illustrated here:
$gpg --gen-key
pg (GnuPG) 1.2.1; Copyright (C) 2002 Free Software Foundation, Inc. This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the file COPYING for details. gpg: keyring '/home/joe/.gnupg/secring.gpg' created gpg: keyring '/home/joe/.gnupg/pubring.gpg' created Please select what kind of key you want: (1) DSA and ElGamal (default) (2) DSA (sign only) (5) RSA (sign only)Your selection?5
What keysize do you want? (1024)2048
Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n yearsKey is valid for? (0)3y
Key expires at Fri Jan 12 00:23:00 2007 CETIs this correct (y/n)?y
You need a User-ID to identify your key; the software constructs the user id from Real Name, Comment and Email Address in this form: "Heinrich Heine (Der Dichter) <[email protected]>" Real name:Joe Doe
Email address:[email protected]
Comment: <nothing> You selected this USER-ID: "Joe Doe <[email protected]>" Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit?o
You need a Passphrase to protect your secret key. Enter passphrase:<passphrase>
Repeat passphrase:<passphrase>
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. ..+++++ +++++ public and secret key created and signed. key marked as ultimately trusted. pub 2048R/97DAFDB2 2004-01-12 Joe Doe <[email protected]> Key fingerprint = 85B2 0933 AC51 430B 3A38 D673 3437 9CAC 97DA FDB2 Note that this key cannot be used for encryption. You may want to use the command "--edit-key" to generate a secondary key for this purpose.
There are two main commands for building packages. dpkg-buildpackage and debuild. You may find cases where one works and the other does not.
dpkg-buildpackage
dpkg-buildpackage [options
]
This command is roughly the same as running the following shell commands, except that dpkg-buildpackage also signs the package files and does a whole lot of other things to check the package.
#debian/rules build
#debian/rules binary
If you run the build as a user, use this option to fake being root for the Debian build scripts.
signing-address
Sign the package with this key, instead of any key that may be specified in the package.
To build a package while not being root, signing with the key belonging to Joe Doe, enter:
dpkg-buildpackage [email protected] -r/usr/bin/fakeroot
debuild
debuild [options
]
debuild is a fully automatic script that generates Debian package files in a form that is ready for upload to a repository. It detects when to use fakeroot, but that option may also be specified on the command line. The command takes dpkg-buildpackage options, so to sign the package with your own key, use -k.
If you run the build as a user, use this option to fake being root for the Debian build scripts.
signing-address
Sign the package with this key, instead of any key that may be specified in the package.
The Debian build process is controlled by two files inside the source directory: debian/control and debian/rules. The control file defines formalities such as the package name, version, and dependencies, in addition to documenting the package. As explained in the previous sections on the dpkg-buildpackage command, the rules file controls building.
In this section we'll take a look at the control file in the package for gzip
.
Source: gzip Section: base Priority: required Maintainer: Bdale Garbee <[email protected]> Build-Depends: debhelper, automake, autoconf (>= 2.52) Standards-Version: 3.5.6.0 Package: gzip Architecture: any Pre-Depends: ${shlibs:Depends} Depends: debianutils (>= 1.6) Essential: yes Description: The GNU compression utility. This is the standard GNU file compression utility, which is also the default compression tool for Debian. It typically operates on files with names ending in '.gz'. . This package can also decompress '.Z' files created with 'compress'.
The file has two main sections, one for the source package and one for the binary package that will be built from it. Building the package requires more than just having it installed, as illustrated by comparing Build-Depends:
and Depends:
. The one bit of magic here is the value of the Pre-Depends:
field. ${shlibs:Depends}
will cause the building helper dh_shlibdeps to find shared library dependencies and dh_gencontrol to fill in the binary package control file with the actual values. debhelper is a package that assists in the build process of this package.
The rules file usually starts out as a automatically generated template file that does all the right things.
1 #!/usr/bin/make -f 2 # Debian rules file for gzip, requires the debhelper package. 3 # Crafted by Bdale Garbee, [email protected], 5 November 2000 4
The rules file is a Makefile.
5 # Comment this to turn off debhelper verbose mode. 6 export DH_VERBOSE=1 7 8 # This is the debhelper compatibility version to use. 9 export DH_COMPAT=3 10 11 CFLAGS="-g -O2 -Wall" 12
The DH_
variables modify the behavior of the debhelper programs. They are documented in the debhelper manpage, and we will skip them here as they are outside our scope. CFLAGS
is the standard Makefile variable that stores options for the C compiler.
13 configure: configure-stamp 14 configure-stamp: 15 dh_testdir 16 CFLAGS=$(CFLAGS) ./configure 17 --prefix=/usr 18 --infodir='pwd'/debian/gzip/usr/share/info 19 --mandir='pwd'/debian/gzip/usr/share/man 20 touch configure-stamp 21
Note the infodir
and mandir
settings and the difference between them and the prefix
setting in the configure
command. The prefix
must be correct so that the program will find itself and its libraries after it is installed. After the configuration is done, the file configure-stamp is created to mark the event. The dh_ programs called during the different steps in this process are debhelper programs that do various bits of setup and work. We'll just take them for granted in this book.
22 build: configure-stamp build-stamp 23 build-stamp: 24 dh_testdir 25 $(MAKE) 26 touch build-stamp 27 28 clean: 29 dh_testdir 30 dh_testroot 31 -rm -f build-stamp configure-stamp 32 make distclean || exit 0 33 dh_clean 34
Not much to say, because the operations are commonplace and are mostly stored in dh_ programs. The setup was done in the configure
target.
35 install: build 36 dh_testdir 37 dh_testroot 38 dh_clean -k 39 dh_installdirs 40 41 make install prefix=debian/gzip/usr bindir=debian/gzip/bin 42 scriptdir=debian/gzip/usr/bin 43 ln debian/gzip/bin/gzip debian/gzip/bin/uncompress 44
The make install command overrides a lot of variables to ensure that the files end up in the packaging directory hierarchy. The ln was added by the package maintainer to make sure there is an uncompress program in the package.
45 binary-indep: build install 46 47 binary-arch: build install 48 dh_testdir 49 dh_testroot 50 dh_installdocs README* TODO 51 dh_installmanpages 52 dh_installinfo gzip.info 53 dh_installchangelogs 54 dh_link 55 dh_strip 56 dh_compress 57 ln -s gunzip.1.gz debian/gzip/usr/share/man/man1/uncompress.1.gz 58 ln -s zgrep.1.gz debian/gzip/usr/share/man/man1/zegrep.1.gz 59 ln -s zgrep.1.gz debian/gzip/usr/share/man/man1/zfgrep.1.gz 60 dh_fixperms 61 # You may want to make some executables suid here. 62 dh_makeshlibs 63 dh_installdeb 64 dh_shlibdeps 65 dh_gencontrol 66 dh_md5sums 67 dh_builddeb 68 69 binary: binary-indep binary-arch
This is the whole build process. As you see, the maintainer has put some ln
commands here too. Other than those, the commands just follow a templated schema to build the package.
70 .PHONY: build clean binary-indep binary-arch binary install configure 71
This just tells make
that the enumerated targets are phony—that is, that they do not refer to files that are meant to be built. The line is not important unless somebody is ornery enough to actually create a file with the name of one of the targets.