[Debian-bootstrap] debhelper seems to be quite build-essential...

[Johannes Schauer]

Hi,

Thorsten Glaser told me a while ago that for his m68k porting efforts,
he already considers debhelper as part of build-essential and has it
always installed in his chroot.

I wanted to verify that, so on top of my code that calculates how much
can be built with the packages that are cross compiled plus arch:all, I
calculated how many packages could be compiled if debhelper would also
be installable.

But first one step back:

The first thing I was doing on top of the old basenocycles.ml was to
write some code that would count which binary package was actually
involved in the most source packages' build dependencies.

For Ubuntu Natty, the most needed packages are:

package: #needed_by
---------------
autoconf: 137
gnome-pkg-tools: 145
python: 153
libx11-dev: 154
zlib1g-dev: 159
intltool: 161
cmake: 177
dpkg-dev: 180
python-support: 183
libgtk2.0-dev: 186
automake: 196
libtool: 197
libglib2.0-dev: 237
perl: 294
pkg-config: 318
autotools-dev: 372
quilt: 378
cdbs: 643
gettext: 873
debhelper: 3209

So debhelper is needed by 3209 of the 3339 source packages.

To recap, with Natty, there are only 24 source packages that can be
built on top of a minimal priority:essential plus build-essential plus
installable arch:all packages system:

authbind, db-defaults, diffutils-doc, dpatch, dput, emacsen-common,
freecdb, gawk-doc, hello, libi dent, liblockfile, linux86, m4, mbr,
procmail, sgmltools-lite, sharutils, trscripts, u-boot, ubunt u-keyring,
unzip, wdiff, zip

I thought, figuring out the package that most things depend upon would
be a good heuristic for deciding which thing to fix first.

So after finding out that debhelper indeed seems to be an important
package to have next, I used Depsolver.edos_install to figure out why
debhelper was not available.

Even though debhelper is arch:all, it has a couple of runtime
dependencies that were not satisfied. Using Depsolver.edos_install I
found that the following source packages had to be built to have all the
required binary packages to have debhelper and its runtime dependencies
available:

file, html2text, po-debconf, bsdmainutils, groff, intltool-debian,
gettext, libcroco, libxml2, glib2.0, libunistring, libpipeline, man-db

Except for gettext (which has a horrible list of build dependencies),
all those source packages require debhelper to be built.

To not complicate things too much, I chose the quick way and decided to
put all the above source packages into the list of packages that would
be cross compiled.

The other option would of course be to make them compile without
debhelper but besides the possible difficulties to do that, there would
be even more troubles with more runtime dependencies that have to be
satisfied.

So for quick verification, putting the packages for cross compilation
must be sufficient for now.

Of course in the long run we dont want to crosscompile many packages but
maybe in some cases, the trouble to make a package cross compile is less
than the trouble to make tons of packages compile without debhelper and
also add tons of install dependencies to the list of problematic
packages (which also have to be built without debhelper).

So maybe in case of debhelper, crosscompiling the above 13 packages is
the best option? But the final decision on that is of course not taken
by me but by someone else.

I now use the file add-cross-sources.list to list source packages that
are also listed to be cross built. The minimal last of cross built
packages is still stored in min-cross-sources.list. Every package that
the user also picks to cross build has to be added to
add-cross-sources.list.

So I added the 13 source packages to add-cross-sources.list and ran the
program again to see how many packages could no be built before running
into cyclic build dependency issues.

So when debhelper is available in addition to the essential and
build-essential packages and installable arch:all packages, the
following amount of packages can be built per round:

round 0: 1393
round 1: 135
round 2: 81
round 3: 6
round 4: 10

After each round it is checked whether more arch:all packages or binary
packages produced by the already compile-able packages can be installed
to compile the maximum amount of source packages per round.

So the availability of debhelper magically lets 1625 out of 3339 be
natively compiled.

If now checking again which binary package's unavailability blocks the
most source packages from being unbuildable, this list is produced:

gnome-pkg-tools: 144
python: 152
libx11-dev: 153
intltool: 160
cmake: 177
automake: 178
python-support: 181
libgtk2.0-dev: 186
libglib2.0-dev: 235
quilt: 298
pkg-config: 316
cdbs: 641

Where gnome-pkg-tools, intltool, automake, python-support, glib2.0 and
quilt (and indirectly pkg-config, gtk+2.0 (both depend on
libglib2.0-dev) and cmake (depends on qt depends on quilt) and libx11
(depends on pkg-config)) all need cdbs.

So cdbs seems to be the next obvious choice.

I will look into cdbs tomorrow.

Having staged build dependencies would maybe be the better choice to get
debhelper and cdbs. I dont know but maybe somebody can tell me how
easy/hard it is to build a couple of packages (especially the ones for
debhelper listed above) without debhelper/cdbs. Crosscompilation should
be the last option but right now for simplicities sake and just to
explore where the problems lie, I limit myself to simple
crosscompilation of "some" packages. I use it to get a better
understanding of how to approach the problem. This is okay and not a
waste of time, right? I mean I figured the results of above email out
within a day so it seems quick enough to be worthwhile for the learning
effect?

So tomorrow I want to extend add-cross-sources.list a bit more so that
also cdbs is available in the system and see how far I get with that and
how much is still left for Natty.

There is one thing though that I didnt manage to fix yet. When I want to
compile the list of how many remaining source packages are in need of a
binary package, I do the following:

I go through the list of all source packages that have not been built
yet. For each source package I get the vpkg list of dependencies and
flatten/concatenate them together. Then I want to filter out all vpkgs
that are already available. In my final list I only want to have those
binary packages listed that are NOT available as of now. I check if a
vpkg requirement is satisfied by my universe using Cudf.mem_installed.
This is the proper way to check if a vpkg is satisfied by a universe,
right? Because in my code, Cudf.mem_installed ALWAYS returns false. I
will look into that problem more detailed tomorrow.

So far for today...

cheers, josch