The arrow of time

Ivan Voras' blog

Notes on pkg_patch

My Google Summer of Code project for 2010 is "pkg_patch" - a utility and a framework for working with binary patches to FreeBSD binary packages. It is necessary because the default way of dealing with 3rd party applications / ports in most BSD systems is to build them from source via various frameworks like ports and pkgsrc, and so far the binary package side has been mostly neglected, at least in FreeBSD.

I strongly feel that the only thing that will help the adoption of binary packages (by making them useful) is to introduce the notion of "stable" ports but since I've repeatedly been told of the uselessness of banging on this particular drum, I'll leave it alone for others.

Anyway, most aspects of pkg_patch are described in the FreeBSD wiki, and these blog posts will deal with daily and miscellaneous information.

Current work-in-progress is available at the FreeBSD p4 repository (the official version) and also temporarily at my private Mercurial repository which can be accessed with:

hg clone http://cosmos.boldlygoingnowhere.org/~ivoras/soc2010

This Mercurial repository will probably not be always available but will probably contain the most recent version until pkg_patch is imported into HEAD) Because the pkg_install module of FreeBSD is different in HEAD and in 8-stable, this code only builds on HEAD.

The first milestone of the project is complete - the part that generates patch packages to upgrade from one version to the next. It is simple in operation - it takes two binary packages (A and B) as input and generates a patch package on the output. By default it just stores files that have changed but it can optionally use bsdiff to create binary patches for these files.

The second milestone is about applying this patch package to a live system with installed package version A to get to the package version B.

So far, I'm laughing at how package metadata is stored for installed packages and am feeling there is more luck than common sense involved in that there are not more problems with the whole thing than there currently are in production. And I've seen all types of problems ranging from dependancy problems (practically an everyday issue in all FreeBSD installations - not because the dependancy information is missing, but because the lack of "stable" tracking makes it less useful), missing package files to missing whole directories under pkg/. Unfortunately, as my other proposal of making the package database an ACID-compliant thing was also refused, I'm also leaving it to be Somebody Else's Problem.

The second milestone is scheduled to be implemented sometime next week, and I think there is good chance that it will be on schedule.

#1 Re: Notes on pkg_patch

Added on 2010-07-03T18:25 by GTodd

This is a good pithy summary of the state of ports/packages in FreeBSD :-)  There's lots of flexibility for power users to fix problems errm created by the flexibility heh heh.

 

Can or will pkg_patch/pkg_snap actually make it easier to *create* STABLE sets of packages?  For example if a release ships with a set of binary packages could a STABLE package set be maintained using binary diffs against that set of packages?  A successuf weekly(daily) run of portsnap and package build on STABLE system could then also produce everything pkg_patch would need to update a set of binaries.  Even if there' was no agreement that the FreeBSD project should do this, it would be useful if it were possible at local sites since it would facilitate tracking and logging a "snaphot" of locally patched binaries against a particular portsnap snapshot.  This is useful for audit DSL/ITIL etc.  I know it's possible to do this with current tools but I'm wondering if your project will make it easier.

Cheers and thanks for your informative blog and your work on FreeBSD


ps: Another sort of related question/comment: I really like pkg_trans and where it seemed to be going (re: audit, tracking changes etc). Will you continue with that work as well?

Post your comment here!

Your name:
Comment title:
Text:
Type "xxx" here:

Comments are subject to moderation and will be deleted if deemed inappropriate. All content is © Ivan Voras. Comments are owned by their authors... who agree to basically surrender all rights by publishing them here :)