There has been some discussion lately about if and how to "revamp" the ports system to make it more usable by general users, which was in part started by a post on PostgreSQL.org. Unfortunately there has been very little feedback from users themselves - which is probably a mistake, but also - there was very little feedback from the population (not a particularily small one) that is the cross-section of users and developers. Some ideas were presented, but at the end it all started revolving around banding the gaps and smaller improvements that will, I think, be practically invisible to the end-users.
I'd like to present some of my ideas here.
The biggest single idea I was hoping to get traction is the necessity of introducing a "stable" ports branch, which doesn't have to include all ports but for starts some more-than-minimal subset (e.g. the "FAPP" set). The main reasons for this are:
- Introducing ABI stability between the ports - so if some popular library changes (like libjpeg, libpng, glib), hundreds of ports on a working production system don't have to be recompiled. (in the defense of the current system - this recompiling works and is automated)
- Introducing binary packages and faster patch time - because maintaining binary packages where everything changes after such an ABI breakage is next to useless.
Unfortunately, the general feeling is that this cannot be done because of infrastructure limitations and because of port maintainers' time - they would be hard pressed to maintain separate sets of ports (the current "HEAD" branch and one or more "stable" branches). I see the infrastructure limitations can be serious but these are merely technical problems - they can be solved automatically one way or the other. The second problem - new demands on ports maintainers' time - is more significant.
I'm not a big ports maintainer - I semi-maintain only two ports - but I don't think it would be particularily hard to maintain a stable branch of the ports tree because of one simple reason: the porting work is already done and "maintaining" the stable branch mostly revolves around updating the ports with newer distfiles (source files), where applicable. Of course there are edge cases - such as that for security reasons some ports (shared libraries mostly) simply *must* break ABI, but I think this could be solved by at least "borrowing" patches and ideas from similar systems that have such a "stable packages" idea successfuly implemented (e.g. most of the big Linux distributions).
To test this, and to prove (mostly to myself) it would be doable, I've proposed a Google Summer of Code project. This is probably the last time I'd be eligible for it. Here's what I proposed:
A frequently requested FreeBSD feature is a more stable ports tree and a binary package distribution mechanism. This proposal aims to offer at least an experimental implementation of both, in a way that offers user a subset of available ports (from the ports tree) in an ABI-stable way which will also be maintained for at least the period of 12 months from the SoC project start, together with their binary packages and a rsync-based distribution method for both. The proposed technologies for building this infrastructure include reliance on existing FreeBSD/unix tools and libraries, with Lua as the glue language (the specifics can be determined later, before the SoC project starts).
The results of this project would be twofold: an infrastructure that could be used for this purpose in the future (with the possibility of import into the base system) if accepted by the community of developers, and the documented experience, made available via a blog, a wiki or similar communication or documentation infrastructure, of the good and bad sides of creating and using such a system. Since the results of this project will be made public ASAP, and it would be a fairly high-profile project, it is expected that users will participate by giving feedback.
A selection of packages will be made from the current ports tree (the everchanging ports' "HEAD"), which will be maintained as close as possible to an an "overlay tree" for the ports tree, with the distinction that the overlay tree will be self-contained in terms of dependencies and will be maintained in an ABI-stable way. The primary goal of maintaining this "stable" tree is to provide users with patches to the ported applications without causing ABI or shared libs breakage, which are currently very visible with the volatile nature of the ports tree. For the sake of possible future development, this "stable" tree will currently be marked as intended for the FreeBSD 8.x series (future possible versions will probably introduce ABI breakage but will be marked for a future version of FreeBSD). The secondary goal of this proposal is to create a robust method of distributing binary packages, possibly by using binary diffs, which will speed up distribution and patching. The binary packages will be built on the earliest supported FreeBSD release from the series.
In no case will there be attempts to circumvent, replace or disable the usual ports tree and binary packages. Full compatibility (with regards to the ports and build infrastructure) between the "stable" tree and the regular ports tree will be maintained. The "stable" tree ports will be updated from the regular tree ports as long as there are no ABI or shared libs inconsistencies between them. Unless an unforseen situation arises, binary packages will be the regular binary packages, compatible with pkg_add et al.
This "stable" ports tree will be maintained for a time period which effectively includes 9-10 months after the SoC project officially ends.
I propose that the selection of ports to maintain in an ABI-stable way consists of (or a superset of): apache22, php5-*, perl, python, sqlite3, postgresql84-*, mysql55-* and their dependencies. The starting points of these ports will be maintained as they are/were when the stable tree "branches" from the regular ports tree. It is possible that some build options (make config) will differ from the ports' defaults for the binary packages, in which case they will be documented and maintained. As is now, users can choose to follow binary packages or build ports from source, with reasonable compatibility between them. It is outside the scope of this project to try to split ports which build or install multiple entities (development libraries, binaries, other resources) into separate ports, or do any such "surgery" on existing ports. Unless the project's scope changes (which would require public discussion and consensus), the system's port database format (/var/db/pkg et al) will not be modified.
A single user-accessible utility will be created to manage the entire client side of the infrastructure, including updating the "stable" ports tree, installing, removing and updating ports and packages. The project/maintainer side might include multiple utilities, as needed.
Hosting for the projects' results (ports, packages, etc.) will be either in the FreeBSD infrastructure or (at least during the initial SoC timeframe) provided by me.
Benefits for FreeBSD:
Possibly the greatest benefits from the FreeBSD project will be the experience and feedback in creating and maintaining this experimental infrastructure, including the evaluation of how hard it is to maintain such infrastructure (and the stable branch), recorded for the purpose of being used by the developers. The entire infrastructure will be created and maintained with the possibility of its inclusion in the FreeBSD base system, if the developer community agrees on it and sufficient port maintainers sign up for the effort.
Of course, absolute stability is not possible - even the systems with very rigid rules (like Debian) occasionally produce bad "stable" packages - we're all human. To err is human, but to have a VCS history of past code to slide to is what computers are for.