Monday, January 21, 2008

Ideas for Asdf and Asdf-install

Gary King, a fellow lisper and diligent blogger started a discussion about making asdf-install better. First: I do not use asdf-install either. Before I tell you why I don't use it let me try to think about the points Gary raised.

1. It doesn't play well with Windows

Gary's first argument is that asdf-install doesn't work well with Windows, because of it's use of Unix utilities for tasks like archiving or signing/verifying. I agree that this is an issue, but nothing that couldn't be solved by just using ports of those utilities for windows.

2. It doesn't play well with itself

The second argument just describe some obvious short-comings of asdf-install. It doesn't handle updating your libraries in any way and it can only download and install tarballs. Gary is right here too; asdf-install is just not enough.

What do I think?

Gary is a smart guy and definitely is on the right track with his idea. I do not agree is his idea to make asdf-install better, but only because I think that asdf-install isn't a good base for anything. I think managing asdf-packages is too important to be left outside of lisp. I think there are enough high-quality libraries to handle the necessary tasks in common lisp and without airy callouts to shell utilities.

Well... but asdf-install is written in lisp!

Technically yes, but if you look into other general purpose lisp-libraries, you will see that a good lisp library should be flexible enough to adapt to many different use cases. A good lisp package manager needs to be extensible in at least four directions:

  1. How to find packages (cliki, cl-user.net...)
  2. How to acquire packages (e. g. tarball, zip, CVS, Darcs, SVN, git, e-mail, local folder)
  3. How to verify integrity of packages
  4. How to authenticate the identity of the supplier of a package

Finding a package should be a decentralized task. It should be possible to easily setup your own package registry. There could be different technologies used, but I think there should be at least a web-based one which doesn't demand more than some static web space. An interesting point here is, that finding a package might not necessarily be a proactive step. Maybe one could register to some typical package feeds, and as soon as a new version is announced, it will be acquired by the system. If the protocols of this component are done right, than there is no perceivable difference in finding a package somewhere in the internet or already installed on your harddrive. There should be no need for symlinks or *central-registry* hacking. The installed packages happen just to be cached on your harddrive.

There is nothing special behind the technologies to acquire the packages. One important thing is the change of notion for an asdf-package. Up to now it was actually a tarball with a certain structure. This does not hold anymore if we want to acquire packages through other channels like version control systems. Maybe an asdf-package should defined as the decompressed folder. The handling of a compressed asdf-package seems obvious to me, regardless of it being archived using tar/gz, zip or whatever.

The most obvious way to verify the integrity of an asdf-package seems to be a digest algorithm like MD5 or SHA. But if we define asdf-packages to be just folders and not tarballs anymore, we have the problem, that there is no obvious way how to compute the digest (because there is no defined order of its components). One idea may be to include a manifest-file with each asdf-package and use the order defined by this manifest to compute the digest; there may certainly be different ideas. A manifest-file could be useful for other things too though. It should be a very easy one step action to bless a package as released and create the necessary stuff like the digest and perhaps the manifest.

I've not thought very much about authentication. There are some evident solutions possible; one of it actually used already by asdf-install. The crux is to find a way which will actually be used, instead of just switched off by most users.

The package manager should understand different versions of a package. In the past the version-info was always encoded in the system-definition. I'm not sure if this is the right place for it. Regardless of this; the package manager has to handle packages with and without versions. The version-number is probably a good idea to order the different package-versions by age. But time is only one dimension in which different versions of a package could be placed. I think there should be something like a unique identifier for each version of a package and the best candidate for this would be the digest for the concrete asdf-package.

The package manager needs to handle dependencies between asdf-packages in a better way than just trying to load it and acquire the package on fail (like it is done in asdf-install). Actually this approach seems caused by a misfeature of asdf itself; there doesn't seem to be a way to ask an asdf-system for its dependencies.

My wishlist in very short: A new package-manager for asdf should:

  1. be written in common lisp
  2. be able to use different package sources
  3. be written in common lisp
  4. not mandate airy-fary use of OS-dependent stuff like symlinks
  5. be written in common lisp
  6. neither makes downloading nor finding packages a centralized task
  7. be written in common lisp
  8. have well-defined CLOS protocols for future extension
  9. and of course: It should be written in plain common lisp!

Thanks for your time! I really want to hear what you think about this ideas.

No comments:

Post a Comment