source: project/wiki/releasing-your-egg @ 39085

== Releasing your egg

[[toc:]]

Let's say you've been reading the [[eggs tutorial]] and you are almost
done writing your first egg.  You've tested that it works locally, and
now you probably want to make it available to others.  If you would
like people to be able to install your egg using {{chicken-install YOUR-EGG}},
there are a few steps you need to take.

=== Note for Subversion users

Before going on details on the process for Subversion,
please note that hosting '''new''' eggs in the '''CHICKEN central
Subversion repository''' is '''deprecated'''.  We recommend using
your own code server or one of the hosts available on the Internet
(e.g., github, bitbucket etc.).

If you host your egg on {{code.call-cc.org}} or you host your own,
you don't ''have'' to do any of the steps below.  Instead, you can
use the release generating script at call-cc.org.  It does all the
steps below automatically.

The URI for an automatically generated release-info for egg YOUR-EGG is
{{http://code.call-cc.org/release-info?egg=YOUR-EGG}}

This URI can be put in the {{egg-locations}} file as described under
[[#publishing-your-egg|"Publishing your egg"]].

If you want more control over the process, please read on.

=== Creating a release-info file

First, you must create a so-called "release-info" file and make it
available on a well-known location via HTTP (discussed below).  This
is a file which describes where the released versions of your egg can
be found.

After everything's set up, releasing a new version of your egg is
simply a matter of tagging your release with your VCS of choice and
adding a line to this file. See [[#helper-scripts|below]] for some tools
to automate even this small step.

It looks something like this:

<enscript highlight="scheme">
(repo git "git://example.com/{egg-name}.git")   ; optional

(uri targz "http://example.com/{egg-name}/releases/{chicken-release}/{egg-name}-{egg-release}.tar.gz")
(release "0.1")
(release "0.2")
(release "1.0")
</enscript>

This example describes where to find the canonical repository and an URI pattern
which describes how to find release tarballs.  It then goes on to declare three
official releases; 0.1, 0.2 and 1.0.  Please note that releases are not supposed
to be removed from the .release-info file, as it would cause consistency issues
in case we want to rebuild caches.

The patterns in the URI enclosed in curly braces are seen as
substitution patterns to be replaced by values.  {{egg-name}} expands
to the current egg's name (which is already known when the
release-info file is being fetched), {{egg-release}} is replaced by
the string in each {{release}} declaration.  {{chicken-release}} is
replaced by the major CHICKEN version for which the egg is being
fetched.  This allows you to supply different packages for each major
CHICKEN release.

The supported types for {{uri}} are currently {{targz}}, {{tarbz2}}, {{zip}},
{{meta-file}} and {{files-list}}.  The first three are all archives expected
to extract to a directory with the egg sources in it (zip files are allowed
to expand directly into the files, but this is not recommended).  The latter
two are special and deserves some more explanation; see the next section for that.

Currently {{repo}} is not used by automated tools, but is intended to
make it easy for users to find the repository.  In preparation for
when it will be used in the future, it's a good idea to use consistent
names, so please use the ''short name'' of the tool.  Generally this is
how it is invoked on the commandline: {{git}}, {{hg}}, {{svn}}, {{fossil}}, {{bzr}}, {{mtn}}, {{darcs}} etc.

==== Meta-file distribution

Sometimes it is impractical to create archives containing your egg's file
contents.  The egg releasing system has been designed to require as few manual
steps as possible, so it is easy for people to release early and often.

A core ideal of the release system is to make it possible to release
simply by tagging your code in a VCS.  Some code hosting sites
automatically make archives available for each tagged version, thereby
making the release available immediately when the code is pushed to
the server.  For those that don't, the meta-file distribution file
type is a way to release by tagging ''without'' having to manually
make a tarball, ''as long as there is a way to directly obtain the raw
sources of a specific release version via HTTP''.

The idea is that each released egg always must have a meta-file to describe
its dependencies, its author and license and so on.  This meta-file can be
used to provide a listing of all the files that are part of an egg.  The system
that manages egg releases automatically will download all these files and put
them in a directory.  When someone requests your egg with {{chicken-install}},
these files are all served up.  Just add a {{files}} entry to your meta-file,
which lists the files (these are resolved relatively to the meta-file itself).

<enscript highlight="scheme">
((files "uri-common.setup" "uri-common.release-info" "uri-common.meta"
        "uri-common.scm" "tests/run.scm")
 (license "BSD")
 (category web)
 (depends (uri-generic 2.3) defstruct matchable)
 (test-depends test)
 (author "Peter Bex")
 (synopsis "Parser for common URI schemes"))
</enscript>

This is a real-world example of the [[/eggref/4/uri-common|uri-common egg]].
It lists all the files which it consists of, and these will be downloaded by
chicken-install.  The disadvantage of this approach is that if you forget a
file, your egg is uninstallable, so if you can please use the archive
distribution types instead.  Another disadvantage is that every time you add,
remove or rename a file you need to remember to change the meta-file.

The {{uri}} entry in uri-common's {{release-info}} file looked like this
before there was an autogenerated one:

<enscript highlight="scheme">
(uri meta-file "http://anonymous@code.call-cc.org/svn/chicken-eggs/release/{chicken-release}/{egg-name}/tags/{egg-release}/{egg-name}.meta")
</enscript>

Because this egg uses subversion and each release has a corresponding
tag, it can simply point to the metafile in the right subdirectory
under tags and it will simply work.  For other version systems this
may require some more messing around.

==== files-list

This is an extremely stupid format containing a manifest of files in
an egg.  It's mostly useful when automatically generating lists of files.
It starts with a base URI and then lists all the files in the egg.
Here's the [[http://code.call-cc.org/files-list?egg=uri-generic;egg-release=1.0;chicken-release=4|files-list for version 1.0 of the uri-generic egg for CHICKEN 4]]:

  http://anonymous:@code.call-cc.org/svn/chicken-eggs/release/4/uri-generic/tags/1.0
  tests/run.scm
  uri-generic.meta
  uri-generic.scm
  uri-generic.setup

The trailing slash after the URI is required.

It's not recommended you use this manually, as this format is subject
to change and only really intended to be used as a communication system
between [[/eggref/4/pseudo-meta-egg-info|pseudo-meta-egg-info]] and
[[/eggref/4/henrietta-cache|henrietta-cache]].

However, if you really need to use this, you can replace the
{{meta-file}} uri type in the example above by {{files-list}}, like
the auto-generated
[[http://code.call-cc.org/release-info?egg=uri-generic|release-info file for uri-generic]],
which starts like this:

  (uri files-list "http://code.call-cc.org/files-list?egg={egg-name};egg-release={egg-release};chicken-release={chicken-release}")
  (release "1.0")
  (release "1.1")

=== Publishing your egg

After creating the release-info file, you need to make it known to the
chicken-install infrastructure that an egg with the given name has a
release info file at the location where you published it.  This step finally
makes it possible for people to say {{chicken-install YOUR-EGG}} and have
it install your egg, or use {{(depends YOUR-EGG)}} in meta-files of their eggs.

Currently this is done by adding a line to the egg-locations file in
Subversion, which can be found at
[[https://code.call-cc.org/svn/chicken-eggs/release/4/egg-locations]]. For security reasons, editing the {{egg-locations}} file requires special permissions.  So, to have an entry for your egg in that file, send a message to the [[http://lists.nongnu.org/mailman/listinfo/chicken-users|chicken-users mailing list]] announcing your egg and requesting the inclusion of its location to {{egg-locations}}.

To keep maintenance to a minimum, it's best to add a release-info
location which will never change; only the release-info file itself
should be changed when you make a new release.  This can most easily
be accomplished by pointing to the "raw file" view of your
trunk/tip/head/master branch in your canonical repository's web
interface.  That way, this file is kept under version control
alongside all the other files in your egg.

If you do have an svn account and don't want to checkout the whole egg
repository to be able to just edit this file, you can work around svn's
limitations like this:

 $ cd /tmp
 $ svn co https://code.call-cc.org/svn/chicken-eggs/release/4 --depth empty
 $ cd 4
 $ svn update egg-locations

If you prefer, you can just publish the file separately via HTTP
somewhere and keep overwriting it.  If your code hoster doesn't
provide an easy way to point to a raw view on a moving "latest
version" pointer via HTTP, you could instead opt to store just the
release-info files of your eggs in the centralised CHICKEN eggs
subversion repository.  Just contact one of the project maintainers or
write to the chicken-users mailinglist.

=== Instructions for popular code hosting methods and VCSes

Now we've explained the basic idea, here's an overview of how to figure
out the correct URIs and how to automate some steps away.

==== Bitbucket (mercurial, git)

===== Location of release-info file

You can use the code browser to figure out the path to your
release-info file.  First, copy the link that says "raw".  This URI is
almost correct but will contain a revision ID hash, so it is pinned to
whatever revision is currently the tip.  However, you can replace it
with the string "tip" and it will still work, and when you visit it again
after making changes it will have picked up those changes.  Example:

Clicking "mini-kanren.release-info" and then copying the "raw" link on
[[https://bitbucket.org/ThatGeoGuy/chicken-minikanren/src]] gives us
[[https://bitbucket.org/ThatGeoGuy/chicken-minikanren/raw/8af0b51f5efac79c8bc0cfe856185ad06d7aa9d0/mini-kanren.release-info]]

Just change it to
[[https://bitbucket.org/ThatGeoGuy/chicken-minikanren/raw/master/mini-kanren.release-info]]
and you have the latest master version.  If you're using Mercurial
instead of git, use {{tip}} instead of {{master}}.

This link should be registered in the egg-locations list for your egg
and it will automatically be able to fetch any new releases as they
are tagged and added to the release-info file.

===== Making releases

The bitbucket code browser also offers a "get source" link that allows
you to fetch the files in that revision as a tarball.  The same trick
as with the raw files works here; just replace the link's revision ID
hash with a symbolic name.  You can use tags and bookmarks as symbolic
names.

So to make a new release, just tag your release with a well-defined
name. If you tag your eggs with the version as tag or bookmark name,
you can use the following release-info file.  Just don't forget to
substitute your bitbucket username!

<enscript highlight="scheme">
(repo hg "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}")
(uri targz "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}/get/{egg-release}.tar.gz")
(release "0.1")
</enscript>

==== GitLab (git)

===== Location of release-info file

Use the code browser ("Files" in the menu) to browse to your master
branch's release-info file and copy the "Raw" link.  For example, for
the [[/eggref/4/sdl2|SDL2 egg]] it looks like this:

[[https://gitlab.com/chicken-sdl2/chicken-sdl2/raw/master/sdl2.release-info]]

===== Making releases

GitLab makes tags available for download as tarballs.  To find the
location, go to the code browser and switch to a tag at the top left.
On the right side there's a drop-down menu that offers a download for
various archive formats.

So to make a new release, just tag your release with a well-defined
name. If you're using gitlab.com's hosted service and tag your eggs
with the version as tag name, you can use the following release-info
file.  Just don't forget to substitute your GitLab project name!

<enscript highlight="scheme">
(repo git "https://gitlab.com/PROJECT-NAME/{egg-name}.git")
(uri targz "https://gitlab.com/PROJECT-NAME/{egg-name}/repository/archive.tar.gz?ref={egg-release}")
(release "0.1")
</enscript>


==== GitHub (git)

===== Location of release-info file

Use the code browser ("source" tab) to browse to your release-info
file and copy the "raw" link.  This link should work as-is, as long as
you do this while looking at the latest revision of the file in the
master branch.  For example, for the [[/eggref/4/kalaha|Kalaha egg]]
it would look like:

[[https://raw.github.com/DerGuteMoritz/kalaha/master/kalaha.release-info]]

===== Making releases

GitHub makes tags available for download as tarballs.  To find the
location, click the big "Downloads" button/link in the code browser.
It will pop up a selection dialog where you can choose between tarball
and "zipball".  It will also offer downloads for each tag, but those
are zipballs only.  However, you can copy the link and replace
"zipball" in the URL by "tarball", or you can first click "switch
tags" and then open the "Downloads" dialog and it will offer you
download links for both tar and zip for that specific tag.

So to make a new release, just tag your release with a well-defined
name. If you tag your eggs with the version as tag name, you can use
the following release-info file.  Just don't forget to substitute your
GitHub username!

<enscript highlight="scheme">
(repo git "git://github.com/YOUR-GITHUB-USERNAME/{egg-name}.git")
(uri targz "https://codeload.github.com/YOUR-GITHUB-USERNAME/{egg-name}/tar.gz/{egg-release}")
(release "0.1")
</enscript>


==== Gitweb (git)

===== Location of release-info file

Navigate to the "tree" view for the master branch of your project and
copy the "raw" link for your release-info file. The link will look
something like this, for an egg "foo":

 http://gitweb/gitweb.cgi?p=foo.git;a=blob_plain;f=foo.release-info

===== Making releases

The download link for a tarball of a specific Git tag is accessible
through Gitweb with a query string like the following:

 http://gitweb/gitweb.cgi?p={egg-name}.git;a=snapshot;h=refs/tags/{egg-release};sf=tgz

So, if you tag each release and use a release-info file like the
following, every new tag will be made available as a tarball at that URL
(make sure to substitute your Git URL and Gitweb host into {{repo}}
and {{uri}} strings):

<enscript highlight="scheme">
(repo git "git://gitweb/{egg-name}.git")
(uri targz "http://gitweb/gitweb.cgi?p={egg-name}.git;a=snapshot;h=refs/tags/{egg-release};sf=tgz")
(release "0.1")
</enscript>


==== Sourcehut (mercurial, git)

===== Location of release-info file

For a Git repository, navigate to the "tree" view for the master branch
of your project, select your release-info file and copy the "View raw"
link from that page. The URL will look something like this, for user
"user" and egg "example":

 https://git.sr.ht/~user/example/blob/master/release-info

For a Mercurial repository, go to your project's "browse" tab, select
your release-info file and copy the "View raw" link. The URL should look
like this:

 https://hg.sr.ht/~user/example/raw/release-info

===== Making releases

Sourcehut provides download URLs for tags in both Mercurial and Git
repositories. So, in either case you can make a new release by simply
tagging your egg.

If you use version numbers as tag names, you can use the following
release-info files, replacing "USERNAME" with your own. Note the leading
tilde, which must be included!

For Git:

<enscript highlight="scheme">
(repo git "https://git.sr.ht/~USERNAME/{egg-name}")
(uri targz "https://git.sr.ht/~USERNAME/{egg-name}/archive/{egg-release}.tar.gz")
(release "0.1")
</enscript>

For Mercurial:

<enscript highlight="scheme">
(repo git "https://hg.sr.ht/~USERNAME/{egg-name}")
(uri targz "https://hg.sr.ht/~USERNAME/{egg-name}/archive/{egg-release}.tar.gz")
(release "0.1")
</enscript>


==== Cgit (git)

===== Location of release-info file

Navigate to the "tree" view for the master branch of your project and
copy the "plain" link for your release-info file (it's all the way to
the right). The link will look something like this, for an egg "foo":

 http://cgit.example.com/foo/plain/foo.release-info

For example, the release info for the [[/eggref/4/hardwood|Hardwood
egg]] looks like this:

[[http://www.upyum.com/cgit.cgi/hardwood/plain/hardwood.release-info]]

The {{cgit.cgi}} part is optional, depending on how cgit was
configured.  For example, the release-info for the
[[/eggref/4/scsh-process|scsh-process egg]] looks like this:

[[http://code.more-magic.net/scsh-process/plain/scsh-process.release-info]]

===== Making releases

The download link for a tarball of a specific Git tag can be found by
visiting "refs", then the tag, following it to the commit and if
snapshots are shown there should be a download link, but it links
directly to the commit hash.  You can simply replace the hash with the
symbolic tag name.

If snapshots are not shown, it's still possible to construct the link
manually, and it will work:

 http://cgit.example.com/cgit.cgi/{egg-name}/snapshot/{egg-name}-{egg-release}.tar.gz

So, if you tag each release and use a release-info file like the
following, every new tag will be made available as a tarball at that
URL (make sure to substitute your Git URL and Cgit host into
{{repo}} and {{uri}} strings):

<enscript highlight="scheme">
(repo git "http://cgit.example.com/cgit.cgi/{egg-name}")
(uri targz "http://cgit.example.com/{egg-name}/snapshot/{egg-name}-{egg-release}.tar.gz")
(release "0.1")
</enscript>

==== hgweb (mercurial) - aka "hg serve"

The built-in web interface for mercurial can be used to serve up eggs,
usually invoked from hgweb and hgwebdir on a "real" server.

===== Location of release-info file

Use the browser to navigate to the latest revision of the release-info
file and click the "raw" link on the left.  Then replace the commit ID
hash with "tip".  Example:

[[http://example.com/my-egg/raw-file/tip/my-egg.release-info]]

===== Making releases

Since this interface doesn't have a way to serve up tarballs, you must
either create your own tarballs and publish them elsewhere or use the
manual "meta-file" way.

See [[#meta-file-distribution|the meta-file section]] at the start of
this wiki page to figure out how to set up a {{files}} section in your
meta-file.  The URI to your meta file is similar to the release-info
file, except this time you can replace "tip" with the release's
tagname.  If you use tagnames that are identical to the release
version, you can base the release-info file on the following example:

<enscript highlight="scheme">
(repo hg "http://example.com/{egg-name}")
(uri meta-file "http://example.com/{egg-name}/raw-file/{egg-release}/{egg-name}.meta")
(release "0.1")
</enscript>

This assumes hgwebdir is running on the domain's root URI and the egg
is available in a repo which has the same name as your egg.

==== Fossil's default web UI

This is the web UI you get when running "fossil serve" or when running
fossil as a CGI script.

There is one tricky part: by default, this interface disallows almost
all outside access; you ''must'' log in, even if you just want to
browse anonymously. To allow everybody to download release zipfiles or
tarballs without login, add the "z" permission for the user "nobody"
in the Admin / Users section of the web UI or with the "fossil user
capabilities" command.

===== Location of release-info file

Fossil's web UI allows serving of documents directly from the
repository under URLs of the form
"http://example.com/my-egg/doc/{version}/{file}". If the files have a
".txt" or ".wiki" extension, they are rendered into HTML, but anything
else is served as is.

For example you can store the release-info file in the repository and
always access the latest version of the default branch at a URL like
"http://example.com/my-egg/doc/trunk/my-egg.release-info".

===== Making releases

Fossil can serve tarballs or zipfiles of repository snapshots. A
release-info file using zipfile packaging may look like this:

<enscript highlight="scheme">
(repo fossil "http://example.com/{egg-name}")
(uri zip "http://example.com/{egg-name}/zip/{egg-name}.zip?uuid=v{egg-release}")
(release "1.0.0")
</enscript>

For tarball packaging, the equivalent {{uri}} entry should look like
this:

<enscript highlight="scheme">
(uri targz "http://example.com/{egg-name}/tarball/{egg-name}.tar.gz?uuid=v{egg-release}")
</enscript>

The releases are referenced using symbolic tags of the form
"v{egg-release}" here. You can add these tags to the changesets
corresponding to released versions of the egg using the "fossil tag
add" command.

==== Launchpad (bazaar)

Unfortunately, Launchpad doesn't currently seem very suitable for
developing eggs in a streamlined way.  Here's a description of one
possible way to integrate with their way of doing release management though.

===== Location of release-info file

You can browse to the release-info file using Loggerhead (the web
interface to bazaar), but there doesn't seem to be a reliable way to
construct a raw download URI for "the latest version of" a file.

A workaround is to create a release series just for the release-info,
and make an (unused) milestone for that.  Then you can upload a
release-info file.  The URI that's listed directly on the downloads
page is stable though (something like
[[http://launchpad.net/my-project/main/release-info/+download/my-egg.release-info]]
if you named the milestone "release-info" in the "main" series)
This link redirects to an unstable URI, so don't paste that one!

When you make a new release you can ''delete'' the file and upload a new
one manually to the same pseudo-"release".


===== Making releases

If you do it through their release management, you just create release
tarballs which you manually upload into the release milestone.

The only automation here is that you can use {{bzr export --format=tgz}}
to create it, and the advantage that you're using someone else's site to
host your code.

You can use this release-info file template if you are consistent in
your milestone naming.  It assumes you used the name "main" for the
series from which releases for chicken-install are made.  You could
alternatively add several {{uri}} entries, one per series, but it's
still required that your egg's release versions are unique for your
egg.  Series are probably more suitable for distinguishing between
major CHICKEN release versions.

<enscript highlight="scheme">
(repo bzr "lp:MY-PROJECT/main")

(uri targz "http://launchpad.net/MY-PROJECT/main/{egg-release}/+download/{egg-name}.tar.gz")
(release "0.1")
(release "0.2")
</enscript>


=== Helper scripts

To make life easier, there are some helper scripts, hooks etcetera
that could be used.  Please add your own links here.

==== Subversion

There's an [[/eggref/4/svn-egg-author|svn-egg-author egg]] that helps
you to update the release-info and meta files and tag a release of
your egg, all with one command.

If you run your own subversion server with Spiffy, you can consider
using [[/eggref/4/pseudo-meta-egg-info|pseudo-meta-egg-info]] instead.
If you do, you do not need to use svn-egg-author.  If you're using the
chicken-eggs repository hosted on call-cc.org, you don't need it either
because call-cc is already running pseudo-meta-egg-info for all its eggs.

==== Mercurial

There's an
[[http://code.more-magic.net/hg-egg-author/plain/egg-author.py|egg-author]]
extension for Mercurial that allows you to say {{hg eggtag 0.1}} and have it
automatically update the release-info file with the new tag name.

==== Git

There's a [[/eggref/4/git-egg-author|git-egg-author egg]] that helps you
keep your release-info file up-to-date and tag a release with one command.

==== Bazaar

There's also a [[https://launchpad.net/bzr-egg-author|bzr-egg-author]]
extension for Bazaar that allows you to say {{bzr eggtag 0.1}} and have it
automatically update the release-info file with the new tag name.

=== Moving an egg

When you decide to migrate to a different code hosting site, and don't
want to move all releases, you can define multiple uris:

<enscript highlight="scheme">
;; Main ("default") repo
(repo hg "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}")
(uri targz "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}/get/{egg-release}.tar.gz")
(release "2.1")
(release "2.0")

;; Old repo, using "old-svn-repo" alias
(repo svn "http://anonymous:@code.call-cc.org/svn/chicken-eggs/release/4/{egg-name}")
(uri files-list "http://code.call-cc.org/files-list?egg={egg-name};egg-release={egg-release};chicken-release={chicken-release}" old-svn-repo)
(release "1.3" old-svn-repo)
(release "1.2" old-svn-repo)
(release "1.1" old-svn-repo)
(release "1.0" old-svn-repo)
(release "0.2" old-svn-repo)
(release "0.1" old-svn-repo)
</enscript>


=== Testing new eggs before publishing them

The initial setup of a new egg is relatively tricky.  Some details have to be
properly configured before the official release.  For example:

* The .release-info file must be publicly available, so that [[/egg/henrietta-cache|henrietta-cache]] can access it.
* The syntax of the .release-info file must be correct, and the releases and tarballs it points to must be consistent.
* The egg code must be compilable and installable.  Usually you test it from the local source code repository, which can be in a different state from the remote source repository -- that leads to problems when making the egg available.
* The egg metadata must be properly specified (e.g., dependencies) -- that's another thing that can cause problems if the egg is tested with the CHICKEN installation that is used for development (on which environment the dependencies are already installed, thus missing specification of dependencies go unnoticed).

Manually testing all these aspects can be boring and prone to mistakes.  To make the life of egg authors easier, we have a tool to test initial versions of eggs: [[/egg/test-new-egg|test-new-egg]].  Using it is very easy: just install it (via chicken-install) and provide the URI for the .release-info file as argument to it:

  $ test-new-egg <egg name> <.release-info file's URI>