source: project/wiki/chicken-5-roadmap-egg-system @ 36118

Last change on this file since 36118 was 33942, checked in by svnwiki, 3 years ago

Anonymous wiki edit for IP [79.241.146.144]: added link to notes

File size: 7.0 KB
Line 
1== CHICKEN 5 roadmap: Improve egg system
2
3This document is part of the [[chicken-5-roadmap|CHICKEN 5 roadmap]].
4
5[[toc:]]
6
7There is currently some work-in-progress in the [[https://code.call-cc.org/cgi-bin/gitweb.cgi?p=chicken-core.git;a=shortlog;h=refs/heads/chicken-5-new-egg-install|chicken-5-new-egg-install]]
8branch of the CHICKEN git repository. See [[chicken-5-new-egg-install-notes|this document]]
9for some very rough information about the current status.
10
11Since we may break backwards compatibility with CHICKEN 5 we could use
12the opportunity to improve the current egg system. What follows is a
13list of possible improvements, including motivation and a potential
14approach to implementing them. You are welcome to extend and comment
15on it.
16
17=== Re-introduce support for static linking
18
19==== Motivation
20
21The current system relies on the egg author to explicitly support
22static linking. This has proven to not work very well in practice and
23lead to static linking of eggs not being officially supported
24anymore. While we still have ''deployment'' as an alternative approach
25for distributing self-contained programs with egg dependencies, static
26linking remains a frequently sought-after feature. Also, some
27platforms such as iOS don't support dynamic linking at all. Hence it
28would be worthwhile to re-introduce official support for static
29linking of eggs.
30
31==== Goal
32
33Static linking should ''just work'', especially for basic extensions
34which only consist of a single module. However, it should also be
35possible to make more complex extensions statically linkable in a
36straightforward fashion. In other words, the new system should rely as
37little as possible on the egg author to explicitly support static
38linking.
39
40==== Approach
41
42One way to achieve this is to make the egg build process declarative
43instead of having a procedural {{setup}} file which requires the egg
44author to include compilation and installation of static objects. For
45example, a declarative build description for a simple single-module
46extension like {{matchable}} could consist of just a single form like
47{{(extension matchable)}}. The build system would then assume that the
48file {{matchable.scm}} defines a module named {{matchable}}, compile
49that into a static object, a shared object as well as an import
50library and finally install all of them. This build description would
51also provide ways to override these default assumptions (e.g. to pass
52special options to the compilation process).
53
54I've drafted up some ideas of what such egg declarations might look
55like in these pastes:
56[[http://paste.call-cc.org/paste?id=b3ebab1e4b1f245115d746e06b894edf8f8a9dbc]]
57--Moritz Heidkamp
58
59=== Consolidate the ''egg'' concept
60
61==== Motivation
62
63Currently the ''egg'' concept is rather ill-defined: Eggs have a
64canonical name (as declared in the central egg index, i.e. the
65{{egg-locations}} file) and version (as declared in the egg's
66{{release-info}} file) but may install various programs and extensions
67of arbitrary other names and versions (as passed to
68{{install-extension}} and {{install-program}} in the egg's {{setup}}
69file). Additional essential egg meta data (such as dependencies) are
70declared in the egg's {{meta}} file.
71
72After installation into the local repository, an egg's canonical name
73and version are lost. This leads to hacks such as "chicken-uninstall"
74removing all libraries that have the given name as a prefix (see
75[[http://bugs.call-cc.org/ticket/1093|#1093]]). Another consequence is
76that when {{chicken-install}} resolves dependency version requirements
77it will check against the versions used in the {{setup}} file rather
78than the canonical version which may differ (often by accident).
79
80==== Goal
81
82There should be fewer places where an egg's meta data live, ideally
83without any redundancy among them so as to reduce accidental
84breakage. Also, eggs should be first-class citizens in the local egg
85repository.
86
87==== Approach
88
89We could make the {{release-info}} file the only place to declare a
90version number. This already works to some extent as it is used in
91case no version is passed explicitly in the {{setup}} file. However,
92that only works when the egg is installed via henrietta. When an egg
93is installed from a local directory it ends up with a version of
94{{unknown}} which might not be ideal.
95
96Programs and extensions could not be installed with an individual name
97and version anymore but would be registered in the local egg
98repository under the egg's canonical name and version. Dependencies
99would then be resolved against these names and versions only. This
100registry would also maintain information about which files belong to
101an egg so that {{chicken-uninstall}} can cleanly remove it again.
102
103A declarative build system would blend well with this approach.
104
105
106=== Restrict capabilities of setup files
107
108==== Motivation
109
110Currently setup-files may run arbitrary Scheme code, potentially even
111as root. This is a trust issue at best and a potential security
112vulnerability at worst.
113
114==== Goal
115
116Restrict what an egg's setup file is allowed to execute to make
117{{chicken-install}} more trustworthy.
118
119==== Approach
120
121A possible approach would be to create a module that exports only the
122list of things we want to support in setup-files.  Then they will
123begin with {{(module () (use setup-files)}} and can still be executed,
124but only the whitelisted operations will be permitted.  --John Cowan
125
126There's a catch which might obviate this whole point: Since
127compilation will evaluate the syntax phase, arbitrary code may be
128executed anyway, so restricting the setup API is not enough. The only
129way I can think of to address this in a comprehensive way is to build
130eggs in a sandbox / jail / chroot / container which clearly lies
131beyond the scope of {{chicken-install}}, though. --Moritz Heidkamp
132
133
134=== Disallow version specification as numbers
135
136We should stop accepting versions as numbers, as they may cause confusion
137in some cases.  For example:
138
139  Version 0.20 is read as 0.2, which is less than 0.19, according to the
140  version comparators.  This can cause problems when chicken-install
141  takes decisions based on version numbers.
142 
143  Although (> 0.20 0.19) => #t,  (version>=? 0.20 0.19) => #f.  However,
144  (version>=? "0.20" "0.19") => #t.
145 
146  That's because versions are (read) by setup-api and tokenized using
147  `.' as separators.  If versions are numbers, they are read as numbers
148  then converted to strings, then parsed by the version API.  So, 0.20
149  is read as 0.2, converted to "0.2" and tokenized as ("0" "2").  Then,
150  converted back to numbers we have (0 2).  If we apply the same to
151  0.19, we have (> 2 19) => #f.
152 
153  By using versions as strings, we have "0.20" read as a string,
154  tokenized as ("0" "20") and converted back to numbers as (0 20).
155  Thus, (> 20 19) => #t.
156
157From [[https://github.com/ckeen/pastiche/commit/862c1d7008342230dcd4a4109376dd381f956207|https://github.com/ckeen/pastiche/commit/862c1d7008342230dcd4a4109376dd381f956207]]
158
159
160=== More notes
161
162[[https://github.com/wasamasa/dotfiles/blob/master/home/wasa/notes/chicken.org#build-system-and-packaging-work|Vasilij Schneidermann's notes on the build system and packaging]]
Note: See TracBrowser for help on using the repository browser.