source: project/chicken/branches/inlining/manual/Extensions @ 15323

Last change on this file since 15323 was 15323, checked in by felix winkelmann, 10 years ago

more intelligent inlining; standard-extension procedure in setup-api

File size: 20.0 KB
Line 
1[[tags: manual]]
2[[toc:]]
3
4== Extensions
5
6=== Extension libraries
7
8Extension libraries (''eggs'') are extensions to the core
9functionality provided by the basic CHICKEN system, to be built and
10installed separately.  The mechanism for loading compiled extensions
11is based on dynamically loadable code and as such is only available on
12systems on which loading compiled code at runtime is
13supported. Currently these are most UNIX-compatible platforms that
14provide the {{libdl}} functionality like Linux, Solaris, BSD, Mac OS X
15and Windows using Cygwin.
16
17Note: Extension may also be normal applications or shell scripts, but
18are usually libraries.
19
20Extensions are technically nothing but dynamically loadable compiled
21files with added meta-data that describes dependencies to other
22extensions, version information and things like the author/maintainer
23of the extension. Three tools provide an easy to use interface for
24installing extensions, removing them and querying the current
25status of installed extensions.
26
27
28=== Installing extensions
29
30To install an extension library, run the {{chicken-install}} program
31with the extension name as argument. The extension archive is
32downloaded, its contents extracted and the contained ''setup'' script
33is executed. This setup script is a normal Scheme source file, which
34will be interpreted by {{chicken-install}}. The complete language
35supported by {{csi}} is available, and the library units {{srfi-1
36regex utils posix tcp}} are loaded. Additional libraries can be loaded
37at run-time.
38
39The setup script should perform all necessary steps to build the new
40library (or application). After a successful build, the extension can
41be installed by invoking one of the procedures {{install-extension}},
42{{install-program}} or {{install-script}}.  These procedures will copy
43a number of given files into the local extension repository or in the
44path where the CHICKEN executables are located (in the case of
45executable programs or scripts). Additionally the list of installed
46files, and user-defined metadata is stored in the repository.
47
48If no extension name is given on the command-line, then all {{.setup}}
49scripts in the current directory are processed, in the order given
50on the command line.
51
52==== Installing extensions that use libraries
53
54Sometimes an extension requires a C library to compile. Compilation
55can fail when your system has this library in a nonstandard
56location. Luckily, normally Chicken searches in the default locations
57{{/usr}} and {{/usr/local}}, and in the prefix where Chicken itself
58was installed. Sometimes this is not enough, so you'll need to supply
59{{chicken-install}} with some extra hints to the C compiler/linker. Here's
60an example:
61
62  CSC_OPTIONS='-I/usr/pkg/include/mysql -L/usr/pkg/lib/mysql -L -R/usr/pkg/lib/mysql' chicken-install mysql
63
64This installs the mysql egg with the extra compiler options -I and -L
65to set the include path and the library search path. The second -L
66switch passes the -R option directly to the linker, which causes the
67library path to get hardcoded into the resulting extension file (for
68systems that do not use {{ld.so.conf}}).
69
70=== Creating extensions
71
72Extensions can be created by creating an (optionally gzipped) {{tar}}
73archive named {{EXTENSION.egg}} containing all needed files plus a
74{{.setup}} script in the root directory.  After {{chicken-install}} has
75extracted the files, the setup script will be invoked. There are no
76additional constraints on the structure of the archive, but the setup
77script has to be in the root path of the archive.
78
79
80=== Procedures and macros available in setup scripts
81
82==== install-extension
83
84 (install-extension ID FILELIST [INFOLIST])
85
86Installs the extension library with the name {{ID}}. All files given in the list of strings
87{{FILELIST}} will be copied to the extension repository. It should be noted here that
88the extension id has to be identical to the name of the file implementing the extension. The
89extension may load or include other files, or may load other extensions at runtime specified
90by the {{require-at-runtime}} property.
91
92{{FILELIST}} may be a filename, a list of filenames, or a list of pairs of
93the form {{(SOURCE DEST)}} (if you want to copy into a particular sub-directory - the
94destination directory will be created as needed). If {{DEST}} is a relative pathname,
95< it will be copied into the extension repository.
96
97The optional argument {{INFOLIST}} should be an association list that
98maps symbols to values, this list will be stored as {{ID.setup-info}} at the same
99location as the extension code. Currently the following properties are used:
100
101===== syntax
102
103 [extension property] (syntax)
104
105Marks the extension as syntax-only. No code is compiled, the extension is intended
106as a file containing macros to be loaded at compile/macro-expansion time.
107
108===== require-at-runtime
109
110 [extension property] (require-at-runtime ID ...)
111
112Specifies extensions that should be loaded (via {{require}}) at runtime. This is mostly
113useful for syntax extensions that need additional support code at runtime.
114
115===== version
116
117 [extension property] (version STRING)
118
119Specifies version string.
120
121===== static
122
123 [extension property] (static STRING)
124
125If the extension also provides a static library, then STRING should
126contain the name of that library. Used by {{csc}} when compiling with
127the {{-static-extensions}} option.
128
129===== static-options
130
131 [extension property] (static-options STRING)
132
133Additional options that should be passed to the linker when linking
134with the static version of an extension (see {{static}} above). Used
135by {{csc}} when compiling with the {{-static-extensions}} option.
136
137All other properties are currently ignored. The {{FILELIST}} argument may also be a single
138string.
139
140==== install-program
141
142 [procedure] (install-program ID FILELIST [INFOLIST])
143
144Similar to {{install-extension}}, but installs an executable program in the
145executable path (usually {{/usr/local/bin}}).
146
147==== install-script
148
149 [procedure] (install-script ID FILELIST [INFOLIST])
150
151Similar to {{install-program}}, but additionally changes the file permissions of all
152files in {{FILELIST}} to executable (for installing shell-scripts).
153
154==== standard-extension
155
156 [procedure] (standard-extension ID VERSION)
157
158A convenience procedure that combines the compilation and installation of
159a simple single-file extension. This is roughly equivalent to:
160
161  (compile -s -O2 -d1 ID.scm -j ID)
162  (compile -c -O2 -d1 ID.scm -j ID -unit ID)
163  (compile -s -O2 -d0 ID.import.scm)
164 
165  (install-extension
166   'ID
167   '("ID.o" "ID.so" "ID.import.so")
168   '((version 1.0)
169     (static "ID.o")))
170
171
172==== run
173
174 [syntax] (run FORM ...)
175
176Runs the shell command {{FORM}}, which is wrapped in an implicit {{quasiquote}}.
177{{(run (csc ...))}} is treated specially and passes {{-v}} (if {{-verbose}} has been given
178to {{chicken-install}}) and {{-feature compiling-extension}} options to the compiler.
179
180==== compile
181
182 [syntax] (compile FORM ...)
183
184Equivalent to {{(run (csc FORM ...))}}.
185
186==== make
187
188 [syntax] (make ((TARGET (DEPENDENT ...) COMMAND ...) ...) ARGUMENTS)
189
190A ''make'' macro that executes the expressions {{COMMAND ...}}, when any of the dependents
191{{DEPENDENT ...}} have changed, to build {{TARGET}}. This is the same as the {{make}}
192extension, which is available separately. For more information, see
193[[http://www.call-with-current-continuation.org/eggs/make.html|make]].
194
195
196==== patch
197
198 [procedure] (patch WHICH REGEX SUBST)
199
200Replaces all occurrences of the regular expression {{REGEX}} with the string {{SUBST}},
201in the file given in {{WHICH}}. If {{WHICH}} is a string, the file will be patched and
202overwritten. If {{WHICH}} is a list of the form {{OLD NEW}}, then a different file named
203{{NEW}} will be generated.
204
205==== copy-file
206
207 [procedure] (copy-file FROM TO)
208
209Copies the file or directory (recursively) given in the string {{FROM}} to the destination
210file or directory {{TO}}.
211
212==== move-file
213
214 [procedure] (move-file FROM TO)
215
216Moves the file or directory (recursively) given in the string {{FROM}} to the destination
217file or directory {{TO}}.
218
219==== remove-file*
220
221 [procedure] (remove-file* PATH)
222
223Removes the file or directory given in the string {{PATH}}.
224
225
226==== find-library
227
228 [procedure] (find-library NAME PROC)
229
230Returns {{#t}} if the library named {{libNAME.[a|so]}} (unix) or {{NAME.lib}} (windows)
231could be found by compiling and linking a test program. {{PROC}} should be the name of a
232C function that must be provided by the library. If no such library was found or the function could not
233be resolved, {{#f}} is returned.
234
235==== find-header
236
237 [procedure] (find-header NAME)
238
239Returns {{#t}} if a C include-file with the given name is available, or {{#f}} otherwise.
240
241==== try-compile
242
243 [procedure] (try-compile CODE #!key cc cflags ldflags compile-only c++)
244
245Returns {{#t}} if the C code in {{CODE}} compiles and links successfully, or {{#f}} otherwise.
246The keyword parameters {{cc}} (compiler name, defaults to the C compiler used to build this system),
247{{cflags}} and {{ldflags}} accept additional compilation and
248linking options. If {{compile-only}} is true, then no linking step takes place.
249If the keyword argument {{c++}} is given and true, then the code will be compiled in C++ mode.
250
251
252==== create-directory
253
254 [procedure] (create-directory PATH)
255
256Creates the directory given in the string {{PATH}}, with all parent directories as needed.
257
258
259==== chicken-prefix
260
261 [parameter] chicken-prefix
262
263The installation prefix specified when CHICKEN was built.
264
265==== installation-prefix
266
267 [parameter] installation-prefix
268
269An alternative installation prefix that will be prepended to extension
270installation paths if specified. It is set by the {{-install-prefix}}
271option or environment variable {{CHICKEN_INSTALL_PREFIX}}.
272
273==== program-path
274
275 [parameter] (program-path [PATH])
276
277Holds the path where executables are installed and defaults to either {{$CHICKEN_PREFIX/bin}},
278if the environment variable {{CHICKEN_PREFIX}} is set or the
279path where the CHICKEN binaries ({{chicken}}, {{csi}}, etc.) are installed.
280
281
282==== setup-root-directory
283
284 [parameter] (setup-root-directory [PATH])
285
286Contains the path of the directory where {{chicken-install}} was invoked.
287
288
289==== setup-install-mode
290
291 [parameter] (setup-install-mode [BOOL])
292
293Reflects the setting of the {{-no-install}} option, i.e. is {{#f}}, if {{-no-install}} was
294given to {{chicken-install}}.
295
296==== required-chicken-version
297
298 [procedure] (required-chicken-version VERSION)
299
300Signals an error if the version of CHICKEN that this script runs under is lexicographically less than
301{{VERSION}} (the argument will be converted to a string, first).
302
303
304==== required-extension-version
305
306 [procedure] (required-extension-version EXTENSION1 VERSION1 ...)
307
308Checks whether the extensions {{EXTENSION1 ...}} are installed and at least of version {{VERSION1 ...}}.
309The test is made by lexicographically comparing the string-representations of the given version with the version
310of the installed extension. If one of the listed extensions is not installed, has no associated version information
311or is of a version older than the one specified.
312
313
314==== cross-chicken
315
316 [procedure] (cross-chicken)
317
318Returns {{#t}} if this system is configured for cross-compilation or {{#f}}
319otherwise.
320
321
322==== host-extension
323
324 [parameter] host-extension
325
326For a cross-compiling CHICKEN, when compiling an extension, then it
327should be built for the host environment (as opposed to the target
328environment). This parameter is controlled by the {{-host-extension}} command-line
329option. A setup script should perform the proper steps of compiling any
330code by passing {{-host}} when invoking {{csc}} or using the {{compile}}
331macro.
332
333
334=== Examples for extensions
335
336The simplest case is a single file that does not export any syntax. For example
337
338<enscript highlight=scheme>
339;;;; hello.scm
340
341(define (hello name)
342  (print "Hello, " name " !") )
343</enscript>
344
345We need a {{.setup}} script to build and install our nifty extension:
346
347<enscript highlight=scheme>
348;;;; hello.setup
349
350;; compile the code into a dynamically loadable shared object
351;; (will generate hello.so)
352(compile -s hello.scm)
353
354;; Install as extension library
355(install-extension 'hello "hello.so")
356</enscript>
357
358Lastly, we need a file {{hello.meta}} defining a minimal set of properties:
359
360<enscript highlight=scheme>
361;;;; hello.meta
362
363((author "Me")
364 (synopsis "A cool hello-world library")
365 (license "GPLv3")
366 (files "hello.scm" "hello.setup"))
367</enscript>
368
369(for more information about available properties, see {{Metafile reference}})
370
371After entering
372
373 $ chicken-install
374
375at the shell prompt (and in the same directory where the two files
376exist), the file {{hello.scm}} will be compiled into a dynamically
377loadable library. If the compilation succeeds, {{hello.so}} will
378be stored in the repository, together with a file named
379{{hello.setup-info}} containing an a-list with metadata (what
380you stored above in {{hello.meta}}).
381If no extension name is given to {{chicken-install}}, it will simply
382execute the any files with the {{.setup}} extension it can find.
383
384Use it like any other CHICKEN extension:
385
386 $ csi -q
387 #;1> (require-extension hello)
388 ; loading /usr/local/lib/chicken/4/hello.so ...
389 #;2> (hello "me")
390 Hello, me!
391 #;3>
392
393Here we create a simple application:
394
395<enscript highlight=scheme>
396;;;; hello2.scm
397
398(print "Hello, ")
399(for-each (lambda (x) (printf "~A " x)) (command-line-arguments))
400(print "!")
401</enscript>
402
403We also need a setup script:
404
405<enscript highlight=scheme>
406;;;; hello2.setup
407
408(compile hello2.scm)  ; compile `hello2'
409(install-program 'hello2 "hello2") ; name of the extension and files to be installed
410</enscript>
411
412<enscript highlight=scheme>
413;;;; hello2.meta
414
415((author "Me")
416 (synopsis "A cool hello-world application")
417 (license "proprietary")
418 (files "hello.scm" "hello.setup"))
419</enscript>
420
421To use it, just run {{chicken-install}} in the same directory:
422
423 $ chicken-install
424
425(Here we omit the extension name)
426
427Now the program {{hello2}} will be installed in the same location as
428the other CHICKEN tools (like {{chicken}}, {{csi}}, etc.), which will
429normally be {{/usr/local/bin}}.  Note that you need write-permissions
430for those locations and may have to run {{chicken-install}} with
431administrative rights or use the {{-sudo}} option.
432
433De-installation is just as easy - use the {{chicken-uninstall}}
434program to remove one or more extensions from the local repository:
435
436 $ chicken-uninstall hello2
437
438When running {{chicken-install}} with an argument {{NAME}}, for which
439no associated {{.setup}} file exists, then it will try to download the
440extension via HTTP from the CHICKEN code repository at
441[[http://chicken.wiki.br/svn/chicken-eggs/]]. Extensions that are
442required to compile and/or use the requested extension are downloaded
443and installed automatically.
444
445To query the list of currently installed extensions, use
446{{chicken-status}}. It can list what extensions are installed and
447what files belong to a particular installed extension.
448
449
450=== chicken-install reference
451
452Available options:
453
454; {{-h   -help}} : show this message and exit
455; {{-v   -version}} : show version and exit
456; {{-force}} : don't ask, install even if versions don't match
457; {{-k   -keep}} : keep temporary files
458; {{-l   -location LOCATION}} : install from given location instead of default
459; {{-t   -transport TRANSPORT}} : use given transport instead of default
460; {{-s   -sudo}} : use {{sudo(1)}} for installing or removing files
461; {{-r   -retrieve}} : only retrieve egg into current directory, don't install
462; {{-n   -no-install}} : do not install, just build (implies {{-keep}})
463; {{-p   -prefix PREFIX}} : change installation prefix to {{PREFIX}}
464; {{-host-extension}} : when cross-compiling, compile extension for host
465; {{-test}} : run included test-cases, if available
466; {{-username USER}} : set username for transports that require this
467; {{-password PASS}} : set password for transports that require this
468; {{-i   -init DIRECTORY}} : initialize empty alternative repository
469; {{-u   -update-db}} : update export database
470
471
472=== chicken-uninstall reference
473
474; {{-h   -help}} : show usage information and exit
475; {{-v   -version}} : show version and exit
476; {{-force}} : don't ask, delete whatever matches
477; {{-s   -sudo}} : use {{sudo(1)}} for deleting files
478
479
480=== chicken-status reference
481
482; {{-h   -help}} : show usage information and exit
483; {{-v   -version}} : show version and exit
484; {{-f   -files}} : list installed files
485
486
487=== Security
488
489When extensions are downloaded and installed one is executing code
490from potentially compromised systems. This applies also when
491{{chicken-install}} executes system tests for required extensions. As
492the code has been retrieved over the network effectively untrusted
493code is going to be evaluated. When {{chicken-install}} is run as
494''root'' the whole system is at the mercy of the build instructions
495(note that this is also the case every time you install software via
496{{sudo make install}}, so this is not specific to the CHICKEN
497extension mechanism).
498
499Security-conscious users should never run {{chicken-install}} as root.
500A simple remedy is to set the environment variable
501{{CHICKEN_REPOSITORY}}, which will transparently place the repository
502at an arbitrary user-selected location (don't forget to initialize it
503first with {{chicken-install -init <directory>}}).  Alternatively
504obtain write/execute access to the default location of the repository
505(usually {{/usr/local/lib/chicken}}) to avoid running as
506root. {{chicken-install}} also provides a {{-sudo}} option to perform
507the last installation steps as root user, but do building and other
508.setup script processing as normal.
509
510
511=== Other modes of installation
512
513It is possible to install extensions directly from a
514[[http://subversion.tigris.org|Subversion]] repository or from a local
515checkout of the repository tree by using the {{-transport}} and
516{{-location}} options when invoking {{chicken-install}}. Three possible
517transport mechanisms are currently supported:
518
519; {{http}} : download extension sources via HTTP from a web-server (this is the default)
520; {{svn}} : perform an {{svn export}} from the central extension repository; this will require a {{svn(1)}} client to be installed on the machine
521; {{local}} : use sources from the local filesystem and build directly in the source directory
522
523The {{-location}} option specifies where to look for the source
524repository and names a web URL, a subversion repository URL or a
525filesystem path, respectively. A list of locations to try when
526retrieving extensions is stored in the file {{setup.defaults}}
527(usually installed in {{/usr/local/share/chicken}}). For {{http}}
528transports, {{chicken-install}} will detect networking timeouts and
529try alternative locations, as listed in the file.
530
531Dependency information, which is necessary to ensure required
532extensions are also installed, is processed automatically.
533
534=== Linking extensions statically
535
536The compiler and {{chicken-install}} support statically linked
537eggs. The general approach is to generate an object file or static
538library (in addition to the usual
539shared library) in your {{.setup}} script and install it along with the
540dynamically loadable extension. The setup properties {{static}}
541should contain the name of the object file (or static library) to be
542linked, when {{csc}} gets passed the {{-static-extensions}} option:
543
544<enscript highlight=scheme>
545 (compile -s -O2 -d1 my-ext.scm)   ; dynamically loadable "normal" version
546 (compile -c -O2 -d1 my-ext -unit my-ext)  ; statically linkable version
547 (install-extension
548   'my-ext
549   '("my-ext.so" "my-ext.o")
550   '((static "my-ext.o")) )
551</enscript>
552
553Note the use of the {{-unit}} option in the second compilation step: static
554linking must use static library units. {{chicken-install}} will perform
555platform-dependent file-extension translation for the file list, but does currently
556not do that for the {{static}} extension property.
557
558To actually link with the static version of {{my-ext}}, do:
559
560 % csc -static-extensions my-program.scm -uses my-ext
561
562The compiler will try to do the right thing, but can not handle all
563extensions, since the ability to statically link eggs is relatively
564new. Eggs that support static linking are designated as being able to
565do so. If you require a statically linkable version of an egg that has
566not been converted yet, contact the extension author or the CHICKEN
567mailing list.
568
569---
570Previous: [[Interface to external functions and variables]]
571
572Next: [[Data representation]]
Note: See TracBrowser for help on using the repository browser.