source: project/wiki/eggref/4/salmonella @ 27377

Last change on this file since 27377 was 27377, checked in by Mario Domenech Goulart, 9 years ago

salmonella (wiki): release notes for salmonella 2.6

File size: 14.5 KB
Line 
1[[tags: egg salmonella]]
2
3== salmonella
4
5[[toc:]]
6
7=== Introduction
8
9Salmonella is a set of tools to test CHICKEN eggs.  Here's a brief
10summary of features:
11
12* very light on dependencies (none at the moment)
13* paralellizable ({{salmonella-epidemy}})
14* can be run with or without Internet access
15
16Note: starting on version 2.0, salmonella is a rewrite of the
17salmonella 1.x series, which was available since CHICKEN 2.x.  The
18documentation for the 1.x series is still available at
19[[http://wiki.call-cc.org/egg/salmonella-1.x|http://wiki.call-cc.org/egg/salmonella-1.x]].
20
21
22=== Author
23
24[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]]
25
26
27=== Repository
28
29salmonella is hosted on [[http://github.com|github]]:
30[[https://github.com/mario-goulart/salmonella|https://github.com/mario-goulart/salmonella]]
31
32
33=== Tools
34
35This egg provides some command line tools.  The next sections describe
36each of them.
37
38
39
40==== salmonella
41
42This is the core tool for testing eggs.  {{salmonella}} can perform
43the following tests:
44
45* installation of eggs
46* dependencies verification
47* {{.meta}} file consistency check
48* egg tests execution
49* general egg consistency (i.e., can be used as an ''egg lint'' to
50avoid common mistakes when publishing a new egg or a new egg
51version)
52
53{{salmonella}} generates a sexpr-based log file as output.  That file
54can be used by tools like
55[[/egg/salmonella-html-report|salmonella-html-report]] to generate a
56pretty output in HTML.
57
58When executing, salmonella prints some basic information on the
59standard output.  Detailed information can be viewed with the
60aforementioned [[/egg/salmonella-html-report|salmonella-html-report]]
61or with {{salmonella-log-viewer}}, a tool which is provided by this
62egg. {{salmonella-log-viewer}} parses the log file generated by
63salmonella and formats the data on the standard output.
64
65===== Command line options
66
67Here's the output of {{salmonella -h}} with an explanation for each
68command line option:
69
70  salmonella [ -h | --help ]
71  salmonella <options> eggs
72
73  <options>:
74  --log-file=<logfile>
75      The name for the log file to be generated by salmonella
76      (default=salmonella.log).
77
78  --chicken-installation-prefix=<prefix dir>
79      If you want to test eggs using a chicken installed on a certain directory,
80      you can use this option (it should point to the same directory as given to
81      `PREFIX' when installing CHICKEN). If omitted, salmonella uses CHICKEN
82      tools from the system PATH variable.
83
84  --chicken-install-args=<install args>
85      This option can be used customize chicken-install's arguments.  You can
86      use <repo> to indicate where you want the actual repository directory
87      to be replaced by salmonella.
88
89  --eggs-source-dir=<eggs dir>
90      By default, salmonella fetches eggs from the egg server.  If you have a
91      local copy of eggs code, you can use this option to point to the directory
92      where they are located.
93
94  --eggs-doc-dir=<doc dir>
95      By default, salmonella checks if documentation for eggs exist by accessing
96      the CHICKEN wiki.  If you have a local copy of the wiki documentation for
97      eggs, you can use this option to point to the directory where they can be
98      found.
99
100  --keep-repo
101      For each egg that salmonella tests, it sets the egg installation repository
102      empty and removes it at the end of its execution.  This option makes
103      salmonella keep the egg installation repository after testing each egg and
104      after finishing its execution.  This option can save a lot of time when
105      testing several eggs, at the cost of potentially making salmonella unable
106      to catch dependencies problems.
107
108  --skip-eggs=<comma-separated list of eggs to skip>
109      A comma-separated list of eggs to be skipped.
110
111  --repo-dir=<path to repo dir to be used>
112      Alternative location for the egg installation directory used by salmonella.
113      By default, salmonella generates a `salmonella-tmp-xxxxx' directory in the
114      current directory.  This option can be useful when used with `--keep-repo'
115      to reuse egg installation repositories for several salmonella executions.
116
117  --verbosity=<number>
118      A number to indicate salmonella's verbosity level.  0 means practically
119      silent. 1 is mostly silent and 2 (default) prints some useful information
120      while salmonella is running.
121
122
123===== Some quick tips by example
124
125Simplest case: testing eggs using the remote egg server and the
126chicken tools ({{chicken-install}} and {{csi}}) in the system {{PATH}}
127
128  $ salmonella big-chicken slice
129
130
131You can test all the available eggs using one of the following
132approaches:
133
1341. If you have CHICKEN >= 4.7.2:
135
136  $ salmonella `chicken-install -list`
137
138
1392. If you don't have CHICKEN >= 4.7.2:
140
141  $ salmonella `wget http://code.call-cc.org/cgi-bin/henrietta.cgi?list=1 -q -O -`
142
143
144If you don't want to test some specific eggs, you can skip them:
145
146  $ salmonella --skip-eggs=macosx,hfs+ `chicken-install -list`
147
148
149You can tell salmonella to use a specific CHICKEN version:
150
151  $ salmonella --chicken-installation-prefix=/usr/local/chicken-4.7.3 big-chicken slice
152
153
154Not cleaning the egg installation repository after installing each egg
155may significantly speed up the salmonella execution time (the default
156behavior is to set the egg installation repository empty after testing
157each egg):
158
159  $ salmonella --keep-repo big-chicken slice
160
161warning: if you use --keep-repo, salmonella will not be able to catch
162dependencies problems.
163
164
165If you want to reuse the same egg installation repository for multiple
166salmonella runs, you can provide a specific directory:
167
168  $ salmonella --repo-dir=my-repo --keep-repo big-chicken slice
169
170
171By default, salmonella generates a log file named {{salmonella.log}} in
172the current directory.  You can change that by using the {{--log-file}}
173command line option:
174
175  $ salmonella --log-file=my-log-file.log big-chicken slice
176
177
178====== Using salmonella as an egg lint tool
179
180Suppose you are working on a new egg and you want to check if it is
181working ok before releasing it (or a new version).  You can use
182salmonella to check it:
183
184  $ cd my-egg    # where your egg code is stored
185  $ salmonella
186
187
188
189==== salmonella-log-viewer
190
191This tool can be used to turn salmonella log files into something
192readable on the standard output.
193
194Just provide a salmonella log file as argument to
195{{salmonella-log-viewer}}:
196
197    $ salmonella-log-viewer salmonella.log
198
199Alternatively, you can use
200[[/egg/salmonella-html-report|salmonella-html-report]] for a prettier
201and more complete format.
202
203
204
205==== salmonella-epidemy
206
207{{salmonella-epidemy}} can be used to run multiple salmonella
208instances in parallel.  It can be handy when you have a multi-core
209machine and you want to make use of all cores, for example.  The
210command line options are basically the same as for {{salmonella}},
211plus a {{--instances=<number>}} to indicate how many {{salmonella}}
212instances you want to run in parallel.
213
214This tool can significantly speed up salmonella execution times on
215multi-core machines.
216
217Usage example:
218
219  $ salmonella-epidemy --instances=2 big-chicken slice spiffy amb
220
221
222
223==== salmonella-log-merger
224
225This tool simply puts more than one salmonella log into a single one,
226so it can be used by tools like {{salmonella-log-viewer}} and
227[[/egg/salmonella-html-report|salmonella-html-report]].
228
229
230Usage example:
231
232  $ salmonella-log-merger --log-file=full.log file1.log file2.log
233
234
235
236==== Log file format
237
238The command line tool writes a log file which contains records in the
239following format:
240
241 (<egg> <action> <status> <message> <duration>)
242
243
244===== <egg>
245
246Can be either a symbol that indicates the egg-name or {{#f}} to
247indicate the {{start}} and {{end}} actions (logged when salmonella is
248started and when it finishes testing, respectively).
249
250===== <action>
251
252A symbol to indicate the action that was executed.  Can be one of the
253following values:
254
255* {{start}}: starting salmonella
256* {{fetch}}: fetching egg
257* {{install}}: installing egg
258* {{test}}: testing egg
259* {{end}}: salmonella has finished
260
261===== <status>
262
263A numeric value indicating the exit status of the executed action.
264When the action is {{test}} and {{status}} is {{-1}}, it means that
265the egg has no tests.
266
267
268===== <message>
269
270The output generated by the commands executed to perform {{<action>}}.
271A string.
272
273
274===== <duration>
275
276The time (in seconds) that was taken to execute {{<action>}}.
277
278For the {{start}} and {{end}} actions, the value is the seconds since
279epoch, so the total salmonella execution time can be determined by
280subtracting the start time from the end time.
281
282
283=== Modules
284
285Salmonella provides two modules:
286
287; {{salmonella}}: the core salmonella functionality and basic data strucuture for logs ({{report}} records)
288
289; {{salmonella-log-parser}}: provides procedures to access log files and compute simple statistics regarding log file data.
290
291
292==== salmonella
293
294===== report
295<record>(report egg action status message duration)</record>
296<procedure>make-report</procedure>
297<procedure>report?</procedure>
298<procedure>report-egg</procedure>
299<procedure>report-action</procedure>
300<procedure>report-status</procedure>
301<procedure>report-message</procedure>
302<procedure>report-duration</procedure>
303<procedure>report-egg-set!</procedure>
304<procedure>report-action-set!</procedure>
305<procedure>report-status-set!</procedure>
306<procedure>report-message-set!</procedure>
307<procedure>report-duration-set!</procedure>
308
309{{report}} objects.  Each log file registry is represented by a
310{{report}} object (serialized as a list).
311
312===== report->list
313<procedure>(report->list report)</procedure>
314
315Convert a {{report}} object to a list.
316
317
318===== log!
319<procedure>(log! report log-file)</procedure>
320
321Print the {{report}} representation to {{log-file}}.
322
323
324===== make-salmonella
325<procedure>(make-salmonella tmp-dir #!key chicken-installation-prefix chicken-install-args eggs-source-dir eggs-doc-dir this-egg?)</procedure>
326
327The salmonella maker.  Returns a procedure that receives symbols
328(''methods'') to indicate the actions to be performed.  The available
329methods are (example considering a {{salmonella}} object returned by
330{{make-salmonella}}):
331
332====== {{(salmonella 'init-repo!)}}
333
334====== {{(salmonella 'clear-repo!)}}
335
336====== {{(salmonella 'fetch <egg>)}}
337
338====== {{(salmonella 'install <egg>)}}
339
340====== {{(salmonella 'test <egg>)}}
341
342====== {{(salmonella 'check-version <egg>)}}
343
344====== {{(salmonella 'env-info)}}
345
346====== {{(salmonella 'meta-data <egg>)}}
347
348====== {{(salmonella 'check-dependencies <egg> <meta data>)}}
349
350====== {{(salmonella 'check-category <egg> <meta data>)}}
351
352====== {{(salmonella 'check-license <egg> <meta data>)}}
353
354====== {{(salmonella 'check-author <egg> <meta data>)}}
355
356====== {{(salmonella 'check-doc <egg>)}}
357
358
359
360==== salmonella-log-parser
361
362===== Reading
363
364<procedure>(read-log-file filename)</procedure>
365
366Reads the log file {{filename}} and returns a list of report records.
367
368===== {{fetch}} action
369
370====== fetch-status
371<procedure>(fetch-status egg log)</procedure>
372
373====== fetch-message
374<procedure>(fetch-message egg log)</procedure>
375
376====== fetch-duration
377<procedure>(fetch-duration egg log)</procedure>
378
379
380
381===== {{install}} action
382
383====== install-status
384<procedure>(install-status egg log)</procedure>
385
386====== install-message
387<procedure>(install-message egg log)</procedure>
388
389====== install-duration
390<procedure>(install-duration egg log)</procedure>
391
392
393
394===== {{check-version}} action
395
396====== check-version-status
397<procedure>(check-version-status egg log)</procedure>
398
399====== check-version-message
400<procedure>(check-version-message egg log)</procedure>
401
402====== egg-version
403<procedure>(egg-version egg log)</procedure>
404
405====== check-version-ok?
406<procedure>(check-version-ok? egg log)</procedure>
407
408
409===== {{test}} action
410
411====== test-status
412<procedure>(test-status egg log)</procedure>
413
414====== test-message
415<procedure>(test-message egg log)</procedure>
416
417====== test-duration
418<procedure>(test-duration egg log)</procedure>
419
420====== has-test?
421<procedure>(has-test? egg log)</procedure>
422
423
424===== {{meta-data}} action
425
426====== meta-data
427<procedure>(meta-data egg log)</procedure>
428
429====== egg-dependencies
430<procedure>(egg-dependencies egg log)</procedure>
431
432====== egg-license
433<procedure>(egg-license egg log #!key with-test-dependencies? with-versions?)</procedure>
434
435
436
437===== {{check-doc}} action
438
439====== doc-exists?
440<procedure>(doc-exists? egg log)</procedure>
441
442
443
444===== {{start}} & {{end}} actions
445
446====== start-time
447<procedure>(start-time log)</procedure>
448
449====== end-time
450<procedure>(end-time log)</procedure>
451
452====== total-time
453<procedure>(total-time log)</procedure>
454
455====== salmonella-info
456<procedure>(salmonella-info log)</procedure>
457
458
459
460===== Statistics
461
462====== count-install-ok
463<procedure>(count-install-ok log)</procedure>
464
465====== count-install-fail
466<procedure>(count-install-fail log)</procedure>
467
468====== count-test-ok
469<procedure>(count-test-ok log)</procedure>
470
471====== count-test-fail
472<procedure>(count-test-fail log)</procedure>
473
474====== count-no-test
475<procedure>(count-no-test log)</procedure>
476
477====== count-total-eggs
478<procedure>(count-total-eggs log #!key with-skipped?)</procedure>
479
480====== count-documented
481<procedure>(count-documented log)</procedure>
482
483====== count-undocumented
484<procedure>(count-undocumented log)</procedure>
485
486
487===== Miscelaneous
488
489====== prettify-time
490<procedure>(prettify-time time)</procedure>
491
492====== sort-eggs
493<procedure>(sort-eggs eggs)</procedure>
494
495====== log-eggs
496<procedure>(log-eggs log)</procedure>
497
498====== log-skipped-eggs
499<procedure>(log-skipped-eggs log)</procedure>
500
501
502
503=== Environment variables
504
505When running, salmonella sets the {{SALMONELLA_RUNNING}} environment
506variable.  If you need to check if your tests code is being run by
507salmonella, this variable can be used.
508
509
510
511=== License
512
513BSD
514
515
516
517=== Version history
518
519==== Version 2.6
520
521* Handle broken .meta files
522* Handle version specification in {{test-depends}} (thanks to [[/users/peter-bex|Peter Bex]] for pointing that out)
523* Fixed filenames for {{chicken}} and {{csi}} on mingw ({{.exe}} extension).  Thanks to Dan Leslie for pointing that out.
524* The {{--this-egg}} command line option has been deprecated
525
526
527==== Version 2.5
528
529*  Log {{check-version}} actions even when fetching eggs from remote servers (i.e., no {{--eggs-source-dir}}), so version numbers can get logged
530
531
532==== Version 2.4
533
534* "Fixed" hack to remove {{-test}} from {{chicken-install}} arguments when installing eggs
535
536
537==== Version 2.3
538
539* Bug fix: reference to unbound variable {{eggs}} (thanks to Shawn Rutledge for noticing that)
540
541
542==== Version 2.2
543
544* Only create the temporary repo dir when it is necessary (i.e., not for handling {{--help}} or when there's nothing to do)
545
546
547==== Version 2.1
548
549* Hours, minutes and seconds formatted with exact numbers ({{prettify-time}})
550
551
552==== Version 2.0
553
554* Rewrite of the 1.x version
Note: See TracBrowser for help on using the repository browser.