source: project/wiki/eggref/4/salmonella @ 25853

[[tags: egg salmonella]]

== salmonella

[[toc:]]

=== Introduction

Salmonella is a set of tools to test CHICKEN eggs.  Here's a brief
summary of features:

* very light on dependencies (none at the moment)
* paralellizable ({{salmonella-epidemy}})
* can be run with or without Internet access

Note: starting on version 2.0, salmonella is a rewrite of the
salmonella 1.x series, which was available since CHICKEN 2.x.  The
documentation for the 1.x series is still available at
[[http://wiki.call-cc.org/egg/salmonella-1.x|http://wiki.call-cc.org/egg/salmonella-1.x]].


=== Author

[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]]


=== Repository

salmonella is hosted on [[http://github.com|github]]:
[[https://github.com/mario-goulart/salmonella|https://github.com/mario-goulart/salmonella]]


=== Tools

This egg provides some command line tools.  The next sections describe
each of them.



==== salmonella

This is the core tool for testing eggs.  {{salmonella}} can perform
the following tests:

* installation of eggs
* dependencies verification
* {{.meta}} file consistency check
* egg tests execution
* general egg consistency (i.e., can be used as an ''egg lint'' to
avoid common mistakes when publishing a new egg or a new egg
version)

{{salmonella}} generates a sexpr-based log file as output.  That file
can be used by tools like
[[/egg/salmonella-html-report|salmonella-html-report]] to generate a
pretty output in HTML.

When executing, salmonella prints some basic information on the
standard output.  Detailed information can be viewed with the
aforementioned [[/egg/salmonella-html-report|salmonella-html-report]]
or with {{salmonella-log-viewer}}, a tool which is provided by this
egg. {{salmonella-log-viewer}} parses the log file generated by
salmonella and formats the data on the standard output.

===== Command line options

Here's the output of {{salmonella -h}} with an explanation for each
command line option:

  salmonella [ -h | --help ]
  salmonella <options> eggs

  <options>:
  --log-file=<logfile>
      The name for the log file to be generated by salmonella
      (default=salmonella.log).

  --chicken-installation-prefix=<prefix dir>
      If you want to test eggs using a chicken installed on a certain directory,
      you can use this option (it should point to the same directory as given to
      `PREFIX' when installing CHICKEN). If omitted, salmonella uses CHICKEN
      tools from the system PATH variable.

  --chicken-install-args=<install args>
      This option can be used customize chicken-install's arguments.  You can
      use <repo> to indicate where you want the actual repository directory
      to be replaced by salmonella.

  --eggs-source-dir=<eggs dir>
      By default, salmonella fetches eggs from the egg server.  If you have a
      local copy of eggs code, you can use this option to point to the directory
      where they are located.

  --eggs-doc-dir=<doc dir>
      By default, salmonella checks if documentation for eggs exist by accessing
      the CHICKEN wiki.  If you have a local copy of the wiki documentation for
      eggs, you can use this option to point to the directory where they can be
      found.

  --keep-repo
      For each egg that salmonella tests, it sets the egg installation repository
      empty and removes it at the end of its execution.  This option makes
      salmonella keep the egg installation repository after testing each egg and
      after finishing its execution.  This option can save a lot of time when
      testing several eggs, at the cost of potentially making salmonella unable
      to catch dependencies problems.

  --skip-eggs=<comma-separated list of eggs to skip>
      A comma-separated list of eggs to be skipped.

  --this-egg
      To be used from withing the egg source directory to check for common errors
      before releasing an egg or a new egg version.

  --repo-dir=<path to repo dir to be used>
      Alternative location for the egg installation directory used by salmonella.
      By default, salmonella generates a `salmonella-tmp-xxxxx' directory in the
      current directory.  This option can be useful when used with `--keep-repo'
      to reuse egg installation repositories for several salmonella executions.

  --verbosity=<number>
      A number to indicate salmonella's verbosity level.  0 means practically
      silent. 1 is mostly silent and 2 (default) prints some useful information
      while salmonella is running.


===== Some quick tips by example

Simplest case: testing eggs using the remote egg server and the
chicken tools ({{chicken-install}} and {{csi}}) in the system {{PATH}}

  $ salmonella big-chicken slice


You can test all the available eggs using one of the following
approaches:

1. If you have CHICKEN >= 4.7.2:

  $ salmonella `chicken-install -list`


2. If you don't have CHICKEN >= 4.7.2:

  $ salmonella `wget http://code.call-cc.org/cgi-bin/henrietta.cgi?list=1 -q -O -`


If you don't want to test some specific eggs, you can skip them:

  $ salmonella --skip-eggs=macosx,hfs+ `chicken-install -list`


You can tell salmonella to use a specific CHICKEN version:

  $ salmonella --chicken-installation-prefix=/usr/local/chicken-4.7.3 big-chicken slice


Not cleaning the egg installation repository after installing each egg
may significantly speed up the salmonella execution time (the default
behavior is to set the egg installation repository empty after testing
each egg):

  $ salmonella --keep-repo big-chicken slice

warning: if you use --keep-repo, salmonella will not be able to catch
dependencies problems.


If you want to reuse the same egg installation repository for multiple
salmonella runs, you can provide a specific directory:

  $ salmonella --repo-dir=my-repo --keep-repo big-chicken slice


By default, salmonella generates a log file named {{salmonella.log}} in
the current directory.  You can change that by using the {{--log-file}}
command line option:

  $ salmonella --log-file=my-log-file.log big-chicken slice


====== Using salmonella as an egg lint tool

Suppose you are working on a new egg and you want to check if it is
working ok before releasing it (or a new version).  You can use
salmonella to check it:

  $ cd my-egg    # where your egg code is stored
  $ salmonella --this-egg



==== salmonella-log-viewer

This tool can be used to turn salmonella log files into something
readable on the standard output.

Just provide a salmonella log file as argument to
{{salmonella-log-viewer}}:

    $ salmonella-log-viewer salmonella.log

Alternatively, you can use
[[/egg/salmonella-html-report|salmonella-html-report]] for a prettier
and more complete format.



==== salmonella-epidemy

{{salmonella-epidemy}} can be used to run multiple salmonella
instances in parallel.  It can be handy when you have a multi-core
machine and you want to make use of all cores, for example.  The
command line options are basically the same as for {{salmonella}},
plus a {{--instances=<number>}} to indicate how many {{salmonella}}
instances you want to run in parallel.

This tool can significantly speed up salmonella execution times on
multi-core machines.

Usage example:

  $ salmonella-epidemy --instances=2 big-chicken slice spiffy amb



==== salmonella-log-merger

This tool simply puts more than one salmonella log into a single one,
so it can be used by tools like {{salmonella-log-viewer}} and
[[/egg/salmonella-html-report|salmonella-html-report]].


Usage example:

  $ salmonella-log-merger --log-file=full.log file1.log file2.log



==== Log file format

The command line tool writes a log file which contains records in the
following format:

 (<egg> <action> <status> <message> <duration>)


===== <egg>

Can be either a symbol that indicates the egg-name or {{#f}} to
indicate the {{start}} and {{end}} actions (logged when salmonella is
started and when it finishes testing, respectively).

===== <action>

A symbol to indicate the action that was executed.  Can be one of the
following values:

* {{start}}: starting salmonella
* {{fetch}}: fetching egg
* {{install}}: installing egg
* {{test}}: testing egg
* {{end}}: salmonella has finished

===== <status>

A numeric value indicating the exit status of the executed action.
When the action is {{test}} and {{status}} is {{-1}}, it means that
the egg has no tests.


===== <message>

The output generated by the commands executed to perform {{<action>}}.
A string.


===== <duration>

The time (in seconds) that was taken to execute {{<action>}}.

For the {{start}} and {{end}} actions, the value is the seconds since
epoch, so the total salmonella execution time can be determined by
subtracting the start time from the end time.


=== Modules

Salmonella provides two modules:

; {{salmonella}}: the core salmonella functionality and basic data strucuture for logs ({{report}} records)

; {{salmonella-log-parser}}: provides procedures to access log files and compute simple statistics regarding log file data.


==== salmonella

===== report
<record>(report egg action status message duration)</record>
<procedure>make-report</procedure>
<procedure>report?</procedure>
<procedure>report-egg</procedure>
<procedure>report-action</procedure>
<procedure>report-status</procedure>
<procedure>report-message</procedure>
<procedure>report-duration</procedure>
<procedure>report-egg-set!</procedure>
<procedure>report-action-set!</procedure>
<procedure>report-status-set!</procedure>
<procedure>report-message-set!</procedure>
<procedure>report-duration-set!</procedure>

{{report}} objects.  Each log file registry is represented by a
{{report}} object (serialized as a list).

===== report->list
<procedure>(report->list report)</procedure>

Convert a {{report}} object to a list.


===== log!
<procedure>(log! report log-file)</procedure>

Print the {{report}} representation to {{log-file}}.


===== make-salmonella
<procedure>(make-salmonella tmp-dir #!key chicken-installation-prefix chicken-install-args eggs-source-dir eggs-doc-dir this-egg?)</procedure>

The salmonella maker.  Returns a procedure that receives symbols
(''methods'') to indicate the actions to be performed.  The available
methods are (example considering a {{salmonella}} object returned by
{{make-salmonella}}):

====== {{(salmonella 'init-repo!)}}

====== {{(salmonella 'clear-repo!)}}

====== {{(salmonella 'fetch <egg>)}}

====== {{(salmonella 'install <egg>)}}

====== {{(salmonella 'test <egg>)}}

====== {{(salmonella 'check-version <egg>)}}

====== {{(salmonella 'env-info)}}

====== {{(salmonella 'meta-data <egg>)}}

====== {{(salmonella 'check-dependencies <egg> <meta data>)}}

====== {{(salmonella 'check-category <egg> <meta data>)}}

====== {{(salmonella 'check-license <egg> <meta data>)}}

====== {{(salmonella 'check-author <egg> <meta data>)}}

====== {{(salmonella 'check-doc <egg>)}}



==== salmonella-log-parser

===== Reading

<procedure>(read-log-file filename)</procedure>

Reads the log file {{filename}} and returns a list of report records.

===== {{fetch}} action

====== fetch-status
<procedure>(fetch-status egg log)</procedure>

====== fetch-message
<procedure>(fetch-message egg log)</procedure>

====== fetch-duration
<procedure>(fetch-duration egg log)</procedure>



===== {{install}} action

====== install-status
<procedure>(install-status egg log)</procedure>

====== install-message
<procedure>(install-message egg log)</procedure>

====== install-duration
<procedure>(install-duration egg log)</procedure>



===== {{check-version}} action

====== check-version-status
<procedure>(check-version-status egg log)</procedure>

====== check-version-message
<procedure>(check-version-message egg log)</procedure>

====== egg-version
<procedure>(egg-version egg log)</procedure>

====== check-version-ok?
<procedure>(check-version-ok? egg log)</procedure>


===== {{test}} action

====== test-status
<procedure>(test-status egg log)</procedure>

====== test-message
<procedure>(test-message egg log)</procedure>

====== test-duration
<procedure>(test-duration egg log)</procedure>

====== has-test?
<procedure>(has-test? egg log)</procedure>


===== {{meta-data}} action

====== meta-data
<procedure>(meta-data egg log)</procedure>

====== egg-dependencies
<procedure>(egg-dependencies egg log)</procedure>

====== egg-license
<procedure>(egg-license egg log #!key with-test-dependencies? with-versions?)</procedure>



===== {{check-doc}} action

====== doc-exists?
<procedure>(doc-exists? egg log)</procedure>



===== {{start}} & {{end}} actions

====== start-time
<procedure>(start-time log)</procedure>

====== end-time
<procedure>(end-time log)</procedure>

====== total-time
<procedure>(total-time log)</procedure>

====== salmonella-info
<procedure>(salmonella-info log)</procedure>



===== Statistics

====== count-install-ok
<procedure>(count-install-ok log)</procedure>

====== count-install-fail
<procedure>(count-install-fail log)</procedure>

====== count-test-ok
<procedure>(count-test-ok log)</procedure>

====== count-test-fail
<procedure>(count-test-fail log)</procedure>

====== count-no-test
<procedure>(count-no-test log)</procedure>

====== count-total-eggs
<procedure>(count-total-eggs log #!key with-skipped?)</procedure>

====== count-documented
<procedure>(count-documented log)</procedure>

====== count-undocumented
<procedure>(count-undocumented log)</procedure>


===== Miscelaneous

====== prettify-time
<procedure>(prettify-time time)</procedure>

====== sort-eggs
<procedure>(sort-eggs eggs)</procedure>

====== log-eggs
<procedure>(log-eggs log)</procedure>

====== log-skipped-eggs
<procedure>(log-skipped-eggs log)</procedure>



=== Environment variables

When running, salmonella sets the {{SALMONELLA_RUNNING}} environment
variable.  If you need to check if your tests code is being run by
salmonella, this variable can be used.



=== License

BSD



=== Version history

==== 2.1

* Hours, minutes and seconds formatted with exact numbers ({{prettify-time}})


==== 2.0

* Rewrite of the 1.x version