Changeset 37794 in project

07/26/19 19:25:17 (5 months ago)
Mario Domenech Goulart

eggref/5/salmonella: add documentation for salmonella for C5

1 edited


  • wiki/eggref/5/salmonella

    r37596 r37794  
     1[[tags: egg salmonella]]
    13== salmonella
    3 Unfortunately, salmonella for CHICKEN 5 has not been released as an
    4 egg yet.  The work-in-progress version for CHICKEN 5 can be installed
    5 from git:
    7   $ git clone
    8   $ cd salmonella
    9   $ git checkout chicken-5+4
    10   $ chicken-install
    12 Please refer to the documentation of
    13 [[/eggref/4/salmonella|salmonella for CHICKEN 4]]
    14 for more information on how to use it.
     7=== Introduction
     9Salmonella is a set of tools to test CHICKEN eggs.  Here's a brief
     10summary of features:
     12* very light on dependencies (none at the moment)
     13* parallelizable ({{salmonella-epidemy}})
     14* can be run with or without Internet access
     16Note: starting on version 2.0, salmonella is a rewrite of the
     17salmonella 1.x series, which was available since CHICKEN 2.x.  The
     18documentation for the 1.x series is still available at
     22=== Author
     24[[/users/mario-domenech-goulart|Mario Domenech Goulart]]
     27=== Repository
     29salmonella is hosted on [[|github]]:
     33=== Tools
     35This egg provides some command line tools.  The next sections describe
     36each of them.
     40==== salmonella
     42This is the core tool for testing eggs.  {{salmonella}} can perform
     43the following tests:
     45* installation of eggs
     46* dependencies verification
     47* {{.egg}} file consistency check
     48* egg tests execution
     49* general egg consistency (i.e., can be used as an ''egg lint'' to
     50avoid common mistakes when publishing a new egg or a new egg
     53{{salmonella}} generates a sexpr-based log file as output.  That file
     54can be used by tools like
     55[[/egg/salmonella-html-report|salmonella-html-report]] to generate a
     56pretty output in HTML.
     58When executing, salmonella prints some basic information on the
     59standard output.  Detailed information can be viewed with the
     60aforementioned [[/egg/salmonella-html-report|salmonella-html-report]]
     61or with {{salmonella-log-viewer}}, a tool which is provided by this
     62egg. {{salmonella-log-viewer}} parses the log file generated by
     63salmonella and formats the data on the standard output.
     65===== Command line options
     67Here's the output of {{salmonella -h}} with an explanation for each
     68command line option:
     70  salmonella [ -h | --help ]
     71  salmonella --version
     72  salmonella [ [ <options> ] eggs ]
     74  When called without eggs in the command line, salmonella will try to
     75  find a .egg file in the current directory and process it (just like
     76  chicken-install).
     78  <options>:
     79  --log-file=<logfile>
     80      The name for the log file to be generated by salmonella
     81      (default=salmonella.log).
     83  --chicken-installation-prefix=<prefix dir>
     84      If you want to test eggs using a chicken installed on a certain directory,
     85      you can use this option (it should point to the same directory as given to
     86      `PREFIX' when installing CHICKEN). If omitted, salmonella uses CHICKEN
     87      tools from the current runtime's installation prefix.
     89  --chicken-install-args=<install args>
     90      This option can be used customize chicken-install's arguments.  You can
     91      use <repo> to indicate where you want the actual repository directory
     92      to be replaced by salmonella.
     94  --eggs-source-dir=<eggs dir>
     95      By default, salmonella fetches eggs from the egg server.  If you have a
     96      local copy of eggs code, you can use this option to point to the directory
     97      where they are located.
     99  --eggs-doc-dir=<doc dir>
     100      By default, salmonella checks if documentation for eggs exist by accessing
     101      the CHICKEN wiki.  If you have a local copy of the wiki documentation for
     102      eggs, you can use this option to point to the directory where they can be
     103      found.
     105  --keep-repo
     106      For each egg that salmonella tests, it sets the egg installation repository
     107      empty and removes it at the end of its execution.  This option makes
     108      salmonella keep the egg installation repository after testing each egg and
     109      after finishing its execution.  This option can save a lot of time when
     110      testing several eggs, at the cost of potentially making salmonella unable
     111      to catch dependencies problems.
     113  --skip-eggs=<comma-separated list of eggs to skip>
     114      A comma-separated list of eggs to be skipped.
     116  --repo-dir=<path to repo dir to be used>
     117      Alternative location for the egg installation directory used by salmonella.
     118      By default, salmonella generates a `salmonella-tmp-xxxxx' directory in the
     119      current directory.  This option can be useful when used with `--keep-repo'
     120      to reuse egg installation repositories for several salmonella executions.
     122  --clear-chicken-home
     123      Remove Scheme files from <chicken-installation-prefix>/share/chicken.
     124      WARNING: use this option with care.  If you don't understand the
     125      consequences of this option, DON'T USE IT.  Extra care when you don't
     126      use --chicken-installation-prefix -- in this case --clear-chicken-home
     127      will remove Scheme files from your "host" CHICKEN installation.
     128      This option is only effective when --keep-repo is NOT used.
     130  --verbosity=<number>
     131      A number to indicate salmonella's verbosity level.  0 means practically
     132      silent. 1 is mostly silent and 2 (default) prints some useful information
     133      while salmonella is running.
     136===== Some quick tips by example
     138Simplest case: testing eggs using the remote egg server and the
     139chicken tools ({{chicken-install}} and {{csi}}) in the system {{PATH}}
     141  $ salmonella big-chicken slice
     144You can test all the available eggs using one of the following
     1471. If you have CHICKEN >= 4.7.2:
     149  $ salmonella `chicken-install -list`
     1522. If you don't have CHICKEN >= 4.7.2:
     154  $ salmonella `wget -q -O -`
     157If you don't want to test some specific eggs, you can skip them:
     159  $ salmonella --skip-eggs=macosx,hfs+ `chicken-install -list`
     162You can tell salmonella to use a specific CHICKEN version:
     164  $ salmonella --chicken-installation-prefix=/usr/local/chicken-4.7.3 big-chicken slice
     167Not cleaning the egg installation repository after installing each egg
     168may significantly speed up the salmonella execution time (the default
     169behavior is to set the egg installation repository empty after testing
     170each egg):
     172  $ salmonella --keep-repo big-chicken slice
     174warning: if you use --keep-repo, salmonella will not be able to catch
     175dependencies problems.
     178If you want to reuse the same egg installation repository for multiple
     179salmonella runs, you can provide a specific directory:
     181  $ salmonella --repo-dir=my-repo --keep-repo big-chicken slice
     184By default, salmonella generates a log file named {{salmonella.log}} in
     185the current directory.  You can change that by using the {{--log-file}}
     186command line option:
     188  $ salmonella --log-file=my-log-file.log big-chicken slice
     191====== Using salmonella as an egg lint tool
     193Suppose you are working on a new egg and you want to check if it is
     194working ok before releasing it (or a new version).  You can use
     195salmonella to check it:
     197  $ cd my-egg    # where your egg code is stored
     198  $ salmonella
     202==== salmonella-log-viewer
     204This tool can be used to turn salmonella log files into something
     205readable on the standard output.
     207Just provide a salmonella log file as argument to
     210    $ salmonella-log-viewer salmonella.log
     212Alternatively, you can use
     213[[/egg/salmonella-html-report|salmonella-html-report]] for a prettier
     214and more complete format.
     218==== salmonella-epidemy
     220Note: this tool is currently not supported on Windows systems.
     222{{salmonella-epidemy}} can be used to run multiple salmonella
     223instances in parallel.  It can be handy when you have a multi-core
     224machine and you want to make use of all cores, for example.  The
     225command line options are basically the same as for {{salmonella}},
     226plus a {{--instances=<number>}} to indicate how many {{salmonella}}
     227instances you want to run in parallel.
     229This tool can significantly speed up salmonella execution times on
     230multi-core machines.
     232Usage example:
     234  $ salmonella-epidemy --instances=2 big-chicken slice spiffy amb
     238==== salmonella-log-merger
     240This tool simply puts more than one salmonella log into a single one,
     241so it can be used by tools like {{salmonella-log-viewer}} and
     245Usage example:
     247  $ salmonella-log-merger --log-file=full.log file1.log file2.log
     251==== salmonella-log-inquirer
     253{{salmonella-log-inquirer}} can be used to query salmonella log files.
     254Below you can see its help message and a usage example.
     256  salmonella-log-inquirer [ -h | -help | --help ]
     257    Print this message.
     259  salmonella-log-inquirer --version
     260    Show version and exit.
     262  salmonella-log-inquirer --log-info <log file>
     263    Show information about the environment where salmonella was executed
     264    to generate the given <log-file> (e.g., environment variables, C
     265    compiler, CHICKEN version, salmonella command line etc.)
     267  salmonella-log-inquirer --statistics <log file>
     268    Show simple statistics on installation, tests and documentation from
     269    the log file .
     271  salmonella-log-inquirer --list-eggs <log file>
     272    List eggs covered in <log file>.
     274  salmonella-log-inquirer --action=<action> --egg=<egg> [ --part=<part> ] <log file>
     275    Query <action> and, optionally, <part> for <egg> in <log file>.
     277  <action>s:
     278  fetch
     279  install
     280  check-version
     281  test
     282  meta-data
     284  <part>s (the default part is "message"):
     285  message
     286  status
     287  duration
     290Usage example:
     292  $ salmonella-log-inquirer --action=fetch --egg=this salmonella.log
     293  '/home/chicken/salmonella/build/salmonella-run-publish/chicken/bin/chicken-install' -r -v -test this 2>&1
     294  this not cached
     295  resolving alias `kitten-technologies' to:
     296  resolving alias `call-cc' to:
     297  trying server ...
     298  downloading this:
     299  connecting to host "", port 80 ...
     300  requesting "/henrietta.cgi?name=this&release=5&mode=default&tests=yes" ...
     301  reading response ...
     302  HTTP/1.1 200 OK
     303  Date: Thu, 25 Jul 2019 11:47:58 GMT
     304  Server: Apache/2.2.31 (Unix) DAV/2 PHP/5.5.36 mod_fastcgi/2.4.6
     305  Connection: close
     306  Transfer-Encoding: chunked
     307  Content-Type: text/plain
     308  reading chunks .
     309  reading files ...
     310    ./
     311    ./this.egg
     312    ./this.release-info
     313    ./this.scm
     314  cp -r '/tmp/temp303b.18990'/* '/home/chicken/salmonella/build/salmonella-run-publish/salmonella-repo/repo/cache/this'
     315  this located at /home/chicken/salmonella/build/salmonella-run-publish/salmonella-repo/repo/cache/this
     318==== Log file format
     320The command line tool writes a log file which contains records in the
     321following format:
     323 (<egg> <action> <status> <message> <duration>)
     326===== <egg>
     328Can be either a symbol that indicates the egg-name or {{#f}} to
     329indicate the {{start}} and {{end}} actions (logged when salmonella is
     330started and when it finishes testing, respectively).
     332===== <action>
     334A symbol to indicate the action that was executed.  Can be one of the
     335following values:
     337* {{start}}: starting salmonella
     338* {{fetch}}: fetching egg
     339* {{install}}: installing egg
     340* {{skip}}: skipping egg
     341* {{test}}: testing egg
     342* {{end}}: salmonella has finished
     344===== <status>
     346A numeric value indicating the exit status of the executed action.
     347When the action is {{test}} and {{status}} is {{-1}}, it means that
     348the egg has no tests.
     351===== <message>
     353The output generated by the commands executed to perform {{<action>}}.
     354A string.
     357===== <duration>
     359The time (in seconds) that was taken to execute {{<action>}}.
     361For the {{start}} and {{end}} actions, the value is the seconds since
     362epoch, so the total salmonella execution time can be determined by
     363subtracting the start time from the end time.
     366=== Modules
     368Salmonella provides two modules:
     370; {{salmonella}}: the core salmonella functionality and basic data strucuture for logs ({{report}} records)
     372; {{salmonella-log-parser}}: provides procedures to access log files and compute simple statistics regarding log file data.
     375==== salmonella
     377===== report
     378<record>(report egg action status message duration)</record>
     392{{report}} objects.  Each log file registry is represented by a
     393{{report}} object (serialized as a list).
     395===== report->list
     396<procedure>(report->list report)</procedure>
     398Convert a {{report}} object to a list.
     401===== log!
     402<procedure>(log! report log-file)</procedure>
     404Print the {{report}} representation to {{log-file}}.
     407===== make-salmonella
     408<procedure>(make-salmonella tmp-dir #!key chicken-installation-prefix chicken-install-args eggs-source-dir eggs-doc-dir this-egg?)</procedure>
     410The salmonella maker.  Returns a procedure that receives symbols
     411(''methods'') to indicate the actions to be performed.  The available
     412methods are (example considering a {{salmonella}} object returned by
     415====== {{(salmonella 'init-repo!)}}
     417====== {{(salmonella 'clear-repo!)}}
     419====== {{(salmonella 'fetch <egg>)}}
     421====== {{(salmonella 'install <egg>)}}
     423====== {{(salmonella 'test <egg>)}}
     425====== {{(salmonella 'check-version <egg>)}}
     427====== {{(salmonella 'env-info)}}
     429====== {{(salmonella 'meta-data <egg>)}}
     431====== {{(salmonella 'check-dependencies <egg> <meta data>)}}
     433====== {{(salmonella 'check-category <egg> <meta data>)}}
     435====== {{(salmonella 'check-license <egg> <meta data>)}}
     437====== {{(salmonella 'check-author <egg> <meta data>)}}
     439====== {{(salmonella 'check-doc <egg>)}}
     443==== salmonella-log-parser
     445===== Reading
     447<procedure>(read-log-file filename)</procedure>
     449Reads the log file {{filename}} and returns a list of report records.
     451===== {{fetch}} action
     453====== fetch-status
     454<procedure>(fetch-status egg log)</procedure>
     456====== fetch-message
     457<procedure>(fetch-message egg log)</procedure>
     459====== fetch-duration
     460<procedure>(fetch-duration egg log)</procedure>
     464===== {{install}} action
     466====== install-status
     467<procedure>(install-status egg log)</procedure>
     469====== install-message
     470<procedure>(install-message egg log)</procedure>
     472====== install-duration
     473<procedure>(install-duration egg log)</procedure>
     477===== {{check-version}} action
     479====== check-version-status
     480<procedure>(check-version-status egg log)</procedure>
     482====== check-version-message
     483<procedure>(check-version-message egg log)</procedure>
     485====== egg-version
     486<procedure>(egg-version egg log)</procedure>
     488====== check-version-ok?
     489<procedure>(check-version-ok? egg log)</procedure>
     492===== {{test}} action
     494====== test-status
     495<procedure>(test-status egg log)</procedure>
     497====== test-message
     498<procedure>(test-message egg log)</procedure>
     500====== test-duration
     501<procedure>(test-duration egg log)</procedure>
     503====== has-test?
     504<procedure>(has-test? egg log)</procedure>
     507===== {{meta-data}} action
     509====== meta-data
     510<procedure>(meta-data egg log)</procedure>
     512====== egg-dependencies
     513<procedure>(egg-dependencies egg log)</procedure>
     515====== egg-license
     516<procedure>(egg-license egg log #!key with-test-dependencies? with-versions?)</procedure>
     520===== {{check-doc}} action
     522====== doc-exists?
     523<procedure>(doc-exists? egg log)</procedure>
     527===== {{start}} & {{end}} actions
     529====== start-time
     530<procedure>(start-time log)</procedure>
     532====== end-time
     533<procedure>(end-time log)</procedure>
     535====== total-time
     536<procedure>(total-time log)</procedure>
     538====== salmonella-info
     539<procedure>(salmonella-info log)</procedure>
     543===== Statistics
     545====== count-install-ok
     546<procedure>(count-install-ok log)</procedure>
     548====== count-install-fail
     549<procedure>(count-install-fail log)</procedure>
     551====== count-test-ok
     552<procedure>(count-test-ok log)</procedure>
     554====== count-test-fail
     555<procedure>(count-test-fail log)</procedure>
     557====== count-no-test
     558<procedure>(count-no-test log)</procedure>
     560====== count-total-eggs
     561<procedure>(count-total-eggs log #!key with-skipped?)</procedure>
     563====== count-documented
     564<procedure>(count-documented log)</procedure>
     566====== count-undocumented
     567<procedure>(count-undocumented log)</procedure>
     570===== Miscelaneous
     572====== prettify-time
     573<procedure>(prettify-time time)</procedure>
     575====== sort-eggs
     576<procedure>(sort-eggs eggs)</procedure>
     578====== log-eggs
     579<procedure>(log-eggs log)</procedure>
     581====== log-skipped-eggs
     582<procedure>(log-skipped-eggs log)</procedure>
     586=== Environment variables
     588When running, salmonella sets the {{SALMONELLA_RUNNING}} environment
     589variable.  If you need to check if your tests code is being run by
     590salmonella, this variable can be used.
     593=== Caveats
     595==== Testing executable files installed by eggs
     597Salmonella sets a private repository to install and test eggs, so when
     598extensions install programs, they get installed into
     599{{<salmonella-repo-dir>/bin}}, not into {{$CHICKEN_PREFIX/bin}}.
     601So, how to invoke (from tests/run.scm, for example) a binary
     602executable file your egg installs?  You have the following options:
     604* you can build up the absolute pathname for the executable, like
     605  {{(make-pathname (program-path) "the-executable")}}.  That is not
     606  going to work with salmonella, since the executable file will get
     607  installed into {{<salmonella-repo-dir>/bin}}, not
     608  {{$CHICKEN_PREFIX/bin}}.  Since {{program-path}} points to
     609  {{$CHICKEN_PREFIX/bin}} (or, when {{$CHICKEN_PREFIX}} is not set, it
     610  falls back to {{C_INSTALL_BIN_HOME}}, which is set during CHICKEN's
     611  build time), salmonella won't find it, as salmonella didn't install
     612  it there.  Salmonella ''may'' find it, in case you have that egg
     613  installed, but in this case you'll be testing a different program.
     615* you can rely on the executable being found in your system's {{PATH}}
     616  variable.  Salmonella actually prepends
     617  {{<salmonella-repo-dir>/bin}} to your system's {{PATH}} variable
     618  when running, so, if you call the executable file installed by your
     619  egg with with {{(system "the-executable")}}, for example, salmonella
     620  will run the right file.  On the other hand, you if run
     621  {{chicken-install}} with the {{-test}} option, and the CHICKEN tools
     622  are not in the system's {{PATH}} (e.g.
     623  {{/some/dir/not/in/PATH/chicken-install -test}}), the executable installed by your egg
     624  will not be found.
     626So far the only "portable" solution is a combination of both options,
     627which happens to be a bit ugly.  Salmonella sets the
     628{{SALMONELLA_RUNNING}} environment variable when it runs, so your test
     629script may check that variable and assume your test executable will be
     630found in {{PATH}} if it is set.  If {{SALMONELLA_RUNNING}} is not set,
     631just build up the absolute path to your executable using
     632{{(program-path)}}.  Here's an example:
     634<enscript highlight=scheme>
     635(use setup-api)
     637(define executable
     638  (make-pathname
     639   (if (get-environment-variable "SALMONELLA_RUNNING")
     640       #f ;; salmonella adds its REPO_PREFIX/bin to PATH
     641       (program-path))
     642   "executable-filename"))
     645==== salmonella-epidemy on Windows
     647{{salmonella-epidemy}} is currently not supported on Windows systems.
     650=== License
     652  Copyright (c) 2010-2019, Mario Domenech Goulart
     653  All rights reserved.
     655  Redistribution and use in source and binary forms, with or without
     656  modification, are permitted provided that the following conditions
     657  are met:
     658  1. Redistributions of source code must retain the above copyright
     659     notice, this list of conditions and the following disclaimer.
     660  2. Redistributions in binary form must reproduce the above copyright
     661     notice, this list of conditions and the following disclaimer in the
     662     documentation and/or other materials provided with the distribution.
     663  3. The name of the authors may not be used to endorse or promote products
     664     derived from this software without specific prior written permission.
     681=== Version history
     683==== Version 3.0.0
     685* CHICKEN 5 support.  Now both CHICKEN 4 and CHICKEN 5 are supported. Most of the code is shared.  Where CHICKEN 4 and 5 are not compatible, {{cond-expand}} is used for version-specific implementations.
     687* New command line tool: {{salmonella-log-inquirer}}.  This tool can be used to query salmonella log files.
     689* More strict command line parser. Now invalid command line options are detected and reported as errors.
     691* Exit non-zero in case of errors.  Salmonella now exits non-zero in case of any check failure.
     693* Add {{--clear-chicken-home}} command line option for {{salmonella}} and {{salmonella-epidemy}}.  Don't use this option unless you know exactly what you are doing.  Check the documentation of this option before using it.
     695* The {{--eggs-source-dir}} option has been removed.  It is still possible to use salmonella with a local directory containing egg sources by configuring {{chicken-install}} ({{-t local -l <eggs-source-dir>}} in CHICKEN 4 and by configuring the {{location}} form in {{PREFIX/share/chicken/setup.defaults}} in CHICKEN 5).
     697* Replace --salmonella-prefix command line option by {{--salmonella-tools-dir}}.  {{salmonella-epidemy}} relies on both {{salmonella}} and {{salmonella-log-merger}}.  Since portably determining the directory of {{salmonella-epidemy}} is not trivial, and we don't want to rely on {{$PATH}} to call external tools, {{--salmonella-tools-dir}} has been created.  With this, {{--salmonella-prefix}} became useless.
     699* Salmonella now copy {{csc}} and {{csi}} from {{<host-prefix>/bin}} to {{<tmp-dir>/bin}}.  With that, salmonella no longer adds <host-prefix>/bin to {{$PATH}}, which would cause salmonella to overlook eggs that rely on executables installed in {{<host-prefix>/bin}} but not declared as dependencies.
     701* Many bugs fixed
     704==== Version 2.7
     706* Command line tools handle {{--version}} (suggested by Michael Stapelberg)
     707* salmonella: prepend {{<repo dir>/bin}} and {{<chicken-installation-prefix>/bin}} to {{PATH}}
     708* salmonella: fix scrutinizer warnings
     709* salmonella: set {{CHICKEN_C_INCLUDE_PATH}}
     710* salmonella: print/log environment variables set when running
     711* salmonella-epidemy: delete repo-dir, unless {{--keep-repo}} is provided
     712* salmonella-epidemy: write per-instance temporary directories in repo-dir
     713* salmonella-epidemy: show when instances are finished
     714* salmonella-epidemy: fix verbosity handling
     715* salmonella-epidemy: properly escape special shell characters in {{--chicken-install-args}}
     716* salmonella-epidemy: remove {{--eggs-source-dir}} support due to concurrency issues
     717* salmonella-epidemy: unless {{--salmonella-prefix}} is provided, salmonella-epidemy calls salmonella from the same prefix
     718* salmonella-epidemy: add {{-h}}|{{--help}} command line options
     719* salmonella-epidemy: correctly handle {{--skip-eggs}}
     722==== Version 2.6
     724* Handle broken .meta files
     725* Handle version specification in {{test-depends}} (thanks to [[/users/peter-bex|Peter Bex]] for pointing that out)
     726* Fixed filenames for {{chicken}} and {{csi}} on mingw ({{.exe}} extension).  Thanks to Dan Leslie for pointing that out.
     727* The {{--this-egg}} command line option has been deprecated
     730==== Version 2.5
     732*  Log {{check-version}} actions even when fetching eggs from remote servers (i.e., no {{--eggs-source-dir}}), so version numbers can get logged
     735==== Version 2.4
     737* "Fixed" hack to remove {{-test}} from {{chicken-install}} arguments when installing eggs
     740==== Version 2.3
     742* Bug fix: reference to unbound variable {{eggs}} (thanks to Shawn Rutledge for noticing that)
     745==== Version 2.2
     747* Only create the temporary repo dir when it is necessary (i.e., not for handling {{--help}} or when there's nothing to do)
     750==== Version 2.1
     752* Hours, minutes and seconds formatted with exact numbers ({{prettify-time}})
     755==== Version 2.0
     757* Rewrite of the 1.x version
Note: See TracChangeset for help on using the changeset viewer.