Opened 13 years ago

Last modified 9 years ago

#630 closed defect

Test scripts that don't properly report exit status — at Version 8

Reported by: Mario Domenech Goulart Owned by:
Priority: major Milestone:
Component: extensions Version: 4.7.x
Keywords: test infrastructure Cc:
Estimated difficulty:

Description (last modified by Christian Kellermann)

The eggs listed bellow don't properly report their exit code to the test infrastructure.

Please, make them exit 0 when all tests pass or a non-zero value when one or more tests fail, so salmonella can properly indicate the tests status. Hint: for those using the test egg, you just need (test-exit) at the end of the test script.

If you test script relies on errors (or assert) to test egg features, you can assume it is ok (I'm sorry for the false alert), since on error the test script will exit 70 (which is ok for the test infrastructure).

I took a look at the last 20 lines of the run.scm files (or at the files they include/load) in the trunk dir for each egg (yeah, it's not exactly a pleasant work). If you don't intend to make a new release of your egg(s) to fix bugs or to add new features soon, please consider making a new release just to fix the tests, so we can have a more accurate idea about the shape of the set of eggs. Also consider that egg tests help to test the compiler too.

bindings
char-set-literals
condition-utils
contracts
coops-utils
couchdb
csv
datatype
digraph
directory-utils
discount
dsssl-utils
elliptic-curves
endian-blob
err5rs-arithmetic
expand-full
filepath
fmt
format-graph
format-textdiff
format
getopt-long
glpk
graph-bfs
graph-cycles
graph-dfs
graph-scc
html-form
iexpr
internet-message
internet-timestamp
interval-digraph
json-abnf
kanren
lazy-ffi
lexgen
list-utils
logpro
lookup-table
macosx
mailbox
mathh
memcached
mime
modular-arithmetic
mpi
mw-core
mw
netstring
numspell
object-graph
openal
operations
packedobjects
pandora
procedure-decoration
progress-indicators
quaternoins
rb-tree
remote-mailbox
ripemd
sassy
sexp-diff
sfht
sha2
smtp
snowdigest
specialized-io
srfi-101
srfi-19
srfi-25
srfi-27
srfi-42
srfi-45
stack
statistics
string-utils
sxml-templates
symbol-utils
synch
system
tigher-hash
tinyclos
tiny-prolog
treap
tuples
typeclass
uuid-ossp
xosd
yasos
yelp

Change History (8)

comment:1 Changed 13 years ago by felix winkelmann

I updated advice, ftl, udp, defstruct, simple-sha1 and coops.

The following eggs don't use test, use assert or perform just simple code execution examples:

big-chicken, bind, dollar, fast-generic, fps, heap-o-rama, htmlprag, inline, loop, protobj, s11n,
spock, sparse-vectors, stalin, z3.

comment:2 Changed 13 years ago by Ivan Raikov

Description: modified (diff)

comment:3 Changed 13 years ago by Ivan Raikov

Description: modified (diff)

comment:4 Changed 13 years ago by Ivan Raikov

Description: modified (diff)

comment:5 Changed 13 years ago by Matt Welland

logpro updated with (test-exit)

comment:6 Changed 13 years ago by sjamaan

Description: modified (diff)

Thank you for working through these things, Mario!

I've updated numbers (which is still not a great testsuite, but more-or-less works as expected now)

prometheus is also an egg that just uses assert and has a few examples.

I've removed these from the list as well as those Felix listed.

comment:7 Changed 13 years ago by Christian Kellermann

Description: modified (diff)

comment:8 Changed 13 years ago by Christian Kellermann

Description: modified (diff)
Note: See TracTickets for help on using tickets.