Opened 6 years ago
Closed 6 years ago
#1358 closed defect (fixed)
(test-exit) returns 0 code for 'error in group outside of tests'
Reported by: | Caolan McMahon | Owned by: | Alex Shinn |
---|---|---|---|
Priority: | major | Milestone: | someday |
Component: | extensions | Version: | 4.12.0 |
Keywords: | Cc: | ||
Estimated difficulty: |
Description
I'd expect the example below to exit with code 1, instead it exits with code 0:
(use test) (test-group "A" (abort "error")) (test-group "B" (test-assert #t)) (test-exit)
$ csi -s run.scm Warning: error in group outside of tests Error: error 1 test completed in 0.0 seconds. 1 error (100%). 0 out of 1 (0%) tests passed. -- done testing A ------------------------------------------------------------ -- testing B ----------------------------------------------------------------- #t ................................................................... [ PASS] 1 test completed in 0.0 seconds. 1 out of 1 (100%) test passed. -- done testing B ------------------------------------------------------------ $ echo $? 0
Change History (4)
comment:1 Changed 6 years ago by
comment:2 Changed 6 years ago by
Perhaps a better fix would involve adding a new errors-outside-test-count
parameter? Because increasing test-failure-count
could result in the impression that there are more tests than actually exist, if the error happens after the last test.
Also, in custom reporters it might be helpful to know that there were errors outside the tests, so the test count is unreliable (because it wouldn't get around to running the remaining tests that follow the code that triggered the error.
comment:3 Changed 6 years ago by
a separate count might be better, but we only use the current count for test-exit right now, and I'm busy :P
comment:4 Changed 6 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
And here is my suggested fix: