source: project/wiki/testbase @ 9766

Last change on this file since 9766 was 9766, checked in by svnwiki, 12 years ago

Changes applied for rlpowell (64.81.66.189) through svnwiki:

File size: 96.7 KB
Line 
1[[tags: egg]]
2
3== testbase
4
5[[toc:]]
6
7=== Description
8
9Macro based unit-testing facility
10
11=== Author
12
13[[Kon Lovett]], with portions from test-infrastructure 1.0 by Peter Keller
14
15=== Requirements
16
17* [[vector-lib]]
18* [[syntax-case]]
19* [[rlimit]]
20* [[srfi-37]]
21* [[srfi-42]]
22* [[args]]
23* [[testbase-results]]
24
25=== Download
26
27[[http://www.call-with-current-continuation.org/eggs/testbase.egg|testbase.egg]]
28
29=== Documentation
30
31This extension provides a macro based unit testing facility based upon
32expectations concerning evaluations of expressions. Expectations are grouped
33into test containers, such as test-suite and test/case. In addition to
34expectation and container test forms a set of annotation forms are supplied.
35All test forms, be they containers, annotations, or expectations, return a test
36result object. Result objects generated by the test forms enclosed by a test
37container are composed by the container into the container's test result
38object. The set of test results forms a tree.
39
40This result tree is then passed to either user defined functions which
41traverse the tree manipulating it in any way desired. API functions to deal
42with the result types are supplied. Examples of existing functions that
43traverse the result are the test results output routines.
44
45
46==== Test File
47
48A {{test-file}} is a Scheme source file consisting of testbase forms and
49necessary auxiliary definitions. The general template is:
50
51<enscript highlight=scheme>
52(require-extension testbase)
53.. other extensions, includes, imports, etc. as needed ..
54
55#| TestBase
56.. see "testbase-driver" documentation ..
57|#
58
59.. any expressions necessary to testing ..
60
61(define-test foo-test ... )
62
63(run-test "Foo Tests")
64</enscript>
65
66While possible to use testbase forms without invoking {{run-test}} command line
67processing is not performed.
68
69Command Line Options:
70
71; ---failure-exit CODE : Specifiy an exit code & set the early-exit-upon-failure-option. See {{test::exit-on-failure?}}, {{test::run}}, and {{run-test}}.
72
73; -b ---batch : Specify batch mode. See {{run-test}}.
74
75; -i ---interactive : Specify interactive (or driven) mode. See {{run-test}}.
76
77; -V ---version : Display the version.
78
79; -h ---help : Display help message.
80
81
82==== Macro and Procedure Naming Conventions
83
84A distinction is made between forms that are to be used within a test, and
85those that are to be used outside of a test. In general, those forms that are
86for use within a test are macros, and those for use outside a test are
87procedures.
88
89Procedures have a {{test::}} namespace prefix, such as {{test::timing}}
90
91Macros do not have a {{test::}} namespace prefix. Since they can
92only pollute the namespace of a single file a looser naming convention is
93used.
94
95
96==== Option Expansion Control
97
98The expanded source of the test code is controlled by a set of syntax
99constants. To drop or add an option place the appropriate syntax form below in
100a test file.
101
102{{FLAG}} is a boolean, {{#t}} or {{#f}}.
103
104===== test::timing
105
106<macro>(define-for-syntax test::timing FLAG)</macro>
107
108{{test::timing}} controls the expectation timing option. Default is {{#f}}.
109
110===== test::filtering
111
112<macro>(define-for-syntax test::filtering FLAG)</macro>
113
114{{test::filtering}} controls the test result filtering
115option. Default is {{#f}}.
116
117===== test::catching
118
119<macro>(define-for-syntax test::catching FLAG)</macro>
120
121{{test::catching}} controls the expectation error/exception
122trapping option. Default is {{#t}}.
123
124Dropping error/exception trapping is not suggested in the general
125case, since any intermediate errors will abort the entire test. Does
126not affect {{expect-exception}}.
127
128===== test::selecting
129
130<macro>(define-for-syntax test::selecting FLAG)</macro>
131
132{{test::selecting}} controls the test selective evaluation option. Default is {{#f}}.
133
134===== test::limiting
135
136<macro>(define-for-syntax test::limiting FLAG)</macro>
137
138{{test::limiting}} controls the test resource limit option. Default is {{#f}}.
139
140
141==== Multiline String Construction
142
143Due to a bug the Chicken multiline string forms cannot be used. This
144macro is provided as a workaround. For use in a test-file.
145
146===== MULTILINE-STRING
147
148<macro>(MULTILINE-STRING STRING ...)</macro>
149
150Yes, the macro name is uppercase.
151
152{{STRING}} is any scheme string. The strings are suffixed with a
153newline and concatenated.
154
155
156==== Immediate Test
157
158This macro will evaluate in a left to right fashion the clauses inside
159it and output the results. The clauses are a sequence of {{(Expression
160Expectation)}} forms. A clause is evaluated as {{(expect-equal
161EXPRESSION EXPECTED)}}. For use with a REPL
162(Read-Eval-Print-Loop).
163
164The results are not kept and no other forms are allowed in the body.
165
166===== test
167
168<macro>(test (EXPECTED EXPRESSION) ...)</macro>
169
170{{EXPRESSION}} is any scheme expression.
171
172{{EXPECTED}} is any scheme expression, the expected result of
173{{EXPRESSION}}.
174
175==== Test Procedure Definition
176
177All test forms are grouped into a test procedure. Test forms cannot be used
178outside of a test procedure without generating an error.
179
180===== define-test
181
182<macro>(define-test PROCEDURE-VARIABLE [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)</macro>
183
184<macro>(define-test (PROCEDURE-VARIABLE VARIABLE ...) [TEST-NAME] [(initial FORM ...)] [(final FORM ...)] CLAUSE ...)</macro>
185
186Defines a variable, {{PROCEDURE-VARIABLE}}, with a {{TEST-PROCEDURE}}
187value.
188
189{{TEST-NAME}} must evaluate to a Scheme {{atom}}. Must be unique
190in the test-file. When missing a string name is formed from the
191{{PROCEDURE-VARIABLE}}.
192
193{{VARIABLE}} is a lambda variable, a formal argument to the
194test-procedure.
195
196When supplied the {{(initial FORM ...)}} will cause each {{FORM}} to
197be expanded at the beginning of the test procedure.  {{FORM}} can be
198any Scheme form legal at the beginning of a {{lambda}} body. The
199{{initial}} section will be evaluated before the first {{CLAUSE}}.  If define forms occur in this section, the variables will be in scope for the whole test.
200
201When supplied the {{(final FORM ...)}} will cause each {{FORM}} to be
202expanded at the end of the test procedure.  {{FORM}} can be any Scheme
203form legal at the end of a {{lambda}} body. The {{final}} section will
204be evaluated after the last {{CLAUSE}} and before the destructor is
205activated!
206
207The forms in the {{initial}} and {{final}} sections do not need to
208return test result objects; they are not considered part of the test
209body. In other words, a {{FORM}} is not a {{CLAUSE}}.
210
211{{CLAUSE}} is any of the test API elements. Actually {{CLAUSE}} can be
212any Scheme {{FORM}} but be careful. A {{CLAUSE}} must return a test
213result object. Unless this can be guaranteed for a {{FORM}} a runtime
214error can be generated.
215
216The {{DTOR-NAME}} will be {{test-dtor}}. See {{test-suite}}.
217
218The {{ESCR-NAME}} will be {{test-escr}}. See {{test-suite}}.
219
220Can be called directly, as {{(PROCEDURE-VARIABLE)}} or
221{{(PROCEDURE-VARIABLE ARGUMENT ...)}}, and returns a test-result.
222
223A parameterized test definition should be complemented by one or more test
224declarations, a {{declare-test}} form. Otherwise the test cannot be
225invoked by {{test::run}}.
226
227===== declare-test
228
229<macro>(declare-test PROCEDURE-VARIABLE TEST-NAME ARGUMENT ...)</macro>
230
231Declares an actual test for the {{PROCEDURE-VARIABLE}}.
232
233{{ARGUMENT}} is a scheme object. An actual argument to the
234corresponding test-procedure formal argument.
235
236{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
237
238Must occur lexically after the {{PROCEDURE-VARIABLE}} definition
239in the test-file!
240
241An actual-test is not a test-procedure and cannot be called directly. The
242actual-test is tied to the {{PROCEDURE-VARIABLE}}, and can only be
243invoked by {{test::run}}.
244
245==== Test Procedure Operations
246
247{{TEST-IDENTIFIER}} is either a {{PROCEDURE-VARIABLE}}
248bound to a {{TEST-PROCEDURE}} or a {{TEST-NAME}}. When a
249{{TEST-NAME}} the corresponding {{TEST-PROCEDURE}} will be
250substituted.
251
252===== test::procedure?
253
254<procedure>(test::procedure? OBJECT)</procedure>
255
256Is the {{OBJECT}} a defined test-procedure object?
257
258===== test::formal-arguments
259
260<procedure>(test::formal-arguments TEST-IDENTIFIER)</procedure>
261
262Returns the formal argument list of the test-procedure.
263
264===== test::declarations
265
266<procedure>(test::declarations TEST-IDENTIFIER)</procedure>
267
268Returns a list of {{(test-procedure actual-test-name
269actual-test-arguments)}} for the test-procedure.
270
271{{TEST-IDENTIFIER}} should identify a parameterized test-procedure.
272
273===== test::procedure
274
275<procedure>(test::procedure TEST-IDENTIFIER)</procedure>
276
277Returns a test-procedure or {{#f}}.
278
279===== test::name
280
281<procedure>(test::name TEST-IDENTIFIER)</procedure>
282
283Returns the test-name of the test-procedure.
284
285===== test::procedure-variable
286
287<procedure>(test::procedure-variable TEST-IDENTIFIER)</procedure>
288
289Returns the defined symbol of the test-procedure.
290
291===== test::styler
292
293<procedure>(test::styler TEST-IDENTIFIER)</procedure>
294
295Returns the default output style procedure for the test-procedure, or
296{{#f}} when no default styler is set.
297
298===== test::styler-set!
299
300<procedure>(test::styler-set! TEST-IDENTIFIER [STYLER])</procedure>
301
302Sets the default output style procedure for the test-procedure. When
303missing or {{#f}} then no default styler is set.
304
305===== test::tolerance
306
307<procedure>(test::tolerance TEST-IDENTIFIER)</procedure>
308
309Returns the current in-exact number comparison tolerance of the
310test-procedure.
311
312===== test::tolerance-set!
313
314<procedure>(test::tolerance-set! TEST-IDENTIFIER TOLERANCE)</procedure>
315
316Sets the current in-exact number comparison tolerance of the test-procedure to
317{{TOLERANCE}}.
318
319===== test::reset!
320
321<procedure>(test::reset! TEST-IDENTIFIER)</procedure>
322
323Sets the state of the test-procedure to the initial settings.
324
325===== test::structure
326
327<procedure>(test::structure TEST-IDENTIFIER)</procedure>
328
329Returns the internal structure of test-procedure as a tree.
330
331The tree is the test hierarchy, where each node of the tree is
332{{(suite | case | expect TEST-NAME ...)}}, where {{...}} are any
333sub-trees.
334
335The top-level {{suite}} element value refers to the test-procedure.
336
337===== test::add-filter!
338
339<procedure>(test::add-filter! TEST-IDENTIFIER FILTER-PROCEDURE)</procedure>
340
341Places {{FILTER-PROCEDURE}} as the head of the result filter chain
342for the test-procedure.
343
344{{FILTER-PROCEDURE}} must be a procedure of one argument, a test
345result object, and it must return a test result object. Not necessarily the
346same one.
347
348Filtering occurs ''during'' test element evaluation.
349
350===== test::remove-filter!
351
352<procedure>(test::remove-filter! TEST-IDENTIFIER FILTER-PROCEDURE)</procedure>
353
354Removes {{FILTER-PROCEDURE}} from the result filter chain for the
355test-procedure.
356
357===== test::add-echo!
358
359<procedure>(test::add-echo! TEST-IDENTIFIER [PORT] [KEY VALUE ...])</procedure>
360
361Causes all test results for the test-procedure to be passed to the
362''echo-procedure''. The signature of an echo-procedure is
363{{(result-object port association-list -> unspecified)}}. The
364{{association-list}} holds options for the echo-procedure.
365
366When {{PORT}} missing the {{(current-output-port)}} is assumed.
367
368The {{PORT}} is unique. Only one echo per {{PORT}}.
369
370{{KEY}} is a symbol. The reserved keys are:
371
372* {{echo-procedure}}, where the {{VALUE}} is a procedure with an echo-procedure signature.
373* {{include-expectation-specifics}}, where the {{VALUE}} is a boolean indicating whether the value fields for an expectation are to be echoed, default is {{#f}}.
374
375When the {{KEY}} {{echo-procedure}} is missing the default
376echo-procedure is used. The test results are echoed to the {{PORT}} as
377an association list. One list per line.
378
379The echo-procedure is invoked after each test element evaluation. As
380such it must deal with test-container result objects in an incomplete
381state. Specifically the result-list field will be {{#f}} when a
382test-container is entered. It also must deal with ignore-results,
383which are stripped from the test-container result-list upon exit from
384the container.
385
386===== test::remove-echo!
387
388<procedure>(test::remove-echo! TEST-IDENTIFIER [PORT])</procedure>
389
390Removes the {{PORT}} echo for the test-procedure.
391
392===== test::set-echo-option!
393
394<procedure>(test::set-echo-option! TEST-IDENTIFIER [PORT] [KEY VALUE ...])</procedure>
395
396Modifies the echo specification for the specified {{PORT}}.
397
398The {{PORT}}, {{KEY}}, and {{VALUE}} parameters have the same interpretation as the {{test::add-echo!}} procedure.
399
400===== test::run-mode
401
402<procedure>(test::run-mode)</procedure>
403
404Returns the test run mode, either {{batch}}, {{interactive}} or {{driven}}.
405
406===== test::selection-mode
407
408<procedure>(test::selection-mode TEST-IDENTIFIER)</procedure>
409
410Returns the test selection mode, either {{take}} or {{skip}}.
411
412When the mode is {{take}} then unless a test is to be specifically skipped, it is taken.
413
414When the mode is {{skip}} then unless a test is to be specifically taken, it is skipped.
415
416The test procedure is always taken.
417
418===== test::select!
419
420<procedure>(test::select! SELECTION-MODE TEST-IDENTIFIER [TEST-NAME | (TEST-TYPE TEST-NAME)] ...)</procedure>
421
422{{SELECTION-MODE}} is one of {{'take}} or {{'skip}}.
423
424{{TEST-TYPE}} is one of {{'suite}}, {{'case}}, or {{'expect}}.
425
426Select the test element(s) identified by {{(TEST-TYPE TEST-NAME)}} of
427the test-procedure. When only {{TEST-NAME}} is provided any test type
428will match.
429
430When no tests are supplied all existing selections are removed and the test
431selection mode is set to the {{SELECTION-MODE}}.
432
433<enscript highlight=scheme>
434(test::select! 'skip my-test-proc)
435;; Now every test in 'my-test-proc' will be skipped.
436
437(test::select! 'take my-test-proc '(case "Top Case") "Foo")
438;; Now every test in 'my-test-proc' except the test-case "Top Case",
439;; and any test named "Foo".
440</enscript>
441
442When a test container is selected all contained tests are similarly selected.
443
444'''Independent of the {{(skip CLAUSE ...)}} form.'''
445
446===== test::selections
447
448<procedure>(test::selections TEST-IDENTIFIER)</procedure>
449
450Returns a list of the test selections, or {{#f}} when there are no selections.
451
452The list is composed of sublists - {{(#f|test|suite|case|expect TEST-NAME take|skip)}}.
453
454===== test::resource-limit
455
456<procedure>(test::resource-limit TEST-IDENTIFIER RLIMIT)</procedure>
457
458Returns the current value of {{RLIMIT}} for the test-procedure.  The
459return value will either be a number or {{#f}}, indicating no set
460limit.
461
462RLIMIT is one of {{'cpu}}, {{'fsize}}, {{'nofile}}, {{'nproc}},
463{{'data}}.
464
465'''Only operational on *NIX platforms.'''
466
467===== test::resource-limit-set!
468
469<procedure>(test::resource-limit-set! TEST-IDENTIFIER RLIMIT VALUE)</procedure>
470
471Sets the current value of {{RLIMIT}} for the test-procedure to
472{{VALUE}}.
473
474RLIMIT is one of {{'cpu}}, {{'fsize}}, {{'nofile}}, {{'nproc}},
475{{'data}}.
476
477{{VALUE}} is a positive integer or {{#f}}, indicating no
478set limit.
479
480When a {{'cpu}} or {{'fsize}} limit is reached the test will terminate
481with a termination result indicating the resource limit exceeded.
482
483When a {{'nofile}}, {{'nproc}}, or {{'data}} limit is reached the test
484will terminate with a termination result.
485
486'''Only operational on *NIX platforms.'''
487
488===== test::procedures
489
490<procedure>(test::procedures)</procedure>
491
492Returns a list of all the test-procedures.
493
494===== test::for-each
495
496<procedure>(test::for-each PROCEDURE [TEST-IDENTIFIER ...])</procedure>
497
498Calls {{PROCEDURE}} with every supplied test procedure, or all test
499procedures when nothing supplied.
500
501{{PROCEDURE}} takes one argument, a test-procedure.
502
503===== test::map
504
505<procedure>(test::map PROCEDURE [TEST-IDENTIFIER ...])</procedure>
506
507Calls {{PROCEDURE}} with every supplied test procedure, or all test
508procedures when nothing supplied. Returns a list of the procedure results.
509
510{{PROCEDURE}} takes one argument, a test-procedure.
511
512===== test::forget!
513
514<procedure>(test::forget! [TEST-IDENTIFIER ...])</procedure>
515
516Causes the specified test-procedures, or all when nothing specified, to be
517dropped.
518
519'''Really only for use during test development with a REPL.'''
520
521===== test::exit-on-failure?
522
523<procedure>(test::exit-on-failure? [CODE-OR-FLAG])</procedure>
524
525When {{CODE-OR-FLAG}} is supplied sets the early-exit-upon-failure-option. Otherwise
526returns the current setting.
527
528{{CODE-OR-FLAG}} is either {{#f}}, to reset the option, or an integer, to set the option.
529
530The initial setting is {{#f}}.
531
532===== test::run
533
534<procedure>(test::run [MESSAGE [TEST-IDENTIFIER ...]])</procedure>
535
536{{MESSAGE}} is some identity for the set of the test-procedures.
537When missing or {{#f}} the process command name is used.
538
539Displays a header with {{MESSAGE}}, and outputs the result of
540calling each test-procedure with the {{test::styler}}. When a
541{{test::styler}} is not set a simple success/failure message is
542given.
543The test-procedures are called in left to right order of the
544{{TEST-IDENTIFIER}} arguments. When a {{TEST-IDENTIFIER}} is not
545specified all the defined test-procedures are called in the order of
546definition in the test-file.
547
548For parameterized test-procedures the corresponding test declarations
549are run.
550
551When {{(test::exit-on-failure?)}} and some test fails then testbase will exit
552with the exit-code.
553
554When interactive mode without a console there is no output by default. See
555{{test::add-echo!}}.
556
557
558
559==== Run Tests
560
561Run test-procedures and output test results.
562
563===== run-test
564
565<macro>(run-test [MESSAGE] [FORM ...])</macro>
566
567Usually the last expression in a test file. Performs command-line processing.
568
569{{MESSAGE}} is some identity for the test. When missing the
570process command name is used. Use {{""}} for no message.
571
572{{FORM ...}} are instances of Scheme forms. When missing {{(test::run
573MESSAGE)}} is used.
574
575This macro will check the process command line for a {{batch}}
576mode (-b or --batch) or {{interactive}} mode (-i --interactive)
577option. When {{interactive}} mode without an attached console the mode
578is called {{driven}}. The default is {{batch}} mode.
579
580When {{batch}} mode the {{FORM ...}} will be evaluated.
581When {{FORM ...}} are missing then <em>every</em> test-procedure is
582run with test result output using the test-procedure styler. See
583{{test::run}}.
584
585When {{interactive}} mode a REPL will be entered.
586
587When {{driven}} mode a REPL will be entered. Test results will be
588reported as for {{test::add-echo!}}.
589
590'''Test macros are not automatically available in the REPL. They must be
591explicitly loaded.'''
592
593==== Test Macros
594
595Test macros are used to generate a test result object. A test
596procedure body is composed of a sequence of test macros.
597
598The test macros are useful only within a test procedure or a test
599container. A runtime error will result when used outside of a test
600procedure.
601
602==== Test Containers
603
604Test containers are test forms that group test forms. Test container
605are available in long and short forms. The short form provides fewer
606options. A test container imposes an evaluation termination mode on
607the contained tests. Test containers may nest.
608
609===== test-suite
610
611<macro>(test-suite TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro>
612
613<macro>(test/suite TEST-NAME [((VARIABLE VALUE) ...)] CLAUSE ...)</macro>
614
615Evaluates all contained test forms, {{CLAUSE}}, in a left to right order, thru any failures.
616
617{{TEST-NAME}} must evaluate to a Scheme {{atom}}. Note that {{#f}} is
618interpreted as a missing name. Further, {{#f}} is assumed when a name
619is missing. Thus, the name of all unnamed test elements is the same -
620{{#f}}.
621
622{{DTOR-NAME}} is an unquoted symbol for an automatic destructor object
623that gets called when the test suite completes for any reason. This
624symbol is bound to a destructor object and is available to you in the
625{{CLAUSE ...}} section of the test suite. See below for the
626description of the destructor object interface.
627
628{{test/suite-dtor}} is the {{test/suite}} destructor name.
629
630{{ESCR-NAME}} is an unquoted symbol for an escape procedure
631available in the body of the test suite. Usually this escape procedure is
632passed to {{(terminate ...)}} which calls it for you and performs
633other tasks. It is not recommended to call the escape procedure directly.
634
635{{test/suite-escr}} is the {{test/suite}} escape procedure name.
636
637{{(warn MESSAGE)}} allows you to specify a warning object, usually a
638string, that gets associated with the test suite. The {{warn}}
639function name is actually a syntax reserved word in the macro.
640
641{{((VARIABLE VALUE) ...)}} are let-style bindings that you may create
642and exist in the lexical scope of the test suite.
643
644{{CLAUSE ...}} are uses of a test container, annotation, or
645expectation macros. The clauses are evaluated in a left to right
646fashion.
647
648While you may use the expectation macros directly in a test suite,
649doing so has a drawback. If the expectation fails, the test suite
650macro will continue evaluating until all clauses are evaluated or the
651escape procedure mechanism is activated. This is different from a
652{{test-case}} macro where upon discovery of a failed expectation,
653evaluation stops immediately.
654
655===== test-case
656
657<macro>(test-case TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro>
658
659<macro>(test/case TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] CLAUSE ...)</macro>
660
661Evaluates all contained test forms, {{CLAUSE}}, in a left to right
662order, until failure.
663
664{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
665
666{{DTOR-NAME}} is an unquoted symbol for an automatic destructor object
667that gets called when the test case completes for any reason. This
668symbol is bound to a destructor object and is available to you in the
669{{CLAUSE ...}} section of the test case. See below for the description
670of the destructor object interface.
671
672{{test/case-dtor}} is the {{test/case}} destructor name.
673
674{{ESCR-NAME}} is an unquoted symbol for an escape procedure available
675in the body of the test case. Usually this escape procedure is passed
676to {{(terminate ...)}} which calls it for you and performs other
677tasks. It is not recommended to call the escape procedure directly.
678
679{{test/case-escr}} is the {{test/case}} escape procedure name.
680
681{{(warn MESSAGE)}} allows you to specify a warning object, usually a
682string, that gets associated with the test case. The {{warn}} function
683name is actually a syntax reserved word in the macro.
684
685{{((VARIABLE VALUE) ...)}} are let-style bindings that you may create
686and exist in the lexical scope of the test case.
687
688{{CLAUSE ...}} are uses of a test container, annotation, or
689expectation macros. The clauses are evaluated in a left to right
690fashion.
691
692'''Note''': Upon discovery of a failed expectation, the test case
693stops its evaluation and returns with the test results up thru the
694failed expectation. This behavior can be changed using the
695{{test-case-evaluate-thru-failure}} macro. Reverts to default at the
696start of each {{test-case}}.
697
698===== test-collect
699
700<macro>(test-collect TEST-NAME DTOR-NAME ESCR-NAME [(warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)</macro>
701
702<macro>(test/collect TEST-NAME (warn MESSAGE)] [((VARIABLE VALUE) ...)] EXPRESSION ...)</macro>
703
704A special version of {{test-case}}. Since every test form must
705evaluate to a test-result the use of looping forms to generate test
706values is very difficult. The collect container provides a method to
707gather the results of such generated tests.
708
709{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
710
711{{DTOR-NAME}} is an unquoted symbol for an automatic destructor object
712that gets called when the test collect completes for any reason. This
713symbol is bound to a destructor object and is available to you in the
714{{CLAUSE ...}} section of the test collect. See below for the
715description of the destructor object interface.
716
717{{test/collect-dtor}} is the {{test/collect}} destructor name.
718
719{{ESCR-NAME}} is an unquoted symbol for an escape procedure available
720in the body of the test collect. Usually this escape procedure is
721passed to {{(terminate ...)}} which calls it for you and performs
722other tasks. It is not recommended to call the escape procedure
723directly.
724
725{{test/collect-escr}} is the {{test/collect}} escape procedure name.
726
727{{(warn MESSAGE)}} allows you to specify a warning object, usually a
728string, that gets associated with the test collect. The {{warn}}
729function name is actually a syntax reserved word in the macro.
730
731{{((VARIABLE VALUE) ...)}} are let-style bindings that you may create
732and exist in the lexical scope of the test collect.
733
734{{EXPRESSION}} is either an arbitrary Scheme expression or the form
735{{(collect-test CLAUSE)}}, where {{CLAUSE}} is a use of a test
736container, annotation, or expectation macro. The expressions are
737evaluated in a left to right fashion.
738
739The {{collect-test}} macro "appends" the test result of the {{CLAUSE}}
740to the results of the enclosing test collect container.
741
742'''Note''': Upon discovery of a failed expectation, the test collect
743stops its evaluation and returns with the test results up thru the
744failed expectation. This behavior can be changed using the
745{{test-case-evaluate-thru-failure}} macro. Reverts to default at the
746start of each {{test-case}}.
747
748
749==== Test Flags
750
751These macros set test execution flags.
752
753===== test-case-evaluate-thru-failure
754
755<macro>(test-case-evaluate-thru-failure BOOLEAN)</macro>
756
757This macro will set the test-case evaluation continuation flag.
758
759The starting value is {{#f}}.
760
761===== test-timing
762
763<macro>(test-timing BOOLEAN)</macro>
764
765This macro will set the expectation timing flag.
766
767The starting value is {{#f}}.
768
769Note that if timing expansion is turned off setting this flag has no effect.
770
771===== test-evaluation-mode?
772
773<macro>(test-evaluation-mode?)</macro>
774
775This macro will expand into a test for the evaluation mode.
776
777Potentially useful within test procedure {{initial}} and {{final}} forms.
778
779==== Test Binding Forms
780
781===== test-letrec
782
783<macro>(test-letrec ((VARIABLE VALUE) ...) CLAUSE ...)</macro>
784
785Defines {{VARIABLE}}, in a new lexical scope, as the result of
786evaluating {{EXPRESSION}}.
787
788{{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of
789the definitions.
790
791The last {{CLAUSE ...}} in the clause body <em>must</em> be a test
792form of some kind, such as an expectation.
793
794===== test-let*
795
796<macro>(test-let* ((VARIABLE VALUE) ...) CLAUSE ...)</macro>
797
798Defines {{VARIABLE}}, in a new lexical scope, as the result of
799evaluating {{EXPRESSION}}.
800
801{{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of
802the definitions.
803
804The last {{CLAUSE ...}} in the clause body ''must'' be a test form of
805some kind, such as an expectation.
806
807===== test-let
808
809<macro>(test-let ((VARIABLE VALUE) ...) CLAUSE ...)</macro>
810
811Defines {{VARIABLE}}, in a new lexical scope, as the result of
812evaluating {{EXPRESSION}}.
813
814{{CLAUSE ...}} are as for {{test-suite}}. Evaluated in the scope of
815the definitions.
816
817The last {{CLAUSE ...}} in the clause body <em>must</em> be a test
818form of some kind, such as an expectation.
819
820==== Test Expectations
821
822An expectation at its core simply evaluates its arguments and check to
823see if it matches the expectation. The positive or negative result is
824encapsulated, along with other things such as the unevaluated
825expressions being checked and some messages supplied with each
826expectation into a particular type of black box object that one can
827query with the appropriate API calls (detailed below).
828
829Expectations all have a descriptive message that can be bound to them,
830along with an optional warning syntax detailed below. A design
831decision was made to supply expectation macros for the usual types of
832expectations a user needs because this reduced the abstractness of an
833expectation into something more manageable.
834
835Expectations are normally evaluated with an exception catcher. An
836unexpected exception is treated as expectation failure. The option may
837be defeated by redefining {{test::if-catching}}.
838
839Any exceptions in setup or teardown expressions will be caught and
840treated as an abnormal termination of the entire test.
841
842===== insist
843
844<macro>(insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION => EXPECTED)</macro>
845
846<macro>(insist [TEST-NAME] [(warn MESSAGE)] EXPRESSION (=> PREDICATE) EXPECTED)</macro>
847
848This expectation checks to see if the evaluated expression passed to
849it meets the predicate condition.
850
851{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
852
853{{(warn MESSAGE)}} allows you to specify a warning object, usually a
854string, that gets associated with the expectation. The {{warn}}
855function name is actually a syntax reserved word in the macro.
856
857{{PREDICATE}} is a procedure of two arguments to perform the test. The
858default is {{equal?}}.
859
860{{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}}
861<em>must</em> match via {{PREDICATE}} in order for this expectation to
862return a positive result.
863
864{{EXPRESSION}} should return a value meeting the conditions of the
865predicate.
866
867===== expect-unary
868
869<macro>(expect-unary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPRESSION)</macro>
870
871This expectation checks to see if the evaluated expression passed to
872it meets the predicate condition. Allows unary and binary predicates.
873
874{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
875
876{{(warn MESSAGE)}} allows you to specify a warning object, usually a
877string, that gets associated with the expectation. The {{warn}}
878function name is actually a syntax reserved word in the macro.
879
880{{KIND}} is a string stating the kind of test.
881
882{{PREDICATE}} is a procedure of one argument to perform the test.
883
884{{EXPRESSION}} should return a value meeting the conditions of the
885predicate.
886
887===== expect-binary
888
889<macro>(expect-binary [TEST-NAME] [(warn MESSAGE)] KIND PREDICATE EXPECTED EXPRESSION)</macro>
890
891This expectation checks to see if the evaluated expression passed to
892it meets the predicate condition. Allows unary and binary predicates.
893
894{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
895
896{{(warn MESSAGE)}} allows you to specify a warning object, usually a
897string, that gets associated with the expectation. The {{warn}}
898function name is actually a syntax reserved word in the macro.
899
900{{KIND}} is a string stating the kind of test.
901
902{{PREDICATE}} is a procedure of two arguments to perform the test.
903
904{{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}}
905<em>must</em> match via {{PREDICATE}} in order for this expectation to
906return a positive result.
907
908{{EXPRESSION}} should return a value meeting the conditions of the
909predicate.
910
911===== expect-zero
912
913<macro>(expect-zero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
914
915This expectation checks to see if the evaluated expression passed to
916it is numerically equal to zero.
917
918{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
919
920{{(warn MESSAGE)}} allows you to specify a warning object, usually a
921string, that gets associated with the expectation. The {{warn}}
922function name is actually a syntax reserved word in the macro.
923
924{{EXPRESSION}} should return a number.
925
926===== expect-nonzero
927
928<macro>(expect-nonzero [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
929
930This expectation checks to see if the evaluated expression passed to it is
931numerically not equal to zero.
932
933{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
934
935{{(warn MESSAGE)}} allows you to specify a warning object, usually
936a string, that gets associated with the expectation. The {{warn}}
937function name is actually a syntax reserved word in the macro.
938
939{{EXPRESSION}} should return a number.
940
941===== expect-positive
942
943<macro>(expect-positive [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
944
945This expectation checks to see if the evaluated expression passed to
946it is a positive value greater than zero.
947
948{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
949
950{{(warn MESSAGE)}} allows you to specify a warning object, usually
951a string, that gets associated with the expectation. The {{warn}}
952function name is actually a syntax reserved word in the macro.
953
954{{EXPRESSION}} should return a number.
955
956===== expect-negative
957
958<macro>(expect-negative [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
959
960This expectation checks to see if the evaluated expression passed to
961it is a negative value less than zero.
962
963{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
964
965{{(warn MESSAGE)}} allows you to specify a warning object, usually
966a string, that gets associated with the expectation. The {{warn}}
967function name is actually a syntax reserved word in the macro.
968
969{{EXPRESSION}} should return a number.
970
971===== expect-near
972
973<macro>(expect-near [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)</macro>
974
975This expectation checks to see if {{(< (abs (- EXPECTED EXPRESSION))
976TOL)))}} is true.
977
978{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
979
980{{(warn MESSAGE)}} allows you to specify a warning object, usually
981a string, that gets associated with the expectation. The {{warn}}
982function name is actually a syntax reserved word in the macro.
983
984{{EXPECTED}} is evaluated and represents the value the
985{{EXPRESSION}} <em>must</em> be "near" to in order for this
986expectation to return a positive result.
987
988{{EXPRESSION}} should return a number.
989
990{{TOL}} is a single expression which, when evaluated, must return
991a tolerance value, a small inexact number. When missing the current tolerance
992value is used, usually {{0.0001}}.
993
994The {{(expect-near EXPECTED TOL EXPRESSION)}} variant is not
995supported!
996
997===== expect-true
998
999<macro>(expect-true [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1000
1001This expectation checks to see if the evaluated expression passed to
1002it is the value {{#t}}.
1003
1004{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1005
1006{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1007string, that gets associated with the expectation. The {{warn}}
1008function name is actually a syntax reserved word in the macro.
1009
1010{{EXPRESSION}} should return {{#t}}.
1011
1012===== expect-false
1013
1014<macro>(expect-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1015
1016This expectation checks to see if the evaluated expression passed to
1017it is the value {{#f}}.
1018
1019{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1020
1021{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1022string, that gets associated with the expectation. The {{warn}}
1023function name is actually a syntax reserved word in the macro.
1024
1025{{EXPRESSION}} should return {{#f}}.
1026
1027===== expect-success
1028
1029<macro>(expect-success [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1030
1031This expectation checks to see if the evaluated expression passed to
1032it is not the value {{#f}}, or a non-error exception occurred.
1033
1034{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1035
1036{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1037string, that gets associated with the expectation. The {{warn}}
1038function name is actually a syntax reserved word in the macro.
1039
1040{{EXPRESSION}} should return {{#t}}, or generate a non-error
1041exception.
1042
1043===== expect-failure
1044
1045<macro>(expect-failure [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1046
1047This expectation checks to see if the evaluated expression passed to
1048it is the value {{#f}}, or an error exception occurred.
1049
1050{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1051
1052{{(warn MESSAGE)}} allows you to specify a warning object, usually
1053a string, that gets associated with the expectation. The {{warn}}
1054function name is actually a syntax reserved word in the macro.
1055
1056{{EXPRESSION}} should return {{#f}}, or generate an error exception.
1057
1058===== expect-not-false
1059
1060<macro>(expect-not-false [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1061
1062This expectation checks to see if the evaluated expression passed to
1063it is not the value {{#f}}.
1064
1065{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1066
1067{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1068string, that gets associated with the expectation. The {{warn}}
1069function name is actually a syntax reserved word in the macro.
1070
1071{{EXPRESSION}} should not return {{#f}}, any other value is accepted.
1072
1073===== expect-not-null
1074
1075<macro>(expect-not-null [TEST-NAME] [(warn MESSAGE)] EXPRESSION)</macro>
1076
1077This expectation checks to see if the evaluated expression passed to
1078it is not the value {{#f}}.
1079
1080{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1081
1082{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1083string, that gets associated with the expectation. The {{warn}}
1084function name is actually a syntax reserved word in the macro.
1085
1086{{EXPRESSION}} should not return {{#f}}, any other value is accepted.
1087
1088===== expect-eq
1089
1090<macro>(expect-eq [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro>
1091
1092This expectation checks to see if {{(eq? EXPECTED EXPRESSION)}} is
1093true.
1094
1095{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1096
1097{{(warn MESSAGE)}} allows you to specify a warning object, usually
1098a string, that gets associated with the expectation. The {{warn}}
1099function name is actually a syntax reserved word in the macro.
1100
1101{{EXPECTED}} is evaluated and represents the value the {{EXPRESSION}}
1102''must'' be eq? to in order for this expectation to return a
1103positive result.
1104
1105{{EXPRESSION}} is a single expression which, when evaluated must
1106return an object where an eq? of this result and the {{EXPECTED}}
1107expression is {{#t}}.
1108
1109The result object this macro produce shall contain the unevaluated
1110{{EXPRESSION}} expression as a field, but not an unevaluated
1111{{EXPECTED}} expression.
1112
1113===== expect-eqv
1114
1115<macro>(expect-eqv [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro>
1116
1117This expectation checks to see if {{(eqv? EXPECTED EXPRESSION)}}
1118is true.
1119
1120{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1121
1122{{(warn MESSAGE)}} allows you to specify a warning object, usually
1123a string, that gets associated with the expectation. The {{warn}}
1124function name is actually a syntax reserved word in the macro.
1125
1126{{EXPECTED}} is evaluated and represents the value the
1127{{EXPRESSION}} <em>must</em> be eqv? to in order for this expectation
1128to return a positive result.
1129
1130{{EXPRESSION}} is a single expression which, when evaluated must
1131return an object where an eqv? of this result and the {{EXPECTED}}
1132expression is {{#t}}.
1133
1134The result object this macro produce shall contain the unevaluated
1135{{EXPRESSION}} expression as a field, but not an unevaluated
1136{{EXPECTED}} expression.
1137
1138===== expect-equal
1139
1140<macro>(expect-equal [TEST-NAME] [(warn MESSAGE)] EXPECTED EXPRESSION)</macro>
1141
1142This expectation checks to see if {{(equal? EXPECTED EXPRESSION)}} is
1143true.
1144
1145{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1146
1147{{(warn MESSAGE)}} allows you to specify a warning object, usually
1148a string, that gets associated with the expectation. The {{warn}}
1149function name is actually a syntax reserved word in the macro.
1150
1151{{EXPECTED}} is evaluated and represents the value the
1152{{EXPRESSION}} <em>must</em> be equal? to in order for this
1153expectation to return a positive result.
1154
1155{{EXPRESSION}} is a single expression which, when evaluated must
1156return an object where an equal? of this result and the {{EXPECTED}}
1157expression is {{#t}}.
1158
1159The result object this macro produce shall contain the unevaluated
1160{{EXPRESSION}} expression as a field, but not an unevaluated
1161{{EXPECTED}} expression.
1162
1163===== expect-unary/values
1164
1165<macro>(expect-unary/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) EXPRESSION)</macro>
1166
1167This expectation checks to see if the evaluated multi-valued expression
1168passed to it meets the predicate condition. Allows unary and binary
1169predicates.
1170
1171{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1172
1173{{(warn MESSAGE)}} allows you to specify a warning object, usually
1174a string, that gets associated with the expectation. The {{warn}}
1175function name is actually a syntax reserved word in the macro.
1176
1177{{KIND}} is a string stating the kind of test.
1178
1179{{PREDICATE}} is a procedure taking one argument and returning
1180{{#t}} or {{#f}}.
1181
1182{{EXPRESSION}} has a multi-valued result.
1183
1184===== expect-near/values
1185
1186<macro>(expect-near/values [TEST-NAME] [(warn MESSAGE)] KIND (PREDICATE ...) VALUES EXPRESSION)</macro>
1187
1188This expectation checks to see if the evaluated multi-valued expression
1189passed to it meets the predicate condition. Allows unary and binary
1190predicates.
1191
1192{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1193
1194{{(warn MESSAGE)}} allows you to specify a warning object, usually
1195a string, that gets associated with the expectation. The {{warn}}
1196function name is actually a syntax reserved word in the macro.
1197
1198{{KIND}} is a string stating the kind of test.
1199
1200{{PREDICATE}} is a procedure taking two arguments and returning {{#t}}
1201or {{#f}}.
1202
1203{{VALUES}} are the expected values.
1204
1205{{EXPRESSION}} has a multi-valued result.
1206
1207===== expect-near/values
1208
1209<macro>(expect-near/values [TEST-NAME] [(warn MESSAGE)] EXPECTED [TOL] EXPRESSION)</macro>
1210
1211This expectation checks to see if {{(< (abs (- EXPECTED[i]
1212EXPRESSION[i])) TOL)))}} is true.
1213
1214{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1215
1216{{(warn MESSAGE)}} allows you to specify a warning object, usually
1217a string, that gets associated with the expectation. The {{warn}}
1218function name is actually a syntax reserved word in the macro.
1219
1220{{EXPECTED}} is evaluated and represents the values the
1221{{EXPRESSION}} <em>must</em> be "near" to in order for this
1222expectation to return a positive result.
1223
1224{{EXPRESSION}} should return one or more inexact or exact number.
1225
1226{{TOL}} is a single expression which, when evaluated, must return
1227a tolerance value, a small inexact number. When missing the current tolerance
1228value is used, usually {{0.0001}}.
1229
1230The {{(expect-near/values EXPECTED TOL EXPRESSION)}} variant is not
1231supported!
1232
1233===== expect-eq/values
1234
1235<macro>(expect-eq/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro>
1236
1237This expectation checks to see if the result of the evaluated multi-valued
1238expression passed to it is element-wise eq? to the VALUES list. The count of
1239returned multiple values must match the length of the VALUES list.
1240
1241{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1242
1243{{(warn MESSAGE)}} allows you to specify a warning object, usually
1244a string, that gets associated with the expectation. The {{warn}}
1245function name is actually a syntax reserved word in the macro.
1246
1247{{VALUES}} is the expected values list.
1248
1249{{EXPRESSION}} has a multi-valued result.
1250
1251===== expect-eqv/values
1252
1253<macro>(expect-eqv/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro>
1254
1255This expectation checks to see if the result of the evaluated multi-valued
1256expression passed to it is element-wise eqv? to the VALUES list. The count of
1257returned multiple values must match the length of the VALUES list.
1258
1259{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1260
1261{{(warn MESSAGE)}} allows you to specify a warning object, usually
1262a string, that gets associated with the expectation. The {{warn}}
1263function name is actually a syntax reserved word in the macro.
1264
1265{{VALUES}} is the expected values list.
1266
1267{{EXPRESSION}} has a multi-valued result.
1268
1269===== expect-equal/values
1270
1271<macro>(expect-equal/values [TEST-NAME] [(warn MESSAGE)] VALUES EXPRESSION)</macro>
1272
1273This expectation checks to see if the result of the evaluated
1274multi-valued expression passed to it is element-wise equal? to the
1275VALUES list. The count of returned multiple values must match the
1276length of the VALUES list.
1277
1278{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1279
1280{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1281string, that gets associated with the expectation. The {{warn}}
1282function name is actually a syntax reserved word in the macro.
1283
1284{{VALUES}} is the expected values list.
1285
1286{{EXPRESSION}} has a multi-valued result.
1287
1288===== expect-exception
1289
1290<macro>(expect-exception [TEST-NAME] [(warn MESSAGE)] EXCEPTION-PATTERN EXPRESSION)</macro>
1291
1292This expectation checks to see if the evaluated expression passed to
1293it signals an exception that matches the supplied exception pattern
1294{{EXCEPTION-PATTERN}}.
1295
1296{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1297
1298{{(warn MESSAGE)}} allows you to specify a warning object, usually a
1299string, that gets associated with the expectation. The {{warn}}
1300function name is actually a syntax reserved word in the macro.
1301
1302{{EXCEPTION-PATTERN}} is a specialized logical expression
1303describing the expected exception, a condition as defined by
1304[[http://srfi.schemers.org/srfi-12/srfi-12.html|SRFI-12]].
1305
1306{{EXPRESSION}} should signal an exception matching {{EXCEPTION-PATTERN}}.
1307
1308  EXCEPTION-PATTERN : COND-EXN
1309                    | SIMP-EXN
1310                    | ()
1311 
1312  SIMP-EXN : KINDKEY
1313           | (property KINDKEY PROP-EXPR ...)
1314           | (SIMP-EXN ...)
1315 
1316  COND-EXN : (and EXN ...)
1317           | (or EXN ...)
1318           | (not EXN)
1319 
1320  PROP-EXPR : COND-PROP-EXPR
1321            | SIMP-PROP-EXPR
1322            | ()
1323 
1324  SIMP-PROP-EXPR : PROPKEY
1325                 | (PROPKEY ...)
1326                 | (PROPKEY VALUE)
1327                 | (PREDICATE PROPKEY VALUE)
1328 
1329  COND-PROP-EXPR : (and PROP-EXPR ...)
1330                 | (or PROP-EXPR ...)
1331                 | (not PROP-EXPR)
1332
1333An example:
1334
1335<enscript highlight=scheme>
1336(define-test ...
1337  ...
1338  (test-case "exn1" d e (
1339      [exn1 (make-composite-condition
1340              (make-property-condition 'abc 'cbs "pbs")
1341              (make-property-condition 'foo 'bar "zip"))])
1342 
1343    (expect-exception "1" (foo abc) (signal exn1))
1344 
1345      ;; These are the same test, the 2nd has an explicit predicate
1346    (expect-exception "2" ((property foo (bar "zip"))
1347                           (property abc (cbs "pbs"))) (signal exn1))
1348    (expect-exception "3" ((property foo (equal? bar "zip"))
1349                           (property abc (equal? cbs "pbs"))) (signal exn1))
1350  )
1351  ...
1352)
1353</enscript>
1354
1355===== expect-ec
1356
1357<macro>(expect-ec [TEST-NAME] [(warn MESSAGE)] QUALIFIER ... EXPECTED [=> PREDICATE] EXPRESSION)</macro>
1358
1359This expectation checks to see if {{(PREDICATE EXPECTED
1360EXPRESSION)}} is true for every state produced by the set of
1361{{QUALIFIER}}.
1362
1363{{TEST-NAME}} must evaluate to a Scheme {{atom}}.
1364
1365{{(warn MESSAGE)}} allows you to specify a warning object, usually
1366a string, that gets associated with the expectation. The {{warn}}
1367function name is actually a syntax reserved word in the macro.
1368
1369{{EXPRESSION}} and {{EXPECTED}} are expressions.
1370
1371{{PREDICATE}} is a two argument procedure returning a boolean,
1372{{equal?}} is used when missing.
1373
1374{{QUALIFIER}} is a SRFI-42 <qualifier>. An eager
1375comprehension implementation, as described by
1376[[http://srfi.schemers.org/srfi-42/srfi-42.html|SRFI-42]] must be
1377provided to use this expectation.
1378
1379===== expect-set!
1380
1381<macro>(expect-set! [TEST-NAME] VARIABLE EXPRESSION)</macro>
1382
1383Evaluates {{EXPRESSION}} as in {{expect-success}}. If successful the
1384variable {{VARIABLE}} is bound to the result of evaluating the
1385{{EXPRESSION}}.
1386
1387{{EXPRESSION}} cannot evaluate to {{#f}} or a {{condition}}.
1388
1389==== Define New Expectations
1390
1391An API to define new expectation macros. May only be used at the
1392toplevel.
1393
1394===== define-expect-unary
1395
1396<macro>(define-expect-unary PREDICATE [SUFFIX [KIND]])</macro>
1397
1398Defines a new unary expectation.
1399
1400{{PREDICATE}} is the name of a procedure of one argument to
1401perform the test.
1402
1403{{SUFFIX}} is a symbol to create the expectation macro name. When
1404missing the {{PREDICATE}} is used, without a {{?}} suffix, if
1405any.
1406
1407{{KIND}} is a string stating the kind of test. When missing the
1408string form of the {{SUFFIX}} is used.
1409
1410===== define-expect-binary
1411
1412<macro>(define-expect-binary PREDICATE [SUFFIX [KIND]])</macro>
1413
1414Defines a new binary expectation.
1415
1416{{PREDICATE}} is the name of a procedure of two arguments to
1417perform the test.
1418
1419{{SUFFIX}} is a symbol to create the expectation macro name. When
1420missing the {{PREDICATE}} is used, without a {{?}} suffix, if
1421any.
1422
1423{{KIND}} is a string stating the kind of test. When missing the
1424string form of the {{SUFFIX}} is used.
1425
1426===== define-expect-unary/values
1427
1428<macro>(define-expect-unary/values PREDICATE [SUFFIX [KIND]])</macro>
1429
1430Defines a new multivalued unary expectation.
1431
1432{{PREDICATE}} is the name of a procedure of one list argument to
1433perform the test.
1434
1435{{SUFFIX}} is a symbol to create the expectation macro name. When
1436missing the {{PREDICATE}} is used, without a {{?}} suffix, if
1437any.
1438
1439{{KIND}} is a string stating the kind of test. When missing the
1440string form of the {{SUFFIX}} is used.
1441
1442===== define-expect-binary/values
1443
1444<macro>(define-expect-binary/values PREDICATE [SUFFIX [KIND]])</macro>
1445
1446Defines a new multivalued binary expectation.
1447
1448{{PREDICATE}} is the name of a procedure of two list arguments to
1449perform the test.
1450
1451{{SUFFIX}} is a symbol to create the expectation macro name. When
1452missing the {{PREDICATE}} is used, without a {{?}} suffix, if
1453any.
1454
1455{{KIND}} is a string stating the kind of test. When missing the
1456string form of the {{SUFFIX}} is used.
1457
1458===== define-expect-nary
1459
1460<macro>(define-expect-nary (SUFFIX ARG ...) EXPR ...)</macro>
1461
1462Defines a new n-ary expectation with the name {{expect-SUFFIX}}.
1463
1464{{(expect-SUFFIX MESSAGE [(warning WARNING)] VAL ...)}}. Unlike other
1465expectations either the MESSAGE or a WARNING must be supplied.
1466
1467{{SUFFIX}} is a symbol.
1468
1469{{ARG ...}} are the parameter variable symbols.
1470
1471{{EXPR ...}} is the body, which must return {{#f}} for failure.
1472
1473==== Testeez Synonyms
1474
1475Syntax forms mimicking some of the '''testeez''' test framework forms.
1476Expanded into the corresponding {{expect-*}} macros.
1477
1478The formal argument names match those in the testeez documentation.
1479
1480Following the testeez semantics all test forms are always evaluated as
1481having possible multiple valued returns.
1482
1483===== test-eval
1484
1485<macro>(test-eval [TEST-NAME] EXPRESSION)</macro>
1486
1487===== test/eq
1488
1489<macro>(test/eq [TEST-NAME] EXPRESSION EXPECTED)</macro>
1490
1491===== test/eqv
1492
1493<macro>(test/eqv [TEST-NAME] EXPRESSION EXPECTED)</macro>
1494
1495===== test/equal
1496
1497<macro>(test/equal [TEST-NAME] EXPRESSION EXPECTED)</macro>
1498
1499==== Test Setup (Side-Effect)
1500
1501The side effecting evaluates all of its arguments as in a {{(begin
1502...)}} form, it returns a result that is completely ignored by the
1503system and unavailable to the output analysis code.
1504
1505===== side-effect
1506
1507<macro>(side-effect CLAUSES)</macro>
1508
1509This macro expands into a begin form with the clauses in order. When
1510it finishes evaluating them, a result is returned that is silently
1511ignored by the testbase system.
1512
1513Usually this is used in conjunction with {{(set! ...)}} or with
1514complicated situations where a lot of setup work must happen for an
1515expectation to be performed.
1516
1517Define forms cannot be expressed here.
1518
1519===== setup
1520
1521<macro>(setup CLAUSES)</macro>
1522
1523A synonym for {{side-effect}}.
1524
1525==== Test Teardown (Destructor)
1526
1527The destructor object allows for you to create helper functions which
1528clean up in case of aborting from a test container.
1529
1530For example, suppose you are testing whether or not file writing to a
1531file works correctly in a test case, so, you'd perform an expectation
1532to open the file, and then queue a function in the destructor to
1533remove the file, and then perform the expectation of the write. If the
1534write (or subsequent) expectation fails, then the test case will
1535''automatically'' invoke the helper cleanup function specified in the
1536destructor object that removes the file.
1537
1538===== destructor-atexit!
1539
1540<macro>(destructor-atexit! [DTOR-NAME] CALL ...)</macro>
1541
1542<macro>(destructor-atexit! DTOR-NAME PROC ARGUMENT ...)</macro>
1543
1544{{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing.
1545
1546A {{CALL}} is a list where the first element is a {{PROC}} and the
1547tail elements are the argument list; i.e. {{(PROC ARGUMENT ...)}}.
1548
1549Each {{CALL}} is added to the destructor queue, promised to be
1550invoked when the {{DTOR-NAME}} destructor object is activated.
1551
1552The second form will insert a promise to invoke {{(PROC ARGUMENT
1553...)}} into a queue in the {{DTOR-NAME}} destructor object.
1554
1555Multiple invocations of this API call will continue to queue up
1556{{(PROC ARGS ...)}} promises indefinitely.
1557
1558'''Note''': The {{PROC}} and {{ARGUMENT ...}}  parameters are
1559evaluated at the point of enqueue; i.e. the current lexical, and
1560dynamic, environment of the macro expansion. This means a variable
1561referenced will have this value when the destructor is activated, and
1562not the "current" value! So no "forward" references.
1563
1564This macro returns a special ''ignore-result'' type that is ignored by
1565the testbase system.
1566
1567===== teardown
1568
1569<macro>(teardown [DTOR-NAME] (FUNC ARGUMENT ...) ...)</macro>
1570
1571A {{(FUNC ARGUMENT ...)}} for is also refered to as a {{CALL}}.
1572
1573A synonym of {{destructor-atexit!}}.
1574
1575<macro>(destructor-activate! [DTOR-NAME])</macro>
1576
1577{{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing.
1578
1579This macro will call, in order of queueing, all the promises embedded
1580into this destructor object, and then delete the queue. The destructor
1581is ''always'' called at the completion of a test container; so be
1582careful that the destructor object doesn't contain anything harmful.
1583
1584This macro returns a special ''ignore-result'' type that is ignored by
1585the testbase system.
1586
1587===== destructor-clear!
1588
1589<macro>(destructor-clear! [DTOR-NAME])</macro>
1590
1591{{DTOR-NAME}} is the destructor to use, {{test-dtor}} when missing.
1592
1593This macro completely removes all of the promises associated with the
1594destructor object {{DTOR-NAME}}.
1595
1596This macro returns a special ''ignore-result'' type that is ignored by
1597the testbase system.
1598
1599===== destructor-dump
1600
1601<macro>(destructor-dump [DTOR-NAME])</macro>
1602
1603{{DTOR-NAME}} is the destructor to use, {{test-dtor}}
1604when missing.
1605
1606This macro, mostly used for debugging purposes, prints out a simple
1607representation of the queued atexit functions to the current port.
1608
1609This macro returns a special ''ignore-result'' type that is ignored by
1610the testbase system.
1611
1612==== Gloss
1613
1614The purpose of the gloss API is to allow the author of a test suite
1615the ability to record messages into the result tree purely for
1616documentation purposes.
1617
1618===== gloss
1619
1620<macro>(gloss MESSAGE [(warn WARNING)])</macro>
1621
1622{{MESSAGE}} can be any scheme object, though usually it is a
1623string.
1624
1625{{(warn WARNING)}} allows you to specify a warning object, usually
1626a string, that gets associated with the gloss. The {{warn}} function
1627name is actually a syntax reserved word in the macro.
1628
1629==== Skip
1630
1631The purpose of the skip API is to allow the author of a test suite to
1632completely skip evaluation of a set of expressions.
1633
1634===== skip
1635
1636<macro>(skip MESSAGE [(warn WARNING)] CLAUSES)</macro>
1637
1638{{MESSAGE}} can be any scheme object, though usually it is a
1639string.
1640
1641{{(warn WARNING)}} allows you to specify a warning object, usually
1642a string, that gets associated with the gloss. The {{warn}} function
1643name is actually a syntax reserved word in the macro.
1644
1645{{CLAUSES}} can be more than one expression (as in a lambda form)
1646that does <em>NOT</em> get evaluated at any time.
1647
1648==== Termination
1649
1650When executing in a test suite or a test case, one might discover some
1651catastrophic failure of such proportions that it is utterly impossible
1652to continue executing the test case or test suite. When that happens
1653you can use the termination facility to exit the test case or test
1654suite. Of course, no more expressions will be evaluated in the scope
1655of the termination. It is recommended that you use this method of
1656terminating the test procedure, test case or test suite evaluation
1657since it wraps some contextual information up into the termination
1658result so you can figure out what happened (and where) later when
1659analyzing the result tree.
1660
1661===== terminate
1662
1663<macro>(terminate MESSAGE)</macro>
1664
1665<macro>(terminate MESSAGE FINDING)</macro>
1666
1667<macro>(terminate ESCR-NAME MESSAGE)</macro>
1668
1669<macro>(terminate ESCR-NAME MESSAGE FINDING)</macro>
1670
1671This is the recommended termination method for a test procedure, test
1672case, or a test suite.
1673
1674{{ESCR-NAME}} is the name of the termination procedure that was
1675specified in a test procedure, test case or test suite. You may pass
1676any test suite or test case termination function available to you in
1677the lexical scope in which you call this function. The termination
1678will take effect in the scope of the created termination function.
1679
1680{{MESSAGE}} can be any scheme object, though usually it is a string.
1681
1682{{FINDING}} can be any scheme object, though usually it is a boolean.
1683
1684==== Todo
1685
1686The purpose of the todo API is to allow the author of a test suite the
1687ability to record into the result tree for later analysis that
1688something still needs to be done. This way you can count/manipulate
1689this information at a later date.
1690
1691===== todo
1692
1693<macro>(todo MESSAGE [(warn WARNING)])</macro>
1694
1695{{MESSAGE}} can be any scheme object, though usually it is a string.
1696
1697{{(warn WARNING)}} allows you to specify a warning object, usually a
1698string, that gets associated with the todo. The {{warn}} function name
1699is actually a syntax reserved word in the macro.
1700
1701==== Test Result Object
1702
1703Test procedures, test containers, expectations, and helper macros
1704({{gloss}}, {{todo}}, etc) all return an object that contains the
1705results and other various aspects of the action performed which
1706ultimately get wired together to form the result tree. This collection
1707of functions forming the rest of the testbase API allows manipulation
1708of these results in an abstracted way as to allow changing of the
1709representation in the future.
1710
1711
1712==== Any Test Result Object
1713
1714This section contains a few functions whose purpose is to simplify
1715certain kinds of manipulations of result objects. Will generate an
1716error for an invalid result object.
1717
1718===== test::result?
1719
1720<procedure>(test::result? RESULT-OBJ)</procedure>
1721
1722This function will return {{#t}} if {{RESULT-OBJ}} is any
1723kind of an evaluated result object. Otherwise, it shall return
1724{{#f}}.
1725
1726===== test::ignore-result?
1727
1728<procedure>(test::ignore-result? RESULT-OBJ)</procedure>
1729
1730This function will return {{#t}} if {{RESULT-OBJ}} is an
1731ignore result object. Otherwise, it shall return {{#f}}. An ignore
1732object is not considered an evaluated result object.
1733
1734===== test::result-passed?
1735
1736<procedure>(test::result-passed? RESULT-OBJ)</procedure>
1737
1738This function will return whether the {{boolean>}} finding of any
1739kind of {{RESULT-OBJ}} passed to it if it is indeed a true result
1740object.
1741
1742===== test::result-finding
1743
1744<procedure>(test::result-finding RESULT-OBJ)</procedure>
1745
1746This function will return the finding of any kind of {{RESULT-OBJ}}
1747passed to it if it is indeed a true result object.
1748
1749The finding is a variant type: {{boolean}} | {{condition}}.
1750
1751===== test::result-name
1752
1753<procedure>(test::result-name RESULT-OBJ)</procedure>
1754
1755This function will return the {{TEST-NAME}} of any kind of
1756{{RESULT-OBJ}} passed to it if it is indeed a true result object.
1757
1758===== test::result-message
1759
1760<procedure>(test::result-message RESULT-OBJ)</procedure>
1761
1762This function will return the {{MESSAGE}} of any kind of
1763{{RESULT-OBJ}} passed to it if it is indeed a true result object.
1764
1765===== test::result-warning?
1766
1767<procedure>(test::result-warning? RESULT-OBJ)</procedure>
1768
1769This function will return whether a warning is defined if
1770{{RESULT-OBJ}} is any kind of an evaluated result object.
1771
1772===== test::result-warning
1773
1774<procedure>(test::result-warning RESULT-OBJ)</procedure>
1775
1776This function will return the warning of any kind of {{RESULT-OBJ}}
1777passed to it if it is indeed a true result object.
1778
1779===== test::result-timing?
1780
1781<procedure>(test::result-timing? RESULT-OBJ)</procedure>
1782
1783This function will return whether a timing is defined if
1784{{RESULT-OBJ}} is any kind of an evaluated result object.
1785
1786===== test::result-timing
1787
1788<procedure>(test::result-timing RESULT-OBJ)</procedure>
1789
1790This function will return the timing values of any kind of
1791{{RESULT-OBJ}} passed to it if it is indeed a true result object.
1792
1793The timing values will be a pair of the milliseconds spent in user
1794code and in system code, for the test corresponding to this
1795test::result-object.
1796
1797For example, the list {{(6 . 1)}} indicates 6 milliseconds spent in
1798user code and 1 millisecond spent in system code.
1799
1800===== test::result-timing-message
1801
1802<procedure>(test::result-timing-message RESULT-OBJ)</procedure>
1803
1804This function will return the timing string representation of any kind of
1805{{RESULT-OBJ}} passed to it if it is indeed a true result object.
1806
1807===== test::result-kind
1808
1809<procedure>(test::result-kind RESULT-OBJ)</procedure>
1810
1811This function will return the kind of any {{RESULT-OBJ}} passed to it
1812if it is indeed a true result object.
1813
1814==== Test Container Result Object
1815
1816==== Test Suite Result Object
1817
1818If any of these API functions, except ''test::test-suite-result?'', are
1819passed something that isn't a test suite result object, they will return an
1820undefined value.
1821
1822===== test::test-suite-result?
1823
1824<procedure>(test::test-suite-result? RESULT)</procedure>
1825
1826If {{RESULT}} is a result object from the invocation of a test
1827suite macro, then this function will return {{#t}}. Otherwise, it will
1828return {{#f}}.
1829
1830===== test::test-suite-result-passed?
1831
1832<procedure>(test::test-suite-result-passed? RESULT)</procedure>
1833
1834Returns the boolean result associated with the test suite
1835{{RESULT}} object.
1836
1837===== test::test-suite-result-finding
1838
1839<procedure>(test::test-suite-result-finding RESULT)</procedure>
1840
1841Returns the finding result associated with the test suite {{RESULT}}
1842object.
1843
1844===== test::test-suite-result-name
1845
1846<procedure>(test::test-suite-result-name RESULT)</procedure>
1847
1848Returns the message object associated with the test suite {{RESULT}} object.
1849
1850===== test::test-suite-result-list
1851
1852<procedure>(test::test-suite-result-list RESULT)</procedure>
1853
1854Returns the list of result objects associated with the test suite
1855{{RESULT}} object.
1856
1857===== test::test-suite-result-warning?
1858
1859<procedure>(test::test-suite-result-warning? RESULT)</procedure>
1860
1861If a warning had been attached to this test suite, this function will
1862return {{#t}}, otherwise it will be {{#f}}.
1863
1864===== test::test-suite-result-warning
1865
1866<procedure>(test::test-suite-result-warning RESULT)</procedure>
1867
1868If a warning had been attached to this test suite, this function will
1869return the warning message object supplied by the user, otherwise it shall
1870return {{'()}}.
1871
1872===== test::test-suite-result-timing?
1873
1874<procedure>(test::test-suite-result-timing? RESULT)</procedure>
1875
1876If a timing had been attached to this test suite, this function will return
1877{{#t}}, otherwise it will be {{#f}}.
1878
1879===== test::test-suite-result-timing
1880
1881<procedure>(test::test-suite-result-timing RESULT)</procedure>
1882
1883If a timing had been attached to this test suite, this function will return
1884the timing pair, otherwise it shall return {{'(0 . 0)}}.
1885
1886===== test::test-suite-results-passed?
1887
1888<procedure>(test::test-suite-results-passed? RESULT-LIST)</procedure>
1889
1890Returns {{#t}} if every single contained result in the
1891{{RESULT-LIST}} passed, or {{#f}} otherwise.
1892
1893==== Test Case Result Object
1894
1895If any of these API functions, except {{test::test-case-result?}}, are
1896passed something that isn't a test case result object, they will
1897return an undefined value.
1898
1899===== test::test-case-result?
1900
1901<procedure>(test::test-case-result? RESULT)</procedure>
1902
1903If {{RESULT}} is a result object from the invocation of a test
1904case macro, then this function will return {{#t}}. Otherwise, it will
1905return {{#f}}.
1906
1907===== test::test-case-result-passed?
1908
1909<procedure>(test::test-case-result-passed? RESULT)</procedure>
1910
1911Returns the boolean result associated with the test case {{RESULT}} object.
1912
1913===== test::test-case-result-finding
1914
1915<procedure>(test::test-case-result-finding RESULT)</procedure>
1916
1917Returns the finding result associated with the test case {{RESULT}} object.
1918
1919===== test::test-case-result-name
1920
1921<procedure>(test::test-case-result-name RESULT)</procedure>
1922
1923Returns the message object associated with the test case {{RESULT}} object.
1924
1925===== test::test-case-result-list
1926
1927<procedure>(test::test-case-result-list RESULT)</procedure>
1928
1929Returns the list of result objects associated with the test case
1930{{RESULT}} object.
1931
1932===== test::test-case-result-warning?
1933
1934<procedure>(test::test-case-result-warning? RESULT)</procedure>
1935
1936If a warning had been attached to this test case, this function will
1937return {{#t}}, otherwise it will be {{#f}}.
1938
1939===== test::test-case-result-warning
1940
1941<procedure>(test::test-case-result-warning RESULT)</procedure>
1942
1943If a warning had been attached to this test case, this function will
1944return the warning message object supplied by the user, otherwise it
1945shall return {{'()}}.
1946
1947===== test::test-case-result-timing?
1948
1949<procedure>(test::test-case-result-timing? RESULT)</procedure>
1950
1951If a timing had been attached to this test case, this function will return
1952{{#t}}, otherwise it will be {{#f}}.
1953
1954===== test::test-case-result-timing
1955
1956<procedure>(test::test-case-result-timing RESULT)</procedure>
1957
1958If a timing had been attached to this test case, this function will return
1959the timing pair, otherwise it shall return {{'(0 . 0)}}.
1960
1961===== test::test-case-results-passed?
1962
1963<procedure>(test::test-case-results-passed? RESULT-LIST)</procedure>
1964
1965Returns {{#t}} if every single contained result in the {{RESULT-LIST}}
1966passed, or {{#f}} otherwise.
1967
1968==== Expectation Result Objects
1969
1970Procedures used to access the various kinds of expectation result
1971object.
1972
1973==== Unary Expectation
1974
1975These expectations all take the form of passing a single expression to
1976them to see if they match some a priori expectation. If any of these
1977API functions, except {{test::expect-result?}}, are passed something
1978that isn't a single clause style expectation result object, they will
1979return {{'not-an-expect-result}}.
1980
1981===== test::expect-result?
1982
1983<procedure>(test::expect-result? RESULT)</procedure>
1984
1985If {{RESULT}} is a single clause style result object from the
1986invocation of an expectation macro, then this function will return
1987{{#t}}. Otherwise, it will return {{#f}}.
1988
1989===== test::expect-result-finding
1990
1991<procedure>(test::expect-result-finding RESULT)</procedure>
1992
1993Returns the boolean result associated with the single clause style
1994expectation {{RESULT}} object.
1995
1996===== test::expect-result-kind
1997
1998<procedure>(test::expect-result-kind RESULT)</procedure>
1999
2000This retrieves the "kind" field of a particular single clause style
2001expectation. For example, if you had a result object from an
2002invocation of a {{(expect-zero "foobar" (- 1 1))}} expectation, then
2003the "kind" field of the expectation result object will be the string
2004{{"zero"}}. Here is a table describing what the "kind" fields are for
2005each kind of single clause style expectation:
2006
2007<table border="1">
2008<tr><th>Unary Expectation</th><th>Associated Specific String</th></tr>
2009<tr><td>expect-zero</td>                     <td>"zero"</td></tr>
2010<tr><td>expect-nonzero</td>                  <td>"nonzero"</td></tr>
2011<tr><td>expect-true</td>                     <td>"true"</td></tr>
2012<tr><td>expect-false</td>                    <td>"false"</td></tr>
2013<tr><td>expect-not-false</td>                <td>"not-false"</td></tr>
2014<tr><td>expect-not-null</td>                 <td>"not-null"</td></tr>
2015<tr><td>expect-positive</td>                 <td>"positive"</td></tr>
2016<tr><td>expect-negative</td>                 <td>"negative"</td></tr>
2017<tr><td>expect-unary</td>                    <td>{{KIND}}</td></tr>
2018<tr><td>expect-unary/values</td>             <td>{{KIND}}</td></tr>
2019</table>
2020
2021===== test::expect-result-name
2022
2023<procedure>(test::expect-result-name RESULT)</procedure>
2024
2025Returns the message object associated with the single clause style
2026expectation {{RESULT}} object.
2027
2028===== test::expect-result-unevaled
2029
2030<procedure>(test::expect-result-unevaled RESULT)</procedure>
2031
2032Returns the unevaluated expression supplied to a single clause style
2033expectation macro.
2034
2035===== test::expect-result-evaled
2036
2037<procedure>(test::expect-result-evaled RESULT)</procedure>
2038
2039Returns the evaluated expression supplied to a single clause style expectation macro.
2040
2041===== test::expect-result-warning?
2042
2043<procedure>(test::expect-result-warning? RESULT)</procedure>
2044
2045If a warning had been attached to this expectation, this function will
2046return {{#t}}, otherwise it will be {{#f}}.
2047
2048===== test::expect-result-warning
2049
2050<procedure>(test::expect-result-warning RESULT)</procedure>
2051
2052If a warning had been attached to this expectation, this function will
2053return the warning message object supplied by the user, otherwise it shall
2054return {{'()}}.
2055
2056===== test::expect-result-timing?
2057
2058<procedure>(test::expect-result-timing? RESULT)</procedure>
2059
2060If a timing had been attached to this expectation, this function will
2061return {{#t}}, otherwise it will be {{#f}}.
2062
2063===== test::expect-result-timing
2064
2065<procedure>(test::expect-result-timing RESULT)</procedure>
2066
2067If a timing had been attached to this expectation, this function will
2068return the timing pair, otherwise it shall return {{'(0 . 0)}}.
2069
2070
2071==== Binary Expectation
2072
2073These expectations all take the form of passing a two expressions, the
2074"left hand side" and the "right hand side" to them to see if they
2075match some a priori equivalence. The left hand side is that which you
2076expect the right hand side to be equivalent. If any of these API
2077functions, except {{test::expect-equivalence-result?}}, are passed
2078something that isn't a single clause style expectation result object,
2079they will return {{'not-an-expect-equivalence-result}}.
2080
2081===== test::expect-equivalence-result?
2082
2083<procedure>(test::expect-equivalence-result? RESULT)</procedure>
2084
2085If {{RESULT}} is a comparison style result object from the
2086invocation of an expectation macro, then this function will return
2087{{#t}}. Otherwise, it will return {{#f}}.
2088
2089===== test::expect-equivalence-result-finding
2090
2091<procedure>(test::expect-equivalence-result-finding RESULT)</procedure>
2092
2093Returns the boolean result associated with the comparison style expectation
2094{{RESULT}} object.
2095
2096===== test::expect-equivalence-result-kind
2097
2098<procedure>(test::expect-equivalence-result-kind RESULT)</procedure>
2099
2100This retrieves the "kind" field of a particular equivalence style
2101expectation. For example, if you had a result object from an invocation of a
2102{{(expect-equal? "foobar" 0 (- 1 1))}} expectation, then the "kind"
2103field of the expectation result object will be the string {{"equal"}}.
2104Here is a table describing what the "kind" fields are for each kind of
2105equivalence style expectation:
2106
2107<table border="1"><tr><th>Binary Expectation</th><th>Associated Specific String</th></tr>
2108<tr><td>expect-eq</td>                     <td>"eq"</td></tr>
2109<tr><td>expect-eqv</td>                    <td>"eqv"</td></tr>
2110<tr><td>expect-equal</td>                  <td>"equal"</td></tr>
2111<tr><td>expect-exception</td>              <td>"exception"</td></tr>
2112<tr><td>expect-binary</td>                 <td>{{KIND}}</td></tr>
2113<tr><td>expect/values</td>                 <td>{{"values"}}</td></tr>
2114<tr><td>expect-eq/values</td>              <td>"eq/values"</td></tr>
2115<tr><td>expect-eqv/values</td>             <td>"eqv/values"</td></tr>
2116<tr><td>expect-equal/values</td>           <td>"equal/values"</td></tr>
2117<tr><td>expect-binary/values</td>          <td>{{KIND}}</td></tr>
2118</table>
2119
2120===== test::expect-equivalence-result-name
2121
2122<procedure>(test::expect-equivalence-result-name RESULT)</procedure>
2123
2124Returns the message object associated with the equivalence style
2125expectation {{RESULT}} object.
2126
2127===== test::expect-equivalence-result-lhs-evaled
2128
2129<procedure>(test::expect-equivalence-result-lhs-evaled RESULT)</procedure>
2130
2131Returns the evaluated "left hand side" expression supplied to an
2132equivalence style expectation.
2133
2134===== test::expect-equivalence-result-rhs-unevaled
2135
2136<procedure>(test::expect-equivalence-result-rhs-unevaled RESULT)</procedure>
2137
2138Returns the unevaluated "right hand side" expression supplied to an
2139equivalence style expectation.
2140
2141===== test::expect-equivalence-result-rhs-evaled
2142
2143<procedure>(test::expect-equivalence-result-rhs-evaled RESULT)</procedure>
2144
2145Returns the evaluated "right hand side" expression supplied to an
2146equivalence style expectation.
2147
2148===== test::expect-equivalence-result-warning?
2149
2150<procedure>(test::expect-equivalence-result-warning? RESULT)</procedure>
2151
2152If a warning had been attached to this expectation, this function will
2153return {{#t}}, otherwise it will be {{#f}}.
2154
2155===== test::expect-equivalence-result-warning
2156
2157<procedure>(test::expect-equivalence-result-warning RESULT)</procedure>
2158
2159If a warning had been attached to this expectation, this function will
2160return the warning message object supplied by the user, otherwise it shall
2161return {{'()}}.
2162
2163===== test::expect-equivalence-result-timing?
2164
2165<procedure>(test::expect-equivalence-result-timing? RESULT)</procedure>
2166
2167If a timing had been attached to this expectation, this function will
2168return {{#t}}, otherwise it will be {{#f}}.
2169
2170===== test::expect-equivalence-result-timing
2171
2172<procedure>(test::expect-equivalence-result-timing RESULT)</procedure>
2173
2174If a timing had been attached to this expectation, this function will
2175return the timing pair, otherwise it shall return {{'(0 . 0)}}.
2176
2177==== Tolerance Style Expectation
2178
2179This is a specialized expectation which accepts three expressions and
2180checks to see if the "right hand side" is within a "tolerance" of the
2181"left hand side". There is only one expectation in the tolerance style
2182currently. If any of these API functions, except
2183{{test::expect-tolerance-result?}}, are passed something that isn't a
2184tolerance style expectation result object the result is unspecified.
2185
2186===== test::expect-tolerance-result?
2187
2188<procedure>(test::expect-tolerance-result? RESULT)</procedure>
2189
2190If {{RESULT}} is a tolerance style result object from the
2191invocation of an expectation macro, then this function will return
2192{{#t}}. Otherwise, it will return {{#f}}.
2193
2194===== test::expect-tolerance-result-finding
2195
2196<procedure>(test::expect-tolerance-result-finding RESULT)</procedure>
2197
2198Returns the boolean result associated with the tolerance style expectation
2199{{RESULT}} object.
2200
2201===== test::expect-tolerance-result-kind
2202
2203<procedure>(test::expect-tolerance-result-kind RESULT)</procedure>
2204
2205This retrieves the "kind" field of a particular tolerance style
2206expectation. For example, if you had a result object from an
2207invocation of a {{(expect-near? "foobar" 100 .01 100.001)}}
2208expectation, then the "kind" field of the expectation result object
2209will be the string {{"near"}}. Here is a table describing what the
2210"kind" fields are for each kind of tolerance style expectation:
2211
2212<table border="1"><tr><th>Tolerance Style Expectation</th><th>Associated Specific String</th></tr>
2213<tr><td>expect-near</td>                 <td>"near"</td></tr>
2214</table>
2215
2216===== test::expect-tolerance-result-name
2217
2218<procedure>(test::expect-tolerance-result-name RESULT)</procedure>
2219
2220Returns the message object associated with a tolerance style
2221expectation {{RESULT}} object.
2222
2223===== test::expect-tolerance-result-lhs-evaled
2224
2225<procedure>(test::expect-tolerance-result-lhs-evaled RESULT)</procedure>
2226
2227Returns the evaluated "left hand side" expression supplied to a
2228tolerance style expectation.
2229
2230===== test::expect-tolerance-result-lhs-tol-evaled
2231
2232<procedure>(test::expect-tolerance-result-lhs-tol-evaled RESULT)</procedure>
2233
2234Returns the evaluated "tolerance" expression supplied to a tolerance
2235style expectation.
2236
2237===== test::expect-tolerance-result-rhs-unevaled
2238
2239<procedure>(test::expect-tolerance-result-rhs-unevaled RESULT)</procedure>
2240
2241Returns the unevaluated "right hand side" expression supplied to a
2242tolerance style expectation.
2243
2244===== test::expect-tolerance-result-rhs-evaled
2245
2246<procedure>(test::expect-tolerance-result-rhs-evaled RESULT)</procedure>
2247
2248Returns the evaluated "right hand side" expression supplied to a
2249tolerance style expectation.
2250
2251===== test::expect-tolerance-result-warning?
2252
2253<procedure>(test::expect-tolerance-result-warning? RESULT)</procedure>
2254
2255If a warning had been attached to this expectation, this function will
2256return {{#t}}, otherwise it will be {{#f}}.
2257
2258===== test::expect-tolerance-result-warning
2259
2260<procedure>(test::expect-tolerance-result-warning RESULT)</procedure>
2261
2262If a warning had been attached to this expectation, this function will
2263return the warning message object supplied by the user, otherwise it shall
2264return {{'()}}.
2265
2266===== test::expect-tolerance-result-timing?
2267
2268<procedure>(test::expect-tolerance-result-timing? RESULT)</procedure>
2269
2270If a timing had been attached to this expectation, this function will
2271return {{#t}}, otherwise it will be {{#f}}.
2272
2273===== test::expect-tolerance-result-timing
2274
2275<procedure>(test::expect-tolerance-result-timing RESULT)</procedure>
2276
2277If a timing had been attached to this expectation, this function will
2278return the timing pair, otherwise it shall return {{'(0 . 0)}}.
2279
2280===== Gloss Result Object
2281
2282===== test::gloss-result?
2283
2284<procedure>(test::gloss-result? RESULT)</procedure>
2285
2286If {{RESULT}} is a gloss result object from the invocation of the
2287gloss macro, then this function will return {{#t}}. Otherwise, it will
2288return {{#f}}.
2289
2290===== test::gloss-result-message
2291
2292<procedure>(test::gloss-result-message RESULT)</procedure>
2293
2294Returns the message object associated with the gloss {{RESULT}}
2295object.
2296
2297===== test::gloss-result-warning?
2298
2299<procedure>(test::gloss-result-warning? RESULT)</procedure>
2300
2301If a warning had been attached to this gloss, this function will
2302return {{#t}}, otherwise it will be {{#f}}.
2303
2304
2305===== test::gloss-result-warning
2306
2307<procedure>(test::gloss-result-warning RESULT)</procedure>
2308
2309If a warning had been attached to this gloss, this function will
2310return the warning message object supplied by the user, otherwise it
2311shall return {{'()}}.
2312
2313===== Skip Result Object
2314
2315===== test::skip-result?
2316
2317<procedure>(test::skip-result? RESULT)</procedure>
2318
2319If {{RESULT}} is a skip result object from the invocation of the skip
2320macro, then this function will return {{#t}}. Otherwise, it will
2321return {{#f}}.
2322
2323===== test::skip-result-message
2324
2325<procedure>(test::skip-result-message RESULT)</procedure>
2326
2327Returns the message object associated with the skip {{RESULT}}
2328object. Hopefully, it was stated why this set of clauses had been
2329skipped.
2330
2331===== test::skip-result-warning?
2332
2333<procedure>(test::skip-result-warning? RESULT)</procedure>
2334
2335If a warning had been attached to this skip, this function will return
2336{{#t}}, otherwise it will be {{#f}}.
2337
2338===== test::skip-result-warning
2339
2340<procedure>(test::skip-result-warning RESULT)</procedure>
2341
2342If a warning had been attached to this skip, this function will return
2343the warning message object supplied by the user, otherwise it shall
2344return {{'()}}.
2345
2346===== Termination Result Object
2347
2348When using the manipulation API for a terminate result, if you pass a result
2349to one of these function that is not a terminate result, it will return an
2350unspecified value.
2351
2352===== test::terminate-result?
2353
2354<procedure>(test::terminate-result? RESULT)</procedure>
2355
2356If {{RESULT}} is a termination result object from the invocation
2357of a termination function, then this function will return {{#t}}.
2358Otherwise, it will return {{#f}}.
2359
2360===== test::terminate-result-finding
2361
2362<procedure>(test::terminate-result-finding RESULT)</procedure>
2363
2364Returns the result associated with the termination function {{RESULT}}
2365object.
2366
2367===== test::terminate-result-scope
2368
2369<procedure>(test::terminate-result-scope RESULT)</procedure>
2370
2371The "scope" of the termination result is exactly the {{TEST-NAME}}
2372parameter supplied to the test case or test suite associated with the
2373{{TERMFUNC}}.
2374
2375===== test::terminate-result-container
2376
2377<procedure>(test::terminate-result-container RESULT)</procedure>
2378
2379The "container" of the termination result is going to be either
2380{{'test-suite}} or {{'test-case}} depending upon which the {{TERMFUNC}} was
2381associated.
2382
2383===== test::terminate-result-message
2384
2385<procedure>(test::terminate-result-message RESULT)</procedure>
2386
2387Returns the message object associated with the termination {{RESULT}}
2388object.
2389
2390===== Todo Result Object
2391
2392===== test::todo-result?
2393
2394<procedure>(test::todo-result? RESULT)</procedure>
2395
2396If {{RESULT}} is a todo result object from the invocation of a todo
2397macro, then this function will return {{#t}}. Otherwise, it will
2398return {{#f}}.
2399
2400===== test::todo-result-message
2401
2402<procedure>(test::todo-result-message RESULT)</procedure>
2403
2404Returns the message object associated with the todo {{RESULT}} object.
2405
2406===== test::todo-result-warning?
2407
2408<procedure>(test::todo-result-warning? RESULT)</procedure>
2409
2410If a warning had been attached to this todo, this function will return
2411{{#t}}, otherwise it will be {{#f}}.
2412
2413===== test::todo-result-warning
2414
2415<procedure>(test::todo-result-warning RESULT)</procedure>
2416
2417If a warning had been attached to this todo, this function will return the
2418warning message object supplied by the user, otherwise it shall return
2419{{'()}}.
2420
2421==== Miscellaneous
2422
2423===== test::walk-structure
2424
2425<procedure>(test::walk-structure WALKER TEST-STRUCTURE)</procedure>
2426
2427This procedure will enumerate the {{TEST-STRUCTURE}} association list,
2428invoking the {{WALKER}} for each element. The hierarchy level value is
2429the tree depth at the point of call.
2430
2431{{WALKER}} is a three argument procedure. The first argument is the
2432hierarchy level, counting from 0, the outermost level. The second
2433argument is the type of the test element. The third argument is the
2434name of the test element.
2435
2436==== Statistics Generation
2437
2438===== test::stat-result-statistics
2439
2440<procedure>(test::stat-result-statistics RESULT-TREE)</procedure>
2441
2442This function will compute and return a vector of result
2443{{STATISTICS}}. Use the procedures below to access the elements by
2444name.
2445
2446===== test::stat-test-suites
2447
2448<procedure>(test::stat-test-suites STATISTICS)</procedure>
2449
2450===== test::stat-test-suite-warnings
2451
2452<procedure>(test::stat-test-suite-warnings STATISTICS)</procedure>
2453
2454===== test::stat-test-suites-passed
2455
2456<procedure>(test::stat-test-suites-passed STATISTICS)</procedure>
2457
2458===== test::stat-test-suites-failed
2459
2460<procedure>(test::stat-test-suites-failed STATISTICS)</procedure>
2461
2462===== test::stat-test-suites-timing
2463
2464<procedure>(test::stat-test-suites-timing STATISTICS)</procedure>
2465
2466===== test::stat-test-suites-terminated
2467
2468<procedure>(test::stat-test-suites-terminated STATISTICS)</procedure>
2469
2470===== test::stat-test-cases
2471
2472<procedure>(test::stat-test-cases STATISTICS)</procedure>
2473
2474===== test::stat-test-case-warnings
2475
2476<procedure>(test::stat-test-case-warnings STATISTICS)</procedure>
2477
2478===== test::stat-test-cases-passed
2479
2480<procedure>(test::stat-test-cases-passed STATISTICS)</procedure>
2481
2482===== test::stat-test-cases-failed
2483
2484<procedure>(test::stat-test-cases-failed STATISTICS)</procedure>
2485
2486===== test::stat-test-cases-timing
2487
2488<procedure>(test::stat-test-cases-timing STATISTICS)</procedure>
2489
2490===== test::stat-test-cases-terminated
2491
2492<procedure>(test::stat-test-cases-terminated STATISTICS)</procedure>
2493
2494===== test::stat-all-expectations
2495
2496<procedure>(test::stat-all-expectations STATISTICS)</procedure>
2497
2498===== test::stat-all-expectation-warnings
2499
2500<procedure>(test::stat-all-expectation-warnings STATISTICS)</procedure>
2501
2502===== test::stat-all-expectations-passed
2503
2504<procedure>(test::stat-all-expectations-passed STATISTICS)</procedure>
2505
2506===== test::stat-all-expectations-failed
2507
2508<procedure>(test::stat-all-expectations-failed STATISTICS)</procedure>
2509
2510===== test::stat-all-expectations-timing
2511
2512<procedure>(test::stat-all-expectations-timing STATISTICS)</procedure>
2513
2514===== test::stat-single-expectations
2515
2516<procedure>(test::stat-single-expectations STATISTICS)</procedure>
2517
2518===== test::stat-single-expectation-warnings
2519
2520<procedure>(test::stat-single-expectation-warnings STATISTICS)</procedure>
2521
2522===== test::stat-single-expectations-passed
2523
2524<procedure>(test::stat-single-expectations-passed STATISTICS)</procedure>
2525
2526===== test::stat-single-expectations-failed
2527
2528<procedure>(test::stat-single-expectations-failed STATISTICS)</procedure>
2529
2530===== test::stat-single-expectations-timing
2531
2532<procedure>(test::stat-single-expectations-timing STATISTICS)</procedure>
2533
2534===== test::stat-tol-expectations
2535
2536<procedure>(test::stat-tol-expectations STATISTICS)</procedure>
2537
2538===== test::stat-tol-expectation-warnings
2539
2540<procedure>(test::stat-tol-expectation-warnings STATISTICS)</procedure>
2541
2542===== test::stat-tol-expectations-passed
2543
2544<procedure>(test::stat-tol-expectations-passed STATISTICS)</procedure>
2545
2546===== test::stat-tol-expectations-failed
2547
2548<procedure>(test::stat-tol-expectations-failed STATISTICS)</procedure>
2549
2550===== test::stat-tol-expectations-timing
2551
2552<procedure>(test::stat-tol-expectations-timing STATISTICS)</procedure>
2553
2554===== test::stat-equiv-expectations
2555
2556<procedure>(test::stat-equiv-expectations STATISTICS)</procedure>
2557
2558===== test::stat-equiv-expectation-warnings
2559
2560<procedure>(test::stat-equiv-expectation-warnings STATISTICS)</procedure>
2561
2562===== test::stat-equiv-expectations-passed
2563
2564<procedure>(test::stat-equiv-expectations-passed STATISTICS)</procedure>
2565
2566===== test::stat-equiv-expectations-failed
2567
2568<procedure>(test::stat-equiv-expectations-failed STATISTICS)</procedure>
2569
2570===== test::stat-equiv-expectations-timing
2571
2572<procedure>(test::stat-equiv-expectations-timing STATISTICS)</procedure>
2573
2574===== test::stat-todos
2575
2576<procedure>(test::stat-todos STATISTICS)</procedure>
2577
2578===== test::stat-todo-warnings
2579
2580<procedure>(test::stat-todo-warnings STATISTICS)</procedure>
2581
2582===== test::stat-skips
2583
2584<procedure>(test::stat-skips STATISTICS)</procedure>
2585
2586===== test::stat-skip-warnings
2587
2588<procedure>(test::stat-skip-warnings STATISTICS)</procedure>
2589
2590===== test::stat-glosses
2591
2592<procedure>(test::stat-glosses STATISTICS)</procedure>
2593
2594===== test::stat-gloss-warnings
2595
2596<procedure>(test::stat-gloss-warnings STATISTICS)</procedure>
2597
2598===== test::stat-terminations
2599
2600<procedure>(test::stat-terminations STATISTICS)</procedure>
2601
2602==== Printing Routines
2603
2604These procedures are used in the generation of test result reports.
2605
2606===== test::write-object
2607
2608<procedure>(test::write-object OBJECT)</procedure>
2609
2610===== test::display-objects
2611
2612<procedure>(test::display-objects [OBJECT ...])</procedure>
2613
2614===== test::display-objects-newline
2615
2616<procedure>(test::display-objects-newline [OBJECT ...])</procedure>
2617
2618===== test::display-indent
2619
2620<procedure>(test::display-indent INDENT)</procedure>
2621
2622===== test::display-indented-objects
2623
2624<procedure>(test::display-indented-objects INDENT [OBJECT ...])</procedure>
2625
2626===== test::display-indented-objects-newline
2627
2628<procedure>(test::display-indented-objects-newline INDENT [OBJECT ...])</procedure>
2629
2630{{INDENT}} is the number of spaces to the left.
2631
2632===== test::display-underlined
2633
2634<procedure>(test::display-underlined [OBJECT ...])</procedure>
2635
2636Prints each supplied {{OBJECT}} to the
2637{{(current-output-port)}}, with a {{#\space}} interspersed,
2638on a single line. Then prints a line of underscores, {{#\_}}.
2639
2640===== test::display-structure
2641
2642<procedure>(test::display-structure TEST-STRUCTURE)</procedure>
2643
2644Prints the supplied {{TEST-STRUCTURE}} tree to the
2645{{(current-output-port)}} with indentation.
2646
2647==== Test Report Generation
2648
2649These functions will display a rendering of the {{RESULT-TREE}} and
2650returns the {{RESULT-TREE}}.
2651
2652===== test::output-style-compact
2653
2654<procedure>(test::output-style-compact RESULT-TREE [PORT|FILE-NAME|#t])
2655
2656Prints a simple listing of the test results. Details are only printed for
2657failed tests.
2658
2659===== test::output-style-human
2660
2661<procedure>(test::output-style-human RESULT-TREE [PORT|FILE-NAME|#t])</procedure>
2662
2663Prints a verbose listing of the test results.
2664
2665===== test::output-style-html
2666
2667<procedure>(test::output-style-html RESULT-TREE [PORT|FILE-NAME|#t])</procedure>
2668
2669Prints a verbose listing of the test results as HTML.
2670
2671===== test::output-style-minimal
2672
2673<procedure>(test::output-style-minimal RESULT-TREE [PORT|FILE-NAME|#t])</procedure>
2674
2675Prints a summary of the test result statistics.
2676
2677The output style procedures are ''not'' loaded automatically. They must
2678be loaded explicitly, as in:
2679
2680<enscript highlight=scheme>
2681(require-extension testbase-output-human)
2682
2683(require-extension testbase-output-html)
2684
2685(require-extension testbase-output-compact)
2686
2687(require-extension testbase-output-minimal)
2688</enscript>
2689
2690This procedure is useful with a test report procedure. The actual
2691report procedures of the above report generators are examples.
2692
2693===== test::with-output-style
2694
2695<procedure>(test::with-output-style STYLER RESULT-TREE [PORT|FILE-NAME|#t])</procedure>
2696
2697{{STYLER}} is a procedure with a single argument, a {{RESULT-TREE}},
2698to print a report.
2699
2700The default destination is the {{(current-output-port)}}.  Otherwise
2701the destination specifies how the output is targeted.
2702
2703A {{PORT}} destination will print to the port.
2704
2705A {{FILE-NAME}} destination will print to the specified file.  Should
2706the file exist the result is implementation dependent.
2707
2708A {{#t}} destination will return the output as a string.
2709
2710The {{STYLER}} will be called with {{RESULT-TREE}} as the only actual
2711argument.
2712
2713
2714==== Notes & Caveats
2715
2716To generate the tree structure of a test procedure the test body is
2717evaluated using a special test mode, {{report}}. The expectation
2718expressions, test container {{let}} bindings, and {{test-letrec}}
2719bindings <em>are not</em> evaluated. This means there should not be
2720any expressions in the test body that will be evaluated, such as the
2721construction of a {{TEST-NAME}}, that depend on these bindings.
2722
2723However, test procedure {{initial}} and {{final}} forms ''are''
2724evaluated.
2725
2726Be careful when using test selection that dependencies between tests
2727are not broken. For example, an expectation expression could depend on
2728the result of an {{expect-set!}}, but the associated variable may
2729never be bound because the {{expect-set!}} was skipped.
2730
2731
2732==== Examples
2733
2734Assume the following are the contents of the file "testbase-example.scm":
2735
2736<enscript highlight=scheme>
2737(require-extension testbase testbase-output-human)
2738
2739(define-expect-binary =)
2740
2741(define-test example "Arithmetic Operators"
2742  (test/case "Testing '+'"
2743    (expect-= "Adding two positive numbers" 2 (+ 1 1))
2744    (expect-= "Adding two negative numbers" -2 (+ -1 -1))
2745    (expect-zero "Adding positive and negative" (+ -1 1)) )
2746  (test/case "Testing '-'"
2747    (expect-zero "Subtracting two positive numbers" (- 1 1))
2748    (expect-zero "Subtracting two negative numbers" (- -1 -1))
2749    (expect-= "Subtracting positive and negative" -2 (- -1 1)) ) )
2750
2751(test::styler-set! example test::output-style-human)
2752(run-test "TestBase Example")
2753</enscript>
2754
2755Then the following entered from a command line interface:
2756
2757  csi -n -s testbase-example.scm
2758
2759Produces the following output:
2760
2761  ** Running TestBase Example **
2762 
2763  Test Suite: Arithmetic Operators
2764 
2765    Test Case: Testing '+'
2766 
2767      Expectation: Adding two positive numbers
2768      Expect =
2769         Expected: 2
2770      Unevaluated: (+ 1 1)
2771        Evaluated: 2
2772      Pass: Adding two positive numbers
2773 
2774      Expectation: Adding two negative numbers
2775      Expect =
2776         Expected: -2
2777      Unevaluated: (+ -1 -1)
2778        Evaluated: -2
2779      Pass: Adding two negative numbers
2780 
2781      Expectation: Adding positive and negative
2782      Expect zero
2783      Unevaluated: (+ -1 1)
2784        Evaluated: 0
2785      Pass: Adding positive and negative
2786 
2787    Pass: Testing '+'
2788 
2789    Test Case: Testing '-'
2790 
2791      Expectation: Subtracting two positive numbers
2792      Expect zero
2793      Unevaluated: (- 1 1)
2794        Evaluated: 0
2795      Pass: Subtracting two positive numbers
2796 
2797      Expectation: Subtracting two negative numbers
2798      Expect zero
2799      Unevaluated: (- -1 -1)
2800        Evaluated: 0
2801      Pass: Subtracting two negative numbers
2802 
2803      Expectation: Subtracting positive and negative
2804      Expect =
2805         Expected: -2
2806      Unevaluated: (- -1 1)
2807        Evaluated: -2
2808      Pass: Subtracting positive and negative
2809 
2810    Pass: Testing '-'
2811 
2812  Pass: Arithmetic Operators
2813 
2814  ALL TESTS SUCCESSFUL!
2815
2816And the following entered from a command line interface:
2817
2818  echo "(test::resource-limit-set! example 'cpu 0) (example)" | csi -n -s testbase-example.scm -- -i
2819
2820Produces the following output:
2821
2822  (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))
2823  (begin (id tr1) (name "Testing '+'") (result #f) (kind "case"))
2824  (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2))
2825  (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2))
2826  (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0))
2827  (end (id tr1) (name "Testing '+'") (result #t) (kind "case"))
2828  (begin (id tr5) (name "Testing '-'") (result #f) (kind "case"))
2829  (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0))
2830  (error (id tr7) (message "Resource Limit Exceeded: Processor-Time") (result #f) (kind "terminate") (container test-suite) (location "Arithmetic Operators"))
2831  (end (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))
2832
2833And the following entered from a command line interface:
2834
2835  echo "(test::run)" | csi -n -s testbase-example.scm -- -i
2836
2837Produces the following output:
2838
2839  (begin (id tr0) (name "Arithmetic Operators") (result #f) (kind "suite"))
2840  (begin (id tr1) (name "Testing '+'") (result #f) (kind "case"))
2841  (expectation (id tr2) (name "Adding two positive numbers") (result #t) (kind "=") (unevaluated (+ 1 1)) (expected 2) (actual 2))
2842  (expectation (id tr3) (name "Adding two negative numbers") (result #t) (kind "=") (unevaluated (+ -1 -1)) (expected -2) (actual -2))
2843  (expectation (id tr4) (name "Adding positive and negative") (result #t) (kind "zero") (unevaluated (+ -1 1)) (actual 0))
2844  (end (id tr1) (name "Testing '+'") (result #t) (kind "case"))
2845  (begin (id tr5) (name "Testing '-'") (result #f) (kind "case"))
2846  (expectation (id tr6) (name "Subtracting two positive numbers") (result #t) (kind "zero") (unevaluated (- 1 1)) (actual 0))
2847  (expectation (id tr7) (name "Subtracting two negative numbers") (result #t) (kind "zero") (unevaluated (- -1 -1)) (actual 0))
2848  (expectation (id tr8) (name "Subtracting positive and negative") (result #t) (kind "=") (unevaluated (- -1 1)) (expected -2) (actual -2))
2849  (end (id tr5) (name "Testing '-'") (result #t) (kind "case"))
2850  (end (id tr0) (name "Arithmetic Operators") (result #t) (kind "suite"))
2851
2852A {{test-collect}} Example:
2853
2854<enscript highlight=scheme>
2855(require-extension testbase testbase-output-human)
2856
2857(define-test tcx "A test-collect example"
2858  (test/collect "Are 0 - 9 integers?"
2859    (do ([i 0 (add1 i)])
2860        [(= i 10)]
2861      (collect-test (expect-true (integer? i))) ) ) )
2862
2863(test::styler-set! tcx test::output-style-human)
2864(run-test "TestBase Example")
2865</enscript>
2866
2867Produces the following output:
2868
2869  ** Running TestBase Example **
2870  Test Suite: A test-collect example
2871 
2872    Test Case: Are 0 - 9 integers?
2873 
2874      Expectation:
2875      Expect true
2876      Unevaluated: (integer? i)
2877        Evaluated: #t
2878      Passed:
2879 
2880      Expectation:
2881      Expect true
2882      Unevaluated: (integer? i)
2883        Evaluated: #t
2884      Passed:
2885 
2886      Expectation:
2887      Expect true
2888      Unevaluated: (integer? i)
2889        Evaluated: #t
2890      Passed:
2891 
2892      Expectation:
2893      Expect true
2894      Unevaluated: (integer? i)
2895        Evaluated: #t
2896      Passed:
2897 
2898      Expectation:
2899      Expect true
2900      Unevaluated: (integer? i)
2901        Evaluated: #t
2902      Passed:
2903 
2904      Expectation:
2905      Expect true
2906      Unevaluated: (integer? i)
2907        Evaluated: #t
2908      Passed:
2909 
2910      Expectation:
2911      Expect true
2912      Unevaluated: (integer? i)
2913        Evaluated: #t
2914      Passed:
2915 
2916      Expectation:
2917      Expect true
2918      Unevaluated: (integer? i)
2919        Evaluated: #t
2920      Passed:
2921 
2922      Expectation:
2923      Expect true
2924      Unevaluated: (integer? i)
2925        Evaluated: #t
2926      Passed:
2927 
2928      Expectation:
2929      Expect true
2930      Unevaluated: (integer? i)
2931        Evaluated: #t
2932      Passed:
2933 
2934    Passed: Are 0 - 9 integers?
2935 
2936  Passed: A test-collect example
2937 
2938  ALL TESTS SUCCESSFUL!
2939 
2940  ** Passed TestBase Example **
2941
2942==== Tips, Tricks, And Idioms
2943
2944==== Migration from test-infrastructure
2945
2946Changes:
2947
2948* {{test-group}} removed.
2949* {{test-apply}} removed.
2950* {{test-it}} -> {{test}}.
2951* {{test-package}} -> {{test-suite}}.
2952* {{expect}} -> {{expect-unary}}.
2953* {{expect-values}} -> {{expect-unary/values}}.
2954* {{expect-equiv}} -> {{expect-binary}}.
2955* {{expect-equiv-values}} -> {{expect-binary/values}}.
2956* {{expect-values-foo}} -> {{expect-foo/values}}.
2957* {{test:foo}} -> {{test::foo}}.
2958* {{foo-ref}} -> {{foo}}.
2959* {{*-foo}} -> {{test::result-foo}}.
2960* Test containers cannot be used outside of a {{define-test}}!
2961
2962Forms like:
2963
2964  (test::apply test::output-style-compact
2965      (skip ...)
2966      test-proc)
2967
2968become
2969
2970  (test::take! test-proc '(skip ...))
2971  (test::styler-set! test-proc test::output-style-compact)
2972  (test-proc)
2973
2974Forms like
2975
2976  (define (test-something)
2977    (test-suite "Something" pd pe
2978      ... test elements ...
2979    )
2980  )
2981
2982become
2983
2984  (define-test test-something
2985    (test-suite "Something" pd pe
2986      ... test elements ...
2987    )
2988  )
2989
2990or even
2991
2992  (define-test test-something "Something"
2993    ... test elements ...
2994    <where any use of 'pd' and 'pe' becomes 'test-dtor' and 'test-escr'>
2995  )
2996
2997Forms like
2998
2999  (let ((result
3000          (test-suite "Something" pd pe
3001            ... test elements ...)))
3002    ...)
3003
3004become
3005
3006  (define-test something
3007    (test-suite "Something" pd pe
3008      ... test elements ...
3009    )
3010  )
3011 
3012  (let ((result (something)))
3013    ...)
3014
3015==== Bugs & Limitations
3016
3017Because a sharp-syntax reader is created to handle "unprintable"
3018results the {{"#<<TAG"}} and {{"#<#TAG"}} multiline string constant
3019read syntax cannot be used within a test-file. Use {{(MULTILINE-STRING
3020STRING ...)}}.
3021
3022
3023=== Contributions
3024
3025{{output-text-compact}} by [[Patrick Brannan]].
3026
3027=== Changelog
3028
3029* 1.5.2 Added early-exit-upon-failure-option
3030* 1.5.1 Fix for non-false expect predicate result
3031* 1.5 Added {{insist}} expectation form
3032* 1.41 Cygwin has resource limiting
3033* 1.4 Expansion options are {{#f}} by default, expcept {{test::catching!}}
3034* 1.304 Bug fix for {{test::forget!}}
3035* 1.303 Removed macro expansion time atom check for {{TEST-NAME}}
3036* 1.302 Added {{test-let}} & {{test-let*}}
3037* 1.301 Dropped {{:optional}}
3038* 1.3 Added {{define-expect-nary}} and {{test-collect}}
3039* 1.2 Bugfix for {{define-expect-*/values}}
3040* 1.1 Added echo filter options
3041* 1.0 Initial release
3042
3043
3044=== License
3045
3046Portions Copyright (c) 2007, Kon Lovett
3047Portions Copyright (c) 2000-2005, Peter Keller - All rights reserved.
3048
3049Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following
3050conditions are met:
3051
3052  Redistributions of source code must retain the above copyright notice, this list of conditions and the following
3053    disclaimer.
3054  Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
3055    disclaimer in the documentation and/or other materials provided with the distribution.
3056  Neither the name of the author nor the names of its contributors may be used to endorse or promote
3057    products derived from this software without specific prior written permission.
3058
3059THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
3060OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
3061AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR
3062CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
3063CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
3064SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
3065THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
3066OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
3067POSSIBILITY OF SUCH DAMAGE.
Note: See TracBrowser for help on using the repository browser.