source: project/wiki/test-infrastructure @ 8850

Last change on this file since 8850 was 8850, checked in by sjamaan, 12 years ago

Remove b0rkage with double apostrophes

File size: 60.3 KB
Line 
1[[tags: egg]]
2
3== test-infrastructure
4
5[[toc:]]
6
7=== Description
8
9Macro based unit-testing facility
10
11=== Author
12
13[[Peter Keller]]
14
15=== Requirements
16
17Requires the [[syntax-case]] egg, and Chicken 2.225 or later.
18
19=== Download
20
21[[http://www.call-with-current-continuation.org/eggs/test-infrastructure.egg|test-infrastructure.egg]]
22
23=== Documentation
24
25This extension can currently only be used with the [[syntax-case]]
26(highlevel) macro system.
27
28This extension provides a macro based unit testing facility based upon
29expectations concerning evaluations of expressions. These functions
30return tagged lists which contain the results of the test package,
31test case, or expectations evaluated (there are a few other types of
32results dealing with the gloss, todo, and skip macros detailed
33below). These result lists are wired together during evaluation to
34form a large hierarchical tree in memory. This result tree is then
35passed to either user defined functions which traverse the tree
36manipulating it in any way desired, or passed to a supplied function
37which manipulates it in a simple way usually producing human readable
38or html generated output.  API functions to deal with the result types
39are supplied and the representation of the result is black boxed to
40the user. It is a violation of encapsulation to inspect the
41representation directly, and it may change unpredictably in the
42future.
43
44The term {{MESSAGE}} below means the name given to a test
45element. While this can be any Scheme object {{#f}} is interpreted as
46a missing name. Further, {{#f}} is assumed when a name is
47missing. Thus, the name of all un-named test elements is the same -
48{{#f}}.
49
50==== The Immediate Test Macro API
51
52<macro>(test-it (EXPRESSION EXPECTED) ...)</macro>
53
54In the above, {{EXPRESSION}} and {{EXPECTED}} can be any scheme
55expression.
56
57This macro will evaluate in a left to right fashion the clauses
58inside it and output the results. The clauses are a sequence of
59{{(EXPRESSION EXPECTATION)}} forms. A clause is evaluated as
60{{(expect-equal 'EXPRESSION EXPECTATION EXPRESSION)}}. For use
61in a REPL.
62
63The results are not kept and no other forms are allowed in the body.
64
65==== The Quick Test Macro API
66
67This macro will execute the test procedure and send the results to a
68stylist procedure. The specifiers can be used to select and de-select
69test packages, groups, cases, and expectations.
70
71<macro>(test-apply [STYLER] [SPECIFIER ...] THUNK)</macro>
72
73{{STYLER}} is a single argument procedure taking the test result
74object, {{output-style-minimal}} when missing.  {{SPECIFIER}} is
75{{(skip MESSAGE ...)}} or {{(take MESSAGE ...)}}, where MESSAGE is a
76test element name as below. The {{skip}} specifier causes the named
77test elements not to be performed and the {{take}} specifier is the
78opposite. Do not use these forms together, they override each
79other. Only named test elements can be selected. Unnamed elements are
80always performed. Also, {{(skip CLAUSE ...)}} forms, see below, still
81apply.  {{THUNK}} is a zero argument procedure returning a test result
82object.
83
84Do not de-select all, or omit to select least one, test packages or
85groups. Otherwise the result will be ill-formed.
86
87The results are not kept.
88
89==== The Test Message Introspection API
90
91Provides information on the tree structure of a test.
92
93This macro will walk the test procedure(s), only capturing the type
94and message of each test element. An association list is returned
95where the key is the element type ({{thunk}}, {{package}}, {{case}},
96{{expect}}) and the value is the message (which will be {{#f}} when
97the element is not named). The {{thunk}} element value is the symbolic
98form of the {{THUNK}}.
99
100<macro>(test-named-structure THUNK ...)</macro>
101
102It is best if the {{THUNK}} argument is a procedure variable, and not
103a procedure object.
104
105The association list is a pre-order of the test hierarchy.
106
107<procedure>(walk-test-named-structure PROCEDURE NAMED-STRUCTURE)</procedure>
108
109This procedure will enumerate the {{NAMED-STRUCTURE}} association list,
110invoking the {{PROCEDURE}} for each element. The hierarchy level value
111is the tree depth at the point of call.
112
113{{PROCEDURE}} is a three argument procedure. The first argument is the
114hierarchy level, counting up from 0. The second argument is the type
115of the test element, as above. The third argument is the name of the
116test element.
117
118==== The Simple Test Package Macro API
119
120This macro will evaluate in a left to right fashion the clauses inside
121it. Clauses can only be certain things, detailed below. All of the
122clauses are executed, except of course if you bail out of the test
123group with the escape procedure mechanism. Note that a test group is a
124test package.
125
126The destructor object name is {{destructor}}. The escape procedure
127name is {{escape}}. Each nesting will lexically shadow the previous
128such definitions of an outer test group.
129
130<macro>(test-group [MESSAGE] [(BINDINGS)] CLAUSES)</macro>
131
132{{MESSAGE}} can be any scheme object, though usually it is a string.
133{{BINDINGS}} are let-style bindings that you may create and exist in
134the lexical scope of the test package.  {{CLAUSES}} are uses of
135{{(test-group ...)}}, {{(test-package ...)}}, {{(test-case ...)}},
136{{(expect* ...)}} along with {{(gloss ...)}}, {{(todo ...)}}, {{(skip
137...)}}, and {{(terminate ...)}} macros.  If the expectation fails, the
138test group macro will continue evaluating until all clauses are
139evaluated or the escape procedure mechanism is activated. This is
140different than a test-case macro where upon discovery of a failed
141expectation, evaluation stops immediately.
142
143==== The Test Package Macro API
144
145This macro will evaluate in a left to right fashion the clauses
146inside it. Clauses can only be certain things, detailed below. All of
147the clauses are executed, except of course if you bail out of the test
148package with the escape procedure mechanism. Test groups and packages
149may nest indefinitely.
150
151<macro>(test-package MESSAGE DESTNAME TERMNAME [(warn MESSAGE)] [(BINDINGS)] CLAUSES)</macro>
152
153{{MESSAGE}} can be any scheme object, though usually it is a string.
154
155{{DESTNAME}} is an unquoted symbol for an automatic destructor
156object that gets called when the test package completes for any reason.
157This symbol is bound to a destructor object and is available to you in
158the CLAUSES section of the test package. See below for the description
159of the destructor object interface.
160
161{{TERMNAME}} is an unquoted symbol for an escape procedure
162available in the body of the test package, usually, this escape
163procedure is passed to {{(terminate ...)}} which calls it
164for you and performs other tasks. It is not recommended to call the escape
165procedure directly.
166
167{{(warn MESSAGE)}} allows you to specify a warning object, usually
168a string, that gets associated with the test package. The
169{{warn}} function name is actually a syntax reserved word
170in the macro.
171
172{{BINDINGS}} are let-style bindings that you may create and exist in
173the lexical scope of the test package.
174
175{{CLAUSES}} are uses of {{(test-group ...)}}, {{(test-package ...)}},
176{{(test-case ...)}} macros along with {{(gloss ...)}}, {{(todo ...)}},
177{{(skip ...)}}, and {{(terminate ...)}} macros. While you may use the
178{{(expect* ...)}} style macros directly in a test package, doing so is
179not recommended. If the expectation fails, the test package macro will
180continue evaluating until all clauses are evaluated or the escape
181procedure mechanism is activated. This is different than a test-case
182macro where upon discovery of a failed expectation, evaluation stops
183immediately.
184
185==== The Test Case Macro API
186
187This macro will evaluate in a left to right fashion the clauses inside
188it, ''stopping at the first failed expectation''. Clauses can only
189be certain things as detailed below. You may also stop the execution of
190expectations if you bail out of the test case with the escape procedure
191mechanism. Test cases may ''NOT'' nest.
192
193<macro>(test-case MESSAGE DESTNAME TERMNAME [(warn MESSAGE)] [(BINDINGS)] CLAUSES)</macro>
194
195{{MESSAGE}} can be any scheme object, though usually it is a string.
196
197{{DESTNAME}} is an unquoted symbol for an automatic destructor
198object that gets called when the test case completes for any reason.
199This symbol is bound to a destructor object and is available to you in
200the CLAUSES section of the test package. See below for the description
201of the destructor object interface.
202
203{{TERMNAME}} is an unquoted symbol for an escape procedure
204available in the body of the test case, usually, this escape
205procedure is passed to {{(terminate ...)}} which calls it
206for you and performs other tasks. It is not recommended to call the escape
207procedure directly.
208
209{{(warn MESSAGE)}} allows you to specify a warning object, usually
210a string, that gets associated with the test case. The
211{{warn}} function name is actually a syntax reserved word
212in the macro.
213
214{{BINDINGS}} are let-style bindings that you may create and exist in
215the lexical scope of the test case.
216
217{{CLAUSES}} are uses of {{(expect* ...)}} macros
218along with {{(gloss ...)}}, {{(todo ...)}},
219{{(skip ...)}}, and {{(terminate ...)}} macros. It is
220important to note that upon discovery of a failed expectation,
221the test case stops its evaluation and returns with the previous
222successful, and including the failed, expectations. This behavior can be
223changed using the {{test-continue?}} form. This behavior reverts to
224default at the start of each test-case.
225
226==== The Expectation Macro API
227
228An expectation at its core simply evaluates its arguments and check to
229see if it matches the expectation. The positive or negative result is
230encapsulated, along with other things such as the unevaluated
231expressions being checked and some messages supplied with each
232expectation into a particular type of black box object that one can
233query with the appropriate API calls (detailed below).
234
235Expectations all have a descriptive message that can be bound to them,
236along with an optional warning syntax detailed below. A design
237decision was made to supply expectation macros for the usual types of
238expectations a user needs because this reduced the abstractness of an
239expectation into something more manageable.
240
241Expectations are evaluated under an exception handler. An unexpected
242exception is treated as expectation failure. Note that only the
243{{CLAUSE}} expression evaluation traps exceptions. Any exceptions in
244setup or teardown expressions will not be caught by the
245test-infrastructure.
246
247===== expect
248
249<macro>(expect [MESSAGE] [(warn MESSAGE)] KIND PREDICATE CLAUSE)</macro>
250
251This expectation checks to see if the evaluated expression passed to it
252meets the predicate condition.
253
254{{MESSAGE}} can be any scheme object, though usually it is a string.
255
256{{(warn MESSAGE)}} allows you to specify a warning object, usually
257a string, that gets associated with the expectation. The
258{{warn}} function name is actually a syntax reserved word
259in the macro.
260
261{{KIND}} is a string stating the kind of test.
262
263{{PREDICATE}} is a procedure of one argument to perform the test.
264
265{{CLAUSE}} is a single expression which should return a value meeting the
266conditions of the predicate.
267
268===== expect-zero
269
270<macro>(expect-zero [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
271
272This expectation checks to see if the evaluated expression passed to it
273is numerically equal to the exact integer zero.
274
275{{MESSAGE}} can be any scheme object, though usually it is a string.
276
277{{(warn MESSAGE)}} allows you to specify a warning object, usually
278a string, that gets associated with the expectation. The
279{{warn}} function name is actually a syntax reserved word
280in the macro.
281
282{{CLAUSE}} is a single expression which should return a exact or
283inexact integer.
284
285===== expect-nonzero
286
287<macro>(expect-nonzero [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
288
289This expectation checks to see if the evaluated expression passed to it
290is numerically not equal to the exact integer zero.
291
292{{MESSAGE}} can be any scheme object, though usually it is a string.
293
294{{(warn MESSAGE)}} allows you to specify a warning object, usually
295a string, that gets associated with the expectation. The
296{{warn}} function name is actually a syntax reserved word
297in the macro.
298
299{{CLAUSE}} is a single expression which should return an exact or
300inexact integer.
301
302===== expect-true
303
304<macro>(expect-true [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
305
306This expectation checks to see if the evaluated expression passed to it
307is the value {{#t}}.
308
309{{MESSAGE}} can be any scheme object, though usually it is a string.
310
311{{(warn MESSAGE)}} allows you to specify a warning object, usually
312a string, that gets associated with the expectation. The
313{{warn}} function name is actually a syntax reserved word
314in the macro.
315
316{{CLAUSE}} is a single expression which should return {{#t}}.
317
318===== expect-false
319
320<macro>(expect-false [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
321
322This expectation checks to see if the evaluated expression passed to it
323is the value {{{#f}}.
324
325{{MESSAGE}} can be any scheme object, though usually it is a string.
326
327{{(warn MESSAGE)}} allows you to specify a warning object, usually
328a string, that gets associated with the expectation. The
329{{warn}} function name is actually a syntax reserved word
330in the macro.
331
332{{CLAUSE}} is a single expression which should return {{#f}}.
333
334===== expect-success
335
336<macro>(expect-success [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
337
338This expectation checks to see if the evaluated expression passed to it
339is the value {{#t}}, or a non-error exception occured.
340
341{{MESSAGE}} can be any scheme object, though usually it is a string.
342
343{{(warn MESSAGE)}} allows you to specify a warning object, usually
344a string, that gets associated with the expectation. The
345{{warn}} function name is actually a syntax reserved word
346in the macro.
347
348{{CLAUSE}} is a single expression which should return {{#t}}, or
349generate a non-error exception.
350
351===== expect-failure
352
353<macro>(expect-failure [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
354
355This expectation checks to see if the evaluated expression passed to it
356is the value {{#f}}, or an error exception occured.
357
358{{MESSAGE}} can be any scheme object, though usually it is a string.
359
360{{(warn MESSAGE)}} allows you to specify a warning object, usually a
361string, that gets associated with the expectation. The {{warn}}
362function name is actually a syntax reserved word in the macro.
363
364{{CLAUSE}} is a single expression which should return {{#f}}, or
365generate an error exception.
366
367===== expect-not-false
368
369(expect-not-false [MESSAGE] [(warn MESSAGE)] CLAUSE)
370
371This expectation checks to see if the evaluated expression passed to
372it is not the value {{#f}}.
373
374{{MESSAGE}} can be any scheme object, though usually it is a string.
375
376{{(warn MESSAGE)}} allows you to specify a warning object, usually
377a string, that gets associated with the expectation. The
378{{warn}} function name is actually a syntax reserved word
379in the macro.
380
381{{CLAUSE}} is a single expression which should not return
382{{#f}}, any other value is accepted.
383
384===== expect-equiv
385
386<macro>(expect-equiv [MESSAGE] [(warn MESSAGE)] KIND PREDICATE EXPECTED CLAUSE)</macro>
387
388This expectation checks to see if {{(PREDICATE EXPECTED CLAUSE)}} is true.
389
390{{MESSAGE}} can be any scheme object, though usually it is a string.
391
392{{(warn MESSAGE)}} allows you to specify a warning object, usually
393a string, that gets associated with the expectation. The
394{{warn}} function name is actually a syntax reserved word
395in the macro.
396
397{{EXPECTED}} is a single expression which is evaluated and represents
398the value the {{CLAUSE}} ''must'' be equiv? to in order for this
399expectation to return a positive result.
400
401{{CLAUSE}} is a single expression which, when evaluated must return
402an object where an equiv? of this result and the {{EXPECTED}} expression
403is {{#t}}.
404
405{{KIND}} is a string stating the kind of test.
406
407{{PREDICATE}} is a procedure of two arguments to perform the test.
408
409The result object this macro produce shall contain the unevaluated
410{{CLAUSE}} expression as a field, but not an unevaluated {{EXPECTED}}
411expression.
412
413===== expect-eq
414
415<macro>(expect-eq [MESSAGE] [(warn MESSAGE)] EXPECTED CLAUSE)</macro>
416
417This expectation checks to see if {{(eq? EXPECTED CLAUSE)}} is true.
418
419{{MESSAGE}} can be any scheme object, though usually it is a string.
420
421{{(warn MESSAGE)}} allows you to specify a warning object, usually
422a string, that gets associated with the expectation. The
423{{warn}} function name is actually a syntax reserved word
424in the macro.
425
426{{EXPECTED}} is a single expression which is evaluated and represents
427the value the {{CLAUSE}} ''must'' be {{eq?}} to in order for this
428expectation to return a positive result.
429
430{{CLAUSE}} is a single expression which, when evaluated must return
431an object where an eq? of this result and the {{EXPECTED}} expression
432is {{#t}}.
433
434The result object this macro produce shall contain the unevaluated
435{{CLAUSE}} expression as a field, but not an unevaluated {{EXPECTED}}
436expression.
437
438===== expect-eqv
439
440<macro>(expect-eqv [MESSAGE] [(warn MESSAGE)] EXPECTED CLAUSE)</macro>
441
442This expectation checks to see if {{(eqv? EXPECTED CLAUSE)}} is true.
443
444{{MESSAGE}} can be any scheme object, though usually it is a string.
445
446{{(warn MESSAGE)}} allows you to specify a warning object, usually
447a string, that gets associated with the expectation. The
448{{warn}} function name is actually a syntax reserved word
449in the macro.
450
451{{EXPECTED}} is a single expression which is evaluated and represents
452the value the {{CLAUSE}} ''must'' be {{eqv?}} to in order for this
453expectation to return a positive result.
454
455{{CLAUSE}} is a single expression which, when evaluated must return
456an object where an eqv? of this result and the {{EXPECTED}} expression
457is {{#t}}.
458
459The result object this macro produce shall contain the unevaluated
460{{CLAUSE}} expression as a field, but not an unevaluated {{EXPECTED}}
461expression.
462
463===== expect-equal
464
465<macro>(expect-equal [MESSAGE] [(warn MESSAGE)] EXPECTED CLAUSE)</macro>
466
467This expectation checks to see if {{(equal? EXPECTED CLAUSE)}} is true.
468
469{{MESSAGE}} can be any scheme object, though usually it is a string.
470
471{{(warn MESSAGE)}} allows you to specify a warning object, usually
472a string, that gets associated with the expectation. The
473{{warn}} function name is actually a syntax reserved word
474in the macro.
475
476{{EXPECTED}} is a single expression which is evaluated and represents
477the value the {{CLAUSE}} ''must'' be equal? to in order for this
478expectation to return a positive result.
479
480{{CLAUSE}} is a single expression which, when evaluated must return
481an object where an equal? of this result and the {{EXPECTED}} expression
482is {{#t}}.
483
484The result object this macro produce shall contain the unevaluated
485{{CLAUSE}} expression as a field, but not an unevaluated {{EXPECTED}}
486expression.
487
488===== expect-values
489
490<macro>(expect-values [MESSAGE] [(warn MESSAGE)] PREDICATE CLAUSE)</macro>
491
492This expectation checks to see if the evaluated multi-valued expression passed to it
493meets the criteria of the single argument procedure {{PREDICATE}}.
494
495{{MESSAGE}} can be any scheme object, though usually it is a string.
496
497{{(warn MESSAGE)}} allows you to specify a warning object, usually
498a string, that gets associated with the expectation. The
499{{warn}} function name is actually a syntax reserved word
500in the macro.
501
502{{PREDICATE}} is a procedure taking a single list argument and returning {{#t}} or {{#f}}.
503
504{{CLAUSE}} is a single expression which has a multi-valued result.
505
506===== expect-values-eq
507
508<macro>(expect-values-eq [MESSAGE] [(warn MESSAGE)] VALUES CLAUSE)</macro>
509
510This expectation checks to see if the result of the evaluated
511multi-valued expression passed to it is element-wise {{eq?}} to the
512{{VALUES}} list. The count of returned multiple values must match the
513length of the {{VALUES}} list.
514
515{{MESSAGE}} can be any scheme object, though usually it is a string.
516
517{{(warn MESSAGE)}} allows you to specify a warning object, usually
518a string, that gets associated with the expectation. The
519{{warn}} function name is actually a syntax reserved word
520in the macro.
521
522{{VALUES}} is a list of values.
523
524{{CLAUSE}} is a single expression which has a multi-valued result.
525
526===== expect-values-eqv
527
528<macro>(expect-values-eqv [MESSAGE] [(warn MESSAGE)] VALUES CLAUSE)</macro>
529
530This expectation checks to see if the result of the evaluated multi-valued
531expression passed to it is element-wise eqv? to the {{VALUES}} list. The count of
532returned multiple values must match the length of the {{VALUES}} list.
533
534{{MESSAGE}} can be any scheme object, though usually it is a string.
535
536{{(warn MESSAGE)}} allows you to specify a warning object, usually
537a string, that gets associated with the expectation. The
538{{warn}} function name is actually a syntax reserved word
539in the macro.
540
541{{VALUES}} is a list of values.
542
543{{CLAUSE}} is a single expression which has a multi-valued result.
544
545===== expect-values-equal
546
547<macro>(expect-values-equal [MESSAGE] [(warn MESSAGE)] VALUES CLAUSE)</macro>
548
549This expectation checks to see if the result of the evaluated
550multi-valued expression passed to it is element-wise {{equal?}} to the
551{{VALUES}} list. The count of returned multiple values must match the
552length of the {{VALUES}} list.
553
554{{MESSAGE}} can be any scheme object, though usually it is a string.
555
556{{(warn MESSAGE)}} allows you to specify a warning object, usually
557a string, that gets associated with the expectation. The
558{{warn}} function name is actually a syntax reserved word
559in the macro.
560
561{{VALUES}} is a list of values.
562
563{{CLAUSE}} is a single expression which has a multi-valued result.
564
565===== expect-near
566
567<macro>(expect-near [MESSAGE] [(warn MESSAGE)] EXPECTED TOL CLAUSE)</macro>
568
569This expectation checks to see if {{(< (abs (- EXPECTED CLAUSE)) TOL)}} is true.
570
571{{MESSAGE}} can be any scheme object, though usually it is a string.
572
573{{(warn MESSAGE)}} allows you to specify a warning object, usually
574a string, that gets associated with the expectation. The
575{{warn}} function name is actually a syntax reserved word
576in the macro.
577
578{{EXPECTED}} is a single expression which is evaluated and represents
579the value the {{CLAUSE}} ''must'' be "near" to in order for this
580expectation to return a positive result.
581
582{{CLAUSE}} is a single expression which should return an inexact or
583exact number.
584
585{{TOL}} is a single expression which, when evaluated must return a
586tolerance value (usually a small inexact number like .0001).
587
588The result object this macro produce shall contain the unevaluated
589{{CLAUSE}} expression as a field, but not the unevaluated {{EXPECTED}}
590or {{TOL}} expression.
591
592===== expect-positive
593
594<macro>(expect-positive [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
595
596This expectation checks to see if the evaluated expression passed to it
597is a positive value greater than zero.
598
599{{MESSAGE}} can be any scheme object, though usually it is a string.
600
601{{(warn MESSAGE)}} allows you to specify a warning object, usually
602a string, that gets associated with the expectation. The
603{{warn}} function name is actually a syntax reserved word
604in the macro.
605
606{{CLAUSE}} is a single expression which should return an inexact or
607exact number.
608
609
610===== expect-negative
611
612<macro>(expect-negative [MESSAGE] [(warn MESSAGE)] CLAUSE)</macro>
613
614This expectation checks to see if the evaluated expression passed to it
615is a negative value less than zero.
616
617{{MESSAGE}} can be any scheme object, though usually it is a string.
618
619{{(warn MESSAGE)}} allows you to specify a warning object, usually
620a string, that gets associated with the expectation. The
621{{warn}} function name is actually a syntax reserved word
622in the macro.
623
624{{CLAUSE}} is a single expression which should return an inexact or
625exact number.
626
627===== expect-exception
628
629<macro>(expect-exception [MESSAGE] [(warn MESSAGE)] EXN CLAUSE)</macro>
630
631This expectation checks to see if the evaluated expression passed to it
632signals an exception that matches the supplied exception pattern {{EXN}}.
633
634{{MESSAGE}} can be any scheme object, though usually it is a string.
635
636{{(warn MESSAGE)}} allows you to specify a warning object, usually a
637string, that gets associated with the expectation. The {{warn}}
638function name is actually a syntax reserved word in the macro.
639
640{{EXN}} is a specialized logical expression describing the expected
641exception, a condition as defined by SRFI-12.
642
643{{CLAUSE}} is a single expression which should signal an exception
644matching EXN.
645
646  EXN : COND-EXN
647      | SIMP-EXN
648      | ()
649 
650  SIMP-EXN : KINDKEY
651           | (property KINDKEY PROP-EXPR ...)
652           | (SIMP-EXN ...)
653 
654  COND-EXN : (and EXN ...)
655           | (or EXN ...)
656           | (not EXN)
657 
658  PROP-EXPR : COND-PROP-EXPR
659            | SIMP-PROP-EXPR
660            | ()
661 
662  SIMP-PROP-EXPR : PROPKEY
663                 | (PROPKEY ...)
664                 | (PROPKEY VALUE)
665                 | (PREDICATE PROPKEY VALUE)
666 
667  COND-PROP-EXPR : (and PROP-EXPR ...)
668                 | (or PROP-EXPR ...)
669                 | (not PROP-EXPR)
670
671Here's an example:
672
673<example>
674(test-case "exn1" d e
675    ([exn1 (make-composite-condition
676             (make-property-condition 'abc 'cbs "pbs")
677             (make-property-condition 'foo 'bar "zip"))])
678 
679  (expect-exception "1" (foo abc) (signal exn1))
680 
681  ;; These are the same test, the 2nd has an explicit predicate
682  (expect-exception "2" ((property foo (bar "zip"))
683                         (property abc (cbs "pbs"))) (signal exn1))
684  (expect-exception "3" ((property foo (equal? bar "zip"))
685                         (property abc (equal? cbs "pbs"))) (signal exn1)))
686</example>
687
688
689===== expect-ec
690
691<macro>(expect-ec [MESSAGE] [(warn MESSAGE)] QUALIFIER ... EXPECTED [=> PREDICATE] CLAUSE)</macro>
692
693This expectation checks to see if {{(PREDICATE EXPECTED CLAUSE)}} is
694true for every state produced by the set of {{QUALIFIER}}.
695
696{{MESSAGE}} can be any scheme object, though usually it is a string.
697
698{{(warn MESSAGE)}} allows you to specify a warning object, usually
699a string, that gets associated with the expectation. The
700{{warn}} function name is actually a syntax reserved word
701in the macro.
702
703{{CLAUSE}} and {{EXPECTED}} are expressions.
704
705{{PREDICATE}} is a two argument procedure returning a
706boolean, {{equal?}} is used when missing.
707
708{{QUALIFIER}} is a SRFI-42 {{<qualifier>}. An eager
709comprehension implementation, as described by
710[[http://srfi.schemers.org/srfi-42/srfi-42.html|SRFI-42]] must be
711provided to use this expectation.
712
713===== expect-set!
714
715<macro>(expect-set! ID EXPR)</macro>
716
717Evaluates {{EXPR}} under an {{expect-success}}. When successful
718a {{test:set!}} is performed of the {{EXPR}} to {{ID}}.
719The {{EXPR}} is therefore evaluated '''twice'''.
720
721===== expect-successful-failure
722
723<macro>(expect-successful-failure EXPR)</macro>
724
725Same as {{(expect-success "Failure" EXPR)}}.
726
727==== Testeez-like API
728
729A suite of synonyms for the {{expect-*}} macros. Note that the {{(warn
730WARNING)}} argument form is not supported. Here {{DESC}} is the same
731as {{MESSAGE}}, and {{EXPR}} is the same as {{CLAUSE}}.
732
733* <macro>(test/zero [DESC] EXPR)</macro>
734* <macro>(test/nonzero [DESC] EXPR)</macro>
735* <macro>(test/true [DESC] EXPR)</macro>
736* <macro>(test/false [DESC] EXPR)</macro>
737* <macro>(test/success [DESC] EXPR)</macro>
738* <macro>(test/failure [DESC] EXPR)</macro>
739* <macro>(test/positive [DESC] EXPR)</macro>
740* <macro>(test/negative [DESC] EXPR)</macro>
741* <macro>(test/not-false [DESC] EXPR)</macro>
742* <macro>(test/equiv [DESC] EXPR EXPECTED PRED)</macro>
743* <macro>(test/eq [DESC] EXPR EXPECTED)</macro>
744* <macro>(test/eqv [DESC] EXPR EXPECTED)</macro>
745* <macro>(test/equal [DESC] EXPR EXPECTED)</macro>
746* <macro>(test/near [DESC] EXPR EXPECTED [TOL 0.0001])</macro>
747* <macro>(test/values [DESC] EXPR EXPECTED)</macro>
748* <macro>(test/values-eq [DESC] EXPR EXPECTED)</macro>
749* <macro>(test/values-eqv [DESC] EXPR EXPECTED)</macro>
750* <macro>(test/values-equal [DESC] EXPR EXPECTED)</macro>
751* <macro>(test/exception [DESC] EXPR EXPECTED)</macro>
752* <macro>(test/ec [DESC] QUALIFIER ... EXPECTED [=> PRED] EXPR)</macro>
753
754{{test/ec}} is just a synonym. Cannot follow testeez argument order
755pattern.
756
757==== Result Object API
758
759Expectations, test cases, test packages, and helper macros ({{gloss}},
760{{todo}}, etc) all return an object that contains the results and
761other various aspects of the action performed which ultimately get
762wired together to form the result tree. This collection of functions
763forming the rest of the test infrastructure API allows manipulation of
764these results in an abstracted way as to allow changing of the
765representation in the future.
766
767===== Test Package Result Object API
768
769If any of these API functions, except ''test-package-result?'', are
770passed something that isn't a test package result object, they will
771return {{'not-a-test-package-result}}.
772
773<procedure>(test-package-result? RESULT)</procedure>
774
775If {{RESULT}} is a result object from the invocation
776of a test package macro, then this function will return
777{{#t}}. Otherwise, it will return {{#f}}.
778
779<procedure>(test-package-result-result-ref RESULT)</procedure>
780
781Returns the boolean result associated with the test package {{RESULT}}
782object.
783
784<procedure>(test-package-result-message-ref RESULT)</procedure>
785
786Returns the message object associated with the test package {{RESULT}}
787object.
788
789<procedure>(test-package-result-exps-ref RESULT)</procedure>
790
791Returns the list of result objects associated with the test package
792{{RESULT}} object.
793
794<procedure>(test-package-result-warning? RESULT)</procedure>
795
796If a warning had been attached to this test package, this function
797will return {{#t}}, otherwise it will be {{#f}}.
798
799<procedure>(test-package-result-warning-ref RESULT)</procedure>
800
801If a warning had been attached to this test package, this function
802will return the warning object supplied by the user, otherwise it
803shall return {{'()}}.
804
805===== Test Case Result Object API
806
807If any of these API functions, except {{test-case-result?}},
808are passed something that isn't a test case result object, they will
809return {{'not-a-test-case-result}}.
810
811<procedure>(test-case-result? RESULT)</procedure>
812
813If {{RESULT}} is a result object from the invocation
814of a test case macro, then this function will return
815{{#t}}. Otherwise, it will return {{#f}}.
816
817<procedure>(test-case-result-result-ref RESULT)</procedure>
818
819Returns the boolean result associated with the test case {{RESULT}}
820object.
821
822<procedure>(test-case-result-message-ref RESULT)</procedure>
823
824Returns the message object associated with the test case {{RESULT}}
825object.
826
827<procedure>(test-case-result-expectations-ref RESULT)</procedure>
828
829Returns the list of expectation result objects associated with the test
830case {{RESULT}} object.
831
832<procedure>(test-case-result-warning? RESULT)</procedure>
833
834If a warning had been attached to this test case, this function
835will return {{#t}}, otherwise it will be {{#f}}.
836
837<procedure>(test-case-result-warning-ref RESULT)</procedure>
838
839If a warning had been attached to this test case, this function
840will return the warning object supplied by the user, otherwise it
841shall return {{'()}}.
842
843===== Expect Result Object API: Single Clause Style Expectation
844
845These expectations all take the form of passing a single expression to
846them to see if they match some a priori expectation. If any of these
847API functions, except {{expect-result?}}, are passed something that
848isn't a single clause style expectation result object, they will return
849{{'not-an-expect-result}}.
850
851<procedure>(expect-result? RESULT)</procedure>
852
853If {{RESULT}} is a single clause style result object from
854the invocation of an expectation macro, then this function will
855return {{#t}}. Otherwise, it will return {{#f}}.
856
857<procedure>(expect-result-result-ref RESULT)</procedure>
858
859Returns the boolean result associated with the single clause
860style expectation {{RESULT}} object.
861
862<procedure>(expect-result-specific-ref RESULT)</procedure>
863
864This retrieves the {{specific}} field of a particular single clause
865style expectation. For example, if you had a result object from
866an invocation of a {{(expect-zero? "foobar" (- 1 1))}}
867expectation, then the {{specific}} field of the expectation
868result object will be the string {{"zero"}}. Here is a
869table describing what the {{specific}} fields are for each kind
870of single clause style expectation:
871
872<table border="1">
873<tr><th>Single Clause Style Expectation</th><th>Associated Specific String</th></tr>
874<tr><td>expect-zero</td>                     <td>"zero"</td></tr>
875<tr><td>expect-nonzero</td>                  <td>"nonzero"</td></tr>
876<tr><td>expect-true</td>                     <td>"true"</td></tr>
877<tr><td>expect-not-false</td>                <td>"not-false"</td></tr>
878<tr><td>expect-false</td>                    <td>"false"</td></tr>
879<tr><td>expect-positive</td>                 <td>"positive"</td></tr>
880<tr><td>expect-negative</td>                 <td>"negative"</td></tr>
881<tr><td>expect</td>                          <td>{{KIND}}</td></tr>
882</table>
883
884<procedure>(expect-result-message-ref RESULT)</procedure>
885
886Returns the message object associated with the single clause
887style expectation {{RESULT}} object.
888
889<procedure>(expect-result-unevaled-ref RESULT)</procedure>
890
891Returns the unevaluated expression supplied to a single clause style
892expectation macro.
893
894<procedure>(expect-result-evaled-ref RESULT)</procedure>
895
896Returns the evaluated expression supplied to a single clause style
897expectation macro.
898
899<procedure>(expect-result-warning? RESULT)</procedure>
900
901If a warning had been attached to this expectation, this function
902will return {{#t}}, otherwise it will be {{#f}}.
903
904<procedure>(expect-result-warning-ref RESULT)</procedure>
905
906If a warning had been attached to this expectation, this function
907will return the warning object supplied by the user, otherwise it
908shall return {{'()}}.
909
910===== Expect Result Object API: Equivalence Style Expectation
911
912These expectations all take the form of passing a two expressions,
913the "left hand side" and the "right hand side" to them to see
914if they match some a priori equivalence. The left hand side is that
915which you expect the right hand side to be equivalent. If any of these
916API functions, except {{expect-equivalence-result?}}, are passed
917something that isn't a single clause style expectation result object,
918they will return {{'not-an-expect-equivalence-result}}.
919
920<procedure>(expect-equivalence-result? RESULT)</procedure>
921
922If {{RESULT}} is a comparison style result object from
923the invocation of an expectation macro, then this function will
924return {{#t}}. Otherwise, it will return {{#f}}.
925
926<procedure>(expect-equivalence-result-result-ref RESULT)</procedure>
927
928Returns the boolean result associated with the comparison style
929expectation {{RESULT}} object.
930
931<procedure>(expect-equivalence-result-specific-ref RESULT)</procedure>
932
933This retrieves the "specific" field of a particular equivalence
934style expectation. For example, if you had a result object from
935an invocation of a {{(expect-equal? "foobar" 0 (- 1 1))}}
936expectation, then the ``specific'' field of the expectation
937result object will be the string {{"equal"}}. Here is a
938table describing what the "specific" fields are for each kind
939of equivalence style expectation:
940
941<table border="1"><tr><th>Equivalence Style Expectation</th><th>Associated Specific String</th></tr>
942<tr><td>expect-eq</td>                     <td>"eq"</td></tr>
943<tr><td>expect-eqv</td>                    <td>"eqv"</td></tr>
944<tr><td>expect-equal</td>                  <td>"equal"</td></tr>
945<tr><td>expect-exception</td>              <td>"exception"</td></tr>
946<tr><td>expect-equiv</td>                  <td>{{KIND}}</td></tr>
947</table>
948
949<procedure>(expect-equivalence-result-message-ref RESULT)</procedure>
950
951Returns the message object associated with the equivalence
952style expectation {{RESULT}} object.
953
954<procedure>(expect-equivalence-result-lhs-evaled-ref RESULT)</procedure>
955
956Returns the evaluated "left hand side" expression supplied to
957an equivalence style expectation.
958
959<procedure>(expect-equivalence-result-rhs-unevaled-ref RESULT)</procedure>
960
961Returns the unevaluated "right hand side" expression supplied
962to an equivalence style expectation.
963
964<procedure>(expect-equivalence-result-rhs-evaled-ref RESULT)</procedure>
965
966Returns the evaluated "right hand side" expression supplied
967to an equivalence style expectation.
968
969<procedure>(expect-equivalence-result-warning? RESULT)</procedure>
970
971If a warning had been attached to this expectation, this function
972will return {{#t}}, otherwise it will be {{#f}}.
973
974<procedure>(expect-equivalence-result-warning-ref RESULT)</procedure>
975
976If a warning had been attached to this expectation, this function
977will return the warning object supplied by the user, otherwise it
978shall return {{'()}}.
979
980===== Expect Result Object API: Tolerance Style Expectation
981
982This is a specialized expectation which accepts three expressions and
983checks to see if the "right hand side" is within a "tolerance"
984of the "left hand side". There is only one expectation in the
985tolerance style currently. If any of these API functions, except
986{{expect-tolerance-result?}}, are passed something that isn't
987a tolerance style expectation result object, they will return
988{{'not-an-expect-tolerance-result}}.
989
990<procedure>(expect-tolerance-result? RESULT)</procedure>
991
992If {{RESULT}} is a tolerance style result object from
993the invocation of an expectation macro, then this function will
994return {{#t}}. Otherwise, it will return {{#f}}.
995
996<procedure>(expect-tolerance-result-result-ref RESULT)</procedure>
997
998Returns the boolean result associated with the tolerance style
999expectation {{RESULT}} object.
1000
1001<procedure>(expect-tolerance-result-specific-ref RESULT)</procedure>
1002
1003This retrieves the {{specific}} field of a particular tolerance
1004style expectation. For example, if you had a result object from
1005an invocation of a {{(expect-near? "foobar" 100 .01 100.001)}}
1006expectation, then the {{specific}} field of the expectation
1007result object will be the string {{"near"}}. Here is a
1008table describing what the {{specific}} fields are for each kind
1009of tolerance style expectation
1010
1011<procedure>(expect-tolerance-result-message-ref RESULT)</procedure>
1012
1013Returns the message object associated with a tolerance
1014style expectation {{RESULT}} object.
1015
1016<procedure>(expect-tolerance-result-lhs-evaled-ref RESULT)</procedure>
1017
1018Returns the evaluated "left hand side" expression supplied to
1019a tolerance style expectation.
1020
1021<procedure>(expect-tolerance-result-lhs-tol-evaled-ref RESULT)</procedure>
1022
1023Returns the evaluated {{tolerance}} expression supplied to
1024a tolerance style expectation.
1025
1026<procedure>(expect-tolerance-result-rhs-unevaled-ref RESULT)</procedure>
1027
1028Returns the unevaluated "right hand side" expression supplied
1029to a tolerance style expectation.
1030
1031<procedure>(expect-tolerance-result-rhs-evaled-ref RESULT)</procedure>
1032
1033Returns the evaluated "right hand side" expression supplied
1034to a tolerance style expectation.
1035
1036<procedure>(expect-tolerance-result-warning? RESULT)</procedure>
1037
1038If a warning had been attached to this expectation, this function
1039will return {{#t}}, otherwise it will be {{#f}}.
1040
1041<procedure>(expect-tolerance-result-warning-ref RESULT)</procedure>
1042
1043If a warning had been attached to this expectation, this function
1044will return the warning object supplied by the user, otherwise it
1045shall return {{'()}}.
1046
1047==== Various Helper API
1048
1049These upcoming macros and functions allow the author of the test suite to
1050better control both the execution flow of the test suite and ``decoration''
1051of the test suite with important information like things yet to do, or just
1052plain documentation.
1053
1054===== Termination API
1055
1056When executing in a test package or a test case, one might discover
1057some catastrophic failure of such proportions that it is utterly
1058impossible to continue executing the test case or test package. When
1059that happens you can use the termination facility to exit the test
1060case or test package. Of course, no more expressions will be evaluated
1061in the scope of the termination. It is recommended that you use this
1062method of terminating the test case or test package evaluation since
1063it wraps some contextual information up into the termination result so
1064you can figure out what happened (and where) later when analyzing the
1065result tree.
1066
1067When using the manipulation API for a terminate result, if you pass a
1068result to one of these function that is not a terminate result, it will
1069return 'not-a-terminate-result.
1070
1071<procedure>(terminate TERMFUNC MESSAGE)</procedure>
1072
1073This is the recommended termination method for a test case or a test
1074package.
1075
1076{{TERMFUNC}} is the name of the termination procedure that you
1077specified in a test case or test package. You may pass any test
1078package or test case termination function available to you in the
1079lexical scope in which you call this function. The termination will
1080take effect in the scope of the created termination function.
1081
1082{{MESSAGE}} can be any scheme object, though usually it is a string.
1083
1084<procedure>(terminate-result? RESULT)</procedure>
1085
1086If {{RESULT}} is a termination result object from the
1087invocation of a termination function, then this function will
1088return {{#t}}. Otherwise, it will return {{#f}}.
1089
1090<procedure>(terminate-result-result-ref RESULT)</procedure>
1091
1092Returns the boolean result associated with the termination function
1093{{RESULT}} object. This is currently hard coded to be {{#f}}.
1094
1095<procedure>(terminate-result-scope-ref RESULT)</procedure>
1096
1097The ''scope'' of the termination result is exactly the {{MESSAGE}}
1098parameter supplied to the test case or test package associated with
1099the {{TERMFUNC}}.
1100
1101<procedure>(terminate-result-container-ref RESULT)</procedure>
1102
1103The ''container'' of the termination result is going to be either
1104{{'test-package}} or {{'test-case}} depending upon which the
1105{{TERMFUNC}} was associated.
1106
1107<procedure>(terminate-result-message-ref RESULT)</procedure>
1108
1109Returns the message object associated with the termination
1110{{RESULT}} object.
1111
1112===== Destructor Object API
1113
1114The destructor object allows for you to create helper functions
1115which clean up for you usually in case of aborting of a test case
1116or package. For example, suppose you are testing whether or not file
1117writing to a file works correctly in a test case, so, you'd perform an
1118expectation to open the file, and then queue a function in the destructor
1119to remove the file, and then perform the expectation of the write. If
1120the write (or subsequent) expectation fails, then the test case will
1121''automatically'' invoke the helper cleanup function specified in
1122the destructor object that removes the file.
1123
1124'''NOTE:''' This API is still a little experimental in the sense that
1125eventually he destructor object should return a typed result that
1126contains the success of the individual destructor calls. But for now,
1127it is functional for what it does. Also, be ''VERY CAREFUL'' that you
1128specify the arguments to these API calls correctly since due to lambda
1129functions not being comparable, this API cannot guarantee that a true
1130destructor object name had been passed to it. So if you call one of
1131the following API calls incorrectly, the behavior will be undefined.
1132
1133<procedure>(destructor-atexit! DESTNAME FUNC ARGS ...)</procedure>
1134
1135This will insert a promise to calculate the {{FUNC}} with the supplied
1136{{ARGS ...}} into a queue in the {{DESTNAME}} destructor
1137object. Multiple invocations of this API call will continue to queue
1138up {{(FUNC ARGS ...)}} promises indefinitely.  This function returns a
1139special ''ignore'' type that is ignored by the test infrastructure
1140system.
1141
1142<procedure>(destructor-activate! DESTNAME)</procedure>
1143
1144This function will call, in order of queueing, all the promises
1145embedded into this destructor object, and then delete the queue. This
1146function is ''ALWAYS'' called at the completion of a test package or
1147test case; so be careful that the destructor object doesn't contain
1148anything harmful. However, you may call it yourself and if you do, it
1149will execute all of the queued promises and then {{clear}}
1150itself. This function returns a special ''ignore'' type that is
1151ignored by the test infrastructure system.
1152
1153<procedure>(destructor-clear! DESTNAME)</procedure>
1154
1155This function completely removes all of the promises associated with
1156the destructor object {{DESTNAME}}. This function returns a special
1157''ignore'' type that is ignored by the test infrastructure system.
1158
1159<procedure>(destructor-dump DESTNAME)</procedure>
1160
1161This function, mostly used for debugging purposes, prints out a simple
1162representation of the queued atexit functions to the current
1163port. This function returns a special ''ignore'' type that is ignored
1164by the test infrastructure system.
1165
1166==== Todo API
1167
1168The purpose of the todo API is to allow the author of a test suite the
1169ability to record into the result tree for later analysis that
1170something still needs to be done. This way you can count/manipulate
1171this information at a later date. Todo macro invocations can occur
1172inside of test cases or test packages.
1173
1174<macro>(todo MESSAGE)</macro>
1175<macro>(todo (warn WARNING) MESSAGE)</macro>
1176
1177{{MESSAGE}} can be any scheme object, though usually it is a string.
1178
1179{{(warn WARNING)}} allows you to specify a warning object, usually a
1180string, that gets associated with the todo. The {{warn}} function name
1181is actually a syntax reserved word in the macro.
1182
1183<procedure>(todo-result? RESULT)</procedure>
1184
1185If {{RESULT}} is a todo result object from the invocation of a todo
1186macro, then this function will return {{#t}}. Otherwise, it will
1187return {{#f}}.
1188
1189<procedure>(todo-result-message-ref RESULT)</procedure>
1190
1191Returns the message object associated with the todo
1192{{RESULT}} object.
1193
1194<procedure>(todo-result-warning? RESULT)</procedure>
1195
1196If a warning had been attached to this todo, this function
1197will return {{#t}}, otherwise it will be {{#f}}.
1198
1199<procedure>(todo-result-warning-ref RESULT)</procedure>
1200
1201If a warning had been attached to this todo, this function
1202will return the warning object supplied by the user, otherwise it
1203shall return {{'()}}.
1204
1205==== Gloss API
1206
1207The purpose of the gloss API is to allow the author of a test suite the
1208ability to record messages into the result tree purely for documentation
1209purposes. Gloss macro invocations can occur inside of test cases or
1210test packages.
1211
1212<macro>(gloss MESSAGE)</macro>
1213<macro>(gloss (warn WARNING) MESSAGE)</macro>
1214
1215{{MESSAGE}} can be any scheme object, though usually it is a string.
1216
1217{{(warn WARNING)}} allows you to specify a warning object, usually
1218a string, that gets associated with the gloss. The {{warn}}
1219function name is actually a syntax reserved word in the macro.
1220
1221<procedure>(gloss-result? RESULT)</procedure>
1222
1223If {{RESULT}} is a gloss result object from the invocation of the
1224gloss macro, then this function will return {{#t}}. Otherwise, it will
1225return {{#f}}.
1226
1227<procedure>(gloss-result-message-ref RESULT)</procedure>
1228
1229Returns the message object associated with the gloss {{RESULT}}
1230object.
1231
1232<procedure>(gloss-result-warning? RESULT)</procedure>
1233
1234If a warning had been attached to this gloss, this function
1235will return {{#t}}, otherwise it will be {{#f}}.
1236
1237<procedure>(gloss-result-warning-ref RESULT)</procedure>
1238
1239If a warning had been attached to this gloss, this function
1240will return the warning object supplied by the user, otherwise it
1241shall return {{'()}}.
1242
1243==== Skip API
1244
1245The purpose of the skip API is to allow the author of a test suite to
1246completely skip evaluation of a set of expressions. Skip macro
1247invocations can occur inside of test cases or test packages.
1248
1249<macro>(skip MESSAGE CLAUSES)</macro>
1250<macro>(skip (warn WARNING) MESSAGE CLAUSES)</macro>
1251
1252{{MESSAGE}} can be any scheme object, though usually it is a string.
1253
1254{{(warn WARNING)}} allows you to specify a warning object, usually
1255a string, that gets associated with the gloss. The {{warn}}
1256function name is actually a syntax reserved word in the macro.
1257
1258{{CLAUSES}} can be more than one expression (as in a lambda form)
1259that does ''NOT'' get evaluated at any time.
1260
1261<procedure>(skip-result? RESULT)</procedure>
1262
1263If {{RESULT}} is a skip result object from the
1264invocation of the skip macro, then this function will
1265return {{#t}}. Otherwise, it will return {{#f}}.
1266
1267<procedure>(skip-result-message-ref RESULT)</procedure>
1268
1269Returns the message object associated with the skip
1270{{RESULT}} object. Hopefully, it was stated why this
1271set of clauses had been skipped.
1272
1273<procedure>(skip-result-warning? RESULT)</procedure>
1274
1275If a warning had been attached to this skip, this function
1276will return {{#t}}, otherwise it will be {{#f}}.
1277
1278<procedure>(skip-result-warning-ref RESULT)</procedure>
1279
1280If a warning had been attached to this skip, this function
1281will return the warning object supplied by the user, otherwise it
1282shall return {{'()}}.
1283
1284==== Side Effect API
1285
1286This section of the API  is considered a little experimental for now.
1287The side effecting evaluates all of its arguments as in a {{(begin
1288...)}} form, it returns a result that is completely ignored by the
1289system and unavailable to the output analysis code.
1290
1291<macro>(side-effect CLAUSES)</macro>
1292
1293This macro expands into a begin form the clauses in order and when it
1294finishes evaluating them, returns a result that is silently ignored by
1295the testing infrastructure system. Usually this is used in conjunction with
1296{{(set! ...)}} or with complicated situations where a lot of setup
1297work must happen for an expectation to be performed.
1298
1299<macro>(test-continue? BOOLEAN)</macro>
1300
1301This macro will set the test-case evaluation continuation flag to the boolean value.
1302
1303<macro>(test-timing? BOOLEAN)</macro>
1304
1305This macro will set the expect timing flag.
1306
1307<macro>(test:set! VAR EXPR)</macro>
1308
1309Sets the local variable to the result of the expression.
1310
1311==== Miscellaneous API
1312
1313This section contains a few functions whose purpose is to simplify certain
1314kinds of manipulations of result objects.
1315
1316<procedure>(*-result? RESULT-OBJ)</procedure>
1317
1318This function will return {{#t}} of {{RESULT-OBJ}} is any kind of an
1319evaluated result object. Otherwise, it shall return {{#f}}.
1320
1321<procedure>(*-result-ref RESULT-OBJ)</procedure>
1322
1323This function will return the result of any kind of {{RESULT-OBJ}}
1324passed to it if it is indeed a true result object. Otherwise, it
1325shall emit an error to the current port, and return {{#f}}.
1326
1327<procedure>(*-result-timing-values-ref RESULT-OBJ)</procedure>
1328
1329This function will return the timing values of any kind of
1330{{RESULT-OBJ}} passed to it if it is indeed a true result
1331object. Otherwise, it shall emit an error to the current port, and
1332return {{#f}}.
1333
1334The timing values will either be a list of the milliseconds spent in
1335user code and in system code, or {{#f}} when no timing was performed,
1336for the test corresponding to this test-result-object.
1337
1338For example, the list {{(6 1)}} indicates 6 milliseconds
1339spent in user code and 1 millisecond spent in system code.
1340
1341<procedure>(all-testpackage-results-true? RESULT-LIST)</procedure>
1342
1343This function takes the result of a call to
1344{{(test-package-result-result-ref PACKAGE-RESULT-OBJ)}} and returns
1345{{#t}} if every single contained result in that list had been true, or
1346{{#f}} otherwise.
1347
1348<procedure>(all-testcase-expectations-true? RESULT-LIST)</procedure>
1349
1350This function takes the result of a call to
1351{{(test-case-result-result-ref CASE-RESULT-OBJ)}} and returns {{#t}}
1352if every single contained result in that list had been true, or {{#f}}
1353otherwise.
1354
1355==== Analysis of the Result Tree
1356
1357Once a result tree has been evaluated and constructed in memory, what
1358do you do with it? Well, you can do anything you want with it if you
1359choose to write the analysis or output generation functions and pass it
1360the evaluated result tree, and this is, in fact, encouraged. However,
1361usually you just want to print out the tree in a human readable format so
1362you can check things before the serious analysis code gets written. So,
1363to save work for the test suite designer, a tiny API has been written
1364to produce human readable output given a tree of results rooted in a
1365single test package. Of course the single test package may have many
1366other test packages and test cases embedded within it.
1367
1368===== Statistics Generation API
1369
1370<procedure>(stat-compute-statistics RESULT-TREE)</procedure>
1371
1372This function will compute and return a vector of result statistics. Use the
1373procedures below to access the elements by name.
1374
1375<procedure>(stat-packages-ref STATISTICS)</procedure>
1376<procedure>(stat-package-warnings-ref STATISTICS)</procedure>
1377<procedure>(stat-packages-passed-ref STATISTICS)</procedure>
1378<procedure>(stat-packages-failed-ref STATISTICS)</procedure>
1379<procedure>(stat-packages-timing-ref STATISTICS)</procedure>
1380<procedure>(stat-packages-terminated-ref STATISTICS)</procedure>
1381<procedure>(stat-cases-ref STATISTICS)</procedure>
1382<procedure>(stat-case-warnings-ref STATISTICS)</procedure>
1383<procedure>(stat-cases-passed-ref STATISTICS)</procedure>
1384<procedure>(stat-cases-failed-ref STATISTICS)</procedure>
1385<procedure>(stat-cases-timing-ref STATISTICS)</procedure>
1386<procedure>(stat-cases-terminated-ref STATISTICS)</procedure>
1387<procedure>(stat-all-expectations-ref STATISTICS)</procedure>
1388<procedure>(stat-all-expectation-warnings-ref STATISTICS)</procedure>
1389<procedure>(stat-all-expectations-passed-ref STATISTICS)</procedure>
1390<procedure>(stat-all-expectations-failed-ref STATISTICS)</procedure>
1391<procedure>(stat-all-expectations-timing-ref STATISTICS)</procedure>
1392<procedure>(stat-single-expectations-ref STATISTICS)</procedure>
1393<procedure>(stat-single-expectation-warnings-ref STATISTICS)</procedure>
1394<procedure>(stat-single-expectations-passed-ref STATISTICS)</procedure>
1395<procedure>(stat-single-expectations-failed-ref STATISTICS)</procedure>
1396<procedure>(stat-single-expectations-timing-ref STATISTICS)</procedure>
1397<procedure>(stat-tol-expectations-ref STATISTICS)</procedure>
1398<procedure>(stat-tol-expectation-warnings-ref STATISTICS)</procedure>
1399<procedure>(stat-tol-expectations-passed-ref STATISTICS)</procedure>
1400<procedure>(stat-tol-expectations-failed-ref STATISTICS)</procedure>
1401<procedure>(stat-tol-expectations-timing-ref STATISTICS)</procedure>
1402<procedure>(stat-equiv-expectations-ref STATISTICS)</procedure>
1403<procedure>(stat-equiv-expectation-warnings-ref STATISTICS)</procedure>
1404<procedure>(stat-equiv-expectations-passed-ref STATISTICS)</procedure>
1405<procedure>(stat-equiv-expectations-failed-ref STATISTICS)</procedure>
1406<procedure>(stat-equiv-expectations-timing-ref STATISTICS)</procedure>
1407<procedure>(stat-todos-ref STATISTICS)</procedure>
1408<procedure>(stat-todo-warnings-ref STATISTICS)</procedure>
1409<procedure>(stat-skips-ref STATISTICS)</procedure>
1410<procedure>(stat-skip-warnings-ref STATISTICS)</procedure>
1411<procedure>(stat-glosses-ref STATISTICS)</procedure>
1412<procedure>(stat-gloss-warnings-ref STATISTICS)</procedure>
1413<procedure>(stat-terminations-ref STATISTICS)</procedure>
1414
1415===== Output Generation API
1416
1417These procedures help in the printing of test result reports.
1418{{THUNK}} is a zero argument procedure to print a report.
1419{{INDENT}} is the number of spaces to left indent. Use
1420{{test:write-object}} when printing a data object.
1421
1422<procedure>(test:output-apply THUNK [PORT|STRING|#t])</procedure>
1423<procedure>(test:write-object OBJECT)</procedure>
1424<procedure>(test:display-objects [OBJECT ...])</procedure>
1425<procedure>(test:display-objects-newline [OBJECT ...])</procedure>
1426<procedure>(test:display-indent INDENT)</procedure>
1427<procedure>(test:display-indented-objects INDENT [OBJECT ...])</procedure>
1428<procedure>(test:display-indented-objects-newline INDENT [OBJECT ...])</procedure>
1429
1430===== Output Style API
1431
1432These functions will display a rendering of the {{RESULT-TREE}} and
1433return the toplevel package result. The default destination is the
1434{{(current-output-port)}}. Otherwise the destination specifies how the
1435output is targeted. A {{PORT}} destination will display to the port. A
1436{{STRING}} destination will display to the specified file.  Should the
1437file exist the result is implementation dependent. A {{#t}}
1438destination will return the output as a string.
1439
1440<procedure>(output-style-human RESULT-TREE [PORT|STRING|#t])</procedure>
1441<procedure>(output-style-html RESULT-TREE [PORT|STRING|#t])</procedure>
1442<procedure>(output-style-compact RESULT-TREE [PORT|STRING|#t])</procedure>
1443<procedure>(output-style-minimal RESULT-TREE [PORT|STRING|#t])</procedure>
1444
1445==== Example Usages of the Test Suite Infrastructure
1446
1447This section contains some simple examples of how to author test suites.
1448
1449Here is a simple example:
1450
1451<example>
1452(let ((result
1453  ;; output-style-human function requires a single test package
1454  ;; to encapsulate everything.
1455  (test-package "Arithmetic Operators" pd pe
1456
1457    (test-case "Testing '+'" d e
1458      (expect-equal "Adding two positive numbers" 2 (+ 1 1))
1459      (expect-equal "Adding two negative numbers" -2 (+ -1 -1))
1460      (expect-zero "Adding positive and negative" (+ -1 1)))
1461
1462    (test-case "Testing '-'" d e
1463      (expect-zero "Subtracting two positive numbers" (- 1 1))
1464      (expect-zero "Subtracting two negative numbers" (- -1 -1))
1465      (expect-equal "Subtracting positive and negative" -2 (- -1 1))))))
1466
1467  (output-style-human result))
1468</example>
1469
1470The above example, when evaluated, will produce some human readable
1471output and the a {{#t}} value as the result of the top level
1472package. The {{result}} variable contains the tree of evaluated
1473expectations, test cases, and the package arranged in a hierarchy
1474extremely similar to the nesting of the above macros. If you desire to
1475manipulate the result tree yourself, you may use the various APIs to
1476manipulate the various results. Please see the implementation of
1477{{(output-style-human)}} in the test-infrastructure source to for an
1478example of this.
1479
1480The variables: {{pe}}, {{e}} are the escape functions you may use with the
1481{{(terminate)}} procedure if you wish to abort the above code
1482somewhere.
1483
1484The variables: {{pd}}, {{d}} allow use of a {{"destructor"}} object
1485which allows you to run side effecting functions at the
1486{{"finishing"}} point of the evaluation of the test case or test
1487package. These functions are run no matter if the code succeeded
1488correctly or not, so be careful to manage the destructor object
1489carefully so you don't perform unwanted side effects. The names of the
1490destructor objects you supply are lexically scoped in the bodies of
1491the test case or test package.
1492
1493Now, here is some example output that {{(output-style-human)}} might
1494generate with the above testing code:
1495
1496  Test Package: Arithmetic Operators
1497 
1498    Test Case: Testing '+'
1499 
1500      Expectation: Adding two positive numbers
1501      Expect equal
1502           Expected: 2
1503        Unevaluated: (+ 1 1)
1504          Evaluated: 2
1505      Pass: Adding two positive numbers
1506 
1507      Expectation: Adding two negative numbers
1508      Expect equal
1509           Expected: -2
1510        Unevaluated: (+ -1 -1)
1511          Evaluated: -2
1512      Pass: Adding two negative numbers
1513 
1514      Expectation: Adding positive and negative
1515      Expect zero
1516        Unevaluated: (+ -1 1)
1517          Evaluated: 0
1518      Pass: Adding positive and negative
1519 
1520    Pass: Testing '+'
1521 
1522    Test Case: Testing '-'
1523 
1524      Expectation: Subtracting two positive numbers
1525      Expect zero
1526        Unevaluated: (- 1 1)
1527          Evaluated: 0
1528      Pass: Subtracting two positive numbers
1529 
1530      Expectation: Subtracting two negative numbers
1531      Expect zero
1532        Unevaluated: (- -1 -1)
1533          Evaluated: 0
1534      Pass: Subtracting two negative numbers
1535 
1536      Expectation: Subtracting positive and negative
1537      Expect equal
1538           Expected: -2
1539        Unevaluated: (- -1 1)
1540          Evaluated: -2
1541      Pass: Subtracting positive and negative
1542 
1543    Pass: Testing '-'
1544 
1545  Pass: Arithmetic Operators
1546 
1547  #t
1548
1549=== Contributions
1550
1551{{output-text-compact}} by [[Patrick Brannan]].
1552
1553=== Changelog
1554
1555* 1.92 'expect-*' timing, see 'test-timing?' macro [Kon Lovett]
1556* 1.91 Helper macros, some testeez-like forms [Kon Lovett]
1557* 1.9 Exports [Kon Lovett]
1558* 1.8 Added test-named-structure [Kon Lovett]
1559* 1.7 Added expect-equiv, expect-ec, test-group, test-apply [Kon Lovett]
1560* 1.6 MESSAGE argument for expectations optional [Kon Lovett]
1561* 1.5 Rename of expect-fail, fix of condition-property bug, added test-it [Kon Lovett]
1562* 1.4 Addition of expect-fail, expect-success [Kon Lovett]
1563* 1.3 Slimming of output, addition of expect, test-continue? [Kon Lovett]
1564* 1.2 Wrapping of expectations in an exception catcher [Kon Lovett]
1565* 1.1 Addition of exception, multi-valued, not-false expectations [Kon Lovett]
1566* 1.0 Initial release
1567
1568=== License
1569
1570  Copyright (c) 2000-2005, Peter Keller - All rights reserved.
1571 
1572  Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following
1573  conditions are met:
1574 
1575    Redistributions of source code must retain the above copyright notice, this list of conditions and the following
1576      disclaimer.
1577    Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
1578      disclaimer in the documentation and/or other materials provided with the distribution.
1579    Neither the name of the author nor the names of its contributors may be used to endorse or promote
1580      products derived from this software without specific prior written permission.
1581 
1582  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
1583  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
1584  AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR
1585  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
1586  CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
1587  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
1588  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
1589  OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
1590  POSSIBILITY OF SUCH DAMAGE.
1591 
Note: See TracBrowser for help on using the repository browser.