Opened 6 years ago

Closed 6 years ago

#1103 closed defect (fixed)

pty test script doesn't report failures (and one test fails)

Reported by: Mario Domenech Goulart Owned by: Alex Shinn
Priority: major Milestone: someday
Component: extensions Version: 4.8.x
Keywords: pty, tests Cc: Peter.Bex@…
Estimated difficulty:

Description

See http://salmonella-linux-x86-64.call-cc.org/master-debugbuild/gcc/linux/x86-64/2014/03/06/salmonella-report/test/pty.htmlz (note that salmonella reports tests status as "OK", although at least one test failed).

For salmonella to be able to detect when tests succeed or failed, it uses the run.scm exit code. It seems that pty's run.scm is not properly reporting the exit code (we usually use (test-exit) at the end of tests).

Change History (3)

comment:1 Changed 6 years ago by Alex Shinn

Can I assume anything about the salmonella environment?
The interesting uses of pty (where the program actually
requires a terminal, not stdout) probably won't be available
anyway. I'm testing units(1), which is probably not there
either. Will /bin/echo always exist?

comment:2 Changed 6 years ago by Mario Domenech Goulart

Cc: Peter.Bex@… added

On the Linux salmonella machines, here's what you can assume to be installed: https://github.com/mario-goulart/setup-salmonella-environment/blob/master/debian-wheezy.sh

On the FreeBSD machine, the current list is https://github.com/CodeYellowBV/setup-salmonella-environment/blob/master/freebsd.sh.

If you need something to be installed on the salmonella environment, please let me know and we'll install it.

/bin/echo is available, and I've just installed units on all Linux salmonella machines (I'm adding Peter to the Cc list, as he maintains the FreeBSD salmonella machine).

Unfortunately, test scripts don't have a real terminal, since their output goes to a file (the salmonella log).

You can check your egg on salmonella on your own system, if you want. It's just a matter of running salmonella without arguments on the egg sources directory (just like chicken-install, to install eggs from their sources).

In case you want to skip some tests on salmonella, you can condition the execution of tests against the SALMONELLA_RUNNING environment variable (salmonella sets it when it runs). Anyway, exiting zero when tests succeed or non-zero when any test fails is what salmonella needs to determine if tests were successful or not.

comment:3 Changed 6 years ago by Alex Shinn

Resolution: fixed
Status: newclosed

Well, fortunately the pty doesn't need a terminal, it makes its own ;)
I'll try to think of a good "real" test later. ssh is the classic example
but hard to test. I'd been using pty for a nethack bot but that's really
complicated.

Anyway, added test-exit.

Note: See TracTickets for help on using tickets.