posix: can't collect exit status and close all process ports at the same time

[Moritz Heidkamp]

It is currently impossible to call process-wait on a PID to collect the exit status of child process and close all associated input and output ports afterwards. This is due to close-[input|output]-port implicitly calling process-wait on process pipe ports. This leads to errors like

Error: (process-wait) waiting for child process failed - No child processes: 10872

because the process has already been purged by the OS at that point. This could be worked around by calling file-close on the ports' file descriptors. However, port->fileno does not work on process ports.

A possible solution might be to ​encapsulate processes in process objects similar to Scsh (or ​subprocess values as Racket calls them) and change process-wait to retain the process' exit status once it has been collected, allowing for it to be called multiple times. This would of course break backwards compatibility and thus should be handled as a Change Request. See also: ​Common Lisp external-program library. Comments welcome.

[Moritz Heidkamp]

Here's a naive drop-in replacement of process* which works around the issue:

(define (process* #!optional cmd args env)
  (let*-values
      (((in-in   in-out) (create-pipe))
       ((out-in out-out) (create-pipe))
       ((err-in err-out) (create-pipe))
       ((pid) (process-fork
               (lambda ()
                 (duplicate-fileno in-in fileno/stdin)
                 (duplicate-fileno out-out fileno/stdout)
                 (duplicate-fileno err-out fileno/stderr)
                 (file-close in-out)
                 (file-close in-in)
                 (file-close out-in)
                 (file-close out-out)
                 (file-close err-in)
                 (file-close err-out)
                 (process-execute cmd args env)))))
    (file-close in-in)
    (file-close out-out)
    (file-close err-out)
    (values (open-input-file*  out-in)
            (open-output-file* in-out)
            pid
            (open-input-file*  err-in))))

[Moritz Heidkamp]

The #!optional marker has to be moved one position to the right, of course.

[sjamaan]

Wrapping them is the way I deal with it in scsh-process. I think it could be a good idea to put something like this in core. There are various annoying issues with the Unix model in a Scheme program. For example, the (process-wait (process-fork (lambda () ...))) idiom will fail when this is done in a library and the user decides to (process-wait) in another thread. If you're unlucky, a context switch could happen in between, and the process-wait call in the library would fail, while the user would be getting the status code of an unexpected process.

Doing it "the SCSH way" makes code more composable. Finally, I think fork should only copy the current thread and abandon all others. This may be trickier than we think, though!

[sjamaan]

We should probably focus on this for 5.0 or maybe even later

[sjamaan]

This code is incredibly hairy in scsh-process (had to remove annoying bugs several times which were triggered by various race conditions), so I'm not convinced this should go into core.