read-line produces inconsistent results on DOS-style line endings when buffer is cut off at an unfortunate point

[sjamaan]

This showed up when trying to use chicken-install, I got this:

% /tmp/chicken-snapshot/bin/chicken-install -test coops
retrieving ...
resolving alias `kitten-technologies' to: http://chicken.kitten-technologies.co.uk/henrietta.cgi
connecting to host "chicken.kitten-technologies.co.uk", port 80 ...
requesting "/henrietta.cgi?name=coops&mode=default&tests=yes" ...
reading response ...
HTTP/1.1 200 OK
Date: Sun, 17 Apr 2011 13:07:39 GMT
Server: Apache
Connection: close
Transfer-Encoding: chunked
Content-Type: text/plain
reading chunks SIZE: 1 line: "1"
...

Error: (zero?) bad argument type: #f

Oddly, it worked when I omitted the -test or tried the call-cc server instead!

It turned out that in setup-download.scm in read-chunks the value of size was #f. After adding some debugging code I found out that read-line returned "0\r" instead of the expected "0", and string->number failed on that.

The actual problem is in an interaction between unit TCP's code for reading lines from a TCP port and ##sys#scan-buffer-line, which it calls. The attached code reproduces this issue 100% reliably for me and triggers what's happening in the kernel on Alaric's machine; it happens to buffer the output just in such a way that my machine reads only a few characters exactly in the middle of the \r and \n line ending markers.

This causes ##sys#scan-buffer-line to be invoked like this:

; read buffer, get two characters in it: "0\r"..
(##sys#scan-buffer-line "0\r" 2 0 ...)
; read buffer, get one character in it: "\n"..
(##sys#scan-buffer-line "\n" 1 0 ...)

At the first call, the one-character lookahead for a \n following the \r fails because it's not present in the buffer. At the next call, it forgot that it encountered the \r in the previous buffer and just returns the empty string. The \r is never discarded.

This can theoretically happen with other types of buffered ports too, but so far I think it only happens with TCP ports.

[sjamaan]

server code

[sjamaan]

Client code

[sjamaan]

The output of the client should be:

I just read: "0"
I just read: "0"

But it is:

I just read: "0"
I just read: "0\r"

[felix winkelmann]

A workaround (thanks to sjamaan) has been committed and merged into "prerelease" and "experimental".

[sjamaan]

Fix

[sjamaan]

Fixed by 002ea4128f8b04c7e6d63b6b7a2bdbcd957b785b (and that was fixed by fa9ccaa030cf7acaceb15378a4d6c33464f0eb1f)