[panic] out of memory - heap full while resizing - execution terminated (awful-picman)

[Mario Domenech Goulart]

I frequently get out [panic] out of memory - heap full while resizing - execution terminated errors when running ​awful-picman (a pictures manager).

I don't have a simple test case, and I can't deterministically reproduce the problem, but it' not very difficult to trigger it.

Here are the steps to install awful-picman, run it and set up a test case:

git clone https://github.com/mario-goulart/awful-picman.git
cd awful-picman
git checkout 45a0a1e5f7245b0ea17b163746362aca87e77d95
chicken-install
mkdir -p test/pics
cd test/pics
wget http://parenteses.org/mario/misc/DSC00065.jpg
for i in `seq 100`; do cp DSC00065.jpg $i.jpg; done
cd ..
awful-picman --init

(--init is only necessary for the first run.)

Then use you browser to request ​http://localhost:8080/folders/pics . Wait until the thumbnails generation is finished then keep reloading the page which shows the pictures. After a couple of requests awful-picman will crash with [panic] out of memory - heap full while resizing - execution terminated.

Sometimes it crashes when the heap is growing, sometimes when the heap is shrinking. The call trace is usually different among crashes.

I can reproduce this problem with CHICKENs 4.7.0, 4.8.0, 4.8.0.3 and the 4.8.2 dev-snapshot tarball (Linux x86-64).

It crashes more frequently with 4.7.0 (usually on the first request).

[sjamaan]

I think it's very important we fix this before 4.9.0 is out

[Alaric Snell-Pym]

I have a reproduction case for this using the Ugarit unit-test suite, too, if it helps. Basically, Ugarit runs a child process to manage actual storage operations, and talks to it via s-expressions over standard input and output. The test suites run fine, except for if I run the same sequence of operations as other tests use, but using an SHA hash rather than a Tiger one; this is a difference in the parent process, observable to the child process purely as the hash strings it is sent being different (with a different length, I note), but in that case, the child gives the same panic error message; but only after it's been sent the command to terminate itself cleanly. More details available upon request!

Ugarit also suffers a problem (in the parent process this time) where some srfi-4 function complains that what it's being passed isn't a u8vector, but printing it out exactly before the srfi-4 call shows that it is, which also suggests that memory is being corrupted somehow. Maybe related. C-Keen is able to reproduce this, but I've not seen it myself.

However, I do feel it is quite likely that Chicken has at least one underlying subtle memory corruption problem here!

Previous mailing-list discussions on this and similar mysteries that look like memory corruption (I don't have the message ID, sorry) resulted in the idea of Felix adding optional logging hooks to the lowest-level runtime operations relating to garbage collection, if somebody else would write tools to process the logs, checking various complex invariants to see if an inconsistency can be found.