CHICKEN Scheme

Summary

chicken.base#implicit-exit-handler reporting segmentation violation with non-default heap size

Metadata

Id: 1edda6351d2cf19a3546721b6cd97c36f9975b98
Trac id: 1726
Type: defect
Reporter: vjorlikowski
Owner:
Cc:
Status: new
Component: core libraries
Estimated difficulty:
Resolution:
Priority: minor
Milestone: someday
Version: 5.1.0
Changetime: 2023-11-14 18:48:38 UTC
Created: 2021-01-15 04:19:31 UTC
Keywords:

Attachments

1edda6351d2cf19a3546721b6cd97c36f9975b98/attachments/test.scm

Description

I was experimenting with the "Man-or-boy" test (http://rosettacode.org/wiki/Man_or_boy_test) using Chicken Scheme; my variant looks like this:

 vjo@glaucus/pts/11:chicken/man_or_boy % cat test.scm
 (import (chicken process-context)
         (chicken fixnum))
 
 (define (A k x1 x2 x3 x4 x5)
   (let
       ([+ fx+]
        [- fx-]
        [<= fx<=])
     (define (B)
       (set! k (- k 1))
       (A k B x1 x2 x3 x4))
     (if (<= k 0)
         (+ (x4) (x5))
         (B))))
 
 (define k (let ([args (command-line-arguments)])
             (if (> (length args) 0)
                 (string->number (car args))
                 10)))
 (print (A k
           (lambda () 1)
           (lambda () -1)
           (lambda () -1)
           (lambda () 1)
           (lambda () 0)))

This compiles fine, and runs great with the default heap size up to a value of k=27:

 vjo@glaucus/pts/11:chicken/man_or_boy % csc -O4 test.scm
 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 26
 -21051458
 ./test 26  25.20s user 1.31s system 99% cpu 26.577 total
 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 27
 -46750171
 ./test 27  101.91s user 4.42s system 99% cpu 1:46.66 total

At k=28, however, we run out of heap, with the default settings:

 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 28
 [panic] out of memory - heap has reached its maximum size - execution terminated
 
 ...more...
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:13: x4
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:13: x4
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A  <--
 ./test 28  22.39s user 3.79s system 99% cpu 26.282 total

No problem, say I; let me re-compile with an increased heap. That works well - and the code gives back the correct answer - but, I now encounter the bug I wish to report:

 vjo@glaucus/pts/11:chicken/man_or_boy % csc -O4 -heap-size 4096M test.scm
 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 28
 -103821058
 
 Error: segmentation violation
 
         Call history:
 
         test.scm:13: x5
         test.scm:11: A
         test.scm:13: x4
         test.scm:13: x5
         test.scm:13: x5
         test.scm:11: A
         test.scm:13: x4
         test.scm:11: A
         test.scm:13: x4
         test.scm:13: x5
         test.scm:13: x5
         test.scm:11: A
         test.scm:13: x4
         test.scm:13: x5
         test.scm:20: chicken.base#print
         chicken.base#implicit-exit-handler              <--
 ./test 28  90.91s user 1.71s system 99% cpu 1:32.65 total

It *appears* that the implicit-exit-handler triggers a segmentation violation for any binary compiled with a non-default heap size specified; as an illustrative example, here is the same code compiled with two different heap sizes and executed using k=26. The first shows what happens with an insufficient heap size, the second shows what happens when the heap size is *just barely* sufficient, but non-default value:

 vjo@glaucus/pts/11:chicken/man_or_boy % csc -O4 -heap-size 928M test.scm
 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 26
 [panic] out of memory - heap full - execution terminated
 
 ...more...
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:13: x4
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:13: x4
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A
 test.scm:14: B
 test.scm:11: A  <--
 ./test 26  5.01s user 0.44s system 99% cpu 5.471 total
 
 vjo@glaucus/pts/11:chicken/man_or_boy % csc -O4 -heap-size 929M test.scm
 vjo@glaucus/pts/11:chicken/man_or_boy % time ./test 26
 -21051458
 
 Error: segmentation violation
 
         Call history:
 
         test.scm:13: x5
         test.scm:13: x5
         test.scm:11: A
         test.scm:13: x4
         test.scm:11: A
         test.scm:13: x4
         test.scm:11: A
         test.scm:13: x4
         test.scm:13: x5
         test.scm:13: x5
         test.scm:13: x5
         test.scm:11: A
         test.scm:13: x4
         test.scm:13: x5
         test.scm:20: chicken.base#print
         chicken.base#implicit-exit-handler              <--
 ./test 26  25.87s user 0.73s system 99% cpu 26.620 total

The versions of all software involved have been determined below:

 vjo@glaucus/pts/11:chicken/man_or_boy % csc -O4 -kv test.scm          
 '/usr/bin/chicken' 'test.scm' -output-file 'test.c' -optimize-level 4 -verbose
 'x86_64-linux-gnu-gcc' 'test.c' -o 'test.o' -c  -fno-strict-aliasing -fwrapv
 -DHAVE_CHICKEN_CONFIG_H -DC_ENABLE_PTABLES -g -O2
 -fdebug-prefix-map=/build/chicken-8b5V5S/chicken-5.1.0=. -fstack-protector-strong -Wformat
 -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include/chicken
 'x86_64-linux-gnu-gcc' 'test.o' -o 'test' -L/usr/lib -Wl,-R/usr/lib -lchicken -lm -ldl
 
 vjo@glaucus/pts/11:chicken/man_or_boy % x86_64-linux-gnu-gcc --version
 x86_64-linux-gnu-gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
 Copyright (C) 2019 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
 vjo@glaucus/pts/11:chicken/man_or_boy % csc -version                  
 CHICKEN
 (c) 2008-2019, The CHICKEN Team
 (c) 2000-2007, Felix L. Winkelmann
 Version 5.1.0 (rev 8e62f718)
 linux-unix-gnu-x86-64 [ 64bit dload ptables ]

Changes and comments

[2021-01-15 04:20:16 UTC] vjorlikowski attached test.scm (description=Man-or-boy test file source)

[2021-01-15 08:51:59 UTC] sjamaan changed milestone from someday to 5.3

[2021-01-15 10:25:59 UTC] sjamaan wrote:

NOTE: This only happens when compiling with `-maximum-heap-size`. When passing `-:hm` at runtime, I could not reproduce this issue.

[2021-01-15 10:52:06 UTC] sjamaan wrote:

This looks like some sort of corruption in the GC, as the pointer to the continuation of `cleanup-before-exit` seems to be broken.

[2021-01-15 11:54:58 UTC] sjamaan wrote:

Possibly there's a bug in the `reintern` option of `C_set_or_change_heap_size`, as that differs between `-:h` and `-heap-size`

[2021-01-16 13:03:40 UTC] megane wrote:

Just compiling an empty program with -heap-size gives the same result.

[2021-01-17 19:43:00 UTC] sjamaan wrote:

It looks like when `C_set_or_change_heap_size` is invoked with `reinit=1`, it only recreates the symbol table, but the termination continuation is still allocated on the old heap which is now in freed memory. This was allocated by `CHICKEN_initialize`.

I tried to copy that bit into the `if(reintern) ...` block of `C_set_or_change_heap_size` so that the termination continuation is pointing into the correct heap, but that didn't seem to fix the situation. I also tried resetting the temporary stack before doing this, but that didn't help either. I guess by the time the toplevel is called, the continuation pointer that's passed to it is still pointing into the old heap.

I think we need to longjump back and re-invoke the toplevel. But then it needs to know that it shouldn't try to resize the heap *again*. Or somehow we must do something with the termination continuation. Maybe not allocate it on the heap?

Possibly instead treat the heap reallocation as a reallocating GC?