Used heap reported by memory-statistics is too big after heap resize

[megane]

The reported used heap just after heap resize seems to be total_allocated + last_heap.

I would expect the value to be 2 * total_allocated. In the code below this means (/ total-allocated used) should be close to 0.5.

The actual used memory as repoted by top indicate that the problem is just with the reported used value; there's no phantom objects on the heap.

Here's code that I used to test. The report is from C4, but the results are similar with C5.

(cond-expand
 (chicken-5 (import (chicken base) (chicken gc) (chicken blob)))
 (else (begin)))

(define stop-megs 400)
(define stop-before-resize? #t) ; #f -> stop just after resize

(define (heap-ref stats) (vector-ref stats 0))
(define (used-ref stats) (vector-ref stats 1))

(let ([l '()]
      [total-alloc 0]
      [stats (memory-statistics)])
  (define last-heap (heap-ref stats))

  (let lp ()
    (let* ([cur (memory-statistics)]
           [bytes (max 500
                       (inexact->exact
                        (floor
                         (* .5 (- (heap-ref cur) (used-ref cur))))))]
           [cur-used (used-ref cur)]
           [cur-heap (heap-ref cur)])
      (unless (or (and (< (* stop-megs 1000 1000) cur-used)
                       (and stop-before-resize? (< cur-heap (+ cur-used bytes))))
                  (< (* 2 stop-megs 1000 1000) cur-used))

        ;; alloc
        (set! l (cons (make-blob bytes) l))
        (set! total-alloc (+ bytes total-alloc))

        (let* ([stats-after (memory-statistics)])
          (print)
          (print "heap - used = "(- (heap-ref cur) (used-ref cur)))
          (print "ALLOC " bytes " expected used: " (+ (used-ref stats-after) bytes)
                 " diff: " (- (used-ref stats-after) (+ (used-ref cur) bytes)))
          (print "used / heap: " (used-ref stats-after) " / " (heap-ref stats-after)
                 " total-alloc: " total-alloc)
          (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref stats-after))))
          (unless (= last-heap (heap-ref stats-after))
            (set! last-heap (heap-ref stats-after))
            (print "#################### Heap resized ####################"))
          (if (and (not stop-before-resize?)
                   (not (= (heap-ref cur) (heap-ref stats-after)))
                   (< (* stop-megs 1000 1000) cur-used))
              'stop
              (lp))))))

  (print "\nSTOP")
  (let ([cur (memory-statistics)])
    (print "used / heap: " (used-ref cur) " / " (heap-ref cur)
           " total-alloc: " total-alloc)
    (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref cur))) ))
  (read)
  (print (length l)))


;;; (define stop-before-resize? #f):

;; heap - used = 696
;; ALLOC 500 expected used: 402653572 diff: 84
;; used / heap: 402653072 / 402653184 total-alloc: 200922244
;; (/ total-alloc used) = 0.498995929677149
;;
;; heap - used = 160
;; ALLOC 500 expected used: 603980700 diff: 201326676
;; used / heap: 603980200 / 805306368 total-alloc: 200922744
;; (/ total-alloc used) = 0.332664454894382
;; #################### Heap resized ####################
;;
;; STOP
;; used / heap: 603980008 / 805306368 total-alloc: 200922744
;; (/ total-alloc used) = 0.332664560645524
;; 133

;; top shows:
;; 804724 400360   5716 S   0.0  1.2   0:02.29 gctest


;;; (define stop-before-resize? #t):

;; ...
;; heap - used = 696
;; ALLOC 500 expected used: 402653572 diff: 84
;; used / heap: 402653072 / 402653184 total-alloc: 200922244
;; (/ total-alloc used) = 0.498995929677149
;;
;; STOP
;; used / heap: 402652880 / 402653184 total-alloc: 200922244
;; (/ total-alloc used) = 0.49899616761713
;; 132

;; top shows:
;; 411504 400212   5548 S   0.0  1.2   0:02.19 gctest

[sjamaan]

Getting ready for 5.1, moving tickets which won't make it in to 5.2.

[megane]

This might have been caused by the faulty heap size calculation in memory-statistics. I'll check if it was.

[megane]

Seems to work now. memory-statistics returns semi-space sizes correctly.
Most likely fixed by 126a315f1ffba0bd33709c05132063affddee266.