CHICKEN Scheme

Summary

Used heap reported by memory-statistics is too big after heap resize

Metadata

Id: 76450c2d2d6ed01cc459c81eda6c4ddf56db0cdb
Trac id: 1540
Type: task
Reporter: megane
Owner: megane
Cc:
Status: closed
Component: core libraries
Estimated difficulty: medium
Resolution: fixed
Priority: minor
Milestone: 5.3
Version: 5.0.0
Changetime: 2019-12-13 12:07:26 UTC
Created: 2018-09-29 17:35:55 UTC
Keywords: gc heap memory

Description

The reported used heap just after heap resize seems to be `total_allocated + last_heap`.

I would expect the value to be `2 * total_allocated`. In the code below this means `(/ total-allocated used)` should be close to `0.5`.

The actual used memory as repoted by `top` indicate that the problem is just with the reported used value; there's no phantom objects on the heap.

Here's code that I used to test. The report is from C4, but the results are similar with C5.

 (cond-expand
  (chicken-5 (import (chicken base) (chicken gc) (chicken blob)))
  (else (begin)))
 
 (define stop-megs 400)
 (define stop-before-resize? #t) ; #f -> stop just after resize
 
 (define (heap-ref stats) (vector-ref stats 0))
 (define (used-ref stats) (vector-ref stats 1))
 
 (let ([l '()]
       [total-alloc 0]
       [stats (memory-statistics)])
   (define last-heap (heap-ref stats))
 
   (let lp ()
     (let* ([cur (memory-statistics)]
            [bytes (max 500
                        (inexact->exact
                         (floor
                          (* .5 (- (heap-ref cur) (used-ref cur))))))]
            [cur-used (used-ref cur)]
            [cur-heap (heap-ref cur)])
       (unless (or (and (< (* stop-megs 1000 1000) cur-used)
                        (and stop-before-resize? (< cur-heap (+ cur-used bytes))))
                   (< (* 2 stop-megs 1000 1000) cur-used))
 
         ;; alloc
         (set! l (cons (make-blob bytes) l))
         (set! total-alloc (+ bytes total-alloc))
 
         (let* ([stats-after (memory-statistics)])
           (print)
           (print "heap - used = "(- (heap-ref cur) (used-ref cur)))
           (print "ALLOC " bytes " expected used: " (+ (used-ref stats-after) bytes)
                  " diff: " (- (used-ref stats-after) (+ (used-ref cur) bytes)))
           (print "used / heap: " (used-ref stats-after) " / " (heap-ref stats-after)
                  " total-alloc: " total-alloc)
           (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref stats-after))))
           (unless (= last-heap (heap-ref stats-after))
             (set! last-heap (heap-ref stats-after))
             (print "#################### Heap resized ####################"))
           (if (and (not stop-before-resize?)
                    (not (= (heap-ref cur) (heap-ref stats-after)))
                    (< (* stop-megs 1000 1000) cur-used))
               'stop
               (lp))))))
 
   (print "\nSTOP")
   (let ([cur (memory-statistics)])
     (print "used / heap: " (used-ref cur) " / " (heap-ref cur)
            " total-alloc: " total-alloc)
     (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref cur))) ))
   (read)
   (print (length l)))
 
 
 ;;; (define stop-before-resize? #f):
 
 ;; heap - used = 696
 ;; ALLOC 500 expected used: 402653572 diff: 84
 ;; used / heap: 402653072 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.498995929677149
 ;;
 ;; heap - used = 160
 ;; ALLOC 500 expected used: 603980700 diff: 201326676
 ;; used / heap: 603980200 / 805306368 total-alloc: 200922744
 ;; (/ total-alloc used) = 0.332664454894382
 ;; #################### Heap resized ####################
 ;;
 ;; STOP
 ;; used / heap: 603980008 / 805306368 total-alloc: 200922744
 ;; (/ total-alloc used) = 0.332664560645524
 ;; 133
 
 ;; top shows:
 ;; 804724 400360   5716 S   0.0  1.2   0:02.29 gctest
 
 
 ;;; (define stop-before-resize? #t):
 
 ;; ...
 ;; heap - used = 696
 ;; ALLOC 500 expected used: 402653572 diff: 84
 ;; used / heap: 402653072 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.498995929677149
 ;;
 ;; STOP
 ;; used / heap: 402652880 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.49899616761713
 ;; 132
 
 ;; top shows:
 ;; 411504 400212   5548 S   0.0  1.2   0:02.19 gctest