Custom Query (1630 matches)

The major garbage collector runs too frequently if you happen to get unfortunate "correct" combination of

• heap size,
• total size of "permanent objects", and
• you're generating lots of garbage objects.

(By permanent objects I mean the set of objects that survives a major garbage collection.)

In this situation you can effectively get a major GC for every N allocated bytes that survive the 1st generation.

This N is basically just heap_size - permanent_objects, i.e. the free heap space just after a major collection.

In practice, the N may get quite small. In this case the program is mostly just doing garbage collection.

One incarnation of this is a program that gradually gets slower and slower, and then suddenly gets fast again. In this case the sum of permanent objects gets bigger and bigger until the objects no longer fit in the heap. At this point the heap is grown and everything gets smooth again.

One possible workaround is to manually specify a big heap with -:h or -:hi. This has the side-effect of

1. Making finalizers run less often (they are checked during each major collection)
2. Causing memory fragmentation. (Objects are compacted during the major GC.)
3. You have to choose a size. With -:hi you can still run into the problem if you choose too small size.

One example of affected program: ​https://stackoverflow.com/questions/62826083/chicken-scheme-read-line-taking-too-long/63215403

(define *corpus*
  (call-with-input-file "largeish_file.txt"
    (lambda (input-file)
      (let loop ([line (read-line input-file)]
                 [tokens '()])
        (if (eof-object? line)
            tokens
            (loop (read-line input-file)
              (append tokens (string-split line))))))))

This program generates lots of garbage by creating lots of temporary lists. Using big heap (using -:h) makes things a lot faster.

I've posted a patch here: ​https://lists.nongnu.org/archive/html/chicken-hackers/2020-01/msg00002.html This works by growing the heap if it's too full after a major collection.

See ​http://salmonella-linux-arm64.call-cc.org/master/clang/linux/arm64/2020/07/28/salmonella-report/test/salt.html

The installation of genann fails with CHICKEN 5.2.0 and more recent versions (see ​http://salmonella-linux-arm64.call-cc.org/master/clang/linux/arm64/2020/07/28/salmonella-report/install/genann.html). This issue is probably related to ​http://bugs.call-cc.org/ticket/1655.

It seems that the patch by Peter has already been applied to the repository (​https://github.com/dieggsy/genann/commit/750a1cdfd93ee63dc5b91ab0666e3c852844456e). Maybe you just have to tag a release?