Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#874 closed defect (fixed)

Compiling some files may cause infinite optimization loops, eating all available memory

Reported by: Mario Domenech Goulart Owned by: felix winkelmann
Priority: major Milestone: 4.9.0
Component: compiler Version: 4.7.x
Keywords: Cc:
Estimated difficulty:

Description

chicken (master -- e5f341a81cb2d5b8c4f7d8eedabcd35c1ecb7281) dies with a segmentation fault when attempting to compile the attached file (srfi-14-test.scm).

While compiling, it uses more than 2GB of RAM. It crashed after running for more than 9 minutes on a 2.4GHz processor. The system is Linux x86.

I tried building chicken with DEBUGBUILD=1 and running it under gdb, but it didn't crash in more than 10 hours, when I stopped it (it was still using 2GB of RAM).

Attachments (1)

srfi-14-test.scm (10.1 KB) - added by Mario Domenech Goulart 12 years ago.
srfi-14-test.scm

Download all attachments as: .zip

Change History (12)

Changed 12 years ago by Mario Domenech Goulart

Attachment: srfi-14-test.scm added

srfi-14-test.scm

comment:1 Changed 12 years ago by sjamaan

Milestone: 4.8.0
Summary: High memory usage for compilingCompiling some files may cause infinite optimization loops, eating all available memory

csc -debug o srfi-14.scm shows that it gets stuck contracting the same procedures over and over.

This is potentially a severe bug affecting other cases too so I'm promoting this to milestone 4.8.0.

comment:2 Changed 12 years ago by sjamaan

It's not an infinite loop, but it keeps contracting the same procedure more than once. If you rip out all the tests under "shiver's tests" the problem goes away. Put some of them back, it takes longer. Put more back, it takes even longer etc.

It's not the macro either. Looks like the cond is causing the problem. Looks like an O(n2) behavior with n being the number of branches in the cond.

Here's a simplified case that clearly shows what's going on:

(cond ((not (= 1 2)) (print 2))
      ((not (= 5 6)) (print 4))
      (else (print 'nothing)))
csc -debug o test.scm
Removed `not' forms: 2 
contracted procedure: k31 
folded constant expression: (= (quote 5) (quote 6)) 
contracted procedure: k22 
contracted procedure: k31 
folded constant expression: (= (quote 5) (quote 6)) 
folded constant expression: (= (quote 1) (quote 2)) 
replaced variables: 21 
contracted procedure: k22 
removed binding forms: 2 
substituted constant variable: r23 
contracted procedure: k31 
removed binding forms: 4 
removed conditional forms: 1 
removed binding forms: 1 

Note the double occurrance of the folded expression, which is probably a result of the fact that the procedure k31 that's generated by cond's expansion is contracted twice. Add a line and you get three entries for the newly added line:

(cond ((not (= 1 2)) (print 2))
      ((not (= 5 6)) (print 4))
      ((not (= 7 8)) (print 5))
      (else (print 'nothing)))
csc -debug o test.scm
Removed `not' forms: 3 
contracted procedure: k40 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k31 
contracted procedure: k40 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
contracted procedure: k22 
contracted procedure: k40 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k31 
contracted procedure: k40 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
folded constant expression: (= (quote 1) (quote 2)) 
replaced variables: 53 
contracted procedure: k22 
removed binding forms: 2 
substituted constant variable: r23 
contracted procedure: k31 
removed binding forms: 4 
removed conditional forms: 1 
removed binding forms: 1 

And another line:

(cond ((not (= 1 2)) (print 2))
      ((not (= 5 6)) (print 4))
      ((not (= 7 8)) (print 5))
      ((not (= 9 10)) (print 6))
      (else (print 'nothing)))
csc -debug o test.scm
Removed `not' forms: 4 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k40 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k31 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k40 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
contracted procedure: k22 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k40 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k31 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k40 
contracted procedure: k49 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
folded constant expression: (= (quote 1) (quote 2)) 
replaced variables: 117 
contracted procedure: k22 
removed binding forms: 2 
substituted constant variable: r23 
contracted procedure: k31 
removed binding forms: 4 
removed conditional forms: 1 
removed binding forms: 1 

Interestingly, it does not happen when you remove the "not".

comment:3 Changed 12 years ago by sjamaan

Further simplification: if you convert it to a nested if the problem still occurs:

(if (not (= 1 2))
    (print 2)
    (if (not (= 5 6))
        (print 4)
        (if (not (= 7 8))
            (print 5)
            (if (not (= 9 10))
                (print 6)
                (print 'nothing)))))
$ csc -debug o test.scm
Removed `not' forms: 4 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k37 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k28 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k37 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
contracted procedure: k19 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k37 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
contracted procedure: k28 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
contracted procedure: k37 
contracted procedure: k46 
folded constant expression: (= (quote 9) (quote 10)) 
folded constant expression: (= (quote 7) (quote 8)) 
folded constant expression: (= (quote 5) (quote 6)) 
folded constant expression: (= (quote 1) (quote 2)) 
replaced variables: 117 
contracted procedure: k19 
removed binding forms: 2 
substituted constant variable: r20 
contracted procedure: k28 
removed binding forms: 4 
removed conditional forms: 1 
removed binding forms: 1 

comment:4 Changed 12 years ago by sjamaan

Could this be due to the fact that perform-pre-optimization! mutates the nodes? Perhaps this creates an inconsistency between the analysis database and the node tree. Felix?

comment:5 Changed 12 years ago by felix winkelmann

Owner: set to felix winkelmann
Status: newassigned

Multiple contractions of the same function is already incorrect ("contraction" means inlining of a function called once). I think the real problem are nested contractions. In the last example above, k46 is nested in k37. The former gets contracted, then the latter (in the same optimization pass). In this case it seems to happens repeatedly. For inlining this case (an inlined procedure being the target of additional inlining) is taken care of by having the inline-target db entry - we probably need this (or something similar) for contraction, too.

comment:6 Changed 12 years ago by sjamaan

Do multiple contractions of the same form even happen for programs without perform-pre-optimization!?

comment:7 Changed 12 years ago by sjamaan

In other words, do you think the bug is in the parts of the optimizer that handle contractions rather than perform-pre-optimization!?

comment:8 in reply to:  7 Changed 12 years ago by felix winkelmann

Replying to sjamaan:

In other words, do you think the bug is in the parts of the optimizer that handle contractions rather than perform-pre-optimization!?

I do, yes. There is a check for the inline-target db entry, which is done for procedures where inlining takes place. I think this must also be done for contractions, but I have to try it out first (real soon).

comment:9 Changed 12 years ago by felix winkelmann

This is actually not related to nested contractions or inlining conflicts. The problem appears to be that copy-propagation inserts references to contracted procedures.

comment:10 Changed 12 years ago by Mario Domenech Goulart

Resolution: fixed
Status: assignedclosed

Fixed by 285f53dbca729cffb4c4d9ee84e4ba893c882546

comment:11 Changed 12 years ago by felix winkelmann

Milestone: 4.8.04.9.0

Milestone 4.8.0 deleted

Note: See TracTickets for help on using tickets.