Opened 8 months ago

Closed 7 months ago

Last modified 7 months ago

#1457 closed defect (fixed)

module/import information in statically linked executables has massive runtime overhead

Reported by: felix Owned by:
Priority: major Milestone: 5.0
Component: extensions Version: 5.0.0
Keywords: Cc: sjamaan, evhan, kristianlm
Estimated difficulty: hard


While experimenting with statically linked eggs, klm noticed that a simple empty program has a considerable startup time (several seconds). So, for example, the following program:

(import parley)
(print 'hello)

compiled like this:

csc -static hello.scm

then ./hello -:d will show the initialization sequence, where particular units need quite long to initialize. All of the overhead is very likely to be caused by module registration code and import resolution.

If I instrument chicken-install (actually egg-compile.scm) to pass -no-module-registration, then the overhead is gone. But this will prevent the use of import in evaluated code for all but the core modules (those provided by the eval-modules unit).

I'm not sure what to do now. I personally think it's acceptable to require for statically linked executables not to use eval, but that does of course not apply to all users. An alternative would be to trigger the module registration lazily, on the first use of import, but that may be difficult to change in the compiler or may result in registration ordering issues.

Another option would be a build-time option that decides whether static extensions should be compiled with -no-module-registration.

Change History (5)

comment:1 Changed 8 months ago by felix

  • Version changed from 4.13.0 to 5.0

comment:2 Changed 8 months ago by sjamaan

This is probably merge-se's quadratic complexity biting us in the ass again. We really need to fix this.

comment:3 Changed 7 months ago by sjamaan

Yeah, it's due to merge-se.

It is triggered by import libraries being present in static binaries. In binaries which are linked with shared libraries you pay most of the cost at compile time, which you don't notice as easily because it is (usually!) dwarfed by the compilation time itself.

Each import library will contain forms like (eval '(import scheme chicken.base ...)) to ensure the syntactic environment of any macros exported by the module match the environment inside the module (which is conceptually somewhat broken in itself, but that's something for another day).

When a module exports no syntactic forms, this is 100% wasteful, so we can avoid emitting eval for all modules that export only regular values. Patch sent to chicken-hackers.

comment:4 Changed 7 months ago by sjamaan

  • Milestone changed from someday to 5.1
  • Resolution set to fixed
  • Status changed from new to closed

The worst part is fixed by 36be0fc. We should still look into getting rid of the eval altogether, but I made a new ticket for that: #1466.

comment:5 Changed 7 months ago by sjamaan

  • Milestone changed from 5.1 to 5.0
Note: See TracTickets for help on using tickets.