CR: overhaul environment representation in the evaluator

[felix winkelmann]

The current implementation of environments is completely syntax-agnostic and also ignores the module system. It would (IMHO) make sense to use something like SE's (syntactic-environments) instead of the currently uses hash-tables, which only provide a value store.
Actually, I think a value store for environments isn't needed at all, only a mapping of identifiers to the single global environment shared by all other code. "Fresh" environments could be created just like modules, using a different module-prefix.

Additionally, mapping modules to environments (for example by something like "(module-environment MODULENAME)") would be a nice feature.

[felix winkelmann]

A new implementation of environments is available in the new-environments branch. Since this change will break the environments egg and all eggs that depend on it, the merge is on currently on hold.

[felix winkelmann]

As I understand R5RS, evaluation environments do not necessarily imply first-class environments (what the current implementation provides). The work done in new-environments does not use first class environments anymore and maps the visible global value identifiers directly to their actual toplevel variable. This means the structure and semantics of environment records have completely changed and are not compatible with the old way of handling environments. interaction-environment effectively uses the global environment, just like calling eval without any environment argument. The environments returned by scheme-report-environment and null-environment are immutable, assignments to toplevel variables are not allowed and new toplevel definitions are not possible. This is compliant to R5RS and keeps the implementation simple. Use-defined modules can be converted to environments, which gives the user more freedom to provide additional toplevel bindings, but these are still immutable.

If first-class environments (which are problematic in general and will always produce inconsistencies between evaluated and compiled code) are considered to be important, I can try to think of a way to customize the evaluator more extensively - some preparation for this is already being done in the bossa-nova branch.

[Alaric Snell-Pym]

I'd be interested to know more about the implications of this change.

In particular, my favourite drum to beat is sandboxing: what will this mean for eval-ing untrusted code, if the global environment is exposed by interaction-environment, and the other environments have new toplevel definitions are not possible? Will we be able to make an extendible but existing-bindings-immutable (or existing-bindings-locally-mutable) clone of scheme-report-environment N?

As for first-class environments - I don't see them as so important, as IMHO the useful cases of programmatically building up an environment starting from a base that provides extra functions for use by eval-ed code can be done by doing so in an alist, then converting said alist to a big let that is wrapped around the code to eval, as long as objects like procedures are allowed as self-evaluating literals!

[felix winkelmann]

Replying to alaric:

In particular, my favourite drum to beat is sandboxing: what will this mean for eval-ing untrusted code, if the global environment is exposed by interaction-environment, and the other environments have new toplevel definitions are not possible? Will we be able to make an extendible but existing-bindings-immutable (or existing-bindings-locally-mutable) clone of scheme-report-environment N?

That's correct: it will not be possible to extend the toplevel environment for scheme-report-environment and null-environment (note that this is what R5RS specifies). Creating an extensible environment will not be possible directly, but since I see the need for some sort of sansdbox as well, I thought about providing a different facility (a small framework for extending the interpreter) separate from the core system, but with the core system providing the necessary hooks.

[felix winkelmann]

Poll(Should the implementation of environments be changed in this way, probably risking compatibility problems with code using the `environments` egg)?; Yes; No)?

[felix winkelmann]

New-environments has been merged into master branch.