Ticket #736: 00repospec.txt

File 00repospec.txt, 6.9 KB (added by Jim Ursetto, 8 years ago)

repospec musings

Line 
1other things to check out:
2  virtualenv (python)  (see EOF)
3  (single-path) alt repo script: http://paste.call-cc.org/?id=fc11a30f00b18360f1957c9ffca8797a811ed8b3
4
5warnings:
6   eval.scm needs data-structures, a new dependency.
7
8notes:
9 - .setup-info file list contains redundant full path to each file (for whatever reason). 
10   Therefore, you cannot simply copy an extension into a repository; chicken-status will
11   list the wrong installed files, and chicken-uninstall will uninstall the wrong files.
12
13 - Possibly we should allow chicken-status to sort eggs by repo, preserving all eggs instead
14   of discarding later ones in the path.
15
16
17 - Host/target extensions may not be handled properly.  Examples:
18   - chicken-status prints entire repo-path interspersed by ; when cross-chicken.  Target extensions
19     appear to be hardcoded to C_TARGET_LIB_HOME/chicken/6 only; repository-pathspec should only
20     affect host extensions.  I believe this is handled correctly.
21
22 - Private repository should perhaps be the "dpath" default repository, inserted when there is
23   an empty element, instead of overriding the entire pathspec.
24
25 - Changing LOAD to follow repository path may change behavior when repository is set to (say) foo/,
26   but the loaded extension is not in foo; currently, it will try . as a last resort.
27   - Specifically, ##sys#find-extension automatically adds "." to end of search path, or
28     to beginning of search path if -setup-mode is set.
29   - Example: may be implicit in -ignore-repository option in batch-driver.scm, which sets
30     repository-path to #f -- although not currently working, it may be intended to search "."
31     only and would need to be set to (".") instead.
32   - If env var is unset, pathspec may need to default to (list dpath ".") to simulate current behavior.
33   - If env var is set, it will probably break the CWD last resort behavior, as I don't want to
34     default to searching CWD.
35   - Setting pathspec to include "." at end is not exactly equivalent to current behavior.
36     For example, neither modules.db nor types.db are currently searched for in CWD, but they
37     would be if "." was set in pathspec.  Also, extension-information never searches CWD.
38     However, resolve-include-filename does search CWD before trying other include paths (which include
39        the repo only when when using -extend, and -inline-global).
40
41
42
43- modules.db creation requires that modules be imported, since import files can define arbitrary modules
44  not necessarily named correctly.   This is quite rare; see prometheus which defines srfi-8, etc.
45  We cannot rely on .import.scm name to check module path / shadowing.
46  Problem: we can't track which repo a module was imported from.  Possible solution: modify module record
47  to contain the repository path from which it was imported (if any).  Once all import files are loaded
48  in path order, write out per-path modules.db.
49  - Note: modules.db is only (currently) used for module import suggestion when the compiler cannot find a module.
50    It is currently impossible to track back a module to its .import.scm (i.e. library name) except by
51    assuming its name is equivalent to its library name.  In fact, modules.db contains spurious entries
52    like (receive syntax srfi-8) when prometheus is installed, even though srfi-8 is an internal artifact,
53    and (use srfi-8) makes no sense outside of prometheus' context.
54  - Also, these "internal" modules can already shadow each other if their names conflict and their eggs
55    are installed in the same repository.
56  - Therefore, it could be argued that only the module named like the import file should be placed in modules.db.
57    It's possible the current behavior of traversing the module table exists just because it's easy.
58  - Or we could just assume "import name == module name" and use this to determine which main modules are
59    shadowed, while still writing the whole table to individual modules.db.  Unfortunately we would have no
60    idea where to put the "internal" modules as they have no filename analog; we might even get confused if
61    an internal module name conflicts with a main module name.
62  "
63     02:57:11 < zbigniew> hmm; it seems that the behavior of modules.db may not even be right currently, with respect to import files that register multiple modules
64     02:59:41 < zbigniew> given that modules.db is used to make possible import suggestions like "suggesting: (import colorize)"
65     03:01:00 < zbigniew> it generally makes little sense to suggest importing a module that is buried inside a differently-named import file -- as this will almost always fail!
66     03:02:43 < zbigniew> as an interesting example, if you were to compile the simple file:
67     03:02:46 < zbigniew> (module foo () (import scheme) (receive))
68     03:02:55 < zbigniew> and prometheus is installed, you will get:
69     03:03:10 < zbigniew> Warning: reference to possibly unbound identifier `receive'
70     03:03:10 < zbigniew> Warning:    suggesting one of:
71     03:03:10 < zbigniew> Warning:    (import srfi-8)
72     03:03:10 < zbigniew> Warning:    (import chicken)
73     03:03:48 < zbigniew>
74     03:05:43 < zbigniew> which is essentially nonsense; so I wonder if slavishly adhering to the current behavior is even necessary, if it entails intrusive changes.
75  "
76
77
78
79* virtualenv / pip
80
81setuptools (and pip) have multiple paths but are a bit schizophrenic because the functionality was added over a long time
82Native python has only sys.path (afaik) module lookup, without any versioning or dependencies.
83
84- multiple installed versions are supported via runtime manipulation of the path
85- installs get their own dedicated and versioned directory and old versions are not necessarily deleted after upgrade
86  - active install is specified, I *believe*, by .pth file in every site-packages/ dir
87- scripts are (always?) stub trampolines to an entry point in the main code, which resides in the code proper
88- setuptools creates the stub scripts for you, given instructions in the setup.py file
89- scripts are installed apart from main code, unversioned, in a dedicated bin directory.  Bin dir overridable by user in config file.
90- you can have multiple script versions installed, but you have to manually `cp script script-ver` after every version install;
91  the stub code itself refers to the egg version, so the correct code library version will still be called
92- pkg_resources API is used to indirectly locate files included with the egg.  Primarily this allows resources
93  to be transparently contained within a zip file.  In Chicken we could use a similar technique with a special resources egg.
94- (UNTESTED) pkg_resources always refers to resources in a versioned code dir, I think, so you should have one full copy of
95  all resources with each installed version.
96- (UNKNOWN) Can you write to a resource directory at runtime, or how else do you find the path to variable files?
97  And if you did write to a resource dir, I assume your changes are wiped out (or inaccessible) when you upgrade the egg, due to versioning.