Opened 18 months ago

Last modified 15 months ago

#1287 new defect

csc -deploy fails with modules

Reported by: nxg Owned by:
Priority: minor Milestone: someday
Component: unknown Version: 4.10.x
Keywords: Cc:
Estimated difficulty: hard

Description

csc -deploy fails when using modules.

Consider the following:

% cat hello.scm                                   
(module hello (hello)
  (import scheme)
  (define (hello)
    (display "Hello, world")
    (newline)))
% csc -version                                    
(c) 2008-2015, The CHICKEN Team
(c) 2000-2007, Felix L. Winkelmann
Version 4.10.0 (rev b259631)
macosx-unix-clang-x86-64 [ 64bit manyargs dload ptables ]
compiled 2015-08-04 on yves.more-magic.net (Linux)

Enter `chicken -help' for information on how to use the compiler,
or try `csc' for a more convenient interface.

Run `csi' to start the interactive interpreter.
% csc -emit-all-import-libraries -shared hello.scm
% cat main.scm
(use hello)
(hello)
% csc -deploy main.scm
% main/main -:d
[debug] application startup...
[debug] heap resized to 1048576 bytes
[debug] stack bottom is 0x7fff5865a820.
[debug] entering toplevel toplevel...
[debug] entering toplevel library_toplevel...
[debug] entering toplevel build_2dversion_toplevel...
[debug] entering toplevel eval_toplevel...
[debug] entering toplevel expand_toplevel...
[debug] entering toplevel modules_toplevel...
[debug] resizing mutation-stack from 8k to 16k ...
[debug] entering toplevel chicken_2dsyntax_toplevel...
; loading ./hello.so ...
[debug] loading compiled module `./hello.so' (handle is 0x00007fc93150daa0)
[panic] nursery is too small - try higher setting using the `-:s' option - execution terminated

% otool -L main/main
main/main:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1153.18.0)
	@executable_path/libchicken.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

Expected behaviour:

Running main/main would type "Hello, world"

Actual behaviour:

Running main/main produces a panic.

Notes:

This is on OS X 10.10

The file hello.scm is a simplified version of the module in page 'Modules' of the manual; the use of the -deploy option is as suggested on the page http://wiki.call-cc.org/man/4/Deployment

Copying hello.so into the directory main/ (worth a try...) produces the same result.

Guessing the following doesn't work either:

% main/main -:s100000000 
[panic] nursery is too small - try higher setting using the `-:s' option - execution terminated

% 

Attachments (1)

build-boot-chicken.log (100.8 KB) - added by nxg 18 months ago.
Build log of bootstrap build of Chicken, using gcc

Download all attachments as: .zip

Change History (13)

comment:1 Changed 18 months ago by sjamaan

Could you please try CHICKEN 4.11.0rc2? We've fixed several OS X/linking related things.

We're about to release 4.11, so you could also wait for that.

comment:2 Changed 18 months ago by nxg

No change, I'm afraid.

With hello.scm and main.scm as before:

% csc -version
(c) 2008-2016, The CHICKEN Team
(c) 2000-2007, Felix L. Winkelmann
Version 4.11.0rc2 ((detached from 4.11.0rc2)) (rev e910197)
macosx-unix-clang-x86-64 [ 64bit manyargs dload ptables ]
compiled 2016-04-28 on waldrop (Linux)

Enter `chicken -help' for information on how to use the compiler,
or try `csc' for a more convenient interface.

Run `csi' to start the interactive interpreter.
% csc -emit-all-import-libraries -shared hello.scm
% csc -deploy main.scm
% main/main -:d
[debug] application startup...
[debug] heap resized to 1048576 bytes
[debug] stack bottom is 0x7fff5cbdd7f0.
[debug] entering toplevel toplevel...
[debug] entering toplevel library_toplevel...
[debug] entering toplevel build_2dversion_toplevel...
[debug] entering toplevel eval_toplevel...
[debug] entering toplevel expand_toplevel...
[debug] entering toplevel modules_toplevel...
[debug] resizing mutation-stack from 8k to 16k ...
[debug] entering toplevel chicken_2dsyntax_toplevel...
; loading ./hello.so ...
[debug] loading compiled module `./hello.so' (handle is 0x00007ffae8d0d880)
[panic] nursery is too small - try higher setting using the `-:s' option - execution terminated
% otool -L main/main
main/main:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1153.18.0)
	@executable_path/libchicken.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
% uname -a
Darwin ptolemy.astro.gla.ac.uk 14.5.0 Darwin Kernel Version 14.5.0: Mon Jan 11 18:48:35 PST 2016; root:xnu-2782.50.2~1/RELEASE_X86_64 x86_64
%

This is OS X 10.10.5

comment:3 Changed 18 months ago by sjamaan

Could you run gdb on this, break on panic, and show us the output of backtrace?

comment:4 Changed 18 months ago by nxg

Happy to, but ermmmm: could you advise me how best to do that? The various breakpoint specifications I thought would work, in both gdb and lldb, don't seem to have any effect (see below; I get the same odd lack of effect in lldb), and I'm wondering if there's something subtle about breakpoints in shared libraries that I haven't encountered before.

I'm sure I'm being dim here, and my google-fu seems weak today, but any pointers would be appreciated.

[...]
(gdb) break runtime.c:panic
No symbol table is loaded.  Use the "file" command.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (runtime.c:panic) pending.
(gdb) run
Starting program: /private/tmp/chicken/main/main 
warning: `/BinaryCache/coreTLS/coreTLS-35.40.1~1/Objects/coretls.build/coretls.build/Objects-normal/x86_64/system_coretls_vers.o': can't open to read symbols: No such file or directory.
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_ciphersuites.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_handshake.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_record.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_stream_parser.a"
[panic] nursery is too small - try higher setting using the `-:s' option - execution terminated

[Inferior 1 (process 73628) exited with code 01]
(gdb) quit

comment:5 Changed 18 months ago by sjamaan

Oh, I forgot to mention: you need to build your CHICKEN by passing DEBUGBUILD=1 to make. This adds debugging symbols to libchicken.

comment:6 Changed 18 months ago by nxg

I'm afraid that didn't work for me either – see gdb output below. I also tried building it using clang and using lldb, but with no more success. I also tried a bootstrap build, for both gcc/gdb and clang/lldb. I'll attach the build log to this comment. I can see #define DEBUGBUILD 1 in chicken-defaults.h, so I don't think I've made any typos.

quaxo:/tmp/chicken> which csc
/Data/tools/chicken-4.11.0rc2/bin/csc
quaxo:/tmp/chicken> csc -emit-all-import-libraries -shared hello.scm
quaxo:/tmp/chicken> csc -deploy main.scm
quaxo:/tmp/chicken> sudo gdb main/main
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin14.5.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from main/main...
warning: `/private/tmp/chicken/main.o': can't open to read symbols: No such file or directory.
(no debugging symbols found)...done.
(gdb) break runtime.c:panic
No symbol table is loaded.  Use the "file" command.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (runtime.c:panic) pending.
(gdb) run
Starting program: /private/tmp/chicken/main/main 
warning: `/BinaryCache/coreTLS/coreTLS-35.40.1~1/Objects/coretls.build/coretls.build/Objects-normal/x86_64/system_coretls_vers.o': can't open to read symbols: No such file or directory.
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_ciphersuites.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_handshake.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_record.a"
warning: Could not open OSO archive file "/BinaryCache/coreTLS/coreTLS-35.40.1~1/Symbols/BuiltProducts/libcoretls_stream_parser.a"
Error in re-setting breakpoint 1: Cannot access memory at address 0x735c00
warning: `/private/tmp/chicken/hello.o': can't open to read symbols: No such file or directory.
Error in re-setting breakpoint 1: Cannot access memory at address 0x735c00
[panic] nursery is too small - try higher setting using the `-:s' option - execution terminated

[Inferior 1 (process 86691) exited with code 01]
(gdb) quit

Changed 18 months ago by nxg

Build log of bootstrap build of Chicken, using gcc

comment:7 Changed 18 months ago by zbigniew

I have no idea how deploy is supposed to work but I can say 2 things:

1) can reproduce the problem on 4.8.0.6 with OS X 10.11.4
2) why does hello.so refer to the absolute path of the libchicken.dylib, while main refers to the @executable_path?

$ otool -L hello.so
hello.so:

/Users/jim/local/chicken/4.8.0.6/lib/libchicken.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

$ otool -L main
main:

/System/Library/Frameworks/CoreFoundation?.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1258.1.0)
@executable_path/libchicken.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

comment:8 Changed 18 months ago by zbigniew

OK, the problem is you're trying to use an extension (hello.so) that hasn't been deployed. When you built hello.so, it wasn't linked for deployment (as evidenced by the fact that it didn't get put into the main/ directory).

I don't know if it's possible to deploy a shared library directly. You may have to make it into an extension and deploy it into the main/ directory with chicken-install. People with experience in using -deploy are welcome to chime in.

comment:9 Changed 18 months ago by zbigniew

  • Priority changed from major to minor

comment:10 Changed 18 months ago by nxg

For completeness, I should note that this doesn't work if I simply drop hello.so and hello.import.scm into the main/ directory; so that doesn't count as 'deployment' of the library (not entirely surprisingly).


But you mentioned making this into an extension...

quaxo:/tmp/chicken> cat hello.scm 
(module hello (hello)
  (import scheme)
  (define (hello)
    (display "Hello, world")
    (newline)))
quaxo:/tmp/chicken> cat hello.meta
((author "Me")
 (synopsis "A cool hello-world library")
 (license "BSD")
 (files "hello.scm" "hello.setup"))
quaxo:/tmp/chicken> cat hello.setup
(standard-extension "hello")
quaxo:/tmp/chicken> cat main.scm
(use hello)
(hello)
quaxo:/tmp/chicken> chicken-install -deploy -prefix $PWD/main
retrieving ...
checking platform for `hello' ...
checking dependencies for `hello' ...
install order:
("hello")
installing hello: ...
changing current directory to .
  '/Data/tools/chicken-4.11.0rc2-clang/bin/csi' -bnq -e "(require-library setup-api)" -e "(import setup-api)" -e "(setup-error-handling)" -e "(extension-name-and-version '(\"hello\" \"\"))" -e "(destination-prefix \"/tmp/chicken/main\")" -e "(runtime-prefix \"/tmp/chicken/main\")" -e "(deployment-mode #t)" 'hello.setup'
  '/Data/tools/chicken-4.11.0rc2-clang/bin/csc' -feature compiling-extension    -deployed -dynamic -optimize-level 3 -debug-level 1 hello.scm -emit-import-library hello
  '/Data/tools/chicken-4.11.0rc2-clang/bin/csc' -feature compiling-extension    -deployed -dynamic -optimize-level 3 -debug-level 0 hello.import.scm
  mkdir -p '/tmp/chicken/main'
  chmod a+x '/tmp/chicken/main'
  cp -r 'hello.so' '/tmp/chicken/main/hello.so'
  chmod a+r '/tmp/chicken/main/hello.so'
  cp -r 'hello.import.so' '/tmp/chicken/main/hello.import.so'
  chmod a+r '/tmp/chicken/main/hello.import.so'
  chmod a+r '/tmp/chicken/main/hello.setup-info'
quaxo:/tmp/chicken> rm hello.import.so
quaxo:/tmp/chicken> csc -deploy main.scm                            
quaxo:/tmp/chicken> main/main
Hello, world
quaxo:/tmp/chicken> 

I'm slightly surprised that I need to remove hello.import.so (csc -deploy main.scm fails otherwise), but this appears to work.

Perhaps, therefore, this should be recategorised as a documentation bugreport. At the end of http://wiki.call-cc.org/man/4/Deployment it mightW would be useful to note that if one wishes to deploy a program which uses modules, then it is necessary to wrap them up as (local) extensions, as described in http://wiki.call-cc.org/man/4/Extensions#a-simple-library

Also, if there were a way to detect this situation, and have a less panicky error message, that would be most civil.

Hooray!

comment:11 Changed 15 months ago by sjamaan

  • Estimated difficulty set to easy

comment:12 Changed 15 months ago by sjamaan

  • Estimated difficulty changed from easy to hard
Note: See TracTickets for help on using tickets.