Building on OS X: egg uses 1 deprecated function and 1 missing function

[Norman Gray]

Building the sqlite3 egg on OS X (10.11.6) produces a warning and an error -- see below for full trace.

Specifically:

'sqlite3_enable_shared_cache' has been explicitly marked deprecated here

The SQLite docs note ​https://www.sqlite.org/c3ref/enable_shared_cache.html that ‘This method is disabled on MacOS X 10.7 and iOS version 5.0 and will always return SQLITE_MISUSE’

Also, the function sqlite3_enable_load_extension appears not to be available on OS X. The SQLite docs ​https://www.sqlite.org/c3ref/enable_load_extension.html seem to recommend a different route to disabling this, on security grounds, but I confess I don't understand the implications of that. This function is still mentioned in the system sqlite3.h header, but it appears to be absent from (been deliberately removed from?) the system sqlite3 library.

There appear to be alternative documented routes to both of the intended functionalities, so I've tentatively marked this as 'easy' difficulty.

Building with:

chicken-install -n -D disable-shared-cache,disable-load-extension

appears to work OK, though I haven't tested fully.

Original build output...

% chicken-install -version
4.11.0
% chicken-install -n
retrieving ...
checking platform for `sqlite3' ...
checking dependencies for `sqlite3' ...
install order:
("sqlite3")
installing sqlite3: ...
changing current directory to .
  '/Data/tools/chicken-4.11.0/bin/csi' -bnq -setup-mode -e "(require-library setup-api)" -e "(import setup-api)" -e "(setup-error-handling)" -e "(extension-name-and-version '(\"sqlite3\" \"\"))" -e "(keep-intermediates #t)" -e "(setup-install-mode #f)" 'sqlite3.setup'
  '/Data/tools/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode -k   -O2 -d1 -s sqlite3.scm -lsqlite3 -j sqlite3

Warning: imported identifier doesn't exist in module chicken: dynamic-wind
sqlite3.c:310:23: warning: 'sqlite3_enable_shared_cache' is deprecated: first deprecated
      in macOS 10.7 [-Wdeprecated-declarations]
C_r=C_int_to_num(&C_a,sqlite3_enable_shared_cache(t0));
                      ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include/sqlite3.h:5424:31: note: 
      'sqlite3_enable_shared_cache' has been explicitly marked deprecated here
SQLITE_API int SQLITE_STDCALL sqlite3_enable_shared_cache(int) __OSX_AVAILABLE_BUT...
                              ^
1 warning generated.
Undefined symbols for architecture x86_64:
  "_sqlite3_enable_load_extension", referenced from:
      _f_8972 in sqlite3.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Error: shell command terminated with non-zero exit status 256: '/Applications/Xcode.app/Contents/Developer/usr/bin/gcc' 'sqlite3.o' -o 'sqlite3.so' -m64 -bundle -headerpad_max_install_names -lsqlite3 -L/Data/tools/chicken-4.11.0/lib -lchicken -lm

Error: shell command failed with nonzero exit status 256:

  '/Data/tools/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode -k   -O2 -d1 -s sqlite3.scm -lsqlite3 -j sqlite3


Error: shell command terminated with nonzero exit code
17920
"'/Data/tools/chicken-4.11.0/bin/csi' -bnq -setup-mode -e \"(require-library set...

[Norman Gray]

This workaround no longer works for Chicken 5.2.0 on macOS 11.5.1. Output below

% chicken-install -D disable-shared-cache,disable-load-extension sqlite3
building sqlite3
   /Data/tools/chicken-5.2.0/bin/csc -host -D compiling-extension -J -s -regenerate-import-libraries -setup-mode -I /Users/norman/.cache/chicken-install/sqlite3 -C -I/Users/norman/.cache/chicken-install/sqlite3 -O2 -d1 -L -lsqlite3 sqlite3.scm -o /Users/norman/.cache/chicken-install/sqlite3/sqlite3.so
/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c:275:23: error: implicit declaration of function 'sqlite3_enable_load_extension' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
C_r=C_int_to_num(&C_a,sqlite3_enable_load_extension(t0,t1));
                      ^
/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c:282:23: warning: 'sqlite3_enable_shared_cache' is deprecated: first deprecated in macOS 10.7 - Not supported [-Wdeprecated-declarations]
C_r=C_int_to_num(&C_a,sqlite3_enable_shared_cache(t0));
                      ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sqlite3.h:6341:16: note: 'sqlite3_enable_shared_cache' has been explicitly marked deprecated here
SQLITE_API int sqlite3_enable_shared_cache(int);
               ^
1 warning and 1 error generated.

Error: shell command terminated with non-zero exit status 256: '/Applications/Xcode.app/Contents/Developer/usr/bin/gcc' '/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c' -o '/Users/norman/.cache/chicken-install/sqlite3/sqlite3.o' -c  -fno-strict-aliasing -fwrapv -fno-common -DHAVE_CHICKEN_CONFIG_H -DC_ENABLE_PTABLES -Os -fomit-frame-pointer -fPIC -DPIC -DC_SHARED -I/Users/norman/.cache/chicken-install/sqlite3 -I/Data/tools/chicken-5.2.0/include/chicken

Error: shell command terminated with nonzero exit code
256
"sh /Users/norman/.cache/chicken-install/sqlite3/sqlite3.build.sh"

[Vasilij Schneidermann]

Here's a patch. Note that the invocation needs to be chicken-install -n -D disable-shared-cache -D disable-load-extension.

[Norman Gray]

I'm afraid it's not just a matter of the mistaken call -- see below (the comma in the argument to -D was a goof: a typo on my part when creating the issue).

In any case, that's not really the issue, but instead the point that the egg is calling a long-deprecated (and now, it appears, finally disabled) interface. When I first looked at this, I recall that there was an alternative recommended by the SQLite docs, and (if I recall correctly) it didn't look terribly hard to switch to. Unfortunately however, I'm not volunteering to make the change, since I'm no longer using Chicken much.

% chicken-install -D disable-shared-cache -D disable-load-extension sqlite3
building sqlite3
   /Data/tools/chicken-5.2.0/bin/csc -host -D compiling-extension -J -s -regenerate-import-libraries -setup-mode -I /Users/norman/.cache/chicken-install/sqlite3 -C -I/Users/norman/.cache/chicken-install/sqlite3 -O2 -d1 -L -lsqlite3 sqlite3.scm -o /Users/norman/.cache/chicken-install/sqlite3/sqlite3.so
objc[1075]: Class AMSupportURLConnectionDelegate is implemented in both /usr/lib/libauthinstall.dylib (0x1f447fad8) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x118cec2b8). One of the two will be used. Which one is undefined.
objc[1075]: Class AMSupportURLSession is implemented in both /usr/lib/libauthinstall.dylib (0x1f447fb28) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x118cec308). One of the two will be used. Which one is undefined.
/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c:275:23: error: implicit declaration of function 'sqlite3_enable_load_extension' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
C_r=C_int_to_num(&C_a,sqlite3_enable_load_extension(t0,t1));
                      ^
/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c:282:23: warning: 'sqlite3_enable_shared_cache' is deprecated: first deprecated in macOS 10.7 - Not supported [-Wdeprecated-declarations]
C_r=C_int_to_num(&C_a,sqlite3_enable_shared_cache(t0));
                      ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sqlite3.h:6341:16: note: 'sqlite3_enable_shared_cache' has been explicitly marked deprecated here
SQLITE_API int sqlite3_enable_shared_cache(int);
               ^
1 warning and 1 error generated.

Error: shell command terminated with non-zero exit status 256: '/Applications/Xcode.app/Contents/Developer/usr/bin/gcc' '/Users/norman/.cache/chicken-install/sqlite3/sqlite3.c' -o '/Users/norman/.cache/chicken-install/sqlite3/sqlite3.o' -c  -fno-strict-aliasing -fwrapv -fno-common -DHAVE_CHICKEN_CONFIG_H -DC_ENABLE_PTABLES -Os -fomit-frame-pointer -fPIC -DPIC -DC_SHARED -I/Users/norman/.cache/chicken-install/sqlite3 -I/Data/tools/chicken-5.2.0/include/chicken

Error: shell command terminated with nonzero exit code
256
"sh /Users/norman/.cache/chicken-install/sqlite3/sqlite3.build.sh"

[Vasilij Schneidermann]

If you apply the patch, the workaround should work again.

From what I understand, shared cache shouldn't be used in the first place and extension loading should be done via sqlite3_load_extension and sqlite3_db_config with SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION. The existing enable-shared-cache! and enable-load-extension! procedures could be changed to raise errors. To support loading extensions there would need to be a replacement API using the aforementioned C functions.

[Vasilij Schneidermann]

I wrote a patch implementing the suggested changes.