Summary

Packaging in OpenEmbedded and Arch broken by CHICKEN 4.11.0

Metadata

Description

Otavio Salvador and Tom Willemse reported packaging issues in OpenEmbedded and Arch Linux, respectively. The symptom is that some files are getting packaged into multiple packages. Package managers complain about that when installing packages (same file is provided by multiple packages).

In the case of OpenEmbedded, the issue was observed when installing eggs using the cross-chicken. The code which implements the call to chicken-install is https://github.com/OSSystems/meta-chicken/blob/master/classes/chicken_install.bbclass#L44

Changes and comments

[2016-08-31 14:02:35 UTC] mario set difficulty to medium

[2016-08-31 14:02:35 UTC] mario changed milestone from someday to 4.12.0

[2016-12-18 22:51:18 UTC] mario wrote:

Below you can find a relatively simple way to demonstrate the difference in behavior between CHICKEN 4.10.0 and 4.11.0, which is likely to be causing the issue reported in this ticket.

The following script will be used to demonstrate the problem:

 $ cat chicken-install-wrapper.sh
 #! /bin/sh
 
 eggsdir=$1
 shift
 
 binversion=`csi -p '(##sys#fudge 42)'`
 mkdir -p $eggsdir/lib/chicken/$binversion
 
 if [ ! -e "$eggsdir/lib/chicken/$binversion/srfi-13.import.so" ]; then
     chicken-install -i $eggsdir/lib/chicken/$binversion
     mkdir -p $eggsdir/share/chicken/
     cp `csi -p '(chicken-home)'`/setup.defaults $eggsdir/share/chicken/
 fi
 
 CHICKEN_C_INCLUDE_PATH=$eggsdir/include/chicken \
 CHICKEN_INCLUDE_PATH=$eggsdir/share/chicken \
 chicken-install -p $eggsdir $@

To reproduce the issue (assuming CHICKENs are installed under $HOME/local) using the channels egg (only miscmacros is a dependency):

 # Install the egg and dependencies in the canonical local repo
 $HOME/local/chicken-4.10.0/bin/chicken-install channels
 $HOME/local/chicken-4.11.0/bin/chicken-install channels

Run chicken-install-wrapper.sh with 4.10.0:

 $ rm -rf ~/eggs-4.10.0
 $ PATH=$HOME/local/chicken-4.10.0/bin:$PATH chicken-install-wrapper.sh ~/eggs-4.10.0 channel
 copying required files to /home/mario/eggs-4.10.0/lib/chicken/7 ...
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/setup-api.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/setup-api.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/setup-download.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/setup-download.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/chicken.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/lolevel.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-1.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-4.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/data-structures.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/ports.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/files.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/posix.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-13.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-69.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/extras.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-14.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/tcp.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/foreign.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/srfi-18.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/utils.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/csi.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/irregex.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7'
   cp -r '/home/mario/local/chicken-4.10.0/lib/chicken/7/types.db' '/home/mario/eggs-4.10.0/lib/chicken/7'
 retrieving ...
 connecting to host "chicken.kitten-technologies.co.uk", port 80 ...
 requesting "/henrietta.cgi?name=channel&mode=default" ...
 reading response ...
 HTTP/1.1 200 OK
 Date: Sun, 18 Dec 2016 21:39:13 GMT
 Server: Apache/2.2.31 (Unix) DAV/2 PHP/5.5.36 mod_fastcgi/2.4.6
 Connection: close
 Transfer-Encoding: chunked
 Content-Type: text/plain
 reading chunks .
 reading files ...
   ./channel.meta
   ./set.scm
   ./channel.release-info
   ./channel.scm
   ./channel.setup
  channel located at /tmp/temp35e7.31743/channel
 checking platform for `channel' ...
 checking dependencies for `channel' ...
 install order:
 ("channel")
 installing channel:0.0.3 ...
 changing current directory to /tmp/temp35e7.31743/channel
   '/home/mario/local/chicken-4.10.0/bin/csi' -bnq -setup-mode -e "(require-library setup-api)" -e "(import setup-api)" -e "(setup-error-handling)" -e "(extension-name-and-version '(\"channel\" \"0.0.3\"))" -e "(destination-prefix \"/home/mario/eggs-4.10.0\")" -e "(runtime-prefix \"/home/mario/eggs-4.10.0\")" 'channel.setup'
   '/home/mario/local/chicken-4.10.0/bin/csc' -feature compiling-extension -setup-mode    -s -d0 -O3 -J channel.scm
   '/home/mario/local/chicken-4.10.0/bin/csc' -feature compiling-extension -setup-mode    -s -d0 -O3 channel.import.scm
   cp -r 'channel.so' '/home/mario/eggs-4.10.0/lib/chicken/7/channel.so'
   chmod a+r '/home/mario/eggs-4.10.0/lib/chicken/7/channel.so'
   cp -r 'channel.import.so' '/home/mario/eggs-4.10.0/lib/chicken/7/channel.import.so'
   chmod a+r '/home/mario/eggs-4.10.0/lib/chicken/7/channel.import.so'
   chmod a+r '/home/mario/eggs-4.10.0/lib/chicken/7/channel.setup-info'

Now run chicken-install-wrapper.sh with 4.11.0:

 $ rm -rf ~/eggs-4.11.0
 $ PATH=$HOME/local/chicken-4.11.0/bin:$PATH chicken-install-wrapper.sh ~/eggs-4.11.0 channel
 copying required files to /home/mario/eggs-4.11.0/lib/chicken/8 ...
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/setup-api.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/setup-api.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/setup-download.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/setup-download.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/chicken.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/lolevel.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-1.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-4.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/data-structures.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/ports.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/files.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/posix.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-13.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-69.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/extras.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-14.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/tcp.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/foreign.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/srfi-18.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/utils.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/csi.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/irregex.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8'
   cp -r '/home/mario/local/chicken-4.11.0/lib/chicken/8/types.db' '/home/mario/eggs-4.11.0/lib/chicken/8'
 retrieving ...
 connecting to host "chicken.kitten-technologies.co.uk", port 80 ...
 requesting "/henrietta.cgi?name=channel&mode=default" ...
 reading response ...
 HTTP/1.1 200 OK
 Date: Sun, 18 Dec 2016 21:39:28 GMT
 Server: Apache/2.2.31 (Unix) DAV/2 PHP/5.5.36 mod_fastcgi/2.4.6
 Connection: close
 Transfer-Encoding: chunked
 Content-Type: text/plain
 reading chunks .
 reading files ...
   ./channel.meta
   ./set.scm
   ./channel.release-info
   ./channel.scm
   ./channel.setup
  channel located at /tmp/temp6a1d.31838/channel
 checking platform for `channel' ...
 checking dependencies for `channel' ...
  missing: miscmacros
 retrieving ...
 connecting to host "chicken.kitten-technologies.co.uk", port 80 ...
 requesting "/henrietta.cgi?name=miscmacros&mode=default" ...
 reading response ...
 HTTP/1.1 200 OK
 Date: Sun, 18 Dec 2016 21:39:28 GMT
 Server: Apache/2.2.31 (Unix) DAV/2 PHP/5.5.36 mod_fastcgi/2.4.6
 Connection: close
 Transfer-Encoding: chunked
 Content-Type: text/plain
 reading chunks .
 reading files ...
   ./debian/
   ./debian/changelog
   ./debian/control
   ./debian/copyright
   ./debian/rules
   ./miscmacros.setup
   ./miscmacros.meta
   ./miscmacros.scm
  miscmacros located at /tmp/temp6a1d.31838/miscmacros
 checking platform for `miscmacros' ...
 checking dependencies for `miscmacros' ...
 install order:
 ("miscmacros" "channel")
 installing miscmacros:2.96 ...
 changing current directory to /tmp/temp6a1d.31838/miscmacros
   '/home/mario/local/chicken-4.11.0/bin/csi' -bnq -setup-mode -e "(require-library setup-api)" -e "(import setup-api)" -e "(setup-error-handling)" -e "(extension-name-and-version '(\"miscmacros\" \"2.96\"))" -e "(destination-prefix \"/home/mario/eggs-4.11.0\")" -e "(runtime-prefix \"/home/mario/eggs-4.11.0\")" 'miscmacros.setup'
   '/home/mario/local/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode    -s -O3 -d0 miscmacros.scm -j miscmacros
   '/home/mario/local/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode    -s -O3 -d0 miscmacros.import.scm
   cp -r 'miscmacros.so' '/home/mario/eggs-4.11.0/lib/chicken/8/miscmacros.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/miscmacros.so'
   cp -r 'miscmacros.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8/miscmacros.import.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/miscmacros.import.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/miscmacros.setup-info'
 installing channel:0.0.3 ...
 changing current directory to /tmp/temp6a1d.31838/channel
   '/home/mario/local/chicken-4.11.0/bin/csi' -bnq -setup-mode -e "(require-library setup-api)" -e "(import setup-api)" -e "(setup-error-handling)" -e "(extension-name-and-version '(\"channel\" \"0.0.3\"))" -e "(destination-prefix \"/home/mario/eggs-4.11.0\")" -e "(runtime-prefix \"/home/mario/eggs-4.11.0\")" 'channel.setup'
   '/home/mario/local/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode    -s -d0 -O3 -J channel.scm
   '/home/mario/local/chicken-4.11.0/bin/csc' -feature compiling-extension -setup-mode    -s -d0 -O3 channel.import.scm
   cp -r 'channel.so' '/home/mario/eggs-4.11.0/lib/chicken/8/channel.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/channel.so'
   cp -r 'channel.import.so' '/home/mario/eggs-4.11.0/lib/chicken/8/channel.import.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/channel.import.so'
   chmod a+r '/home/mario/eggs-4.11.0/lib/chicken/8/channel.setup-info'

Note that 4.11.0 also installs miscmacros in the directory specified with -p (~/eggs-4.11.0). 4.10.0 does not install miscmacros in the directory specified with -p (~/eggs-4.10.0).

[2016-12-26 16:31:35 UTC] mario wrote:

The behavior change has been introduced by e090c524b782e3bc982299b4c021a5fc96db36e4 . On a second thought, the behavior in 4.11 makes sense, as eggs installed in an alternative prefix by 4.10 are basically useless if they have dependencies and dependencies are not explicitly installed in the alternative prefix.

On the other hand, the behavior in 4.11 makes packaging individual eggs very difficult for packagers.

[2016-12-26 16:57:14 UTC] mario wrote:

Maybe the solution for packagers is using information from .setup-info files. It's not a perfect solution because egg authors still can install stuff without using install-{extension,script,program}.

[2017-01-22 18:01:17 UTC] sjamaan changed status from new to closed

[2017-01-22 18:01:17 UTC] sjamaan set resolution to fixed

[2017-01-22 18:01:17 UTC] sjamaan wrote:

Should be fixed with the new -no-install-deps option introduced by ea1b8f4.

[2017-01-22 18:01:17 UTC] sjamaan wrote:

1485104489720464