source: project/gazette/src/issues/21.wiki @ 25557

((title . "Issue 21")
 (authors "Andy Bennett")
 (date . 1321465864))

== 0. Introduction

Welcome to the 21st issue of the Chicken Gazette. While the gazette
was on hiatus the chicken community was definitely not.

With the stabilisation of Chicken 4.7 well underway since the last
gazette...

Some people have also started to set up salmonella farms and C-Keen
has completed his "Chicken Zoo".

Enjoy!


== 1. Chickens Out & About

=== Nuremberg (December 2011)
[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] wrote
([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00114.html|2011/10/22 17:38Z]]) to
confirm the planned Chicken Hacking Sprint scheduled for the beginning of December.

Interested and available Chickenauts will meet in Nuremberg on the weekend of 9th, 10th and 11th
of December. If you haven't done so already, you can let us know that you will be joining us by
placing a note at the foot of the [[https://wiki.call-cc.org/chicken-sprint-nuremberg|wiki page]].
If you can't attend in person then we'll be keeping an eye on IRC so feel free to book yourself
a slot in your cave at home and join us throughout the weekend and into the evenings, Central European Time.

For those of you keeping score, this will bring the total number of Chicken meets this year to
5 (FOSDEM, Cologne, FrOSCon, T-DOSE and Nuremberg).


=== Cologne (March 2011)

In March, Chickens met up at the Chaos Computer Club in Cologne, Germany for a Chicken Hacking Weekend. Here's
[[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]]'s report on what they got up to.

The usual suspects [[user:peter-bex|Peter Bex]],
[[user:felix-winkelmann|Felix Winkelmann]],
[[user:christian-kellermann|Christian Kellermann]] and me met for two
and a half days of exchanging ideas and hacking. While we also fixed a
good share of bugs our main goal for this weekend was to create a
working prototype of a distributed egg system. By the end of Sunday,
mostly thanks to Peter's effort, we had something that was already
pretty similar to what is in use today (see
[[http://lists.nongnu.org/archive/html/chicken-users/2011-03/msg00160.html|his announcement on the mailing list]]). Apart
from that some interested visitors stopped by every now and then. Some
nice chats, food and the occasional beers were had which hopefully
helped to spread the Chicken word.


=== FrOSCon (August 2011)
At FrOSCon in August we sported not only a project booth but a presentation room as well so we took the
opportunity to lay on a couple of talks for the delegates.

[[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]] presented
"[[http://programm.froscon.org/2011/events/869.html|An introduction to Lisp]]:
Why to talk to computers in parentheses" whilst
[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] offered
"[[http://programm.froscon.org/2011/events/867.html|A guided tour through the republic of CHICKEN]]:
get up to speed with the practical scheme implementation".
[[http://wiki.call-cc.org/users/christian-kellermann|Christian]] has been keeping his talk up-to-date and
you can find the latest version [[http://wiki.call-cc.org/chickenista-guide|on the wiki]].

Naturally, there are [[http://www.call-cc.org/pictures/froscon2011/|pictures]].


=== T-DOSE 2011 (November 2011)
European Chickens met up at T-DOSE in Eindhoven, The Netherlands on the weekend of the 5th, 6th and 7th of
November. [[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] arranged for us to contribute a "Chicken
Scheme Project Booth" and we subsequently lured a few Dutch hackers into #chicken on IRC.

[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] knocked up some impressive demos
of [[http://en.wikipedia.org/wiki/Pong|PONG]] using Chicken's [[http://wiki.call-cc.org/eggref/4/cairo|Cairo Egg]].

[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]] finished off the tests for his
[[http://wiki.call-cc.org/eggref/4/ugarit|Ugarit]] backup egg which allowed him to release version 1.0.

...and [[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]] started on a replacement for the
[[http://wiki.call-cc.org/eggref/4/environments|Environments Egg]] as it has unfortunately has been overtaken
by developments in the Chicken development branch.

Pictures are up in the [[http://www.call-cc.org/pictures/t-dose2011/|usual place]] on the Chicken website.



== 2. Chicken Talk

=== R7RS to be dedicated to the memory of John McCarthy

Matt Welland wrote in with the sad news that the "Father of Lisp", John McCarthy had sadly passed away ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00115.html|2011/10/25 15:59Z]]). [[http://wiki.call-cc.org/users/john-cowan|John Cowan]], member of the R7RS working group, told us that R7RS,
the upcoming version of the Scheme standard, would be dedicated to his memory ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00116.html|16:22Z]])


=== Parallel build patch for Chicken Makefile

A lively discussion broke out on the [[https://wiki.call-cc.org/mailing-lists|Chicken Users mailing list]] regarding
[[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00025.html|parallelisation of the Chicken build process]].

Vitaly Magerya supplied a [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00025.html|patch]] (2011/10/06 12:37Z) to fix some problems
he'd been having with builds failing when specifying the {{-j}} to {{gmake}}.
[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] noted that this functionality was already being
addressed by [[http://bugs.call-cc.org/ticket/526|ticket 526]]
([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00027.html|2011/10/06 12:39Z]]) and Vitaly was keen to see it committed
soonly as he maintains the FreeBSD port and people had been asking him for parallel builds ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00028.html|2011/10/06 14:24Z]]). Whilst still keen, [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00029.html|advised caution]] as he wanted to see it proven ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00031.html|2011/10/06 14:40Z]]).

Vitaly was keen to push things forward ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00033.html|2011/10/06 15:45Z]]) and
offered continued testing under FreeBSD 8.2-RELEASE on x86_64. He noted that, as far as he could tell, {{gmake check}} passed when using his
patch, modified with [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00026.html|Moritz Wilhelmy's suggestions]]. Moreover, he
offered to try the patch from [[http://bugs.call-cc.org/ticket/526|ticket 526]] as well.

[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00034.html|suggested]]
that using the [[http://bugs.call-cc.org/ticket/526|ticket 526]] patch was a good idea as it had, as far as he knew, received testing under Linux and MacOSX.

Toby Thain spoke in favour of the patch, saying that a working parallel build gives confidence that the Makefile is, in fact, correct.
He offered to test on three platforms, namely Solaris 10 SPARC SMP and both PowerPC and Intel version of MacOSX SMP.

Vitaly updated the [[http://bugs.call-cc.org/ticket/526|ticket 526]] patch to the then current sources and confirmed that the build worked at {{-j8}}
and the tests passed in his FreeBSD 8.2-RELEASE, x86_64 environment. He did, however, note that parallelisation does not work for the install phase
([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00048.html|2011/10/08 12:29Z]]).

Toby Thain reported successful builds on the 64-bit version MacOSX-10.6 as well as MacOSX-10.5 on a dual-processor G5. He noted "{{make -j8}}
reduces the make step from 3:39 to 0:47 on my 8-core system". Unfortunately {{make check}} did not pass on either system, with or without the patch
([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00071.html|2011/10/12 01:59Z]] &
[[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00100.html|2011/10/15 19:53Z]])

[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] noted that it might necessary to install the compiled binaries
([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00101.html|2011/10/15 20:07Z]]) however, Toby did not report back as to whether this
had any effect on the results of the tests.


=== 64-bit SPARC Build

Nicolas Pelletier came to the [[https://wiki.call-cc.org/mailing-lists|mailing list]]
unable to build for Solaris 10 running on SPARC64 due to a missing {{apply-hack.S}}
([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00012.html|2011/11/09 11:48Z]]).

It turned out that there was no SPARC64 support for Chicken at all and [[http://wiki.call-cc.org/users/john-cowan|John Cowan]]
came to the rescue ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00013.html|15:36Z]])
with instructions for how to add it to the supported architectures list and disable the
apply hack. [[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] chimed in with affirmations
around the utility of using Chicken without an apply hack ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00014.html|15:54Z]])

Nicolas struggled with disabling the apply hack ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00015.html|2011/11/10 11:10Z]])
due to the Two Lies Of The README. Over the course of the next few minutes [[http://wiki.call-cc.org/users/christian-kellermann|Christian]]
and [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]]]
confirmed that the lies, which related to the way bootstrap compilers are distributed and invoked, had
already been fixed in the development versions.

Meanwhile, Toby Thain tried the build on his SPARC64 ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00020.html|12:40Z]])
and discovered that GNU make 3.82 is required as well as the addition of {{-lrt}} in {{Makefile.solaris}}. Toby's
build eventually succeeded and appeared to be good, albeit having only used 1 of his 4 CPUs and with a couple of {{make check}} errors.

Later on the Thursday, [[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] noted ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00021.html|16:47Z]])
that the [[http://bugs.call-cc.org/raw-attachment/ticket/526/parallel-build-2.diff.txt|parallel build patch]] in
[[http://bugs.call-cc.org/ticket/526|ticket 526]] still works on Chicken 4.7.0.3-st. Toby [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00022.html|confirmed]]
that the patch did indeed work on Solaris and build time was reduced to 10 just of his Earth minutes from the original 40.
Of course, user time across the four CPUs was still in the region of 40 minutes in total. :-)

To round everything off, Toby offered [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00028.html|binary SYSV packages for Solaris 10 on SPARC64]].
Contact him via the [[https://wiki.call-cc.org/mailing-lists|mailing list]] if you're interested.


=== Static Linking & Standalone Binaries.

Serg Kozhemyakin [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00105.html|asked]] whether it is possible to build static
binaries after having trouble using the {{-static}} flag to {{csc}}. He also wondered about building standalone binaries for Windows.
[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00106.html|advised]]
against static linking as many of the eggs cannot cope with it. He did, however, suggest that the {{-deploy}} option may well solve both
issues as it ensures that all the DLLs (including the Chicken runtime and other shared objects) are bundled together into a single directory.


=== Tracing with {{,tr}} moved to an egg

[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00007.html|helped]]
Curtis Cooley in his [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00006.html|search for the {{,tr}} command]].
The functionality had been moved from the core into an egg in Chicken 4.3.0.


=== {{gmake check}} sometimes fails.

Pekka Niiranen wrote to tell us about
[[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00009.html|problems with {{gmake check}} in Chicken 4.7.0-st]]
on OpenBSD 5.0.

[[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00010.html|said]]
that one of the failures had been fixed in the stability branch after the Chicken 4.7.0-st release and he filed
[[https://bugs.call-cc.org/ticket/724|ticket 724]] for the segmentation fault issue.

[[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] mentioned that the [[https://bugs.call-cc.org/ticket/724|ticket 724]] segmentation
fault had also been observed under NetBSD and Arch Linux. He said that "It's known to be broken" but that it is not properly reproducible
as it only happens some of the time ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00011.html|2011/11/06 09:51|]]).


== 3. Salmonella and other tests
Salmonella is Chicken's egg testing framework.

For some time we have had daily salmonella runs of the master development
version of Chicken under Linux on x86. Throughout November, MacOSX-x86_64 has also
been regularly experiencing the runs. [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] who maintains the testing
effort at [[http://tests.call-cc.org/]], as well as the [[https://github.com/mario-goulart/salmonella|Salmonella codebase]]
itself, is always on the lookout for more feeds. We are aware that testing on
other platforms is being organised but if you've got a private Salmonella
running regularly on an interesting operating system or version of Chicken then
give us a shout and we'll see if we can get it included with the other reports
on [[http://tests.call-cc.org/]]

There is also [[https://github.com/mario-goulart/salmonella|an entirely new version of Salmonella]] over at github so test that too. The new version is more modular -- some features that were previously built in salmonella are now independent eggs (e.g., [[https://github.com/mario-goulart/salmonella-html-report|salmonella-html-report]] and [[https://github.com/mario-goulart/salmonella-feeds|salmonella-feeds]].  There's also a new egg that provides a new feature: [[https://github.com/mario-goulart/salmonella-diff|salmonella-diff]] which generates HTML output to render differences between two salmonella logs.

Here's short list of new salmonella features:

* egg lint mode: salmonella can now easily check for common egg mistakes before the code hits the egg repository.  Just run {{salmonella --this-egg}} from the egg directory
* a new execution mode that can significantly reduce salmonella execution times.  In this mode, salmonella won't set the temporary egg installation directory empty before installing new eggs.  While this mode won't spot egg dependencies problems, it can drastically effect the execution time
* a new tool to take advantage of multi-core systems ({{salmonella-epidemy}})
* a simple text mode log file viewer ({{salmonella-log-viewer}})
* [[https://github.com/mario-goulart/salmonella-html-report|salmonella-html-report]] generates reverse dependencies graphs.  That can be useful, for
example, to ilustrate how many eggs depend on a given egg.


[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]] is one of the people who has been setting up a Chicken testing
environment. After [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00068.html|some discussion]], he chose Chicken 4.5.0 under
NetBSD ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00035.html|2011/10/06 21:18Z]]). He's been keeping track of all the
Egg dependencies and plans to publish them as a list of NetBSD pkgsrc packages.

Thanks to [[http://www.bevuta.com/|bevuta IT GmbH]], the CHICKEN test
infrastructure will soon have more machines available to test the
CHICKEN compiler, tools and eggs.  A new salmonella environment for
Linux/x86-64 is currently being set up.

In addition to salmonella, work on the
[[http://wiki.call-cc.org/playground|Chicken Playground]] project has
been resumed.  The Chicken Playground project provides environments
that can be used by tools like {{chroot}} to test CHICKEN tools and
eggs.  Those environments contain tools and libraries required to
build and test most eggs.  Currently a Linux/x86 environment is
available. A Linux/x86-64 environment is under development and should
be available soon.

The new test infrastructure, based on new versions of salmonella and
Chicken Playground and running on new platforms will probably go into
production during the Nuremberg hackaton.


== 4. Core Development

Since the last issue most core development took place in the
"scrutinizer", the flow-analysis pass used for determining the number
and types of values as they propagate through Scheme expressions. The
results of this analysis is now used to replace various primitive
operations by more efficient code, taking advantage of the obtained
type information. Over the last months the typing and rewrite rules of
all core library procedures (which can be found in the installed
"types.db" file) have been heavily improved and debugged. Depending on
the nature of the code, these optimizations can result in significant
performance improvements. For more information on specialisation see
the respective section at
[[http://library.readscheme.org/page10.html|readscheme.org]].

A couple of months ago the CHICKEN team has formalised the development
process a bit as can be seen on
[[http://wiki.call-cc.org/development-process|its wiki page]]. Others
already joined in the discussion of patches on chicken-hackers and you
are more than welcome to join in.

As more people are trying git checkouts, the question of how to
bootstrap a git checkout properly keeps reappearing on the mailing
list. So the steps are:

* Build a [[http://code.call-cc.org|stable release]] or a [[http://code.call-cc.org/dev-snapshots|development snapshot]]
* Checkout chicken-core from git (see [[http://code.call-cc.org]])
* Build a boot-chicken: {{make boot-chicken}}
* Use this chicken to build the real chicken: {{make CHICKEN=./chicken-boot}}
* Install the chicken and run tests: {{make install && make check}}


sparc64
mario, felix
lots of bug fixes, sparc64 support

 * On origin/master:
   * (e1a5437): when conditional branches differ in the number of results, do not merge the results (felix, 2 days ago)
   * (bae7d92): clustering optimization added (felix, 5 weeks ago)
   * (b1907c5): use different default stack-sizes on 32/64 bit platforms (felix, 6 days ago)
   * (986bd3f): disable apply-hack on sparc64 (felix, 6 days ago)
   * (28a36bf): Remove scripts/make-egg-rss-feed.scm (imported into the chicken-infrastructure repo) (Mario Domenech Goulart, 5 days ago)
   * (fbeffd1): scripts/tools.scm: needs regex for `string-match' (Mario Domenech Goulart, 5 days ago)
   * (1fca9c6): use equal? in case MSYSTEM is not defined (makedist.scm) (felix, 6 days ago)
   * (42a8263): don't remove build-info files on make confclean or it is not possible to build from a tarball without preinstalled chicken (felix, 9 days ago)
   * (3bb9645): write debugging output for optimizations into generated source file (felix, 3 weeks ago)
   * (e7fe2fb): allow procedure in operator-position of evaluated list (#705, suggested by John Cowan) (felix, 2 weeks ago)
   * (e2fb31c): fix subtype check for fixed-size list/vector types in scrutinizer (felix, 10 days ago)
   * (fd018df): strip std-envs of non-std identifiers, ignore ##sys#macro-environment when evaluating with an explicit environment (felix, 13 days ago)
   * (fe8f63c): minor change in port-tests.scm to handle case when HOME is not defined (Windows) (felix, 2 weeks ago)
   * (723a469): cut of union types above a certain length (felix, 2 weeks ago)
   * (919a7c7): fixes in build + Windows test script, added note to README (thanks to mario) (felix, 2 weeks ago)
   * (ddb96f9): Also added runtests.bat to manifest. (felix, 3 weeks ago)
   * (0df9bb8): another trivial fix for making boot-chicken work on cmd.exe (felix, 3 weeks ago)
   * (cea668f): trivial Windows-related corrections in rules.make, found by Mario - not yet working, though (felix, 3 weeks ago)
   * (1bc311d): undocumented scrutinize declaration (obsolete), reported by ckeen (felix, 3 weeks ago)
   * (10494f6): fixed stupid paren-matching error (felix, 3 weeks ago)
   * (f9a6702): disable debug output in scrutinizer (felix, 5 weeks ago)
   * (2483f8d): check not necessary for lambda and  warn if using keyword as variable in binding form (felix, 3 weeks ago)
   * (aa6b2bb): cleaning up uninitialized variables in C_reclaim, add debug output to C_mutate on m-stack resize, grow m-stack at higher rate (felix, 3 weeks ago)
   * (1e04156): use write-string port-class method (felix, 3 weeks ago)
   * (59cdaaf): remove obsolete C_h_... allocation functions (pointed out by Joerg Wittenberger) (felix, 3 weeks ago)
   * (159611d): handle macro-aliases when resolving declared identifier, making sure things are done right inside modules. (felix, 4 weeks ago)
   * (0cc88a4): removed obsolete file (felix, 4 weeks ago)
   * (1f56f3a): ##core#type makes subtype-check optional; quit compile when type-mismatches in strict mode (felix, 5 weeks ago)


== 5. Stability is for Chickens

[[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] has set up a Chicken Stability Branch
effort over at [[http://wiki.call-cc.org/stability]]. Jim describes it as "a fork of the Chicken
Scheme core which backports certain useful bugfixes and minor features from the future to 'stable'
maintenance versions". Releases for stability/4.7.0 have version numbers like {{4.7.0.x-st}} (for
example, {{4.7.0.1-st}}). If you're interested in running a stable Chicken in production, we recommend
that you check out Jim's work.



== 6. Hatching Farm

There has been a lot of egg activity over the course of the past few months. I've selected a few,
focussing on ones that were announced or discussed on the [[https://wiki.call-cc.org/mailing-lists|mailing list]].

TODO: Write about the distributed egg repository

By far the most notable thing to have happened is the announcment and implementation of the new
Egg management system.

* [[http://wiki.call-cc.org/users/ivan-raikov|Ivan Raikov]]'s [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00058.html|proposal]]
  to recategorise eggs filed under "Uncategorized or invalid" and "Miscellaneous" was met with enthusiasm with a number of people pitching in to
  help as well as [[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] suggesting that the category list be revisited
  ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00064.html|2011/10/11 08:12Z]]). [[http://wiki.call-cc.org/users/peter-bex|Peter]]
  specifically suggested the addition of a "Communication"s category.

* [[http://wiki.call-cc.org/users/thomas-chust|Thomas Chust]] announced some new eggs: [[http://www.chust.org/fossils/webkit|bindings to WebKitGTK+ and JavaScriptCore]], [[http://www.chust.org/fossils/webgate|webgate: a tiny (S)CGI application framework]] and a [[http://www.chust.org/fossils/berkeley-db|small binding for the BerkelyDB library]]

* [[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]]
  [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00103.html|fixed some bugs]] in the
  [[http://wiki.call-cc.org/eggref/4/cairo|Cairo Egg]] and tagged version 0.1.11.

* [[http://wiki.call-cc.org/users/alyn.post|Alan Post]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00001.html|patched]] the
  [[http://wiki.call-cc.org/eggref/4/sandbox|Sandbox Egg]] to bring it up-to-date with API changes in the Chicken core. His patch was duly accepted.
  
* [[user:moritz-heidkamp|Moritz Heidkamp]] released the [[egg:stemmer|stemmer]] egg which provides bindings for the [[http://snowball.tartarus.org/|Snowball project's]] libstemmer.



== 7. Donations

It's now possible to donate to your favourite scheme (as long as it's Chicken)!

[[http://wiki.call-cc.org/donate]]

We've already had two donations since the account opened a month ago so get donating before it fills up! :-)

Thank you [[http://wiki.call-cc.org/users/sven-hartrumpf|Sven Hartrumpf]] and Ross Lonstein!


== 8. Omelette Recipes

This issue's omelette is the second installment of
[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]]'s
"financial book-keeping in scheme" series. If you missed the first part,
check it out [[http://gazette.call-cc.org/issues/20.html#omelette-recipes|here]].

Now, to make things easy, I parse the ledger by just defining a heap
of procedures and macros, and then diving into the ledger with
eval. It's just Scheme code that, as it is executed, builds up the
data structures. We'll need some helpers to set up third parties
properly:

<enscript>
(define (define-third-party name address group)
  (let* ((balance-account
          (make-account
           (string-append name ".balance")
           'balance
           group
           #f))
         (expenses-account
          (make-account
           (string-append name ".expenses")
           'delta
           'expenses-reclaimed
           #f))
         (third-party
          (make-third-party
           name address balance-account expenses-account)))
    (account-third-party-set! balance-account third-party)
    (account-third-party-set! expenses-account third-party)
    (register-account! balance-account)
    (register-account! expenses-account)
    (set! (hash-table-ref *third-parties* name) third-party)))
</enscript>

Now we can work on a way of representing bills and invoices. A nice
input syntax would be:

<enscript>
(define-third-party "clients.widgetcorp" "123 Any Street" 'clients)
(register-account! (make-account "income.work" 'delta 'income #f))
(register-account! (make-account "expenses.travel" 'delta 'travel #f))
(register-account! (make-account "stock.balance" 'balance 'stock #f))
(register-account! (make-account "cash" 'balance 'cash #f))
(register-account! (make-account "taxes.vat" 'balance 'vat #f))

(invoice "INV005" "clients.widgetcorp" (ymd 2011 04 25)
         (service "income.work" 800 (vat20) "Router setup and installation")
         (sale "stock.balance" 350 (vat20) "1 of LX300 router, serial number 0343248")
         (expense (ymd 2011 03 02) "expenses.travel" "cash" 35 () "Travel to site"))
</enscript>

The idea is that the sales taxes incurred on a line are specified as a
list after the amount. If there's no taxes due, then we use an empty
list. Otherwise we have a list of taxes, which are either plain tax
names (to have the system compute the tax due itself) or two-element
lists joining a tax name to a precomputed amount (often, when we pass
on an expense, we know the tax we paid as it's on the receipt, so we
should use that (even if they made a mistake working it out) rather
than calculating our own).

A nice way to implement that might be to make "invoice" a macro that
absorbs its first three arguments as an invoice code, the name of the
third party to invoice, and the date; then treats the rest as a body
to be wrapped in a dynamic environment in which a parameter allows
{{sale}}, {{expense}}, and {{service}} to add lines to the
invoice. This is easily arranged:

<enscript>
(define-record date year month day)

(define-record invoice
  name date third-party lines)

(define (register-line! invoice line)
  (invoice-lines-set! invoice
                      (cons line (invoice-lines invoice))))

;; Compute sales taxes
(define (compute-tax tax amount)
  (case tax
    ((vat20) (* 0.20 amount)) ;; Current UK rate
    ((vat15) (* 0.15 amount)) ;; Previous UK rate
    ((vat175) (* 0.175 amount)))) ;; Previous UK rate

;; Expand a list of taxes, some of which might be bare symbols
;; naming taxes to work out, or (<tax> <amount>) lists for
;; ready-computed taxes, into an alist of tax names to tax amounts
(define (resolve-taxes amount taxes)
  (map (lambda (tax-desc)
         (if (list? tax-desc)
             (cons (car tax-desc) (cadr tax-desc))
             (cons tax-desc (compute-tax tax-desc amount))))
       taxes))

(define-record invoice-service-line
  income-account amount taxes description)

(define-syntax service
  (syntax-rules ()
    ((service income-account amount taxes description)
     (register-service! (*current-invoice*) income-account amount 'taxes description))))

(define (register-service! invoice income-account amount taxes description)
  (let ((service
         (make-invoice-service-line
          (find-account income-account)
          amount
          (resolve-taxes amount taxes)
          description)))
    (register-line! invoice service)))

(define-record invoice-sale-line
  stock-account amount taxes description)

(define (register-sale! invoice stock-account amount taxes description)
  (let ((sale
         (make-invoice-sale-line
          (find-account stock-account)
          amount
          (resolve-taxes amount taxes)
          description)))
    (register-line! invoice sale)))

(define-syntax sale
  (syntax-rules ()
    ((sale stock-account amount taxes description)
     (register-sale! (*current-invoice*) stock-account amount 'taxes description))))

(define-record invoice-expense-line
  expense-account payment-account amount taxes description)

(define (register-expense! invoice date expense-account payment-account amount taxes description)
  (let ((expense
         (make-invoice-expense-line
          (find-account expense-account)
          (find-account payment-account)
          amount
          (resolve-taxes amount taxes)
          description)))
    (register-line! invoice expense)))

(define-syntax expense
  (syntax-rules ()
    ((expense (ymd year month day) expense-account payment-account amount taxes description)
     (register-expense!
      (*current-invoice*)
      (make-date year month day)
      expense-account
      payment-account
      amount
      'taxes
      description))))

(define *current-invoice* (make-parameter #f))
(define *invoices* (make-hash-table))

(define-syntax invoice
  (syntax-rules (service sale expense)
    ((invoice name third-party (ymd year month day) body ...)
     (let ((inv
            (make-invoice name
                          (make-date year month day)
                          (hash-table-ref *third-parties* third-party)
                          '())))
       (parameterize
        ((*current-invoice* inv))
        (begin body ...))
       (set! (hash-table-ref *invoices* name) inv)
       (generate-invoice-transactions! inv)))))
</enscript>

We end the expansion of the {{invoice}} macro with a call to
{{generate-invoice-transactions!}}, which will do the task of creating
the double-entry transactions for the invoice. Other types of summary
structure can be added by calling additional generator procedures at
this point. This is largely a matter of going through the invoice
lines, handling them on a case-by-case basis to generate lists of
transaction splits that we can append together to generate the invoice
transaction. The case of expense lines is interesting, in that an
extra transaction has to be generated for each expense, to record its
initial spending, as well as a split to record the expense being
claimed in the invoice.

For now, let's just handle one case:

<enscript>
(define (generate-invoice-transactions! inv)
  (register-txn! (make-txn
    (invoice-date inv)
    (string-append "Invoice " (invoice-name inv) " for "
                   (third-party-full-name (invoice-third-party inv)))
    (let ((txn-balance-account
           (third-party-balance-account
            (invoice-third-party inv))))
      (flatten
       (map
        (lambda (line)
          (cond
           ((invoice-expense-line? line)
            (list)) ;; FIXME: Not implemented
           ((invoice-sale-line? line)
            (list)) ;; FIXME: Not implemented
           ((invoice-service-line? line)
            (list
             (make-txn-split
              (invoice-service-line-income-account line)
              (- (invoice-service-line-amount line))
              (invoice-service-line-description line))
             (make-tax-splits
              (invoice-service-line-taxes line)
              txn-balance-account)
             (make-txn-split
              txn-balance-account
              (invoice-service-line-amount line)
              #f)))))
        (invoice-lines inv)))))))

(define (make-tax-splits taxes txn-balance-account)
  (map (lambda (tax)
         (let ((tax-type (car tax))
               (tax-amount (cdr tax)))
           (case tax-type
             ((vat20 vat15 vat175)
              (list
               (make-txn-split
                (find-account "taxes.vat")
                (- tax-amount)
                #f)
               (make-txn-split
                txn-balance-account
                tax-amount
                #f))))))
       taxes))
</enscript>

Feeding in the above example invoice, then checking out the resulting
double-entry transaction list, shows that it worked:

<enscript>
(hash-table-for-each *txns*
                     (lambda (date txns)
                       (for-each
                        (lambda (txn)
                          (printf "Date: ~A Desc: ~A\n"
                                  (txn-date txn)
                                  (txn-description txn))
                          (for-each (lambda (split)
                                      (printf "Acct: ~A Delta: ~A Notes: ~A\n"
                                              (account-name (txn-split-account split))
                                              (txn-split-amount split)
                                              (txn-split-notes split))) (txn-splits txn)))
                        txns)))
</enscript>

   date: #<date> Desc: Invoice INV005 for Widget Corp
   Acct: income.work Delta: -800 Notes: Router setup and installation
   Acct: taxes.vat Delta: -160.0 Notes: #f
   Acct: clients.widgetcorp.balance Delta: 160.0 Notes: #f
   Acct: clients.widgetcorp.balance Delta: 800 Notes: #f

We've ended up with multiple splits for the same account, as we record
that both VAT and the money due for the service are to come from the
client's balance account - and other splits will add plenty more. To
fix this, we need to write a procedure that canonicalises a list of
splits, and call that on the splits before calling
{{make-txn}}. Canonicalisation consists of finding all the splits that
refer to the same account and have the same notes (be it {{#f}} or a
string) and merging them into one with the total of the amounts. But
I'll leave that (along with implementing bills, payments, and some
actual reports) as an exercise to the reader... It's easy to imagine
how to generate a VAT report from the list of transactions, by
filtering them for membership of the required date range and looking
for splits involving "taxes.vat", or to generate a nicely formatted
invoice by extracting a single invoice record, or to work out the
balance of an account at any point in time by adding up all the
transaction splits that involve it up to that point in time. Also, the
core engine needs to be wrapped up in a module that only exposes the
required bindings, and hides internals.

Having automated one's book-keeping and financial reporting, many
operations (such as the VAT returns) can be done without involving an
accountant; in my case, the accountant is only needed to help with the
annual corporation tax computation and filing of official accounts,
which requires deep understanding of the UK tax system to do
everything properly. Having said that, if I studied the system
properly (and tracked the changes each year), I'm sure I could
automate that, too...

== 9. About the Chicken Gazette

The Gazette is produced occasionally by a volunteer from the Chicken
community. The latest issue can be found at
[[http://gazette.call-cc.org]] or you can follow it in your feed
reader at [[http://gazette.call-cc.org/feed.atom]]. If you'd like to
write an issue, [[http://wiki.call-cc.org/gazette|consult the wiki]]
for the schedule and instructions!