source: project/gazette/src/issues/21.wiki @ 25553

Last change on this file since 25553 was 25553, checked in by Christian Kellermann, 9 years ago

gazette #21: add core dev section, pending comments from felix

File size: 35.5 KB
Line 
1((title . "Issue 21")
2 (authors "Andy Bennett")
3 (date . 1321465864))
4
5== 0. Introduction
6
7Welcome to the 21st issue of the Chicken Gazette. While the gazette
8was on hiatus the chicken community was definitely not.
9
10With the stabilisation of Chicken 4.7 well underway since the last
11gazette...
12
13Some people have also started to set up salmonella farms and C-Keen
14has completed his "Chicken Zoo".
15
16Enjoy!
17
18
19== 1. Chickens Out & About
20
21=== Nuremberg (December 2011)
22[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] wrote
23([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00114.html|2011/10/22 17:38Z]]) to
24confirm the planned Chicken Hacking Sprint scheduled for the beginning of December.
25
26Interested and available Chickenauts will meet in Nuremberg on the weekend of 9th, 10th and 11th
27of December. If you haven't done so already, you can let us know that you will be joining us by
28placing a note at the foot of the [[https://wiki.call-cc.org/chicken-sprint-nuremberg|wiki page]].
29If you can't attend in person then we'll be keeping an eye on IRC so feel free to book yourself
30a slot in your cave at home and join us throughout the weekend and into the evenings, Central European Time.
31
32For those of you keeping score, this will bring the total number of Chicken meets this year to
335 (FOSDEM, Cologne, FrOSCon, T-DOSE and Nuremberg).
34
35
36=== Cologne (March 2011)
37
38In March, Chickens met up at the Chaos Computer Club in Cologne, Germany for a Chicken Hacking Weekend. Here's
39[[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]]'s report on what they got up to.
40
41The usual suspects [[user:peter-bex|Peter Bex]],
42[[user:felix-winkelmann|Felix Winkelmann]],
43[[user:christian-kellermann|Christian Kellermann]] and me met for two
44and a half days of exchanging ideas and hacking. While we also fixed a
45good share of bugs our main goal for this weekend was to create a
46working prototype of a distributed egg system. By the end of Sunday,
47mostly thanks to Peter's effort, we had something that was already
48pretty similar to what is in use today (see
49[[http://lists.nongnu.org/archive/html/chicken-users/2011-03/msg00160.html|his announcement on the mailing list]]). Apart
50from that some interested visitors stopped by every now and then. Some
51nice chats, food and the occasional beers were had which hopefully
52helped to spread the Chicken word.
53
54
55=== FrOSCon (August 2011)
56At FrOSCon in August we sported not only a project booth but a presentation room as well so we took the
57opportunity to lay on a couple of talks for the delegates.
58
59[[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]] presented
60"[[http://programm.froscon.org/2011/events/869.html|An introduction to Lisp]]:
61Why to talk to computers in parentheses" whilst
62[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] offered
63"[[http://programm.froscon.org/2011/events/867.html|A guided tour through the republic of CHICKEN]]:
64get up to speed with the practical scheme implementation".
65[[http://wiki.call-cc.org/users/christian-kellermann|Christian]] has been keeping his talk up-to-date and
66you can find the latest version [[http://wiki.call-cc.org/chickenista-guide|on the wiki]].
67
68Naturally, there are [[http://www.call-cc.org/pictures/froscon2011/|pictures]].
69
70
71=== T-DOSE 2011 (November 2011)
72European Chickens met up at T-DOSE in Eindhoven, The Netherlands on the weekend of the 5th, 6th and 7th of
73November. [[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] arranged for us to contribute a "Chicken
74Scheme Project Booth" and we subsequently lured a few Dutch hackers into #chicken on IRC.
75
76[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] knocked up some impressive demos
77of [[http://en.wikipedia.org/wiki/Pong|PONG]] using Chicken's [[http://wiki.call-cc.org/eggref/4/cairo|Cairo Egg]].
78
79[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]] finished off the tests for his
80[[http://wiki.call-cc.org/eggref/4/ugarit|Ugarit]] backup egg which allowed him to release version 1.0.
81
82...and [[http://wiki.call-cc.org/users/moritz-heidkamp|Moritz Heidkamp]] started on a replacement for the
83[[http://wiki.call-cc.org/eggref/4/environments|Environments Egg]] as it has unfortunately has been overtaken
84by developments in the Chicken development branch.
85
86Pictures are up in the [[http://www.call-cc.org/pictures/t-dose2011/|usual place]] on the Chicken website.
87
88
89
90== 2. Chicken Talk
91
92=== R7RS to be dedicated to the memory of John McCarthy
93
94Matt Welland wrote in with the sad news that the "Father of Lisp", John McCarthy had sadly passed away ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00115.html|2011/10/25 15:59Z]]). [[http://wiki.call-cc.org/users/john-cowan|John Cowan]], member of the R7RS working group, told us that R7RS,
95the upcoming version of the Scheme standard, would be dedicated to his memory ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00116.html|16:22Z]])
96
97
98=== Parallel build patch for Chicken Makefile
99
100A lively discussion broke out on the [[https://wiki.call-cc.org/mailing-lists|Chicken Users mailing list]] regarding
101[[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00025.html|parallelisation of the Chicken build process]].
102
103Vitaly Magerya supplied a [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00025.html|patch]] (2011/10/06 12:37Z) to fix some problems
104he'd been having with builds failing when specifying the {{-j}} to {{gmake}}.
105[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] noted that this functionality was already being
106addressed by [[http://bugs.call-cc.org/ticket/526|ticket 526]]
107([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00027.html|2011/10/06 12:39Z]]) and Vitaly was keen to see it committed
108soonly as he maintains the FreeBSD port and people had been asking him for parallel builds ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00028.html|2011/10/06 14:24Z]]). Whilst still keen, [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00029.html|advised caution]] as he wanted to see it proven ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00031.html|2011/10/06 14:40Z]]).
109
110Vitaly was keen to push things forward ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00033.html|2011/10/06 15:45Z]]) and
111offered continued testing under FreeBSD 8.2-RELEASE on x86_64. He noted that, as far as he could tell, {{gmake check}} passed when using his
112patch, modified with [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00026.html|Moritz Wilhelmy's suggestions]]. Moreover, he
113offered to try the patch from [[http://bugs.call-cc.org/ticket/526|ticket 526]] as well.
114
115[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00034.html|suggested]]
116that using the [[http://bugs.call-cc.org/ticket/526|ticket 526]] patch was a good idea as it had, as far as he knew, received testing under Linux and MacOSX.
117
118Toby Thain spoke in favour of the patch, saying that a working parallel build gives confidence that the Makefile is, in fact, correct.
119He offered to test on three platforms, namely Solaris 10 SPARC SMP and both PowerPC and Intel version of MacOSX SMP.
120
121Vitaly updated the [[http://bugs.call-cc.org/ticket/526|ticket 526]] patch to the then current sources and confirmed that the build worked at {{-j8}}
122and the tests passed in his FreeBSD 8.2-RELEASE, x86_64 environment. He did, however, note that parallelisation does not work for the install phase
123([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00048.html|2011/10/08 12:29Z]]).
124
125Toby Thain reported successful builds on the 64-bit version MacOSX-10.6 as well as MacOSX-10.5 on a dual-processor G5. He noted "{{make -j8}}
126reduces the make step from 3:39 to 0:47 on my 8-core system". Unfortunately {{make check}} did not pass on either system, with or without the patch
127([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00071.html|2011/10/12 01:59Z]] &
128[[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00100.html|2011/10/15 19:53Z]])
129
130[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] noted that it might necessary to install the compiled binaries
131([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00101.html|2011/10/15 20:07Z]]) however, Toby did not report back as to whether this
132had any effect on the results of the tests.
133
134
135=== 64-bit SPARC Build
136
137Nicolas Pelletier came to the [[https://wiki.call-cc.org/mailing-lists|mailing list]]
138unable to build for Solaris 10 running on SPARC64 due to a missing {{apply-hack.S}}
139([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00012.html|2011/11/09 11:48Z]]).
140
141It turned out that there was no SPARC64 support for Chicken at all and [[http://wiki.call-cc.org/users/john-cowan|John Cowan]]
142came to the rescue ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00013.html|15:36Z]])
143with instructions for how to add it to the supported architectures list and disable the
144apply hack. [[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] chimed in with affirmations
145around the utility of using Chicken without an apply hack ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00014.html|15:54Z]])
146
147Nicolas struggled with disabling the apply hack ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00015.html|2011/11/10 11:10Z]])
148due to the Two Lies Of The README. Over the course of the next few minutes [[http://wiki.call-cc.org/users/christian-kellermann|Christian]]
149and [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]]]
150confirmed that the lies, which related to the way bootstrap compilers are distributed and invoked, had
151already been fixed in the development versions.
152
153Meanwhile, Toby Thain tried the build on his SPARC64 ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00020.html|12:40Z]])
154and discovered that GNU make 3.82 is required as well as the addition of {{-lrt}} in {{Makefile.solaris}}. Toby's
155build eventually succeeded and appeared to be good, albeit having only used 1 of his 4 CPUs and with a couple of {{make check}} errors.
156
157Later on the Thursday, [[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] noted ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00021.html|16:47Z]])
158that the [[http://bugs.call-cc.org/raw-attachment/ticket/526/parallel-build-2.diff.txt|parallel build patch]] in
159[[http://bugs.call-cc.org/ticket/526|ticket 526]] still works on Chicken 4.7.0.3-st. Toby [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00022.html|confirmed]]
160that the patch did indeed work on Solaris and build time was reduced to 10 just of his Earth minutes from the original 40.
161Of course, user time across the four CPUs was still in the region of 40 minutes in total. :-)
162
163To round everything off, Toby offered [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00028.html|binary SYSV packages for Solaris 10 on SPARC64]].
164Contact him via the [[https://wiki.call-cc.org/mailing-lists|mailing list]] if you're interested.
165
166
167=== Static Linking & Standalone Binaries.
168
169Serg Kozhemyakin [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00105.html|asked]] whether it is possible to build static
170binaries after having trouble using the {{-static}} flag to {{csc}}. He also wondered about building standalone binaries for Windows.
171[[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00106.html|advised]]
172against static linking as many of the eggs cannot cope with it. He did, however, suggest that the {{-deploy}} option may well solve both
173issues as it ensures that all the DLLs (including the Chicken runtime and other shared objects) are bundled together into a single directory.
174
175
176=== Tracing with {{,tr}} moved to an egg
177
178[[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00007.html|helped]]
179Curtis Cooley in his [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00006.html|search for the {{,tr}} command]].
180The functionality had been moved from the core into an egg in Chicken 4.3.0.
181
182
183=== {{gmake check}} sometimes fails.
184
185Pekka Niiranen wrote to tell us about
186[[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00009.html|problems with {{gmake check}} in Chicken 4.7.0-st]]
187on OpenBSD 5.0.
188
189[[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00010.html|said]]
190that one of the failures had been fixed in the stability branch after the Chicken 4.7.0-st release and he filed
191[[https://bugs.call-cc.org/ticket/724|ticket 724]] for the segmentation fault issue.
192
193[[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] mentioned that the [[https://bugs.call-cc.org/ticket/724|ticket 724]] segmentation
194fault had also been observed under NetBSD and Arch Linux. He said that "It's known to be broken" but that it is not properly reproducible
195as it only happens some of the time ([[http://lists.nongnu.org/archive/html/chicken-users/2011-11/msg00011.html|2011/11/06 09:51|]]).
196
197
198== 3. Salmonella and other tests
199Salmonella is Chicken's egg testing framework.
200
201For some time we have had daily salmonella runs of the master development
202version of Chicken under Linux on x86. Throughout November, MacOSX-x86_64 has also
203been regularly experiencing the runs. [[http://wiki.call-cc.org/users/mario-domenech-goulart|Mario Domenech Goulart]] who maintains the testing
204effort at [[http://tests.call-cc.org/]], as well as the [[https://github.com/mario-goulart/salmonella|Salmonella codebase]]
205itself, is always on the lookout for more feeds. We are aware that testing on
206other platforms is being organised but if you've got a private Salmonella
207running regularly on an interesting operating system or version of Chicken then
208give us a shout and we'll see if we can get it included with the other reports
209on [[http://tests.call-cc.org/]]
210
211There is also [[https://github.com/mario-goulart/salmonella|an entirely new version of Salmonella]] over at github so test that too. The new version is more modular -- some features that were previously built in salmonella are now independent eggs (e.g., [[https://github.com/mario-goulart/salmonella-html-report|salmonella-html-report]] and [[https://github.com/mario-goulart/salmonella-feeds|salmonella-feeds]].  There's also a new egg that provides a new feature: [[https://github.com/mario-goulart/salmonella-diff|salmonella-diff]] which generates HTML output to render differences between two salmonella logs.
212
213Here's short list of new salmonella features:
214
215* egg lint mode: salmonella can now easily check for common egg mistakes before the code hits the egg repository.  Just run {{salmonella --this-egg}} from the egg directory
216* a new execution mode that can significantly reduce salmonella execution times.  In this mode, salmonella won't set the temporary egg installation directory empty before installing new eggs.  While this mode won't spot egg dependencies problems, it can drastically effect the execution time
217* a new tool to take advantage of multi-core systems ({{salmonella-epidemy}})
218* a simple text mode log file viewer ({{salmonella-log-viewer}})
219* [[https://github.com/mario-goulart/salmonella-html-report|salmonella-html-report]] generates reverse dependencies graphs.  That can be useful, for
220example, to ilustrate how many eggs depend on a given egg.
221
222
223[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]] is one of the people who has been setting up a Chicken testing
224environment. After [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00068.html|some discussion]], he chose Chicken 4.5.0 under
225NetBSD ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00035.html|2011/10/06 21:18Z]]). He's been keeping track of all the
226Egg dependencies and plans to publish them as a list of NetBSD pkgsrc packages.
227
228Thanks to [[http://www.bevuta.com/|bevuta IT GmbH]], the CHICKEN test
229infrastructure will soon have more machines available to test the
230CHICKEN compiler, tools and eggs.  A new salmonella environment for
231Linux/x86-64 is currently being set up.
232
233In addition to salmonella, work on the
234[[http://wiki.call-cc.org/playground|Chicken Playground]] project has
235been resumed.  The Chicken Playground project provides environments
236that can be used by tools like {{chroot}} to test CHICKEN tools and
237eggs.  Those environments contain tools and libraries required to
238build and test most eggs.  Currently a Linux/x86 environment is
239available. A Linux/x86-64 environment is under development and should
240be available soon.
241
242The new test infrastructure, based on new versions of salmonella and
243Chicken Playground and running on new platforms will probably go into
244production during the Nuremberg hackaton.
245
246
247== 4. Core Development
248
249Since the last issue two main things happened in the chicken core:
250Improvements to the scrutinizer and hand in hand to that so called
251specialisation (or partial evaluation). What does this do? The first
252lead to a flow analysis pass in the compiler. This made it possible to
253give the programmer more hints at potential programming errors, such
254as unused code, unreachable branches, and type misuses. The latter is
255done by having a central database of core procedure types. It is
256installed as types.db and has been there on your chicken system for a
257quite a while now. With more improvements going on the false positives
258(warnings about non-issues) are much fewer than before, also lots
259additional procedures and types have been added. One example for this
260are procedures using ports which is now even checked for the right
261port type (input or output port).
262
263Now that the flow analysis pass is already there it can also be used
264to optimise code, usually resulting in omitting redundant code such as
265superfluous type checks. In the end this should lead to faster code
266altogether. Over the last months more and more specialisations have
267been added. For more information on specialisation see the respective
268section at
269[[http://library.readscheme.org/page10.html|readscheme.org]].
270
271A couple of months ago the CHICKEN team has formalised the development
272process a bit as can be seen on
273[[http://wiki.call-cc.org/development-process|its wiki page]]. Others
274already joined in the discussion of patches on chicken-hackers and you
275are more than welcome to join in.
276
277As more people are trying git checkouts, the question of how to
278bootstrap a git checkout properly keeps reappearing on the mailing
279list. So the steps are:
280
281* Build a stable or development release
282* Checkout chicken-core from git (see [[http://code.call-cc.org]])
283* Build a boot-chicken: {{make boot-chicken}}
284* Use this chicken to build the real chicken: {{make CHICKEN=./chicken-boot}}
285* Install the chicken and run tests: {{make install && make check}}
286
287
288sparc64
289mario, felix
290lots of bug fixes, sparc64 support
291
292 * On origin/master:
293   * (e1a5437): when conditional branches differ in the number of results, do not merge the results (felix, 2 days ago)
294   * (bae7d92): clustering optimization added (felix, 5 weeks ago)
295   * (b1907c5): use different default stack-sizes on 32/64 bit platforms (felix, 6 days ago)
296   * (986bd3f): disable apply-hack on sparc64 (felix, 6 days ago)
297   * (28a36bf): Remove scripts/make-egg-rss-feed.scm (imported into the chicken-infrastructure repo) (Mario Domenech Goulart, 5 days ago)
298   * (fbeffd1): scripts/tools.scm: needs regex for `string-match' (Mario Domenech Goulart, 5 days ago)
299   * (1fca9c6): use equal? in case MSYSTEM is not defined (makedist.scm) (felix, 6 days ago)
300   * (42a8263): don't remove build-info files on make confclean or it is not possible to build from a tarball without preinstalled chicken (felix, 9 days ago)
301   * (3bb9645): write debugging output for optimizations into generated source file (felix, 3 weeks ago)
302   * (e7fe2fb): allow procedure in operator-position of evaluated list (#705, suggested by John Cowan) (felix, 2 weeks ago)
303   * (e2fb31c): fix subtype check for fixed-size list/vector types in scrutinizer (felix, 10 days ago)
304   * (fd018df): strip std-envs of non-std identifiers, ignore ##sys#macro-environment when evaluating with an explicit environment (felix, 13 days ago)
305   * (fe8f63c): minor change in port-tests.scm to handle case when HOME is not defined (Windows) (felix, 2 weeks ago)
306   * (723a469): cut of union types above a certain length (felix, 2 weeks ago)
307   * (919a7c7): fixes in build + Windows test script, added note to README (thanks to mario) (felix, 2 weeks ago)
308   * (ddb96f9): Also added runtests.bat to manifest. (felix, 3 weeks ago)
309   * (0df9bb8): another trivial fix for making boot-chicken work on cmd.exe (felix, 3 weeks ago)
310   * (cea668f): trivial Windows-related corrections in rules.make, found by Mario - not yet working, though (felix, 3 weeks ago)
311   * (1bc311d): undocumented scrutinize declaration (obsolete), reported by ckeen (felix, 3 weeks ago)
312   * (10494f6): fixed stupid paren-matching error (felix, 3 weeks ago)
313   * (f9a6702): disable debug output in scrutinizer (felix, 5 weeks ago)
314   * (2483f8d): check not necessary for lambda and  warn if using keyword as variable in binding form (felix, 3 weeks ago)
315   * (aa6b2bb): cleaning up uninitialized variables in C_reclaim, add debug output to C_mutate on m-stack resize, grow m-stack at higher rate (felix, 3 weeks ago)
316   * (1e04156): use write-string port-class method (felix, 3 weeks ago)
317   * (59cdaaf): remove obsolete C_h_... allocation functions (pointed out by Joerg Wittenberger) (felix, 3 weeks ago)
318   * (159611d): handle macro-aliases when resolving declared identifier, making sure things are done right inside modules. (felix, 4 weeks ago)
319   * (0cc88a4): removed obsolete file (felix, 4 weeks ago)
320   * (1f56f3a): ##core#type makes subtype-check optional; quit compile when type-mismatches in strict mode (felix, 5 weeks ago)
321
322
323== 5. Stability is for Chickens
324
325[[http://wiki.call-cc.org/users/jim-ursetto|Jim Ursetto]] has set up a Chicken Stability Branch
326effort over at [[http://wiki.call-cc.org/stability]]. Jim describes it as "a fork of the Chicken
327Scheme core which backports certain useful bugfixes and minor features from the future to 'stable'
328maintenance versions". Releases for stability/4.7.0 have version numbers like {{4.7.0.x-st}} (for
329example, {{4.7.0.1-st}}). If you're interested in running a stable Chicken in production, we recommend
330that you check out Jim's work.
331
332
333
334== 6. Hatching Farm
335
336There has been a lot of egg activity over the course of the past few months. I've selected a few,
337focussing on ones that were announced or discussed on the [[https://wiki.call-cc.org/mailing-lists|mailing list]].
338
339TODO: Write about the distributed egg repository
340
341By far the most notable thing to have happened is the announcment and implementation of the new
342Egg management system.
343
344* [[http://wiki.call-cc.org/users/ivan-raikov|Ivan Raikov]]'s [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00058.html|proposal]]
345  to recategorise eggs filed under "Uncategorized or invalid" and "Miscellaneous" was met with enthusiasm with a number of people pitching in to
346  help as well as [[http://wiki.call-cc.org/users/peter-bex|Peter Bex]] suggesting that the category list be revisited
347  ([[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00064.html|2011/10/11 08:12Z]]). [[http://wiki.call-cc.org/users/peter-bex|Peter]]
348  specifically suggested the addition of a "Communication"s category.
349
350* [[http://wiki.call-cc.org/users/thomas-chust|Thomas Chust]] announced some new eggs: [[http://www.chust.org/fossils/webkit|bindings to WebKitGTK+ and JavaScriptCore]], [[http://www.chust.org/fossils/webgate|webgate: a tiny (S)CGI application framework]] and a [[http://www.chust.org/fossils/berkeley-db|small binding for the BerkelyDB library]]
351
352* [[http://wiki.call-cc.org/users/christian-kellermann|Christian Kellermann]]
353  [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00103.html|fixed some bugs]] in the
354  [[http://wiki.call-cc.org/eggref/4/cairo|Cairo Egg]] and tagged version 0.1.11.
355
356* [[http://wiki.call-cc.org/users/alyn.post|Alan Post]] [[http://lists.nongnu.org/archive/html/chicken-users/2011-10/msg00001.html|patched]] the
357  [[http://wiki.call-cc.org/eggref/4/sandbox|Sandbox Egg]] to bring it up-to-date with API changes in the Chicken core. His patch was duly accepted.
358 
359* [[user:moritz-heidkamp|Moritz Heidkamp]] released the [[egg:stemmer|stemmer]] egg which provides bindings for the [[http://snowball.tartarus.org/|Snowball project's]] libstemmer.
360
361
362
363== 7. Donations
364
365It's now possible to donate to your favourite scheme (as long as it's Chicken)!
366
367[[http://wiki.call-cc.org/donate]]
368
369We've already had two donations since the account opened a month ago so get donating before it fills up! :-)
370
371Thank you [[http://wiki.call-cc.org/users/sven-hartrumpf|Sven Hartrumpf]] and Ross Lonstein!
372
373
374== 8. Omelette Recipes
375
376This issue's omelette is the second installment of
377[[http://wiki.call-cc.org/users/alaric-blagrave-snellpym|Alaric Snell-Pym]]'s
378"financial book-keeping in scheme" series. If you missed the first part,
379check it out [[http://gazette.call-cc.org/issues/20.html#omelette-recipes|here]].
380
381Now, to make things easy, I parse the ledger by just defining a heap
382of procedures and macros, and then diving into the ledger with
383eval. It's just Scheme code that, as it is executed, builds up the
384data structures. We'll need some helpers to set up third parties
385properly:
386
387<enscript>
388(define (define-third-party name address group)
389        (let* ((balance-account
390                (make-account
391                 (string-append name ".balance")
392                 'balance
393                 group
394                 #f))
395               (expenses-account
396                (make-account
397                 (string-append name ".expenses")
398                 'delta
399                 'expenses-reclaimed
400                 #f))
401               (third-party
402                (make-third-party
403                 name address balance-account expenses-account)))
404          (account-third-party-set! balance-account third-party)
405          (account-third-party-set! expenses-account third-party)
406          (register-account! balance-account)
407          (register-account! expenses-account)
408          (set! (hash-table-ref *third-parties* name) third-party)))
409</enscript>
410
411Now we can work on a way of representing bills and invoices. A nice
412input syntax would be:
413
414<enscript>
415(define-third-party "clients.widgetcorp" "123 Any Street" 'clients)
416(register-account! (make-account "income.work" 'delta 'income #f))
417(register-account! (make-account "expenses.travel" 'delta 'travel #f))
418(register-account! (make-account "stock.balance" 'balance 'stock #f))
419(register-account! (make-account "cash" 'balance 'cash #f))
420(register-account! (make-account "taxes.vat" 'balance 'vat #f))
421
422(invoice "INV005" "clients.widgetcorp" (ymd 2011 04 25)
423         (service "income.work" 800 (vat20) "Router setup and installation")
424         (sale "stock.balance" 350 (vat20) "1 of LX300 router, serial number 0343248")
425         (expense (ymd 2011 03 02) "expenses.travel" "cash" 35 () "Travel to site"))
426</enscript>
427
428The idea is that the sales taxes incurred on a line are specified as a
429list after the amount. If there's no taxes due, then we use an empty
430list. Otherwise we have a list of taxes, which are either plain tax
431names (to have the system compute the tax due itself) or two-element
432lists joining a tax name to a precomputed amount (often, when we pass
433on an expense, we know the tax we paid as it's on the receipt, so we
434should use that (even if they made a mistake working it out) rather
435than calculating our own).
436
437A nice way to implement that might be to make "invoice" a macro that
438absorbs its first three arguments as an invoice code, the name of the
439third party to invoice, and the date; then treats the rest as a body
440to be wrapped in a dynamic environment in which a parameter allows
441{{sale}}, {{expense}}, and {{service}} to add lines to the
442invoice. This is easily arranged:
443
444<enscript>
445(define-record date year month day)
446
447(define-record invoice
448  name date third-party lines)
449
450(define (register-line! invoice line)
451  (invoice-lines-set! invoice
452                      (cons line (invoice-lines invoice))))
453
454;; Compute sales taxes
455(define (compute-tax tax amount)
456  (case tax
457    ((vat20) (* 0.20 amount)) ;; Current UK rate
458    ((vat15) (* 0.15 amount)) ;; Previous UK rate
459    ((vat175) (* 0.175 amount)))) ;; Previous UK rate
460
461;; Expand a list of taxes, some of which might be bare symbols
462;; naming taxes to work out, or (<tax> <amount>) lists for
463;; ready-computed taxes, into an alist of tax names to tax amounts
464(define (resolve-taxes amount taxes)
465  (map (lambda (tax-desc)
466         (if (list? tax-desc)
467             (cons (car tax-desc) (cadr tax-desc))
468             (cons tax-desc (compute-tax tax-desc amount))))
469       taxes))
470
471(define-record invoice-service-line
472  income-account amount taxes description)
473
474(define-syntax service
475  (syntax-rules ()
476    ((service income-account amount taxes description)
477     (register-service! (*current-invoice*) income-account amount 'taxes description))))
478
479(define (register-service! invoice income-account amount taxes description)
480  (let ((service
481         (make-invoice-service-line
482          (find-account income-account)
483          amount
484          (resolve-taxes amount taxes)
485          description)))
486    (register-line! invoice service)))
487
488(define-record invoice-sale-line
489  stock-account amount taxes description)
490
491(define (register-sale! invoice stock-account amount taxes description)
492  (let ((sale
493         (make-invoice-sale-line
494          (find-account stock-account)
495          amount
496          (resolve-taxes amount taxes)
497          description)))
498    (register-line! invoice sale)))
499
500(define-syntax sale
501  (syntax-rules ()
502    ((sale stock-account amount taxes description)
503     (register-sale! (*current-invoice*) stock-account amount 'taxes description))))
504
505(define-record invoice-expense-line
506  expense-account payment-account amount taxes description)
507
508(define (register-expense! invoice date expense-account payment-account amount taxes description)
509  (let ((expense
510         (make-invoice-expense-line
511          (find-account expense-account)
512          (find-account payment-account)
513          amount
514          (resolve-taxes amount taxes)
515          description)))
516    (register-line! invoice expense)))
517
518(define-syntax expense
519  (syntax-rules ()
520    ((expense (ymd year month day) expense-account payment-account amount taxes description)
521     (register-expense!
522      (*current-invoice*)
523      (make-date year month day)
524      expense-account
525      payment-account
526      amount
527      'taxes
528      description))))
529
530(define *current-invoice* (make-parameter #f))
531(define *invoices* (make-hash-table))
532
533(define-syntax invoice
534  (syntax-rules (service sale expense)
535    ((invoice name third-party (ymd year month day) body ...)
536     (let ((inv
537            (make-invoice name
538                          (make-date year month day)
539                          (hash-table-ref *third-parties* third-party)
540                          '())))
541       (parameterize
542        ((*current-invoice* inv))
543        (begin body ...))
544       (set! (hash-table-ref *invoices* name) inv)
545       (generate-invoice-transactions! inv)))))
546</enscript>
547
548We end the expansion of the {{invoice}} macro with a call to
549{{generate-invoice-transactions!}}, which will do the task of creating
550the double-entry transactions for the invoice. Other types of summary
551structure can be added by calling additional generator procedures at
552this point. This is largely a matter of going through the invoice
553lines, handling them on a case-by-case basis to generate lists of
554transaction splits that we can append together to generate the invoice
555transaction. The case of expense lines is interesting, in that an
556extra transaction has to be generated for each expense, to record its
557initial spending, as well as a split to record the expense being
558claimed in the invoice.
559
560For now, let's just handle one case:
561
562<enscript>
563(define (generate-invoice-transactions! inv)
564  (register-txn! (make-txn
565    (invoice-date inv)
566    (string-append "Invoice " (invoice-name inv) " for "
567                   (third-party-full-name (invoice-third-party inv)))
568    (let ((txn-balance-account
569           (third-party-balance-account
570            (invoice-third-party inv))))
571      (flatten
572       (map
573        (lambda (line)
574          (cond
575           ((invoice-expense-line? line)
576            (list)) ;; FIXME: Not implemented
577           ((invoice-sale-line? line)
578            (list)) ;; FIXME: Not implemented
579           ((invoice-service-line? line)
580            (list
581             (make-txn-split
582              (invoice-service-line-income-account line)
583              (- (invoice-service-line-amount line))
584              (invoice-service-line-description line))
585             (make-tax-splits
586              (invoice-service-line-taxes line)
587              txn-balance-account)
588             (make-txn-split
589              txn-balance-account
590              (invoice-service-line-amount line)
591              #f)))))
592        (invoice-lines inv)))))))
593
594(define (make-tax-splits taxes txn-balance-account)
595  (map (lambda (tax)
596         (let ((tax-type (car tax))
597               (tax-amount (cdr tax)))
598           (case tax-type
599             ((vat20 vat15 vat175)
600              (list
601               (make-txn-split
602                (find-account "taxes.vat")
603                (- tax-amount)
604                #f)
605               (make-txn-split
606                txn-balance-account
607                tax-amount
608                #f))))))
609       taxes))
610</enscript>
611
612Feeding in the above example invoice, then checking out the resulting
613double-entry transaction list, shows that it worked:
614
615<enscript>
616(hash-table-for-each *txns*
617                     (lambda (date txns)
618                       (for-each
619                        (lambda (txn)
620                          (printf "Date: ~A Desc: ~A\n"
621                                  (txn-date txn)
622                                  (txn-description txn))
623                          (for-each (lambda (split)
624                                      (printf "Acct: ~A Delta: ~A Notes: ~A\n"
625                                              (account-name (txn-split-account split))
626                                              (txn-split-amount split)
627                                              (txn-split-notes split))) (txn-splits txn)))
628                        txns)))
629</enscript>
630
631   date: #<date> Desc: Invoice INV005 for Widget Corp
632   Acct: income.work Delta: -800 Notes: Router setup and installation
633   Acct: taxes.vat Delta: -160.0 Notes: #f
634   Acct: clients.widgetcorp.balance Delta: 160.0 Notes: #f
635   Acct: clients.widgetcorp.balance Delta: 800 Notes: #f
636
637We've ended up with multiple splits for the same account, as we record
638that both VAT and the money due for the service are to come from the
639client's balance account - and other splits will add plenty more. To
640fix this, we need to write a procedure that canonicalises a list of
641splits, and call that on the splits before calling
642{{make-txn}}. Canonicalisation consists of finding all the splits that
643refer to the same account and have the same notes (be it {{#f}} or a
644string) and merging them into one with the total of the amounts. But
645I'll leave that (along with implementing bills, payments, and some
646actual reports) as an exercise to the reader... It's easy to imagine
647how to generate a VAT report from the list of transactions, by
648filtering them for membership of the required date range and looking
649for splits involving "taxes.vat", or to generate a nicely formatted
650invoice by extracting a single invoice record, or to work out the
651balance of an account at any point in time by adding up all the
652transaction splits that involve it up to that point in time. Also, the
653core engine needs to be wrapped up in a module that only exposes the
654required bindings, and hides internals.
655
656Having automated one's book-keeping and financial reporting, many
657operations (such as the VAT returns) can be done without involving an
658accountant; in my case, the accountant is only needed to help with the
659annual corporation tax computation and filing of official accounts,
660which requires deep understanding of the UK tax system to do
661everything properly. Having said that, if I studied the system
662properly (and tracked the changes each year), I'm sure I could
663automate that, too...
664
665== 9. About the Chicken Gazette
666
667The Gazette is produced occasionally by a volunteer from the Chicken
668community. The latest issue can be found at
669[[http://gazette.call-cc.org]] or you can follow it in your feed
670reader at [[http://gazette.call-cc.org/feed.atom]]. If you'd like to
671write an issue, [[http://wiki.call-cc.org/gazette|consult the wiki]]
672for the schedule and instructions!
Note: See TracBrowser for help on using the repository browser.