source: project/gazette/src/issues/ @ 25513

Last change on this file since 25513 was 25513, checked in by andyjpb, 9 years ago

Initial version of the 21st gazette.

File size: 46.9 KB
1((title . "Issue 21")
2 (authors "Andy Bennett")
3 (date . 1321465864))
5== 0. Introduction
7Welcome to the 21st issue of the Chicken Gazette. While the gazette
8was on hiatus the chicken community was definitely not.
10With the stabalisation of Chicken 4.7 well underway since the last
13Some people have also started to set up salmonella farms and C-Keen
14has completed his "Chicken Zoo".
16Peter Bex has been working on irregex.
18Thanks for the [[#omelette-recipes|omelette recipe]] go to [[|Alaric Snell-Pym]].
23== Chickens Out & About
25=== Nuremburg (December 2011)
26[[|Christian Kellermann]] wrote
27([[|2011/10/22 17:38Z]]) to
28confirm the planned Chicken Hacking Sprint scheduled for the beginning of December.
30Interested and available Chickenauts will meet in Nuremburg on the weekend of 9th, 10th and 11th
31of December. If you haven't done so already, you can let us know that you will be joining us by
32placing a note at the foot of the [[|wiki page]].
33If you can't attend in person then we'll be keeping an eye on IRC so feel free to book yourself
34a slot in your cave at home and join us throughout the weekend and into the evenings, Central European Time.
36For those of you keeping score, this will bring the total number of Chicken meets this year to
374 (FOSDEM, FrOSCon, T-DOSE and Nuremburg).
40=== FrOSCon (August 2011)
41At FrOSCon in August we sported not only a project booth but a presentation room as well so we took the
42opportunity to lay on a couple of talks for the delegates.
44[[|Moritz Heidkamp]] presented
45"[[|An introduction to Lisp]]:
46Why to talk to computers in parentheses" whilst
47[[|Christian Kellermann]] offered
48"[[|A guided tour through the republic of CHICKEN]]:
49get up to speed with the practical scheme implementation".
50[[|Christian]] has been keeping his talk up-to-date and
51you can find the latest version [[|on the wiki]].
53Naturally, there are [[|pictures]].
56=== T-DOSE 2011 (November 2011)
57European Chickens met up at T-DOSE in Eindhoven, The Netherlands on the weekend of the 5th, 6th and 7th of
58November. [[|Peter Bex]] arranged for us to contribute a "Chicken
59Scheme Project Booth" and we subsequently lured a few Dutch hackers into #chicken on IRC.
61[[|Christian Kellermann]] knocked up some impressive demos
62of [[|PONG]] using Chicken's [[|Cairo Egg]].
64[[|Alaric Snell-Pym]] finished off the tests for his
65[[|Ugarit]] backup egg which allowed him to release version 1.0.
67...and [[|Moritz Heidkamp]] started on a replacement for the
68[[|Environments Egg]] as it has unfortunately has been overtaken
69by developments in the Chicken development branch.
71Pictures are up in the [[|usual place]] on the Chicken website.
75== 2. Chicken Talk
77=== R7RS to be dedicated to the memory of John McCarthy
79Matt Welland wrote in with the sad news that the "Father of Lisp", John McCarthy had sadly passed away ([[|2011/10/25 15:59Z]]). [[|John Cowan]], member of the R7RS working group, told us that R7RS,
80the upcoming version of the Scheme standard, would be dedicated to his memory ([[|16:22Z]])
83=== Parallel build patch for Chicken Makefile
85A lively discussion broke out on the [[|Chicken Users mailing list]] regarding
86[[|parallelisation of the Chicken build process]].
88Vitaly Magerya supplied a [[|patch]] (2011/10/06 12:37Z) to fix some problems
89he'd been having with builds failing when specifying the {{-j}} to {{gmake}}.
90[[|Mario Domenech Goulart]] noted that this functionality was already being
91addressed by [[|ticket 526]]
92([[|2011/10/06 12:39Z]]) and Vitaly was keen to see it committed
93soonly as he maintains the FreeBSD port and people had been asking him for parallel builds ([[|2011/10/06 14:24Z]]). Whilst still keen, [[|Mario]] [[|advised caution]] as he wanted to see it proven ([[|2011/10/06 14:40Z]]).
95Vitaly was keen to push things forward ([[|2011/10/06 15:45Z]]) and
96offered continuted testing under FreeBSD 8.2-RELEASE on x86_64. He noted that, as far as he could tell, {{gmake check}} passed when using his
97patch, modified with [[|Moritz Wilhelmy's suggestions]]. Moreover, he
98offered to try the patch from [[|ticket 526]] as well.
100[[|Mario]] [[|suggested]]
101that using the [[|ticket 526]] patch was a good idea as it had, as far as he knew, recieved testing under Linux and MacOSX.
103Toby Thain spoke in favour of the patch, saying that a working parallel build gives confidence that the Makefile is, in fact, correct.
104He offered to test on three platforms, namely Solaris 10 SPARC SMP and both PowerPC and Intel version of MacOSX SMP.
106Vitaly updated the [[|ticket 526]] patch to the then current sources and confirmed that the build worked at {{-j8}}
107and the tests passed in his FreeBSD 8.2-RELEASE, x86_64 environment. He did, however, note that parallelisation does not work for the install phase
108([[|2011/10/08 12:29Z]]).
110Toby Thain reported successful builds on the 64-bit version MacOSX-10.6 as well as MacOSX-10.5 on a dual-processor G5. He noted "{{make -j8}}
111reduces the make step from 3:39 to 0:47 on my 8-core system". Unfortunately {{make check}} did not pass on either system, with or without the patch
112([[|2011/10/12 01:59Z]] &
113[[|2011/10/15 19:53Z]])
115[[|Christian Kellermann]] noted that it might necessary to install the compiled binaries
116([[|2011/10/15 20:07Z]]) however, Toby did not report back as to whether this
117had any effect on the results of the tests.
120=== 64-bit SPARC Build
122Nicolas Pelletier came to the [[|mailing list]]
123unable to build for Solaris 10 running on SPARC64 due to a missing {{apply-hack.S}}
124([[|2011/11/09 11:48Z]]).
126It turned out that there was no SPARC64 support for Chicken at all and [[|John Cowan]]
127came to the rescue ([[|15:36Z]])
128with instructions for how to add it to the supported architecutes list and disable the
129apply hack. [[|Christian Kellermann]] chimed in with affirmations
130around the utility of using Chicken without an apply hack ([[|15:54Z]])
132Nicolas struggled with disabling the apply hack ([[|2011/11/10 11:10Z]])
133due to the Two Lies Of The README. Over the course of the next few minutes [[|Christian]]
134and [[|Mario Domenech Goulart]]]
135confirmed that the lies, which related to the way bootstrap compilers are distributed and invoked, had
136already been fixed in the development versions.
138Meanwhile, Toby Thain tried the build on his SPARC64 ([[|12:40Z]])
139and discovered that GNU make 3.82 is required as well as the addition of {{-lrt}} in {{Makefile.solaris}}. Toby's
140build eventually succeeded and appeared to be good, albeit having only used 1 of his 4 CPUs and with a couple of {{make check}} errors.
142Later on the Thursday, [[|Jim Ursetto]] noted ([[|16:47Z]])
143that the [[|parallel build patch]] in
144[[|ticket 526]] still works on Chicken Toby [[|confirmed]]
145that the patch did indeed work on Solaris and build time was reduced to 10 just of his Earth minutes from the original 40.
146Of course, user time across the four CPUs was still in the region of 40 minutes in total. :-)
148To round everything off, Toby offered [[|binary SYSV packages for Solaris 10 on SPARC64]].
149Contact him via the [[|mailing list]] if you're interested.
152=== Static Linking & Standalone Binaries.
154Serg Kozhemyakin [[|asked]] whether it is possible to build static
155binaries after having trouble using the {{-static}} flag to {{csc}}. He also wondered about building standalone binaries for Windows.
156[[|Christian Kellermann]] [[|advised]]
157against static linking as many of the eggs cannot cope with it. He did, however, suggest that the {{-deploy}} option may well solve both
158issues as it ensures that all the DLLs (including the Chicken runtime and other shared objects) are bundled together into a single directory.
161=== Tracing with {{,tr}} moved to an egg
163[[|Mario Domenech Goulart]] [[|helped]]
164Curtis Cooley in his [[|search for the {{,tr}} command]].
165The functionality had been moved from the core into an egg in Chicken 4.3.0.
168=== {{gmake check}} sometimes fails.
170Pekka Niiranen wrote to tell us about
171[[|problems with {{gmake check}} in Chicken 4.7.0-st]]
172on OpenBSD 5.0.
174[[|Jim Ursetto]] [[|said]]
175that one of the failures had been fixed in the stability branch after the Chicken 4.7.0-st release and he filed
176[[|ticket 724]] for the segmentation fault issue.
178[[|Peter Bex]] mentioned that the [[|ticket 724]] segmentation
179fault had also been observed under NetBSD and Arch Linux. He said that "It's known to be broken" but that it is not properly reproducible
180as it only happens some of the time ([[|2011/11/06 09:51|]]).
183== 3. Salmonella and other tests
184Salmonella is Chicken's egg testing framework.
186For some time we have had daily salmonella runs of the master development
187version of Chicken under Linux on x86. Throughout November, MacOSX-x86_64 has also
188been regularly experiencing the runs. [[|Mario Domenech Goulart]] who maintains the testing
189effort at [[]], as well as the [[|Salmonella codebase]]
190itself, is always on the lookout for more feeds. We are aware that testing on
191other platforms is being organised but if you've got a private Salmonella
192running regularly on an interesting operating system or version of Chicken then
193give us a shout and we'll see if we can get it included with the other reports
194on [[]]
196There is also [[|an entirely new version of Salmonella]] over at github so test that too.
198[[|Alaric Snell-Pym]] is one of the people who has been setting up a Chicken testing
199environment. After [[|some discussion]], he chose Chicken 4.5.0 under
200NetBSD ([[|2011/10/06 21:18Z]]). He's been keeping track of all the
201Egg dependencies and plans to publish them as a list of NetBSD pkgsrc packages.
206== 2. Core Development
208I need some blurb on this from people Who Know.
209I'd like something on the scrutinizer and stability work that's been going on as well as the proposal to bug fix until r7rs
210and then do Chicken 5.0.
213mario, felix, ckeen
214lots of bug fixes, sparc64 support, scrutinizer fixes
216 * On origin/master:
217   * (e1a5437): when conditional branches differ in the number of results, do not merge the results (felix, 2 days ago)
218   * (bae7d92): clustering optimization added (felix, 5 weeks ago)
219   * (b1907c5): use different default stack-sizes on 32/64 bit platforms (felix, 6 days ago)
220   * (986bd3f): disable apply-hack on sparc64 (felix, 6 days ago)
221   * (28a36bf): Remove scripts/make-egg-rss-feed.scm (imported into the chicken-infrastructure repo) (Mario Domenech Goulart, 5 days ago)
222   * (fbeffd1): scripts/tools.scm: needs regex for `string-match' (Mario Domenech Goulart, 5 days ago)
223   * (1fca9c6): use equal? in case MSYSTEM is not defined (makedist.scm) (felix, 6 days ago)
224   * (42a8263): don't remove build-info files on make confclean or it is not possible to build from a tarball without preinstalled chicken (felix, 9 days ago)
225   * (3bb9645): write debugging output for optimizations into generated source file (felix, 3 weeks ago)
226   * (e7fe2fb): allow procedure in operator-position of evaluated list (#705, suggested by John Cowan) (felix, 2 weeks ago)
227   * (e2fb31c): fix subtype check for fixed-size list/vector types in scrutinizer (felix, 10 days ago)
228   * (fd018df): strip std-envs of non-std identifiers, ignore ##sys#macro-environment when evaluating with an explicit environment (felix, 13 days ago)
229   * (fe8f63c): minor change in port-tests.scm to handle case when HOME is not defined (Windows) (felix, 2 weeks ago)
230   * (723a469): cut of union types above a certain length (felix, 2 weeks ago)
231   * (919a7c7): fixes in build + Windows test script, added note to README (thanks to mario) (felix, 2 weeks ago)
232   * (ddb96f9): Also added runtests.bat to manifest. (felix, 3 weeks ago)
233   * (0df9bb8): another trivial fix for making boot-chicken work on cmd.exe (felix, 3 weeks ago)
234   * (cea668f): trivial Windows-related corrections in rules.make, found by Mario - not yet working, though (felix, 3 weeks ago)
235   * (1bc311d): undocumented scrutinize declaration (obsolete), reported by ckeen (felix, 3 weeks ago)
236   * (10494f6): fixed stupid paren-matching error (felix, 3 weeks ago)
237   * (f9a6702): disable debug output in scrutinizer (felix, 5 weeks ago)
238   * (2483f8d): check not necessary for lambda and  warn if using keyword as variable in binding form (felix, 3 weeks ago)
239   * (aa6b2bb): cleaning up uninitialized variables in C_reclaim, add debug output to C_mutate on m-stack resize, grow m-stack at higher rate (felix, 3 weeks ago)
240   * (1e04156): use write-string port-class method (felix, 3 weeks ago)
241   * (59cdaaf): remove obsolete C_h_... allocation functions (pointed out by Joerg Wittenberger) (felix, 3 weeks ago)
242   * (159611d): handle macro-aliases when resolving declared identifier, making sure things are done right inside modules. (felix, 4 weeks ago)
243   * (0cc88a4): removed obsolete file (felix, 4 weeks ago)
244   * (1f56f3a): ##core#type makes subtype-check optional; quit compile when type-mismatches in strict mode (felix, 5 weeks ago)
248== 1. Hatching Farm
250* [[|Ivan Raikov]]'s [[|proposal]]
251  to recatagorise eggs filed under "Uncategorized or invalid" and "Miscellaneous" was met with enthusiasm with a number of people pitching in to
252  help as well as [[|Peter Bex]] suggesting that the category list be revisited
253  ([[|2011/10/11 08:12Z]]). [[|Peter]]
254  specifically suggested the addition of a "Communication"s catagory.
256* [[|Thomas Chust]] [[|announced]]
257  a [[|small binding for the BerkelyDB library]]
259* [[|Christian Kellermann]]
260  [[|fixed some bugs]] in the
261  [[|Cairo Egg]] and tagged version 0.1.11.
263* [[|Alan Post]] [[|patched]] the
264  [[|Sandbox Egg]] to bring it up-to-date with API changes in the Chicken core. His patch was duly accepted.
267(18:54:58) mario-goulart: andyjpb: you can use your favorite svn client to get a diff for /release/4/egg-locations
269Write about the distributed egg repository
272The following new extensions were added:
274* [[egg:canvas-draw|canvas-draw]]: Bindings to the CanvasDraw graphics library ([[user:thomas-chust|Thomas Chust]])
275* [[egg:bindings|bindings]]: light-weight alternative to the matchable egg together with some enhancements ([[user:felix-winkelmann|Felix Winkelmann]])
276* [[egg:tuples|tuples]]: a tuple datatype implementation ([[user:felix-winkelmann|Felix Winkelmann]])
277* [[egg:svn-egg-author|svn-egg-author]]: makes it easier for people still using svn to tag/release eggs ([[user:peter-bex|Peter Bex]])
278* [[egg:memcached|memcached]]: memcached client interface ([[user:ivan-raikov|Ivan Raikov]])
279* [[egg:tree-rewrite|tree-rewrite]]: initial import of tree-rewrite, an adaptation of Oleg Kiselyov's term rewriting system PostL ([[user:ivan-raikov|Ivan Raikov]])
280* [[egg:srfi-71|srfi-71]] '''1.0''': import some of srfi-71 (evhan)
282The following new versions have been released:
284* [[egg:hen|hen]] '''0.5''': Create tag 0.5 (
285* [[egg:sequences|sequences]] '''0.3''': bugfixes; replicate ([[user:felix-winkelmann|Felix Winkelmann]])
286* [[egg:fastcgi|fastcgi]] '''1.1.1''': pointer->c-pointer fix; output strings can contain NULs (thanks to Peter Danenberg)
287* [[egg:contracts|contracts]] '''0.4''': some enhancements ([[user:felix-winkelmann|Felix Winkelmann]])
288* [[egg:sequences|sequences]] '''0.4''': performance tuning, some renaming ([[user:felix-winkelmann|Felix Winkelmann]])
289* [[egg:message-digest|message-digest]] '''2.3.5''': reverted def alloc of ctx & no str use in u8vector restyp creation. ([[user:kon-lovett|Kon Lovett]])
290* [[egg:md5|md5]] '''2.5.1''': Rel 2.5.1 w/ minor bug fix. ([[user:kon-lovett|Kon Lovett]])
291* [[egg:sha1|sha1]] '''2.3.1''': Fix correct context field clear (was not walking over memory since sufficient padding in struct, still it was wrong.) Rmvd ctx clear since unneeded. ([[user:kon-lovett|Kon Lovett]])
292* [[egg:spock|spock]] '''0.03''': renamed some API procedures; bugfix in driver (thanks to Supermario) ([[user:felix-winkelmann|Felix Winkelmann]])
293* [[egg:amb|amb]] '''2.1.4''': Wrong dep verno for condition-utils & check-errors in .meta. Missing import for srfi-1 member use in amb-extras [reported by Markus Klotzbuecher]. ([[user:kon-lovett|Kon Lovett]])
294* [[egg:mojo|mojo]] '''0.0.1''': first release ([[user:moritz-heidkamp|Moritz Heidkamp]])
295* [[egg:sdl|sdl]] '''0.5.3''': new release ([[user:christian-kellermann|Christian Kellermann]])
296* [[egg:tuples|tuples]] '''0.2''': tuples 0.2 ([[user:felix-winkelmann|Felix Winkelmann]])
297* [[egg:progress-indicators|progress-indicators]] '''0.3''': added missing licensing information to meta file (thanks to mario) ([[user:felix-winkelmann|Felix Winkelmann]])
298* [[egg:spiffy-request-vars|spiffy-request-vars]] '''0.13''': test fixes ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
299* [[egg:html-tags|html-tags]] '''0.10''': added generate-sxml? parameter support ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
300* [[egg:blas|blas]] '''3.0''': Chicken 4 cleanup ([[user:ivan-raikov|Ivan Raikov]])
301* [[egg:atlas-lapack|atlas-lapack]] '''2.0''': Chicken 4 cleanup ([[user:ivan-raikov|Ivan Raikov]])
302* [[egg:probdist|probdist]] '''1.7''': updated use of blas and atlas-lapack ([[user:ivan-raikov|Ivan Raikov]])
303* [[egg:hyde|hyde]] '''0.15''': simpler atom feed generation ([[user:moritz-heidkamp|Moritz Heidkamp]])
304* [[egg:signal-diagram|signal-diagram]] '''1.1''': added reduce combinator ([[user:ivan-raikov|Ivan Raikov]])
305* [[egg:awful-sql-de-lite|awful-sql-de-lite]] '''0.4''': See for the latest changes. ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
306* [[egg:awful-postgresql|awful-postgresql]] '''0.4''': See for the latest changes. ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
307* [[egg:awful-sqlite3|awful-sqlite3]] '''0.4''': See for the latest changes. ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
308* [[egg:awful|awful]] '''0.31''': See for the latest awful changes. ([[user:mario-domenech-goulart|Mario Domenech Goulart]])
309* [[egg:spock|spock]] '''0.04''': removed explicit version number from setup script (reported to mario)
310* [[egg:spock|spock]] '''0.05''': compiled code runs somewhat on IE; slightly more verbose setup-script ([[user:felix-winkelmann|Felix Winkelmann]])
311* [[egg:memcached|memcached]] '''1.0''': first release ([[user:ivan-raikov|Ivan Raikov]])
312* [[egg:locale|locale]] '''0.6.11''': check-locale-component for 'name had bad value ('name itself rather than value variable). with-tzset was leaving TZ set when no original TZ value. Proc for envvar LANGUAGE (gnu) was using code that deref'ed nonexisting 'language locale component; part of code to substitute primary locale region for possible nonexisting 'country in LANGUAGE locale, which can be subset - just language. (This caused the Error: (string-parse-start+end) bad argument type - not a string: #f from string-upcase.) ([[user:kon-lovett|Kon Lovett]])
313* [[egg:locale|locale]] '''0.6.12''': Use 'current locale category for primary locale, rather than 'messages. locale-timezone has proc for offset & name. ([[user:kon-lovett|Kon Lovett]])
314* [[egg:matrix-utils|matrix-utils]] '''1.12''': updated use of blas ([[user:ivan-raikov|Ivan Raikov]])
315* [[egg:iexpr|iexpr]] '''1.2''': fixed unit tests ([[user:ivan-raikov|Ivan Raikov]])
316* [[egg:mpi|mpi]] '''1.8''': ported to Chicken 4 ([[user:ivan-raikov|Ivan Raikov]])
317* [[egg:bloom-filter|bloom-filter]] '''1.1.4''': Use of round -> ceiling (can't lose a bit). Added different way to calc M & K. Added variant signature for make-bloom-filter to take N & P. ([[user:kon-lovett|Kon Lovett]])
318* [[egg:ripemd|ripemd]] '''1.0.2''': Added ident to digest prim. ([[user:kon-lovett|Kon Lovett]])
319* [[egg:setup-helper|setup-helper]] '''1.3.1''': Fix for trashing the .o when static+shared module setup. "static module" added. ([[user:kon-lovett|Kon Lovett]])
320* [[egg:slime|slime]] '''1.1''': add swank:documentation-symbol (from Jean-Christophe Petkovich) and preliminary inspector implementation ([[user:christian-kellermann|Christian Kellermann]])
321* [[egg:format|format]] '''3.1.5''': first release ([[user:felix-winkelmann|Felix Winkelmann]])
322* [[egg:neuromorpho|neuromorpho]] '''1.12''': small fixes ([[user:ivan-raikov|Ivan Raikov]])
323* [[egg:yelp|yelp]] '''1.0.2''': add note to tests about Yelp API key (ddp)
324* [[egg:format-compiler-base|format-compiler-base]] '''17017.1''': Replaces "use" of "extras" unit with "import", as the current implementation modified the imported binding to "format" and thus overwrote the toplevel binding. ([[user:felix-winkelmann|Felix Winkelmann]])
325* [[egg:embedded-test|embedded-test]] '''17018.1''': fixed long open bug and tagged 17018.1 ([[user:felix-winkelmann|Felix Winkelmann]])
326* [[egg:ssax|ssax]] '''5.0.4''': added export for ssax:warn and tagged 5.0.4 ([[user:felix-winkelmann|Felix Winkelmann]])
327* [[egg:rpc|rpc]] '''1.1.2''': removed use of project procedure ([[user:felix-winkelmann|Felix Winkelmann]])
328* [[egg:flsim|flsim]] '''1.4''': fixes related to Octave code generation ([[user:ivan-raikov|Ivan Raikov]])
329* [[egg:opengl|opengl]] '''1.19''': opengl 1.19: properly link -lGLU (thanks to Moritz Heidkamp) ([[user:felix-winkelmann|Felix Winkelmann]])
330* [[egg:iset|iset]] '''1.8''': fix typo in optimization option ([[user:felix-winkelmann|Felix Winkelmann]])
331* [[egg:hen|hen]] '''0.6''': various fixes and additions (Joseph Gay)
337== Donations
339It's now possible to donate to your favourite scheme (as long as it's Chicken)!
343We've already had two donations since the account opened a month ago so get dontating before it fills up! :-)
345Thank you [[|Sven Hartrumpf]] and Ross Lonstein!
348== 3. Omelette Recipes
350The dream of many Schemers is to write Scheme for a living. Due to the
351regrettable tendency of employers to frown upon good ideas, the best
352way to achieve this is to start your own company, so you can be your
353own boss, and frown upon the good ideas of others instead.
355However, running your own company comes at a price; and a big part of
356that price is the horror of "book keeping"; the requirement to track
357all flows of money and other, more abstract, forms of value in and out
358of your company.
360Some people will tell you book-keeping is simple. "Just keep all your
361receipts and bank statements and bills and invoices", they say. "Then
362send them to your accountant at the end of the year."
364"But what about value-added tax?", you ask (or "sales tax" in some
365countries). "And what about income tax paid at source for my employees
366(including myself)?". "And why do I need to pay an accountant so much
367to do a job that a computer can easily do in milliseconds?" And then
368the smug smile slowly drips from the face of the "Oh it's easy" crowd.
370Clearly, book-keeping is complicated. And yet also simple, in that it
371is determined by sets of rules.
373We know what to do, don't we? Let's get coding!
375The problem is that book-keeping involves several different kinds of
376inputs - bills (that people send you), invoices (that you send
377people), transfers of money (bills and invoices being paid), loans (to
378and from the company), employees being paid, interest payments from
379the bank, dividend payments, and so on; while it also involves several
380different outputs - tax reports for the various taxes involved (in the
381UK, I had to deal with VAT every three months, income tax and national
382insurance when paying myself and my wife as employees every month, an
383annual filing fee, and annual corporation tax and dividend payments),
384statutory filing of certain financial summaries (generally annually),
385and internal reporting: How much was I spending? How much did each
386client owe? How much should be in the bank by when? Plus, it's nice to
387be able to generate nice invoices to send to folks who owe you
388money. That's a form of specialised report, too, just reporting on a
389single invoice.
391Each of the output reports depend in complex ways on different
392information from the inputs. The VAT reports mainly have to add up how
393much VAT I've paid when being billed by others, and how much VAT I've
394charged when invoicing - meaning that VAT needs to be tracked on all
395bills and invoices so it can be extracted. They also want to know
396totals of actual money in and out of the company in the period (even
397stuff where VAT isn't an issue), presumably to check up on
398me. Meanwhile, end of year reports tend to need to know how much I've
399invoiced for various different kinds of work, and what I've spent on
400what kinds of things: buying equipment that will last me for several
401years is handled differently to expenses like travel, or buying stuff
402that I eventually resell to clients (so in our invoices, we need to
403keep track of money charged for services separately to money charged
404for things).
406Some reports care about virtual money moving hands. As soon as I
407invoice somebody, then the company now has a virtual asset - some
408money owed to it. That's worth as much as cash in the bank from some
409perspectives (generally, I have to pay tax on it as soon as it's
410invoiced, even if I've not been paid). And yet some care only about
411actual cash changing hands (working out my bank balance, for instance).
413Sometimes our clients invite us to incur expenses in doing work for
414them (such as extra travel) and then invoice them on for those
415expenses, so they pay us back - in which case, expenses need to be
416able to be tied to invoices, as well. Sometimes we decide to cancel an
417invoice, which can't be done by just pretending it never existed, for
418audit-trail reasons; we need to issue a "negative" invoice called a
419credit note.
421Just to complicate matters more, the actual movement of money isn't
422atomic. If I invoice somebody on date A, they might post me a cheque
423which arrives on date B, which I pay into the bank on date C, which
424actually clears into the account (and thereby appears on my bank
425statement, when I get it) on date D. So at date A the company now has
426a "we are owed" pretend-money asset, which goes through various stages
427until it finally turns into money in the bank on date D.
429I handled my book-keeping with some hacky scripts written in Chicken
430Scheme. What I'm going to document here is partly what I've done, and
431partly what I should have done - it was a very iterative process,
432refining the best way to handle stuff, and there's lots of
433improvements I've wanted to make but not had time to. So I'm going to
434describe the ideal case, not the hacky half-way house I actually have
435right now!
437The approach I took was to have a file called a "ledger" that I enter
438all my invoices and so on into. This is parsed to build up a bunch of
439data structures in memory, from which the various reports can easily
440be obtained. Firstly, for each kind of input object (invoices, bills,
441etc) there's a list of them, or more often a hashmap to make them easy
442to find by some identifier (I can give my invoices unique symbolic
443names, for instance). That contains the raw data as parsed from the
444ledger file. But then we also create summary structures, which are
445used by the more general reports to generate their output without
446having to special-case each and every different input object type, and
447to enable sharing of common functionality between reports.
449The main summary structure is the double-entry transaction list, which
450models the entire financial activity of the company as transfers
451between accounts.
453Imagine I invoice Widget Corp for setting up and installing a router:
455  INVOICE INV005: Issued 2011-04-25
456  Router setup and installation: GBP 800
457  1 router from my stock: GBP 350
458  1 train ticket for me to go to their site: GBP 35 (no VAT due)
459  Subtotal: GBP 1,185
460  VAT on the above: GBP 230
461  Total due: GBP 1,415
463As part of the work, I lose a router (worth GBP 350) from my stock,
464and have to spend GBP 35 on a train fare.
466This might expand into the following transactions:
468 * 2011-03-02: "Expense for Widget Corp (INV005)"
469   * +35 "Travel to site"
470   * cash -35
472 * 2011-04-25 "Invoice Widget Corp (INV005)"
473   * -800 (Router set up and installation)
474   * stock.balances -350 (1 of LX300 router, serial number 0343248)
475   * clients.widgetcorp.expenses -35 (Travel 2011-03-02)
476   * taxes.vat -230
477   * clients.widgetcorp.balance +1415
479And, eventually, they might pay me, which hits my bank account some
480time later:
482 * 2011-05-07 "Payment from Widget Corp (INV005)"
483   * clients.widgetcorp.balance -1415
484   * bank.balance +1415
486And then one day I'll pay my VAT bill, which will look something like:
488 * 2011-06-01 "VAT payment for period from 2011-03-01 to 2011-06-01"
489   * taxes.vat 230
490   * bank.balance -230
492Note a few tricky things. Each transactions "splits", as the lines
493within them are known, have to sum to zero for everything to balance
494correctly, which tells us that nothing has gone missing. So when we
495start being owed GBP 1,415 by Widget Corp, we need to account for
496where that asset has come from. Special accounts with names such as
497"" (for value generated by me working) and
498"clients.widgetcorp.expenses" (for previously-paid expenses that, as
499of this invoice, I can charge the client for) pop into
500existence. "taxes.vat" looks as if VAT is a form of income for me, as
501money comes "from" it in the transaction - which is sort of true; I'm
502charging Widget Corp for some VAT alongside for the actual work
503done. Figuring out what signs to put on all the items in the invoice
504is mind-bending and painful, but if you just concentrate on making it
505all add up to zero in the end and starting from things that are
506obvious (is money going into or out of the bank account, or the "owed
507to me by this customer" account?), you can figure it out.
509From the above, we can start to flesh out some data structures:
512(define-record txn
513               date customer code description splits)
515(define-record txn-split
516               account amount notes)
519(define *txns* (make-hash-table))
520(define (register-txn! txn)
521  (if (hash-table-exists? *txns* (txn-date txn))
522      (begin
523        (set! (hash-table-ref *txns* (txn-date txn))
524              (cons txn
525                    (hash-table-ref *txns* (txn-date txn)))))
526      (begin
527        (set! (hash-table-ref *txns* (txn-date txn))
528              (list txn)))))
531What is an "account"? There's a few kinds, and what kind of account it
532is matters in reporting. Accounts might be assets within the company -
533such as "clients.widgetcorp.balance" or "bank.balance" or
534"stock.balance". Or they may be places where money (be it real or
535virtual) is created from or destroyed by (from the perspective of the
536company), such as "" and "". The important
537difference is that balance-type accounts have a balance that is
538increased when money is sent to them and decreased when it's taken
539out, and that balance is part of the value of the company, while the
540income/expense type accounts don't. In my terminology, these are
541"balance" accounts and "delta" accounts. Each account also begins to a
542group, used to aggregate them in reports: there's income accounts,
543bank accounts, client accounts, and so on. And accounts may be tied to
544a third party - I've given an example of a client above, but also, the
545organisations that send me bills have balances (the money I owe
546them). In general, every third party (be they ones that bill me, or
547ones that I invoice, or both - I've interacted with other freelancers,
548sometimes working for them, sometimes vice versa) has a set of
549accounts attached to them for their balance, expenses I can claim from
550them, and so on. That implies another set of record types:
553(define-record third-party
554               name address balance-account expenses-account)
556(define-record account
557               name type group third-party)
560(define *third-parties* (make-hash-table))
561(define *accounts* (make-hash-table))
563(define (find-account acct-name)
564  (hash-table-ref *accounts* acct-name))
566(define (register-account! acct)
567  (set! (hash-table-ref *accounts* (account-name acct)) acct))
570An account's {{third-party}} slot may be {{#f}} if it's not part of a
571third party.
573Now, to make things easy, I parse the ledger by just defining a heap
574of procedures and macros, and then diving into the ledger with
575eval. It's just Scheme code that, as it is executed, builds up the
576data structures. We'll need some helpers to set up third parties
580(define (define-third-party name address group)
581        (let* ((balance-account
582                (make-account
583                 (string-append name ".balance")
584                 'balance
585                 group
586                 #f))
587               (expenses-account
588                (make-account
589                 (string-append name ".expenses")
590                 'delta
591                 'expenses-reclaimed
592                 #f))
593               (third-party
594                (make-third-party
595                 name address balance-account expenses-account)))
596          (account-third-party-set! balance-account third-party)
597          (account-third-party-set! expenses-account third-party)
598          (register-account! balance-account)
599          (register-account! expenses-account)
600          (set! (hash-table-ref *third-parties* name) third-party)))
603Now we can work on a way of representing bills and invoices. A nice
604input syntax would be:
607(define-third-party "clients.widgetcorp" "123 Any Street" 'clients)
608(register-account! (make-account "" 'delta 'income #f))
609(register-account! (make-account "" 'delta 'travel #f))
610(register-account! (make-account "stock.balance" 'balance 'stock #f))
611(register-account! (make-account "cash" 'balance 'cash #f))
612(register-account! (make-account "taxes.vat" 'balance 'vat #f))
614(invoice "INV005" "clients.widgetcorp" (ymd 2011 04 25)
615         (service "" 800 (vat20) "Router setup and installation")
616         (sale "stock.balance" 350 (vat20) "1 of LX300 router, serial number 0343248")
617         (expense (ymd 2011 03 02) "" "cash" 35 () "Travel to site"))
620The idea is that the sales taxes incurred on a line are specified as a
621list after the amount. If there's no taxes due, then we use an empty
622list. Otherwise we have a list of taxes, which are either plain tax
623names (to have the system compute the tax due itself) or two-element
624lists joining a tax name to a precomputed amount (often, when we pass
625on an expense, we know the tax we paid as it's on the receipt, so we
626should use that (even if they made a mistake working it out) rather
627than calculating our own).
629A nice way to implement that might be to make "invoice" a macro that
630absorbs its first three arguments as an invoice code, the name of the
631third party to invoice, and the date; then treats the rest as a body
632to be wrapped in a dynamic environment in which a parameter allows
633{{sale}}, {{expense}}, and {{service}} to add lines to the
634invoice. This is easily arranged:
637(define-record date year month day)
639(define-record invoice
640  name date third-party lines)
642(define (register-line! invoice line)
643  (invoice-lines-set! invoice
644                      (cons line (invoice-lines invoice))))
646;; Compute sales taxes
647(define (compute-tax tax amount)
648  (case tax
649    ((vat20) (* 0.20 amount)) ;; Current UK rate
650    ((vat15) (* 0.15 amount)) ;; Previous UK rate
651    ((vat175) (* 0.175 amount)))) ;; Previous UK rate
653;; Expand a list of taxes, some of which might be bare symbols
654;; naming taxes to work out, or (<tax> <amount>) lists for
655;; ready-computed taxes, into an alist of tax names to tax amounts
656(define (resolve-taxes amount taxes)
657  (map (lambda (tax-desc)
658         (if (list? tax-desc)
659             (cons (car tax-desc) (cadr tax-desc))
660             (cons tax-desc (compute-tax tax-desc amount))))
661       taxes))
663(define-record invoice-service-line
664  income-account amount taxes description)
666(define-syntax service
667  (syntax-rules ()
668    ((service income-account amount taxes description)
669     (register-service! (*current-invoice*) income-account amount 'taxes description))))
671(define (register-service! invoice income-account amount taxes description)
672  (let ((service
673         (make-invoice-service-line
674          (find-account income-account)
675          amount
676          (resolve-taxes amount taxes)
677          description)))
678    (register-line! invoice service)))
680(define-record invoice-sale-line
681  stock-account amount taxes description)
683(define (register-sale! invoice stock-account amount taxes description)
684  (let ((sale
685         (make-invoice-sale-line
686          (find-account stock-account)
687          amount
688          (resolve-taxes amount taxes)
689          description)))
690    (register-line! invoice sale)))
692(define-syntax sale
693  (syntax-rules ()
694    ((sale stock-account amount taxes description)
695     (register-sale! (*current-invoice*) stock-account amount 'taxes description))))
697(define-record invoice-expense-line
698  expense-account payment-account amount taxes description)
700(define (register-expense! invoice date expense-account payment-account amount taxes description)
701  (let ((expense
702         (make-invoice-expense-line
703          (find-account expense-account)
704          (find-account payment-account)
705          amount
706          (resolve-taxes amount taxes)
707          description)))
708    (register-line! invoice expense)))
710(define-syntax expense
711  (syntax-rules ()
712    ((expense (ymd year month day) expense-account payment-account amount taxes description)
713     (register-expense!
714      (*current-invoice*)
715      (make-date year month day)
716      expense-account
717      payment-account
718      amount
719      'taxes
720      description))))
722(define *current-invoice* (make-parameter #f))
723(define *invoices* (make-hash-table))
725(define-syntax invoice
726  (syntax-rules (service sale expense)
727    ((invoice name third-party (ymd year month day) body ...)
728     (let ((inv
729            (make-invoice name
730                          (make-date year month day)
731                          (hash-table-ref *third-parties* third-party)
732                          '())))
733       (parameterize
734        ((*current-invoice* inv))
735        (begin body ...))
736       (set! (hash-table-ref *invoices* name) inv)
737       (generate-invoice-transactions! inv)))))
740We end the expansion of the {{invoice}} macro with a call to
741{{generate-invoice-transactions!}}, which will do the task of creating
742the double-entry transactions for the invoice. Other types of summary
743structure can be added by calling additional generator procedures at
744this point. This is largely a matter of going through the invoice
745lines, handling them on a case-by-case basis to generate lists of
746transaction splits that we can append together to generate the invoice
747transaction. The case of expense lines is interesting, in that an
748extra transaction has to be generated for each expense, to record its
749initial spending, as well as a split to record the expense being
750claimed in the invoice.
752For now, let's just handle one case:
755(define (generate-invoice-transactions! inv)
756  (register-txn! (make-txn
757    (invoice-date inv)
758    (string-append "Invoice " (invoice-name inv) " for "
759                   (third-party-full-name (invoice-third-party inv)))
760    (let ((txn-balance-account
761           (third-party-balance-account
762            (invoice-third-party inv))))
763      (flatten
764       (map
765        (lambda (line)
766          (cond
767           ((invoice-expense-line? line)
768            (list)) ;; FIXME: Not implemented
769           ((invoice-sale-line? line)
770            (list)) ;; FIXME: Not implemented
771           ((invoice-service-line? line)
772            (list
773             (make-txn-split
774              (invoice-service-line-income-account line)
775              (- (invoice-service-line-amount line))
776              (invoice-service-line-description line))
777             (make-tax-splits
778              (invoice-service-line-taxes line)
779              txn-balance-account)
780             (make-txn-split
781              txn-balance-account
782              (invoice-service-line-amount line)
783              #f)))))
784        (invoice-lines inv)))))))
786(define (make-tax-splits taxes txn-balance-account)
787  (map (lambda (tax)
788         (let ((tax-type (car tax))
789               (tax-amount (cdr tax)))
790           (case tax-type
791             ((vat20 vat15 vat175)
792              (list
793               (make-txn-split
794                (find-account "taxes.vat")
795                (- tax-amount)
796                #f)
797               (make-txn-split
798                txn-balance-account
799                tax-amount
800                #f))))))
801       taxes))
804Feeding in the above example invoice, then checking out the resulting
805double-entry transaction list, shows that it worked:
808(hash-table-for-each *txns*
809                     (lambda (date txns)
810                       (for-each
811                        (lambda (txn)
812                          (printf "Date: ~A Desc: ~A\n"
813                                  (txn-date txn)
814                                  (txn-description txn))
815                          (for-each (lambda (split)
816                                      (printf "Acct: ~A Delta: ~A Notes: ~A\n"
817                                              (account-name (txn-split-account split))
818                                              (txn-split-amount split)
819                                              (txn-split-notes split))) (txn-splits txn)))
820                        txns)))
823   date: #<date> Desc: Invoice INV005 for Widget Corp
824   Acct: Delta: -800 Notes: Router setup and installation
825   Acct: taxes.vat Delta: -160.0 Notes: #f
826   Acct: clients.widgetcorp.balance Delta: 160.0 Notes: #f
827   Acct: clients.widgetcorp.balance Delta: 800 Notes: #f
829We've ended up with multiple splits for the same account, as we record
830that both VAT and the money due for the service are to come from the
831client's balance account - and other splits will add plenty more. To
832fix this, we need to write a procedure that canonicalises a list of
833splits, and call that on the splits before calling
834{{make-txn}}. Canonicalisation consists of finding all the splits that
835refer to the same account and have the same notes (be it {{#f}} or a
836string) and merging them into one with the total of the amounts. But
837I'll leave that (along with implementing bills, payments, and some
838actual reports) as an exercise to the reader... It's easy to imagine
839how to generate a VAT report from the list of transactions, by
840filtering them for membership of the required date range and looking
841for splits involving "taxes.vat", or to generate a nicely formatted
842invoice by extracting a single invoice record, or to work out the
843balance of an account at any point in time by adding up all the
844transaction splits that involve it up to that point in time. Also, the
845core engine needs to be wrapped up in a module that only exposes the
846required bindings, and hides internals.
848Having automated one's book-keeping and financial reporting, many
849operations (such as the VAT returns) can be done without involving an
850accountant; in my case, the accountant is only needed to help with the
851annual corporation tax computation and filing of official accounts,
852which requires deep understanding of the UK tax system to do
853everything properly. Having said that, if I studied the system
854properly (and tracked the changes each year), I'm sure I could
855automate that, too...
857== 4. About the Chicken Gazette
859The Gazette is produced occasionally by a volunteer from the Chicken
860community. The latest issue can be found at
861[[]] or you can follow it in your feed
862reader at [[]]. If you'd like to
863write an issue, [[|consult the wiki]]
864for the schedule and instructions!
Note: See TracBrowser for help on using the repository browser.