source: project/wiki/eggref/4/chicken-doc-admin @ 37084

Last change on this file since 37084 was 37084, checked in by Jim Ursetto, 7 months ago

Update Chicken 4 docs for zbigniew eggs

These eggs were ported to Chicken 5 and still support
Chicken 4; docs remain synchronized.

atom chickadee chicken-doc chicken-doc-admin doctype
feature-test socket sql-de-lite tcp6 udp6

File size: 16.2 KB
Line 
1[[tags: egg]]
2== chicken-doc-admin
3
4chicken-doc-admin is a tool for generating and maintaining
5Chicken documentation for the [[chicken-doc]] egg.
6[[toc:]]
7=== Overview
8
9[[chicken-doc-admin]] provides facilities to create and modify a Chicken
10documentation repository for [[chicken-doc]].  It includes a command-line
11tool to convert egg documentation and manual pages from the Chicken
12wiki for use with [[chicken-doc]].  It also provides an API for
13repository manipulation.
14
15The typical mode of using [[chicken-doc-admin]] is to check out a copy of
16the wiki using Subversion, then run the [[chicken-doc-admin]] command
17against the copy's egg and man directories.  Optionally, process
18eggdocs by running [[chicken-doc-admin]] against an egg repository checkout.
19Repeat when either is updated; only changed files will be reexamined.
20
21Eggs can also include bundled documentation rather than sourcing it
22from the wiki; [[chicken-doc-admin]] can parse these automatically
23as well.  Typically, though, official eggs use the wiki.
24
25=== Quick start
26
27The default repository location is in the Chicken install tree and may
28not be writable by your user.  If so, either use {{sudo}} or set an
29alternate writable location with:
30
31 $ export CHICKEN_DOC_REPOSITORY=/path/to/some/writable/directory
32
33Initialize the repository if this is your first time:
34
35 $ chicken-doc-admin -i
36
37Now download and process the wiki documentation:
38
39 $ svn co --username anonymous --password "" \
40      http://code.call-cc.org/svn/chicken-eggs/wiki
41 $ chicken-doc-admin -m wiki/man/4
42 $ chicken-doc-admin -e wiki/eggref/4
43
44==== Locally installed egg documentation
45
46Certain eggs may install wiki documentation during a
47{{chicken-install}}.  Although uncommon, this allows unofficial,
48private or experimental eggs to include documentation without making
49it public on the wiki.  To do this with your own eggs, see [[#Local egg documentation for developers]].
50
51Process all locally installed documentation:
52
53 $ chicken-doc-admin -H
54
55Process docs just for the {{inane}} and {{pathfinder}} eggs:
56
57 $ chicken-doc-admin -H inane pathfinder
58
59Working example:
60
61 hg clone https://bitbucket.org/ursetto/inane
62 cd inane
63 chicken-install
64 cd ..
65 chicken-doc-admin -H inane
66
67The {{cd ..}} is only for illustrative purposes here to show you can
68process installed docs from any directory.  (Since you checked out
69from source, you could instead have done {{chicken-doc-admin -E inane}}
70from the source directory to process {{./inane.wiki}}.)
71
72'''Note:''' If egg documentation is both on the wiki ''and'' locally
73installed, and you process both, then the latest file timestamp wins.
74Unfortunately, these timestamps reflect not the actual update time
75but rather the install time on your local system, and are of
76dubious value in this case. 
77
78==== Optional eggdoc processing
79
80''Almost all users can skip this step'', because there are only a few
81remaining eggs that use eggdoc. The Chicken 5 version of this egg drops
82support for eggdoc.
83
84Download the egg repository and process any eggdocs:
85
86* '''Warning''': eggdocs are executable Scheme code, so if you are
87concerned about untrusted code, then skip this step or do not run
88it as root.  This risk is reasonably low because anonymous uploads
89are not allowed.
90
91 $ svn co --username anonymous --password "" \
92      http://code.call-cc.org/svn/chicken-eggs/release/4 eggs
93 $ chicken-doc-admin -t eggdoc -e eggs
94
95=== Upgrading
96
97After an upgrade of the egg, if you get a message informing you that
98the repository version is invalid, you will need to delete and
99recreate your repository.  For example:
100
101 $ chicken-doc-admin -D
102 $ chicken-doc-admin -i
103 $ chicken-doc-admin -m ...
104 $ chicken-doc-admin -e ...
105
106=== Command-line
107
108 $ chicken-doc-admin COMMAND
109
110   -l           list repository information
111
112Print some information about the repository, such as its location
113and version.
114
115   -i           initialize repository non-destructively
116
117Initialize the repository in the current default location or as
118overridden by {{CHICKEN_DOC_REPOSITORY}}.  The directory will be
119created if it does not exist.  This command must be run before any
120documentation can be processed.  You should usually initialize an
121empty directory; initializing a populated directory won't delete
122anything, and may confuse us.
123
124   -t type      document type
125
126Set the source document type, which may be {{eggdoc}} or {{svnwiki}},
127defaulting to {{svnwiki}}.  This option can be used before -e, -E, -m, -M.
128
129To process type {{eggdoc}}, you must install the {{eggdoc-svnwiki}} extension.
130
131   -e dir       process egg directory DIR
132
133Process a copy of egg directory DIR, adding each egg documentation page
134to a node at toplevel and the identifiers contained in the
135page to that node's children.  Egg names are displayed as
136they are processed.
137
138* For '''svnwiki''' document types, directory is typically {{/path/to/wiki/eggref/4}}.  The node name will be that of the file's basename, so file {{eggref/4/9p}} becomes node {{9p}}.
139* For '''eggdoc''' types, directory is typically {{/path/to/checkout/of/egg/repo}}, which is scanned for .meta files containing (eggdoc) attributes.  The node name is taken from the eggdoc's (name) attribute.
140
141The timestamp of the source file is stored in each node, and files which
142have not changed since they were last processed are skipped.  Use {{-f}}
143to disregard these timestamps and force reprocessing.
144
145   -m dir       process svnwiki manual directory DIR
146
147Process a copy of svnwiki Chicken manual directory DIR (usually
148{{/path/to/wiki/man/4}}).  Adds unit documentation to
149nodes at toplevel (Unit posix -> posix) and Chicken core
150documentation to nodes under toplevel node chicken
151(Parameters -> chicken parameters).
152
153A copy of the manual is usually installed in `(chicken-home)`
154under `doc/manual` -- for example, `/usr/local/share/chicken/doc/manual` --
155and you can use this copy instead of checking it out from SVN.
156
157{{chicken-doc-admin}} internally maps each manual page to a node path,
158based on its filename.
159
160   -E file [path...]     process egg file FILE
161
162Process an individual egg file FILE as in -e.  The resulting node path
163is usually determined from context, but you may set it manually with PATH.
164
165FILE may reside anywhere on disk, not just in a repository checkout.
166
167   -M file [path...]     process svnwiki man file FILE
168
169Process an individual svnwiki manual page FILE as in -m.  The node
170name is determined by an internal map of filenames to node paths.
171Alternatively, you can set it directly with PATH.
172
173   -H               process all installed egg documentation
174
175Process locally installed egg documentation.  Most eggs will keep their
176documentation on the wiki, but it may also be contained within and
177installed with the egg itself.  This is useful for unofficial, private or
178experimental eggs.
179
180   -H egg1 egg2...  process specified installed egg documentation
181
182Like {{-H}}, but only process the specified eggs.
183
184   -T               process all installed target egg documentation
185   -T egg1 egg2...  process specified installed target egg documentation
186
187Like {{-H}}, but when Chicken is built as a cross-compiler, it will process
188eggs installed for the target.  When a regular compiler instead, this has the
189same effect as {{-H}}.
190
191   -r           regenerate indices
192
193Regenerate documentation indices; at the moment, this is just one
194index, the node -> path search map.  Indexing is done automatically,
195so there is no need to use this option unless your index is somehow
196broken.
197
198   -f           force processing (ignore timestamp checks)
199
200The timestamp of the source file is stored in each node, and normally
201files which have not changed since they were last processed are
202skipped.  Use {{-f}} to disregard these timestamps and force
203reprocessing.  This option can be used before {{-e}}, {{-m}}, {{-E}} and {{-M}},
204which all consider stored timestamps by default.
205
206   -d path      delete node path recursively
207
208Delete node path PATH and everything under it.  Useful for removing,
209for example, an entire egg from the repository.
210
211'''WARNING''': If you do not provide a path, the root path () is used,
212which will delete the contents of the entire repository and leave
213a clean repository in its place.
214
215   -D           destroy repository
216
217Recursively deletes the repository base directory.  You must use
218{{-i}} to recreate the repository.
219
220=== Repository
221
222The repository layout produced by chicken-doc-admin's automatic egg
223and man parser is detailed in the documentation for the
224[[chicken-doc]] egg.
225
226To recap, documentation for each egg and unit is placed in a toplevel
227node named after that egg or unit, with procedure, macro,
228etc. identifiers for that unit as the node's children.  Chicken core
229man pages not corresponding to a particular unit are placed
230individually under the "chicken" toplevel node; for example,
231[[Non-standard macros and special forms]] resides under the path
232{{(chicken macros)}}.
233
234As with [[chicken-doc]], you can override the repository location
235by setting an environment variable:
236
237 export CHICKEN_DOC_REPOSITORY=/path/to/repository
238
239This is useful for testing and also if the default location, which is
240located in {{(chicken-home)}}, is not writable by you.  You can verify
241the current repository location with {{chicken-doc-admin -l}}.
242
243==== Low-level repository structure
244
245This structure is subject to change.
246
247 |-- .chicken-doc-repo    Repository magic file; contains repository info
248 |-- id.idx               Alist mapping node identifiers to paths
249 |-- lock                 Lock file to prevent write conflicts
250 |-- root/                Documentation root node ()
251     |-- 9p/              Documentation node (9p)
252     |    |- ,meta         (9p) metadata alist (signature "9p egg", type 'egg)
253     |    |- ,sxml         (9p) sxml document
254     |    |- ,defs         (9p) definitions (index ...) (def ...) (def ...)
255     |
256     |-- foreign/         Documentation node (foreign)
257          |- ,meta         (foreign) metadata alist + sxml document
258          |- access/      Documentation node (foreign access)
259             |- ,meta      (foreign access) metadata
260             |- ,sxml      (foreign access) sxml document
261             |- ,defs      (foreign access) definitions,
262                              e.g. for (foreign access foreign-code)
263
264Certain characters are %-escaped in filenames, such as / and period.
265
266When the {{,sxml}} document is under 3KB, it is packed into the {{,meta}} file.
267
268{{,defs}} nodes are virtual nodes containing definition sexprs extracted
269from the parent sxml document at parse time, plus an index.  They cannot
270be created, modified or destroyed except via their containing document,
271but are otherwise accessed as regular nodes.  {{,defs}} nodes first appeared
272in version 0.4.0; prior to that, they were "real" nodes and one directory
273was created per definition.
274
275Each node contains a timestamp corresponding to that of the source file.
276
277=== Proper wiki documentation
278
279''This section is a work in progress.''
280
281==== The short and skinny
282
283* Use svnwiki type tags (procedure, macro) around all identifiers
284  to be documented.  See [[wiki-syntax-chicken|Chicken-specific wiki syntax]].
285* Place related tagged identifiers on consecutive, non-blank lines;
286  the text description below is then used for all of them.  eggdocs
287  must use the proper grouping form, described below.
288* Ensure all identifier signatures can be read with a call to (read).
289  Don't use characters such as {{|}} (pipe).
290* Ensure open and close tags for bold, italic, links, code and
291  identifiers are located on the same line.
292
293==== svnwiki identifier type tags
294
295Each identifier is assigned a type (such as 'procedure, 'macro)
296corresponding to svnwiki tags "procedure", "macro" etc.  It is also
297assigned a signature which is taken verbatim from the svnwiki tag
298content.  Finally, the identifier name also comes from the signature;
299the signature is parsed using {{(read)}}, and the result may be a
300symbol (for example, a constant), or a list --- typically, a procedure
301or macro signature, in which case the first element of the list is
302used.  With rare exception, as when the signature contains an
303illegal character such as {{|}} (pipe), this strategy works well.
304
305If a signature cannot be parsed as above, the definition is discarded.
306As a special exception, read syntax signatures are used verbatim, so
307that you can look up read syntax like {{#u8}}.
308
309The currently recognized svnwiki tags are:
310
311 (define +identifier-tags+
312   (list "procedure" "macro" "read" "parameter"
313         "record" "string" "class" "method" "constant" "setter"))
314
315==== Identifier descriptions
316
317When creating an identifier description, the svnwiki parser takes all text
318from after the occurrence of the identifier tag up until the next
319section marker, or the next identifier tag.  This follows the informal
320standard for Chicken documentation on the wiki, and is a natural
321way to separate descriptions logically.  Some points to note:
322
323* The description stops even if the section marker denotes a nested
324  section.  This works well in general, although there are a few eggs
325  which do contain important identifier info in nested sections.
326  To get this information, you have to read the full documentation,
327  or rearrange the structure.
328* If several identifier signatures occur on consecutive lines, without
329  any blank lines between them, they are considered part of a "group"
330  of identifiers.  The following text description then applies to
331  every identifier in the group.  Again, this corresponds to common
332  practice on the wiki.
333* If you look up one identifier in a group, they will all appear,
334  along with the single text description.
335
336==== Eggdoc notes
337
338Identifier groups in eggdocs use the somewhat verbose form
339
340 (definition
341   (signatures
342    (signature "procedure" "(foo bar)")
343    (signature "procedure" "(baz bat)")
344    (signature "macro"     "(quux phlox)"))
345   (p "This describes foo, baz and quux."))
346
347Separating the definitions like {{(procedure ...) (procedure ...)}}
348instead defines independent, non-grouped procedures.  Nesting
349{{(procedure ... (procedure ...))}} is illegal.
350
351=== Local egg documentation for developers
352
353To include local egg documentation, create a text file in svnwiki
354format, call it ''{{<eggname>}}''{{.wiki}}, and place it in your
355toplevel source directory.
356
357 foo.setup
358 foo.meta
359 foo.scm
360 foo.wiki
361
362Then in your {{.setup}} file, add the {{.wiki}} file to your
363{{install-extension}} clause:
364
365 (install-extension
366   'foo
367    '("foo.so" "foo.import.so" "foo.wiki")    ;; add "foo.wiki" here
368    '((version "0.0.0")))
369
370'''Note:''' Egg documentation that is obtained from the wiki ''and''
371from installed eggs may cause confusing results for users.  The latest
372file timestamp wins out, but the timestamp reflects local install
373time, not remote change time.  Therefore, the active doc may depend on
374the user's order of execution of {{svn co}} on the wiki,
375{{chicken-install}} of the egg, {{chicken-doc-admin -e}}, and
376{{chicken-doc-admin -H}}.  In this case, the user may want to run
377{{chicken-doc-admin -H eggname}} explicitly instead of processing
378all eggs.
379
380=== API
381
382To be documented.
383
384=== About this egg
385
386==== Source
387
388[[https://github.com/ursetto/chicken-doc-admin]]
389
390==== Author
391
392Jim Ursetto
393
394==== Version history
395
396; 0.5.0 : Support running on Chicken 5. Workaround file locking bug in Chicken 5.0.0. Repo version 4 for r/w invariance; see [[chicken-doc]] 0.6.0. Drop eggdoc support for Chicken 5.
397; 0.4.8 : Support Chicken 5 man page layout
398; 0.4.7 : Summarize errors, warn (don't crash) if svnwiki parser raises exception.
399; 0.4.6 : Fix ID cache bug in 0.4.5 with directory recursion
400; 0.4.5 : Recurse into directories. Nicer output format (A/M/?) and summary.
401; 0.4.4 : Process locally-installed (host or target) egg documentation
402; 0.4.2 : Recognize 'Cross development' manpage
403; 0.4.1 : Ensure old definitions are removed from index when updating a node
404; 0.4.0 : Repository version 3; dedicated ,defs nodes (saves 80% or more space)
405; 0.3.12 : Eliminate manual reindexing; pack page ,sxml into ,meta when < 3072 bytes
406; 0.3.11 : Read syntax signatures are used verbatim (requires Chicken 4.4.2 or later)
407; 0.3.9 : process only new or modified nodes; display statistics
408; 0.3.7 : pack definition ,sxml into ,meta when < 3072 bytes; saves 20+MB
409; 0.3.4 : shared repository cache
410; 0.3.0 : store pages as SXML; improve parse; repo version 2
411; 0.2.2 : Improve manpage handling
412; 0.2 : Support eggdoc
413; 0.1.1 : Initial release
414
415==== License
416
417Copyright (c) 2010-2019 Jim Ursetto.  License: 3-clause BSD.
418
419
Note: See TracBrowser for help on using the repository browser.