source: project/wiki/eggref/5/chicken-doc-admin @ 37636

Last change on this file since 37636 was 37636, checked in by wasamasa, 7 weeks ago

Fix broken link

File size: 16.5 KB
Line 
1[[tags: egg]]
2== chicken-doc-admin
3
4chicken-doc-admin is a tool for generating and maintaining
5Chicken documentation for the [[chicken-doc]] egg.
6[[toc:]]
7=== Overview
8
9[[chicken-doc-admin]] provides facilities to create and modify a Chicken
10documentation repository for [[chicken-doc]].  It includes a command-line
11tool to convert egg documentation and manual pages from the Chicken
12wiki for use with [[chicken-doc]].  It also provides an API for
13repository manipulation.
14
15The typical mode of using [[chicken-doc-admin]] is to check out a copy of
16the wiki using Subversion, then run the [[chicken-doc-admin]] command
17against the copy's egg and man directories.  Optionally, process
18eggdocs by running [[chicken-doc-admin]] against an egg repository checkout.
19Repeat when either is updated; only changed files will be reexamined.
20
21Eggs can also include bundled documentation rather than sourcing it
22from the wiki; [[chicken-doc-admin]] can parse these automatically
23as well.  Typically, though, official eggs use the wiki.
24
25=== Quick start
26
27The default repository location is in the Chicken install tree and may
28not be writable by your user.  If so, either use {{sudo}} or set an
29alternate writable location with:
30
31 $ export CHICKEN_DOC_REPOSITORY=/path/to/some/writable/directory
32
33Initialize the repository if this is your first time:
34
35 $ chicken-doc-admin -i
36
37Check out the wiki documentation:
38
39 $ svn co --username anonymous --password "" \
40      https://code.call-cc.org/svn/chicken-eggs/wiki
41
42To import Chicken 5 documentation:
43
44 $ chicken-doc-admin -m wiki/man/5
45 $ chicken-doc-admin -e wiki/eggref/5
46
47To import Chicken 4 documentation, use 4 instead of 5:
48
49 $ chicken-doc-admin -m wiki/man/4
50 $ chicken-doc-admin -e wiki/eggref/4
51
52{{chicken-doc-admin}} is aware of the structure of both Chicken 4 and Chicken 5
53manual pages and will automatically import them appropriately. Both Chicken 4
54and 5 can serve either documentation version.
55
56
57==== Locally installed egg documentation
58
59Certain eggs may install wiki documentation during a
60{{chicken-install}}.  Although uncommon, this allows unofficial,
61private or experimental eggs to include documentation without making
62it public on the wiki.  To do this with your own eggs, see [[#Local egg documentation for developers]].
63
64Process all locally installed documentation:
65
66 $ chicken-doc-admin -H
67
68Process docs just for the {{inane}} and {{pathfinder}} eggs:
69
70 $ chicken-doc-admin -H inane pathfinder
71
72Working example:
73
74 hg clone https://bitbucket.org/ursetto/inane
75 cd inane
76 chicken-install
77 cd ..
78 chicken-doc-admin -H inane
79
80The {{cd ..}} is only for illustrative purposes here to show you can
81process installed docs from any directory.  (Since you checked out
82from source, you could instead have done {{chicken-doc-admin -E inane}}
83from the source directory to process {{./inane.wiki}}.)
84
85'''Note:''' If egg documentation is both on the wiki ''and'' locally
86installed, and you process both, then the latest file timestamp wins.
87Unfortunately, these timestamps reflect not the actual update time
88but rather the install time on your local system, and are of
89dubious value in this case. 
90
91==== Optional eggdoc processing
92
93''Almost all users can skip this step'', because there are only a few
94remaining eggs that use eggdoc. The Chicken 5 version of this egg drops
95support for eggdoc.
96
97Download the egg repository and process any eggdocs:
98
99* '''Warning''': eggdocs are executable Scheme code, so if you are
100concerned about untrusted code, then skip this step or do not run
101it as root.  This risk is reasonably low because anonymous uploads
102are not allowed.
103
104 $ svn co --username anonymous --password "" \
105      http://code.call-cc.org/svn/chicken-eggs/release/4 eggs
106 $ chicken-doc-admin -t eggdoc -e eggs
107
108=== Upgrading
109
110After an upgrade of the egg, if you get a message informing you that
111the repository version is invalid, you will need to delete and
112recreate your repository.  For example:
113
114 $ chicken-doc-admin -D
115 $ chicken-doc-admin -i
116 $ chicken-doc-admin -m ...
117 $ chicken-doc-admin -e ...
118
119=== Command-line
120
121 $ chicken-doc-admin COMMAND
122
123   -l           list repository information
124
125Print some information about the repository, such as its location
126and version.
127
128   -i           initialize repository non-destructively
129
130Initialize the repository in the current default location or as
131overridden by {{CHICKEN_DOC_REPOSITORY}}.  The directory will be
132created if it does not exist.  This command must be run before any
133documentation can be processed.  You should usually initialize an
134empty directory; initializing a populated directory won't delete
135anything, and may confuse us.
136
137   -t type      document type
138
139Set the source document type, which may be {{eggdoc}} or {{svnwiki}},
140defaulting to {{svnwiki}}.  This option can be used before -e, -E, -m, -M.
141
142To process type {{eggdoc}}, you must install the {{eggdoc-svnwiki}} extension.
143
144   -e dir       process egg directory DIR
145
146Process a copy of egg directory DIR, adding each egg documentation page
147to a node at toplevel and the identifiers contained in the
148page to that node's children.  Egg names are displayed as
149they are processed.
150
151* For '''svnwiki''' document types, directory is typically {{/path/to/wiki/eggref/4}}.  The node name will be that of the file's basename, so file {{eggref/4/9p}} becomes node {{9p}}.
152* For '''eggdoc''' types, directory is typically {{/path/to/checkout/of/egg/repo}}, which is scanned for .meta files containing (eggdoc) attributes.  The node name is taken from the eggdoc's (name) attribute.
153
154The timestamp of the source file is stored in each node, and files which
155have not changed since they were last processed are skipped.  Use {{-f}}
156to disregard these timestamps and force reprocessing.
157
158   -m dir       process svnwiki manual directory DIR
159
160Process a copy of svnwiki Chicken manual directory DIR (usually
161{{/path/to/wiki/man/4}}).  Adds unit documentation to
162nodes at toplevel (Unit posix -> posix) and Chicken core
163documentation to nodes under toplevel node chicken
164(Parameters -> chicken parameters).
165
166A copy of the manual is usually installed in `(chicken-home)`
167under `doc/manual` -- for example, `/usr/local/share/chicken/doc/manual` --
168and you can use this copy instead of checking it out from SVN.
169
170{{chicken-doc-admin}} internally maps each manual page to a node path,
171based on its filename.
172
173   -E file [path...]     process egg file FILE
174
175Process an individual egg file FILE as in -e.  The resulting node path
176is usually determined from context, but you may set it manually with PATH.
177
178FILE may reside anywhere on disk, not just in a repository checkout.
179
180   -M file [path...]     process svnwiki man file FILE
181
182Process an individual svnwiki manual page FILE as in -m.  The node
183name is determined by an internal map of filenames to node paths.
184Alternatively, you can set it directly with PATH.
185
186   -H               process all installed egg documentation
187
188Process locally installed egg documentation.  Most eggs will keep their
189documentation on the wiki, but it may also be contained within and
190installed with the egg itself.  This is useful for unofficial, private or
191experimental eggs.
192
193   -H egg1 egg2...  process specified installed egg documentation
194
195Like {{-H}}, but only process the specified eggs.
196
197   -T               process all installed target egg documentation
198   -T egg1 egg2...  process specified installed target egg documentation
199
200Like {{-H}}, but when Chicken is built as a cross-compiler, it will process
201eggs installed for the target.  When a regular compiler instead, this has the
202same effect as {{-H}}.
203
204   -r           regenerate indices
205
206Regenerate documentation indices; at the moment, this is just one
207index, the node -> path search map.  Indexing is done automatically,
208so there is no need to use this option unless your index is somehow
209broken.
210
211   -f           force processing (ignore timestamp checks)
212
213The timestamp of the source file is stored in each node, and normally
214files which have not changed since they were last processed are
215skipped.  Use {{-f}} to disregard these timestamps and force
216reprocessing.  This option can be used before {{-e}}, {{-m}}, {{-E}} and {{-M}},
217which all consider stored timestamps by default.
218
219   -d path      delete node path recursively
220
221Delete node path PATH and everything under it.  Useful for removing,
222for example, an entire egg from the repository.
223
224'''WARNING''': If you do not provide a path, the root path () is used,
225which will delete the contents of the entire repository and leave
226a clean repository in its place.
227
228   -D           destroy repository
229
230Recursively deletes the repository base directory.  You must use
231{{-i}} to recreate the repository.
232
233=== Repository
234
235The repository layout produced by chicken-doc-admin's automatic egg
236and man parser is detailed in the documentation for the
237[[chicken-doc]] egg.
238
239To recap, documentation for each egg and unit is placed in a toplevel
240node named after that egg or unit, with procedure, macro,
241etc. identifiers for that unit as the node's children.  Chicken core
242man pages not corresponding to a particular unit are placed
243individually under the "chicken" toplevel node; for example,
244[[Non-standard macros and special forms]] resides under the path
245{{(chicken macros)}}.
246
247As with [[chicken-doc]], you can override the repository location
248by setting an environment variable:
249
250 export CHICKEN_DOC_REPOSITORY=/path/to/repository
251
252This is useful for testing and also if the default location, which is
253located in {{(chicken-home)}}, is not writable by you.  You can verify
254the current repository location with {{chicken-doc-admin -l}}.
255
256==== Low-level repository structure
257
258This structure is subject to change.
259
260 |-- .chicken-doc-repo    Repository magic file; contains repository info
261 |-- id.idx               Alist mapping node identifiers to paths
262 |-- lock                 Lock file to prevent write conflicts
263 |-- root/                Documentation root node ()
264     |-- 9p/              Documentation node (9p)
265     |    |- ,meta         (9p) metadata alist (signature "9p egg", type 'egg)
266     |    |- ,sxml         (9p) sxml document
267     |    |- ,defs         (9p) definitions (index ...) (def ...) (def ...)
268     |
269     |-- foreign/         Documentation node (foreign)
270          |- ,meta         (foreign) metadata alist + sxml document
271          |- access/      Documentation node (foreign access)
272             |- ,meta      (foreign access) metadata
273             |- ,sxml      (foreign access) sxml document
274             |- ,defs      (foreign access) definitions,
275                              e.g. for (foreign access foreign-code)
276
277Certain characters are %-escaped in filenames, such as / and period.
278
279When the {{,sxml}} document is under 3KB, it is packed into the {{,meta}} file.
280
281{{,defs}} nodes are virtual nodes containing definition sexprs extracted
282from the parent sxml document at parse time, plus an index.  They cannot
283be created, modified or destroyed except via their containing document,
284but are otherwise accessed as regular nodes.  {{,defs}} nodes first appeared
285in version 0.4.0; prior to that, they were "real" nodes and one directory
286was created per definition.
287
288Each node contains a timestamp corresponding to that of the source file.
289
290=== Proper wiki documentation
291
292''This section is a work in progress.''
293
294==== The short and skinny
295
296* Use svnwiki type tags (procedure, macro) around all identifiers
297  to be documented.  See [[/edit-help|Chicken-specific wiki syntax]].
298* Place related tagged identifiers on consecutive, non-blank lines;
299  the text description below is then used for all of them.  eggdocs
300  must use the proper grouping form, described below.
301* Ensure all identifier signatures can be read with a call to (read).
302  Don't use characters such as {{|}} (pipe).
303* Ensure open and close tags for bold, italic, links, code and
304  identifiers are located on the same line.
305
306==== svnwiki identifier type tags
307
308Each identifier is assigned a type (such as 'procedure, 'macro)
309corresponding to svnwiki tags "procedure", "macro" etc.  It is also
310assigned a signature which is taken verbatim from the svnwiki tag
311content.  Finally, the identifier name also comes from the signature;
312the signature is parsed using {{(read)}}, and the result may be a
313symbol (for example, a constant), or a list --- typically, a procedure
314or macro signature, in which case the first element of the list is
315used.  With rare exception, as when the signature contains an
316illegal character such as {{|}} (pipe), this strategy works well.
317
318If a signature cannot be parsed as above, the definition is discarded.
319As a special exception, read syntax signatures are used verbatim, so
320that you can look up read syntax like {{#u8}}.
321
322The currently recognized svnwiki tags are:
323
324 (define +identifier-tags+
325   (list "procedure" "macro" "read" "parameter"
326         "record" "string" "class" "method" "constant" "setter"))
327
328==== Identifier descriptions
329
330When creating an identifier description, the svnwiki parser takes all text
331from after the occurrence of the identifier tag up until the next
332section marker, or the next identifier tag.  This follows the informal
333standard for Chicken documentation on the wiki, and is a natural
334way to separate descriptions logically.  Some points to note:
335
336* The description stops even if the section marker denotes a nested
337  section.  This works well in general, although there are a few eggs
338  which do contain important identifier info in nested sections.
339  To get this information, you have to read the full documentation,
340  or rearrange the structure.
341* If several identifier signatures occur on consecutive lines, without
342  any blank lines between them, they are considered part of a "group"
343  of identifiers.  The following text description then applies to
344  every identifier in the group.  Again, this corresponds to common
345  practice on the wiki.
346* If you look up one identifier in a group, they will all appear,
347  along with the single text description.
348
349==== Eggdoc notes
350
351Identifier groups in eggdocs use the somewhat verbose form
352
353 (definition
354   (signatures
355    (signature "procedure" "(foo bar)")
356    (signature "procedure" "(baz bat)")
357    (signature "macro"     "(quux phlox)"))
358   (p "This describes foo, baz and quux."))
359
360Separating the definitions like {{(procedure ...) (procedure ...)}}
361instead defines independent, non-grouped procedures.  Nesting
362{{(procedure ... (procedure ...))}} is illegal.
363
364=== Local egg documentation for developers
365
366To include local egg documentation, create a text file in svnwiki
367format, call it ''{{<eggname>}}''{{.wiki}}, and place it in your
368toplevel source directory.
369
370 foo.setup
371 foo.meta
372 foo.scm
373 foo.wiki
374
375Then in your {{.setup}} file, add the {{.wiki}} file to your
376{{install-extension}} clause:
377
378 (install-extension
379   'foo
380    '("foo.so" "foo.import.so" "foo.wiki")    ;; add "foo.wiki" here
381    '((version "0.0.0")))
382
383'''Note:''' Egg documentation that is obtained from the wiki ''and''
384from installed eggs may cause confusing results for users.  The latest
385file timestamp wins out, but the timestamp reflects local install
386time, not remote change time.  Therefore, the active doc may depend on
387the user's order of execution of {{svn co}} on the wiki,
388{{chicken-install}} of the egg, {{chicken-doc-admin -e}}, and
389{{chicken-doc-admin -H}}.  In this case, the user may want to run
390{{chicken-doc-admin -H eggname}} explicitly instead of processing
391all eggs.
392
393=== API
394
395To be documented.
396
397=== About this egg
398
399==== Source
400
401[[https://github.com/ursetto/chicken-doc-admin]]
402
403==== Author
404
405Jim Ursetto
406
407==== Version history
408
409; 0.5.0 : Support running on Chicken 5. Workaround file locking bug in Chicken 5.0.0. Repo version 4 for r/w invariance; see [[chicken-doc]] 0.6.0. Drop eggdoc support for Chicken 5.
410; 0.4.8 : Support Chicken 5 man page layout
411; 0.4.7 : Summarize errors, warn (don't crash) if svnwiki parser raises exception.
412; 0.4.6 : Fix ID cache bug in 0.4.5 with directory recursion
413; 0.4.5 : Recurse into directories. Nicer output format (A/M/?) and summary.
414; 0.4.4 : Process locally-installed (host or target) egg documentation
415; 0.4.2 : Recognize 'Cross development' manpage
416; 0.4.1 : Ensure old definitions are removed from index when updating a node
417; 0.4.0 : Repository version 3; dedicated ,defs nodes (saves 80% or more space)
418; 0.3.12 : Eliminate manual reindexing; pack page ,sxml into ,meta when < 3072 bytes
419; 0.3.11 : Read syntax signatures are used verbatim (requires Chicken 4.4.2 or later)
420; 0.3.9 : process only new or modified nodes; display statistics
421; 0.3.7 : pack definition ,sxml into ,meta when < 3072 bytes; saves 20+MB
422; 0.3.4 : shared repository cache
423; 0.3.0 : store pages as SXML; improve parse; repo version 2
424; 0.2.2 : Improve manpage handling
425; 0.2 : Support eggdoc
426; 0.1.1 : Initial release
427
428==== License
429
430Copyright (c) 2010-2019 Jim Ursetto.  License: 3-clause BSD.
431
432
Note: See TracBrowser for help on using the repository browser.