source: project/wiki/eggref/4/chicken-doc-admin @ 28589

Last change on this file since 28589 was 28589, checked in by zbigniew, 6 years ago

wiki/chicken-doc-admin: 0.4.4: Process locally-installed (host or target) egg documentation

File size: 15.6 KB
Line 
1[[tags: egg]]
2== chicken-doc-admin
3
4chicken-doc-admin is a tool for generating and maintaining
5Chicken documentation for the [[chicken-doc]] egg.
6[[toc:]]
7=== Overview
8
9[[chicken-doc-admin]] provides facilities to create and modify a Chicken
10documentation repository for [[chicken-doc]].  It includes a command-line
11tool to convert egg documentation and manual pages from the Chicken
12wiki for use with [[chicken-doc]].  It also provides an API for
13repository manipulation.
14
15The typical mode of using [[chicken-doc-admin]] is to check out a copy of
16the wiki using Subversion, then run the [[chicken-doc-admin]] command
17against the copy's egg and man directories.  Optionally, process
18eggdocs by running [[chicken-doc-admin]] against an egg repository checkout.
19Repeat when either is updated; only changed files will be reexamined.
20
21Eggs can also include bundled documentation rather than sourcing it
22from the wiki; [[chicken-doc-admin]] can parse these automatically
23as well.  Typically, though, official eggs use the wiki.
24
25=== Quick start
26
27The default repository location is in the Chicken install tree and may
28not be writable by your user.  If so, either use {{sudo}} or set an
29alternate writable location with:
30
31 $ export CHICKEN_DOC_REPOSITORY=/path/to/some/writable/directory
32
33Initialize the repository if this is your first time:
34
35 $ chicken-doc-admin -i
36
37Now download and process the wiki documentation:
38
39 $ svn co --username anonymous --password "" \
40      http://code.call-cc.org/svn/chicken-eggs/wiki
41 $ chicken-doc-admin -m wiki/man/4
42 $ chicken-doc-admin -e wiki/eggref/4
43
44==== Locally installed egg documentation
45
46Certain eggs may install wiki documentation during a
47{{chicken-install}}.  Although uncommon, this allows unofficial,
48private or experimental eggs to include documentation without making
49it public on the wiki.  To do this with your own eggs, see [[#Local egg documentation for developers]].
50
51Process all locally installed documentation:
52
53 $ chicken-doc-admin -H
54
55Process docs just for the {{inane}} and {{pathfinder}} eggs:
56
57 $ chicken-doc-admin -H inane pathfinder
58
59Working example:
60
61 hg clone https://bitbucket.org/ursetto/inane
62 cd inane
63 chicken-install
64 cd ..
65 chicken-doc-admin -H inane
66
67The {{cd ..}} is only for illustrative purposes here to show you can
68process installed docs from any directory.  (Since you checked out
69from source, you could instead have done {{chicken-doc-admin -E inane}}
70from the source directory to process {{./inane.wiki}}.)
71
72'''Note:''' If egg documentation is both on the wiki ''and'' locally
73installed, and you process both, then the latest file timestamp wins.
74Unfortunately, these timestamps reflect not the actual update time
75but rather the install time on your local system, and are of
76dubious value in this case. 
77
78==== Optional eggdoc processing
79
80''Almost all users can skip this step'', because there are only a few
81remaining eggs that use eggdoc.
82
83Download the egg repository and process any eggdocs:
84
85* '''Warning''': eggdocs are executable Scheme code, so if you are
86concerned about untrusted code, then skip this step or do not run
87it as root.  This risk is reasonably low because anonymous uploads
88are not allowed.
89
90 $ svn co --username anonymous --password "" \
91      http://code.call-cc.org/svn/chicken-eggs/release/4 eggs
92 $ chicken-doc-admin -t eggdoc -e eggs
93
94=== Upgrading
95
96After an upgrade of the egg, if you get a message informing you that
97the repository version is invalid, you will need to delete and
98recreate your repository.  For example:
99
100 $ chicken-doc-admin -D
101 $ chicken-doc-admin -i
102 $ chicken-doc-admin -m ...
103 $ chicken-doc-admin -e ...
104
105=== Command-line
106
107 $ chicken-doc-admin COMMAND
108
109   -l           list repository information
110
111Print some information about the repository, such as its location
112and version.
113
114   -i           initialize repository non-destructively
115
116Initialize the repository in the current default location or as
117overridden by {{CHICKEN_DOC_REPOSITORY}}.  The directory will be
118created if it does not exist.  This command must be run before any
119documentation can be processed.  You should usually initialize an
120empty directory; initializing a populated directory won't delete
121anything, and may confuse us.
122
123   -t type      document type
124
125Set the source document type, which may be {{eggdoc}} or {{svnwiki}},
126defaulting to {{svnwiki}}.  This option can be used before -e, -E, -m, -M.
127
128To process type {{eggdoc}}, you must install the {{eggdoc-svnwiki}} extension.
129
130   -e dir       process egg directory DIR
131
132Process a copy of egg directory DIR, adding each egg documentation page
133to a node at toplevel and the identifiers contained in the
134page to that node's children.  Egg names are displayed as
135they are processed.
136
137* For '''svnwiki''' document types, directory is typically {{/path/to/wiki/eggref/4}}.  The node name will be that of the file's basename, so file {{eggref/4/9p}} becomes node {{9p}}.
138* For '''eggdoc''' types, directory is typically {{/path/to/checkout/of/egg/repo}}, which is scanned for .meta files containing (eggdoc) attributes.  The node name is taken from the eggdoc's (name) attribute.
139
140The timestamp of the source file is stored in each node, and files which
141have not changed since they were last processed are skipped.  Use {{-f}}
142to disregard these timestamps and force reprocessing.
143
144   -m dir       process svnwiki manual directory DIR
145
146Process a copy of svnwiki Chicken manual directory DIR (usually
147{{/path/to/wiki/man/4}}).  Adds unit documentation to
148nodes at toplevel (Unit posix -> posix) and Chicken core
149documentation to nodes under toplevel node chicken
150(Parameters -> chicken parameters).
151
152A copy of the manual is usually installed in `(chicken-home)`
153under `doc/manual` -- for example, `/usr/local/share/chicken/doc/manual` --
154and you can use this copy instead of checking it out from SVN.
155
156{{chicken-doc-admin}} internally maps each manual page to a node path,
157based on its filename.
158
159   -E file [path...]     process egg file FILE
160
161Process an individual egg file FILE as in -e.  The resulting node path
162is usually determined from context, but you may set it manually with PATH.
163
164FILE may reside anywhere on disk, not just in a repository checkout.
165
166   -M file [path...]     process svnwiki man file FILE
167
168Process an individual svnwiki manual page FILE as in -m.  The node
169name is determined by an internal map of filenames to node paths.
170Alternatively, you can set it directly with PATH.
171
172   -H               process all installed egg documentation
173
174Process locally installed egg documentation.  Most eggs will keep their
175documentation on the wiki, but it may also be contained within and
176installed with the egg itself.  This is useful for unofficial, private or
177experimental eggs.
178
179   -H egg1 egg2...  process specified installed egg documentation
180
181Like {{-H}}, but only process the specified eggs.
182
183   -T               process all installed target egg documentation
184   -T egg1 egg2...  process specified installed target egg documentation
185
186Like {{-H}}, but when Chicken is built as a cross-compiler, it will process
187eggs installed for the target.  When a regular compiler instead, this has the
188same effect as {{-H}}.
189
190   -r           regenerate indices
191
192Regenerate documentation indices; at the moment, this is just one
193index, the node -> path search map.  Indexing is done automatically,
194so there is no need to use this option unless your index is somehow
195broken.
196
197   -f           force processing (ignore timestamp checks)
198
199The timestamp of the source file is stored in each node, and normally
200files which have not changed since they were last processed are
201skipped.  Use {{-f}} to disregard these timestamps and force
202reprocessing.  This option can be used before {{-e}}, {{-m}}, {{-E}} and {{-M}},
203which all consider stored timestamps by default.
204
205   -d path      delete node path recursively
206
207Delete node path PATH and everything under it.  Useful for removing,
208for example, an entire egg from the repository.
209
210'''WARNING''': If you do not provide a path, the root path () is used,
211which will delete the contents of the entire repository and leave
212a clean repository in its place.
213
214   -D           destroy repository
215
216Recursively deletes the repository base directory.  You must use
217{{-i}} to recreate the repository.
218
219=== Repository
220
221The repository layout produced by chicken-doc-admin's automatic egg
222and man parser is detailed in the documentation for the
223[[chicken-doc]] egg.
224
225To recap, documentation for each egg and unit is placed in a toplevel
226node named after that egg or unit, with procedure, macro,
227etc. identifiers for that unit as the node's children.  Chicken core
228man pages not corresponding to a particular unit are placed
229individually under the "chicken" toplevel node; for example,
230[[Non-standard macros and special forms]] resides under the path
231{{(chicken macros)}}.
232
233As with [[chicken-doc]], you can override the repository location
234by setting an environment variable:
235
236 export CHICKEN_DOC_REPOSITORY=/path/to/repository
237
238This is useful for testing and also if the default location, which is
239located in {{(chicken-home)}}, is not writable by you.  You can verify
240the current repository location with {{chicken-doc-admin -l}}.
241
242==== Low-level repository structure
243
244This structure is subject to change.
245
246 |-- .chicken-doc-repo    Repository magic file; contains repository info
247 |-- id.idx               Alist mapping node identifiers to paths
248 |-- lock                 Lock file to prevent write conflicts
249 |-- root/                Documentation root node ()
250     |-- 9p/              Documentation node (9p)
251     |    |- ,meta         (9p) metadata alist (signature "9p egg", type 'egg)
252     |    |- ,sxml         (9p) sxml document
253     |    |- ,defs         (9p) definitions (index ...) (def ...) (def ...)
254     |
255     |-- foreign/         Documentation node (foreign)
256          |- ,meta         (foreign) metadata alist + sxml document
257          |- access/      Documentation node (foreign access)
258             |- ,meta      (foreign access) metadata
259             |- ,sxml      (foreign access) sxml document
260             |- ,defs      (foreign access) definitions,
261                              e.g. for (foreign access foreign-code)
262
263Certain characters are %-escaped in filenames, such as / and period.
264
265When the {{,sxml}} document is under 3KB, it is packed into the {{,meta}} file.
266
267{{,defs}} nodes are virtual nodes containing definition sexprs extracted
268from the parent sxml document at parse time, plus an index.  They cannot
269be created, modified or destroyed except via their containing document,
270but are otherwise accessed as regular nodes.  {{,defs}} nodes first appeared
271in version 0.4.0; prior to that, they were "real" nodes and one directory
272was created per definition.
273
274Each node contains a timestamp corresponding to that of the source file.
275
276=== Proper wiki documentation
277
278''This section is a work in progress.''
279
280==== The short and skinny
281
282* Use svnwiki type tags (procedure, macro) around all identifiers
283  to be documented.  See [[wiki-syntax-chicken|Chicken-specific wiki syntax]].
284* Place related tagged identifiers on consecutive, non-blank lines;
285  the text description below is then used for all of them.  eggdocs
286  must use the proper grouping form, described below.
287* Ensure all identifier signatures can be read with a call to (read).
288  Don't use characters such as {{|}} (pipe).
289* Ensure open and close tags for bold, italic, links, code and
290  identifiers are located on the same line.
291
292==== svnwiki identifier type tags
293
294Each identifier is assigned a type (such as 'procedure, 'macro)
295corresponding to svnwiki tags "procedure", "macro" etc.  It is also
296assigned a signature which is taken verbatim from the svnwiki tag
297content.  Finally, the identifier name also comes from the signature;
298the signature is parsed using {{(read)}}, and the result may be a
299symbol (for example, a constant), or a list --- typically, a procedure
300or macro signature, in which case the first element of the list is
301used.  With rare exception, as when the signature contains an
302illegal character such as {{|}} (pipe), this strategy works well.
303
304If a signature cannot be parsed as above, the definition is discarded.
305As a special exception, read syntax signatures are used verbatim, so
306that you can look up read syntax like {{#u8}}.
307
308The currently recognized svnwiki tags are:
309
310 (define +identifier-tags+
311   (list "procedure" "macro" "read" "parameter"
312         "record" "string" "class" "method" "constant" "setter"))
313
314==== Identifier descriptions
315
316When creating an identifier description, the svnwiki parser takes all text
317from after the occurrence of the identifier tag up until the next
318section marker, or the next identifier tag.  This follows the informal
319standard for Chicken documentation on the wiki, and is a natural
320way to separate descriptions logically.  Some points to note:
321
322* The description stops even if the section marker denotes a nested
323  section.  This works well in general, although there are a few eggs
324  which do contain important identifier info in nested sections.
325  To get this information, you have to read the full documentation,
326  or rearrange the structure.
327* If several identifier signatures occur on consecutive lines, without
328  any blank lines between them, they are considered part of a "group"
329  of identifiers.  The following text description then applies to
330  every identifier in the group.  Again, this corresponds to common
331  practice on the wiki.
332* If you look up one identifier in a group, they will all appear,
333  along with the single text description.
334
335==== Eggdoc notes
336
337Identifier groups in eggdocs use the somewhat verbose form
338
339 (definition
340   (signatures
341    (signature "procedure" "(foo bar)")
342    (signature "procedure" "(baz bat)")
343    (signature "macro"     "(quux phlox)"))
344   (p "This describes foo, baz and quux."))
345
346Separating the definitions like {{(procedure ...) (procedure ...)}}
347instead defines independent, non-grouped procedures.  Nesting
348{{(procedure ... (procedure ...))}} is illegal.
349
350=== Local egg documentation for developers
351
352To include local egg documentation, create a text file in svnwiki
353format, call it ''{{<eggname>}}''{{.wiki}}, and place it in your
354toplevel source directory.
355
356 foo.setup
357 foo.meta
358 foo.scm
359 foo.wiki
360
361Then in your {{.setup}} file, add the {{.wiki}} file to your
362{{install-extension}} clause:
363
364 (install-extension
365   'foo
366    '("foo.so" "foo.import.so" "foo.wiki")    ;; add "foo.wiki" here
367    '((version "0.0.0")))
368
369'''Note:''' Egg documentation that is obtained from the wiki ''and''
370from installed eggs may cause confusing results for users.  The latest
371file timestamp wins out, but the timestamp reflects local install
372time, not remote change time.  Therefore, the active doc may depend on
373the user's order of execution of {{svn co}} on the wiki,
374{{chicken-install}} of the egg, {{chicken-doc-admin -e}}, and
375{{chicken-doc-admin -H}}.  In this case, the user may want to run
376{{chicken-doc-admin -H eggname}} explicitly instead of processing
377all eggs.
378
379=== API
380
381To be documented.
382
383=== About this egg
384
385==== Author
386
387Jim Ursetto
388
389==== Version history
390
391; 0.4.4 : Process locally-installed (host or target) egg documentation
392; 0.4.2 : Recognize 'Cross development' manpage
393; 0.4.1 : Ensure old definitions are removed from index when updating a node
394; 0.4.0 : Repository version 3; dedicated ,defs nodes (saves 80% or more space)
395; 0.3.12 : Eliminate manual reindexing; pack page ,sxml into ,meta when < 3072 bytes
396; 0.3.11 : Read syntax signatures are used verbatim (requires Chicken 4.4.2 or later)
397; 0.3.9 : process only new or modified nodes; display statistics
398; 0.3.7 : pack definition ,sxml into ,meta when < 3072 bytes; saves 20+MB
399; 0.3.4 : shared repository cache
400; 0.3.0 : store pages as SXML; improve parse; repo version 2
401; 0.2.2 : Improve manpage handling
402; 0.2 : Support eggdoc
403; 0.1.1 : Initial release
404
405==== License
406
407Copyright (c) 2010 Jim Ursetto.  License: 3-clause BSD.
408
409
Note: See TracBrowser for help on using the repository browser.