source: project/wiki/eggref/4/intarweb @ 27238

Last change on this file since 27238 was 27238, checked in by sjamaan, 9 years ago

Fix typo

File size: 41.2 KB
Line 
1[[tags: egg]]
2
3== Intarweb
4
5[[toc:]]
6
7=== Description
8
9Intarweb is an advanced http library.  It parses all headers into more
10useful Scheme values.  It serves as a low-level basis for http servers
11or clients.  For a more high-level server library that is based on
12intarweb, see [[spiffy]].  For a high-level client library based on
13intarweb, see [[http-client]].
14
15You would rarely need to build an application directly on top of raw
16Intarweb, but when using one of the above-mentioned libraries you
17often need to interact with intarweb's API.
18
19=== Author
20
21[[/users/peter-bex|Peter Bex]]
22
23=== Requirements
24
25Requires the [[defstruct]], [[base64]] and [[uri-common]] extensions.
26
27=== Documentation
28
29The intarweb egg is designed to be used from a variety of
30situations. For this reason, it does not try to be a full HTTP client
31or server. If you need that kind of functionality, see eggs like
32[[spiffy]] or [[http-client]].
33
34=== Requests
35
36<procedure>(make-request #!key uri port (method 'GET) (major 1) (minor 1) (headers (make-headers '())))</procedure>
37
38Create a request object (a [[defstruct]]-type record). The client will
39generally write requests, while the server will read them.
40
41The URI defines the entity to retrieve on the server, which should be
42a [[uri-common]]-type URI object. The PORT is the scheme I/O port
43where the request is written to or read from.  The METHOD is a symbol
44that defines the HTTP method to use (case sensitive). MAJOR and MINOR
45identify the major and minor version of HTTP to use. Currently, 0.9,
461.0 and 1.1 are supported (but be careful with 0.9, it has some weird
47consequences and is not widely supported). HEADERS must be a headers
48object.
49
50<procedure>(update-request old-request #!key uri port method major minor headers)</procedure>
51
52Like {{make-request}}, except this takes an {{old-request}} object as
53a template for values which are missing from the parameter list, thereby
54providing a way to do a purely functional update of that object.
55
56
57<procedure>(request-uri REQUEST) => URI</procedure><br>
58<procedure>(request-port REQUEST) => PORT</procedure><br>
59<procedure>(request-method REQUEST) => SYMBOL</procedure><br>
60<procedure>(request-major REQUEST) => NUMBER</procedure><br>
61<procedure>(request-minor REQUEST) => NUMBER</procedure><br>
62<procedure>(request-headers REQUEST) => HEADERS</procedure><br>
63
64An existing request can be picked apart with these accessors.
65
66<procedure>(write-request REQUEST) => REQUEST</procedure>
67
68Write a request line with headers to the server.  In case it
69is a request type that has any body data, this should be written to
70the the request's port. Beware that this port can be modified by
71write-request, so be sure to write to the port as it is returned by
72the write-request procedure!
73
74<procedure>(read-request PORT) => REQUEST</procedure>
75
76Reads a request object from the given input-port.  An optional request
77body can be read from the request-port after calling this procedure.
78
79<parameter>(request-parsers [LIST])</parameter>
80
81Requests are parsed using parse procedures, which can be customized
82by overriding this parameter.
83
84LIST is a list of procedures which accept a request line
85string and produce a request object, or {{#f}} if the
86request is not of the type handled by that procedure.
87
88The predefined request parsers are:
89
90* {{http-0.9-request-parser}}
91* {{http-1.x-request-parser}}
92
93<procedure>(http-0.9-request-parser STRING) => REQUEST</procedure><br>
94<procedure>(http-1.x-request-parser STRING) => REQUEST</procedure><br>
95
96Predefined request parsers for use with {{request-parsers}}.
97
98<parameter>(request-unparsers [LIST])</parameter>
99
100Requests are written using unparse procedures, which can be
101customized by overriding this parameter.
102
103LIST is list of procedures which accept a request object and write
104to the request's output port and return the new, possibly updated
105request object. If the request object is not unparsed by this
106handler, it returns {{#f}}.
107
108The predefined request unparsers are:
109
110* {{http-0.9-request-unparser}}
111* {{http-1.x-request-unparser}}
112
113<procedure>(http-0.9-request-unparser REQUEST) => REQUEST</procedure><br>
114<procedure>(http-1.x-request-unparser REQUEST) => REQUEST</procedure><br>
115
116Predefined request unparsers for use with {{request-unparsers}}.
117They return the request, and as a side effect they write the
118request to the request object's port.
119
120=== Responses
121
122<procedure>(make-response #!key port status (code 200) (reason "OK") (major 1) (minor 1) (headers (headers '())))</procedure>
123
124Create a response, a [[defstruct]]-type record.  A server will usually
125write a response with {{write-response}}; a client will read it
126with {{read-response}}.
127
128You can either supply a status symbol or a code and/or reason to set
129the response status.  If you do, the code will be set to match that
130symbol and the reason is set to the default reason belonging to that
131code, as provided by the HTTP standard(s).  The allowed symbols are
132generally just Schemified versions of the default reason.
133
134Only the code and reason are actual fields in the object; the status
135is a virtual field.
136
137See {{http-status-codes}} for a list of all known default statuses.
138
139<procedure>(update-response old-response #!key port status code reason major minor headers)</procedure>
140
141Like {{make-response}}, except this takes an {{old-response}} object as a
142template for values which are missing from the parameter list, thereby
143providing a way to do a purely functional update of that object.
144
145<procedure>(response-port RESPONSE) => PORT</procedure><br>
146<procedure>(response-code RESPONSE) => NUMBER</procedure><br>
147<procedure>(response-reason RESPONSE) => STRING</procedure><br>
148<procedure>(response-class RESPONSE-OR-CODE) => NUMBER</procedure><br>
149<procedure>(response-major RESPONSE) => NUMBER</procedure><br>
150<procedure>(response-minor RESPONSE) => NUMBER</procedure><br>
151<procedure>(response-headers RESPONSE) => HEADERS</procedure><br>
152<procedure>(response-status RESPONSE-OR-CODE) => SYMBOL</procedure><br>
153
154An existing response can be picked apart using these accessors.
155
156The PORT, MAJOR, MINOR and HEADERS are the same as for requests. CODE
157and REASON are an integer status code and the short message that
158belongs to it, as defined in the spec (examples include: 200 OK, 301
159Moved Permanently, etc).  CLASS is the major class of the response
160code (100, 200, 300, 400 or 500).  {{response-class}} can be called
161either on a response object or directly on a response code number.
162
163{{response-status}} attempts to fetch the symbolic status from the
164response object based on its response code.  If no matching symbol can
165be found in the {{http-status-codes}} parameter, an exception is thrown.
166{{response-status}} can also be called on a response object or directly
167on a response code number.
168
169<procedure>(response-port-set! RESPONSE PORT)</procedure><br>
170<procedure>(response-code-set! RESPONSE NUMBER)</procedure><br>
171<procedure>(response-reason-set! RESPONSE STRING)</procedure><br>
172<procedure>(response-major-set! RESPONSE NUMBER)</procedure><br>
173<procedure>(response-minor-set! RESPONSE NUMBER)</procedure><br>
174<procedure>(response-headers-set! RESPONSE HEADERS)</procedure><br>
175<procedure>(response-status-set! RESPONSE SYMBOL)</procedure><br>
176
177These procedures mutate an existing response object and set the
178corresponding slot. {{response-status-set!}} will attempt to look up
179the code and reason in {{http-status-codes}} and set both slots.
180If the symbolic status is unknown, an exception is thrown.
181
182<parameter>(http-status-codes [ALIST])</parameter>
183
184This is an alist mapping symbolic status indicators to HTTP codes and
185reason strings.
186
187These can be used to make your code a bit more expressive and to
188reduce duplication of hardcoded strings; instead of using a numeric
189"magic number" HTTP code plus the same human-readable string
190everywhere the same code occurs, you can instead use a descriptive
191symbol.
192
193The default value of this mapping is as follows:
194
195<enscript highlight=scheme>
196'((continue . (100 . "Continue"))
197  (switching-protocols . (101 . "Switching Protocols"))
198  (ok . (200 . "OK"))
199  (created . (201 . "Created"))
200  (accepted . (202 . "Accepted"))
201  (non-authoritative-information . (203 . "Non-Authoritative Information"))
202  (no-content . (204 . "No Content"))
203  (reset-content . (205 . "Reset Content"))
204  (partial-content . (206 . "Partial Content"))
205  (multiple-choices . (300 . "Multiple Choices"))
206  (moved-permanently . (301 . "Moved Permanently"))
207  (found . (302 . "Found"))
208  (see-other . (303 . "See Other"))
209  (not-modified . (304 . "Not Modified"))
210  (use-proxy . (305 . "Use Proxy"))
211  (temporary-redirect . (307 . "Temporary Redirect"))
212  (bad-request . (400 . "Bad Request"))
213  (unauthorized . (401 . "Unauthorized"))
214  (payment-required . (402 . "Payment Required"))
215  (forbidden . (403 . "Forbidden"))
216  (not-found . (404 . "Not Found"))
217  (method-not-allowed . (405 . "Method Not Allowed"))
218  (not-acceptable . (406 . "Not Acceptable"))
219  (proxy-authentication-required . (407 . "Proxy Authentication Required"))
220  (request-time-out . (408 . "Request Time-out"))
221  (conflict . (409 . "Conflict"))
222  (gone . (410 . "Gone"))
223  (length-required . (411 . "Length Required"))
224  (precondition-failed . (412 . "Precondition Failed"))
225  (request-entity-too-large . (413 . "Request Entity Too Large"))
226  (request-uri-too-large . (414 . "Request-URI Too Large"))
227  (unsupported-media-type . (415 . "Unsupported Media Type"))
228  (request-range-not-satisfiable . (416 . "Requested range not satisfiable"))
229  (expectation-failed . (417 . "Expectation Failed"))
230  (internal-server-error . (500 . "Internal Server Error"))
231  (not-implemented . (501 . "Not Implemented"))
232  (bad-gateway . (502 . "Bad Gateway"))
233  (service-unavailable . (503 . "Service Unavailable"))
234  (gateway-time-out . (504 . "Gateway Time-out"))
235  (http-version-not-supported . (505 . "HTTP Version not supported")))
236</enscript>
237
238<procedure>(write-response RESPONSE) => RESPONSE</procedure>
239
240Write the response object RESPONSE to the {{response-port}}.
241
242If there is a response body, this must be written to the response-port
243after sending the response headers.
244
245<procedure>(read-response PORT) => RESPONSE</procedure>
246
247Reads a response object from the port. An optional response body can
248be read from the response-port after calling this procedure.
249
250<parameter>(response-parsers [LIST])</parameter>
251
252Responses are parsed using parse procedures, which can be customized
253by overriding this parameter.
254
255LIST is a list one of procedures which accept a response line string
256and produce a response object, or {{#f}} if the response is not of the
257type handled by that procedure.
258
259The predefined response parsers are:
260
261* {{http-0.9-response-parser}}
262* {{http-1.x-response-parser}}
263
264<procedure>(http-0.9-response-parser REQUEST) => REQUEST</procedure><br>
265<procedure>(http-1.x-response-parser REQUEST) => REQUEST</procedure><br>
266
267Predefined response parsers for use with {{response-parser}}.
268
269<parameter>(response-unparsers [LIST])</parameter>
270
271Responses are written using unparse procedures, which can be
272customized by overriding this parameter.
273
274LIST is a list of procedures which accept a response object and write
275to the response's output port and return the new, possibly updated
276response object. If the response object is not unparsed by this
277handler, it returns {{#f}}.
278
279The predefined response unparsers are the following:
280
281* {{http-0.9-response-unparser}}
282* {{http-1.x-response-unparser}}
283
284<procedure>(http-0.9-response-unparser REQUEST) => REQUEST</procedure><br>
285<procedure>(http-1.x-response-unparser REQUEST) => REQUEST</procedure><br>
286
287Predefined response unparsers for use with {{response-unparser}}.
288
289=== Headers
290
291<procedure>(headers ALIST [HEADERS]) => HEADERS</procedure>
292
293This creates a header object based on an input list.
294
295Requests and responses contain HTTP headers wrapped in a special
296header-object to ensure they are properly normalized.
297
298The input list has header names (symbols) as keys, and
299lists of values as values:
300
301<enscript highlight="scheme">
302(headers `((host ("example.com" . 8080))
303           (accept #(text/html ((q . 0.5)))
304                   #(text/xml ((q . 0.1)))))
305          old-headers)
306</enscript>
307
308This adds the named headers to the existing headers in
309{{old-headers}}. The host header is a pair of hostname/port.
310The accept header is a list of allowed mime-type symbols.
311
312As can be seen here, optional parameters or "attributes" can be added
313to a header value by wrapping the value in a vector of length 2. The
314first entry in the vector is the header value, the second is an alist
315of attribute name/value pairs.
316
317<procedure>(headers->list HEADERS) => ALIST</procedure>
318
319This converts a header object back to a list.  See {{headers}}
320for details.
321
322<procedure>(header-values NAME HEADERS) => LIST</procedure>
323
324Obtain the value of header NAME in the HEADERS object.
325
326The NAME of the header is a symbol; this procedure will return all the values
327of the header (for example, the Accept header will have several values
328that indicate the set of acceptable mime-types).
329
330<procedure>(header-value NAME HEADERS [DEFAULT]) => value</procedure>
331
332If you know in advance that a header has only one value, you can use
333{{header-value}} instead of {{header-values}}.  This will return the
334first value in the list, or the provided default if there is no value
335for that header.
336
337<procedure>(header-params NAME HEADERS) => ALIST</procedure>
338
339This will return all the params for a given header, assuming there is
340only one header.  An empty list is returned if the header does not exist.
341
342<procedure>(header-param PARAM NAME HEADERS [DEFAULT]) => value</procedure>
343
344This will return a specific parameter for the header, or DEFAULT
345if the parameter isn't present or the header does not exist.  This
346also assumes there's only one header.
347
348<procedure>(header-contents NAME HEADERS) => LIST</procedure><br>
349<procedure>(get-value VECTOR) => value</procedure><br>
350<procedure>(get-params VECTOR) => ALIST</procedure><br>
351<procedure>(get-param PARAM VECTOR [DEFAULT]) => value</procedure><br>
352
353Procedures such as {{header-values}} are just shortcuts; these are
354the underlying procedures to query the raw contents of a header.
355
356Header contents are lists of 2-element vectors; the first value
357containing the value for the header and the second value containing
358an alist with "parameters" for that header value. Parameters are
359attribute/value pairs that define further specialization of a header's
360value. For example, the {{accept}} header consists of a list of
361mime-types, which optionally can have a quality parameter that
362defines the preference for that mime-type.  All parameter names
363are downcased symbols, just like header names.
364
365Here's a few examples on how to retrieve info from headers:
366
367<enscript highight="scheme">
368;; This would be returned by a server and retrieved via (response-headers r):
369(define example-headers
370  (headers '((accept #(text/html ((q . 0.1)))
371                     #(text/xml ((q . 0.5)))
372                     text/plain)
373             (allow HEAD GET)
374             (content-type #(text/html ((charset . utf-8))))
375             (max-forwards 2))))
376
377;;; Basic procedures
378(define c (header-contents 'accept example-headers))
379c ; => (#(text/html ((q . 0.5))) #(text/xml ((q . 0.1))) #(text/plain ()))
380
381(get-value (car c))
382; => text/html
383(get-params (car c))
384; => ((q . 0.5))
385(get-param 'q (car c))
386; => 0.5
387
388;;; Simplified helpers
389(header-values 'accept example-headers)
390; => (text/html text/xml text/plain)
391(header-values 'max-forwards example-headers)
392; => (2)
393(header-values 'nonexistent-header example-headers)
394; => ()
395
396;; This assumes there's only one value (returns the first)
397(header-value 'max-forwards example-headers)
398; => 2
399(header-value 'nonexistent-header example-headers)
400; => #f
401(header-value 'nonexistent-header example-headers 'not-here)
402; => not-here
403;; Tricky:
404(header-value 'accept example-headers)
405; => text/html
406
407;; This is tricky: this just returns the first, which is not the preferred
408(header-params 'accept example-headers)
409; => ((q . 0.1))
410;; Quick access
411(header-param 'charset 'content-type example-headers)
412; => utf-8
413</enscript>
414
415==== Header types
416
417The headers all have their own different types.  Here follows a list
418of headers with their value types:
419
420<table>
421<tr><th>Header name</th><th>Value type</th><th>Example value</th></tr>
422<tr>
423<td>{{accept}}</td>
424<td>List of mime-types (symbols), with optional {{q}} attribute
425indicating "quality" (preference level)</td>
426<td>{{(text/html #(text/xml ((q . 0.1))))}}</td>
427</tr>
428<tr>
429<td>{{accept-charset}}</td>
430<td>List of charset-names (symbols), with optional {{q}} attribute</td>
431<td>{{(utf-8 #(iso-8859-5 ((q . 0.1))))}}</td>
432</tr>
433<tr>
434<td>{{accept-encoding}}</td>
435<td>List of encoding-names (symbols), with optional {{q}} attribute</td>
436<td>{{(gzip #(identity ((q . 0))))}}</td>
437</tr>
438<tr>
439<td>{{accept-language}}</td>
440<td>List of language-names (symbols), with optional {{q}} attribute</td>
441<td>{{(en-gb #(nl ((q . 0.5))))}}</td>
442</tr>
443<tr>
444<td>{{accept-ranges}}</td>
445<td>List of range types acceptable (symbols). The spec only defines
446{{bytes}} and {{none}}.</td>
447<td>{{(bytes)}}</td>
448</tr>
449<tr>
450<td>{{age}}</td>
451<td>Age in seconds (number)</td>
452<td>{{(3600)}}</td>
453</tr>
454<tr>
455<td>{{allow}}</td>
456<td>List of methods that are allowed (symbols).</td>
457<td>{{(GET POST PUT DELETE)}}</td>
458</tr>
459<tr>
460<td>{{authorization}}</td>
461<td>Authorization information. This consists of a symbol identifying the
462authentication scheme, with scheme-specific attributes.
463{{basic}} is handled specially, as if it were a regular symbol with two
464attributes; {{username}} and {{password}}.</td>
465<td>{{(#(basic ((username . "foo") (password . "bar"))) #(digest ((qop . auth) (username . "Mufasa") (nc . 1))))}}</td>
466</tr>
467<tr>
468<td>{{cache-control}}</td>
469<td>An alist of key/value pairs. If no value is applicable, it is {{#t}}</td>
470<td>{{((public . #t) (max-stale . 10) (no-cache . (age set-cookie)))}}</td>
471</tr>
472<tr>
473<td>{{connection}}</td>
474<td>A list of connection options (symbols)</td>
475<td>{{(close)}}</td>
476</tr>
477<tr>
478<td>{{content-disposition}}</td>
479<td>A symbol indicating the disposition</td>
480<td>{{(#(inline ((filename . "test.pdf"))))}}</td>
481</tr>
482<tr>
483<td>{{content-encoding}}</td>
484<td>A list of encodings (symbols) applied to the entity-body.</td>
485<td>{{(deflate gzip)}}</td>
486</tr>
487<tr>
488<td>{{content-language}}</td>
489<td>The natural language(s) of the "intended audience" (symbols)</td>
490<td>{{(de nl en-gb)}}</td>
491</tr>
492<tr>
493<td>{{content-length}}</td>
494<td>The number of bytes (an exact number) in the entity-body</td>
495<td>{{(10)}}</td>
496</tr>
497<tr>
498<td>{{content-location}}</td>
499<td>A location that the content can be retrieved from (a uri-common object)</td>
500<td>{{(<#uri-common# ...>)}}</td>
501</tr>
502<tr>
503<td>{{content-md5}}</td>
504<td>The MD5 checksum (a string) of the entity-body</td>
505<td>{{("12345ABCDEF")}}</td>
506</tr>
507<tr>
508<td>{{content-range}}</td>
509<td>Content range (pair with start- and endpoint) of the entity-body, if partially sent</td>
510<td>{{((25 . 120))}}</td>
511</tr>
512<tr>
513<td>{{content-type}}</td>
514<td>The mime type of the entity-body (a symbol)</td>
515<td>{{(#(text/html ((charset . iso-8859-1))))}}</td>
516</tr>
517<tr>
518<td>{{date}}</td>
519<td>A timestamp (10-element vector, see {{string->time}}) at which the message originated. ''Important'': Note that you will always need to supply (an empty list of) attributes, because otherwise it is ambiguous whether it's a vector with attribs or a bare timestamp.</td>
520<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
521</tr>
522<tr>
523<td>{{etag}}</td>
524<td>An entity-tag (pair, car being either the symbol weak or strong, cdr being a string) that uniquely identifies the resource contents.</td>
525<td>{{((strong . "foo123"))}}</td>
526</tr>
527<tr>
528<td>{{expect}}</td>
529<td>Expectations of the server's behaviour (alist of symbol-string pairs), possibly with parameters.</td>
530<td>{{(#(((100-continue . #t)) ()))}}</td>
531</tr>
532<tr>
533<td>{{expires}}</td>
534<td>Expiry timestamp (10-element vector, see {{string->time}}) for the entity. Also see the note for {{date}}</td>
535<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
536</tr>
537<tr>
538<td>{{from}}</td>
539<td>The e-mail address (a string) of the human user who controls the client</td>
540<td>{{("info@example.com")}}</td>
541</tr>
542<tr>
543<td>{{host}}</td>
544<td>The host to use (for virtual hosting). This is a pair of hostname and port. The port will be {{#f}} if the port should be the default one for the requested service.</td>
545<td>{{(("example.com" . 8080))}}</td>
546</tr>
547<tr>
548<td>{{if-match}}</td>
549<td>Either {{'*}} (a wildcard symbol) or a list of entity-tags (pair, weak/strong symbol and unique entity identifier string).</td>
550<td>{{((strong . "foo123") (strong . "bar123"))}}</td>
551</tr>
552<tr>
553<td>{{if-modified-since}}</td>
554<td>Timestamp (10-element vector, see {{string->time}}) which indicates since when the entity must have been modified.</td>
555<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
556</tr>
557<tr>
558<td>{{if-none-match}}</td>
559<td>Either {{'*}} (a wildcard symbol) or a list of entity-tags (pair, weak/strong symbol and unique entity identifier symbol).</td>
560<td>{{((strong . foo123) (strong . bar123))}}</td>
561</tr>
562<tr>
563<td>{{if-range}}</td>
564<td>The range to request, if the entity was unchanged</td>
565<td>TODO</td>
566</tr>
567<tr>
568<td>{{if-unmodified-since}}</td>
569<td>A timestamp (10-element vector, see {{string->time}}) since which the entity must not have been modified</td>
570<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
571</tr>
572<tr>
573<td>{{last-modified}}</td>
574<td>A timestamp (10-element vector, see {{string->time}}) when the entity was last modified</td>
575<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
576</tr>
577<tr>
578<td>{{location}}</td>
579<td>A location (an URI object) to which to redirect</td>
580<td>{{(<#uri-object ...>)}}</td>
581</tr>
582<tr>
583<td>{{max-forwards}}</td>
584<td>The maximum number of proxies that can forward a request</td>
585<td>{{(2)}}</td>
586</tr>
587<tr>
588<td>{{pragma}}</td>
589<td>An alist of symbols containing implementation-specific directives.</td>
590<td>{{((no-cache . #t) (my-extension . my-value))}}</td>
591</tr>
592<tr>
593<td>{{proxy-authenticate}}</td>
594<td>Proxy authentication request.  Equivalent to {{www-authenticate}}, for proxies.</td>
595<td>{{(#(basic ((realm . "foo")) ) #(digest ((realm . "foo") (domain . (<#uri object> <#uri object>)) (qop . (auth auth-int)) (nonce . "012345abc"))))}}</td>
596</tr>
597<tr>
598<td>{{proxy-authorization}}</td>
599<td>The answer to a {{proxy-authentication}} request. Equivalent to {{authorization}}, for proxies.</td>
600<td>{{(#(basic ((username . "foo") (password . "bar"))) #(digest ((qop . auth) (username . "Mufasa") (nc . 1))))}}</td>
601</tr>
602<tr>
603<td>{{range}}</td>
604<td>The range of bytes (a pair of start and end) to request from the server.</td>
605<td>{{((25 . 120))}}</td>
606</tr>
607<tr>
608<td>{{referer}}</td>
609<td>The referring URL (uri-common object) that linked to this one.</td>
610<td>{{(<#uri-object ...>)}}</td>
611</tr>
612<tr>
613<td>{{retry-after}}</td>
614<td>Timestamp (10-element vector, see {{string->time}}) after which to retry the request if unavailable now.</td>
615<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
616</tr>
617<tr>
618<td>{{server}}</td>
619<td>List of products the server uses (list of 3-tuple lists of strings; product name, product version, comment. Version and/or comment may be {{#f}}). Note that this is a single header, with a list inside it!</td>
620<td>{{((("Apache" "2.2.9" "Unix") ("mod_ssl" "2.2.9" #f) ("OpenSSL" "0.9.8e" #f) ("DAV" "2" #f) ("mod_fastcgi" "2.4.2" #f) ("mod_apreq2-20051231" "2.6.0" #f)))}}</td>
621</tr>
622<tr>
623<td>{{te}}</td>
624<td>Allowed transfer-encodings (symbols, with optional q attribute) for the response</td>
625<td>{{(deflate #(gzip ((q . 0.2))))}}</td>
626</tr>
627<tr>
628<td>{{trailer}}</td>
629<td>Names of header fields (symbols) available in the trailer/after body</td>
630<td>{{(range etag)}}</td>
631</tr>
632<tr>
633<td>{{transfer-encoding}}</td>
634<td>The encodings (symbols) used in the body</td>
635<td>{{(chunked)}}</td>
636</tr>
637<tr>
638<td>{{upgrade}}</td>
639<td>Product names to which must be upgraded (strings)</td>
640<td>TODO</td>
641</tr>
642<tr>
643<td>{{user-agent}}</td>
644<td>List of products the user agent uses (list of 3-tuple lists of strings; product name, product version, comment. Version and/or comment may be {{#f}}). Note that this is a single header, with a list inside it!</td>
645<td>{{((("Mozilla" "5.0" "X11; U; NetBSD amd64; en-US; rv:1.9.0.3") ("Gecko" "2008110501" #f) ("Minefield" "3.0.3" #f)))}}</td>
646</tr>
647<tr>
648<td>{{vary}}</td>
649<td>The names of headers that define variation in the resource body, to determine cachability (symbols)</td>
650<td>{{(range etag)}}</td>
651</tr>
652<tr>
653<td>{{via}}</td>
654<td>The intermediate hops through which the message is forwarded (strings)</td>
655<td>TODO</td>
656</tr>
657<tr>
658<td>{{warning}}</td>
659<td>Warning code for special status</td>
660<td>TODO</td>
661</tr>
662<tr>
663<td>{{www-authenticate}}</td>
664<td>If unauthorized, a challenge to authenticate (symbol, with attributes)</td>
665<td>{{(#(basic ((realm . "foo"))) #(digest ((realm . "foo") (domain . (<#uri object> <#uri object>)) (qop . (auth auth-int)) (nonce . "012345abc"))))}}</td>
666</tr>
667<tr>
668<td>{{set-cookie}}</td>
669<td>Cookies to set (name/value pair (both strings), with attributes)</td>
670<td>{{(#(("foo" . "bar") ((max-age . 10) (port . '(80 8080))))}}</td>
671</tr>
672<tr>
673<td>{{cookie}}</td>
674<td>Cookies that were set (name/value string pair, with attributes)</td>
675<td>{{(#(("foo" . "bar") ((version . 1) (path . #(uri path: (/ ""))) (domain . "foo.com"))))}}</td>
676</tr>
677<tr>
678<td>{{x-forwarded-for}}</td>
679<td>The chain of IP addresses which each intermediate proxy will add to.  Plain strings representing the IP-address or "unknown" when the proxy couldn't determine the client address or the option is disabled. ''Never accept this value without question; it can easily be spoofed!''</td>
680<td>{{("192.168.1.2" "unknown" "123.456.789.012" "some-made-up-value-by-an-attacker")}}</td>
681</tr>
682
683</table>
684
685Any unrecognised headers are assumed to be multi-headers, and the
686entire header lines are put unparsed into a list, one entry per line.
687
688
689==== Header parsers and unparsers
690
691<parameter>(header-parsers [ALIST])</parameter><br>
692<parameter>(header-unparsers [ALIST])</parameter><br>
693
694The parsers and unparsers used to read and write header values can be
695customized with these parameters.
696
697These (un)parsers are indexed with as key the header name (a symbol)
698and the value being a procedure.
699
700A header parser accepts the contents of the header (a string, without
701the leading header name and colon) and returns a ''list of vectors''
702which represents the values of the header.  For headers that are
703supposed to only have a single value, the last value in the list will
704be stored as the value (as determined by {{single-headers}}).
705
706A header unparser accepts one argument: the header's contents (a
707vector).  It should return a list of strings, each of which represents
708one line's worth of header contents (without the header name).  For
709each entry, a header line will automatically be printed with the
710header name preceding it.
711
712The parser driver will call {{update-header-contents!}} with the
713parser's result.
714
715<parameter>(header-parse-error-handler [HANDLER])</parameter>
716
717When there is an error parsing a given header, this
718parameter's procedure will be invoked.
719
720{{HANDLER}} is a procedure accepting four values: the header name, the
721header contents, the current headers and the exception object.  The
722procedure must return the new headers.  Defaults to a procedure that
723simply returns the current headers.  When an error occurs while
724parsing the header line itself (for example when a colon is missing
725between the header name and contents), the error will not be caught.
726
727In such a case, Servers should return a 400 Bad Request error and
728clients should error out.  The reason that malformed error lines are
729ignored is that there are several servers and clients that send
730headers content values that are slightly off, even though the rest of
731the request is OK.  In the interest of the "robustness principle",
732it's best to simply ignore these headers with "bad" content values.
733
734<procedure>(replace-header-contents NAME CONTENTS HEADERS) => HEADERS</procedure><br>
735<procedure>(replace-header-contents! NAME CONTENTS HEADERS) => HEADERS</procedure><br>
736<procedure>(update-header-contents NAME CONTENTS HEADERS) => HEADERS</procedure><br>
737<procedure>(update-header-contents! NAME CONTENTS HEADERS) => HEADERS</procedure><br>
738
739The {{replace}} procedures replace any existing contents of the named
740header with new ones, the {{update}} procedures add these contents to
741the existing header. The procedures with a name ending in bang are
742linear update variants of the ones without the bang. The header
743contents have to be normalized to be a 2-element vector, with the
744first element being the actual value and the second element being an
745alist (possibly empty) of parameters/attributes for that value.
746
747The update procedures append the value to the existing header if it is
748a multi-header, and act as a simple replace in the case of a
749single-header.
750
751<parameter>(single-headers [LIST])</parameter>
752
753Whether a header is allowed once or multiple times in a request or
754response is determined by this parameter.
755
756The value is a list of symbols that define header-names which are
757allowed to occur only once in a request/response.
758
759<procedure>(http-name->symbol STRING) => SYMBOL</procedure><br>
760<procedure>(symbol->http-name SYMBOL) => STRING</procedure><br>
761
762These procedures convert strings containing the name of a header or
763attribute (parameter name) to symbols representing the same. The
764symbols are completely downcased.  When converting this symbol back to
765a string, the initial letters of all the words in the header name or
766attribute are capitalized.
767
768<procedure>(remove-header name headers) => headers</procedure><br>
769<procedure>(remove-header! name headers) => headers</procedure><br>
770
771These two procedures remove all headers with the given name.
772
773===== Header subparsers and subunparsers
774
775Some headers are modular ''themselves''.  This means they need some
776way to extend them.  This is done through subparsers and subunparsers.
777
778<parameter>(authorization-param-subparsers [ALIST])</parameter>
779
780This is an alist of subtypes for the {{authorization}} header parser.
781A subparser of this kind accepts the string containing the header and
782an integer position in the string.  It should parse from that position
783onwards, and return the parsed contents as an alist of header
784parameters.  Usually, these are actually pseudo-parameters; they don't
785necessarily have to appear in parameter syntax in the header.  The
786unparser should be configured to expect the same parameters and combine
787them back into a string, though.
788
789This parameter defaults to:
790
791<enscript highligh=scheme>
792`((basic . ,basic-auth-subparser)
793  (digest . ,digest-auth-subparser))
794</enscript>
795
796<procedure>(basic-auth-param-subparser STR POS)</procedure>
797
798Parses {{STR}} at {{POS}} by extracting the username and password
799components from a base64-encoded string.  These are returned in its
800first value as an alist with keys {{username}} and {{password}}.  Its
801second return value is the position after which the next header value
802may begin.
803
804<procedure>(digest-auth-param-subparser STR POS)</procedure>
805
806Parses {{STR}} at {{POS}} by reading the various components from a
807parameter list.  These are returned in its first return value as an
808alist with keys {{nc}}, {{uri}}, {{qop}} and {{algorithm}}.  Its
809second return value is the position after which the next header value
810may begin.
811
812<parameter>(authorization-param-subunparsers [ALIST])</parameter>
813
814This is an alist of subtypes for the {{authorization}} header
815unparser.  An unparser of this kind accepts an alist containing the
816parameters that it needs to unparse and should return a string
817containing the raw unparsed parameters only.
818
819This parameter defaults to:
820
821<enscript highligh=scheme>
822`((basic . ,basic-auth-subunparser)
823  (digest . ,digest-auth-subunparser))
824</enscript>
825
826<procedure>(basic-auth-param-subunparser PARAMS)</procedure>
827
828This unparses the {{PARAMS}} alist into a base64-encoded string for
829basic authentication.  It expects {{username}} and {{password}}
830parameters.
831
832<procedure>(digest-auth-param-subunparser PARAMS)</procedure>
833
834This unparses the {{PARAMS}} alist into a string for digest
835authentication. It expects {{username}}, {{uri}}, {{realm}},
836{{nonce}}, {{cnonce}}, {{qop}}, {{nc}}, {{response}}, {{opaque}} and
837{{algorithm}} parameters.  The {{response}} parameter should be
838pre-encoded in the way digest auth expects (this is not done here
839because the MD5 sum of the contents may be required, which is not
840available to the parsers).
841
842TODO: This will probably change in the future; the md5 can be passed
843and all the hard stuff can be done in intarweb.
844
845=== Other procedures
846
847<parameter>(http-line-limit [length])</parameter>
848
849The maximum length of any line that's read by intarweb as part of the
850request/response cycle.  This includes the request and response lines
851as well as the headers.  If this is exceeded, an exception of type
852{{(exn http line-limit-exceeded)}} is raised.
853
854You can set this to {{#f}} to disable this check. However, this will
855open up a resource consumption vulnerability (attackers can cause
856your application to blow up by letting it use all available memory).
857
858Defaults to {{1024}}.
859
860<parameter>(http-header-limit [count])</parameter>
861
862The maximum number of headers that are allowed to be sent, as part
863of a request or response. If this is exceeded, an exception of type
864{{(exn http header-limit-exceeded)}} is raised.
865
866You can set this to {{#f}} to disable this check. However, this will
867open up a resource consumption vulnerability (attackers can cause
868your application to blow up by letting it use all available memory).
869
870Defaults to {{256}}.
871
872<procedure>(keep-alive? request-or-response)</procedure>
873
874Returns {{#t}} when the given request or response object belongs to a
875connection that should be kept alive, {{#f}} if not.  Remember that
876both parties must agree on whether the connection is to be kept alive
877or not; HTTP/1.1 defaults to keep alive unless a {{Connection: close}}
878header is sent, HTTP/1.0 defaults to closing the connection, unless a
879{{Connection: Keep-Alive}} header is sent.
880
881<parameter>(request-has-message-body? [predicate])</parameter>
882
883This parameter holds a predicate which accepts a request object and
884returns {{#t}} when the request will have a message body.  By default
885in HTTP/1.1, this is the case for all requests that have a {{content-length}}
886or {{transfer-coding}} header.
887
888The parameter is useful for servers to determine whether to read a
889request body or not.
890
891<procedure>(read-urlencoded-request-data request [max-length])</procedure>
892
893Convenience procedure to read URLencoded request data (regular POST
894data; ''not'' multipart data!) from the given {{request}} object.  It
895will return an alist, as would be returned by {{form-urldecode}} from
896the [[uri-common]] egg.
897
898You have to take care of checking the request type whether there
899really will be request data yourself (it can optionally use
900{{request-has-message-body?}} for this, but it's probably advisable to
901check the request type anyway).
902
903This will read at most {{max-length}} bytes.  If not specified,
904{{max-length}} defaults to the current value of
905{{http-urlencoded-request-data-limit}}.  If this maximum is exceeded,
906an exception of type {{(exn http urlencoded-request-data-limit-exceeded)}}
907is raised.
908
909<parameter>(http-urlencoded-request-data-limit [length])</parameter>
910
911Set the default limit for request body data.  Defaults to 4194304 (4MB).
912
913<parameter>(response-has-message-body-for-request? [predicate])</parameter>
914
915This parameter holds a predicate which accepts two arguments: a
916response object and a request object.  It returns {{#t}} when the
917response will have a message body for the given request.  By default
918in HTTP/1.1, this is '''not''' the case for responses with a
919response-code of 204 and 304 or in the 1xx class, nor for {{HEAD}}
920requests.  All other responses will have a message body.
921
922The parameter is useful for deciding in clients whether a message body
923will follow (otherwise, trying to read will probably result in an error
924or in case of HTTP pipelining in a synchronisation problem)
925
926<procedure>(safe? request-or-method)</procedure>
927
928Returns {{#t}} when the given request object or symbol (method) is a
929''safe'' method.  A method is defined to be safe when a request of
930this method will have no side-effects on the server.  In practice this
931means that you can send this request from anywhere at any time and
932cause no damage.
933
934'''Important''': Quite a lot of software does not abide by these
935rules!  This is not necessarily a reason to treat all methods as
936unsafe, however.  In the words of the standard "the user did not
937request the side-effects, so therefore cannot be held accountable for
938them".  If a safe method produces side-effects, that's the server-side
939script developer's fault and he should fix his code.
940
941<parameter>(safe-methods [symbols])</parameter>
942
943A list of methods which are to be considered safe.  Defaults to
944{{'(GET HEAD OPTIONS TRACE)}}.
945
946<procedure>(idempotent? request-or-method)</procedure>
947
948Returns {{#t}} when the given request object or symbol (method) is a
949''idempotent'' method.  A method is defined to be idempotent when a
950series of identical requests of this method in succession causes the
951exact same side-effect as just one such request.  In practice this
952means that you can safely retry such a request when an error occurs,
953for example.
954
955'''Important''': Just as with the ''safe'' methods, there is no
956guarantee that methods that ''should be'' idempotent really are
957idempotent in any given web application.  Furthermore, a sequence of
958requests which each are individually idempotent is not necessarily
959idempotent as a whole.  This means that you cannot replay requests
960starting anywhere in the chain.  To be on the safe side, only retry
961the last request in the chain.
962
963<parameter>(idempotent-methods [symbols])</parameter>
964
965A list of methods which are to be considered idempotent.  Defaults to
966{{'(GET HEAD PUT DELETE OPTIONS TRACE)}}.
967
968<procedure>(etag=? a b)</procedure>
969
970Do the etag values {{a}} and {{b}} strongly match?  That is, their
971{{car}} and {{cdr}} must be equal, and neither can have a {{car}} of
972{{weak}} (both must be {{strong}}).
973
974<procedure>(etag=-weakly? a b)</procedure>
975
976Do the etag values {{a}} and {{b}} weakly match?  That is, their
977{{car}} and {{cdr}} must be equal.  A {{car}} of {{weak}} is allowed.
978
979<procedure>(etag-matches? etag matches)</procedure>
980
981Does the {{etag}} strongly match any of the etags in the list
982{{matches}}?  {{matches}} is a plain list of etag values, but it can
983also contain the special symbol {{*}}, which matches any etag.
984
985<procedure>(etag-matches-weakly? etag matches)</procedure>
986
987Does the {{etag}} weakly match any of the etags in the list
988{{matches}}?  {{matches}} is a plain list of etag values, but it can
989also contain the special symbol {{*}}, which matches any etag.
990
991=== Changelog
992
993* trunk Treat the charset attribute for Content-Type header as case-insensitive token for consistency with Accept-Charset header.  Remove dependency on the [[/eggref/4/regex|regex egg]] and improve correctness of a few parsers.  Add {{request-has-message-body?}} and {{response-has-message-body-for-request?}} procedures.  Add parser for Content-Disposition header and improve unparser by adding date support (Thanks to Evan Hanson).  Implement line length and header count limit checking.  Add {{read-urlencoded-request-data}} with built-in limit check.
994* 0.7 Add trivial {{x-forwarded-for}} "parser".  Add easier overriding of {{authorization}} headers through parameter instead of having to rewrite the entire parser.  Add {{content-disposition}} unparser to accommodate the fact that filenames ''must'' always be quoted.  Add {{http-status-codes}} parameter and {{status:}} key to {{update-response}} and {{make-response}} procedures, as well as {{response-status}} and {{response-status-set!}} procedures.
995* 0.6 Change path parameters on cookies to be uri-common objects
996* 0.5 Add regex requirement to make it work with Chicken 4.6.2+
997* 0.4 Don't unparse "attributes" (aka "params") by titlecasing their names. Don't default Host header's port to 80, but use #f
998* 0.3 Add rfc1123 to default unparser list for the 'date' header. Add etag procedures and if-match unparser. Change procedure signature for header unparsers
999* 0.2 Make cookie header parsers/unparsers preserve cookie name case. Change header unparse procedure semantics slightly.
1000* 0.1 Initial version
1001
1002=== License
1003
1004  Copyright (c) 2008-2012, Peter Bex
1005  All rights reserved.
1006 
1007  Redistribution and use in source and binary forms, with or without
1008  modification, are permitted provided that the following conditions are
1009  met:
1010 
1011  Redistributions of source code must retain the above copyright
1012  notice, this list of conditions and the following disclaimer.
1013 
1014  Redistributions in binary form must reproduce the above copyright
1015  notice, this list of conditions and the following disclaimer in the
1016  documentation and/or other materials provided with the distribution.
1017 
1018  Neither the name of the author nor the names of its contributors may
1019  be used to endorse or promote products derived from this software
1020  without specific prior written permission.
1021 
1022  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
1023  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
1024  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
1025  FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
1026  COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
1027  INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
1028  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
1029  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
1030  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
1031  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
1032  ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
1033  OF THE POSSIBILITY OF SUCH DAMAGE.
Note: See TracBrowser for help on using the repository browser.