source: project/wiki/eggref/4/intarweb @ 27109

Last change on this file since 27109 was 27109, checked in by sjamaan, 9 years ago

Fix typo

File size: 39.8 KB
Line 
1[[tags: egg]]
2
3== Intarweb
4
5[[toc:]]
6
7=== Description
8
9Intarweb is an advanced http library.  It parses all headers into more
10useful Scheme values.
11
12=== Author
13
14[[/users/peter-bex|Peter Bex]]
15
16=== Requirements
17
18Requires the [[defstruct]], [[base64]] and [[uri-common]] extensions.
19
20=== Documentation
21
22The intarweb egg is designed to be used from a variety of
23situations. For this reason, it does not try to be a full HTTP client
24or server. If you need that kind of functionality, see eggs like
25[[spiffy]] or [[http-client]].
26
27=== Requests
28
29<procedure>(make-request #!key uri port (method 'GET) (major 1) (minor 1) (headers (make-headers '())))</procedure>
30
31Create a request object (a [[defstruct]]-type record). The client will
32generally write requests, while the server will read them.
33
34The URI defines the entity to retrieve on the server, which should be
35a [[uri-common]]-type URI object. The PORT is the scheme I/O port
36where the request is written to or read from.  The METHOD is a symbol
37that defines the HTTP method to use (case sensitive). MAJOR and MINOR
38identify the major and minor version of HTTP to use. Currently, 0.9,
391.0 and 1.1 are supported (but be careful with 0.9, it has some weird
40consequences and is not widely supported). HEADERS must be a headers
41object.
42
43<procedure>(update-request old-request #!key uri port method major minor headers)</procedure>
44
45Like {{make-request}}, except this takes an {{old-request}} object as
46a template for values which are missing from the parameter list, thereby
47providing a way to do a purely functional update of that object.
48
49
50<procedure>(request-uri REQUEST) => URI</procedure><br>
51<procedure>(request-port REQUEST) => PORT</procedure><br>
52<procedure>(request-method REQUEST) => SYMBOL</procedure><br>
53<procedure>(request-major REQUEST) => NUMBER</procedure><br>
54<procedure>(request-minor REQUEST) => NUMBER</procedure><br>
55<procedure>(request-headers REQUEST) => HEADERS</procedure><br>
56
57An existing request can be picked apart with these accessors.
58
59<procedure>(write-request REQUEST) => REQUEST</procedure>
60
61Write a request line with headers to the server.  In case it
62is a request type that has any body data, this should be written to
63the the request's port. Beware that this port can be modified by
64write-request, so be sure to write to the port as it is returned by
65the write-request procedure!
66
67<procedure>(read-request PORT) => REQUEST</procedure>
68
69Reads a request object from the given input-port.  An optional request
70body can be read from the request-port after calling this procedure.
71
72<parameter>(request-parsers [LIST])</parameter>
73
74Requests are parsed using parse procedures, which can be customized
75by overriding this parameter.
76
77LIST is a list of procedures which accept a request line
78string and produce a request object, or {{#f}} if the
79request is not of the type handled by that procedure.
80
81The predefined request parsers are:
82
83* {{http-0.9-request-parser}}
84* {{http-1.x-request-parser}}
85
86<procedure>(http-0.9-request-parser STRING) => REQUEST</procedure><br>
87<procedure>(http-1.x-request-parser STRING) => REQUEST</procedure><br>
88
89Predefined request parsers for use with {{request-parsers}}.
90
91<parameter>(request-unparsers [LIST])</parameter>
92
93Requests are written using unparse procedures, which can be
94customized by overriding this parameter.
95
96LIST is list of procedures which accept a request object and write
97to the request's output port and return the new, possibly updated
98request object. If the request object is not unparsed by this
99handler, it returns {{#f}}.
100
101The predefined request unparsers are:
102
103* {{http-0.9-request-unparser}}
104* {{http-1.x-request-unparser}}
105
106<procedure>(http-0.9-request-unparser REQUEST) => REQUEST</procedure><br>
107<procedure>(http-1.x-request-unparser REQUEST) => REQUEST</procedure><br>
108
109Predefined request unparsers for use with {{request-unparsers}}.
110They return the request, and as a side effect they write the
111request to the request object's port.
112
113=== Responses
114
115<procedure>(make-response #!key port status (code 200) (reason "OK") (major 1) (minor 1) (headers (headers '())))</procedure>
116
117Create a response, a [[defstruct]]-type record.  A server will usually
118write a response with {{write-response}}; a client will read it
119with {{read-response}}.
120
121You can either supply a status symbol or a code and/or reason to set
122the response status.  If you do, the code will be set to match that
123symbol and the reason is set to the default reason belonging to that
124code, as provided by the HTTP standard(s).  The allowed symbols are
125generally just Schemified versions of the default reason.
126
127Only the code and reason are actual fields in the object; the status
128is a virtual field.
129
130See {{http-status-codes}} for a list of all known default statuses.
131
132<procedure>(update-response old-response #!key port status code reason major minor headers)</procedure>
133
134Like {{make-response}}, except this takes an {{old-response}} object as a
135template for values which are missing from the parameter list, thereby
136providing a way to do a purely functional update of that object.
137
138<procedure>(response-port RESPONSE) => PORT</procedure><br>
139<procedure>(response-code RESPONSE) => NUMBER</procedure><br>
140<procedure>(response-reason RESPONSE) => STRING</procedure><br>
141<procedure>(response-class RESPONSE-OR-CODE) => NUMBER</procedure><br>
142<procedure>(response-major RESPONSE) => NUMBER</procedure><br>
143<procedure>(response-minor RESPONSE) => NUMBER</procedure><br>
144<procedure>(response-headers RESPONSE) => HEADERS</procedure><br>
145<procedure>(response-status RESPONSE-OR-CODE) => SYMBOL</procedure><br>
146
147An existing response can be picked apart using these accessors.
148
149The PORT, MAJOR, MINOR and HEADERS are the same as for requests. CODE
150and REASON are an integer status code and the short message that
151belongs to it, as defined in the spec (examples include: 200 OK, 301
152Moved Permanently, etc).  CLASS is the major class of the response
153code (100, 200, 300, 400 or 500).  {{response-class}} can be called
154either on a response object or directly on a response code number.
155
156{{response-status}} attempts to fetch the symbolic status from the
157response object based on its response code.  If no matching symbol can
158be found in the {{http-status-codes}} parameter, an exception is thrown.
159{{response-status}} can also be called on a response object or directly
160on a response code number.
161
162<procedure>(response-port-set! RESPONSE PORT)</procedure><br>
163<procedure>(response-code-set! RESPONSE NUMBER)</procedure><br>
164<procedure>(response-reason-set! RESPONSE STRING)</procedure><br>
165<procedure>(response-major-set! RESPONSE NUMBER)</procedure><br>
166<procedure>(response-minor-set! RESPONSE NUMBER)</procedure><br>
167<procedure>(response-headers-set! RESPONSE HEADERS)</procedure><br>
168<procedure>(response-status-set! RESPONSE SYMBOL)</procedure><br>
169
170These procedures mutate an existing response object and set the
171corresponding slot. {{response-status-set!}} will attempt to look up
172the code and reason in {{http-status-codes}} and set both slots.
173If the symbolic status is unknown, an exception is thrown.
174
175<parameter>(http-status-codes [ALIST])</parameter>
176
177This is an alist mapping symbolic status indicators to HTTP codes and
178reason strings.
179
180These can be used to make your code a bit more expressive and to
181reduce duplication of hardcoded strings; instead of using a numeric
182"magic number" HTTP code plus the same human-readable string
183everywhere the same code occurs, you can instead use a descriptive
184symbol.
185
186The default value of this mapping is as follows:
187
188<enscript highlight=scheme>
189'((continue . (100 . "Continue"))
190  (switching-protocols . (101 . "Switching Protocols"))
191  (ok . (200 . "OK"))
192  (created . (201 . "Created"))
193  (accepted . (202 . "Accepted"))
194  (non-authoritative-information . (203 . "Non-Authoritative Information"))
195  (no-content . (204 . "No Content"))
196  (reset-content . (205 . "Reset Content"))
197  (partial-content . (206 . "Partial Content"))
198  (multiple-choices . (300 . "Multiple Choices"))
199  (moved-permanently . (301 . "Moved Permanently"))
200  (found . (302 . "Found"))
201  (see-other . (303 . "See Other"))
202  (not-modified . (304 . "Not Modified"))
203  (use-proxy . (305 . "Use Proxy"))
204  (temporary-redirect . (307 . "Temporary Redirect"))
205  (bad-request . (400 . "Bad Request"))
206  (unauthorized . (401 . "Unauthorized"))
207  (payment-required . (402 . "Payment Required"))
208  (forbidden . (403 . "Forbidden"))
209  (not-found . (404 . "Not Found"))
210  (method-not-allowed . (405 . "Method Not Allowed"))
211  (not-acceptable . (406 . "Not Acceptable"))
212  (proxy-authentication-required . (407 . "Proxy Authentication Required"))
213  (request-time-out . (408 . "Request Time-out"))
214  (conflict . (409 . "Conflict"))
215  (gone . (410 . "Gone"))
216  (length-required . (411 . "Length Required"))
217  (precondition-failed . (412 . "Precondition Failed"))
218  (request-entity-too-large . (413 . "Request Entity Too Large"))
219  (request-uri-too-large . (414 . "Request-URI Too Large"))
220  (unsupported-media-type . (415 . "Unsupported Media Type"))
221  (request-range-not-satisfiable . (416 . "Requested range not satisfiable"))
222  (expectation-failed . (417 . "Expectation Failed"))
223  (internal-server-error . (500 . "Internal Server Error"))
224  (not-implemented . (501 . "Not Implemented"))
225  (bad-gateway . (502 . "Bad Gateway"))
226  (service-unavailable . (503 . "Service Unavailable"))
227  (gateway-time-out . (504 . "Gateway Time-out"))
228  (http-version-not-supported . (505 . "HTTP Version not supported")))
229</enscript>
230
231<procedure>(write-response RESPONSE) => RESPONSE</procedure>
232
233Write the response object RESPONSE to the {{response-port}}.
234
235If there is a response body, this must be written to the response-port
236after sending the response headers.
237
238<procedure>(read-response PORT) => RESPONSE</procedure>
239
240Reads a response object from the port. An optional response body can
241be read from the response-port after calling this procedure.
242
243<parameter>(response-parsers [LIST])</parameter>
244
245Responses are parsed using parse procedures, which can be customized
246by overriding this parameter.
247
248LIST is a list one of procedures which accept a response line string
249and produce a response object, or {{#f}} if the response is not of the
250type handled by that procedure.
251
252The predefined response parsers are:
253
254* {{http-0.9-response-parser}}
255* {{http-1.x-response-parser}}
256
257<procedure>(http-0.9-response-parser REQUEST) => REQUEST</procedure><br>
258<procedure>(http-1.x-response-parser REQUEST) => REQUEST</procedure><br>
259
260Predefined response parsers for use with {{response-parser}}.
261
262<parameter>(response-unparsers [LIST])</parameter>
263
264Responses are written using unparse procedures, which can be
265customized by overriding this parameter.
266
267LIST is a list of procedures which accept a response object and write
268to the response's output port and return the new, possibly updated
269response object. If the response object is not unparsed by this
270handler, it returns {{#f}}.
271
272The predefined response unparsers are the following:
273
274* {{http-0.9-response-unparser}}
275* {{http-1.x-response-unparser}}
276
277<procedure>(http-0.9-response-unparser REQUEST) => REQUEST</procedure><br>
278<procedure>(http-1.x-response-unparser REQUEST) => REQUEST</procedure><br>
279
280Predefined response unparsers for use with {{response-unparser}}.
281
282=== Headers
283
284<procedure>(headers ALIST [HEADERS]) => HEADERS</procedure>
285
286This creates a header object based on an input list.
287
288Requests and responses contain HTTP headers wrapped in a special
289header-object to ensure they are properly normalized.
290
291The input list has header names (symbols) as keys, and
292lists of values as values:
293
294<enscript highlight="scheme">
295(headers `((host ("example.com" . 8080))
296           (accept #(text/html ((q . 0.5)))
297                   #(text/xml ((q . 0.1)))))
298          old-headers)
299</enscript>
300
301This adds the named headers to the existing headers in
302{{old-headers}}. The host header is a pair of hostname/port.
303The accept header is a list of allowed mime-type symbols.
304
305As can be seen here, optional parameters or "attributes" can be added
306to a header value by wrapping the value in a vector of length 2. The
307first entry in the vector is the header value, the second is an alist
308of attribute name/value pairs.
309
310<procedure>(headers->list HEADERS) => ALIST</procedure>
311
312This converts a header object back to a list.  See {{headers}}
313for details.
314
315<procedure>(header-values NAME HEADERS) => LIST</procedure>
316
317Obtain the value of header NAME in the HEADERS object.
318
319The NAME of the header is a symbol; this procedure will return all the values
320of the header (for example, the Accept header will have several values
321that indicate the set of acceptable mime-types).
322
323<procedure>(header-value NAME HEADERS [DEFAULT]) => value</procedure>
324
325If you know in advance that a header has only one value, you can use
326{{header-value}} instead of {{header-values}}.  This will return the
327first value in the list, or the provided default if there is no value
328for that header.
329
330<procedure>(header-params NAME HEADERS) => ALIST</procedure>
331
332This will return all the params for a given header, assuming there is
333only one header.  An empty list is returned if the header does not exist.
334
335<procedure>(header-param PARAM NAME HEADERS [DEFAULT]) => value</procedure>
336
337This will return a specific parameter for the header, or DEFAULT
338if the parameter isn't present or the header does not exist.  This
339also assumes there's only one header.
340
341<procedure>(header-contents NAME HEADERS) => LIST</procedure><br>
342<procedure>(get-value VECTOR) => value</procedure><br>
343<procedure>(get-params VECTOR) => ALIST</procedure><br>
344<procedure>(get-param PARAM VECTOR [DEFAULT]) => value</procedure><br>
345
346Procedures such as {{header-values}} are just shortcuts; these are
347the underlying procedures to query the raw contents of a header.
348
349Header contents are lists of 2-element vectors; the first value
350containing the value for the header and the second value containing
351an alist with "parameters" for that header value. Parameters are
352attribute/value pairs that define further specialization of a header's
353value. For example, the {{accept}} header consists of a list of
354mime-types, which optionally can have a quality parameter that
355defines the preference for that mime-type.  All parameter names
356are downcased symbols, just like header names.
357
358Here's a few examples on how to retrieve info from headers:
359
360<enscript highight="scheme">
361;; This would be returned by a server and retrieved via (response-headers r):
362(define example-headers
363  (headers '((accept #(text/html ((q . 0.1)))
364                     #(text/xml ((q . 0.5)))
365                     text/plain)
366             (allow HEAD GET)
367             (content-type #(text/html ((charset . utf-8))))
368             (max-forwards 2))))
369
370;;; Basic procedures
371(define c (header-contents 'accept example-headers))
372c ; => (#(text/html ((q . 0.5))) #(text/xml ((q . 0.1))) #(text/plain ()))
373
374(get-value (car c))
375; => text/html
376(get-params (car c))
377; => ((q . 0.5))
378(get-param 'q (car c))
379; => 0.5
380
381;;; Simplified helpers
382(header-values 'accept example-headers)
383; => (text/html text/xml text/plain)
384(header-values 'max-forwards example-headers)
385; => (2)
386(header-values 'nonexistent-header example-headers)
387; => ()
388
389;; This assumes there's only one value (returns the first)
390(header-value 'max-forwards example-headers)
391; => 2
392(header-value 'nonexistent-header example-headers)
393; => #f
394(header-value 'nonexistent-header example-headers 'not-here)
395; => not-here
396;; Tricky:
397(header-value 'accept example-headers)
398; => text/html
399
400;; This is tricky: this just returns the first, which is not the preferred
401(header-params 'accept example-headers)
402; => ((q . 0.1))
403;; Quick access
404(header-param 'charset 'content-type example-headers)
405; => utf-8
406</enscript>
407
408==== Header types
409
410The headers all have their own different types.  Here follows a list
411of headers with their value types:
412
413<table>
414<tr><th>Header name</th><th>Value type</th><th>Example value</th></tr>
415<tr>
416<td>{{accept}}</td>
417<td>List of mime-types (symbols), with optional {{q}} attribute
418indicating "quality" (preference level)</td>
419<td>{{(text/html #(text/xml ((q . 0.1))))}}</td>
420</tr>
421<tr>
422<td>{{accept-charset}}</td>
423<td>List of charset-names (symbols), with optional {{q}} attribute</td>
424<td>{{(utf-8 #(iso-8859-5 ((q . 0.1))))}}</td>
425</tr>
426<tr>
427<td>{{accept-encoding}}</td>
428<td>List of encoding-names (symbols), with optional {{q}} attribute</td>
429<td>{{(gzip #(identity ((q . 0))))}}</td>
430</tr>
431<tr>
432<td>{{accept-language}}</td>
433<td>List of language-names (symbols), with optional {{q}} attribute</td>
434<td>{{(en-gb #(nl ((q . 0.5))))}}</td>
435</tr>
436<tr>
437<td>{{accept-ranges}}</td>
438<td>List of range types acceptable (symbols). The spec only defines
439{{bytes}} and {{none}}.</td>
440<td>{{(bytes)}}</td>
441</tr>
442<tr>
443<td>{{age}}</td>
444<td>Age in seconds (number)</td>
445<td>{{(3600)}}</td>
446</tr>
447<tr>
448<td>{{allow}}</td>
449<td>List of methods that are allowed (symbols).</td>
450<td>{{(GET POST PUT DELETE)}}</td>
451</tr>
452<tr>
453<td>{{authorization}}</td>
454<td>Authorization information. This consists of a symbol identifying the
455authentication scheme, with scheme-specific attributes.
456{{basic}} is handled specially, as if it were a regular symbol with two
457attributes; {{username}} and {{password}}.</td>
458<td>{{(#(basic ((username . "foo") (password . "bar"))) #(digest ((qop . auth) (username . "Mufasa") (nc . 1))))}}</td>
459</tr>
460<tr>
461<td>{{cache-control}}</td>
462<td>An alist of key/value pairs. If no value is applicable, it is {{#t}}</td>
463<td>{{((public . #t) (max-stale . 10) (no-cache . (age set-cookie)))}}</td>
464</tr>
465<tr>
466<td>{{connection}}</td>
467<td>A list of connection options (symbols)</td>
468<td>{{(close)}}</td>
469</tr>
470<tr>
471<td>{{content-disposition}}</td>
472<td>A symbol indicating the disposition</td>
473<td>{{(#(inline ((filename . "test.pdf"))))}}</td>
474</tr>
475<tr>
476<td>{{content-encoding}}</td>
477<td>A list of encodings (symbols) applied to the entity-body.</td>
478<td>{{(deflate gzip)}}</td>
479</tr>
480<tr>
481<td>{{content-language}}</td>
482<td>The natural language(s) of the "intended audience" (symbols)</td>
483<td>{{(de nl en-gb)}}</td>
484</tr>
485<tr>
486<td>{{content-length}}</td>
487<td>The number of bytes (an exact number) in the entity-body</td>
488<td>{{(10)}}</td>
489</tr>
490<tr>
491<td>{{content-location}}</td>
492<td>A location that the content can be retrieved from (a uri-common object)</td>
493<td>{{(<#uri-common# ...>)}}</td>
494</tr>
495<tr>
496<td>{{content-md5}}</td>
497<td>The MD5 checksum (a string) of the entity-body</td>
498<td>{{("12345ABCDEF")}}</td>
499</tr>
500<tr>
501<td>{{content-range}}</td>
502<td>Content range (pair with start- and endpoint) of the entity-body, if partially sent</td>
503<td>{{((25 . 120))}}</td>
504</tr>
505<tr>
506<td>{{content-type}}</td>
507<td>The mime type of the entity-body (a symbol)</td>
508<td>{{(#(text/html ((charset . iso-8859-1))))}}</td>
509</tr>
510<tr>
511<td>{{date}}</td>
512<td>A timestamp (10-element vector, see {{string->time}}) at which the message originated. ''Important'': Note that you will always need to supply (an empty list of) attributes, because otherwise it is ambiguous whether it's a vector with attribs or a bare timestamp.</td>
513<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
514</tr>
515<tr>
516<td>{{etag}}</td>
517<td>An entity-tag (pair, car being either the symbol weak or strong, cdr being a string) that uniquely identifies the resource contents.</td>
518<td>{{((strong . "foo123"))}}</td>
519</tr>
520<tr>
521<td>{{expect}}</td>
522<td>Expectations of the server's behaviour (alist of symbol-string pairs), possibly with parameters.</td>
523<td>{{(#(((100-continue . #t)) ()))}}</td>
524</tr>
525<tr>
526<td>{{expires}}</td>
527<td>Expiry timestamp (10-element vector, see {{string->time}}) for the entity. Also see the note for {{date}}</td>
528<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
529</tr>
530<tr>
531<td>{{from}}</td>
532<td>The e-mail address (a string) of the human user who controls the client</td>
533<td>{{("info@example.com")}}</td>
534</tr>
535<tr>
536<td>{{host}}</td>
537<td>The host to use (for virtual hosting). This is a pair of hostname and port. The port will be {{#f}} if the port should be the default one for the requested service.</td>
538<td>{{(("example.com" . 8080))}}</td>
539</tr>
540<tr>
541<td>{{if-match}}</td>
542<td>Either {{'*}} (a wildcard symbol) or a list of entity-tags (pair, weak/strong symbol and unique entity identifier string).</td>
543<td>{{((strong . "foo123") (strong . "bar123"))}}</td>
544</tr>
545<tr>
546<td>{{if-modified-since}}</td>
547<td>Timestamp (10-element vector, see {{string->time}}) which indicates since when the entity must have been modified.</td>
548<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
549</tr>
550<tr>
551<td>{{if-none-match}}</td>
552<td>Either {{'*}} (a wildcard symbol) or a list of entity-tags (pair, weak/strong symbol and unique entity identifier symbol).</td>
553<td>{{((strong . foo123) (strong . bar123))}}</td>
554</tr>
555<tr>
556<td>{{if-range}}</td>
557<td>The range to request, if the entity was unchanged</td>
558<td>TODO</td>
559</tr>
560<tr>
561<td>{{if-unmodified-since}}</td>
562<td>A timestamp (10-element vector, see {{string->time}}) since which the entity must not have been modified</td>
563<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
564</tr>
565<tr>
566<td>{{last-modified}}</td>
567<td>A timestamp (10-element vector, see {{string->time}}) when the entity was last modified</td>
568<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
569</tr>
570<tr>
571<td>{{location}}</td>
572<td>A location (an URI object) to which to redirect</td>
573<td>{{(<#uri-object ...>)}}</td>
574</tr>
575<tr>
576<td>{{max-forwards}}</td>
577<td>The maximum number of proxies that can forward a request</td>
578<td>{{(2)}}</td>
579</tr>
580<tr>
581<td>{{pragma}}</td>
582<td>An alist of symbols containing implementation-specific directives.</td>
583<td>{{((no-cache . #t) (my-extension . my-value))}}</td>
584</tr>
585<tr>
586<td>{{proxy-authenticate}}</td>
587<td>Proxy authentication request.  Equivalent to {{www-authenticate}}, for proxies.</td>
588<td>{{(#(basic ((realm . "foo")) ) #(digest ((realm . "foo") (domain . (<#uri object> <#uri object>)) (qop . (auth auth-int)) (nonce . "012345abc"))))}}</td>
589</tr>
590<tr>
591<td>{{proxy-authorization}}</td>
592<td>The answer to a {{proxy-authentication}} request. Equivalent to {{authorization}}, for proxies.</td>
593<td>{{(#(basic ((username . "foo") (password . "bar"))) #(digest ((qop . auth) (username . "Mufasa") (nc . 1))))}}</td>
594</tr>
595<tr>
596<td>{{range}}</td>
597<td>The range of bytes (a pair of start and end) to request from the server.</td>
598<td>{{((25 . 120))}}</td>
599</tr>
600<tr>
601<td>{{referer}}</td>
602<td>The referring URL (uri-common object) that linked to this one.</td>
603<td>{{(<#uri-object ...>)}}</td>
604</tr>
605<tr>
606<td>{{retry-after}}</td>
607<td>Timestamp (10-element vector, see {{string->time}}) after which to retry the request if unavailable now.</td>
608<td>{{(#(#(42 23 15 20 6 108 0 309 #f 0) ()))}}</td>
609</tr>
610<tr>
611<td>{{server}}</td>
612<td>List of products the server uses (list of 3-tuple lists of strings; product name, product version, comment. Version and/or comment may be {{#f}}). Note that this is a single header, with a list inside it!</td>
613<td>{{((("Apache" "2.2.9" "Unix") ("mod_ssl" "2.2.9" #f) ("OpenSSL" "0.9.8e" #f) ("DAV" "2" #f) ("mod_fastcgi" "2.4.2" #f) ("mod_apreq2-20051231" "2.6.0" #f)))}}</td>
614</tr>
615<tr>
616<td>{{te}}</td>
617<td>Allowed transfer-encodings (symbols, with optional q attribute) for the response</td>
618<td>{{(deflate #(gzip ((q . 0.2))))}}</td>
619</tr>
620<tr>
621<td>{{trailer}}</td>
622<td>Names of header fields (symbols) available in the trailer/after body</td>
623<td>{{(range etag)}}</td>
624</tr>
625<tr>
626<td>{{transfer-encoding}}</td>
627<td>The encodings (symbols) used in the body</td>
628<td>{{(chunked)}}</td>
629</tr>
630<tr>
631<td>{{upgrade}}</td>
632<td>Product names to which must be upgraded (strings)</td>
633<td>TODO</td>
634</tr>
635<tr>
636<td>{{user-agent}}</td>
637<td>List of products the user agent uses (list of 3-tuple lists of strings; product name, product version, comment. Version and/or comment may be {{#f}}). Note that this is a single header, with a list inside it!</td>
638<td>{{((("Mozilla" "5.0" "X11; U; NetBSD amd64; en-US; rv:1.9.0.3") ("Gecko" "2008110501" #f) ("Minefield" "3.0.3" #f)))}}</td>
639</tr>
640<tr>
641<td>{{vary}}</td>
642<td>The names of headers that define variation in the resource body, to determine cachability (symbols)</td>
643<td>{{(range etag)}}</td>
644</tr>
645<tr>
646<td>{{via}}</td>
647<td>The intermediate hops through which the message is forwarded (strings)</td>
648<td>TODO</td>
649</tr>
650<tr>
651<td>{{warning}}</td>
652<td>Warning code for special status</td>
653<td>TODO</td>
654</tr>
655<tr>
656<td>{{www-authenticate}}</td>
657<td>If unauthorized, a challenge to authenticate (symbol, with attributes)</td>
658<td>{{(#(basic ((realm . "foo"))) #(digest ((realm . "foo") (domain . (<#uri object> <#uri object>)) (qop . (auth auth-int)) (nonce . "012345abc"))))}}</td>
659</tr>
660<tr>
661<td>{{set-cookie}}</td>
662<td>Cookies to set (name/value pair (both strings), with attributes)</td>
663<td>{{(#(("foo" . "bar") ((max-age . 10) (port . '(80 8080))))}}</td>
664</tr>
665<tr>
666<td>{{cookie}}</td>
667<td>Cookies that were set (name/value string pair, with attributes)</td>
668<td>{{(#(("foo" . "bar") ((version . 1) (path . #(uri path: (/ ""))) (domain . "foo.com"))))}}</td>
669</tr>
670<tr>
671<td>{{x-forwarded-for}}</td>
672<td>The chain of IP addresses which each intermediate proxy will add to.  Plain strings representing the IP-address or "unknown" when the proxy couldn't determine the client address or the option is disabled. ''Never accept this value without question; it can easily be spoofed!''</td>
673<td>{{("192.168.1.2" "unknown" "123.456.789.012" "some-made-up-value-by-an-attacker")}}</td>
674</tr>
675
676</table>
677
678Any unrecognised headers are assumed to be multi-headers, and the
679entire header lines are put unparsed into a list, one entry per line.
680
681
682==== Header parsers and unparsers
683
684<parameter>(header-parsers [ALIST])</parameter><br>
685<parameter>(header-unparsers [ALIST])</parameter><br>
686
687The parsers and unparsers used to read and write header values can be
688customized with these parameters.
689
690These (un)parsers are indexed with as key the header name (a symbol)
691and the value being a procedure.
692
693A header parser accepts the contents of the header (a string, without
694the leading header name and colon) and returns a ''list of vectors''
695which represents the values of the header.  For headers that are
696supposed to only have a single value, the last value in the list will
697be stored as the value (as determined by {{single-headers}}).
698
699A header unparser accepts one argument: the header's contents (a
700vector).  It should return a list of strings, each of which represents
701one line's worth of header contents (without the header name).  For
702each entry, a header line will automatically be printed with the
703header name preceding it.
704
705The parser driver will call {{update-header-contents!}} with the
706parser's result.
707
708<parameter>(header-parse-error-handler [HANDLER])</parameter>
709
710When there is an error parsing a given header, this
711parameter's procedure will be invoked.
712
713{{HANDLER}} is a procedure accepting four values: the header name, the
714header contents, the current headers and the exception object.  The
715procedure must return the new headers.  Defaults to a procedure that
716simply returns the current headers.  When an error occurs while
717parsing the header line itself (for example when a colon is missing
718between the header name and contents), the error will not be caught.
719
720In such a case, Servers should return a 400 Bad Request error and
721clients should error out.  The reason that malformed error lines are
722ignored is that there are several servers and clients that send
723headers content values that are slightly off, even though the rest of
724the request is OK.  In the interest of the "robustness principle",
725it's best to simply ignore these headers with "bad" content values.
726
727<procedure>(replace-header-contents NAME CONTENTS HEADERS) => HEADERS</procedure><br>
728<procedure>(replace-header-contents! NAME CONTENTS HEADERS) => HEADERS</procedure><br>
729<procedure>(update-header-contents NAME CONTENTS HEADERS) => HEADERS</procedure><br>
730<procedure>(update-header-contents! NAME CONTENTS HEADERS) => HEADERS</procedure><br>
731
732The {{replace}} procedures replace any existing contents of the named
733header with new ones, the {{update}} procedures add these contents to
734the existing header. The procedures with a name ending in bang are
735linear update variants of the ones without the bang. The header
736contents have to be normalized to be a 2-element vector, with the
737first element being the actual value and the second element being an
738alist (possibly empty) of parameters/attributes for that value.
739
740The update procedures append the value to the existing header if it is
741a multi-header, and act as a simple replace in the case of a
742single-header.
743
744<parameter>(single-headers [LIST])</parameter>
745
746Whether a header is allowed once or multiple times in a request or
747response is determined by this parameter.
748
749The value is a list of symbols that define header-names which are
750allowed to occur only once in a request/response.
751
752<procedure>(http-name->symbol STRING) => SYMBOL</procedure><br>
753<procedure>(symbol->http-name SYMBOL) => STRING</procedure><br>
754
755These procedures convert strings containing the name of a header or
756attribute (parameter name) to symbols representing the same. The
757symbols are completely downcased.  When converting this symbol back to
758a string, the initial letters of all the words in the header name or
759attribute are capitalized.
760
761<procedure>(remove-header name headers) => headers</procedure><br>
762<procedure>(remove-header! name headers) => headers</procedure><br>
763
764These two procedures remove all headers with the given name.
765
766===== Header subparsers and subunparsers
767
768Some headers are modular ''themselves''.  This means they need some
769way to extend them.  This is done through subparsers and subunparsers.
770
771<parameter>(authorization-param-subparsers [ALIST])</parameter>
772
773This is an alist of subtypes for the {{authorization}} header parser.
774A subparser of this kind accepts the string containing the header and
775an integer position in the string.  It should parse from that position
776onwards, and return the parsed contents as an alist of header
777parameters.  Usually, these are actually pseudo-parameters; they don't
778necessarily have to appear in parameter syntax in the header.  The
779unparser should be configured to expect the same parameters and combine
780them back into a string, though.
781
782This parameter defaults to:
783
784<enscript highligh=scheme>
785`((basic . ,basic-auth-subparser)
786  (digest . ,digest-auth-subparser))
787</enscript>
788
789<procedure>(basic-auth-param-subparser STR POS)</procedure>
790
791Parses {{STR}} at {{POS}} by extracting the username and password
792components from a base64-encoded string.  These are returned in its
793first value as an alist with keys {{username}} and {{password}}.  Its
794second return value is the position after which the next header value
795may begin.
796
797<procedure>(digest-auth-param-subparser STR POS)</procedure>
798
799Parses {{STR}} at {{POS}} by reading the various components from a
800parameter list.  These are returned in its first return value as an
801alist with keys {{nc}}, {{uri}}, {{qop}} and {{algorithm}}.  Its
802second return value is the position after which the next header value
803may begin.
804
805<parameter>(authorization-param-subunparsers [ALIST])</parameter>
806
807This is an alist of subtypes for the {{authorization}} header
808unparser.  An unparser of this kind accepts an alist containing the
809parameters that it needs to unparse and should return a string
810containing the raw unparsed parameters only.
811
812This parameter defaults to:
813
814<enscript highligh=scheme>
815`((basic . ,basic-auth-subunparser)
816  (digest . ,digest-auth-subunparser))
817</enscript>
818
819<procedure>(basic-auth-param-subunparser PARAMS)</procedure>
820
821This unparses the {{PARAMS}} alist into a base64-encoded string for
822basic authentication.  It expects {{username}} and {{password}}
823parameters.
824
825<procedure>(digest-auth-param-subunparser PARAMS)</procedure>
826
827This unparses the {{PARAMS}} alist into a string for digest
828authentication. It expects {{username}}, {{uri}}, {{realm}},
829{{nonce}}, {{cnonce}}, {{qop}}, {{nc}}, {{response}}, {{opaque}} and
830{{algorithm}} parameters.  The {{response}} parameter should be
831pre-encoded in the way digest auth expects (this is not done here
832because the MD5 sum of the contents may be required, which is not
833available to the parsers).
834
835TODO: This will probably change in the future; the md5 can be passed
836and all the hard stuff can be done in intarweb.
837
838=== Other procedures
839
840<parameter>(http-line-limit [length])</parameter>
841
842The maximum length of any line that's read by intarweb as part of the
843request/response cycle.  This includes the request and response lines
844as well as the headers.  If this is exceeded, an exception of type
845{{(exn http line-limit-exceeded)}} is raised.
846
847You can set this to {{#f}} to disable this check. However, this will
848open up a resource consumption vulnerability (attackers can cause
849your application to blow up by letting it use all available memory).
850
851Defaults to {{1024}}.
852
853<parameter>(http-header-limit [count])</parameter>
854
855The maximum number of headers that are allowed to be sent, as part
856of a request or response. If this is exceeded, an exception of type
857{{(exn http header-limit-exceeded)}} is raised.
858
859You can set this to {{#f}} to disable this check. However, this will
860open up a resource consumption vulnerability (attackers can cause
861your application to blow up by letting it use all available memory).
862
863Defaults to {{256}}.
864
865<procedure>(keep-alive? request-or-response)</procedure>
866
867Returns {{#t}} when the given request or response object belongs to a
868connection that should be kept alive, {{#f}} if not.  Remember that
869both parties must agree on whether the connection is to be kept alive
870or not; HTTP/1.1 defaults to keep alive unless a {{Connection: close}}
871header is sent, HTTP/1.0 defaults to closing the connection, unless a
872{{Connection: Keep-Alive}} header is sent.
873
874<parameter>(request-has-message-body? [predicate])</parameter>
875
876This parameter holds a predicate which accepts a request object and
877returns {{#t}} when the request will have a message body.  By default
878in HTTP/1.1, this is the case for all requests that have a {{content-length}}
879or {{transfer-coding}} header.
880
881The parameter is useful for servers to determine whether to write a
882response body or not.
883
884<parameter>(response-has-message-body-for-request? [predicate])</parameter>
885
886This parameter holds a predicate which accepts two arguments: a
887response object and a request object.  It returns {{#t}} when the
888response will have a message body for the given request.  By default
889in HTTP/1.1, this is '''not''' the case for responses with a
890response-code of 204 and 304 or in the 1xx class, nor for {{HEAD}}
891requests.  All other responses will have a message body.
892
893The parameter is useful for deciding in clients whether a message body
894will follow (otherwise, trying to read will probably result in an error
895or in case of HTTP pipelining in a synchronisation problem)
896
897<procedure>(safe? request-or-method)</procedure>
898
899Returns {{#t}} when the given request object or symbol (method) is a
900''safe'' method.  A method is defined to be safe when a request of
901this method will have no side-effects on the server.  In practice this
902means that you can send this request from anywhere at any time and
903cause no damage.
904
905'''Important''': Quite a lot of software does not abide by these
906rules!  This is not necessarily a reason to treat all methods as
907unsafe, however.  In the words of the standard "the user did not
908request the side-effects, so therefore cannot be held accountable for
909them".  If a safe method produces side-effects, that's the server-side
910script developer's fault and he should fix his code.
911
912<parameter>(safe-methods [symbols])</parameter>
913
914A list of methods which are to be considered safe.  Defaults to
915{{'(GET HEAD OPTIONS TRACE)}}.
916
917<procedure>(idempotent? request-or-method)</procedure>
918
919Returns {{#t}} when the given request object or symbol (method) is a
920''idempotent'' method.  A method is defined to be idempotent when a
921series of identical requests of this method in succession causes the
922exact same side-effect as just one such request.  In practice this
923means that you can safely retry such a request when an error occurs,
924for example.
925
926'''Important''': Just as with the ''safe'' methods, there is no
927guarantee that methods that ''should be'' idempotent really are
928idempotent in any given web application.  Furthermore, a sequence of
929requests which each are individually idempotent is not necessarily
930idempotent as a whole.  This means that you cannot replay requests
931starting anywhere in the chain.  To be on the safe side, only retry
932the last request in the chain.
933
934<parameter>(idempotent-methods [symbols])</parameter>
935
936A list of methods which are to be considered idempotent.  Defaults to
937{{'(GET HEAD PUT DELETE OPTIONS TRACE)}}.
938
939<procedure>(etag=? a b)</procedure>
940
941Do the etag values {{a}} and {{b}} strongly match?  That is, their
942{{car}} and {{cdr}} must be equal, and neither can have a {{car}} of
943{{weak}} (both must be {{strong}}).
944
945<procedure>(etag=-weakly? a b)</procedure>
946
947Do the etag values {{a}} and {{b}} weakly match?  That is, their
948{{car}} and {{cdr}} must be equal.  A {{car}} of {{weak}} is allowed.
949
950<procedure>(etag-matches? etag matches)</procedure>
951
952Does the {{etag}} strongly match any of the etags in the list
953{{matches}}?  {{matches}} is a plain list of etag values, but it can
954also contain the special symbol {{*}}, which matches any etag.
955
956<procedure>(etag-matches-weakly? etag matches)</procedure>
957
958Does the {{etag}} weakly match any of the etags in the list
959{{matches}}?  {{matches}} is a plain list of etag values, but it can
960also contain the special symbol {{*}}, which matches any etag.
961
962=== Changelog
963
964* trunk Treat the charset attribute for Content-Type header as case-insensitive token for consistency with Accept-Charset header.  Remove dependency on the [[/eggref/4/regex|regex egg]] and improve correctness of a few parsers.  Add {{request-has-message-body?}} and {{response-has-message-body-for-request?}} procedures.  Add parser for Content-Disposition header and improve unparser by adding date support (Thanks to Evan Hanson).  Implement line length and header count limit checking.
965* 0.7 Add trivial {{x-forwarded-for}} "parser".  Add easier overriding of {{authorization}} headers through parameter instead of having to rewrite the entire parser.  Add {{content-disposition}} unparser to accommodate the fact that filenames ''must'' always be quoted.  Add {{http-status-codes}} parameter and {{status:}} key to {{update-response}} and {{make-response}} procedures, as well as {{response-status}} and {{response-status-set!}} procedures.
966* 0.6 Change path parameters on cookies to be uri-common objects
967* 0.5 Add regex requirement to make it work with Chicken 4.6.2+
968* 0.4 Don't unparse "attributes" (aka "params") by titlecasing their names. Don't default Host header's port to 80, but use #f
969* 0.3 Add rfc1123 to default unparser list for the 'date' header. Add etag procedures and if-match unparser. Change procedure signature for header unparsers
970* 0.2 Make cookie header parsers/unparsers preserve cookie name case. Change header unparse procedure semantics slightly.
971* 0.1 Initial version
972
973=== License
974
975  Copyright (c) 2008-2012, Peter Bex
976  All rights reserved.
977 
978  Redistribution and use in source and binary forms, with or without
979  modification, are permitted provided that the following conditions are
980  met:
981 
982  Redistributions of source code must retain the above copyright
983  notice, this list of conditions and the following disclaimer.
984 
985  Redistributions in binary form must reproduce the above copyright
986  notice, this list of conditions and the following disclaimer in the
987  documentation and/or other materials provided with the distribution.
988 
989  Neither the name of the author nor the names of its contributors may
990  be used to endorse or promote products derived from this software
991  without specific prior written permission.
992 
993  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
994  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
995  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
996  FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
997  COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
998  INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
999  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
1000  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
1001  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
1002  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
1003  ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
1004  OF THE POSSIBILITY OF SUCH DAMAGE.
Note: See TracBrowser for help on using the repository browser.