source: project/wiki/eggref/4/http-client @ 25142

Last change on this file since 25142 was 25142, checked in by sjamaan, 10 years ago

Update changelog for intarweb, http-client and spiffy

File size: 15.5 KB
Line 
1[[tags: egg]]
2
3== http-client
4
5[[toc:]]
6
7=== Description
8
9Http-client is a highlevel HTTP client library.
10
11=== Author
12
13[[/users/peter-bex|Peter Bex]]
14
15=== Requirements
16
17Requires the [[intarweb]] and [[openssl]] extensions.
18
19=== Documentation
20
21This egg is still under development; the API might change a bit in
22future versions.
23
24==== Main request procedures
25
26<procedure>(call-with-response request writer reader)</procedure>
27
28This is the core http-client procedure.  It is only necessary to use
29this when you want the most control over the request/response cycle.
30{{request}} is the request object that contains information about the
31request to perform.  {{reader}} is a procedure that receives the
32response object and should read the ''entire'' request body (any
33leftover data will cause errors on subsequent requests with keepalive
34connections), {{writer}} is a procedure that receives the request
35object and should write the request body.
36
37The {{writer}} should be prepared to be called several times; if the
38response is a redirect or some other status that indicates the server
39wants the client to perform a new request, the writer should be ready
40to write a request body for this new request. In case digest
41authentication with message integrity checking is used, {{writer}} is
42always invoked at least twice, once to determine the message digest of
43the response and once to actually write the response.
44
45Returns three values: The result of the call to {{reader}}, the
46request-uri of the last request and the response object. The
47request-uri is useful because this is to be used as the base uri of
48the document. This can differ from the initial request in the presence
49of redirects.
50
51<procedure>(call-with-input-request uri-or-request writer reader)</procedure>
52
53This procedure is a convenience wrapper around {{call-with-response}}.
54
55It is much less strict - {{uri-or-request}} can be an [[intarweb]]
56request object, but also an uri-common object or even a string with
57the URI in it, in which case a request object will be automatically
58constructed around the URI, using the {{GET}} method when {{writer}}
59is {{#f}} or the {{POST}} method when {{writer}} is not {{#f}}.
60
61{{writer}} can be either {{#f}} (in which case nothing is written and
62the {{GET}} method chosen), a string containing the raw data to send,
63an alist, or a procedure that accepts a port and writes the
64response data to it.  If you supply a procedure, do not forget to set
65the {{content-length}} header!  In the other cases, whenever possible,
66the length is calculated and the header automatically set for you.
67
68If you supplied an alist, the {{content-type}} header is automatically
69set to {{application/x-www-form-urlencoded}} unless there's an alist
70entry whose value is a list starting with the keyword {{file:}}, in
71which case {{multipart/form-data}} is used.  See the examples for
72{{with-input-from-request}} below.
73
74{{reader}} is a procedure which accepts a port and reads out the data.
75If there is data left in the port when it returns, this will be
76automatically discarded to avoid problems.
77
78Returns three values: The result of the call to {{reader}}, the
79request-uri of the last request and the response object.  If the
80response code is not in the 200 class, it will throw an exception of
81type {{(exn http client-error)}}, {{(exn http server-error)}} or
82{{(exn http unexpected-server-response)}}, depending on the response
83code.  This includes {{404 not found}} (which is a {{client-error}}).
84
85When posting multipart form data, the value of a file entry is a list
86of keyword-value pairs.  The following keywords are recognised:
87
88; {{file:}} : This indicates the file to read from.  Can be either a string or a port. This ''must'' be specified, everything else is optional.
89; {{filename:}} : This indicates the filename to pass on to the server.  If not specified or {{#f}}, the {{file:}}'s string (or port-name in case of a port) will be used.
90; {{headers:}} : Additional headers to send for this entry (an [[intarweb]] headers-object).
91
92<procedure>(with-input-from-request uri-or-request writer-thunk reader-thunk)</procedure>
93
94Same as {{call-with-input-request}}, except when you pass a procedure
95as {{reader-thunk}} or {{writer-thunk}} it has to be a thunk (lambda
96of no arguments) instead of a procedure of one argument.  These thunks
97will be executed with the current input (or output) port to the
98request or response port, respectively.
99
100You can still pass {{#f}} for both or an alist or string for
101{{writer-thunk}}.
102
103===== Examples
104
105<enscript highlight="scheme">
106(use http-client)
107
108;; Start with a simple GET request:
109(with-input-from-request "http://wiki.call-cc.org/" #f read-string)
110 => ;; [the chicken wiki page HTML contents]
111
112;; Perform a POST of the key "test" with value "value" to an echo service:
113(with-input-from-request "http://localhost/echo-service"
114                         '((test . "value")) read-string)
115 => "You posted: test=value"
116
117;; Posting a file to the same echo-services:
118(with-input-from-request "http://localhost/echo-service"
119                         '((test . "value")
120                           (test-file file: "/tmp/myfile" filename: "hello.txt"
121                                      headers: (headers
122                                                `((content-type text/plain))))))
123                         read-string)
124 => "You posted: test=value and a file named \"hello.txt\""
125
126
127;; Performing a PUT request (a less commonly used method) requires
128;; constructing your request object manually:
129
130(use intarweb uri-common)  ; Required for "make-request" and "uri-reference"
131
132(with-input-from-request
133  (make-request method: 'PUT
134                uri: (uri-reference "http://example.com/blabla"))
135  (lambda () (print "Page contents"))
136  read-string)
137</enscript>
138
139
140==== Request handling parameters
141
142<parameter>(max-retry-attempts [number])</parameter>
143
144When a request fails because of an I/O or network problem (or simply
145because the remote end closed a persistent connection while we were
146doing something else), the library will try to establish a new
147connection and perform the request again.  This parameter controls how
148many times this is allowed to be done.  If {{#f}}, it will never give up.
149
150Defaults to 1.
151
152<parameter>(retry-request? [predicate])</parameter>
153
154This procedure is invoked when a retry should take place, to determine
155if it should take place at all.  It should be a procedure accepting a
156request object and returning {{#f}} or a true value.  If the value is
157true, the new request will be sent.  Otherwise, the error that caused
158the retry attempt will be re-raised.
159
160Defaults to {{idempotent?}}, from [[intarweb]].  This is because
161non-idempotent requests cannot be safely retried when it is unknown
162whether the previous request reached the server or not.
163
164<parameter>(max-redirect-depth [number])</parameter>
165
166The maximum number of allowed redirects, or {{#f}} if there is no
167limit.  Currently there's no automatic redirect loop detection
168algorithm implemented.  If zero, no redirects will be followed at all.
169
170Defaults to 5.
171
172<parameter>(client-software [software-spec])</parameter>
173
174This is the names, versions and comments of the software packages that
175the client is using, for use in the {{user-agent}} header which is
176automatically added to each request.
177
178Defaults to {{(("Chicken Scheme HTTP-client" VERSION #f))}}, where
179{{VERSION}} is the version of this egg.
180
181
182==== Connection management
183
184<procedure>(close-connection! uri)</procedure>
185
186Close the connection to the server associated with the URI.
187
188<procedure>(close-all-connections!)</procedure>
189
190Close all connections to all servers.
191
192
193==== Cookie management
194
195http-client's cookie management is supposed to be as automatic and
196DWIMmy as possible.  This means it will write any cookie as instructed
197by a server and all stored cookies are automatically sent back to the
198server upon a new request.
199
200However, in some cases you may want to take control of how cookies are
201stored.
202
203The API described here should be considered unstable and it may change
204dramatically when someone comes up with a better way to handle cookies.
205
206<procedure>(get-cookies-for-uri uri)</procedure>
207
208Fetch a list of all cookies which ought to be sent to the given URI.
209Cookies are vectors of two elements: a name/value pair and an alist of
210attributes.  In other words, these are the exact same values you can
211put in a {{cookie}} header.
212
213<procedure>(store-cookie! cookie-info set-cookie)</procedure>
214
215Store a cookie in the cookiejar corresponding to the Set-Cookie header
216given by {{set-cookie}}.  This overwrites any cookie that is equal to
217this cookie, as defined by RFC 2965, section 3.3.3.  Practically, this
218means that when the cookie's name, domain and path are equal to an
219existant one, it will be overwritten by the new one.  These attributes
220are taken from the {{cookie-info}} alist and expected to be there.
221
222Generally, attributes should be taken from {{set-cookie}}, but if
223missing they ought to be taken from the request URI that responded
224with the {{set-cookie}}.
225
226<procedure>(delete-cookie! cookie-name cookie-info)</procedure>
227
228Removes any cookie from the cookiejar that is equal to the given
229cookie (again, in the sense of RFC 2965, section 3.3.3).
230The {{cookie-name}} must match and the {{path}} and {{domain}} values for
231the {{cookie-info}} alist must match.
232
233==== Authentication support
234
235When a 401 Unauthorized response is received, in most interactive
236clients, the user is normally asked to authenticate.  To support this
237type of interaction, http-client offers the following parameter:
238
239<parameter>(determine-username/password [HANDLER])</parameter>
240
241The procedure in this parameter is called whenever the remote
242host requests authentication via a 401 Unauthorized response.
243
244The {{HANDLER}} is a procedure of two arguments; the URI for the
245resource currently being requested and the realm (a string) which
246wants credentials.  The procedure should return two string values:
247the username and the password to use for authentication.
248
249The default value is a procedure which extracts the username and
250password components from the URI.
251
252For proxy authentication support, see {{determine-proxy-username/password}}
253in the next section.
254
255==== Proxy support
256
257http-client has support for sending requests through proxy servers.
258
259<parameter>(determine-proxy [HANDLER])</parameter>
260
261Whenever a request is sent, the library invokes the procedure stored
262in this parameter to determine through what proxy to send the request,
263if any.
264
265The {{HANDLER}} procedure receives one argument, the URI about to be
266requested, and returns either an URI-common absolute URI object
267representing the proxy or {{#f}} if no proxy should be used.
268
269The URI's path and query, if present, are ignored; only the scheme
270and authority (host, port, username, password) are used.
271
272The default value of this parameter is {{determine-proxy-from-environment}}.
273
274If you just want to disable proxy support, you can do:
275
276<enscript highlight="scheme">
277(determine-proxy (constantly #f))   ; From unit data-structures
278</enscript>
279
280<procedure>(determine-proxy-from-environment URI)</procedure>
281
282This procedure implements the common behaviour of HTTP software under
283UNIX:
284
285* First it checks if the requested URI's host (or an asterisk) is listed in the {{NO_PROXY}} environment variable (if suffixed with a port number, the port is also compared).  If a match is found, no proxy is used.
286* Then it will check if the {{$(protocol)_proxy}} or the {{$(PROTOCOL)_PROXY}} variable (in that order) are set.  If so, that's used.  {{protocol}} here actually means "scheme", so the URI's scheme is used, suffixed with {{_proxy}}. This means {{http_proxy}} is used for HTTP requests and {{https_proxy}} is used for HTTPS requests.
287* If there's still no match, it looks for {{all_proxy}} or {{ALL_PROXY}}, in that order. If one of these environment variables are set, that value is used as a fallback proxy.
288* Finally, if none of these checks resulted in a proxy URI, no proxy will be used.
289
290Some UNIX software expects plain hostnames or hostname port
291combinations separated by colons, but (currently) this library expects
292full URIs, like most modern UNIX programs.
293
294<parameter>(determine-proxy-username/password [HANDLER])</parameter>
295
296The procedure in this parameter is called whenever the proxy requests
297authentication via a 407 Proxy Authentication Required response. This
298basically works the same as authentication against an origin server.
299
300The {{HANDLER}} is a procedure of two arguments; the URI for the
301''proxy'' currently being used and the realm (a string) which wants
302credentials.  The procedure should return two string values: the
303username and the password to use for authentication.
304
305The default value is a procedure which extracts the username and
306password components from the proxy's URI.
307
308
309=== Changelog
310
311* 0.5 Improve detection of dropped connections (prevents unneccessary "connection reset" exceptions to propagate into the program). Simplify interface by switching to {{POST}} when a {{writer}} is given to {{with-input-from-request}} and {{call-with-input-request}}.  Add support for multipart forms (file upload). Fix error in case of missing username when authorization was required (introduced by version 0.4.2). Put loop call in tail position (thanks to [[/users/felix-winkelmann|Felix]]) Automatically discard remaining data on the input port, if any, to avoid problems on subsequent requests. Add rudimentary support for parameterizable authentication schemes.
312* 0.4.2 Allow missing passwords in URIs for authentication
313* 0.4.1 Fix connection status check so when the remote end closed the connection we don't try to read from it anymore (thanks to Daishi Kato and Thomas Hintz)
314* 0.4 Fix redirection code on 303, and off-by-1 mistake in redirects count (thanks to Moritz Heidkamp). Add arguments to exn objects (thanks to Christian Kellermann). Also accept an empty alist for POSTdata. Fix URI path comparisons in cookies (thanks to Daishi Kato)
315* 0.3 Fixed handling of missing Path parameters in set-cookie headers. Reported by Hugo Arregui. Improve set-cookie handling by only passing Path and Domain when matching Set-Cookie header included those parameters.
316* 0.2 Added proxy support and many many bugfixes
317* 0.1 Initial version
318
319=== License
320
321  Copyright (c) 2008-2011, Peter Bex
322  Parts copyright (c) 2000-2004, Felix L. Winkelmann
323  All rights reserved.
324 
325  Redistribution and use in source and binary forms, with or without
326  modification, are permitted provided that the following conditions are
327  met:
328 
329  Redistributions of source code must retain the above copyright
330  notice, this list of conditions and the following disclaimer.
331 
332  Redistributions in binary form must reproduce the above copyright
333  notice, this list of conditions and the following disclaimer in the
334  documentation and/or other materials provided with the distribution.
335 
336  Neither the name of the author nor the names of its contributors may
337  be used to endorse or promote products derived from this software
338  without specific prior written permission.
339 
340  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
341  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
342  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
343  FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
344  COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
345  INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
346  (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
347  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
348  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
349  STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
350  ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
351  OF THE POSSIBILITY OF SUCH DAMAGE.
Note: See TracBrowser for help on using the repository browser.