copyright (c) 2000-2001 Franz Inc
This document is a companion to the AllegroServe reference manual. Here we will take you through various examples and demonstrate how to use the facilities of the AllegroServe web server. Refer to the reference manual for more details on the functions we mention here.
AllegroServe is distributed as a single fasl file: AllegroServe.fasl. If the file is installed where require can find it then you need only type
(require :aserve)
to ensure that it's loaded. Otherwise you'll have to call the load function. In the subsequent steps we've assumed that you've loaded AllegroServe into Lisp.
AllegroServe consists of two components: a web server and an html generator. These are located in two Lisp packages: net.aserve and net.html.generator. These are long package names to type so the first thing to do is to create a package that uses these packages as well as the normal Lisp packages. Let's create a package called tutorial and make that the current package:
(defpackage :tutorial
(:use :common-lisp :excl :net.aserve :net.html.generator))
(in-package :tutorial)
Normally you would publish all the pages for your site and then start the web server. That way everyone would see a consistent view of your site. However, for this tutorial we'll start the server first so that we can immediately see the pages we're publishing.
Web servers normally listen on port 80. On Unix port 80 can only be allocated by the the superuser (called root). On Windows any user can open port 80 as long as it's not yet allocated. In order to make this tutorial work on both Unix and Windows (and not require that you run as root on Unix), we'll put our web server on port 8000.
tutorial(4): (start :port 8000) #<wserver @ #x206929aa> tutorial(5):
Now the web server is up and running. Let's assume that we're running AllegroServe on a machine named test.franz.com. If you now go to a web browser and ask for http://test.franz.com you will contact this AllegroServe server and it will respond that whatever you asked for wasn't found on the server (since we haven't published any pages). You can also try http://test and get the same result (although the response message will be slightly different). If you are running the web browser on test.franz.com as well you can ask for http://localhost and get a similar "not found" response. This demonstrates that web servers are known by many names. If you choose to take advantage of that (creating what are known as Virtual Hosts) then AllegroServe will support you . However if you want to create web pages that are served by whatever name can be used to reach the server, then AllegroServe will allow you to do that as well.
Type :proc to Lisp and look at which Lisp lightweight processes are running:
tutorial(6): :proc P Dis Sec dSec Priority State Process Name, Whostate, Arrest * 8 3 3.2 0 runnable Initial Lisp Listener * 2 0 0.0 0 waiting Connect to Emacs daemon, waiting for input * 1 0 0.0 0 inactive Run Bar Process * 1 0 0.0 0 waiting Editor Server, waiting for input * 1 0 0.0 0 waiting AllegroServe-accept-6, waiting for input * 0 0 0.0 0 inactive 1-aserve-worker * 0 0 0.0 0 inactive 2-aserve-worker * 0 0 0.0 0 inactive 3-aserve-worker * 0 0 0.0 0 inactive 4-aserve-worker * 0 0 0.0 0 inactive 5-aserve-worker tutorial(7):
We've emboldened the threads that are part of AllegroServe. The thread named aserve-accept-6 is waiting for an http request. When one arrives it passes it off to one of the aserve-worker threads and then loops back to wait for the next request. The number of worker threads is determined by the :listeners argument to the start function.
The simplest way to publish something is to publish files stored on the disk. Create a file (here we called it /tmp/sample.txt) and put some words in it, and then
tutorial(30): (publish-file :path "/foo" :file "/tmp/sample.txt") #<net.aserve::file-entity @ #x2076e0c2> tutorial(31):
If you are running on Windows then the file will have a name like c:\tmp\sample.txt When this file name is written in a Lisp string it would be "c:\\tmp\\sample.txt" due to the special nature of the backslash character.
Now if we ask a web browser for http://test.franz.com:8000/foo we'll see the contents of the file in the web browser. Since we didn't specify a content-type in the call to publish-file the content-type will be determined by the "txt" file type, which is associated with the "text/plain" content-type.
Because we didn't specify a :host argument to publish-file AllegroServe will return this page to any browser regardless of the host name used to name the machine. So AllegroServe will respond to requests for http://test.franz.com:8000/foo and http://test:8000/foo and http://localhost:8000/foo.
If we do
tutorial(30): (publish-file :path "/foo" :file "/tmp/sample.txt"
:host "test.franz.com")
#<net.aserve::file-entity @ #x2076e0c2>
tutorial(31):
Then AllegroServe will only respond to requests for http://test.franz.com:8000/foo. If we do
tutorial(30): (publish-file :path "/foo" :file "/tmp/sample.txt"
:host ("test" "test.franz.com"))
#<net.aserve::file-entity @ #x2076e0c2>
tutorial(31):
Then AllegroServe will only respond to http://test.franz.com:8000/foo and http://test:8000/foo. This type of restriction is useful if you want to create the illusion that a single machine is really a set of machines, each with its own set of web pages. Suppose that the machine test.franz.com also had the name sales.franz.com. You could publish two different ways to respond to the "/foo" url, depending on the host name specified in the request
tutorial(30): (publish-file :path "/foo" :file "/tmp/test-sample.txt"
:host "test.franz.com")
#<net.aserve::file-entity @ #x2076e0c2>
tutorial(31): (publish-file :path "/foo" :file "/tmp/sales-sample.txt"
:host "sales.franz.com")
#<net.aserve::file-entity @ #x2076e324>
Now you will get different results if you ask for http://test.franz.com:8000/foo and http://sales.franz.com:8000/foo.
The most important reason for using the AllegroServe web server is that you can compute a web page when a request comes in. This allows your program to display the most up-to-date information on the page or tailor the page to each browser. Using the publish function, a lisp function called a response function is associated with a url. When a request comes in that matches that url, the response function is run and it must generate the correct response which is sent back to the browser. The simplest response function is published here:
(publish :path "/hello"
:content-type "text/plain"
:function
#'(lambda (req ent)
(with-http-response (req ent)
(with-http-body (req ent)
(princ "Hello World!" *html-stream*)))))
Response functions take two arguments: a request object and an entity object. The request object contains all of the information about the request (such as the machine from which the request was made, and the headers passed along with the request). The request object is also used to store information about the response that is made to the request. The entity object contains the information passed to the publish function. One important item in the entity is the content-type which serves as the default content-type for the response (it can be overridden by an argument to with-http-response).
A response function must use the with-http-response and with-http-body macros and then send any additional data to the stream *html-stream*. Despite the name of the stream, the data need not always be html. The purpose of with-http-response is to allow AllegroServe to determine how it will setup streams to respond to the request. AllegroServe will also check to see if the browser already has an up-to-date copy of this page cached in which case it will not even run the code in the body of the with-http-response macro. with-http-body is responsible for sending back the response code and headers, and the body of with-http-body is where lisp code can send data which will be the body of the response.
The preceding example sends a very simple plain text string, specifying the content-type to be "text/plain". Typically you'll want to return an html page. AllegroServe has a very concise macro for creating html. Here's a response function that sends html:
(publish :path "/hello2"
:content-type "text/html"
:function
#'(lambda (req ent)
(with-http-response (req ent)
(with-http-body (req ent)
(html
(:html (:head (:title "Hello World Test"))
(:body
((:font :color "red") "Hello ")
((:font :color "blue") "World!"))))))))
While both of the preceding response functions generate their response at request time, they both send back the exact same response every time. That's not a very good demonstration of dynamic web pages. The following page shows how you can implement a simple counter for the number of accesses:
(publish :path "/hello-count"
:content-type "text/html"
:function
(let ((count 0))
#'(lambda (req ent)
(with-http-response (req ent)
(with-http-body (req ent)
(html
(:html
(:head (:title "Hello Counter"))
(:body
((:font :color (nth (random 5)
'("red" "blue"
"green" "purple"
"black")))
"Hello World had been called "
(:princ (incf count))
" times")))))))))
This page counts the number of accesses and also displays the text in a color it selects randomly.
A form displays information, has places for the user to enter information, and has one or more ways for the user to signal that he is done entering information and the the form should be processed. There may be more than one form on a web page but the forms can't be nested or overlap.
When the user clicks on the "Submit" (or equivalent) button and the form data is sent by the browser to the web server, the web server has to process that data and return a web page to display on the browser screen. This is an important situation where using Lisp to process the form data is significantly easier than the alternatives (such as the shell, or perl or some other primitive scripting language).
There are three ways that form data is sent to the web browser
The three attributes of a :form tag that determine how data is sent to the server are:
Let's examine in detail each of the methods for sending form data:
In a url like http://www.machine.com/foo/bar?name=gen&age=28 the characters after the question mark are the query-string. The query string is not used by AllegroServe to determine the entity to handle the request. When the entity begins processing the request it can ask for the request-query of the request object. request-query will return an assoc list where the car of each entry is a string (e.g. "name" in the example) and the cdr is also a string (e.g. "gen" in the example). You can ask for the request-query of any request object and if there is no query string for the request, request-query will return nil.
This is a typical entity handler that generates a form and handles the result of filling out the form:
(publish :path "/queryform"
:content-type "text/html"
:function
#'(lambda (req ent)
(let ((name (cdr (assoc "name" (request-query req)
:test #'equal))))
(with-http-response (req ent)
(with-http-body (req ent)
(if* name
then ; form was filled out, just say name
(html (:html
(:head (:title "Hi to " (:princ-safe name)))
(:body "Your name is "
(:b (:princ-safe name)))))
else ; put up the form
(html (:html
(:head (:title "Tell me your name"))
(:body
((:form :action "queryform")
"Your name is "
((:input :type "text"
:name "name"
:maxlength "20"))))))))))))
In the function above we first check to see what value is associated wtih the tag "name" in the query string. If there is a name then we know that we've been called after the form was filled out, so we process the form data, which in this case means just printing out the name. Note that we use :princ-safe to display the name in html. It's important to use :princ-safe instead of :princ in situations like this where we are printing a string that may contain characters special to html. In the :action attribute for the :form we specified "queryform" instead of "/queryform" since it's best to keep all urls relative rather than absolute in order to make it possible to access the pages through a proxy server that might prepend a string to the root of the url. We could have separated the functionality in the above example so that one entity handler put up the form and another one processed the form. For simple forms it just seems simpler to do it all with one url and one entity function.
When the data from the form is very large or it's important to hide it from view in the url, the typical method to accomplish this is to specify the "POST" method for the form. In this case the data for the form appears in the body of the request. There are two supported encodings of the form data in the body. In this section we'll describe how to handle the default encoding, called: "application/x-www-form-urlencoded". First you must call get-request-body to read and return the body of the request. Second you must call form-urlencoded-to-query to convert the encoded body into an assoc list, where every entry is a cons consisting of a string naming the value and then the string containing the value.
The following sample shows a single entity handler function that can put up a form and can process data from the form. It isn't necessary to use the same handler for putting up and processing the data from a form. In this example we create a form with a big box for entering text. We invite the user to enter text in the box and click on a button when he is finished. At that point the entity handler gets and decodes the body of the request, and finds the data from the text box. It then generates a table showing how often the characters a through z are found in the text selection.
(publish :path "/charcount"
:content-type "text/html"
:function
#'(lambda (req ent)
(let* ((body (get-request-body req))
(text (if* body
then (cdr (assoc "quotation"
(form-urlencoded-to-query body)
:test #'equal)))))
(with-http-response (req ent)
(with-http-body (req ent)
(if* text
then ; got the quotation, analyze it
(html
(:html
(:head (:title "Character Counts")
(:body
(:table
(do ((i #.(char-code #\a) (1+ i)))
((> i #.(char-code #\z)))
(html (:tr
(:td (:princ (code-char i)))
(:td (:princ
(count (code-char i)
text)))))))))))
else ; ask for quotation
(html
(:html
(:head (:title "quote character counter")
(:body
((:form :action "charcount"
:method "POST")
"Enter your favorite quote "
:br
((:textarea
:name "quotation"
:rows 30
:cols 50))
:br
((:input :type "submit"
:name "submit"
:value "count it")))))))))))))
In this example we ask for the body of the request and then the value of the field named "quotation". If that isn't found then we assume that we are being called to display the form. We could have checked the value of (request-method req) which will be :get when we should put up the form and :post when we should analyze data from the form.
The final method of sending form data is as a multipart message. This occurs when you specify a :method of "POST" and an :enctype of "multipart/form-data". The handler for this must detect when it is being called from a :post request and must call a sequence of functions to retrieve each item from the message body. First it calls get-multipart-header to get the next header (or nil if there are no more headers). The header data is an assoc list where the values have different formats as described in the AllegroServe manual. After reading the header the handler must call get-multipart-sequence to read successive chunks of data associated with this header.
An example demonstrating this is too large to include here but can be found in the AllegroServe examples.cl file (search in that file for "getfile-got")
You don't necessarily want to allow everyone to access every page you publish. We will describe common ways to check whether someone has permission to access a page. There are two ways to do the authorization checks. You can write the tests yourself in the entity function, or you can create an authorizer object and attach it to the entity. Below we'll show you how to write the code to do the checks manually. The Allegro AllegroServe manual describes the authorizer objects.
One way to control access to a page is to request that the person at the browser enter a name and password. You can create a form and have the user enter the information and then click on a button to submit it. Another way is to return a 401 (response unauthorized) code to the request made to access your page. When given that response, the web browser will pop up a window requesting a name and password, and after that's entered, the browser resends the request but includes the name and password in the header.
The method you choose for asking for the name and password may depend on how secure you want the response to be. Using a form the name and password are sent to the web server without any encoding at all (other than the simple urlencoding which only affects unusual characters). If your form uses the "GET" method then the name and password appear in the url which makes them very easy to spot, so you at least want to use the "POST" method if you use a form. If on the other hand you use the 401 response code, then the name and password are sent in a more encrypted form (using an encoding called base64) however anyone sufficiently motivated can decrypt this without a lot of trouble. AllegroServe does not yet support md5 authentication which is the most secure way to do authentication in the HTTP/1.1 protocol.
One advantage of using the 401 response to cause the user to enter a name and password is that most web browsers will continue to send the entered name and password along with future requests to the server until the web browser is restarted. Thus you can simultaneously unlock a whole group of pages. If you choose to handle the authentication in a form you may want to use a cookie to make a record that this web browser is now qualified to access a certain group of pages. Cookies aren't hard to store, but some users turn off cookie saving thus you will make your site inaccessible to these people. Another authorization mechanism is to insert hidden fields in forms with values that tell the server that this access is authorized for a certain amount of time.
This is an example of using the 401 response to do user authorization.
(publish :path "/secret"
:content-type "text/html"
:function
#'(lambda (req ent)
(multiple-value-bind (name password) (get-basic-authorization req)
(if* (and (equal name "foo") (equal password "bar"))
then (with-http-response (req ent)
(with-http-body (req ent)
(html (:head (:title "Secret page"))
(:body "You made it to the secret page"))))
else ; cause browser to ask name and password
(with-http-response (req ent :response
*response-unauthorized*)
(set-basic-authorization req "secretserver")
(with-http-body (req ent)))))))
You can determine the address from which a request was made and restrict access based on that address. If the request came through a proxy server then you are really determining the address of the proxy server. The following code only serves the 'secret' page if the request came from a browser running on the same machine as the server, and which is made over the loopback network on the machine. The loopback network is a network that exists entirely inside the operating system of the machine. The convention is that a loopback network address has 127 in the most significant byte, and that is what we test for in the following example:
(publish :path "/local-secret"
:content-type "text/html"
:function
#'(lambda (req ent)
(let ((net-address (ash (socket:remote-host
(request-socket req))
-24)))
(if* (equal net-address 127)
then (with-http-response (req ent)
(with-http-body (req ent)
(html (:head (:title "Secret page"))
(:body (:b "Congratulations. ")
"You are on the local network"))))
else
(with-http-response (req ent)
(with-http-body (req ent)
(html
(:html (:head (:title "Unauthorized"))
(:body
"You cannot access this page "
"from your location")))))))))
To see how this example works differently depending on whether the access is through the loopback network or the regular network, try accessing it via http://localhost:8000/local-secret and http://test.franz.com:8000/local-secret (where we are assuming that you are running on test-franz.com).
AllegroServe can run multiple independent web servers. Each web server listens for requests on a different port. Because each web server can appear to be serving pages for different hosts (using the virtual host facility already described in the discussion of the publish functions), it is usually not necessary to use the multiple server facility we describe here.
All of the information about a web server, including the entities it serves, are stored in a wserver object. The current wserver object is stored in the variable *wserver*. The publish functions use the value of *wserver* as the default server into which they publish entities.
We will describe this in detail when the tutorial is updated. For now read the documentation on net.aserve::debug-on in the AllegroServe manual.