Thursday, 4 July 2013

Setting up a simple vibe.d application: app itself

After few questions in D and vibe.d newsgroups about what functionality people want article about, I have decided to implement this application:
  1. It provides a form to user, validates input and saves values entered last time
  2. Tracks user - different sessions get different values
  3. Shows table with all entered values that were successfully validated
It is simple in its functionality but covers some of topics beginner is likely to encounter doing web development. Impatient ones can simply go to https://github.com/Dicebot/examples/tree/master/vibe.d/habr-basic for complete working example, but it intentionally has not comments ;)

Experienced web developers won't be interested. I have warned.

Layout & minimal app

nginx configuration file from previous article defines simple layout - everything for domain "example.com" sits below /home/http/example.com , request of form example.com/static/* go to /home/http/example.com/static and everything else is proxied to vibe.d application listening on port 8080.

Location of files outside of static sub-folder does not really matter as reverse proxy does not interact with them directly, but by convention all Diet templates are stored in "views" sub-folder. Basic layout will look like this:
example.com
    app.d
    static
        main.css
    views
        main.dt

I will intentionally keep main.css  empty because I suck at web design - feel free to put there any styling you may find suitable for this example ;) vibe.d application is compiled from app.d source file, which looks like this:
import vibe.appmain;
import vibe.http.server;

// Entry point for request handling. Obviously, req contains
// all data about incoming request and res needs to be filled
// by your code in an appropriate way.
void handler(HTTPServerRequest req, HTTPServerResponse res)
{
    // renderCompat() is used instead of plain render() because of DMD bug

    // This single line searches for "main.dt" file in places defined by
    // -J compiler flag and renders HTML text, completely during the
    // compilation time.
    res.renderCompat!("main.dt")();
}

shared static this()
{
    auto settings = new HTTPServerSettings;
    settings.port = 8080; // matches the value in nginx.conf

    // Currently there is only a single valid route with a single handler.
    // You can add any multiple handlers to URLRouter even for the same
    // route, those will get matches one-by-one until one will finally
    // fill the response data.
    auto router = new URLRouter();
    router.any("/", &handler);

    // It looks like a blocking call but in fact this simply
    // registers event to start HTTP server which will be run
    // once main loop gets started. Vibe.d will do it on its own.
    listenHTTP(settings, router);
}
And matching "main.dt" may look similar to this:
!!! 5
html
    head
        title vibe.d example
        link(rel= 'stylesheet', type='text/css', href='static/main.css')
    body
        div
            h1 Hello!
Vibe.d diet templates are mostly inspired by Jade with an exception that D is used instead of JavaScript.

Anyway, that's the start. That should be enough to provide a web application that answers with a pre-rendered HTML page on requests to "/" and with status code 404 to anything else. Of course, provided code is somewhat excessive - there are overloads with shorter syntax in vibe.d for such trivial tasks. However, my goal was to show the tool set that can be used in real application, thus the extra verbosity.

Sessions

Most interactive web pages want to store at least some portion of data about its visitors. Stateless nature of HTTP protocol is usually dealt with using session cookies and vibe.d, of course provides some tools for this out of the box.

Let's upgrade app.d a bit:
// ...
void handler(HTTPServerRequest req, HTTPServerResponse res)
{
    if (req.session is null)
        req.session = res.startSession();

// ...
settings.sessionStore = new MemorySessionStore();
That's all. After session object is created it will act as a data store similar to associative array, tied to unique visitor. That is easy to verify - add logInfo(req.session.id) to the request handler and try visiting your page from two different browsers to emulate different identities.

Form with saved input

Adding form to diet template mentioned earlier should be trivial for anyone familiar with HTML. But polite course of action will be to also to remember last data provided by user in case either validation fails or he navigates away by an accident.

Retrieving POST data from request is straightforward:
auto name = req.form["name"];
Next snippet retrieves old input stored in session, updates it with new data and stores result back into session:
auto name = req.session["name"];
auto new_name = req.form["name"];
if (new_name)
    name = new_name;
// ..do stuff
req.session["name"] = name;
Now it time to update diet template. Obviously, form fields can't be known during compilation an need to be provided to template rendered during run-time:
res.renderCompat!("main.dt", string, "name")(name);
// will become res.render!("main.dt", name)() once bug is fixed
Variables used in rendering can be referenced within template using #{} syntax:
div
    h1 New entry
        form(method="POST")
            p#notification #{notification}
            label
                | Name
                input(name="name" value="#{name}")
            input(type="submit")
That should be enough to make last entered form data persistent - and separate for each user.

Making use of D introspection

Copy-pasting same stuff over and over again sucks. Currently variable name needs to be mentioned to get it from session and form, to store back, when passing to renderer. Add few more values, nice aggregate struct to store them all and things will quickly get dirty.

Language with strong static introspection tools (like D) must be able to figure out most of boilerplate on its own! And it can.

Lets define form content by a simple structure, Entry. Final goal is to provide utility functions that will be able to fill structures fields using form data or session data and store them back. In other words, some limited form of serialization. D does make it easy, but not that obvious:
struct Entry
{
    string name, surname;
    string phone;

    void loadFrom(T)(T source)
    {
        foreach(i, ref elem; this.tupleof)
        {
            enum id = __traits(identifier, typeof(this).tupleof[i]);
            foreach (key, value; source)
            {
                if (key == id)
                    elem = value;
            }
        }
    }

    static Entry loadFrom(T)(T source)
    {
        Entry result;
        result.loadFrom(source);
        return result;
    }
}

// ...

auto entry = Entry.loadFrom(req.session);
Anyone not yet familiar with D meta-programming power may ask "What the hell it is?". Such initial impression may be expected, but one all tools are explained and become familiar, it becomes an incredibly straightforward solution. Hardest part here is that I know no other language that has such capabilities, not at compile time at least.

.tupleof provides a view to aggregate elements in a form of tuple - special sequential that can be manipulated during compile time. Such container can be iterated with foreach which gets completely unrolled in this special case, acting as an automated copy-paste script. i here contains field index in declaration order, elem - struct field itself. Fields are iterated using reference access because I want to write data to them.

While this.tupleof results in a tuple of actual struct instance members, typeof(this).tupleof results in a tuple of struct type members - names and types of fields. As they are also in declaration order i can be used to index a type info that matches current field. And __traits(identifier, symbol) simply provides string name associated with a symbol.

source here is instance of any type that supports loop iteration over key-value pairs. Both form and session do. In proper library code that should be enforced by strict template constraints but I am omitting that right now for the sake of simplicity.

Second static overload serves simply as a syntax sugar for creating a new struct instance from storage.

What result is achieved here? You have a function that can load struct string fields from any key-value container with a iteration interface (range or opApply) using field names as keys. You can change names, add new fields - no additional maintenance is needed, D can take care of boilerplate on its own.

Complete example also has similar code for saving struct content to a container. I may hint that similar techniques can be used to generate form code in Diet template but won't show final code for this. Instead, I really recommend to try implement it on your own after reading this article - it is a perfect exercise that integrates various different topics mentioned.

Saving data and rendering table

No fun in sending form requests if they produce no visible result. Let's combine stuff created so far and add some embedded D code into Diet templates on top.
Similar to Jade templates, you can combine embedded D code with HTML entities to generate parts of page wuth a similar structure in a loop:
div
    h1 Old entries
        table
            tr
                td Name
                td Surname
                td Phone
            - foreach( old_entry; entries )
                tr
                    td #{old_entry.name}
                    td #{old_entry.surname}
                    td #{old_entry.phone}
Of course, for entries array to be used it needs to be provided to renderer:
Entry[] entries;
// ...
private template pair(alias elem)
{
    alias pair = TypeTuple!(typeof(elem), __traits(identifier, elem));
}
// ...
res.renderCompat!(
    "main.dt",
     Entry[], "entries",
     Entry, "entry"
)(entries, entry);
As this example uses only one worker thread, no locking is needed when entries are modified. Using vibe.d in high-load scenarios with multiple worker threads and/or multiple nodes is a distinctive complex topic to cover. Probably in some other article later.

Validation + custom error page

Last step toward final goal is so trivial that I won't even copy the relevant code parts - those can be found in example. I am not aware of any special vibe.d utilities for this so it all comes up to a simple method that checks struct values and provides an error message if those are invalid. And, of course, passing error message string to renderer.

Issue with real error pages (404 Not Found for example) is worth some more attention though. Vibe.d sets proper status codes by default but it also outputs exception trace as a response body which is rarely the behavior you want in production. However, you can easily provide your own error page:
void error(HTTPServerRequest req, HTTPServerResponse res, HTTPServerErrorInfo error)
{
    res.renderCompat!("error.dt")();
}
// ...
settings.errorPageHandler = toDelegate(&error);
Alternative approach is to override error pages using nginx config. Either way you almost certainly don't want to expose your application traces to outer world.

Thanks for your time!

Any questions, improvement proposals or request to cover some specific topics are welcome at public@dicebot.lv

No comments:

Post a Comment