This is another post in my series of posts about designing a web framework using Node. Catch the first two here and here.
A New Website
First a quick update: last night I decided Bomber needed its own place on the web, so I threw together a website for it. It is mostly based on the code for this site, so it was pretty quick. Anyway…
Announcing bomber.obtdev.com! I’m pretty proud of the icon, so make sure to check it out. Now I have a place to put documentation and more details then are appropriate for blog posts.
Today’s installment is going to be about what I am calling “actions”, by which I mean, the code that is run for each request. The last post talked about routing, which was the business of figuring out from the URL which code should be run. So, this is the second half of that equation. For comparison, Rails calls these “actions” and Django calls them “views”.
Conceptually actions are pretty simple. Given a request, it is their role to generate a
response. If you requested an HTML list of blog posts, it is the job of the action to
generate that HTML document. In an MVC framework, the action plays the role of the glue
between all the different parts of application: the database, the templates, the cookie or
session variables, etc. Here is an example of an action from Rails, generated by its
The code is pretty straightforward (even if you aren’t familiar with Rails). It loads a list of photos from the database, and then depending on the requested format it either renders the ‘index.html.erb’ template, or converts the list to an XML document.
Actions have a lot of different needs. The two most basic are dealing with the request and the response:
- An action needs to have access to the details of the request: What HTTP method was it? What headers were sent? What were the query parameters? What was the body of the request? What POST variables were sent? What cookies were sent? What session is it associated with? What session variables are set? Where is the request from? What browser is the request from? And so on.
- An action needs to be able to modify all the details of the response that it is creating. Primarily, that is setting headers, cookies, or session variables and generating the HTML.
Giving users access to the Response and the Request in existing frameworks
Django is very explicit, here’s an example of a simple view from the docs:
You are given a request, and you return a response. This is very intuitive, and powerful but can be a bit of a pain sometimes when you don’t want to be writing all that boilerplate code over and over again. I wish I just didn’t have to type quite so much. (This is a complaint I keep coming back to over and over again with Django, and it really isn’t a strike against Django. Django is this way precisely because they don’t want to pigeon hole you into writing your projects in a specific way. But sometimes I just wish they would pick a way for me)
Rails is a bit more “magical” in the sense that a lot of the code happens behind the
scenes. With Rails, an
ActionController object is iniated for each request, and
then the requested action method is called on that object. And because
optional in Rails it seems like things just come out of nowhere. Here’s an example
What is going on is
all actually methods on the
ActionController object. To return a response you call
a method like
redirect_to. I really do feel like writing applications
in Rails is like writing in a-whole-nother language. Rails has request and response
objects, but you generally don’t use them in favor of the other simpler methods.
With Rails there isn’t a very clear distinction between requests and responses like
there is in Django.
Sinatra works in a very similar way to Rails. And of the Node frameworks, the majority of them are Sinatra like in spirit.
If we already didn’t have enough choices for how to design an API for a web framework, Node then adds an additional level of complexity due to its evented nature.
What does that mean? Well, Node lives by the philosophy that a program should never just sit there and do nothing while it is waiting for input or output to finish. Here’s an example of what not to do from Ryan’s presentation at JSConf this year:
Ryan asks a simple question, “What is the software doing while it queries the database?” Well, with most libraries it is doing nothing, it is just sitting there waiting for the database to do its thing. This is inefficient. An evented library uses callbacks, so instead of waiting for the database to return, it does some other computation, and then when the database query is done, it calls the callback function. Here’s Ryan’s example:
This is pretty straight forward but what if you need to make a second database call depending on the result of the first one, and then use both those results? You’d get something like this:
And what if you need to load in a template file (which would mean reading the file from disk) and use that template to format your results?
There is an additional problem here. How do you keep track of state between all
these different functions? In the previous example it is easy because a function is
aware of all the local variables that exist at the time it is created, so the last
function has access to
result2 but this isn’t always the case.
Especially if you are trying to reuse functions in different parts of the app.
This is very quickly getting unwieldy. And thus far none of the Node frameworks have addressed this issue head on. There have been some talks on the mailing list of adding deferred objects to Node and I think many people hope that this is going to solve the problem of writing evented web frameworks. While I think deferreds would be awesome, I think the solution needs to be a little more integrated then that.
(Note, this is the API I plan to implement in Bomber, but not all of it is completely written yet. Some of it though!)
In Bomber, the solution is
Action objects, which have deferred functionality, but
are a little more aware of the problem they are trying to address. Specifically, an
Action has a list of tasks that should be run to complete the action. And just like
with deferreds, the result of a previous task is passed to the next task. This
way it is easy chain functions to be run after one another. However, (and this is
the real bread and butter of
Actions) unlike with deferreds (at least as far as
I can tell) if the return result of a previous task is itself a deferred (or a
Promise in Node-speak), the
Action will wait for that deferred to finish its task
before calling the next task. Now you can chain up all your tasks, and not worry
about the asynchronicity of the different parts.
Action objects have a couple other tricks up their sleeves.
- They will make sure that all tasks are bound to the action object itself, so you can
keep track of state between tasks by setting variables on
- Every task will be called with the request and the response for that
Actionas the first two arguments. This makes it easy to access those objects so you can get/set cookies, sessions or headers.
- If any task returns a certain kind of object (like a ‘404 error’ object – I haven’t
decided what to call it, yet) Node will stop running the tasks for that
Actionand send that as the response for this request.
- If the result of the last task is a string or an object, Node assumes you want to use
this value for the body of the request. If it is a string it assumes it is HTML and if
it is any thing other than either a string or one of the objects in the previous bullet
point, it converts it to JSON and sends that. This allows you to easily construct
simple responses, but if you want more control you can just return
nulland manage it yourself.
Here is how the Server initializes an
Then a possible
view_function could look like this (to tackle the example from the
section on evented programming):
I have one more post planned about this Bomber design stuff but it might be a little while before I get to it. (I need to figure out how I want templating to work before I can write it! But I have some ideas…)