Published on

Under the Hood of Express.js: A Close Look at the Node.js Web Framework

Authors
  • avatar
    Name
    Youness Hassoune
Reading Time
Under the Hood of Express.js: A Close Look at the Node.js Web Framework

Introduction

Node.js is a popular choice for creating HTTP servers, handling incoming web requests and serving web content. In this article, we'll explore how to create an HTTP server using vanilla Node.js code, examine its limitations, and understand why many developers turn to Express.js as their preferred request-response handler.

After establishing the need for Express.js, we'll dive into its source code architecture, gaining insights into what happens behind the scenes when you create a simple GET request using Express. By the end of this article, you'll have a comprehensive understanding of both vanilla Node.js server development and the advantages Express.js brings to the table and how all this works under the hood.

Table of contents

Table of Contents

Creating a Basic HTTP Server

For this purpose we are going to import the http module provided by Node js ,this module contains all the necessary functionality for creating an HTTP server.

server.js
const http = require("http");

const host = 'localhost';
const port = 8000;

// create a new server object using createServer() function.
const server = http.createServer()

//bind our server to a network address using listen() function
//listen function accepts accepts three arguments  port,host,callback function
server.listen(port, host, () => {
    //When the server starts listening, it triggers the callback function.
    //in this case a its simple log.
    console.log(`Server is running on http://${host}:${port}`);
});
Output
Server is running on http://localhost:8000

After setting up our server and having it listen for incoming requests, let's create a GET request where a user asks for a specific post by its ID. We'll then explore how to handle the user's request and send back the requested post content

How to handle GET Request

To make our example functional, we need a sample dataset that allows us to retrieve posts by their post IDs. Additionally, we should extract and validate the user's request to ensure they are requesting a post and extract the post ID provided by the user, Here's an updated version of our code.

server.js
const http = require("http");

const host = 'localhost';
const port = 8000;

// Sample data: Posts
const posts = {
  1: 'This is the first post.',
  2: 'Another post here.',
  3: 'Yet another post.'
};

const server = http.createServer((req, res) => {
  // Check if the request method is GET and the URL matches the pattern "/post/:id"
  if (req.method === 'GET' && req.url.startsWith('/post/')) {
    // Extract the post ID from the URL
    const postId = req.url.split('/')[2];

    // Check if the requested post ID exists in our data
    if (posts[postId]) {
      // Send a response with the requested post
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end(posts[postId]);
    } else {
      // If the post ID doesn't exist, send a 404 Not Found response
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.end('Post not found');
    }
  } else {
    // If the request doesn't match our expected format, send a 400 Bad Request response
    res.writeHead(400, { 'Content-Type': 'text/plain' });
    res.end('Bad Request');
  }
});

server.listen(port, host, () => {
    console.log(`Server is running on http://${host}:${port}`);
});

now we go ahead and request in our browser http://localhost:8000/post/1

Output
This is the first post.

Now, after creating a simple GET request where users can request a post by its ID, let's achieve the same functionality using Express.js.

Express

What is Express ?

Express.js (or simply Express) is a minimal framework for Node.js, designed to simplify the process of building web applications and APIs. It provides a set of powerful features and tools that make it easier to handle HTTP requests and responses in Node.js applications.

Create an Express Server and How to handle a Get Request

express.js
const express = require('express');
const app = express();
const port = 8000;

// Sample data: Posts
const posts = {
  1: 'This is the first post.',
  2: 'Another post here.',
  3: 'Yet another post.'
};

// Define a route to handle GET requests for specific post IDs
app.get('/post/:id', (req, res) => {
  const postId = req.params.id;

  // Check if the requested post ID exists in our data
  if (posts[postId]) {
    res.send(posts[postId]);
  } else {
    res.status(404).send('Post not found');
  }
});

// Start the Express server
app.listen(port, () => {
  console.log(`Express server is listening on port ${port}`);
});

Express vs Vanilla Node js

From our comparison of the vanilla Node.js code and the Express.js code examples, we can observe the following differences:

  1. Setup:
  • Express is quicker and easier to set up compared to vanilla Node.js.
  1. Amount of Code:
  • The Express.js example requires less code compared to the vanilla Node.js example, thanks to the framework's abstractions and built-in features.
  1. Routing:
  • Express.js provides a structured and easy-to-use way to define routes using app.get(...).
  • In vanilla Node.js, you must manually set up an HTTP server and handle routing and request/response logic yourself, which can be more verbose.
  1. Error Handling:
  • Express.js simplifies error handling with built-in functions like res.status() and res.send() for consistent error responses.
  • In vanilla Node.js, error handling needs to be implemented manually, making it more error-prone and less standardized.

Express Under The Hood

Now, let's dive into the exciting part where we explore the inner workings of Express.js. We'll take a closer look at Express.js's source code structure and understand the behind-the-scenes magic that occurs when you create an Express server.

Reviewing our example, we performed three different actions:

  1. create the express server
  2. create a route for a GET Request
  3. start the express server
express.js
// 1 create the express server
const express = require('express');
const app = express();
const port = 8000;


// 2 Define a route to handle GET requests for specific post IDs
app.get('/post/:id', (req, res) => {
//   getting the post by id and send it back to the client login
});

// 3 Start the Express server
app.listen(port, () => {
  console.log(`Express server is listening on port ${port}`);
});

Create The express Server

Now, let's explore what occurs when we invoke the express() function. To answer this question, we'll dive into the source code.

Taking a look at the code source ,we see that the entry point of the application is index.js. In index.js, the createApplication() function is imported from /lib/express.js, which serves as the primary export and It is the function that gets called when we first call express() in our example.

express/lib/express.js
var EventEmitter = require('events').EventEmitter;
var mixin = require('merge-descriptors');
var proto = require('./application');
var req = require('./request');
var res = require('./response');
/**
 * Expose `createApplication()`.
 */

exports = module.exports = createApplication;

/**
 * Create an express application.
 *
 * @return {Function}
 * @api public
 */

function createApplication() {
  var app = function(req, res, next) {
    app.handle(req, res, next);
  };

  mixin(app, EventEmitter.prototype, false);
  mixin(app, proto, false);

  // expose the prototype that will get set on requests
  app.request = Object.create(req, {
    app: { configurable: true, enumerable: true, writable: true, value: app }
  })

  // expose the prototype that will get set on responses
  app.response = Object.create(res, {
    app: { configurable: true, enumerable: true, writable: true, value: app }
  })

  app.init();
  return app;
}

  • The createApplication() function returns an Express instance represented by the app object, which we utilize in our application code example.

  • The merge-descriptors library is commonly used to add methods and properties from one object (source) to another object (target) in JavaScript, typically used for inheriting or mixing functionalities. So , it's used here to add methods and properties from EventEmitter.prototype and proto to the app object in order to extend the functionality of the app object, making it more feature-rich and capable of handling various tasks.

Define a route to handle GET requests

Express.js defines route handling methods like app.get(...), app.post(...), etc., dynamically using a dynamic approach. These methods are not individually defined as you might expect but are generated using methods.forEach(...) loop that exist in the /lib/application.js. This approach avoids duplicating code for each HTTP method, making the codebase more concise and maintainable.

express/lib/application.js
/**
 * Delegate `.VERB(...)` calls to `router.VERB(...)`.
 */

// methods is an array of HTTP methods: ['get','post',...]
methods.forEach(function(method){
  // defines a method on the app object based on the current HTTP method in the loop (in our case app.get)
  app[method] = function(path){
    if (method === 'get' && arguments.length === 1) {
      // app.get(setting)
      return this.set(path);
    }

    this.lazyrouter();

    var route = this._router.route(path);
    route[method].apply(route, slice.call(arguments, 1));
    return this;
  };
});
  1. The purpose of this.lazyrouter() is to initialize the router if it hasn't been created yet.
express/lib/application.js
app.lazyrouter = function lazyrouter() {
  if (!this._router) {
    //creating new Router
    this._router = new Router({
      caseSensitive: this.enabled('case sensitive routing'),
      strict: this.enabled('strict routing')
    });

    this._router.use(query(this.get('query parser fn')));
    this._router.use(middleware.init(this));
  }
};
express/lib/router/index.js
var proto = module.exports = function(options) {
  var opts = options || {};

  function router(req, res, next) {
    router.handle(req, res, next);
  }

  setPrototypeOf(router, proto)

  router.params = {};
  router._params = [];
  router.caseSensitive = opts.caseSensitive;
  router.mergeParams = opts.mergeParams;
  router.strict = opts.strict;
  router.stack = [];
  //Router object is returned
  return router;
};
  1. After this the route() function in /lib/route/index.js will be called creating a new Route and use a Layer to wrap it and this layer is pushed to the routers stack.
express/lib/router/index.js
proto.route = function route(path) {
  var route = new Route(path);

  var layer = new Layer(path, {
    sensitive: this.caseSensitive,
    strict: this.strict,
    end: true
  }, route.dispatch.bind(route));

  layer.route = route;

  this.stack.push(layer);
  return route;
};
express/lib/router/route.js
function Route(path) {
  this.path = path;
  this.stack = [];

  debug('new %o', path)

  // route handlers for various http methods
  this.methods = {};
}

To provide further clarification, we can represent this structure graphically as follows:

structure.text
App Instance
+-------------------------------------+
|             Router                  |
|      +-----------------------------+
|      |                             |
|      |   Route Stack               |
|      |   +-----------------------+ |
|      |   |                       | |
|      |   |   Layer 1             | |
|      |   |   +-------------------+ |
|      |   |   |                   | |
|      |   |   |   Handler: Route 1 | |
|      |   |   |                   | |
|      |   |   +-------------------+ |
|      |   |                       | |
|      |   |   Layer 2             | |
|      |   |   +-------------------+ |
|      |   |   |                   | |
|      |   |   |   Handler: Route 2 | |
|      |   |   |                   | |
|      |   |   +-------------------+ |
|      |   |                       | |
|      |   |   ...                 | |
|      |   +-----------------------+ |
|      |                             |
|      |   ...                     |
|      +-----------------------------+
+-------------------------------------+

Start the Express server

After we ve established the routing configuration for the server, the next crucial step is to launch the server and make it listen for incoming network requests. In our example, this is accomplished by invoking the app.listen method. To gain a more comprehensive understanding let's explore the implementation details in the /lib/application.js.

express/lib/application.js
app.listen = function listen() {
  var server = http.createServer(this);
  return server.listen.apply(server, arguments);
};

Examining this code, it becomes apparent that app.listen serves as a wrapper that invokes the http.createServer function, which we've previously utilized in our initial example with vanilla Node.js.

What occurs when I eventually send the request?

When a new request arrives the app.handle() method is triggered.

express/lib/application.js
app.handle = function handle(req, res, callback) {
  // final handler
  var done = callback || finalhandler(req, res, {
    env: this.get('env'),
    onerror: logerror.bind(this)
  });

  //...............

  this.router.handle(req, res, done);
};

Then The router.handle() is called.

express/lib/router/index.js
proto.handle = function handle(req, res, out) {
   //.............
};

In summary, the router.handle function will iterate through all the layers within its stack, diligently searching for one that matches the request's path. Once found, it proceeds to execute the handle_request method of that layer, which in turn executes the layer's predefined handler function.

Summary

Express.js is an efficient Node.js framework, simplifying web app and API development. Understanding how Express works under the hood can significantly enhance your ability to leverage its capabilities effectively in your web development projects.