In this post, I take a look at the different patterns that you need to know when using Node.js. These came from my own coding and from a look at the code behind Tim Caswell's flow control libraries. I think it is necessary to know how these basic patterns are implemented even if you use a library..

1. Objects and classes

1.1 Class pattern

// Constructor
var Class = function(value1, value2) {
  this.value1 = value1;
}
// properties and methods
Class.prototype = {
  value1: "default_value",
  method: function(argument) {
    this.value2 = argument + 100;
  }
};
// node.js module export
module.exports = Class;

// constructor call

var object = new Class("Hello", "2");

If the class is long, then instead of doing a single Class.prototype = {...} assignment, it may be split into multiple Class.prototype.method = function () {..} assignments.

Reminder: Assign all your properties some value in your constructor. Otherwise while the resulting object can access the property defined in the prototype, the prototype value is shared among all instances. So in order for your "instance" to actually own it's own copies, you have to explicitly initialize the variables in the constructor, or they will act like static variables in non-prototype-based OOP. It's a stupid mistake, don't make it.

1.2 Accessing global values from objects

// constructor
var Class = function(global, value2) {
  this.global = global;
}
// access using this.global in class methods

1.3 Factory pattern

// Constructor
var Class = function(value1, value2) { ... }
// Factory
Class.factory(value1) { return new Class(value1, "aaa"); }
// properties and methods
Class.prototype = { ... };

1.4 Sharing state between modules

var Common = {
  util: require('util'),
  fs:   require('fs'),
  path: require('path')
};

module.exports = Common;

// in other modules
var Common = require('./common.js');

1.5 Singleton class (added Feb 2011)

var Singleton = (function() {
   var private_variable = 'value';
   function private_function() {
      ...
   }
   function public_function() {
      ...
   }
  return {
      public_function: public_function
  };
})();

2. Parsing requests

2.1 Parsing GET

// parse URL
var url_parts = url.parse(req.url);
// parse query
var raw = querystring.parse(url_parts.query);
// some juggling e.g. for data from jQuery ajax() calls.
var data = raw ? raw : {};
data = raw.data ? JSON.parse(raw.data) : data;

2.2 Parsing POST

if (req.method == 'POST') {
   var fullBody = '';
   req.on('data', function(chunk) {
   // append the current chunk of data to the fullBody variable
   fullBody += chunk.toString();
   });
   req.on('end', function() {
      // parse the received body data
      var decodedBody = querystring.parse(fullBody);
      console.log(decodedBody);
   }
}

3. Concurrency

3.1 Waiting for async stuff to complete before continuing

E.g. when you need to have all the results from the database before you do something.

var wait = function(callbacks, done) {
   var counter = callbacks.length;
   var next = function() {
      if(--counter == 0) {
         done();
      }
   };
   for(var i = 0; i < callbacks.length; i++) {
      callbacksi;
   }
}

Example usage (if you prefer, imagine that these are three database calls and that you are storing the results in some higher-scope variable in each of them and then using that result in function d):

var a = function (next) {
   setTimeout( function() {
      console.log("Done A");
      next();
   }, 3000);
  };

var b = function (next) {
   setTimeout( function() {
      console.log("Done B");
      next();
   }, 2000);
  };

var c = function (next) {
   setTimeout( function() {
      console.log("Done C");
      next();
   }, 1000);
  };

var d = function () {
   console.log("All done!");
  };

wait([a, b, c], d );

Similar libraries include: Tim Caswell's Step and Will Conant's Flow.exec(). This code is simpler so it doesn't use this to pass the function next(); but rather passes it explicitly. Also it needs an array, instead of accepting an arbitrary number of function arguments. The library functions do better error handling and have more features, so you might want to use them / look at them to improve the code.

3.2 Limiting concurrency

E.g. reading a gazillion files but just running 30 reads at a time not to exhaust the available file handles. You have a list of operations to do, you want to do them all but can't start/don't want to have more than max_concurrency number of the operations running simultaneously.

I call this the Pile, but there probably is a better name for it. Put your stuff in the pile, and then run it all, finally call done() when everything is done. Main difference with simple completion counters like Wait() above is that this code limits concurrent execution, which is necessary in some cases (e.g. reading files).

var Pile = function() {
   this.pile = [];
   this.concurrency = 0;
   this.done = null;
   this.max_concurrency = 10;
}
Pile.prototype = {
  add: function(callback) {
   this.pile.push(callback);
  },
  run: function(done, max_concurrency) {
      this.done = done || this.done;
      this.max_concurrency = max_concurrency || this.max_concurrency;
      var target = this.pile.length;
      var that = this;
      var next = function() {
         that.concurrency--;
         (--target == 0 ? that.done() : that.run());
      };
      while(this.concurrency < this.max_concurrency && this.pile.length > 0) {
         this.concurrency++;
         var callback = this.pile.shift();
         callback(next);
      }
   }
};

Example usage (add 20 functions, then run em at concurrency of 5 at a time). Again, imagine that setTimeout an async I/O call.

Note: you have to call next() when you're done.

pilex = new Pile();

var counter = 0;

for(var i = 0; i < 20; i++) {
   pilex.add( function test(next) {
      var now = new Date().getTime();
      setTimeout( function() {
         counter++;
         console.log(counter +" Hello world");
         next();
      }, 5000);
     }
   );
}
pilex.run(function() {console.log("Done "+counter);}, 5);

3.3 Pooling and reusing expensive, persistent resources

3.4 Running arbitrary workflows when dependencies are matched

4. More good basic node.js patterns/snippets?

Comments

windyrobin: very good,it's really very helpful!

John Hurliman: Thanks a bunch for this handy reference. By the way, the technical term for your Pile concept is a semaphore.

zenek: hi, I try to do what u remind in point 1.1, but i cant function a() { } a.prototype = { z :1 }

var a1Obj = new a(); var a2Obj = new a(); a2.z = 2; console.log(a1.z) // 1 console.log(a2.z) // 2

this is not like static. Please help me :D

Mikito Takada: Its "like static" - not exactly the same.

If you assign a value to obj.something, and have a value in obj.prototype.something, then the value in obj.something will take precedence.

So this works: function A() {} A.prototype = { z: 1 }; var aa = new A(); var ab = new A(); console.log(aa.z); // 1 console.log(ab.z); // 1 A.prototype.z = 100; // EXPLICITLY assign to prototype property console.log(aa.z); // 100 console.log(ab.z); // 100

but if you do: aa.z = 123; console.log(aa.z); // 123 (from object property z) console.log(ab.z); // 100 (from object prototype property z)

For a lot more details, see http://dmitrysoshnikov.com/ecmascript/chapter-7-2-oop-ecmascript-implementation/

ik: Thanks, really useful for those studying node.

Rob Baltzer: This stuff is great. I think there is a type on 2.2. Should be:

if (req.method == 'POST') { var fullBody = ''; req.on('data', function(chunk) { // append the current chunk of data to the fullBody variable fullBody += chunk.toString(); }); req.on('end', function() { // parse the received body data var decodedBody = querystring.parse(fullBody); console.log(decodedBody); }); }

zaphod1984: great resource, thanks a lot!

trantor: Should there be a 'var' in front of Common = {...} in 1.4? Or will the Common object become a global? Thanks for yours tutorial!

Mikito Takada: Thanks for noticing - fixed it above. Also, have a look at my book for a more up to date and complete discussion of Node patterns, at http://book.mixu.net/

rich: Is your Wait example above really synchronous (or meant to be).

It seems to just run all the functions and the timeout is what is really just firing.

So you aren't getting what I'd expect which was:

A runs to completion, then calls next (ie B), B runs, then C. so the output should be:

Done A, Done B, Done C

What is happening is

Done C, Done B, Done A no matter what order is in:

wait([A,B,C], D);

same results with wait([C,A,B], D), etc,etc Thanks,

Rich

Mikito Takada: Yeah, all the tasks are started at the same time and run in parallel - and then the last function triggers.

setTimeout is the actual task you want to run. In real code, you would substitute setTimeout() with another async function call, like fs.readFile('./foo.txt', function() { next() });

I guess the function should be called done() and not next().

Have a look at http://book.mixu.net/ch7.html for the updated, longer explanation.

ekanna: Hi,

Is it necessary to use constructor functions in NodeJs?

Why can't we just write simple functions and assign them to exports object for reuse in other modules?

I believe constructor functions main purpose is code reuse. when that is achieved by assigning simple functions to exports object in NodeJS, why do we need constructor functions?

I am new to programming itself! Your book is really excellent.

Thanks ekanna

Marc: Hi,

your patterns have been very useful for me. But I've been missing a Pattern for a Module with an EventEmitter. So here I created one: https://gist.github.com/2012929

Greets Marc

Quest: Invaluable compilation of snippets. I'd been looking for a good recent object example all morning.

The only thing that's missing is a good example of Node object inheritance (older examples use sys.inherits, but it looks like it's been moved to util.inherits).

wiill: Regarding your Class pattern, and your article on Node Fundamentals (ch 9.2 EventEmitters), it's problematic to define class methods in a single prototype hash definition, as in following:

class.prototype = {...}

As it will overwrite all previous / defaults for the prototype, especially when calling:

util.inherits(class, EventEmitter) (I wrote a comment as well on this chapter)

I would then recommend to define methods using your second pattern:

class.prototype.methodName = function(args){...}

(Thanks for correcting me if anything wrong, quite new in JS so I may missing some fundamentals : )