Cascading file loading in Node.js

One of my favorite features of Kohana 3 is it’s cascading filesystem – so I decided to implement it for Node.js. A cascading filesystem is an elegant solution to a common problem: how to provide a mechanism for loading modules and reusing code?

The following image from Kohana 3′s docs shows an example:

Benefits

The key benefits are:

  1. Consistency. All your application files, including views, controllers, models and other data such as translation messages are loaded using one, easy-to-understand mechanism.
  2. Easy reuse. Without a cascading file system, you’ll have to copy and move files around if you want to use someone else’s libraries or modules. With a cascading file system, you just place the module in your application, and enable cascading for that directory.
  3. Transparent extensibility. What if you want to override one part of a module (say, a view) but don’t want to modify your copy of the module (e.g. so that you can update without manually merging changes). A cascading filesystem allows you to selectively replace files in 3rd party code simply by providing your own version of the file.

The code

Load order and file name resolution

The load order for my implementation is:

  1. Application path Р files under ./application/ are always checked first.
  2. Module paths – set modules(['./modules/my-module']) to enable module loading. Files from modules are loaded from in the order they are added.
  3. System path – files under ./system/ are loaded if no alternative exists.

Assumptions about file and class names

Files are assumed to be lowercase. Underscores in class names are replaced by slashes (so Controller_User becomes ./application/classes/ controller/user.js).

Performance impact

Requests are cached, so that additional calls to find_file() do not cause additional stat() calls in the filesystem. This is insignificant anyway, since Node.js servers are persistent so the cascading search is only done once per server instance for each file (not once per request).

Loading 3rd party code

The loaded files do not need to be “compatible” in any way other than layout in the file system. For example, while Cfs.factory(‘some_other_lib’) loads the file from ./application/ classes/some/other/lib.js, that file does not actually need to contain a class named some_other_lib; just that it returns something via module.exports.

Methods

The methods are:

  • Cfs.modules(['./modules/path-to-module']) – set the modules directories to search.
  • Cfs.find_file(dir, file, ext) – Search each path under dir (e.g. ‘classes’, ‘views’) for file (filename) with the extension (ext, default is “.js”).
  • Cfs.factory(class_name) – Return a new instance of the given class after loading the corresponding file from the cascading file system. Note that classes should be in the classes subdirectory.
  • Cfs.load(class_name) – Return whatever require(file-which-contains-the-class) returns. Useful for extending classes, see below for an example

Example usage:

var Cfs = require('./cfs.js');
 
// test class loading:
// e.g. check ./application/classes/test.js
// ./modules/modulename/classes/test.js
// ./system/classes/test.js
var t = Cfs.factory('test');
t.run();
 
// test view loading
// e.g ./application/views/user/index.html
// ./modules/modulename/views/user/index.html
// ./system/views/user/index.html
fs.readFile(Cfs.find_file('views', 'user/index', '.html'), function (err, data) {
  if (err) throw err;
  sys.puts(data);
});

To set modules:

// set only once, before calling any other functions!
Cfs.modules([
         "./modules/testmodule/",
         "./modules/testmodule2/",
         ]);

Extending classes:

// test extending class (see code in /application/classes/controller/extend.js
// to see how extension is achieved)
// e.g. ./application/classes/controller/extend.js
// ./modules/modulename/classes/controller/extend.js
// ./system/classes/controller/extend.js
var t3 = Cfs.factory('Controller_Extend');
t3.run();
t3.run_parent();

Note that if you put cfs.js in ~/node_modules/cfs.js, you don’t need to specify the path to it… see Modules in node.js docs.

// in extend.js:
var Controller_Extend = function () {
}
// extend the class
var util = require('util'), Cfs = require('../../../../cfs.js');
util.inherits(Controller_Extend, Cfs.load('Controller_Base'));
 
Controller_Extend.prototype.run = function() {
   console.log("Controller_Extend from testmodule2.");
};
Controller_Extend.prototype.run_parent = function() {
   // run the parent function
   Controller_Extend.super_.prototype.run();
};
 
module.exports = Controller_Extend;

3 comments


  1. Calling this HMVC is highly misleading. The cascading filesystem has absolutely nothing to do with hierarchical model-view-controller. I strongly recommend that you rename your “HMVC” class to something reasonable, such as “CFS”.

    • Mikito Takada

      @shadowhand I agree and apologize if I have misled people. I had greater ambitions for the module at the time, hence the “HMVC style” reference, but I ended up doing other things. I’ve removed any references to HMVC here (well, except the diagram showing the cascading file system in Kohana but that’s CFS rather than HMVC); I’ll do github as well next time I actually work on that repo…

      Edit: for clarification if someone ends up here: the title of this post used to be “HMVC -style cascading…”

Leave a comment

You must be logged in to post a comment.