Mixu's tech blogWhen I read what I write I learn what I think

I started implementing i18n for my upcoming KO3 application, and implemented a quick patch so that I don’t need to manually find and type translation strings.

What this code does

What the code below does is it checks whether the translation string exists, if not then it saves it into the translation file with the English equivalent. This updated version of the translation string file is saved into /application/i18n/languagename.php, and the old file is saved with a new name containing the current date and time.

Hope this helps!

How to set it up

First set the language using i18n::lang(‘xy-xx’) in bootstrap.php.

Also, add the following as the last line in bootstrap.php:

// Write the updated language file, if necessary
i18n::write();

Finally, add the file /application/classes/i18n.php which overrides i18n::get():

<?php
/**
 * A patch for the Internationalization (i18n) class.
 *
 * @package    I18n
 * @author Mikito Takada
 */
class I18n extends Kohana_I18n {
   // Cache of missing strings
   protected static $_cache_missing = array();
   /**
    * Returns translation of a string. If no translation exists, the original
    * string will be returned.
    *
    * @param   string   text to translate
    * @return  string
    */
   public static function get($string)
   {
      if ( ! isset(I18n::$_cache[I18n::$lang]))
      {
         // Load the translation table
         I18n::load(I18n::$lang);
      }
      // Return the translated string if it exists
      if(isset(I18n::$_cache[I18n::$lang][$string]))
      {         
         return I18n::$_cache[I18n::$lang][$string];
      } else {
         // Translated string does not exist
         // Store the original string as missing - still makes sense to store the English string so that loading the untranslated file will work.
         I18n::$_cache_missing[I18n::$lang][$string] = $string;
         return $string;
      }
   }
 
   public static function write()
   {
      // something new must be added for anything to happen
      if(!empty(I18n::$_cache_missing)) {
         $contents = '<?php defined(\'SYSPATH\') or die(\'No direct script access.\');
/**
 * Translation file in language: '.I18n::$lang.'
 * Automatically generated from previous translation file.
 */
return '.var_export(array_merge(I18n::$_cache_missing[I18n::$lang], I18n::$_cache[I18n::$lang]), true).';';
 
         // save string to file
         $savepath = APPPATH.'/i18n/';
         $filename = I18n::$lang.'.php';
         // check that the path exists
         if(!file_exists($savepath)) {
            // if not, create directory
            mkdir($savepath, 0777, true);
         }
         // rename the old file - if the file size is different.
         if(file_exists($savepath.$filename) && ( filesize($savepath.$filename) != strlen($contents) ) ) {
            $result = rename($savepath.$filename, $savepath.I18n::$lang.'_'.date('Y_m_d_H_i_s').'.php');
            if(!$result) {
               // Rename failed! Don't write the file.
               return;
            }
         }
         // save the file
         file_put_contents($savepath.$filename, $contents);
      }
   }
}

Caveats and notes

There are two things that have to be taken into account:

• First, this is would obviously be inefficient for a production site, since actual files are being rewritten on each request that finds new translation strings.
• Why this is not a problem: My recommendation is that you shouldn’t run this code in production mode, since there is no point and it is very easy to remove the code after developement is completed.
• Second, this approach is less comprehensive than using something like the gettext tools that are available – those tools scan all of the source code, while my approach depends on run-time detection of new strings. This means that a small percentage of strings will not be found automatically (ex. rare errors that never get triggered).
• Why this is not a problem: This approach will still get the vast majority of the strings without requiring any manual hunting for strings, so I think it’ll save you quite a bit of time.

Javascript is an interesting language because it is flexible and surprisingly powerful. Once you grow up from having one static page into having a large number of pages in a dynamic web application, you need conventions on how to organize your code.

In my opinion, there are two parts to the problem:

1. the easy part, which is to apply JS programming patterns
2. the hard part, which is doing so in a consistent and maintainable manner

The easy part: Namespaces, modules and commenting conventions

Namespace and module pattern

Javascript has no explicit syntax for advanced language features such as namespaces and private variables, but these can be implemented using very simple programming patterns.

A lot has been written on the namespace and module pattern, so I will only illustrate it here.

I like to add a “placeholder” function shown below to prevent me from making errors while modifying the module (IE6 freaks out if there is a comma after the last item). This used to happen when I moved or added the methods and forgot to check whether that the commas are correct.

The example code below is adopted from (http://yuiblog.com/blog/2007/06/12/module-pattern/):

// create a namespace
YAHOO.namespace("myProject");
// Assign the return value of an anonymous function to the namespace
/** @namespace */
YAHOO.myProject.myModule = function () {
	/** @private */
	var myPrivateVar = "I can be accessed only from within YAHOO.myProject.myModule.";
 
	/** @private */
	var myPrivateMethod = function () {
		YAHOO.log("I can be accessed only from within YAHOO.myProject.myModule");
	}
 
        /** @scope YAHOO.myProject.myModule */
	return  {
		/** describe myPublicProperty here */
		myPublicProperty: "I'm accessible as YAHOO.myProject.myModule.myPublicProperty."
		/** describe myPublicMethod here */
		myPublicMethod: function () {
			YAHOO.log("I'm accessible as YAHOO.myProject.myModule.myPublicMethod.");
 
			//Within myProject, I can access "private" vars and methods:
			YAHOO.log(myPrivateVar);
			YAHOO.log(myPrivateMethod());
 
			//The native scope of myPublicMethod is myProject; we can
			//access public members using "this":
			YAHOO.log(this.myPublicProperty);
		},
		/**
		* A placeholder function to prevent errors due to not having a comma after the last function in the return statement for this module.
		*/
		placeholder: function () {}
};
 
}(); // the parens here cause the anonymous function to execute and return

Commenting conventions. Pick a commenting tool and stick with it. Comment the intent of the code and any gotchas as well as the expected arguments and purpose of methods.

The two main JavaScript commenting tools are: JsDoc-Toolkit (self-contained Java) and YuiDoc (Python).

File names and splitting code into files. Put all the hard-to-reuse code into its own namespace (ex. Framework. ModuleName. ModulePart.ApplicationName) in a separate file (application.js or custom.js). This way you keep the library clean of any application-specific stuff. It is often hard to avoid implementing some things in a one-off manner, so you might as well organize all the single-use stuff in its own file.

That is, if writing your own extensions to an existing framework, you will most likely have:

1. code that is fully reusable (specialized or pre-configured widgets, utilities etc.)
2. code that is hard or pointless to reuse (page- or application-specific messages, formatting and utilities)

The hard part: structuring code for reuse, logical naming conventions

Structuring code for reuse. Encourage reuse by splitting code into configuration, implementation and customization.

1. Configuration is anything that might need to change on each instantiation of your widget. It should be single section within a module (a private object in JSON notation). Make sure you have good and documented defaults, because this makes it easier to use the code again.
2. Implementation is the main code. It should NEVER contain hardcoded ID’s, messages or other things that will eventually need to be overridden.
3. Customization is a set of page-specific blocks of code that alter the configuration. You only override the few things that are needed for the specific functionality on the page.

Here is an example (from http://www.wait-till-i.com/2008/05/23/script-configuration/):

myProject.myModule = function(){
// CONFIGURATION
  var config = {
    CSS:{
     classes:{
       hover:'hover',
       active:'current',
       jsEnabled:'js'
     },
     ids:{
       container:'maincontainer'
     }
    },
    timeout:2000,
    userID:'chrisheilmann'
  };
 
  // IMPLEMENTATION
  function init(){ };
  // make init and config public
  return {
    init:init,
    config:config
  };
}();
 
// CUSTOMIZATION
// This makes it possible to override stuff before calling init
module.config.CSS.ids.container = 'header';
module.config.userID = 'alanwhite';
module.init();

Naming conventions. I don’t mean deciding whether or not to use camelCase, but rather how the logical organization names of variables and widgets should be done. It is reasonably simple to stop using global variables and functions to avoid name conflicts, but what is harder is to come up with a logical naming convention that allows any code to be reused anywhere.

In particular, when you create new widgets, you often need to be able to connect them to one another in some manner. Being able to identifying the type (class) and retrieve/replace data from a new widget using a standard interface takes some planning and a lot of discipline, but it makes reusing the widgets much easier.

ID and name attribute naming convention. I use “data[widgetName][recordIndex][fieldName]” for input names (because PHP will parse this into arrays automatically) and “widgetName_fieldName_recordIndex” for ID attributes (because it makes string comparisons easier).

Widget instance naming convention. I use YAHOO.ApplicationName.WidgetInstanceName for widget instances, and use WidgetInstanceName to create any related tags with IDs.

I am still looking for a good logical naming convention for widgets, but it seems this topic is not as commonly discussed. If you have any tips, please leave a comment!

Reference stuff

http://stackoverflow.com/questions/211795/are-there-any-coding-standards-for-javascript

http://ajaxian.com/archives/maintainable-javascript-videos-are-now-available

http://yuiblog.com/blog/2007/06/12/module-pattern/

http://www.wait-till-i.com/2008/05/23/script-configuration/

MVC, Model-Controller-View, is all the rage in web development these days. With regards to MVC, I think the right question is to ask not whether you should use an MVC framework but rather which framework fits the kinds of problems you are likely to encounter while developing your web application.

MVC frameworks: full stack vs. glue frameworks

There are the two basic options with MVC frameworks: glue frameworks and full stack frameworks.

1. “Glue frameworks” are collections of components and libraries which you can use to build an application. Most of the functionality is optional and often you must make decisions regarding how you will structure control flow within your application.
2. “Full stack frameworks” are integrated sets of components in which most of the components and libraries are mandatory and most of the design decisions have been made for you in advance.

How full stack frameworks sometimes get in the way: an example

Most frameworks are optimized for the 0-3 month project. All the basic functionality you need is there, but the expectation seems to be that you are creating a custom version of a blog or a CMS and that the most complex bits will be the views (custom code) and the models (posts, users, tags, comments). However, the more you’re dealing with complexity with regards to business logic, integration, regulatory validation and scaling requirements, the more likely it is that a full-stack framework will require manual optimizations and supplementary mechanisms which are not easily implementable in a full stack framework.

Here is one example of how a full stack framework can end up in more complexity than a glue framework. In CakePHP (a full stack framework), if you want to show multiple “flash” messages, you have to rather actively work around the existing mechanism, because it does not support multiple flash messages easily. Yes, you can set one message per action and you can set custom styles, but setting more than one message becomes unnecessarily complex (see this article and have a look at the comments as well) because by default, the Session flash messages overwrite each other rather than appending to the existing set of messages.

This is a very trivial example – and you can use the existing Session classes to create most of the functionality necessary to fix this. But if you do that, you will end up with two “flash” message functionalities: the default one and the one you built to make it easy to set more than one message of each type.

A glue framework might offer the Session functionality without directly supporting flash messages. This does mean that you have to write your own, but you’ll end up with a slightly simpler codebase. These kinds of small things add up if you have a larger project.

Now, the key question to me when deciding which to use is:

What kind of complexity/problems are you tackling in your application?

The optimal choice of framework depends on covering the 80% case effectively (without compromising constraints like performance).

Here are some questions to consider when picking the right framework for your project:

1. Complex views – What are the most probable uses of the system? Make sure you can cover the technical functionality needed to implement the most important client goals in a way that delights or at least does not annoy your customer. What is key here is the user experience you can deliver, not just technical sophistication.
2. Complex data structures – What data structure requirements are most likely to change? If possible, pick an architecture that makes the most likely changes relatively cheap to make. This may imply a tradeoff of some kind with other functionality.
3. Complex business logic – What are the key concepts and processes, and how are these likely to change? How easy is it to create exceptions to the rule and how can rules be verified and tested?
4. Complex integration – What are the key connecting technologies, and how well are they supported by the framework? Ideally, you can use pre-existing and tested code to integrate key areas.
5. Strict performance requirements – What is the everyday workload going to be? What mechanisms will allow you to identify bottlenecks in the system? Is there a possibility to optimize functionality by scaling horizontally or by bypassing some unnecessary functionality?
6. Scaling requirements – How can you partition your workload? How well do the core mechanisms lend to custom enhancements to fulfill scalability requirements?
7. Strict schedule requirements – What is the minimum feature set? How well does the framework support rapid implementation and testing of the minimum feature set? What are the main dependencies, and are there pre-existing modules or plugins for them?

If what you are building is not particularly complex, or the primary complexity is in building the views, then a full-stack framework is likely to be a great choice – particularly if you are have a very limited schedule. An example of this would be a brochure website, which has most of its complexity in the views.

The larger and more complex the application, the less likely you are to obtain significant productivity gains from using a framework. This is simply because having a framework can at best save you a few hundred hours of work (which should be more than what it would take for you to replicate the functionality you use). If the project is big, the time saved doing this becomes less significant compared to the time needed to build the rest of the application (e.g. the useful parts).

I use Netbeans as my primary editor. It’s a great editor with pretty much all the features I would want built in (zero-configuration Mercurial support, code completion+navigation, unit testing support and more) .

The only caveat has been the performance – I was so frustrated I installed (and ultimately rejected) the vast majority of other (PHP) editors. There is a simple fix if you have memory to spare: increase the heap size in the startup options. The Java VM default heap size seems to be way too small, causing a lot of swapping within the Java VM (while not using all the memory available from the OS perspective).

Fixing Netbeans scanning performance

Open /etc/netbeans.conf (e.g. C:\Program Files\Netbeans 6.x\etc\netbeans.conf). If you are on Windows 7, you need to start your text editor with Administrator privileges (right click and select Run as..).

Change the line: netbeans_default_options=”-J-client -J-Xverify:none ….”

by adding the following to the single line (AFTER “-J-client -J-Xverify:none”):

-J-Xmx2048m -J-XX:+UseConcMarkSweepGC -J-XX:+CMSClassUnloadingEnabled
-J-XX:+CMSPermGenSweepingEnabled

The -Xmx option changes the maximum heap size the Java virtual machine can address to 2048M from the default value (which is a “best guess” method). The other three are explained in the conf file.

DONE! Ever since I made this tweak (about two months ago) there have been no additional pauses at all. My theory is that the Netbeans scanning code was exhausting the heap, and a majority of the time “scanning” was instead spent dealing with the heap exhaustion. Now I will not say that it is reasonable for an IDE to consume 2 gigabytes of memory – but if that’s what it takes to fix performance, fine.

Virtualization is awesome in software development and testing. It uses more resources (than having separate physical machines), but significantly reduces the hassles of setting up and running different operating systems on a single machine. Being able to start a new system without rebooting, having virtual servers and being able to store snapshots of different systems has been really useful.

What is virtualization?

Three-sentence definition from Wikipedia: “Virtualization is performed on a given hardware platform by host software (a control program), which creates a simulated computer environment, a virtual machine, for its guest software. The guest software is not limited to user applications; many hosts allow the execution of complete operating systems. The guest software executes as if it were running directly on the physical hardware”.

Which basically means you can run multiple virtual computers on a single physical machine, each with their own separate operating system and access to the network.

How I use virtualization

I use virtualization for the following:

1. Development servers: Running Linux in a virtual machine to provide pre-configured web servers for testing and doing development work. The main alternative would be to run a web server on the local computer (e.g. using Uniform Server). However, the key advantage in using virtualization is that the server installation is practically identical to the deployment installation – this is important since many dependencies of web applications such as PDF and Excel generation may work slightly differently on different operating systems. By using virtualization, these differences are eliminated.

2. Testing IE6. Internet Explorer 6 is unfortunately still used in many corporate environments and still has a significant market share. It is an insecure and deeply broken browser, but unfortunately it still needs to be tested. Instead of keeping it installed on a separate Windows XP system, I keep a virtual machine around with IE6 and use it for testing purposes.

3. Trying out beta software and server software. In particular, alternative Linux setups for servers which are often easier to do using fresh virtual machines rather than on a single server. If there are any problems, I can just revert the whole system to the previous snapshot.

One interesting website related to this is Bitnami, which offers ready-made virtual machines for various open source software tools. Worth trying if you want a particular web application working with minimal hassle.

4. Low-hassle deployment of temporary/personal stuff. Also known as “poor-man’s-Xen-virtualization”. I am planning on doing a small BIND DNS client setup soon to provide hostnames for the home LAN. I will probably build and test out the configuration on the my main computer, then just copy it to an older computer.

While there is a performance loss from deploying on consumer grade hardware with a consumer-oriented virtualization solution, it also allows you to use almost any hardware to host virtual machines (more lightweight hypervisors like Xen have rather specific hardware requirements).

And I am thinking about:

5. Work desktop virtualization (looking into this). I move a lot between computers and hate having to spend time installing and customizing the different applications I use. In the near future, I am planning to move my work setup from a physical machine to a virtual machine, meaning that when I change from one physical machine to another, the only thing I need to do is copy the virtual machine and have all my programs and settings remain intact. There are two things holding me back: one, I would prefer to have my files stored outside VMs (I am a bit worried about losing them if the virtual machine gets corrupted somehow) and two, graphics cards are not supported on current virtualization solutions which is bad for graphics performance.