Mixu's tech blog

Continuing my series on Kohana 3 (see Auth here and Validation here), I’m tackling Kohana 3 internationalization in this post.

How Kohana 3 I18n works

Kohana 3′s i18n functionality is implemented in two files:

• /system/classes/i18n.php – Implements the i18n class, which provides all the functionality.
• /system/base.php – Implements the __() function, which is most commonly used for translating strings.

The Kohana_I18n class provides the following functions:

• I18n::lang($lang = NULL). Gets or sets the target language for translation. Call without parameters to get the current target language to which strings will be translated to.
• I18n::get($string, $lang = NULL). Returns the translation of the source string, optionally specifying what the target language should be. You should not usually use this function, however it has some uses (discussed later).
• I18n::load($lang). Loads the translation table for a given language, caches it and returns it.

The base file provides one translation function, the double underscore:

• __($string, $values, $lang = ‘en-us’). When the current target language is the same as the param $lang, returns the same string. When the current target language is something else, returns the translated string if it exists, or the same string if it does not.The $values array determines replacements made in the string.

__() is the main function you should be using. Just wrap all your strings into it.

Storing and loading the translation strings

As you notice if you look at the source, there are no translation strings here. They are stored under /application/i18n/languagename.php, where languagename is the name of the language you want to use. Kohana uses the string “en-us” as the default target language.

Note how the default language string has two parts: one specifying the language (en for English) and another for the region (us for the US).

Loading is done automatically on-demand when you call __(). The loading function I18n::load() works in a way that it can flexibly search for a file. It explodes the string on the “-” character, so the default target language “en-us” results in a search for the following files:

• /application/i18n/en.php
• /application/i18n/en/us.php

Note that the order means that you can have a single language file for English, and then override some parts of the file – for example with the target language string “en-gb”, Kohana would first load the en.php file and then the /en/gb.php file; this makes it easy to have regional variants for strings (“color” vs. “colour”).

What should I store in the i18n language files?

This is an important question, and Kohana does not make it for you. There are basically two approaches:

Option 1: Storing identifiers

Option 2: Storing the translated string itself

Which one is better? This is a matter of preference, and I strongly advise that you store the translated string itself. I absolutely HATE storing identifiers, having done that and worked with some applications that store identifiers. shadowhand (Kohana’s benevolent dictator) agrees, see this discussion and this discussion.

The problem with identifiers is that they make you remember a million different strings and hunt around for the right string to change in your file. Your translators have to remember what “about.description” meant when they translate your files, and you have to remember what is behind “about.description” when you make code changes. This is a maintenance nightmare, and if you forget to translate a string, the user will see a cryptic identifier.

The pros of identifiers are that the translation files are slightly smaller and use slightly less memory, and that you can change the translation files without changing the code. However, these are in my opionion rather meager advantages compared to the maintenance problems that are created.

Storing the string itself is better, because it allows you to see the text in your files and allows you to edit the translation in-place. This 1) allows for direct editing, and 2) makes it obvious that a new translation is needed since the English text is shown, and 3) if you forget to translate a string, the user will see something in English rather than just a identifier.

The nice thing is that if you write your application in the default target language, no I18n calls are performed when you use __()! So you don’t even pay the cost of a lookup. You do not need to create a translation file for the default language, because strings in the default language will be returned as-is by __().

TL;DR: Store the translated string itself.

How do I tell Kohana I want to translate a string?

Use __(‘This is my string’). If the string is in a language other than en-us, then use __(‘String’, null, ‘language’); you will then need to have a /application/i18n/en.php file for the English equivalent.

If you need to replace items in the string, do something like:

How do I start collecting the i18n strings?

Put a file like this as /application/languagename.php:

and start writing the English-to-your-language strings.

Now that you know how to do that – don’t do it manually! Instead take a look at my automatic I18n string collector, which extends I18n and automatically detects missing translations (for whatever language is currently set as the target), and keeps updating your translation files. It saves some serious amounts of time.

How do I allow the user to switch the language dynamically?

Use a cookie to store the user language, then load the value in /application/bootstrap.php:

The link would look something like Html::anchor(‘/controller/change_language/fi’, ‘Finnish’) to change the language.

How do I get a string in a different language than the one currently in use?

This is the use case for I18n::get($string, $lang). It will get you the string in a different language than the one currently in use. An example would be when you want the user to select the language an invitation email is sent to a friend (who may speak a different language), and you already have the invitation email translated in multiple languages.

Remember, in __($string, $values, $lang) you set the source language (what language is the string in) while in I18n::get($string, $lang) you set the target language (what language you want the string to be returned in).

Update: An important thing to note about newline characters in multi-line strings

I noticed that there is one area which causes problems with translations: newline characters.

Since lookups are character-sensitive, if your newline characters change in the translation file, you may not be able to look up the correct string!

I strongly recommend using UNIX line endings in your translation files and explicitly using “\n” when performing multi-line string lookups (e.g. __(‘”String\nNewline\nSecond newline”‘). Otherwise, there is the risk that you accidentally save with Windows file endings, causing translation lookups to fail for multi-line strings. If you need different line endings (e.g. emails), do a strtr afterwards.

Update: Remember to save as UTF-8

The default output from Kohana is UTF-8 encoded, so make sure your translation files are saved as UTF-8.

How do I make it easier for someone to translate my strings?

Updated: I’ve now released the Kohana-translate module, which is an application for collecting translations for KO3 applications. Read about & download Kohana-translate here.

Questions, comments, thanks, contributions?

Leave a comment below… or ask on the Kohana forums.

Update: I am gradually updating my Kohana 3.0 articles to Kohana 3.1.

Overview of Kohana 3 validation

Validation is done using the Validation class, or via ORM using $model->check() automatically via exceptions. It uses the messages functionality in KO3, which is a system for specifying validation messages for various forms (I will discuss this).

The steps in validating a form are:

1. Initialize the Validation class.
2. Call Validate->check() and check whether the validation succeeded.
3. If the validation failed, call errors($file) to get the error messages.
4. Display the validation errors in the correct context on the form.

The first steps are covered by existing documentation, so I will focus on the last two steps here.

Initializing the Validation class

In Kohana 3.1, the Validation class is simply initialized by setting up rules. The class has also changed from Validate (KO 3.0) to Validation (KO 3.1).

Rules. The Validation class has a nice set of basic validation rules, including rules for email addresses, credit cards, URLs and regexps.

KO 3.0 migration notes

KO 3.1 validation no longer has callbacks or filters. Callbacks have been merged into rules, while filters have been removed altogether.

Filters were used to do things like trim() the input in KO 3.0.x – you can do array_map($array, ‘trim’).

Callbacks are now specified using the rule() function, see this excellent tutorial from Lysender for the details!

Some sample code

 

 

Retrieving and customizing error messages

Validation can be done for both models (on save) and various other forms. Because of this the messages are stored in separate, reusable files. You should put your message files under /application/messages/filename.php.

I think the logic behind this has gotten a bit more complicated for KO 3.1 – in particularly with validation messages from the ORM… have to get back to you on that.

The files look like this:

Each field is a sub-array and each rule has it’s own field. Note that you can use :field to specify the field name.

To retrieve the messages, use $validate->errors($file) where $file is the filename (no extension).

The return value looks something like:

The simplest way to show these is the just print them in as a list. However, I will show one way to show these messages in context on the actual form.

Displaying validation errors in context on the form

Here is how we want to show the forms:

To accomplish this, I have created a new Appform helper which uses the Form class, but wraps its input with application-specific markup for errors.

The basic idea is that you can pass default values, form values and error messages to Appform prior to outputting the fields. It will then create the contextual markup for each of the fields. I have maintained compatibility with the Form API, with additional properties for added functionality.

Note: for Kohana 3.1, you need to move the error messages from _external to the base for Appform to work.

As a side note, I think this is exactly how the division of labor between framework and application developer should be: the framework should not impose a particular markup on the developer, but to keep application-specific code shorter, the developer should be able to create an extended version of the Form helper for the basic form layout. There are too many different preferred styles of markup, and the framework should not try to guess how you like your forms but rather provide the backend. Kohana gets this right.

Here is how a sample invocation would look like (note that the Appform class is not static, since each form has its own contextual data):

You can download my form validation helper from Bitbucket. It’s only a couple of hundred lines so it is easy to improve – a useful start for building more complex functionality for your app and also to reduce the lines of code needed for each form.

ORM and form validation (still need to update to KO 3.1)

The Kohana ORM provides support for field validation. However, it is very much oriented towards each model having one set of validations. With ORM, the function calls are:

You may want to do something different, either using a different set of rules or a different set of features! The classic example is user profile editing, where you do not want to force the user to re-enter their password – so you need to exclude the password rules from the Validation check.

There are many different suggestions regarding how you should handle this case. For example, in the unofficial docs they define two functions which each initialize a different validation object. I did this in my KO3 Auth sample implementation, and biakaveron (author/maintainer of KO3 Auth) suggested it is a “WTF” – since ORM already has the properties and methods for initializing a validation object from the given rules.

I would suggest using the following pattern: rely on the ORM $_rules property when possible, and when you need different types of validation (e.g. create, update, change subset of fields), define a checkCreate/checkUpdate/checkXYZ function on your model which loads the alternative set of $_rules and then calls the ORM check() to actually perform the validation.

Here is an example (model function, see also sample code):

This way you can still keep using the same, or almost identical code e.g. $model->values(), $model->validation->errors() and $model->checkXYZ(), while benefiting from the builtin code.

For example (in the Controller, see also sample code):

A note about ORM: $_ignored_columns

This field is used to specify fields which will not be saved to the database, but which may be written and accessed in while the model class exists. If you use validation through ORM, then you should specify any additional fields which are part of the model validation here (e.g. fields which will be combined during the actual save, but are transmitted separately).

This is the 3rd part of my mini-series on Kohana 3 auth. I figure the best way to show the Kohana Auth module works is to provide a sample application module which uses auth. In this series of posts, I discuss:

1. Setting up the basic Auth in KO3 (part 1)
2. An overview of the functionality provided by the Auth module (part 2)
3. Tips on implementing Auth in a custom application (part 3; this part)
4. Getting started with Useradmin, my Kohana 3 auth admin module (part 4)

The new release is AWESOME , and now is based on the Kohana 3.1.x API. (You can get the old version for 3.0.x via Github.)

Here are a few screenshots:

New UI, more providers supported! Get it from:

• Github: https://github.com/mixu/useradmin
  (git clone git://github.com/mixu/useradmin.git)

This is a custom module, and the code is in /modules/user. The code contains way more comments than usual. I’ve tried to make things clear, and keep the UI basic but pretty enough.

Further, the code is released under the BSD licence, so you can use it in commercial applications. While I don’t restrict you in using the code, it is good karma to send improvements to this module back: in particular, I would like to have the features in the roadmap below.

What functionality is there:

• Use the base Ko3 auth module and do not reinvent the wheel — OK
• Support translation; no strings without __() — OK
• Boilerplate for administration (create user, edit user, delete user, list users and last logins) — OK; roles have to be added via DB but that’s OK
• Boilerplate for users (login, logout, view profile, edit profile, unregister, access denied page; error handling/validation) — OK
• reCAPTCHA optionally supported for registration — OK; implemented for registration, thank you jnbdz!
• Optional support for 3rd party login:
  • Facebook — OK
  • Google — OK
  • Yahoo — OK
  • Twitter — OK (with the caveat that Twitter does not allow you to get the email address via api)
• Login  using username, email or both — OK
• Optional lost password reset via email — OK; workflow is a bit clunky (user has to type the new password one time)
• Optional autologin support via cookies — OK but not documented here; read about it Ko3 docs. Need to add checkbox to UI.
• Optional enforcement of number of failed login attempts — is there but configuration/activation is clunky.
• UI design that doesn’t make your eyes bleed — OK
• Kohana 3.1.x support – OK, thank you gartz!

What needs to be done – contribute this!

• reCAPTCHA optionally supported for too many failed logins
• Optional account activation via confirmation email + expire unactivated account/welcome message via email
• Optional password strength indicator for client
• Separation between module and sample app on Github — e.g. migrate development to Github only

Nice-to-have (perhaps release as compatible modules?)

• Profile picture loading from Facebook, Twitter or uploaded image — not yet.

So if you implement any of those, and don’t it to be too much of a burden to publish them under the BSD licence, let me know via comments or via email (firstname.surname@gmail.com).

Changelog is in the readme file on Github/Bitbucket.

Useful code snippets

Here are some code snippets which show you the basics of Auth:

Create a new user (e.g. if you have not set up any users yet and want to do that programmatically)

Check whether current user has a role

Check whether some other user has a role

Add a role to a user

Remove a role from a user

Retrieve all users that have a particular role

Documentation on the interesting bits

The template view

The template view in /views/default/template.php uses the Thematic-inspired markup which I discuss in this post. You can do a lot by adding a few more divs into the header div. I have tried to follow my own advice about CSS, but the included CSS file is pretty basic. It includes the Yahoo CSS reset.

Controller_App: base controller class

This is the “base” controller I use for all the classes. It handles template autoloading and auth checks. You should inherit all your controllers from Controller_App to benefit from its features.

There is also one additional neat thing: in case of session expiry, Kohana may show an error in session loading. There is an additional check for this in Controller_App, which discards the session in case of loading errors.

Template autoloading loads the file from /views/default/template.php, so that you just have to set $this->template->content = $view->render() in your controller.

Auth checks are configured through two additional properties in the controller:

/**
 * Controls access for the whole controller, if not set to
FALSE we will only allow user roles specified.
 *
 * See Controller_App for how this implemented.
 *
 * Can be set to a string or an array, for example array('login', 'admin') or 'login'
 */
public $auth_required = FALSE;

/** Controls access for separate actions
 *
 *  See Controller_App for how this implemented.
 *
 *  Examples:
 * 'adminpanel' => 'admin' will only allow users with
the role admin to access action_adminpanel
 * 'moderatorpanel' => array('login', 'moderator') will
only allow users with the roles login and moderator to access action_moderatorpanel
 */
public $secure_actions = array();

Controller_User: Controller for user self-management

Particularly interesting code is in the register view, which shows how you can load validation error messages from a file, and how you might alter classes to show errors.

You might want to improve that code, I haven’t had the time to create a proper helper for this.

• action_index shows how to check for a particular role using auth
• action_noaccess is the generic “access denied” view.
• action_profile and action_profile_edit show how to load data, and how to validate data on save. See application/modules/user.php for details on how the validation is done.
• action_register shows how to create a new user.
• action_unregister shows how to delete the current user, and log them off programmatically.
• action_forgot is a basic example of how forgotten passwords might be recovered. I haven’t implemented the email sending functionality for now, so if you do it, please contribute here.
• action_reset resets the user password if the user knows the 32-letter recovery key which should be emailed to them.

Controller_Admin_User: Controller for administering users

You need the admin and login privileges to access this controller.

There is a sample implementation of how to use Kohana 3 pagination in the index view. This could be improved e.g. by a helper that would take into account the sort state.

• action_index is a sample of how pagination can be used.
• action_edit is the administrative edit and add for new users. It shows how to add roles to users and also how to combine editing and adding new users into one action
• action_delete is the administrative delete function for deleting users.

Model_User: Model for users

This code is mostly based on Model_Auth_User (in /modules/auth/classes/model/auth/user.php). However, I extended the login() function to support failing after too many attempts over a 5-minute period.

The “too many recent failed logins” functionality is commented out, since it will need a few additional fields.

Also, you might want to add the “created” and “modified” columns, there is a comment showing how this can be done using ORM functionality.

Questions, comments, thanks, contributions?

Leave a comment below… or ask on the Kohana forums.

The Kohana 3 Auth model implements a set of basic functionality for authentication (login) and authorization (user rights based on roles). In this series of posts, I will discuss:

1. Setting up the basic Auth in KO3 (part 1)
2. An overview of the functionality provided by the Auth module (part 2; this part)
3. Tips on implementing Auth in a custom application (part 3)

In this second post in the series I will discuss the different functionality provided by the Auth module.

An overview of the files in the KO3 Auth module:

• modules/auth/classes/kohana/auth/file.php – contains the file driver for Auth.
• modules/auth/classes/kohana/auth/orm.php – contains the ORM driver for Auth.
• modules/auth/classes/kohana/auth.php – implements the Auth singleton.
• modules/auth/classes/model/auth/user.php – User model (Model_Auth_User).
• modules/auth/classes/model/auth/role.php – Role model (Model_Auth_Role).
• modules/auth/classes/model/auth/user/token.php – User_token model (Model_Auth_User_Token).
• modules/auth/config/auth.php – Auth module configuration.

Different drivers: File and ORM

There are two different drivers for Auth. The ORM driver is most commonly used, and it stores and retrieves data in the database. The File driver is the alternative, which stores credentials in the auth module config file.

ORM functionality vs. Auth functionality

It is important to note that the Auth class models (user, role, token) extend the ORM base class. This means that all the ORM methods can be used e.g. to retrieve users using different conditions in the same way as other ORM models would.

Have a look at the ORM documentation for specifics.  The Auth functionality for the ORM driver is built upon the ORM, and it uses the existing functionality such as relationships ($_has_many), validation rules ($_rules, $_ignored_columns), callbacks ($_callbacks).

Much of what you will be doing with the users model will be adding extended information such as new relationships and new fields. As long as you keep the default relationships, rules and field names, you will be able to flexibly extend the user model to suit your needs.

The Auth singleton

Auth class (/modules/auth/classes/kohana/auth/orm.php):

• Auth::instance(). Retrieves the instance of the Auth singleton.
• logged_in($role = NULL). If called without arguments, checks whether the user is logged in. If an argument is given, then checks whether the logged in  user has the given role (string/Role object) or roles (array of strings / Role objects).
• get_user(). Retrieves the user model for the user who is currently logged in (or FALSE).
• login(array & $array, $redirect = FALSE). The array should contain the password and username. If the redirect parameter is set, then the login will be redirected to the given URL.
• logout(). Logs the current user out.
• force_login($username). Forces the username given as the first argument to log in – without a password.
• auto_login(). Logs in the user automatically.

The User model

Model_Auth_User defines two relationships:

// Relationships
protected $_has_many = array
(
	'user_tokens' => array('model' => 'user_token'),
	'roles'       => array('model' => 'role', 'through' => 'roles_users'),
);

These relationships connect the user to Roles and tokens. Roles are particular sets of rights, such as “admin”. Tokens are used for two purposes: automatic login using cookies and password reset using password reset tokens. A user can have multiple tokens.

Model_Auth_User defines additional rules and callbacks for the username, email and password fields.

	// Rules
	protected $_rules = array
	(
		'username'			=> array
		(
			'not_empty'		=> NULL,
			'min_length'		=> array(4),
			'max_length'		=> array(32),
			'regex'			=> array('/^[-\pL\pN_.]++$/uD'),
		),
		'password'			=> array
		(
			'not_empty'		=> NULL,
			'min_length'		=> array(5),
			'max_length'		=> array(42),
		),
		'password_confirm'	=> array
		(
			'matches'		=> array('password'),
		),
		'email'				=> array
		(
			'not_empty'		=> NULL,
			'min_length'		=> array(4),
			'max_length'		=> array(127),
			'validate::email'	=> NULL,
		),
	);

It also provides a number of functions:

• login(). First argument is an array containing username and password, the second argument is an optional URL to redirect to.
• change_password(). Allows the password to be changed. First argument is an array containing password and password_confirm, second is an optional URL which if given causes the value to be saved immediately and redirected to the given URL.
• username_available(). A callback used in the validation process which checks that the username is available when creating a user.
• email_available(). A callback used in the validation process which checks that the email address is unique.
• save(). A save() function that causes the password to be hashed before saving if it was changed.

The Role model

Roles have a name and a description (a line of text). They are associated with users through the roles_users table. The model does not implement any functions, only a number of rules stored in $_rules.

The User_token model

User_tokens have a unique token and an expiry date. The token can be generated by using create_token() and housekeeping (removing the expired tokens) can be performed using delete_expired(). When the token is saved, the user agent of the current request is saved.

Tokens are essentially unique authorizations either related to password resets or to auto login.