- Posted by liammclennan on May 15, 2012
Why Do We Need Modules
Organising JavaScript into modules makes it easier to reason about programs and makes it possible to test. To test code that is not organised into modules you have to identitfy all of the code's dependencies and remove those dependencies. Use modules to make the dependencies explicit.
Modules Systems
There are three competing module systems at the moment: CommonJS, AMD and ECMAscript Harmony modules.
CommonJS Modules
CommonJS modules are the module specification used by node.js. Each module is declared in it's own file and has it's own private scope. Modules explicitly export their API by attaching to an 'exports' object.
Although CommonJS is mostly a server-side technology it can be used in the browser via stitch or browserify. They make it difficult to debug in the browser because they require that all modules be combined into a single script.
AMD
AMD modules use a function wrapper instead of files like CommonJS. AMD modules can specify the modules that they depend on which are then provided when the module is resolved. For this reason AMD modules provide a primitive (but still very useful) implementation of dependency injection. The most popular implementation of AMD is require.js.
ECMAScript Harmony Modules
ECMAScript Harmony is the proposed next version of JavaScript. It includes a module specification.
Or Role Your Own
I couldn't bring myself to add a dependency on require.js when all I needed was a simple module system. The AMD format is very simple so I implemented my own simple version. It supports recursive resolution of module dependencies.
For more on this topic see Writing Modular Js by Addy Osmani.
- Posted by liammclennan on April 21, 2012
Today I updated my JavaScript Koans project, adding a new topic (about 'this') and updating an existing topic (about reflection).
The about reflection topic had some weird examples that needed fixing and while I was in the code I added a test about object's 'constructor' property.
The 'about this' topic explores the various ways that JavaScript's 'this' keyword can behave.
- Posted by liammclennan on April 18, 2012
The web is trending towards single page applications. It won't happen overnight - but it will happen. Until then, the fully client-side web application is not appropriate for every scenario. There is a tradeoff between the ultimate user experience and development effort. The continuum ranges from the traditional server-side web application through to the fully client-side single page application that may not require a dynamic server at all.
The Server-Side Application
It's hard to imagine an application so unimaginative and boring that it requires no client-side enhancements. I doubt if I will ever build another one of these.
The Server-Side Application With Some Client-Side Enhancement
If the functionality of your application fits neatly into the MVC pattern then you can get away with minimal client-side code. This begins to fall apart when you need to maintain client-side state (or the illusion of client-side state) between server interactions. An example of this is a tabbed interface. From the user's perspective all of the tabs are in the same context. They expect to be able to change tabs, and change back, without losing their work.
The Server-Side Application With Embedded Client-Side Applications
Sometimes, most of your application can be handled server-side but small complicated parts can be implemented as embedded client-side applications using backbone.js, spine.js, ember.js or one of the others. It's like java applets, but less shit.
The Client-Side Single Page Application
The ultimate solution is to move the application to the client, reducing the servers responsibilities to:
- serving the application files
- providing a data api
The server/client state problem is solved but the cost is some added friction coding and testing your application.
Summary
The extreme ends of the continuum both provide a clean way to build maintainable, quality applications. The hybrid approaches are more common but unfortunately they tend to be a bit of a mess. When starting a new application it is important to carefully and pragmatically consider the requirements and chose the least harmful design.
- Posted by liammclennan on April 6, 2012
Backbone.js is a JavaScript library that provides helpful types for organizing interactive JavaScript user interfaces. From their website:
Backbone.js gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface.
Backbone provides models, collections, views and routers. The relationships between the components is shown in the following diagram.
A simple demo of models and views
The Model
Models represent the application's data. The model required for this application doesn't need any special behaviour at all so we can simply extend Backbone's model type with an empty object:
var Rectangle = Backbone.Model.extend({});
Now we have a new type (Rectangle) to represent our shapes. We can instantiate a new Rectangle with whatever properties we like:
new Rectangle({
width: 100,
height: 60,
position: {
x: 300,
y: 150
},
color: '#ff0000'
});
The View
Views are used to render models. The View is where the magic happens in this example. tagName, className, events and render are special Backbone properties. tagName and className define the html element type and css class of the element that will be rendered for this view. render is the function that will be called when we want to render the view. It gives us a chance to customise the rendered element. Events is a hash that maps event handlers to functions. In this example we are saying that when the 'click' event fires on the element rendered by this view then the 'move' function should be called. Here is the complete code for the view:
Assembly
To make this work we have to place the code in a html page that has references to jQuery (or zepto), backbone.js and underscore.js. jQuery (or zepto) and underscore are required dependencies of backbone. When the document has loaded we can render some rectangles by creating views (passing them models) and calling their render method.
$('div#canvas').append(new RectangleView({model: model}).render().el);
el is a backbone property that contains the html element rendered by the view.
That is the very basics of backbone. This example did not include any backbone events (only DOM events) and did not demonstrate collections or routing. There is much more to backbone.
- Posted by liammclennan on March 26, 2012
A large portion of client-side JavaScript is dependent on running in a browser and is difficult to test without a browser. Consider the following client-side JavaScript (check the 'HTML' tab too):
It is a simple function that copies the value of a textbox into a span. To be able to test this function we must satisfy its dependencies:
- jQuery
- a browser / DOM environment
- specific markup
The Problem
When client-side script depends on specific markup it makes testing difficult. It can be done by declaring an html page, setting up the markup and runing tests with something like jasmine or selenium. This is bad because you need (at least) one html file for every script in your application. And don't forget the maintenance burden caused by duplicating all of your markup in tests. And these tests will be slow.
If yours is a primarily server-side application then accept this situation as the inevitable friction caused by an architecture that can't decide what it wants to be and continue to be a happy and productive member of society. If your user interface is rendered client-side then we can do much better.
The Solution
Have scripts render the markup on which they depend.
Let the server render an empty page and json data. Use client-side JavaScript to render the complete UI. If my simple script had been capable of rendering its markup then its dependencies are reduced to:
- jQuery
- a browser / DOM environment
Taking a dependency on jQuery is not a problem for user interface code. It's a fundamental dependency like server-side code that depends on the base class library / standard library of your environment.
A single html file, using a library like mocha, can test all of your application's user interface code if each script renders the markup that it requires. This is a natural pattern when using client-side frameworks such as backbone.js or spine.js.
Making it Better
As good as this is, there is still a small problem. To be useful tests must be run, and tests that require manually opening a browser don't get run by programmers and usually cannot be run by a continuous build server. What we need is a browser environment that we can interact with programatically - I use zombie.js. Zombie.js is a headless browser environment. Using zombie the process is:
- write scripts that generate the markup they require
- test scripts with mocha in a single html page
- use zombie to programatically open the test page and inspect the result of the tests
To make this perfectly simple I put together a node.js package called browsertest that exposes a nice api for programatically running browser dependent tests. Here is how you use it:
The 'url' argument to the browsertest function identifies the html file containing mocha tests. Zombie will load this page, wait for the tests to execute, and report the results. Because this process doesn't use an actual browser it can be used in a continuous build without any problems.
The result of running 'mocha -t 5000' (5 second timeout) is:
The End
This post is a summary of the result of work that I have been doing to address the following goals in the new single-page web application world:
- JavaScript is desperately in need of structure, discipline and testing. How do we test UI JavaScript?
- If our JavaScript needs a browser environment for testing how do we tame the process so that it can be part of a regular built-test cycle?
I believe that these goals have been achieved with the combination of: JavaScript modules that generate the markup they require, brower-based tests using mocha, and a headless browser environment like zombie.js. Browsertest makes this combination easier.
- Posted by liammclennan on March 24, 2012
Lately I've been thinking about testing JavaScript so I will write some posts about it. My primary premise is that we don't do it much, and we should.
The ruby community has a reputation for being test oriented. Testing is built-in to many popular projects and rubyists have produced their share of testing innovation.
The truth is, new rubyists quickly discover that writing code in a dynamic, interpreted language without a test safety net is batshit crazy. The tests are as much a language necessity as a sign of community maturity.
Until now the enterprise developers have avoided treating JavaScript as a real language. That's fine when you're adding two lines of jQuery to a page to focus on a form field, but not when most of your user interface is written in JavaScript. Flash and Silverlight are on the way out and rich JavaScript interfaces are becoming more popular. If you are planning on building a rich client-side application remember the following:
- JavaScript UI code deserves the same respect and care as your compiled C# code. Modularity, single responsibility principle, and OO design principles all apply exactly the same.
- Q. Do I need to test this? A. Does it need to work?
- If you say something like, "I don't know JavaScript very well, but I can get by" that is probably not ok. You wouldn't have that attitude for your server-side C#/Java/Ruby. You might benefit from some quality JavaScript training to cover the basics (OMG that was shameless ;P).
- Posted by liammclennan on September 7, 2011
When you write an NHibernate Linq query you may want to include a predicate to check that a property/field has a value, which might look like this:
person.Email != ""
NHibernate will convert this to a sql expression
Email <> ''
Because oracle evaluates empty strings as null this does not work. The query will never return any rows. The best solution I could find was to use the string length
person.Email.Length > 0
- Posted by liammclennan on August 21, 2011
I have tried to solve many problems in my life, with the solutions created falling roughly into two categories: those that I am proud of and those that I would prefer to forget. Attempting to bias my future towards the former category I spent some time examinging what those successfull solutions had in common, and devised the following heuristic that anecdotally appears to work for me. YMMV. I'm probably stupid. The fact is you may be better off to do exactly the opposite of anything I say.
Heuristic for Successful Problem Solving
Break It Down
Start by decomposing a big hairy problem into many small fuzzy problems. Refine or restart this process until a promising design emerges.
Solve a Little Problem
Pick a small fuzzy problem that provides a logical first (or next) step. Learn as much as possible about the problem domain including everything needed to solve the small fuzzy problem and then a little bit more. This means giving up the implied rule that if you're not typing you're not working. It means reading. And prototyping. And decompiling. I know that I need more knowledge and less typing when I find myself slipping into trial and error.
Record What You Learn
When you're on a role or under pressure it can be difficult but I have found it valuable to make sure that I record what I learn. For a web programmer this is often as simple as recording a significant URL for later reference. Here are some concrete examples:
- Write what you have learnt in setting concrete (that's a little joke)
- Begin each spike with a set of questions that the spike aims to answer. I use a readme file located somewhere obvious (such as /project/spikes/name_of_spike/readme). When the spike is complete append the answer to each question to the readme.
- Record notes in a personal wiki. I have a tiddlywiki in my dropbox. I have been recording notes in the same wiki since July 2005.
Do Not Compromise: Measure Twice, Cut Once
Uncle Bob never tires of reminding us that cutting corners makes us move slower. My anecdotal experience suggests that compromising on the solutions to the small fuzzy problems is a mistake. It makes me slower in the medium and long term. Take the time to get the best solution. Rewrite things. To get it right I often have to do things more than once. Many small things working perfectly are how we build big complicated things that work at all.
Management tend to work by intuition. And refusing to compromise on quality intuitively feels like gold-platting that will hurt budgets and timeframes. But it won't. If the goal is the best solution possible then it will help budgets and timeframes. It may be the only way to hit budgets and timeframes. Your boss may say that you are not being pragmatic because she doesn't realise that doing it properly IS pragmatic. The best solutions are produced when the goal is perfection. The old 'be pragmatic' get it done and out the door attitude leads to mediocraty. This is why I like to work on products, where quality is everything.
If You Are Not Winning Stop
If it's not working, for the love of atheist god, quit, walk away, give up, throw in the towel, and then burn the towel.
It's a little known fact that all of Einstein's greatest breakthroughs occured on the can.
Again, I am writing this down because I know that it is true, yet I find it hard to practive, because it is counter intuitive. It always feel like just one more hack will get me over the line. Once more, slipping into trial and error is a sure sign that things are not right. So stop. Get someone else to take a look at it, leave it until tomorrow or take a walk.
- Posted by liammclennan on August 17, 2011
Rich text editors, by which I mostly mean the WYSIWYG text editors that allow people to create formatted HTML, are a blight upon the internet. Every application I have ever seen that included a rich text editor resulted in the creation of eye-bleedingly hideous content. Take this average example of what happens when information workers get their hands on a WYSIWYG editor:
Now doesn’t that make you want to cry?
This is not an edge case. Content creators consistently give birth to these monsters and no amount of gentle (or rough) cajoling will help. It’s not their fault, they only do what comes naturally and what they should be expected to do. It’s our fault for providing tools with inherent flaws.
Why Does This Happen?
We include rich text editors in applications because they provide instant visual feedback, giving the user a feeling of power and control. But it’s like giving a light saber to a two year old. What follows is my list of reasons why rich text editors must die.
When Everything Is Highlighted, Nothing Is Highlighted
This is the most common and acute symptom or rich text editors. Authors wish to highlight an important point, so they make it bold. But of course the other things they have to say are important too. All of which leads to the truly important idea. By the time they are done, nothing stands out.
I Know It When I See It
A moderately talented information worker is capable of examining published material and expressing an intelligent opinion on the effectiveness of the design. They intuitively ‘feel’ when a design works, but this does not translate into an ability to create good design themself. When it comes to their own work their design-dar is completely useless. The majority of the people creating content do not have the skills to present that content with good design.
Shotgun Surgery
Shotgun Surgery is a software development anti-pattern described as “making a number of small changes to a number of different areas in the code, in order to effect a single, coherent change in behavior”. This situation arises in content formatting if the content creators have been able to include formatting within their content. Imagine having a library of thousands of help files and wanting to change the heading font. If the authors have individually set the heading fonts in each document then you have a big problem.
Consistency
Related to shotgun surgery is the difficulty of enforcing consistency when authors are able to format content themselves. Writing about his experience with documentation in his workplace, Jon Combe discussed:
inconsistencies between documents that make styling and branding across documents a project in itself.
Information Loss
When text is saved as HTML information is lost. Automatic operations against the content becomes difficult. Consider the common example of searching documents. Its easy to search text but searching HTML is tricky. You first need to extract the text from the markup. It would not do to return thousands of results for ‘div’.
Consider the merging of documents. It’s easy to merge text documents programmatically. Merging HTML is problematic because the merged document may be semantically incorrect or malformed. Merging Microsoft Word of PDF documents is close to impossible.
Separation of Concerns
As programmers we know that we should not mix orthogonal concerns, such as content and presentation. They should be defined separately and they should be able to vary independently.
Some Ideas That Almost Work
The democratization of content creation and publishing is one of the drivers of the current technology revolution, so this problem cannot be solved by taking away their crayons. But what if we took away just the coloured ones? Here are some strategies in use today that reduce, but don’t solve the problem.
Reduced Formatting Options
Reduce the text-editor gone mad by disabling as many of the formatting controls as you can get away with. The less power authors have, the less damage they can do.
Reduce Formatting to Styles
The logical conclusion of reducing formatting options is to prevent all formatting other than the application of CSS styles. This actually solves a number of my objections to rich text formatting (but not all).
But What Can I Do?
Step away from the black pit of despair, for their is a solution.
Create Content Using A Semantic Text Language – Such As Markdown
Markdown is an easy-to-read, easy-to-write lightweight markup language. It is extremely close to plain text, with just the most basic formatting commands included. It encourages authors to focus on content and leave presentation as a separate step.
Jon Combe described why markdown (or similar) is the best solution to the rich text editor blues:
Anyone, technical or otherwise, can work with it, we can edit a stylesheet once and keep all documents looking consistent across the product range, and use it to export to HTML and PDF (via pandoc). Sure, Word can do the above, but at the expense of ugly, bloated HTML and inconsistencies between documents that make styling and branding across documents a project in itself.
Jon publishes a markdown editor that you can use to experiment with the syntax. If you use Github, Stack Overflow or FunnelWeb then you have probably already used markdown.
This is an option that will not be popular with users. They WANT full control over the presentation of the document they are creating because:
Like most medicine it might not taste good at first, but the benefits are enormous, especially to organisations that publish large volumes of material.
I have a dream that one day we will see the end of rich text editors, Microsoft Word and their kind.
I have a dream that one day the overzealous formatters and the semantic text aficionados will be able to sit down together at the table of brotherhood.
PS. I feel a tad guilty about appropriating one of the twentieth centuries greatest speeches to make an insignificant point about content creation, storage and manipulation. To salve my conscience here is a link to the full text of Martin Luther King’s I Have A Dream speech. Read it.
- Posted by liammclennan on August 12, 2011
It's not exactly news now, but earlier this year I published a course for pluralsight called 'JavaScript Fundamentals'. The course has been doing very well so thank you to everyone who completed it. It's not free - I think you need a $30 monthly subscription to view it.