Monthly Archives: February 2015

Video: JavaScript coding tips

By Axel Rauschmayer

The following video is a recording of the talk “JavaScript coding tips”, which I held 2014-05-16 at the Sud Web Conference in Toulouse, France.

Bonus:

  • See how I feel about comma first style (I was much meaner than I remember). I don’t personally like it, but I respect how creatively it solves a problem.
  • Hear me speak French after the talk.

Material:

Source:: 2ality

Preventing function signature mismatch in ES5 and ES6

By Axel Rauschmayer

In some cases, using a function (or method) with a callback can lead to surprising results – if the signature of the latter does not match the expectations of the former. This blog post explains this phenomenon and suggests fixes.

Function signature mismatch

Let’s look at an example [1]:

    > ['1','2','3'].map(parseInt)
    [1,NaN,NaN]

Here, map() expects the following signature:

    callback(element, index, array)

But parseInt() has the signature:

    parseInt(string, radix?)

It’s not a problem that parseInt‘s arity is less than the 3 expected by map; JavaScript does not complain if you ignore arguments. However, index and the optional radix don’t match semantically.

Whenever you are using a library function as a callback, you are taking a risk: its signature may not match semantically, it may even change later on.

Preventing mismatch

Prevention via arrow functions

In ECMAScript 6, arrow functions [2] give you the means to be explicit about the signature of a callback, without too much verbosity:

    > ['1', '2', '3'].map(x => parseInt(x))
    [1, 2, 3]

I like using arrow functions for this purpose: it’s compact and you immediately see how the code works.

Prevention via a helper function

Another option for preventing signature mismatch is to use a higher-order helper function, e.g.:

    > ['1', '2', '3'].map(passArgs(parseInt, 0))
    [1, 2, 3]

passArgs has the following signature:

    passArgs(toFunction, argIndex0, argIndex1, ...)

The indices indicate which of the input parameters toFunction receives and in which order. The following is an implementation for ECMAScript 5.

    function passArgs(toFunction /* argIndex0, argIndex1, ... */) {
        var indexArgs = arguments;
        return function () {
            var applyArgs = new Array(indexArgs.length-1);
            for(var i=0; i < applyArgs.length; i++) {
                var index = indexArgs[i+1];
                applyArgs[i] = arguments[index];
            }
            return toFunction.apply(this, applyArgs);
        };
    }

The following is an implementation for ECMAScript 6. Note how much simpler it is, due to rest parameters and arrow functions.

    function passArgs(toFunction, ...argIndices) {
        return function (...inputArgs) {
            var passedArgs = argIndices
                .map(argIndex => inputArgs[argIndex]);
            return toFunction.apply(this, passedArgs);
        };
    }

References

  1. Pitfall: Unexpected Optional Parameters” in Speaking JavaScript
  2. ECMAScript 6: arrow functions and method definitions

Source:: 2ality

Standardizing (a flavor of) Markdown

By Axel Rauschmayer

Update 2014-09-05: Jeff Atwood renames “Standard Markdown” to “Common Markdown” and apologizes to John Gruber. Details: “Standard Markdown is now Common Markdown”.

On September 3, Jeff Atwood announced a new standardization effort for John Gruber’s Markdown: Standard Markdown.

It is amazing how ubiquitous Markdown has become and it’s great that there is now a standard for it. Highlights:

  • The working group comprises representatives from: GitHub, Reddit, Stack Exchange, the open source community.
  • There is a proper specification that also describes a parsing strategy. This should help tremendously with writing a parser that can handle all content that complies with the standard. Given all the incompatible Markdown dialects in existence, that is currently a very difficult task.
  • There are reference implementations in JavaScript and C and a validation test suite.
    • You can try out the reference implementation online.

Background:

A few more thoughts:

  • The use case “publishing” is currently a bit underrepresented in the working group. Members of the Ghost blogging platform and of Leanpub would be great additions.
  • Mid-term to long-term, I’d like a more extensive standard to build on this one: It should comprise the Asciidoc features that are currently missing from Markdown (but maybe in more of a formal syntax instead of something ASCII art-ish). Rationale: better support Markdown for publishing, especially books.

Source:: 2ality

ECMAScript 6 modules: the final syntax

By Axel Rauschmayer

At the end of July 2014, TC39 [1] had another meeting, during which the last details of the ECMAScript 6 (ES6) module syntax were finalized. This blog post gives an overview of the complete ES6 module system.

Module systems for current JavaScript

JavaScript does not have built-in support for modules, but the community has created impressive work-arounds. The two most important (and unfortunately incompatible) standards are:

  • CommonJS Modules: The dominant implementation of this standard is in Node.js (Node.js modules have a few features that go beyond CommonJS). Characteristics:
    • Compact syntax
    • Designed for synchronous loading
    • Main use: server
  • Asynchronous Module Definition (AMD): The most popular implementation of this standard is RequireJS. Characteristics:
    • Slightly more complicated syntax, enabling AMD to work without eval() (or a compilation step).
    • Designed for asynchronous loading
    • Main use: browsers

The above is but a grossly simplified explanation of the current state of affairs. If you want more in-depth material, take a look at “Writing Modular JavaScript With AMD, CommonJS & ES Harmony” by Addy Osmani.

ECMAScript 6 modules

The goal for ECMAScript 6 modules was to create a format that both users of CommonJS and of AMD are happy with:

  • Similar to CommonJS, they have a compact syntax, a preference for single exports and support for cyclic dependencies.
  • Similar to AMD, they have direct support for asynchronous loading and configurable module loading.

Being built into the language allows ES6 modules to go beyond CommonJS and AMD (details are explained later):

  • Their syntax is even more compact than CommonJS’s.
  • Their structure can be statically analyzed (for static checking, optimization, etc.).
  • Their support for cyclic dependencies is better than CommonJS’s.

The ES6 module standard has two parts:

  • Declarative syntax (for importing and exporting)
  • Programmatic loader API: to configure how modules are loaded and to conditionally load modules

An overview of the ES6 module syntax

There are two kinds of exports: named exports (several per module) and default exports (one per module).

Named exports (several per module)

A module can export multiple things by prefixing their declarations with the keyword export. These exports are distinguished by their names and are called named exports.

    //------ lib.js ------
    export const sqrt = Math.sqrt;
    export function square(x) {
        return x * x;
    }
    export function diag(x, y) {
        return sqrt(square(x) + square(y));
    }
    
    //------ main.js ------
    import { square, diag } from 'lib';
    console.log(square(11)); // 121
    console.log(diag(4, 3)); // 5

There are other ways to specify named exports (which are explained later), but I find this one quite convenient: simply write your code as if there were no outside world, then label everything that you want to export with a keyword.

If you want to, you can also import the whole module and refer to its named exports via property notation:

    //------ main.js ------
    import * as lib from 'lib';
    console.log(lib.square(11)); // 121
    console.log(lib.diag(4, 3)); // 5

The same code in CommonJS syntax: For a while, I tried several clever strategies to be less redundant with my module exports in Node.js. Now I prefer the following simple but slightly verbose style that is reminiscent of the revealing module pattern:

    //------ lib.js ------
    var sqrt = Math.sqrt;
    function square(x) {
        return x * x;
    }
    function diag(x, y) {
        return sqrt(square(x) + square(y));
    }
    module.exports = {
        sqrt: sqrt,
        square: square,
        diag: diag,
    };
    
    //------ main.js ------
    var square = require('lib').square;
    var diag = require('lib').diag;
    console.log(square(11)); // 121
    console.log(diag(4, 3)); // 5

Default exports (one per module)

Modules that only export single values are very popular in the Node.js community. But they are also common in frontend development where you often have constructors/classes for models, with one model per module. An ECMAScript 6 module can pick a default export, the most important exported value. Default exports are especially easy to import.

The following ECMAScript 6 module “is” a single function:

    //------ myFunc.js ------
    export default function () { ... };
    
    //------ main1.js ------
    import myFunc from 'myFunc';
    myFunc();

An ECMAScript 6 module whose default export is a class looks as follows:

    //------ MyClass.js ------
    export default class { ... };
    
    //------ main2.js ------
    import MyClass from 'MyClass';
    let inst = new MyClass();

Note: The operand of the default export declaration is an expression, it often does not have a name. Instead, it is to be identified via its module’s name.

Having both named exports and a default export in a module

The following pattern is surprisingly common in JavaScript: A library is a single function, but additional services are provided via properties of that function. Examples include jQuery and Underscore.js. The following is a sketch of Underscore as a CommonJS module:

    //------ underscore.js ------
    var _ = function (obj) {
        ...
    };
    var each = _.each = _.forEach =
        function (obj, iterator, context) {
            ...
        };
    module.exports = _;
    
    //------ main.js ------
    var _ = require('underscore');
    var each = _.each;
    ...

With ES6 glasses, the function _ is the default export, while each and forEach are named exports. As it turns out, you can actually have named exports and a default export at the same time. As an example, the previous CommonJS module, rewritten as an ES6 module, looks like this:

    //------ underscore.js ------
    export default function (obj) {
        ...
    };
    export function each(obj, iterator, context) {
        ...
    }
    export { each as forEach };
    
    //------ main.js ------
    import _, { each } from 'underscore';
    ...

Note that the CommonJS version and the ECMAScript 6 version are only roughly similar. The latter has a flat structure, whereas the former is nested. Which style you prefer is a matter of taste, but the flat style has the advantage of being statically analyzable (why that is good is explained below). The CommonJS style seems partially motivated by the need for objects as namespaces, a need that can often be fulfilled via ES6 modules and named exports.

The default export is just another named export

The default export is actually just a named export with the special name default. That is, the following two statements are equivalent:

    import { default as foo } from 'lib';
    import foo from 'lib';

Similarly, the following two modules have the same default export:

    //------ module1.js ------
    export default 123;
    
    //------ module2.js ------
    const D = 123;
    export { D as default };
Why do we need named exports?

You may be wondering – why do we need named exports if we could simply default-export objects (like CommonJS)? The answer is that you can’t enforce a static structure via objects and lose all of the associated advantages (described in the next section).

Design goals

If you want to make sense of ECMAScript 6 modules, it helps to understand what goals influenced their design. The major ones are:

  • Default exports are favored
  • Static module structure
  • Support for both synchronous and asynchronous loading
  • Support for cyclic dependencies between modules

The following subsections explain these goals.

Default exports are favored

The module syntax suggesting that the default export “is” the module may seem a bit strange, but it makes sense if you consider that one major design goal was to make default exports as convenient as possible. Quoting David Herman:

ECMAScript 6 favors the single/default export style, and gives the sweetest syntax to importing the default. Importing named exports can and even should be slightly less concise.

Static module structure

In current JavaScript module systems, you have to execute the code in order to find out what the imports and exports are. That is the main reason why ECMAScript 6 breaks with those systems: by building the module system into the language, you can syntactically enforce a static module structure. Let’s first examine what that means and then what benefits it brings.

A module’s structure being static means that you can determine imports and exports at compile time (statically) – you only have to look at the source code, you don’t have to execute it. The following are two examples of how CommonJS modules can make that impossible. In the first example, you have to run the code to find out what it imports:

    var mylib;
    if (Math.random()) {
        mylib = require('foo');
    } else {
        mylib = require('bar');
    }

In the second example, you have to run the code to find out what it exports:

    if (Math.random()) {
        exports.baz = ...;
    }

ECMAScript 6 gives you less flexibility, it forces you to be static. As a result, you get several benefits [2], which are described next.

Benefit 1: faster lookup

If you require a library in CommonJS, you get back an object:

    var lib = require('lib');
    lib.someFunc(); // property lookup

Thus, accessing a named export via lib.someFunc means you have to do a property lookup, which is slow, because it is dynamic.

In contrast, if you import a library in ES6, you statically know its contents and can optimize accesses:

    import * as lib from 'lib';
    lib.someFunc(); // statically resolved
Benefit 2: variable checking

With a static module structure, you always statically know which variables are visible at any location inside the module:

  • Global variables: increasingly, the only completely global variables will come from the language proper. Everything else will come from modules (including functionality from the standard library and the browser). That is, you statically know all global variables.
  • Module imports: You statically know those, too.
  • Module-local variables: can be determined by statically examining the module.

This helps tremendously with checking whether a given identifier has been spelled properly. This kind of check is a popular feature of linters such as JSLint and JSHint; in ECMAScript 6, most of it can be performed by JavaScript engines.

Additionally, any access of named imports (such as lib.foo) can also be checked statically.

Benefit 3: ready for macros

Macros are still on the roadmap for JavaScript’s future. If a JavaScript engine supports macros, you can add new syntax to it via a library. Sweet.js is an experimental macro system for JavaScript. The following is an example from the Sweet.js website: a macro for classes.

    // Define the macro
    macro class {
        rule {
            $className {
                    constructor $cparams $cbody
                    $($mname $mparams $mbody) ...
            }
        } => {
            function $className $cparams $cbody
            $($className.prototype.$mname
                = function $mname $mparams $mbody; ) ...
        }
    }
    
    // Use the macro
    class Person {
        constructor(name) {
            this.name = name;
        }
        say(msg) {
            console.log(this.name + " says: " + msg);
        }
    }
    var bob = new Person("Bob");
    bob.say("Macros are sweet!");

For macros, a JavaScript engine performs a preprocessing step before compilation: If a sequence of tokens in the token stream produced by the parser matches the pattern part of the macro, it is replaced by tokens generated via the body of macro. The preprocessing step only works if you are able to statically find macro definitions. Therefore, if you want to import macros via modules then they must have a static structure.

Benefit 4: ready for types

Static type checking imposes constraints similar to macros: it can only be done if type definitions can be found statically. Again, types can only be imported from modules if they have a static structure.

Types are appealing because they enable statically typed fast dialects of JavaScript in which performance-critical code can be written. One such dialect is Low-Level JavaScript (LLJS). It currently compiles to asm.js.

Benefit 5: supporting other languages

If you want to support compiling languages with macros and static types to JavaScript then JavaScript’s modules should have a static structure, for the reasons mentioned in the previous two sections.

Support for both synchronous and asynchronous loading

ECMAScript 6 modules must work independently of whether the engine loads modules synchronously (e.g. on servers) or asynchronously (e.g. in browsers). Its syntax is well suited for synchronous loading, asynchronous loading is enabled by its static structure: Because you can statically determine all imports, you can load them before evaluating the body of the module (in a manner reminiscent of AMD modules).

Support for cyclic dependencies between modules

Two modules A and B are cyclically dependent on each other if both A (possibly indirectly/transitively) imports B and B imports A. If possible, cyclic dependencies should be avoided, they lead to A and B being tightly coupled – they can only be used and evolved together.

Why support cyclic dependencies?

Cyclic dependencies are not inherently evil. Especially for objects, you sometimes even want this kind of dependency. For example, in some trees (such as DOM documents), parents refer to children and children refer back to parents. In libraries, you can usually avoid cyclic dependencies via careful design. In a large system, though, they can happen, especially during refactoring. Then it is very useful if a module system supports them, because then the system doesn’t break while you are refactoring.

The Node.js documentation acknowledges the importance of cyclic dependencies [3] and Rob Sayre provides additional evidence:

Data point: I once implemented a system like [ECMAScript 6 modules] for Firefox. I got asked for cyclic dependency support 3 weeks after shipping.

That system that Alex Fritze invented and I worked on is not perfect, and the syntax isn’t very pretty. But it’s still getting used 7 years later, so it must have gotten something right.

Let’s see how CommonJS and ECMAScript 6 handle cyclic dependencies.

Cyclic dependencies in CommonJS

In CommonJS, if a module B requires a module A whose body is currently being evaluated, it gets back A’s exports object in its current state (line #1 in the following example). That enables B to refer to properties of that object inside its exports (line #2). The properties are filled in after B’s evaluation is finished, at which point B’s exports work properly.

    //------ a.js ------
    var b = require('b');
    exports.foo = function () { ... };
    
    //------ b.js ------
    var a = require('a'); // (1)
    // Can't use a.foo in module body,
    // but it will be filled in later
    exports.bar = function () {
        a.foo(); // OK (2)
    };
    
    //------ main.js ------
    var a = require('a');

As a general rule, keep in mind that with cyclic dependencies, you can’t access imports in the body of the module. That is inherent to the phenomenon and doesn’t change with ECMAScript 6 modules.

The limitations of the CommonJS approach are:

  • Node.js-style single-value exports don’t work. In Node.js, you can export single values instead of objects, like this:
    module.exports = function () { ... }
    If you did that in module A, you wouldn’t be able to use the exported function in module B, because B’s variable a would still refer to A’s original exports object.

  • You can’t use named exports directly. That is, module B can’t import a.foo like this:
    var foo = require('a').foo;
    foo would simply be undefined. In other words, you have no choice but to refer to foo via the exports object a.

CommonJS has one unique feature: you can export before importing. Such exports are guaranteed to be accessible in the bodies of importing modules. That is, if A did that, they could be accessed in B’s body. However, exporting before importing is rarely useful.

Cyclic dependencies in ECMAScript 6

In order to eliminate the aforementioned two limitations, ECMAScript 6 modules export bindings, not values. That is, the connection to variables declared inside the module body remains live. This is demonstrated by the following code.

    //------ lib.js ------
    export let counter = 0;
    export function inc() {
        counter++;
    }
    
    //------ main.js ------
    import { inc, counter } from 'lib';
    console.log(counter); // 0
    inc();
    console.log(counter); // 1

Thus, in the face of cyclic dependencies, it doesn’t matter whether you access a named export directly or via its module: There is an indirection involved in either case and it always works.

More on importing and exporting

Importing

ECMAScript 6 provides the following ways of importing [4]:

    // Default exports and named exports
    import theDefault, { named1, named2 } from 'src/mylib';
    import theDefault from 'src/mylib';
    import { named1, named2 } from 'src/mylib';
    
    // Renaming: import named1 as myNamed1
    import { named1 as myNamed1, named2 } from 'src/mylib';
    
    // Importing the module as an object
    // (with one property per named export)
    import * as mylib from 'src/mylib';
    
    // Only load the module, don't import anything
    import 'src/mylib';

Exporting

There are two ways in which you can export things that are inside the current module [5]. On one hand, you can mark declarations with the keyword export.

    export var myVar1 = ...;
    export let myVar2 = ...;
    export const MY_CONST = ...;
    
    export function myFunc() {
        ...
    }
    export function* myGeneratorFunc() {
        ...
    }
    export class MyClass {
        ...
    }

The “operand” of a default export is an expression (including function expressions and class expressions). Examples:

    export default 123;
    export default function (x) {
        return x
    };
    export default x => x;
    export default class {
        constructor(x, y) {
            this.x = x;
            this.y = y;
        }
    };

On the other hand, you can list everything you want to export at the end of the module (which is once again similar in style to the revealing module pattern).

    const MY_CONST = ...;
    function myFunc() {
        ...
    }
    
    export { MY_CONST, myFunc };

You can also export things under different names:

    export { MY_CONST as THE_CONST, myFunc as theFunc };

Note that you can’t use reserved words (such as default and new) as variable names, but you can use them as names for exports (you can also use them as property names in ECMAScript 5). If you want to directly import such named exports, you have to rename them to proper variables names.

Re-exporting

Re-exporting means adding another module’s exports to those of the current module. You can either add all of the other module’s exports:

    export * from 'src/other_module';

Or you can be more selective (optionally while renaming):

    export { foo, bar } from 'src/other_module';
    
    // Export other_module's foo as myFoo
    export { foo as myFoo, bar } from 'src/other_module';

Module meta-data

ECMAScript 6 also provides a way to access information about the current module (such as the module’s URL) from inside that module. This is done as follows.

    import { url } from this module;
    console.log(url);

this module is simply a token indicating that we import the meta-data “as a module”. It could just as well be module_meta_data.

You can also access the meta-data via an object:

    import * as metaData from this module;
    console.log(metaData.url);

Node.js uses module-local variables such as __fileName for this kind of meta-data.

eval() and modules

eval() does not support module syntax. It parses its argument according to the Script grammar rule and scripts don’t support module syntax (why is explained later). If you want to evaluate module code, you can use the module loader API (described next).

The ECMAScript 6 module loader API

In addition to the declarative syntax for working with modules, there is also a programmatic API. It allows you to:

  • Programmatically work with modules and scripts
  • Configure module loading

Loaders handle resolving module specifiers (the string IDs at the end of import...from), loading modules, etc. Their constructor is Reflect.Loader. Each platform keeps a customized instance in the global variable System (the system loader), which implements its specific style of module loading.

Importing modules and loading scripts

You can programmatically import a module, via an API based on ES6 promises:

    System.import('some_module')
    .then(some_module => {
        // Use some_module
    })
    .catch(error => {
        ...
    });

System.import() enables you to:

  • Use modules inside elements (where module syntax is not supported, consult Sect. “Further information” for details).
  • Load modules conditionally.

System.import() retrieves a single module, you can use Promise.all() to import several modules:

    Promise.all(
        ['module1', 'module2', 'module3']
        .map(x => System.import(x)))
    .then(([module1, module2, module3]) => {
        // Use module1, module2, module3
    });

More loader methods:

Configuring module loading

The module loader API has various hooks for configuration. It is still work in progress. A first system loader for browsers is currently being implemented and tested. The goal is to figure out how to best make module loading configurable.

The loader API will permit many customizations of the loading process. For example:

  1. Lint modules on import (e.g. via JSLint or JSHint).
  2. Automatically translate modules on import (they could contain CoffeeScript or TypeScript code).
  3. Use legacy modules (AMD, Node.js).

Configurable module loading is an area where Node.js and CommonJS are limited.

Further information

The following content answers two important questions related to ECMAScript 6 modules: How do I use them today? How do I embed them in HTML?

  • Using ECMAScript 6 today gives an overview of ECMAScript 6 and explains how to compile it to ECMAScript 5. If you are interested in the latter, start reading in Sect. 2. One intriguing minimal solution is the ES6 Module Transpiler which only adds ES6 module syntax to ES5 and compiles it to either AMD or CommonJS.

  • Embedding ES6 modules in HTML: The code inside elements does not support module syntax, because the element’s synchronous nature is incompatible with the asynchronicity of modules. Instead, you need to use the new element. The blog post “ECMAScript 6 modules in future browsers” explains how works. It has several significant advantages over and can be polyfilled in its alternative version .

  • CommonJS vs. ES6:JavaScript Modules” (by Yehuda Katz) is a quick intro to ECMAScript 6 modules. Especially interesting is a second page where CommonJS modules are shown side by side with their ECMAScript 6 versions.

Benefits of ECMAScript 6 modules

At first glance, having modules built into ECMAScript 6 may seem like a boring feature – after all, we already have several good module systems. But ECMAScript 6 modules have features that you can’t add via a library, such as a very compact syntax and a static module structure (which helps with optimizations, static checking and more). They will also – hopefully – end the fragmentation between the currently dominant standards CommonJS and AMD.

Having a single, native standard for modules means:

  • No more UMD (Universal Module Definition): UMD is a name for patterns that enable the same file to be used by several module systems (e.g. both CommonJS and AMD). Once ES6 is the only module standard, UMD becomes obsolete.
  • New browser APIs become modules instead of global variables or properties of navigator.
  • No more objects-as-namespaces: Objects such as Math and JSON serve as namespaces for functions in ECMAScript 5. In the future, such functionality can be provided via modules.

Acknowledgements: Thanks to Domenic Denicola for confirming the final module syntax. Thanks for corrections of this blog post go to: Guy Bedford, John K. Paul, Mathias Bynens, Michael Ficarra.

References

  1. A JavaScript glossary: ECMAScript, TC39, etc.
  2. Static module resolution” by David Herman
  3. Modules: Cycles” in the Node.js API documentation
  4. Imports” (ECMAScript 6 specification)
  5. Exports” (ECMAScript 6 specification)

Source:: 2ality

ECMAScript 6 promises (1/2): foundations

By Axel Rauschmayer

This blog post explains foundations of asynchronous programming in JavaScript. It is first in a series of two posts and prepares you for part two, which covers promises and the ECMAScript 6 promise API.

The JavaScript call stack

When a function f calls a function g, g needs to know where to return to (inside f) after it is done. This information is usually managed with a stack, the call stack. Let’s look at an example.

    function h(z) {
        // Print stack trace
        console.log(new Error().stack); // (A)
    }
    function g(y) {
        h(y + 1); // (B)
    }
    function f(x) {
        g(x + 1); // (C)
    }
    f(3); // (D)
    return; // (E)

Initially, when the program above is started, the call stack is empty. After the function call f(3) in line (D), the stack has one entry:

  • Location in global scope

After the function call g(x + 1) in line (C), the stack has two entries:

  • Location in f
  • Location in global scope

After the function call h(y + 1) in line (B), the stack has three entries:

  • Location in g
  • Location in f
  • Location in global scope

The stack trace printed in line (A) shows you what the call stack looks like:

    Error
        at h (stack_trace.js:2:17)
        at g (stack_trace.js:6:5)
        at f (stack_trace.js:9:5)
        at <global> (stack_trace.js:11:1)

Next, each of the functions terminates and each time the top entry is removed from the stack. After function f is done, we are back in global scope and the call stack is empty. In line (E) we return and the stack is empty, which means that the program terminates.

The browser event loop

Simplifyingly, each browser tab runs (in) a single process: the event loop. This loop executes browser-related things (so-called tasks) that it is fed via a task queue. Examples of tasks are:

  1. Parsing HTML
  2. Executing JavaScript code in script elements
  3. Reacting to user input (mouse clicks, key presses, etc.)
  4. Processing the result of an asynchronous network request

Items 2–4 are tasks that run JavaScript code, via the engine built into the browser. They terminate when the code terminates. Then the next task from the queue can be executed. The following diagram (inspired by a slide by Philip Roberts [1]) gives an overview of how all these mechanisms are connected.

The event loop is surrounded by other processes running in parallel to it (timers, input handling, etc.). These processes communicate with it by adding tasks to its queue.

Timers

Browsers have timers. setTimeout() creates a timer, waits until it fires and then adds a task to the queue. It has the signature:

    setTimeout(callback, ms)

After ms milliseconds, callback is added to the task queue. It is important to note that ms only specifies when the callback is added, not when it actually executed. That may happen much later, especially if the event loop is blocked (as demonstrated later in this post).

setTimeout() with ms set to zero is a commonly used work-around to add something to the task queue right away. However, some browsers do not allow ms to be below a minimum (4 ms in Firefox); they set it to that minimum if it is.

Displaying DOM changes

For most DOM changes (especially those involving a re-layout), the display isn’t updated right away. “Layout happens off a refresh tick every 16ms” (@bz_moz) and must be given a chance to run via the event loop.

There are ways to coordinate frequent DOM updates with the browser, to avoid clashing with its layout rhythm. Consult the documentation on requestAnimationFrame() for details.

Run-to-completion semantics

JavaScript has so-called run-to-completion semantics: The current task is always finished before the next task is executed. That means that each task has complete control over all current state and doesn’t have to worry about concurrent modification.

Let’s look at an example:

    setTimeout(function () { // (A)
        console.log('Second');
    }, 0);
    console.log('First'); // (B)

The function starting in line (A) is added to the task queue immediately, but only executed after the current piece of code is done (in particular line (B)!). That means that this code’s output will always be:

    First
    Second

Blocking the event loop

As we have seen, each tab (in some browers, the complete browser) is managed by a single process – both the user interface and all other computations. That means that you can freeze the user interface by performing a long-running computation in that process. The following code demonstrates that. You can try it out online.

    <a id="block" href="">Block for 5 seconds</a>
    <p>
    <button>Simple button</button>
    <script>
        ...
        function onClick(event) {
            event.preventDefault();
    
            console.log('Blocking...');
            sleep(5000);
            console.log('Done');
        }
        function sleep(milliseconds) {
            var start = Date.now();
            while ((Date.now() - start) < milliseconds);
        }
    </script>

Whenever the link at the beginning is clicked, the function onClick() is triggered. It uses the – synchronous – sleep() function to block the event loop for five seconds. During those seconds, the user interface doesn’t work. For example, you can’t click the “Simple button”.

Avoiding blocking

You avoid blocking the event loop in two ways:

First, you don’t perform long-running computations in the main process, you move them to a different process. This can be achieved via the Worker API.

Second, you don’t (synchronously) wait for the results of a long-running computation (your own algorithm in a Worker process, a network request, etc.), you carry on with the event loop and let the computation notify you when it is finished. In fact, you usually don’t even have a choice in browsers and have to do things this way. For example, there is no built-in way to sleep synchronously (like the previously implemented sleep()). Instead, setTimeout() lets you sleep asynchronously.

The next section explains techniques for waiting asynchronously for results.

Receiving results asynchronously

Two common patterns for receiving results asynchronously are: events and callbacks.

Asynchronous results via events

In this pattern for asynchronously receiving results, you create an object for each request and register event handlers with it: one for a successful computation, another one for handling errors. The following code shows how that works with the XMLHttpRequest API:

    var req = new XMLHttpRequest();
    req.open('GET', url);
    
    req.onload = function() {
        if (req.status == 200) {
            processData(req.response);
        } else {
            console.log('ERROR', req.statusText);
        }
    };
    
    req.onerror = function() {
        console.log('Network Error');
    };
    
    req.send(); // Add request to task queue

Note that the last line doesn’t actually perform the request, it adds it to the task queue. Therefore, you could also call that method right after open(), before setting up onload and onerror. Things would work the same, due to JavaScript’s run-to-completion semantics.

If you are used to multi-threaded programming languages, IndexedDB requests look like they might be prone to race conditions. However, run to completion makes the following code safe in JavaScript:

    var openRequest = indexedDB.open('test', 1);
    
    openRequest.onsuccess = function(event) {
        console.log('Success!');
        var db = event.target.result;
    };
    
    openRequest.onerror = function(error) {
        console.log(error);
    };

open() does not immediately open the database, it adds a task to the queue, which is executed after the current task is finished. That is why you can (and in fact must) register event handlers after calling open().

Asynchronous results via callbacks

If you handle asynchronous results via callbacks, you pass callback functions as trailing parameters to asynchronous function or method calls.

The following is an example in Node.js. We read the contents of a text file via an asynchronous call to fs.readFile():

    // Node.js
    fs.readFile('myfile.txt', { encoding: 'utf8' },
        function (error, text) { // (A)
            if (error) {
                // ...
            }
            console.log(text);
        });

If readFile() is successful, the callback in line (A) receives a result via the parameter text. If it isn’t, the callback gets an error (often an instance of Error or a sub-constructor) via its first parameter.

The same code in classic functional programming style would look like this:

    // Functional
    readFileFunctional('myfile.txt', { encoding: 'utf8' },
        function (text) { // success
            console.log(text);
        },
        function (error) { // failure
            // ...
        });

Continuation-passing style

The programming style of using callbacks (especially in the functional manner shown previously) is also called continuation-passing style (CPS), because the next step (the continuation) is explicitly passed as a parameter. This gives an invoked function more control over what happens next and when.

The following code illustrates CPS:

    console.log('A');
    identity('B', function step2(result2) {
        console.log(result2);
        identity('C', function step3(result3) {
           console.log(result3);
        });
        console.log('D');
    });
    console.log('E');
    
    // Output: A E B D C
    
    function identity(input, callback) {
        setTimeout(function () {
            callback(input);
        }, 0);
    }

For each step, the control flow of the program continues inside the callback. This leads to nested functions, which are sometimes referred to as callback hell. However, you can often avoid nesting, because JavaScript’s function declarations are hoisted (their definitions are evaluated at the beginning of their scope). That means that you can call ahead and invoke functions defined later in the program. The following code uses hoisting to flatten the previous example.

    console.log('A');
    identity('B', step2);
    function step2(result2) {
        // The program continues here
        console.log(result2);
        identity('C', step3);
        console.log('D');
    }
    function step3(result3) {
       console.log(result3);
    }
    console.log('E');

[3] contains more information on CPS.

Composing code in CPS

In normal JavaScript style, you compose pieces of code via:

  1. Putting them one after another. This is blindingly obvious, but it’s good to remind ourselves that concatenating code in normal style is sequential composition.
  2. Array methods such as map(), filter() and forEach()
  3. Loops such as for and while

The library Async.js provides combinators to let you do similar things in CPS, with Node.js-style callbacks. It is used in the following example to load the contents of three files, whose names are stored in an array.

    var async = require('async');
    
    var fileNames = [ 'foo.txt', 'bar.txt', 'baz.txt' ];
    async.map(fileNames,
        function (fileName, callback) {
            fs.readFile(fileName, { encoding: 'utf8' }, callback);
        },
        // Process the result
        function (error, textArray) {
            if (error) {
                console.log(error);
                return;
            }
            console.log('TEXTS:n' + textArray.join('n----n'));
        });

Pros and cons of callbacks

Using callbacks results in a radically different programming style, CPS. The main advantage of CPS is that its basic mechanisms are easy to understand. However, it has disadvantages:

  • Error handling becomes more complicated: There are now two ways in which errors are reported – via callbacks and via exceptions. You have to be careful to combine both properly.

  • Less elegant signatures: In synchronous functions, there is a clear separation of concerns between input (parameters) and output (function result). In asynchronous functions that use callbacks, these concerns are mixed: the function result doesn’t matter and some parameters are used for input, others for output.

  • Composition is more complicated: Because the concern “output” shows up in the parameters, it is more complicated to compose code via combinators.

Callbacks in Node.js style have three disadvantages (compared to those in a functional style):

  • The if statement for error handling adds verbosity.
  • Reusing error handlers is harder.
  • Providing a default error handler is also harder. A default error handler is useful if you make a function call and don’t want to write your own handler. It could also be used by a function if a caller doesn’t specify a handler.

Looking ahead

The second part of this series covers promises and the ECMAScript 6 promise API. Promises are more complicated under the hood than callbacks. In exchange, they bring several significant advantages.

Further reading

  1. Help, I’m stuck in an event-loop” by Philip Roberts (video).
  2. Event loops” in the HTML Specification.
  3. Asynchronous programming and continuation-passing style in JavaScript

Reviewers

I’d like to thank the following people for reviewing this post.

Source:: 2ality

ECMAScript 6 promises (2/2): the API

By Axel Rauschmayer

This blog post is an introduction to asynchronous programming via promises in general and the ECMAScript 6 (ES6) promise API in particular. It is second in a series of two posts – part one explains foundations of asynchronous programming (which you may need to learn in order to fully understand this post).

Given that the ECMAScript 6 promise API is easy to polyfill for ECMAScript 5, I’m mainly using function expressions and not ECMAScript 6 arrow functions, even though the latter are much less verbose.

Promises

Promises are a pattern that helps with one particular kind of asynchronous programming: functions (or methods) that return their results asynchronously. To implement such a function, you return a promise, an object that is a placeholder for the result. The caller of the function registers callbacks with the promise to be notified once the result has been computed. The function sends the result via the promise.

The de-facto standard for JavaScript promises is called Promises/A+ [1]. The ECMAScript 6 promise API follows that standard.

A first example

Let’s look at a first example, to give you a taste of what working with promises is like.

With Node.js-style callbacks, reading a file asynchronously looks like this:

    fs.readFile('config.json',
        function (error, text) {
            if (error) {
                console.error('Error while reading config file');
            } else {
                try {
                    var obj = JSON.parse(text);
                    console.log(JSON.stringify(obj, null, 4));
                } catch (e) {
                    console.error('Invalid JSON in file');
                }
            }
        });

With promises, the same functionality is implemented like this:

    readFilePromisified('config.json')
    .then(function (text) { // (A)
        var obj = JSON.parse(text);
        console.log(JSON.stringify(obj, null, 4));
    })
    .catch(function (reason) { // (B)
        // File read error or JSON SyntaxError
        console.error('An error occurred', reason);
    });

There are still callbacks, but they are provided via methods that are invoked on the result (then() and catch()). The error callback in line (B) is convenient in two ways: First, it’s a single style of handling errors. Second, you can handle the errors of both readFilePromisified() and the callback from line (A).

Creating and using promises

Let’s look at how promises are operated from the producer and the consumer side.

Producing a promise

As a producer, you create a promise and send a result via it:

    var promise = new Promise(
        function (resolve, reject) { // (A)
            ...
            if (...) {
                resolve(value); // success
            } else {
                reject(reason); // failure
            }
        });

A promise is always in either one of three (mutually exclusive) states:

  • Pending: the result hasn’t been computed, yet
  • Fulfilled: the result was computed successfully
  • Rejected: a failure occurred during computation

A promise is settled (the computation it represents has finished) if it is either fulfilled or rejected. A promise can only be settled once and then stays settled. Subsequent attempts to settle it have no effect.

The parameter of new Promise() (starting in line (A)) is called an executor:

  • If the computation went well, the executor sends the result via resolve(). That usually fulfills the promise (it may not, if you resolve with a promise, as explained later).
  • If an error happened, the executor notifies the promise consumer via reject(). That always rejects the promise.

Consuming a promise

As a consumer of promise, you are notified of a fulfillment or a rejection via reactions – callbacks that you register with the method then():

    promise.then(
        function (value) { /* fulfillment */ },
        function (reason) { /* rejection */ }
    );

What makes promises so useful for asynchronous functions (with one-off results) is that once a promise is settled, it doesn’t change anymore. Furthermore, there are never any race conditions, because it doesn’t matter whether you invoke then() before or after a promise is settled:

  • In the former case, the appropriate reaction is called as soon as the promise is settled.
  • In the latter case, the promise result (fulfillment value or rejection value) is cached and handed to the appropriate reaction “immediately” (queued as a task).

Only handling fulfillments or rejections

If you are only interested in fulfillments, you can omit the second parameter of then():

    promise.then(
        function (value) { /* fulfillment */ }
    );

If you are only interested in rejections, you can omit the first parameter. The method catch() is a more compact way of doing the same thing.

    promise.then(
        null,
        function (reason) { /* rejection */ }
    );
    
    // Equivalent:
    promise.catch(
        function (reason) { /* rejection */ }
    );

It is recommended to use then() exclusively for fulfillments and catch() for errors, because it nicely labels callbacks and because you can handle the rejections of multiple promises at the same time (how is explained later).

Examples

Let’s use these basic building blocks in a few examples.

Example: promisifying XMLHttpRequest

The following is a promise-based function that performs an HTTP GET via the event-based XMLHttpRequest API:

    function httpGet(url) {
        return new Promise(
            function (resolve, reject) {
                var request = new XMLHttpRequest();
                request.onreadystatechange = function () {
                    if (this.status === 200) {
                        // Success
                        resolve(this.response);
                    } else {
                        // Something went wrong (404 etc.)
                        reject(new Error(this.statusText));
                    }
                }
                request.onerror = function () {
                    reject(new Error(
                        'XMLHttpRequest Error: '+this.statusText));
                };
                request.open('GET', url);
                request.send();    
            });
    }

This is how you use httpGet():

    httpGet('http://example.com/file.txt')
    .then(
        function (value) {
            console.log('Contents: ' + value);
        },
        function (reason) {
            console.error('Something went wrong', reason);
        });

Example: delaying an activity

Let’s implement setTimeout() as the promise-based function delay() (similar to Q.delay()).

    function delay(ms) {
        return new Promise(function (resolve, reject) {
            setTimeout(resolve, ms); // (A)
        });
    }
    
    // Using delay():
    delay(5000).then(function () { // (B)
        console.log('5 seconds have passed!')
    });

Note that in line (A), we are calling resolve with zero parameters, which is the same as calling resolve(undefined). We don’t need the fulfillment value in line (B), either and simply ignore it. Just being notified is enough here.

Example: timing out a promise

    function timeout(ms, promise) {
        return new Promise(function (resolve, reject) {
            promise.then(resolve);
            setTimeout(function () {
                reject(new Error('Timeout after '+ms+' ms')); // (A)
            }, ms);
        });
    }

Note that the rejection after the timeout (in line (A)) does not cancel the request, but it does prevent the promise being fulfilled with its result.

Using timeout() looks like this:

    timeout(5000, httpGet('http://example.com/file.txt'))
    .then(function (value) {
        console.log('Contents: ' + value);
    })
    .catch(function (reason) {
        console.error('Error or timeout', reason);
    });

Chaining then()

The result of a method call

    P.then(onFulfilled, onRejected)

is a new promise Q. That means that you can keep the promised-based control flow going by invoking then() on Q:

  • Q is resolved with what is returned by either onFulfilled or onRejected.
  • Q is rejected if either onFulfilled or onRejected throw an exception.

Resolving with normal values

If you resolve the promise Q returned by then() with a normal value, you can pick up that value via a subsequent then():

    asyncFunc()
    .then(function (value1) {
        return 123;
    })
    .then(function (value2) {
        console.log(value2); // 123
    });

Resolving with thenables

You can also resolve the promise Q returned by then() with a thenable R. A thenable is any object that has a promise-style method then(). Thus, promises are thenable. Resolving with R (e.g. by returning it from onFulfilled) means that it is inserted “after” Q: R’s settlement is forwarded to Q’s onFulfilled and onRejected callbacks. In a way, Q becomes R.

The main use for this mechanism is to flatten nested then() calls, like in the following example:

    asyncFunc1()
    .then(function (value1) {
        asyncFunc2()
        .then(function (value2) {
            ...
        });
    })

The flat version looks like this:

    asyncFunc1()
    .then(function (value1) {
        return asyncFunc2();
    })
    .then(function (value2) {
        ...
    })

Error handling

As mentioned previously, whatever you return in an error handler becomes a fulfillment value (not rejection value!). That allows you to specify default values that are used in case of failure:

    retrieveFileName()
    .catch(function () {
        // Something went wrong, use a default value
        return 'Untitled.txt';
    })
    .then(function (fileName) {
        ...
    });

Catching exceptions

Exceptions in the executor are passed on to the next error handler.

    new Promise(function (resolve, reject) {
        throw new Error();
    })
    .catch(function (err) {
        // Handle error here
    });

As are exceptions that are thrown in either one of then‘s parameters:

    asyncFunc()
    .then(function (value) {
        throw new Error();
    })
    .catch(function (reason) {
        // Handle error here
    });

Chaining errors

There can be one or more then() method calls that don’t provide an error handler. Then the error is passed on until there is an error handler.

    asyncFunc1()
    .then(asyncFunc2)
    .then(asyncFunc3)
    .catch(function (reason) {
        // Something went wrong above
    });

Composition

This section describes how you can compose existing promises to create new ones. We have already encountered one way of composing promises: sequential chaining via then(). Promise.all() and Promise.race() provide additional ways of composing.

map() via Promise.all()

One nice thing about promises is that many synchronous tools still work, because promise-based functions return results. For example, you can use the array method map():

    var fileUrls = [
        'http://example.com/file1.txt',
        'http://example.com/file2.txt'
    ];
    var promisedTexts = fileUrls.map(httpGet);

promisedTexts is an array of promises. Promise.all() takes an array of promises (thenables and other values are converted to promises via Promise.resolve()) and, once all of them are fulfilled, it fulfills with an array of their values:

    Promise.all(promisedTexts)
    .then(function (texts) {
        texts.forEach(function (text) {
            console.log(text);
        });
    })
    .catch(function (reason) {
        // Receives first rejection among the promises
    });

Timing out via Promise.race()

Promise.race() takes an array of promises (thenables and other values are converted to promises via Promise.resolve()) and returns a promise P. The first of the input promises that is settled passes its settlement on to the output promise.

As an example, let’s use Promise.race() to implement a timeout:

    Promise.race([
        httpGet('http://example.com/file.txt'),
        delay(5000).then(function () {
            throw new Error('Timed out')
        });
    ])
    .then(function (text) { ... })
    .catch(function (reason) { ... });

Promises are always async

A promise library has complete control over whether results are delivered to promise reactions synchronously (right away) or asynchronously (after the current continuation, the current piece of code, is finished). However, the Promises/A+ specification demands that the latter mode of execution be always used. It states so via the following requirement (2.2.4) for the then() method:

onFulfilled or onRejected must not be called until the execution context stack contains only platform code.

That means that you code can rely on run-to-completion semantics (as explained in part 1) and that chaining promises won’t starve other tasks of processing time.

Cheat sheet: the ECMAScript 6 promise API

This section gives an overview of the ECMAScript 6 promise API, as described in the specification.

Glossary

The promise API is about delivering results asynchronously. A promise object (short: promise) is a stand-in for the result, which is delivered via that object.

States:

  • A promise is always in either one of three mutually exclusive states:
    • Before the result is ready, the promise is pending.
    • If a result is available, the promise is fulfilled.
    • If an error happened, the promise is rejected.
  • A promise is settled if “things are done” (if it is either fulfilled or rejected).
  • A promise is settled exactly once and then remains unchanged.

Reacting to state changes:

  • Promise reactions are callbacks that you register with the promise method then(), to be notified of a fulfillment or a rejection.

  • A thenable is an object that has a promise-style then() method. Whenever the API is only interested in being notified of settlements, it only demands thenables.

Changing states: There are two operations for changing the state of a promise. After you have invoked either one of them once, further invocations have no effect.

  • Rejecting a promise means that the promise becomes rejected.
  • Resolving a promise has different effects, depending on what value you are resolving with:
    • Resolving with a normal (non-thenable) value fulfills the promise.
    • Resolving a promise P with a thenable T means that P can’t be resolved anymore and will now follow T’s state, including its fulfillment or rejection value. The appropriate P reactions will get called once T settles (or are called if T is already settled).

Constructor

The constructor for promises has the following signature:

    var p = new Promise(executor(resolve, reject))

It creates a promise whose behavior is determined by the callback executor. It can use its parameters to resolve or reject p:

  • resolve(x) resolves p with x:
    • If x is thenable, its settlement is forwarded to p (which includes triggering reactions registered via then()).
    • Otherwise, p is fulfilled with x.
  • reject(e) rejects p with the value e (often an instance of Error).

Static methods

All static methods of Promise support subclassing: they create new instances via their receiver (think: new this(...)) and also access other static methods via it (this.resolve(...) versus Promise.resolve(...)).

Creating promises

The following two methods create new instances of their receiver (their this).

  • Promise.resolve(x):

    • If x is thenable, it is converted to a promise (an instance of the receiver).
    • If x is a promise, it is returned unchanged.
    • Otherwise, return a new instance of the receiver that is fulfilled with x.
  • Promise.reject(reason): creates a new promise that is rejected with the value reason.

Composing promises

Intuitively, the static methods Promise.all() and Promise.race() compose iterables of promises to a single promise. That is:

  • They take an iterable. The elements of the iterable are converted to promises via this.resolve().
  • They return a new promise. That promise is a fresh instance of the receiver.

The methods are:

  • Promise.all(iterable): returns a promise that…

    • is fulfilled if all elements in iterable are fulfilled.
      Fulfillment value: array with fulfillment values.
    • is rejected if any of the elements are rejected.
      Rejection value: first rejection value.
  • Promise.race(iterable): the first element of iterable that is settled is used to settle the returned promise.

Instance prototype methods

Promise.prototype.then(onFulfilled, onRejected):

  • The callbacks onFulfilled and onRejected are called reactions.
  • onFulfilled is called immediately if the promise is already fulfilled or as soon as it becomes fulfilled. Similarly, onRejected is informed of rejections.
  • then() returns a new promise Q (created via the constructor of the receiver):
    • If either of the reactions returns a value, Q is resolved with it.
    • If either of the reactions throws an exception, Q is rejected with it.
  • Omitted reactions:
    • If onFulfilled has been omitted, a fulfillment of the receiver is forwarded to the result of then().
    • If onRejected has been omitted, a rejection of the receiver is forwarded to the result of then().

Default values for omitted reactions could be implemented like this:

    function defaultOnFulfilled(x) {
        return x;
    }
    function defaultOnRejected(e) {
        throw e;
    }

Promise.prototype.catch(onRejected):

  • Same as then(null, onRejected).

Pros and cons of promises

The pros

Unifying asynchronous APIs

One important advantage of promises is that they will increasingly be used by asnychronous browser APIs and unify currently diverse and incompatible patterns and conventions. Let’s look at two upcoming promise-based APIs.

The fetch API is a promise-based alternative to XMLHttpRequest:

    fetch(url)
    .then(request => request.text())
    .then(str => ...)

fetch() returns a promise for the actual request, text() returns a promise for the content as a string.

The ECMAScript 6 API for programmatically importing modules is based on promises, too:

     System.import('some_module.js')
    .then(some_module => {
        ...
    })
Promises versus events

Compared to events, promises are better for handling one-off results. It doesn’t matter whether you register for a result before or after it has been computed, you will get it. This advantage of promises is fundamental in nature. On the flip side, you can’t use them for handling recurring events. Chaining is another advantage of promises, but one that could be added to event handling.

Promises versus callbacks

Compared to callbacks, promises have cleaner function (or method) signatures. With callbacks, parameters are used for input and output:

    fs.readFile(name, opts?, function (err, data))

With promises, all parameters are used for input:

    readFilePromisified(name, opts?)
        .then(dataHandler, errorHandler)

Additional promise advantages include better error handling (which integrates exceptions) and easier composition (because you can reuse some synchronous tools such as Array.prototype.map()).

The cons

Promises work well for for single asynchronous results. They are not suited for:

  • Recurring events: If you are interested in those, take a look at reactive programming, which add a clever way of chaining to normal event handling.
  • Streams of data: A standard for supporting those is currently in development.

ECMAScript 6 promises lack two features that are sometimes useful:

  • You can’t cancel them.
  • You can’t query them for how far along they are (e.g. to display a progress bar in a client-side user interface).

The Q promise library has support for the latter and there are plans to add both capabilities to Promises/A+.

Promises and generators

With the help of a utility function such as Q.spawn(), you can use promise-based functions inside shallow coroutines, implemented via generators. This has the important advantage that the code looks synchronous and that you can use synchronous mechanisms such a try-catch:

    Q.spawn(function* () {
        try {
            let [foo, bar] = yield Promise.all([ // (A)
                httpGet('foo.json'),
                httpGet('bar.json')
            ]);
            render(foo);
            render(bar);
        } catch (e) {
            console.log('Read failed: ' + e);
        }
    });

The parameter of Q.spawn() is a generator function [7]. If the yield operator is used, the following things happen:

  1. Execution of the function is paused.
  2. The operand of yield is “returned” by the function. (It’s not exactly a “return”, but ignore that for now.)
  3. Later, the function can be resumed with a value or an exception. In the former case, execution continues where it was previously paused and yield returns the value. In the latter case, an exception is thrown inside the function, as if it were thrown “inside” yield‘s operand.

Thus, it’s clear what Q.spawn() has to do: When the generator function yields a promise, spawn registers reactions and waits for a settlement. If the promise is fulfilled, the generator is resumed with the result. If the promise is rejected, an exception is thrown inside the generator.

There is a proposal to add support for spawning to JavaScript, via the new syntactic construct “async functions”. The previous example as an async function looks as follows. Under the hood, there is not much of a difference – async functions are based on generators.

    async function () {
        try {
            let [foo, bar] = await Promise.all([
                httpGet('foo.json'),
                httpGet('bar.json')
            ]);
            render(foo);
            render(bar);
        } catch (e) {
            console.log('Read failed: ' + e);
        }
    }

Debugging promises

The main challenge with debugging asynchronous code is that it contains asynchronous function and method calls. Asynchronous calls originate in one task and are carried out in a new task. If something goes wrong in the new task, a stack trace will only cover that task and not contain information about previous tasks. Thus, you have to make do with much less debugging information in asynchronous programming.

Google Chrome recently got the ability to debug asynchronous code [6]. It doesn’t completely support promises, yet, but it’s impressive how well it handles normal asynchronous calls. For example, in the following code, first asynchronously calls second which in turn calls third.

    function first() {
        setTimeout(function () { second('a') }, 0); // (A)
    }
    function second(x) {
        setTimeout(function () { third('b') }, 0); // (B)
    }
    function third(x) {
        debugger;
    }
    first();

As you can see in the screen shot, the debugger shows a stack trace that contains all three functions. It even includes the anonymous functions in line (A) and (B).

The internals of promises

In this section, we will approach promises from a different angle: Instead of learning how to use the API, we will look at a simple implementation of it. This different angle helped me greatly with making sense of promises.

The promise implementation is called DemoPromise and available on GitHub. In order to be easier to understand, it doesn’t completely match the API. But it is close enough to still give you much insight into the challenges that actual implementations are facing.

DemoPromise is a constructor with three instance prototype methods:

  • DemoPromise.prototype.resolve(value)
  • DemoPromise.prototype.reject(reason)
  • DemoPromise.prototype.then(onFulfilled, onRejected)

That is, resolve and reject are methods (versus functions handed to a callback parameter of the constructor).

A stand-alone promise

Our first implementation is a stand-alone promise with minimal functionality:

  • You can create a promise.
  • You can resolve or reject a promise and you can only do it once.
  • You can register reactions (callbacks) via then(). The method does not support chaining, yet – it does not return anything. It must work independently of whether the promise has already been settled or not.

This is how this first implementation is used:

    var dp = new DemoPromise();
    dp.resolve('abc');
    dp.then(function (value) {
        console.log(value); // abc
    });

The following diagram illustrates how our first DemoPromise works:

Let’s examine then() first. It has to handle two cases:

  • If the promise is still pending, it queues invocations of onFulfilled and onRejected, to be used when the promise is settled.
  • If the promise is already fulfilled or rejected, onFulfilled or onRejected can be invoked right away.
    DemoPromise.prototype.then = function (onFulfilled, onRejected) {
        var self = this;
        var fulfilledTask = function () {
            onFulfilled(self.promiseResult);
        };
        var rejectedTask = function () {
            onRejected(self.promiseResult);
        };
        switch (this.promiseState) {
            case 'pending':
                this.fulfillReactions.push(fulfilledTask);
                this.rejectReactions.push(rejectedTask);
                break;
            case 'fulfilled':
                addToTaskQueue(fulfilledTask);
                break;
            case 'rejected':
                addToTaskQueue(rejectedTask);
                break;
        }
    };
    function addToTaskQueue(task) {
        setTimeout(task, 0);
    }

resolve() works as follows: If the promise is already settled, it does nothing (ensuring that a promise can only be settled once). Otherwise, the state of the promise changes to 'fulfilled' and the result is cached in this.promiseResult. All fulfillment reactions that have been enqueued so far must be triggered now.

    Promise.prototype.resolve = function (value) {
        if (this.promiseState !== 'pending') return;
        this.promiseState = 'fulfilled';
        this.promiseResult = value;
        this._clearAndEnqueueReactions(this.fulfillReactions);
        return this; // enable chaining
    };
    Promise.prototype._clearAndEnqueueReactions = function (reactions) {
        this.fulfillReactions = undefined;
        this.rejectReactions = undefined;
        reactions.map(addToTaskQueue);
    };

reject() is similar to resolve().

Chaining

The next feature we implement is chaining:

  • then() returns a promise that is resolved with what either onFulfilled or onRejected return.
  • If onFulfilled or onRejected are missing, whatever they would have received is passed on to the promise returned by then().

Obviously, only then() changes:

    DemoPromise.prototype.then = function (onFulfilled, onRejected) {
        var returnValue = new DemoPromise(); // (A)
        var self = this;
    
        var fulfilledTask;
        if (typeof onFulfilled === 'function') {
            fulfilledTask = function () {
                var r = onFulfilled(self.promiseResult);
                returnValue.resolve(r); // (B)
            };
        } else {
            fulfilledTask = function () {
                returnValue.resolve(self.promiseResult); // (C)
            };
        }
    
        var rejectedTask;
        if (typeof onRejected === 'function') {
            rejectedTask = function () {
                var r = onRejected(self.promiseResult);
                returnValue.resolve(r); // (D)
            };
        } else {
            rejectedTask = function () {
                // Important: we must reject here!
                // Normally, result of `onRejected` is used to resolve
                returnValue.reject(self.promiseResult); // (E)
            };
        }
        ...
        return returnValue; // (F)
    };

then() creates and returns a new promise (lines (A) and (F)). Additionally, fulfilledTask and rejectedTask are set up differently: After a settlement…

  • The result of onFulfilled is used to resolve returnValue (line (B).

    • If onFulfilled is missing, we use the fulfillment value to resolve returnValue (line (C)).
  • The result of onRejected is used to resolve (not reject!) returnValue (line (D)).

    • If onRejected is missing, we use the rejection value to reject returnValue (line (E)).

Flattening

Flattening is mostly about making chaining more convenient: Normally, returning a value from a reaction passes it on to the next then(). If we return a promise, it would be nice if it could be “unwrapped” for us, like in the following example:

    asyncFunc1()
    .then(function (value1) {
        return asyncFunc2(); // (A)
    })
    .then(function (value2) {
        // value2 is fulfillment value of asyncFunc2() promise
        console.log(value2);
    });

We returned a promise in line (A) and didn’t have to nest a call to then() inside the current method, we could invoke then() on the method’s result. Thus: no nested then(), everything remains flat.

We implement this by letting the resolve() method do the flattening:

  • Resolving a promise P with a promise Q means that Q’s settlement is forwarded to P’s reactions.
  • P becomes “locked in” on Q: it can’t be resolved (incl. rejected), anymore. And its state and result are always the same as Q’s.

We can make flattening more generic if we allow Q to be a thenable (instead of only a promise).

To implement locking-in, we introduce a new boolean flag this.alreadyResolved. Once it is true, this is locked and can’t be resolved anymore. Note that this may still be pending, because its state is now the same as the promise it is locked in on.

    DemoPromise.prototype.resolve = function (value) {
        if (this.alreadyResolved) return;
        this.alreadyResolved = true;
        this._doResolve(value);
        return this; // enable chaining
    };

The actual resolution now happens in the private method _doResolve():

    DemoPromise.prototype._doResolve = function (value) {
        var self = this;
        // Is `value` a thenable?
        if (value !== null && typeof value === 'object'
            && 'then' in value) {
            addToTaskQueue(function () { // (A)
                value.then(
                    function onFulfilled(value) {
                        self._doResolve(value);
                    },
                    function onRejected(reason) {
                        self._doReject(reason);
                    });
            });
        } else {
            this.promiseState = 'fulfilled';
            this.promiseResult = value;
            this._clearAndEnqueueReactions(this.fulfillReactions);
        }
    };

The flattening is performed in line (A): If value is fulfilled, we want self to be fulfilled and if value is rejected, we want self to be rejected. The forwarding happens via the private methods _doResolve and _doReject, to get around the protection via alreadyResolved.

Promise states in more detail

With chaining, the states of promises become more complex (as covered by Sect. 25.4 of the ECMAScript 6 specification):

If you are only using promises, you can normally adopt a simplified worldview and ignore locking-in. The most important state-related concept remains “settledness”: a promise is settled if it is either fulfilled or rejected. After a promise is settled, it doesn’t change, anymore (state and fulfillment or rejection value).

If you want to implement promises then “resolving” matters, too and is now harder to understand:

  • Intuitively, “resolved” means “can’t be (directly) resolved anymore”. A promise is resolved if it is either settled or locked in. Quoting the spec: “An unresolved promise is always in the pending state. A resolved promise may be pending, fulfilled or rejected.”
  • Resolving does not necessarily lead to settling: you can resolve a promise with another one that is always pending.
  • Resolving now includes rejecting (i.e., it is more general): you can reject a promise by resolving it with a rejected promise.

Exceptions

As our final feature, we’d like our promises to handle exceptions in user code as rejections. For now, “user code” means the two callback parameters of then().

The following excerpt shows how we turn exceptions inside onFulfilled into rejections – by wrapping a try-catch around its invocation in line (A).

    var fulfilledTask;
    if (typeof onFulfilled === 'function') {
        fulfilledTask = function () {
            try {
                var r = onFulfilled(self.promiseResult); // (A)
                returnValue.resolve(r);
            } catch (e) {
                returnValue.reject(e);
            }
        };
    } else {
        fulfilledTask = function () {
            returnValue.resolve(self.promiseResult);
        };
    }

Revealing constructor pattern

If we wanted to turn DemoPromise into an actual promise implementation, we’d still need to implement the revealing constructor pattern [5]: ES6 promises are not resolved and rejected via methods, but via functions that are handed to the executor, the callback parameter of the constructor.

If the executor throws an exception then “its” promise must be rejected.

Two useful additional promise methods

This section describes two useful methods that are easy to add to ES6 promises. Many of the more comprehensive promise libraries have them.

done()

When you chain several promise method calls, you risk silently discarding errors. For example:

    function doSomething() {
        asyncFunc()
        .then(f1)
        .catch(r1)
        .then(f2); // (A)
    }

If then() in line (A) produces a rejection, it will never be handled anywhere. The promise library Q provides a method done(), to be used as the last element in a chain of method calls. It either replaces the last then() (and has one to two arguments):

    function doSomething() {
        asyncFunc()
        .then(f1)
        .catch(r1)
        .done(f2);
    }

Or it is inserted after the last then() (and has zero arguments):

    function doSomething() {
        asyncFunc()
        .then(f1)
        .catch(r1)
        .then(f2)
        .done();
    }

Quoting the Q documentation:

The Golden Rule of done vs. then usage is: either return your promise to someone else, or if the chain ends with you, call done to terminate it. Terminating with catch is not sufficient because the catch handler may itself throw an error.

This is how you would implement done() in ECMAScript 6:

    Promise.prototype.done = function (onFulfilled, onRejected) {
        this.then(onFulfilled, onRejected)
        .catch(function (reason) {
            setTimeout(() => { throw reason }, 0);
        });
    };

While done‘s functionality is clearly useful, it has not been added to ECMAScript 6, because this kind of check can be performed automatically by debuggers in the future (a thread on es-discuss provides further background).

finally()

Sometimes you want to perform an action independently of whether an error happened or not. For example, to clean up after you are done with a resource. That’s what the promise method finally() is for, which works much like the finally clause in exception handling. Its callback receives no arguments, but is notified of either a resolution or a rejection.

    createResource(...)
    .then(function (value1) {
        // Use resource
    })
    .then(function (value2) {
        // Use resource
    })
    .finally(function () {
        // Clean up
    });

This is how Domenic Denicola proposes to implement finally():

    Promise.prototype.finally = function (callback) {
        let p = this.constructor;
        // We don't invoke the callback in here,
        // because we want then() to handle its exceptions
        return this.then(
            // Callback fulfills: pass on predecessor settlement
            // Callback rejects: pass on rejection (=omit 2nd arg.)
            value  => p.resolve(callback()).then(() => value),
            reason => p.resolve(callback()).then(() => { throw reason })
        );
    };

The callback determines how the settlement of the receiver (this) is handled:

  • If the callback throws an exception or returns a rejected promise then that becomes/contributes the rejection value.
  • Otherwise, the settlement of the receiver becomes the settlement of the promise returned by finally(). In a way, we take finally() out of the chain of methods.

Example 1 (by Jake Archibald): using finally() to hide a spinner. Simplified version:

    showSpinner();
    fetchGalleryData()
    .then(data => updateGallery(data))
    .catch(showNoDataError)
    .finally(hideSpinner);

Example 2 (by Kris Kowal): using finally() to tear down a test.

    var HTTP = require("q-io/http");
    var server = HTTP.Server(app);
    return server.listen(0)
    .then(function () {
        // run test
    })
    .finally(server.stop);

ES6-compatible promise libraries

There are many promise libraries out there. The following ones conform to the ECMAScript 6 API, which means that you can use them now and easily migrate to native ES6 later.

  • RSVP.js” by Stefan Penner is a superset of the ES6 promise API
.
    • ES6-Promises” by Jake Archibald extracts just the ES6 API out of RSVP.js.
  • Native Promise Only (NPO)” by Kyle Simpson is “a polyfill for native ES6 promises, as close as possible (no extensions) to the strict spec definitions”.
  • Lie” by Calvin Metcalf is “a small, performant, promise library implementing the Promises/A+ spec”.
  • Q.Promise by Kris Kowal implements the ES6 API.
  • Lastly, the “ES6 Shim” by Paul Millr includes Promise.

Interfacing with legacy asynchronous code

When you are using a promise library, you sometimes need to use non-promise-based asynchronous code. This section explains how to do that for Node.js-style asynchronous functions and jQuery deferreds.

Interfacing with Node.js

The promise library Q has several tool functions for converting functions that use Node.js-style (err,result) callbacks to ones that return a promise (there are even functions that do the opposite – convert promise-based functions to ones that accept callbacks). For example:

    var readFile = Q.denodeify(FS.readFile);
    
    readFile('foo.txt', 'utf-8')
    .then(function (text) {
        ...
    });

denodify is a micro-library that only provides the “nodification” functionality and complies with the ECMAScript 6 promise API.

Interfacing with jQuery

jQuery has deferreds which are similar to promises, but have several differences that prevent compatibility. Their method then() is almost like that of ES6 promises (main difference: it doesn’t catch errors in reactions). Thus, we can convert a jQuery deferred to an ES6 promise via Promise.resolve():

    Promise.resolve(
        jQuery.ajax({
            url: 'somefile.html', 
            type: 'GET'
        }))
    .then(function (data) {
        console.log(data);
    })
    .catch(function (reason) {
        console.error(reason);
    });

Further reading

Source:: 2ality

Statically typed JavaScript via Microsoft TypeScript, Facebook Flow and Google AtScript

By Axel Rauschmayer

Update 2014-11-18: Facebook Flow has been released as open source. Its website is flowtype.org. The site mentions plans for Flow’s future.

This blog post looks at three initiatives for adding static typing to JavaScript: Microsoft’s TypeScript, Facebook’s Flow and Google’s AtScript.

Typing

Let’s first clarify some terminology related to typing (excerpted from “Speaking JavaScript”).

Static versus dynamic

In the context of language semantics and type systems, static usually means “at compile time” or “without running a program,” while dynamic means “at runtime.”

Static typing versus dynamic typing

In a statically typed language, variables, parameters, and members of objects (JavaScript calls them properties) have types that the compiler knows at compile time. The compiler can use that information to perform type checks and to optimize the compiled code.

Even in statically typed languages, a variable also has a dynamic type, the type of the variable’s value at a given point at runtime. The dynamic type can differ from the static type. For example (Java):

    Object foo = "abc";

The static type of foo is Object; its dynamic type is String.

Normal JavaScript is dynamically typed; types of variables are generally not known at compile time.

Benefits of static typing

Does JavaScript really need static typing? Aren’t unit tests enough? Static typing isn’t magic: it does add complexity and visual clutter. This is especially true if you need to manually specify a type for everything (like in most of Java). Thankfully, neither of the three variants of typed JavaScript mentioned in this post force you to do so: if you don’t explicitly type an entity, they try to infer its type, by how the entity is used. Three examples:

  • If a variable is initialized with a value then the variable’s type is probably the same as the value’s type.
  • A number being passed as an argument leads to the initial assumption that the corresponding formal parameter has the type number.
  • The multiplication operator being applied to a parameter means that the parameter’s type is probably number.

Static typing offers the following benefits:

  • You get more errors at compile time. Early errors are good. This is usually faster than running unit tests and tends to catch a different category of errors.
  • It helps IDEs with auto-completion. Years ago, I used GWT (Java, compiled to JavaScript on the front end). Its statically typed DOM made it easy to explore that API.
  • Type annotations are useful for documenting parts of an API. I occasionally mention types in my JSDoc comments. If that information helps a static type checker then that is a nice side effect.
  • Checking and documenting types helps large teams collaborate, because it gives you an additional way of specifying what you require from or provide for your collaborators.

One more advantage is political: static typing (complemented by classes and modules) makes JavaScript more palatable for programmers coming from static languages such as Java and C# (i.e., many enterprise programmers). There is a danger of those programmers missing some of the subtleties of JavaScript, because it looks too familiar. However, more people liking a language that is still very recognizably JavaScript is a win, in my opinion.

Microsoft TypeScript

TypeScript [1] is a subset of ECMAScript 6 [2] plus optional static typing. There are several ECMAScript 6 (ES6) features it doesn’t support yet (e.g. let, destructuring, string templates, promises, iterators and for-of loops) and it still uses an older version of the ES6 module syntax [3]. Both divergences from ES6 will disappear by version 2.0 [4], meaning that TypeScript will be a strict superset of ES6 (give or take ES6 features that are difficult to compile to ES5, such as generators and proxies).

As mentioned, static typing is optional in TypeScript and supported via:

  • Type annotations: You can annotate parameters, function results and properties to declare their types.

  • Generics: TypeScript supports generic type variables, generic types, generic classes and generic constraints.

  • Interfaces: enable you to describe the structure of a value. Interfaces match structurally (“duck typing”). Therefore, you can introduce interfaces for “existing” values – externally and without changing how those values are created (no need to “implement” like in Java or C#).

  • Visibility modifiers for instance properties declared in classes. If a property is marked private, it can only be accessed from “within” a class, external accesses produce compiler errors.

Let’s look at examples.

Type annotations. In the following code, the parameters x and y and the function results are declared to have the type number. Thus, the compiler shows an error for the function call in the last line.

    function add(x : number, y : number) : number {
        return x + y;
    }
    add('a', 'b'); // compiler error

TypeScript compiles the function to the following ECMAScript 5 code. That is, all type information is gone at runtime, the end result is normal JavaScript.

    function add(x, y) {
        return x + y;
    }

Interfaces. In the following code, we demand that objects passed via the parameter x must have the number-valued property length.

    function getLength(x : { length: number }) {
        return x.length;
    }
    console.log(getLength('abcd')); // 4
    console.log(getLength(['a', 'b', 'c'])); // 3

Visibility modifiers. In the following code, x and y are private properties of Point instances. They can be accessed by the method dist(), but not from outside.

    class Point {
        constructor(private x, private y) {
        }
        dist() {
            return Math.sqrt(this.x*this.x + this.y*this.y);
        }
    }
    var p = new Point(3, 4);
    console.log(p.x); // compiler error: Point.x is inaccessible
    console.log(p.dist()); // 5

The class Point is translated to this ECMAScript 5 code.

    function Point(x, y) {
        this.x = x;
        this.y = y;
    }
    Point.prototype.dist = function () {
        return Math.sqrt(this.x * this.x + this.y * this.y);
    };

.d.ts files

TypeScript allows you to provide static type information for existing (untyped) code via external files, which have the file name extension .d.ts. The website DefinitelyTyped provides such files for many libraries. For example: jQuery, Backbone.js, Esprima, Express, gulp and Jasmine. That means that working with those libraries becomes more convenient if an IDE supports TypeScript. IDEs that do so are: Visual Studio, WebStorm, Eclipse (via TypEcs) and others.

As an example (one of several in the TypeScript manual) let’s assume that this is how the animalFactory API is used:

    animalFactory.create("dog");
    animalFactory.create("giraffe", { name: "ronald" });
    animalFactory.create("panda", { name: "bob", height: 400 });
    // Invalid: name must be provided if options is given
    animalFactory.create("cat", { height: 32 });

The .d.ts file for this API would be:

    interface AnimalOptions {
        name: string;
        height?: number;
        weight?: number;
    }
    function create(name: string, animalOptions?: AnimalOptions): Animal;

Facebook Flow

Flow [5] is a type checker for ECMAScript 6 that is based on flow analysis. As such, it only adds optional type annotations to the language and infers and checks types. It does not help with compiling ECMAScript 6 to ECMAScript 5. Flow is already in use at Facebook and will be open-sourced “later this year”.

React lets you use Flow’s annotations by removing them while compiling its JavaScript dialect to plain JavaScript. Quoting the React Blog:

And lastly, on the heels of announcing Flow at @Scale yesterday, we’re adding the ability to strip TypeScript-like type annotations as part of the jsx transform. To use, simply use the --strip-types flag on the command line, or set stripTypes in the options object when calling the API.

The Flow type system has support for extensible objects, open methods, prototypes, tuples, enums, generics, nullable types, union types, intersection types and more. All of these features are motivated by staying true to JavaScript’s nature, by the desire to capture how current JavaScript code is (implicitly) typed. Nullable types help prevent type errors caused by accessing properties if a value is undefined or null. You explicity specify whether, for example, a given parameter can be null (or undefined) or not. In the former case, the compiler forces you to check for null every time you access the parameter. In the latter case, the compiler warns you if you pass null or a nullable value.

Why not TypeScript? Flow’s type annotation syntax being compatible with TypeScript’s makes you wonder why Facebook doesn’t use TypeScript. Avik Chaudhuri mentioned [5] the following reasons:

  • Flow currently scales better than TypeScript. That is, it is faster for large programs.
  • Flow can infer more types, which means that it is more useful as a consistency checker for existing (completely unannotated) code bases.
  • Flow’s type system is richer. For example, TypeScript does not have non-nullable types.
  • Controlling the type checker enables Facebook to support their own technologies (e.g. React and its custom JSX syntax).

In order to scale, Flow runs as a server in the background and keeps itself up to date. Tools query the server. Flow’s type analysis is incremental, meaning that it can check modules in isolation. It supports many ECMAScript 6 features such as arrows, destructuring, optional parameters, promises, etc. Facebook is committed to track JavaScript standards as they evolve.

Google AtScript

The preferred way to code AngularJS 2.0 will be AtScript [6]. It is compiled to ECMAScript 5 (for now) and all of the AngularJS 2 features will be accessible from ES5 code.

AtScript is ECMAScript 6 plus the following extensions:

  • Type annotations for variables, parameters and properties (with TypeScript-compatible syntax).
  • Meta-data annotations (which are called “annotations” in Java and “decorators” in Python).

In contrast to TypeScript and Flow, both data is available at runtime.

Runtime type checks

You can turn type annotations into runtime type checks [6]: The following is AtScript code.

    class MyClass {
        methodA(name : string) : int {
            var length : int = name.length;
            return length;
        }
    }

The above code is equivalent to this ECMAScript 6 code:

    import * as rtts from 'rtts';
    class MyClass {
        methodA(name) {
            rtts.types(name, rtts.string);
            var length = tts.type(name.length, rtts.int);
            return rtts.returnType(length, rtts.int);
        }
    }

The idea is to use runtime type checks during development, because they help catch errors when you are working with untyped code. For deployment, you wouldn’t insert them into the code.

Runtime type information

Meta-data annotations mean that data is attached to annotated entities. For example, the following code uses two annotations, @Component and @Inject.

    @Component({selector: 'foo'})
    class MyComponent {
      @Inject()
      constructor(server:Server) {}
    }

It is translated to:

    class MyComponent {
      constructor(server) {}
    }
    MyComponent.parameters = [{is:Server}];
    MyComponent.annotate = [
      new Component({selector: 'foo'}),
      new Inject()
    ];

AngularJS uses the runtime type information for dependency injection and to configure constructs such as directives. For example [7]:

    @Directive({
      selector: ['[blink]']
    })
    class Blink {
      constructor(element:Element,
                  options:Options,
                  timeout:Timeout) {
        var selectors:Array<CssSelectors> =
            someThirdPartyAPI();
        element.query(selectors.join(','))
               .forEach(e => options.apply(e));
       }
    }

Thus, while TypeScript and Flow throw type data away after compilation, AtScript keeps it around. This is the only way to make it available to runtime mechanisms such as dependency injection. It also enables you to type-check JSON you load at runtime.

Compiling to ECMAScript 5 and Dart

What surprised me is that AtScript (.ats files) can be compiled to two target languages:

  • ECMAScript 5 (.js files), via Traceur [2], which supports AtScript’s language extensions
  • Dart (.dart files)

Given that AngularJS 2.0 is written completely in AtScript that means that there will be a single code base for both JavaScript and Dart.

A common standard?

The teams of TypeScript, Flow and AtScript seem eager to collaborate. Avik Chaudhuri says so in his talk [5] and the TypeScript team mentions it on their blog [4]:

The TypeScript team is working with both the Flow and AtScript teams to help ensure that resources that have already been created by the JavaScript typing community can be used across these tools. […] In the long term, we will also be working to fold the best features of these tools into ECMAScript, the standard behind JavaScript.

Furthermore, the following timeline is given [7] for AtScript’s features (without mentioning specific dates for each step):

  1. Runtime type checking: compile AtScript to .js and .dart
  2. Static type checking: IDE support
  3. Align with TypeScript
  4. ECMAScript proposal
  5. Browser support
  6. ECMAScript standard

Obviously, steps 5 and 6 are somewhat beyond the control of the AtScript team and depend on how things develop in the future.

Further reading

  1. Welcome to TypeScript”, the official TypeScript homepage
  2. Using ECMAScript 6 today
  3. ECMAScript 6 modules: the final syntax
  4. TypeScript and the Road to 2.0” by Jonathan Turner for the TypeScript Blog
  5. Video: “JavaScript Testing and Static Type Systems at Scale” by Avik Chaudhuri and Jeff Morrison
  6. AtScript Primer” by Miško Hevery
  7. Slides: “Keynote: AtScript” from the ng-europe conference

Source:: 2ality

Meta programming with ECMAScript 6 proxies

By Axel Rauschmayer

This blog post explains the ECMAScript 6 (ES6) feature proxies. Proxies enable you to intercept and customize operations performed on objects (such as getting properties). They are a meta programming feature.

The code in this post occasionally uses other ES6 features. Consult “Using ECMAScript 6 today” for an overview of all of ES6.

Before we can get into what proxies are and why they are useful, we first need to understand what meta programming is.

Programming versus meta programming

In programming, there are levels:

  • At the base level (also called: application level), code processes user input.
  • At the meta level, code processes base level code.

Base and meta level can be diffent languages. In the following meta program, the meta programming language is JavaScript and the base programming language is Java.

    let str = 'Hello' + '!'.repeat(3);
    console.log('System.out.println("'+str+'")');

Meta programming can take different forms. In the previous example, we have printed Java code to the console. Let’s use JavaScript as both meta programming language and base programming language. The classic example for this is the eval() function, which lets you evaluate/compile JavaScript code on the fly. There are very few actual use cases for eval(). In the interaction below, we use it to evaluate the expression 5 + 2.

    > eval('5 + 2')
    7

Other JavaScript operations may not look like meta programming, but actually are, if you look closer:

    // Base level
    let obj = {
        hello() {
            console.log('Hello!');
        }
    };
    
    // Meta level
    for (let key of Object.keys(obj)) {
        console.log(key);
    }

The program is examining its own structure while running. This doesn’t look like meta programming, because the separation between programming constructs and data structures is fuzzy in JavaScript. All of the Object.* methods can be considered meta programming functionality.

Kinds of meta programming

Reflective meta programming means that a program processes itself.
Kiczales et al. [2] distinguish three kinds of reflective meta programming:

  • Introspection: you have read-only access to the structure of a program.
  • Self-modification: you can change that structure.
  • Intercession: you can redefine the semantics of some language operations.

Let’s look at examples.

Example: introspection. Object.keys() performs introspection (see previous example).

Example: self-modification. The following function moveProperty moves a property from a source to a target. It performs self-modification via the bracket operator for property access, the assignment operator and the delete operator. (In production code, you’d probably use property descriptors for this task.)

    function moveProperty(source, propertyName, target) {
        target[propertyName] = source[propertyName];
        delete source[propertyName];
    }

Using moveProperty():

    > let obj1 = { prop: 'abc' };
    > let obj2 = {};
    > moveProperty(obj1, 'prop', obj2);
    
    > obj1
    {}
    > obj2
    { prop: 'abc' }

JavaScript doesn’t currently support intercession, proxies were created to fill that gap.

An overview of proxies

ECMAScript 6 proxies bring intercession to JavaScript. They work as follows. There are many operations that you can perform on an object obj. For example:

  • Getting a property prop (via obj.prop)
  • Listing enumerable own properties (via Object.keys(obj))

Proxies are special objects that allow you to provide custom implementations for some of these operations. A proxy is created with two parameters:

  • handler: For each operation, there is a corresponding handler method that – if present – performs that operation. Such a method intercepts the operation (on its way to the target) and is called a trap (a term borrowed from the domain of operating systems).
  • target: If the handler doesn’t intercept an operation then it is performed on the target. That is, it acts as a fallback for the handler. In a way, the proxy wraps the target.

In the following example, the handler intercepts the operations get (getting properties) and ownKey (retrieving the own property keys).

    let target = {};
    let handler = {
        get(target, propKey, receiver) {
            console.log('get ' + propKey);
            return 123;
        },
        ownKeys(target) {
            console.log('ownKeys');
            return ['hello', 'world'];
        }
    };
    let proxy = new Proxy(target, handler);

When we get property foo, the handler intercepts that operation:

    > proxy.foo
    get foo
    123

Similarly, Object.keys() triggers ownKeys:

    > Object.keys(proxy)
    ownKeys
    [ 'hello', 'world' ]

The handler doesn’t implement the trap set (setting properties). Therefore, setting proxy.bar is forwarded to target and leads to target.bar being set.

    > proxy.bar = 'abc';
    > target.bar
    'abc'

Function-specific traps

If the target is a function, two additional operations can be intercepted:

  • apply: Making a function call, triggered via proxy(···), proxy.call(···), proxy.apply(···).
  • construct: Making a constructor call, triggered via new proxy(···).

The reason for only enabling these traps for function targets is simple: You wouldn’t be able to forward the operations apply and construct, otherwise.

Revocable proxies

ECMAScript 6 lets you create proxies that can be revoked (switched off):

    let {proxy, revoke} = Proxy.revocable(target, handler);

On the left hand side of the assignment operator (=), we are using destructuring to access the properties proxy and revoke of the object returned by Proxy.revocable().

After you call the function revoke for the first time, any operation you apply to proxy causes a TypeError. Subsequent calls of revoke have no further effect.

    let target = {}; // Start with an empty object
    let handler = {}; // Don't intercept anything
    let {proxy, revoke} = Proxy.revocable(target, handler);
    
    proxy.foo = 123;
    console.log(proxy.foo); // 123
    
    revoke();
    
    console.log(proxy.foo); // TypeError: Revoked

Proxies as prototypes

A proxy proto can become the prototype of an object obj. Some operations that begin in obj may continue in proto. One such operation is get.

    let proto = new Proxy({}, {
        get(target, propertyKey, receiver) {
            console.log('GET '+propertyKey);
            return target[propertyKey];
        }
    });
    
    let obj = Object.create(proto);
    obj.bla; // Output: GET bla

The property bla can’t be found in obj, which is why the search continues in proto and the trap get is triggered there. There are more operations that affect prototypes, they are listed at the end of this post.

Forwarding operations

Operations whose traps the handler doesn’t implement are automatically forwarded to the target. Sometimes there is some task you want to perform in addition to forwarding the operation. For example, a handler that intercepts all operations and logs them, but doesn’t prevent them from reaching the target:

    let handler = {
        deleteProperty(target, propKey) {
            console.log('DELETE ' + propKey);
            return delete target[propKey];
        },
        has(target, propKey) {
            console.log('HAS ' + propKey);
            return propKey in target;
        },
        // Other traps: similar
    }

For each trap, we first log the name of the operation and then forward it by performing it manually. ECMAScript 6 has the module-like object Reflect that helps with forwarding: for each trap

    handler.trap(target, arg_1, ···, arg_n)

Reflect has a method

    Reflect.trap(target, arg_1, ···, arg_n)

If we use Reflect, the previous example looks as follows.

    let handler = {
        deleteProperty(target, propKey) {
            console.log('DELETE ' + propKey);
            return Reflect.deleteProperty(target, propKey);
        },
        has(target, propKey) {
            console.log('HAS ' + propKey);
            return Reflect.has(target, propKey);
        },
        // Other traps: similar
    }

Now what each of the traps does is so similar that we can implement the handler via a proxy:

    let handler = new Proxy({}, {
        get(target, trapName, receiver) {
            // Return the handler method named trapName
            return function (...args) {
                // Slice away target object in args[0]
                console.log(trapName.toUpperCase()+' '+args.slice(1));
                // Forward the operation
                return Reflect[trapName](...args);
            }
        }
    });

For each trap, the proxy asks for a handler method via the get operation and we give it one. That is, all of the handler methods can be implemented via the single meta method get. It was one of the goals for the proxy API to make this kind of virtualization simple.

Let’s use this proxy-based handler:

    > let target = {};
    > let proxy = new Proxy(target, handler);
    > proxy.foo = 123;
    SET foo,123,[object Object]
    > proxy.foo
    GET foo,[object Object]
    123

The following interaction confirms that the set operation was correctly forwarded to the target:

    > target.foo
    123

Use cases for proxies

This section demonstrates what proxies can be used for. That will also give you the opportunity to see the API in action.

Implementing the DOM in JavaScript

The browser Document Object Model (DOM) is usually implemented as a mix of JavaScript and C++. Implementing it in pure JavaScript is useful for:

  • Emulating a browser environment, e.g. to manipulate HTML in Node.js. jsdom is one library that does that.
  • Speeding the DOM up (switching between JavaScript and C++ costs time).

Alas, the standard DOM can do things that are not easy to replicate in JavaScript. For example, most DOM collections are live views on the current state of the DOM that change dynamically whenever the DOM changes. As a result, pure JavaScript implementations of the DOM are not very efficient. One of the reasons for adding proxies to JavaScript was to help write more efficient DOM implementations.

Accessing a restful web service

A proxy can be used to create an object on which arbitrary methods can be invoked. In the following example, the function createWebService creates one such object, service. Invoking a method on service retrieves the contents of the web service resource with the same name. Retrieval is handled via an ECMAScript 6 promise.

    let service = createWebService('http://example.com/data');
    // Read JSON data in http://example.com/data/employees
    service.employees().then(json => {
        let employees = JSON.parse(json);
        ···
    });

The following code is a quick and dirty implementation of createWebService in ECMAScript 5. Because we don’t have proxies, we need to know beforehand what methods will be invoked on service. The parameter propKeys provides us with that information, it holds an array with method names.

    function createWebService(baseUrl, propKeys) {
        let service = {};
        propKeys.forEach(function (propKey) {
            Object.defineProperty(service, propKey, {
                get: function () {
                    return httpGet(baseUrl+'/'+propKey);
                }
            });
        });
        return service;
    }

The ECMAScript 6 implementation of createWebService can use proxies and is simpler:

    function createWebService(baseUrl) {
        return new Proxy({}, {
            get(target, propKey, receiver) {
                return httpGet(baseUrl+'/'+propKey);
            }
        });
    }

Both implementations use the following function to make HTTP GET requests (how it works is explained in the 2ality blog post on promises).

    function httpGet(url) {
        return new Promise(
            (resolve, reject) => {
                let request = new XMLHttpRequest();
                Object.assign(request, {
                    onreadystatechange() {
                        if (this.status === 200) {
                            // Success
                            resolve(this.response);
                        } else {
                            // Something went wrong (404 etc.)
                            reject(new Error(this.statusText));
                        }
                    },
                    onerror() {
                        reject(new Error(
                            'XMLHttpRequest Error: '+this.statusText));
                    }
                });
                request.open('GET', url);
                request.send();    
            });
    }

Tracing property accesses

The example in this section is inspired by Brendan Eich’s talk “Proxies are Awesome”: We want to trace when a given set of properties is read or changed. To demonstrate how that works, let’s create a class for points and trace accesses to the properties of an instance.

    class Point {
        constructor(x, y) {
            this.x = x;
            this.y = y;
        }
        toString() {
            return 'Point('+this.x+','+this.y+')';
        }
    }
    // Trace accesses to properties `x` and `y`
    let p = new Point(5, 7);
    p = tracePropAccess(p, ['x', 'y']);

Getting and setting properties of p now has the following effects:

    > p.x
    GET x
    5
    > p.x = 21
    SET x=21
    21

Intriguingly, tracing also works whenever Point accesses the properties, because this now refers to the proxy, not to an instance of Point.

    > p.toString()
    GET x
    GET y
    'Point(21,7)'

In ECMAScript 5, you’d implement tracePropAccess() as follows. We replace each property with a getter and a setter that traces accesses. The setters and getters use an extra object, propData, to store the data of the properties. Note that we are destructively changing the original implementation, which means that we are meta programming.

    function tracePropAccess(obj, propKeys) {
        // Store the property data here
        let propData = Object.create(null);
        // Replace each property with a getter and a setter
        propKeys.forEach(function (propKey) {
            propData[propKey] = obj[propKey];
            Object.defineProperty(obj, propKey, {
                get: function () {
                    console.log('GET '+propKey);
                    return propData[propKey];
                },
                set: function (value) {
                    console.log('SET '+propKey+'='+value);
                    propData[propKey] = value;
                },
            });
        });
        return obj;
    }

In ECMAScript 6, we can use a simpler, proxy-based solution. We intercept property getting and setting and don’t have to change the implementation.

    function tracePropAccess(obj, propKeys) {
        let propKeySet = new Set(...propKeys);
        return new Proxy(obj, {
            get(target, propKey, receiver) {
                if (propKeySet.has(propKey)) {
                    console.log('GET '+propKey);
                }
                return Reflect.get(target, propKey, receiver);
            },
            set(target, propKey, value, receiver) {
                if (propKeySet.has(propKey)) {
                    console.log('SET '+propKey+'='+value);
                }
                return Reflect.set(target, propKey, value, receiver);
            },
        });
    }

Warning about unknown properties

When it comes to accessing properties, JavaScript is very forgiving. For example, if you try to read a property and misspell its name, you don’t get an exception, you get the result undefined. You can use proxies to get an exception in such a case. This works as follows. We make the proxy a prototype of an object.

If a property isn’t found in the object, the get trap of the proxy is triggered. If the property doesn’t even exist in the prototype chain after the proxy, it really is missing and we throw an exception. Otherwise, we return the value of the inherited property. We do so by forwarding the get operation to the target, whose prototype is the prototype of the proxy.

    let PropertyChecker = new Proxy({}, {
        get(target, propKey, receiver) {
            if (!(propKey in target)) {
                throw new ReferenceError('Unknown property: '+propKey);
            }
            return Reflect.get(target, propKey, receiver);
        }
    });

Let’s use PropertyChecker for an object that we create:

    > let obj = { __proto__: PropertyChecker, foo: 123 };
    > obj.foo  // own
    123
    > obj.fo
    ReferenceError: Unknown property: fo
    > obj.toString()  // inherited
    '[object Object]'

If we turn PropertyChecker into a constructor, we can use it for ECMAScript 6 classes via extends:

    function PropertyChecker() { }
    PropertyChecker.prototype = new Proxy(···);
    
    class Point extends PropertyChecker {
        constructor(x, y) {
            this.x = x;
            this.y = y;
        }
    }
    
    let p = new Point(5, 7);
    console.log(p.x); // 5
    console.log(p.z); // ReferenceError

If you are worried about accidentally creating properties, you have two options: You can either create a proxy that traps set. Or you can make an object obj non-extensible via Object.preventExtensions(obj), which means that JavaScript doesn’t let you add new (own) properties to obj.

Negative array indices

Some array methods let you refer to the last element via -1, to the second-to-last element via -2, etc. For example:

    > ['a', 'b', 'c'].slice(-1)
    [ 'c' ]

Alas, that doesn’t work when accessing elements via the bracket operator ([]). We can, however, use proxies to add that capability. The following function createArray() creates arrays that support negative indices. It does so by wrapping proxies around array instances. The proxies intercept the get operation that is triggered by the bracket operator.

    function createArray(...elements) {
        let handler = {
            get(target, propKey, receiver) {
                let index = Number(propKey);
                // Sloppy way of checking for negative indices
                if (index < 0) {
                    propKey = String(target.length + index);
                }
                return Reflect.get(target, propKey, receiver);
            }
        };
        // Wrap a proxy around an array
        let target = [];
        target.push(...elements);
        return new Proxy(target, handler);
    }
    let arr = createArray('a', 'b', 'c');
    console.log(arr[-1]); // c

Acknowledgement: The idea for this example comes from a blog post by hemanth.hm.

Data binding

Data binding is about syncing data between objects. One popular use case are widgets based on the MVC (Model View Controler) pattern: With data binding, the view (the widget) stays up-to-date if you change the model (the data visualized by the widget).

To implement data binding, you have to observe and react to changes made to an object. In the following code snippet, I sketch how that could work for an array.

    let array = [];
    let observedArray = new Proxy(array, {
        set(target, propertyKey, value, receiver) {
            console.log(propertyKey+'='+value);
            target[propertyKey] = value;
        }
    });
    observedArray.push('a');

Output:

    0=a
    length=1

Data binding is a complex topic. Given its popularity and concerns over proxies not being performant enough, a dedicated mechanism has been created for data binding: Object.observe(). It will probably be part of ECMAScript 7 and is already supported by Chrome.

Consult Addy Osmani’s article “Data-binding Revolutions with Object.observe()” for more information on Object.observe().

Revocable references

Revocable references work as follows: A client is not allowed to access an important resource (an object) directly, only via a reference (an intermediate object, a wrapper around the resource). Normally, every operation applied to the reference is forwarded to the resource. After the client is done, the resource is protected by revoking the reference, by switching it off. Henceforth, applying operations to the reference throws exceptions and nothing is forwarded, anymore.

In the following example, we create a revocable reference for a resource. We then read one of the resource’s properties via the reference. That works, because the reference grants us access. Next, we revoke the reference. Now the reference doesn’t let us read the property, anymore.

    let resource = { x: 11, y: 8 };
    let {reference, revoke} = createRevocableReference(resource);
    
    // Access granted
    console.log(reference.x); // 11
    
    revoke();
    
    // Access denied
    console.log(reference.x); // TypeError: Revoked

Proxies are ideally suited for implementing revocable references, because they can intercept and forward operations. This is a simple proxy-based implementation of createRevocableReference:

    function createRevocableReference(target) {
        let enabled = true;
        return {
            reference: new Proxy(target, {
                get(target, propKey, receiver) {
                    if (!enabled) {
                        throw new TypeError('Revoked');
                    }
                    return Reflect.get(target, propKey, receiver);
                },
                has(target, propKey) {
                    if (!enabled) {
                        throw new TypeError('Revoked');
                    }
                    return Reflect.has(target, propKey);
                },
                ···
            }),
            revoke() {
                enabled = false;
            },
        };
    }

The code can be simplified via the proxy-as-handler technique from the previous section. This time, the handler basically is the Reflect object. Thus, the get trap normally returns the appropriate Reflect method. If the reference has been revoked, a TypeError is thrown, instead.

    function createRevocableReference(target) {
        let enabled = true;
        let handler = new Proxy({}, {
            get(dummyTarget, trapName, receiver) {
                if (!enabled) {
                    throw new TypeError('Revoked');
                }
                return Reflect[trapName];
            }
        });
        return {
            reference: new Proxy(target, handler),
            revoke() {
                enabled = false;
            },
        };
    }

However, you don’t have to implement revocable references yourself, because ECMAScript 6 lets you create proxies that can be revoked. This time, the revoking happens in the proxy, not in the handler. All the handler has to do is forward every operation to the target. As we have seen that happens automatically if the handler doesn’t implement any traps.

    function createRevocableReference(target) {
        let handler = {}; // forward everything
        let { proxy, revoke } = Proxy.revocable(target, handler);
        return { reference: proxy, revoke };
    }
Membranes

Membranes build on the idea of revocable references: Environments that are designed to run untrusted code wrap a membrane around that code to isolate it and keep the rest of the system safe. Objects pass the membrane in two directions:

  • The code may receive objects from the outside.
  • Or it may hand objects to the outside.

In both cases, revocable references are wrapped around the objects. Objects returned by wrapped functions or methods are also wrapped.

Once the untrusted code is done, all of those references are revoked. As a result, none of its code on the outside can be executed anymore and outside objects that it has cease to work, as well. The Caja Compiler is “a tool for making third party HTML, CSS and JavaScript safe to embed in your website”. It uses membranes to achieve this task.

Other use cases

There are more use cases for proxies. For example:

  • Local placeholders that forward method invocations to remote objects. Similar: web service example.
  • Data access objects for databases: reading and writing to the object reads and writes to the database. Similar: web service example.
  • Profiling: Intercept method invocations to track how much time is spent in each method. Similar: tracing example.
  • Type checking: Nicholas Zakas has used proxies to type-check objects.

The design of the proxy API

In this section, we go deeper into how proxies work and why they work that way.

Stratification: keeping base level and meta level separate

Firefox has allowed you to do some interceptive meta programming for a while: If you define a method whose name is __noSuchMethod__, it is notified whenever a method is called that doesn’t exist. The following is an example of using __noSuchMethod__.

    let obj = {
        __noSuchMethod__: function (name, args) {
            console.log(name+': '+args);
        }
    };
    // Neither of the following two methods exist,
    // but we can make it look like they do
    obj.foo(1);    // Output: foo: 1
    obj.bar(1, 2); // Output: bar: 1,2

Thus, __noSuchMethod__ works similarly to a proxy trap. In contrast to proxies, the trap is an own or inherited method of the object whose operations we want to intercept. The problem with that approach is that base level and meta level are mixed. Base-level code may accidentally invoke or see a meta level method and there is the possibility of accidentally defining a meta level method.

Even in standard ECMAScript 5, base level and meta level are sometimes mixed. For example, the following meta programming mechanisms can fail, because they exist at the base level:

  • obj.hasOwnProperty(propKey): This call can fail if a property in the prototype chain overrides the built-in implementation. For example, it fails if obj is { hasOwnProperty: null }. Safe ways to call this method are Object.prototype.hasOwnProperty.call(obj, propKey) and its abbreviated version {}.hasOwnProperty.call(obj, propKey).
  • func.call(···), func.apply(···): For these two methods, problem and solution are the same as with hasOwnProperty.
  • obj.__proto__: In most JavaScript engines, __proto__ is a special property that lets you get and set the prototype of obj. Hence, when you use objects as dictionaries, you must be careful to avoid __proto__ as a property key.

By now, it should be obvious that making (base level) property keys special is problematic. Therefore, proxies are stratified – base level (the proxy object) and meta level (the handler object) are separate.

Virtual objects versus wrappers

Proxies are used in two roles:

  • As wrappers, they wrap their targets, they control access to them. Examples of wrappers are: revocable resources and tracing proxies.

  • As virtual objects, they are simply objects with special behavior and their targets don’t matter. An example is a proxy that forwards method calls to a remote object.

An earlier design of the proxy API conceived proxies as purely virtual objects. However, it turned out that even in that role, a target was useful, to enforce invariants (which is explained later) and as a fallback for traps that the handler doesn’t implement.

Transparent virtualization and handler encapsulation

Proxies are shielded in two ways:

  • It is impossible to determine whether an object is a proxy or not (transparent virtualization).
  • You can’t access a handler via its proxy (handler encapsulation).

Both principles give proxies considerable power for impersonating other objects. One reason for enforcing invariants (as explained later) is to keep that power in check.

If you do need a way to tell proxies apart from non-proxies, you have to implement it yourself. The following code is a module lib.js that exports two functions: one of them creates proxies, the other one determines whether an object is one of those proxies.

    // lib.js
    
    let proxies = new WeakSet();
    
    export function createProxy(obj) {
        let handler = {};
        let proxy = new Proxy(obj, handler);
        proxies.add(proxy);
        return proxy;
    }
    
    export function isProxy(obj) {
        return proxies.has(obj);
    }

This module uses the ECMAScript 6 data structure WeakSet for keeping track of proxies. WeakSet is ideally suited for this purpose, because it doesn’t prevent its elements from being garbage-collected.

The next example shows how lib.js can be used.

    // main.js
    
    import { createProxy, isProxy } from './lib.js';
    
    let p = createProxy({});
    console.log(isProxy(p)); // true
    console.log(isProxy({})); // false

The meta object protocol and proxy traps

This section examines how JavaScript is structured internally and how the set of proxy traps was chosen.

The term protocol is highly overloaded in computer science. One definition is:

A prototcol is about achieving tasks via an object, it comprises a set of methods plus a set of rules for using them.

Note that this definition is different from viewing protocols as interfaces (as, for example, Objective C does), because it includes rules.

The ECMAScript specification describes how to execute JavaScript code. It includes a protocol for handling objects. This protocol operates at a meta level and is sometimes called the meta object protocol (MOP). The JavaScript MOP consists of own internal methods that all objects have. “Internal” means that they exist only in the specification (JavaScript engines may or may not have them) and are not accessible from JavaScript. The names of internal methods are written in double square brackets.

The internal method for getting properties is called [[Get]]. If we pretend that property names with square brackets are legal, this method would roughly be implemented as follows in JavaScript.

    // Method definition
    [[Get]](propKey, receiver) {
        let desc = this.[[GetOwnProperty]](propKey);
        if (desc === undefined) {
            let parent = this.[[GetPrototypeOf]]();
            if (parent === null) return undefined;
            return parent.[[Get]](propKey, receiver); // (*)
        }
        if ('value' in desc) {
            return desc.value;
        }
        let getter = desc.get;
        if (getter === undefined) return undefined;
        return getter.[[Call]](receiver, []);
    }

The MOP methods called in this code are:

  • [[GetOwnProperty]] (trap getOwnPropertyDescriptor)
  • [[GetPrototypeOf]] (trap getPrototypeOf)
  • [[Get]] (trap get)
  • [[Call]] (trap apply)

In line (*) you can see why proxies in a prototype chain find out about get if a property isn’t found in an “earlier” object: If there is no own property whose key is propKey, the search continues in the prototype parent of this.

Fundamental versus derived operations. You can see that [[Get]] calls other MOP operations. Operations that do that are called derived. Operations that don’t depend on other operations are called fundamental.

The MOP of proxies

The meta object protocol of proxies is different from that of normal objects. For normal objects, derived operations call other operations. For proxies, each operation is either intercepted by a handler method or forwarded to the target.

What operations should be interceptable via proxies? One possibility is to only provide traps for fundamental operations. The alternative is to include some derived operations. The advantage of derived traps is that they increase performance and are more convenient: If there wasn’t a trap for get, you’d have to implement its functionality via getOwnPropertyDescriptor. One problem with derived traps is that they can lead to proxies behaving inconsistently. For example, get may return a value that is different from the value stored in the descriptor returned by getOwnPropertyDescriptor.

Selective intercession: what operations should be interceptable?

Intercession by proxies is selective: you can’t intercept every language operation. Why were some operations excluded? Let’s look at two reasons.

First, stable operations are not well suited for intercession. An operation is stable if it always produces the same results for the same arguments. If a proxy can trap a stable operation, it can become unstable and thus unreliable. Strict equality (===) is one such stable operation. It can’t be trapped and its result is computed by treating the proxy itself as just another object. Another way of maintaining stability is by applying an operation to the target instead of the proxy. As explained later, when we look at how invariants are enfored for proxies, this happens when Object.getPrototypeOf() is applied to a proxy whose target is non-extensible.

A second reason for not making more operations interceptable is that intercession means executing custom code in situations where that normally isn’t possible. The more this interleaving of code happens, the harder it is to understand and debug a program.

Traps: “get” versus “invoke”

If you want to create virtual methods via ECMAScript 6 proxies, you have to return functions from a get trap. That raises the question: why not introduce an extra trap for method invocations (e.g. invoke)? That would enable us to distinguish between:

  • Getting properties via obj.prop (trap get)
  • Invoking methods via obj.prop() (trap invoke)

There are two reasons for not doing so.

First, not all implementations distinguish between get and invoke. For example, Apple’s JavaScriptCore doesn’t.

Second, extracting a method and invoking it later via call() or apply() should have the same effect as invoking the method via dispatch. In other words, the following two variants should work equivalently. If there was an extra trap invoke then that equivalence would be harder to maintain.

    // Variant 1: call via dynamic dispatch
    let result = obj.m();
    
    // Variant 2: extract and call directly
    let m = obj.m;
    let result = m.call(obj);

Only possible with invoke. Some things can only be done if you are able to distinguish between get and invoke. Those things are therefore impossible with the current proxy API. Two examples are: auto-binding and intercepting missing methods.

First, by making a proxy the prototype of an object obj, you can automatically bind methods:

  • Retrieving the value of a method m via obj.m returns a function whose this is bound to obj.
  • obj.m() performs a method call.

Auto-binding helps with using methods as callbacks. For example, variant 2 from the previous example becomes simpler:

    let boundMethod = obj.m;
    let result = boundMethod();

Second, invoke lets a proxy emulate the previously mentioned __noSuchMethod__ mechanism that Firefox supports. The proxy would again become the prototype of an object obj. It would react differently depending on how an unknown property foo is accessed:

  • If you read that property via obj.foo, no intercession happens and undefined is returned.
  • If you make the method call obj.foo() then the proxy intercepts and, e.g., notifies a callback.

Enforcing invariants for proxies

Before we look at what invariants are and how they are enforced for proxies, let’s review how objects can be protected via non-extensibility and non-configurability.

Protecting objects

There are two ways of protecting objects:

  • non-extensibility protects objects
  • non-configurability protects properties (or rather, their attributes)

Non-extensibility. If an object is non-extensible, you can’t add properties and you can’t change its prototype:

    'use strict'; // switch on strict mode to get TypeErrors
    
    let obj = Object.preventExtensions({});
    console.log(Object.isExtensible(obj)); // false
    obj.foo = 123; // TypeError: object is not extensible
    Object.setPrototypeOf(obj, null); // TypeError: object is not extensible

Non-configurability. All the data of a property is stored in attributes. A property is like a record and attributes are like the fields of that record. Examples of attributes:

  • The attribute value holds the value of a property.
  • The boolean attribute writable controls whether a property’s value can be changed.
  • The boolean attribute configurable controls whether a property’s attributes can be changed.

Thus, if a property is both non-writable and non-configurable, it is read-only and remains that way:

    'use strict'; // switch on strict mode to get TypeErrors
    
    let obj = {};
    Object.defineProperty(obj, 'foo', {
        value: 123,
        writable: false,
        configurable: false
    });
    console.log(obj.foo); // 123
    obj.foo = 'a'; // TypeError: Cannot assign to read only property
    
    Object.defineProperty(obj, 'foo', {
        configurable: true
    }); // TypeError: Cannot redefine property

For more details on these topics (including how Object.defineProperty() works) consult the following sections in “Speaking JavaScript”:

Enforcing invariants

Traditionally, non-extensibility and non-configurability are:

  • Universal: they work for all objects.
  • Monotonic: once switched on, they can’t be switched off again.

These and other characteristics that remain unchanged in the face of language operations are called invariants. With proxies, it is easy to violate invariants, as they are not intrinsically bound by non-extensibility etc.

The proxy API prevents proxies from violating invariants by checking the parameters and results of handler methods. Non-extensibility and non-configurability are enforced by using the target object for bookkeeping. The following are a few examples of invariants (for an arbitrary object obj) and how they are enforced for proxies (an exhaustive list is given at the end of this post):

  • Invariant: Object.isExtensible(obj) must return a boolean.
    • Enforced by coercing the value returned by the handler to a boolean.
  • Invariant: Object.getOwnPropertyDescriptor(obj, ···) must return an object or undefined.
    • Enforced by throwing a TypeError if the handler doesn’t return an appropriate value.
  • Invariant: If Object.preventExtensions(obj) returns true then all future calls must return false and obj must now be non-extensible.
    • Enforced by throwing a TypeError if the handler returns true, but the target object is not extensible.
  • Invariant: Once an object has been made non-extensible, Object.isExtensible(obj) must always return false.
    • Enforced by throwing a TypeError if the result returned by the handler is not the same (after coercion) as Object.isExtensible(target).

Enforcing invariants has the following benefits:

  • Proxies work like all other objects with regard to extensibility and configurability. Therefore, universality is maintained. This is achieved without preventing proxies from virtualizing (impersonating) protected objects.
  • A protected object can’t be misrepresented by wrapping a proxy around it. Misrepresentation can be caused by bugs or by malicious code.

The following sections give examples of invariants being enforced.

Example: the prototype of a non-extensible target must be represented faithfully

In response to the getPrototypeOf trap, the proxy must return the target’s prototype if the target is non-extensible.

To demonstrate this invariant, let’s create a handler that returns a prototype that is different from the target’s prototype:

    let fakeProto = {};
    let handler = {
        getPrototypeOf(t) {
            return fakeProto;
        }
    };

Faking the prototype works if the target is extensible:

    let extensibleTarget = {};
    let ext = new Proxy(extensibleTarget, handler);
    console.log(Object.getPrototypeOf(ext) === fakeProto); // true

We do, however, get an error if we fake the prototype for a non-extensible object.

    let nonExtensibleTarget = {};
    Object.preventExtensions(nonExtensibleTarget);
    let nonExt = new Proxy(nonExtensibleTarget, handler);
    Object.getPrototypeOf(nonExt); // TypeError
Example: non-writable non-configurable target properties must be represented faithfully

If the target has a non-writable non-configurable property then the handler must return that property’s value in response to a get trap. To demonstrate this invariant, let’s create a handler that always returns the same value for properties.

    let handler = {
        get(target, propKey) {
            return 'abc';
        }
    };
    let target = Object.defineProperties(
        {}, {
            foo: {
                value: 123,
                writable: true,
                configurable: true
            },
            bar: {
                value: 456,
                writable: false,
                configurable: false
            },
        });
    let proxy = new Proxy(target, handler);

Property target.foo is not both non-writable and non-configurable, which means that the handler is allowed to pretend that it has a different value:

    > proxy.foo
    'abc'

However, property target.bar is both non-writable and non-configurable. Therefore, we can’t fake its value:

    > proxy.bar
    TypeError: Invariant check failed

Reference: the proxy API

This section serves as a quick reference for the proxy API: the global objects Proxy and Reflect.

Creating proxies

There are two ways to create proxies:

  • proxy = new Proxy(target, handler)
    Creates a new proxy object with the given target and the given handler.

  • {proxy, revoke} = Proxy.revocable(target, handler)
    Creates a proxy that can be revoked via the function revoke. revoke can be called multiple times, but only the first call has an effect and switches proxy off. Afterwards, any operation performed on proxy leads to a TypeError being thrown.

Handler methods

This subsection explains what traps can be implemented by handlers and what operations trigger them. Several traps return boolean values. For the traps has and isExtensible, the boolean is the result of the operation. For all other traps, the boolean indicates whether the operation succeeded or not.

Traps for all objects:

  • defineProperty(target, propKey, propDesc)boolean
    • Object.defineProperty(proxy, propKey, propDesc)
  • deleteProperty(target, propKey)boolean
    • delete proxy[propKey]
    • delete proxy.foo // propKey = 'foo'
  • enumerate(target)Iterator
    • for (x in proxy) ···
  • get(target, propKey, receiver)any
    • receiver[propKey]
    • receiver.foo // propKey = 'foo'
  • getOwnPropertyDescriptor(target, propKey)PropDesc|Undefined
    • Object.getOwnPropertyDescriptor(proxy, propKey)
  • getPrototypeOf(target)Object|Null
    • Object.getPrototypeOf(proxy)
  • has(target, propKey)boolean
    • propKey in proxy
  • isExtensible(target)boolean
    • Object.isExtensible(proxy)
  • ownKeys(target)Array
    • Object.getOwnPropertyPropertyNames(proxy)
    • Object.getOwnPropertyPropertySymbols(proxy)
    • Object.keys(proxy)
  • preventExtensions(target)boolean
    • Object.preventExtensions(proxy)
  • set(target, propKey, value, receiver)boolean
    • receiver[propKey] = value
    • receiver.foo = value // propKey = 'foo'
  • setPrototypeOf(target, proto)boolean
    • Object.setPrototypeOf(proxy, proto)

Traps for functions (available if target is a function):

  • apply(target, thisArgument, argumentsList)any
    • proxy.apply(thisArgument, argumentsList)
    • proxy.call(thisArgument, ...argumentsList)
    • proxy(...argumentsList)
  • construct(target, argumentsList)Object
    • new proxy(..argumentsList)
Fundamental operations versus derived operations

The following operations are fundamental, they don’t use other operations to do their work: apply, defineProperty, deleteProperty, getOwnPropertyDescriptor, getPrototypeOf, isExtensible, ownKeys, preventExtensions, setPrototypeOf

All other operations are derived, they can be implemented via fundamental operations. For example, for data properties, get can be implemented by iterating over the prototype chain via getPrototypeOf and calling getOwnPropertyDescriptor for each chain member until either an own property is found or the chain ends.

Invariants

Invariants are safety constraints for handlers. This subsection documents what invariants are enforced by the proxy API and how. Whenever you read “the handler must do X” below, it means that a TypeError is thrown if it doesn’t. Some invariants restrict return values, others restrict parameters. Ensuring the correct return value of a trap is ensured in two ways: Normally, an illegal value means that a TypeError is thrown. But whenever a boolean is expected, coercion is used to convert non-booleans to legal values.

This is the complete list of invariants that are enforced (source: ECMAScript 6 specification):

  • apply(target, thisArgument, argumentsList)
    • No invariants are enforced.
  • construct(target, argumentsList)
    • The result returned by the handler must be an object (not null or a primitive value).
  • defineProperty(target, propKey, propDesc)
    • If the target is not extensible then propDesc can’t create a property that the target doesn’t already have.
    • If propDesc sets the attribute configurable to false then the target must have a non-configurable own property whose key is propKey.
    • If propDesc was used to (re)define an own property for the target then that must not cause an exception. An exception is thrown if a change is forbidden by the attributes writable and configurable.
  • deleteProperty(target, propKey)
    • Non-configurable own properties of the target can’t be deleted.
  • enumerate(target)
    • The handler must return an object.
  • get(target, propKey, receiver)
    • If the target has an own, non-writable, non-configurable data property whose key is propKey then the handler must return that property’s value.
    • If the target has an own, non-configurable, getter-less accessor property then the handler must return undefined.
  • getOwnPropertyDescriptor(target, propKey)
    • The handler must return either an object or undefined.
    • Non-configurable own properties of the target can’t be reported as non-existent by the handler.
    • If the target is non-extensible then exactly the target’s own properties must be reported by the handler as existing (and none of them as missing).
    • If the handler reports a property as non-configurable then that property must be a non-configurable own property of the target.
    • If the result returned by the handler were used to (re)define an own property for the target then that must not cause an exception. An exception is thrown if the change is not allowed by the attributes writable and configurable. Therefore, the handler can’t report a non-configurable property as configurable and it can’t report a different value for a non-configurable non-writable property.
  • getPrototypeOf(target)
    • The result must be either an object or null.
    • If the target object is not extensible then the handler must return the prototype of the target object.
  • has(target, propKey)
    • A handler must not hide (report as non-existent) a non-configurable own property of the target.
    • If the target is non-extensible then no own property of the target may be hidden.
  • isExtensible(target)
    • The result returned by the handler is coerced to boolean.
    • After coercion to boolean, the value returned by the handler must be the same as target.isExtensible().
  • ownKeys(target)
    • The handler must return an object, which treated as array-like and converted into an array.
    • Each element of the result must be either a string or a symbol.
    • The result must contain the keys of all non-configurable own properties of the target.
    • If the target is not extensible then the result must contain exactly the keys of the own properties of the target (and no other values).
  • preventExtensions(target)
    • The result returned by the handler is coerced to boolean.
    • If the handler returns a truthy value (indicating a successful change) then target.isExtensible() must be false afterwards.
  • set(target, propKey, value, receiver)
    • If the target has an own, non-writable, non-configurable data property whose key is propKey then value must be the same as the value of that property (i.e., the property can’t be changed).
    • If the target has an own, non-configurable, setter-less accessor property then a TypeError is thrown (i.e., such a property can’t be set).
  • setPrototypeOf(target, proto)
    • The result returned by the handler is coerced to boolean.
    • If the target is not extensible, the prototype can’t be changed. This is enforced as follows: If the target is not extensible and the handler returns a truthy value (indicating a successful change) then proto must be the same as the prototype of the target. Otherwise, a TypeError is thrown.

The prototype chain

The following operations of normal objects perform operations on objects in the prototype chain (source: ECMAScript 6 specification). Therefore, if one of the objects in that chain is a proxy, its traps are triggered. The specification implements the operations as internal own methods (that are not visible to JavaScript code). But in this section, we pretend that they are normal methods that have the same names as the traps. The parameter target becomes the receiver of the method call.

  • target.enumerate()
    Traverses the prototype chain of target via getPrototypeOf. Per object, it retrieves the keys via ownKeys and examines whether a property is enumerable via getOwnPropertyDescriptor.
  • target.get(propertyKey, receiver)
    If target has no own property with the given key, get is invoked on the prototype of target.
  • target.has(propertyKey)
    Similarly to get, has is invoked on the prototype of target if target has no own property with the given key.
  • target.set(propertyKey, value, receiver)
    Similarly to get, set is invoked on the prototype of target if target has no own property with the given key.

All other operations only affect own properties, they have no effect on the prototype chain.

Reflect

The global object Reflect implements all interceptable operations of the JavaScript meta object protocol as methods. The names of those methods are the same as those of the handler methods, which, as we have seen, helps with forwarding operations from the handler to the target.

  • Reflect.apply(target, thisArgument, argumentsList)any
    Better version of Function.prototype.apply().
  • Reflect.construct(target, argumentsList)Object
    The new operator as a function.
  • Reflect.defineProperty(target, propertyKey, propDesc)boolean
    Similar to Object.defineProperty().
  • Reflect.deleteProperty(target, propertyKey)boolean
    The delete operator as a function.
  • Reflect.enumerate(target)Iterator
    Returns an iterater over all enumerable string property keys of target. In other words, the iterator returns all values that the for-in loop would iterate over.
  • Reflect.get(target, propertyKey, receiver?)any
    A function that gets properties.
  • Reflect.getOwnPropertyDescriptor(target, propertyKey)PropDesc|Undefined
    Same as Object.getOwnPropertyDescriptor().
  • Reflect.getPrototypeOf(target)Object|Null
    Same as Object.getPrototypeOf().
  • Reflect.has(target, propertyKey)boolean
    The in operator as a function.
  • Reflect.isExtensible(target)boolean
    Same as Object.isExtensible().
  • Reflect.ownKeys(target)Array
    Returns all own property keys (strings and symbols!) in an array.
  • Reflect.preventExtensions(target)boolean
    Similar to Object.preventExtensions().
  • Reflect.set(target, propertyKey, value, receiver?)boolean
    A function that sets properties.
  • Reflect.setPrototypeOf(target, proto)boolean
    The new standard way of setting the prototype of an object. The current non-standard way that works in most engines is to set the special property __proto__.

Several methods have boolean results. For has and isExtensible, they are the results of the operation. For the remaining methods, they indicate whether the operation succeeded.

Apart from forwarding operations, why is Reflect useful [4]?

  • Different return values: Reflect duplicates the following methods of Object, but its methods return booleans indicating whether the operation succeeded (where the Object methods return the object that was modified).
    • Object.defineProperty(obj, propKey, propDesc)Object
    • Object.preventExtensions(obj)Object
    • Object.setPrototypeOf(obj, proto)Object
  • Operators as functions: The following Reflect methods implement functionality that is otherwise only available via operators:
    • Reflect.construct(target, argumentsList)Object
    • Reflect.deleteProperty(target, propertyKey)boolean
    • Reflect.get(target, propertyKey, receiver?)any
    • Reflect.has(target, propertyKey)boolean
    • Reflect.set(target, propertyKey, value, receiver?)boolean
  • The for-in loop as an iterator: This is rarely useful, but if you need it, you can get an iterator over all enumerable (own and inherited) string property keys of an object.
    • Reflect.enumerate(target)Iterator
  • Shorter version of apply: The only safe way to invoke the built-in function method apply is via Function.prototype.apply.call(func, thisArg, args) (or similar). Reflect.apply(func, thisArg, args) is cleaner and shorter.

State of implementations

As usual, Kangax’ ES6 compatibility table is the best way of finding out how well engines support proxies. As of December 2014, Internet Explorer has the most complete support and Firefox supports some of the API (caveats: get doesn’t work properly, getPrototypeOf is not supported yet and Reflect is empty). No other browser or engine currently supports proxies.

Conclusion

This concludes our in-depth look at the proxy API. For each application, you have to take performance into consideration and – if necessary – measure. Proxies may not always be fast enough. On the other hand, performance is often not crucial and it is nice to have the meta programming power that proxies give us. As we have seen, there are numerous use cases they can help with.

Acknowledgements

Thanks go to Tom Van Cutsem: his paper [1] is the most important source of this blog post and he kindly answered questions about the proxy API that I had.

Technical Reviewers:

Further reading

  1. On the design of the ECMAScript Reflection API” by Tom Van Cutsem and Mark Miller. Technical report, 2012.
  2. The Art of the Metaobject Protocol” by Gregor Kiczales, Jim des Rivieres and Daniel G. Bobrow. Book, 1991.
  3. Putting Metaclasses to Work: A New Dimension in Object-Oriented Programming” by Ira R. Forman and Scott H. Danforth. Book, 1999.
  4. Harmony-reflect: Why should I use this library?” by Tom Van Cutsem.

Source:: 2ality

One JavaScript: avoiding versioning in ECMAScript 6

By Axel Rauschmayer

What is the best way to add new features to a language? This blog post describes the approach taken by ECMAScript 6 [3], the next version of JavaScript. It is called One JavaScript, because it avoids versioning.

Versioning

In principle, a new version of a language is a chance to clean it up, by removing outdated features or by changing how features work. That means that new code doesn’t work in older implementations of the language and that old code doesn’t work in a new implementation. Each piece of code is linked to a specific version of the language. Two approaches are common for dealing with versions being different.

First, you can take an “all or nothing” approach and demand that, if a code base wants to use the new version, it must be upgraded completely. Python took that approach when upgrading from Python 2 to Python 3. A problem with it is that it may not be feasible to migrate all of an existing code base at once, especially if it is large. Furthermore, the approach is not an option for the web, where you’ll always have old code and where JavaScript engines are updated automatically.

Second, you can permit a code base to contain code in multiple versions, by tagging code with versions. On the web, you could tag ECMAScript 6 code via a dedicated Internet media type. Such a media type can be associated with a file via an HTTP header:

    Content-Type: application/ecmascript;version=6

It can also be associated via the type attribute of the element (whose default value is text/javascript):

    <script type="application/ecmascript;version=6">
        ···
    </script>

This specifies the version out of band, externally to the actual content. Another option is to specify the version inside the content (in-band). For example, by starting a file with the following line:

    use version 6;

Both ways of tagging are problematic: out-of-band versions are brittle and can get lost, in-band versions add clutter to code.

A more fundamental issue is that allowing multiple versions per code base effectively forks a language into sub-languages that have to be maintained in parallel. This causes problems:

  • Engines become bloated, because they need to implement the semantics of all versions. The same applies to tools analyzing the language (e.g. style checkers such es JSLint).
  • Programmers need to remember how the versions differ.
  • Code becomes harder to refactor, because you need to take versions into consideration when you move pieces of code.

Therefore, versioning is something to avoid, especially for JavaScript and the web.

Evolution without versioning

But how can we get rid of versioning? By always being backwards-compatible. That means we must give up some of our ambitions w.r.t. cleaning up JavaScript:

  • We can’t introduce breaking changes: Being backwards-compatible means not removing features and not changing features. The slogan for this principle is: “don’t break the web”.
  • We can, however, add new features and make existing features more powerful.

As a consequence, no versions are needed for new engines, because they can still run all old code. David Herman calls this approach to avoiding versioning One JavaScript (1JS) [1], because it avoids splitting up JavaScript into different versions or modes. As we shall see later, 1JS even undoes some of a split that already exists, due to strict mode.

Supporting new code on old engines is more complicated. You have to detect in the engine what version of the language it supports. If it doesn’t support the latest version, you have to load different code: your new code compiled to an older version. That is how you can already use ECMAScript 6 in current engines: you compile it to ECMAScript 5 [3]. Apart from performing the compilation step ahead of time, you also have the option of compiling in the engine, at runtime.

Detecting versions is difficult, because many engines support parts of versions before they support them completely. For example, this is how you’d check whether an engine supports ECMAScript 6’s for-of loop – but that may well be the only ES6 feature it supports:

    function isForOfSupported() {
        try {
            eval("for (var e of ['a']) {}");
            return true;
        } catch (e) {
            // Possibly: check if e instanceof SyntaxError
        }
        return false;
    }

Mark Miller describes how the Caja library detects whether an engine supports ECMAScript 5. He expects detection of ECMAScript 6 to work similarly, eventually.

One JavaScript does not mean that you have to completely give up on cleaning up the language. Instead of cleaning up existing features, you introduce new, clean, features. One example for that is let, which declares block-scoped variables and is an improved version of var. It does not, however, replace var, it exists alongside it, as the superior option.

One day, it may even be possible to eliminate features that nobody uses, anymore. Some of the ES6 features were designed by surveying JavaScript code on the web. Two examples (that are explained in more detail later) are:

  • let is available in non-strict mode, because let[x] rarely appears on the web.
  • Function declarations do occasionally appear in non-strict blocks, which is why the ES6 specification describes measures that web browsers can take to ensure that such code doesn’t break.

Strict mode and ECMAScript 6

Strict mode was introduced in ECMAScript 5 to clean up the language. It is switched on by putting the following line first in a file or in a function:

    'use strict';

Strict mode introduces three kinds of breaking changes:

  • Syntactic changes: some previously legal syntax is forbidden in strict mode. For example:
    • The with statement is forbidden. It lets users add arbitrary objects to the chain of variable scopes, which slows down execution and makes it tricky to figure out what a variable refers to.
    • Deleting an unqualified identifier (a variable, not a property) is forbidden.
    • Functions can only be declared at the top level of a scope.
    • More identifiers are reserved: implements interface let package private protected public static yield
  • More errors. For example:
    • Assigning to an undeclared variable causes a ReferenceError. In sloppy mode, a global variable is created in this case.
    • Changing read-only properties (such as the length of a string) causes a TypeError. In non-strict mode, it simply has no effect.
  • Different semantics: Some constructs behave differently in strict mode. For example:
    • arguments doesn’t track the current values of parameters, anymore.
    • this is undefined in non-method functions. In sloppy mode, it refers to the global object (window), which meant that global variables were created if you called a constructor without new.

Strict mode is a good example of why versioning is tricky: Even though it enables a cleaner version of JavaScript, its adoption is still relatively low. The main reasons are that it breaks some existing code, can slow down execution and is a hassle to add to files (let alone interactive command lines). I love the idea of strict mode and don’t nearly use it often enough.

Supporting sloppy mode

One JavaScript means that we can’t give up on sloppy mode: it will continue to be around (e.g. in HTML attributes). Therefore, we can’t build ECMAScript 6 on top of strict mode, we must add its features to both strict mode and non-strict mode (a.k.a. sloppy mode). Otherwise, strict mode would be a different version of the language and we’d be back to versioning. Unfortunately, two ECMAScript 6 features are difficult to add to sloppy mode: let declarations and block-level function declarations. Let’s examine why that is and how to add them, anyway.

let declarations in sloppy mode

let enables you to declare block-scoped variables. It is difficult to add to sloppy mode, because let is only a reserved word in strict mode. That is, the following two statements are legal in ECMAScript 5 sloppy mode:

    var let = [];
    let[0] = 'abc';

In strict ECMAScript 6, you get an exception in line 1, because you are using the reserved word let as a variable name. And the statement in line 2 is interpreted as a let variable declaration.

In sloppy ECMAScript 6, the first line does not cause an exception, but the second line is still interpreted as a let declaration. The pattern in that line is rare enough that ES6 can afford to make this interpretation. Other ways of writing let declarations can’t be mistaken for existing sloppy syntax:

    let foo = 123;
    let {x,y} = computeCoordinates();

Block-level function declarations in sloppy mode

ECMAScript 5 strict mode forbids function declarations in blocks. The specification allowed them in sloppy mode, but didn’t specify how they should behave. Hence, various implementations of JavaScript support them, but handle them differently.

ECMAScript 6 wants a function declaration in a block to be local to that block. That is OK as an extension of ES5 strict mode, but breaks some sloppy code. Therefore, ES6 provides “web legacy compatibility semantics” for browsers that lets function declarations in blocks exist at function scope.

Other keywords

The identifiers yield and static are only reserved in ES5 strict mode. ECMAScript 6 uses context-specific syntax rules to make them work in sloppy mode:

  • In sloppy mode, yield is only a reserved word inside a generator function.
  • static is currently only used inside class literals, which are implicitly strict (see below).

Implicit strict mode

The bodies of modules and classes are implicitly in strict mode in ECMAScript 6 – there is no need for the 'use strict' marker. Given that virtually all of our code will live in modules in the future, ECMAScript 6 effectively upgrades the whole language to strict mode.

The bodies of other constructs (such as arrow functions and generator functions) could have been made implicitly strict, too. But given how small these constructs usually are, using them in sloppy mode would have resulted in code that is fragmented between the two modes. Classes and especially modules are large enough to make fragmentation less of an issue.

It is interesting to note that, inside a element, you can’t declaratively import modules via an import statement. Instead, there will be a new element, which may be called , whose insides are much like a module [2]: Modules can be imported asynchronously and code is implicitly strict and not in global scope (variables declared at the top level are not global).

Another way of importing a module, that works inside both elements, is the programmatic System.import() API that returns a module asynchronously, via a promise.

Things that can’t be fixed

The downside of One JavaScript is that you can’t fix existing quirks, especially the following two.

First, typeof null should return the string 'null' and not 'object'. But fixing that would break existing code. On the other hand, adding new results for new kinds of operands is OK, because current JavaScript engines already occasionally return custom values for host objects. One example are ECMAScript 6’s symbols:

    > typeof Symbol.iterator
    'symbol'

Second, the global object (window in browsers) shouldn’t be in the scope chain of variables. But it is also much too late to change that now. At least you won’t be in global scope in modules and within elements.

Conclusion

One JavaScript means making ECMAScript 6 completely backwards compatible. It is great that that succeeded. Especially appreciated is that modules (and thus most of our code) are implicitly in strict mode.

In the short term, adding ES6 constructs to both strict mode and sloppy mode is more work when it comes to writing the language specification and to implementing it in engines. In the long term, both the spec and engines profit from the language not being forked (less bloat etc.). Programmers profit immediately from One JavaScript, because it makes it easier to get started with ECMAScript 6.

Further reading

  1. The original 1JS proposal (warning: out of date): “ES6 doesn’t need opt-in” by David Herman.
  2. ECMAScript 6 modules in future browsers
  3. Using ECMAScript 6 today (overview plus links to more in-depth material)

Source:: 2ality

ECMAScript 6: new OOP features besides classes

By Axel Rauschmayer

Classes [2] are the major new OOP feature in ECMAScript 6 [1]. However, it also includes new features for object literals and new utility methods in Object. This blog post describes them.

New features of object literals

Method definitions

In ECMAScript 5, methods are properties whose values are functions:

    var obj = {
        myMethod: function () {
            ···
        }
    };

In ECMAScript 6, methods are still function-valued properties, but there is now a more compact way of defining them:

    let obj = {
        myMethod() {
            ···
        }
    };

Getters and setters continue to work as they did in ECMAScript 5 (note how syntactically similar they are to method definitions):

    let obj = {
        get foo() {
            console.log('GET foo');
            return 123;
        },
        set bar(value) {
            console.log('SET bar to '+value);
            // return value is ignored
        }
    };

Let’s use obj:

    > obj.foo
    GET foo
    123
    > obj.bar = true
    SET bar to true
    true

There is also a way to concisely define properties whose values are generator functions [3]:

    let obj = {
        * myGeneratorMethod() {
            ···
        }
    };

This code is equivalent to:

    let obj = {
        myGeneratorMethod: function* () {
            ···
        }
    };

Property value shorthands

Property value shorthands let you abbreviate the definition of a property in an object literal: If the name of the variable that specifies the property value is also the property key then you can omit the key. This looks as follows.

    let x = 4;
    let y = 1;
    let obj = { x, y };

The last line is equivalent to:

    let obj = { x: x, y: y };

Property value shorthands work well together with destructuring [4]:

    let obj = { x: 4, y: 1 };
    let {x,y} = obj;
    console.log(x); // 4
    console.log(y); // 1

One use case for property value shorthands are multiple return values [4].

Computed property keys

Remember that there are two ways of specifying a key when you set a property.

  1. Via a fixed name: obj.foo = true;
  2. Via an expression: obj['b'+'ar'] = 123;

In object literals, you only have option #1 in ECMAScript 5. ECMAScript 6 additionally provides option #2:

    let propKey = 'foo';
    let obj = {
        [propKey]: true,
        ['b'+'ar']: 123
    };

This new syntax can also be combined with a method definition:

    let obj = {
        ['h'+'ello']() {
            return 'hi';
        }
    };
    console.log(obj.hello()); // hi

The main use case for computed property keys are symbols: you can define a public symbol and use it as a special property key that is always unique. One prominent example is the symbol stored in Symbol.iterator. If on object has a method with that key, it becomes iterable [3]. The method must return an iterator, which is used by constructs such as the for-of loop to iterate over the object. The following code demonstrates how that works.

    let obj = {
        * [Symbol.iterator]() { // (A)
            yield 'hello';
            yield 'world';
        }
    };
    for (let x of obj) {
        console.log(x);
    }
    // Output:
    // hello
    // world

Line A starts a generator method definition with a computed key (the symbol stored in Symbol.iterator).

New methods of Object

Object.assign(target, source_1, source_2, ···)

This method merges the sources into the target: It modifies target, first copies all enumerable own properties of source_1 into it, then all own properties of source_2, etc. At the end, it returns the target.

    let obj = { foo: 123 };
    Object.assign(obj, { bar: true });
    console.log(JSON.stringify(obj));
        // {"foo":123,"bar":true}

Let’s look more close at how Object.assign() works:

  • Both kinds of property keys: Object.assign() supports both strings and symbols as property keys.

  • Only enumerable own properties: Object.assign() ignores inherited properties and properties that are not enumerable.

  • Copying via assignment: Properties in the target object are created via assignment (internal operation [[Put]]). That means that if target has (own or inherited) setters, those will be invoked during copying. An alternative would have been to define new properties, an operation which always creates new own properties and never invokes setters. There originally was a proposal for a variant of Object.assign() that uses definition instead of assignment. That proposal has been rejected for ECMAScript 6, but may be reconsidered for later editions.

Use cases for Object.assign()

Let’s look at a few use cases. You can use Object.assign() to add properties to this in a constructor:

    class Point {
        constructor(x, y) {
            Object.assign(this, {x, y});
        }
    }

Object.assign() is also useful for filling in defaults for missing properties. In the following example, we have an object DEFAULTS with default values for properties and an object options with data.

    const DEFAULTS = {
        logLevel: 0,
        outputFormat: 'html'
    };
    function processContent(options) {
        options = Object.assign({}, DEFAULTS, options); // (A)
        ···
    }

In line A, we created a fresh object, copied the defaults into it and then copied options into it, overriding the defaults. Object.assign() returns the result of these operations, which we assign to options.

Another use case is adding methods to objects:

    Object.assign(SomeClass.prototype, {
        someMethod(arg1, arg2) {
            ···
        },
        anotherMethod() {
            ···
        }
    });

You could also assign functions, but then you don’t have the nice method definition syntax and need to mention SomeClass.prototype each time:

    SomeClass.prototype.someMethod = function (arg1, arg2) {
        ···
    };
    SomeClass.prototype.anotherMethod = function () {
        ···
    };

One last use case for Object.assign() is a quick way of cloning objects:

    function clone(orig) {
        return Object.assign({}, orig);
    }

This way of cloning is also somewhat dirty, because it doesn’t preserve the property attributes of orig. If that is what you need, you have to use property descriptors.

If you want the clone to have the same prototype as the original, you can use Object.getPrototypeOf() and Object.create():

    function clone(orig) {
        let origProto = Object.getPrototypeOf(orig);
        return Object.assign(Object.create(origProto), orig);
    }

Object.getOwnPropertySymbols(obj)

In ECMAScript 6, the key of a property can be either a string or a symbol. There are now five tool methods that retrieve the property keys of an object obj:

  • Object.keys(obj)Array
    retrieves all string-valued keys of all enumerable own properties.

  • Object.getOwnPropertyNames(obj)Array
    retrieves all string-valued keys of all own properties.

  • Object.getOwnPropertySymbols(obj)Array
    retrieves all symbol-valued keys of all own properties.

  • Reflect.ownKeys(obj)Array
    retrieves all keys of all own properties.

  • Reflect.enumerate(obj)Iterator
    retrieves all string-values keys of all enumerable properties.

Object.is(value1, value2)

The strict equals operator (===) treats two values differently than one might expect.

First, NaN is not equal to itself.

    > NaN === NaN
    false

That is unfortunate, because it often prevents us from detecting NaN:

    > [0,NaN,2].indexOf(NaN)
    -1

Second, JavaScript has two zeros, but strict equals treats them as if they were the same value:

    > -0 === +0
    true

Doing this is normally a good thing.

Object.is() provides a way of comparing values that is a bit more precise than ===. It works as follows:

    > Object.is(NaN, NaN)
    true
    > Object.is(-0, +0)
    false

Everything else is compared as with ===.

If we combine Object.is() with the new ECMAScript 6 array method findIndex() [5], we can find NaN in arrays:

    > [0,NaN,2].findIndex(x => Object.is(x, NaN))
    1

Object.setPrototypeOf(obj, proto)

This method sets the prototype of obj to proto. The non-standard way of doing so in ECMAScript 5, that is supported by many engines, is via assinging to the special property __proto__. The recommended way of setting the prototype remains the same as in ECMAScript 5: during the creation of an object, via Object.create(). That will always be faster than first creating an object and then setting its prototype. Obviously, it doesn’t work if you want to change the prototype of an existing object.

References

  1. Using ECMAScript 6 today
  2. ECMAScript 6: classes
  3. Iterators and generators in ECMAScript 6
  4. Multiple return values in ECMAScript 6
  5. ECMAScript 6’s new array methods

Source:: 2ality