Thursday, 24 November, 2016 UTC

ES6 Iterators and Generators in Practice

ES6 Iterators and Generators in Practice

Summary

This article is a section from the course ES6 in Practice. I created this course during the last couple of months, because there is an evident need for a resource that helps JavaScript developers put theory into practice.
This course won’t waste your time with long hours of video, or long pages of theory that you will never use in practice. We will instead focus on learning just enough theory to solve some exercises. Once you are done with the exercises, you can check the reference solutions, and conclude some lessons. In fact, some of the theory are sometimes placed in the reference solutions.
If you like this article, check out the course here.
This section is about iterators and generators.
It is worth for you to learn about iterators, especially if you are a fan of lazy evaluation, or you want to be able to describe infinite sequences. Understanding iterators also helps you understand generators, promises, sets, and maps better.
Once we cover the fundamentals of iterators, we will use our knowledge to understand how generators work.

Iterables and Iterators

ES6 comes with the iterable protocol. The protocol defines iterating behavior of JavaScript objects.
An iterable object has an iterator method with the key Symbol.iterator. This method returns an iterator object.
let iterableObject = {
  [Symbol.iterator]() { return iteratorObject; }  
};
Symbol.iterator is a well known symbol. If you don’t know what well known symbols are, read the lesson about symbols.
We will now use Symbol.iterator to describe an iterable object. Note that we are using this construct for the sake of understanding how iterators work. Technically, you will hardly ever need Symbol.iterator in your code. You will soon learn another way to define iterables.
An iterator object is a data structure that has a next method. When calling this method on the iterator, it returns the next element, and a boolean signalling whether we reached the end of the iteration.
// Place this before iterableObject
let iteratorObject = {
    next() {
        return {
            done: true,
            value: null
        };
    }    
};
The return value of the next function is an object with two keys:
  • done is treated as a boolean. When done is truthy, the iteration ends, and value is not considered in the iteration
  • value is the upcoming value of the iteration. It is considered in the iteration if and only if done is falsy. When done is truthy, value becomes the return value of the iterator
Let’s create a countdown object as an example:
let countdownIterator = {
    countdown: 10,
    next() {
        this.countdown -= 1;
        return {
            done: this.countdown === 0,
            value: this.countdown
        };
    }    
};  

let countdownIterable = {
    [Symbol.iterator]() {
        return Object.assign( {}, countdownIterator ) 
    }
};

let iterator = countdownIterable[Symbol.iterator]();

iterator.next();
> Object {done: false, value: 9}

iterator.next();
> Object {done: false, value: 8}
Note that the state of the iteration is preserved.
The role of Object.assign is that we create a shallow copy of the iterator object each time the iterable returns an iterator. This allows us to have multiple iterators on the same iterable object, storing their own internal state. Without Object.assign, we would just have multiple references to the same iterator object:
let secondIterator = countdownIterable[Symbol.iterator]();
let thirdIterator = countdownIterable[Symbol.iterator]();

console.log( secondIterator.next() );
> Object {done: false, value: 9}

console.log( thirdIterator.next() );
> Object {done: false, value: 9}

console.log( secondIterator.next() );
> Object {done: false, value: 8}
We will now learn how to make use of iterators and iterable objects.

Consuming iterables

Both the for-of loop and the spread operator can be used to perform the iteration on an iterable object.
for ( let element of iterableObject ) {
    console.log( element );
}

console.log( [...iterableObject] );
Using the countdown example, we can print out the result of the countdown in an array:
[...countdownIterable]
> [9, 8, 7, 6, 5, 4, 3, 2, 1]
Language constructs that consume iterable data are called data consumers. We will learn about other data consumers soon.

Built-in Iterables

Some JavaScript types are iterables:
  • Arrays are iterables, and work well with the for-of loop
  • Strings are iterables as arrays of 2 to 4 byte characters
  • DOM data structures are also iterables. If you want proof, just open a random website, and execute [...document.querySelectorAll('p')] in the console
  • Maps and Sets are iterables. See the next section for more details
Let’s experiment with built-in iterables a bit.
let message = 'ok';

let stringIterator = message[Symbol.iterator]();
let secondStringIterator = message[Symbol.iterator]();

stringIterator.next();
> Object {value: "o", done: false}

secondStringIterator.next();
> Object {value: "o", done: false}

stringIterator.next();
> Object {value: "k", done: false}

stringIterator.next();
> Object {value: undefined, done: true}

secondStringIterator.next();
> Object {value: "k", done: false}
Before you think how cool it is to use Symbol.iterator to get the iterator of built-in datatypes, I would like to emphasize that using Symbol.iterator is generally not cool. There is an easier way to get the iterator of built-in data structures using the public interface of built-in iterables.
You can create an ArrayIterator by calling the entries method of an array. ArrayIterator objects yield an array of [key, value] in each iteration.
Strings can be handled as arrays using the spread operator:
let message = [...'ok'];

let pairs = message.entries();

for( let pair of pairs ) {
    console.log( pair );
}

> [0, "o"]
> [1, "k"]

Iterables with Sets and Maps

The entries method is defined on sets and maps. You can also use the keys and values method on a set or map to create an iterator/iterable of the keys or values. Example:
let colors = new Set( [ 'red', 'yellow', 'green' ] );
let horses = new Map( [[5, 'QuickBucks'], [8, 'Chocolate'], [3, 'Filippone']] );

console.log( colors.entries() );
> SetIterator {["red", "red"], ["yellow", "yellow"], ["green", "green"]}

console.log( colors.keys() );
> SetIterator {"red", "yellow", "green"}

console.log( colors.values() );
> SetIterator {"red", "yellow", "green"}

console.log( horses.entries() );
> MapIterator {[5, "QuickBucks"], [8, "Chocolate"], [3, "Filippone"]}

console.log( horses.keys() );
> MapIterator {5, 8, 3}

console.log( horses.values() );
> MapIterator {"QuickBucks", "Chocolate", "Filippone"}
You don’t need these iterators though to perform the iteration. Sets and maps are iterable themselves, therfore, they can be used in for-of loops.
A common destructuring pattern is to iterate the keys and values of a map using destructuring in a for-of loop: 
for ( let [key, value] of horses ) {
    console.log( key, value );
}
> 5 "QuickBucks"
> 8 "Chocolate"
> 3 "Filippone"
When creating a set or a map, you can pass any iterable as an argument, provided that the results of the iteration can form a set or a map:
let s = new Set( countdownIterable );
> Set {9, 8, 7, 6, 5, 4, 3, 2, 1}

The role of the iterable interface

We can understand iterables a bit better by concentrating on data flow:
  • The for-of loop, the ... operator, and some other language constructs are data consumers. They consume iterable data
  • Iterable data structures such as arrays, strings, dom data structures, maps, and sets are data sources
  • The iterable interface specifies how to connect data consumers with data sources
  • Iterable objects are created according to the iterable interface specification. Iterable objects can create iterator objects that facilitate the iteration on their data source, and prepare the result for a data consumer
We can create independent iterator objects on the same iterable. Each iterator acts like a pointer to the upcoming element the linked data source can consume.
In the lesson on sets and maps, we have learned that it is possible to convert sets to arrays using the spread operator:
let arr = [...set];
You now know that a set is an iterable object, and the spread operator is a data consumer. The formation of the array is based on the iterable interface. ES6 makes a lot of sense once you start connecting the dots.

Generators

There is a relationship between iterators and generators: a generator is a special function that returns an iterator. There are some differences between generator functions and regular functions:
  • There is an * after the function keyword,
  • Generator functions create iterators
  • We use the yield keyword in the created iterator function. By writing yield v, the iterator returns { value: v, done: false } as a value
  • We can also use the return keyword to end the iteration. Similarly to iterators, the returned value won’t be a enumerated by a data consumer
  • The yielded result is the next value of the iteration process. Execution of the generator function is stopped at the point of yielding. Once a data consumer asks for another value, execution of the generator function is resumed, by executing the statement after the last yield
Example:
function *getLampIterator() {
    yield 'red';
    yield 'green';
    return 'lastValue';
    // implicit: return undefined;
}

let lampIterator = getLampIterator();

console.log( lampIterator.next() );
> Object {value: "red", done: false}

console.log( lampIterator.next() );
> Object {value: "green", done: false}

console.log( lampIterator.next() );
> Object {value: "lastValue", done: true}
When we reach the end of a function, it automatically returns undefined. In the above example, we never reached the end, as we returned 'lastValue' instead.
If the return value was missing, the function would return {value: undefined, done: true}.
Use generators to define custom iterables to avoid using the well known symbol Symbol.iterator.

Generators return iterators that are also iterables

Recall our string iterator example to refresh what iterable objects and iterators are:
let message = 'ok';

let stringIterator = message[Symbol.iterator]();
We call the next method of stringIterator to get the next element:
console.log( stringIterator.next() );
> Object {value: "o", done: false}
However, in a for-of loop, we normally use the iterable object, not the iterator:
for ( let ch of message ) {
    console.log( ch );
}
> o
> k
Iterable objects have a [Symbol.iterator] method that returns an iterator.
Iterator objects have a next method that returns an object with keys value and done.
Generator functions return an object that is both an iterable and an iterator. Generator functions have:
  • a [Symbol.iterator] method to return their iterator,
  • a next method to perform the iteration
As a consequence, the return value of generator functions can be used in for-of loops, after the spread operator, and in all places where iterables are consumed.
function *getLampIterator() {
    yield 'red';
    yield 'green';
    return 'lastValue';
    // implicit: return undefined;
}

let lampIterator = getLampIterator();

console.log( lampIterator.next() );
> Object {value: "red", done: false}

console.log( [...lampIterator] );
> ["green"]
In the above example, [...lampIterator] contains the remaining values of the iteration in an array.

Iterators and destructuring

When equating an array to an iterable, iteration takes place.
let lampIterator = getLampIterator();

let [head,] = lampIterator;

console.log( head, [...lampIterator] );
> red []
The destructuring assignment is executed as follows:
  • first, lampIterator is substituted by an array of form [...lampIterator]
  • then the array is destructured, and head is assigned to the first element of the array
  • the rest of the values are thrown away
  • as lampIterator was used to build an array with all elements on the right hand side, [...lampIterator] is empty in the console log

Combining generators

It is possible to combine two sequences in one iterable. All you need to do is use yield * to include an iterable, which will enumerate all of its values one by one.
let countdownGenerator = function *() {
    let i = 10;
    while ( i > 0 ) yield --i;
}

let lampGenerator = function *() {
    yield 'red';
    yield 'green';
}

let countdownThenLampGenerator = function *() {
    yield *countdownGenerator();
    yield *lampGenerator();
}

console.log( [...countdownThenLampGenerator()] );
> [9, 8, 7, 6, 5, 4, 3, 2, 1, 0, "red", "green"]

Passing parameters to iterables

The next method of iterators can be used to pass a value that becomes the value of the previous yield statement.
let greetings = function *() {
    let name = yield 'Hi!';
    yield `Hello, ${ name }!`;
}

let greetingIterator = greetings();

console.log( greetingIterator.next() );
> Object {value: "Hi!", done: false}

console.log( greetingIterator.next( 'Lewis' ) );
> Object {value: "Hello, Lewis!", done: false}
The return value of a generator becomes the value of a yield * expression:
let sumSequence = function *( num ) {
    let sum = 0;
    for ( let i = 1; i <= num; ++i ) {
        sum += i;
        yield i;
    }
    return sum;
}

let wrapSumSequence = function *( num ) {
    let sum = yield *sumSequence( num );
    yield `The sum is: ${ sum }.`;
}

for ( let elem of wrapSumSequence( 3 ) ) {
    console.log( elem );
}
> 1
> 2
> 3
> The sum is: 6.

Practical applications

You now know everything to be able to write generator functions. This is one of the hardest topics in ES6, so you will get a chance to solve more exercises than usual.
After practicing the foundations, you will find out how to use generators in practice to:
  • define infinite sequences (exercise 5),
  • create code that evaluates lazily (exercise 6).
For the sake of completeness, it is worth mentioning that generators can be used for asynchronous programming. Running asynchronous code is outside the scope of this lesson. We will use promises for handling asynchronous code.

Exercises

These exercises help you explore in more depth how iterators and generators work. You will get a chance to play around with iterators and generators, which will result in a higher depth of learning experience for you than reading about the edge cases.
You can also find out if you already know enough to command these edge cases without learning more about iterators and generators.
I will post an article with the solutions of the exercises. I will hide the solutions for a couple of days so that you can try solving these exercises yourself.
If you liked this lesson, check out the course by clicking the book below, or visiting this link.
Exercise 1. What happens if we use a string iterator in a for-of loop?
let message = 'ok';
let messageIterator = message[Symbol.iterator]();

messageIterator.next();

for ( let item of messageIterator ) {
    console.log( item );
}
Exercise 2. Create a countdown iterator that counts from 9 to 1. Use generator functions!
let getCountdownIterator = // Your code comes here

console.log( [ ...getCountdownIterator() ] );
> [9, 8, 7, 6, 5, 4, 3, 2, 1]
Exercise 3. Make the following object iterable:
let todoList = {
    todoItems: [],
    addItem( description ) {
        this.todoItems.push( { description, done: false } );
        return this;
    },
    crossOutItem( index ) {
        if ( index < this.todoItems.length ) {
            this.todoItems[index].done = true;
        }
        return this;
    }
};

todoList.addItem( 'task 1' ).addItem( 'task 2' ).crossOutItem( 0 );

let iterableTodoList = // ???;

for ( let item of iterableTodoList ) {
    console.log( item );
}

// Without your code, you get the following error:
// Uncaught TypeError: todoList[Symbol.iterator] is not a function
Exercise 4. Determine the values logged to the console without running the code. Instead of just writing down the values, formulate your thought process and explain to yourself how the code runs line by line.
let errorDemo = function *() {
    yield 1;
    throw 'Error yielding the next result';
    yield 2;
}

let it = errorDemo();

// Execute one statement at a time to avoid
// skipping lines after the first thrown error.

console.log( it.next() );

console.log( it.next() );

console.log( [...errorDemo()] );

for ( let element of errorDemo() ) {
    console.log( element );
}
Exercise 5. Create an infinite sequence that generates the next value of the Fibonacci sequence.
The Fibonacci sequence is defined as follows:
  • fib( 0 ) = 0
  • fib( 1 ) = 1
  • for n > 1, fib( n ) = fib( n - 1 ) + fib( n - 2 )
Exercise 6. Create a lazy filter generator function. Filter the elements of the Fibonacci sequence by keeping the even values only.
function *filter( iterable, filterFunction ) {
    // insert code here
}
Would you like to learn ES6?
Strengthen your JavaScript knowledge with marketable skills!
YesORNo
Close
Get the Course "ES6 in Practice"!
Learn Marketable Skills.

Verify your knowledge with real world exercises.
I'm In!
Close
Thank you for your subscription.
Please check your inbox to access the first lesson.
Close

... more @ zsoltnagy.eu

zsoltnagy.eu