More often than not, interviewers ask about Generator Functions, the Yield Keyword, and Iterators. Most developers have no idea what they mean.

Through this article, I will explain those complicated concepts using the fundamentals. Even I didn’t know they existed. I read about them in a QNA where developers were supposed to submit the names of intricate concepts.

However, Generators have the same reputation as Closures. They are complicated because the crowd says so. Let us understand what Generators mean, the Yield keyword, and Iterators.

What are Iterators

By definition, iterators represent recurrence. In JavaScript, as per MDN, Iterators and Generators bring the concept of iteration, i.e., repetition, directly into the core language.

If you remember, we have loops in JavaScript to perform the repetition of code statements. We have methods, like map(), too.

However, the concept of Iterators in this context alongside Generators is different. Iterators that are custom and Generators use loops to perform their function. Generators and custom Iterators provide a mechanism for modifying the behaviour of the default iterators in the language, like loops.

const exampleArr = [1, 2, 3, 4, 5];

for(const val of exampleArr) {
    console.log(val);
}

You are looking at a standard for-of loop. It doesn’t keep track of the current index or iteration number. It doesn’t know whether it should begin from the left or right-hand side. Unlike the traditional for-loop, we don’t mention a starting or ending index in the for-of loop.

The loop moves from left to right sequentially because the array contains an iterator function instructing the for-of loop to iterate through a specific array in that sequence. It is the default behaviour.

The for-of loop is an iterator because it executes the same statements on each array element repetitively. However, the for-of loop is a default iterator.

Sometimes, you require a custom iterator that repeats specific code statements or traverses through a datatype in a specified manner. In massive applications, developers create custom data types. Those datatypes require custom iterators to access that data and manipulate them. Iterators are crucial when writing custom libraries.

In the new context, an iterator is an object that defines the sequence of the iteration and potentially the values that will get returned when the code statements terminate. We can create custom iterators using the Iterator Protocol.

Every iterator implements the Iterator Protocol with a next() method that gets returned by it. The next() method seems logical because the job of iterators is to traverse through elements from one index to another in a specific sequence. The method returns an object with two properties —

1. value — It holds the next value in the iteration sequence.

2. done — It holds a boolean value. It represents whether the last value in the sequence has been traversed or consumed. If the .value and .done properties are present in an object, they represent the iterator’s return value.

The most common iterators in JavaScript are forEach(), filter(), map(), etc., which return each value in a sequence. Those are default iterators. Now, let us create a custom iterator.

Custom Iterator

Let’s reciprocate the for-loop and create a sequence that executes code statements for each iteration of an imaginary datatype. Remember, this is not how we make custom iterators. We use Generators, and the following example is the complicated approach. However, it is required to understand Generator Functions.

function customIterate(start = 0, end = Infinity, step = 1) {
  let nextIndex = start;
  let iterationCount = 0;

  return {
    // Returns a method to iterate over the iterator
    next() {
      // Stores an object throughout the iteration to keep track of the value and progress
      let result;
      // Indicates that iteration isn't finished
      if (nextIndex < end) {
        // Assigning those traditional value and done properties
        result = {
          value: nextIndex,
          done: false,
        };

        // Incrementing the index based on the steps
        nextIndex += step;
        // Incrementing the iteration count
        iterationCount++;

        // Returning the object
        return result;
      } else {
        return { value: iterationCount, done: true };
      }
    },
  };
}

// Storing the instance of the Iterator
const instanceOfIterator = customIterate(0, 20, 5);

// Invoking the next() method that iterates through the iterator
let result = instanceOfIterator.next();

// Till the .done property returns true
while (!result.done) {
  console.log(result.value);

  // invoking the next() method to update the result value for the while-loop and the iterator
  result = instanceOfIterator.next();
}

I will start with a function customIterate(). It accepts your desired starting index, ending index, and step size as inputs. We will use default values if the user fails to provide them.

Now, imagine an invisible runner flashing along numbers. Initially, they stand at the starting index. Inside the function, we prepare two trackers: nextIndex tells us where the runner is now, and iterationCount keeps tabs on how many laps they completed.

The real magic happens in the next() method. It is the engine that drives the iteration. If our runner has not crossed the finish line, it hands you the current position and prompts the running to continue with a .done flag set to false. Then, it hops forward based on the step size and keeps track of one more lap.

But when the runner reaches the end, things change. Instead of another position, I offer the total number of laps as a farewell and wave a .done flag of true. The .done property lets you know the exercise is over.

To put this runner to work, you create an iterator object using customIterate(). Then, you kick things off with a manual call to the next(). The manual call gives you the initial position and gets the runner moving.

The next step is a familiar friend: the while loop. As long as the runner keeps going, you can access their current location through the result.value property, do whatever cool stuff you want with it, and then hand them a new next() call to keep the journey going.

This custom iterator approach works, but it is like building a bicycle from scratch when a sleek, ready-made option exists. Now, this is where generator functions come in. They offer a much simpler and safer way to handle iterations, and I will dive into their world next.

What are Generator Functions

We cannot create iterators that use the mechanism of the for-loop with standard Iterator Protocol. The solution is to use Generator Functions, but that's not the only reason anyone would want to use them.

Standard Iterators store explicit state values inside their iterator functions. For example, the nextStart and iterationCount variables store the current progress or state of the iterator.

Maintaining these states manually with variables is complicated without careful programming principles. Generator Functions are a powerful alternative that can maintain the states.

Generator functions allow you to write the same iterative algorithm by creating a single function whose execution is not continuous. We can pause and resume at random intervals. These functions have a separate syntax.

To define generators, we use the function* keyword. There is an extra asterisk to the standard syntax. We're still trying to iterate through custom datatype but without explicit states. In addition to the strange syntax, JavaScript introduces a new keyword called yield.

When we invoke generator functions, they don't initially execute the code. Instead, they return a specific object called a Generator. We store that object inside another variable and invoke the next() method or access the .done and .value properties through that object. Remember, Generators are beyond Iterators.

Generator functions can run infinite loops because they can pause and resume their execution. We cannot have infinite loops without them because that would freeze our respective machines. The loop will continuously execute without any pauses.

Generators allow us to insert those pauses and run infinite loops without worrying about freezing our computers. Until the next() method gets called, the loop stays on hold after discovering the yield keyword.

You will understand this in a bit. Depending on your application, you can use Generators to mimic the async/await functionality and deal with asynchronous code. Not to mention, they are also used to implement infinite scrolls on the front end.

What is the Yield Keyword

By definition, the yield keyword pauses and resumes the generator execution. The yield keyword pauses the execution of the generator till it encounters another next() method. Or till the next iteration comes around while using a loop. It is a distinctive sort of return statement keyword used inside a generator.

The yield keyword is similar to async / await. Without the async keyword, we cannot utilize await and vice-versa. Similarly, we cannot use the yield keyword without a generator function.

// Generator Function
function* iterable() {
  yield 1;
  yield 2;
  yield 3;
  yield 4;
}

// Storing the returned object of the Generator
const iterableVal = iterable();

// Iterating
for (const val of iterableVal) {
  console.log(val);
}

If you run this code, you will get a sequence of values printed to the console from 1 to 4. It prints all the values provided inside the generator function with the yield keyword. Why? Let me explain.

Using the for-of loop, JavaScript prints the results sequentially. The JS Engine detects a generator function that represents iterable and repetitive elements. The yield keyword works in harmony with the function.

Generators accept values and repeatedly return a new output. It returns the output using the yield keyword sequentially with pauses at different intervals. The yield keyword is like the return keyword. Hence, those numbers show up in the console.

Generators inherit the idea of an iterator that iterates and executes the same code on each element to get an output. Whenever JS encounters a yield keyword, it prints the value followed by the keyword and resumes the execution. When it encounters another yield keyword, it repeats the same process.

Let us expand on that example. I will implement the same logic we had implemented earlier when understanding Iterators.

// Example 2.0

// Generator Function
function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

// Creates a Generator Object using the Generator Function
const generatorObject = genIterator(1, 20);

// Loops through the values of the Generator Object
for (const val of generatorObject) {
  console.log(val);
}

I defined a function with a for-loop, passing in the same parameters. We do not need if-statements or manual step-by-step increments; the for-loop does the heavy lifting, returning an integer for each iteration.

I stored the outcome of invoking the generator function inside another variable because generator functions return a Generator object. In our case, we stored the object inside the generatorObject variable after invoking genIterator().

The yield statement returns the value stored in i after each iteration. This lightweight approach lets you iterate through custom data types without crafting logic for each step. 

You can fetch information from the database and call the yield keyword to pause and return the values instead of iterating continuously.

Think of it this way: instead of gulping down a whole database at once, you can use yield to savour each piece of information, pausing between bites. It ensures efficient handling of large datasets and provides ultimate control over the iteration process.

If we do not use the yield keyword, we can remove the for-of loop and invoke the next() method repeatedly and manually. Since the yield keyword pauses the generator, we must execute the next() method to resume the execution. We use the for-of loop for the same reason.

The for-loop replaces the massive logic inside the loop, and the for-of loop remains the same to continue the iterator. You can access the same .done and .value properties too.

function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 5);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);

A few iterations are skipped in this approach because the next() method gets executed. The paused iterations are resumed and then skipped while printing them. Instead, you can do the following.

function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 5);

console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());

The simplest way to access the values directly without skipping iterations and manually executing the next() method is to use a for-of loop, as shown in Example 2.0.

If you still cannot understand the difference, let me explain. We all know that loops do not halt at any point to execute code statements. Once they start, they only stop at the end.

We can use the break keyword to stop the loop entirely or the continue keyword to skip iterations. However, we can never pause the execution of a loop.

It is where the yield keyword becomes a convenient mechanism. We are using the for-loop inside the generator function. Instead of letting the for-loop complete its execution immediately, we use the yield keyword to pause and resume its execution once we attain the desired data. 

It helps us manage database transactions, manipulation or handling data. Let me do DOM manipulation using a Generator Function.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Generator Functions</title>
  </head>
  <body>
    <button class="yield-result">Click Here</button>

    <script src="script.js"></script>
    <script src="example.js"></script>
  </body>
</html>

function* genIterator(start, end, step = 1) {
  for (let i = start; i < end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 10);
const btn = document.querySelector(".yield-result");

btn.addEventListener("click", () => {
  btn.textContent = generatorObject.next().value;
});

I created a button element inside the HTML document. I attached an event listener, used the values fetched from the generator function after each pause and displayed it through the button.

The last returned object from the next() with the .done property contained true and indicated its competition after the entire generator completes its execution. The .value property remained undefined after resuming the last yield statement because there are no more values to return.

When you invoke the function, it returns nothing except a Generator object. Your code remains unexecuted. Previously, I had four yield statements inside a generator function and configured a for-of loop to iterate through them. Here’s how they worked.

Generator Function Explanation | Medium Article - YouTube

This is a small explanation for a concept in JavaScript called Generator Function. The entire video is meant for my medium article.

youtube.com

The last yield statement doesn't return true for the .done property because the generator function gets paused and doesn't finish the execution.

We can have multiple generators at the same time. You can create another instance or object of the same or another generator and use a loop or manually invoke the next() method.

// First Generator
function* firstGen() {
  yield 1;
}

// Second Generator
function* secondGen() {
  yield 1;
}

const firstObj = firstGen();
const secondObj = firstGen();

// First Object
console.log(firstObj.next());
// Second Object
console.log(secondObj.next());

Each instance of the same generator is separate and does not share values. You can create many of them. Let’s create an infinite loop and control it using a generator.

function* infiniteLoop() {
  const username = "@whyafan";

    // Infinite Loop
  while (true) {
    yield username;
    console.log("Yielded!");
  }

    // Ignored because infinite loop never ends
  console.log("Can JS reach this statement?");
}

const infiniteExample = infiniteLoop();

console.log(infiniteExample.next());
console.log(infiniteExample.next());
console.log(infiniteExample.next());
console.log(infiniteExample.next());

I inserted a while-loop and conditioned it to execute the code statements inside it till the condition represented a truthy value. Of course, true will infinitely return a truthy value, so the loop will never stop.

I yielded my username on all social media platforms through the while-loop. Whenever the returned object gets printed, the value will be @whyafan.

Remember, the .done property will never return true because this is an infinite loop. I invoked the next() method four times and received the returned object in the same limit because the yield keyword pauses the generator till an approaching next() method gets discovered in the wild.

Let us see how we can use the generator function for an array. We will access the array elements sequentially.

function* arrayExample(start, end = Infinity, step = 1, array) {
  for (let i = start; i <= end; i += step) {
    yield array[i];
  }
}

const array = [1, 2, 3, 4, 5, 6, 7, 8, 9];
const arrayObj = arrayExample(0, 10, 2, array);

console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());

Lastly, let us see how we can use the yield keyword to accept values passed as a parameter to the next() method.

function* passingToNext() {
  let id = 1;

  while (true) {
        // This is where JS received the params
    const increment = yield id;

    if (increment !== null || increment !== undefined) {
      id += increment;
    } else {
      id++;
    }
  }
}

const yieldObj = passingToNext();
console.log(yieldObj.next());
console.log(yieldObj.next(2));
console.log(yieldObj.next(3));
console.log(yieldObj.next(4));

I created an infinite while loop like the previous examples. I purposely stored the result of the yield keyword inside another variable because when you pass a value as a parameter to the next() method, it takes that value and gives it to the yield keyword.

When I passed 2 to the next() method, it received that integer through the id variable with the yield keyword and stored that inside the increment variable. Then, I checked whether the received value is null or undefined. If yes, I incremented the id variable without the value received from the user.

Otherwise, I incremented the id variable with the integer received by the user when they passed it as a value to the next() method.

You cannot pass a value to the first next() method execution because yield doesn’t exist during that iteration. It returns a result at the second iteration onwards.

Remember, Generator functions can be a property of an object or the method of a class.