asynchronous functions and how to avoid the callback hell

I last time gave the advice to learn about asynchronous programming. Finding the right information can be hard. With this article I want to make it a little easier for you.

We will learn what is a callback and when and when it is not asynchronous. We will look at different pattern for asynchronous programming and look how we can structure our source-code to work with the complexity.

Buy first we start with the question why?.

Javascript if you run it in the browser or on the server using node.js, is single threated. That means only one function can run at the same time. To not feel this limitation, tasks like making an ajax network request or loading a file are done asynchronous. That means these input output tasks, are not blocking the single thread. The actual work of loading is not done by JS, only initiated, and once the resource is loads, we can process the data with javascript. That has the advantage, that while the data is loading, the javascript process is free to process other functions, such as from user events.

Now let us look at some code.

var myList = ['apples', 'bananas', 'peach'];
myList.forEach(function(fruit) {
  console.log(fruit)
});
console.log('this is all my fruits');

This code is using a callback. But this code is not async. Often people misunderstand the use of callback functions in javascript with asynchronous programming. So how can we see, that this code is not async, but synchronous. Here is the output from the console.

# node printfruits.js
apples
bananas
peach
this is all my fruits
#

We can see that first all the fruits are printed and then the sentence this is all my fruits. We will now compare this with an asynchronous function.

var myList = ['apples', 'bananas', 'peach'];
myList.forEach(function(fruit) {
  setTimeout(function callback(){
    console.log(fruit)
  }, 0);
});
console.log('this is all my fruits');

I added the asynchronous function setTimeout to print print the fruits asynchronous. SetTimeout is usually used to just wait in javascript programs. But for tasks that need a callback function the usage is the same.

By the way: the callback function does not need to be named ‘callback’, but can have any other name. It is good practice to give your callback functions a name. That can help you in bigger programs to find and fix bugs.

Here is the output.

# node printfruitsAsyncron.js
this is all my fruits
apples
bananas
peach
#

You can see that this time, this is all my fruits was logged first. This shows, that when using an asynchronous function, javascript is able to process other functions.

This is very cool, because the program behaves like we have a parallel executing threats. The effect of processing many tasks at the same time, but in only one thread is often called ‘concurrent’.

examples for asynchronous functions

In the examples before I show you the setTimeout function. It will wait the given time and execute the function once. Similar, there is the setInterval. SetInterval also need a function and a number as arguments, but it will call the function again and again, as long as the program is running.

jQuery

In jQuery we can add event-listeners on elements in the HTML. (in JS ‘event listener’ is only a synonym for a callback function)

$('.likeButton').on('click', function(e) {
  // do something
});
$('.dislikeButton').on('click', function(e) {
  // do something else
});

You see, the code can register many functions to all the elements on the page, but the event listener is only executed once the user click at one of the DOM nodes.

When loading extra data from the or u
to an update to the server, you can use jQueries Ajax function. This example show an update when a like button is clicked.

$('.likeButton').on('click', function(e) {
    $.ajax('/articles/like, {
      method: 'POST',
      dataType: 'json',
      body: {
        artitleId: currentArticle.jd
      },
      success: function(){
        $('.likeButton').addClass('liked');
      },
      error: function() {
        alert('something went wrong when liking this article.')
      }
    });
});

Can you see, first we add a function to the on click event of the like button. You can understand it like telling the browser ‘once the user click this button, execute this function’. Than, once the user actually click the button, we are calling the $.ajax method with a url and some settings. the settings specify that we are doing a POST request, want to send and receive data as json, the actual information we want to send, that is like the argument that the server expect, and again, we add callback functions or event listeners to the success and error property.

Once the request was successful, jquery will call the success method, otherwise the error method. These functions are asynchronous. That means the browser can in the meantime to other things on the page, such as animating css or responding to other user activities.

node.js callbacks

In the last section we assigned callback functions to a parameter. We had two different functions, one for success one for error. Different libraries and javascript frameworks have different pattern. In node.js most modules follow the same pattern. Look at this example for reading a file:

var fs = require('fs');
fs.readFile('./data.json', function(err, dataBuffer) {
  if (err) {
    console.log(err)
  }
  var data = JSON.parse(dataBuffer.toString());
  // we can do something with the data.
});

The fs module is build in to node.js. its functions expect the usual arguments and as a last parameter a single callback function. This function has as its first argument a possible error. Only when the first argument is undefined the callback receive valid result data in the second argument. Look at a second example, inserting data (maybe load from a file) into a mysql database.

var mysql = require('mysql');
var connection = mysql.createConnection({
  host: 'localhost',
  user: 'me',
  password: 'secret',
  database: 'my_db'
});
connection.connect();
 
connection.query('INSERT INTO data set ?', data, function (error, results) {
  if (err) console.log(err);
  console.log(results.insertId);
});

When calling the query function like this, inside an API controller, node.js will be able to serve other requests, while our current request is waiting for its result. To serve many requests like this, other programing languages use additional operating system threads. This is a huge waste in memory, and needs to create more connections to the database server, slowing down the database server as well.

callback hell

Now that I have shown you what and why, I want to show you how to organize the code. because organizing the source code with many callbacks can become hard to read, maintain and test.

It is a good Idea, to define some rules. and the first is, ypu should not define a callback function inside a loop. when defining a function inside a loop, a new function object for every iteration will be created. This is not only slow, but what is more important is, is that it is hard to reason about the code.

A second rule, is to never define a callback function inside an other callback function. Look one more time at the second example about the jQuery ajax function. This example violates the rule, but you can see where this code can lead to. It can lead to very deep nesting and indentation. This kind of structure is very hard to debug and maintain.

There are libraries to help you follow these two rules.

In node.js many programmers use the async library. Here is an example for avoiding a callback inside a loop, while processing an array of data.

var async = require('async');
async.each(datas, function processOneData(data, done) {
  connection.query('update ...', data, function(err){
    done()
  })
}, function allDone(err){
  cosole.log('done', err);
});

This is processing all items at the same time, concurrently or parallel, this makes this code very fase.

var data;
async.series([
  function one (cb){
    fs.readFile('data.json', function(err, _datas){data=_data; cb(err);})
  },
  function two(cb){
    connection.query('INSERT INTO data set ?',    data, cb);
  },
 function three(){
    if (err) console.log(err);
    console.log(results.insertId);
  }
]);

This example show a series of three functions. the first load a file, the second query data and the third print the result. Do you notice thay the function one violates the second rule? It defines a callback inside an other callback.

I did that on purpose, to show you how to in function two, how this can be avoided. Do you remember, that in node.js callbacks should always follow the structure of first argument is the error second the result. because that structure is also used by the cb function, we can directly pass it into the argument of the function we are calling.

Promises

Promises are an other way, to avoid a deeply nested callback hell. Maybe I show them in an other article, because with the tips I already show you in this article, you will be able to program your application server.

I promise.