How To Build Distributed Apps

By

Today I want to share how to build distributed apps. After making a small test, I was able to implement a distributed message broadcasting application, in less then about 50 lines of JavaScript. The idea stuck for a while now in my head, after studying the code of naivecoin. For this article, some concept is simply extracted what I learned from that project. Here, without the blockchain. And because the concept and code is so simple, everyone of you, will be able to build distributed apps on your own.

With this article, I want to help you to understand how to build distributed applications. The code we look at will be very simple, so that everyone can follow along and understand how to build sophisticated applications that run not only on the servers of one organization, but can be hosted by everyone. This sounds like a touch job, but once we break the problem down, it gets manageable.

Today, we live in a world of giant monolithic applications. Applications that lock the users data into silos. Where the provider has all the influence and decision power, not the user. In the beginning, when the internet started to develop, most services where designed to be operated by the user or the users community/organization.

In this Article we will build a distributed message broadcasting application. So let’s start with a standard express js boilerplate. Also we will use node-fetch, to communicate via http between the servers. 

1
2
3
4
5
6
7
const express = require('express');
const fetch = require('node-fetch');

const app = express();
app.listen(process.env.PORT || 3000);

app.use(express.json({}));

Our database in this project consist of two arrays. A list of peer servers, second a message array for our actual application. 

1
2
const servers = [];
const messages = [];

For connecting distributed servers, we only setup three routs. First is ping, what we use later to verify if the other peer server is there. Using /servers, we can look into the list of known other server. A POST request /server will deliver a URL, by simply trying out this URL, it get verified. The verified URL get stored in db. 

1
2
3
4
5
6
7
8
9
10
11
12
app.get('/ping', (req,res) => res.json(true));
app.get('/servers', (req,res) => res.json(servers));
app.post('/server', async (req,res) => {
  try {
    await ( await fetch(req.body.url+'/ping')).json();
  } catch (err) {
    console.log(err);
    return res.status(400).json(false);
  }
  servers.push({url: req.body.url});
  res.json(true);
});

This setup is very simple, but it contains everything to maintain a list of peer servers. Now it would be up to you, to modify this example, to meet more restrictive requirements on feature side and security. For example the domains from the URL should be cached, otherwise, now it is a nice server that others can miss use as a proxy for a denial of service attack. Also, you could want to classify servers, for different features. like in a hyperledger Fabric blockchain network, where you have servers of the types orderer, peer, endorser or certificate authority.

now let’s get to our application, that we want to build with this simple setup. A message distributions application.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
app.get('/messages',(req,res) => res.json(messages));
app.post('/message',(req,res)=>{
  if(typeof req.body.message ==='string'){
    messages.push({text:req.body.message, time:Date.now()});
    res.json(true);
  } else {
    res.status(400).json(false)
  }
})

app.post('/broatcast',async (req,res)=>{
  if(typeof req.body.message !=='string'){
    return res.status(400).json(false);
  }
  try{
    await Promise.all(servers.map(async server=>{
      console.log('request:',server.url+'/message')
      await fetch(server.url+'/message', {
        method:'POST',
        body:JSON.stringify({
          message: req.body.message,
        }),
        headers: { 'Content-Type': 'application/json' },
      }).then(r=>{
        console.log('get response')
        return r.json();
      });
    }))
  }catch(err){
    console.log(err)
  }
  res.json(true);
});

First, clients could regularly pull all messages,to keep some UI up to date. Messages should be POSTed to /broadcast. Then the Server will forward the message to all servers using the /message route. On the /message route, the message will just be stored, without broadcasting it further.
Here I used two routes, to avoid re-sending known messages. you could use a different mechanism to verify a message. But you get the point, how messages will end up in our messages array.
you also see, that the performance and storage capabilities, are all determined by the limitations of a simple JavaScript array. you might want to develop a strategy to remove old messages or query only new messages. Also, I am sure, you would want to add some authentication and users, maybe post boxes or channels and rooms.

Use Our Distributed App

For now, the server only exposes an API. So we can interact with it programatically from other applications. You could also use the express.static middleware, to serve some HTML and client side JS, to interact on a web GUI with the server.

To start the server use the following command, if you saved all the code in a file called server.js.

1
2
PORT=5555 node server.js &
PORT=6666 node server.js &

It will be running twice on two ports. Maybe a friend of yours is interested to run the second or third server on his computer or online hosted web server.

Now we can add the servers know each other. 

1
2
3
curl -X POST localhost:5555/server -H "Content-Type: application/json" -d '{"url":"http://localhost:6666"}'

curl -X POST localhost:6666/server -H "Content-Type: application/json" -d '{"url":"http://localhost:5555"}'

You now can check the servers array, using GET /servers.

Let’s have some fun with sending messages, by broadcasting to one server and reading a message on the other. 

1
2
3
curl -X POST localhost:6666/broadcast -H "Content-Type: application/json" -d '{"message":"Hallo World!!!"}'

curl localhost:5555/messages

And? Is that fun? Sending messages on this app probably not, but it is fun to see how easy it can be to build a distributed application. I hope you learned something and gain some confidence, when you want to promote building a distributed application.

Of course, you want to use more professional architectures in production. Use a real database, implement strategies to explore new servers in the network, handle errors, when a node get removed from the network. Use Authentication, encryption and more verifications as needed, but the base of storing a url or IP to maintain the peers is very easy to do.

I wish you fun and success implementing distributed apps yourself.

Contents
  1. 1. Use Our Distributed App