Rapid Prototyping with Json File Database

What do you think? How many good ideas never got started? Rapid Prototyping, is the art of getting up an initial version of an app. A version good enough for showing an idea, proving that an idea is feasible.

That often is a frontend. How fast can you run create react app, add a material UI, maybe with the help of material or ant design let users login, insert and present data.

For me as a backend engineer it is often more about the nodejs app. How quick can I create an express app with session middleware and some endpoints to insert and query data. Can it be faster setup using lookback?

Personally I never used loopback, sails or such productive, they both have an inMemory datastore. I mostly use less opinionated libraries.

The part that stopped most of my projects from actually getting started is setting up the DB. Docker is not stable on my mac nor on my Windows PC. That is why I sometimes look into file datastores.

For frontend developers there is the fantastic json-server. It provides rest API to a json file very quickly in a single command. As a backend developer we can use SQLite, nedb (outdated/not maintained), low-db that is inside json-server.

But the problem I found is, that the code written with these solutions is much different from actual production applications made with a sophisticated database.

Solution

That is why I created the node module trdb. A json file database that feels like a real db with asynchronous API and the data inside the db are isolated objects, that will not unexpectedly mutate.

While json-server let you build the frontend before the API is ready, With trdb you can implement the API before the db is ready.

// create a db
const db = newFileDB('./db.json');

// create a collection
const users = db.collection('users');

// insert a user (see without id!)
const user = await users.insert({
    name: 'Tobias Nickel',
    job: 'technical lead',
});

// update and save the user
user.favoriteHobby = 'programming';
await users.save(user);

await users.insertMany([
    { name: 'Sebastian Sanchez' },
    { name: 'Firat Evander' },
    { name: 'Silpa Janz' }
]);

// When inserting or saving, internally a new copy of the object is created
// next the reloaded user represents the same user in DB, but is a separate object.
// this is the same behavior you get from a real databases or API.
const userReloaded = await users.findOne({id: user.id});
console.log(user !== userReloaded); // true

// pass in the properties all the resulting items should have.
// also allow to pass arrays for then the objects value need to be included.
const searchedUsers = await users.find({ name: ['Firat Evander', 'Tobias Nickel']});
console.log(searchedUsers.length);

// removing items just works just like search, if first search then removes.
await users.remove({ name: 'Tobias Nickel' });
await users.remove({ id: user.id });

// Feel free to create multiple dbs to put collections in separate files.
// This example also shows the options for custom idName and
// newId. This the newId defaults to uuid v4, and provides a method for
// generating autoIncrementIds. You can implement your own ID functions,
// returning and primitive like numbers and strings. 
const postsDB = newFileDB('posts.json', {
    idName: 'postId',
    newId: newAutoIncrementId
});
const posts = db.collection('posts');

The db also watches the file. during development and testing you can always see what is inside the db file. When you edit that file, it is instandly loaded into the server process.

Result

I was about to study oauth2 and open connectId. To play with that, you need three services. The authentication/authorization service, an API service and the app service. Each want to have its own db. With trdb setting up these services was quick and I can concentrate on the logic, rather than setup.

What do you think of my solution? How do you build quick prototypes.