Talking about some technology is difficult. You are passionate about some technology, just some technique, some idea how you want to structure your code, a lib or anything that you learned to know and think that could be useful for your projects. You like that, what ever it is. You think that might be changing your work. Then of course you want tell someone about that. But they don’t get the point.

That might have different reasons.

lack of interest
lack of understanding
is currently passionate about something else

Pretending to understand and talk about very general/low level stuff

Pretending he had a similar solution by doing something different and telling you about his interesting stuff, that you don’t care

Pretending to be interested, knowing they can never catch up in a reasonable time to understand

you find someone who understand it, who is giving you further good thoughts, but they are so busy doing other stuff.

Find someone else, find someone who got the point and is willing to use it in a project with you.

XML is loosing traction, it seems that every new API uses JSON or alike. But still there are big data sets only available in XML. As I already had a great XML-parser, that is extremely fast, has a clean API and provides an AST/DOM, I had to fill the last important feature missing for that parser. Until now it was only possible to parse about 10-40 Megabyte strings.

I thought long and I started a few times implementing a parser that is still very fast and has all the features and on top of that is able to parse though large files. Such as a Wikipedia-dump or the openStreetMap-world file. But now about at the fifed try, I found a solution to handle streams. I solved it by taking assumptions about the shape of large xml-files. Large XML files usually consist of a root element, containing a long list of items. The new parser is going to provide items one by one. It uses a nodejs-Streams. That makes it possible to use a stream reader for compressed files and use the plane-data stream for the xml-parser.

Each item provided from the xml is an ast on its own and can easily be simplified by the framework. That makes working with xml-files not just much faster then ever before in JS it is also much more convenient. Developer have an API they are probable already familiar with.

If I would be you and have to work with XML data, I would definitely use tXml!!! If you can really choose, choose JSON.

Currently I am an intensive user of visual studio code. It is an editor, developed by microsoft, that is extendable with many plugins. My favorit feature is the typescript integreation and its intelisence.

Recently it supports type recognition based on JSdoc comments. reason enough to look deeper into JSdoc. And I found it awefull to defined types. That is so much to type and by that error prone. Below you find a tool, that let you generate some JSdoc comment, from a JSON object. That is usefull for database or API responses.

Restful is a pattern to provide an API to manage resources on a server, providing a uniform deal to create, access and change data. In a previous post I have shown the json-sever. With that server, I made several tests and even extended its API. The json-server takes a JSON-file and provide an access through a auto-generated API.

But I never made a complete app using that server, until recently. Because every time I meet some issues that would require a lot of coding that I was not willing to spend for small side projects. But I could not get RESTapis out of my mind, so I studied studied some resources to answer all questions that are open. In this small series of posts I want to talk about some.

The first is this one, about Authentication, it is the one you are currently reading. The next will be about actual designs, means about how parameter and responses should look. It will also provide good resources with good API-definitions and actual implementations. After that, I will actually take a look of implementing a restAPI in nodejs and auto generated APIs, with additional feature: how objects will be validated, security guarantied and business rules applied.

Authentication

Often I had the question, how should I do authentication and how does resources look, that are related to the current user. Messages to me, messages from me, my photos, my results, orders, what ever. On top of that, I was asking myself how should a restful API actually look like. The JSON-Server is so simple that it is instantly fun to play with it. But quickly I reached several points, that throw questions that need to be answered but for the small side projects they are to big.

Login

First was the authentication, is making a session actually restful? My answer to that is actually yes. Many people come to different opinions, but it is not to important. The important thing is that you know how to do authentication. Typically I had some auth-module that has provided an RPC method for login and an other to get information about the current session.

For my first complete restful app, I actually provided a kind of a virtual resource. Many frameworks would name that a Controller or API-Handler. The Controller that I made received login information through a POST-request and provided the current session information to the GET and the DELETE for logout.

token + signatures

When the API is not meant to be called by a browser, working with sessions is not comfortable and when using web-APIs I never saw that. So the authentication is send on every request. Depending on the importance of that API, there was just a token. So the provider can monitor my request and limit the results and number of requests. Sometimes it is fine to just use https for encryption. But because many http implementations do not validate the certificates APIs require to add a signature to a request.

implementation

Using nodejs with express, both types of authentication can be ensured using middleware that is running before the middleware for the actual API is executed. No matter if you are using sails.js, JSON-server or an other rest-API providing framework, you can use standard middleware such as express-session, curf or express-body-parser. With your applications specific authentication middleware you only need to invest once and the authentication will be solved.

Programming with server applications is very complex. One big discipline is using databases. The most common kind of databases are SQL databases. A best practice is saying: you should not write SQL in string-notation mixed into your code. This Post is about moving sql queries out of your sourcecode.

When I used J2EE, I also used JQL, the java query language. There you write he queries right above your actual function and give it a name, that you then can access inside that function. But in my opinion the query then is still inside the code. just on a different position and added to some variable or object.

An other common approach are query builder. There you are going to use function to assemble the querystring. In nodejs there are modules for that, squel or knex, to name two. in both, you write something like that: squel.select().from(“tablename”,”t”).join(“othertable”, “t2”,”t.c=t2.c”).where(“t.b>2”). but actually, again, in my opinion the best practice is not met. The query is still written between your application code(hopefully in a separate module). You don’t write SQL, but still a complex syntax, to query the database, that you have to learn on top of the database and SQL. I think the value of using those libraries directly is very limited.

But they are often used by some the third way, that want to solve that issue. ORMs often use query builder to work on different databases with different dialects of SQL. For example Sequilize as well as bookshelf are using the knex-query-builder. In addition knex provide handy methods to load, update and insert objects. But as soon as you need to execute a more complex query, you still use the underlying querybuilder and the problem is the same again. you need to learn an ORM + its query-builder.

An other type is query-mapping libraries. Actually the mysql module for node can already be seen as one of this kind. The result is mapped into an array of plain objects, where the values can get accessed by the same name as the field on the table. But this alone is not helping to move the query out of your JS file or PHP file. At my company we are using bearcat-dao. It is actually describing itself as a query-mapping-framework. Instead to simple objects it also can map to defined classes. In bearcar-dao the SQL-queries are written into separate .sql-files. When booting the app, bearcat dao will read all .sql-files and provide those queries by there name. So when you want to execute a query, you choose one by name in your code and the query is written in an SQL file, what is also giving you nice syntax highlighting in editors.

This actually meets the best practice of moving SQL-queries into a separate file. But the implementation has a drawback. All queries across all SQL-files share the same namespace and if you gave two queries the same name, you don’t even get a warning. This has lead me to make an sql-file loader. That makes it possible to have a namespace for each single SQL-file. My new module tsqlreader, can load SQL-files. it can handle comments and has support for templates or fragments.

So if you think using an ORM or query-builder is not appropriate, you can try the tsqlreader to load your SQL queries from separate files. But please, try to not repeat yourself to much, when you write your SQL. That will be the next post about.