building the minimal rpc with msgpack over udp
I wanted to change the protocol from tcp to udp. But the API between the net module and the dgram module have more differences than expected.
But first one more insights I was not aware of with tcp.
tcp frames and packages
Every time when calling write there is send a new frame, the frame get divided into chunks or segments.
| Frame | ||
|---|---|---|
| segment | segment | segment |
However on the receiving site we have in node.js only the onData event each giving us one segment.
In msgpack5 new chunks get concatenated and parsed, until parsing was successful. This works, because the final segment/chunk will end at the end of a message(serialized object).
I am not sure now how this works in a file stream, maybe there is an additional end character on each message/object.
Using the dgram udp module
On a tcp or net module server in node.js there is a onConnection event and data get passed into and read from this connection object. Each connection representing one client.
When using the dgram module however, there is only the message event. that give us data as well as ip and port for the client. When Data does not get send on a connection object, but through the server and specifying the IP and Port as an argument.
When the message is send, we do not get any error, if the send information has actually arrived had failed.
When the client shutdown or disconnect, there is no event that the client disapeared.
I think even with UDP we can construct a mechanis to implement a similar API as already done with tcp, however some extra overhead is needed:
- check if the connection is still there, pissible via regular ping messages
- for bigger messages some flow controll is needed, as in UDP the order of messages can change(rarly but surely)
- pick up lost packages(give messages some id)
When implementing the additional guaranties, we have a granular way to decide what guaranties we need back, compared to TCP. The good part, depending on the problem at hand, it is possible, not to implement all guaranties, and therefor gain other advantages, such as latency or lower traffic.
Also, as there are so many drawbacks compared to tcp, I believe it is seldom useful for the logic part of an enterprise application. For a logging system however I think this can be very useful. UDP is more useful in scenarios where the information flow can be unidirectional (one direction)
Unidirectional like in:
- updates on a realtime game
- writing logs into log server
- live updates from IOT devices
Did you like this article? You also might be interested in the schema less API.