zlacker

TIBCO Rednezvous, https://www.tibco.com/products/tibco-rendezvous, is the first thing that came to my mind from previous experience in the financial industry working with real-time market data. Although I'm not sure if it has built-in KV support for dealing with large payloads like NATS does, TIBCO RV and their related software packages are worth checking out to see what an long time established commercial product offers. Which leads me to...

... the protocol is text-based like HTTP with CR LF for field both for the client, https://docs.nats.io/reference/reference-protocols/nats-prot..., and cluster protocols, https://docs.nats.io/reference/reference-protocols/nats-serv... -- which means encoding overhead if your payloads are binary. So depending on your definition of performance, ymmv.

I really do not see how implementing an API across multiple languages is easier by making a new linefeed-based protocol, https://github.com/nats-io/nats-server/blob/0421c65c888bf381..., than just using code-generated JSON or gRPC (Protobuf or Flatbuffers). One could then write subscriptions/clustering algorithms in a protocol-neutral library.

>>no_cir+(OP)
I don't think there's encoding overhead. It's like http, like you said, but just like http body encoding it seems that actual payloads are preceded with their length so that you just read n followed by n bytes. Look at the docs for `pub` for example. Json would be less efficient; gRPC adds tons of complexity and overhead.

>>c-cube+z1
Correct. Text based is more than fine these days, as long as you use length prefix so you can avoid escaping. But you do still need to parse lines which can be tricky in bare or poorly tooled environments. It’s the best trade off today, because debugging binary formats is a serious obstacle.

> Json would be less efficient; gRPC adds tons of complexity and overhead.

Indeed. There aren’t many suitable specs around, and this protocol, albeit custom, is very easy to implement. Which is proven by the fact that there are well maintained Nats clients in many different languages.

>>klabb3+x3
The only developers that would likely need to debug binary protocols are the framework developer themselves. It would be an extremely rare occurrence for someone to need to use a binary/hex viewer to figure out what is going wrong with their "app". Perhaps you are thinking of telemetry/introspection of one's own messages? In that case there usually is a library call to convert them to a human readable format like JSON, e.g., https://protobuf.dev/programming-guides/proto3/#json.

IMO it is a real waste of developer time to code up a new transport protocol without determining that the existing ones don't work or don't perform as well as needed. Multiply that by all the programming languages that need to be supported... when instead the client APIs could have been mostly code generated.

Although may only apply to the "client-side" API, what is going in the message payload? JSON probably, or some other serialization format. I don't think a lot of developers are hand writing parsers for their own payloads. The overhead of the JSON or Protobuf parser is already in there.

>>no_cir+(OP)
I'm not sure how much it will reduce your skepticism, but I believe NATS was actually created by the same person who created TIBCO RV: https://twitter.com/derekcollison/status/1163299967089254402

Derek seems like the kind of person who might know a thing or two about messaging systems.

I really like what NATS has become, and I do appreciate the simplicity of the protocol.