zlacker

[parent] [thread] 5 comments
1. lmm+(OP)[view] [source] 2021-11-26 11:41:01
> All businesses operate on the current view of data.

All businesses operate in response to events. Most of the things you do are because x happened rather than because the current state of the world is y.

> In our "live datafows" etc. we use a pre-determined set of queries that are guaranteed to run multiple orders of magnitude faster in a relational database on the current view of data than having to reconstruct all the data from an unbounded stream of raw events.

If you have a pre-determined set of queries, you can put together a corresponding set of stream transformations that will compute the results you need much faster than querying a relational database.

> I roll back the transaction. As simple as that.

And then what, completely discard the attempt without even a record that it happened?

replies(1): >>dmitri+e1
2. dmitri+e1[view] [source] 2021-11-26 11:50:06
>>lmm+(OP)
> All businesses operate in response to events.

Yes, but once an event happens, business needs access to current state of data.

> If you have a pre-determined set of queries, you can put together a corresponding set of stream transformations that will compute the results you need much faster than querying a relational database.

No, it won't. Because you won't be able to run "a corresponding set of transformations" on, say, a million clients.

You can, however, easily query this measly set on a laptop with an "overengineered" relational database.

> completely discard the attempt without even a record that it happened?

Somehow in your world audit logging doesn't exist.

replies(1): >>lmm+84
◧◩
3. lmm+84[view] [source] [discussion] 2021-11-26 12:16:34
>>dmitri+e1
> No, it won't. Because you won't be able to run "a corresponding set of transformations" on, say, a million clients.

Of course you can. It's a subset of the same computation, you're just doing it in a different place.

> Somehow in your world audit logging doesn't exist.

If you have to use a separate "audit logging" datastore to augment your relational database then I think you've proven my point.

replies(2): >>dmitri+37 >>eklavy+j9
◧◩◪
4. dmitri+37[view] [source] [discussion] 2021-11-26 12:44:02
>>lmm+84
> > No, it won't. Because you won't be able to run "a corresponding set of transformations" on, say, a million clients.

> Of course you can.

Of course, you can't. Because you can't run a million transformations. Whereas querying specific data for any of the one million clients? It's trivial on a relational database.

Moreover. If you need new queries into data, it's again trivial. Because you have the current view of your data, and you don't need to recalculate everything from the beginning of time just because your requirements ever so slightly changed.

> If you have to use a separate "audit logging" datastore to augment your relational database then I think you've proven my point.

No, I haven't.

It's funny, however, that you think that businesses don't require a current view of data and need to re-calc everything from scratch.

◧◩◪
5. eklavy+j9[view] [source] [discussion] 2021-11-26 13:03:44
>>lmm+84
I would like to disagree, in my experience eventing/cqrs are wonderful solutions to a set of problems (specially where event by event playback is a primary functionality). In most other cases it’s overkill and maintaining a snapshot of state, which like you said is inevitable even in the event log case, is imperative.

There are just too many scenarios where not having transactions is dog slow or really really unwieldy.

replies(1): >>lmm+QI6
◧◩◪◨
6. lmm+QI6[view] [source] [discussion] 2021-11-29 01:10:12
>>eklavy+j9
In my experience when you do something that requires transactions - i.e. some complicated calculation based on the current state of the world that you can't reduce to a sequence of events and transformations between them - you always end up regretting it. Almost by definition, you can't reproduce what the transaction was supposed to do, and if there are bugs in your logic then you can't fix them; often you can't even detect that they happened.
[go to top]