zlacker

[parent] [thread] 8 comments
1. KirinD+(OP)[view] [source] 2019-05-27 15:08:55
It's exactly the same case as SQS in that if your disk interface dies in mid write of a message you lose it the same as if your interface to SQS or SQS itself dies mid-write, you lose it.

The rest of the sentence is a challenge, "show me a data loss bug in SQS that extends beyond the failure cases you've already implicitly agreed to using postgres itself."

replies(2): >>gmueck+9c >>crypto+tQ
2. gmueck+9c[view] [source] 2019-05-27 16:48:04
>>KirinD+(OP)
What possibilities are there for a write to fail that the database server cannot react to? I am under the impression that a write error like this would be reported as a failure to run the statement and the application is responsible for handling that.
replies(1): >>KirinD+wi
◧◩
3. KirinD+wi[view] [source] [discussion] 2019-05-27 17:35:04
>>gmueck+9c
The box turning off, the disk interface failing, and/or the link to the database failing in mid instruction.

Same as SQS.

replies(3): >>xyzzy_+tv >>z3t4+wy >>mcquee+jD
◧◩◪
4. xyzzy_+tv[view] [source] [discussion] 2019-05-27 19:22:29
>>KirinD+wi
All of those would result in a write failure from the application's perspective, which is fine, and must be accounted for regardless (e.g. retry, two phase commit, log an error, whatever).
◧◩◪
5. z3t4+wy[view] [source] [discussion] 2019-05-27 19:53:24
>>KirinD+wi
Also the FS might report the data as written but its actually in a write cache and will be lost if the plug is pulled.
replies(1): >>gmueck+5I
◧◩◪
6. mcquee+jD[view] [source] [discussion] 2019-05-27 20:44:33
>>KirinD+wi
But you have to explicitly delete the message from SQS, right? You'd only delete after confirming you processed the message, right? So if you die mid-instruction in processing a message, the message just re-appears in the SQS queue after the visibility timeout.
replies(1): >>KirinD+HV
◧◩◪◨
7. gmueck+5I[view] [source] [discussion] 2019-05-27 21:31:53
>>z3t4+wy
Not an issue if you follow th e recommendations in the PostgeSQL documentation. The way to comfigure write caches is described there.
8. crypto+tQ[view] [source] 2019-05-27 23:11:44
>>KirinD+(OP)
SKIP LOCKED is a read-side feature. PG is a problem for u/andrewstuart because they have to manage it themselves (unless they're getting a PG service from a cloudy provider, say) and that's harder than it is for Amazon to manage SQS.
◧◩◪◨
9. KirinD+HV[view] [source] [discussion] 2019-05-28 00:34:39
>>mcquee+jD
That's a feature, and one your have to replicate in your postgres queue.
[go to top]