zlacker

>>goranm+(OP)
I'm pretty skeptical of the tipping section. Sure, it might work, but the two examples are a bit suspect. The first example relies on a tweet lacking in context that doesn't actually show the system prompts and outputs. (The author's "reproduction" appears to be something completely different and n=1.) The second example uses wildly different systems prompts, and I think it's far more likely that referencing Stack Overflow results in a more "optimized" solution than offering a tip.

>>whakim+8S
Yeah, the folks working on aider (AI pair programming) [1] found that these kind of tricks reduced performance for them.

I’m pretty confident there will be situations where you can measure a statistically significant performance improvement by offering a tip or telling the model you have no hands, but I’m not convinced that it’s a universal best practice.

A big issue is that a lot of the advice you see around prompting is (imo) just the output of someone playing with GPT for a bit and noticing something cool. Without actual rigorous evals, these findings are basically just superstitions

[1]: https://aider.chat/docs/unified-diffs.html