zlacker

fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up.

I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities.

I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like stripe) through Envoy as a mitm proxy which injects the real creds after verifying the macaroon.

It is such a powerful pattern. I'm always worried about leaking sensitive keys through prompt injection attacks (or just sending them to anthropic), but in this model you can attenuate the keys (both capabilities & validity window) client side. The Envoy proxy lives inside my flycast network so it can't be accessed externally.

It would be so cool if fly built something like this into sprites.dev (though I can see how it would be spooky to have fly install their own certs for stripe, etc...)

>>dtkav+(OP)
If you read Ben Toews work on the tokenizer you have a good sense of where I want Sprites to go with key leaks and prompt injection:

https://fly.io/blog/tokenized-tokens/

>>tptace+s
Awesome stuff! Thanks for the reply.

Tokenizer is an explicit proxy though right?

My use case is very similar, but I wanted a transparent proxy so I could run unmodified scripts. It is a tricky design decision though.

I also mount a little fuse filesystem that mints macaroon on read (with a shorter lifetime, probably inspired by y'all but i forget from where).

I work on realtime collaboration of markdown files (currently in Obsidian), which has become a shared-context substrate for agents, skills, etc.. Our own company workspace has skills that have scoped access to fly, stripe, gmail, etc. We're definitely drinking the file-over-app personal-software-for-teams Kool-Aid, so the problem space for us includes access control and auditing.

Love your work :)

>>dtkav+74
We have enough control over the execution environment in a Sprite (unlike a Fly Machine, where the implied Linux contract we have with our users gets in the way) that we can trivially hide explicit proxies.

We can also attach Macaroons to Fly Machines and Sprites for configurable ambient privileges, something I've wanted us to expose as a feature for a very long time.

>>tptace+Fq1
Awesome, i look forward to that. I think that could be a major differentiator for sprites. I wish i could work on that problem at fly.io scale.

What is the contract with sprites? Is it just built-with-linux but not promising Linux? Or is it more like a machine but y'all control the container image?

>>dtkav+xP1
There's no "formal" contract in either place but people running on Fly Machines expect that there's nothing at all between them and the kernel, and we don't have that expectation in Sprites; we can do whatever we want. :)

I don't want to get too far into the rest of the details only because I'm writing this up for next week. They're not that interesting technically, but they're a really big deal for us in other ways.

>>tptace+IP1
Great, i look forward to reading it.

>>dtkav+(OP)
Did you write up anything about this? Is this off the shelf behavior for Envoy or did you create this API yourself?

>>CGames+Ge2
I can open source it next week when i get a chance.

>>CGames+Ge2
sorry for the delayed response. I ended posting on this [0] thread where they (Formal) are doing something similar.

Here's the repo [1]. I modified it a bit to post publicly and remove the details of my setup within my tailnet/flycast network.

[0] >>46605155

[1] https://github.com/dtkav/agent-creds