Once: Cache CLI commands (github.com)

85 pointsby baquero12 hours ago37 comments

ShakataGaNai 4 hours ago

OMG. `op read` with agents. Holy shit.

I first clicked this with just a mild curiosity. But as soon as I saw the 1password example. Lightning.

I try to do everything "the right way", no secrets on disk, gitops, the whole shebang. Even for my personal projects. But when working with API keys and agents it can get so frustrating. I refuse to admit to the number of times where I've told the agent "Please write the 1Password creds to .env and use .env instead because I'm tired of clicking approve on my watch 15 times for every test run."

alex0ptr 3 hours ago

A brother from another mother. I was afraid no one has similar workflows. :)

aktau 10 hours ago

These kinds of tools are very useful. I have my own called memo (which is just a shell script: https://github.com/aktau/dotfiles/blob/master/bin/memo).

It was discussed in https://news.ycombinator.com/item?id=45670052 and others chimed in with their own (like bkt(1) and up(1)).

The main differences I see:

  - memo doesn't need to be built (its a shell script)
  - once keeps output cache in a running daemon. By contrast, memo stores content under /tmp with whatever the best compression available is (it prefers zstd). There's trade-offs in that. More pareto-optimal from the security front may be to have a sort of session key and to compress-then-encrypt files on disk.

thomascountz 7 hours ago

Hey! I use your shell script after seeing it on HN awhile ago. It's very handy (I find myself reaching for it as if it's a built-in) and I appreciate its design (I learned a few things reading it)!

__MatrixMan__ 8 hours ago

I dream of applications that expose sufficient metadata about their outputs such that the OS can know if rerunning is necessary without me needing to specify.

Nixos does this for builds, but I've not seen it generalized to arbitrary processes.

jchw 5 hours ago

You can, in many cases, mangle other use cases into the Nix build system, to varying degrees of success, though there are plenty of complexities with that.

jedbrooke 5 hours ago

this is kind of the idea behind functional programming. A pure function is one whose output depends only on its input, and no other hidden state, so that let’s you just cache a lut of input -> output for anything you’ve computed before. The tricky thing is anything that reads/writes to memory/disk/network etc. is no longer a pure function and can’t be cached easily

PinkSheep 4 hours ago

...unless you marry it to a CoW fs that can let it travel back in time :)

__MatrixMan__ 4 hours ago

Seems like your filesystem could keep track of whether the depended-on bits have changed and invalidate a cache if they did.

And your network stack could handle things like: "hey last time you gave me 59MB with hash, 0xabcdef123455, can I give my caller a cached copy or has that changed?" ...so that while you're iterating on a bash pipeline that pulls data and transforms it, the data only gets pulled once.

Even attached peripherals like a scanner are in a position to know whether the old thing is still in there or whether a new thing has been loaded such that they only re-scan when there's a new thingy present to scan.

I love FP but having it at the OS level would be something new.

xuhu 10 hours ago

I wish this worked without prefixing the commands with "once". Especially if I ran a command with verbose output and afterwards decided I wanted to grep something from it. Terminals have the output in their scrollback buffer and maybe it's just a matter of writing an "output" command that lets me run:

    $ cmake ..
    $ output | grep "libssl version"

ctippett 9 hours ago

You could probably achieve something like that using Fish shell and their fish_preexec and fish_postexec event hooks.

Edit: Nope, looks like you can't. You only get the invoked command and no output.

stirfish 8 hours ago

Could you use preexec + tee?

lelanthran 5 hours ago

I run bash from inside vim and nvim. At anytime I can drop into vim and use the entire scrolled history as a buffer.

deadbunny 10 hours ago

I've always wanted to deal with cache invalidation in my terminal.

_doctor_love 5 hours ago

Call it "memoized CLI" instead, you'll sound super cool

hecomo 6 hours ago

This looks awesome. Thanks for making this.

Been using this, for similar cli output catching. https://github.com/dimo414/bkt

Wondering what you think about the two, and what are the good reasons to use one vs the other? (Maybe: once is more actively developed? bkt hasn't been active for a year).

alex0ptr 3 hours ago

Wow this is incredibly similar. I guess it seems that the notable difference is that the cached results are written to a directory - which is exactly what I wanted to avoid. So depends on what you want to cache I guess.

giancarlostoro 8 hours ago

> The typical use case: reading secrets from 1Password without approving every single read with your fingerprint.

Uh I dont know about that one chief.

alex0ptr 8 hours ago

Yes - but I'm out of ideas. How else support long running agents without leaving secrets in files or by default exposed in the environment. That way I get notified the first time before they ask for credentials.

giancarlostoro 8 hours ago

The same agents that could potentially leak your secrets? I would rather not give a hacker a cached session that unlocks the keys to the kingdom.

I'll take security by inconvenience over building what becomes the primary reason for a security incident.

alex0ptr 8 hours ago

If I already approved it once I already have to assume it could have been leaked. The cache doesn't change much about it if the agent / tenant is asking for something it already has.

giancarlostoro 3 hours ago

It slightly opens what should be a really short and brief window of access.

mehackernewsacc 8 hours ago

Does https://secretspec.dev/ address your use case?

alex0ptr 8 hours ago

That looks pretty cool - even supports caching. Will take a deeper look. Thx

devmor 7 hours ago

Don't give secrets to agents at all that you don't plan on revoking immediately after.

If you have allowed an agent to access any kind of credential, you should assume it is no longer private.

stryan 6 hours ago

Fnox (by jdx of mise) supports fetching secrets and caching the results either in local age encrypted files or in a background daemon (in memory only) to minimize repeated gets. The daemon is per terminal instance too I believe so you fetching a secret once doesn't store it for the whole session.

It's not a perfect fix but it keeps secrets out of env with (so far for me) minimal inconvenience.

ShakataGaNai 4 hours ago

I 100% love this idea and example and I very much appreciate it.

baquero 12 hours ago

Do you have CLI calls that you want to cache? Here you go!

stingraycharles 10 hours ago

Seems well-designed, but what’s the use case? I’m trying to think of them but my creativity is failing me.

frizlab 10 hours ago

From the example in the Readme, I guess retrieving a token from an API, using a secret fetched from a secure vault that requests a password or TouchID validation.

Not sure if there could be other interesting uses. I can’t think of one anyways.

alex0ptr 8 hours ago

Yes, exactly. This is especially frustrating when I leave an agent running for a long time and it gets stuck on my build scripts because it's waiting for my approval.

I just want to avoid leaving my credentials and secrets on the filesystem.

EDIT: I use a lot of direnv / mise. So reloading credentials with different values is common.

whilenot-dev 10 hours ago

I have the same question, especially since any CLI output can be stored explicitly in some variable or temp file. What's the advantage of storing the output implicitly?

alex0ptr 8 hours ago

I don't want to store them in a file since I don't trust my agents with that data. Instead, the daemon secures the values using/under an HMAC key, making them virtually impossible to guess.

adregan 10 hours ago

I have a small fish function that does something similar. It saves output to a file in XDG_CACHE_HOME. I wouldn’t use it for sensitive items like passwords or tokens in the examples, but I do use it for scripts that need to fetch some data from the network.

For example, I have a script for automating the creation of PRs which fetches the available labels for a repo from github and presents them with fzf multi select. I store the labels with a TTL of a week so that I don’t have to fetch them every time and the script compares the file’s age against the desired TTL to invalidate.

I find it useful for augmenting other programs, but I’m not typically using it on the cli directly.

sleepybrett 2 hours ago

I might use this when i make massive `kubectl get -o json` calls to store the output so i can noodle with some jq i'm using to filter it. Right now it looks like OUTPUT=$(kubectl... -o json); echo $OUTPUT | jq '.blahblahblah' .. this seems like it might be more useful and if i'm working so long my ttl expires it will refetch say every 2m.. if i'm reading this properly.

1vuio0pswjnm7 5 hours ago

I just use tmux buffers

But I'm not running agents, I have different needs

newadays 7 hours ago

Congratulations. What is Word Error Rate? We have a few use cases for something like that at newcare.app