I read "Turning the database inside out" by Martin Kleppman. I've also been thinking about better ways of providing concise context for LLMs. Both have to do with stream processing. Especially struck by the Kleppman description of state made up of derived data from an append-only log of immutable facts. Which I think is also how people are starting to do memory for agents.

For the agents to use that memory they then do an embedding (+ keyword?) search over the immutable facts and put them in context. Similarly you could have an LLM step that takes really a lot of the relevant facts and summarizes them and then feeds that summary as context to the LLM. Some questions of where you do the cutoff? Top 20 would work most of the time but maybe not always?

Fun to spot the similarities, anyway. I kind of want to experiment with a really transparent memory mechanism - maybe a constraint systems example where anybody can contribute a memory - where a memory is almost like a twitter post and then you can talk to an LLM that checks memory first... also showing which memories were used.

Relates to the fact that a log of immutable facts also has a lot in common with something like a person's social media feed. You might want to do the same sort of stream processing to something like a twitter feed.

The questions are again about which level to run it at. Do you do it on demand? Do you generate a summary and update it at regular intervals? How do you let the user modify it without it being re-overwritten on updates?