Morning 10 minute freewrite scrawl. I want to return to the timer concept today. Build out the frontend with the iframe portals. Reduce data scope down to just startTime, duration, and label (leave calendar time blocks for later). I think the shuffle mechanism will be the real small proof of what shared data/structure could do for generated apps.

Remembered walking today that the two pieces should be separate - timer is in charge of displaying the apps but the iframe links could go anywhere. I'll probably have a convenience repo to start with. I wonder how much focus I should put on getting a live react playground working with it - I wonder if I could use one of the LLM canvas playgrounds to actually run it - with the share link - though how stable are those share links really?

Some of my recent work experience from submissions should come in handy - it's always surprising how the most mundane projects have a way of working themselves back in.

Ideas-wise also really curious about this guide to living structures by Ken Isaacs that was referenced on Merveilles. Have to get my printer connection on this laptop working and then I'll print it out.

Some of his other work reminded me about discussions about transparency layers. Think I should get some of those again - will also be useful for planning out tattoos.

I also had the related idea for a drawing/image editing app where each layer is an iframe. This follows up the work I'm doing with scrape. The worry with scrape is just that the layer business is overwhelming the scrape effect which is my main focus. Not the first time that's happened so of course I'm looking for a place to modularize or generalize it. Reminds me again of 'carving at the joints'. I want to do a better job of remembering key touchstone phrases - I should add carving at the joints to the quote page. It might be the one I think of most.

That got me thinking about the connection between frames and layers. Based partly on my other work on frame trying to isolate GIF and video frames. I can't tell if I have too many unfinished constraint systems projects right now or it's just the natural mess of working things out.

I really like combining - or playing with combining - frames and layers. Frames are time based, or sequential based? Layers have an order too. Playing between the two a practical problem is there's often probably just too many frames - if I explode them all there's going to be too little difference between them and too many to run. But FPS would be the natural solve for that right? Maybe even playing with the idea of keyframes.

I've been thinking about what makes an app or program feel coherent. Like they have an underlying logic.

I think our appreciation of coherency comes from interaction with the natural world, where coherency emerges out of the underlying physics.

Interfaces are not chained to physics but they use it as metaphor a lot.

I've always been drawn to idea that coherency in software apps should come from "going with the grain", sticking to how the underlying software underneath works. That maybe there's something there that could give a foundation like the underlying physics - though it's definitely a different shape of thing than physics is.

I do a lot of this with the image apps, where I try and use drawImage, because it feels like that is the underlying structure of computer drawing - copying bitmaps from one place to another. Because it's 'natural' it can also be done very fast, and that opens up interesting possibilities in software.

A program can feel coherent if it feels like it was made by someone with a point-of-view. Like they thought this is how this process should work. This is connected to the idea of coherency and as emerging from an underlying process, but it's not necessarily the same, I think?

I think about this in relation to music. What makes music feel powerful is if it feels like it comes from a point-of-view. It can take wildly different forms but I think that point-of-view is always what's interesting and compelling to me. Again this seems connected to what makes for interesting software, but it's not the same.

A point-of-view comes across from decisions made at different scales (big-picture and small-picture). Both should have influenced each other in the making.

how long (with all of the generative media) before the fact of a camera capturing only what is present before it becomes novel and more interesting again?

I guess it would be more like "the act of arranging the physical world to portray an image" which is then captured by the camera that is what will be interesting. and already is?

extremely fuzzy territory. but I want to keep thinking about it.

One of the big sticking points in my "react apps that use canvas" experiments keeps being "where does the canvas live". Most often I put it in the .tsx and use a ref to access it in functions and useEffects. It works but never feels great. If I have canvases that are never going to be rendered I can just create them where needed - caching behind a ref to make sure I don't end up creating each call (possibly it wouldn't matter performance-wise if I did).

Maybe I should sort of allocate and create all my canvases in the functional code and then also render "render" canvases in the UI that I draw the "real" ones too. But sometimes (like my current project) the number of canvases is dynamic - dependent on something else. In that case I need to update that dynamic number in both DOM and drawing code - having it in react state so it reliably triggers a DOM rerender helps, although sometimes I need to run drawing code without waiting for the state update to clear, leading to more helper refs...

the stack and the heap but the heap is an infinite canvas (is this something)?

the reverse chronological list and the infinite canvas.

One reason to make things is to develop a deeper relationship with the things you put into them.

When songs I put on mix CDs - particularly ones I thought hard about sequencing or mixing into another song - come on I'm transported back to when I was making them.

I just got an email from college where it showed the college logo and it brought me back to trying to find a place for that logo on the college band CDs we made.

And it wasn't something inherit to those songs or logos that fixed those memories for me - it was me trying to use them - where I had to get a more in-depth feel for them their shape and edges, as I tried to fit them into something else.

How do you handle image resolution and file size in a spatial canvas.

In a file browser image files all feel about the same, even though they can vary widely in resolution and size. When you drop them into a spatial canvas web app, that app has to decide how to size. The most 'honest' way would probably be to render them at the resolution of the files - but i think this would often be surprising to people - they won't understand why some images are twice the size of others.

So you want to normalize, probably pick a max size and scale big ones down to that - then you have to think about how you want to manage that for performance. Do you use canvas to resize it down 'for real'? That means if it was bigger you lose that extra resolution. What if the user later actually wants in that big? Probably you want to keep the original size in case needed - but then how do you indicate to the user that some of the images on the canvas could size up without degradation and some would look rough. You list the resolution of course - but if you're a user working on arranging things it feels like its forcing you to switch between this spatial flow to think through math resolution.

You could show a red outline or something when something gets resized above resolution. But while that makes sense in isolation, in the context of an app with a bunch of other things going on I think it could be confusing noise. Plus sometimes if it's just getting scaled up a little it's probably fine - and you don't want to create the feeling of a problem where there isn't one.

My impulse is always to find a way to give the user information more directly so they can then build intuition around it. But it's hard in this one.

You almost want to indicate heaviness or something like that for big files.

It's a knotty problem but that's also probably means there's some fun solutions (or at least experiments) to be done with it.

At heart a question of whether it should be "don't make the user think about it" or "give the user enough info to think about it clearly". Is it real complexity or incidental complexity. And of course this partly dependes on where the output is headed... to be printed? To be imported into another program?

I think I pinpointed a misgiving I have about general collaborative LLM use: that the LLM will never stop and say "I don't know" or "I don't have this mental model, can you help me understand it". That's the fundamental limitation of being trained to plausibly (not correctly) generate text.

I guess it gets fuzzier with the reinforcement learning layered on top, right? But can you really reinforcement train it to say "I don't know"? You can train it to say "I don't know" based on patterns of when it's training data shows people saying "I don't know". But it's against its grain, right?

If I had a collaborator who never said "I don't know" that would be a bad collaborator - I'd feel like I never had a steady foundation with them. Sometimes this does kind of happen because people feel embarrassed not to know things - they may commit code they don't understand and not want to admit it. But I feel like a key part of me getting to know someone and working with them is getting past that to where we can talk about what doesn't make sense - because a solid, shared mental model benefits us both.

I definitely use LLMs to speed up my development, but mostly in the form of autosuggest, which to me feels like the most 'honest' or 'with the grain' format for it. I'll also have it generate more generic components or bash scripts or database logic. But even then I feel like I really want to keep thinking of it as a form of 'codegen', of 'predictive text', and not a collaborator.

I don't know the consequences of all this - maybe accuracy crosses a threshold where the hallucination tradeoff is negligible. Maybe grounding in documentation helps a lot. And possibility you can do some chaining of an evaluator model... Ideally I think you'd get a confidence or 'novelty' score from the model itself, that you'd learn to factor into your intuition about how to treat what it's generated, but I don't see a lot of progress or talk about that lately.

Maybe this is just an argument that there's some metaphor beyond chat is called for. Something that signals its going to 'run with idea' or 'play it out', not 'answer your question definitively'.

One of my favorite pieces of writing is The Web's Grain by Frank Chimero https://frankchimero.com/blog/2015/the-webs-grain/. Lately I've been trying to think about what the grain of LLM-related development could be.

There's the interface for using the LLM, where Cursor has done a lot of work. But I want to think about what kind of websites and software you can build with LLM assistance.

There's using LLMs as a piece of the pipeline - unstructured data into a JSON schema, for example. There are possibilities there, I think. The dilemma is probably giving the user a mental model of what they can do - LLMs can unlock a new level of fuzzy input, but will it be clear to the user where and how that input breaks down? Will they run to the edges and then be frustrated there?

Then there's codegen and vibe-coding. This is such a knotty feedback-loop. It's safe to say LLMs are good at boilerplate, particularly the most popular project boilerplate. So one approach to going with the grain could be picking a stable tech stack and coding according to convention.

A really interesting piece of this is how libraries fit in. I think a lot of people's favorite vibe-coded examples use libraries like three.js or matter.js, to simulate worlds or physics. I think in general there's an imprecise linking of a model's 'intelligence' and physics sims, where we imagine a model has an understanding of the world, but what it actually has is ability to hook into a carefully designed library intended to simulate the world.

But I do love physics sims, and I think there is some energy there where we've always wanted physics sims in our software, but the complexity and the clash between digital logic and physical world logic has been to much to contain. Maybe LLMs can be the glue that helps us meld those two together.

Another place LLMs have been useful for me is approaching FFMPEG syntax commands. Another powerful library case where it allows me to dip into the powerful capabilities there and dip out without having to puzzle over syntax.

There's definitely a flow where you mainly use LLMs for boilerplate and for tech adjacent to your main focus. I've done that on this blog with most of the database logic being codegen. I can see a near-term future where my focus is experimental UIs, stuff that can't be easily generated, and I lean on LLMs to generate the surrounding tech.

But I don't think that's exactly approaching LLMs as Chimero approaches websites in the article - it's not looking at what 'LLMs' want to make. For that I don't have an answer currently beyond popular web stacks and this interesting relationship to libraries. Going to keep thinking, though.

I've been working on building rough shelves out of 2x4s in our basement. Partly for storage, partly to get more of a feel for how to frame things with 2x4s. The big improvement on the second set was using 3" deck screws. The process made me think about learning and how it feels different to have solved a problem versus avoiding having a problem.

On the first set I had just got some long phillips head screws from the hardware store, but they kept stripping out. I had some idea that construction screws used other drive heads - and it became clear that this was probably the reason.

I switched to star head screws and those drove better, if still a little rough. Then I watched this https://www.youtube.com/watch?v=SG6FJCWYeRU video and followed the advice to get 3" deck screws.

For the second set of shelves I used the deck screws, they drove easier, and the space at the top of the screw meant the two 2x4s pulled tight.

Thinking about learning: I could have got all this information at the start, but it feels so much more real to have gone through the process of encountering the problem - really felt that problem - and then the difference of the solution. I love that process.

I think that sort of feeling is a key part of building intuition.

Another interesting puzzle for me is why intuition is hard to pass on - having experienced this issue could I now tell someone else that the secret to building 2x4 shelves is to use deck screws? I would probably mention it. But I think the tricky part is there's lots of other parts to building shelves, which the person I'm talking to might know more or less about. It's a huge set of information. I (and I think a lot of people) need to be in the process and solving problems step-by-step versus being given a giant checklist at the start. At least that's how I enjoy doing it.