2023-07-14 09:33:09 <KipIngram   > Still puzzling over exactly how to handle the sync/async nature of thread to thread communication in this new system.
2023-07-14 09:33:32 <KipIngram   > I think I want the default to be synchronous - non-buffered - pipes.  But there are places where there need to be buffers.
2023-07-14 09:33:59 <KipIngram   > For example, a process needs to be able to write an output byte to the console without blocking even if it doesn't currently own the physical console.
2023-07-14 09:34:04 <KipIngram   > Buffering seems reuired there.
2023-07-14 09:34:50 <KipIngram   > I'm picture all such processes will have a large enough output buffer to paint an entire screen, so that when I switch the console over to a process I can put the right stuff on-screen.
2023-07-14 09:36:14 <KipIngram   > That's a fairly large buffer - my screen will hold 10193 characters, and with utf8 in play that more bytes.  Probably 16kB would do, but I could concoct a screen that would take more than that.
2023-07-14 09:36:54 <KipIngram   > In most cases a smaller one would work - one 4kB block would probably work 99% of the time.
2023-07-14 09:41:31 <KipIngram   > I'm also struggling a little with how all of this connects up with the other thoughts I had re: event-driven interfaces.  That conversation we had a few months ago that led me to consider the SDL2 stuff.
2023-07-14 09:42:30 <KipIngram   > With a traditional console interface, that output buffer is just a continuous stream of bytes.  But SDL2 scrambles that all up.
2023-07-14 09:42:54 <KipIngram   > Since you can move the cursor around all over the place.
2023-07-14 09:43:41 <KipIngram   > You could still do something similar, though - the output stream would just record all of the EVENTS that happened, in sequnetial order, and you could still "replay" that in a similar sort of way.
2023-07-14 09:44:02 <KipIngram   > But it does wipe out the possibility of slapping that whole buffer to the screen with one syscall.
2023-07-14 09:45:17 <thrig       > latency vs. throughput, again
2023-07-14 09:45:29 <KipIngram   > Again - yah.  :-(
2023-07-14 09:45:47 <KipIngram   > I won't be happy with it if it's sluggish.
2023-07-14 09:46:50 <KipIngram   > I want to think that many years ago I saw stuff that implied you could set boundaries of a window on a terminal, and the text you sent to it would confine itself to that window - that window would scroll, and so on, in the usual way.
2023-07-14 09:47:04 <KipIngram   > Is that actually a possibility, or am I just making that up?
2023-07-14 09:48:02 <KipIngram   > It would be great if I could just send the window boundaries into the system and then have it just "work" as though that window was a minature terminal.
2023-07-14 09:49:15 <KipIngram   > I'm also noting that I might have a window on the screen that is serving as my console interface, in which case when I hit Enter I want the next text to be interpreted in the usual way, or I might have a window that is just a text block I'm editing.  Those are "almost" the same thing - it's just the processing of certain events that's different.
2023-07-14 09:59:21 <KipIngram   > Oh, hmmm.  There is an app in my package repository - vttest.
2023-07-14 09:59:28 <KipIngram   > It runs various terminal capability tests.
2023-07-14 09:59:42 <KipIngram   > I ran it with terminator, terminal, and kitty - none of them did "everything" right.
2023-07-14 10:04:15 <thrig       > probably because there's a bazillion escape sequences to implement
2023-07-14 10:05:10 <Zarutian_iPad> just for graphics there is NAPLPS, RipScrip, and ReGIS
2023-07-14 10:06:12 <KipIngram   > thrig: Yeah, the big emphasis has been on GUIs for decades - I imagine careful terminal support has suffered.
2023-07-14 10:06:17 <Zarutian_iPad> then there is mouse support, text screen cursor movement, and all that jas
2023-07-14 10:06:26 <KipIngram   > Wonder if there even IS a fully compliant terminal program even out there.
2023-07-14 10:06:54 <KipIngram   > Zarutian_iPad - exactly; I'm trying to think out how all this stuff gets "blended."
2023-07-14 10:07:06 <KipIngram   > It's quite a large change from a fully serial terminal setup.
2023-07-14 10:07:42 <KipIngram   > But it seems like what I want is to have the normal console behavior, but to be able to "put it in an arbitrary window."
2023-07-14 10:08:09 <KipIngram   > I KNOW that it's snappy if I paint an entire screen with one syscall.
2023-07-14 10:08:21 <KipIngram   > I THINK it's probably snappy enough if I paint line by line.
2023-07-14 10:08:33 <KipIngram   > But it can't be char by char - that just won't perform well enough.
2023-07-14 10:08:44 <Zarutian_iPad> then there VNC too
2023-07-14 10:08:52 <KipIngram   > That's the reason I implement EMIT on top of TYPE instead of TYPE on top of EMIT.
2023-07-14 10:14:27 <KipIngram   > Also,  looking forward to my ultimate goals, the idea here is to use this software on things I build someday, which I will likely want to communicate with USING MY NOTEBOOK.  So the actual console won't be on the gadget.  So all of this needs to work over a serial type communication link.  "Terminal" is the right model.
2023-07-14 10:14:48 <KipIngram   > In my current system things are hopelessly mixed up - I don't have those aspects separated the way they really need to be.
2023-07-14 10:15:22 <KipIngram   > And I'm not clear at all on exactly what this all implies.
2023-07-14 10:15:54 <KipIngram   > Probably that the "core system" just communicates with the outside owrld via these pipes.
2023-07-14 10:16:03 <KipIngram   > And what's on the other end of the pipe is entirely irrelevant.
2023-07-14 10:16:35 <KipIngram   > The big question is whether the "terminal" part of it is just some chosen existing terminal emulator, or whether it's something I write custom.
2023-07-14 10:22:16 <thrig       > custom terminator, huh
2023-07-14 10:22:20 <thrig       > I mean, terminal
2023-07-14 10:22:59 <KipIngram   > There are features of kitty I like - it supports the delivery and display of images, in-terminal.
2023-07-14 10:23:19 <KipIngram   > In fact, you can even dribble them across slowly, in chunks, and then have them ready to display when you're ready.
2023-07-14 10:23:44 <KipIngram   > It's like there's a "side channel" that is available for communicating with kitty from the application, separate from the normal displayed stream.
2023-07-14 10:23:53 <Zarutian_iPad> sixels!
2023-07-14 10:24:03 <KipIngram   > Not exactly, but right - along those lines.
2023-07-14 10:24:26 <KipIngram   > It has always bugged me that terminals offer pretty much zero image support.
2023-07-14 10:24:34 <KipIngram   > Once again, GUIs have taken all the attention.
2023-07-14 10:24:46 <Zarutian_iPad> and ya sure it just support of one or many of the gfx stuff I mentioned?
2023-07-14 10:25:03 <KipIngram   > Seems like there ought to be a lucrative "middle ground" in there somewhere.
2023-07-14 10:25:18 <KipIngram   > Let me check.
2023-07-14 10:26:32 <KipIngram   > I don't see any of them explicitly mentioned.
2023-07-14 10:29:05 <KipIngram   > So, let's take as the base case a set of several console windows open into different processes running on the gadget.  That means the gadget will be sinding several streams of output to the host, and the host will have to know how to route those to the correct window on-screen.
2023-07-14 10:29:18 <KipIngram   > Gadget doesn't care what's being donw with that output, or even if ANYTHING is being done with it.
2023-07-14 10:29:37 <KipIngram   > So the channel between the gadget and the host needs to support stream multiplexing.
2023-07-14 10:30:05 <KipIngram   > The histories needed to re-draw screens when they're made visible again all lives on the host.
2023-07-14 10:31:17 <KipIngram   > If everything is running on my notebook, then I guess that means there are separate threads for gadget side and host side that are in communication in a similar way.
2023-07-14 10:31:37 <KipIngram   > Same structure - just happeing to be on the same system.
2023-07-14 10:48:44 <KipIngram   > Anyway, it's clearly POSSIBLE for the hardware to deliver the kind of performance I want, because there exist things like terminator, tmux, and so on that do a perfectly fine job of showing multiple separate windows on the screen in a high-performance way.
2023-07-14 10:49:13 <KipIngram   > So it CAN be done - doesn't mean that any of those tools make the capability available at an application level the way I want.
2023-07-14 10:49:25 <thrig       > it might involve a lot of curses
2023-07-14 10:49:32 <KipIngram   > I may wind up having to write my own terminal emulator to get everything I want.
2023-07-14 10:51:35 <KipIngram   > There are other complexities too - in the case of big powerful "gadgets" I may want my source code on the gadget, but on the other hand I may be doing something with a really low end processor and want the source on the host - a more "tethered" thing.
2023-07-14 10:51:52 <KipIngram   > So I feel like I need to have clean support for either option.
2023-07-14 10:52:25 <KipIngram   > What it's kind of shaping into is a distributed system that can more or less have pieces on either side of the river, without it really mattering in any important way.
2023-07-14 10:53:53 <KipIngram   > Reminds me of the little chat we had that day about laying blocks on top of ipfs.  You load a block - but where it comes from isn't terribly relevant.
2023-07-14 10:54:37 <thrig       > might be if Bob has to rummage around in a tape warehouse first to find the tape
2023-07-14 11:07:38 <KipIngram   > :-)
2023-07-14 11:07:58 <KipIngram   > Well, it just brings up the idea of distributed storage, which looks like it applies here too.
2023-07-14 11:08:42 <KipIngram   > Maybe it's a device level thing - maybe I can have "locally backed" device vs. "remote backed device."
2023-07-14 11:09:02 <KipIngram   > A block device.
2023-07-14 11:10:43 <KipIngram   > Or maybe one assigns a range of blocks to some "supporting capability."  So still one set of blocks, but some might be local and some remote.
2023-07-14 11:11:26 <KipIngram   > At the moment that last option seems appealing to me, but I haven't given it much thought yet.
2023-07-14 11:12:18 <KipIngram   > It would keep Forth's block model intact - having to specify a device, or flip back and forth between a pair of devices, would be a fairly significant change.
2023-07-14 11:12:57 <KipIngram   > It would require processing the block number to decide what to do, though.
2023-07-14 11:13:34 <KipIngram   > That wouldn't have to be in the fast path, though.
2023-07-14 11:13:46 <KipIngram   > Could be done at the blk-rd / blk-wr level.
2023-07-14 11:16:55 <KipIngram   > This would all need to work extended to include file systems, too, since I do plan on adding that later.  Distributed file system, spread across multiple machines, all running "this system."
2023-07-14 11:17:16 <KipIngram   > Accessing a file you wouldn't actually need to know where it was stored.
2023-07-14 11:17:29 <KipIngram   > This feels a little plan9 like.
2023-07-14 11:17:59 <thrig       > and probably tricky and hard to implement
2023-07-14 11:19:00 <KipIngram   > Hence the need to spend plenty of time thinking about it.
2023-07-14 11:19:30 <KipIngram   > I'm not writing a file system today, but when doing the early parts of it I'd rather try to avoid painting myself into a corner.
2023-07-14 11:19:50 <KipIngram   > This kind of thing is exactly why I like to think a lot before implementing.
2023-07-14 11:20:05 <KipIngram   > Which unfortunately often becomes "never implementing."  :-(
2023-07-14 11:20:38 <KipIngram   > But if I go off on this kind of thing half-cocked I'll almost certainly hit a wall at some point.
2023-07-14 11:20:54 <KipIngram   > Not necessarily one that keeps me from "getting there," but one that makes things ugly.
2023-07-14 11:29:30 <thrig       > that waterfall does look kinda high and scary, wouldn't want to go over it too hasty
2023-07-14 11:30:39 <thrig       > https://pbfcomics.com/comics/raft-friends/
2023-07-14 11:42:08 <KipIngram   > Ouch.
2023-07-14 13:03:28 <KipIngram   > Ok, I suspect there will be several ways to access off-system storage.  I do like the "block pool" idea, where some range of blocks gets moved in from or out to a remote system.  But I imagine there will be a separate remote file system capability, that will likely look a lot like plan9's.
2023-07-14 13:03:35 <KipIngram   > Just different things.
2023-07-14 13:04:19 <KipIngram   > And if they're different they won't interact in troublesome ways.
2023-07-14 16:00:18 <veltas      > KipIngram: I've said before, I don't think switching threads is frequent enough that you need to worry about overhead so much
2023-07-14 16:00:39 <veltas      > Doesn't hurt but I don't think redundant pushes etc will break the bank
2023-07-14 16:09:01 <KipIngram   > True, but in this case avoiding it is trivially easy, so... why not?
2023-07-14 16:20:04 <veltas      > Yeah why not
2023-07-14 16:20:22 <veltas      > I don't know, only so much time in the world lol