2023-02-15 08:40:06 <KipIngram   > joe9: I think the difference between a concurrent process approach to this sort of thing vs. a "library call" is that it insulates the application from the implementation.  When the app uses a channel to talk to another process to get some mouse information or something, he may be talking to an entity that handles the bare metal mouse.  Or he may be talking to an intermediate entity that represents
2023-02-15 08:40:08 <KipIngram   > multiplexed access - that process he talks to turns around and interacts with yet another process that's handling the bare metal mouse.
2023-02-15 08:40:20 <KipIngram   > It seems to me it achieves a better modularization of functionality.
2023-02-15 08:41:16 <KipIngram   > Not saying, of course, that you couldn't achieve the same multi-level effect the other way too, but the code would be more complex, with conditional tests (am I talking to bare metal or not) etc.
2023-02-15 08:42:19 <KipIngram   > One of the things Pike emphasized in his paper on using concurrent processes to implement a window manager was that the approach allows you to run the window manager itself in a window, if you want to.  Or a window of a window, etc.
2023-02-15 08:45:25 <KipIngram   > The concurrent approach to the keyboard has the very  nice advantae that the code that is watching the keyboard is *always running*.  You don't want for hte keyboard to be ignored for long periods of time while your app crunches on something, because you won't get good timing of keyboard events, and you need that to discern the escape key from  escape sequences.
2023-02-15 09:00:11 <KipIngram   > Anyway, my hope is that by taking this approach to things I wind up with simpler code, based on the fact that code doing these device specific things are *only* doing that and are guaranteed to run in a "attentive" way.
2023-02-15 09:06:19 <KipIngram   > I did browse the sam paper last night, and noticed that a lot of implementation concepts Pike talked about were very similar to the kind of thoughts I'd had about a b-tree based file system.
2023-02-15 09:07:16 <KipIngram   > I'm starting to think that all this is more likely to work if I go ahead and deal with the file system at the base layer.  So, process switching, process communication, simple memory manager, and basic file system - that may all need to be at the ground floor.
2023-02-15 09:08:40 <KipIngram   > And if I'm going to be working with files, I need to be able to manipulate file names, so something better than "nothing" may be called for on string handling as well.
2023-02-15 10:20:46 <KipIngram   > Turns out Pike covers some territory that I've thought about in the past.  When considering implementation of a console in a fundamentally graphical environment, I definitely did think about having processes maintain data structure that described the purely textual appearance of the screen, and then there would be system software that did the work of making that content appear on the graphical screen.
2023-02-15 10:21:07 <KipIngram   > And that textual data structure would be READABLE, so you could at any time go read what's on the screen at some particular location.
2023-02-15 10:21:24 <thrig       > curses!
2023-02-15 10:21:26 <KipIngram   > So to the application, the console looks like a text device.
2023-02-15 10:21:31 <KipIngram   > Yeah.
2023-02-15 10:21:46 <KipIngram   > The whole business of the bitmaps and so on is just hidden behind the scenes.
2023-02-15 10:22:06 <KipIngram   > Though it is exposed through another device - /dev/draw.
2023-02-15 10:22:49 <KipIngram   > I've griped a lot about how obtuse and hard to get one's head around modern graphics are - this kind of thinking just makes a lot more sense to me.
2023-02-15 10:22:56 <thrig       > fastest /dev/draw East of Tallahassee
2023-02-15 10:23:16 <KipIngram   > That data structure isn't "just an array," like it used to be back in the DOS days, but it is nonetheless a textual representation.
2023-02-15 10:28:09 <Zarutian_iPad> what? you put a draw list there or something?
2023-02-15 10:28:37 <Zarutian_iPad> display list is also another name for it
2023-02-15 10:28:47 <KipIngram   > Well, I don't think I can describe the structure perfectly, but yeah - basically it's a "list of lists."  Each inner list describes line-high boxes across the display.
2023-02-15 10:28:54 <KipIngram   > And the outer list moves down the screen.
2023-02-15 10:29:08 <KipIngram   > Each item is a chunk of text and a description of the rectangle it lives in.
2023-02-15 10:29:18 <KipIngram   > So it's not restricted to proportionally spaced fonts.
2023-02-15 10:29:38 <Zarutian_iPad> oh, a bit like Apple QuickDraw masks are stored but more complex
2023-02-15 10:29:42 <KipIngram   > that description may not be perfect - I need to read more closely.
2023-02-15 10:29:59 <dlowe       > QuickDraw Masks were just RLE bitmaps
2023-02-15 10:30:12 <Zarutian_iPad> so a “line” has the same height through out?
2023-02-15 10:30:30 <KipIngram   > But if you went to the starting point of a rectangle and marched across the text snip, slapping character rectangles up there and advancing to the right, you'd successfully image that snip of text to the screen.
2023-02-15 10:30:46 <dlowe       > I think Apple Quartz is The Way, though.  We should have all moved to resolution independent representations a long time ago.
2023-02-15 10:30:49 <KipIngram   > You just do that for all the parts of your window that are exposed.
2023-02-15 10:31:04 <Zarutian_iPad> dlowe: yeb but still rather ingenious idea for storing bitmap masks
2023-02-15 10:31:09 <KipIngram   > Yeah, this is certainly resolution dependent.
2023-02-15 10:31:50 <KipIngram   > I would imagine this is simpler to implement than a resolution independent method.
2023-02-15 10:31:58 <KipIngram   > don't know how much, though.
2023-02-15 10:32:12 <Zarutian_iPad> I am just wondering how one would write a gpu fragment shader to render it
2023-02-15 10:32:19 <dlowe       > Sure, but we have ubiquitous GPUs now, so anything that pushes load off the CPU should be done
2023-02-15 10:32:22 <KipIngram   > Meanwhile, you can also use this data structure to find out what text is in a certain location on the screen.
2023-02-15 10:32:56 <KipIngram   > Yeah, but I do really like the concept of being able to "read the screen."
2023-02-15 10:33:22 <KipIngram   > Just let the user manipulate the screen, and then when he trigers it, you go read it.
2023-02-15 10:33:48 <KipIngram   > Why do we have to keep up with what "has been put on the screen" in some app-dependent way?
2023-02-15 10:34:05 <dlowe       > It's a neat idea but I think you could do it with vector representations rather than boxes
2023-02-15 10:34:20 <dlowe       > and you could query the system "what intersects this box?"
2023-02-15 10:34:31 <KipIngram   > Well, how you draw these boxes - I don't think the method makes any particular way necessary.
2023-02-15 10:34:39 <Zarutian_iPad> I am thinking about how to pack this datastructure into a texture for that shader to sample
2023-02-15 10:34:45 <KipIngram   > You certainly good use vectors to put chars on the screen instead of boxes of pixels.
2023-02-15 10:35:14 <KipIngram   > Can you?  This data structure stores TEXT, at least partially.
2023-02-15 10:35:38 <Zarutian_iPad> textures are just arrays in a sense
2023-02-15 10:35:46 <KipIngram   > dlowe: I think you could change this to a vector method fairly easily, now that I think about it.
2023-02-15 10:36:17 <KipIngram   > Those vectors are still going to fill a particular rectangle on the screen, and you'd still index across the characters - that would just tell you what vectors to draw instead of what pixel blocks to grab.
2023-02-15 10:36:19 <Zarutian_iPad> where on uses x y (and sometimes z) coords as index into it
2023-02-15 10:36:35 <Zarutian_iPad> one*
2023-02-15 10:37:11 <KipIngram   > Well, I think you could compute the location on the screen a particular character would be by adding the sum of the widths of the prior chars to the left coordinate of the rectangle.
2023-02-15 10:37:39 <KipIngram   > And again, each char will have a width regardless of whether you draw it with pixels or vectors.
2023-02-15 10:37:50 <Zarutian_iPad> not what I meant. A texture is often used as a two dimensional array
2023-02-15 10:38:53 <KipIngram   > Ok.
2023-02-15 10:40:18 <Zarutian_iPad> so one trick is to have one line of the texture tell the max heights of each text line
2023-02-15 10:43:20 <Zarutian_iPad> heck even using relative pointer trick by having the texture line be as wide as the ‘screen’ is and each item in the line offset point to the next part of the data structure
2023-02-15 10:44:37 <Zarutian_iPad> the in that part you repeat the trick but to find the correct char box
2023-02-15 10:46:29 <Zarutian_iPad> then in the char box item you have the x, y, width, height, of the char box and pointer to the glyph data
2023-02-15 10:48:45 <Zarutian_iPad> this is mostly because the fragment shader function just returns an colour for the pixel and is run for every pixel
2023-02-15 10:49:16 <KipIngram   > Are you talking about a way to read the TEXT back from a texture?
2023-02-15 10:49:23 <KipIngram   > Do textures store that kind of thing?
2023-02-15 10:49:50 <Zarutian_iPad> no, I am talking how to cheaply render the text using the GPU
2023-02-15 10:49:56 <KipIngram   > I think going the other way is fine - exactly what we'd be doing would be to modify some small rectangle of a texture to add a char at a particular place.
2023-02-15 10:50:10 <KipIngram   > For bitmap display that's just copying a small rectangle, from another texture that has all of your characters.
2023-02-15 10:52:19 <KipIngram   > Yeah, it would be great if we could push that process wholly onto the GPU.
2023-02-15 10:52:30 <KipIngram   > But I don't know how to, offhand.
2023-02-15 10:53:21 <Zarutian_iPad> I was describing the datastructure the fragment shader gets the text from
2023-02-15 10:53:27 <KipIngram   > I was just happy to not have to send the character BITMAPS over at the time I did the screen update.
2023-02-15 10:53:37 <KipIngram   > Just a simple command to get them from another texture.
2023-02-15 10:54:34 <KipIngram   > For a proportional spaced font that's really simple - you can calculate the location of a given char box in the font texture.
2023-02-15 10:54:45 <KipIngram   > For a non-proportional spaced font, a table would have to be used.
2023-02-15 11:44:19 <KipIngram   > Well, god damn it.
2023-02-15 11:44:46 <KipIngram   > Looks like I've got covid.  Started feeling hot flashes / chills last night.  Just took a home test and it's clearly positive.
2023-02-15 11:44:51 <KipIngram   > Damn it, damn it.
2023-02-15 11:45:07 <KipIngram   > We'll see how bad it gets, I guess.  So far it's bearable.
2023-02-15 11:47:04 <thrig       > better than corvid?
2023-02-15 13:31:43 <KipIngram   > joe9: sam seems like a pretty nice editor.  I think it's something i could get comfortable with, once I was on board with a mouse.  But, I think I'd likely not implement it "completely" - instead, what I'd go for would be to have my "sam window" just be a window with the Forth interpreter running in it, which had a "sam vocabularly" as the first item in its search path.
2023-02-15 13:32:01 <KipIngram   > And I might give some thought to finding an RPN-friendly format for the commands.
2023-02-15 13:32:23 <KipIngram   > Regular expressions seem vital to it.  I've long thought that I'd like to have "some sort of suitable" regular expression support in my Forth.
2023-02-15 13:32:31 <KipIngram   > I don't really know what that is, though.
2023-02-15 13:33:11 <KipIngram   > There's a very good paper out there somewhere, that I wish I could put my hands on, that gets fairly deep into the implementation of regular expression engines.
2023-02-15 13:33:50 <KipIngram   > The author was of the firm opinion that a number of the most popular regex engines have gone "off the reservation" and incorporated features that can create NP-complete complexity conditions in the operation.
2023-02-15 13:33:59 <dlowe       > There's one from golang's implementation that's good
2023-02-15 13:34:13 <KipIngram   > According to him, the original conception of regular expressions was free of that.  But people started adding features, and weren't careful about what they added.
2023-02-15 13:34:29 <KipIngram   > And some of those added features can't be done without a complexity explosion in pathological conditions.
2023-02-15 13:35:12 <KipIngram   > Sounds like a big argument - this guy would say "get rid of those - don't go there," whereas other folks would say "oh God, you can't give THAT up..."
2023-02-15 13:36:25 <thrig       > many modern regular expression engines aren't actually regular expressions
2023-02-15 13:36:25 <KipIngram   > The immediate issue that arises with Forth and regular expressions is that regular expressions aren't space delimited - they have multiple function units rammed up against each other.
2023-02-15 13:41:48 <KipIngram   > You know, finding stuff about sam online is a pita - apparently Amazon Web Services has something called SAM, and all the searches find that.
2023-02-15 13:42:09 <KipIngram   > joe9: I built a github sam, but I can't figure out how to run it.
2023-02-15 13:42:27 <KipIngram   > It's got two binaries, sam and samterm, and apparently I need them both, connected somehow.
2023-02-15 13:42:39 <KipIngram   > If I run samterm, I get a window, but it's a dead window.
2023-02-15 13:43:15 <thrig       > maybe there's a package for https://9fans.github.io/plan9port/ ?
2023-02-15 13:45:41 <KipIngram   > I grabbed that last night, but haven't installed it.
2023-02-15 13:45:57 <KipIngram   > I was hoping for something I could just build locally and try out, without scattering all around my os.
2023-02-15 18:58:22 <KipIngram   > What I will say about this "text to bitmap" algorithm is that there's absolutely nothing remarkable about it.  It's just as simple as it sounds.
2023-02-15 18:59:34 <KipIngram   > When you insert a delete a substring (the file is regarded as one long string, and those are the only primitive operations), you just break and coalesce character boxes as makes sense.  New line "boxes" just fill up whaatever space is left to the far right edge.  Tab boxes just have their width.
2023-02-15 19:04:48 <KipIngram   > So, say if inserting a character somewhere makes the box at the right end stick past the edge, you have to split it, or remove some characters from it and put them in the first box of the next line down, and so on, until you come to a line where you can just narrow the width of a newline.
2023-02-15 19:26:29 <thrig       > ye olde line wrapping problem
2023-02-15 20:48:33 <KipIngram   > So, question for you formal verification guys.  I contend that one can look at this line of c:
2023-02-15 20:48:39 <KipIngram   > z = x + y;
2023-02-15 20:48:48 <KipIngram   > And "just know" that it is correct.
2023-02-15 20:48:56 <KipIngram   > presuming that's what you want to do, of course.
2023-02-15 20:49:01 <thrig       > are them signed?
2023-02-15 20:49:39 <KipIngram   > So, the question.  How do you draw a boundary between code that you can just inspect, as a qualified individual, and have confidence in its correctness, vs. code that you need to use some kind of tool to verify?
2023-02-15 20:50:03 <thrig       > because I've read that signed int overflow is a little bit undefined
2023-02-15 20:50:25 <KipIngram   > x = 20; y = 10;
2023-02-15 20:50:30 <KipIngram   > z = x + y;
2023-02-15 20:52:31 <KipIngram   > I guess I'm just taking the position that *up to a point* a good programmer can just study his code and be confident in its correctness.
2023-02-15 20:52:49 <KipIngram   > It's also quite clear to me that beyond that point that breaks down.
2023-02-15 20:53:06 <KipIngram   > Which is part of why Forth encourages us to factor.
2023-02-15 20:53:28 <thrig       > diligence is a bit lacking, hence a lot of bugs in C code
2023-02-15 20:53:54 <KipIngram   > That's fair - it's like any other job; you actually have to  DO it.
2023-02-15 20:54:33 <KipIngram   > If you're on your sofa with the Playboy channel on while you inspect code, you're probably not going to get good results.
2023-02-15 20:55:01 <KipIngram   > And to do it well  you'd  need to guard against fatigue.
2023-02-15 20:55:06 <KipIngram   > And that's a problem right there.
2023-02-15 20:55:13 <KipIngram   > Given how hard companies push their developers.
2023-02-15 20:57:18 <KipIngram   > Also, there's the question of who verifies the verification tool.
2023-02-15 22:30:20 <KipIngram   > I was thinking about file systems today, and I still think a tree structure is the way to go.  In fact, I thought of the notion of reserving the final seven bytes of each block as a 32-bit "parent block," a 12-bit byte count, and a 12-bit "content type" field.
2023-02-15 22:30:34 <KipIngram   > Then you could pick up any block and walk back up to the root of the tree.
2023-02-15 22:30:49 <KipIngram   > The contents of the block would depend on that 12-bit type field.
2023-02-15 22:31:01 <KipIngram   > But I like the "generality" of that.
2023-02-15 22:32:06 <KipIngram   > A file would have a directory entry somewhere, and up to size 4089 bytes that directory entry would just point to a block and the file would be in that block.
2023-02-15 22:32:34 <KipIngram   > Once the size went over 4089, you'd split the content and put it in two blocks.  Then those two would have a parent and the directory entry would point at it.
2023-02-15 22:32:59 <KipIngram   > and that first level parent could support quite a number of child blocks, so that would get you up to a much larger file size.
2023-02-15 22:33:35 <KipIngram   > For even bigger files (which I'd likely never have), you'd have a new layer, and I really can't imagine using up the file size capacity of that.
2023-02-15 22:35:53 <KipIngram   > Anyway, that's basically a b-tree.
2023-02-15 22:36:12 <KipIngram   > And I envision a second tree to store these links I've talked about.
2023-02-15 22:36:31 <KipIngram   > each end of the link would be specified by a block, a 12-bit starting byte, and a 12-bit length byte.
2023-02-15 22:36:48 <KipIngram   > I'd actually have two link trees - one sorted by origin location and the other by destination location.
2023-02-15 22:37:06 <KipIngram   > because when I modify a block I have to be able to quickly find any links that affects and modify them.
2023-02-15 22:37:36 <KipIngram   > But only within the block containing the change - I wouldn't have to visit the whole remainder of the file.
2023-02-15 22:38:19 <KipIngram   > So basically a link will be able to have some block of text, somewhere, as its origin and another block of text, somewhere, as its destination.
2023-02-15 22:38:23 <KipIngram   > Completely arbitrary.
2023-02-15 22:41:22 <KipIngram   > So a link will be two 32-bit block numbers and four 12-bit offset or size fields.  So that's 80 bits.  I'd  probably use the other 16 bits out to 96 for "something," which means a block could specify 4089/12 = 340 links.
2023-02-15 22:41:55 <KipIngram   > I'd do the same t hing with them as with file data blocks - when one gets full, split it into two and modify the tree accordingly.
2023-02-15 22:46:09 <KipIngram   > Pike paper on "structured regular expressions":
2023-02-15 22:46:10 <KipIngram   > http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf
2023-02-15 22:46:34 <KipIngram   > He's advocating against the "line oriented" nature of general Linux tools (grep, etc.)
2023-02-15 22:46:56 <KipIngram   > And that's why he wrote sam with the notion that a file was just one long string.
2023-02-15 22:55:40 <KipIngram   > So, why do Linux file systems put some information in the directory entry and other information somewhere else (inode)?
2023-02-15 22:55:59 <KipIngram   > What is gained in doing that, as opposed to just having your directory entry contain all of the info re: a file?
2023-02-15 22:56:50 <KipIngram   > The information still takes up the same amount of room either way, and with inodes you've got to call out the inode number in the directory entry.
2023-02-15 22:56:55 <KipIngram   > So its less space efficient.
2023-02-15 22:59:46 <dave0       > KipIngram: i think it's done like that for hard links .. you can have multiple names pointing to the same inode
2023-02-15 23:00:11 <dave0       > but i'm not sure if hard links came before or after they designed the filesystem
2023-02-15 23:00:18 <dave0       > so it might be ret-conned
2023-02-15 23:07:27 <KipIngram   > The down side of that parent pointer idea, in the leaves, is that it deprives me of "fully data" 4kB blocks.
2023-02-15 23:07:33 <KipIngram   > So I don't know if I will do that yet.
2023-02-15 23:08:03 <KipIngram   > I'd have such a field in all the non-leaf nodes of the tree; I definitely want to be able to walk the tree in both directions.
2023-02-15 23:08:14 <KipIngram   > But I may make the leaf nodes "just up to 4096 bytes of data."
2023-02-15 23:08:36 <KipIngram   > dave0: That makes sense.
2023-02-15 23:08:44 <KipIngram   > I don't think I've ever used a hard link - ever.
2023-02-15 23:08:50 <KipIngram   > I use soft links all the time, though.
2023-02-15 23:10:47 <dave0       > i can't think of a time that i've used links for own data
2023-02-15 23:11:23 <dave0       > it's just never come up
2023-02-15 23:13:42 <KipIngram   > So, a block and a byte count (the needed things for designating data blocks that are part of the file) takes 44 bits.  So a block could hold 92 of those.  So "leaf only" would get me to 4096 bytes filesize.  One interior node level would get me 92*4096 = 376,832 bytes, though you normally wouldn't have all the blocks full.
2023-02-15 23:14:09 <KipIngram   > Two interior levels would get me to something like 92*92*4096 = 34.6 MB.
2023-02-15 23:14:25 <KipIngram   > That's a guestimate - I don't know that the "records" would be the same size in those two levels.
2023-02-15 23:15:02 <KipIngram   > But anyway, once you add one interior block to a file, that gets you up to 92 blocks of file data.  So the overhead is a little over 1%.
2023-02-15 23:15:49 <KipIngram   > Even with a file system implemented, I still expect to HAVE blocks, and may still use them - may use block to actually haul data in, and the file system as a "roadmap."
2023-02-15 23:16:13 <KipIngram   > So that roadmap block would tend to get BLOCKED in and then hang around.
2023-02-15 23:18:27 <KipIngram   > In my most recent system, choice of disk buffer doesn't exist.  There are 255 buffers, and when a block is read, it goes in buffer <blocknum> 255 mod.
2023-02-15 23:18:38 <KipIngram   > I use a cute trick from Hacker's Delight to do that modulo.
2023-02-15 23:19:05 <KipIngram   > It would be nice to have a couple of buffers eligible to hold a block - would reduce my collisions quite a lot.
2023-02-15 23:21:21 <KipIngram   > But it would add more work to the residency test, and I want it to be as fast as I can possibly make it.
2023-02-15 23:27:30 <KipIngram   > Ugh.  Some browser tab when flakey on me - fans are over 4500 RPM.
2023-02-15 23:27:42 <KipIngram   > when that happens I just close Chrome and re-open it; it usually fixes it.
2023-02-15 23:50:38 <KipIngram   > Maybe a Forth could accept regular expressions as "literals."  Search the dictionary, try to convert to a number, then parse it as a regular express, and if it's valid you could either store at as a string and return the address.
2023-02-15 23:55:49 <KipIngram   > Another possibility (I've mentioned this before) would be for the system to contain a pointer (initially null) to an array of regular expressions.  If you type something in the input line that satisfies one of the expressions, it would be considered valid input, and likely a particular word you wrote for the purpose would be called to "do something" with it.