2022-07-16 00:10:34 <KipIngram   > Anyway, I started pecking together a new framework this afternoon.  Trying to kind of put together sectorforth and planckforth ideas - a single sector system that bootstraps the way planckforth does.  No idea yet whether it'll prove feasible.
2022-07-16 00:16:09 <KipIngram   > The "first interpreter loop," for the single-char commands, is just five cells long - it's basically 'char calc exec jump <offset>'
2022-07-16 00:16:32 <KipIngram   > "calc" is to convert the char to an xt, which I'll try to be clever about.
2022-07-16 00:19:01 <KipIngram   > Seems like there are two basic ways to go about it - the "obvious" one would involve a pointer table.  The other would be a table of code snips, with the snips evenly spaced in the table.  So no pointer overhead.  But the snips surely wouldn't all be the same length, so there's a not-yet-know amount of post-snip waste in that approach.  Hard to say which one would be smaller.
2022-07-16 00:20:43 <thrig       > gperf is usually for speed, not a small lookup table
2022-07-16 00:25:04 <KipIngram   > Is that a name for what I just described?
2022-07-16 00:25:28 <thrig       > gperf does perfect hashing (maybe)
2022-07-16 00:25:43 <KipIngram   > Yeah, just looked it up.
2022-07-16 00:25:45 <KipIngram   > Hmmm.
2022-07-16 00:26:04 <KipIngram   > I was thinking about something a few months ago that needed something like that.
2022-07-16 00:26:23 <KipIngram   > I think it had to do with hashed dictionary lookups.
2022-07-16 00:27:18 <KipIngram   > Loosely, the idea was that all the plethora of RAM we have on our regular computers could be used to accelerate compilation on a  Forth, by caching xts in a has table.
2022-07-16 00:27:38 <KipIngram   > So you'd look up a word the first time in the usual linked list way, but every time you saw it after that you could get it from a hash table.
2022-07-16 00:27:47 <KipIngram   > That you just threw RAM at.
2022-07-16 00:28:22 <thrig       > some unix shells do something like that to avoid PATH searchs for known program names
2022-07-16 00:28:33 <KipIngram   > That idea came to mind because I added some stuff to my system that let me do some interesting timing measurements, and one of the things I measured was compilation time.
2022-07-16 00:28:57 <KipIngram   > Compared mine to GForth's, and GForth just kicked my ass.  By such a large margin that I just couldn't believe they do a linked list search.
2022-07-16 00:29:08 <KipIngram   > It seemed TOO fast to me.
2022-07-16 00:29:22 <KipIngram   > For there to be such a search going on, that is.
2022-07-16 00:29:39 <KipIngram   > so I was speculating on how they might do that.
2022-07-16 00:31:15 <KipIngram   > But that wouldn't be a "perfect" hash - it would just be a hash table that was big enough for the collision rate to be low.
2022-07-16 00:31:21 <KipIngram   > It would have a lot of empty space.
2022-07-16 00:31:35 <KipIngram   > I don't see how you could have a perfect hash when you didn't know in advance all of your names.
2022-07-16 00:34:12 <KipIngram   > Mine compared decently well with GForth on execution speed, though, and on a couple of the things I timed it did better, which I think arose from some of the extra stuff I have in mind (conditional returns and so on).  Just let me put together a shorter definition for whatever that test was.
2022-07-16 00:34:45 <KipIngram   > have in "mine"
2022-07-16 01:06:21 <imode       > 1m processes in half a gig.
2022-07-16 01:06:35 <imode       > each with 64 words worth of memory.
2022-07-16 01:36:39 <KipIngram   > imode: What is L, "load immediate" doing in the planckforth system?
2022-07-16 01:37:02 <KipIngram   > Does that mean load from the input stream, and if so, how is that different from  "read a character"?
2022-07-16 01:37:15 <KipIngram   > Does that let you embed binary numbers in the stream?
2022-07-16 01:38:47 <KipIngram   > Would someone like me ever do a million processes?  I get it that there are such applications.
2022-07-16 01:39:12 <KipIngram   > But the overwhelming majority of OS deployments on the planet or "just us folks."
2022-07-16 01:39:53 <KipIngram   > I feel fairly strongly that the main thing that gets done with most computer sold is "run OS code."
2022-07-16 01:40:18 <KipIngram   > As opposed to "running application code."
2022-07-16 01:40:19 <imode       > you'll do a million processes if that's how you represent data and compute.
2022-07-16 01:41:06 <KipIngram   > Oh, well, sure - I guess we might have a paradigm shift.
2022-07-16 01:41:25 <KipIngram   > Right now I seem to be running 338.
2022-07-16 01:41:40 <KipIngram   > At least according to ps ax | wc.
2022-07-16 01:42:22 <imode       > a single string, for example, would be composed of multiple processes.
2022-07-16 01:42:37 <imode       > in sequence.
2022-07-16 01:42:43 <imode       > or rather, linked together.
2022-07-16 01:45:32 <KipIngram   > There's also command S, load string literal.  Now sure I see what that one does either.  Maybe these become important later in the abstraction process.
2022-07-16 01:48:50 <imode       > I believe it's simply an XT.
2022-07-16 01:49:38 <imode       > line 86 onward explains the initial portion of the bootstrapping sequence.
2022-07-16 01:49:42 <imode       > and that's the only place it's used.
2022-07-16 01:49:53 <imode       > well, apart from some places in word defs.
2022-07-16 01:56:16 <KipIngram   > Ah - yes; I see where L is used now.
2022-07-16 01:58:02 <KipIngram   > I think I see - it's not used in the bootstrap code itself - it's not "load a literal from the input stream."  It's "load a literal" just like we do in Forth - it's used to construct new definitions.
2022-07-16 01:58:55 <imode       > yep.
2022-07-16 01:59:00 <KipIngram   > So k loads a byte from the input stream - L loads a literal from the instruction stream that's executing in RAM.
2022-07-16 02:13:56 <KipIngram   > I also see now why PlanckForth may be bigger.
2022-07-16 02:14:39 <KipIngram   > It has to be functionally capable in two ways - it has to have input stream processing powerful enough to let it build definitions in RAM, and it ALSO has to have the code necessary for the primitives that will run later, when it's a mature RAM-based Forth.
2022-07-16 02:14:48 <KipIngram   > So it's a two front war, whereas sectorForth is only one.
2022-07-16 02:15:01 <imode       > youuuuu got it.
2022-07-16 02:15:37 <dave0       > planck forth?? i must google
2022-07-16 02:16:44 <dave0       > oh it has heaps of builtin words!
2022-07-16 02:18:23 <KipIngram   > Yes, but it has a VERY interesting twist.
2022-07-16 02:18:33 <KipIngram   > The way it works is quite a lot of fun.
2022-07-16 02:18:38 <KipIngram   > I wanna make one.
2022-07-16 02:19:39 <KipIngram   > imode: For a lot of the words, one bit of code will take care of both situations.  It's only the words that deal with "inline" arguments that have to have an input stream version and a RAM definition version.
2022-07-16 02:19:59 <KipIngram   > Words like +, -, etc. - those don't relate to inline material, so we only need them once.
2022-07-16 02:22:17 <KipIngram   > k is the only one that takes inline data from the bootstrap stream.  And we need both L and S because inline integers and inline strings are different.
2022-07-16 02:22:25 <KipIngram   > Makes total sense now.
2022-07-16 02:22:53 <KipIngram   > And the jump operations take inine data as well.
2022-07-16 02:24:34 <KipIngram   > dave0: https://github.com/nineties/planckforth
2022-07-16 02:34:21 <dave0       > KipIngram: nice!
2022-07-16 02:34:40 <dave0       > KipIngram: did you end up looking at my zip file i sent you?
2022-07-16 02:35:57 <KipIngram   > A little, and also looked at the online sectorForth.
2022-07-16 02:36:56 <dave0       > KipIngram: my sectorforth is nearly the same as cesarblum's on github, except i use - instead of +
2022-07-16 02:37:28 <KipIngram   > dave0: I'm interested in trying to "merge" the sectorForth and PlanckForth ideas.  A boot sector to start with, but bootstrap it similarly to PlanckForth.
2022-07-16 02:37:42 <dave0       > KipIngram: the interesting part is the    hello.forth    that builds up a hello world
2022-07-16 02:37:45 <KipIngram   > BTW, in the PlanckForth git repo, the file "bootstrap.fs" is the really interesting one.
2022-07-16 02:39:15 <dave0       > ugh i'd have to deep dive on planckforth
2022-07-16 02:39:50 <dave0       > sectorforth has proper words, not single letters
2022-07-16 02:40:24 <imode       > planckforth builds those up.
2022-07-16 02:40:59 <dave0       > imode: i just got to the   `alias-builtin`    in bootstrap.fs :-)
2022-07-16 02:41:06 <imode       > hehe.
2022-07-16 02:42:59 <KipIngram   > Yeah, but what it does with those single letters is fascinating.
2022-07-16 02:43:11 <KipIngram   > It eventually becomes "regular words."
2022-07-16 02:43:31 <dave0       > yup it's a lot easier to read after alias-builtin
2022-07-16 02:43:54 <KipIngram   > I'm tinkering around with the one-char name list right now, trying to reduce it to a smaller range without giving up the "intuitive names."
2022-07-16 02:45:28 <KipIngram   > I started by moving every lower case letter that didn't have it's upper case counterpart in the list to upper case.  That only left a handful of lower case chars.
2022-07-16 02:49:11 <dave0       > planckforth has a `nop` word
2022-07-16 02:49:25 <KipIngram   > That was fairly painless most of the way, but there's one lower case letter left - "r."
2022-07-16 02:49:34 <dave0       > nop is one of my machine code primitives ehehe
2022-07-16 02:49:44 <KipIngram   > It's handy to have around.
2022-07-16 02:50:06 <KipIngram   > It also has some stuff I just don't "need."  Like a word to print a version string.
2022-07-16 02:50:20 <KipIngram   > Word to get argc and argv, which won't exist in my context.
2022-07-16 02:50:42 <KipIngram   > dave0: I installed qemu earlier - I can use it to test this.
2022-07-16 02:50:51 <dave0       > KipIngram: ah nice
2022-07-16 02:51:46 <dave0       > qemu is quite large
2022-07-16 02:51:49 <KipIngram   > Aside from that pesky r, I have it down to a collation range of 47.
2022-07-16 02:52:06 <KipIngram   > Well, looks like it comes with a whole slew of platform options.
2022-07-16 02:52:39 <KipIngram   > Different processors, different BIOS, etc.
2022-07-16 02:52:55 <KipIngram   > Like it's got a ready to go virtualization of a "386 based PC."
2022-07-16 02:53:02 <KipIngram   > Complete with BIOS.
2022-07-16 02:53:18 <KipIngram   > It was easy to install and easy to use and test out.
2022-07-16 02:54:21 <KipIngram   > Wow - it's nearly 2am.  Better get some sack time.
2022-07-16 02:54:26 <KipIngram   > Have good one, guys.
2022-07-16 02:55:17 <dave0       > nite KipIngram !
2022-07-16 12:57:36 <KipIngram   > Ok, so, I realized after I lay down to sleep last night that that whole endeavor of trying to find a clever "table arrangement" to make the planckforth style engine compact was misguieded.
2022-07-16 12:58:41 <KipIngram   > It can't be that way - it HAS to be an actual search of a list, because I'm about to start adding new items to that list.  The very first thing PlanckForth does is add support for spaces, newlines, and the \ whole line comment character.  So the way it finds things has to be a way it can grow.
2022-07-16 12:59:00 <KipIngram   > And that removes the need for any sort of re-arranging to try to reduce the "range" of the list.
2022-07-16 12:59:46 <eris[m]     > you can preallocate a table for builtins and switch to a list?
2022-07-16 13:04:53 <KipIngram   > Could, that's true.
2022-07-16 13:58:52 <KipIngram   > Ooops.
2022-07-16 13:59:33 <KipIngram   > imode: % is included twice in the PlanckForth one-char command list, with two different meanings.
2022-07-16 13:59:46 <KipIngram   > It's used for modulo and for arithmetic right shift.
2022-07-16 14:08:05 <KipIngram   > Maybe that's just a typo - > is unused, so it could be right s hift.
2022-07-16 14:08:37 <thrig       > or redshift if the forth is accelerating away from you
2022-07-16 14:18:45 <KipIngram   > You know how they really ought to make credit cards work?  That chip in your card would generate a long random number for each use.  The card company would be able to verify it came from your card.  That way your actual "seed" number would never be distributed.
2022-07-16 14:19:21 <KipIngram   > And if you wanted to do an online transaction, you'd be able to go to their website and do the same thing.  For recurring payments the website would let you specify some "lifetime" for the generated number.
2022-07-16 14:19:30 <KipIngram   > As far as I can tell that should be totally secure.
2022-07-16 14:20:03 <KipIngram   > At one point in time Bank of America let their card holders do something like that, except the number you generated was a standard looking card number.
2022-07-16 14:20:19 <KipIngram   > They don't do that anymore, though, and I kind of see why - there are only so many such numbers.
2022-07-16 14:20:35 <KipIngram   > But if it was set up to use a long number then it could be arranged for it to never run out.
2022-07-16 14:21:12 <KipIngram   > Maybe a future generation of commerce tech will work that way.
2022-07-16 15:21:25 <KipIngram   > I like how sectorForth puts the next code near the middle of the sector - makes it accessible with a two byte jump from most of the sector.
2022-07-16 15:21:55 <KipIngram   > Next itself if five bytes in 16-bit code, so jumping to it saves three bytes per word.
2022-07-16 15:23:29 <KipIngram   > I guess a two-byte jump can only go from 128 backward to 127 forward, right?  So I guess you can't have it work that way from the whole sector.  You'd arrange things so that as many bits of word code as possible in that range.
2022-07-16 15:24:02 <KipIngram   > Of course it would still work - you'd just get three bytes instead of two for the more distant words.
2022-07-16 15:24:47 <thrig       > it may depend on the CPU and how efficient or not or big or not the jumps are
2022-07-16 15:31:22 <KipIngram   > Yeah - I was speaking only about 16-bit mode x86.  I guess there are just two opcodes - one uses a byte and the other uses two for the offset.
2022-07-16 15:31:35 <KipIngram   > Assembler just uses the smallest one that will get the job done.
2022-07-16 15:42:31 <KipIngram   > Ok, here's a nice trick.  Default header structure is a 2-byte link field, a count byte, the name bytes, then code (which for primitives will just be the first instruction of the primitive).
2022-07-16 15:42:53 <KipIngram   > But for a PlanckForth work-alike, we can use the LSB of the link field to indicate a "built-in" word.
2022-07-16 15:43:10 <KipIngram   > And in that case, the link is just a single byte and there is no count byte - the name is known to be one byte long.
2022-07-16 15:43:27 <KipIngram   > So there's just a one-byte link and a one-byte name compare byte, and then code.
2022-07-16 15:43:40 <KipIngram   > So all the built-ins will just have two bytes of header overhead.
2022-07-16 15:43:51 <KipIngram   > I don't see any reason that wouldn't work just fine.
2022-07-16 15:44:36 <KipIngram   > The initial FIND won't even know how to work with the full headers anyway - it will just assume the built-in headers and will have to be replaced eventually.
2022-07-16 15:48:29 <eris[m]     > lots of things do get replaced
2022-07-16 15:48:35 <eris[m]     > honestly id love to write a planckforth
2022-07-16 15:48:35 <KipIngram   > Yes.
2022-07-16 15:48:55 <imode       > I'm glad this conversation is continuing. :P
2022-07-16 15:49:00 <KipIngram   > I'm going to try to follow through on it.  I'm kind of in the "pondering" phase right now, which I tend to do a lot of.
2022-07-16 15:49:15 <KipIngram   > Yes - thank you for floating that by for ufs.
2022-07-16 15:49:19 <KipIngram   > for us
2022-07-16 15:49:19 <thrig       > walk the plankforth, matey
2022-07-16 15:49:26 <KipIngram   > lmao
2022-07-16 15:50:26 <KipIngram   > I don't think I can implement the full list of built-ins that PlanckForth and still stay in one sector.  I'll have to see if I can think of some way to skin it down some.
2022-07-16 15:50:49 <KipIngram   > I think the rule, though, needs to be that building machine code words later isn't allowed.
2022-07-16 15:50:59 <KipIngram   > I mean, eventually there might be an assembler.
2022-07-16 15:51:16 <KipIngram   > But that feels like cheating, if I use that to "build the system."
2022-07-16 15:51:39 <KipIngram   > I want the bootstrap to be portable.
2022-07-16 15:51:54 <eris[m]     > I'd personally go for a miniforth type approach with a very minimal system used to bootstrap to assembler
2022-07-16 15:52:12 <KipIngram   > I guess that's the right way to say "the rule" - the bootstrap has to run on any system that properly implements the built-ins.
2022-07-16 15:52:19 <KipIngram   > Without change.
2022-07-16 15:52:26 <eris[m]     > because filling in the remaining few primitives and jumps probably isnt an issue, imo
2022-07-16 15:52:31 <imode       > what you're describing is a virtual machine.
2022-07-16 15:52:39 <KipIngram   > Sure.
2022-07-16 15:53:30 <imode       > https://mikeinnes.io/2017/09/13/brainforth
2022-07-16 15:54:11 <KipIngram   > If all you wanted was "a way any way" you could have a way of ticking bytes into RAM and updating HERE and LATEST, and that with a few arithmetic primitives would pretty much do it.  But you'd be ticking system dependent machine code in.
2022-07-16 16:06:37 <KipIngram   > And regarding "replacing most everything," I think that's write.  The boot sector gets loaded at address 0x7c00, and the BIOS uses the first 0x500 bytes.  So I imagine you'd build the final system starting there at 0x500 - do it in a way that eventually abandons the stuff at 0x7c00.
2022-07-16 16:06:57 <KipIngram   > Copy primitives to the new low RAM system as needed.