2023-08-19 01:49:29 <xelxebar    > next4th: What are you up to this weekend?
2023-08-19 02:24:23 <next4th     > xelxebar: as usual, playing with computing (guix, forth, games etc.) and maybe some reading
2023-08-19 04:34:27 <veltas      > xelxebar: It seems like a lot of overhead but if you have a few good optimised code words then the whole system runs well usually
2023-08-19 04:35:21 <veltas      > I've got a tokenised and direct threaded (build option) forth for the ZX Spectrum which runs wayyy faster than the built-in BASIC for most things
2023-08-19 04:38:51 <veltas      > Part of why it runs well is if you have 'critical path' stuff in code then 90% of your runtime is optimised assembly ... at least that's the theory
2023-08-19 04:39:27 <veltas      > An unoptimised direct threaded forth is fast but not as fast as an optimised compiler or raw assembly
2023-08-19 04:39:56 <veltas      > There are reasonable optimisers for Forth and you can do "subroutine threading" which is basically just raw machine code output
2023-08-19 04:40:44 <veltas      > Combine that with some optimisations and you've got something within an order of magnitude of C ....
2023-08-19 04:41:18 <veltas      > (or a lot faster depending on problem and if you write the 'busy' part yourself in assembly) ;-)
2023-08-19 04:41:23 <xelxebar    > Yeah, SmithForth is subroutine threaded. Could be interesting to do some benchmarks against C.
2023-08-19 04:42:14 <xelxebar    > Would also be curious to try writing an inlining compiler and comparing with that.
2023-08-19 04:42:26 <xelxebar    > Dictionary would be huge. lol
2023-08-19 04:43:29 <veltas      > subroutine threaded is pretty much like "very very bad unoptimised C output" in the colon defs, but the code words will be optimised
2023-08-19 04:44:09 <veltas      > But it is actually machine code, there's not "next function", you use a return instruction to finish a code word
2023-08-19 04:45:38 <veltas      > xelxebar: Implementing inlining is not actually that hard, and doesn't require that many words
2023-08-19 04:45:56 <xelxebar    > Yeah, that's the thing. For "bunch of one-line defs" forths, I'd naively expect that jumps (via calls) are a significant part of your code. So performance comes down to how well your branch predictor works.
2023-08-19 04:46:27 <veltas      > Yeah that's not actually strictly true
2023-08-19 04:46:32 <veltas      > It depends what you're doing
2023-08-19 04:46:52 <veltas      > Let's say you're doing block transfer, then most of CPU time will be spent in your MOVE code
2023-08-19 04:47:08 <veltas      > Or if it's using disk then it will be I/O-bound
2023-08-19 04:47:10 <xelxebar    > Ah, okay, if you're i/o bound, then yeah.
2023-08-19 04:47:28 <veltas      > If it's not disk then you won't be limited by Forth, you'll be limited by MOVE
2023-08-19 04:47:35 <veltas      > Because most of the time will be spent in MOVE
2023-08-19 04:47:59 <veltas      > So if MOVE is optimised well then it won't matter how much overhead your threading has... unless it's especially bad somehow
2023-08-19 04:48:23 <veltas      > That's the thing about performance, it's not simple, usually the best thing is to profile and find what needs improvement
2023-08-19 04:55:50 <veltas      > But if in profiling you're spending like 30% or more of your time in the NEXT routine then yeah it's bound by the threading model
2023-08-19 04:56:04 <xelxebar    > Yeah, there are definitely lots of moving pieces, so workload matters, but we can still affirmatively say that spamming 5 nops between every functional block of your asm is worse than without, right?
2023-08-19 04:56:37 <veltas      > I would say so
2023-08-19 04:56:46 <veltas      > But 'worse' is not always 'significantly worse'
2023-08-19 04:57:03 <veltas      > That's the important thing in performance is to understand what is and isn't premature or significant
2023-08-19 04:57:17 <veltas      > Because the effort of optimisation is a real cost too
2023-08-19 04:57:31 <xelxebar    > If your branch predictor were perfect, then small sequences of calls effectively become 1-cycle "nops" compared to 100% inlined code.
2023-08-19 04:57:32 <veltas      > And the overhead of maintaining optimisation code is considered in the Forth stack
2023-08-19 04:57:49 <veltas      > The branch predictor would help in 'busy' code yeah
2023-08-19 04:58:10 <veltas      > The proof is in the pudding
2023-08-19 04:58:30 <xelxebar    > Yeah, in a real life situation, you stare at flame graphs and stuff. I'm mostly just trying to feel out what Forth applications feel like when executing.
2023-08-19 05:00:00 <xelxebar    > Of course, the fully general answer is, "it can be anything!" so for concreteness, I'm thinking of a subroutine threaded forth (really, SmithForth) and application that's basically a big collection of one-line defs.
2023-08-19 05:02:39 <xelxebar    > Since SmithForth looks like a compliant Forth 2012 implementation, I'd kind of like to throw a medium-sized application at it and do some profiling.
2023-08-19 05:02:50 <xelxebar    > Not sure where to find such an app though.
2023-08-19 05:10:07 <next4th     > i think something for live coding / music synthsis could be a suitable one, realtime sythesis need to be fast, and forth is good at interactive & programmable.
2023-08-19 05:11:57 <next4th     > or games, which can be anything :)
2023-08-19 05:12:40 <xelxebar    > Music synth... That's a cool idea. There used to be a concatenative programming language for music synthesis called Sporth, I think.
2023-08-19 05:14:28 <xelxebar    > next4th: Have you gotten to the primitive FIND def yet?
2023-08-19 05:17:42 <next4th     > not yet, i'm still at bi..  what i do is random read through it, and try to write out hex without looking at its source.
2023-08-19 05:18:59 <next4th     > when i stuck i'll look at source or manual, copy a bit, and later start again..
2023-08-19 05:20:07 <xelxebar    > Oh, cool. You're going in the asm -> x86 direction?
2023-08-19 05:22:18 <next4th     > um, i write hex, read SmithForth's asm comment for refs, this is what i have in my editor now: https://envs.sh/20q.hex
2023-08-19 05:23:14 <next4th     > run in blinkenlights, so without ELF header, and mem layout is a little different.
2023-08-19 05:26:38 <xelxebar    > Ah, I see, so you're copying over bit-by-bit as you understand.
2023-08-19 05:28:43 <MrMobius    > ya making something you're really interested in is a great way to learn any language
2023-08-19 05:30:53 <next4th     > i think a next big project for me could be a irc client in duskos to join here lol
2023-08-19 05:32:03 <xelxebar    > Ooooo. That would be tons of fun!
2023-08-19 05:32:40 <xelxebar    > Does DuskOS already have a reliable network stack?
2023-08-19 05:34:40 <next4th     > no yet, but maybe i can talk via serial to znc, or learn some networking...
2023-08-19 05:37:34 <next4th     > seems i dreaming too much, should actually put time into learn/practice..
2023-08-19 05:40:10 <xelxebar    > Kind of liking the idea of an IRC client on top of SmithForth.
2023-08-19 09:29:01 <KipIngram   > Regarding performance, I agree with what veltas said about profiling.  In most applications where performance really matters, you're spending almost all your time in a percent or two of your source code.  So profile, and optimize those parts, using hand assmbly if necessary.
2023-08-19 09:29:17 <KipIngram   > In most cases you can get close to "as fast as anything."
2023-08-19 09:29:29 <KipIngram   > Especially if you mean "for all practical purposes."
2023-08-19 09:30:11 <KipIngram   > So, I'm planning to include a nice profiling capability  in my system eventually.
2023-08-19 09:30:37 <KipIngram   > I already have a pretty good idea of how I'll do that.
2023-08-19 09:34:36 <KipIngram   > I think that principle is proven by the fact that people write heavy numerical processing tools using Python.  What's going on is that 99% of your time is spent doing the core numerical parts, like doing stuff with matrices and so on, and that code is in a package you call and has usually been written in C and optimized.
2023-08-19 09:35:00 <KipIngram   > The actual Python just winds up being the "connective tissue" of the application, and you just don't spend much of the time there.
2023-08-19 09:35:34 <KipIngram   > So even though Python is kind of a dog performance wise, it winds up not mattering much.
2023-08-19 09:36:32 <KipIngram   > This is also why I've historically preferred indirect threading, even though it's thought of as the less performant of the three main models.
2023-08-19 09:36:45 <KipIngram   > Models of Forth architecture.
2023-08-19 10:37:30 <xelxebar    > Definitely. Just to be clear, though, my questions are not *really* about performance at this stage. I'm just trying to understand the shape and implications of Forth architecture(s).
2023-08-19 10:39:35 <xelxebar    > I need to re-read Moving Forth again, now that I have a handle on one implementation.
2023-08-19 11:01:36 <KipIngram   > Well, I think good Forth code is clearly very heavy in terms of call/return operations.  So, step one is to make those as efficient as they can possibly be, and Forth is pretty good on that.  And there are mitigating steps you can take - a Forth system can fairly easily (if you write it to do it) "unwind" short definitions - inline them into higher levels.  And in a code threading system you can take that all
2023-08-19 11:01:37 <KipIngram   > the way to the level of inlining machine code.
2023-08-19 11:02:13 <KipIngram   > You can get into trouble doing that "unwinding" - particularly if you're doing anything to the return stack you might et a situation where it doesn't work.  But we're assuming the programmer knows what he's doing.
2023-08-19 11:03:26 <KipIngram   > For example, we were talking about how ['] uses the return stack to get at the location of the word it's supposed to operate on - if the compiler unwound that word into it's components, that wouldn't work.
2023-08-19 11:05:23 <KipIngram   > Anywhere you call several primitives in a row, that sequence could be replaced by a new primitive that did the "business part" of all of those original primitives - that would save you running NEXT for every single one of them.  And that's something that could potentially be automated, though there are some memory management issues to deal with.
2023-08-19 11:06:07 <KipIngram   > So those are the two major performance hindering things in Forth: 1) call return overhead and 2) all those executions of NEXT.
2023-08-19 11:07:27 <KipIngram   > I timed NEXT on my current implementation - it seems to represent order of 1 nanosecond.
2023-08-19 11:08:17 <KipIngram   > How big a hit it is depends on how much work the  primitive does.  1+ is just an inc instruction, so compared to that NEXT is a big hit.
2023-08-19 11:09:16 <KipIngram   > So 1+ is an obvious candidate for inlining in a code threaded system.
2023-08-19 11:24:31 <KipIngram   > I feel good enouh about Forth's performance that I'm about to undertake a system with an additional layer - I'm going to make this F18A-like emulator and then build my system on that vm.  So right out the chute I'll have a layer that cuts performance some.
2023-08-19 11:26:29 <KipIngram   > However, I don't have a prediction about how the actual timing will turn out - I'm going to be able to get five / occasionally six virtual instructions into a cell - the code that I'd call NEXT in this system won't actually have to fetch anything from RAM in order to figure out where to go next.  I'll only need to fetch a new cell / increment IP every 5-6 instructions.  So it may wind up being comparable in
2023-08-19 11:26:31 <KipIngram   > performance.
2023-08-19 11:26:49 <KipIngram   > I'll know when I time it.
2023-08-19 12:45:29 <KipIngram   > So, the idea here is that the vm part of this will be platform specific, written using the native instruction set.  But then the rest will be written using vm instructions, and that would be binary portable across all common-endieness systems.
2023-08-19 12:46:25 <KipIngram   > So, I've just started a Python project to construct that universal part of the system.  I mean for it to actually be executable, using a Python emulation of the vm.  That way I'll be able to actually use it to develop.
2023-08-19 12:46:53 <KipIngram   > The idea would then be to write a specific platform's vm and then glue the results of this Python development to it - and expect that to run.
2023-08-19 12:49:48 <KipIngram   > I'm storing the image as a simple Python list; items will be unsigned integers that fit in 32 bits.  Eventually I'll write a function to export that in some suitable format.
2023-08-19 12:55:16 <MrMobius    > KipIngram: just curious, how do you plan to benchmark?
2023-08-19 12:55:43 <MrMobius    > for the comparison
2023-08-19 13:05:13 <KipIngram   > Well, obviously this Python thing will crawl.
2023-08-19 13:05:55 <KipIngram   > But I guess I could only compare things that I can run both of on a platform.  Comparing across platforms wouldn't say much about the architectures.
2023-08-19 13:06:10 <KipIngram   > So I may or may not have a good opportunity to do that.
2023-08-19 13:07:07 <KipIngram   > But some insight could be gotten from just studying the bits of code that carry me from machine code section to machine code section in two different approaches.
2023-08-19 13:08:38 <KipIngram   > In this new type of system, NEXT is along the lines of mov regA, regB ; shr regA, 6 ; and regB 0x3F ; jmp [tableReg+4*regB]
2023-08-19 13:08:59 <KipIngram   > That's not very much code.
2023-08-19 13:10:14 <KipIngram   > It should be faster than my current NEXT, because my current NEXT makes more RAM references.
2023-08-19 13:10:56 <KipIngram   > Of course, I will still have to use IP to fetch the next cell, but only every 5-6 times.
2023-08-19 13:11:29 <KipIngram   > Eventually 0x000000 will shift into the active position, and the table-accessed code for that will reload regA.
2023-08-19 13:11:40 <KipIngram   > I'm sorry - I meant 0b000000
2023-08-19 13:12:27 <KipIngram   > But that operation doesn't actually take up an instruction slot, so it will be in the role of a 6th or 7th operation in a pure-opcode cell.
2023-08-19 13:12:32 <MrMobius    > oh. I thought you might have meant comparing the speed of the vm on your microcontroller to C or assembly
2023-08-19 13:12:58 <KipIngram   > Oh, well, once I have it running on the microcontroller I'll be able to measure how much time the overhead code takes.
2023-08-19 13:13:23 <KipIngram   > I've already done such measurements, by writing a test word, timing it, then adding the bit I want to measure to it and timing it again.
2023-08-19 13:13:38 <KipIngram   > Loop over that like a billion times to get a nice measureable difference.
2023-08-19 13:13:43 <KipIngram   > Divide that by a billion.
2023-08-19 13:13:54 <MrMobius    > that's part of it but equivalent code in two languages would probably tell you more since you can avoid the DUP/SWAP/ROT stack thrashing stuff
2023-08-19 13:14:02 <MrMobius    > for example
2023-08-19 13:14:43 <KipIngram   > Oh, I see what you mean.  I'm not particularly interested in comparing to C, for example.  I know C is faster.  I've done some rough measurements of that sort on my system, and it came in around a third of C speed.
2023-08-19 13:15:14 <KipIngram   > But I haven't done it over a broad range of applications or anything.
2023-08-19 13:22:36 <KipIngram   > I'm thinking about putting my headers back inline with my definitions.  Now that I've got this notion of a vocabularly sitting in its own RAM region, it seems it would simplify things if that vocabulary was just one linear blob of stuff, instead of me needing to keep up with sections within it.
2023-08-19 13:23:16 <KipIngram   > And yes, that would mean if I wanted to run definitions together (finish a def with [ instead of ;) the compiler would have to recognize when to compile a jump around an intervening header.
2023-08-19 13:23:34 <KipIngram   > That's a pretty modest disadvantage, thouh.
2023-08-19 13:23:37 <KipIngram   > though
2023-08-19 13:23:43 <KipIngram   > I swear, my g key is dying.
2023-08-19 13:24:04 <KipIngram   > I may go all the way back to a straight FIG layout, per vocabulary.
2023-08-19 13:24:21 <KipIngram   > It is "the simplest way."
2023-08-19 13:24:33 <KipIngram   > And the most compact.
2023-08-19 13:25:52 <KipIngram   > link(2), count(1), name(?*4), cfa(4), <vm code>
2023-08-19 13:28:31 <KipIngram   > Actually that may not be exactly right - this is going to have the form of a code threaded system and I'm unfamiliar with them.  There may be differences.
2023-08-19 13:29:08 <KipIngram   > But anyway, you get the idea.  No extra layers of pointers like I've used in the past.
2023-08-19 13:30:18 <KipIngram   > How does one normally structure, say, a variable in a code threaded system?
2023-08-19 13:41:23 <MrMobius    > you mean the header in the dictionary?
2023-08-19 13:41:55 <MrMobius    > and i assume you mean subroutine threaded by code threaded
2023-08-19 13:43:19 <MrMobius    > sorry, just clarifying not being pedantic
2023-08-19 13:55:49 <KipIngram   > Yes and yes.
2023-08-19 13:56:16 <KipIngram   > This will essentially be a subroutine threaded system - just using vm code instead of machine code.
2023-08-19 13:56:55 <KipIngram   > That is, the vm has a literal "call" opcode, that's followed by a value specifying the call target.
2023-08-19 13:57:42 <KipIngram   > The F18A I'm using as the basis for this has its instruction set in section 2.3 here:
2023-08-19 13:57:44 <KipIngram   > https://www.greenarraychips.com/home/documents/greg/DB001-221113-F18A.pdf
2023-08-19 13:58:08 <KipIngram   > I'm going to use 6-bit opcodes instead of 5-bit, because I want to add a small number of additional instructions.
2023-08-19 13:58:17 <KipIngram   > But it will operate in very much the same way.
2023-08-19 13:59:28 <KipIngram   > He has 18 bit cells and packs opcodes three per cell, with a fourth occasionally possible if that fourth one is one of a select group.
2023-08-19 14:00:02 <KipIngram   > In my case I'll use 32-bit cells, so will get at least five 6-bit slots in a cell, with a sixth possible if it happens to be one of four opcodes.
2023-08-19 14:00:17 <KipIngram   > An opcode with the four high bits zero.
2023-08-19 14:00:58 <KipIngram   > I'm going to use a logical right shift in the NEXT code, so zero bits will stream in from the top.  Sooner or later I'll always get a 0b000000 opcode, and that will guide me to the next cell.
2023-08-19 14:00:59 <thrig       > the four high bits of the alpacalypse!
2023-08-19 14:01:40 <KipIngram   > So that's one of the four instructions - I'll be left with three I can choose as I like.
2023-08-19 14:02:00 <KipIngram   > 0b000001, 0b000010, and 0b000011.
2023-08-19 14:02:24 <KipIngram   > The compiler will recognize opportunities to exploit that.
2023-08-19 14:02:58 <KipIngram   > A call or jump opcode will always be the last opcode in a cell - the remainder of the cell will be taken as the operand.
2023-08-19 14:03:45 <KipIngram   > So that means how far you can call or jump will depend on how much of the cell you've already used - the compiler will handle that too, and if necessary will zero the rest of the cell and move to the next one.
2023-08-19 14:05:20 <KipIngram   > It will be a more complicated compiler than Forth normally requires, given all these things it will have to keep track of.
2023-08-19 14:05:29 <KipIngram   > But it's all "straightforward."
2023-08-19 14:08:24 <KipIngram   > I toyed with the idea of having a call opcode that didn't use up the whole remainder of the cell (more opcodes could come after the target spec), but that would necessitate saving not only the IP on the return stack on calls but also the current partially shifted out instruction word.  I decided I didn't want to add that much overhead to call/return.
2023-08-19 14:08:49 <KipIngram   > It just means occasionally wasting a few bits of potentially usable code space.  But this is already going to be a quite compact system.
2023-08-19 14:11:16 <KipIngram   > I've got 128k of RAM on this MAX32655 - with this approach that will hold an awful lot of functionality.
2023-08-19 15:21:06 <vms14       > how do languages achieve to be implemented in themselves?
2023-08-19 15:21:46 <vms14       > I'm doing it in a wrong way as always, which is to check if a word exists and provide a default fallback if not
2023-08-19 15:23:47 <vms14       > I have interpret_element which is the eval function of the lang (not string eval), this will check first if the element is a word and execute it, if not it will try to find some words depending on what kind of element it is, if the words are defined the element is pushed on the stack and the word executed, if not a builtin behavior is used instead
2023-08-19 15:24:18 <vms14       > but it's not how I should be doing it xd
2023-08-19 15:24:45 <vms14       > for example I can : interpret.list some code to interpret a list as code ;
2023-08-19 15:24:59 <vms14       > now when the interpreter evaluates a list will execute that word
2023-08-19 15:26:13 <vms14       > [ 1 2 3 ] eval    this would push the list and execute 'interpret.list'
2023-08-19 15:26:15 <vms14       > and allows to make a handler for any type, even objects defined by the user
2023-08-19 15:26:50 <vms14       > : interpret.meh 'hahaha ;   some.word.that.returns.a.meh eval
2023-08-19 15:27:36 <vms14       > but this is not how metacircular and meta forths are made for example
2023-08-19 15:27:51 <vms14       > they usually define some primitives and the rest is from there
2023-08-19 15:28:10 <vms14       > still not even know properly what a metacircular forth means
2023-08-19 15:30:06 <thrig       > that you could be going in circles
2023-08-19 15:54:02 <vms14       > but circular data blows my borrowed return stack :/
2023-08-19 16:13:48 <KipIngram   > vms14: Well, think about it for a minute.  A computer program is a pattern of bytes in RAM.  A computer language provides you with a way of manipulating RAM.  So, you just have your program put the right bytes into RAM, guided by some sort of source code representation of what that should be.
2023-08-19 16:14:18 <KipIngram   > Forth obviously lets you compile Forth source code into RAM - why couldn't that source code specify RAM contents that will then run as a Forth system?
2023-08-19 16:15:11 <vms14       > yeah forth is a jump table
2023-08-19 16:15:23 <KipIngram   > You would start using your Forth's assembly language tools, to lay down the code for initialization, the handling of variables, constants, definitions, and so on.
2023-08-19 16:15:23 <vms14       > able to add new entries
2023-08-19 16:15:37 <vms14       > hmm
2023-08-19 16:16:04 <KipIngram   > Often CREATE is set up so that you can have it put its results into an arbitrary image buffer, rather than into the dictionary.
2023-08-19 16:16:32 <KipIngram   > That flexibility has to be designed in, of course - you have CREATE use variables to guide its actions, and the default value for those variables is "add to the running dictionary."
2023-08-19 16:16:43 <KipIngram   > But you can modify them to have the constructed headers placed elsewhere.
2023-08-19 16:17:07 <KipIngram   > Similarly the compiler can have the flexibility to put the compiled cell values in an alternate image buffer.
2023-08-19 16:17:50 <KipIngram   > Lots of details you have to keep straight, differentiating between the "old, running" system and the "new, target" system.  But, with enough cleverness it can be done - it's straightforward and "magic free."
2023-08-19 16:19:05 <KipIngram   > when I say I want my system to be able to recompile itself, I just mean that I want a set of source code to feed to the thing that will result in an image that matches the running image.  Ideally I'd be able to md5 hash image that I start up with, md5 hash the generated image, and have them match.
2023-08-19 16:21:40 <KipIngram   > Just to keep it simple, consider a FIG system where everything goes into contiguous memory.  There's a variable called DP which holds the address of the next available byte in the dictionary.
2023-08-19 16:22:11 <KipIngram   > If you change DP to point into some buffer, then , (the word that ticks content into the dictionary) will put the content in the new buffer.
2023-08-19 16:22:23 <KipIngram   > CREATE, : and so on will inherit that, since they use , ultimately.
2023-08-19 16:22:45 <KipIngram   > Just be sure to save your original DP value so you can put it back - otherwise you'll lose the ability to modify your dictionary.
2023-08-19 16:23:38 <KipIngram   > Also, when meta-compiling you have to keep straight where to search for words.
2023-08-19 16:24:08 <KipIngram   > When you're searching for a word to RUN, you look in your running, standard dictonary.  When searching for a word TO COMPILE, you need to look in the dictionary you're BUILDING.
2023-08-19 16:24:18 <KipIngram   > That involves further variables beyond DP, but it's the same idea.
2023-08-19 16:24:28 <KipIngram   > Just reset all the right variables and process your source.
2023-08-19 16:25:05 <KipIngram   > If that's successful you can then jump to the right starting point in the new image (if it's for the same processor) and it will work.
2023-08-19 16:25:27 <KipIngram   > But of course you could do this for an altogether different platform - different processor, etc.
2023-08-19 16:25:47 <KipIngram   > Your assembly words would just have to match that target processor.
2023-08-19 16:26:00 <KipIngram   > does that make it seem less mysterious?
2023-08-19 16:26:21 <vms14       > KipIngram: did you start with the metacompiler?
2023-08-19 16:26:51 <KipIngram   > No, my system that runs currently is built in assembly language using nasm.
2023-08-19 16:27:06 <KipIngram   > I haven't ever actually achieved this goal.  But I want to.
2023-08-19 16:27:46 <KipIngram   > And you can't just "start with a metacompiler" - before you can RE build a system you have to have that system running so it can do the building.
2023-08-19 16:28:29 <KipIngram   > It's helpful, though, that large parts of my system are basically expressed as high level Forth.
2023-08-19 16:28:51 <KipIngram   > It's high level Forth hand-assembled using nasm, but nonetheless it's Forth definitions.
2023-08-19 16:28:59 <KipIngram   > And I have the Forth source in comments above each word.
2023-08-19 16:29:23 <KipIngram   > So when I do get meta-compile in "basic operation," I'll be able to take that source and use it as part of my metacompilation source.
2023-08-19 16:29:33 <KipIngram   > So I don't have to really "re-write" everythin.
2023-08-19 16:30:22 <KipIngram   > But to the extent that I have stuff written in assembly language, in nasm format, yeah - I'll have to re-implement all of that in a form that my Forth can process.
2023-08-19 16:35:39 <vms14       > as I do the stuff in such a different way I have to translate your stuff into my world
2023-08-19 16:36:01 <vms14       > usually I just can't as they're two different world
2023-08-19 16:36:20 <vms14       > but the transpiling thing
2023-08-19 16:36:47 <vms14       > you told me something like you would have the definition of the host language and the definition of the target lang
2023-08-19 16:37:19 <vms14       > and I'm like, ok you can do that since you can know what the instruction codes are for the target
2023-08-19 16:37:27 <vms14       > and make the word be those numbers
2023-08-19 16:37:48 <vms14       > same way does forth to provide an assembler in itself
2023-08-19 16:38:20 <vms14       > host machine*
2023-08-19 16:38:45 <vms14       > in my case instead of a machine is a language and I can somehow make the words be code
2023-08-19 16:40:27 <vms14       > I have to play with that more
2023-08-19 18:49:11 <KipIngram   > Right, given you high reliance on external components it might be hard for you - just depends on how your "wiring" into those things works.  You don't actually have a "raw image" that defines your system - you're all wired into the underpinnings of Perl and so on.
2023-08-19 18:49:37 <KipIngram   > But I can think of my full system as just an array of bytes.
2023-08-19 18:50:37 <KipIngram   > I'm approaching this Python emulator as an array of 32768 32-bit cells.  Initialized to all zeroes, and then as I advance DP through it I "OR in" content.
2023-08-19 18:51:11 <KipIngram   > So it's easy to write Python functions compile(op) and compile_val(op, val).
2023-08-19 18:51:47 <KipIngram   > I've pretty much got that simple bit written now - it handles checking how much is left of the current cell and advancing to the next cell internally, so I won't have to think about that at higher levels.
2023-08-19 18:51:56 <KipIngram   > I'm just about to write the bit that stuffs headers in there.
2023-08-19 18:52:36 <KipIngram   > For the instruction compilation I move dp by 4 when necessary - for the header section I need to work with bytes.  But I'll guarantee that I have dp back to a multiple of four after the header.
2023-08-19 18:53:06 <KipIngram   > After this construction stuff is all working I'll write some more stuff that will actually emulate execution.
2023-08-19 18:54:57 <KipIngram   > So, while it will be slow, this tool will be a functionin system that I can use to develop code.
2023-08-19 21:11:01 <vms14       > I'm playing with the transpile interpret thing
2023-08-19 21:11:57 <vms14       > It's easy if I put a stack on the target too, so there's two data stacks, at compile time on the host and at runtime of the generated code
2023-08-19 21:12:22 <vms14       > The funny thing is stuff like conditionals, loops, etc
2023-08-19 21:14:17 <vms14       > And there are some constraints imposed by perl itself, like I have to declare all variables I'll use on the generated program and  every time I use eval to generate the function to use when interpreting
2023-08-19 21:14:36 <vms14       > I assume the key is to start making words that help me generate code
2023-08-19 21:16:44 <vms14       > Also now I want the compile flag on the interpreter I guess
2023-08-19 21:36:38 <KipIngram   > You're talking about STATE?
2023-08-19 21:52:44 <vms14       > Yes
2023-08-19 22:37:00 <xelxebar    > : POSTPONE  ' DUP imm IF COMPILE, ELSE [COMPILE] LITERAL   ['] COMPILE, COMPILE, THEN ; IMMEDIATE
2023-08-19 22:38:01 <xelxebar    > So if you want to POSTPONE n times, you need 2^n POSTPONEs. Fun.
2023-08-19 22:39:57 <KipIngram   > I've never really figured out POSTPONE.
2023-08-19 22:40:04 <KipIngram   > It was a later addition to Forth.
2023-08-19 22:42:02 <xelxebar    > Just postpones execution of the word until the next call, so ABC is immediate then POSTPONE ABC gets compiled in.
2023-08-19 22:42:27 <xelxebar    > And if ABC isn't immediate, then it turns into code that compiles ABC into place at runtime.
2023-08-19 22:42:53 <xelxebar    > At least, that's how I'm interpreting the code above.
2023-08-19 22:43:58 <xelxebar    > imm is SmithForth specific but just means ( xt -- is-immediate )
2023-08-19 22:46:38 <xelxebar    > I'm guessing that postpone is a tactic to avoid the proliferation of bracketed pairs of words.
2023-08-19 22:47:32 <xelxebar    > Different approach, but in spirit, similar to how you avoid the explosion of parenthesized/bare pairs.