2023-05-09 06:01:45 <veltas      > KipIngram: I think that's a terminal option, whether \n generates the corresponding carriage return
2023-05-09 06:03:20 <veltas      > And this is how C on UNIX systems implements new lines in the terminal, because in the C standard \n must generate a new line (not just a line feed in terminal)
2023-05-09 06:03:31 <veltas      > so they just leave that option on by default
2023-05-09 06:04:02 <veltas      > In Windows I would imagine that instead the putc() et al functions check for \n and generate \r\n when in text mode
2023-05-09 06:04:11 <veltas      > And in binary mode you need to use \r\n
2023-05-09 06:41:13 <KipIngram   > Yeah, probably so.
2023-05-09 06:41:28 <KipIngram   > (terminal option).
2023-05-09 07:26:05 <KipIngram   > I started to pull together a little set of block editing words last night that looked fairly promising.  Basically a line editor - word for displaying the block, words to move the target line around in the block, insert a new line somewhere, delete a line, etc.
2023-05-09 07:26:27 <KipIngram   > Won't be as smooth as my previous editor, but so far it feels like it would get the job done.
2023-05-09 07:27:04 <KipIngram   > Uses newline chars and not 64-char lines.
2023-05-09 07:27:48 <KipIngram   > And it capitalizes on the ability a component of my EXPECT has to carry the whole remainder of the block along for the ride while I'm editing a line.
2023-05-09 07:29:07 <vms14       > I've started to fake compile by default, althought it's not good yet
2023-05-09 07:29:34 <vms14       > but everything becomes a function, a colon word becomes a function that executes an array of functions xD
2023-05-09 07:30:03 <vms14       > it's not good cause I'm not caching compiling, so everything is compiled every time
2023-05-09 07:30:17 <vms14       > I have to fix that, but somehow I see it a bit more similar than forth xD
2023-05-09 07:30:46 <vms14       > instead of a pointer to assembly instructions, it's a function
2023-05-09 07:30:59 <KipIngram   > There's a Forth out there I've seen that does interpretation that way - it compiles the whole line and then executes it.  Just doesn't save it unless you're making a definition.
2023-05-09 07:31:09 <vms14       > also realized forth wants to be written in assembly, but my lang wants to be written in a high level lang
2023-05-09 07:31:18 <KipIngram   > The nice thing about it is that you can use any words in "interpret" mode, since it's really compile mode under the hood.
2023-05-09 07:31:23 <KipIngram   > So you can have loops and so on.
2023-05-09 07:31:40 <vms14       > because it assumes a call stack, environments, hashes, lists and a gc exist
2023-05-09 07:32:08 <KipIngram   > Yeah, you are using some of the abstract things your underlying language offers.
2023-05-09 07:32:12 <vms14       > KipIngram: yeah, I had it but as optional way, in order to provide a bit of efficiency
2023-05-09 07:32:34 <vms14       > but this time I'm not saving words
2023-05-09 07:32:53 <vms14       > because I need to change the functions for objects
2023-05-09 07:33:13 <vms14       > or whatever it is that gives me metadata, for example to see if it's an immediate function
2023-05-09 07:33:31 <vms14       > so yeah, they need a header
2023-05-09 07:33:32 <vms14       > xD
2023-05-09 07:34:18 <vms14       > the cool thing is if I save the compiled version of a word, and this word is tail recursive, node can give it tco
2023-05-09 07:35:06 <vms14       > when compiling the definition of a word, if I find the name of the word I'm compiling, I return a function that will execute the function of the future cached version
2023-05-09 07:35:21 <vms14       > as in a recursive definition the word isn't yet compiled
2023-05-09 07:36:35 <KipIngram   > I'm really planning to start a new system, but I'm givin serious consideration to trying to use the existing system to develop it.  Just get this metacompile thing tackled.
2023-05-09 07:36:36 <vms14       > I was amazed that node was able to provide tco with anon subs that come from a "hash"
2023-05-09 07:36:51 <vms14       > KipIngram: oh, forth metacompilers interest me
2023-05-09 07:37:04 <KipIngram   > It's mostly just a bookkeeping problem.
2023-05-09 07:37:21 <vms14       > I want to learn about them sooner or later
2023-05-09 07:37:26 <KipIngram   > You need two sets of various pointers, and you have to know which set to use at various times.
2023-05-09 07:38:38 <vms14       > KipIngram: how do you turn a recursive word into a loop even if it's not tail recursive?
2023-05-09 07:39:04 <vms14       > oh you don't need that, you have a call stack
2023-05-09 07:39:33 <vms14       > I tried to do it, once the recursion ended it was missing to execute the remainder of the word
2023-05-09 07:39:52 <vms14       > I assume this is why tail call is usually optimized, they convert it in a loop
2023-05-09 07:40:13 <vms14       > but since it's a stack based language, I think I can take the remainder of the recursion before starting the loop
2023-05-09 07:40:23 <vms14       > once the loop ends, execute the remainder
2023-05-09 07:40:42 <KipIngram   > Um, there is a way to do that, but it's been a long time since I've looked into it.
2023-05-09 07:40:52 <KipIngram   > There's an "algorithm" for the conversion, sort of.
2023-05-09 07:41:09 <KipIngram   > Basically you have to provide the equivalent of the standard stack that the recursion is taking advantage of.
2023-05-09 07:41:10 <vms14       > I'm not sure if "saving" the remainder of the word before starting recursion, and execute it later will be enough
2023-05-09 07:41:16 <vms14       > it looks like it would work for me
2023-05-09 07:41:37 <vms14       > it would be cool as then I have recursion without blowing the stack
2023-05-09 07:41:44 <KipIngram   > You can't dodge having to save the information - you just do it explicitly instead of having it happen transparently using the standard stack.
2023-05-09 07:42:23 <KipIngram   > Sometimes your standard stack has limited capacity - if you allocate the necessary memory yourself, you avoid running into that limit.
2023-05-09 07:42:41 <vms14       > my stack has the same limit as the memory available
2023-05-09 07:42:46 <vms14       > it's perl again
2023-05-09 07:42:47 <vms14       > xD
2023-05-09 07:42:49 <KipIngram   > It's just particularly easy in the tail recursion case, because you don't have to save any information in that case.
2023-05-09 07:43:17 <KipIngram   > Yeah, if your system stack is unlimited, then you can go either way - recursive or iterative.
2023-05-09 07:43:30 <vms14       > I thought perl was a playground and I had to make a C version once I had an idea of what I'm doing
2023-05-09 07:43:55 <vms14       > but there's no reason to use C, unless I need performance
2023-05-09 07:44:11 <vms14       > and I never wrote a program that needs performance
2023-05-09 07:44:23 <KipIngram   > Or if you just want to learn to solve some of the things Perl gives you for free.
2023-05-09 07:44:33 <vms14       > I tried, cause for example if I compare the perl version with the js one I cry
2023-05-09 07:44:48 <vms14       > KipIngram: that's the best benefit I have
2023-05-09 07:44:49 <KipIngram   > I've written some things worth crying over over the years.
2023-05-09 07:45:04 <vms14       > I can focus on whatever I want and don't have to be distracted implementing stuff I already have
2023-05-09 07:45:14 <vms14       > this is also the reason it differs so much from forth
2023-05-09 07:45:31 <KipIngram   > Sure.
2023-05-09 07:45:43 <KipIngram   > It's not a Forth - it's a stack-oriented "something else."
2023-05-09 07:45:46 <vms14       > yeah xD
2023-05-09 07:46:11 <KipIngram   > But you've had fun, and I can tell you've learned some things.
2023-05-09 07:46:38 <vms14       > and it never stops changing
2023-05-09 07:46:51 <vms14       > I don't know how many times I've rewritten
2023-05-09 07:46:58 <vms14       > even when I'm not using rust
2023-05-09 07:47:15 <KipIngram   > Re: metacompiling, there are really two issues.  When you do that, you have two dictionaries - the one you're running from and the one you're creating.
2023-05-09 07:47:31 <vms14       > but it will be cross compiling?
2023-05-09 07:47:34 <KipIngram   > So you need two pointers to the next free byte in the dictionary.
2023-05-09 07:47:37 <vms14       > so it will be on other machine?
2023-05-09 07:47:56 <KipIngram   > When you compile a meta word, you want to compile it into the new dictionary, and advance that dictonary's pointer.
2023-05-09 07:48:11 <KipIngram   > And you also want to SEARCH that dictionary for the words in the definition you're building.
2023-05-09 07:48:18 <KipIngram   > That's all - you just have to keep those things straight.
2023-05-09 07:48:43 <KipIngram   > You don't want the target dictionary pointing at anything in hte host dictionary.
2023-05-09 07:49:07 <KipIngram   > Meanwhile, though, your host system has to continue to run properly
2023-05-09 07:49:42 <KipIngram   > And if you've written the host system without having that in mind, it's probably not going to be awfully easy.
2023-05-09 07:50:04 <vms14       > so you just decide whether to add stuff on the host or target dictionary?
2023-05-09 07:50:17 <KipIngram   > I'm not expecting it to be particularly easy i nmy case here - I will probably have to make some adjustments to the host system I've got in order to make it w ork.
2023-05-09 07:50:40 <vms14       > that's why you wanted to make a new system
2023-05-09 07:50:57 <KipIngram   > Yes, correct.  The "easiest" way would be to use different words to compile to the different dictionary, but you do want the target result to have the right names for its words and so on.
2023-05-09 07:51:05 <KipIngram   > Like, if you see : foo ... ;
2023-05-09 07:51:13 <KipIngram   > are you adding a word to the host, or to the target?
2023-05-09 07:51:29 <vms14       > I expected to have different syntax to decide that
2023-05-09 07:51:30 <KipIngram   > You have to have a way to tell the host which thing to do.
2023-05-09 07:51:39 <vms14       > well a flaag would also do
2023-05-09 07:51:40 <KipIngram   > Then your new system has a different syntax.
2023-05-09 07:51:48 <KipIngram   > Yes - you need a flag-based way.
2023-05-09 07:52:13 <KipIngram   > So that the two systems can share various bits of naming/syntax.
2023-05-09 07:52:28 <vms14       > hmm
2023-05-09 07:52:32 <KipIngram   > could just be as "simple" has having words HOST and TARGET.
2023-05-09 07:52:43 <KipIngram   > TARGET : foo ... ;
2023-05-09 07:52:49 <vms14       > in my case metacompiling on the same system does not make much sense
2023-05-09 07:52:56 <vms14       > but "metatranspiling" does
2023-05-09 07:53:03 <vms14       > also transpiling
2023-05-09 07:53:15 <vms14       > haha I like the term
2023-05-09 07:53:19 <vms14       > metatranspiling
2023-05-09 07:53:29 <KipIngram   > Yes - I'm interested in building a new system that runs on the same machine, but once you've got this all set up and working that's not a "requirement" - you certainly could cross-compile.
2023-05-09 07:53:39 <vms14       > in your case it does make sense
2023-05-09 07:53:45 <vms14       > in mine I just add overhead
2023-05-09 07:54:39 <vms14       > KipIngram: how would you cross compile?
2023-05-09 07:55:53 <KipIngram   > Well, in addition to keeping track of your two dictionaries, you also have the ability to write primitives for the other architecture.
2023-05-09 07:56:26 <KipIngram   > And you might have a different cell size, or endianess, or whatever on the new system - you just have to "handle all that."
2023-05-09 07:56:53 <KipIngram   > Like, when you compile a primitive, you use the right instruction set.
2023-05-09 07:57:56 <KipIngram   > So, 1) which dictionary do you search, 2) which dictionary do you add new stuff to, and 3) what processor / system platform architecture do you use.
2023-05-09 07:58:30 <KipIngram   > One host/target flag can tell you what you're doing - you just have to wire that flag to all the right bits of functionality.
2023-05-09 07:59:52 <KipIngram   > Also, you're probably saving this target image in disk buffers, and later you're going to want to load it to some arbitrary place in the target's RAM and run it.  That's probably going to require some adjustment to the actual image to account for where you put it in RAM when you want to run it.
2023-05-09 08:00:05 <KipIngram   > So somehow your image has to tell you how to do all that.
2023-05-09 08:00:35 <KipIngram   > That's the same as on Linux, though - the loadable image format has records in it that tell Linux how to properly load it into RAM.
2023-05-09 08:00:35 <vms14       > hmm
2023-05-09 08:00:44 <KipIngram   > The ELF executable spec.
2023-05-09 08:01:18 <vms14       > so I just need a way to "insert" primitives
2023-05-09 08:01:22 <KipIngram   > In my case, for example, those tables that hold CFAs and PFAs will probably have "offsets" in the image, but when I load it I'll need to modify all those entries to make them addresses.
2023-05-09 08:01:35 <KipIngram   > Which will just mean adding the base address I load at to every entry.
2023-05-09 08:01:37 <vms14       > well that's mainly the whole thing about transpiling
2023-05-09 08:02:13 <vms14       > what you explained kind of made a bulb on my head almost be on
2023-05-09 08:02:19 <vms14       > :D
2023-05-09 08:02:31 <KipIngram   > If you knew exactly where you were going to load it ahead of time, you could just output the addresses right into the image.  But unless you're making something for some embedded device where you control the whole address space, you probably don't know that.
2023-05-09 08:02:38 <KipIngram   > In Linux there's no teling where it will get put.
2023-05-09 08:04:38 <KipIngram   > This is part of why you rarely see Forth allow you to "save compiled programs."  It dodges that problem altogether - you just compile your source every time, and since you're compiling it directly to where it will run, you can just set those addresses as you go.
2023-05-09 08:04:56 <veltas      > On modern arches you want to put code separate to data, because most modern machines have separate instruction and data L1 caches
2023-05-09 08:05:09 <KipIngram   > Yes.
2023-05-09 08:05:30 <KipIngram   > Which raises the interesting question of "what is a colon definition"?
2023-05-09 08:05:42 <KipIngram   > To the processor it looks like data, at least on a threaded design.
2023-05-09 08:05:47 <KipIngram   > direct or indirect, that is.
2023-05-09 08:06:03 <veltas      > Yeah it's data unless you're STC
2023-05-09 08:06:13 <veltas      > Or return-threaded as well
2023-05-09 08:06:27 <veltas      > Well actually no sorry
2023-05-09 08:06:30 <veltas      > Just STC
2023-05-09 08:06:53 <KipIngram   > I generally put my primitives and my definitions in the same zone, but in a "separated" way.  Right where they meet it might overlap, but "most" of the code is separated from "most" of the definition data.
2023-05-09 08:06:54 <veltas      > Return threaded code the colon def is just a list of code addresses, so it's like direct threading
2023-05-09 08:07:07 <KipIngram   > Yes.
2023-05-09 08:07:29 <KipIngram   > I could always stick an align at the boundary, I guess - align to a fresh cache line.
2023-05-09 08:07:40 <KipIngram   > But ideally they'd be in separate regions.
2023-05-09 08:07:56 <KipIngram   > Because if I added new primitives to my system the way it is now, then they'd start to get mixed up.
2023-05-09 08:08:03 <veltas      > Unnecessary alignment sounds bad
2023-05-09 08:08:16 <KipIngram   > Yeah, I try not to do too much of it.
2023-05-09 08:08:54 <KipIngram   > But one between "all the code" and "all the defs" wouldn't be too bad.
2023-05-09 08:08:55 <veltas      > You probably want code unaligned and contiguous, only align really hot bits of code to the size of the instruction (i.e. loop restart point)
2023-05-09 08:09:06 <KipIngram   > Yes, I don't align within code.
2023-05-09 08:09:31 <KipIngram   > I do align between headers, so odd length names won't throw out the alignment of stuff coming after.
2023-05-09 08:09:38 <veltas      > Sections tend to have a reasonably large alignment
2023-05-09 08:10:09 <veltas      > I think often they start at 4KB alignment?
2023-05-09 08:10:12 <KipIngram   > I think so.  Maybe 4k?  I think the hardware memory permission stuff works on 4k page boundaries.
2023-05-09 08:10:15 <veltas      > Or 1KB at least
2023-05-09 08:10:51 <KipIngram   > Maybe as I do this new system I'll fully separate machine code from definitions.
2023-05-09 08:11:18 <KipIngram   > One of the advantages of developing it in Forth instead of nasm is that I can do anything of that sort I want, without having to figure out how to get nasm to cooperate.
2023-05-09 08:11:30 <veltas      > It's an optimisation that's not too hard, so probably worth doing
2023-05-09 08:15:40 <veltas      > I have actually converted ilo-amd64 to gas format because I got fed up with nasm
2023-05-09 08:16:04 <veltas      > And converting I found a bug where nasm had silently 'fixed' an invalid effective address
2023-05-09 08:16:24 <KipIngram   > Ooops.
2023-05-09 08:16:34 <KipIngram   > I haven't run into any terrible problems with nasm thus far.
2023-05-09 08:16:39 <veltas      > So I will probably avoid nasm from now on
2023-05-09 08:16:48 <veltas      > The thing is that GNU as is just far more battle-tested than nasm
2023-05-09 08:17:32 <veltas      > It's got a very important job so it absolutely 'works' and supports things I think nasm is lacking like anonymous labels
2023-05-09 08:17:46 <KipIngram   > vms14: Another possible way to handle the meta compiling would be to just write a completely new target compiler, instead of trying to trick the system compiler into doing it for you.  Something like this:
2023-05-09 08:17:52 <KipIngram   > TARGET{
2023-05-09 08:17:59 <KipIngram   > : foo ... ;
2023-05-09 08:18:02 <KipIngram   > : bar ... ;
2023-05-09 08:18:05 <KipIngram   > ...
2023-05-09 08:18:09 <KipIngram   > }
2023-05-09 08:18:11 <unjust      > doesn't nasm represent local labels with a leading .?
2023-05-09 08:18:27 <KipIngram   > I'm not sure - I don't actually use them.
2023-05-09 08:18:30 <veltas      > unjust: Yeah and it generates symbols for them and there's no way to turn that off, which is annoying
2023-05-09 08:18:33 <KipIngram   > Sounds reasonable, though.
2023-05-09 08:18:36 <unjust      > not quite the 1b / 1f that gas offers though
2023-05-09 08:18:40 <veltas      > Yeah
2023-05-09 08:19:00 <veltas      > 1b / 1f don't generate extra symbols
2023-05-09 08:19:14 <KipIngram   > vms14: So TARGET{ would just take control and wouldn't give it up until it saw that closing }.
2023-05-09 08:19:21 <veltas      > Better debugging output, less noise in the ELF
2023-05-09 08:19:37 <KipIngram   > And if you want to use } for something else, it could be TARGET} at the tail end.
2023-05-09 08:19:55 <veltas      > You can even use intel syntax in gas now
2023-05-09 08:20:11 <veltas      > Although I've not done that because 'when in Rome'
2023-05-09 08:20:14 <KipIngram   > That's a fair bit of code to write, though - TARGET{ would have to have a FIND and so on.
2023-05-09 08:20:22 <vms14       > KipIngram: but what target would really do
2023-05-09 08:20:25 <KipIngram   > But it would just completely solve the juggling issue.
2023-05-09 08:20:34 <vms14       > everything I assume
2023-05-09 08:20:35 <veltas      > KipIngram: I think it's necessary to allow handling literals properly
2023-05-09 08:20:54 <KipIngram   > It would call WORD to parse words out of the source, search for them in the target dictionary, act on them to compile the results to the target dictionary, using the target architecture, etc.
2023-05-09 08:21:07 <KipIngram   > It would do pretty much what the normal system compiler does.
2023-05-09 08:21:08 <vms14       > I mean, not only put those definitions on the target, but has to care about everything
2023-05-09 08:21:29 <KipIngram   > Yeah, it has to do a fair bit.  It's just a way to solve the bookkeeping problem cleanly.
2023-05-09 08:21:34 <veltas      > It's not that big though
2023-05-09 08:21:40 <vms14       > at least that way you can use your host normally
2023-05-09 08:21:47 <vms14       > and have this almost as an extension
2023-05-09 08:21:54 <veltas      > You've already had most of the work done for you, you just need to rewrite the high level compiler-interpretre
2023-05-09 08:21:55 <KipIngram   > No, it wouldn't be "awful," and if you've got the source for your host system you could capitalize on it quite a lot.
2023-05-09 08:22:20 <vms14       > I tend to add words like that in order to not mess with the interpreter code
2023-05-09 08:22:37 <KipIngram   > It's probably what I will do, given that I didn't really plan thoroughly for metacompile while writing my current system.
2023-05-09 08:22:45 <vms14       > I like the interpreter to be as dumb as possible
2023-05-09 08:22:51 <veltas      > It would be helpful to vectorise COMPILE, and LITERAL for this
2023-05-09 08:22:54 <vms14       > read a word, recognize it
2023-05-09 08:23:17 <KipIngram   > And you could have a word TARGET-LOAD such that you didn't have to actually have TARGET{ and TARGET} in your source blocks themselves.
2023-05-09 08:23:21 <veltas      > And then add new definitions for some words like IF/THEN
2023-05-09 08:23:48 <KipIngram   > Right - it would need to handle those properly as well.  But it's a "clean problem" now, instead of a noodly mess.
2023-05-09 08:23:58 <veltas      > Hmmmm
2023-05-09 08:24:03 <veltas      > nasm - noodle assembler
2023-05-09 08:24:11 <KipIngram   > Heh.
2023-05-09 08:24:52 <KipIngram   > It feels more tractable to me than fighting with my existing code trying to fix all the things I left out while writing it.
2023-05-09 08:26:39 <unjust      > until nasm allows you to prevent local symbol export, you could remove the locals with objcopy if they bothered you enough
2023-05-09 08:27:26 <veltas      > unjust: That's just one reason, I think gas is a better choice for many reasons
2023-05-09 08:27:48 <KipIngram   > I'm hoping I won't be going back to nasm anymore.
2023-05-09 08:30:00 <KipIngram   > vms14: Another reason I like this TARGET{ idea is that I am planning a pretty different dictionary layout, and the existing FIND for example assumes a particular layout that the new system won't have.
2023-05-09 08:30:19 <KipIngram   > It would get to be a real mess trying to make the existing FIND conditionally support either/or.
2023-05-09 08:30:51 <KipIngram   > I'm even planning a different xt size.
2023-05-09 08:31:28 <KipIngram   > I will try to write the new system with native metacompile support in mind, though.
2023-05-09 08:33:43 <vms14       > it could be better than just a flag
2023-05-09 08:33:54 <vms14       > but somehow equivalent
2023-05-09 08:34:18 <vms14       > anyways the more you put on words and less on the "core" the better
2023-05-09 08:34:22 <KipIngram   > Well, it eliminates needing a flag, since you're no longer coaxing your system into double duty.
2023-05-09 08:34:51 <vms14       > every word is a world
2023-05-09 08:35:00 <KipIngram   > Then if the new system will run on your host system, you could have TARGET-RUN, which would load the image from disk buffer into a run buffer and launch it.
2023-05-09 08:35:27 <KipIngram   > And then you'd be running the new system, and BYE would drop you back to the host system.
2023-05-09 08:35:47 <KipIngram   > Once you get it that far along.
2023-05-09 08:36:01 <KipIngram   > A common step for me in early development is just getting the system to run and exit without throwing an error.
2023-05-09 08:36:03 <vms14       > KipIngram: why did you want metacompiling?
2023-05-09 08:36:09 <vms14       > just cause it was interesting?
2023-05-09 08:36:12 <KipIngram   > That would work here too.
2023-05-09 08:36:25 <KipIngram   > Well, yes, but mainly so that I no longer have to use external development tools.
2023-05-09 08:36:31 <KipIngram   > I'd like to "cut that tether."
2023-05-09 08:37:21 <KipIngram   > So I can see in the early phase of this trying to get to where I can TARGET-RUN and it just loads the image, runs it, and pops right back out.
2023-05-09 08:37:29 <KipIngram   > Just a "no error pass" through the new stuff.
2023-05-09 08:38:06 <veltas      > Just make sure you have offsite backups of your binaries because you can't build from source anymore
2023-05-09 08:38:44 <KipIngram   > For sure.
2023-05-09 08:39:37 <KipIngram   > I'd keep the last nasm source and the blocks.dat file that it can use to build the new system of course.
2023-05-09 08:39:51 <KipIngram   > But you're right - keeping several different options around all that would be good.
2023-05-09 08:40:54 <KipIngram   > And I'd want a "golden" image somewhere in blocks.dat too, because I'm sure I will break my dev image many times along the way.
2023-05-09 08:41:48 <KipIngram   > I imagine having a little Linux executable that lets me say loader <blocks.dat> <block num>"
2023-05-09 08:42:14 <KipIngram   > It would allocate a bunch of RAM, load the right part of blocks.dat into it, and jump to it.
2023-05-09 08:49:35 <KipIngram   > veltas: Here's question re: separating code and data.
2023-05-09 08:50:01 <KipIngram   > In this new plan of mine, I will have registers pointing at each of the tables (CFA and PFA pointer tables).
2023-05-09 08:50:35 <KipIngram   > In theory I wouldn't need a register pointing at the code / definition zone(s) - the CFA/PFA pointers themselves will point into those zones.
2023-05-09 08:51:24 <KipIngram   > In my existing system, though, I have a register pointing to that area, and I let it do double duty by putting NEXT first so that that register points to it too, and a jmp <reg> instruction serves as my next in primitives.
2023-05-09 08:51:48 <KipIngram   > To avoid allocating a third register, I'd contemplated putting next at the base of  one of the tables.
2023-05-09 08:51:55 <KipIngram   > How big an issue do you see that as?
2023-05-09 08:52:03 <KipIngram   > Probably the most frequently executed code in the system.
2023-05-09 08:52:18 <KipIngram   > We'd only be talking about one cache line.
2023-05-09 08:52:38 <KipIngram   > And none of the "data" in the table that was shared in that cache line will be written to, ever.
2023-05-09 08:53:04 <KipIngram   > It doesn't feel to me like it would be problematic, but I think you're better tuned into this aspect of things than I am.
2023-05-09 08:53:23 <KipIngram   > Wouldn't I just wind up with copies of that line in both caches?
2023-05-09 08:54:15 <KipIngram   > And there I would put an align after the next code, just so all the table entries were aligned.
2023-05-09 08:54:22 <KipIngram   > Probably only a 16-bit align, though.
2023-05-09 08:54:35 <KipIngram   > No wait - sorry.  A 64-bit align.
2023-05-09 08:54:42 <KipIngram   > Table entries are 64 bits.
2023-05-09 08:54:48 <KipIngram   > xt's in definitions will be 16 bits.
2023-05-09 08:55:15 <KipIngram   > So that would look like
2023-05-09 08:55:28 <KipIngram   > <next code> <64 bit align> <table entry> <table entry> ...
2023-05-09 08:55:56 <KipIngram   > Then all the rest of the machine code would go off over in the machine code section.
2023-05-09 08:59:18 <veltas      > Not 100% sure what you mean by putting NEXT at bottom of table
2023-05-09 09:01:34 <KipIngram   > I'll have a region for a table of CFA addresses.  A register will point to it.  I'm just considering putting that one bit of code - the code for NEXT - right at the start of that region, and then what followed would be table entries.
2023-05-09 09:01:48 <KipIngram   > This is just to have an already-allocated register pointing to the next code.
2023-05-09 09:02:01 <KipIngram   > Instead of allocating a third register, above and beyond the two for the tables.
2023-05-09 09:02:25 <KipIngram   > In a perfect world I'd just have a third register point to the machine code region and put next there.
2023-05-09 09:02:34 <KipIngram   > Just trying to save a register.
2023-05-09 09:02:45 <KipIngram   > I use practically all of them for various things.  :-)
2023-05-09 09:03:22 <KipIngram   > Which both pleases me (I'm taking advantage of the resources) and irritates me (more to save on thread switches).
2023-05-09 09:04:28 <KipIngram   > Oh, by the way, speaking of thread switches... most of the stuff I've dredged up so far on memory allocation doesn't account for multi-threading.  Probably pretty important to get that right so that I avoid a bunch of locking and contention around memory resources.
2023-05-09 09:06:17 <KipIngram   > Anyway, if I put that bit of code at the base of the table, then I have one place where code is right by data.  My guess is that that line would just wind up in the instruction cache and the data cache, but it seems to me that ought to be ok since that data will really never get written and thus invalidated.
2023-05-09 09:06:36 <KipIngram   > Kicking next out of the cache would be the worst thing I could do.
2023-05-09 09:07:24 <KipIngram   > I see that that would be wasting most of one line of instruction cache, but it would be only one.
2023-05-09 09:07:27 <veltas      > How long is your next code anyway?
2023-05-09 09:07:38 <KipIngram   > Only 10-11 bytes.
2023-05-09 09:07:46 <veltas      > I'd just inline it
2023-05-09 09:07:52 <KipIngram   > Well, maybe a bit longer than that, but it's pretty short.
2023-05-09 09:07:52 <veltas      > That's not long
2023-05-09 09:08:03 <veltas      > Can you paste the code somewhere and link it?
2023-05-09 09:08:09 <KipIngram   > I think it's nine instructions.
2023-05-09 09:08:16 <KipIngram   > Yeah, one second.
2023-05-09 09:10:19 <KipIngram   > https://pastebin.com/rVWYz5gM
2023-05-09 09:10:35 <KipIngram   > That does next, docol, and the tick counting for thread switching.
2023-05-09 09:10:58 <KipIngram   > TICXT is the xt of the code that actually does the thread switching, that gets called now and again.
2023-05-09 09:11:37 <KipIngram   > So next  itself is only two instructions.
2023-05-09 09:11:44 <KipIngram   > But all of that is 9.
2023-05-09 09:12:43 <KipIngram   > For this to work, the high bits of rax must be kept zero.
2023-05-09 09:12:56 <KipIngram   > Because lodsw doesn't clear them when it picks up a 16-bit quantity.
2023-05-09 09:13:05 <KipIngram   > lodsd, 32 bits, does.
2023-05-09 09:13:21 <KipIngram   > So any primitive that uses rax is responsible for clearing it before next-ing.
2023-05-09 09:13:51 <KipIngram   > This actually doesn't decrement the tick counter every next - it decrements it every docol.
2023-05-09 09:14:08 <KipIngram   > But the code was just particularly clean this way; decided I liked it.
2023-05-09 09:15:12 <veltas      > What are the register assignments?
2023-05-09 09:15:36 <veltas      > I'm not a huge fan of renaming registers in assembly :(
2023-05-09 09:17:46 <KipIngram   > Oh, sorry.  Um, rrIP is rsi, so that lodsw will work.  rrW is rax.
2023-05-09 09:17:52 <KipIngram   > The others don't really matter.
2023-05-09 09:18:08 <KipIngram   > It's just that lodsw mechanism that drives those to "musts."
2023-05-09 09:20:35 <KipIngram   > UGH.
2023-05-09 09:20:41 <KipIngram   > Those *two* musts.  :-(
2023-05-09 09:21:16 <KipIngram   > I particularly liked the way it worked out for handling the tick call by just making an extra pass through docol.
2023-05-09 09:21:30 <KipIngram   > It just "slips in" a call to TICXT.
2023-05-09 09:22:10 <KipIngram   > So I can write that all in Forth, and just vector it in.
2023-05-09 09:22:42 <KipIngram   > And if I wanted an uninterrupted period I could just revector it to a noop.
2023-05-09 09:23:24 <KipIngram   > Well, I guess it would need to at least re-load the counter.
2023-05-09 09:36:41 <KipIngram   > But anyway, the real evil of mixing code and data would be writing to the data and evicting the code from the cache, wouldn't it?  And also the fact that the data would take up some space in an instruction cache line?
2023-05-09 09:38:22 <veltas      > Yeah it's bad for performance
2023-05-09 09:38:48 <KipIngram   > But if I can guarantee those data items will never be written to, I'd avoid that eviction, right?
2023-05-09 09:39:06 <KipIngram   > I'm willing to carry a part of one cache line not used for code.
2023-05-09 09:41:48 <veltas      > I'd expect the NEXT code to frequently be cached, it containing mostly data would probably be bad
2023-05-09 09:42:05 <veltas      > Probably hard to notice without profiling though
2023-05-09 09:42:56 <KipIngram   > Well, I always could align the table data up to the next cache line, I guess.  Then that part of the next cache line would just be wasted, but it's already wasted the other way too, at least insofar as the instruction cache goes.
2023-05-09 09:45:45 <vms14       > 0 1 1000000 range [ + ] do.list .   =>   1.47 real
2023-05-09 09:46:00 <vms14       > it improved a bit, it was 3 seconds last time
2023-05-09 09:46:17 <KipIngram   > Hey, that's more than "a bit."
2023-05-09 09:46:24 <KipIngram   > Cutting in half is good.
2023-05-09 09:46:32 <vms14       > it's because of the fake compile
2023-05-09 09:46:43 <vms14       > it mainly avoids the interpreter overhead
2023-05-09 09:46:57 <vms14       > the interpreter overhead is at compile time
2023-05-09 09:47:12 <vms14       > the recognition of every word, etc
2023-05-09 09:47:25 <KipIngram   > Sure.
2023-05-09 09:47:38 <vms14       > but still it's wrong
2023-05-09 09:47:57 <vms14       > and defeats the purpose of making the whole fake compiling
2023-05-09 09:48:06 <vms14       > I need to cache the compiled versions
2023-05-09 09:48:36 <vms14       > this will remove a flaw it has, which is that dynamic behavior of words, using always the last definition
2023-05-09 09:48:53 <KipIngram   > Caching is a funny word to use for it - to me that's just compiling to the dictionary.
2023-05-09 09:48:58 <vms14       > yeah xD
2023-05-09 09:49:08 <vms14       > caching for me is because they end in a hash table
2023-05-09 09:49:11 <unjust      > isn't the word for that memo'isation?
2023-05-09 09:50:06 <KipIngram   > Yeah, that's used to.  I guess we could think of a Forth definition as memoizing.
2023-05-09 09:50:28 <KipIngram   > too
2023-05-09 09:50:35 <unjust      > vms14: congrats on 2x speed up
2023-05-09 09:50:39 <KipIngram   > Having problems with "to's" this morning.  :-|
2023-05-09 09:51:04 <vms14       > unjust: the js version stills being much faster even without that fake compile
2023-05-09 09:51:05 <vms14       > xD
2023-05-09 09:51:12 <vms14       > js takes 0.5 seconds for that
2023-05-09 09:51:29 <vms14       > but I wasn't aiming at performance
2023-05-09 09:51:41 <vms14       > I need to get rid of a flaw, and compiling is one of the solutions
2023-05-09 09:51:46 <vms14       > "compiling"
2023-05-09 09:54:00 <veltas      > Yeah I agree with KipIngram, sounds like you're just reinventing compiling and dictionaries
2023-05-09 09:54:41 <vms14       > but it's not efficient anyways
2023-05-09 09:54:46 <vms14       > 0 1 1000000 range [ 1 2 3 drop drop drop + ] do.list .
2023-05-09 09:55:09 <vms14       > this takes 3.57 real
2023-05-09 09:55:48 <unjust      > running on what though?
2023-05-09 09:56:00 <vms14       > unjust: what?
2023-05-09 09:56:08 <vms14       > ah, on a gpd micropc
2023-05-09 09:56:15 <vms14       > want to benchmark?
2023-05-09 09:56:43 <vms14       > https://termbin.com/t7hj
2023-05-09 09:56:52 <vms14       > time perl this.file
2023-05-09 09:56:58 <KipIngram   > veltas: Regarding register aliasing, I totally hear what you're saying.  My reasoning, though, was that I was aiming at code that would eventually evolve into something portable.  What I didn't show you when I posted that was another column of "portable code" over on the right, that I had toyed with.
2023-05-09 09:57:06 <vms14       > it contains that  [ 1 2 3 drop drop drop + ] code
2023-05-09 09:57:17 <KipIngram   > And I certainly found it easier while writing primitives to not have to remember what register I'd picked for what purpose.
2023-05-09 09:57:42 <KipIngram   > The aliases I gave them were really easy for me to keep in mind.
2023-05-09 09:59:15 <vms14       > unjust: the system is netbsd, I don't know how much would differ if it was linux
2023-05-09 09:59:38 <vms14       > and perl is v5.36.0
2023-05-09 10:01:26 <vms14       > I expect most of the other computers have a better result
2023-05-09 10:01:41 <vms14       > this is a computer that fits on my pocket :D
2023-05-09 10:02:39 <veltas      > KipIngram: Yeah I just find it easier in my experience to bite the bullet and memorise the allocation
2023-05-09 10:02:41 <unjust      > real   0m53.781s
2023-05-09 10:02:51 <unjust      > Linux 4.9.226-perf on armv7l
2023-05-09 10:02:53 <KipIngram   > :-)
2023-05-09 10:03:05 <veltas      > Which was a bit annoying in ilo-amd64 because I changed my allocation half way through :)
2023-05-09 10:03:20 <unjust      > perl v5.36.0
2023-05-09 10:03:30 <veltas      > When I realised it was probably worth using rbx for the data stack register
2023-05-09 10:03:40 <veltas      > And r15 for memory pointer
2023-05-09 10:04:50 <unjust      > real   0m3.576s (on x86-64)
2023-05-09 10:08:07 <vms14       > xd
2023-05-09 10:08:25 <unjust      > most likely that build of perl isn't as optimised as well as it could be for armv7l, but not sure it would get near 4s even if it were
2023-05-09 10:09:08 <vms14       > it's a fake compilation, the best way to get performance on a perl implementation would be to write perl code
2023-05-09 10:09:16 <vms14       > I'm also interested on that
2023-05-09 10:09:24 <veltas      > What sort of armv7l machine is it?
2023-05-09 10:09:36 <vms14       > I tried to do it and it kind of worked xD
2023-05-09 10:12:56 <unjust      > veltas: a phone with an armv7l targetted distribution of android, but running on qualcomm snapdragon 215 (armv8 native)
2023-05-09 10:14:20 <veltas      > vms14: Is your abomination code online somewhere?
2023-05-09 10:14:41 <unjust      > it's in the last link he posted
2023-05-09 10:14:43 <vms14       > veltas: only in form of a paste
2023-05-09 10:15:06 <vms14       > https://termbin.com/6c07 now it's almost the same
2023-05-09 10:15:20 <vms14       > but it's "caching" the compiled version of a word
2023-05-09 10:15:32 <vms14       > still it's the same performance for that code
2023-05-09 10:16:06 <veltas      > Have you rewritten it in C?
2023-05-09 10:16:09 <vms14       > no
2023-05-09 10:16:17 <vms14       > I was going to, but nope
2023-05-09 10:16:40 <vms14       > I realized I have no reason to use any other language
2023-05-09 10:16:52 <vms14       > except maybe js for getting it into the browser
2023-05-09 10:17:07 <vms14       > I don't want performance, so why I would want C
2023-05-09 10:17:14 <vms14       > I only want Xlib from C
2023-05-09 10:17:23 <vms14       > I can get it anyways
2023-05-09 10:18:18 <veltas      > Other advantages of C or asm would be to reduce dependencies, produce smaller overall stack
2023-05-09 10:18:32 <veltas      > Makes it easier to fix things and maintain it properly
2023-05-09 10:18:37 <vms14       > the only dependency is perl
2023-05-09 10:18:43 <vms14       > and it makes it quite portable
2023-05-09 10:18:57 <vms14       > even if I want to use external libraries
2023-05-09 10:19:29 <vms14       > It's a toy language after all :D
2023-05-09 10:19:32 <veltas      > Yeah
2023-05-09 10:19:48 <vms14       > it has environments also, which mean closures and alike
2023-05-09 10:19:48 <veltas      > There's nothing wrong with that, it's your own recreational thing
2023-05-09 10:20:11 <vms14       > you can put words on the environment
2023-05-09 10:20:19 <vms14       > and every word has its own environment
2023-05-09 10:20:35 <vms14       > they inherit from other words
2023-05-09 10:20:57 <vms14       > C only gets on the way to implement those features
2023-05-09 10:21:11 <vms14       > and gives me performance in return, which I don't want
2023-05-09 10:21:12 <vms14       > :D
2023-05-09 10:22:07 <veltas      > I wonder how well it runs on my laptop
2023-05-09 10:22:24 <veltas      > It's an Intel Core 2 Duo
2023-05-09 10:22:28 <KipIngram   > So you're basically treating each word like its own process?
2023-05-09 10:22:46 <vms14       > veltas: you have the benchmark already on that file
2023-05-09 10:22:55 <vms14       > you just need to "time perl this.file"
2023-05-09 10:23:25 <vms14       > KipIngram: no, I have a variable named "environment" on perl
2023-05-09 10:23:53 <KipIngram   > Ah, I see.  Different environment.
2023-05-09 10:24:03 <vms14       > when I execute a word, the environment is now a new environment that inherits from the old environment and the internal environment of a word
2023-05-09 10:24:11 <vms14       > every word has it's own internal environment
2023-05-09 10:24:23 <vms14       > when I exit the world I restore the environment to the old one
2023-05-09 10:24:53 <veltas      > vms14: I'll have to try when I'm home, at work
2023-05-09 10:25:04 <vms14       > this allows me to create closures and accumulators
2023-05-09 10:25:34 <vms14       > veltas: I started from scratch like this morning, so it's quite incomplete
2023-05-09 10:25:55 <vms14       > and likely to have bugs
2023-05-09 10:27:14 <vms14       > for example the global environment exists, but not the word environment yet
2023-05-09 10:27:31 <vms14       > it's likely I'll rewrite it again xD
2023-05-09 11:14:15 <KipIngram   > Rewriting - story of my Forth life.  :-)
2023-05-09 11:14:46 <KipIngram   > That's part of why I'm pushing some here to set up to do it using Forth this time - at least that will be "something different" instead of hacking down the same jungle path yet again.
2023-05-09 11:26:57 <vms14       > I'm trying to patch what I wrote
2023-05-09 11:27:01 <vms14       > but it's a mess
2023-05-09 11:27:36 <vms14       > for some reason the ':' word is executing twice
2023-05-09 11:27:42 <vms14       > :D
2023-05-09 11:28:37 <vms14       > I'll try again making the interpreter first
2023-05-09 11:28:40 <vms14       > then compiling
2023-05-09 11:28:58 <vms14       > or idk
2023-05-09 11:29:09 <vms14       > but this is broken as fuck
2023-05-09 11:30:19 <vms14       > rewriting in my case is needed cause I'm constantly "researching", trying stuff and new ideas, etc
2023-05-09 11:30:49 <vms14       > the language is usually a bunch of tests that ended working together, so it needs to be written properly
2023-05-09 11:31:07 <vms14       > but then new ideas or changes come and the cycle repeats
2023-05-09 11:31:38 <vms14       > I'll take a break for a while, sometimes is good
2023-05-09 11:31:42 <vms14       > see you guys
2023-05-09 11:44:58 <veltas      > KipIngram: Rewriting is often productive
2023-05-09 11:45:13 <KipIngram   > Yeah, I usually feel like I take steps forward.
2023-05-09 11:45:15 <veltas      > The second, third, fourth time round you understand the problem better and how to write it better
2023-05-09 11:45:46 <thrig       > I could see some bias for the fourth time here
2023-05-09 11:45:53 <veltas      > And it's fine if you're rewriting something that's practical to rewrite. Rewriting a massive app might not be, but a Forth env you absolutely can
2023-05-09 11:46:09 <veltas      > thrig: ;)