2022-03-05 11:19:11 I'm trying to use ffi from gforth 2022-03-05 11:19:42 I try to s" -R /usr/pkg/lib/" add-cflags and s" -L /usr/pkg/lib/" add-cflags 2022-03-05 11:19:50 but gforth says this word does not exist 2022-03-05 11:20:23 and it should exist https://gforth.org/manual/Declaring-OS_002dlevel-libraries.html#index-add_002dcflags 2022-03-05 11:20:48 why the heck the word does not exist 2022-03-05 11:21:05 and how I can add some flags to the ffi compiling phase 2022-03-05 13:51:34 General question for you guys. In traditional Forth, when we BLOCK data from storage into a buffer, we get an address we can use to get at the data. The general "rule" is that it's our responsibility to know when we need to use BLOCK on that same data again - that is, we just have to know when something we do might invalidate that address value. This is well and good in a single-process system; we just 2022-03-05 13:51:37 know that as long as we haven't used BLOCK or LOAD again since then, the address is still valid. What sort of arrangements have you guys seen made in more complicated systems, where there might be multiple tasks and processes and so on? I can think of a number of ways to proceed - just wondering if there's already some "best practice" in use out there. 2022-03-05 13:52:55 One path I see would involve enforcing some "atomicity" - BLOCK would lock the contents of that buffer, and you'd have to explicitly unlock it to make it available for re-use. That would require some syntax for doing that be invented. 2022-03-05 13:53:18 Another path would be to design it so one processes actions couldn't invalidate a buffer held by another process. 2022-03-05 13:53:50 Then the usual rule would continue to apply, but buffer management would become less flexible. 2022-03-05 13:56:03 Also, separating the processes too much could cause issues too - what if I *want* to processes to both be working on the same block? 2022-03-05 13:56:19 And I know I've written them such that that will work out? 2022-03-05 13:57:50 Generally speaking, most Forth materials I've ever read seem to have the orientation that your "one stream of Forth" is the only thing happening on the system. 2022-03-05 13:58:48 But if my goal was a Forth-implemented OS that I could use much the way I use Linux (and that is my long-term goal), that's just an unacceptable way to look at things. 2022-03-05 14:01:03 I'm also reaching the limits of my Linux knowledge here - Linux processes really don't "explicitly share RAM," do they? If they are in communication with one another that's generally done through something like mmap, isn't it? So it's like each one maps some file or portion of file onto their own address space? 2022-03-05 14:01:55 And then there's "processes" vs. "tasks/threads" - different rules for those isn't there? 2022-03-05 14:02:22 I'd assume that two threads belonging to the same process both shared an address space, so to speak. 2022-03-05 14:02:43 And so could just directly use global variables any way they liked? 2022-03-05 14:32:37 KipIngram: check out the ostep book. You might enjoy it. 2022-03-05 14:46:19 also, the polyforth manual and the nemo book are good reads too. 2022-03-05 14:49:25 joe9: Thanks! You're referring to this: 2022-03-05 14:49:27 https://pages.cs.wisc.edu/~remzi/OSTEP/ 2022-03-05 14:49:30 ? 2022-03-05 14:49:47 yes. 2022-03-05 14:50:41 all the memory questions above seem to be due to paging and it's effects. inferno takes the route of no paging (all linear memory). 2022-03-05 14:56:24 I just realized that I am likely confusing myself by not distinguishing properly between "processes" and "sessions." When I say "session" I'm referring to me sitting there in front of a GNU screen window, for example. And I understand that is a process, but I think it's better for this reasoning for me to use the word "process" to refer to the commands I run in that screen session. 2022-03-05 14:57:02 I think if one of those command processes allocates dynamic RAM or something, that RAM just needs to stay put until the process releases it. So those addresses would be perpetually valid, unless the process got rid of them. 2022-03-05 14:57:38 One of the things I ultimately want is to be able to "archive" a session to disk. Put that work away, and come back to it later, and have it so when I re-load it I can pick up right where I left off. 2022-03-05 14:57:56 But I think a rule I want is that a session that has running processes can't be archived. 2022-03-05 14:58:06 Running child processes. 2022-03-05 14:58:22 I'm probably using horrible terminology here - I'm not really a "practitioner in this field." 2022-03-05 14:59:06 But distinguishing between those things lets me draw a line of sorts for RAM management purposes. 2022-03-05 15:00:38 Or maybe a better way to say it is that a process that has dynamically allocated memory active can't be archived - wouldn't matter whether it allocated itself or had children that had allocated it. 2022-03-05 15:02:28 When I wrote my most recent stuff, I was working on MacOS, and it required me to jump some relocatability hoops. Specifically, where a typical Forth would be working with addresses, I'm often working with offsets. I'd thought that now that I'm back to Linux, I might set that aside and do things in a more customary way - it would be a bit more efficient. But then I realized that having everything as "offset 2022-03-05 15:02:30 based" as I do actually has some advantages - it makes it possible to move an image around in RAM, and as long as I properly update my base addresses (which I keep in registers), it would still work. 2022-03-05 15:02:44 That allows the possibility for archiving - when I reload the image later it will likely be at a different address. 2022-03-05 15:04:10 But in spite of it being written that way, I do wind up with hard addresses on the stack at times. Like when I execute a variable name, I get its *address* on the stack, just like always. The dictionary references to that variable are offsets, and the "dovar" routine adds in the proper base value from a register, but THEN it's an address. 2022-03-05 15:04:33 So if I archived a session while it had addresses on its stack, that would be problematic. I do de-reference those offsets from time to time. 2022-03-05 15:05:14 So I really only have like 99% of an ability to do this "archiving." And that 1% might turn out to be very troublesome. 2022-03-05 15:06:10 Any linked list structures I built - if they use actual addresses for links, then they become immobile. 2022-03-05 15:07:05 All of these things are things that, were I really developing a full-on OS, I'd be able to use CPU memory management hardware to help me. But so long as I'm running "under" some other system I don't have access to those methods. 2022-03-05 15:07:45 So it may just be silly to try to solve these problems until such time as I am "really" writing an OS, and have access to all of my resources. 2022-03-05 15:09:07 All of the data in my return stack are full-fledged addresses too. 2022-03-05 15:09:26 Well, aside from things I put there with >r. 2022-03-05 15:10:09 you seem to be talking about swapping(?) 2022-03-05 15:11:27 Yeah, it's at least in that neighborhood. That's what I meant by having CPU assistance in the long run. One problem here is that I'm sort of murkily stubmling around a handful of different arenas. 2022-03-05 15:11:33 Trying to see if something clever emerges. 2022-03-05 15:12:17 I was pretty happy with how I wound up satisfying MacOS - I was able to confine those offsets to a fairly low number of places, so the overall impact on efficiency isn't that bad. 2022-03-05 15:13:01 NEXT has a register add instruction in it that wouldn't be there without that requirement, and there are a few other places where I have to remember to add in or subtract off the offset. But it's fairly limited. 2022-03-05 15:13:39 dovar, docol - that sort of place. Each of them has like one extra operation beyond what it would in a straight address-based approach. 2022-03-05 15:13:54 And since the necessary adjustment is in a register, that price isn't very high. 2022-03-05 15:15:33 The archiving thing may just not be feasible. If it turned out that the return stack was the only place I had addresses, and if I knew that it held ONLY addresses, I could adjust them out when I archived and put them back when I re-loaded. The return stack is a well-defined thing. 2022-03-05 15:15:48 But there would be a lot of "gotchas" to watch out for. 2022-03-05 15:17:50 in unix/linux or paging-using systems, a process has the whole memory to itself. 2022-03-05 15:18:07 Right - a whole virtual address space. 2022-03-05 15:18:30 yes. so, the addresses are always the same. 2022-03-05 15:18:48 but, it is a problem with forth where the addresses are absolute. 2022-03-05 15:22:31 I am not sure how polyforth manages swapping. 2022-03-05 17:03:34 Well, I can only think of three ways to cope with it - 1) have cpu hardware machinery for it, 2) correct it whenever you establish the image at a new location, or 3) deal with it at time of use. 2022-03-05 17:05:19 The MacOS "macho" executable format won't *allow* any absolute addresses in the on-disk image. So anything that I needed to be part of the binary had to be relative. Once Forth was *running*, it's able to create absolute addresses and use them just fine - so long as I'm running from that given image location. 2022-03-05 17:06:22 I was able to identify one way to get the actual address of an item at run-time that the assembler and linker would accept. It used the lea (load effective address) instruction. 2022-03-05 17:08:04 So I lea'd the addresses of labels at the origin of each of my sections, and slapped those values into a couple of registers. Everywhere else my assembly basically used quantities like "label - base_label" to get an offset, and when I picked those up for use I'd add in the appropriate base register to get a new "real address." 2022-03-05 17:08:39 Fortunately I was able to use macros to hide a lot of that "label-base_label" stuff away, so the source isn't ugly. 2022-03-05 17:10:27 As an extra perk I put my NEXT code at the very beginning of the code section, so the register that holds that base address points right at it. Then primitives can just end with "jmp " to send them to NEXT. 2022-03-05 17:10:58 And yes, it would be a hair faster if I just copied NEXT to each place, but there are some benefits to having a spot you know execution will pass frequently through. 2022-03-05 17:11:15 Makes it easier to set up things like profiling, pre-emptive task suspension, and so on. 2022-03-05 17:11:54 I can just replace the first few bytes of NEXT with a new jump instruction, and I can intercept execution at the end of the next primitive. 2022-03-05 17:16:00 And if I ever do write any code that just absolutely positively has to be as fast as possible (which I wouldn't know without being able to profile), I can always optimize those little bits. 2022-03-05 17:31:03 I wrote a short gemlog about the Raspberri Pi Pico and Mecrisp Forth: 2022-03-05 17:31:07 https://portal.mozz.us/gemini/gem.librehacker.com/gemlog/tech/20220305-0.gmi?inline=1 2022-03-05 17:31:55 It has been an enjoyable experience so far, as an introduction to ARM programming 2022-03-05 17:38:34 Oh, neat - I should check that out. I'm sure I'll get around to wanting to run something on ARM at some point. 2022-03-05 17:47:17 KipIngram: I like your idea of offsets. I think that could work. though could slow things down a little. 2022-03-05 18:05:15 http://www.greenarraychips.com/home/documents/greg/DB005-120825-PF-REF.pdf 2022-03-05 18:06:12 Well, I felt some angst over it, becuase of course it's a small performance penalty. But it still wound up pretty fast. Keeping the base addresses in a pair of registers was worth it, I think. 2022-03-05 18:06:55 I use two RAM regions - basically one for headers and the other for bodies. My headers aren't interspersed with the definitions; I did that so I could have definitions "fall through" into one another if I wanted to: 2022-03-05 18:07:13 : foo ...code... : bar ...code... ; 2022-03-05 18:08:12 Tail recursion words (some of which are conditional) jump to the most recently defined name. 2022-03-05 18:09:21 Of course I could tail recurse by just using the name bar and tail optimizing it, but I have a special symbol for it ("me") and I can have versions of that are conditional, like 0=me for example. 2022-03-05 18:10:04 That's my only looping construct - I don't have the usual standard looping words. 2022-03-05 18:10:26 I don't have IF/THEN/ELSE, either - I factor and use conditional returns. 2022-03-05 18:11:14 Instead of 0= IF ... THEN I would have : word 0<>; ... ; 2022-03-05 18:12:53 My definitions became shorter when I started coding like that. 2022-03-05 19:02:50 thinking forth talks recommends this style too 2022-03-05 19:43:16 Oh, are there conditional returns in TF? 2022-03-05 19:43:33 Or do you just mean the additional factoring? 2022-03-05 19:56:55 lispmacs[work]: nice, short article you wrote there :) 2022-03-05 19:57:14 I should get myself a Pico sometime.. 2022-03-05 19:58:56 I've got a little Cortex M4 dongle I keep meaning to try to make a port to. It came out of the box with a Python setup, but I think when I looked I saw how to replace it all the way down. 2022-03-05 19:59:42 It kind of astounded me; it's about a half inch wide and two inches long, and I remember noticing how thoroughly it would kick the first IBM PC I ever owned in the pants. 2022-03-05 20:00:31 Comparable RAM, and 32 bits instead of 8 and 20x the clock speed. 2022-03-05 21:46:21 Oh, regarding that ostep book - this was a nice find: 2022-03-05 21:46:24 https://gist.github.com/adyavanapalli/55dffc51cfef38eac2d6b71b0712502a 2022-03-05 21:46:38 It's a script that will download the whole thing and assemble it into a finished pdf. 2022-03-05 21:46:54 I ran it - it works. 2022-03-05 21:47:06 Poof... instant 700+ page book. 2022-03-05 21:52:50 Check out footnote 7 at the bottom of page 9. :-) :-) 2022-03-05 21:53:51 I used emacs, once long ago. Liked it quite a lot. Eventually I gave in and shifted to vim because it just felt to me like vim was the focus of more active development. 2022-03-05 21:54:25 More tools and plugins and so on. Clearly the ostep author... disagrees with my decision there. 2022-03-05 21:54:41 And doesn't mind being outspoken about it. 2022-03-05 21:55:35 It was back then I started swapping Ctrl and CapsLock on every PC I had to use. I still do that, even though it's not AS important as it was when I was trying to fly emacs. 2022-03-05 23:34:11 this channel has no forth bot? 2022-03-05 23:34:25 a bot that can evaluate forth code