2022-03-23 04:53:24 KipIngram: Just use the info for C programs and apply it to your syscalls in assembly 2022-03-23 04:53:37 What's the current problem? (in 1 line please :P) 2022-03-23 05:40:08 I expect the current problem is just that I haven't digested the material well enough. I really only started looking down this hole in earnest yesterday. I just found the C-centrism kind of annoying. 2022-03-23 05:41:21 Also, there was a whiff of "standard-itis" in the whole thing too - like doing something with your C compiler that's not guaranteed to work on every C compiler is some kind of a criminal action or something. 2022-03-23 05:43:44 Someone asks a question, and instead of answering the question a lot of people just talk about why you shouldn't be trying to do that. 2022-03-23 05:45:37 you're talking about #c channel? 2022-03-23 06:06:13 No, mostly just about Stack Overflow conversations. It wouldn't have occurred to me to go to #c with it, since I don't regard it as a c-related question. To me it's a linux question. 2022-03-23 06:07:06 I did a fair degree of research figuring out how to do this system without linking in any C baggage at all. 2022-03-23 06:07:29 I wanted a "syscalls only" system, and wanted to minimize that. 2022-03-23 06:08:13 I do think I'm going to add mprotect to the error recovery image snapshot, though. It doesn't really matter how fast I do that - it's all fast compared to me typing, so it'll be fine. 2022-03-23 06:08:44 i went a different way and have some c functions for i/o and (you could call it a foreign function interface) from forth to c 2022-03-23 06:08:59 Sure - that's a worthy goal. 2022-03-23 06:10:23 By the way, last night I mentioned being able to re-vector NEXT by just changing a register. I realized later that I can't do that where I am today - the register that addresses NEXT is also the register that contains my "system base" address that gets added to all the offsets when necessary. I got double-duty from it by placing NEXT at the base of the system. So I can't change that register. 2022-03-23 06:10:45 KipIngram: whoops! 2022-03-23 06:10:51 However, being able to seems so desirable to me in terms of adding advanced features that I may invest in another register and split those functions. 2022-03-23 06:11:08 KipIngram: Have you tried using the syscalls directly then? 2022-03-23 06:11:24 Yeah, that's what I do do. 2022-03-23 06:11:39 I just need to do more digging on the signal related syscalls. 2022-03-23 06:11:42 i remember KipIngram having problems with syscalls in macos because of a lack of documentation 2022-03-23 06:11:52 linux is better documented 2022-03-23 06:12:03 Yeah, Applel really doesn't want you to do that, apparently. 2022-03-23 06:12:27 There's some library they want to impose on you. 2022-03-23 06:13:12 Finding all the right "magic numbers" for the termios adjustments on the Mac - that was quite a research project. 2022-03-23 06:14:09 I didn't even think about changing NEXT via a register change until last night - previously I'd assumed I'd just slap a jump instruction at the start of it to re-vector it. 2022-03-23 06:14:17 Downside of that is that it changes it for everything. 2022-03-23 06:14:30 Whereas a register change can be "per task." 2022-03-23 06:14:53 do you have something like jmp r10 to do NEXT ? 2022-03-23 06:15:15 Yeah. I use r15, actually, but that's arbitrary. 2022-03-23 06:15:32 but how large is your NEXT ? 2022-03-23 06:15:43 shouldn't be more than 2 or 3 instructions? 2022-03-23 06:16:03 next_c: geti rrW, rrIP ; Fetch next icell 2022-03-23 06:16:06 radd rrW, rrHB ; Offset -> address 2022-03-23 06:16:08 inci rrIP ; Bump instruction pointer 2022-03-23 06:16:10 geti rrTMP, rrW ; Fetch CFA offset 2022-03-23 06:16:12 radd rrTMP, rrBB ; Offset -> address 2022-03-23 06:16:14 rjmp rrTMP ; Go there 2022-03-23 06:16:16 Some of those are macros, but they're just one-line macros. 2022-03-23 06:16:24 Sugar to make the Forth vm more explicit. 2022-03-23 06:16:50 it is indirect threaded? 2022-03-23 06:16:54 Yes. 2022-03-23 06:17:29 geti rrW rrIP --> mov r8, dword [r15], etc. 2022-03-23 06:17:50 did you write your own assembler?? 2022-03-23 06:18:20 Well, I use nasm for the main work, but yeah, I had a couple of blocks of code that would encode quite a lot of x86_64 instructions. 2022-03-23 06:18:44 i'm gonna try something 2022-03-23 06:18:53 Tackling the entire x86_64 instruction set would be a nightmare, but some of the simple stuff, like moves, arithmetic, and so on actually has a sensible encoding that you can ferret out with not too much effort. 2022-03-23 06:19:45 So when the dust all settles I expect to have a simple "assembler" that handles the portion of the instruction set I need to build the system. 2022-03-23 06:21:21 Though for general porting purposes I also want a mechanism that lets me just specify a set of bytes to be emitted. Like a "raw binary" macro. 2022-03-23 06:21:44 And the lowest layer of a port would be one of those for each of those "portable instructions." 2022-03-23 06:21:59 Then the primitives would be written using those. 2022-03-23 06:22:14 Hopefully portably - that's the idea at least. 2022-03-23 06:23:43 i just checked and JMP can't use the fancy address modes without fetching a word from memory .. i thought i might be able to do JMP rax+rbx but no 2022-03-23 06:24:49 KipIngram: you can combine radd rrW,rrHB and geti rrTMP,rrW 2022-03-23 06:25:37 KipIngram: (intel mnemonics, but your register names) mov rrTMP,[rrHB+rrW] 2022-03-23 06:26:19 i thought i might have been able to use it in JMP too but no 2022-03-23 06:28:42 Well, I want the rrW reg to have that full value in it. It's used sometimes. But you may be right - maybe it's better to just remember to do that adjustment in both places. 2022-03-23 06:29:01 oooh you use rrW later? 2022-03-23 06:29:05 didn't think of that 2022-03-23 06:29:18 docol uses it, but I could adjust there too. 2022-03-23 06:29:35 But I'd have to time it to see which worked out faster. 2022-03-23 06:29:53 I haven't fretted overly about the speed - it seems plenty fast to me. 2022-03-23 06:30:10 But it's an optimizaiton that would be worth looking into, I guess. 2022-03-23 06:30:41 What I wind up with there is rrW pointing to the CFA field of my word, and the PFA field that docol needs is immediately following that. 2022-03-23 06:31:02 That's not where the definition is - that PFA field points to the definition. 2022-03-23 06:31:43 i was only thinking to minimize NEXT using the x86 address modes 2022-03-23 06:31:45 Those two pointers (CFA, PFA) are right by the name string in the header section. 2022-03-23 06:31:52 Yeah - good observation. 2022-03-23 06:32:44 It would speed up primitives a little, I imagine. 2022-03-23 06:33:25 i was thinking more of inlining NEXT 2022-03-23 06:33:54 that depends on a lot of things 2022-03-23 06:34:04 Right - jumping to it is an obvious performance hit. I just decided that having a place in the code where everything passed through was useful. 2022-03-23 06:34:17 Ultimately it's how I'll implement profiling and maybe some other features. 2022-03-23 06:35:09 Also it might be useful to give a specific task a next with some extra hooks in it that gave me better control over it. 2022-03-23 06:35:40 It would make it easy to put tasks to sleep and that kind of thing. 2022-03-23 06:35:42 basically what you do for NEXT (jmp r15) i do for DOCOL (jmp rbp) 2022-03-23 06:36:42 KipIngram: i read a forth story on low latency interrupts by modifying NEXT ! 2022-03-23 06:37:08 i wonder if i can find that again.. hang on i'll google 2022-03-23 06:38:01 this might be it.. http://6502.org/tutorials/zero_overhead_forth_interrupts.html 2022-03-23 06:38:11 Ah, fun. 2022-03-23 06:41:28 i think that's what i remember.. it's a long read.. not super important 2022-03-23 06:43:16 basically he polled for interrupts in NEXT which slowed it down, but it also reduced overhead of servicing interrupts, which sped it up if there was many interrupts :-) 2022-03-23 06:43:39 a side effect was the primitives became atomic 2022-03-23 06:43:48 which is interesting 2022-03-23 06:44:57 my slowest primitive is ROLL 2022-03-23 06:45:10 oh maybe multiply and divide is slower heh 2022-03-23 06:45:50 but ROLL has a loop where the number of iterations is passed to it.. so the actual time it takes depends on the argument, which is kinda bad 2022-03-23 06:46:10 :swap 1 roll ; : rot 2 roll ; 2022-03-23 06:46:35 whereas while multiply and divide takes a long time, the actual time is fixed 2022-03-23 06:47:29 Yeah, I'll be regarding primitives as atomic. 2022-03-23 06:48:05 Which will be useful for doing locks and so on. 2022-03-23 06:49:00 i <3 atomic instructions 2022-03-23 06:49:16 i found 2 more primitives that use loops... ERASE and MOVE 2022-03-23 06:53:03 i have them because x86 has hardware acceleration for that... rep stos and rep movs 2022-03-23 06:53:46 that's a nice feature of forth, being able to take advantage of the hardware in a way that's difficult in at least c 2022-03-23 06:53:57 you never know what you're gonna get with c 2022-03-23 06:54:29 Right. 2022-03-23 06:54:50 I don't really think of rep'd operations as "loops" per se, but I suppose in a way they are. 2022-03-23 06:55:54 I've only implemented cmove> and i followed the 2012 standard 2022-03-23 06:57:28 So, I may be oversimplifying, but I'm wondering if this may turn out to be as simple as writing a bit of code that throws the right error and using the sigaction syscall to specify it as the handler for SYSSEGV. 2022-03-23 06:58:15 When I get around to child tasks that might need to revision, but that may be all it takes for the main process. 2022-03-23 06:58:58 Ugh. I should make coffee - I hate it when I write sentences that are just baldly wrong because I'm mentally writing two different sentences. 2022-03-23 06:59:20 i have a package that might be helpful 2022-03-23 06:59:22 wait 2022-03-23 07:00:19 http://ftp.gnu.org/gnu/libsigsegv/libsigsegv-2.14.tar.gz 2022-03-23 07:00:25 The way I have this set up, WARM resets the data stack and then calls the snapshot routine. Immediately after that is an entry point that ERR uses - it "unsnaps" (so WARM basically snaps then immediately unsnaps) and runs QUIT. 2022-03-23 07:00:44 WARM is a good name 2022-03-23 07:01:01 So ERR prints the appropriate message and then restarts the snapped state. 2022-03-23 07:01:08 WARM is a FIG Forth legacy. 2022-03-23 07:01:14 COLD, WARM, QUIT. 2022-03-23 07:01:20 ooh 2022-03-23 07:01:23 nice 2022-03-23 07:01:49 The way I've split that up is COLD initializes everything - I'm back to the initial load situation - WARM resets both stacks, QUIT resets just the return stack. 2022-03-23 07:02:27 i have to refactor my code to properly handle signals.. last week i rewrote the forth->c interface 2022-03-23 07:03:12 and i want ctrl-c to forcefully stop and drop me into QUIT 2022-03-23 07:03:52 it doesn't do that, ctrl-c kills everything back to the unix shell 2022-03-23 07:04:09 i have to completely rewrite signal handling 2022-03-23 07:06:55 I probably would also want to have that bit of handler code set the processor stack pointer back to it's normal value. I don't use it, but the system code probably would have stacked a bunch of stuff up by the time my handler gets called. 2022-03-23 07:07:28 ctrl-c returns key code 3 for me. 2022-03-23 07:07:49 But I think in the terminator terminal emulation program I use it *also* gets caught by the terminal and does "copy" 2022-03-23 07:08:02 ctrl-v is getting caught as well, and doesn't seem to give me a code. 2022-03-23 07:08:20 They both work just fine in kitty, though - it uses ctrl-shift-c and -v for copy and paste. 2022-03-23 07:08:32 cute name :-) 2022-03-23 07:08:43 I think I could tell terminator to do that too, but I'm awfully used to ctrl-c and -v for copy/paste. 2022-03-23 07:08:51 i want ctrl-c to actually interrupt the code, for example forth goes into an endless loop 2022-03-23 07:09:05 yeah linux is pretty crap like that 2022-03-23 07:09:09 Part of my termios changes were to get all of the ctrl-letter keys to behave and just be "keystrokes." 2022-03-23 07:09:47 Once I get signal handling working, I might reconsider ctrl-c; it might be nice to be able to interrupt a running word. 2022-03-23 07:10:07 that is what emacs does... which is fine, i even use an emacs clone called jove for all my editing 2022-03-23 07:10:21 but it's a lousy user interface 2022-03-23 07:10:26 I used to use emacs, but some years ago switched over to vim. 2022-03-23 07:10:53 People fight about them (religious wars are the worst), but I figure they're both damn good editors. 2022-03-23 07:11:34 i only know how to quit without saving changes from vi ;-) 2022-03-23 07:11:39 :q! 2022-03-23 07:11:50 then i run emacs ;-) 2022-03-23 07:13:03 but currently there's no way to stop an endless loop in my forth without killing the whole process 2022-03-23 07:13:29 also i haven't written any code to do a stack trace 2022-03-23 07:13:33 Same for me. 2022-03-23 07:13:52 I've got some extensive debugger plans. Someday. 2022-03-23 07:14:24 Maybe soon. Don't know exactly what order I'll do things in. 2022-03-23 07:14:24 i couple of weeks ago something in my brain flipped, and i thought it's time to at least try and make my program user friendly 2022-03-23 07:14:34 :-) 2022-03-23 07:14:44 same 2022-03-23 07:14:47 so many things to do 2022-03-23 07:15:24 brb 2022-03-23 07:21:18 Man, it looks like signal handlers CAN get really, really involved. 2022-03-23 07:21:51 I don't THINK I'm interested in any info the system passes to me in the handler, but I guess time will tell. 2022-03-23 07:22:12 Basically I just want to abandon whatever was happening. 2022-03-23 07:23:28 I think I'll make the handler a primitive so I can test it out just by running it. Honestly it's just a machine code entry point for ERR that supplies its own error number. 2022-03-23 07:29:34 Ok, that was simple. 2022-03-23 07:29:46 Now to call sigaction during init. 2022-03-23 07:37:46 someone in #c asked about signals just then :-p 2022-03-23 07:37:55 they really are messy 2022-03-23 07:48:18 Wow - the sigaction structure size is 152 bytes. :-| 2022-03-23 08:00:12 I don't get that. The man pages defines the sigaction structure like so: 2022-03-23 08:00:16 struct sigaction { 2022-03-23 08:00:18 void (*sa_handler)(int); 2022-03-23 08:00:20 void (*sa_sigaction)(int, siginfo_t *, void *); 2022-03-23 08:00:22 sigset_t sa_mask; 2022-03-23 08:00:24 int sa_flags; 2022-03-23 08:00:26 void (*sa_restorer)(void); 2022-03-23 08:00:28 }; 2022-03-23 08:00:30 I don't see how that leads to 152 bytes. 2022-03-23 08:00:40 But printf-ing sizeof(struct sigaction) says 152. 2022-03-23 08:01:31 I set it up and made the syscall with a 152-byte buffer of zeros (so NULL handler) - it didn't like something about that; returned EINVAL error. 2022-03-23 08:04:41 This is the definition of the structure from /usr/include/bits/sigaction.h: 2022-03-23 08:04:45 struct sigaction 2022-03-23 08:04:48 { 2022-03-23 08:04:50 /* Signal handler. */ 2022-03-23 08:04:52 #if defined __USE_POSIX199309 || defined __USE_XOPEN_EXTENDED 2022-03-23 08:04:54 union 2022-03-23 08:04:56 { 2022-03-23 08:04:58 /* Used if SA_SIGINFO is not set. */ 2022-03-23 08:05:00 __sighandler_t sa_handler; 2022-03-23 08:05:02 /* Used if SA_SIGINFO is set. */ 2022-03-23 08:05:04 void (*sa_sigaction) (int, siginfo_t *, void *); 2022-03-23 08:05:06 } 2022-03-23 08:05:08 __sigaction_handler; 2022-03-23 08:05:10 # define sa_handler __sigaction_handler.sa_handler 2022-03-23 08:05:12 # define sa_sigaction __sigaction_handler.sa_sigaction 2022-03-23 08:05:14 #else 2022-03-23 08:05:16 __sighandler_t sa_handler; 2022-03-23 08:05:18 #endif 2022-03-23 08:05:20 /* Additional set of signals to be blocked. */ 2022-03-23 08:05:22 __sigset_t sa_mask; 2022-03-23 08:05:24 /* Special flags. */ 2022-03-23 08:05:26 int sa_flags; 2022-03-23 08:05:28 /* Restore handler. */ 2022-03-23 08:05:30 void (*sa_restorer) (void); 2022-03-23 08:05:32 }; 2022-03-23 08:05:34 That does not look like 152 bytes to me. 2022-03-23 08:07:55 ah 2022-03-23 08:07:59 check __sigset_t 2022-03-23 08:08:06 that might be an array 2022-03-23 08:08:38 __sigset_t sa_mask; i suspect this is some kind of array of bits 2022-03-23 08:09:06 Oh, good thought. 2022-03-23 08:09:15 I just saw "mask" and figured it was... well, a mask. 2022-03-23 08:11:09 Now I get to find where those are defined. :-) Maybe signal.h. 2022-03-23 08:11:41 Ah. bits/types/sigset_t.h 2022-03-23 08:12:48 Ohhh... dave0, nice job: 2022-03-23 08:12:51 typedef struct 2022-03-23 08:12:53 { 2022-03-23 08:12:55 unsigned long int __val[_SIGSET_NWORDS]; 2022-03-23 08:12:57 } __sigset_t; 2022-03-23 08:13:07 God, what a mess. :-) 2022-03-23 08:13:51 I'm half tempted to just build that in yet-to-be-used RAM instead of having a permanent buffer for one-time use. 2022-03-23 08:15:30 Looks like that's 16 words long. 1024 bits. 2022-03-23 08:17:57 Well, I think this framework is all set up - I just have to figure out what to set all of that structure to to get it to accept it. 2022-03-23 08:21:59 Holy cow - it just ran with no error. I found an example online that led me to set one of the parameters to 8. I formerly had it 0. 2022-03-23 08:22:17 But - still segfaults. :-) 2022-03-23 08:54:49 Ok, so if that whole structure is 152 bytes, then the last field - sa_restorer which is a void pointer - will start 144 bytes in, right? And then the int flags field which immediately preceeds it should start at 136? 2022-03-23 08:55:23 i think 140 for the int 2022-03-23 08:55:30 oh i'm not sure 2022-03-23 08:55:34 int is 4 bytes 2022-03-23 08:55:44 you're on amd64? pointer is 8 bytes, int is 4 2022-03-23 08:55:58 i know 2022-03-23 08:56:10 write a c program that prints the offset 2022-03-23 08:56:12 hang on 2022-03-23 08:57:39 I've got such a program already - I'll add a line. 2022-03-23 08:58:35 Yeah - it's 4. Thanks. 2022-03-23 08:58:57 cool 2022-03-23 08:59:03 i just quickly whipped it up 2022-03-23 08:59:25 printf("%d ", (int) offsetof(struct sigaction, sa_flags)); 2022-03-23 08:59:35 Yeah, that was necessary for getting the termios stuff all figured out, and I've just added to it since then. 2022-03-23 09:00:00 No joy still. Success return from the handler installation call, but no sign that I'm intercepting anything on fault. 2022-03-23 09:00:53 i think in terms of c .. i would write a c program, and after it works, read the assembly 2022-03-23 09:01:34 gcc -S program.c gives you the assembly, and the raw numbers 2022-03-23 09:03:58 I think it needs to be 136. Here's the structure again: 2022-03-23 09:04:11 struct sigaction 2022-03-23 09:04:14 { 2022-03-23 09:04:16 /* Signal handler. */ 2022-03-23 09:04:18 #if defined __USE_POSIX199309 || defined __USE_XOPEN_EXTENDED 2022-03-23 09:04:20 union 2022-03-23 09:04:22 { 2022-03-23 09:04:24 /* Used if SA_SIGINFO is not set. */ 2022-03-23 09:04:26 __sighandler_t sa_handler; 2022-03-23 09:04:28 /* Used if SA_SIGINFO is set. */ 2022-03-23 09:04:30 void (*sa_sigaction) (int, siginfo_t *, void *); 2022-03-23 09:04:32 } 2022-03-23 09:04:34 __sigaction_handler; 2022-03-23 09:04:36 # define sa_handler __sigaction_handler.sa_handler 2022-03-23 09:04:38 # define sa_sigaction __sigaction_handler.sa_sigaction 2022-03-23 09:04:40 #else 2022-03-23 09:04:42 __sighandler_t sa_handler; 2022-03-23 09:04:44 #endif 2022-03-23 09:04:46 /* Additional set of signals to be blocked. */ 2022-03-23 09:04:48 __sigset_t sa_mask; 2022-03-23 09:04:50 /* Special flags. */ 2022-03-23 09:04:52 int sa_flags; 2022-03-23 09:04:54 /* Restore handler. */ 2022-03-23 09:04:56 void (*sa_restorer) (void); 2022-03-23 09:04:58 }; 2022-03-23 09:05:00 The first field is a pointer - 8 bytes. Then we have sigset_t sa_mask, and the test program says sizeof(sigset_t) is 0x80. The flags come right after, so at 0x88. That's 136. 2022-03-23 09:05:25 I'm not sure how we get to 152, though. 2022-03-23 09:05:27 ok 2022-03-23 09:05:35 offsetof will remove all doubt 2022-03-23 09:05:46 Oh. I wasn't aware of that. 2022-03-23 09:05:49 Heh heh. 2022-03-23 09:05:55 printf("%d ", (int) offsetof(struct sigaction, sa_flags)); 2022-03-23 09:05:57 Checking... 2022-03-23 09:06:05 #include 2022-03-23 09:09:02 https://ideone.com/9UYZY7 2022-03-23 09:09:25 it looks like ideone uses linux 2022-03-23 09:09:31 but yeah that's the code 2022-03-23 09:09:37 i'll try and see if i can do sizes too 2022-03-23 09:09:51 0x88 = 136. 2022-03-23 09:11:09 yep 2022-03-23 09:11:14 what else do you need to know? 2022-03-23 09:11:32 Nothing right now - you're a fountain of wisdom this morning. 2022-03-23 09:11:38 I like learning new pony tricks. 2022-03-23 09:11:40 lol thanks 2022-03-23 09:11:55 i've been doing c a long time 2022-03-23 09:11:59 too long sigh 2022-03-23 09:12:13 it has warped my brain, that's why forth is so interesting 2022-03-23 09:12:24 So, I suppose it may be actually working but I'm somehow sending it off into never never land - which might just produce the same segfault message. 2022-03-23 09:12:41 It looks happy with the syscall at intialization; I'm getting back a 0 return for that. 2022-03-23 09:12:50 So one would thing it registered *something*. 2022-03-23 09:13:49 i have never tried even thinking about writing a signal handler in assembly 2022-03-23 09:14:05 i think it will be hurculean 2022-03-23 09:14:26 Well, the complexity is all in the contents of this struct sigaction. 2022-03-23 09:14:40 herculean 2022-03-23 09:14:40 The actual call and its setup is pretty simple, it seems. 2022-03-23 09:14:52 ah cool 2022-03-23 09:15:07 I found a nice example to cull. 2022-03-23 09:15:27 Here: 2022-03-23 09:15:30 https://board.flatassembler.net/topic.php?t=13490 2022-03-23 09:17:38 ah looks interesting 2022-03-23 09:18:16 i used to write assembly but that was 20 years ago lol 2022-03-23 09:18:29 and for amiga, much simpler than todays OS'es 2022-03-23 09:18:48 i can't really imagine it nowdays 2022-03-23 09:21:58 everything appears to be set up the way I want it. I'm inspecting the sigact structure from the running system. It's pointing at the fault routine, and the int starting at 0x88 is the flags I set. 2022-03-23 09:25:45 That example uses sa_restorer, but the man page says that's obsolete and not to use it. And I think it has to do with "returning" from the sig handler. 2022-03-23 09:27:09 Hmmm. The example also has sa_flags immediately following the handler. 2022-03-23 09:28:41 And defines sa_flags as a dq, that's 8 bytes. 2022-03-23 09:28:54 but either way the low order half winds up in the same spot. 2022-03-23 09:34:57 PUBLIC SERVICE ANNOUNCEMENT: Please DO NOT calculate average bucket length to determine how good your hash is, the average bucket length will be pretty much the same regardless of hash function 2022-03-23 09:35:38 You should plot the results on a graph, or if you know what you're doing use standard deviation or another metric 2022-03-23 09:37:22 I have seen two otherwise intelligent people do this now, including it being mentioned in an article published by FIG, guys if you don't have university grade education in interpreting data please at least plot the graphs and *look* at what you're doing 2022-03-23 09:39:30 Wise words. 2022-03-23 09:48:39 Ah I think this FIG article is actually looking at average traversal length travelled when looking up a sample data set? That would be more sane. But once again the proof is in the pudding, why not profile it? And I wonder if their data is at all representative? 2022-03-23 09:50:57 I think when we all read advice online we need to remember that there are no refunds on free advice :P 2022-03-23 10:38:12 Aha - I think I AM getting the signal call, but am somehow just failing to do a proper restart. 2022-03-23 10:38:42 I ran my program and sent it a kill -57 signal - it stopped, reporting "Real time signal 23." 2022-03-23 10:39:01 Then I changed my handler set up to signal 57 instead of SYSSEGV and re-built. 2022-03-23 10:39:04 Did the same thing. 2022-03-23 10:39:19 And THIS time I got a Segmentation fault message. 2022-03-23 10:39:26 It *behaved differently*. 2022-03-23 10:46:49 I wonder if perhaps the problem is that when the OS transfers control to my handler, some registers are not set right. 2022-03-23 10:47:12 Specifically, r15, my "base of system" register. 2022-03-23 11:34:10 Ok, I'm convinced that was something to do with it. I think control lands there in the fault handler, and r14 (header section pointer) and r15 (body section pointer) are not right. Which means I can't find ANYTHING - because it's all living in some block of RAM I allocated somewhere and had r15 pointed to. 2022-03-23 11:35:07 When I try to use IP relative addressing to recover it, it seems to somehow be using my originally loaded code, because my calculations are off by the difference between the original load point and my allocated buffer. I have no way to find out what the original load point was, so that a bummer. 2022-03-23 11:36:01 Finally, I put a pair of 64-bit mov r15, val; mov r14, val instructions there at the beginning of the fault handler and then manually plugged the right immediate values in after the system was running. I could then run the fault handler as a normal Forth word and everything hung together. 2022-03-23 11:36:12 but I still segfault on signal 11. 2022-03-23 11:37:23 I can't think of any other register that matters. The fault handler is explicitly setting the Forth instruction pointer, and as soon as it starts Forth back up Forth is resetting the data and return stack pointers. None of the other registers matter for what I'm trying to do this morning. 2022-03-23 11:41:43 Ok, I take it back. I just pointed the handler at my actual code entry point - the starting point of the whole shooting match. Still segfaults. So it is NOT transfering control to my handler. 2022-03-23 11:50:14 KipIngram: r15 isn't going to be set correctly, signal handlers are like other threads, they don't preserve registers 2022-03-23 11:50:25 Right. 2022-03-23 11:50:36 I'm going to have to arrange to restore the things that need restoring. 2022-03-23 11:50:38 Changing to handle 57 it did segfault message because the default segfault signal handler executed 2022-03-23 11:51:12 Yes, right - I was just interested that the behavior differed on sig 57 depending on whether or not I'd installed a handler for that signal. 2022-03-23 11:51:22 That implied that I'd accomplished SOMETHING, at least. 2022-03-23 11:51:57 You just need to set RIP to ' COLD 2022-03-23 11:52:00 r15 is the critical value - once I have it right, all the other values I need are stored somewhere I can find them. 2022-03-23 11:52:33 Where is r15 loaded from? Is it dynamically allocated? 2022-03-23 11:52:36 Well, that's not where I want to pick up. I want to retain my dictionary and so on. 2022-03-23 11:52:56 Yes, in my startup code I point it to the buffer I allocate for the system to run in. 2022-03-23 11:53:02 And copy the image into that buffer. 2022-03-23 11:53:15 Up there everything is read/write/execute, so I'm free to do anything I want. 2022-03-23 11:53:21 I would use RIP relative addressing on a variable, so it's got to be 32-bit offset from the instructions 2022-03-23 11:53:29 That makes it easier then 2022-03-23 11:53:36 Put a copy of r15 in a variable 2022-03-23 11:53:43 I'm assuming it doesn't change otherwise? 2022-03-23 11:53:56 The problem is that I can't find that variable without having r15 set. 2022-03-23 11:54:09 RIP relative 2022-03-23 11:54:16 I tried that. 2022-03-23 11:54:17 i.e. relative to the running code 2022-03-23 11:54:23 So it doesn't need r15 2022-03-23 11:54:25 Or I thought I did. 2022-03-23 11:54:31 How would you express that in code? 2022-03-23 11:54:43 Depends on the assembler 2022-03-23 11:54:45 I tried mov rax [rel label] 2022-03-23 11:54:50 and that did assemble. 2022-03-23 11:55:10 G2G 2022-03-23 11:55:22 But it seemed to somehow be considering the original load image address - the ultimate value I came up with for r15 was off by exactly the difference between my allocated buffer address and my original load address. 2022-03-23 11:55:41 Oh wait no I'm getting texted when I need to leave 2022-03-23 11:56:02 I have a klutzy way to solve this. 2022-03-23 11:56:03 Yes of course it would use the original address 2022-03-23 11:56:24 You can patch the relative offset dynamically yes 2022-03-23 11:56:32 I can put mov r15, 0x0123456789abcdef in my fault handler, and then plug that argument with the right r15 value in my start up code. 2022-03-23 11:56:41 That too 2022-03-23 11:56:46 Since in the allocated buffer my code is writeable. 2022-03-23 11:56:48 That's easier 2022-03-23 11:56:55 Yeah go for it 2022-03-23 11:57:38 I did, and the fault handler ran fine as a forth word - I "survived" passing through it and it behaved exactly as designed. 2022-03-23 11:57:46 But then I still segfault on signal 11. 2022-03-23 11:57:55 I'm missing something. 2022-03-23 11:58:04 Maybe I somehow need to "clear" the signal. 2022-03-23 11:58:16 Somehow "acknowledge it." 2022-03-23 11:58:28 Maybe just having a handler isn't enough. 2022-03-23 11:58:58 Find some C code to do this with system calls and look at generated assembly? 2022-03-23 11:59:24 Actually it may be that I need to restore the data stack pointer too. 2022-03-23 11:59:37 But that will be easy - once I have r15 I've got the data stack based stored somewhere. 2022-03-23 11:59:54 Yeah - that's likely it. Ok; here we go again. 2022-03-23 12:01:36 I would try and look up the docs but the apple developer website seems to be down right now 2022-03-23 12:03:38 I'm not sure how the signal returns to the program, and wouldn't be surprised if the C signal handling code has a shim that allows you to simply return instead of doing a system call, you might need to do a system call to end 2022-03-23 12:06:59 KipIngram: There's a syscall on Linux called "sys_rt_sigreturn", I am guessing there's some kind of sigreturn on mac as well 2022-03-23 12:11:15 A bunch of websites are down right now, I guess AWS or some cloud thing is down? Or a coincidence? 2022-03-23 12:11:28 I thought stackoverflow was self-hosted 2022-03-23 12:18:36 I'll look into it. I'm not on a Mac anymore, fortunately. 2022-03-23 12:18:49 Still use one for work, but this stuff is on my Fedora box now. 2022-03-23 12:19:10 I'd seen that sigreturn, but I figured that was probably if you wanted to try to continue what you were doing. 2022-03-23 12:19:26 I'm not - I'm "starting over." :-) 2022-03-23 12:20:03 But it really does seem like there's something needed that I'm msising. 2022-03-23 12:20:10 I having "made the system happy" yet. 2022-03-23 12:20:14 haven't 2022-03-23 12:22:27 Ok - 2022-03-23 12:22:29 https://man7.org/linux/man-pages/man2/sigreturn.2.html 2022-03-23 12:23:12 Looks much more involved. I think that the "handler" needs to fuss around with a copy of my system's state that the system has saved on the stack somewhere. Looks like the normal routine here is that everything is designed to restart you from where you left off. 2022-03-23 12:23:25 There's ANOTHER bit of code referred to as a "signal trampoline." 2022-03-23 12:24:03 So I'm going to have to go through that, and the system will be trying to send me back to right where I left off. I don't want that. So I need to find the information it's saved with which to do that and doctor it, so that when it sends me back it will send me where I want to go. 2022-03-23 12:24:11 the good news is that it will restore my registers and so on. 2022-03-23 12:26:27 This is trying to do more than I really need, but I guess it is what it si. 2022-03-23 12:47:42 KipIngram: Oh right I thought you were on MacOS 2022-03-23 12:47:47 Just noise I guess 2022-03-23 12:47:52 Linux is easier 2022-03-23 12:48:04 Maybe look at how gforth does it 2022-03-23 12:55:50 Actually don't, it will probably be at the C level 2022-03-23 13:00:11 :-) 2022-03-23 13:00:41 I'm kind of worn out on it today, I think - maybe I'll pick it up again tomorrow. No point making it an ordeal. I think I have a more clear picture than I did this morning, at least. 2022-03-23 13:01:06 I assume the way the system saves my context is documented in some way, and I think as I've been nosing around I've seen some "context" data structures in play. 2022-03-23 13:01:28 So likely it will be easy enough to patch that context up to re-direct my path on return, and then just do the sigreturn. 2022-03-23 14:45:42 Ok, so I found a *working example* that I can assemble and run. 2022-03-23 14:45:44 https://gist.github.com/rexim/c172f43ffdc73a14e8b0f83f4cbe0f5a#file-sigaction-asm-L1 2022-03-23 14:46:01 they have line 41 commented out - I needed to uncomment that to make it work. 2022-03-23 14:46:39 But that appears to work flawlessly - it's set up to intercept ctrl-c. If I install the handler, it does so - both the handler and restorer functions print messages that get interspersed with the normal output. 2022-03-23 14:47:01 If I do not install the handler, ctrl-c (or kill -2) terminates it in the usual way. 2022-03-23 14:47:07 So that's a basis. 2022-03-23 14:47:14 I'm trying to port it into the Forth now. 2022-03-23 14:49:35 I can't just print messages in those functions, though, and don't want to. The third parameter passed to the handler, which should be addressed by rdx, is that "context structure." I want to adjust the rip value there to "next" and the Forth rrIP value to an error point. 2022-03-23 14:49:52 So far no luck, but SURELY with a working example in hand I can figure it out. 2022-03-23 14:53:40 I would check that adjusting RIP there actually makes a difference 2022-03-23 14:53:50 The changes to that struct might be ignored 2022-03-23 14:54:11 It explicitly said it was used to restore the user state. 2022-03-23 14:54:16 Ah good 2022-03-23 14:54:40 If there's not a way to change where the system returns to, then I'm beaten. 2022-03-23 14:54:54 It will just keep trying to do whatever caused the problem in the first place. 2022-03-23 14:55:05 No there definitely is, or there'd be no way to recover a SIGSEGV 2022-03-23 14:55:37 Well, often what's done there is set up a new RAM page, so when the intruction retries it works without a fault. 2022-03-23 14:56:03 The whole point in those cases is for the user program to run exactly as it was written. 2022-03-23 14:56:07 When you get this working you might need to check if there's a guard page mechanism on Linux for stack growth, because if so you'll need to re-add the guard page on the stack 2022-03-23 14:56:21 If you're even using the system-supplied stack... 2022-03-23 14:57:01 It's a thing on Windows anyway, might not be a thing on Linux https://devblogs.microsoft.com/oldnewthing/20220203-00/?p=106215 2022-03-23 15:11:30 I am using the system stack. I don't really do anything with it in my system, except a little juggle necessary to let COLD refresh the run image from the load image. There are some values that need to be passed through there, and nothing in the initial load image is writable. 2022-03-23 15:25:25 This is all discussed here: 2022-03-23 15:25:28 https://man7.org/training/download/spintro_signal_handlers_slides.pdf 2022-03-23 15:26:23 I still don't have this working. I'm sure my installation call and the structure I'm passing mirrors the one in the working example. 2022-03-23 15:27:14 Really the main difference is that I'm touching that context in the handler. 2022-03-23 15:27:22 It says very clearly here: 2022-03-23 15:27:24 https://aaronbloomfield.github.io/pdr/book/x86-64bit-ccc-chapter.pdf 2022-03-23 15:27:51 in the section "The C Calling Convention / caller's rules" that the third parameter to a function is passed in rdx. 2022-03-23 15:28:08 And that context structure is clearly documented in include files - it's just a string of dq's. 2022-03-23 15:31:22 The working example actually performs screen writes from the handler and the trampoline - you're really not supposed to do that. Not supposed to call any "not guaranteed asynch safe" functions. 2022-03-23 15:31:39 But it works in spite of that. 2022-03-23 15:33:16 I never recover from the situation that's supposed to trigger my function, so there's no very good way to investigate whether that context is really there where I think it is or not. 2022-03-23 15:33:45 "there are some values that need to be passed through there" unless you're talking about environment variables and command-line arguments, you don't need to use the provided stack at all 2022-03-23 15:34:03 And you can copy those to another buffer as well 2022-03-23 15:35:13 Well, I just meant I needed some way to pass them. Any way. There is no writable RAM in the load image. So when I jump back down there to re-copy the image to the allocated buffer, I just pass some data needed to know the parameters for the copy operation. 2022-03-23 15:35:22 There might be some other way to do it, but this way worked. 2022-03-23 15:37:00 Oh... I think I just successfully used gdb to put a breakpoint in the signal handler. I can inspect the registers now. 2022-03-23 15:37:31 rdx certainly does NOT point at a structure. 2022-03-23 15:38:12 Might be passed on the stack? 2022-03-23 15:38:30 Might be. I'ma gonna go prospecting for it. 2022-03-23 15:38:54 x/100x $sp 2022-03-23 15:38:59 I need to run it again, though, and note down some interesting values for comparison with the regs. 2022-03-23 15:41:45 Oh, hmmm. r14 and r15 have their correct value, even in the handler. 2022-03-23 15:42:00 Not something you can rely on I think 2022-03-23 15:42:06 Agreed. 2022-03-23 15:42:08 Which signal are you trapping now? 2022-03-23 15:42:12 2 2022-03-23 15:42:23 Just because that's the one the example used. 2022-03-23 15:42:47 I'm not sure 2 will get the same data as SIGSEGV, unless the example was inspecting RIP? 2022-03-23 15:42:50 Ok, so there is a bit that tells the system how you want your handler called. One way just passes you an int. 2022-03-23 15:43:00 Ah that's it 2022-03-23 15:43:01 The other way passes three parameters, one of which is that structure pointer. 2022-03-23 15:43:09 the SIGINFO bit. 2022-03-23 15:43:17 Which I thought I was setting, but perhaps not. 2022-03-23 15:43:35 This isn't your first x86-64 syscall right? 2022-03-23 15:43:51 Not sure what you mean. 2022-03-23 15:44:02 I've done a lot of syscalls for I/O and stuff before. 2022-03-23 15:44:07 Okay then 2022-03-23 15:44:09 And iocotls. 2022-03-23 15:44:16 This is my first signal handler, though. 2022-03-23 15:44:31 I was going to suggest something but it's not that if you've done syscalls properly before 2022-03-23 15:47:34 KipIngram: On my system it's 4 2022-03-23 15:58:30 KipIngram: rt_sigaction has 4 arguments, are you setting all 4? 2022-03-23 16:32:31 I was talking about the parameters passed to the handler. 2022-03-23 16:34:21 The example was actually setting only two arguments, with a third commented out, on the rt_sigaction call. The handler, the flags, and I uncommented a "restorer" argument (which the man pages say is depracated, but it didn't work without it and did work with it). 2022-03-23 16:34:27 What is your fourth argument? 2022-03-23 16:34:52 My main syscall reference page here: 2022-03-23 16:34:55 https://hackeradam.com/x86-64-linux-syscalls/ 2022-03-23 16:34:58 shows four args as well. 2022-03-23 16:35:25 I actually am setting that fourth one. It's zero. 2022-03-23 16:35:31 The "old action." 2022-03-23 16:36:21 I have to specify 1) signal number, 2) handler, 3) old handler receptacle (I make it zero), and 4) a size, which seems to want to be 8. 2022-03-23 16:36:45 At least if I don't set it to 8, I get EINVAL error. 2022-03-23 16:55:04 "I was talking about the parameters passed to the handler." yeah I know that 2022-03-23 16:55:51 Yes the fourth argument is a size that has to be a fixed size, I'll take your word it's 8 2022-03-23 16:56:34 Just trying to help figure out why rdx isn't getting set properly 2022-03-23 17:03:23 I don't think gdb is handling this break right. If just break at my machine code nop word, that works - the program counter is one byte past the break address. 2022-03-23 17:03:50 But when I put the break at the start of the signal handler and send the signal, the program counter points to a completely different place. 2022-03-23 17:04:17 Maybe you can't break in a handler? Maybe it registers it but doesn't break until later? 2022-03-23 17:05:39 One problem here is that the sa_sigaction structure passed in by the working example is laid out differently from the numbers I get from printing offsetof() in my C test program. 2022-03-23 17:06:27 And the man pages say that "restorer" thing is depracated. 2022-03-23 17:06:41 That's basically the trampoline. 2022-03-23 17:07:28 Some of the docs also say one should never actually call the sigreturn function - somehow that's supposed to happen automatically. 2022-03-23 17:07:47 But they do in the example, and even print a message from it, so it is getting run. 2022-03-23 17:09:21 So the whole situation is something of a mess. 2022-03-23 17:09:42 It's a little hard to see how it ever got this confused. 2022-03-23 18:07:51 KipIngram: The sigreturn function is called magically I think by your C library or kernel's trampoline, or something like that 2022-03-23 18:08:09 But in assembly/FORTH you're going to have to call that 2022-03-23 18:47:19 Yes, that call must come from user-space, so I have to explicitly write it. But apparently when the OS calls my handler, it arranges the return stack so that when I return from the handler it will return into the trampoline. Seems a little silly in assembly - just as easy to have me do it myself, but I can see how it let them "hide that" in C. 2022-03-23 18:48:19 I just broke down and posted a Stack Overflow question on it. We'll see how that goes - over the years I've encountered some pretty pretentious people on that platform. 2022-03-23 18:49:33 Anyway, if it actually works the way I'm trying to make it work now, then I could have just done the sigret call from the handler, instead of doing the extra backflip. 2022-03-23 19:10:29 I can see that setting a gdb breakpoint on my signal handler changes its first byte to 0xCC, which is the "INT3" opcode. So that's promising. 2022-03-23 19:11:42 Oddly, when I hit a breakpoint (even one in an innocuous word like NOP), it's as if I hit ctrl-z - I'm put into bash with the debug session in the background. I have to run "fg" to get back to gdb. 2022-03-23 19:12:43 And as I mentioned earlier, with a breakpoint in nop, the instruction pointer is positioned one byte past the break location, but on the signal handler it's nowhere close. From the looks of the code, if I disassemble, it looks like I'm at the tail end of KEY - probably just got back from the input syscall. 2022-03-23 19:14:19 That actually makes sense, though - I read somewhere this afternoon that the handler has to run in user space, so if you're in a syscall it's postponed until you return to userland. 2022-03-23 19:22:19 Ok - here's a good write-up on gdb and signals: 2022-03-23 19:22:22 https://undo.io/resources/gdb-watchpoint/how-work-signals-gdb/ 2022-03-23 19:23:10 gdb itself intercepts the SIGINT signal, and never passes it to the application. So I am not triggering at a breakpoint - I'm just stopping the program wherever gdb happens to catch it. 2022-03-23 19:23:16 I can tell it to pass that through, I think. 2022-03-23 19:26:36 rip still isn't at the breakpoint. 2022-03-23 19:27:07 Ah, because the program's dead. 2022-03-23 19:27:18 So it never made it to my handler. 2022-03-23 19:40:11 Ok. So right - it's crashing when it tries to call my handler. Never getting there. So no wonder rdx had the wrong value; I wasn't at the right place. 2022-03-23 19:40:26 Sure is nice to be able to run this and the example with gdb and compare behaviors. 2022-03-23 20:33:15 Hey! Progress. It's not throwing the error message the way it should, but it's no longer crashing. 2022-03-23 20:51:52 Ok, it's working. I have the context diddling commented out right now, but when I send the SIGINT signal it's passing through the handler and trampoline, with no effect at all on the running system. 2022-03-23 20:53:11 If I try it with signal 11, though, the system freezes up. it no loner segfaults - just hangs. 2022-03-23 20:53:29 But - that's a bad error and I'm suppressing it without doing anything about it. So that doesn't surprise me too much. 2022-03-23 20:55:43 Aha - rdx now contains an address. 2022-03-23 20:55:52 And I am breaking IN THE HANDLER. 2022-03-23 21:06:32 I see the context structure. It looks to me like the registers are NOT mapped onto that the way the documentation made it appear. 2022-03-23 21:33:19 Man, this makes no sense. I see registers in that structure. A few of them I can identify just based on the contents, and poking around and inspecting offsets in my code in another window. 2022-03-23 21:33:34 But they DO NOT MATCH UP with the include files. They're in a different order. 2022-03-23 21:34:58 r14 and r15, for example, are, according to the includes, the seventh and eighth items in the structure. 2022-03-23 21:35:22 But I see them in the 10th and 11th slots. 2022-03-23 21:35:55 Ok, well - wait. Maybe there's something in the structure before the registers. 2022-03-23 21:36:10 So lemme see how that works out. Let me MAKE them be 7th and 8th. 2022-03-23 21:45:01 That parameter is just documented as void *, so we don't really get told what it points to.