2022-12-09 10:52:01 does anyone have pointers on existing words or tools to handle organization of blocks? 2022-12-09 12:35:43 Chuck talks about some techniques in that early book of his, but doesn't off any particular code. 2022-12-09 12:35:59 Nothing fancy - just stuff you'd probably write down in your first few minutes of thinking about it. 2022-12-09 12:38:27 The "most simple" way of doing is to just link the blocks together, either singly or doubly, but that means allocating a bit of each block for those pointers. 2022-12-09 12:38:48 Otherwise you need a map of some kind. I've thought about doing it with b-trees. 2022-12-09 12:39:12 Then you have to decide whether or not a block can be "partially used." 2022-12-09 12:40:02 That information could also go either in each block or in a map/b-tree. 2022-12-09 12:40:19 But I'll hush now, because I doubt I'm telling you anything that isn't obvious to you. :-| 2022-12-09 13:13:27 I'm mostly looking for things to help in moving blocks around, to consolidate scattered things, and keep things closer together. But not to extend to anything like an actual file system. 2022-12-09 13:16:21 I now have a collection of several thousand blocks, but over the last year the main store of these has become quite disorganized and scattered. 2022-12-09 13:16:23 Before I go down the road of writing a set of tools to help structure them, I'm hoping to review existing approaches. 2022-12-09 13:22:16 Makes sense. 2022-12-09 13:22:39 I guess you can tell from the content somehow what goes with what? 2022-12-09 13:23:26 every block has a title line with a group identifier, which helps 2022-12-09 13:37:43 did you think about making forth your shell? 2022-12-09 13:38:06 I've made my lang to be the shell, now scripts will be just colon word definitions 2022-12-09 13:38:31 it will also make it grow and be useful as is now my main interface to the system 2022-12-09 13:38:58 tried in the past but had issues and didn't know I just need to set the PATH 2022-12-09 13:39:02 not exactly. Most of the tools I use on my unix system are written in Forth, but I still use a traditional shell. 2022-12-09 13:39:11 I think forth can be good as a shell 2022-12-09 13:39:19 most of the time for the last year I've been running in a purely block-based environment 2022-12-09 13:39:21 commands are just a colon word 2022-12-09 13:39:36 you can set up some init files or whatever 2022-12-09 13:39:45 and it makes you use the lang much more 2022-12-09 13:40:42 I'm not really interested in building a normal-ish shell in forth 2022-12-09 13:40:52 for now I only have [ command and arguments ] exec/system 2022-12-09 13:41:06 crc I always wanted to make my own shell 2022-12-09 13:41:23 I'm happy now that I just set the PATH and everything works 2022-12-09 13:42:01 also my lang is not a real forth and is made on top of the best unix scripting language I know 2022-12-09 13:42:09 so it has potential to be a shell 2022-12-09 13:42:16 my two main systems are the newer block-based one, and one in which I can only run forth programs (runs in a chroot, the only binary is the forth vm, all programs are spawned in a forked vm and compiled on load) 2022-12-09 13:42:17 and has fancy colors :D 2022-12-09 13:54:09 I'm happy 2022-12-09 14:17:16 https://termbin.com/rqpm5 this is the init file 2022-12-09 14:17:28 I'm just adding words that call exec/system 2022-12-09 14:17:45 # is to interpolate an element in a list xD 2022-12-09 15:40:39 vms14: I've thought at times that I would like a Forth shell. 2022-12-09 15:41:26 However, I think one of the strongest features of bash as a shell is the use of pipes to chain commnds together into pipelines. I've thought about how to do that in Forth, but it hasn't led me anywhere particularly pleasing yet. 2022-12-09 15:43:02 I have a pipe() function in perl 2022-12-09 15:43:20 and socketpair which is a better pipe function using unix sockets instead 2022-12-09 15:44:05 Well, making a pipe - the literal plumbing - seems straightforward enough. It's exactly how it would "serve" in a stack-based system like Forth that I haven't seen a clear "happy architecture" for. 2022-12-09 15:44:14 also I can "open" a command and get a filehandle to stdin and stdout 2022-12-09 15:44:37 Yes; again I can see how to bolt stdin and stdout onto Forth processes. 2022-12-09 15:44:43 hmm 2022-12-09 15:44:45 My system doesn't "do" any of that yet. 2022-12-09 15:44:55 But I think I could put it together without too much trouble. 2022-12-09 15:45:04 so you don't know how to design the "api" 2022-12-09 15:45:09 or the syntax 2022-12-09 15:45:20 But when I think about how I actually use pipes in Linux, it's hard to see how to port that style of getting things done over to Forth. 2022-12-09 15:45:36 Forth seems to lean toward taking *commands* from the input, and *data* from the stack. 2022-12-09 15:45:55 there's nothing corresponding to the stack in the bash shell envrironment. 2022-12-09 15:46:19 there are stacks, but they're for history or pushd or such 2022-12-09 15:46:25 you can put the memory address of a 'process' object 2022-12-09 15:46:31 When you make a Linux pipeline, that *data* you're usually sending through that pipe. 2022-12-09 15:46:46 actually Forth already does that, except it uses stack data. 2022-12-09 15:47:05 You can very much think of a line of Forth as a pipeline, with the stack carrying data a long down the line. 2022-12-09 15:47:34 yeah, my stack can have strings so it could be even easier 2022-12-09 15:47:37 Sure - all of those mechanical details I see how one would do. 2022-12-09 15:47:44 It's WHAT TO DO WITH IT that's unclear to me. 2022-12-09 15:47:50 Use cases that really resonate with me. 2022-12-09 15:48:24 hmm 2022-12-09 15:48:47 you just need to focus on how you want to send input/output to an existing process 2022-12-09 15:49:03 and how to read it 2022-12-09 15:49:21 that's all what a pipe does after all 2022-12-09 15:49:28 What I've tried to do is think about how I use pipelines in Linux, and try to imagine that same functionality happening in Forth. 2022-12-09 15:49:35 It just hasn't come together yet for me. 2022-12-09 15:49:56 probably need to go on walkabout 2022-12-09 15:50:20 Maybe I'm focusing too much on thinking of the first process's stdin as being the keyboard; usually in linux you send in the initial input from somewhere else, like a file. 2022-12-09 15:50:55 We CALL it stdin, but usually the first command in a pipeline does something to a file. Like maybe greps on the lines of a file, etc. 2022-12-09 15:51:20 Actually I do think that might clarify some things; I'll have to ponder that this weekend. 2022-12-09 15:51:33 what if you treat them like file handles 2022-12-09 15:51:48 a word to take the input file handle of a process 2022-12-09 15:51:56 another for the output file handle 2022-12-09 15:52:06 just read/write form/to them 2022-12-09 15:52:14 That still feels like mechanical details to me; not the problem. 2022-12-09 15:53:17 Maybe I need a way to plumb a block, or a range of blocks, onto the left end of the pipeline as the initial source of the first command's stdin. 2022-12-09 15:53:48 " ls" process output read.until.eof 2022-12-09 15:54:04 xD 2022-12-09 15:54:12 100 block 4096 type | cmd | cmd | cmd 2022-12-09 15:56:12 I'm not willing to implement pipes, but I'd just use the perl open2 function which starts a process and gives me two file handles for read/write 2022-12-09 15:56:29 both would be pushed on the stack or whatever 2022-12-09 15:56:33 Sure. But remember I'm really not using anything that comes with such tools. 2022-12-09 15:56:44 I've even minimized OS syscalls as much as possible. 2022-12-09 15:56:48 still you say that's not the problem 2022-12-09 15:57:25 Sure - I could make my KEY and EMIT words so that they could draw data from / send data to something other than the console. 2022-12-09 15:57:35 oh 2022-12-09 15:57:44 I could arrange for each process to have a vectored word to do those functions. 2022-12-09 15:57:55 Those would represent its stdin and stdout. 2022-12-09 15:58:09 I have a word named attach that makes . print to a filehandle instead of stdin 2022-12-09 15:58:24 the filehandle can be a process input, file or a socket 2022-12-09 15:58:39 or even a variable as I can open variables as a file handle xD 2022-12-09 15:58:40 And I could add "buffers" that could take output from stdout of some process and send it to some other process's stdin. 2022-12-09 15:59:21 a process that found its stdin empty would block until data appeared in it. Likewise if there was a capacity limit on stdout and it got full. 2022-12-09 15:59:23 I pty the fool who tries to reinvent that 2022-12-09 15:59:32 :-) 2022-12-09 15:59:45 It's just a queue, right? 2022-12-09 16:00:03 with some sharp edges like things blocking forever 2022-12-09 16:01:06 Well, sure - you have to do it right. I could cut myself with my woodworking chisel too. 2022-12-09 16:01:25 it will happen always when you write a line and expect to read a line, but the process wants to read the whole thing before output 2022-12-09 16:01:44 Nice thing about such a queue, though is that each pointer (write pointer, read pointer) would be managed by only one process. so there wouldn't be any contention around those. 2022-12-09 16:02:26 If read pointer == write pointer, it's empy; if write pointer is one less than read pointer, it's full. 2022-12-09 16:02:34 ring buffer again. 2022-12-09 16:02:40 on unix the streams are shared between processes for extra fun(TM) 2022-12-09 16:02:54 With "one less than" being treated appropriately modulo. 2022-12-09 16:02:59 mainly for convenience 2022-12-09 16:03:10 that's why pipe exists mainly xD 2022-12-09 16:03:27 Yeah - if you started putting more than one process on each end it would get trickier wouldn't it? 2022-12-09 16:03:43 you create a pipe and then fork 2022-12-09 16:03:59 the child will get that pipe so you can talk with your beloved child 2022-12-09 16:04:05 and he can ask you for money 2022-12-09 16:04:11 then you close the file handle 2022-12-09 16:04:28 I'm not immediately thinking of an application for such a thing. I know that tee is useful sometimes, to let you see what passes by a spot in a pipe. 2022-12-09 16:04:33 if does not ignore sigpipe will die 2022-12-09 16:04:51 Heh. 2022-12-09 16:05:20 That actually is something I think of as slightly weak in Linux. It doesn't really offer any way of "sharing variables" 2022-12-09 16:05:46 You can give your children an initial copy, but then they leave home and they never answer the phone. 2022-12-09 16:06:00 I'm thinking of environment variables here. 2022-12-09 16:06:41 Sometimes it would be dang useful to have shared environment variables. 2022-12-09 16:06:50 I'd also like ipc to be better 2022-12-09 16:07:05 in this stuff I've done recently I used files. On process writes to a file; another monitors it. 2022-12-09 16:07:05 posix_shm 2022-12-09 16:07:06 :) 2022-12-09 16:07:09 Unidirectional. 2022-12-09 16:07:18 Via ssh; processes are on different systems. 2022-12-09 16:07:24 you cna always serialize stuff, but meh 2022-12-09 16:07:41 SHM 2022-12-09 16:07:43 OVER 2022-12-09 16:07:46 THE NETWORK 2022-12-09 16:07:48 after all, that's why protols are made 2022-12-09 16:09:45 but you KipIngram with your own forth can provide something better 2022-12-09 16:09:55 and share memory between processes 2022-12-09 16:10:21 fork always seemed wrong to me 2022-12-09 16:10:27 threads are nicer 2022-12-09 16:11:13 Yes, if I want to. 2022-12-09 16:11:19 but "you call fork, if returned 0 you're the child so you wrap everything in an if" does not look like a good way to work 2022-12-09 16:11:37 Or it could be done with BLOCK. 2022-12-09 16:11:57 If two processes load the same block, they'll both have the address of that data. 2022-12-09 16:12:13 KipIngram: won't you end with your own system? 2022-12-09 16:12:17 then you will want to 2022-12-09 16:12:18 xD 2022-12-09 16:12:34 Well, hopefully. That's the idea. 2022-12-09 16:12:47 If I ever get around to finishing it. 2022-12-09 16:12:53 I figure I probably will. 2022-12-09 16:12:58 It's pretty far along at this point. 2022-12-09 16:13:21 But it's been a while since I've worked on it. 2022-12-09 16:13:48 my main goal with my lang is to make it replace any other lang I want to use 2022-12-09 16:14:03 making it my default shell seems like a good step in that direction 2022-12-09 16:14:12 Yeah, me too, except not for work or anything. Just for future things I do. 2022-12-09 16:14:35 for now it sucks as a shell, but I assume it will be better eventually 2022-12-09 16:14:56 I'm just making colon words that call exec XD 2022-12-09 16:16:38 which now that I think about I should really start making those pipes 2022-12-09 17:27:14 KipIngram: 2022-12-09 17:27:17 [ ls ] process drop drop :input input: read.line dup [ input: read.line dup .? cr ] while 2022-12-09 17:27:19 XD 2022-12-09 17:27:34 don't parse ls sheesh 2022-12-09 17:27:35 quite a bit shitty but it does read all the output of ls 2022-12-09 17:27:58 the 2 drops are cause process returns input output pid 2022-12-09 17:28:33 you're supposed to wait for the children to exit, but meh 2022-12-09 17:28:42 he will die with me 2022-12-09 17:28:57 or you could ignore SIGCHLD 2022-12-09 17:29:16 what does happen if I do 2022-12-09 17:30:23 Setting $SIG{CHLD} to `IGNORE' on such a platform has the effect of not creating zombie processes 2022-12-09 17:30:25 oh 2022-12-09 17:30:59 this is nice 2022-12-09 17:31:06 ignoring your children so they die 2022-12-09 17:31:38 Zombies annoy me. Sometimes I can't kill them, even as root, and it's just offensive to me that there's ANYTHING I can't do as root. 2022-12-09 17:31:48 I mean, WHY? 2022-12-09 17:32:04 because that hard nfs mount might come back up? 2022-12-09 17:32:14 That seems like a bug to me. 2022-12-09 17:32:22 But I don't think it's considered one. 2022-12-09 17:32:29 unix ipc is not nice after all 2022-12-09 17:32:48 most of it looks like an afterthought or a hot fix 2022-12-09 17:33:00 only pipes are fine xD 2022-12-09 17:33:05 But, it's a process, and I have its PID. And I'm root. If I tell it to die it should die. 2022-12-09 17:33:07 End of story. 2022-12-09 17:33:25 Wait - wrong words. 2022-12-09 17:33:25 you can shutdown the computer so they'll die 2022-12-09 17:33:35 If I tell the system to KILL it, it should do so. 2022-12-09 17:33:38 I know that. 2022-12-09 17:33:43 But I don't like rebooting. 2022-12-09 17:34:07 rebooting is great. firmware updates, checking whether the system still boots correctly, etc 2022-12-09 17:34:08 It's better than it used to be, because these days Chrome remembers open tabs. 2022-12-09 17:34:56 But it is a little bit of work to get everything set back up just right after a reboot. 2022-12-09 17:35:19 I'll happily go for months between reboots if I can. 2022-12-09 17:35:59 not sure if linux can go for months, unless you're hotpatching the kernel or ignoring the updates 2022-12-09 17:36:17 I occasionally apply updates. 2022-12-09 17:36:24 But I don't worry about doing it instantly. 2022-12-09 17:36:44 But at any rate, the point is that I don't see WHY the system won't kill a process no matter what if I tell it to. 2022-12-09 17:36:52 kill -9, I mean. 2022-12-09 17:37:04 blocking I/O, etc 2022-12-09 17:37:46 Well, if an IO comes in later, that's simple. Oh, he killed that recipent; drop this on the floor. 2022-12-09 17:38:07 it seems that only the parent process can kill a zombie 2022-12-09 17:38:22 but being root and not able to do that, yes, looks like a bug 2022-12-09 17:38:31 bug/misfeature/design flaw 2022-12-09 17:38:37 Well, my console session should be some ancestor. Parent, grandparent, etc. 2022-12-09 17:38:44 I'd say design flaw as this is mostly how I see unix ipc 2022-12-09 17:38:50 But I don't see why even that should matter. 2022-12-09 17:38:52 the only good things are pipes and sockets 2022-12-09 17:38:59 Root should be root. God. 2022-12-09 17:39:10 if the console dies, then it will go to pid 1 2022-12-09 17:39:57 I mean if the parent of a zombie dies, the zombie it's goes to the orphanage which is 1 2022-12-09 17:40:27 still makes no sense, but it's like it was done like that and it won't change 2022-12-09 17:40:49 why are we structuring running programs hierarchically. 2022-12-09 17:40:57 kinda dumb. 2022-12-09 17:41:06 or at least, ignorant. 2022-12-09 17:41:15 cause fork 2022-12-09 17:41:32 fork without program-mediated supervision ala erlang is a bad idea. 2022-12-09 17:41:51 fork is a good idea, but bad implemented 2022-12-09 17:42:04 supervision trees are a great idea. 2022-12-09 17:42:05 well half of fork is a good idea xd 2022-12-09 17:42:06 fork is a bad idea. 2022-12-09 17:42:53 for example to a server is good to be able to fork an instance that shares opened sockets 2022-12-09 17:43:17 threads are like a second thought on fork 2022-12-09 17:44:04 go use erlang for a while. 2022-12-09 17:44:16 you will be infinitely frustrated with everything else after you do. 2022-12-09 17:44:33 erlang has reputation for being good with concurrency 2022-12-09 17:44:53 and processes are a form of concurrent execution. 2022-12-09 17:44:56 hence. 2022-12-09 17:51:45 hahha in perl threads are officially discouraged xD 2022-12-09 17:52:08 they were hacked in to emulate fork on a certain platform 2022-12-09 17:52:17 they're actually a fork in disguise 2022-12-09 17:52:44 that is disgusting. 2022-12-09 17:59:03 well it's not a real thread, but at least has a better api than fork xD 2022-12-09 17:59:44 but it makes a whole copy of the perl interpreter and all the data at that time like fork 2022-12-09 18:00:10 yes, because Microsoft lacks fork, so therefore... 2022-12-09 18:01:04 you can't fork on windows? 2022-12-09 18:02:26 pretty sure some curl author was complaining about windows being the hardest OS to support 2022-12-09 18:04:35 that makes sense. much different API set. 2022-12-09 18:05:56 https://news.ycombinator.com/item?id=33798708 2022-12-09 18:39:38 Erlang is like all about threads, right? That's kind of its point? 2022-12-09 18:39:56 Don't they make Erlang applications with like a million threads or something like that? 2022-12-09 18:39:58 not threads 2022-12-09 18:40:20 threads are generally OS level and preemptively scheduled by the OS 2022-12-09 18:40:20 Ok, what do they call them? 2022-12-09 18:40:23 uhh 2022-12-09 18:40:25 processes 2022-12-09 18:40:32 Ok. 2022-12-09 18:40:37 erlang processes are network transparent and dont share memory 2022-12-09 18:40:38 But they're super light, right? 2022-12-09 18:40:41 uber 2022-12-09 18:40:42 yes 2022-12-09 18:40:51 tiny heap by default 2022-12-09 18:41:07 What does "network transparent" mean? 2022-12-09 18:41:32 Does that mean it doesn't matter where they run? 2022-12-09 18:42:52 Like, do all the participating systems form up some kind of "meta environment"? 2022-12-09 18:44:23 yes 2022-12-09 18:44:32 That's kind of a cool idea. 2022-12-09 18:44:58 I think I watched a YouTube video by the guy who came up with that. 2022-12-09 19:05:30 KipIngram: Erlang uses the actor model 2022-12-09 19:09:24 https://en.wikipedia.org/wiki/Actor_model 2022-12-09 19:09:51 they're kind of an object, but in terms of concurrency 2022-12-09 19:14:42 A newly spawned Erlang process uses 309 words of memory [4] 2022-12-09 19:14:51 http://blog.richardalucas.com/2017/01/09/Actor-Model-in-Erlang/ 2022-12-09 19:15:29 Spawning 20,000 processes took an average of 3.0 microseconds/process of CPU time and 3.4 microseconds of elapsed (wall-clock) time. 2022-12-09 19:19:58 Yeah, that's the sort of thing I'd heard - just really really really light processes. 2022-12-09 19:20:18 I probably should read a little about how that all works; it sounds like it has to be pretty clever. 2022-12-09 19:21:12 I like learning about ideas like that and then trying to fuse a bunch of them together. Pick up nice bits from all over. 2022-12-09 19:21:53 Some folks like gathering code from various places and integrating - I like gathering ideas. 2022-12-09 19:26:46 KipIngram: if you're going to implement something like threads or processes, the actor model seems to be a good ide 2022-12-09 19:26:48 idea 2022-12-09 19:27:08 the actor model is why erlang shines so much with concurrency 2022-12-09 19:27:26 so it seems it's better than the approaches we already knew 2022-12-09 19:27:47 they avoid stuff like race conditions, locking, shared memory, etc 2022-12-09 19:28:31 if I were going to make an operating system, fork and threads is obviously a no go 2022-12-09 19:28:54 I'd go for the actor model, just because it seems to pay off 2022-12-09 19:29:47 they're just a unit of computation, like an object when a language is fully object oriented 2022-12-09 19:30:27 they can spawn other actors and die when they finish their task, they can talk with other actors via messages 2022-12-09 19:30:57 erlang just bases on that 2022-12-09 19:31:45 I wouldn't care to even try to think about fork or threads 2022-12-09 19:32:05 you already know everyone hates them and find them difficult to use 2022-12-09 19:32:23 semaphores, locks, race conditions, etc 2022-12-09 19:33:22 but at the end is just to avoid shared memory and provide a way to send messages