2022-04-30 04:27:53 <veltas      > KipIngram: I think the best way to do UTF-8 is to not do UTF-8
2022-04-30 04:28:20 <veltas      > It's already an efficient storage format, you just need to 'remember' it's UTF-8 when it counts, which is not often
2022-04-30 04:29:12 <veltas      > And UTF-32 doesn't actually describe 'characters' per-se, because Unicode code points can combine
2022-04-30 04:29:41 <veltas      > So you've got a multi-byte or multi-word format... might as well use a more intelligent multi-byte format
2022-04-30 04:29:53 <veltas      > More space-efficient anyway
2022-04-30 04:36:24 <dave0       > i tried to write a utf8<->utf32 function (but in c not forth :-| ) but i haven't properly tested it
2022-04-30 04:37:26 <dave0       > there's some rules to follow which makes the code hard to read
2022-04-30 04:41:08 <KipIngram   > veltas: that's exactly how I felt too, until I started tinkering with UTF-8 and started wanting access to those special characters.
2022-04-30 04:41:36 <KipIngram   > I had no interest whatsoever.
2022-04-30 04:42:05 <KipIngram   > The simplicity and straightforwardness of ASCII is very pleasing.
2022-04-30 04:42:42 <KipIngram   > But  now I want them, and I also want to support the escape sequences that some of the special keys send.
2022-04-30 04:43:25 <KipIngram   > Two different things, but doing the escape keys without "inventinga kludge" also involves going beyond one byte per character.
2022-04-30 04:43:58 <KipIngram   > For the escape keys only I could probably cook myself up some use of the >127 and make it work.
2022-04-30 04:44:28 <KipIngram   > I might even be able to do that with the APL symbols, but that feels kind fo back-flippy.
2022-04-30 04:45:06 <veltas      > Again, escape sequences are some kind of multi-byte encoding
2022-04-30 04:45:11 <KipIngram   > Also, if I do either thing that way I then have data that I can't look at very easily outside of my Forth.
2022-04-30 04:45:24 <KipIngram   > Right - that's how they come in.
2022-04-30 04:45:40 <veltas      > I will say that a lot of parsing and processing stuff in literature makes you think "if I can break this into tokens it will be easier"
2022-04-30 04:45:54 <KipIngram   > I find them to be particularly ad hoc - everything from two bytes to six or seven.
2022-04-30 04:46:00 <KipIngram   > Surely there was a better way...
2022-04-30 04:46:08 <veltas      > But really you shouldn't ever actually physically break into tokens in memory, at most you should have control flow tied to tokens like words that parse a token etc
2022-04-30 04:46:16 <veltas      > But don't ever actually put tokens in an array etc
2022-04-30 04:46:25 <veltas      > Real compilers don't do this either
2022-04-30 04:46:33 <KipIngram   > Well, I didn't invent either of these things.
2022-04-30 04:46:59 <KipIngram   > I just think it was really ill-advised to begin those escape sequences with a byte that there is a KEY FOR on the keyboard.
2022-04-30 04:47:05 <KipIngram   > Escape is a damn nusance.
2022-04-30 04:47:08 <KipIngram   > nuisance
2022-04-30 04:47:28 <KipIngram   > To properly handle those keys I'm going to have to TIME the keyboard input.
2022-04-30 04:47:32 <veltas      > Yeah it made it unnecessarily complicated perhaps, but it doesn't make parsing fundamentally harder
2022-04-30 04:47:47 <veltas      > And unfortunately "KEY" might be a bit too simplistic a model
2022-04-30 04:47:48 <KipIngram   > No, it doesn't.  It's just a state machine.
2022-04-30 04:47:57 <KipIngram   > Just one that's more complicated than it probably needed to be.
2022-04-30 04:48:11 <KipIngram   > I'm sure it "made sense at the time," given whatever constraints whoever did it was under.
2022-04-30 04:48:17 <veltas      > But I think KEY will just have to parse potentially more than one character when it reaches the start of a multi-byte etc
2022-04-30 04:48:45 <KipIngram   > Yes, I have KEY and EMIT stand-alone working just fine for UTF8 now.
2022-04-30 04:48:49 <veltas      > Probably the way that vim does it, with a (noticeable) timeout if the sequence doesn't complete fast enough
2022-04-30 04:48:53 <KipIngram   > UTF8 is nice and systematic and wasn't that hard to code.
2022-04-30 04:48:59 <KipIngram   > And it's clearly recognizable.
2022-04-30 04:49:11 <KipIngram   > You can set that timeout.
2022-04-30 04:49:18 <veltas      > Yeah I need to set it lower
2022-04-30 04:49:20 <KipIngram   > I turned mine down because I could notice it.
2022-04-30 04:49:28 <veltas      > No machine is going to be that slow
2022-04-30 04:49:38 <KipIngram   > I think it was screen that was doing it to me - I had to adjust it in .screenrc.
2022-04-30 04:49:43 <veltas      > vim's timeout is almost so long it's designed for you to manually enter an escape sequence
2022-04-30 04:49:57 <veltas      > Yeah maybe it's tmux
2022-04-30 04:49:58 <KipIngram   > Yes - that makes sense.
2022-04-30 04:50:02 <KipIngram   > But I don't need to do that.
2022-04-30 04:50:06 <veltas      > Yup
2022-04-30 04:50:23 <KipIngram   > At least all UTF8 characters are clearly outside of the ASCII codes.
2022-04-30 04:51:03 <veltas      > Yeah it makes JSON easier to parse
2022-04-30 04:51:17 <KipIngram   > Anyway, my hope yesterday didn't quite materialize.  Turns out I did use one variable in expect to represent both "cursor distance right of starting point" and "working position in the string buffer."
2022-04-30 04:51:23 <KipIngram   > That's going to have to be split into two items.
2022-04-30 04:51:51 <KipIngram   > I was going to have to do that for my block editor anyway, because I want to support having color and attribute sequences in the edit buffer.
2022-04-30 04:52:11 <veltas      > Have you looked at extended character stuff in the forth standard?
2022-04-30 04:52:18 <KipIngram   > So this may make the editor more straightforward.
2022-04-30 04:52:41 <KipIngram   > No, I'm afraid I've mostly just diddled with Forths of my own over the years.
2022-04-30 04:53:19 <KipIngram   > I did my learning mostly on FIG and Forth 79, and somewhere upstairs I have a copy of the 79 standard, and maybe a copy of the 83 standard.
2022-04-30 04:53:33 <KipIngram   > Since then, though, I've been largely "detached."
2022-04-30 04:54:02 <KipIngram   > My Forth is probably more like FIG than anything else.  Most of the differences are things I've invented on my own.
2022-04-30 04:54:18 <veltas      > Well look at it and see how they've structured handling UTF-8, because that's really what it's designed for
2022-04-30 04:54:35 <KipIngram   > "invented" - you know what I mean - I conceieved it myself, but turned out someone else had done it at some point.  :-)
2022-04-30 04:54:43 <KipIngram   > The kind of inventions that make you no profit.
2022-04-30 04:56:01 <KipIngram   > Well, ok, but what am I going to get from it?  Is there going to be someway to have word names comprised of APL characters without actually supporting those characters in EXPECT?
2022-04-30 04:56:19 <KipIngram   > Conveniently and without jumping extra hoops?
2022-04-30 04:56:31 <KipIngram   > Just handling it in EXPECT seemed like the most straightforward way to me.
2022-04-30 04:57:00 <veltas      > Well they might have made a couple interface decisions that are better than what you're doing
2022-04-30 04:57:12 <veltas      > You wouldn't want standard forth to be better than your forth right?
2022-04-30 04:57:12 <KipIngram   > If I do this right, then I'll be able to use the major components of EXPECT to handle "each line" in the editor.
2022-04-30 04:57:27 <KipIngram   > :-)
2022-04-30 04:57:29 <veltas      > No harm in looking anyway
2022-04-30 04:57:40 <KipIngram   > Ok - where exactly do you recommend looking?
2022-04-30 04:57:55 <KipIngram   > And no - I have no shame whatsoever about taking good ideas wherever I find them.
2022-04-30 04:59:04 <KipIngram   > The big working part of EXPECT is READ.  It's designed to handle a whole bunch of characters - "line" editing characters - and return if it gets a character it can't handle.  But it's designed to be "re-entrant."
2022-04-30 04:59:20 <KipIngram   > I can start it on a buffer that already has text in it, at a specified cursor position.
2022-04-30 04:59:41 <KipIngram   > So the editor should just wrap that in something that handles the non-line-oriented characters.
2022-04-30 05:00:09 <KipIngram   > And QUERY will wrap it with a command history handler.
2022-04-30 05:00:52 <KipIngram   > Or if you can give me some effective google search terms - that would help too.
2022-04-30 05:02:53 <dave0       > KipIngram: do you know this site?   https://vt100.net/   https://vt100.net/emu/    it has a lot of information on vt100 emulators like xterm, it's mostly about displaying control codes, but it's also got recieving control codes
2022-04-30 05:03:24 <KipIngram   > I am thinking about just re-writing EXPECT (or READ, rather) instead of trying to "adjust" what I've got.  It was awfully close to being fine, but this merging of those two different things is a little deeply engrained in it.
2022-04-30 05:03:37 <KipIngram   > So I may as well re-write it as smart as possible.
2022-04-30 05:03:55 <KipIngram   > No, I wasn't familiar with that one - thanks.
2022-04-30 05:04:43 <KipIngram   > I tried once to "nudge" it last night, but eventually broke it and reverted back to a saved source version.
2022-04-30 05:05:11 <KipIngram   > I'm sure if I re-write I'll harvest a lot of the helper words I've already got - a lot of it doesn't need any change.
2022-04-30 05:07:32 <KipIngram   > Oh, I like how that site is laid out.  Bet that one won't spin up my fan.  :-)
2022-04-30 05:07:41 <KipIngram   > Looks nice and traditional.
2022-04-30 06:14:35 <KipIngram   > Oh... well, that's kind of cute...
2022-04-30 06:14:38 <KipIngram   > x:4746454443424140 emit @ABCDEFG ok
2022-04-30 06:15:13 <KipIngram   > Unplanned side effect of extending EMIT to support UTF8 chars.
2022-04-30 06:16:20 <KipIngram   > As long as the cell handed to emit is "just a byte," it works exactly as always.  But those seven bytes ARE there...
2022-04-30 06:17:23 <dave0       > KipIngram: ehehehe nice feature :-)
2022-04-30 06:18:51 <KipIngram   > Have to be sure the high byte is never >127, though - then it would hang.
2022-04-30 06:19:07 <KipIngram   > I'd never get to 0.
2022-04-30 06:20:20 <KipIngram   > -1 8 >> . -1  ok
2022-04-30 06:21:42 <KipIngram   > Maybe I need u>> and use that in emit instead of >>
2022-04-30 06:24:23 <KipIngram   > Good.  Did that.
2022-04-30 06:24:41 <dave0       > i belive java uses >>> for unsigned right shift
2022-04-30 06:25:12 <dave0       > but on the other hand forth uses u prefix for unsigned like u<
2022-04-30 06:25:17 <dave0       > or um*
2022-04-30 06:25:50 <KipIngram   > Right.  I have u versions of all of my conditional returns and stuff.
2022-04-30 06:26:05 <dave0       > ah cool
2022-04-30 06:35:19 <KipIngram   > Looks like rshift in gforth is unsigned.
2022-04-30 06:35:26 <KipIngram   > logical shift
2022-04-30 06:35:53 <KipIngram   > In nasm you have shr and sar
2022-04-30 06:35:55 <dave0       > yeah that's in the standard
2022-04-30 06:36:51 <dave0       > nasm seems nice, i made my sectorforth homage compatible with nasm
2022-04-30 06:49:19 <KipIngram   > I'm really pleased with nasm's macro facilities.
2022-04-30 06:49:37 <KipIngram   > They do all of my dictionary construction heavy lifting.
2022-04-30 06:49:49 <KipIngram   > God, I used to do all that by hand.
2022-04-30 07:09:03 <KipIngram   > Ugh.  Dead battery in one of the cars.  It wouldn't crank yesterday - I put it on the charger overnight, but it still won't.
2022-04-30 07:09:12 <KipIngram   > Guess I'm going to the auto shop later.
2022-04-30 07:19:43 <KipIngram   > You know, things like this READ word can just really open a can of worms.  It seems like such a nice, self-contained function.  But... exactly how do I want to use it?  Do I want to use it in a typical scrolling tty type environment?  Or do I want to use it in a situation where I've got the screen divided into several panels and I want each one to behave like a self-contained scrolling tty?
2022-04-30 07:20:21 <KipIngram   > If the latter, then a lot of intelligence about what parts of the screen it's allowed to draw on wind up having to be considered, and exactly how to set that up is an involved thought process.
2022-04-30 07:20:47 <KipIngram   > The standard answer a lot of people would give would be "use ncurses."
2022-04-30 07:21:08 <KipIngram   > But I'm trying not to use any libraries - I'm trying to write something that can port to a bare metal environment.
2022-04-30 07:21:29 <KipIngram   > Although, even in those cases I'll often be connecting with a terminal emulator, so...
2022-04-30 07:22:04 <KipIngram   > I can try to "think of everything up front," but that most likely will be wrong and isn't very "Forth honoring."
2022-04-30 07:23:20 <KipIngram   > So, do I want it to have a boundary on the right that it won't go beyond?  And if I do, what do I do with lines that are longer than that?  To I write it so they horizontal scroll?
2022-04-30 07:23:25 <KipIngram   > Lot of little things to consider.
2022-04-30 07:25:18 <KipIngram   > Forth says don't try to solve problems you don't have yet.
2022-04-30 07:25:25 <KipIngram   > Just plan to re-write.
2022-04-30 08:55:47 <KipIngram   > You know, I don't actually see why my cursor is getting out of whack to start with.  Printing those three bytes (a utf8 sequence) only moves the cursor one notch.  I can't tell where the other motion is even coming from.
2022-04-30 08:56:10 <KipIngram   > I do print my whole string between keystrokes, but I save/hide and restore/unhide the cursor around that.
2022-04-30 08:56:25 <KipIngram   > Obviously it's in there somehwere - just gotta find it.
2022-04-30 09:19:04 <veltas      > KipIngram: The extended character wordset section, in Forth 2012 or a more recent pdf
2022-04-30 10:41:36 <KipIngram   > I've got it mostly working now.
2022-04-30 10:42:05 <eris[m]     > thinking about the design of an array forth
2022-04-30 10:42:09 <KipIngram   > Straight typing works with regular and utf8 chars.  Cursor control, backspace, home, end. "didn't break" for regular characters.
2022-04-30 10:42:22 <KipIngram   > So all that's  left is to carry those last items to utf8 as well.
2022-04-30 10:42:37 <KipIngram   > Oh fun - that's a better thing to talk about.
2022-04-30 10:42:43 <KipIngram   > Please share when you're ready.
2022-04-30 10:42:46 <eris[m]     > the main thing is doing nesting without tags
2022-04-30 10:43:19 <eris[m]     > oh, ive not done anything yet
2022-04-30 10:43:26 <eris[m]     > im just toying with the idea
2022-04-30 10:43:32 <KipIngram   > That's the best time to talk.  :-)
2022-04-30 10:44:28 <eris[m]     > scalar (unnested) maps and folds are easy enough with the machineforth registers
2022-04-30 10:45:14 <eris[m]     > assuming that the array address is in a and the xt in b, the core loops are as follows
2022-04-30 10:46:12 <eris[m]     > : map a@ b execute a!+ ;
2022-04-30 10:46:35 <eris[m]     > : fold a@+ b execute
2022-04-30 10:47:22 <eris[m]     > doing nesting is... harder
2022-04-30 10:52:41 <eris[m]     > of course, this isnt particularly forthy
2022-04-30 10:53:23 <eris[m]     > i feel you'd be better off with adding defining words that introduce iteration
2022-04-30 10:53:59 <eris[m]     > like,
2022-04-30 10:54:09 <eris[m]     > fold: sum + ;
2022-04-30 10:56:59 <KipIngram   > Yeah, that's the impression I've gotten of array stuff from APL.  Obviously there's still an iteration going on -  it's just hidden from of us.
2022-04-30 10:58:42 <eris[m]     > mhm
2022-04-30 10:58:50 <eris[m]     > that is the entire point
2022-04-30 10:59:16 <KipIngram   > they've got "bracket" indexing, but they seem to shun it.
2022-04-30 10:59:22 <KipIngram   > As much as possible.
2022-04-30 11:00:45 <eris[m]     > besides, an array model seems... almost wasteful?  when you have decent primitives
2022-04-30 11:01:20 <KipIngram   > I think I know what you mean - it does sometimes seem like that "array style convenience" carries some excess labor with it.
2022-04-30 11:01:30 <eris[m]     > makes sense in idk, ans, but machineforth's address register is simply delightful
2022-04-30 11:01:36 <KipIngram   > Easy to "do," but it's going to not be a terribly efficient computation.
2022-04-30 11:01:56 <KipIngram   > Yeah, the address registers were a good idea.
2022-04-30 11:02:49 <eris[m]     > just need to find something a bit nicer than do...loop and begin...end
2022-04-30 11:02:50 <KipIngram   > I'm going to use si and di for those, since they work with the rep instructions.  I haven't done much with them yet, though.
2022-04-30 11:03:58 <eris[m]     > is there any way to do postincrement on x86?
2022-04-30 11:04:34 <KipIngram   > Yes, rsi and rdi will do that with the movsb, lodsb, stosb, and some other instructions.
2022-04-30 11:05:04 <KipIngram   > Those instructions will increment or decrement the registers, and you can put rep in front of them and do it rcx times.
2022-04-30 11:05:21 <eris[m]     > huh
2022-04-30 11:06:12 <KipIngram   > It's actually movs lods stos - the b means you work on bytes, increment by 1.  You can go all the way up to movsq, lodsq, etc.
2022-04-30 11:06:19 <KipIngram   > Which is 64 bits at a shot.
2022-04-30 11:06:30 <KipIngram   > That adjusts the increment size.
2022-04-30 11:06:43 <KipIngram   > But you can't do any arithmetic or anything that way.
2022-04-30 11:07:03 <KipIngram   > You could lodsq do math stosq, but that can't be "repped."
2022-04-30 11:07:57 <KipIngram   > Have you looked into the SSE/AVX instructions?
2022-04-30 11:08:06 <KipIngram   > Those work with special 'vector' registers.
2022-04-30 11:08:57 <eris[m]     > so a@+ can be lodsw reg push eax
2022-04-30 11:08:58 <eris[m]     > roughly?
2022-04-30 11:09:11 <eris[m]     > i try to avoid thinking about sind
2022-04-30 11:10:12 <KipIngram   > Yeah, lodsq for 64 bits.  That gets an 8-byte item into rax, which you can then add to your top of stack register.
2022-04-30 11:10:23 <KipIngram   > Which is probably in another register.
2022-04-30 11:10:43 <KipIngram   > So for me it would be lodsq; add rcx, rax; next
2022-04-30 11:11:35 <KipIngram   > But you could also use the AVX stuff to do that to eight elements of a vector in one go.  If you had like a "vector stack" and one of hte vector registers was the top of it.
2022-04-30 11:11:48 <eris[m]     > the (E)SI register is incremented or decremented automatically according to the setting of the DF flag in the EFLAGS register.
2022-04-30 11:11:54 <eris[m]     > nmmmmmmm
2022-04-30 11:11:56 <eris[m]     > no like
2022-04-30 11:12:05 <KipIngram   > Yeah.
2022-04-30 11:12:14 <KipIngram   > CLD clears it - you get increment.
2022-04-30 11:12:42 <KipIngram   > The only time I've ever used decrement is for moving a string to a higher address that overlaps the starting string.
2022-04-30 11:12:52 <KipIngram   > You start that at the high end and work down.
2022-04-30 11:13:40 <eris[m]     > theres no real reason to have a@- so i guess i dont have to worry
2022-04-30 11:14:13 <KipIngram   > Yeah, that overlapping string thing is the only case I know.
2022-04-30 11:14:33 <KipIngram   > But, you do need to do exactly that to open up a spot in the middle of a string to insert into.  Nice to be able to do that with rep.
2022-04-30 13:09:19 <KipIngram   > cursor left and right and home seem to be working.
2022-04-30 13:09:27 <KipIngram   > Backspace and "end" to go.
2022-04-30 13:09:38 <KipIngram   > I hardly ever use home and end anyway - don't really know why I put them in.
2022-04-30 13:11:18 <KipIngram   > eris: I've been interested in the SSE/AVX stuff recently.  Really seems ike that's probably the way to go an an array Forth.  Either that or GPU.
2022-04-30 13:33:36 <Zarutian_HTC> KipIngram: does SSE have scatter and gather instructions?
2022-04-30 13:33:56 <KipIngram   > Yes; I'm not an expert or anything, but I did notice that it had those.
2022-04-30 13:35:45 <Zarutian_HTC> conditional select or move? that is given a bitvector, source vector, and destination vector, the src[i] gets copied into dest[i] iff bitvector[i] is 1 m
2022-04-30 13:36:06 <Zarutian_HTC> s/ m/?/
2022-04-30 13:39:14 <KipIngram   > It has an emphasis on working with masks.  The article I read talked all about how to use mask vectors to selectively do operations on elements of a vector.
2022-04-30 13:39:35 <KipIngram   > Let me see if I can find the article; one sec.
2022-04-30 13:39:56 <Zarutian_HTC> s/bitvector/mask/
2022-04-30 13:40:53 <Zarutian_HTC> this ties in with that implement Forth VMs with SIMD
2022-04-30 13:43:47 <KipIngram   > This isn't the one I wanted to find, but it's one I did read the other day:
2022-04-30 13:43:48 <KipIngram   > http://www.cs.uu.nl/docs/vakken/magr/2017-2018/files/SIMD%20Tutorial.pdf
2022-04-30 13:55:36 <Zarutian_HTC> I rather write directly in macro assembler than C or variants
2022-04-30 13:59:16 <Zarutian_HTC> just a stray tought I had: use that GPT-3 Copilot thing to translate psuedo code into compilable code
2022-04-30 14:30:01 <KipIngram   > Looks to me like there is just know way to handle the "end" key (that sends you to the end of the line) without actually scanning the string.
2022-04-30 14:30:45 <KipIngram   > For cursor left you know your column on the screen is going to increase by 1, but for "jump to end" you don't know how far it's going to move, and you can't without looking at every character between you and there.
2022-04-30 14:32:10 <KipIngram   > Ugh.
2022-04-30 14:32:21 <KipIngram   > NO way.  No way, not know way.  :-(
2022-04-30 17:44:50 <eris[m]     > KipIngram: i dont feel like simd is the way to go in forth, it's too abtruse
2022-04-30 17:48:39 <eris[m]     > aww
2022-04-30 17:48:54 <eris[m]     > do you not know the length of the string you're in?
2022-04-30 19:21:24 <KipIngram   > I found it.
2022-04-30 19:23:09 <KipIngram   > Of course it was obvious and simple once I saw it - that's how it almost always is.
2022-04-30 19:29:18 <eris[m]     > what was it?
2022-04-30 19:34:23 <Zarutian_HTC> we are agog
2022-04-30 19:36:18 <KipIngram   > It was stupid.  I re-print my line on each keystroke, and for some reason quite a long time ago I printed two spaces after.  This was being called inside a higher level cursor save restore - I just wanted to clean off characters left on the screen when I shortened a string.
2022-04-30 19:36:42 <KipIngram   > As part of what I was doing more recently I called that word from another place, but this time I wanted the cursor motion it was going to give me.
2022-04-30 19:36:53 <KipIngram   > I did NOT want the two spaces.
2022-04-30 19:37:07 <KipIngram   > I just moved the two spaces up a level in the call tree where I did want them.
2022-04-30 19:38:01 <KipIngram   > So that when I call from the new place I get only the actual string print.
2022-04-30 19:38:23 <eris[m]     > ohhh
2022-04-30 19:38:25 <KipIngram   > This is all connected with the fact that you can no longer CALCULATE a large cursor jump.
2022-04-30 19:38:34 <KipIngram   > Not if some of the chars in between may be utf8.
2022-04-30 19:38:46 <KipIngram   > There's no way to deal with it withough handling ever character.
2022-04-30 19:38:53 <KipIngram   > every
2022-04-30 19:39:14 <KipIngram   > Which of course is totally obvious once you pin the idea down.
2022-04-30 20:17:29 <Zarutian_HTC> here is an idea, write a word that given an utf-8 string returns an bytearray of how many bytes each codepoint consumes
2022-04-30 21:11:18 <KipIngram   > I think it depends just on how many bits is required to write the codepoint down.
2022-04-30 21:11:36 <KipIngram   > You then encode using the fewest number of bytes that will hold that many data bits.
2022-04-30 21:15:43 <Zarutian_HTC> yebb, but the word is simpler, it just goes through the utf-8 string and tallies up the bytes used by each codepoint, even if some funkyness occured at encoding time
2022-04-30 21:26:01 <KipIngram   > Well, how does it do that tally?  This was all described in the wikipedia article.  According to here: https://en.wikipedia.org/wiki/UTF-8#Encoding, one byte gets any 7-bit code, 2 bytes gets any 8-11 bit code, 3 bytes gets any 12-16 bit code, and four bytes gets any 17-21 bit code.  It's required to use the shortest possible byte count.  Nothing has more than 21 bits in its code point value.
2022-04-30 21:26:27 <KipIngram   > So I would just note the bit position of the highest 1 and apply those ranges.
2022-04-30 21:27:15 <KipIngram   > I think EXPECT is working properly on UTF-8 now, though all I've done is just the trickiest test patterns I could think of on the fly.  No sort of "exhaustive" verification.
2022-04-30 21:27:31 <KipIngram   > But all the editing keys seem to work as one would expect.
2022-04-30 21:30:36 <KipIngram   > I wound up handling "end of line" by computing "string length minus insert point" and calling my cursor right routine that many times.  That "tacitly" scans the pertinent part of the string, since cursor right has to check to make sure it doesn't stop over a "not first" byte of a character.
2022-04-30 21:34:37 <KipIngram   > So, back when ASCII was first conceived and deployed, why were the averse to having the MSB of the char bytes set?  Was that used as an extra parity bit or something?
2022-04-30 21:35:01 <KipIngram   > To me these days it feels like they just threw away half their code space.
2022-04-30 21:35:51 <KipIngram   > I'm not young, but I'm not THAT old - I wasn't around for that.  :-)
2022-04-30 21:37:21 <Zarutian_HTC> 
ACTION hands KipIngram an 7 hole papertape from ASR 35 as a explanation

2022-04-30 21:37:46 <Zarutian_HTC> I was curious about it too
2022-04-30 21:39:39 <KipIngram   > AHA.
2022-04-30 21:39:43 <KipIngram   > That makes sense!
2022-04-30 21:39:50 <KipIngram   > That's a pleasing explanation.
2022-04-30 21:40:03 <KipIngram   > Good god, and we're STILL carrying the baggage.
2022-04-30 21:40:17 <KipIngram   > That's the same thing we were talking about re: processors the other day.
2022-04-30 21:40:25 <KipIngram   > The burdens of legacy.
2022-04-30 21:40:59 <Zarutian_HTC> yeah the octal fetish of the original x86 still shines through
2022-04-30 21:41:54 <Zarutian_HTC> and I say nothing of the wierdness of ARM, 65C816, PowerPC, or RISC-V
2022-04-30 21:44:11 <Zarutian_HTC> hence why I am so interested in BitGrid, and making a BitGrid based fpga/cpld using sram (or fram/memristors) for the memory cells
2022-04-30 21:45:38 <Zarutian_HTC> I think open source design of BitGrid chips would be handy, specially for the opensource ecology project
2022-04-30 21:47:15 <Zarutian_HTC> even at micron node (read: minimum feature size) such a design will be quite usefull
2022-04-30 21:47:59 <KipIngram   > I am all for open source / grass roots anything.  The more things we can pry out of the "exclusive hands" of big corporations the better.
2022-04-30 21:48:26 <KipIngram   > I've always so wished that the DIY grassroots IC fabrication stuff could do better.
2022-04-30 21:48:34 <Zarutian_HTC> specially if you can do corondium (aloxide) or rubium (copperoxide) substrate based chips for rad harded chips
2022-04-30 21:48:39 <KipIngram   > It's "there" - there are some projects - but they're woefully behind the curve.
2022-04-30 21:49:14 <KipIngram   > Which means the corporations get to dictate to us what we get to do with silicon.
2022-04-30 21:49:49 <Zarutian_HTC> what I have in mind are chips fabricatable in garage type setup
2022-04-30 21:49:57 <KipIngram   > YES.
2022-04-30 21:50:00 <KipIngram   > Me too.
2022-04-30 21:50:37 <KipIngram   > I'd like to see "kitchen witches" making chips and gadgets and stuff.
2022-04-30 21:50:54 <Zarutian_HTC> not the bleeding edge, more toward blunt stone ax direction, (See John Tree of Primitive Technology on youtube)
2022-04-30 21:51:10 <KipIngram   > That's really where all of the truly truly great ideas come from most of the time anyway.
2022-04-30 21:51:53 <KipIngram   > Yeah, I understand that we're not going to suddenly be making state of the art stuff at home.  But if we could even do, say, 20 years back tech.  That would be fabulous.
2022-04-30 21:52:04 <KipIngram   > I'm talking here about process geometry.
2022-04-30 21:52:37 <KipIngram   > And the advantage we'd have is that we don't need huge yield on massive production runs.  We just need to be able to get a working chip without TOO much trouble.
2022-04-30 21:52:50 <KipIngram   > We don't need production quantity scale gear.
2022-04-30 21:53:20 <KipIngram   > How about an ion implantation gun we can maneuver like a printer head?
2022-04-30 21:53:26 <Zarutian_HTC> what about process geometry? I think there are good enough designs from the 80's or 90's available from universities
2022-04-30 21:53:30 <KipIngram   > Just "paint" our chips.
2022-04-30 21:53:45 <KipIngram   > Inside a chamber the size of an aquarium or something.
2022-04-30 21:54:11 <KipIngram   > I haven't checked in on it in quite a while.
2022-04-30 21:54:27 <KipIngram   > The last time I did, the "bragging project" was etching someone's name on a wafer.
2022-04-30 21:54:45 <Zarutian_HTC> oh, I see. You can use cathodray gun for the implantation and even electrography
2022-04-30 21:54:49 <KipIngram   > Pretty small, but... it also wasn't terribly pretty.
2022-04-30 21:55:16 <KipIngram   > I need to dip back into that.  It really is something I get pretty excited about.
2022-04-30 21:56:00 <KipIngram   > This is the tech that has changed the world more than any other in the last few decades - we need to grass roots it.