2023-02-19 00:39:46 <KipIngram   > Wow.
2023-02-19 00:39:56 <KipIngram   > fzf is reall cool.
2023-02-19 00:40:13 <KipIngram   > Try it out - install the fzf package, and then from your ~ director run
2023-02-19 00:40:17 <KipIngram   > vim $(fzf)
2023-02-19 00:40:37 <KipIngram   > It'll bring up a list of pretty much your whole file tree, and then you can do a "fuzzy search" on those.
2023-02-19 00:40:54 <KipIngram   > You just type anything you can remember, and as you type words it'll scope down the remaining list.
2023-02-19 00:41:09 <KipIngram   > Soon as you can see what you want, cursor to it and hit enter, and it'll open that file in vim.
2023-02-19 00:41:39 <KipIngram   > If you want an exact literal match, start off your typing with '
2023-02-19 00:41:53 <KipIngram   > But then you're doing something you could do pretty easily with locate.
2023-02-19 00:42:04 <KipIngram   > It's the fuzzy aspect that makes it really powerful.
2023-02-19 00:42:25 <KipIngram   > You can use it more or less anywhere you'd type a pathname.
2023-02-19 00:56:42 <KipIngram   > Hah - just installed fzf on my irc machine here.  What a great way to search for stuff in logs.
2023-02-19 00:57:14 <KipIngram   > I just aimed it at our log and typed bitgrid and it went right to all of our mentions of that.
2023-02-19 00:57:56 <KipIngram   > For a while now I've had all of my Dresden Files books text-exported and I have a hotkey that will pop open an fzf console window that starts with the whole collection - one line per paragraph.
2023-02-19 00:58:10 <KipIngram   > I can find practically anything in seconds.
2023-02-19 01:01:40 <KipIngram   > If you install it from github it will offer to integrate into your system in a variety of ways.
2023-02-19 01:01:50 <KipIngram   > https://github.com/junegunn/fzf#using-linux-package-managers
2023-02-19 01:01:58 <KipIngram   > Ooops.
2023-02-19 01:02:09 <KipIngram   > https://github.com/junegunn/fzf
2023-02-19 01:02:11 <KipIngram   > Better.
2023-02-19 01:02:49 <KipIngram   > Unfortunately it seems to have a limit on how much it can load up at once - I wasn't able to just throw everything in my weechat logs directory at it.
2023-02-19 01:02:55 <KipIngram   > Got an "argument list too long" error.
2023-02-19 01:03:09 <KipIngram   > I couldn't even throw all of the ##forth log at it.
2023-02-19 01:03:18 <KipIngram   > I had to tail off a bunch of it.
2023-02-19 01:05:27 <KipIngram   > Oh, I take it back - it will absorb all of the logs.
2023-02-19 01:05:33 <KipIngram   > I just had to do it the right way.
2023-02-19 01:05:40 <KipIngram   > cat *.weechatlog | fzf
2023-02-19 01:06:02 <KipIngram   > Whereas fzf <$(cat *.weechatlog) is no bueno.
2023-02-19 01:06:52 <KipIngram   > cd
2023-02-19 01:07:02 <KipIngram   > Sorry.
2023-02-19 04:13:01 <veltas      > man page isn't very good to be fair
2023-02-19 04:13:40 <veltas      > It's not obvious from man page how to use it, doesn't have instructions anywhere, typically badly documented unix tool
2023-02-19 04:13:53 <veltas      > I guess these days people just google everything
2023-02-19 10:58:14 <KipIngram   > Yeah
2023-02-19 11:21:07 <KipIngram   > I really caught on that damn interestng things could be done with it by seeing examples online, not by reading its docs.
2023-02-19 11:21:31 <KipIngram   > What's nice about it is it's so damn fast and responsive.
2023-02-19 11:23:30 <thrig       > cat *.blah will (eventually) run into ARGV_MAX, even if Linux did embiggen that buffer so Chrome could get off the ground
2023-02-19 12:08:31 <KipIngram   > See, that seems  poor to me.  These are *streams* - even if only a portion of the data can be handled at a time, it should still be possible to stream an arbitrary size file through a pipeline.
2023-02-19 12:09:04 <KipIngram   > There's not a *need* to hold the entire file at once at any stage of the process.
2023-02-19 12:12:00 <thrig       > no, the *.blah expansion is not a stream
2023-02-19 12:12:24 <KipIngram   > It's still something that could be processed incrementally, though.
2023-02-19 12:12:41 <thrig       > no, *.blah is first expanded by the shell, then passed to cat
2023-02-19 12:12:46 <KipIngram   > I'm not denying that the extant implementation may have limits.
2023-02-19 12:13:17 <thrig       > with cat \*.blah then cat could incrementally expand things (which is how DOS tools handle it)
2023-02-19 12:13:18 <KipIngram   > Ok.  But conceptually what I'm wanting to do is clear, and it would be better for it to be achievable.
2023-02-19 12:13:45 <KipIngram   > Ok.  Like I said, I'm not questioning your statement of "how it is."
2023-02-19 12:13:59 <KipIngram   > But maybe it could be better.
2023-02-19 12:15:56 <KipIngram   > I mean, it's obvious that whether it works or not depends on exactly how I write it, because the first way I tried it last night didn't work, but an alternate way did.
2023-02-19 12:16:27 <KipIngram   > cat * | fzf wound up working, though.
2023-02-19 12:17:13 <KipIngram   > I've had this weechat sitting here, sometimes with over a dozen channels open, 24/7 for years, for the most part, so there's quite a lot of stuff in that log directory.
2023-02-19 12:17:17 <thrig       > it will break when too many files are present (or also **environ is too big, which is less likely)
2023-02-19 12:17:21 <KipIngram   > Lemme see how much...
2023-02-19 12:18:24 <KipIngram   > Looks like about 57 MB.
2023-02-19 12:18:40 <KipIngram   > Which is a lot of text, but... not a particularly huge amount of data.
2023-02-19 12:19:10 <KipIngram   > kip@localhost:~/.weechat$ cat logs/* | wc
2023-02-19 12:19:12 <KipIngram   >  689337 8964767 58998205
2023-02-19 12:32:08 <KipIngram   > I do see what you're saying, though, and yes - a generic bash process may need access to all of its command line parameters at once, so processing the list of them incrementally isn't sufficient.
2023-02-19 13:13:48 <thrig       > strace it and see what happens in some exec* call
2023-02-19 14:37:26 <KipIngram   > So, has anyone ever given thought to the notion of having the dictionary (I mean the header parts - not the compiled bodies) be treated virtually and reside in block space?
2023-02-19 14:37:56 <KipIngram   > That would make it possible on a limited memory system to swap that stuff out during execution of a memory hungry program.
2023-02-19 14:38:24 <KipIngram   > I haven't really plumbed into it - just had it occur to me.
2023-02-19 14:44:00 <KipIngram   > In my design the header region is a separate RAM region anyway, so treating offsets within that region as virtual disk addresses would be straightforward.
2023-02-19 14:44:37 <KipIngram   > What triggered the thought in the first place was musing that searching a file directory for a particular name is an essentially similar task to searching a Forth vocabulary for a particular word name.
2023-02-19 14:45:40 <KipIngram   > Instead of the search yielding an xt (a table offset), it would yield an inode type value.
2023-02-19 14:58:34 <KipIngram   > So, does the total number of available inodes get set at format time?
2023-02-19 14:58:52 <KipIngram   > Or maybe I should say file system creation time?
2023-02-19 15:00:17 <thrig       > file creation time isn't available on various unix filesystems
2023-02-19 15:00:39 <KipIngram   > No, I mean when you first establish the file system, not when you create an individual file.
2023-02-19 15:00:45 <KipIngram   > mkfs
2023-02-19 15:01:00 <KipIngram   > Does mkfs reserve a particular amount of space in which to save inodes?
2023-02-19 15:01:11 <thrig       > oh, lots of filesystems set a particular number of inodes at that time
2023-02-19 15:01:21 <thrig       > others (ZFS) are dynamical about it
2023-02-19 15:01:23 <KipIngram   > Ok.
2023-02-19 15:01:40 <KipIngram   > Yeah, I know it is possible in some cases to run out, though it's not a very often encountered situation.
2023-02-19 15:02:08 <thrig       > there's an IBM filesystem intended for big files that has limited inodes; this went... poorly... with genome data who have billions of files
2023-02-19 15:02:08 <KipIngram   > /dev/nvme0n1 on my notebook reports it has 2^20 total inodes.
2023-02-19 15:02:48 <KipIngram   > I figure if I were doing it, I'd set aside a continguous range of blocks to hold that information when I first set up a file system.
2023-02-19 15:03:38 <KipIngram   > Which is one of those "hard to swallow" pills, because that's a fair amount of space that you've just assigned, before you ever create even one piddly file.
2023-02-19 15:04:11 <thrig       > the genome scientists also complained about the filesystem overhead tax
2023-02-19 15:04:37 <thrig       > sad times if you lost that one superblock you needed
2023-02-19 15:04:46 <KipIngram   > The pain of one-size-fits-all approaches...
2023-02-19 15:05:21 <thrig       > or where mysql was hella fast, until it went sideways and needed a 12 hour recovery
2023-02-19 15:05:43 <KipIngram   > Yeah, sometimes a database needs some tlc.
2023-02-19 15:05:55 <KipIngram   > Usually you need to take the time to make some new indices.
2023-02-19 15:06:21 <KipIngram   > When MySQL is shining it can be pretty impressive.
2023-02-19 15:06:43 <thrig       > a race car can also be pretty impressive
2023-02-19 15:22:14 <KipIngram   > I've occasionally lamented the fact that creation time isn't stored - there are times that would be useful.
2023-02-19 15:36:36 <KipIngram   > So, if I create a new file in Linux, and open it and use seek to write some data, say, 1MB into the file, and then close it, ls will tell me I have about a 1MB file.  But has Linux actually allocated blocks for all that empty space?  Or does it do that only when I actually put data on one of the pages?
2023-02-19 15:37:28 <KipIngram   > Does it have any notion of "empty" space in a file?
2023-02-19 15:38:11 <KipIngram   > Seems to me that if you wanted to work with a big disk-resident hash table, for example, that it would be nice if it allocated space only for atually written data.
2023-02-19 15:51:06 <KipIngram   > We used to not only track  "not yet written" pages, but also explicitly detect written zero pages in our drive, and rather than actually write anything to flash we'd just set a bit in the LPT.
2023-02-19 15:51:29 <KipIngram   > Makes those pages really fast to read, since you don't have to muck about with the flash at all.
2023-02-19 15:51:47 <KipIngram   > Somewhere along the way we dropped the new zero page detection and now we only do that for not yet written pages.
2023-02-19 15:52:06 <KipIngram   > There was some reason having to do with RAID that drove that change, but I can't quite remember what it was.
2023-02-19 15:58:58 <KipIngram   > I.e., used to be writing a page of zeros resulted in the same action as a trim/unmap.  Those are different things now, though.
2023-02-19 16:30:09 <thrig       > sparse files can be created by seeking somewhere and then write, or pwrite
2023-02-19 16:31:48 <KipIngram   > Ok, so then space is not allocated for the unwritten parts?
2023-02-19 16:31:55 <KipIngram   > That's how it seems like it ought to be to me.
2023-02-19 16:32:21 <thrig       > hilarity can happen if you backup said file as if it were solid
2023-02-19 16:32:27 <KipIngram   > And it doesn't *feel* like something that should be terribly difficult to arrange, though I've not tried or anything.
2023-02-19 16:33:45 <KipIngram   > In a b-tree implementation I'd there would just need to be a way to express "no mapping" byte ranges in the file's space usage tree.
2023-02-19 21:29:20 <Zarutian_iPad> KipIngram: re regexp: and here is another thing. The regexp “syntaxes” are not the same between diffrent regexp implementations/definitons.
2023-02-19 21:39:36 <KipIngram   > Yeah; go figure.  :-)