2023-11-29 04:45:09 <zelgomer    > KipIngram: we are well into the age of 5th gen warfare now
2023-11-29 04:46:20 <olle        > forth gen warfare...?
2023-11-29 04:50:16 <zelgomer    > i think forth is more suited to 5gw :)
2023-11-29 04:51:54 <olle        > Hehe
2023-11-29 05:09:07 <zelgomer    > being that disinfo is a large part of 5gw, i wonder if some of the claims about stuxnet have been embelished
2023-11-29 08:59:39 <MrMobius    > there is a really good documentary on stuxnet. when they eventually dissected it there were like 8-10 zero days stacked up which would cost several hundred thousand dollars so they knew it was created at a government level
2023-11-29 09:10:19 <zelgomer    > zero days in the virus itself or that it was exploiting?
2023-11-29 09:16:10 <Zarutian_iPad> ever since reading https://www.schneier.com/academic/archives/1998/06/environmental_key_ge.html in 1998 I had waited for a worm/virus that used techniques from that paper. Stuxnet was discovered in 2011 so it took thirteen years or so.
2023-11-29 09:17:33 <Zarutian_iPad> meanwhile polymorphic exploits had been used for ten years at that point.
2023-11-29 09:57:46 <KipIngram   > My interpretation of what I watched was that it had four zero-days that it was able to exploit to infect the machines it came into contact with.
2023-11-29 09:58:03 <KipIngram   > And seven methods for spreading.
2023-11-29 10:05:57 <zelgomer    > this part confuses me: "which would cost several hundred thousand dollars so they knew it was created at a government level"  how does exploiting a zero day have a cost, and how does an investigator determine this cost?
2023-11-29 10:07:23 <Zarutian_iPad> zelgomer: indeed. A zero day exploit can _sell_ for quite a lot but most of the time they get discovered by someone looking
2023-11-29 10:08:02 <thrig       > especially when gov'ts can demand access to private code
2023-11-29 10:08:05 <Zarutian_iPad> paying for that salary/cost isnt much
2023-11-29 10:08:52 <Zarutian_iPad> thrig: you mean the source code? Lot of exploits are discovered purely by disassembling executables
2023-11-29 10:10:38 <Zarutian_iPad> which is more likely as some compilers are wont to "optimize" away safety/security checks/measures that might exist in the source code
2023-11-29 10:18:35 <MrMobius    > zelgomer: if you're lucky enough to discover 8 zero days by yourself (or 4 as KipIngram said) then you can use them for free
2023-11-29 10:19:03 <MrMobius    > if you want to buy a zero day exploit from an illicit source, you may need a whole lot of money
2023-11-29 10:27:45 <zelgomer    > but how would an investigator know how much they were bought for without already knowing who bought them?
2023-11-29 10:28:44 <Zarutian_iPad> zelgomer: via "journalistic infersnce" which is mostly errornous or made up
2023-11-29 10:29:06 <Zarutian_iPad> s/rsn/ran/
2023-11-29 10:34:24 <rendar      > KipIngram, Iran has "off" switch?
2023-11-29 10:34:28 <rendar      > what you mean exactly?
2023-11-29 11:19:11 <Zarutian_iPad> 
ACTION is going through https://pdf1.alldatasheet.com/datasheet-pdf/view/1232222/MOTOROLA/MC68B54.html to figgure out the guts of Econet

2023-11-29 11:27:10 <KipIngram   > rendar: I saw a video last night that asserted StuxNet had been a cyber assault against Iran's uranium enrichment facility, targeting its centrifuge units, and that there exists another program run by the US, the UK, and Israel called Nitro Zeus that is capable of shutting down massive parts of Iran's infrastructure.  It's power grid, it's anti-aircraft defense grid, etc.
2023-11-29 11:27:32 <KipIngram   > The implication being that we can just tap a few keys in Washington and then fly right into Iran with impunity and do anything we want.
2023-11-29 11:28:36 <rendar      > KipIngram, you know that Iran leaders wants to *erase* Israel, and the Western World, and that kill people on the street like candies, right?
2023-11-29 11:29:00 <KipIngram   > Yes, I do know they have it in for Israel.
2023-11-29 11:29:05 <thrig       > America is not above regieme change, and kills people on the street like candies
2023-11-29 11:29:20 <Zarutian_iPad> KipIngram: one of those yt videos that alphabet soup agencies pump out?
2023-11-29 11:29:28 <rendar      > thrig, ahgahahahahahahhaha
2023-11-29 11:29:34 <rendar      > are you joking, right?
2023-11-29 11:29:50 <KipIngram   > I wouldn't even begin to try to assess the integrity of the various vlogsters on YouTube.
2023-11-29 11:29:56 <rendar      > thrig, where is now that guy who killed Floyd? IN JAIL
2023-11-29 11:30:09 <KipIngram   > This particular one was a Brit dude who goes by the moniker "Thoughty2."
2023-11-29 11:30:25 <rendar      > where is that police *woman* who killed that girl in Iran because she refused to wear the jihab? she's free
2023-11-29 11:30:26 <thrig       > road deaths are rather high in America. Americans seem mostly "meh" on the slaughter by cars issue. candies.
2023-11-29 11:30:53 <KipIngram   > https://www.youtube.com/watch?v=kFYKyZvu2Bo
2023-11-29 11:30:54 <Zarutian_iPad> KipIngram: the guy is probably pants on head crazy Tory Brexiter
2023-11-29 11:30:55 <rendar      > thrig, you're confusing a western country with HUMAN RIGHTS with a brutal dictatorship
2023-11-29 11:31:00 <rendar      > it's different thing
2023-11-29 11:31:17 <rendar      > you should read more books about people who fled from Iran to the US
2023-11-29 11:31:30 <rendar      > or western countries
2023-11-29 11:31:30 <KipIngram   > At least a few of his videos imply at least some conservative views.  I haven't seen him discuss Brexit directly.
2023-11-29 11:31:33 <thrig       > can't do much about iran. can complain about car sitters trying to murder me
2023-11-29 11:31:52 <thrig       > (no car sitters tried to murder me in pakistan, unlike in america)
2023-11-29 11:31:58 <KipIngram   > But he came down pretty hard on the modern trend to be "offended" by everything.
2023-11-29 11:32:36 <KipIngram   > Well, not the right to be offended, but rather the right to try to push other people around over it.
2023-11-29 11:33:09 <Zarutian_iPad> here is a thing though that seems to pass many "inteligence" peeps and their groupie politicans: there are criminal elements and if your operations or whatever cause them enough persistant annoyance then do not be surprised when an assasin or such shows up
2023-11-29 11:33:10 <KipIngram   > The thing is, though, he makes videos about damn near everything - his subject matter is certainly not exclusively political.  It's very far-ranging.
2023-11-29 11:35:02 <KipIngram   > If I were to guess, I'd say that only a percent or two of his videos I've wathced are "controversial topic" related.
2023-11-29 11:35:50 <Zarutian_iPad> does he cover each topic comprehensively?
2023-11-29 11:35:52 <KipIngram   > So it's hard for me to view him as any sort of "activist."
2023-11-29 11:36:09 <KipIngram   > I don't know how to rate that - you'd have to watch some for yourself and decide.
2023-11-29 11:36:32 <KipIngram   > I haven't felt like I'm being "played," but who knows - maybe he's just that good.
2023-11-29 11:37:30 <KipIngram   > About the best you can do these days, though, is expose yourself to a whole lot of different sources and try to extract some sense of what's real out of all of it.
2023-11-29 11:37:37 <KipIngram   > The net is just packed with liars of all flavors.
2023-11-29 11:37:59 <Zarutian_iPad> btw if anyone is wondering why I am bothering with Econet it is because it might have worth ideas stealing for CommanderX16
2023-11-29 11:38:15 <KipIngram   > If his goal is to disinform people, though, he sure spends a lot of time on topics that don't contribute to that goal.
2023-11-29 11:39:07 <Zarutian_iPad> KipIngram: being seeing ads for something called Ground News, might be something you want to take a look at
2023-11-29 11:39:59 <thrig       > of course riling the natives up about BRUTAL DICTATORSHIPS is a good way to distract them from pressing issues at home
2023-11-29 11:40:19 <user51      > I never tell the truth! ..wait a sec :)
2023-11-29 11:40:36 <Zarutian_iPad> thrig: the old 'barbarians at the city gates' roman tactic?
2023-11-29 11:41:04 <thrig       > external war for internal unification is something I have read about
2023-11-29 11:45:53 <KipIngram   > To the extent he has touched on the controversial stuff, I've tended to find myself in agreement with him.  But I'm complicated - I wind up landing all over the map.  I annoyed zelgomer the other day by advocating some amount of redistribution, but on the other hand I'm a practically RABID free speech defender.
2023-11-29 11:46:06 <KipIngram   > Basically neither of the "extreme party" wings would have me.
2023-11-29 11:47:02 <KipIngram   > When the left tried to label Jordan Peterson as  "alt right" I just had to laugh - that is the most insane notion I've ever heard.
2023-11-29 11:47:14 <KipIngram   > If Peterson leans any direction I'm 100% sure it would be toward the left.
2023-11-29 11:47:24 <KipIngram   > He just happens to take free speech seriously too.
2023-11-29 11:48:22 <KipIngram   > His feeling is that if we're not allowed to speak our  minds on controversial issues, we are well and truly hosed.
2023-11-29 11:48:32 <KipIngram   > And I agree with that.
2023-11-29 11:50:12 <KipIngram   > I hold the old school position on such things - the "sticks and stones" philosphy.
2023-11-29 11:51:44 <KipIngram   > And the hypocrisy of it all is nuts - you can't criticize the left; that's quickly a hate crime.  But then they'll turn right around and call people Nazis.  Apparently that's NOt a hate crime.
2023-11-29 11:51:54 <KipIngram   > That's legitimate criticism.
2023-11-29 11:54:30 <Zarutian_iPad> this, the hatecrime thing, actually touches on something I have been puzzled by for a long time: prooving intent and why regulating 'intent' does not lead to 'thoughtcrime' slippery slope.
2023-11-29 11:54:58 <olle        > Same with murder vs manslaughter
2023-11-29 11:55:05 <olle        > Thoughts matter :)
2023-11-29 11:55:06 <KipIngram   > oh yes - if the technology existed for them to know what you were thinking it would become thought crime.  I have no doubt.
2023-11-29 11:56:03 <KipIngram   > Because it's really not about anything "wholesome" - it's about silencing those who think differently from them.
2023-11-29 11:56:08 <Zarutian_iPad> olle: actions speak louder than words, and words louder than thoughts
2023-11-29 11:56:09 <KipIngram   > Using any means at their disposal.
2023-11-29 11:56:48 <KipIngram   > I don't mean to be praising the right here - they have their own set of problems.
2023-11-29 11:56:53 <KipIngram   > They're just slightly different problems.
2023-11-29 11:57:08 <KipIngram   > and i'm referring mostly to the extreme camps in both cases.
2023-11-29 11:57:13 <KipIngram   > not "everyone."
2023-11-29 11:58:09 <KipIngram   > I definitely believe there's a segment of the right that is overtly racist etc.  And I'd like to slice them off and discard them just as much as I'd like to be rid of the extreme left.
2023-11-29 11:58:12 <Zarutian_iPad> KipIngram: hmm, I have seen this particular in-group/out-group thinking pattern in many 'left' zealots adhere to
2023-11-29 11:58:18 <KipIngram   > Those are bad/dangerous people in both cases.
2023-11-29 11:58:51 <KipIngram   > But somehow they manage to dominate the broad discussion in the media.
2023-11-29 11:59:10 <KipIngram   > The (very large, I think) middle camp just gets ignored for the most part.
2023-11-29 12:00:06 <KipIngram   > I guess "reasonable people" just aren't interesting enough for the news peeps to pay any attention.
2023-11-29 12:00:08 <KipIngram   > Boring.
2023-11-29 12:00:15 <Zarutian_iPad> KipIngram: because they get so much attention, cause they rile up the audience. The same yellow paper journalism that has metasized into ragement metrics of social media
2023-11-29 12:00:58 <KipIngram   > Yeah.
2023-11-29 12:01:09 <KipIngram   > They serve the goals of the media.
2023-11-29 12:01:30 <Zarutian_iPad> which is to sell advertisement space
2023-11-29 12:01:43 <KipIngram   > When I was young I tried very hard to make sure I watched the news every day, but I gave that up a long time ago - they're all mostly worthless.
2023-11-29 12:01:58 <Zarutian_iPad> which leads to 'brand damage risks'
2023-11-29 12:02:10 <KipIngram   > Yeah - the almighty dollar.  Or pound, or euro, or whatever.
2023-11-29 12:02:15 <thrig       > in Pakistan one would read the Indian and the Pakistani editorials and between the two...
2023-11-29 12:02:46 <KipIngram   > Yes, when I am interested in some issue that's exactly what I try to do.  Look at the info from across the spectrum, and then just "do my best.'
2023-11-29 12:03:19 <Zarutian_iPad> KipIngram: then you really need to look at that Ground News thing
2023-11-29 12:03:25 <thrig       > some editorials did go on about a hindu american zionist conspiracy
2023-11-29 12:03:52 <KipIngram   > I have peeked at that.
2023-11-29 12:04:45 <KipIngram   > Thing is, even if it's unbiased today it might not be tomorrow; gotta stay vigilent on that front.  I think it's risky to ever start relying on a single source.
2023-11-29 12:05:16 <Zarutian_iPad> oh, I agree with that
2023-11-29 12:05:43 <KipIngram   > But if a particular source is largely unbiased, you can probably figure that out with a little time.
2023-11-29 12:05:50 <KipIngram   > Just by looking at what they're telling you.
2023-11-29 12:06:40 <KipIngram   > And of course I don't expect publications to  "have no opinion."  It's just that the "opinion" is supposed to be confined to the editorial page.
2023-11-29 12:08:50 <KipIngram   > Anyway, re: Thoughty2, my guess is that he has free speech beliefs similar to my own, and I couldn't even start to tell you what his opinion might be on other fronts, like welfare etc.  It hasn't really come up in any of the videos I've watched.
2023-11-29 12:10:52 <KipIngram   > The main thing is that I've found him entertaining.
2023-11-29 12:11:40 <KipIngram   > So he's on my list of YouTube channels I'll generally check out, right along with Veratasium, Sabine Hossenfelder, and a few others.
2023-11-29 12:15:34 <KipIngram   > BTW, this site allows some interesting function visualization:
2023-11-29 12:15:36 <KipIngram   > https://c3d.libretexts.org/CalcPlot3D/index.html
2023-11-29 14:42:48 <KipIngram   > Ok, I've had a damnably odd problem pop up on a test I ran last week.
2023-11-29 14:43:04 <KipIngram   > These tests basically run on two separate computers, both sending traffic into the same SSD.
2023-11-29 14:43:38 <KipIngram   > I go to great trouble to syncrhonize the starting of each test, and I can watch that happen onscreen.
2023-11-29 14:43:48 <KipIngram   > The two computers are network time synchronized.
2023-11-29 14:44:06 <KipIngram   > Each one produces a set of performance logs with each line time stamped.
2023-11-29 14:44:25 <KipIngram   > Later, when the job is done, I gather all those files up and aggregate the pairs of files into drive total results.
2023-11-29 14:45:05 <KipIngram   > On this one test, out of four I ran last week, late in the job the two log files for the last tests of the job started showing a time mismatch.
2023-11-29 14:45:29 <KipIngram   > That caused my aggregation program to not add them up properly, and those particular curves showed only half the performance they should have.
2023-11-29 14:45:43 <KipIngram   > Now, the easiest thing to assume is that the OS clocks got out of synch.
2023-11-29 14:46:09 <thrig       > do you test that NTP or whatever isn't broken?
2023-11-29 14:46:10 <KipIngram   > But - there were two drives under test on that stand, and the other one was definitely running during this time period, and its log files don't show the problem AT ALL.
2023-11-29 14:46:20 <KipIngram   > ^^ that's what I'm ruling out right there.
2023-11-29 14:46:34 <KipIngram   > There was another test subject to the same NTP time sync, and it is fully in sync.
2023-11-29 14:46:44 <thrig       > because I saw a server boot four years into the future (and NTP was broken because of an unrelated firewall thing)
2023-11-29 14:46:49 <KipIngram   > That would have been the easiest explanation, but it appears that it's not applicable.
2023-11-29 14:47:15 <KipIngram   > There is some more digging I can do, but right now I'm mystified.
2023-11-29 14:47:48 <KipIngram   > And I've been using this stand with this time sync method on it for six months or so and have never seen this previously.
2023-11-29 14:48:13 <thrig       > bios can go bad
2023-11-29 14:48:29 <KipIngram   > Well, again, something at that level should have affected both of the two tests.
2023-11-29 14:49:20 <KipIngram   > I have two new tests running on the stand that I started Monday.  I looked at the most current collected log files on those runs and see well-aligned timestamps in the logs.
2023-11-29 14:49:59 <KipIngram   > I can take some defensive action - I can add a check for this in my test harness and confirm at the start of each test that the time sync is good.
2023-11-29 14:50:17 <KipIngram   > But, only if the system time is actually responsible, which I kind of think it's not.
2023-11-29 14:50:50 <KipIngram   > And the other thing I can do is just ignore time stamps in my slave log file - use the master log's timestamp and just match the two files up line by line instead of by timestamp.
2023-11-29 14:51:03 <KipIngram   > That's not terribly satisfying, but it would "work."
2023-11-29 14:51:31 <KipIngram   > I have session logs of all of this.  When the currently running tests are done I'll open those up and look back at the point in time where this happened and see if I see anything there.
2023-11-29 14:51:46 <KipIngram   > I just prefer not to poke at the live stuff while tests are underway.
2023-11-29 14:52:44 <KipIngram   > I know precisely where it first showed up in the sequence of tests, and the session log files will tell me exactly what time the system thought it was when it launched those tests.
2023-11-29 14:53:33 <KipIngram   > Although the session logs are only collected on the master canister - I won't actually have a record of what time the slave canister thought it was.  I'll add a capture of that for tests going forward.
2023-11-29 14:54:03 <KipIngram   > Oh, wait - scratch that.  I do have session logs for both canisters.
2023-11-29 14:54:12 <KipIngram   > Losing track of my own design here.
2023-11-29 14:54:37 <KipIngram   > so yeah - when the system's idle I'll go in and look and see what things looked like then.  And, I can also go look at the other drives's session logs.
2023-11-29 14:54:57 <KipIngram   > There's a small possibility that it was doing some "idle rest" or something during that period of time.
2023-11-29 14:55:09 <KipIngram   > But I can know for sure from the session logs.
2023-11-29 14:55:21 <KipIngram   > They just accumulate over time across all jobs.
2023-11-29 14:56:33 <KipIngram   > Anyway, that will tell me for sure whether the system clock is involved in this in any way.
2023-11-29 15:21:33 <KipIngram   > I think I'm not going to muck about with any of my scripts just yet.  Not until I have a more complete picture of exactly where and how this problem entered into the system.
2023-11-29 15:22:12 <KipIngram   > I see a number of strange of strange things going on in those test results.
2023-11-29 15:22:46 <KipIngram   > We did have a network issue last week at the office, so I suppose it's possibly related, but I still don't see why that wouldn't have affected both tests.
2023-11-29 15:23:10 <thrig       > there was a funny story from NASA on ycombinator recently
2023-11-29 15:23:11 <KipIngram   > If it affected one, that is.
2023-11-29 15:23:18 <KipIngram   > yeah?
2023-11-29 15:24:05 <thrig       > https://www.chrislewicki.com/articles/failurestory
2023-11-29 15:24:18 <thrig       > https://news.ycombinator.com/item?id=38452959
2023-11-29 15:24:29 <KipIngram   > Oh, is that the one with the metric/imperial mismatch?
2023-11-29 15:24:34 <thrig       > nope
2023-11-29 15:29:51 <KipIngram   > Oh ouch, that looks kind of unfortunate.  Or at least he was thinking so where I'm reading now.
2023-11-29 15:31:58 <KipIngram   > Man, I'm so happy for that kid that things didn't go as badly as they could have.
2023-11-29 15:32:36 <KipIngram   > Of course, a process that lets a single person error kill a half billion dollar piece of equipment is just not an acceptable process, so if the worst had happened the fault would have gone far beyond that one young fellow.
2023-11-29 15:33:11 <KipIngram   > Humans MUST be regarded as fallible components when you're considering single events.
2023-11-29 15:37:16 <thrig       > 12+ hour working sessions also aren't good at keeping the error rate down
2023-11-29 15:38:51 <KipIngram   > Not at all.
2023-11-29 15:48:26 <KipIngram   > Ok, interesting.  I did go ahead and look at the session log, at least a little.  I didn't check the "good' test yet, but the problematic one did show system clock mismatch at times.
2023-11-29 15:48:55 <KipIngram   > In fact, I think the clock must have jumped "during" some tests.  Because, for example, one test that was supposed to run for 900 seconds only seems to have run for 110 seconds.
2023-11-29 15:49:01 <KipIngram   > It decided it was done, prematurely.
2023-11-29 15:49:22 <KipIngram   > And that is not something I can work around after the fact.
2023-11-29 15:50:17 <KipIngram   > I need to add some further diagnostics to my harness.  I need to have the slave canister report its system time back to the master, on each test, and after each test I need to check the line counts of the log files and make sure they are roughly the right size.
2023-11-29 15:50:21 <KipIngram   > Things like that.
2023-11-29 15:50:48 <KipIngram   > And if anything is ammiss, I need to declare that a "failure" - the system will already repeat test steps that are considered "failure."
2023-11-29 15:50:55 <KipIngram   > I just need to expand my inventory of failure modes.
2023-11-29 15:51:25 <KipIngram   > Repeating a 180 second test or even a 900 second test isn't that big of an issue.
2023-11-29 15:51:39 <KipIngram   > I only have a couple of tests that are long enough to make that noticeably painful.
2023-11-29 15:51:52 <thrig       > you could also query and log what ntp thinks it's state it
2023-11-29 15:52:01 <KipIngram   > Yes.
2023-11-29 15:52:36 <KipIngram   > And if I find the canisters out of sync, I could potentially resync them.  I don't know exactly how to do that properly, but I can learn.
2023-11-29 15:52:41 <thrig       > I had a motherboard where the motherboard drifted too much for ntpd to deal with
2023-11-29 15:52:51 <thrig       > derp, system where the motherboard...
2023-11-29 15:53:39 <KipIngram   > Yeah, this particular stand is one of my new ones, and it uses something different from ntp.  Something supposedly newer.  The guys over in Hursley told me how to configure it.
2023-11-29 15:54:04 <KipIngram   > We did have those network problems last week - maybe they were injecting this behavior somehow.
2023-11-29 15:54:24 <KipIngram   > I wasn't actually working last week, so I don't know a lot about the details.
2023-11-29 15:54:35 <KipIngram   > I just know when I logged on to spin my tests the network was still down.
2023-11-29 15:54:41 <KipIngram   > So I didn't get the new tests started until Monday.
2023-11-29 15:56:21 <KipIngram   > If this time sync engine is "iffy," I could potentially have the master send its local time over to the slave as it starts each test, and the slave would just SET its time to match as it starts its test.
2023-11-29 15:56:47 <KipIngram   > All that really matters is that they agree with each other - they don't necessarily have to agree with the world.
2023-11-29 15:57:13 <KipIngram   > So I could cut my reliance on the network sync completely.
2023-11-29 15:57:13 <thrig       > or the server is the ntp server for the clients
2023-11-29 15:57:33 <KipIngram   > Yeah, or that.  I currently have them using the same remote server.
2023-11-29 15:57:39 <KipIngram   > But that's not necessary either.
2023-11-29 15:57:57 <KipIngram   > It's "pleasing" to me to be able to brag that they're spot on with the world, but not essential.
2023-11-29 15:58:20 <KipIngram   > Gotta run a quick errand; back shortly.
2023-11-29 17:16:24 <KipIngram   > Ok, IT guy says the problem last week definitely could have injected time sync errors.
2023-11-29 17:16:31 <KipIngram   > So, that's comforting.
2023-11-29 17:16:46 <KipIngram   > I don't like things that just sail in out of the blue.
2023-11-29 17:18:23 <thrig       > I've had to fix NTP like at every job I've had
2023-11-29 17:41:28 <gordonjcp   > only time I've had to fix NTP at work was when the GPS antenna got blown off the tower
2023-11-29 18:08:46 <KipIngram   > Well, some of the symptoms I'm seeing definitely seem to be  indicating that the time jumped mid-test in some cases, and kept them from runnin for the correct length of time.  I'm still contemplating the notion of cutting myself loose from it completely and just "self-sync" my canisters.
2023-11-29 18:09:28 <KipIngram   > Stability is more important to me than precise time accuracy.
2023-11-29 18:10:05 <thrig       > ntp helps avoid drift from motherboards
2023-11-29 18:22:22 <KipIngram   > Sure - it has lots of benefits.  And, I may not change anything.  They went right back in sync when the problem was resolved.
2023-11-29 18:23:03 <KipIngram   > The thing is, one would think that losing contact with the time server should just cause the system to fall back on its own clock, starting from the last sync.  It shouldn't just suddenly jump many minutes or something like that.
2023-11-29 18:23:13 <KipIngram   > It really still should have been ok, but apparently it wasn't.
2023-11-29 18:29:11 <thrig       > ntpd often has a drift file that shows how bad the host is
2023-11-29 18:29:28 <KipIngram   > I'll check for that tomorrow.
2023-11-29 18:29:54 <KipIngram   > I thought that was all "crystal accurate."
2023-11-29 18:30:01 <KipIngram   > Non-network time, I mean.
2023-11-29 18:30:09 <thrig       > (if it were real bad, ntpd would have given up. crystals can be cheap and/or broken)
2023-11-29 18:30:10 <KipIngram   > I mean, it's just counting cycles of a crystal oscillator isn't it?
2023-11-29 18:30:52 <KipIngram   > And quartz crystals are supposedly very, very good.
2023-11-29 18:31:56 <KipIngram   > I don't think the machine caused these jumps - I saw indications in my results of the canisters being within a second of one another at the start of one test and then five minutes later being out by 1000 or 2000 seconds.
2023-11-29 18:32:07 <KipIngram   > I can't believe the machine's clock would be that bad.
2023-11-29 18:32:22 <KipIngram   > I think somehow the network issues "confused' ntp.
2023-11-29 18:32:47 <KipIngram   > My old stand uses ntpd; the new pair of stands use chronyd.
2023-11-29 18:51:55 <thrig       > it is rare, but I have seen a motherboard with a lot of drift
2023-11-29 19:15:46 <KipIngram   > I guess crystals could be "defective."  Maybe have a hairline crack or something like that.  I don't know much about them, really.
2023-11-29 19:16:23 <thrig       > it was a cheap motherboard, too
2023-11-29 19:19:39 <KipIngram   > At one company I worked for we made down-hole oilwell tools.  They weren't mass-produced; each one was "named" and handled as an individual entity.  We had a room in the back where we aged the crystals.  We bought large quantities of parts, and put them through out own testing process to select out the ones that were in the right side tail of their "quality" bell curve, and sold the rest to the spare parts
2023-11-29 19:19:40 <KipIngram   > market.
2023-11-29 19:20:12 <KipIngram   > It was a business process the owner of the place had concocted and I always found it pretty impressive - we got far more out of those parts than they were rated to provide.
2023-11-29 19:20:28 <KipIngram   > Higher temperature range, better crystal accuracy, etc.
2023-11-29 19:21:12 <KipIngram   > We drove them out of their manufacturer specified working ranges, but WE knew they could do it because we'd tested for it.