The Archive v1.0.5/v1.0.8: Reacting to external file changes (bug hunt!)

ctietze · May 2018

While external changes were always picked up by the app, external changes were not displayed in all cases, e.g. after you began editing a note.

As @Basil often pointed out, this is kinda weird when the note as shown by The Archive is out-of-sync.

In 1.0.4 and 1.0.5 (from the Cutting Edge update channel), I changed how the app reacts to external changes. This needs more stress testing before I push it to the main update channel.

What's the problem with external file changes? -- They are indistinguishable from file changes that are effects of auto-saving in The Archive. So you might type away until auto-save is triggered, the file on disk is changed (while you continue to type), the file update is read in by the app and presented as new, and your changes since the auto-save are lost. A trivial approach to make this work is to distinguish between typing and idling, and not updating the displayed text while the user is typing. But the problem of timing remains in principle and will reveal itself again when you begin to type during about the same half second when the app shows the new file contents.

So please try to type, wait 1 second (which triggers idling), and type again. Type erratically. Move the cursor around a bit -- do stuff in the text editor and try to find a glitch

rene · May 2018

imho: the way i would do this, is: remember time of last (auto-)save for each open file. if file mtime is suddenly greater than last save time, an external change must have happened. this might be totally inapplicable to the framework or code you use that might trigger file change / auto-save. just 2c

ctietze · May 2018

I already do that But every write onto the disk necessarily happens after the internal updates, with a likely non-negligible milliseconds delay on spinning disk drives. And when you sync to Dropbox, you'll receive even more file change events due to metadata changes right after the real write operation that cannot be distinguished properly from content changes. (In theory, the FSEvent could report which kind of change happens, but the file system info doesn't help that much in practice.) So the modification date on its own is not a 100% reliable indicator, though it'll get you pretty far in most cases.

Basil · May 2018

I just realized that v1.0.6 seems to have (partially?) regressed back to the pre-v1.0.4 behavior of not always refreshing the currently displayed note when it is changed externally.

Steps:
1. Edit a note in The Archive
2. Wait for 10 seconds
3. Edit the same note externally

Result:
The Archive does not display the recently made changes

Is v1.0.6 from a different branch than v1.0.4 and v1.0.5, or did you accidentally "unfix" the bug?

miscellane · May 2018

I have the same document I'm working on in The Archive also open in Typora another screen as my markdown preview, so the sync issue is coming up a lot. It seems to be working so far to save in Typora, flick back to The Archive, select another note, then go back to the target note. Changes are usually registered by then. Whether or not going to another note refreshes The Archive or whether it just gives me something to do while waiting, I'm not sure...but it's looking like the former.

Basil · May 2018

Since this is a pretty important issue and we haven't heard back from you, I want to make sure that this does not fall through the cracks:

Are you aware that v1.0.6 has regressed to the pre-v1.0.4 behavior of not handling external note changes properly? Is v1.0.6 from a different branch than v1.0.4, or did you accidentally "unfix" the bug?

ctietze · May 2018

Whoops, yes: 1.0.4 actually was less precise, so I am surprised it works better. Once I finish the currently unfinished theme tasks, I'll focus on this one again. (My throughput has decreased quite a bit, so things do take longer for me to finish than they did in April.)

Basil · May 2018

Sounds good, thanks for the reply.

I'm not sure whether v1.0.4 worked universally better, but if you followed the steps I have described further up in this thread, then it would properly update the currently displayed note.

As always, if you need any help with testing, just let us know.

ctietze · July 2018

@Basil, @miscellane, and folks, please check out v1.0.8 and tell me if external changes are now picked up as expected, and also if you run into display glitches for internal changes. My tests have been solid so far, but you know how everybody else's machine seems to be infested with bug-producing components, right?

Basil · July 2018

First, thank you so much for continuing to work on this issue! I can imagine how much fun dealing with something like this is.

As usual, I seem to have one of those "machines that is infested with bug-producing components"...

Version 1.0.8 definitely is an improvement, but in many instances, the currently viewed note will still not get updated. The bug always depended on whether focus was inside the note and on whether there was an active search present. In that 2x2 matrix, TA used to handle one cell properly, and now it does two:

There is no active search and the note is selected but focus is not inside the note -> note is properly updated in TA
There is no active search and focus is inside the note -> note is not updated properly updated (this is new!)
There is an active search and the note is selected but focus is not inside the note -> note is not updated
There is an active search and focus is inside the note -> note is not updated

(with TA v1.0.8 on macOS 10.12.6)

I'm somewhat (but not 100%) sure that v1.0.4 handled all four scenarios properly.

miscellane · July 2018

Forgive my ignorance of how the update process works... My version of The Archive is coming up as 1.01, and when I select Check For Updates, it tells me I have the most up to date version. Where should I be looking for 1.08?

Basil · July 2018

@miscellane, you would need to switch from the Release Update Channel to the Cutting Edge Update Channel in the Update section of The Archive's preferences. Once you do that, Check for Updates will offer you v1.0.8.

Versions 1.0.2 through 1.0.8 have all been designated cutting edge / beta versions, meaning that @ctietze considered them slightly experimental and did not want to push them to regular users who did not opt-in to using potentially slightly less stable versions. My experience with these builds has been pretty solid, though, with only occasional small hiccups.

miscellane · July 2018

Thanks, I'll check it out today.

ctietze · July 2018

I have to report a new bug myself: now after the editor releases its lock, undo information are lost. So once you stop typing for a couple of seconds, you cannot go back. Bummer. Will fix that.

@Basil I'll put the app under stress test for the 2 missing cases and see if that helps finding the problem. Thanks for the feedback! If you want to give v1.0.4 a shot again to see if it holds up to your nostalgia, you can still download it from https://update.christiantietze.de/thearchive/v1/TheArchive-v1.0.4.zip

Basil · July 2018

Ha! Apparently, I was so excited when the bug had seemingly been fixed in v1.0.4 that I did not thoroughly test all four scenarios. Turns out, v1.0.4 only properly handles the two scenarios when there is no active search, which is just like v1.0.8 behaves now.

I chuckled a bit about your usage of "my nostalgia for v1.0.4", but apparently that was spot-on! So, sorry for the misinformation.

On the plus side, now that I've realized that we can easily download older versions by modifying the version number in the link that you have posted, it will be much easier to test such hypotheses about older versions in the future. I did briefly consider trying to fish v1.0.4 out of my networked Time Machine backups before I made that statement, but since browsing through old backups is slow as molasses, I decided against it and relied on my (faulty) memory.

ctietze · July 2018

I secretly had hoped to be wrong and you'd be right, dang

Thanks for all the testing effort you put into this! Instead of fishing in your backups, you can always, you know, ask me for stuff. Spending a couple minutes on your email is the least I could do to quench your thirst for outdated versions of the stuff I produce, which is weird enough in and of itself, but much appreciated

Basil · July 2018

You don't understand. It's my job to save you as much time as possible, so you can spend that time on further improving the application we all love. Bugging you with small, avoidable requests would run counter to that responsibility.

Seriously, though, I tremendously appreciate all the hard work you are putting into The Archive and how responsive you are. I tried helping to make nvALT better through bug reports, feature requests, and donations, but five or six years later, the application had improved about as much as The Archive typically improves in the span of two months.

Anyhow, if there is anything else we can help you with, just let us know.

Oh, one more thing:
It looks like v1.0.8 introduced a minor regression that causes the note text to flicker briefly after editing a note. I just cross checked (see, that link posting of yours is already paying off big time! ) and this issue is neither present in v1.0.7 nor v1.0.4.

ctietze · July 2018

Check out v1.0.9 and, finger's crossed, that should be the end of this story arc.

Basil · July 2018

I can confirm: Version 1.0.9 checks off all 4 boxes of the Basil Matrix, and the note-flashing regression also seems to be gone!

Fantastic work, @ctietze!

ctietze · July 2018

Thank goodness. Glad to have had your support in testing this stuff

Zettelkasten Forum

The Archive v1.0.5/v1.0.8: Reacting to external file changes (bug hunt!)

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion