Archiving external sources

grayen

Context is important. Most text only make sense when put into context, and so it is for Zettels as well. A Zettel might be nicely written and self contained, but what is written, was still written with a particular context in mind, and ideas, thoughts, information, etc, might be lost when this context is lost as well.

For example, when I properly get my Zettelkasten going, I will probably write some long overdue Zettels on one of my favorite science fiction short novels, however, a few years ago, the website that hosted it, took it down. Luckily I was able to find a mirror (alternative host) for it, but what if I hadn't and I had already written those Zettels? Aside from losing the novel, which would be painful, without the source, the Zettels would lose much of their value too.

I know of the web clippers / archivers, but do not be too focused on this particular example of an external source, they could be PDFs, images, anything really. I am not on MacOS, so I do not know what The Archive exactly offers for this, but I was wondering what others do to tackle this particular problem, what tools you use.

I am trying not to overthink my use of software too much, since I know from past experience I get stuck on trying to make the perfect tool, causing me lose track of the original goal of note taking, so I end up with a few tens of notes at most, until I create a new collection of notes elsewhere. Eventually not looking into them much or simply forgetting about them existing.

After hearing about this community I am trying to do better and just keep it simple at start making Zettels. My most successful collection of notes so far have been a pile of paper, some were even drawn on the back of opened envelopes, but they contained notes I kept going back to and adding stuff over time. I still have this pile and I know I will find it interesting to read again, even now after some years. The other successful collection were 3 simple .txt files that I simply only appended to, growing huge, but they are among my most treasured notes. Whenever I read them again, I come up with new insights or they allow me to go back to my core ideas and original thoughts on them when I might have strayed too far from them. This shows me the value of Zettelkasten, which is mostly the same, but applied on a bigger scale and more organized (in the method).

On topic again: Although it might cause me think too much about getting the best technological solution for it, I do think the need to archive external sources somehow is a something that needs a good technological solution for it to work, and is needed regardless of how simple you might want to keep things. What do you think and what solutions do you have to this problem?

Will

@grayen, you describe one of my use-cases for a Zettelkasten that I'll tell you what I do. I have a directory next to my Zettelkasten directory that I store any digital file that I want to refer to in my notes. One big file with no organization. How I find anything is via its reference in my Zettelkasten. This could be ANY digital file mp3, pdf, Docx, HTML, anything. These become clickable links in my Zettelkasten.
I place a wiki link to it like this

"A Hanging" by George Orwell ![](media/Orwell_Hanging.pdf)

I have a long history with Evernote, you may use one of the other GUI note apps. I use a protocol handler formatted like below as a link in my Zettelkasten note to a note in Evernote.

evernote:///view/597091/s5/d48fe8bd-e206-4432-9814-a8f70378ebfe/d48fe8bd-e206-4432-9814-a8f70378ebfe/

Various apps have different protocol handlers. some use /open/ or /edit/ or somesuch instead of /view/, you have to look at the documentation. Some but not all Zettelkasten software editors make one or both of these types of links clickable in notes.

I use a macro generator to help with the creation of these links.
Good Luck

grayen

@Will Thanks for explaining your setup! I had been thinking of a similar setup, but when I look at my old prototype for this, I seem to have used e.g. 2MqBcWKWYDYtramReh5v8x as file name, which thinking about it now, is bad, because the lack of file extension might cause it to not be opened by the right software, or other such issues. So I have file extension now as a requirement of my setup for this.

I use a protocol handler formatted like below as a link in my Zettelkasten note to a note in Evernote.

Nice idea! I have seen custom protocol used sometimes, but when I was thinking how to create URLs to notes and files within my Zettelkasten, I was stuck thinking about domain names and IP addresses, using a custom protocol seems much nicer and flexible though!

I still have some open questions with the setup I am thinking of at the moment:

• What to use as custom protocol name? I could use the name I assign to the current Zettelkasten software I am developing, but this is likely to change in the future. Although it makes sense on some level, since e.g. in the case of your Evernote links, it is a specific protocol used by Evernote, but most protocols are meant to be software agnostic, meaning maybe zettel://<UID> would be better.

• How should files be named? The reason for using UIDs to link to Zettels also applies to files, so to not use UIDs for them feels wrong to me. You could claim that you can just do a search and replace if the filename changes, but the same arguments for and against this have been discussed around Zettels. For some people this is perfectly fine, since they never link from the outside to their Zettelkasten, removing the need to have permalinks. However, I do realize that filenames may hold much more importance to some files compared to Zettels, since in Zettels this information is given to you through the contents, which might not be the case for files (images for example). To solve this, I rule out any solution that is not software agnostic, like having a database mapping UID to filename, however I do have some ideas that might work:

  • Prefix files with their UID. However with this approach you might lose the whole point of having filenames, since the UID takes up precious space when the name is shown. Using the UID as a suffix might not have this problem, but looks much uglier to me.
  • Make a directory with the UID as name and in it you can name the file (or directory for that matter) however you like.

• Should every file be accompanied by a Zettel for meta information? There a multiple ways to go about this. If you require every file to be accompanied by a Zettel, you could have the title of the Zettel be the filename (maybe after some automatic processing). The nice thing is that you can still keep the nice unprocessed file name available as the title in your Zettel, while with just the file, you will only have the processed title to work with, the original is lost in translation. It would also make it possible to have links between files, even if the file format itself does not allow for this. You could have a separate UID for the file and Zettel and have the connection between the two through metadata, but this feels wrong to me, because you lose permalinks again (the meta Zettel could change the file UID it references to at any time).

rhubarb

@grayen said:

• What to use as custom protocol name? I could use the name I assign to the current Zettelkasten software I am developing, but this is likely to change in the future. Although it makes sense on some level, since e.g. in the case of your Evernote links, it is a specific protocol used by Evernote, but most protocols are meant to be software agnostic, meaning maybe zettel://<UID> would be better.

A few years ago, I created a custom protocol/url-scheme zettel://<UID> (just as you suggest above). I made this software agnostic by introducing a layer between my Zettelkasten software and my protocol-based links to zettels.

I implemented it as a simple AppleScript that I exported as a macOS application using the standard ScriptEditor that comes with macOS. The important part is that the exported app needs to know that it has an url-scheme, which can be set in its Info.plist file¹. The AppleScript itself is very simple and just calls a python script zettelscheme.py that is embedded in the final application:

on open location this_URL
    set zettelscheme_script to (path to me)'s POSIX path & "Contents/Resources/Scripts/zettelscheme.py"
    do shell script "python " & zettelscheme_script & " " & "'" & this_URL & "'"
end open location

This python script does a few things for me, but in its most basic form, it would simply open the zettel with the UID. It could look as follows:

import subprocess
import sys
import urlparse

if __name__ == '__main__':
    parsed = urlparse.urlparse(sys.argv[1])
    if parsed.scheme != 'zettel':
        raise(Exception('Invalid URL scheme ' + parsed.scheme))
        sys.exit(-1)

    zid = parsed.netloc
    subprocess.call(["open", 'thearchive://match/{}'.format(zid)])

Currently, I just delegate all the action to “The Archive”, which I call via its own url-scheme theachive://. This is very convenient. Before I switched to The Archive, however, the python script had an embedded logic (also very simple) for finding the zettel with the UID in the directory where my Zettelkasten is located and then opened it with Sublime Text.

The whole approach is software agnostic². When I switched to The Archive, all I had to do was to update the python script within the App (which, after finding the information on the thearchive url-scheme, took me two minutes). All my links to zettels that I created in this way still work³, but now open the zettel in The Archive instead of Sublime Text.

---

1. The whole approach is based on a solution I found in this answer on StackOverflow). ↩︎

2. The approach is macOS only, but I am sure that it is easily possible to create similar solutions on other platforms as well. ↩︎

3. One way I use this protocol is as a reference from calendar entries to note-style zettels in my zettelkasten. When I take notes in a meeting, I simply add zettel://n20200421-1538 to the URL-field of the calendar-appointment in my calendar application. This is basically what the acclaimed journaling software Agenda implemented (years later ;-). ↩︎

ctietze

I didn't even know you could bundle AppleScripts as .app bundles and then set a custom URI scheme! This is nuts! Great work!

grayen

@rhubarb Thank you for sharing this! It is nice to see that it indeed helps making permalinks software agnostic yet still allow nice interlinking between software (e.g. your use of them in your calendar).

I thought about using them too in my Zettels themselves, e.g. [Automatically generated based on title](zettel://202004211540), but after thinking some more about the importance of meaningful links, I realize this would lead me to become lazy in that regard, e.g. thinking the title is enough to explain the relation, yet the title can change over time.