If you asked me last week, I'd have told you I had a dozen or so orphaned zettel.
Well, I'm embarrassed. I ran a little script to see, revealing I was confused. Classic Blind Spot Bias. I hope to help. I don't worry if things aren't perfect. I aim to share my path as I discover little things that help. Some of you don't have any orphans; even if you did, this wouldn't be interesting. I consider myself a journeyman knowledge worker and can continually improve.
egrep -L -e ' \[\[|^\[\[|\.\[\[' *.md | wc -l
It showed 620 zettel with no outgoing links. Many of these have inbound links. It would be interesting to write a script that shows which zettel had no inbound links. Writing such a script is above my skill level. 620 is 20.5% of my ZK. It spotlights my laziness. Most of the zettel were older zettel, created when I didn't know better, and coding notes, but there are plenty of exceptions. I noticed a mass of class notes having no links. Maybe because they proliferated at hurricane force, and I didn't take the time to integrate them into my ZK fully.
I'm turning over a new leaf and starting a new practice. I've created a script that lists all the zettel without atleast one outgoing link. When I morning journal, the template populates with three random zettel from that list. I then spend a few minutes refactoring and adding outgoing links, integrating them into my ZK. If I do the higher math, I see that at a rate of three a day, getting the 620 orphan zettel integrated into the herd will take me 200.66666667 days or until May 16, 2023.
Zero orphans by mid-May. That is my publicly stated goal. Hold me to it.
Here is the script in one go. I've split it into two, one part creates the list once a week, and the second finds the three random zettel wanting attention.
#!/bin/zsh
cd /Users/will/Dropbox/zettelkasten/
# Reset the orphan's zettel list.
rm '/Users/will/Dropbox/zettelkasten/Orphan-Naked Ideas 202208222105.md'
for filename in *.md; do
# For each note, various positions of a link (not the UUID "›[["), piping the output to orphan zettel list reformating filename into The Archive link.
egrep -L -e ' \[\[|^\[\[|\.\[\[' $filename | while read i; do echo $i | sed -E 's/(.*) ([[:digit:]]{12})(\.md)/[[\1 \2]]/' >> "Orphan-Naked Ideas 202208222105.md" ; done
done
# "shuf" is a UNIX random tool '3' is the number of lines returned.
#while reading random (shuf) file names, reformat for Bear.
cat "Orphan-Naked Ideas 202208222105.md" | shuf -n 3 | while read i; do echo $i | sed -E 's/\[\[(.*) ([[:digit:]]{12})\]\]/- [\1](thearchive:\/\/match\/\1 \2)/' ; done
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
Howdy, Stranger!
Comments
@Will I try to track that as I go along using the tag "#Unlinked". That tag gets applied as part of the zettel creation macro I "borrowed" from you, that is, every new zettel has that tag included. I don't remove it until there are at least two outbound links. And every week or so I do a search on that tag, and process a few zettels so that they have outgoing links. However - I have way fewer zettels than you do - perhaps my method is too clunky when your ZK gets large?
@GeoEng51, thanks for the tip. You've mentioned this before, and it didn't resonate as it does now. I'm implementing a #Unlinked tag alongside my other prompts in my zettel creation template so that when I get my ZK 'cleansed,' I can keep my Zk looking like a home rather than an orphanage.
I think I'll remove the #Unlinked tag when forming the first outbound link. The first inbound link is the structure note that the zettel gets hung on. I've been thinking about scoring each zettel by watching the ratio of inbound to outbound links. I've been convinced that scoring a zettel on its link ratio is a fool's game. Wasting too many cognitive cycles for a paltry return.
Checking for orphans is a way of randomizing zettel refactoring. It is probably as good as any queue for refactoring.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
@Will Do you feel that all zettels should be "hung" on a structure note? If so, what are the advantages of doing that?
Love the shuffle routine, to me it reads as: yup some items here, but no biggie as far as order is concerned. And this folks, only reinforces my thinking that plain text is the defacto way to handle data - no plugins needed, simply use your existing toolkit & noggin.
And UUID? Unique User ID? Wish there was a glossary board in the forum to help disambiguate local acronyms, mnemonics and such.
Tinybase: plain text database for BSD, Linux, Windows (& hopefully Mac soon)
Off-topic, but yes. See prior discussions.
Did you run
egrep -L -e ' \[\[|^\[\[|\.\[\[' *.md | wc -lin your zettelkasten directory? What were the results?Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
Sorry for the acronym soup. UUID is a stand-in for Universal User ID. This gets the referred to zettel without its links. I find it indispensable when writing scripts and programming in python, and I'm looking for a specific zettel. Using a UID produces every zettel with a link to the UUID, not just the specific note. This conundrum can probably be solved more elegantly via other formatting protocols, but this seemed most straightforward way back when, and I've just stuck with it.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
At least with the Folgezettel technique I don't have more than one or two orphans, since I start by deciding where the next note should be linked. The orphans are either interrupted notes or documents that probably should reside elsewhere.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
I get the result "0".
Arguments could be made that the connectiveness of the Archive is a lead variable for its ability to function as a thinking aid: If there is always a link present the association never stops.
I am a Zettler
Looks Nice & readable to me Will. Very idiomatic of the timeless "garden hose analogy", where the output of x serves as the input of y.
Tinybase: plain text database for BSD, Linux, Windows (& hopefully Mac soon)
Congratulations. Every one of your zettel are integrated into your ZK.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
Congratulations on choosing a technique that automatically links a note into an idea stream through an inherent characteristic of the filename.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
Maybe this isn't a help to anyone else. Maybe I'm the only one concerned with zettel orphans. I'm admitting my naivety.
Sorry for the digression. Carry on.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
It think this is indeed a recommended practice: To have 0 orphans.
I am a Zettler
Haha! Well, I only have about 350 zettels and I've spent a lot of time "maintaining" them. So in my case, it makes sense. Others will have different purposes for their ZK and a different pace, and may well have some zettels that are not integrated into the whole.
I agree with @Sascha , that the ideal would be zero, but that has to be weighed against other priorities in the creation and maintenance of your ZK.
I'm just getting started, so almost all my zettels are orphans now. I'm still working out my linking system, but based on what you said, I'll need to find a good way to search for zettels which aren't linked to anything.
I'm using OneNote, which I generally really like, but it doesn't have a way (that I know of) to search for pages with no links. I'll probably have to write a macro for it.
Thanks for the idea.
It isn't automatic, it's the habit that comes with deciding what the left-hand, Folgezettel portion of the ID should be. The right-hand timestamp portion is automatic. One exception is when I backdate an ID to make a sibling that should occur earlier in the sequence in Zettlr's linear file pane view. There is also the first H1 header which has the title visible in the file pane, at least in Zettlr as it is configured. Scrivener will display at least this much in the cork board view. And as arduous as that is, I make some effort to read what I am linking to.
As for idea streams, I'm not sure much of anything in my ZK rises to the level of ideas. Perhaps one thing that I am working on. But it's more of a depleted aquifer than a stream. The rest of the effluent gurgles to the bottom of the sluice, perhaps never to be seen again. But the effluvium hangs in the air.
It is useful--please don't say that! I have an entirely parallel situation.
Because Zettlr won't generate my IDs (only the right-hand portion) I have an analogous trouble for which I wrote a script. The filenames that Zettlr generates are 14-character dotted timestamps in the form
YYYY.MMDD.hhmm. I define the ID in the first H1 header, followed by the title. The corrected ID will never match the filename unless I copy the corrected ID from the H1 header, type ctrl-r for rename, and paste the ID into the file renaming dialog with ctrl-v. I wrote a python script to list all the Zettels where the ID in the H1 header doesn't match the filename.It helps to deceive myself that this is eufriction, to use Bob Doto's term. I often find myself in a state of inanition after working with the ZK. And to counteract that, I compound the self-deception with a belief in unbounded willpower and energy. This follows the rule of threes (eufriction, willpower, energy) which I continually forget.
Thanks for your regex also.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
I did it!
This morning I opened my journal, and it showed me the following:
The six orphans are all in my proofing/inbox. I'm all caught up and have ZERO ORPHANS. A couple of days ahead of the scheduled date of May 16, 2023. 620 at three a day.
Oops, I tweaked my shoulder, patting myself on the back.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
@Will: I can report something similar: the ID in the H1 header of each markdown file matches the filename. Nineteen files don't have H1 headers. Those can be fixed, deleted, or moved. I have 486 Zettels.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Well done, @Will and @ZettelDistraction ! I'm impressed by your dedication to working it through.
Given the effort involved and the back-and-forth character of my forum writing, the following quotation -- whose intentionally omitted source will not be lost on the forum participants--is apt.
Now that the Zettelkasten is in order, the real work of "processing" sources into Zettels--a term that strikes me as insufficiently deferential to the totality of human knowledge--may perhaps to begin. Yes?
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Best of luck with your ZK "real work." My next step is refactoring my larger notes. Atomizing one a day. Refactoring is proving to be more time-consuming but no less interesting/exciting than refactoring orphans with links.
It's a messy business.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
@Will Thank you. I too have the interminable work of factoring, refactoring, partitioning, consolidating, expanding, laminating, curing, emulsifying and demulsifying my Zettels. Special Zettels with identifiers of the form Rpt.folgezettel.0.timestamp signify a report to the Zettelkasten. Occasionally the Zettelkasten responds. The occasion of fixing the broken links of the ZK led to a report to that effect, with the punchline. After every improvement, or perceived improvement, the "real work" begins anew.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
@Will @GeoEng51 Now at 509 Zettels.
I could track down files without tags. That might be useful. The "show the first H1 header as the filename" community plugin for Obsidian stopped working thanks to several software incompatibilities. The actual name for this is "file explorer markdown headers," but that's less descriptive than Zettlr's "Display files using Title or first heading level 1 if applicable," which is not as concise as "show the first H1 header as the filename." I might work on a plugin.
The ZK has given me at least two software projects to work on. At this point, I can use the ZK to direct my work by checking what fits into the note sequences.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Forgive my ignorance of the ZK methods, I am new and learning. I'm wondering why the absence of an outgoing link is necessarily a bad thing. Since they have incoming links, maybe they are "leaves" in your tree. Unworthy or unnecessary of further elaboration? If they are rarely (never?) traversed following your existing links, could that be an indication that the notes aren't serving the purpose of your ZK? Maybe just leave them alone and let the ZK grow up around them like a field of grass. While you are fixing those old notes, you could have been writing better, new ones.
For those of you using a ZK, how do you measure the quality of your ZK and the quality of your connections? What makes making a connection "worthy?" While I haven't done research in this aspect of memory in many, many years, I'm not sure a fully connected network (or 'overconnected' network) is a good thing (each note having at least one outgoing and incoming link). Partially connected networks with strong links are usually going to be better because of things like traversal costs, decision costs on which path to take, etc.
I think of it like a path through the grass between buildings at a university. Over time, common paths get worn into the grass making a stronger and stronger link between those 2 locations. Is there a way in ZK to "strengthen" a link? That seems more powerful than simply connecting things more "blindly," or maybe with less thought or consideration is a better way to put it. I'm afraid if you just run a script and find all the ones with no outgoing links, the process becomes mechanical. I would think there would be a cost to making links you'll have to pay for later when you are figuring out which one to take because there are so many.
Dan, thanks for the opportunity to expose my thinking. Most of this will come together for us as I write. So bear with me as I try to answer your seven questions. I'm a beginner too. I took the red pill and am striving to improve my skills incrementally. I aim for but will never be at the top of my game. If I had more time I'd have written you a shorter reply.
When I first read your post, I imagined my notes without any outgoing links and thought, 'why not?' After writing this response, my answer is that quality links feed nutrients through idea sequences back through the ZK.
The absence of an outing link in a note doesn't necessarily diminish the value of the idea expressed in the note. It does make it hard to follow any idea thread. Ideas are not something that exists in isolation.
The incoming links on one note are the outgoing links on another note. To continue with your tree metaphor, the leaf needs a path to send nutrients back to the stem, branch, bole, and, eventually, the roots.
What is an example of an idea that is "unworthy or unnecessary of further elaboration?" Looking closely at orphan notes can help determine whether or not a note might be culled. It surprising how few old orphan notes qualify as 'unnecessary.' (I culled 3 notes completely, found a handful of duplicates, and a dozen more that I combined.) The time I spend reviewing and refactoring established notes is among the most enjoyable times in the ZK.
Link Quality (LQ) and Link Weight (LW) are moving targets without a single strategy suitable in all cases. The criteria for a high-quality link are in the titling of the note and in documenting the link's context. Titling is a skill, and it takes time to develop. Adding context to the link is adding a one-sentence 'summary' to the link. This is the form of my links.
- A Note About Notes [[202305170552]] * A note's "passability" is a spectrum of the reader's desire, excitement, and intention to share the idea. Even if they don't spread the idea or pass it forward.A simple link with the UUID becomes meaningful only after selecting it. In contrast, one with context makes for a quick decision on whether it will be fruitful. The work done during the capture/creation phase will pay dividends in the research phase.
The paths you describe are called 'desire lines or social trails' and, in the best cases, are used to determine where future sidewalks are built. Using the metaphor of the path, the more a link is traversed, the more value it gets. ★★I'm stealing this idea! - Program a way to track the number of times a link is traversed and add that number to the links LW.★★
Yes, I agree that just running some script is nothing but mechanical. I don't want to sound too defensive. The magic is in what you do with the output of the script. In my case, the script output a 620-line file. One note per line. I then divided the workload, looking at 3 notes a day. I spent 15-30 mins reading, refactoring, and adding links to other established notes. This was a feast of fulfilling fun. I felt very involved with my ZK. I hope I'm not deluding myself too much; I spent over 100 hours on this project over the last 4 months.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
Thanks for the answers, it helps. I tend to ask a lot of questions when I’m learning something new. My intro post has been sitting waiting to be approved for a few days, so I can’t ask my own questions yet, so I’m trying to learn in these responses. Your problem is exactly what I’m struggling with.
In my current style, I have the following issue that mirrors your situation:
Source 1: 3 notes that cover topics A, B, C.
Source 2: 3 notes that cover the same topics, slightly differently.
… Source N: ditto
I deciding on one of the following currently:
1. Create 3 notes per source, 1 per topic. Create a topic summary note that links to each source note covering that topic. I would probably never elaborate the individual source notes. Your thread made me think about it.
2. Never add the source notes. Just leave them in Zotero. Just add the Summary note and use source citations. This is what I do now in my non-ZK workflow.