Help: Multiple interpretations, and when to stop digging?

alkhan

Hello Zettelkasten friends,

How do you make notes on an experimental study when you have multiple interpretations of it? For example, here is a note I got from "Human memory: A proposed system and its control processes":

Montague, Adams, and Kiess (1966) found that associating a word — called a Natural Language Mediator increased recall rate. Specifically, when people forgot the NLM assigned to a pair of nonsense words, retention rate was negligible. In contrast, when people remembered the NLM assigned to a pair of nonsense words, it allowed them subjects to remember the word pair at 70% success rate.

Associating this note to my own experience in learning strategies, it leads me to two interpretations:

1. I was right in my experience that memory techniques still need to be rehearsed. They act like NLM's as used in the study, so when you forget your memory techniques, you've basically "burned the bridge" to remembering the encoded info -- making it harder or even impossible to retrieve it
2. This could also be an example of chunking, or organizing information into a single retrievable unit, because the NLM used in this were assigned to the pair of words

Actually, there is a third -- that is, simply associating doesn't lead to perfect retention.

And of course, this quote came to my mind:

If you read a book, you mostly read secondary literature. That means that you read a text that is about texts. Scientific reviews, self-help literature, history books, you name it. Most of what we read is second-hand knowledge. In the beginning of my Zettelkasten work I once read all the studies mentioned in footnotes of a book on nutrition that was famous back then. I was quite surprised at the bias and flawed approaches of the authors. So I pledged myself I would never rely on someone else’s interpretation of phenomena. (See first example.) To this day, I am happy with that pledge and its consequences. It takes some time to process a book, but with the barbell method of reading I am happy with my productivity.

I started to get influenced by Sascha's stance on this when a year ago, he mentioned that the time you're spending here is enriching your knowledge, or "educating your Zettelkasten." This led me to notice that a lot of books out there (especially Essentialism, please AVOID that one and just read the studies referenced) cite a lot of scientific papers yet having a shallow level of interpretation -- not to mention a skewed one to fit their narrative.

But what if you're interpreting it yourself and you have multiple interpretations? How do you approach or think about this? [Edited this question -- see TL;DR]

Also, while this is a primary source, sometimes these authors also interpret other papers to fit their own narrative. I think this creates a tendency to misinterpret the cited paper, so by that assumption, you'd have to dig deeper to the primary source to determine the truth. But then again, I suspect that this quest of digging will lead you eventually to papers from 1930's or something. If your goal isn't to become an expert in this research field, this could easily be impractical, so here goes the question:

What are some of your heuristics to determine when you should stop digging?

TL; DR:

1. How do you make notes on your multiple interpretations for experimental results? New question: Can you give examples of your notes from these different 'layers of evidence'? One problem I have is naming my notes 😂
2. If your goal isn't to become an expert in this research field, what are some of your heuristics to determine when you should stop digging?

EDIT

I figured this was a stupid question -- the first question was answered here: https://zettelkasten.de/posts/layers-of-evidence/

Your notes should always reflect those different layers of evidence. They build on each other, not only in an abstract way. Either a note belongs to one of the layers, or it itself is divided according to those three layers. In the first case, it only contains phenomena, interpretations, or integrations. The notes are then linked to each other: An interpretation note links to a phenomenon note as reference, the integration notes link to the interpretation notes. In the second case, the note contains a description of phenomena at the top, followed up with the interpretation of the phenomena and the integration into the bigger picture. [...] It is not always possible to be that clean. But take it as an ideal to strive for. It is also a tool for thinking which is not an incidence. The Zettelkasten Method is the concrete manifestation of the abstract principles of good thinking practices and knowledge creation.

alkhan

I'm actively solving this problem after posting, and here's a bit of progress: (the arrows indicate connections)

It took me a while to generate these, but is this how you'd do it? It's more time consuming, for sure, but I believe this is due to the lack of skill than the process itself.

If I follow correctly, there should (ideally) be notes that deal with:
1. Facts/phenomenon
2. My interpretation of facts
3. How these facts fit into the big picture (synthesis)

I'd suspect that I should have less notes that synthesize interpretations and phenomena since they're naturally more generalized.

ctietze

Of course I can't judge the notes themselves, but the progress you show looks promising: separating observation from interpretation is still a 'killer feature' for me during note-taking. The difficulty you experience may wear off with time -- but this process is also intrinsically hard because you have to think, and thinking is difficult, and it's more difficult when things are new (both method and content). This is not a merely mechanic solution that requires fingers but no brain, unlike what people around the web seem to be looking for.

alkhan

Of course I can't judge the notes themselves, but the progress you show looks promising: separating observation from interpretation is still a 'killer feature' for me during note-taking. The difficulty you experience may wear off with time -- but this process is also intrinsically hard because you have to think, and thinking is difficult, and it's more difficult when things are new (both method and content). This is not a merely mechanic solution that requires fingers but no brain, unlike what people around the web seem to be looking for.

Thanks — and yes, it was indeed difficult! But I felt like it was more fruitful and "cleaner" than my prior notes. It's totally worth the 'extra brains and fingers' because now I can use multiple "lenses" to create interpretations and therefore maximizing the ideas generated from just a single paper.

However, two questions still remain

First, I still have problems naming the notes taken from observational studies. I don't know if this is good enough:

Second, what are some of your heuristics to determine when you should stop digging for the primary sources? In relation to above, I feel like when I take notes on a cited experimental study, (e.g. from a theory) it's considered "secondary." What's your opinion on this?

Will

@improveism said:
I'm actively solving this problem after posting, and here's a bit of progress: (the arrows indicate connections)

It took me a while to generate these, but is this how you'd do it? It's more time consuming, for sure, but I believe this is due to the lack of skill than the process itself.

If I follow correctly, there should (ideally) be notes that deal with:
1. Facts/phenomenon
2. My interpretation of facts
3. How these facts fit into the big picture (synthesis)

I'd suspect that I should have less notes that synthesize interpretations and phenomena since they're naturally more generalized.

I'd approach this in one of two ways.
First, I'd consider pretending a symbol to the title like when separating Phenomenom, interpretation, and synthesis into three notes:
For the phenomenom piece
⦽ Associating a Natural Language Mediator to a pair of words improved recall rate.
For the interpretation piece
⨍ Associating a Natural Language Mediator to a pair of words improved recall rate.
For the synthesis piece
Ω Associating a Natural Language Mediator to a pair of words improved recall rate.
Maybe they'd all have the same title verbiage maybe not. This has the advantage of being able at a glance to tell whether a note is observational or an interpretation (mine or others).

The second way would, and my preferred way, is to keep these three aspects of the idea in the same atomic note as they are intimately related, but I clearly mark each section of the note with headers and other callouts. It would be easier and clearer in the future when looking for this idea to tell what was who's doing the interpretation, who's doing the observation, and who's synthesis is being reported. But they would be co-mingled.

@improveism said:

However, two questions still remain

First, I still have problems naming the notes taken from observational studies. I don't know if this is good enough:

You might consider NLM instead of Natural Language Mediator to shorten the title. Otherwise, I think it tickles the findings and would lead me in the future to investigate.
I can't remember who I stole the idea from, sorry, but I try to make titles such that they help resurface the idea when I need it. And you seem to have done it.

Will

I found it thanks to my zettelkasten! A great testimonial to my zettelkasten's power and utility! I was linking a note I'm curating titled Separating Observation From Interpretation From Synthesis In Notes on this topic and stumbled on this!

I found where I stole the idea of a workflow of calling out "all the various interpretations and analytics." @Sascha alludes to various notes with interpretations and analytics in them, but I thought I could handle this in the same note for atomized ideas.

https://forum.zettelkasten.de/discussion/comment/7662/#Comment_7662

Will

@ctietze said:
...separating observation from interpretation is still a 'killer feature' for me during note-taking.

Tell us more. I want this "killer feature." I must not understand because I see interpretation as a synonym for observation. Maybe you've written about this before using different terms? Or maybe I'm just confused.

ctietze

@Will
Apart from Sascha's post about layers of evidence linked above, I don't think we have a central place where we discuss the distinction. At least I associate this more with remarks that were made in passing. A summary is also here: https://zettelkasten.de/posts/overview/#scaling-your-note-archive

It's the separation of describing the 'thing' first: that's the observation, like "every time Sascha kicks me in the groin, I topple over" (#domesticabuse); an interpretation would be "It seems that the groin is an area that flips the center of mass 90 degrees; activation via strong external force causes falling." Could both be in the same note, but practicing the distinction helped me figure out why rephrasing content sometimes was hard: I didn't re-tell 100% of the observation story, of the phenomena, and the original author didn't back the interpretation properly, so there was indeed a disconnect that I could uncover.

@improveism
When you process the paper, I suspect you'll end up dividing Associating a Natural Language Mediator to a pair of words improved recall rate even further. If you have a "Natural Language Mediator" concept definition note, then you can write things like Foo Bar et al found that associating an NLM[[202105210901]] with [what actually?] improves recall in a [observational/double-blind/...] study of [how many] participants over [how long] by [how many]% or something like that. I'm not good at that stuff because I process so little academic papers, though.

alkhan

@Will I've always loved your idea of signifiers! Super useful — thanks for the idea. Before reading your post, I thought that phenomena could just be determined with "past tense" but it doesn't hurt to have a common search pattern, too. (i.e. the signifiers)

Based on what you said above, I guess all of those concepts can be chunked up into a single pattern: have distinctions between phenomena/interpretation/synthesis or integration and let the content decide the refactoring, is that right?

BTW what do the "A-" and "B-" mean in your list? Curious about that.

@ctietze I was also thinking about refactoring it — but after processing, I've decided not to do it (yet...) because NLM didn't show up in most studies, and it was just a fancy term for an "arbitrarily assigned word that's part of our language" 😂

---

Just a quick testimony of the progress from the concept...it was extremely draining to incorporate these distinctions, but I now feel that I have a stronger opinion of the studies cited by most authors in their 'bestselling' books.

For example, I just processed Sophie Leroy's paper on attention residue, probably the "most cited paper on multitasking." Turns out that attention residue is just a form of "need for closure" and only affects, say, Task B when the "attentional pull" of Task A is much stronger than Task B. This "attentional pull" is determined by completion rate and your motivation to finish it.

The implication is that as long as you're not switching to another task that consumes your attention too much, then that "deep work" factor is still maintained.

---

And I have a new question, but I'll post it in another thread 😁 Thanks a lot for the help!

Will

@ctietze said:
It's the separation of describing the 'thing' first: that's the observation, like "every time Sascha kicks me in the groin, I topple over" (#domesticabuse); an interpretation would be "It seems that the groin is an area that flips the center of mass 90 degrees; activation via strong external force causes falling." Could both be in the same note, but practicing the distinction helped me figure out why rephrasing content sometimes was hard: I didn't re-tell 100% of the observation story, of the phenomena, and the original author didn't back the interpretation properly, so there was indeed a disconnect that I could uncover.

I love the humor (that is, if it is not a confession and a cry for help!! #domesticabuse.) Humor is a social grease that could and should be used more.

The distinction I'd be interested in and the one I watch for is "Who is doing the interpretation?" Is it an expert in her domain, a talking head on Fox, my neighbor telling me about the road crew, an author with little background in the topic, an author writing from experience, some yahoo on the web, friends, family, or even myself? I interpret what I observe, and there are flaws in my interpretation. I have to try and objectively weigh the value of the various interpretations I'm exposed to. There can be a breakdown in the validity of claims in any domain from any source, now or in the future. Navigating this messy maze is the hard work of building knowledge.

@improveism said:
Based on what you said above, I guess all of those concepts can be chunked up into a single pattern: have distinctions between phenomena/interpretation/synthesis or integration and let the content decide the refactoring, is that right?

BTW what do the "A-" and "B-" mean in your list? Curious about that.

No special mystery.

I don't use signifiers to split the distinctions between phenomena/interpretation/synthesis or other parts of ideas. I use them currently to separate types of structure notes. I'm rethinking this. I use these prepended signifiers as a visual queue when looking at the note list. I'm queued about the "link density," the form, and the context of the note. It also helps with finding the best candidate note for hanging a new note on.

A- is the onboarding of an article or paper.
A-Understanding Mathematical Creativity 202104240939
Which is what I'm currently factoring into my zettelkasten.
Henri Poincare's essay on Mathematical Creation from The Philosophy of Math Journal

B- is the signifier I use for books.
B-The Lone Ranger And Tonto Fistfight In Heaven 202102140705
Sherman Alexie's great humorous/tragic book on life on the Spokane Indian Reservation.

</distrations!>

I whipped up this script to see how many notes with prepended signifiers I have in my archive. This provided an hour's worth of distraction. I learned about doing math with variables in bash and rounding percentages.

# Find the number of notes with a prepended signifier.
#!/bin/bash
cd ~/Dropbox/zettelkasten/
folder_to_count="~/Dropbox/zettelkasten/"
note_count=$(ls | egrep -E -e '^.-|^[^0-9a-zA-Z]' |  wc -l)
note_total=$(ls *.md | wc -l)
percentage=$(printf %.2f%% "$((10**3 * 100 * $note_count / $note_total))e-3")

clear
echo ★★★★★
echo $note_count notes with signifiers preprended in $folder_to_count and that represents $percentage of the $note_total Total Notes in zettelkasten.

Saved as a file prepend.sh and run on the command line as sh prepend.sh.

The output is 414 notes with signifiers prepended in ~/Dropbox/zettelkasten/ and that represents 20.10% of the 2060 Total Notes in zettelkasten.

Interesting? Does this mean 79.90% of my notes are atomic?

<distractions!/>