Is the Philosophy of Knowledge Valuable for Practice?

System · June 10

This discussion was created from comments split from: The Iceberg Model and Atomicity.

tomp · June 9

@Andy said:

@tomp said that he didn't want to get into the meaning of knowledge, but that may be a very helpful issue to address briefly.

In a previous discussion, a book about knowledge graphs was criticized for defining them as "a graph of data intended to accumulate and convey knowledge of the world". The counterexamples of fiction and mathematics were mentioned as not being "knowledge of the world". I suggested that philosopher Ernst von Glasersfeld's definition of knowledge could be broad enough to encompass subjects such as fiction and mathematics: "knowledge refers to conceptual structures that epistemic agents, given the range of present experience within their tradition of thought and language, consider viable."

I'm going to illustrate why I said I didn't want to get into the meaning of "knowledge". I cannot understand Glasersfeld's definition, certainly not in any actionable way. And "viable" seems to be some kind of code word that may or may not be made clear in other parts of his work.

The most common current definition knowledge (by philosophers) seems to be (source: https://link.springer.com/chapter/10.1007/978-3-319-26551-3_2)

"traditionally defined as true justified belief"

and

"what is knowledge? Philosophers disagree; but they do not disagree about the truth-condition; a false belief cannot be knowledge"

But even so:

Almost all epistemologists, at the time and since, have agreed that Gettier disproved the justified-true-belief conception of knowledge

[Gettier produced some examples of someone having a belief that turned out to be true, but was true for reasons that had nothing to do with the spurious justifications that the person had in mind].

That's not helpful either.

Back to @andy:

Using von Glasersfeld's definition of knowledge, the name "knowledge building block" need not mean that the building blocks are used to assemble knowledge (although this is very often what a ZK is used for: even writing a story could be considered assembling knowledge in von Glasersfeld's sense); it can mean instead that the building blocks are conceptual structures that are viable for a given purpose.

Basically, this seems to mean whatever structures you like, use them. I don't need to read philosophy to see the point of this...

If you're doing science, you need conceptual structures that are viable for doing science; if you're writing fiction, you need conceptual structures that are viable for writing fiction.

In a personal knowledge base, what counts as viable is the business of the person creating the personal knowledge base, but if the purpose is one that other "epistemic agents" have shared in the past, it may help to learn from and adapt their knowledge—the conceptual structures that they found to be viable.

That sounds fine but I'm a physicist and engineer, and I couldn't really tell you what "building blocks" of science I would use in my Zettelkasten for notes about, lets say, the basics of quantum mechanics, or whether we "know" that dark matter exists. Perhaps they would turn out to be congruent with Sasha's six building blocks, but I'm not at all sure.

In my experience, inside or outside of a ZK, you muddle around for a while, then some things start to get clear, then maybe you see something to generalize. Sometimes you can solve a problem by first generalizing, then applying the generalization to the specific case. Sometimes you want to prove something by a syllogism but more often that's not possible, if only because most inputs to what might otherwise be a syllogism don't fit into a binary mold.

Philosophical discussions of "knowledge" aren't helping me.

Andy · June 9

@tomp said:

I'm going to illustrate why I said I didn't want to get into the meaning of "knowledge".

OK, but you used the word knowledge above ("The story as a whole is not going to be an assembled work of knowledge") which seems to imply that the word knowledge means something to you. If you're going to use the word, wouldn't you want to be able to say what you mean by it? Defining what knowledge means would help clarify what you're saying.

The most common current definition knowledge (by philosophers) seems to be [...] "true justified belief"

That quote from the abstract of the source you cited is imprecise. The full text of the source says, more precisely: "The most common view among philosophers [...] is that propositional knowledge is justified true belief" (JTB). But propositional knowledge is only one kind of knowledge. I understand von Glasersfeld's definition to be more general, presumably applying to any kind of knowledge. (I don't claim that it's the best definition.) And von Glasersfeld's definition bypasses the controversy around the JTB definition that you understandably want to avoid. Philosophy need not be so esoteric.

And "viable" seems to be some kind of code word that may or may not be made clear in other parts of his work. [..] Basically, this seems to mean whatever structures you like, use them. I don't need to read philosophy to see the point of this...

There's nothing technical or esoteric about the word viable; it means what my English dictionary says: "capable of working successfully". (This is what a lot of philosophy of science boils down to: as Joe Morrison put it, "Logical empiricists thought that we are justified in accepting hypotheses on the basis that they work... falsificationists think we are justified in accepting hypotheses just so long as they continue to work...") Talking about viability implies at least some criteria of viability or working successfully, and the lack of explicit criteria in the definition is what makes the definition so general, but the definition implies that if you have no criteria of viability that you have used to evaluate your conceptual structures then you have no ground for calling your conceptual structures knowledge. The definition differentiates conceptual structures that have been evaluated for viability and those that have not. So use whatever viable conceptual structures you like, but if you don't know what is viable in your domain, then you have to do some research into that. It doesn't mean use whatever structures.

Philosophical discussions of "knowledge" aren't helping me.

That's too bad. They have helped me!

tomp · June 9

@Andy said:
@tomp said:

I'm going to illustrate why I said I didn't want to get into the meaning of "knowledge".

...And "viable" seems to be some kind of code word that may or may not be made clear in other parts of his work. [..] Basically, this seems to mean whatever structures you like, use them. I don't need to read philosophy to see the point of this...

There's nothing technical or esoteric about the word viable; it means what my English dictionary says: "capable of working successfully".

And Mirriam-Webster on line says "capable of growing or developing", pretty close.

(This is what a lot of philosophy of science boils down to: as Joe Morrison put it, "Logical empiricists thought that we are justified in accepting hypotheses on the basis that they work... falsificationists think we are justified in accepting hypotheses just so long as they continue to work...") Talking about viability implies at least some criteria of viability or working successfully, and the lack of explicit criteria in the definition is what makes the definition so general, but the definition implies that if you have no criteria of viability that you have used to evaluate your conceptual structures then you have no ground for calling your conceptual structures knowledge. The definition differentiates conceptual structures that have been evaluated for viability and those that have not. So use whatever viable conceptual structures you like, but if you don't know what is viable in your domain, then you have to do some research into that. It doesn't mean use whatever structures.

I would say, whatever structures seem (to you) to be useful when you want to apply them. Actually, I don't think we are very far apart. I just don't think we need to depend on philosophical theories of "knowledge" or "structures". But however that balances out, it takes practice to develop skill in how to identify structure (or at least patterns) that work well for our notes in the context of why they are written.

Sascha · June 10

@Andy said:
@tomp said:

I'm going to illustrate why I said I didn't want to get into the meaning of "knowledge".

OK, but you used the word knowledge above ("The story as a whole is not going to be an assembled work of knowledge") which seems to imply that the word knowledge means something to you. If you're going to use the word, wouldn't you want to be able to say what you mean by it? Defining what knowledge means would help clarify what you're saying.

I think that there this call needs to be qualified. I am regularly torn between supporting the call to analyse the words we use. On the flip side, I share Taleb's anti intellectualism. A quote (hopefully, the original quote):

My dear Socrates … you know why they are putting you to death? It is because you make people feel stupid for blindly following habits, instincts, and traditions. You may be occasionally right. But you may confuse them about things they’ve been doing just fine without getting in trouble. You are destroying people’s illusions about themselves. You are taking the joy of ignorance out of the things we don’t understand. And you have no answer; you have no answer to offer them. source

I had such discussions with students of natural science (physics, chemistry, biology) and they were always sceptical about the value of scepticism. The possibility of being wrong was absolutely no reason for them to ditch the default naive realism that I accused them of.

The open task is the mass seduction to the metacognitive inquiry that is philosophy done right. The problem seems to follow the pattern of religious argumentation. The value of deep language analysis is not obvious and needs both rational and emotional justification.

This encapsulates the challenge perfectly:

Philosophical discussions of "knowledge" aren't helping me.

That's too bad. They have helped me!

An example of my coaching: I advertised systems thinking to a client regularly. He is smart, ambitious and surprisingly self-aware. But only when I showed him how to use a simplified version of the stock-flow model patterns presented by Donella Meadows in Thinking in Systems to create a bottom-up analysis of the current field of physical AI automation, he crossed the threshold of motivation to engage with the book.

@tomp said:
That sounds fine but I'm a physicist and engineer, and I couldn't really tell you what "building blocks" of science I would use in my Zettelkasten for notes about, lets say, the basics of quantum mechanics, or whether we "know" that dark matter exists. Perhaps they would turn out to be congruent with Sasha's six building blocks, but I'm not at all sure.

I would be surprised if these structures weren't present:

Concepts define a specific part of the world. You draw a boundary and say, “This is X.”
Arguments transfer the truth of a set of statements to another via a logical structure.
Counter-arguments disrupt the transfer of truth provided by arguments.
Models relate entities to each other and provide part-to-part relationships and part-to-whole relationships, often to map a part of reality or a fictional reality.
Hypotheses and theories formulate statements on how reality actually is. The difference is that a hypothesis is an isolated statement, while a theory comes with an inventory of methods.
Empirical observations are results of sensory probing on how reality actually is.

Don't you

define specific parts of the world ("with X I mean Y")
transfer the truth of a set of statements to another via a logical structure ("if this is true, then this is true also")
disrupt the transfer of truth ("it's not correct that if this is true, then that is true also")
relate entities to each other and provide part-to-part relationships and part-to-whole relationships
formulate statements on how reality actually is
conduct observations

?

I definitely share the reservations against the direct value of a lot of philosophical inquiry. However, based on my experience, I can attest that @Andy is right. What @Andy is suggesting is to engage in metacognition.

This is a short essay from 2012 of mine (machine translation):

I choose these terms because the terms I consider more appropriate are already established within academic philosophy. The distinction between practical and theoretical philosophy lies in the question “What should we do?” draws its boundary. Everything connected to this question belongs to practical philosophy; everything else belongs to theoretical philosophy.

Some questions of practical philosophy are:

What should I do? (Moral philosophy)
What is “oughtness”? (Ethics)
How should people be governed? (Political philosophy)

Some questions of theoretical philosophy are:

What exists in the world? (Ontology)
What can I know, and how can I know it? (Epistemology, Philosophy of Science)
What is language, and how does it exist in the world? (Philosophy of Language)

An incident during my philosophy studies led me to draw a new distinction. I wrote a term paper on the question of whether personality can serve as a criterion for personal identity. Put simply, the paper dealt with whether a person’s personality can change so profoundly that they are no longer the same person.

As an example of such a transformation of personality, I wanted to use the events of the Stanford Prison Experiment conducted by Philip Zimbardo. My intention was that a real case study—something that had actually happened—would have significant relevance to our lives. I assumed that my argument would therefore be especially convincing, because I was not referring to some far-fetched fiction but to the world as it actually exists around us.

My instructor’s counterproposal was to replace the experiment with a thought experiment in which a mad scientist, using some extraordinary device, radically alters a person’s personality. He was right that this example would be far less vulnerable to criticism. In the thought experiment, the personality change itself cannot be challenged, because it is simply stipulated. In the real experiment, by contrast, one could still question the alleged changes, the methods, and so forth. Academically speaking, my argument would have been stronger.

Nevertheless, I was not satisfied at all. I wrote the paper with the aspiration that, in some circumstances, I could also approach non-philosophers and non-academics and say: “Look here. Personality and personhood are related in this way.”

I wanted to formulate an answer that might have relevance for legal theory, or even for philosophical counseling. I wanted to formulate an answer that had something to do with life itself. A mad scientist with absurd machines—just imagine a lawyer presenting that in court as an argument for a reduced sentence on the grounds of radical personality change. Incidentally, Philip Zimbardo actually attempted something along those lines. I wanted to provide philosophical support for that argument.

This is what marks the distinction. Academic philosophy plays only by the rules of the university, of science, of the academy itself. It does so without necessarily concerning itself with whether it actually answers the question, “What should I do?” Philosophy of practice, by contrast, seeks to provide an answer to that question that can directly inform action. The distinction follows the same pattern by which basic research is separated from applied science.

The gap I see is this: applied philosophy is not taught, even though the question “What should I do?” is so central to human life. The result is that, on one side, there is a vast philosophical wasteland of academic research, and on the other side there are people who are searching for answers and genuinely need them, yet cannot find them because they mistake uncultivated land for barren land.

Thus the philosopher is confined to the ivory tower, even though a good philosopher belongs in the tent where the shaman once sat.

harr · June 10

@tomp said:
Philosophical discussions of "knowledge" aren't helping me.

I came to the same conclusion. :-)

@Andy said:
The definition differentiates conceptual structures that have been evaluated for viability and those that have not. So use whatever viable conceptual structures you like, but if you don't know what is viable in your domain, then you have to do some research into that.

Your recommendation prompted useful questions. What domains do I care about? What tools do I need in these domains?

I'm interested in domains like history, psychology and politics. I enjoy discussions about epistemology in those fields. "How do we know what we know?" is an important and difficult question in those fields. How do we know the politician X actually said Y? How do we know that there was a war in Z? How do we know that a therapy works? These are not abstract philosophical discussions about "knowledge" in general. These are practical discussion with real-life consequences.

So when it comes to personal note-making in a "Zettelkasten" (in the 20th century German sense of the word), I care about "knowledge" in a practical sense. I care about the kind of knowledge that can be managed in a personal "knowledge management system". I care about a tool for "knowledge work" in Peter Drucker's practical sense.

@ctietze once summarized the other view (emphasis added):

@Sascha and I, pioneered by Sascha's work on the ZK book and writing here, talk less about notes and more about ideas (in the Platonic sense, and in a sense that naturally fits our German language intuition, but which apparently isn't shared universally). Notes are the vessels, the representations; it's not about organizing notes, really. That's a chore, that can be busywork. It's about thinking, aided by a tool.
For me, that was a powerful shift in communication. Removing the conflation of "note" and the "thought" or impulse and association behind it helped bring the useful part into focus. Sascha stresses this in posts when he writes about creative knowledge work and thinking tools in a toolbox -- and how you're not doing "creative notes work" and "management".

I don't think that this shift in communication was beneficial. For me "Zettelkasten" is not about "thinking, aided by a tool". For me "Zettelkasten" is about "a tool that aids thinking".

Sascha's 2015 book Die Zettelkastenmethode introduced the six blocks in the context of a reading technique. I prefer other reading techniques. :-) I do enjoy bold statements about "discrete" knowledge building blocks and a "universal" inventory. But as I don't see how they would apply to the domains I care about, they become a distraction.

I find knowledge management tools and methods more useful, that do not interfere with domain-specific conceptual structures.

tomp · June 10

@Sascha said:

As an example of such a transformation of personality, I wanted to use the events of the Stanford Prison Experiment conducted by Philip Zimbardo. My intention was that a real case study—something that had actually happened—would have significant relevance to our lives. I assumed that my argument would therefore be especially convincing, because I was not referring to some far-fetched fiction but to the world as it actually exists around us.

Ironically it has turned out that the experiment was conducted and reported in a way that was close to being fraudulent and whatever it may have demonstrated was not what the Principal Investigator claimed.

My instructor’s counterproposal was to replace the experiment with a thought experiment in which a mad scientist, using some extraordinary device, radically alters a person’s personality. He was right that this example would be far less vulnerable to criticism. In the thought experiment, the personality change itself cannot be challenged, because it is simply stipulated. In the real experiment, by contrast, one could still question the alleged changes, the methods, and so forth. Academically speaking, my argument would have been stronger.

Much of Einstein's greatest work grew out of thought experiments. I wouldn't disparage them, but it's true making a good thought experiment is not easy.

For myself, I tend to be a modeler. That fits in very well with systems engineering, which is one area I worked in for some time. I work hard to devise the leanest model that seems to have a good chance of being realistic, with the fewest assumptions, and then try to make the most out of that. IMO there is an element of art in arriving at a good model, even if it's in a technical area.

And if you get right down to it, a good thought experiment is essentially a model, or at least a model underlies the thought experiment. At some point one will probably need to close the loop by checking the thought experiment's conclusions against the actual world being modeled.

Sascha · June 11

@tomp said:

@Sascha said:

As an example of such a transformation of personality, I wanted to use the events of the Stanford Prison Experiment conducted by Philip Zimbardo. My intention was that a real case study—something that had actually happened—would have significant relevance to our lives. I assumed that my argument would therefore be especially convincing, because I was not referring to some far-fetched fiction but to the world as it actually exists around us.

Ironically it has turned out that the experiment was conducted and reported in a way that was close to being fraudulent and whatever it may have demonstrated was not what the Principal Investigator claimed.

Can you give me a link?

My instructor’s counterproposal was to replace the experiment with a thought experiment in which a mad scientist, using some extraordinary device, radically alters a person’s personality. He was right that this example would be far less vulnerable to criticism. In the thought experiment, the personality change itself cannot be challenged, because it is simply stipulated. In the real experiment, by contrast, one could still question the alleged changes, the methods, and so forth. Academically speaking, my argument would have been stronger.

Much of Einstein's greatest work grew out of thought experiments. I wouldn't disparage them, but it's true making a good thought experiment is not easy.

For myself, I tend to be a modeler. That fits in very well with systems engineering, which is one area I worked in for some time. I work hard to devise the leanest model that seems to have a good chance of being realistic, with the fewest assumptions, and then try to make the most out of that. IMO there is an element of art in arriving at a good model, even if it's in a technical area.

100% agree. My point is that that academics (this is not specific to philosophers) avoid contact with reality way to often.

And if you get right down to it, a good thought experiment is essentially a model, or at least a model underlies the thought experiment. At some point one will probably need to close the loop by checking the thought experiment's conclusions against the actual world being modeled.

@harr said:

@tomp said:
Philosophical discussions of "knowledge" aren't helping me.

I came to the same conclusion. :-)

@Andy said:
The definition differentiates conceptual structures that have been evaluated for viability and those that have not. So use whatever viable conceptual structures you like, but if you don't know what is viable in your domain, then you have to do some research into that.

Your recommendation prompted useful questions. What domains do I care about? What tools do I need in these domains?

I'm interested in domains like history, psychology and politics. I enjoy discussions about epistemology in those fields. "How do we know what we know?" is an important and difficult question in those fields. How do we know the politician X actually said Y? How do we know that there was a war in Z? How do we know that a therapy works? These are not abstract philosophical discussions about "knowledge" in general. These are practical discussion with real-life consequences.

Especially, in those domains a hefty dose of the tools of analytical philosophy and metacognitive practices would greatly improve the work.

There is no need to dive into the complete historical discourse about the philosophy of knowledge. But the level of metacognitive skill in psychology and politics is, on average, abysmal.

Take one entry point: Haller and Krauss (2002) showed that students and their teachers share the same systematic misinterpretations of statistical significance — in a domain where inductive reasoning makes up the vast majority of cognitive demand. No wonder psychology was hit so hard by the replication crisis.

Haller, H. und Krauss, Stefan http://orcid.org/0000-0002-7440-9154 (2002) Misinterpretations of Significance: A problem students share with their teachers? Methods of Psychological Research Online 7 (1), S. 1-20.

Philosophy failed to make a tempting offer. But psychology and politics compounded that failure: their players actively defend the cozy, self-sealing bed of unscientific practice.

@ctietze once summarized the other view (emphasis added):

@Sascha and I, pioneered by Sascha's work on the ZK book and writing here, talk less about notes and more about ideas (in the Platonic sense, and in a sense that naturally fits our German language intuition, but which apparently isn't shared universally). Notes are the vessels, the representations; it's not about organizing notes, really. That's a chore, that can be busywork. It's about thinking, aided by a tool.
For me, that was a powerful shift in communication. Removing the conflation of "note" and the "thought" or impulse and association behind it helped bring the useful part into focus. Sascha stresses this in posts when he writes about creative knowledge work and thinking tools in a toolbox -- and how you're not doing "creative notes work" and "management".

I don't think that this shift in communication was beneficial. For me "Zettelkasten" is not about "thinking, aided by a tool". For me "Zettelkasten" is about "a tool that aids thinking".

Sascha's 2015 book Die Zettelkastenmethode introduced the six blocks in the context of a reading technique. I prefer other reading techniques. :-) I do enjoy bold statements about "discrete" knowledge building blocks and a "universal" inventory. But as I don't see how they would apply to the domains I care about, they become a distraction.

I find knowledge management tools and methods more useful, that do not interfere with domain-specific conceptual structures.

Don't you

define specific parts of the world ("with X I mean Y")
transfer the truth of a set of statements to another via a logical structure ("if this is true, then this is true also")
disrupt the transfer of truth ("it's not correct that if this is true, then that is true also")
relate entities to each other and provide part-to-part relationships and part-to-whole relationships
formulate statements on how reality actually is
conduct observations

?

harr · June 11

@Sascha said:
Especially, in those domains a hefty dose of the tools of analytical philosophy and metacognitive practices would greatly improve the work.

Do you have concrete recommendations for such tools and practices?

@Sascha said:
There is no need to dive into the complete historical discourse about the philosophy of knowledge. But the level of metacognitive skill in psychology and politics is, on average, abysmal.

Bold claim. :-)

@Sascha said:
Philosophy failed to make a tempting offer. But psychology and politics compounded that failure: their players actively defend the cozy, self-sealing bed of unscientific practice.

Bold claim. :-)

@Sascha said:
Don't you

define specific parts of the world ("with X I mean Y")

transfer the truth of a set of statements to another via a logical structure ("if this is true, then this is true also")

disrupt the transfer of truth ("it's not correct that if this is true, then that is true also")

relate entities to each other and provide part-to-part relationships and part-to-whole relationships

formulate statements on how reality actually is

conduct observations

?

I'd like to add context for readers who aren't aware of the context. The six items in the list refer to Sascha's inventory of knowledge building blocks. You can find the most recent version of the inventory in the section "What is Atomicity?" of The Complete Guide to Atomic Note-Taking.

Concepts: I don't define parts of the world, I define words. (I agree with the description in your 2025 book: "Eine Definition ist eine Beschreibung davon, was wir meinen, wenn wir ein Wort benutzen." Machine translation: "A definition is a description of what we mean when we use a word.")
Arguments: "truth" is a difficult concept in the domains that I'm interested in. The concept of a "transfer of truth" assumes that I can somehow determine that individual statements or sets of statements are true. Propositional logic and argumentation frameworks are useful, but not always applicable.
Counter-arguments: see 2.
Models: I do relate entities to each other. But depending on the purpose of my notes, these relationships can be captured in various forms, eg a simple definition or semantic links between nodes.
Hypotheses and theories: I formulate claims that I find useful for a particular purpose. I'm not a philosopher who ponders how reality "actually is". :-)
Empirical observations: I do collect observations and experimental data. But in the domains that I'm interested in they can be rather fuzzy and complex and interrelated. I think more in terms of sources of information.

tomp · June 11

@Sascha said:

@tomp said:

@Sascha said:

As an example of such a transformation of personality, I wanted to use the events of the Stanford Prison Experiment conducted by Philip Zimbardo. My intention was that a real case study—something that had actually happened—would have significant relevance to our lives. I assumed that my argument would therefore be especially convincing, because I was not referring to some far-fetched fiction but to the world as it actually exists around us.

Ironically it has turned out that the experiment was conducted and reported in a way that was close to being fraudulent and whatever it may have demonstrated was not what the Principal Investigator claimed.

Can you give me a link?

It's not where I first learned about the problems with the experiment and its reporting, but Wikipedia has an article with some references (which I haven't looked at yet, except for the Ars Technica one):

https://en.wikipedia.org/wiki/Stanford_prison_experiment

Other researchers have found it difficult to reproduce the study, especially given those constraints.[5]

Certain critics have described the study as unscientific and fraudulent.[3][6] In particular, Thibault Le Texier has established that the guards were asked directly to behave in certain ways in order to confirm Zimbardo's conclusions, which were largely written in advance of the experiment. Zimbardo claimed that Le Texier's article was mostly ad hominem and ignored available data that contradicts his counterarguments, but the original participants, who were interviewed for the National Geographic documentary The Stanford Prison Experiment: Unlocking the Truth, have largely confirmed many of Le Texier's claims.[7]

harr · June 11

One issue with the knowledge building blocks (KBB) might be, that they conflate two tasks: reading a text and structuring zettels.

I see some value in the KBB as a reading aid. I agree with @ctietze's comment that it is helpful to "define & recognize elements of information in (…) sources."

However, I'm not convinced that KBB should determine the granularity of zettels.

Sascha · June 12

@harr said:

@Sascha said:
Especially, in those domains a hefty dose of the tools of analytical philosophy and metacognitive practices would greatly improve the work.

Do you have concrete recommendations for such tools and practices?

@Sascha said:
There is no need to dive into the complete historical discourse about the philosophy of knowledge. But the level of metacognitive skill in psychology and politics is, on average, abysmal.

Bold claim. :-)

@Sascha said:
Philosophy failed to make a tempting offer. But psychology and politics compounded that failure: their players actively defend the cozy, self-sealing bed of unscientific practice.

Bold claim. :-)

Take one entry point: Haller and Krauss (2002) showed that students and their teachers share the same systematic misinterpretations of statistical significance — in a domain where inductive reasoning makes up the vast majority of cognitive demand. No wonder psychology was hit so hard by the replication crisis.

Haller, H. und Krauss, Stefan http://orcid.org/0000-0002-7440-9154 (2002) Misinterpretations of Significance: A problem students share with their teachers? Methods of Psychological Research Online 7 (1), S. 1-20.

These are not mere "bold" claims but supported by evidence of the outcome (replication crisis) and the mechanism (lack of essential skills in inductive reasoning in a domain where inductive reasoning makes up the vast majority of cognitive demand).

You present with your post a very good example of misleading framing. You cut out the source I am referencing to support my statements (first frame). Then you merely comment with "bold claim. :-)" (second frame).

But I don't know why.

harr · June 12

@Sascha said:
These are not mere "bold" claims but supported by evidence of the outcome (replication crisis) and the mechanism (lack of essential skills in inductive reasoning in a domain where inductive reasoning makes up the vast majority of cognitive demand).

The claims I highlighted are much broader. Let's pick one:

@Sascha said:
(…) the level of metacognitive skill in psychology and politics is, on average, abysmal.

When I read this claim, I wonder:

What does Sascha mean by "metacognitive skill"? (Definition)
How does he determine "the level" of such skills? (Metric)
What group of people in "psychology and politics" does he have in mind? (Scope)
Why does he emphasize "on average"? (Statistics)
What does he mean by "abysmal"? (Definition)

When I read your reference to the paper, I wonder:

Why doesn't Sascha provide a direct link to the paper, like this one? (An interesting paper by the way; thanks for the tip!)
How does this one paper about one particular aspect of psychological research support the much broader claim?

Interestingly, the paper supports the point I made with KKB 2 and 3. In psychology, the concept of a "transfer of truth" doesn't make much sense, when it's so difficult to know what's "true" in the first place. And it supports the point I made with KKB 6 that observations and experimental data in psychology "can be rather fuzzy and complex and interrelated."

@Sascha said:
You present with your post a very good example of misleading framing. You cut out the source I am referencing to support my statements (first frame). Then you merely comment with "bold claim. :-)" (second frame).

That my post is an example of "misleading framing" is another bold claim. :-)

@Sascha said:
But I don't know why.

First you're working from a wrong assumption. This is not about framing, but about highlighting.

I do exactly what you and Christian recommend. I recognize elements of information in a source. In this case claims.

I qualify them as "bold" because of tone and scope. In a book I'd underline such claims and write in the margin "bold claim!!!" :-)

Second you seem to misunderstand my quoting style. I quote only the parts of the post, that I'm replying to. It makes replies easier to read. The original context of the quote is easily accessible in a forum like this one. The reader just has to scroll back on the same page.

I don't know why others quote complete posts, when they're responding to only one sentence. :-)

harr · June 12

Does the paper support your generalized claim, that "their players actively defend the cozy, self-sealing bed of unscientific practice"?

I don't think so. I'd classify the paper as a counter-example. Apparently there are "players" who actively challenge a bad practice. :-)

Does the paper support the claim about "metacognitive skill"?

I don't think so. The authors sum up their findings:

Teaching statistics to psychology students should not only consist of teaching calculations, procedures and formulas, but should focus much more on statistical thinking and understanding of the methods. Although it can be assumed that this is an implicit aim of lectures on statistics, our survey suggests that these efforts are not sufficient. Since thinking as well as understanding are genuine matters of scientific psychology, it is astounding that these issues have been largely neglected in the methodology instruction of psychology students.

I don't think that "statistical thinking" is a "metacognitive skill", but a higher‑order, domain‑specific cognitive skill. Maybe you and I have different definitions of "metacognitive skill" in mind?

tomp · June 12

@Sascha said:
Take one entry point: Haller and Krauss (2002) showed that students and their teachers share the same systematic misinterpretations of statistical significance — in a domain where inductive reasoning makes up the vast majority of cognitive demand. No wonder psychology was hit so hard by the replication crisis.

Haller, H. und Krauss, Stefan http://orcid.org/0000-0002-7440-9154 (2002) Misinterpretations of Significance: A problem students share with their teachers? Methods of Psychological Research Online 7 (1), S. 1-20.

Not available there but I found it here:

https://www.researchgate.net/publication/27262211_Misinterpretations_of_Significance_A_Problem_Students_Share_with_Their_Teachers

Sascha · June 12

Second you seem to misunderstand my quoting style. I quote only the parts of the post, that I'm replying to.

Your quoting "style" is the mechanism by which you frame.

I am out of this conversation.

harr · June 12

@Sascha said:
Your quoting "style" is the mechanism by which you frame.

Interesting idea. I haven't thought about it in terms of framing. Another potential rabbit hole to jump into. :-)

I noticied that you were selective in your quoting yourself. :-) In comment 25418 you didn't quote the part where I directly replied to your question "Don't you …"

Personally, I would have preferred to stay on the topic of the six knowledge building blocks.

But I find the replication crisis also quite relevant for a thread about the practical value of the philosophy of knowledge:

How can the philosophy of knowledge help overcome the replication crisis?

And where do the six knowledge building blocks fit in all this?

tomp · June 12

@harr said:

But I find the replication crisis also quite relevant for a thread about the practical value of the philosophy of knowledge:

How can the philosophy of knowledge help overcome the replication crisis?

The replication crisis involves much more than misusing p-values. Best not to get into it here.

harr · June 12

@tomp said:
The replication crisis involves much more than misusing p-values.

Yes! And that makes it relevant here.

I had a little chat with my favorite bot and learned a new word: binary bias.

I don't understand it fully yet, but I wanted to share the idea.

One reason for the replication crisis seems to be that researchers are biased toward binary thinking. They like to organize their knowledge in discrete boxes. They like statements that are either true or false.

The article The Binary Bias: A Systematic Distortion in the Integration of Information (https://www.jstor.org/stable/26957509) says:

Our studies demonstrate a pervasive bias to treat evidence as binary. Summaries of data are systematically distorted because of a failure to properly weigh the strength of a given piece of evidence and instead evaluate it in an all-or-none manner. The errors presented in these studies are not due to complete misinterpretations of graphs, as sometimes occurs when laypeople misread a graph's entire message (Vekiri, 2002). Instead, they reflect more subtle errors that might influence even the most sophisticated researcher. These errors are not due to any particular visual feature of data visualization but occur even when people are considering information that is not explicitly statistical or visual in any way, suggesting that the error is a domain-general cognitive illusion.

This makes me wonder:

How can we think about knowledge, so that we don't fall for this illusion?

My bot had some interesting ideas:

For someone who prefers binary categories, the replication crises imply at least three shifts:
- You need to treat “true/false” more like “better or worse supported right now,” and accept that the status of many findings can legitimately change as new replications come in.
- You have to live with grey zones where we neither fully accept nor fully reject a claim, because available evidence is mixed or low‑quality.
- You must recognize that failures to replicate do not automatically make a result “false”; they often signal that the effect is smaller, more fragile, or more dependent on context than originally thought.
Psychologically, this can be uncomfortable, because binary thinking offers a feeling of certainty and control, while the actual epistemic situation in many sciences is messy and gradational. But philosophically, the replication crises push us toward a picture of knowledge where beliefs have degrees of justification, not just on/off labels, and where responsible trust in science means tracking how evidence accumulates rather than demanding once‑and‑for‑all verdicts.

What does this mean for our zettelkastens? How do we capture messy, gradational and probabilistic knowledge?

tomp · June 12

@harr said:

@tomp said:
The replication crisis involves much more than misusing p-values.

Yes! And that makes it relevant here.

[snip]

What does this mean for our zettelkastens? How do we capture messy, gradational and probabilistic knowledge?

Start exploring Debora Mayo's blog Error Statistics Philosophy. You should skim several years worth of posts. Mayo is a philosopher of science with a special interest in statistics and how it can be used to draw well-founded conclusions. But even this area is only a piece of a bigger problem. A growing and bigger part of the problem is with slop in the publication channels - not just AI (yet, though that part is growing fast).

Mayo emphasizes taking pains to be sure that a hypothesis has been "severely tested".

You might also get a lot out of reading Andrew Gelman's blog Statistical Modeling, Causal Inference, and Social Science. It goes back a long way and he has many many posts on various aspects of what used to be called the replication crisis.

Remember that you cannot assign a probability to the truth of a hypothesis. You can only talk about how well it has been tested. Of course, in the context of statistical tests, "hypothesis" has a very limited meaning compared with the way most of us would use the term.

harr · June 13

Interesting, but i'm looking for something much simpler. :-)

I learned that a technique that I've already been using has a name: hedging, vague writing or cautious writing:

For example:

"Psychology has a replication crisis."
"Insufficient inductive reasoning skills of academic researchers might contribute to the replication crisis."

I'm pretty sure about the first one, considering the amount of research addressing that crisis. The second one makes sense intuitively. However, the relationship between cause and effect is unclear. I have no evidence that supports the claim. So I use a language that indicates that I'm not sure (yet).

tomp · June 13

@harr said:
Interesting, but i'm looking for something much simpler. :-)

I learned that a technique that I've already been using has a name: hedging, vague writing or cautious writing:

https://wiki.uni-koeln.de/!gsgs/lib/exe/fetch.php?media=start:information_sheet_08_-_hedging.pdf

https://www.bristol.ac.uk/academic-language/media/BEAP/5.4/index.html

https://www.eapfoundation.com/writing/skills/hedging/

For example:

"Psychology has a replication crisis."

"Insufficient inductive reasoning skills of academic researchers might contribute to the replication crisis."

Hedging is one of those practices that gives academia a bad name. The example in your third link is especially egregious:

Although duration of smoking is also important when considering risk, it is highly correlated with age, which itself is a risk factor, so separating their effects can be difficult; however, large studies tend to show a relation between duration and risk. Because light smoking seems to have dramatic effects on cardiovascular disease, shorter duration might also be associated with a higher than expected risk.

This passage says nothing of any value and actually contradicts itself. It is worthless. Hedged and worthless statements contribute to the haze of slop hovering around some fields. Hedging is not the same as acknowledging uncertainty. Hedging is about trying to avoid blame for any possible error or later contradiction.

If you want to create a z-card that makes a statement but you think it's uncertain, you could add a field to the card's header. E.g., confidence: low.
Or `status: uncertain. Or you could link to another card that covers the reasons for the uncertainty. Just please don't get into hedging. You don't need to be protecting yourself from criticism from your future self.

harr · June 14

@tomp said:
Hedging is one of those practices that gives academia a bad name.

Does it? Does academia have a bad name? In what circles? Have you examples, where hedging was mentioned in particular?

I have the opposite impression. I believe that the replication crisis is mostly a consequence of over-confidence. Weak findings are presented as truths. Not to mention outright faking and manipulation of data, so that a study produces the expected results. A bit more humility in research and communicating of what we actually know should go a long way.

@tomp said:
The example in your third link is especially egregious: (…) This passage says nothing of any value and actually contradicts itself. It is worthless.

I don't find the example particularly well written either. But I don't find it worthless.

@tomp said:
Hedged and worthless statements contribute to the haze caof slop hovering around some fields.

Maybe it's the other way round? The fields deal with vague knowledge and hedging communicates that vagueness honestly?

@tomp said:
Hedging is not the same as acknowledging uncertainty. Hedging is about trying to avoid blame for any possible error or later contradiction.

This is an example where hedging becomes useful. Compare:

"Hedging is about trying to avoid blame for any possible error or later contradiction."
"I think that hedging is about trying to avoid blame for any possible error or later contradiction."
"Hedging might also be about trying to avoid blame for any possible error or later contradiction."

I read the first sentence as binary proposition. It's either true or false. You seem to believe it's true. Do you want me to accept it as a non-negotiable truth? If yes, there's nothing more to say. I read the second as an invitation to a dialog: this is what I think, what do you think? The third takes other aspects of hedging into account. It opens up a wide space of options on how to proceed.

@tomp said:
If you want to create a z-card that makes a statement but you think it's uncertain, you could add a field to the card's header. E.g., confidence: low.
Or `status: uncertain. Or you could link to another card that covers the reasons for the uncertainty.

I already tried to label my zettels as you described. I found the process too bureaucratic.

@tomp said:
Just please don't get into hedging.

I find hedging an excellent writing practice, because it reminds me of potential gaps in my knowledge. It makes it easy to ask relevant follow-up questions.

Hedging also works in other contexts like forum discussions. How else would you communicate concisely that you know that your claim is uncertain or has little evidence?

Unfortunately I don't remember where I learned to talk and write this way. I think it was in another context.

I encourage you to try out hedging. It makes it much easier to manage vague knowledge.

Also, there's no need to hedge everything. I enjoy bold claims. They can shake things up in a good way.

@tomp said:
You don't need to be protecting yourself from criticism from your future self.

I don't want to fool my future self. I want to be honest with my future self. In my personal notes I acknowledge what I don't know.

I consider the hedges an invitation for future research, an invitation to replicate, an invitation to reevaluate.

harr · June 14

I asked my favorite bot about parallels between hedging and the philosophy of knowledge. It mentioned these keywords:

fallibilism - knowledge is revisable
epistemic humility - acknowledge limits of own knowledge
skepticism - all claims can be challenged
social epistemology - collaborate with others

These are aspects of the philosophy of knowledge that I find valuable for practice.

harr · June 17

@harr said:

@tomp said:
Philosophical discussions of "knowledge" aren't helping me.

I came to the same conclusion. :-)

And I changed my mind. :-)

When we're talking about Sascha's and Christian's Zettelkasten Method (ZKM), there is a practical value in being aware of relevant philosophical traditions. It helps resolve misunderstandings. When Sascha talks about "knowledge", "ideas" and "thoughts", he seems to have a particular branch of Analytical Philosophy in mind.

I find philosophical discussions helpful, when they help understand something. In this case I was struggling with some core assumptions of ZKM, because I didn't understand the philosophical foundation. I still disagree with some core principles of ZKM, because I don't find them practical for my purposes and because I don't share Sascha's priorities.

But on the way I learned to also better understand my own priorities.

Philosophical discussions are helpful for becoming aware of other viewpoints and other ways to think. They can help people get unstuck. They can open new paths.

ZettelDistraction · June 21

I use my Zettelkasten to record definitions, technical notes, configurations that I am likely to forget, and what you might call "gists."

I'm too busy with projects to cultivate the Zettelkasten for its own sake. It's the repository of last resort. Everything else has failed: index cards, Markup, Markdown, YAML, emails to myself, wikis, and transcranial direct-current stimulation.

My Zettelkasten has become a roach motel for ideas: they check in, but don't check out. Note-taking proceeds from projects to Zettelkasten. My Zettelkasten isn't what it says on the tin: an incubator for a mind teeming with original thought.

I have yet to encounter evidence that philosophical distinctions improve what enters the system or what comes out. I've never been surprised by following links in my Zettelkasten. Appalled and embarrassed, yes, but surprised, never.

tomp · June 21

@ZettelDistraction said:
I use my Zettelkasten to record definitions, technical notes, configurations that I am likely to forget, and what you might call "gists."

I'm too busy with projects to cultivate the Zettelkasten for its own sake. It's the repository of last resort. Everything else has failed: index cards, Markup, Markdown, YAML, emails to myself, wikis, and transcranial direct-current stimulation.

My Zettelkasten has become a roach motel for ideas: they check in, but don't check out. Note-taking proceeds from projects to Zettelkasten. My Zettelkasten isn't what it says on the tin: an incubator for a mind teeming with original thought.

I posted in another thread my idea of the kinds of thinking support a PKM-like or ZK-like system can offer. This is not related to Sascha's levels; it is all about how the system can support the user.

Level	Type of Support	Example of Thinking Support
Level 1	Facts	What paint I used last time I painted this room
Level 2	Threads	Chains of thought, steps of an argument, related facts or ideas
Level 3	Extension	Create new threads, e.g., for writing
Level 4	Expansion	Finding new connections
Level 5	Emergence	Spawning new discovery and creation

Most of what @ZettelDistraction describes seems like Level 1 to me. We all need to save and find Level 1 information, usually a lot of it, and why not in one's ZK? Then at least one knows where it is and how to go looking for it. I do the same.

With the right system, there is no good reason why such notes should interfere with using a ZK for higher levels of support. And if, with some not-quite-so-right system, there should be too much interference, one could use one ZK for all those level 1 notes and another for levels 2 and up.

For myself, I keep Level 1 notes in my ZK, and my browser bookmarks in a near-ZK system which is almost the same but slightly specialized for bookmarks.

ZettelDistraction · June 21

@tomp said:

I posted in another thread my idea of the kinds of thinking support a PKM-like or ZK-like system can offer. This is not related to Sascha's levels; it is all about how the system can support the user.

Level Type of Support Example of Thinking Support
Level 1 Facts What paint I used last time I painted this room
Level 2 Threads Chains of thought, steps of an argument, related facts or ideas
Level 3 Extension Create new threads, e.g., for writing
Level 4 Expansion Finding new connections
Level 5 Emergence Spawning new discovery and creation
Most of what @ZettelDistraction describes seems like Level 1 to me. We all need to save and find Level 1 information, usually a lot of it, and why not in one's ZK? Then at least one knows where it is and how to go looking for it. I do the same.

With the right system, there is no good reason why such notes should interfere with using a ZK for higher levels of support.

That taxonomy is useful. My description was incomplete: I also link notes when they form a chain of thought, and occasionally use such chains when I write, so my Zettelkasten reaches Level 2 and sometimes Level 3. I keep one Zettelkasten; the Level 1 material does not interfere with the rest.

The direction is still mostly projects → Zettelkasten. Projects generate the threads; the Zettelkasten records them. I have not observed the links themselves carrying me into expansion or emergence.

The levels suggest the need for a benchmark: agreed tasks, a simpler baseline system, and measurable outcomes. Without such a benchmark, claims about what a Zettelkasten offers often devolve into unfalsifiable philosophical speculation. Success can be credited to the method, while failure can be attributed to the system not being quite right—or to the user not quite getting it.

I have started updating my public flengyel/Zettel GitHub repository to document the system I actually use.

tomp · June 21

@ZettelDistraction said:

@tomp said:

I posted in another thread my idea of the kinds of thinking support a PKM-like or ZK-like system can offer. This is not related to Sascha's levels; it is all about how the system can support the user.

Level Type of Support Example of Thinking Support
Level 1 Facts What paint I used last time I painted this room
Level 2 Threads Chains of thought, steps of an argument, related facts or ideas
Level 3 Extension Create new threads, e.g., for writing
Level 4 Expansion Finding new connections
Level 5 Emergence Spawning new discovery and creation
Most of what @ZettelDistraction describes seems like Level 1 to me. We all need to save and find Level 1 information, usually a lot of it, and why not in one's ZK? Then at least one knows where it is and how to go looking for it. I do the same.

With the right system, there is no good reason why such notes should interfere with using a ZK for higher levels of support.

That taxonomy is useful. My description was incomplete: I also link notes when they form a chain of thought, and occasionally use such chains when I write, so my Zettelkasten reaches Level 2 and sometimes Level 3. I keep one Zettelkasten; the Level 1 material does not interfere with the rest.

The direction is still mostly projects → Zettelkasten. Projects generate the threads; the Zettelkasten records them. I have not observed the links themselves carrying me into expansion or emergence.

Level 4/5 support isn't limited to links and keywords. It's very dependent on the practitioner and the moment. In fact, noticing a new relationship most likely means there wasn't a previous link annotated between the corresponding notes. Here's an example contained in one of my z-cards:

Even a phonetic similarity can ignite if user is receptive

Example:
I search for "bugle" and as I look at the responses I think of "beagle". I wonder what I have on beagles. I search through notes on beagles and on dogs. This could eventually lead to interesting thoughts on dogs and music, was howling the (evolutionarily) earliest kind of dog vocalization, or who knows what else!

The user has to be open to noticing associations. This can be learned.

What does the ZK system contribute in a case like this, where there are no pre-existing links? It's the ability to quickly and with low friction make a query that shows its results in a semantically helpful way. In some cases, e.g., if the system uses an outline, an interesting connection can be noticed by closeness in the outline.

In this example, the system might contain some cards on the origin of certain sounds in folk music, but didn't include dog howls. It turns out that my own system only has one possibly related card: Musical analogies and metaphors. Even that one card is stimulating thoughts in my head already, wondering how musical analogies are related to natural sounds.

It may also be interesting to know that this latter card is located at (or has the compound keyword of) Current Interest-->Placeholders-->Zettelkastens. I set this up when I was doing some work on the nature of Zettelkastens. It is still an open question on whether I want to remove it from its provisional status, but even being provisional, it's turning out to be interesting and stimulating in ways I didn't expect when I wrote the card.

Another example is unfolding as I write - I looked back at the bugle-beagle example and realized that at a deeper level the connection is about unexpected or unusual uses of sounds. So i searched for "sound" and up came one card: Computers-->Operating Systems-->Linux-->Sounds-->Play a sound. It's a Level 1 card to remind me of the syntax to play a sound file in Linux. Now that my brain is thinking about unusual ways to use sounds, I started to wonder about writing a computer program in iambic pentameter or as a haiku. The latter reminds me of the Obfuscated C contests. That in turn reminds me of some computer languages that were developed to be as obscure as possible, like brainf*ck

By now I have so many ideas floating around my head that I could start half a dozen new lines of inquiry, if I were interested enough and had the energy to follow them up.

This is well supported by my system, which emphasizes low cognitive friction, low-latency search and navigation, and showing results in their semantic neighborhoods (so that topics that are somehow related are surfaced easily).

ZettelDistraction · June 21

@tomp said:

...

That is a useful example of associative expansion, but it does not yet demonstrate Level 5 emergence. It shows low-friction search prompting an associative user. Pre-existing links do no work in these examples, and I do not yet see a Zettelkasten-specific the effect.

What baseline would distinguish the contribution of the system from ordinary full-text search or an outline? And what result would count against the claim that the system supports expansion or emergence?

ZettelDistraction · June 21

Let me put it this way. A useful theory would say:

Under task $(T)$, feature $(X)$ improves outcome $(Y)$ over baseline $(B)$, and result $(R)$ would count against this claim.

I am willing to update my Zettel GitHub repository to create and host a benchmark. What I am not prepared to do is run all the experiments myself, given the work involved. The repository can host the protocol, implementation, tasks, and results, but without some collaboration the ostrich won't take off.

My claim is that without a benchmark or reference system, these accounts remain anecdotes. They may accurately report an experience, but they cannot establish what caused it, whether a simpler system would have worked as well, or what result would count against the explanation.

Zettelkasten Forum

Is the Philosophy of Knowledge Valuable for Practice?

Comments

Even a phonetic similarity can ignite if user is receptive

Howdy, Stranger!

Quick Links

Categories

In this Discussion