Why Categories for Your Note Archive are a Bad Idea

ZettelDistraction

@Sascha said:

@ZettelDistraction said:
Whatever @Sascha said can be disregarded, on account of:

ignore naysayers and those with purist tendencies…
– Chris Aldrich. Comment on Does anyone else work in project-based systems instead? : Zettelkasten (reddit.com).

Can you explain? I don't understand.

I cannot follow the precept of @chrisaldrich and offer an explanation.

I'll wait for a scholar to weigh in.

I love your lust for authority.

Sascha, we are on the Internet, a place where the epistemic authority of experts has been undermined. By joking about a genuine social problem (maybe it's much worse in the US than in Germany, which could be), we are forgetting what the scholars at Universität Bielefeld (and everywhere else) are up against:

Sustaining epistemic authority depends, crucially, on social institutions that inculcate reliable second-order norms about whom to believe about what. The traditional media were crucial, in the age of mass democracy, with promulgating and sustaining such norms. The Internet has obliterated the intermediaries who made that possible (and, in the process, undermined the epistemic standing of experts), while even the traditional media in the U.S., thanks to the demise of the “Fairness Doctrine,” has contributed to the same phenomenon.

The previous quotation was from the abstract of Leiter, Brian, The Epistemology of the Internet and the Regulation of Speech in America (January 9, 2022). Georgetown Journal of Law & Public Policy, 2022, Available at SSRN: https://ssrn.com/abstract=3939948 or http://dx.doi.org/10.2139/ssrn.3939948.

But why are you waiting for a scholar to weigh in when Luhmann stated this himself?

This isn't how professional scholarship works. The Luhmann Archive project at Bielefeld would grind to a halt and lose its funding (such as it is) if they were to just take Luhmann's word for it. A scholar would approach this differently: they would look at the entire Zettelkasten, the archives and the published record to see how Luhmann worked. Did each book and paper that Luhmann derived from his Zettelkasten constitute a project in its own right? Or were they all to be regarded as waystations or milestones under the single project, "The Theory of Society." "Well, he said he had one project, so that's that." Do we make the scholarly point by appending a smiley face?

Sascha

@ZettelDistraction said:

@Sascha said:

@ZettelDistraction said:
Whatever @Sascha said can be disregarded, on account of:

ignore naysayers and those with purist tendencies…
– Chris Aldrich. Comment on Does anyone else work in project-based systems instead? : Zettelkasten (reddit.com).

Can you explain? I don't understand.

I cannot follow the precept of @chrisaldrich and offer an explanation.

But perhaps you can explain what you are trying to communicate. I don't get how the quote is grounds to disregard what I wrote.

I'll wait for a scholar to weigh in.

I love your lust for authority.

Sascha, we are on the Internet, a place where the epistemic authority of experts has been undermined. By joking about a genuine social problem (maybe it's much worse in the US than in Germany, which could be), we are forgetting what the scholars at Universität Bielefeld (and everywhere else) are up against:

Sustaining epistemic authority depends, crucially, on social institutions that inculcate reliable second-order norms about whom to believe about what. The traditional media were crucial, in the age of mass democracy, with promulgating and sustaining such norms. The Internet has obliterated the intermediaries who made that possible (and, in the process, undermined the epistemic standing of experts), while even the traditional media in the U.S., thanks to the demise of the “Fairness Doctrine,” has contributed to the same phenomenon.

The previous quotation was from the abstract of Leiter, Brian, The Epistemology of the Internet and the Regulation of Speech in America (January 9, 2022). Georgetown Journal of Law & Public Policy, 2022, Available at SSRN: https://ssrn.com/abstract=3939948 or http://dx.doi.org/10.2139/ssrn.3939948.

I don't have any mercy for the scholars at the university. They are in a similar relationship like bodybuilders in the beginning of the internet:

When YouTube as a platform started to conquer more and more shares of the attention market (cf. Tim Wu - Attention Merchants), professional bodybuilders were marginalized. Dilettantes, who perhaps could not perform (flesh mass and resistance of the liver to poisoning by drugs), but knew how to get into direct contact with their audience, received much more attention than the Mr. Olympias.

The bitterness set in completely when these dilettantes, their sometimes provocatively displayed lack of respect for established norms, began to earn much more money with much less work and effort.

The old bodybuilders could no longer rely on gatekeepers and arbiters of epistemic authority. Anyone was free to pick up a camera and shoot a video, or post an article on a blog.

Now bodybuilding has been primarily for introverts. Bodybuilding is a solitary sport. It isolates socially because you are out of sync with your social environment. You eat differently, sleep earlier, get up earlier, you're no longer open to pleasures like eating out or partying. And now people like Zyzz came along and violated fundamental beliefs of the bodybuilding scene.

This was the time of a great selection: there were people who understood this stress as information and others disappeared.

The mind can go either direction under stress-toward positive or toward negative: on or off. Think of it as a spectrum whose extremes are unconsciousness at the negative end and hyperconsciousness at the positive end. The way the mind will lean under stress is strongly influenced by training. - Frank Herbert, Dune

There were some serious disappointments in the time when I thought I understood what a scholar was doing at the university. Two memorable ones for me are:

1. I witnessed intellectual theft on several occasions. A graduate student used my paper in a seminar and simply appropriated it without mentioning me. At the time, I thought nothing of it. Later I understood that my student thoughts were considered "free real estate". There were some scandals that professors and teachers stole among themselves. The university and its scholars do not strive for knowledge, but use knowledge to play the game of power.
2. there is absolute inefficiency at the university. I have long felt sorry for the "publish or perish" culture. Today I am open to the proposition that this is not a necessary evil to push the overall operation of the university in a proper direction.
3. one of the main motivations of many scholars is to escape the market and its pressures. The university, unfortunately, is an all too popular hiding place for people who want to escape the world.

This violation of virtue-ethical principles is one thing. But, at least in Germany, public funding is another: The state, a soulless bureaucratic apparatus, extracts money from my mother. She has no choice, because the state enforces this demand for money by force if necessary. Part of this money is then put into the university apparatus. And a part of this money is then played into the hands of people who deal with completely obscure partial aspects of a special debate of a special debate, because they don't want that their work is also evaluated on its usefulness (e.g. some answer to an answer about the question of the difference between knowing-that and knowing-how).

In short, my mother is being forced by force to give her hard-earned money to people who are setting up a cozy hideaway in the university. This goes beyond my virtue-ethical outrage, which is my private business. The state here acts even more reprehensibly than the mafia, which at least shows up in person to collect the money for its imposed services (in some places, the protective service provided by the mafia is nothing more than a private protective service). The state, on the other hand, is a soulless bureaucracy that trains people to reduce other people to numbers in a spreadsheet or forms to be processed.

Interim: Classical bodybuilders and university scholars have in common the loss of external gatekeepers and institutions that give them epistemic authority.

Given the multiple instances of corruption (falsified data, falsifying research for industry, fraud for personal gain, etc.) that have done considerable damage to humanity, the aforementioned is sufficient for me to welcome a democratization of collective research. After all, too large and tyrannical institutions will have some of their power taken away.

Another commonality to bodybuilders is that especially introverts and/or people with poor (often self-inflicted) social skills complain about having their protective cocoon taken away. This is where I personally can feel some sympathy. When what one takes for granted and relies on is taken from one, it is hard to understand. But then I have to confront it and grow from it. For people who do not take on this task, I feel no pity. Those who fail in ability fail honorably. Those who fail in character do not.

The loss of epistemic authority is also a democratization of research. And both must adapt:

1. researchers must learn to present, monetize, and expose themselves to free market forces. This also means admitting that what used to be a leisure pursuit for the wealthy was leisure pursuit for good reasons. History, most of philosophy, literary criticism -- all of these belong back in the realm of leisure, not serious research.
2. learners need to learn the basic tools of the trade. What empiricism even means, how arguments work, and how to work on concepts until they are embraced as useful tools.

Researchers need to give up the notion that they don't have to worry about their epistemic authority and learners that everything is chewed up for them.

Nietzsche might say: only those who sharpen their own fangs can become truly mature.

(Translated by DeepL. Some wording might be off)

But why are you waiting for a scholar to weigh in when Luhmann stated this himself?

This isn't how professional scholarship works. The Luhmann Archive project at Bielefeld would grind to a halt and lose its funding (such as it is) if they were to just take Luhmann's word for it. A scholar would approach this differently: they would look at the entire Zettelkasten, the archives and the published record to see how Luhmann worked.

Did each book and paper that Luhmann derived from his Zettelkasten constitute a project in its own right? Or were they all to be regarded as waystations or milestones under the single project, "The Theory of Society."

This is just an arbitrary decision on how you draw boundaries. This is not worth any dime of my mother's pockets. This is hobbiest works

"Well, he said he had one project, so that's that." Do we make the scholarly point by appending a smiley face?

Perhaps, just research in a more valuable direction.

ZettelDistraction

Ok. I was referring to professional scholars with a modicum of integrity. You moved the goalpost on Luhmann. I never learn. Merry Christmas.

Sascha

@ZettelDistraction said:
Merry Christmas.

Merry Christmas to you, too. And a happy new year.

ZettelDistraction

Maybe I will try an index consisting of 26 cards (or fewer) instead of categories. Some of the note sequences would blend into other categories if they weren't stuck in the one I started with. At some point there must a discontinuity in the ID assignment from one category to another.

ZettelDistraction

I had written that I did away with categories and category notes in favor of an alphabetic index. But categories and category notes were eliminated in name only. What an alphabetic index gives you is a collection of "soft" category notes, or a collection of structure notes which organize the notes of your Zettelkasten alphabetically.

Here's my index now. The ID numbers are chosen so that the index is at the top of the file listing.

The rule for assigning an ID to a note is that the first place of the ID is an alphanumeric sequence consisting of at least one and at most four of the initial letters or digits of one of the words or numbers of the title. The rest could be a timestamp. The note links back to the index note containing the first letter of the ID.

This does mean that I will need "structure notes" (AKA "category notes"--see below) if I want to follow threads within a single note. Folgezettel are still possible, but they will be less useful. Against that, the IDs will not get "stuck" under deeply nested categories, since they will tend to move around alphabetically. And finding them in the file listing is possible by looking for the classifying initial sequence of the ID. I could not do this with timestamp IDs alone.

I have jumbled up my ZK with this maneuver. It might stabilize, eventually.

A distinction without much difference in practice

There isn't that much difference between a category note and a structure note, or a hub note. As long as you have a structure note, links from it form a category by definition: the set of notes linked to by the structure note. Typically there is an organizing notion that binds the links of a structure note together, which gives you a category. Conversely, a category note is already a structure note--if there are only backlinks from other notes to the category note, you could convert the category note into a structure note by adding all of those backlinks to the category note. This process is reversible like a change of coordinates, so you might as well call a category note a structure note. ^†

The argument over structure notes versus category notes is over a distinction without that much of a difference. I will say though that starting with an alphabetic index seems to be preferable to starting with a selection of categories in the long run. But the alphabetic index is another kind of collection of category notes, under which other notes that start with a given letter are in the same category. Or else it's another collection of structure notes, under which other notes that contain words starting with a given letter (in the title or elsewhere) are linked to from the same structure note, or which link back to the same hub note.^‡

---

^†Perhaps categories are less general than structure notes, which can be entirely random. I leave such fine distinctions to the analytic philosophers.
^‡It's possible that an alphabetic index doesn't box you in as much as an initial list of categories, relying on internal ramification to find subcategories of these, and so on. A hub note which serves only as a target for backlinks can save some typing (in the body of the hub note). The notes such a hub note classifies are visible in a "what links here" pane in some ZK editors. We are past the point of diminishing returns.

Will

@ZettelDistraction I've been following your discussion of category notes both here and here. Interesting, brain-stunning, and thought-provoking, as usual.

A distinction without much difference in practice

The semantics of zettelkasting is arcane and convoluted. We'll see how your shift away from folgezettel-style titled category notes to an alphabetical index plays out as you use it. Please report back to us your findings.

I do have a couple of questions.
1. Do you use 'sub-category' notes? Below is a screenshot of a category note from my ZK called "G-Poetry 202301290800". In your system, it would be in the 0000.0000.00.6 P-Q-R bin. In my system, it is a hub note. It contains a small listing of connected sub-hubs. They were sections of the main hub that grew in complexity, warranting their own note. This is my approach to keeping links so they "will not get "stuck" under deeply nested categories." A link is on a hub or a sub-hub.
2. Can you show us what the note 0000.0000.00.9 Y-Z looks like? This might help clarify.

I have jumbled up my ZK with this maneuver, and I hope it stabilizes...this seems about right.

I empathize. Making a major shift in the overall structure of our ZK is time-consuming and we don't know if things will be better once the dust settles.

Typically, there is an organizing notion that binds the links of a structure note together, which gives you a category.

/#definition_of_sturcture_note

I will say though that starting with an alphabetic index seems to be preferable to starting with a selection of categories in the long run. But the alphabetic index is another kind of collection of category notes, under which other notes that start with a given letter are in the same category.

I'd find it hard to use an alphabetical system like this. I'd never remember if I called the note I was looking for "Online and Built-In Proofreading Tools 202301241040" or "Built-in and Online Proofreading Tools 202301241040" or if I put it in the category of Writing tips? But I can't remember the last time I might have stumbled around looking for a note without using ★SEARCH★, so it wouldn't matter where it was categorized, I'd find it.

ZettelDistraction

@Will said:
@ZettelDistraction I've been following your discussion of category notes both here and here. Interesting, brain-stunning, and thought-provoking, as usual.

Shucks, thank you @Will for your kind words.

A distinction without much difference in practice

The semantics of zettelkasting is arcane and convoluted. We'll see how your shift away from folgezettel-style titled category notes to an alphabetical index plays out as you use it. Please report back to us your findings.

Will do.

I do have a couple of questions.
1. Do you use 'sub-category' notes? Below is a screenshot of a category note from my ZK called "G-Poetry 202301290800". In your system, it would be in the 0000.0000.00.6 P-Q-R bin. In my system, it is a hub note. It contains a small listing of connected sub-hubs. They were sections of the main hub that grew in complexity, warranting their own note. This is my approach to keeping links so they "will not get "stuck" under deeply nested categories." A link is on a hub or a sub-hub.

Yes, sub-category notes are structure notes, and I had those in the previous system, where the categories broke down and I needed to insert new category notes (AKA structure notes) under the existing top-level categories.

2. Can you show us what the note 0000.0000.00.9 Y-Z looks like? This might help clarify.

Apologies, I must have changed my indexing after you wrote this, so our messages crossed. Here is the new index:

Ideally I would have gotten further with the ID reassignments before I show you the contents of an index note. But here is one in its current state.

I had to redo the ID format for this. The regular expression that matches my
IDs is

((\w{1,4}\.){2,}\d\w{3})

This regular expression has several properties:

1. The regex is backward-compatible with all but 22 of the pre-existing IDs so far. The 22 that don't match don't present a problem. Linking them again will be interesting..
2. The regex will match the IDs of the index above
3. I can use four-letter words or abbreviations in IDs (exception: three letters after a digit at the end) such as "Cite.1234.5678.0key" for a note about cite keys. That's useful because the filename tells me something about the note. This is not possible with timestamp IDs serving as filenames.
4. The regex is unlikely to match strings of symbols used for any other purpose in any note.
5. The regex allows three-letter words or letter combinations at the end.
6. It limits the length of Folgezettel to at most 4. (However, I can extend them after a period if I wanted). This tends to push me toward links and hub notes.
7. The periods between alphanumeric groups (of length between one and four) are compatible with common operating system filename conventions, such as Linux, Windows and MacOS.
8. It is not possible to end an ID with a common filename extension, such as ".md", ".pdf", ".txt", ".png" etc., because the length of the last group of five characters is a period followed by a digit, followed by three alphanumeric characters. The common file extensions consist of two or three alphanumeric characters.
9. Zettlr can identify IDs in notes with this regex. Some longer regexes exceed Zettlr's capacity (if I am not mistaken).

The reason why point 4 matters is that Zettlr will highlight anything that matches the regex. This indicates that an expression is an ID.

Note to self: update the Obsidian plugin to match this.

I have jumbled up my ZK with this maneuver, and I hope it stabilizes...this seems about right.

I empathize. Making a major shift in the overall structure of our ZK is time-consuming and we don't know if things will be better once the dust settles.

I've throttled the numbering enough to completely confuse Obsidian. It could be that I need to change a regex in an Obsidian plugin though--I was focusing on Zettlr first.

As far as notes go, so far I have

flengyel@DESKTOP-M2MA3HI:~/pythonics$ ./zkmismatch.py
Total # of Zettels: 0
Well-formed Zettels: 0
Malformed Zettels: 0
Non-Zettel or invalid format: 0

That can't be right--I need to fix the regular expression in zkmismatch.py too.

flengyel@DESKTOP-M2MA3HI:~/pythonics$ ./zkmismatch.py
MISMATCH! H1 ID: C.1.1b.22.0306 Filename: 0.1.1b.22.0306.md Title: Cross-cutting categories in Zettelkasten
MISMATCH! H1 ID: Rply.1b3a.0.22.1227 Filename: 0.1b3a.0.22.1227.md Title: Reply to a comment on YouTube video.
MISMATCH! H1 ID: Inst.2.0.22.0430.1656 Filename: 0.2.0.22.0430.1656.md Title: Instrumentalism in Ahrens
MISMATCH! H1 ID: Rpt.3.2c5.0.22.0702 Filename: 0.3.2c5.0.22.0702.md Title: Progress Report 4
H1 header missing or invalid format: 0.3.2c7a1.0.22.1016.md
H1 header missing or invalid format: 0.3.2c7a2.0.22.1016.md
H1 header missing or invalid format: 0.3.2c7a3.0.22.1103.md
MISMATCH! H1 ID: 0000.0000.0ABC Filename: 0000.0000.00.1.md Title: A-B-C
MISMATCH! H1 ID: 0000.0000.0DEF Filename: 0000.0000.00.2.md Title: D-E-F
MISMATCH! H1 ID: 0000.0000.0GHI Filename: 0000.0000.00.3.md Title: G-H-I
MISMATCH! H1 ID: 0000.0000.0JKL Filename: 0000.0000.00.4.md Title: J-K-L
MISMATCH! H1 ID: 0000.0000.0MNO Filename: 0000.0000.00.5.md Title: M-N-O
MISMATCH! H1 ID: 0000.0000.0PQR Filename: 0000.0000.00.6.md Title: P-Q-R
MISMATCH! H1 ID: 0000.0000.0STU Filename: 0000.0000.00.7.md Title: S-T-U
MISMATCH! H1 ID: 0000.0000.0VWX Filename: 0000.0000.00.8.md Title: V-W-X
MISMATCH! H1 ID: 0000.0000.00YZ Filename: 0000.0000.00.9.md Title: Y-Z
MISMATCH! H1 ID: Will.1a.0.21.0517 Filename: 1.1a.0.21.0517.md Title: Will Weston Drawing Process
H1 header missing or invalid format: 2.1e1a1.0.22.0212.md
H1 header missing or invalid format: 2.1e2a1.0.22.0525.md
H1 header missing or invalid format: 2.1e2a1.0.22.0612.md
H1 header missing or invalid format: 2.1e2a1a.0.22.0924.md
H1 header missing or invalid format: 2.1e2a1b.0.22.0925.md
H1 header missing or invalid format: 2.1e2a1c.0.22.0926.md
H1 header missing or invalid format: 2.1e2a2.0.22.0913.md
H1 header missing or invalid format: 2.2a3b1.023.0123.md
H1 header missing or invalid format: 2.2a3d.0.23.01232.md
H1 header missing or invalid format: 2.2a3d1.0.23.0123.md
H1 header missing or invalid format: 2.2h2c1.0.22.1009.md
H1 header missing or invalid format: 20210726235001.md
H1 header missing or invalid format: 20211230141430.md
H1 header missing or invalid format: 2022.1026.1514.md
H1 header missing or invalid format: 20220104200700.md
MISMATCH! H1 ID: 2.2i.0.23.0128 Filename: 2023.0128.1336.md Title: LinkedIn Headline (saved)
MISMATCH! H1 ID: 2.7a.0.23.0129 Filename: 2023.0129.0043.md Title:  Sudoko Solver
H1 header missing or invalid format: 4.1.0.21.1004.14.md
MISMATCH! H1 ID: 4.0.23.0125.1225 Filename: 4.2.0.23.0125.1225.md Title: Vocal Music Index
H1 header missing or invalid format: 7.1a2.0.23.01.02.md
MISMATCH! H1 ID: Phys.0.22.1008.1332 Filename: 8.0.22.1008.1332.md Title: How to become a successful physicist
MISMATCH! H1 ID: 8.2.0.23.0124.1200 Filename: 8.2.0.23.0124.md Title: Amateur Radio
MISMATCH! H1 ID: Lora.2a.0.22.1206 Filename: 8.2a.0.22.1206.md Title: LoRa
H1 header missing or invalid format: Combinatorial Matrix Homology.md
MISMATCH! H1 ID: IDft.1.0.22.0305.1829 Filename: I.1.0.22.0305.1829.md Title: ID Format
MISMATCH! H1 ID: Id.1a1.0.22.0827 Filename: I.1a1.0.22.0827.md Title: ID Regexp
H1 header missing or invalid format: MatrixBoundaryHomology220503.md
H1 header missing or invalid format: Untitled 1.md
H1 header missing or invalid format: Untitled.md
H1 header missing or invalid format: vtl-example.md
Total # of Zettels: 333
Well-formed Zettels: 311
Malformed Zettels: 22
Non-Zettel or invalid format: 25

Looks like I have my work cut out for me.

Typically, there is an organizing notion that binds the links of a structure note together, which gives you a category.

/#definition_of_sturcture_note

Yes, exactly.

I'd find it hard to use an alphabetical system like this. I'd never remember if I called the note I was looking for "Online and Built-In Proofreading Tools 202301241040" or "Built-in and Online Proofreading Tools 202301241040" or if I put it in the category of Writing tips? But I can't remember the last time I might have stumbled around looking for a note without using ★SEARCH★, so it wouldn't matter where it was categorized, I'd find it.

That's why I allow limited-length key words at the beginning and end of IDs.