Rethinking my use of UIDs
I’ve tried to follow the discussions around Folgezettel and higher-order notes, but the most interesting thing for me was that it seemed to surface the various different approaches to unique identifiers.
When I first began to think of my notes as a Zettelkasten rather than a bunch of random notes, I read a few forum posts and quickly adopted the idea of each note having a title that consisted of a UID and a brief word or phrase. I also adopted a structure for the top of the note, based again on discussion in the forums, so:
# 201811241024 zettelkasten Tags: #zettelkasten #tag2
I made a simple little snippet to do that.
The tags are not always as redundant as this example, but what this recent discussion made me realise is that the UID is doing me very little good there in the Title of the note. Because I have it in a heading, it cannot act as a link, even if I surround it with the requisite [[]]. An obvious solution would then be to remove the heading and make it a link, and that certainly works. But I think I would prefer to leave the title as a heading, because it improves legibility, and create the link on the next line.
As I understand it, the main benefit of having the UID as a link is that it provides a very simple way of seeing all the other notes that link to this note, a facility that I have not been making nearly enough use of as I build connections between notes. As I used a macro to create the front matter on all new notes, they all have an identical structure, and so I ought to be able to work out a grep search and replace to make the changes I want to see. And for the time being, I don’t actually need to change the filename of each note, though that would be easy too.
I think I am going to make a copy of my Zettel and give it a try for a week or so.
Or is this a really bad idea?
Howdy, Stranger!
Comments
I did it exactly as you describe without problems. I used macro to extend filename to first line as linked UID + title on subsequent line. Later I added tags. My Zettels had following structure:
filename:
2018091546 TitleAt the moment, I am leaving UIDs altogether - https://forum.zettelkasten.de/discussion/1039/why-are-uid-necessary-used#latest
I may misunderstand but ...
I followed that very same earlier discussion about note headers and creating them with Keyboard Maestro or Alferd. I still use a macro to build a new zettel moving the UID to the YAML footer like so.
You can see that I've moved the UID to the bottom along with other meta information. What is nice about this arrangement is that visually the note has a less cluttered heading which I can freely edit without losing any reference as the original title is also listed in the YAML block. A note can have several titles, and a couple of my notes do have multiple titles. If for some reason I want to use a different UID or numbering sequence I can just add it to the YAML without losing and old references.
Your "Front Matter" is what I'd call a YAML block and I just chose to put my and the end.
I'm not sure why, if the UID is in the note 'front matter' or 'YAML block', we place the UID in the file name. Maybe it is for software independence? If we wanted to migrate to another platform is might become important. Also, I'm finding when building Keyboard Maestro macros having a UID in the file name comes in real handy.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
I see @Will requesting an option to toggle display of UID in note filenames on and off. That would certainly improve sidebar display for those who want it. I don't honestly mind having the UID in the name of the Zettel. I just do not want it in the title of the note.
@Jeremy, we can use UID's and link to them in The Archive without having to include them in the titles. Just include them somewhere in the note text. Because double bracket links and tags are basically clickable Omnibar searches.
I wonder if we mostly assume if we use a UID or some other type of unique identifier that it has to be in the title. I did in the beginning.
Having them in the title gives a reliable way of sorting by creation date if you are using UID's at the beginning of the note list title/filename. If The Archive had three Sort By options, Title, Modification, Date and File System creation date it might help to not need to include UID's at the beginning of a Titles. Maybe there are other benefits for having them in the title, although I can preserve the creation date value by including it as data within the note.
I too don't mind the UID in the title but with the limited sort options, I have to in order use sort by creation date ordering.
I do hope @Will's request gets some traction.
I have to say for me the UID in the sidebar is a major turn off. My brain can't see past the UID, or at least doing so uses some up some brain processing that ultimately just represents a distraction. It's the #1 reason I keep turning away from the Archive, despite its clear strengths.
I love the idea of front matter moving to end matter. Putting it first never made sense to me since it is a bit distracting. Sadly, I think most of the parsers only accept it literally being at the front.
@kmac yes, "most of the parsers only accept it literally being at the front." I frame my YAML block within special characters so I can quickly and safely. This makes it possible to move the YAML block for those few notes I want to run through parsers with a simple bash script. Easy peasy.
I found the info in the YAML block to be a distraction when reviewing. Yet the info is valuable for searching and linking. Moving it to the bottom of the note just made sense. I've not found an argument otherwise - YET.
This is an eye thing. My aesthetic. Funny cause I like seeing the UID's in the note list in The Archive.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
@JKF wrote:
I'm with you on this. It is very distracting to me as well. The fact that @Will suggested adding the ability to hide the UIDs in the note title display suggests that it might be a widespread issue.
I think that recently in another thread @Sascha said that DTIDs are not strictly needed for linking in terms of the technology, but can be a sort of backup way of finding a note (can't find the comment now, unfortunately), and above @MikeBraddock suggests the option of including them in note text.
If it's right that UIDs are not needed technologically for linking notes (as long as notes have unique file names), then we might think of the UID issue in a different way - not LIDs vs DTIDs, but LIDs vs no IDs (other than file name). This is akin to asking, should you have a system with a content-based note structure in the note titles, which would be more or less browesable based on your system, actual use, and number of notes? The practice of having such a structure visible in note titles, which is being debated on this forum as ways to implement folgezettel, is not exclusive with having structure notes.
This angle brings another thought to my mind: coming up with a file name that is both unique (for linking) and brief (for easy browsing) produces a content-based challenge to the user that may serve a feedback purpose similar to that of Luhmann numbering. It forces you to think, "what is the very heart or core idea of this note, and how does it differ from other notes?". Summarizing notes into titles is in some ways similar to the process of forcing yourself to summarize reading content into notes, with potentially similar advantages.
The more I think about it the less I'm inclined to see the value in turning the UID's off in the note list. I find them invaluable in placing the note in a temporal context before opening it. Now that I have more time as a Zettelnaut when I search on a topic, for example, if I search for 'writing' I get 180 hits with dates spread out since Nov 2018. And this will grow as I add to my Zettelkasten. Being able to see the dates I can quickly sort the wheat from the chaff as my ideas about skillful writing have significantly evolved.
I view this like I view "Show File Extensions" options. Ok, for casual users to be ignorant of but something a power user needs to know. I have a few Keyboard Maestro macros that depend on the UID's so I've become comfortable seeing them and know their value.
I'm sure once you have a thousand and more notes, you'll see the UID differently. I certainly do.
I don't think this is a widespread issue. It hasn't been till now to my knowledge.
Will Simpson
My zettelkasten is for my ideas, not the ideas of others. I don’t want to waste my time tinkering with my ZK; I’d rather dive into the work itself. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
kestrelcreek.com
I doubt it. I have one of 'those' brains that struggles with filtering. I can't filter noise, either, or people moving in the office. I highly doubt I'll ever get to a thousand notes because, once again, I've had to turn away from TA towards something cleaner feeling because every time I open it, I feel physically overwhelmed.
As an aside, I'm rather bothered by the way my brain gets distracted and was in discussions with my GP for possible ADD tests before the current crisis - I often find that what other people take to or adopt easily is very difficult for me for odd reasons (like this one). I find note taking / 'gasping' information extraordinarily difficult / overwhelming, compared to peers of similar overall intelligence, and always have.
That said, I do wonder whether I'm the wrong audience for TA. There's so much discussion on here of scripts etc and it's clear that so many are from a programming background. Maybe it suits that type of brain vest. I'm a (female) middle aged humanities student trying to fit all this around a day job (well, not at the mo, thank you global crisis 🙄🙄) and feeling overwhelmed with it all.
I switched to Notion yesterday as a trial (I get a free account as a part time student) and made more progress in one day in terms of processing notes than I have for all the months I've been looking at TA. I prefer Roam most of all but I find the 'it's free now but won't be and it won't necessarily be cheap' a huge distraction (again, for my brain).
That said, there's something about TA and these forums I'm super drawn to. I find as I'm trying more COTS tools I'm thinking 'Oh - NOW I understand how I can implement that concept in TA. But it doesn't come easily to me and I've certainly spent more time working with the tool / method than making progress on studies.
A unique ID for a Zettel note is very helpful as it provides a stable, unique (and rather short) identifier that doesn't break. If instead, you're using a textual ID (such as the Zettel note's file name), you have to make sure that it stays unique and that links pointing to it won't break should you want to rename it. While some software tools may help you to globally rename a textual ID and all links pointing to it, external links are usually harder to rename. But if you don't plan to use your links in external applications, then this may be less of a problem. OTOH, being able to link into your Zettelkasten from the outside is a huge opportunity that I wouldn't want to miss.
While I don't think a unique ID is strictly necessary in the title of the Zettel note, it may help in environments where you cannot sort by creation date using any other means. Personally, I prefer titles to stay focused on the note content (i.e., w/o the ID). But I think it's crucial to at least include a unique ID:
IMO it's also of tremendous help (esp. long-term) if the Zettel note’s ID can be found/matched by a machine/regex pattern. Ideally, all important knowledge elements should be human readable & machine parseable.
Yes, I get your points.
Are you saying 1 OR 2, or 1 AND 2?
@cobblepot wrote:
I meant 1 AND 2.
OR will work if the noise of the UID In the title creates friction for you as in @JKF’s case.
There is more flexibility in The Archive and the ZK Method then we may imagine.
If you have the UID in the filename, then why would you also want it in the note content/body?
Redundancy to make the system (1) resilient and (2) receptive to more techniques and workflows.
Example: If you have the ID in the title of the note content you can search for "# ID" and just have the exact file returned. If you search for the ID only you'll have the file and its backlinks.
Historically, the ID in the filename was the addition because some applications did just allow for title search. This is quite helpful when I access my Zettelkasten via the Dropbox website.
I am a Zettler
I understand this, but why should the note have its own ID in its body text/content in addition to its filename? It would still be found in your example, yes?
Exactly what @Sascha says. You can leave the ID out in one location but this makes your system less robust, and you potentially loose the chance to work with your Zettel notes in various or automated ways.
IMO, Zettel notes should be as self-contained as possible. I.e., the Zettel note's body text (or its metadata) should contain all relevant information (like its tags, creation date, linking & citation info, related file info, and of course the Zettel ID). This way, you can do lots of things with your Zettel notes but don't loose anything. All relevant info always travels with the Zettel. E.g., you could transfer your notes somewhere where there's no file path info, like a database, a single text file, or an app that stores files internally using its own naming scheme. Your notes would still remain intact and can be retrieved again w/o loss.
A concrete example: The app that I'm using for my notes doesn't offer any batch search & replace yet, and has no regular expression support. But I can select multiple Zettel notes and drag them all into a new editor window of my text editor. The notes get concatenated with
\r---\r, and all contain their note IDs (in a way that's machine readable w/o error). I can then go ahead and search & replace things as I like while still being able to see all notes at the same time in a single file, and jump between notes easily. I can even add new notes there. When I'm done, I simply drag them back into my Zettel note app. The app will split the notes again, update existing notes as necessary, and add new ones. This is only possible due to the note IDs being contained in the note body itself. This is, of course, just an example which could also be solved otherwise. But it shows the flexibility that self-contained notes will give you.@msteffens , thanks for the concrete example. I haven't seen anything like that in present software implementations yet, but I now understand the idea.
If it wouldn't be in the body of the text you couldn't use the additional syntax to perform a macro like THIS
I am a Zettler
In the old days people used a command called sharfile that create a single file out of many. This is how people shared files on Usenet or mailed them to others. You run this one file through /bin/sh to get those files back (mainly source files. For binary files you have to first encode them to ascii only characters).
In other words, you don't need IDs in the file (if the files are named with their IDs) at least for your purpose.
As an on-again/off-again database developer, I see this thread in terms of a surrogate key vs a natural key.
In the database world, a surrogate key would be the date id/UID. The natural key would be the filename. The problem with natural keys is they aren't very easy to use when creating relationships, but surrogate keys have no meaning (except in this case they are timestamps) and have no reason to ever change.
The UX wannabe in me says the files should be the surrogate key and the DISPLAY {sh,c}ould be the title of the document. The beauty of text files is they can adapt to your tool if the tool uses text files (emacs, nvAlt, visual studio code, etc). If for some reason your tool didn't support the sidebar title display, you could rename your files w/ a trivial script to have both.
Even then, the title might be too verbose to convey meaning when truncated (I never use a full sized window), then you need a short title. And tags. And on it goes.
After all this, my recommendation is hide the sidebar and learn to love the Omni Bar. I'm going to try it at least!
My subconscious tells me this quote is relevant. Your understanding might be significantly better than mine!
I can only agree with JKF. Whilst I have some programming skills I don't want to be using them in this context. I just want to be writing and storing notes. A Zettelkasten is a note archive that, in a computer-based world, should be neither difficult to understand nor difficult to implement.
Moving to a plain-text system to avoid an impenetrable, locked-in system only to make extracting the information almost as impenetrable is defeating the point completely. We're in danger of achieving that.
We must also remember that JKF has no interest in the arcane workings of any kind of database, she just wants to be able to store, manipulate and extract information. I'm sure she realises, and accepts, there is a learning curve but, from reading this discussion, it's easy to see too many trees in the way, not all of them natural growths.
Plain text should mean some level of simplicity (that is its beauty) as should The Archive and nvAlt. I think she, and I, have a right to expect simplicity in interrogating the information and not get bogged down in seeming minutiae.
Having said all that, reading this has cleared up the problem of UIDs in my mind at least. Put the UID in the filename and in the title. It may not look pretty but we really shouldn't care. These are just notes and not finished products.
I've been studying this thread because I have been trying to decide how to handle similar issues after I migrated my ZK from The Archive to Drafts.
First regarding lingo: What might be called a note or file or zettel in The Archive is called a "draft" in Drafts. For the purposes of this posting, the terms can be used interchangeably without loss of meaning. Second, the Drafts UI is similar to that of The Archive. There is a list on the left and an edit panel in the center. Unlike The Archive there is a third panel on the right that is used to display "Actions."
I had two problems: (a) Dealing with a set of notes that comprised my first attempt at a ZK - that I imported into Drafts, and (b) Organizing drafts for use in future ZKs.
My major issue - as it seems to be with others in this thread - is how to deal with the UID. There appear to be three concerns: (1) Uniqueness, (2) Sorting by creation date, and (3) Avoiding problems with corrupted links in the event that a note's title is changed.
With respect to a future ZK developed in Drafts:
For all these reasons, as I go forward, I'll not use a UID. I think that UIDs are mainly a crutch that TA uses because it doesn't have a Date Created sorting capability. The uniqueness and "change the title" issues seem like a "kill a fly with a 16 inch gun" approach. IMO, these issues simply don't arise often enough to warrant the use of a UID. Maybe the overhead costs would be justified if you envision developing a 15,000 note ZK over two decades. Personally, I am more likely to develop a half-dozen ZK's with a few hundred notes in each.
Importing my first ZK from TA posed a different problem. The import was fast and easy, but all the newly imported drafts were given the same Date Created - the date they were imported. One option I considered was to leave all the leading UIDs in the titles and links.
But, I like to move through my ZK on my iPhone and horizontal space is at a premium. The UIDs really got in the way. On the other hand, I'd like to have some idea of when I created each note.
Luckily, when working in TA I routinely followed the title by a "plain language" date. This format:
202005231235 This is a title
May 23, 2020
Rest of the note
The second line is the date created - not in metadata where I wish it resided - but it is there. Unlike TA, Drafts allows more than one piece of data to be shown in the left-panel list. I chose Title, the first few lines of the body, and tags. With spaces, the "body" was restricted to the date. See the attached screenshot from my iPhone.
I toyed with the idea of reducing the 12 digit UID to 6 digits yymmdd - e.g., 200523. In the end I wanted more horizontal space for the title and was happy with the second line giving me the creation date.
For future ZK's I'll replace the "body" - with the actual date created as it exists in metadata. I'll be able to sort by that date without use of a UID.
I keep coming back to your comment @cowgap. I think you've articulated a tension I've always noticed in this community, lovely as it is--namely that between what, for want of better terms, we can call the database junkies and the word/content junkies. The former group spend a lot of time back-and-forthing about the technical challenges of automating workflows and working with configurable text editors to make those workflows more efficient. The latter--well, they tend to lurk in the background by and large, because the forum seems to attract more of the former. Perhaps the database junkies hail predominantly from science and tech backgrounds, and the latter less so; again, I'm generalizing.
As with all generalizations of course, this paints a broad-brushstroke portrait of what is a pretty diverse community united by a shared love of the ZK method and its possibilities, and misses the complexities of what are multiple and overlapping populations here on the forum. I used the word 'tension' in my first sentence, but I don't what this to be misconstrued as a negative characterization, and I'm certainly not trying to start a food fight. It's a productive tension, after all, and there are those who effortlessly straddle my simplistic typology (@Will springs to mind) by having feet planted firmly in both camps. I know I could do with learning a lot more about Keyboard Maestro and markdown and so on, as my workflows remain stubbornly old school.
I'd be interested in hearing what the rest of you think.
Phil
Started ZK 4.2018. "The path is at your feet, see? Now carry on."
@Phil I agree with your line of thinking. It is basically this tension that gives Christian and me a productive dynamic. Many times, I say something like "Oh no. Not this again. Get you nerdy ass out and come back if you have something appropriate for normal people."
What are the old school elements that you are suspicious of possible modernisation?
I am a Zettler
It was my musing that started us off down this route so I thought it might be useful to say a little more. I am interested, in a secondary way, in the extended possibilities of TA, but I'm not a coder, and I suspect the mindset and skillset of coding really helps, both in terms of understanding how best to adopt a zettelkasten approach and (obviously) how to implement / adapt scripts. And, for that matter, with feeling comfortable with the UI of TA.
There are three concurrent things I'm trying to learn / develop / assess: 1) the general art of note taking (which I'm finding challenging), 2) the particular art of note taking via zettelkasten (and / or deciding whether it is for me), and 3) using TA and / or other tools to support 1) and / or 2). Dealing with these three things at the same time was proving too much for me and I had to drop one. So, for now, I've dropped TA.
I'm currently using Notion and am making enough process in 1) to allow me to make some progress in 2). Why has this been the case? Simply because Notion has some of the functionality built in that TA requires third party scripts for. I burned way too much time trying to get TA working for me that I should have been using on reading and processing notes.
An obvious example is the need to format links as '[[ID#]] Title' rather than '[[ID# Title]]') and this requiring a manual adjustment or a script. And the different use cases for creating a link - new zettel as you type, linking to an existing zettel, turning a selection into a new zettel - each of which need a script. It burns time and mental energy to find, implement and test what you need and if, like me, you don't have that natural mindset / skillset, it burns perhaps too much time and energy that should have been used actually processing notes. (The reason it's important, in terms of this, that I'm also trying to develop skills both in note taking and zettelkasten is that I don't always know what I need, I just know something isn't quite working for me but not whether that's process side or functionality side, so I can easily slide into a loop of distraction).
And, yes, as a non tech user, I'd hugely prefer to see 'This is considered in more detail in [[name of topic]]' than 'This is considered in more detail in [[2020050823]] name of topic'. One is nearly smooth 'natural language', the other disrupts thinking to look beyond the ID# and find the topic name to understand the connection. As Phil and Cowgap point towards, I do understand the database issues behind this - but as an end user I care less for those than I do for my needs of a smooth workflow.
Now, equally as both @Phil and @cowgap, and I myself, have pointed towards, this is no criticism of TA or this forum at all. I'm impressed by - and grateful for - the thought and skill that continues to pour into both, and I understand that this forum is a kind of technical debate about both Zettelkasten and how to develop the tool Although TA 'is' commercial software, in respect of the fact it is purchased, it's almost more of an open source (that's not the right term, but I can't think of the right one) offering, and the forum reflects that. I'm really just following up on Phil's and Cowgap's thoughts that there are different kinds of user, and occasionally I think this forum forgets that with responses like 'Ah, you quickly learn to look past that' or 'you are not thinking about this in the right way' or 'that's not how software / databases work'.
@JKF Totally get where you're coming from!
Just so you know more about The Archive's handling of links, and what it can and cannot do: other kinds of wiki links work fine as well. You can use
[[ID# Title]], and you can just use[[Title]]. It's just that Sascha and I aren't advocating this because when you rename a note, you have to rename all your linksFor example: Back in the old days of wikis, people used CamelCaseWords to link to wiki pages (because whitespace is ... ugh). Say you used this convention and create
AristotleMetaphysicsOverview.txt, then a link tosee [[AristotleMetaphysicsOverview]] for morewill bring the note up just fine. It's a unique enough letter combination.But a link like
see [[the overview of Aristotle's metaphysics]]is ambiguous -- it executes a search, and if you don't have a note of exactly that name, you have to fish for the file calledAristotleMetaphysicsOverview.txtit in the search results yourself.Author at Zettelkasten.de • https://christiantietze.de/
Thanks @ctietze, I do understand that, and I also understand why you and Sascha give the advice that you do. And for what it's worth, having now used Notion for a while I'm beginning to see a lot more of Sascha's points about the ID# being VERY useful for searches in tools (like Notion) that don't have automatic bidirectional linking. Plus Notion is SLOW compared to TA 😂 Unless my current experiments with linked Notion databases totally commits me to that direction, I suspect after my exams I'll come back to TA and start from scratch with the scripts, this time making notes on what I've implemented and the triggers, so that it uses less brain power.
I guess I'm an annoying (to myself) kind of middle user. I'm essentially, in @Phil 's description, here to become a word / content junkie, and as a mature humanities student, my efforts need to be focused on content and concepts. In my other (working) life, I'm a (general) Business Analyst, so I take the advice from you and Sascha very seriously as I know, in the long term, it's what's needed. If I were a true non-techy, I'd just ignore you 😂😂