Metadata: embedded or not?

aduncan

On another forum, someone questioned me about the practice of embedding tags and other note metadata in the text of the note vs relying on the management software that I use for my Zettelkasten. I didn't have a conclusive answer. The questioner's issue with the practice was that the software application maintains a robust record of all the note metadata that I would embed in the note in YAML. But the counterargument is that embedding metadata as YAML allows the note to be more portable should I choose to use a different software application to manage my notes. I'm on the fence. Am I missing some other important argument for the practice?

Will

"...the counterargument is that embedding metadata as YAML allows the note to be more portable should I choose to use a different software application to manage my notes." Wins!

What software is the questioner trying to defend?

YAML is used for more than just metadata.
1. note specific printing parameters
2. scripting tokenization
3. hiding stuff

Here is part of a note with my YAML.

I include the note's UUID, the cdate, and my tags when I use them. Tags don't work in the exported notes I share, so not including them makes sense.

One reason for YAML is that the information is centrally located, not scattered around the app or operating system. A second reason for YAML is that the tokens can be formatted in a human-readable format.

The best and most exciting reason for YAML is that it's can be used as tokenization for scripts, shell scripts, and Keyboard Maestro.

r1tger

@Will said:
YAML is used for more than just metadata.
1. note specific printing parameters

This is the big one for me. I've started including some pandoc specific fields in the metadata. This lets me collect notes into one big markdown file, clean up and convert to a pdf using pandoc.

zk_1000

You don't need to go all the way in one direction, I'm using a mix of both. I'm also using YAML because it is widely used. My header looks similar to @Will's. Even If you keep it simple like that it is not robust. Occasionally you'll create an invalid YAML header.

If you blow it up with tenths of keys on each note it is only getting worse.

The software developer is not going to be overly happy about this solution, because it is much less efficient compared to a dedicated record. Plus, the overhead of supporting all variants of ID, UID, UUID, etc.

Additionally, he needs to treat the data record as something very sensible sinde it belongs the user. The formatting must be preserved, key values, order, etc. Adding or removing records is also something that not every user likes.

I suggest for a software to

• do not trust a YAML header
• allow the user to use it as he please
• limit your support

My personal opinion from a users perspective is to treat my YAML header as read-only.

aduncan

@Will said:

What software is the questioner trying to defend?

In this case, it's DEVONthink. I do understand the perspective of "don't repeat yourself." His argument is that if the software is keeping track of titles, dates, and tags (that are in turn exposed in the macOS file system) then why repeat them in the preamble of the note? Furthermore (one could argue) if the application has robust scripting capabilities then porting notes to a new environment should not be terribly difficult, if it comes to that.

On the other hand, there are other elements of metadata, or as @Will notes, print directives, etc. that have no home in the application and if they are to be maintained, it can only be in the context of the note.

I have quite a large number of notes, so I have to decide whether it's worth the effort to add these headers to previous notes or simply do it on an opportunistic basis. Thank you @zk_1000 @Will @r1tger for allowing me to think out loud.

@Will what is the cdate - I see it come up in your posts, but I'm drawing a blank in my search... (I mean what is the origin of the term - that it's a human readable date, I understand.)

MartinBB

@aduncan cdate is just a handy abbreviation of creation date.

aduncan

@MartinBB said:
cdate is just a handy abbreviation of creation date.

Ah, OK. Thank you.

ctietze

Dedicated management apps like DEVONthink can do a better job preserving metadata in their internal database. In a regular file system, creation date will be gone once you copy your file via USB thumb drive to a Windows computer and back, to give just one example.

From a technical perspective, the question then changes to: if you want to leave DEVONthink in the future, can you export your stuff with the metadata from the DT database intact, or will you lose that info?

A rule of thumb: put into filename and file content what you want to have around 'forever' (i.e. as long as you don't lose the file).

If you don't have a file, yet, and use another storage system (or a DB around the files like DEVONthink), then ensuring this could be deferred to the export step. But you'll need to have that figured out early on, and not postpone knowing how to keep your data intact, otherwise you'll be just losing what you originally wanted to keep.