Zettels and Org-mode

mediapathic

This is mostly a question for @sfast, but anyone else who has opinions is welcome to chime in.

I've recently been bitten by the org-mode bug, largely because a reasonable IOS client, beorg, recently came out. So, I'm playing around with it (via Spacemacs, which I can recommend if you're a vim person wanting to play with org-mode).

I guess what I'm looking for here is tips and tricks for how you integrate your org files with an extant (markdown) Zettelkasten. I don't want to convert my Zettel to org-mode; I am very pleased with the way The Archive and Sublime Text (thanks again @rene) handle that. Mostly I think what I'm wondering is best practices for, say, having a TODO item that links to a file in my Zettel (as well, to specific points/ markdown headers in larger text documents outside the Zettel, for example "TODO rewrite this scene").

In my current setup, I've created a /org directory inside my archive. Is this reasonable for future scalability? Is there any advantage/disadvantage to just putting my .org files in my Zettel directory instead?

As well, general hints as to how to use emacs as a writer/ knowledge worker, as opposed to a lisp programmer, would be appreciated. Highly usful key combinations? Interesting minor-modes? Particularly nice markdown-friendly themes? Anything at all!

Thanks in advance!

mondayrobot

Hey,

I don't know if you know about it, there's a relatively new emacs package that has a lot of potential for knowledge work: org-brain. It's a cross between a concept mapping tool and a wiki, but it allows you to have more than one type of relationship between notes (zettels). It also allows you to present these relations visually, while remaining strictly plain text.

Combine this with deft (a package inspired by nvalt) and you could have a nice system for knowledge work in emacs.

https://github.com/Kungsgeten/org-brain
https://jblevins.org/projects/deft/

mediapathic

@mondayrobot oho! This is very much worth looking into. Thank you!

ctietze

I do not use Org-Mode, so I'm curious what kind of links you can create from org to Zettel note archive.

• How do you link to files?
• Is it beneficial to put the org/ folder inside your archive to use relative paths?
• How many org files do you have, and when do you create new ones? Per project?
• Are org files for task management only, or do you write your text outlines in that format, too?

General remarks on project management

On my Mac, when I create a new window in Finder, it takes me to ~/Pending -- that's my folder of work-in-progress projects and files. When I finish a project, like the book cover design of Sascha's latest book, I move the folder out of ~/Pending and into ~/Documents/Project Archive/. It syncs to Dropbox, by the way.

Writing projects start as project folders in ~/Pending. I put scans/pictures of hand drawn illustrations in there, save the outline to that folder, put the final Markdown files there, and assemble the PDF from LaTeX from inside the project folder, too. (All in sub-directories, of course.)

I don't benefit from the path to my projects or the notes in any way. (I do not really want to strongly couple my writing to the notes on the level of file system information). -- Links to my Zettel notes are created using the ID convention. The outline contains references to Zettel notes that I manually resolve (or use the outline compilation script for).

From my point of view, it doesn't (and shouldn't!) matter where you put your org files in terms of productivity. It should not ruin your writing if you move the org files away, or else your file coupling was bad. If you want to see the plain text org files in your archive management app of choice, putting them in the same folder makes sense, though, but for different reasons.

All that being said, I do have project files in my archive. Outlines in my project folders never start in a vacuum, they are just the next step of extracting structure notes from the archive and making a coherent outline from them. Even though my projects are mostly written as code (which permanently live in ~/Coding/, by the way) instead of prose, I still end up with textual notes about them which I need to keep in order to help me think, so they go into the Zettelkasten, not into the work-in-progress folders.

Sascha

I strictly seperate my task management and my knowledge work. These are two different modes of thinking and actions.

Flow from Archive to Task Management

In my org-files, I have tasks. For example:

1. I write a note on the difficulty of leaning to work with a schedule.
2. I go to my org-file on writing. And go through the various areas (knowledge work, health and fitness, philosophy etc.)
3. I create a new task, copy the ID and title of the note as reference and create a small outline for the text.

My org-files

1. Writing. All tasks related to writing.
2. Trainer. All tasks related to my work with clients.
3. Everyday life. All tasks related to my personal lifestyle
4. A couple of current projects. E.g. The Archive, "Reflexion und Analyse" (my current book) etc.

Integration into my daily task management

No big method applied. Org is just for handling tasks and information related to tasks that are not part of knowledge work. I have seperate files for my clients that are not part of kowledge work or similar stuff.

I center my work around projects and daily routines. In the morning, I mostly write. In the afternoon, I do my client related tasks and other stuff that doesn't need unbroken hours.

I don't have a smartphone and never will. I have an iPod but never saw any benefit of being able to take notes on that thing. I take notes with a good ol' paper notebook. To me, smartphones and tablets just add to the mess and not to the organisational clarity. Therefore, I don't bother with synchronisation or anything like that.

zwhaley

@ctietze can you discuss a bit further your reasoning for storing textual notes about coding into you Zettelkasten? Are you doing this only for the textual coding notes you believe will be helpful to you in the future? I'm trying to make the distinction between the notes that help you code but you just keep in the relevant project folder vs. the ones that help code but you put in your ZK. I'm guessing it comes down to future usefullness? If information that supports a project may be useful for another project in the future should it go into the ZK?

ctietze

@zwhaley I blog and write books about programming, that's why I picked up the habit of collecting some how-to solutions. Not every trivial function, but stuff that puzzled me and for which I needed to consult the developer documentation a lot.

These notes are similar to other Zettel notes, like interpretations of books/articles -- only in the case of "code" the book is an API documentation and my explanation ends with a sample code block to illustrate instead of a list of arguments, say.

As always, future usefulness is one factor. Clarity of thought another. Sometimes I end up finding out where I missed a detail when I explain the connection between parts of foreign code to myself in prose.

zwhaley

@ctietze ahhh I see. Does your Zettel then seem to have a target of items that could assist in writing? If you were a 9-5 worker, would you include items and connect for items that assisted with your day job? The only concern here is if you switched jobs then wouldn't most of those notes and connections become irrelevant?

Work -->Personal-->Extracurricular , it all seems interrelated. Except work in the "desk-job" sense seems to be the one thing that doesn't necessarily live forever, and does it belong in the same Zettelkasten?

Let me point out not all "desk job" knowledge is org-material, i.e. project or task specific. Some of this knowledge is information that enhances your ability to think and make connections within that role and may span multiple projects. But definitely separate knowledge from anything that could support a book or article in the future....

Not sure if this has been addressed yet but I do feel the ZK has been targeted to true "knowledge work" and I'm wondering if the term "knowledge" is applicable on the broader sense of the term, or if something like this really would call for two ZK.

ctietze

@zwhaley Tough question! I think the answer should be "Yes": my Zettel are text fragments that can be composed into a whole any time. So everything I put in, I put in to potentially pull it out again and write about it. (I wouldn't use it to learn vocabulary, for example, even though I have to "know" the words.)

Recently, @Sascha and I were discussing how to phrase what our app The Archive does, and is. "Knowledge work" apparently is a very broad subject, including filing reference material and all of Getting Things Done. The Zettelkasten and our app is targeted more towards writers, authors, creative workers. Without the purpose of eventual creation in mind, including passing a test at University with good grades, I have a hard time fathoming what you'd use a Zettelkasten for. Maybe Sascha has something better to say about non-creative, non-writing use cases

zwhaley

@ctietze I must most respectfully disagree! I am an entrepreneur and find HUGE potential in recording work material for the ability to connect people to concepts to ideas to industry to research and beyond. Without any intention of writing about it. (Unless you call that creative work.) But the Zettelkasten is capable of much more, in my opinion, and exploring that concept would allow one to open this amazing technique up to a huge other market. The MAJORITY of the market in fact could stand to benefit. The average non-writer can benefit in their life from just being able to connect concepts in this manner.

"Creativity is just connecting things." —Steve Jobs

According to Jobs, then, by using a ZK (properly) you become a creative by default.

Sascha

It think the problem lies in the layers of intepretation that you have to remove before you can see the actual thing.

The Archive for example is designed to manipulate text and text files in a very agile way. You cannot really say that it is for this or for that.

You can compare it to the human body. It is designed to move efficiently. But should you play soccer or practice martial arts? This is not a question that is possible to answer.

The Archive operates on a similar way. Therefore, you can tweak it to be capable to do almost anything text related. I read it in a post on Emacs I think:

As a writer you should be really good at manipulating text.

This is one of the deeper layers of the archive. It is about manipulating text. This is at least very next to the core. Because of this I accepted nvALT over all the other apps because it was the most honest approach without all the ideology that other apps embbedd into their structure.

zwhaley

@Sascha said:
Org is just for handling tasks and information related to tasks that are not part of knowledge work. I have separate files for my clients that are not part of knowledge work or similar stuff.

I'm still struggling with what you consider knowledge work and what you consider "information related to tasks that are not knowledge work." Here you say files for your clients are not knowledge work. I'm confused because:

1. Your work to assist clients falls under the category of knowledge work activity, i.e. you are dubbed a "knowledge worker" in doing this kind of work
2. Files for clients then are a). notes written to assist in the completion of knowledge work and b). are written to be referenced later

If either of my assumptions are wrong please correct me; but if the assumptions are correct then why do they not go into the Archive?

This also seems to follow the quote from @ctietze that Archive notes are typically being written with the "purpose of eventual creation in mind"...in this case the creation is the deliverable of advice you will provide to your client, or whatever else you have promised.

Sascha

I think a use case is the best to clarify:

I have a client who suffers from gut dysbiosis. That means that he has some gut bacteria which don't belong there and some are missing.

• When I am working on his case everything first is in my org.
• If I stumble over something useful I file it in my archive.
• Sometimes, I do research for my clients. This is in my archive. In my orgs are the specifics for their case.

For me, it is something about the mode of working. But I think your confusion is justified. The seperation is somewhat arbitrary. I could do everything in the archive.

I even could do it all in one big text file - to increase confusion. (I think I'll write a small blogpost about that topic in the middle future).

zwhaley

@Sascha thank you for using this example. It seems the research FOR your clients go in the archive, but the notes ON your clients do not, because (I'm assuming) you have deemed this information a). not useful to enhance your archive and/or b). not useful post project.

Do you have a use case example in which you ended up inserting a note specifically about a client into the archive and what made this one different?

Sascha

Case example: I discussed with a client how you organise yourself in advanced and on the importance of keeping commitments. Some of the email got into my archive under the umbrella of "self-organisation" and "self-determination".

mediapathic

Quick update:

Hi, I've been working on implementing a Zettel in org-mode. I will be doing a full write up on this when I get it working a bit more cleanly (it works fine, but some of my processes are kind of awkward, which I'm sure I can fix by learning a bit more elisp). But, short form, using deft and custom IDs I've built a system that feels really quite robust.

I'll let you all know when I have something that's good enough for public consumption!

ctietze

Yes please, that would be great!

jimsen

Hi (and Hello to the forum!),

I already got a Zettelkasten working with org-mode. I also use deft, custom IDs (date+time) and a nice org-capture template to start typing immediately. Unfortunately, my config is not yet ready to be published to the whole world, but if you have any specific configuration questions please don't hesitate to ask.

I use my org-mode Zettelkasten for scientific work. Having all the other modes at your finger tips such as org-ref or even magit is pretty cool. Not to forget how easy it is to write some nice looking papers in org-mode.

ctietze

I don't yet know how nice that'd be, so if you want to share this with us someday, that'd be great @jimsen!

trmd

Hi, I've been trying to use deft in Emacs as an nvALT alternative to accessing my zettelkasten, but it is very slow on my several thousand number of notes. Have anyone run into this problem and solved it?

magnus

If you use spacemacs, you have a bunch of shortcuts handy for moving through your Zettelkasten already. I tried to recreate some of the sublime zettelkasten from @rene there.

• SPC p F will search in your project folder for files with the word under the cursor and go directly there if its a unique hit. So for example, f2 SPC p F (forward to the next 2 as in timestamp in the year 2000, open link) directly opens a zettel.

• SPC s P searches the content of files in your project folder for the word under the cursor. Place it on a timestamp to find other Zettels linking to the same note, or on a unique word to search other notes talking about the same thing (works on pandoc citekeys for example).

• To find other notes linking to the same zettel as you are at, copy the filepath with SPC f y and then run a script that removes the full path except the uid. Mine looks like this:

xclip -selection c -o|ag -o "[0-9]{12}"|tr -d '\n'|xclip -selection c

Then you can search in your files with SPC s p and paste this timestamp.

These are just some examples. spacemacs with its combination of helm or ivy is so good for moving, switching, searching, cutting and pasting that you can get a pretty good workflow with just the standard tools.

EFLS

Hello all

I'm new here, but have been interested in Zettelkasten for some time. Recently, I've been tinkering a bit with deft and some elisp to create some Zettelkasten functions.

Check the results here, if you are interested: https://github.com/EFLS/zetteldeft.
I've documented everything thoroughly, as I wouldn't recommend to copy everything blindly, but rather select some functions to make your own.

I'm happy to receive any comments, but I can't guarantee I can respond to anything in depth, as my elisp skills are very limited.

Cheers
EFLS

magnus

@EFLS I think Zetteldeft is such a well thought through package. Really excellent work! I'm trying to get it to work under spacemacs but it throws errors works perfectly!

Got some help to set up a layer now. The utility functions to assemble a single file for export from linked files is brilliant!

~~
I tried to do the use-package instructions you have in my "(defun dotspacemacs/user-config ()" and then call it under additional-packages, but when I try to use it it says things like "command-execute: Wrong type argument: commandp, zd-deft-new-search".

Would you mind guiding me a bit here. Is there an easier way to do this? Have you even made a spacemacs layer of zetteldeft? Can I bombard you with questions somewhere? ~~

mediapathic

@magnus said:
If you use spacemacs, you have a bunch of shortcuts handy for moving through your Zettelkasten already. I tried to recreate some of the sublime zettelkasten from @rene there.

Can you give me a hint what spacemacs is using for the underlying search to do these things? Those sound like useful things to implement in my own vanilla (is it really vanilla if you have a massive init?) emacs. Is it like avy or helm-ag or some kind of projectile fun?

EFLS

@magnus said:
@EFLS I think Zetteldeft is such a well thought through package. Really excellent work! I'm trying to get it to work under spacemacs but it throws errors works perfectly!

Got some help to set up a layer now. The utility functions to assemble a single file for export from linked files is brilliant!

~~
I tried to do the use-package instructions you have in my "(defun dotspacemacs/user-config ()" and then call it under additional-packages, but when I try to use it it says things like "command-execute: Wrong type argument: commandp, zd-deft-new-search".

Would you mind guiding me a bit here. Is there an easier way to do this? Have you even made a spacemacs layer of zetteldeft? Can I bombard you with questions somewhere? ~~

Glad that you got it to work! zetteldeft and its functionalities were written as I required them for my personal workflow, so I'm happy that others find it useful.

I don't often visit these forums (well at least not on a regular basis), but if you run into problems feel free to submit an issue over on github.

Also, could you share (in an issue on github) how you set up the spacemacs layer? It would be nice to include some more info for spacemacs users in the readme somewhere.

magnus

@mediapathic said:
Can you give me a hint what spacemacs is using for the underlying search to do these things? Those sound like useful things to implement in my own vanilla (is it really vanilla if you have a massive init?) emacs. Is it like avy or helm-ag or some kind of projectile fun?

Yes, this is spacemacs with ivy (helm works as well) and some use of projectile.

I use:

• SPC p F

Find file with under cursor as input. Basically works as follow link. Since the ID is unique, it will just immediately to the linked file. If it's a non-unique string, you get a lift of files containing that string.

• SPC s P

search in zettelkasten with under cursor input, for example for other notes that reference a zettel or ref.

• SPC s p

Search contents of zettelkasten

• SPC f f

Incremental search on filenames in zettelkasten. SPC b b is same but with recent buffers.

• SPC f y

Copy file path. Then I have a script that removes everything except the ID (201902142149). I can use this with SPC s p to search for any files that contain links to the current file.

But I think the functions in Zetteldeft does these and more much better

@EFLS said:
Also, could you share (in an issue on github) how you set up the spacemacs layer? It would be nice to include some more info for spacemacs users in the readme somewhere.

>

Will do that!

mediapathic

@magnus said:
Yes, this is spacemacs with ivy (helm works as well) and some use of projectile.

I use:

Sorry, my question was ambiguous. What I meant was more what functions are these things calling? For example for this one:

• SPC p F

Find file with under cursor as input. Basically works as follow link. Since the ID is unique, it will just immediately to the linked file. If it's a non-unique string, you get a lift of files containing that string.

I went looking for something like ivy-find-file and that doesn't exist. So, what are you actually calling with that? That's the intent of my question.

Thanks!

magnus

SPC p F

projectile-find-file-dwim

SPC s P

search-project-auto-region-or-symbol

SPC s p

search-project-auto

SPC f f

counsel-find-file

SPC f y

show-and-copy-buffer-filename

If you have helm installed instead of ivy, there are equivalent function for that interface.

mediapathic

@magnus lovely! Thanks much.

mediapathic

@magnus on further experimentation: it looks like the search-project ones are specific to spacemacs, but helm-ag works as an approximation (it doesn't search at point though, it'd be nice to figure that one out). But the projectile-find-file-dwim is an excellent tool to add to my aresenal. Thank you!

pinkhurricanegown

I'm going to necropost into this thread rather than opening a new one --

For other folks using Deft as their zettelkasten on Emacs: how on earth do you deal with the slowness? It drives me nuts! And just limiting the number of matches returned for the search using deft-file-limit -- which seems to be the only recommended solution -- just doesn't improve things at all. And I'm on a relatively beefy [albeit, Windows] machine!

Doesn't matter whether I'm running my own init, or a clean emacs.d that only loads deft -- same slowdown. Originally, I was using one big org file and searching headlines with org-velocity and tags with the agenda tag browser -- it wasn't a great solution, but at least it was usable.

Are there any other ways to implement ZK in Emacs?

And, if not, are there any decent solutions for ZK on Windows? Everything I've tried has been either terrible, or so early in development that I can't justify investing in it.