Insert a List of Backlinks Into All Your Notes

nickmilo22 · March 2020

I'm wrapping my head around how to best take advantage of contextual backlinks. It seems to work best if your [[workflow]] uses wiki-style links as you type. So any time you're make a zettle on [[workflow]], you just type it in your sentence, or at the end. [[workflow]].

Then your zettel called 201901019999 Workflow would have a back link that looks like:

[[201802159999 Procedure notes from John Smith's book]]
- After step one is complete, do step two. [[workflow]]
[[202003071035 Example Forum Post on Backlinks]]
- I'm wrapping my head around these backlinks might best. It seems best to work if your [[workflow]] uses wiki-style links as you type. So any time you're make a zettle on [[workflow]], you just type it in your sentence, or at the end. [[workflow]].

Maybe I'm way off. I'm sure there are other uses.

That said, I find automatic contextual backlinks enticing, but dangerous. It just adds noise—at least to my zettels.

I'd rather expend the effort to manually create backlinks. Manual effort is another rep. Reps are good. Just like improving a skill, each rep strengthens the mental connection.

Still, I'm very interested to see where and how this feature develops over the months and years.

piotr · March 2020

I'm using [[...]] instead of tags to have more auto connections. For example systemic healthcare failure note has on top links to health and healthcare issues and therefore is listed in their backlinks.
In backlinks of this note I see that it was itself linked from content of the note about infectious diseases and soon will be linked on Covid-19.
I'm using macros (published by other users on this forum) to help me with finding pages that I want to link ([[ will show the input with autocompletion), and I'm currently experimenting with Sublime ZK where it is even easier and built-in.
I also created (programmaticaly) daily notes for next 6 months and I'm linking a lot from there. You can watch some videos on RoamResearch to find more about that workflow. To me it's superior in every way to classical note-taking, but YMMV, I'm not writer or researcher.

Eiff · March 2020

@piotr Awesome! Sounds like a great workflow. Thanks for sharing.
I've been playing with Roam and bought the course. Super interesting ideas, I just hate waiting for web apps to load. Soon perhaps. Thanks again for all you do!

ctietze · March 2020

Found this set of related scripts to rename [[202003200830 this text]] into [[202003200830]] this text, and also insert level-1-headings that match the file name if they don't exist already. That's handy. https://gist.github.com/JohnAtl/6f719f0249c881a5f7ff4c465909ae42

Heads up! These scripts will change your notes, so make a backup and try it on a copy of your archive folder first.

John · March 2020

@ctietze said:
Found this set of related scripts to rename [[202003200830 this text]] into [[202003200830]] this text, and also insert level-1-headings that match the file name if they don't exist already. That's handy. https://gist.github.com/JohnAtl/6f719f0249c881a5f7ff4c465909ae42

Hi Christian, do you just use the command line to invoke the script pointed at one certain note (.txt or .md extension or what have you) ?

What would be your use case to do so?

ctietze · March 2020

The linked scripts were made by the author for bulk changes. You execute them from inside the note archive folder.

I think they are a good base to make more useful per-file variants that actually take a file name argument. (Should be simple to separate the logic from the loop in the Ruby script, for example.)

I don't use the scripts myself; it's just for reference to quench the thirst of those folks who ask for "sync headings with the filename" and "change links to only contain the ID".

GBC · April 2020

@ctietze said:
Found this set of related scripts to rename [[202003200830 this text]] into [[202003200830]] this text, and also insert level-1-headings that match the file name if they don't exist already. That's handy. https://gist.github.com/JohnAtl/6f719f0249c881a5f7ff4c465909ae42

Heads up! These scripts will change your notes, so make a backup and try it on a copy of your archive folder first.

Apologies (again) to ask, but I am unfamiliar with scripts and don't know what I need to do with these. This sounds like functionality that would be helpful for me, could someone point me in the right direction as to a) how to run them and b) how to trigger them?

If there's a learning resource on scripts so I can stop asking these questions, I'd be grateful to be pointed in that direction, too.

Huge thanks

GBC · April 2020

Further to the above, I've worked out how to execute the .sh files (not how to get them to work in the way I thought they would, but at least to execute, lol) but not the .rb file.

GBC · April 2020

And me again. I've now run the .rb and it does seem to be doing something - that is, creating a new title of all documents to match the file name, and then renaming all existing documents in the folder as. bak.

What I'm noticing is that the newly inserted heading at the top of each document contains the file extension, which just adds visual clutter.

Nick · April 2020

Ya I recently downloaded Andy Matuschak note link janitor. It doesn't work for me because I am using Zettlr and note having a filename number combo, just using numbers within the document.

From a philosophical standpoint I find this interesting and ran into the issue when using back links on TiddlyBlink. They can be very helpful for easily seeing connections, sometimes ones you didn't realize existed. But they can also get tedious and fill up the note with redundant information.

Eiff · April 2020

@JKF said:
What I'm noticing is that the newly inserted heading at the top of each document contains the file extension, which just adds visual clutter.

Do your filenames begin with a 14 digital zettelkasten ID? From the script header: "Assumes filename begins with a 14 digit zettelkasten ID."

Here's a tweaked version of the script that doesn't make that assumption. Hopefully should stop the extension from coming into the header.

https://gist.github.com/davideiffert/b24b36266a575bf42ff1b5d2f15a73d5

GBC · April 2020

@Eiff said:.

Do your filenames begin with a 14 digital zettelkasten ID? From the script header: "Assumes filename begins with a 14 digit zettelkasten ID."

Good point! It's just the Archive default numbering system that I use, I didn't think to check.

Thank you - very much appreciated 😀

Dan_Lardi · April 2020

@piotr said:
I pasted my response two times before, hopefully third time will be successful

Sure, @JKF , are you familiar with terminals? Terminal.app is the default on Mac.
First of all you need to install node. Download '13.9.0 Current' from https://nodejs.org/ and double click to open installer and click on "yes" etc. couple of times until it ends. You can check in your terminal if node --version gives the correct output.
Next thing is yarn https://classic.yarnpkg.com/en/docs/install#mac-stable . If brew install yarn command doesn't work in your terminal, try with curl -o- -L https://yarnpkg.com/install.sh | bash. After that see if yarn --version command gives you version and no errors.
Next download and unpack https://github.com/panpiotrs/note-link-janitor/archive/master.zip
You will need to go to that unpacked directory in your terminal, by default it's most likely in Downloads, so cd /Users/YOURUSERNAME/Downloads/note-link-janitor-master should work.
When you're in there use yarn install to trigger installation of required libraries, and after that yarn build.
Now you can use this project with this command: dist/index.js /path/to/folder/containing/md/files each time you want to generate the backlinks. You can automate it with macro I posted before or some other way, for example to run it every couple of minutes or hours, but I can't help you on that route, never did that.

Good day!

I followed everything here, but when I came to the part where I enter this command, dist/index.js /path/to/folder/containing/md/files, this is the result I get...

(node:49842) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, scandir '/path/to/folder/containing/md/files'
(node:49842) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:49842) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Joses-MacBook-Pro:note-link-janitor-master danlardi$

Question: What should be my next step after I install the yarn build?

Eiff · April 2020

@Dan_Lardi said:
I followed everything here, but when I came to the part where I enter this command, dist/index.js /path/to/folder/containing/md/files, this is the result I get...

Are you sure the path to your folder containing markdown files is correct in the command? This is a silly question, but you're editing that command to point correctly to your folder containing markdown files right?

For example, I run it with this command:

/Users/davideiffert/Dropbox/bin/note-link-janitor-piotr2/dist/index.js /Users/davideiffert/Dropbox/Messx/ZTXT

The first half is the location of the index.js script and the second half is the folder that contains my markdown files. (Note that you can use the FULL path to the script like I have and that will enable you to run it without being in the correct directory in terminal.)

Is that helpful? If not, reply back with the command you're typing into terminal and I'll take a look.

Dan_Lardi · April 2020

@Eiff said:

@Dan_Lardi said:
I followed everything here, but when I came to the part where I enter this command, dist/index.js /path/to/folder/containing/md/files, this is the result I get...

Are you sure the path to your folder containing markdown files is correct in the command? This is a silly question, but you're editing that command to point correctly to your folder containing markdown files right?

For example, I run it with this command:
/Users/davideiffert/Dropbox/bin/note-link-janitor-piotr2/dist/index.js /Users/davideiffert/Dropbox/Messx/ZTXT
The first half is the location of the index.js script and the second half is the folder that contains my markdown files. (Note that you can use the FULL path to the script like I have and that will enable you to run it without being in the correct directory in terminal.)

Is that helpful? If not, reply back with the command you're typing into terminal and I'll take a look.

You are awesome!

I literally just copy-pasted the command without really indicating the path.

Now it works! I'm wondering if you have a Youtube channel where you teach or share your approach with The Archive?

Eiff · April 2020

@Dan_Lardi said:
Now it works! I'm wondering if you have a Youtube channel where you teach or share your approach with The Archive?

Glad to help! No YouTube channel or anything like that at the moment. Something to think about.

Eiff · April 2020

For those of you who want to REMOVE the backlinks section here's a simple script that will do so:

cd /Users/davideiffert/Dropbox/Zettelkasten/
perl -i -0pe 's/## Backlinks[\s\S]*## ...//gim' *.md

Just update the path on the top line to your Zettelkasten note directory, paste it into terminal, and hit enter.

Sometimes I like having the backlinks but other times I like seeing my notes clean without them. This lets me toggle between the two without losing anything.

Note: This must be used with piotr's forked version discussed earlier in this thread. It can be installed using his installation instructions here.

avggeek · May 2020

Hello. Recently got started with Zettelkasten and I feel like this tool is great for folks like me to help build links between different Notes.

I use @EFLS Zettelkasten but am struggling to get it working with the fork from @piotr.

Here is a note that will be referenced in another:

2020-05-11-0851 Ref.md
# Reference

This is the meta note that can be used for linking unprocessed notes.

Over time this note should not have any notes linked to it.

Here is a note that links to the one above using the timestamp:

2020-05-03-1807 cite.md
# Using Reference

[[2020-05-11-0851]]

Here is one note citing the other.

With the above syntax, there is no back-link generated in 2020-05-11-0851 Ref.md. The only combination that seems to work is:

2020-05-03-1807 cite.md
# Using Reference

[[Reference]]

Here is one note citing the other.

When I use the title of the first Zettel, then I can see a back-link appear.

# Reference

This is the meta note that can be used for linking unprocessed notes.

Over time this note should not have any notes linked to it. 
## Backlinks
* [[2020-05-03-1807 Reference]]
    * [[Reference]]

## ...

This is a bit worrying since I cannot be sure that I will never use the same Title between different Zettels. In addition, I notice that there is no new-lines before the start of the backlinks section, but in an earlier comment @piotr mentioned he added that feature.

Can anyone help me figure out what I'm doing wrong?

ctietze · May 2020

The tool, IIRC, expects your note to start with a heading that contains the timestamp or ID again. So the file 2020-05-11-0851 Ref.md should read:

# 2020-05-11-0851 Reference

This is the meta note that can be used for linking unprocessed notes.

Over time this note should not have any notes linked to it.

And then it should work.

Lookup by file name might be a possible addition, too, as a fall-back mechanism. Though don't ask me how to hack that in

avggeek · May 2020

@ctietze said:
The tool, IIRC, expects your note to start with a heading that contains the timestamp or ID again. So the file 2020-05-11-0851 Ref.md should read:

Thanks for the information Christian!

I did try what you suggested and updated 2020-05-11-0851 Ref.md but it seems like cite.md requires the entire heading to be included before the tool will work. However, my understanding was that a reference like [[2020-050-11-0851]] Reference should work, as you have mentioned here.

That was my understanding of @piotr's fork, but so far it seems like I only get a "full" Heading reference like the original tool expects to work.

piotr · May 2020

It recognizes only 12 or 14 digits ids. Without dashes or anything else between digits.

avggeek · May 2020

@piotr said:
It recognizes only 12 or 14 digits ids. Without dashes or anything else between digits.

Hi @piotr I was looking at the regex in the index.ts but could not figure it out. But through a process of trial and error, I did eventually figure out that a %Y%m%d%H%M% string would work. I then went through the slightly painful process of renaming my existing zettels (easy) and then changing my links in the files (not so easy). But now the script works across all my notes. I just need to create a batch script so I can leave it running in the background.

Thanks for making the revised version available!

shauxesis · May 2020

@piotr @Eiff I'm running into some trouble with this setup. Followed the instructions to a T, but I'm getting the following error:

(node:3546) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:3546) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Terminal Command used:

shande@Shandes-MacBook-Pro / % /Users/shande/Downloads/note-link-janitor-master/index.js /Users/shande/Downloads/testing-md-backlinks

(using the index.ts kicks up permission errors, so I'm guessing that's the wrong one to use)

For testing purposes, note-link-janitor-master is located in my Downloads folder, along with a duplicate copy of my archive.

I start at root level: shande@Shandes-MacBook-Pro / %

Then drag the index.js file (located: note-link-janitor-master/dist/index.js) onto the Terminal window, followed by the testing-md-backlinks folder.

I'd be grateful for any help or insight you might be able to offer.

Much appreciated in advance!

bjbarry · May 2020

This functionality is available in slightly different form in Drafts. A Drafts action that locates backlinks is freely available for download. No configuration is necessary. When you tap on the Action it locates all backlinks to the note currently being edited and displays them in a list.

The Action can be activated - optionally - by pressing a named button, using an icon in lieu of a named button, or by assigning a keyboard shortcut to the Action. Works in both macOS and iOS versions of Drafts.

You don't have to update anything because the Action only displays backlinks when you ask to see them - and finds them based on a (very fast) search conducted at that time.

avggeek · May 2020

I have a strange "heisen"-bug to mention around line endings. "heisen"-bug as I'm not able to replicate the behavior but I've seen it happen a couple of times now.

I have the note-link-janitor dist/index.js running a scheduled task in Windows using a Batch file. The sequence of events that I'm seeing is:

Create a new Zettel #1 in Emacs (saved with Unix line endings)
Create another Zettel #2 on another device that links to Zettel #1. This is also saved with Unix line endings.
On first run of the scheduled task (maybe?), the script will update the backlinks in Zettel #1 correctly. It also seems to change the line endings for the existing content from Unix style to Windows style. However, the backlinks section continues to have Unix style line endings.
- This causes Emacs specifically to start printing ugly "^M" characters in the file to warn of mixed line endings.
I've tried deleting the backlinks and re-creating them while having the file open but when I do so the line endings don't change. Also tried this by triggering the scheduled task manually but that hasn't made a difference.

Mentioning it here in case some node.js gurus have seen this type of odd cross-platform behavior.

tantrig · September 2020

Hello!

Thank you so much @piotr for your work, it is great.

I have, however, two issues:

I use Typora and thus my wikilinks need a "trick" to work: they are actually written that way "[filename]". Do you think that I could change your script to make that change? Right now the script does get the [[wikilink]], the only part it misses is (filename.md) that should be appended to the end of the links.
I use quite a lof of math in my notes using Mathjax. Thus, my notes have a lot of strings that look like $\beta_{i=1}$ etc etc. When the script takes the "context" of the link, it seems that it doubles every backslash, thus breaking all of the Mathjax code. Is there any way to correct that?

Edit: actually I think the script also adds backslash in front of underscore (_) which further breaks the Mathjax code.

tantrig · September 2020

Sorry I meant "[filename] (filename.md)" which is a standard link in markdown.

tantrig · September 2020

Sorry for the triple post: I create a modified version of the script by Piotr that doesn't include "context" around the links and have links formated as [[note_title]](note_title.md) (works with Typora).

Available here: https://github.com/raphaelhuleux/note-link-janitor-Typora/blob/master/README.md.

blundin · October 2020

I tried to install this script (both the original and @piotr's mod) and I'm finding some strange behavior. I wonder if any of you have seen this?

First, the original seems to do nothing to any of the notes in the folder. Second, when I run @piotr's script, it only modifies three of my notes, even though there are many more notes with actual backlinks. One of these backlinks sections look like this:

## Backlinks
* [[Evaluating technology for adoption]]
    * This is in particular regard to the [[Digital Declutter]] Newport recommends to start with [[Digital Minimalism]]

## ...

But the note linked to there on evaluating technology is actually named: "201903301057 evaluating technology for adoption.md". Also, the first line of the linked to note is:

# Evaluating technology for adoption

None of the created links in the Backlinks section include the time-base IDs. Is this a bug? Am I missing something?

rajesh · January 2021

I created a Github Actions workflow to automatically trigger Andy Matuschak’s note-link-janitor on every push to the repository (which contains all the md files for version control).

Using this means, your repo is maintained without the necessity of local cron-jobs or installation of yarn.

All you need to do is copy this workflow to the /.github/workflows/ folder.

Zettelkasten Forum

Insert a List of Backlinks Into All Your Notes

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion