Visual Studio Code (VS Code) extensions for ZK
Based on this comment by @mlevison and comments by @jay_pee_ , let's start a thread to discuss pro/con and best practices for doing a ZK with VS Code. I have found four extensions that explicitly claim to be designed for ZK:
I am brand new to VS Code so I don't have opinions on them yet, but I would love to see discussion here and have people let us know about any other directly relevant extensions (but let's not discuss all of the 100s of general Markdown extensions available unless you have a particular ZK workflow you want to share).
I'm also interested to hear whether anyone knows which of these extension developers seem to actively engage with users and are open to feedback.
Howdy, Stranger!
Comments
I've tried off and on to use VS Code (or it's twin, https://vscodium.com/), and I'm also interested. So far, one of my big problems is the extensions either use their own custom link format for linking (so, I would need to translate things), or want the entire filename in the double square bracket (so... I would need to translate things). It would be nice to have one that either used The Archive format, or that let you define the ID for the links. I thought this would let me do that, but I couldn't make it work.
As for visuals, I liked VS Code Markdown Notes the most, it highlighted the inter-links great, and other things. None of them really did source code for Markdown (though not a lot did that either among Markdown extensions).
Professionally, I use Visual Studio Code for lots of things, so using for this would be great (my fingers already have some muscle memory for certain operations). I haven't been brave enough to try my hand at a rolling my own extension for what I want just yet.
Though, I'm trying to cope with the loss of an "omnibar" or "insta-search"...
Here, you are talking about importing existing notes, correct? I don't use TheArchive (no Mac) but is there not some type of regular expression that could fix all the links?
Does "search across files" (ctrl+shift+f) not work well enough?
Yeah, importing existing notes is one part. I use nvPY for now, and it uses the same format as The Archive (at least, my modified version does...). While I could (and have) write scripts to convert things around, I like links just being the ID without the filename, such that renames also don't require more scripts and babysitting.
Search Across Files can work, but will be slower for much larger archives (as we all hope to get to one day). It can work "well enough", but I would still miss the faster searching (and search operators) that I've grown used to.
None of this a deal breaker of course, but if I have to make tradeoffs between one solution and another, it's good to list those out and decide which set is what people really want (and if they can be fixed via plugins, or just worked around with a new method).
If you have time, please contribute to the nvPY row of this software comparison table directly or in this thread
I have no experience with searches across many VS Code files. Can you give me a sense of how large a group of plain text files has to be before this type of search gets slow or difficult?
How often do you need to search in the file content rather than just the file name? If it's just the name, have you tried something like the Fuzzy search extension? That seems super-fast.
Do you know of any extensions that allow indexing of file contents in a certain folder for faster search?
Yes, totally agreed!
I'll try to add some details to the reddit thread, sure.
I've not experimented in awhile with larger groups of files, but given the regular way this works, VS Code would have to open/search/and then display results from disk for the larger set of files (as it doesn't keep all files of a project in memory from what I can tell). As such, that will just be slower than other methods that keep either all file content in memory, or that utilize an index, as you said. I don't know of any other extensions that use an index either, though I may see what I can find.
File name searches aren't so bad, and Fuzzy search/fzf is ok there. Main use I have for it, is for when getting ready to link a note around the archive, searching for key terms that are related seems more thorough than just following trails of links. It's a benefit of being digital that analog systems couldn't realistically do: just look through everything for a word, and pull all those out to review. For me, it counts as a poor man's "Back links" when I search for the link ID of a given note, and all the notes that contain that link show up.
As you said, everything here is a balancing act. Interface speed is very important, but I wonder how much delay would annoy me. Is 2 seconds OK for a full content search? 5 seconds would be too slow for me.
Since you use VS Code at work, maybe you could run a search on a folder of a bunch (100+? 1000+?) plain text files as an experiment.
I found three (one free, two paid), but you may be able to find more due to your experience. I can't evaluate the free one, but maybe you can:
Code Index Search
Entrian Source Search $30 USD
FastFind $20 USD
It also occurred to me that if VS Code is the system default editor for .md files, a third-party search program with 1) a global hotkey 2) fast search 3) ability to index only my ZK folder rather than the whole drive might be an acceptable search method (for me, at least).
The fzf videos I saw had literally no delay. Were these misleading, or do you have insanely high standards?
Not sure how long you've done the ZK, but this concern might be lessened as you either get better at remembering links or as you get more comfortable with not linking to 100% of all possibly relevant notes. Remember, you can always add more links later if you discover additional connections.
Thankfully, Christian provided a git repo with 10000 markdown files for people to test things with.
As for tests, system setup and current load affect things a lot. I can test, but you can run the same tests and get different results just due to that alone. I have tried to run "dumb" tests before (mainly taking the above repo, and searching for words I know aren't in there just so the only thing I see is time to search, not any extra bit of time rendering/remembering what hits were found), and it was more than 5 seconds for a full search.
I may try that free indexer on it, but being a "Code Indexer", I worry it may try to "help" with code structures, but maybe not. Either way, should be fun to play with.
I am more than happy with fzf performance and similar (using the Control/Command+P menu is extremely fast to do file name searches, no extensions needed). I didn't mean to sound like it wasn't good enough, it works for that.
As for the FOMO, it is a real problem, in collecting and connecting, at least for me. I'm trying to come to terms with the idea of notes "laying dormant" (sounds better) for some time before possibly being rediscovered. But, sometimes I can remember having a particular note on a subject, but I do not remember much of anything that I might have linked it to (working on that as well). Having that search as a fall back is nice for those times where you think "I know I've seen this before...", to either find it, or prove to yourself you didn't note it down (but probably should now).
I did add details about nvPY to the reddit table, and I'll try to play with the Code Index Search this weekend (nothing else to do...). It might be worth editing all my files to make use of the link style the extensions we have now use to play with this more, or finally decide it's worth trying to make/edit an extension myself.
To add some info to the Code Index Search: it's based on Apache Lucene, and it might be worth looking for benchmarks that compare Lucene with SQLite Full Text Search (FTS) 4. Because it's prohibitively simple to set up a SQLite search index, and SQLite is available on virtually every operating system without much setup. The database is stored in a single file, that's it.
One comparison of a Microsoft SQL FTS implementation vs Lucene: https://www.dbbest.com/blog/lucene-vs-sql-server-fts/ -- MS SQL FTS appears to be good for "simple queries", and I don't know why Zettelkasten users would want to query a word index for complex terms all day, every day, so it might be a worthwhile trade-off.
If anyone wants to experiment with SQLite FTS and is capable of installing SQLite, start a database like this:
Go through all your notes and store them in the
zetteltable.The FTS virtual table will be populated as well. Then you can query the search index:
Note that FTS tables don't automatically update when their external content tables change. I can provide SQLite
TRIGGERs for that if needed. But this should get you started with "static" experiments!Author at Zettelkasten.de • https://christiantietze.de/
When working with the ZK, how often are you searching just for a filename vs. doing a full text file content search? I'm hoping others might answer this also - it's important for figuring out the search time/resource trade-offs.
If it's just an occasional fall-back, then 5 seconds isn't big deal, for me at least. If it's several times a day, then that's too much.
Thanks!
Well, if you do decide to make one, obviously I'm happy to help test and/or in any other way I can...um, other than actually doing the programming, of course!
@ctietze , I'm not a programmer, so I can't quite parse this:
But can you tell me: will it work with individual plain text files, or only for text stored in single database? Or does it make a database from individual files to do the search?
I tried that extension a bit. It failed to even install, missing a file in the zip, so I didn't get far. The instructions also seemed to need a server address to reach lucene or something like it, so that also seemed like a steep price to pay.
I might play with sqlite, it's been awhile since I tinkered with it. I would love any triggers or things for it.
The latter: both Lucene search and SQLite FTS depend on a database that mirrors the notes on disk, as far as I know.
Author at Zettelkasten.de • https://christiantietze.de/
Yeah, as far as I know, every solution will have to build some form of index, and search over that. They usually also store the full document as well, so a reindex is just from their own sources.
As a CLI-like option, I found this talk awhile back that could be modified to work similarly:
https://www.iaria.org/conferences2018/filesDBKDA18/AndreasSchmidt_Tutorial_SearchEngine.pdf
I added recently VS Code as an external editor to The Archive.
For me, VS Code offers the best experience for scientific writing (Markdown + MathJax, and integrated Jupyper Notebooks) - without the need to learn Emacs.
Here is an exemple with VS Code (in French):
Hello all, I started a vscode extension recently that uses lucene to do full-text search of my notes. It takes about 10 seconds to index https://github.com/Zettelkasten-Method/10000-markdown-files . Search is very fast - at least I can't come up with a slow query. I haven't spent any time trying to improve the indexing performance, so there may be some quick wins to be made there.
Or course, I started my extension before looking for existing options. The other extensions seem more focused on other note-taking efficiency features, which my extension currently lacks.
I might reach out to the other extension developers to see if they want to add full text search to their extension.
My extension: https://marketplace.visualstudio.com/items?itemName=uozuaho.note-searcher . It currently requires Java, but I'm thinking about trying https://lunrjs.com/ to remove that dependency.
@uozu_aho Nice project! I am working on my own VS Code extension for Zettelkasten at the moment too, searching was low on my TODO list, so I haven't looked into it yet, so I appreciate being able to learn from your project
. My project is also one of those extensions focused on making note-taking more efficient, but it also focused on navigating my Zettelkasten, so search will come up at some point too.
Little late to this thread, but I developed a VSCode extension called "Zettel Archive (Markdown Notes) that has very similar capabilities to The Archive. I did this only because The Archive was not available for Linux or MacOS and I have to switch to those platforms during the day.
I developed this mainly for myself so there's not many settings you can adjust, but if you like The Archive, you should like this too if you need to use other platforms.
@cobblepot @mleo2003 @ctietze @grayen Feel free to check it out and let me know what ya'll think!