Zettelkasten Forum


The Archive: Plugin System dev log

edited November 2023 in The Archive

In the spirit of transparency and buzz and excitement, I want to share with you some progress I'm making on The Archive's plugin system.


Starting with an odd one to not spoil all too much at once :)

Today was the first time I used the very much work-in-progress development view to verify a couple of JavaScript quirks in the interactive REPL. It's basically line-by-line code execution to test algorithms and poke around things:

The top part is the actual script editing part and filled with a placeholder comment. Nothing to see there yet.

So here's a verified rumor -- the scripting language will be JavaScript by default. Reason for that is the excellent and secure environment Apple provides out of the box. (Other languages I would have preferred would require us to ship the language environment or depend on the user installing everything. So this is currently the only sane way to get started. Let's see what the future holds, though.)

Author at Zettelkasten.de • https://christiantietze.de/

Comments

  • Yahoo!

    I get to learn another programming language. JavaScript isn't the most popular programming language for no reason. Supposedly easy to understand and comes preinstalled are great attributes. This will make the sharing of plugins a no-brainer. What language do Obsidian, Roam, and Zettlr use?

    We'll see how easy JavaScript is to learn. The challenge is locked and loaded.

    Will Simpson
    My zettelkasten is for my ideas, not the ideas of others. I will try to remember this. I must keep doing my best even though I'm a failure. My peak cognition is behind me. One day soon, I will read my last book, write my last note, eat my last meal, and kiss my sweetie for the last time.
    kestrelcreek.com

  • @Will Looking forward to see what you'll be exploring! :) Coming from Python, the syntax is different, but the core principles and paradigms are similar enough. So I'm confident that you'll become productive in no time.

    Author at Zettelkasten.de • https://christiantietze.de/

  • Is the plugin system TypeScript-compatible? I guess the Javascript-compiled output would be...

    GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.

  • @ZettelDistraction The system doesn't perform any pre-processing step for you, only plain JavaScript (ES6, I believe? Need to check!). To get started, it all needs to be contained in 1 file, so for complex plugins, Babel or whatever needs to be used? Using require(...) to import files from the plugin bundle is on my list, but that's beyond a first version to do things with :)

    If you're a TypeScript user, I'd eventually really like to explore how to make the workflow bearable. Delivering type annotations, for example.

    Author at Zettelkasten.de • https://christiantietze.de/

    • The JavaScript code without syntax highlighting is, uh, a bit on the ugly and confusing side.
    • Running the script with insufficient access permissions correctly produces an error (see the console). I then enabled the expected input ("All Notes") and output ("New File").
    • The configuration inspector to the right is almost complete. That's all the inputs and outputs in their boring glory at the moment. But the file visiting one's are bound to be made user configurable instead of the script author's decision.

    The script I executed is a built-in one: "Statistics of All Tags":

    let tags = {};
    
    // match[0] is the whole hashtag, match[1] the tag name without the hash
    let tagRegex = /(?:\s|^)#(\w*[A-Za-z_]+\w*)/g;
    for (let note of input.notes.all) {
        for (let tagMatch of note.content.matchAll(tagRegex)) {
            let tag = tagMatch[1].toString().toLowerCase();
            tags[tag] = (tags[tag] === undefined) ? 1 : tags[tag] + 1;
        }
    }
    
    // Object.entries produces [["tagname",123],...]
    let sortedTagCountPairs = Object.entries(tags).sort((lhs, rhs) => lhs[1] < rhs[1]);
    let body = "Tags:\n";
    for (let tagCount of sortedTagCountPairs) {
        body += tagCount[1] + " #" + tagCount[0].toString() + "\n";
    }
    output.newFile.content = body;
    

    It's done for quite some time (2022-08-06) and the plugin management window shows it's info:

    Author at Zettelkasten.de • https://christiantietze.de/

  • The "Copy Settings as JavaScript" button is meant to kick-start development with a template that exposes all the inputs and outputs.

    The mechanisms are a bit simple for now. If you selected all the options, you'd get this

    // Each note object has these properties:
    // - `path`: The absolute POSIX path to that file.
    // - `filename`: The filename without extension. 
    //    (Can be considered the title.)
    // - `content`: The full text of the note.
    
    // Array of all notes in the user's archive.
    const allNotes = input.notes.all;
    
    // Array of selected notes. Can be empty, the currently visited (edited) note,
    // or multiple notes for bulk operations.
    const selectedNotes = input.notes.selected;
    
    // Full text of the currently visited note.
    // (Is the same as `selectedNotes[0].content` when a single note is selected.)
    const editorContents = input.text.all;
    
    // Selected text in the currently visited note. Can be empty
    // when no note is visited or the selection is empty
    // (that is, when the cursor blinks normally).
    const selectedText = input.text.selected;
    
    // 📝 Your Script Goes Here!
    
    // Set the text to insert on behalf of the user
    // in the editor when the script has finished.
    // (Affects the currently visited file, not the one you'll be creating.)
    output.insert = "";
    
    // Set the contents of a new file that the app should
    // create when the script has finished.
    output.newFile.content = "";
    
    

    Author at Zettelkasten.de • https://christiantietze.de/

  • Writing JavaScript code without any visual guidance is quite the challenge, I found. I'm spoiled :)

    Luckily, to get rudimentary syntax highlighting for a well-known language like JavaScript to work requires just the inclusion of a couple of simple open source libraries, and things are much more readable:

    Author at Zettelkasten.de • https://christiantietze.de/

  • Very nice.

    GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.

  • I presume development of The Archive is in Swift using Xcode? (More of a question than an actual presumption lol.)

    What open source libraries are you using to facilitate the JavaScript execution and editing features?

    -- ER

  • @EternalRecursion It is! JavaScriptCore is a first-party (read: Apple-made) library for this. It's basically the JavaScript engine built into browser via the WebKit engine by default. The plumbing to make The Archive accessible from JavaScript is custom.

    Author at Zettelkasten.de • https://christiantietze.de/

  • edited July 14

    Can't wait for it to happen :) I do have a few cases where extra custom logic would be handy

  • Progress Report 2024-07-14

    In my last update, I talked about the editor/development console. To rapidly iterate (and not compile for hours) I worked on that in a separate module. In December I began the integration work, then worked on other issues in parallel and cleaned up the text editor a bit, so it took the past ~2--3 days to integrate everything for real and resolve issues. (I'm not particularly good at interpreting the kind of errors you get there by build tools, but I'm much better at it than I was 1 or 2 years ago, thankfully.)

    Evaluating the first script in 2024

    With the dev tool available from the app, the result of my first script execution is this:

    You see a selection of notes in The Archive at the top (pointing at my dev/test note collection where, apparently, I also debugged issues with image inclusion that @Will reported a while ago :)).

    The code in the dev tool is:

    input.notes.selected.map((n) => n.filename)
    

    This obtains all selected notes and from each note picks the filename. That's the highlighted text at the bottom: a JavaScript representation of an array of the filenames as strings.

    The dev tool allows to experiment with things on the fly and remote-control The Archive to some extent. This example is just reading selected notes. So ...

    Producing text on my behalf

    A very simple and rather safe changes is to make a script compute 1 large text output and insert it on the user's behalf:

    output.insert.text = "Hello, World!"
    

    That will 'type' the text Hello, World! into the currently frontmost editor window where the cursor is.

    Safety first

    This kind of output is limited to 1 change per script run. That means consecutive setter statements will overwrite previous ones.

    For example, this will only produce "Bye" after execution:

    output.insert.text = "Hello"
    output.insert.text = "Bye"
    

    This is the current design for what'll become the lowest risk kind of plugins. Plugins that offer simple automations where the assembly of information can be as convoluted as you like, but the users only needs to understand whether the script will e.g. insert text once into the editor or create a new file.

    That's fairly limiting for power users, so that's not all there is to it, but that's all I want to show today :)

    Author at Zettelkasten.de • https://christiantietze.de/

Sign In or Register to comment.