The Archive: Plugin System dev log

ctietze

In the spirit of transparency and buzz and excitement, I want to share with you some progress I'm making on The Archive's plugin system.

---

Starting with an odd one to not spoil all too much at once

Today was the first time I used the very much work-in-progress development view to verify a couple of JavaScript quirks in the interactive REPL. It's basically line-by-line code execution to test algorithms and poke around things:

The top part is the actual script editing part and filled with a placeholder comment. Nothing to see there yet.

So here's a verified rumor -- the scripting language will be JavaScript by default. Reason for that is the excellent and secure environment Apple provides out of the box. (Other languages I would have preferred would require us to ship the language environment or depend on the user installing everything. So this is currently the only sane way to get started. Let's see what the future holds, though.)

Will

Yahoo!

I get to learn another programming language. JavaScript isn't the most popular programming language for no reason. Supposedly easy to understand and comes preinstalled are great attributes. This will make the sharing of plugins a no-brainer. What language do Obsidian, Roam, and Zettlr use?

We'll see how easy JavaScript is to learn. The challenge is locked and loaded.

ctietze

@Will Looking forward to see what you'll be exploring! Coming from Python, the syntax is different, but the core principles and paradigms are similar enough. So I'm confident that you'll become productive in no time.

ZettelDistraction

Is the plugin system TypeScript-compatible? I guess the Javascript-compiled output would be...

ctietze

@ZettelDistraction The system doesn't perform any pre-processing step for you, only plain JavaScript (ES6, I believe? Need to check!). To get started, it all needs to be contained in 1 file, so for complex plugins, Babel or whatever needs to be used? Using require(...) to import files from the plugin bundle is on my list, but that's beyond a first version to do things with

If you're a TypeScript user, I'd eventually really like to explore how to make the workflow bearable. Delivering type annotations, for example.

ctietze

• The JavaScript code without syntax highlighting is, uh, a bit on the ugly and confusing side.
• Running the script with insufficient access permissions correctly produces an error (see the console). I then enabled the expected input ("All Notes") and output ("New File").
• The configuration inspector to the right is almost complete. That's all the inputs and outputs in their boring glory at the moment. But the file visiting one's are bound to be made user configurable instead of the script author's decision.

The script I executed is a built-in one: "Statistics of All Tags":

let tags = {};

// match[0] is the whole hashtag, match[1] the tag name without the hash
let tagRegex = /(?:\s|^)#(\w*[A-Za-z_]+\w*)/g;
for (let note of input.notes.all) {
    for (let tagMatch of note.content.matchAll(tagRegex)) {
        let tag = tagMatch[1].toString().toLowerCase();
        tags[tag] = (tags[tag] === undefined) ? 1 : tags[tag] + 1;
    }
}

// Object.entries produces [["tagname",123],...]
let sortedTagCountPairs = Object.entries(tags).sort((lhs, rhs) => lhs[1] < rhs[1]);
let body = "Tags:\n";
for (let tagCount of sortedTagCountPairs) {
    body += tagCount[1] + " #" + tagCount[0].toString() + "\n";
}
output.newFile.content = body;

It's done for quite some time (2022-08-06) and the plugin management window shows it's info:

ctietze

The "Copy Settings as JavaScript" button is meant to kick-start development with a template that exposes all the inputs and outputs.

The mechanisms are a bit simple for now. If you selected all the options, you'd get this

// Each note object has these properties:
// - `path`: The absolute POSIX path to that file.
// - `filename`: The filename without extension. 
//    (Can be considered the title.)
// - `content`: The full text of the note.

// Array of all notes in the user's archive.
const allNotes = input.notes.all;

// Array of selected notes. Can be empty, the currently visited (edited) note,
// or multiple notes for bulk operations.
const selectedNotes = input.notes.selected;

// Full text of the currently visited note.
// (Is the same as `selectedNotes[0].content` when a single note is selected.)
const editorContents = input.text.all;

// Selected text in the currently visited note. Can be empty
// when no note is visited or the selection is empty
// (that is, when the cursor blinks normally).
const selectedText = input.text.selected;

// 📝 Your Script Goes Here!

// Set the text to insert on behalf of the user
// in the editor when the script has finished.
// (Affects the currently visited file, not the one you'll be creating.)
output.insert = "";

// Set the contents of a new file that the app should
// create when the script has finished.
output.newFile.content = "";

ctietze

Writing JavaScript code without any visual guidance is quite the challenge, I found. I'm spoiled

Luckily, to get rudimentary syntax highlighting for a well-known language like JavaScript to work requires just the inclusion of a couple of simple open source libraries, and things are much more readable:

ZettelDistraction

Very nice.

EternalRecursion

I presume development of The Archive is in Swift using Xcode? (More of a question than an actual presumption lol.)

What open source libraries are you using to facilitate the JavaScript execution and editing features?

-- ER