# Rene's Setevi Converter Script

This discussion was created from comments split from: Setevi: expandable note link trees.

Author at Zettelkasten.de • https://christiantietze.de/

• It is coming together. The first prototype gives a #tags and notes overview. Code is on github, requires python3; I didn't create standalone executables yet.

Well, it is a start, but lots of formatting work still needs to be done.

Little example:

[[201802200331]] Note title
The above leads to duplicate note titles when the note id
is replaced by its title.

[[201802200331 Note title]]
The above should eliminate that problem if my code
worked correctly :-)


Due to the nature of Setevi, bulleted lists that contain links probably also deserve a special treatment:

* [[201802191807]] First test note
* [[201802191808]] Second test note
* [[201802191810]] A third note


I have to split text from 'links' to create clickable nodes. That destroys bulleted lists.

So I played with this Zettelkasten:

which can be found on github in the scratch/zksetevi folder; the resulting HTML file is in the output/ directory.

Here are some screenshots:

• It's much better now that I place links after the paragraph containing them and added a bit of color.

See the difference:

• Cool! Played around with it and am surely looking forward to more of this At the moment, running the conversion script repeatedly changes some connection IDs, but I didn't investigate why this behaves unpredictable at the moment. I guess you'll have that fixes in an hour or two anyway

Author at Zettelkasten.de • https://christiantietze.de/

• @ctietze Thx. Whaaat? Connection IDs are being changed? Can you elaborate? No need for a github issue but can you provide more details? I will look into the code to check for oddities... Unfortunately and fortunately I have day job, too, so I cannot make any promises regarding how much time I can dedicate to this. So, maybe not in an hour or two, but ...

• edited February 2018

Had time for a little update: citations!

I have pretty long citekeys:

Next up is a tab next to notes and tags, so you can find notes that cite a given source.

But that'll probably have to wait until after dancing class ...

• @ctietze said:
At the moment, running the conversion script repeatedly changes some connection IDs, but I didn't investigate why this behaves unpredictable at the moment.

Maybe a factor of unpredictability is: When you link to a note id and you have multiple files whose filenames start with that exact note id, then only the first one found will be picked. But I'll dig into this once I can reproduce. The code is probably a bit confusing if you're new to the setevi node structure and how I "did" things, but if you find sth, I'll look at it.

• A bit of progress:

• tags are now also displayed inside notes
• citations not only show the source but also other citing notes
• there is a "@citations" "tab"/"node" at the top

Here a few screenshots of the current state:

## Citation open inside note

• @rene said:

@ctietze said:
At the moment, running the conversion script repeatedly changes some connection IDs, but I didn't investigate why this behaves unpredictable at the moment.

Maybe a factor of unpredictability is: When you link to a note id and you have multiple files whose filenames start with that exact note id, then only the first one found will be picked.

I have to correct myself. This is a python feature: Not all my lists (sets, dicts) are ordered, so the order in which I process nodes is not guaranteed (especially when I iterate over dictionary keys). Hence node_ids can vary from run to run which is reflected in the connections.

I didn't dig into this deeply. If you think the generated HTML links to notes it shouldn't, then please tell me I am wrong .

• I hope you don't mind the spamming.

When converting my real ZK, I was astonished how cool browsing it with Setevi is (by tag, citekey, related notes...) . The notes look quite OK, and with all the links in place I could really follow my web of thoughts and get carried away. Awesome!

Here is a more realistic example of a ZK note style markdown document:

And even cooler:

# ZK on Android

• On with the spam!

I incorporated pandoc parser support. This gives us tables and fenced code blocks for free.

If you want to try it, make sure you have the pypandoc package installed (which I just discovered, makes life so much easier):

pip install pypandoc
# or: pip3 install pypandoc, depending on your system


Once I make more progress, I'll generate OS specific executables so nobody has to mess around with python and packages. I want to incorporate more parsers, maybe I'll find a decent multimarkdown capable one.

Also note, that the pandoc parser is rather slow. So don't process hundreds of notes with it if you don't want to wait for a couple of minutes, staring at the progress messages.

Here are 2 sample screen shots:

• We support local images (via pandoc):

• ## Big ones

• Fast Multimarkdown parser
• Syntax highlighting (pygments) in fenced code blocks

I guess, soon it will be time to create executables so everyone can try it out.

I am also in a discussion with @fbb to switch to a pure vertical layout in Setevi.

#### Multimarkdown or Pandoc?

The Multimarkdown parser is way faster than spawning a pandoc process.

The script converts pandoc citations (@citekey) and pandoc fenced code blocks (~~~code~~~) to multimarkdown automatically so pandoc can be avoided.

So the only reason I can think of that requires to select the pandoc parser is if you're using pandoc style tables.

To be fair to pandoc: It's still my favourite way to write markdown. It is also still used for creating the sources of citations

Here's what it looks like:

mmd.png 509.4K
• I have just published a pre-release!

Now even non-python-freaks can convert their Zettelkästen into the current Setevi format, too! (BTW: Maybe @fbb will update it soon, for a more natural reading experience with a strict vertical layout).

It contains standalone executables for both the macOS and Windows 10 versions of the command line tool. I hope they work on other computers, too.

If anyone's brave enough to try them out, I'd be happy to take feedback whether there are some unresolved dependencies or other problems.

A GUI version will be in the works, soon.

## 3rd party tools

You'll need pandoc installed (creates citation sources).

## Usage

usage: zk2setevi.py [-h] [-b FILE] [-e EXTENSION] [-l {single,double,§}]
[-p PARSER] [-w MAX_IMG_WIDTH]
input_folder output_folder

Convert a Zettelkasten into a Setevi HTML page

positional arguments:
output_folder         Output: Folder to write output HTML to

optional arguments:
-h, --help            show this help message and exit
-b FILE, --bibfile FILE
.bib file to use for citations if none is in your
Zettelkasten folder (default: None)
-e EXTENSION, --extension EXTENSION
extension of your markdown files (default: .md)
(default: double)
-p PARSER, --parser PARSER
markdown parser: mmd=internal Multimarkdown,
pandoc=pandoc, native=native (default: mmd)
-w MAX_IMG_WIDTH, --max-img-width MAX_IMG_WIDTH
maximum width of images (default: 320)


This tool provides/uses 3 different markdown parsers:

• native: the fastest but least capable parser

• handles pandoc and multimarkdown citations
• handles pandoc and multimarkdown style fenced code blocks with syntax coloring
• no tables
• supports images
• multimarkdown (default): fast parser, should always be used

• handles pandoc and multimarkdown citations
• handles pandoc and multimarkdown style fenced code blocks with syntax coloring
• handles multimarkdown tables
• supports images
• pandoc: slow parser but can handle pandoc markdown

• handles pandoc and multimarkdown citations
• handles pandoc and multimarkdown style fenced code blocks with syntax coloring
• handles multimarkdown tables
• handles pandoc tables
• supports images

You should always use the default mmd parser, even if your text uses pandoc syntax.

The only exception is: If your text contains pandoc style tables, then go for the pandoc parser. Pandoc [@citations] and pandoc ~~~fenced cod blocks~~~ will be converted internally automatically by the tool, so that the multimarkdown parser can handle them.

• Quite some progress, man!

Couldn't the image width be solved with

img { max-width: 100%; width: auto; }


... and surrounding <img> tags in links to open in a new window?

Author at Zettelkasten.de • https://christiantietze.de/

• edited February 2018

@ctietze said:
Quite some progress, man!

Couldn't the image width be solved with

img { max-width: 100%; width: auto; }


... and surrounding <img> tags in links to open in a new window?

Awesome suggestion!!! My HTML is sooooo rusty. You wouldn't believe what a challenge it was for me to figure out how to embed CSS into the body (<div id="scoped-content"><style type="text/css">) for code highlighting

I've just pushed the change!

• Feel free to open an issue in your own repo and ping me if in doubt

Author at Zettelkasten.de • https://christiantietze.de/

• # BOOOOM !!!

We now have a GUI !!!

It will take me a bit to prepare the binary release but this is awesome! Now it's easy to use for anyone

• Oh, I also renamed the GitHub repository (old references still work but should be updated) to semantic_zk since nobody had claimed it for anything better and I had named the GUI script that way. Goes well with sublime_zk, too

• Phew !

The binaries for both command line and GUI version are released!

It has nothing to do with this release, but I just like this animation:

• @rene you are beast.

I am a Zettler

• edited February 2018

MacOS can really be a pain sometimes.

I had to update the binaries of the release for macOS. I didn't want to create a full-blown release for that.

Now the graphical tool is a "real .app" and doesn't open a terminal anymore. Also some PATH issues have been fixed, in order to find pandoc.

In addition, should anything go wrong, a stack trace is reported for debugging.

The new structure of the macOS release is:

• macOS:
• semantic_zk-pre-0.1b-macOS.zip containing
• command_line/zk2setevi (command line tool)
• semantic_zk-0.1.app (graphical tool)

As before, a few instructions are in the README

• @sfast said:
@rene you are beast.

No worries, I can be a beauty, too

• # BOOOM See it in action!

Semantic_ZK is now documented in Zettelkasten style and has converted its own documentation .

See it in action here!

• ### New Release!

I've just uploaded a new release.

Now you can specify a base URL if you plan to publish the exported HTML to a remote server. This is necessary to get the image links right:

• edited February 2018

The layout in the generated HTML is now all vertical! Thx @fbb for the setevi update! . This makes for far better readable notes.

It's all vertical

xjf.png 411.8K
• Oh boy, this escalated quickly! Great work, @rene and @fbb!

Author at Zettelkasten.de • https://christiantietze.de/

• # The best release ever!

We now have support to export only part of a Zettelkasten!

We can combine the following filters:

• only notes containing specific tags
• never notes containing specific tags
• only notes not older than date(+time)
• only notes not younger than date(+time)

## Command Line Example:

zk2setevi docs_zk docs                         \
--only-tags='#setevi #zettelkasten'  \
--never-tags='#start'                \
--from=20180224                      \
--to=20180225

• convert the docs_zk folder
• write output to the docs folder
• only consider notes tagged with either #setevi or #zettelkasten
• never consider notes tagged with #start
• only consider notes from 2018-02-24 to 2018-02-25 (incl.)

## GUI example

Get it here! (Binary Release)

See it here! (Resulting HTML export)

• edited October 2020

... Setevi is now Sems: http://sems.inetwebspace.de/sems/
(At the moment only in German language)