Comments

• Andy

  June 2023

  I'm not a mathematician or other formal scientist, but I must say: the idea of a LaTeX-based Zettelkasten was begging to be implemented, so I bow down to this amazing-looking implementation!

  Having said that (and I mean it—what an amazing idea), I would think that Pandoc Markdown would still be the best way to go for most people these days. Have you tried Pandoc Markdown? It has pretty good inline LaTeX support. @ZettelDistraction is a mathematician who has used Pandoc Markdown in Zettlr, and who seemed pretty enthusiastic about it, at least until recently.
• alfred

  June 2023 edited June 2023

  @Andy said:
  I'm not a mathematician or other formal scientist, but I must say: the idea of a LaTeX-based Zettelkasten was begging to be implemented, so I bow down to this amazing-looking implementation!

  Thanks!

  Having said that (and I mean it—what an amazing idea), I would think that Pandoc Markdown would still be the best way to go for most people these days. Have you tried Pandoc Markdown? It has pretty good inline LaTeX support. @ZettelDistraction is a mathematician who has used Pandoc Markdown in Zettlr, and who seemed pretty enthusiastic about it, at least until recently.

  I did have a look into pandoc but decided it would be easier to just write the LaTeX myself. The issue is that I want the links between the rendered pdf (or html) documents to be clickable (I want to read the notes in rendered form not the source markdown/tex) and I couldn't find a way of handling links between external documents with pandoc. The implementation of this in LaTeX is fairly non trivial - you need to keep track of all the files in a .tex document that defines how to reference other files. I'm not sure if there is an existing automated tool for doing this. A benefit of using LaTeX to handle the links between notes if I then want to produce a draft of a paper, all the references automatically work if I paste all the notes into one document. I imagine there would be some markdown specific references to parse if you wanted to do the same thing with pandoc / markdown.

  Is the main benefit of using pandoc + markdown that it is quicker to write markdown than LaTeX?
• Andy

  June 2023

  @alfred said:

  The issue is that I want the links between the rendered pdf (or html) documents to be clickable (I want to read the notes in rendered form not the source markdown/tex) and I couldn't find a way of handling links between external documents with pandoc.

  This is basically a user-interface issue that can be solved by modifying your zettel-editor as needed. Perhaps @ZettelDistraction can indicate how this works in Zettlr.

  A benefit of using LaTeX to handle the links between notes if I then want to produce a draft of a paper, all the references automatically work if I paste all the notes into one document. I imagine there would be some markdown specific references to parse if you wanted to do the same thing with pandoc / markdown.

  Yes, but I think the Markdown-specific reference parsing would be trivial if not already supported by Pandoc. The pandoc-eqnos filter is one way of handling equation numbers in Pandoc Markdown.

  Is the main benefit of using pandoc + markdown that it is quicker to write markdown than LaTeX?

  Yes, Pandoc Markdown is more lightweight than HTML or LaTeX, which is why many people prefer to write in it. Daniel Allington expounded the anti-LaTeX position pretty strongly: "The LaTeX fetish (Or: Don’t write in LaTeX! It’s just for typesetting)" (2016).

  You seem super-happy with LaTeX so I'm not saying that you should change your workflow, but you might be interested to look deeper into Pandoc. I think you would find it interesting even if you choose to stay with your LaTeX Zettelkasten.
• ZettelDistraction

  June 2023

  @Andy said:
  This is basically a user-interface issue that can be solved by modifying your zettel-editor as needed. Perhaps @ZettelDistraction can indicate how this works in Zettlr.

  There are Pandoc filters to convert [[WikiLinks]] to Markdown links that Pandoc can then convert to LaTeX hyperlinks--but I have yet to try these. Until I saw @alfred's example of a PDFLaTeX Zettelkasten with clickable links (I'm impressed!), it did not occur to me to attempt to generate a PDFLaTeX Zettelkasten, let alone one from a Markdown+LaTex Zettelkasten.

  Since Zettlr calls Pandoc with YAML default files and a bibliography file, in my case, exported from Zotero in Better CSL JSON format (using the Better BibTeX plugin for Zotero), we can set the Zettlr editor aside. We could reuse the YAML default files for each of the exported output formats--at least those that produce LaTeX, PDFLaTeX, and HTML (with some loss of detail--I'm less interested in this). Since I use the header-includes: YAML variable in the default export files to define a default font and to include several LaTeX packages that I use, I added the more-header-includes: YAML variable to the Pandoc LaTeX template so that individual Markdown files could declare their own LaTeX packages and macro definitions in addition to the defaults for the exported output.

  There are two kinds of export--one to generate a standalone PDFLaTex Zettelkasten and another by combining notes into a single paper. Here's where a tradeoff occurs between a LaTeX-only system and one with Markdown: the YAML headers of the individual Markdown files issue commands to Pandoc, resolved at the LaTeX compilation step. These might be incompatible, and it occurs when the individual Markdown files include LaTeX directives in the YAML header. Some preprocessing of the individual LaTeX files selected for inclusion in a paper to combine the YAML headers and discard redundancies seems necessary. However, I will sidestep this workflow aspect until I have a better idea, or I have to repeatedly remove YAML from the combined file by hand.

  I will think about this. I, too, moved away from Obsidian because I found the core and community plugins awkward to use, and the opportunity cost of rewriting them, especially after an API overhaul that made Obsidian less helpful to me, was higher than writing VS Code extensions to do similar things. There are helpful commands in Obsidian, such as renaming links if a filename changes, but this is another feature that I probably will add to a VS Code extension. Likewise, a workable Zettel graph exists as a VS Code extension. I spend more time programming than doing any mathematics. When I retire, I will have more time.

  GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.
• alfred

  June 2023

  @Andy said:
  You seem super-happy with LaTeX so I'm not saying that you should change your workflow, but you might be interested to look deeper into Pandoc. I think you would find it interesting even if you choose to stay with your LaTeX Zettelkasten.

  Thanks for your suggestions, I had assumed most people would be happiest writing in LaTeX but I suppose this is not the case. For the project I guess it depends on the user, for practicing researchers we can probably assume the requisite level and familiarity with LaTeX for this not to be an issue - I would assume they would prefer the control writing directly in LaTeX and be happy not to learn markdown syntax that represents LaTeX commands that they already know. If, say, students or people less proficient with LaTeX wanted to use the project then I agree that it would be worth looking in to. I can also see that it may be much quicker to write in markdown. Daniel Allington's post is mostly about the fairly steep learning curve for using LaTeX (and that it isn't worth it if you're not going to typeset mathematics).

  Having thought for a bit I think it might be possible for the project to support both markdown + pandoc and writing in pure LaTeX. Since you have to specify the header of the .tex file when using pandoc I don't think it would be too difficult to integrate with the templates that I already use. A possible workflow for writing in markdown would be to set it up so that the markdown is rendered by pandoc into the format that I currently write by hand.

  This is basically a user-interface issue that can be solved by modifying your zettel-editor as needed.

  I agree with this, but perhaps the cleanest way of doing this is to actually render the LaTeX, using something like the above. The eqnos filter is interesting and would work for referencing within one file. My main concern that makes implementing pandoc support not completely trivial is the interplay between the interdocument referencing in markdown and LaTeX.
• ZettelDistraction

  June 2023 edited June 2023

  @alfred said:
  My main concern that makes implementing pandoc support not completely trivial is the interplay between the interdocument referencing in markdown and LaTeX.

  This is not a concern in my case, since the ID occurring in the unique level-one header of a markdown document is the basename of its filename ID.md. The level-one header always has the form # ID Title.

  GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.
• alfred

  June 2023 edited June 2023

  @ZettelDistraction said:
  There are two kinds of export--one to generate a standalone PDFLaTex Zettelkasten and another by combining notes into a single paper. Here's where a tradeoff occurs between a LaTeX-only system and one with Markdown: the YAML headers of the individual Markdown files issue commands to Pandoc, resolved at the LaTeX compilation step. These might be incompatible, and it occurs when the individual Markdown files include LaTeX directives in the YAML header. Some preprocessing of the individual LaTeX files selected for inclusion in a paper to combine the YAML headers and discard redundancies seems necessary. However, I will sidestep this workflow aspect until I have a better idea, or I have to repeatedly remove YAML from the combined file by hand.

  I got around this issue with my implementation of exporting a paper by making the user declare the LaTeX header that they want to use in the export. Then it effectively just pastes in the commands from the main body and if it doesn't render then its the user's fault. Another way to ensure compatibility is to make all the notes use the same header, I think it would be quite rare that you really need two incompatible headers. A very annoying quirk of LaTeX headers is that the order if the imports matters and this could be an issue if you combine headers from different files.

  There are Pandoc filters to convert [[WikiLinks]] to Markdown links that Pandoc can then convert to LaTeX hyperlinks--but I have yet to try these.

  @ZettelDistraction said:
  This is not a concern in my case, since the ID occurring in the unique level-one header of a markdown document is the basename of its filename ID.md. The level-one header always has the form # ID Title.

  I was under the impression that pandoc doesn't have support for the LaTeX package xr-hyper which is what I use to get the links in the PDFLaTeX render of the Zettelkasten, so one would need to convert from the markdown [[WikiLinks]] to the relevant LaTeX commands to get this to work, and this is what I wouldn't know how to do.
• Andy

  June 2023

  @alfred said:

  I can also see that it may be much quicker to write in markdown. Daniel Allington's post is mostly about the fairly steep learning curve for using LaTeX (and that it isn't worth it if you're not going to typeset mathematics).

  Yes, Allington's post isn't so relevant to mathematicians. Nevertheless, people who need to typeset mathematics but don't use very much of it can find Pandoc an attractive alternative to LaTeX. For such people, the long-term maintainability of the more-lightweight Markdown is far superior to LaTeX. Here at zettelkasten.de there is generally an emphasis on maintaining a lifelong note system, not just a note system for a single project like a dissertation. When your note system scales to many thousands of notes over decades, the difference between the maintainability of Markdown and LaTeX is significant.

  Having thought for a bit I think it might be possible for the project to support both markdown + pandoc and writing in pure LaTeX. Since you have to specify the header of the .tex file when using pandoc I don't think it would be too difficult to integrate with the templates that I already use. A possible workflow for writing in markdown would be to set it up so that the markdown is rendered by pandoc into the format that I currently write by hand.

  Yes, I was thinking the same thing. If it seems easy to implement, I would put it on your planned features list.

  I agree with this, but perhaps the cleanest way of doing this is to actually render the LaTeX, using something like the above.

  If "cleanest" refers to the visual aesthetics of the rendered PDF, I agree. But again, there is the issue of how well the process of rendering a duplicate set of PDFs scales as your note system scales to many thousands of notes. For a smaller number of mathematics-heavy notes, LaTeX-Zettel is indeed a very nice solution.
• ZettelDistraction

  June 2023 edited June 2023

  @alfred said:
  I was under the impression that pandoc doesn't have support for the LaTeX package xr-hyper which is what I use to get the links in the PDFLaTeX render of the Zettelkasten, so one would need to convert from the markdown [[WikiLinks]] to the relevant LaTeX commands to get this to work, and this is what I wouldn't know how to do.

  True. My approach has been to turn off LaTeX macro expansion in Pandoc (apologies for not mentioning that design decision upfront--those decisions don't come instantly to mind, and they didn't come instantly as I stumbled through my setup). A preprocessing script to convert [[WikiLinks]] to xr-hyper would work better than having Pandoc do any LaTeX macro expansion because of the side effects:

  1. The Pandoc extension latex_macros causes Pandoc to expand LaTeX macro definitions as it converts to LaTeX. See Pandoc extension: latex_macros.

  2. The Pandoc manual recommends disabling the latex_macros extension when converting to LaTeX and PDF.

  When latex_macros is disabled, the raw LaTeX and math will not have macros applied. This is usually a better approach when you are targeting LaTeX or PDF.

  3. Zettlr's default Pandoc configuration turns latex_macros on. This is non-standard; it can make re-using some example macros difficult and complicates the already complicated handling of macro expansion in LaTeX.

  Post edited by ZettelDistraction on June 2023

  GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.
• Andy

  June 2023

  @alfred: I found a nice Pandoc cheat sheet by Keefer Rourke, "Typesetting with Pandoc Markdown: How I Take Notes" (2021), which is written for "other students pursuing CS and/or math education" who want to write "the bare-minimum of LaTeX to get by". It's a very clear brief guide that may give you some ideas for Markdown support in LaTeX-Zettel.
• alfred

  June 2023

  @Andy said:
  @alfred: I found a nice Pandoc cheat sheet by Keefer Rourke, "Typesetting with Pandoc Markdown: How I Take Notes" (2021)

  Thanks, I've have been looking into pandoc and have been able to get some reasonable results with the amsthm filter just making basic documents that have theorems etc as a proof of concept. In my brief spell testing pandoc, I'm not really convinced of

  the long-term maintainability of the more-lightweight Markdown [being] far superior to LaTeX

  when using Pandoc. The problem being that the use of certain filters would make the project rely on them being actively maintained. For example, pandoc-eqnos hasn't been updated for a number of years and so is not compatible with the current versions of pandoc. I'm not sure of the best way to go about introducing markdown support in a way that will be easy to maintain and for anyone who uses LaTeX-Zettel to use easily.

  Ideally I think a good design for the notes would be something like

  # Note Title
  
  ::: Theorem {#theorem:theorem label}
  Theorem ...
  :::
  
  ::: Proof.
  Proof of theorem
  :::
  
  Reference [#theorem:theorem label] in this document. Reference to [[another_document#theorem:another theorem]].
  

  And have this render into the LaTeX Zettel format

  \RequirePackage(...)
  \subimport{../template}{preamble.tex}
  
  \title{Note Title}
  
  \begin{document}
  \maketitle \currentdoc{note}
  \begin{theorem} \label{theorem:theorem label}
  Theorem ...
  \end{theorem}
  \begin{proof}
  Proof of theorem
  \end{proof}
  
  Reference \cref{theorem:theorem label} in this document. Reference to \excref[theorem:another theorem]{AnotherDocument}.
  \end{document}
  

  Hopefully this will be fairly straightforward to get working.
• alfred

  June 2023

  @Andy said:
  @alfred: I found a nice Pandoc cheat sheet by Keefer Rourke, "Typesetting with Pandoc Markdown: How I Take Notes" (2021)

  Thanks, I've have been looking into pandoc and have been able to get some reasonable results with the amsthm filter just making basic documents that have theorems etc as a proof of concept.

  I think a reasonable design for a LaTeX-Zetel markdown note design would be something like

  # Note Title
  
  ::: Theorem {#theorem:example theorem}
  Example theorem
  :::
  
  ::: Proof
  Proof of the theorem
  :::
  
  And references something like [#theorem:example theorem] and [[another_document#theoorem:other theorem]] for external references.
  

  and have this render into the LaTeX-Zettel note

  \RequirePackage{import}
  \subimport{../templace}{preamble.tex}
  
  \title{Note Title}
  
  \begin{document}
      \maketitle \currentdoc{note}
      <%note>
      \begin{theeorem} \label{theorem:example theorem}
          Example theorem
      \end{theorem}
      \begin{proof}
          Proof of the theorem.
      \end{proof}
      And references something like \cref{theorem:example theorem} and \excref{theorem:other theorem}{AnotherDocument} for external references.
      </note>
  \end{document}
  

  This should be easy enough to implement with pandoc. Ideally I'd like to use minimal dependencies.
• Andy

  June 2023

  @alfred said:

  Thanks, I've have been looking into pandoc and have been able to get some reasonable results with the amsthm filter just making basic documents that have theorems etc as a proof of concept.

  That pandoc-amsthm filter is interesting, and the code should be helpful for you.

  In your sample Markdown note design, unless I'm mistaken, it seems that the single square brackets in [#theorem:example theorem] were a typo? In any case, shouldn't you stick to standard [[WikiLink]] and Markdown link syntax? As mentioned above, you'll probably have to implement a preprocessing script to filter link syntax.

  You will also want to include in your sample Markdown note design some of the features that Keefer Rourke showed in his cheat sheet that I mentioned above. Especially citations, e.g. [@citekey]. Running pandoc with the --biblatex option should give the result you want.
• Andy

  June 2023

  Also note that Pandoc Markdown supports implicit reference links in the following form:

  Here's a reference to [Example Theorem][] and to [Other Theorem][] in another document.
  
  [Example Theorem]: #theorem:example-theorem
  [Other Theorem]: another_document#theorem:other-theorem
  

  This may be the most readable form for the kind of links you want to make.
• saf_dmitry

  June 2023

  Jon Sterling provides some interesting thoughts about atomicity and general structure of mathematical notes and general design of tools for mathematical thought:

  https://forest.jonmsterling.com/tfmt-0001.xml

  The website itself is built using his own tool Forester, which is an implementation example of his ideas.
• Andy

  June 2023 edited August 2023

  @saf_dmitry: Amazing, it's another LaTeX Zettelkasten?! But it's mostly powered by Ocaml instead of @alfred's Python. Here is the source for the "Designing tools for mathematical thought" page that you mentioned.

  Are those Markdown links that I see mixed with LaTeX in his source? Like the "evergreen notes" link here:

  \p{A “tool for mathematical thought” could be many things, but it must be a tool for the \strong{development} and \strong{interlinking} of mathematical ideas in a way that facilitates \strong{authoring}, \strong{publishing}, \strong{teaching}, \strong{learning}, and the maintenance of \strong{[evergreen notes](tfmt-0003)}.}
  

  ---

  Edited 2023-08-12 by @ctietze: Fixed links

  Post edited by ctietze on August 2023
• Andy

  June 2023

  I skimmed through Jon Sterling's "Designing tools for mathematical thought" (mentioned above), and I have some comments about its conceptual content:

  It is a nice summary of some of the major software-design issues for a Zettelkasten for mathematics. One of his recurring references is Andy Matuschak (not me), which is not surprising since Matuschak is a software designer.

  As a non-mathematician, I see one area for improvement in Sterling's exposition, although Sterling might consider these remarks to be out of scope for his project: I would emphasize that many of the issues that he addresses are not unique to mathematics but must be addressed by anyone who aspires to rigorous reasoning, where reasoning, in the words of psychologist David Moshman, "is epistemologically self-constrained thinking in which the application and coordination of inferences is guided by a metacognitive commitment to what are deemed to be justifiable inferential norms".¹

  In the pursuit of rigorous reasoning, fields such as argumentation theory, information science, knowledge organization, and some subfields of philosophy (e.g. conceptual engineering) have also developed relevant conceptual structures, partly in conversation with mathematicians.

  From the point of view of an intellectual generalist, it would be better that the node "Designing tools for mathematical thought" were a subnode of a more general node called something like "Designing tools for rigorous reasoning" so that issues in the design of a personal knowledge base for mathematics could be connected to issues in the design of a personal knowledge base in general, building on conceptual resources from any relevant fields such as those previously mentioned.

  ---

  1. David Moshman (2004). "From inference to reasoning: the construction of rationality". Thinking & Reasoning, 10(2), 221–239. https://digitalcommons.unl.edu/edpsychpapers/44 and https://doi.org/10.1080/13546780442000024 ;↩︎
• alfred

  July 2023

  @Andy said:
  That pandoc-amsthm filter is interesting, and the code should be helpful for you.

  Yeah it's good. I've been able to get the example mostly working using a custom pandoc template - by default pandoc doesn't at all optimize for readable LaTeX output.

  I still have a few issues for which I'm not sure if there are easy fixes. The main problem is that pandoc-amsthm always writes the \newtheorem definitions at the start of the text, even if you run pandoc without the --standalone flag which is a bit irritating. So the output looks like

  \Requirepackage{import}
  \subimport{..}
  
  \title{Note Title}
  
  \begin{document}
  \maketitle \currentdoc{note}
  
  \theoremstyle{definition}
  \newtheorem[Theorem]{theorem}{theorem}
  \newtheorem[Definition]{theorem}{theorem}
  ... etc...
  

  And I would prefer all of these to be in the latex header template and not in the rendered LaTeX notes. A rather pedantic issue also is that I'd like the markdown to be

  ::: Theorem
  Example theorem.
  :::
  

  and have the output be

  \begin{theorem}
  Example theorem.
  \end{theorem}
  

  but by default pandoc-asmthm uses \begin{Theorem} (with a capital T) and these have to match definitions given in the metadata. Not exactly the end of the world, but it would be better to have the option of a dictionary between the markdown classes and LaTeX theorem environments. Also I'm not a fan of the syntax

  ::: {.Theorem #theorem:label}
  

  For labeling the theorem, I would prefer

  ::: Theorem #theorem:label
  

  Going back to pandoc, I'm not sure if there is an easy way to get pandoc to render $$ maths $$ as

  \begin{equation*}
  maths
  \end{equation*}
  

  rather than

  \[
  maths
  \]
  

  Again, not a big deal, but I'd prefer the equation environments.

  My plan is to have a go at editing the pandoc-amsthm filter to add some extra configuration options.

  In your sample Markdown note design, unless I'm mistaken, it seems that the single square brackets in [#theorem:example theorem] were a typo? In any case, shouldn't you stick to standard [[WikiLink]] and Markdown link syntax? As mentioned above, you'll probably have to implement a preprocessing script to filter link syntax.

  I'm currently undecided about this - maybe its a bad idea to have two different markdown link styles. My thinking was to differentiate between external and internal links, although just not having the file name before the # would be sufficient so maybe that is better. Writing in latex I have the option of using my own \excref command and the default \cref, the former adds in a prefix with the name of the note being referenced while the other doesn't, and it might be nice to have this functionality in the markdown.

  You will also want to include in your sample Markdown note design some of the features that Keefer Rourke showed in his cheat sheet that I mentioned above. Especially citations, e.g. [@citekey]. Running pandoc with the --biblatex option should give the result you want.

  True, I forgot about this, I've been able to get the citations to work with no issues.> @Andy said:

  Also note that Pandoc Markdown supports implicit reference links in the following form:

  Here's a reference to [Example Theorem][] and to [Other Theorem][] in another document.
  
  [Example Theorem]: #theorem:example-theorem
  [Other Theorem]: another_document#theorem:other-theorem
  

  This may be the most readable form for the kind of links you want to make.

  This is quite nice, but I think I prefer the inline style links that don't depend on the definition later in the note. I do also have a plan to automate the addition of a backrefs and references section to the end of the PDFLaTeX output.
• alfred

  July 2023

  @saf_dmitry said:
  Jon Sterling provides some interesting thoughts about atomicity and general structure of mathematical notes and general design of tools for mathematical thought:

  https://forest.jonmsterling.com/tfmt-0001.xml

  The website itself is built using his own tool Forester, which is an implementation example of his ideas.
  

  Thanks - I think I saw this before I decided to start LaTeX-Zettel. I couldn't very easily follow the documentation so wasn't able to get much from the forester project. I just tried and failed to run the example given here, and I don't really know enough go to try and debug the problem.

  I think many of the points in tools for mathematical thought line up with how I'm currently using LaTeX-Zettel.
• Andy

  July 2023

  @alfred said:

  I still have a few issues for which I'm not sure if there are easy fixes. The main problem is that pandoc-amsthm always writes the \newtheorem definitions at the start of the text, even if you run pandoc without the --standalone flag which is a bit irritating.

  I haven't looked closely enough at pandoc-amsthm to give you a solution for the \newtheorem definitions placement problem, but I imagine that this problem (and the problem of the capitalization of theorem environment names) could be be fixed in your own filter.

  Also I'm not a fan of the syntax

  ::: {.Theorem #theorem:label}
  

  For labeling the theorem, I would prefer

  ::: Theorem #theorem:label
  

  The syntax that you don't like is Pandoc's fenced-div syntax. This is the syntax that Pandoc uses to create the Div element [ Div ( "theorem:label" , [ "Theorem" ] , [] ) [] in its abstract syntax tree (AST).

  It probably would be best to accept this syntax if you want to use Pandoc for Markdown support, because this syntax allows you or others to use Pandoc to easily convert the same Markdown source into other formats beside LaTeX.

  Going back to pandoc, I'm not sure if there is an easy way to get pandoc to render $$ maths $$ as

  \begin{equation*}
  maths
  \end{equation*}
  

  Yes, this can be done with a filter. For example, the pandoc-crossref filter does it.

  Pandoc's internal AST representation of $$ maths $$ is [ Para [ Math DisplayMath " maths " ] ].

  Applying the pandoc-crossref filter renders $$ maths $$ {#eq:maths-example} to LaTeX as \begin{equation}\protect\hypertarget{eq:maths-example}{}{ maths }\label{eq:maths-example}\end{equation}.

  The pandoc-crossref filter is just serving as a proof of concept here; it's written in Haskell. It wouldn't be difficult to write a filter in Python that does something similar. The pandoc-xnos suite is written in Python and does the same thing as pandoc-crossref, so the code of pandoc-xnos could be helpful for you. See also this section of the Pandoc docs.

  My thinking was to differentiate between external and internal links, although just not having the file name before the # would be sufficient so maybe that is better. Writing in latex I have the option of using my own \excref command and the default \cref, the former adds in a prefix with the name of the note being referenced while the other doesn't, and it might be nice to have this functionality in the markdown. [...]

  I think I prefer the inline style links that don't depend on the definition later in the note. I do also have a plan to automate the addition of a backrefs and references section to the end of the PDFLaTeX output.

  My mention of implicit reference links was a bit of a distraction, since both that style of link and an ordinary inline link get translated into a Link element in Pandoc's internal AST. Like the filter examples that I mentioned above, the best way to deal with links in Pandoc is to let Pandoc parse the varieties of link syntax that it supports into its AST, and apply a filter to get the output that you want.

  Both styles of Markdown link (e.g. echo "[Example Theorem](#theorem:example-theorem)" | pandoc -f markdown -t native) are translated into the same Pandoc AST:

  [ Para
      [ Link
          ( "" , [] , [] )
          [ Str "Example" , Space , Str "Theorem" ]
          ( "#theorem:example-theorem" , "" )
      ]
  ]
  

  However, WikiLinks (e.g. echo "[[#theorem:example-theorem|Example Theorem]]" | pandoc -f markdown+wikilinks_title_after_pipe -t native) are slightly different; they have an extra attribute in the AST:

  [ Para
      [ Link
          ( "" , [] , [] )
          [ Str "Example Theorem" ]
          ( "#theorem:example-theorem" , "wikilink" )
      ]
  ]
  

  Probably the best solution for most of the problems you need to solve will be to create your own Pandoc filter based on what you can learn from the pandoc-amsthm filter and pandoc-xnos filter, and put it in a new git repo called pandoc-latex-zettel or something like that.
• ZettelDistraction

  July 2023

  My advice is to turn off $(\LaTeX)$ macro expansion by Pandoc, and use defaults YAML, JSON (etc) bibliography files, and a lightly modified Pandoc latex template for the export formats that you want. Otherwise, you will have to roll your own Pandoc filter as @Andy suggests. That might be a too clever detour to a bridge too far by half.

  Here's a post I wrote to show what can go wrong if you enable $(\LaTeX)$ macro expansion in Pandoc.

  GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.
• Andy

  July 2023

  @ZettelDistraction said:

  and a lightly modified Pandoc latex template for the export formats that you want. Otherwise, you will have to roll your own Pandoc filter as @Andy suggests. That might be a too clever detour to a bridge too far by half.

  I'm confused by the phrase "for the export formats that you want" here. While @ZettelDistraction uses Pandoc to export from a primarily Markdown-based Zettelkasten (if I'm not mistaken), @alfred's use of Pandoc here is to import from Markdown files to a primarily LaTeX-based Zettelkasten. To translate from Markdown to the amsthm theorem environments that LaTeX-Zettel uses, I don't see a better alternative to rolling his own filter. Making a filter isn't very difficult—I've done it myself before, and I'm a low-skill programmer.

  I've been looking more at Jon Sterling's Zettelkasten—I mean, "forest of evergreen notes"—and he has a pretty interesting implementation of hierarchical structures that seems to be more sophisticated than what @alfred has done in LaTeX-Zettel so far. It reminds me to some degree of what the Breadcrumbs plugin does to support hierarchical relationships in Obsidian. @alfred may want to think about how to better support such structures in LaTeX-Zettel and in the Markdown input to LaTeX-Zettel.
• ZettelDistraction

  July 2023 edited July 2023

  @Andy said:
  @ZettelDistraction said:

  and a lightly modified Pandoc latex template for the export formats that you want. Otherwise, you will have to roll your own Pandoc filter, as @Andy suggests.

  I'm confused by the phrase "for the export formats that you want" here. While @ZettelDistraction uses Pandoc to export from a primarily Markdown-based Zettelkasten (if I'm not mistaken), @alfred's use of Pandoc here is to import from Markdown files to a primarily LaTeX-based Zettelkasten.

  Pandoc doesn't recognize its italicized role as an importer or an exporter. You mean that the configuration files are different for the different applications. I can think of two export formats that could go to a LaTeX-based Zettelkasten--the same two formats I generally use: LaTeX source and PDFLaTeX. This is what I do and what Zettlr does (up to renaming directories).

  For LaTeX output:

  pandoc --defaults=export.latex.yaml --bibliography media\MyLibrary.json Hom20221026.md -o Hom20221026.tex

  For PDFLaTeX:

  pandoc --defaults=export.pdf.yaml --bibliography media\MyLibrary.json Hom20221026.md -o Hom20221026.pdf

  The Pandoc LaTeX template, template.tex is lightly modified. The modification is the addition of a Pandoc YAML variable named more-header-includes: to include any $(\LaTeX)$ that I wanted in Markdown files. The built-in YAML variable header-includes is already used in the YAML defaults files--I added links to my GitHub versions of these files in the command line examples above.

  My approach has been to use LaTeX within Markdown and turn off latex_macros. If I wanted to output PDFlatex in which wikilinks resolve to filenames, I would need to use xr-hyper (for example), which would add a preprocessing step. You suggest augmented Markdown syntax that Pandoc would transform to LaTeX. Well, OK--now I understand.

  Apologies--I'm somewhat behind in this thread and Pandoc programming. I will have to ask my tutor, ChatGPT4, for help. I intended this as an admittedly weak example.

  Post edited by ZettelDistraction on July 2023

  GitHub. Erdős #2. CC BY-SA 4.0. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein.
• Andy

  July 2023

  @ZettelDistraction said:

  Pandoc doesn't recognize its italicized role as an importer or an exporter.

  True, I was thinking the same thing but stated it poorly. All that is happening is conversion/translation of some kind: input → AST → output. Or adding a filter: input → AST → AST → output. I recommended to @alfred to avoid inventing his own custom syntax because that would probably add another layer of conversion (unless he wrote his own Pandoc reader, which would be too much work): custom syntax → input → AST → AST → output. But if he hated the available syntax enough, he might choose to do that.
• jonsterling

  August 2023

  Hi all, I just wanted to thank you for your interested and feedback about my work on Forester and related topics. The link to my document "Designing tools for mathematical thought" is now broken, as I have moved my forest to the www subdomain of my website here: https://www.jonmsterling.com/tfmt-0001.xml.

  The software can be a little hard to set up for beginners at the moment because I haven't written much documentation, and it is constantly changing. But if anyone is very interested, I am happy to provide assistance getting started; you can post to my mailing list here: https://lists.sr.ht/~jonsterling/forester-discuss.
• Andy

  August 2023

  @jonsterling said:

  Hi all, I just wanted to thank you for your interested and feedback about my work on Forester and related topics.

  Jon, your Forester project and the way you modeled it after the Stacks Project (which I wasn't familiar with since I'm not a mathematician) has provided me a ton of food for thought about alternative ways of structuring a personal knowledge base. So thank YOU for that.

  Can't you set up an HTTP 301 redirect from your subdomain forest to www to avoid the broken links? That's really important. If you can't do that with GitHub pages, you should switch your forest subdomain to a web host that supports 301 redirects.
• alfred

  October 2023

  Some updates to the project.

  First of all I renamed the project to texnotes, just so that the name means more to people who aren't familiar with the zettelkasten method.

  I've made quite a lot of progress and changes to the project so I'll list here the most significant ones. I also made a second youtube video about the project and the new features. The introduction shows me importing notes from obsidian and rendering these with texnotes.

  1. I've been able to get pandoc working to convert between markdown notes and texnotes latex documents. I basically did what @ZettelDistraction suggested. The system works by preprocessing the markdown to convert wikilinks style links to my latex referencing commands and then for the main conversion I wrote a lua filter. This allows you to define theorem environments using the markdown syntax of level three headings so ### Theorem (Theorem Name) {#theorem-label} will be converted to an amsthm theorem environment. This behavior can be configured in the document metadata. The proceeding paragraph then becomes the theorem body. I decided to denote proof environments in markdown by writing _Proof._ (content of proof) _Q.E.D_, with _Q.E.D_ being included because proofs will probably be more than one paragraph long in general.

  2. I added some python code that adds a section to the bottom of each note with links to the notes that reference it.

  3. I fixed html rendering with make4ht, so it is possible to make a static html site with nicely rendered latex (including basically any LaTeX package like tikz for example) with all the latex clever and hyper referencing, including clever referencing between pages.

  4. I added a transclude feature, as in @jonsterling 's Forester, where you can include part or all of another texnote in a note, and I improved the way that notes can be exported to create paper drafts. This involved making a new sort of note called a project, which is intended to transclude a load of zettels. These project files can be rendered just using pdflatex. Texnotes provides a function to export these as standalone latex documents where all the latex code is copied into the notes.

  5. I've added some functions that handle rendering the notes an running bibber automatically. Texnotes now tracks labels, links and citations in the files and so can work out which notes it needs to recompile when some notes are changed. For example, if a new label is added then you need to compile both the file with the label in as well as any notes that reference the label to get working hyperlinks. Texnotes now handles all of this.

  Thanks @Andy, @ZettelDistraction for your comments above and @jonsterling for some inspiration on extra features to add to the project.
• ctietze

  October 2023

  @alfred Do you have anything visual to share how things look nowadays?

  Author at Zettelkasten.de • https://christiantietze.de/
• alfred

  October 2023 edited October 2023

  @ctietze The overall look hasn't changed much, but I can show what the markdown notes look like, and how they look when they are rendered.

  For example, I've made two notes called example_note and example_references. The workflow for doing this is to run ./manage.py newnote_md example_note and then ./manage.py newnote_md example_references from the base directory. The idea is that you can setup a text editor (eg visual studio code or vim) to execute this for you.

  To show some of the features I added this to example note:

  # Example Note 
  
  This is a texnote written using markdown. We can use theorem environments:
  
  ### Theorem
  This is an example theorem. 
  
  At the moment, theorem environments only capture the first paragraph. I think this should be fine for most applications, but it might be nice to somehow allow multi paragraph theorems.
  
  The available theorem environments are given in the metadata file `pandoc/defaults.yaml`. If the first word of the heading matches an environment defined in the metadata, the filter replaces level three headings with the appropriate \LaTeX code. We can also add a label to theorems using standard markdown links.
  
  ### Theorem {#theorem-label}
  This theorem has a label.
  
  _Proof._
  Proof environments can be indicated using empf (by starting the proof with `_Proof._` and ending with `_Q.E.D._`.
  
  
  These work over multiple paragraphs.
  _Q.E.D._
  
  Theorems can be referenced using clever references: [[#theorem-label]].
  
  ### Definition (My Definition)
  We can also add names to the theorem environments using brackets after the theorem name
  
  

  and added this to example_references.

  # Example References
  
  This is an example of referencing with markdown. We can reference using wikilinks style links. For example, creates a reference [[example]] to the previous note we just made. We can add clever references to theorems, e.g. [[example#theorem-label]] or add a [[example#theorem-label|hyperlink to the theorem]].
  
  
  Citations are managed using biber (through the pandoc `--bibtex` flag). E.g. [@ahrensHowTakeSmart2022].
  

  These then get converted to LaTeX, and when they're rendered with pdflatex (throught texnotes) they give

  

  and