MWE for using Zettels in a TeX environment. Please contribute!

Sociopoetic

A few different conversations here have me thinking about how I might use ZK notes (as .md files) in my longer papers. While this isn't a perfect use, for the (La)TeX users here, this should offer a start.

First, I created a folder in the TeX project called "zettels" and uploaded the some example zettels. I did this so that I was using a duplicate of a zettel, not my original. Note that for now I am using Overleaf.

To each uploaded zettel, I had to add \begin{markdown} two lines above the beginning of the note (i.e., it was on line 1, then an empty line followed). Two lines after the end of the zettel, I added \end{markdown}, like so:

\begin{markdown}

# Header

- List
- List
- List
- *emphasized*

## A subheader



  This is block text


### A nested subheader

A rule:

---


\end{markdown}

Then the TeX packages markdown and import did the rest:

\documentclass{article}
\usepackage[utf8]{inputenc}

\usepackage{import}
\usepackage[hybrid=false]{markdown}

\title{Editing Zettels into a \LaTeX\ paper}
\author{SocioPoetic}
\date{December 2020}

\begin{document}
\import{zettels}{zettel.md} % this indicates {folder}{file}
\import{zettels}{another-zettel.md}

\end{document}

A few known issues & comments:

• The only way this worked was to edit the Zettel in order for TeX to read it as markdown. Perhaps there is a better solution here?
• I had to be meticulous with the Markdown text in order for it to render properly in the Zettel. It might honestly be simpler to copy/paste the Zettel into my .tex document, but that defeats the purpose of figuring this out.
• There are many different settings for the markdown package, so I still need to figure out how to handle references, etc. The package documentation indicates that markdown inline images can convert to TeX figures, but they don't have as many nice features (in my discipline, references and cross-references to figures is very important). Multimarkdown is not yet supported, so there is no table support.

I also share all of this as someone who has not yet quite mastered pandoc; using Overleaf has allowed me to focus on learning the TeX needed for my academic discipline, but at the expense of being not as independent in my workflow.

Hopefully this is helpful for others, at least as a starting point. Any input and improvements are welcome! I begin writing my MS thesis in about 5 months...

ctietze

I didn't know you could import Markdown in LaTeX using a package! Neato!

I can see how this would be useful for maybe an essay or a letter of sorts. I don't think that verbatim imports of un-edited note files is what you will end up with, though. Concatenating 10 note files doesn't automatically make a text; it still needs filler, edits, striking out repetitions, etc etc -- so a manuscript that is more than and different from the notes is required down the road.

Lemme explain a bit. It might save you some time and trouble

When we assembled books from Markdown notes in the past, we eventually went from many Markdown notes to 1 Markdown manuscript file and converted that to 1 LaTeX file (some search & replace was necessary here to fix some conversion issues, and table and image inclusion looked odd, so I had to edit that). Since the output is not print-ready, yet, and needs to be proof-read and edited, the LaTeX file was split up into multiple files to stay manageabe and they all were included in the master file where the frontmatter lived.

Markdown has so many weird edge cases that actually are features and not bugs, I bet the LaTeX import thingie just doesn't do a good job at keeping up a high % of compatibility and that's where your issues stem from. Trying out pandoc until it converts from Markdown to TeX is worth the half hour of dabbling around because pandoc does a predictably good job converting. It's not always 100% what we expected, but it worked.

Downside: you end up with ~3 versions of your text. One is in the form of your original Markdown notes; changing these later in the process will not propagate downstream to LaTeX automatically; one is the Markdown manuscript file you can use for printing to proof-read, but changes there also don't propagate (so you're probably better off just trashing this file once you have the LaTeX version); and lastly, one version is the LaTeX manuscript.

To propagate changes in your Markdown notes to LaTeX after you assembled a draft, you might want to include comments that include the original note's ID.

% Based on: 202012030819 Banana bread recipe
While raspberries are tasty, \textbf{bananas} are superior in every way. ...

That works in both directions:

1. When you see a pice of text that needs changing or fact-checking, you can look up the "source" note;
2. when you change a note, you can look up where it was used in the manuscript.

Sociopoetic

@ctietze: Wow, thank you! You did an excellent job explaining this. I am a little more inspired now to learn pandoc, too (which as an aside, may be necessary for professors who demand .doc files rather than PDF). Including the UIDs as comments is probably the Occam's Razor sort of answer, too. Really, it just seemed tedious to copy/paste notes from The Archive to LaTeX, but maybe there's no shortcut there. Tracking the notes with the UID at least makes the process more organized.

On the other hand, I may keep the markdown TeX package as a part of my default preamble for manuscripts just to stay focused on writing.

Regarding the proofreading of texts, one feature I like in Overleaf is the preview pane similar to Marked (and there's also a rich text editor if preferred, but I'm not as fond).

Maybe you can offer some insight here to my underlying issue: My final reports (with diagrams, maps, tables, footnotes (as comments, not citations) and lengthy reference lists all benefit handsomely from TeX. Much of my content in those reports (paraphrasing of articles, explanations of concepts, etc.) is the my ZK. But getting from the ZK markdown files to my manuscript TeX files often is where most of my time is spent. Knowing that my ZK is always where I start and TeX always where I end, I'm trying to make the journey between as fruitful as possible.

ctietze

So this sounds like you want to reduce friction in the process.

For initial document assembly, I copy and paste the notes from my ZK in one large file and then convert that to LaTeX, e.g. using pandoc.

Then I fix the LaTeX, which means I cannot just re-generate the TeX output again. That would overwrite my changes. I adjust my process after the initial LaTeX assembly.

To reduce friction of amendments and edits (or what programmers might call "patches"), converting a MD paragraph to TeX has to be easier than the initial mass conversion via the pandoc toolchain.

On Mac, you can customize the right-click menu (contextual menu) by adding "Services" that operate on text selection. I use one that converts MMD to HTML all the time to e.g. assemble release notes HTML from the Markdown running list of changes I make. You could do that for TeX as well and whip up a service where you can select text in your ZK, right click, select "Services > Convert MMD to LaTeX", and then get the result in your clipboard automatically for easy pasting into your LaTeX document/draft.

The MMD-to-HTML thingie is from Brett Terpstra's tools:
https://brettterpstra.com/projects/markdown-service-tools/

Mac users can pry his MDD-to-HTML service open by using Automator.app to open the service, and then try to use pandoc or mmd2tex instead of the mmd2html of Brett's service.

Sociopoetic

Hi friends,

I learned a new trick today, thanks to the support team at Overleaf. If you are using the markdown package for LaTeX, you can also have a portion of your document like this:

\markdownBegin
*your Markdown Zettel here*
\markdownEnd

This would reduce some of the reformatting, but allow for editing of the markdown texts into a coherent paper, as @ctietze was saying. If you do want to directly input the Zettel as-is, there's the command \markdownInput instead, that eliminates the need to add any TeX formatting to the Zettel copy in the project (like I did in my original post).