Lurker ready to say Hello and ask about referencing

James27

Hi Everyone,

First, a thank you to the creators of this forum! I've learned a lot over the past few weeks here. I am a writer and prospective PhD student (literature) and stumbled upon the idea of the ZK through DEVONThink. I found my way here via a series of links (I feel there's a joke here somewhere). For the past while I've been researching trying to determine the best tools to use (Mac) and, since I am a long-time Ulysses user, have settled on Bookends for reference management and The Archive (which is awesome) for my ZK.

My ideal workflow: Bookends pdf annotations on iOS - TA buffer + zettel - draft in Ulysses - final edit in Word + implement bibliography and formatted in-text citations. (I'm trying to avoid what I see as the unnecessary step of learning LaTeX tools only to write prose. Comments on this proposed workflow welcome.)

I haven't yet figured out a clear way to get these all to talk to each other, but I'm slowly piecing things together.

One thing I'm struggling with is figuring out the workflow between Bookends and the ZK. Right now I'm unclear on the necessity / usefulness of the [#authorDate] / [#authorDateDescriptor] format as used by @ctietze . As I understand, the desirability of such a # format is so that the citation becomes its own searchable key in the ZK (?). Bookends auto-outputs temporary citations in the format {author, date, #uniqueCode}. It is similar to the [#authorDate] style in that the code becomes a tag, but the tag itself, like a UID, is rather meaningless. Is this a drawback worth working around? Is it advisable to (somehow) set up Bookends to output a citation to TA in the format [#authorDate]? One reason why a manual workaround wouldn't work is that Bookends needs (many steps later, when these temporary citations finally appear in Word) to be able to 'Scan' the document and produce the bibliography and having an unrecognised temporary citation won't work for that. Bookends would need to be set up to correctly generate the temporary citation in the format [#authorDate] to later recognise it during its scan in Word Based on some posts by @John , I'm aware that there are workarounds if you have the paid version of Alfred (which I don't). I'm curious whether the citation style mentioned is favoured for the reasons I've outlined above or if I've missed something...

I see my introductory post has become rather long. My apologies and thanks in advance.

Darryl

Welcome @James27! I too use Bookends + TA in my workflow. I currently have Bookends setup to use the format {#authorDateDescriptor}. To do so I just checked the "Enable BibTeX for references" option in the system pref pane in Bookends under the "BibText" tab (also set to "Warn" if the field is not unique). This enables the Key field which I manually populate each time I add a reference. I chose use that format rather than the random generated ID from Bookends just in case development of the app ever ceases.

That said, just as an aside, although I love Bookends and the developer is very responsive, I am in the process of wrapping my head around Bibdesk (trying to implement CMS17 and other custom entries) to utilize a plain text approach (as is advocated by our forum hosts). Thinking about a couple of decades worth of upcoming research and writing has prompted me to consider a safer bet with the tools I choose today.

Hope that helps!

Will

Hi, @James27, I'm just a beginner and I have similar questions. I'm an aspiring non-fiction writer so I feel my needs span from no need for a bib reference to full-on academic situations.

@James27 said:
I'm trying to avoid what I see as the unnecessary step of learning LaTeX tools only to write prose.

My understanding, which is far from perfect, the tool pandoc which can take a bib file and with it convert all you textual cite keys to a full reference section. Not many "steps". Beautiful and standardized. Not full-on LaTex, a bit less fiddly. Most journal publishers and most dissertations have a required reference format. If you never plan on publishing then maybe this is mote.

@James27 said:
I'm unclear on the necessity / usefulness of the [#authorDate] / [#authorDateDescriptor] format as used by @ctietze . As I understand, the desirability of such a # format is so that the citation becomes its own searchable key in the ZK (?).

My understanding, again less than perfect, is that using the cite key is the way text-based apps handle bib references. By using a cite key in reference to a bib file managed my a bib file manager, your zettels are future proof and this prevents falling into vendor lock-in. One thing I use the cite key for is as a substitute for a tag to search for and bring all the zettels with a common cite key into the note list.

Maybe I've made such a mess of my explanation that we'll both be saved by someone here with real experience with publishing, pandoc, or LaTex.

ctietze

Welcome on board, @James27 !

Here's some background on the citation stuff:

[#somethingsomething] is the MultiMarkdown way to provide a bibliography reference using a bibliography citekey -- that's the somethingsomething; the # is just do distinguish this kind of reference from regular links and footnotes (footnotes use ^, as in [^footnote]).

So [p 123][#kant1793] of [#kant1793][p 123] is an actual citation. [#kant1793][]/[][#kant1793] is a whole-book reference. You need both pairs of brackets to make a citation, but the order is irrelevant.

MultiMarkdown suggests you plug in bibliographies via BibTeX, a plain text format for which you can easily get compatible reference managers. But with the advent of pandoc, as @Will mentioned, options have exploded a bit in the plain text academic writing market Back in the day, MultiMarkdown was the only variant that supported citations (and which I knew). You could adapt all of this to your needs if you are positive that Bookends is the best thing for the rest of your life (or ZK project span)

James27

Thank you all for your warm messages of welcome and very helpful replies!

I've spent most of today trying to fill in the rather glaring holes in my knowledge. Here's the path your hints sent me down:

1. MultiMarkdown is rather different from vanilla markdown (a distinction whose importance I hadn't appreciated until @ctietze pointed it out - so thank you).
2. Compiling zettel from TA into Ulysses is perhaps not the smartest workflow, since Ulysses doesn't appear to support MultiMarkdown and so (I suppose) won't play as nicely with TA as a mmd-supporting editor will (like iA). Plaintext approach rules.
3. pandoc is insanely powerful and awesome.
4. Bookends is not necessarily the best choice. BibDesk looks very simple but offers most of what I need (which, being a literature student, isn't very much).

I've downloaded and installed BibDesk and made test entries, found and installed the latest mmd package, MacTex, and pandoc, and have played around exporting various formats from the Terminal. Some questions remain, though, particularly around the workflow you kindly outlined @Will . How does one get the .txt file containing the correct temporary citations in the mmd format to a pdf in which those citations have been replaced with the correct e.g. MLA / CMS17 citations and bibliography? Do you use TexShop to bring the .bib file .tex file together or this operation also done within pandoc? And where would one set up the citation style rules?

Sorry for all the questions. I've been trawling documentation about mmd and pandoc trying to educate myself but with my very limited knowledge of all these tools (even the command line is new to me!) I am still coming up short.

Once again, a huge thank you in advance for your help. Also, please let me know if I should move my questions out of this thread. I realise this is supposed to be for introductions

Will

@James27 said:
Some questions remain, though, particularly around the workflow you kindly outlined @Will . How does one get the .txt file containing the correct temporary citations in the mmd format to a pdf in which those citations have been replaced with the correct e.g. MLA / CMS17 citations and bibliography? Do you use TexShop to bring the .bib file .tex file together or this operation also done within pandoc? And where would one set up the citation style rules?

I'm still coming to grips with this workflow my self. This is what I've learned so far. Check out https://pandoc.org/MANUAL.html#citation-rendering. If you use BibDesk, it keeps all your citations in a .bib file and you want to call it when doing a conversion from plaintext to PDF.

The following command calls pandoc and tells it to use the biblio.bib file, BibDesk's DB, filtering it through a program called pandoc-citeproc looking at markdown citations then format them using a 'csl' template called "chicago-fullnote-bibliography.csl" using your input file and writing the output to output.pdf. It might look something like, pandoc -s --bibliography biblio.bib --filter pandoc-citeproc --csl chicago-fullnote-bibliography.csl input.md --pdf-engine=xelatex -o output.pdf
This is taken from https://pandoc.org/demos.html.

The citation style rules and in the .csl template. Available on the web. A long-winded explanation. I have not yet tried this. I think converting to PDF has its own quirks. I've mostly been converting to .docx and .epub.

ctietze

I haven't gone "full pandoc" to PDF, yet. Instead, I convert MultiMarkdown manuscripts to PDF via LaTeX. It's a markup language for accurate typesetting. There's a MultiMarkdown-to-LaTeX converter mmd2tex, and pandoc can do this, too.

The difference to @Will's approach, I believe, is that with the LaTeX code, my citations are transformed from [#foobar123][chapter 12] MMD to \cite[chapter 12]{foobar123} in LaTeX, and that's it. I use my old LaTeX templates to configure documents to look the way I want, be it MLA, APA or whatever-AuthorYear-style. The .bib file from BibDesk then serves as a bibliography input file to the LaTeX-to-PDF converter: it provides all available data, and the LaTeX-to-PDF pipeline takes whatever it needs for the style I want. That's where the .bib BibTeX format originated, by the way.

The pure LaTeX/BibTeX approach requires you lean yet another toolchain, and it's probably not going to be for everybody. I can just say that it works fine once you figure out how to debug odd-looking lines and the like and save templates for papers, letters, and books for later reuse.

Sönke

Hey @James27,
I am truly not very well versed in these things but I recently adopted a similar workflow for my university work since I hate writing in Word. Maybe my meager understanding can be of some help...

I write Markdown text in Atom and use Zotero for the citations. Since I needed a special citations style I created a .csl style. I use the BetterBibTex extension in Zotero for the cite-key (#authorYear) that I use in my writing. Then I use pandoc to convert my .md file to a word (.docx) file to make some style changes in the end (exact page numbers etc.) I tried LaTeX but didn't feel entirely comfortable with it. When I am done with all the changes I export my .docx file to PDF.

Over all I am very happy with my workflow at the moment. It took some time to implement but my writing and editing has been much more enjoyable since I switched away from Microsoft Word. I like having Word as my end point though, as I sometimes need the .docx file in university.

For Zettels in The Archive I use the same cite-key from the BetterBibTex extension in Zotero. I have not adopted MMD and I don't need it at the moment (I think). So when I am using Zettels in my work in Markdown I only need to change the # to @ and I am good to go.

This is a screenshot of my working document right now. @heusinger2009 is the cite-key and [ ]automatically makes a footnote.

This is a screenshot of one of my Zettel. I use the same cite-key with a # instead of an @ so the citekey #schwinekoper1967becomes searchable in The Archive.

The Pandoc command I run: zunft.bib.is the bibliography file I exported from Zotero (BetterBibTex keeps it updated if I make changes or add another Book). .cslis the "style-file". vorlage.docxis a reference file I created with the styles I want (12pt. 1,5 spacing etc.)
pandoc --bibliography=zunft.bib --csl=geschichte.csl --reference-doc vorlage.docx -o test.docx test.md

This is how the paragraph looks in word after the pandoc-magic:

This setup gives me the flexibility I need and the security I crave to get my work done efficiently. Hope this mildly helpful to someone out there...

James27

Thank you all for the useful feedback!

I spent yesterday getting up to speed with the command line.

@Will Thanks for your tips. I fiddled with the commands you supplied but discovered that in order for my input.md to be parsed correctly and turned into citation + bibliography in the pdf, I had to change the citation style from mmd style [#citekey] to [@citekey]. Am I missing something here? Must one first convert from mmd to some other md format before running this command on the source? (I have multimarkdown 6 installed, if that means anything.)

@ctietze Your comment intrigued me, so I've begun reading Knuth's TeXBook, which is both hilarious and informative. I doubt I'll need one hundredth of the knowledge in it, but it is interesting nonetheless. One question I have is around the mmd2tex command you mentioned. I note it produces a .tex file that looks completely different from the one pandoc produces. It also fails to typeset properly in TeXShop, which is strange. That said, since I've only just begun Knuth's book, I really have very little idea of what I'm looking at. Do you know what might be happening?

@Sönke Thank you for your very helpful little tutorial - I tried it out and it works like a charm! The only drawback, as you point out, is that you must somehow transform all your citekeys stored in zettel in TA into the format [@citekey] but I suppose it is easy enough to find and replace all instances of [# with [@ in your writing program before exporting to Word.

Regarding templates—I've fiddled around a bit and persuaded Word to adjust the input text to the desired destination style in most cases. One situation where this doesn't work for me is when trying to force paragraph indentation. Even when I supply an indented style, the indentation will not render. Do you know whether this can be done? I've googled for info and come up short.

Sönke

Hey @James27
I am really sorry... I just saw your question.

Unfortunately I had the same issue with some word-style-components in my reference document. Somehow some style changes don't get registered. I am sorry that I could not be of much help.

avggeek

@James27 welcome to the pandoc (ab)users club! I've been wrangling it to help change my ZK notes into something that can be published into a blog and I've truly come to understand why pandoc is the Swiss army knife of document manipulation - since it has a lot of sharp edges you can cut yourself on!

I fiddled with the commands you supplied but discovered that in order for my input.md to be parsed correctly and turned into citation + bibliography in the pdf, I had to change the citation style from mmd style [#citekey] to [@citekey]. Am I missing something here?

There is mention of this in the Pandoc manual but it can be hard to find. I will extract the relevant bits here for your reference:

Citations go inside square brackets and are separated by semicolons. Each citation must have a key, composed of ‘@’ + the citation identifier from the database, and may optionally have a prefix, a locator, and a suffix.