0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[0.2d.0.22.0407]] More header includes and Asterism
[[0.2e.0.22.0407]] Mike Shulman's twocell code
[[5.4.1.1.0.22.0410]] Replacing macros with styles
The macro environment was adapted from the definition of +bmatrix in tabularray.sty This is my first ever modification of a macro from a LaTeX package.
NOTE: The LaTeX package expl3 and the \ExplSyntaxOn and \ExplSyntaxOff commands cannot be eliminated! Wilfing through tex.stackexchange.com and related LaTeX sites eventually led to expl3 and environ (not to mention libraries I ended up not needing, such as xparse). TikZ styles don't seem to work with the Pandoc command line setting -latex_macros [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.
I learned something. I hate when that happens.™
What if we omit the optional argument? Is there an error? No, but perhaps the code should check.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Oh well. I'll subject it to severe criticism myself then:
The lines that were modified weren't called out!
Aside from the \RenewEnviron, which replaces original \NewEnviron in tabularray.sty, the modified line was rowsep={1pt},row{1}={#1},row{2}={#1},
Didn't say what prompted this detour through LaTeXology!
The application was to color entries in a matrix, for a math project that is under wraps for the moment. Coloring rows and columns of matrices in LaTeX documents can be done in several ways. However, two methods I used (setting a style with tikzset and calling a macro within optional square bracket arguments in a LaTeX environment) stopped working after I modified the Pandoc reader configuration in the Zettlr configuration files export.pdf.yaml and export.latex.yaml. The modification was to prevent Pandoc from expanding LaTeX macros while converting from markdown to latex. What you ordinarily want is for LaTeX to expand its macros, not for Pandoc to preprocess these. The setting to stop Pandoc from expanding macros before LaTeX gets the opportunity is reader: markdown-latex_macros. However, that meant that macro expansion within square bracket arguments never occurred during LaTeX processing. This is a general problem that people tend to encounter. I had a macro that set a few row and column colors that no longer expanded when called from the optional argument in +bmatrix, for example: begin{+bmatrix}[\myspec{lightskyblue}]. I set out to understand the more general problem and solve it for this case.
Are you sure about using tikzset to set styles with tikz graphics?
No, but uh, I wanted to get Mike Shulman's twocell code to work, and for that I needed to turn off Pandoc LaTeX expansion anyway.
What's stopping you?
From going down that rabbit hole? I'm not using tikzpicture so there is no local environment in which to set styles, and I'm not sure that I want to set global styles with tikzset. How do you know this issue is germane enough to warrant your hectoring! Give me a break!
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[0.2d.0.22.0407]] More header includes and Asterism
[[0.2e.0.22.0407]] Mike Shulman's twocell code
[[5.4.1.1.0.22.0410]] Replacing macros with styles
Not being able to look at these, I don't know why you list them. Is this a replacement for the "twocell code"? Is it using that? You call the target by name, but don't explain what the relation is.
The macro environment was adapted from the definition of +bmatrix in tabularray.sty This is my first ever modification of a macro from a LaTeX package.
That's a good summary of the "what". I'd be missing an intro paragraph about the "why". Like: `I tried to display tables with 20 short columns with package X but they looked ugly. Here's how I adapted a better-looking package Y to behave like X." -- Or whatever actually happens
Code is code, that's good enough. Maybe you can inline the "NOTE"s below as warnings into the code if you believe you need that warning when you want to use the code:
\ExplSyntaxOn % This cannot be eliminated!
I'd argue that if you take for granted that the code in your notes is as small as possible, then this wouldn't be necessary. But:
NOTE: The LaTeX package expl3 and the \ExplSyntaxOn and \ExplSyntaxOff commands cannot be eliminated!
I would personally think that whatever the code is listing is necessary. (Why else include it?) So what is the note/warning here intending to tell you?
Wilfing through tex.stackexchange.com and related LaTeX sites eventually led to ...
(I'd scrap that as noise; in a blog post I likely would add this myself, but not in a Zettel)
... expl3 and environ (not to mention libraries I ended up not needing, such as xparse).
Add links to the 2 packages you used? Document why xparse was a candidate but then not needed, if you mention it at all? (You likely also didn't need 99.99% of the rest of the libs that exist on CTAN, but you did decide to call xparse by name, why?)
TikZ styles don't seem to work with the Pandoc command line setting -latex_macros [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.
This sounds quite important. Put it onto its own line so it stands out from the backstory as an actionable warning? Is it an important deviation from your regular workflow that it might be better off before the code?
To me, it's not clear if the link is about the warning before the [[...]], or if it's just about Customizing Pandoc LaTeX in Zettlr. I adhere to the same conventions as @Sascha explains in https://zettelkasten.de/posts/embrace-dark-folgezettel-power/, so I would expect the link at this point to be about Pandoc command line setting \-latex_macros``, i.e. a note about the pandoc setting. Which sounds like a useful reference note to have. Otherwise I'd write this as:
TikZ styles don't seem to work with the Pandoc command line setting `-latex_macros`. I use this when customizing Pandoc LaTeX in Zettlr [[0.2c.0.21.0617.1059]].
I learned something. I hate when that happens.™
Again, blog post or forum material, but not Zettel
What if we omit the optional argument? Is there an error? No, but perhaps the code should check.
Is this a TODO for you? As a note, I'd document this in the code, actually; not sure about TeX conventions, but maybe like this:
...
\ExplSyntaxOn
%% Omitting the optional argument does not error-out but is undefined behavior.
\RenewEnviron{ +bmatrix }[1][]{
\left[\begin{+array}[expand = \BODY]{
...
The "NOTES" paragraph at the bottom, I'd consider splitting up into a list for easier parsing, and put it at the top to explain dependencies, so the code copypasta comes last:
TikZ styles don't seem to work with the Pandoc command line setting `-latex_macros`[[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.
Required libraries:
- `expl3` for [INSERT REASON] [INSERT LINK]
- `environ` for [INSERT REASON] [INSERT LINK]
I also tried `xparse` for [INSERT REASON] but didn't need it because [???].
<CODE HERE>
@ctietze Thank for your extensive comments. I'm going to follow some of your suggestions. Perhaps I could take "feeting" notes. I tend to work interactively. That could be my downfall.
My notes aren't self-contained. Their function is to jog my memory. Also, my attitude to code differs from yours. Here I was performing surgery on a macro. Whatever understanding I have is local, situational. My attitude is that code is not code, that is, there is no reason to expect that everything present is necessary and sufficient. I came to this not knowing what was needed. I tried several approaches over three days. I happened to be playing with expl3, xparse, environ and several other things in a scattered effort to grasp at what might work. My reason for trying them? I felt like lt. They seemed to touch on possibly relevant aspects of macro evaluation in TeX and LaTeX, a system that has led TeX and LaTeX package designers to invent alternatives.
Perhaps this haphazard approach isn't methodical or efficient. I don't expect to be. I rely on unconscious processes beyond my control, on blindsight. It helps to cultivate blindsight. I don't insist on having reasons. The 2500 year history of normative philosophy is the history of disagreement among philosophers over what constitutes a reason. We are discussing programs, though, and these are supposed to have preconditions that can be stated, computational complexity notwithstanding. I grant you that. As for what led me to try one thing or another? I don't know. I was driven to try them.
Why not mention the things that didn't work, or that I tried or thought about or tried to think about before some clarity emerged from the mind shrouded in tenebrous vapors? Nothing that I had read contained the solution to this problem. There were hints about possible approaches here and there.
There are two kinds of programmers in the world: those who think code is there for a reason and should be enough, and those who don't think "should“ is a term of computer science. OK the other group includes Literate Programming, after D. E. Knuth. Avoid both of them. I'm in the second camp.
It was sheer luck that I happened to have enclosed the ‘\RenewEnviron’ macro in ‘\ExplSyntaxOn’ and ‘\ExplSyntaxOff’. The renewed macro definition worked with them present, and refused to compile without them. Perhaps expl3 is a prerequisite in tabularray.tex. I didn't see it in tabularray.sty, which contains the original macro definition. In any case, I had been attempting to understand expl3 previously, when I decided that I needed to rewrite the macro whose behavior changed once I prevented Pandoc from expanding LaTeX macros while converting from markdown to LaTeX. In retrospect that would have been the correct initial approach. But then I would have had to have known that ExplSyntaxOn and ExplSyntaxOff were somehow needed. Unless this is an inadvertently cached dependency left behind in some auxilliary intermediate compilation file after some long abandoned experiment. The approach is operational.
The macro has one optional argument, indicated by the presence of square brackets. LaTeX won't evaluate macros in square bracketed arguments to other macros, but it will recognize constants. This is a common problem if you need to set a series of comma-separated key-value pairs within optional arguments. I didn't want to have to copy the key-value pairs only to change the value of two keys to the same constant. The constant varied depending on the term of an equation in which it appeared.
Instead I defined a macro. It worked until I told Pandoc not to do LaTeX's job of evaluating macros. I needed to do this for Mike Shulman's macros to work. That was more important than my macro. Ordinarly you want LaTeX to compile LaTeX--it's nonstandard to rely on Pandoc for part of the compilation. Publishers won't accept documents that won't compile. But then my macro was left unevaluated.
Anyway, my preference is to telegraph a few things to reconstruct. Perhaps I could err closer in the direction of self-containment. But that is already Zettel Distraction from the mathematical project that led down this primrose path...
And as for omitting jokes, digressions and noise from Zettels, I'll never have any fun.
Post edited by ZettelDistraction on
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[0.2d.0.22.0407]] More header includes and Asterism
[[0.2e.0.22.0407]] Mike Shulman's twocell code
[[5.4.1.1.0.22.0410]] Replacing macros with styles
Not being able to look at these, I don't know why you list them. Is this a replacement for the "twocell code"? Is it using that? You call the target by name, but don't explain what the relation is.
No, the reason for listing Mike Shulman's code is that to get that code to work, the latex_macros extension to Pandoc has to be turned off in the invocation to the markdown reader, like so:
The idea is that Pandoc's macro processing is useful for targeting formats other than LaTeX. But as the documentation says, "[w]hen latex_macros is disabled, the raw LaTeX and math will not have macros applied. This is usually a better approach when you are targeting LaTeX or PDF."
I don't want non-standard macro expansion behavior when creating LaTeX and PDF documents. But when Pandoc lets LaTeX parse and expand its own macros, a macro that I had written and that I had unwittingly assumed would continue to work stopped working. My macro was \my_macro[1]
I was using it to create colored matrices.
\begin{+bmatrix}[\my_macro{apricot}]
a & b \\
c & d
\end{+bmatrix}
The optional square bracket arguments in LaTeX macros lead to trouble. If the argument is a macro, in this case \my_macro (not its real name), it won't expand. This is the expected behavior in LaTeX. So now I was wading into the icy waters of LaTeX macro expansion.
In the end, I needed to replace a macro with a constant. Macros just won't expand inside square brackets, as far as I know (or need to know at this point).
The macro environment was adapted from the definition of +bmatrix in tabularray.sty This is my first ever modification of a macro from a LaTeX package.
That's a good summary of the "what". I'd be missing an intro paragraph about the "why". Like: `I tried to display tables with 20 short columns with package X but they looked ugly. Here's how I adapted a better-looking package Y to behave like X." -- Or whatever actually happens
I disagree. Code isn't code, unless you are a machine. I err on the side of literate programming. Programming Masters of the Art of Binary can log into a BBS by screeching into a phone in lieu of an acoustic modem. But my gifts are somewhat more limited. I often rely on more human (or in my case mutant) readable presentations...
Maybe you can inline the "NOTE"s below as warnings into the code if you believe you need that warning when you want to use the code:
\ExplSyntaxOn % This cannot be eliminated!
I'd argue that if you take for granted that the code in your notes is as small as possible, then this wouldn't be necessary.
I never make that assumption. Never. There are two kinds of programmers in the world: those who take it for granted that their code is as small as possible, and those who don't. As Schopenhauer said, "Avoid both of them."
But:
NOTE: The LaTeX package expl3 and the \ExplSyntaxOn and \ExplSyntaxOff commands cannot be eliminated!
I would personally think that whatever the code is listing is necessary. (Why else include it?)
Because I felt like it. I never assume anything in code is necessary or sufficient just because I listed it. I tried removing them. The code wouldn't compile. I need to be reminded in no uncertain terms not to do that again. In neon lights.
So what is the note/warning here intending to tell you?
Imagine my surprise when I was playing around with expl3 and left the enclosing \ExplSyntaxOn and \ExplSyntaxOff when I decided it was time to use \RenewEnviron to rewrite the +bmatrix environment. It worked! I didn't know at the time that the ExplSyntax macros were needed, and removed them. Big mistake. By accident I had done the right thing. I wish to preserve the awe and mystery that reaches from the inner mind to The Outer Limits.
It turns out that tabularry.sty requires expl3.sty. Presumably this would have been included along with tabularry.sty, which is included in my markdown and in the Pandoc-transformed LaTeX output, but this wasn't the case for the macro definition in the markdown itself. In any case there is a dependency.
Wilfing through tex.stackexchange.com and related LaTeX sites eventually led to ...
(I'd scrap that as noise; in a blog post I likely would add this myself, but not in a Zettel)
It happened. It will get me in the mood. Sorry it's not working for you. I am hinting at the scattershot learning process (if it rises to the level of a process) of a LaTeX macro-expansion novitiate.
... expl3 and environ (not to mention libraries I ended up not needing, such as xparse).
Add links to the 2 packages you used?
Good idea.
Document why xparse was a candidate but then not needed, if you mention it at all? (You likely also didn't need 99.99% of the rest of the libs that exist on CTAN, but you did decide to call xparse by name, why?)
Something I toyed with, among those 99.999% I never touched or thought of during the exercise. I might return to it one day. Why not mention it? Is not mentioning what you ended up not using a theorem? Why only include what is necessary? What about the human element? Is it really necessary to create the misleading impression of linear progress? That would be an unconscionable lie. It's a disgrace! But there you are.
Why not hint at paths not taken (not all of them, but the ones under consideration)? Why not suggest things picked up through the mysterious mechanism of blindsight while investigating something else? Isn't the cultivation of such connections among the virtues of Zettelkasten? No, they have to be suppressed! Stamped out! Erased! Stricken from the record!
Nothing I looked at online had the solution I needed, though several discussions involving such libraries related to LaTeX3 touched on aspects of the problem I was attempting to solve.
It was necessary to try various libraries, driven by forces beyond human agency, until the subconscious (I have faith in this) led me to something that worked. Life is intrinsically wasteful. Most of our energy is wasted as heat. This process is no different.
TikZ styles don't seem to work with the Pandoc command line setting -latex_macros [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.
This sounds quite important. Put it onto its own line so it stands out from the backstory as an actionable warning? Is it an important deviation from your regular workflow that it might be better off beforzxx bve the code?
That probably should be its own Zettel. In fact, I have such a Zettel. TikZ styles aren't really relevant here for a couple of reasons. First, I'm not using tikzpicture, so there is no style to define locally. Second, the alternative is to define global styles, but this would be pointless because I need to change the colors of the matrices individually, not globally for all matrices. I'm cutting it.
To me, it's not clear if the link is about the warning before the [[...]], or if it's just about Customizing Pandoc LaTeX in Zettlr. I adhere to the same conventions as @Sascha explains in https://zettelkasten.de/posts/embrace-dark-folgezettel-power/, so I would expect the link at this point to be about Pandoc command line setting \-latex_macros``, i.e. a note about the pandoc setting. Which sounds like a useful reference note to have. Otherwise I'd write this as:
TikZ styles don't seem to work with the Pandoc command line setting `-latex_macros`. I use this when customizing Pandoc LaTeX in Zettlr [[0.2c.0.21.0617.1059]].
I guess I'm using reverse Polish notation, or something. I'm going to cut the reference to Tikz styles. That really isn't relevant (in my opinion).
I learned something. I hate when that happens.™
Again, blog post or forum material, but not Zettel
Killjoy! I'll never have one iota of fun with this.
What if we omit the optional argument? Is there an error? No, but perhaps the code should check.
Is this a TODO for you? As a note, I'd document this in the code, actually; not sure about TeX conventions, but maybe like this:
Actually the omission is in the call, not the code. There seems to be no side effect. There are conditional tests for no value...
...
\ExplSyntaxOn
%% Omitting the optional argument does not error-out but is undefined behavior.
\RenewEnviron{ +bmatrix }[1][]{
\left[\begin{+array}[expand = \BODY]{
...
The "NOTES" paragraph at the bottom, I'd consider splitting up into a list for easier parsing, and put it at the top to explain dependencies, so the code copypasta comes last:
```markdown
TikZ styles don't seem to work with the Pandoc command line setting -latex_macros[[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.
Required libraries:
expl3 for [INSERT REASON] [INSERT LINK]
Required by tabularray for the definition of +bmatrix and RenewEnviron, below
environ for [INSERT REASON] [INSERT LINK]
needed for \RenewEnviron, to redefine the +bmatrix environment so that it accepts an xcolor constant.
I also tried xparse for [INSERT REASON] but didn't need it because [???].
I felt like trying a lot of things....until I gained enough understanding...
This is very helpful. I'll rewrite. Thank you.
PS. Your lukewarm tea is now cold. I'm making more and will pour it into my microphone, so position your teacup next to your speakers.
Post edited by ZettelDistraction on
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
This is more than I would want to spend on an incidental technical problem related to a larger project, but you were gracious enough to offer sound criticism of my execrable Zettel. Here it is revised.
0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[2.0a.0.22.0412]] LaTeX
The redefinition of the +bmatrix environment given here was motivated by two problems below. It all started with an attempt to get Mike Shulman's twocell macro [[0.2e.0.22.0407]] to compile in LaTeX within an exported markdown document written in Zettlr.
Problem 1. Zettlr calls Pandoc with latex_macros enabled
The Pandoc extension latex_macros causes Pandoc to expand LaTeX macro definitions as it converts to LaTeX. See Pandoc extension: latex_macros.
The Pandoc manual recommends disabling the latex_macros extension when converting to LaTeX and PDF.
When latex_macros is disabled, the raw LaTeX and math will not have macros applied. This is usually a better approach when you are targeting LaTeX or PDF.
Zettlr's default Pandoc configuration turns latex_macros on. This is non-standard, it can make re-using some example macros difficult, and it complicates the already complicated handling of macro expansion in LaTeX.
LaTeX programmers have been dissatisfied with latex macro handling and have written macro expansion packages with improved interfaces. One such library is expl3 (see below).
Among other side-effects, the default Zettlr Pandoc configuration caused Mike Shulman's twocell macro [[0.2e.0.22.0407]] to fail. Disabling latex_macros in Zettlr's LaTeX and PDF export configuration files enables the twocell macro to compile to valid LaTeX. See [[0.2d.0.22.0407]] More header includes and Asterism for details on modifying markdown reader behavior in Zettlr, and on fencing LaTeX code in the YAML document header variable more-header-includes: for injecting LaTeX code in the preamble of the exported LaTeX. (These should be split out.)
Fixing Problem 1 led to Problem 2.
Problem 2. macros relying on Pandoc's latex_macros extension could fail.
With Zettlr and Pandoc leaving macro LaTeX macro expansion to LaTeX, other problems surfaced. I had written the following macro to set the color of the rows and columns of a matrix. A minimum working example is shown below.
The code was used in the optional argument of the +bmatrix environment defined in tabularray.sty as follows. Again this is a minimum working example.
{\begin{+bmatrix}[\my_macro{apricot}]
a & b \\
c & d
\end{+bmatrix}}
The macro \my_macro{apricot} is invoked within the matching square bracketed optional argument of the +bmatrix environment. With the Pandoc extension latex_macros disabled, LaTex no longer expands \my_macro, and the rows and columns of the matrix are no longer colored.
The macro environment definition below was adapted from the definition of +bmatrix in tabularray.sty This is my first ever modification of a macro from a LaTeX package.
\usepackage{expl3} % tabularray.sty requires expl3
\usepackage{environ} % Needed for \RenewEnviron
\ExplSyntaxOn % Defined in expl3 and needed for the RenewEnviron below
\RenewEnviron{ +bmatrix }[1][]{ % +bmatrix is defined in tabularray.sty (loaded in template.tex)
\left[\begin{+array}[expand = \BODY]{
column{1} = {leftsep = 0pt}, column{Z} = {rightsep = 0pt},
rowsep={1pt},row{1}={#1},row{2}={#1}, % This line replaces the macro expansion
cells = {c}, ##1
}
\BODY
\end{+array}\right]
\ignorespacesafterend
}
\ExplSyntaxOff % Defined in expl3
I found out about them through the definitive online encyclopaedia of software documentation bar none (excluding the great D. E. Knuth himself), StackExchange.
The neat happy ending is unmitigated bullshit
I'm presenting this solution as if pulling a rabbit out of a hat. The solution came after three days of confused efforts to understand the LaTeX macro system and to get a feeling for various packages written to improve upon macro handling in LaTeX. Nothing that I had read contained the solution to this problem. There were hints about possible approaches here and there.
I had been experimenting with expl3 and absent-mindedly left behind the pair of \ExplSyntaxOn and \ExplSyntaxOff macros when it occurred to me to renew the +bmatrix environment with the \RenewEnviron command of the environ package. This worked. When I removed the ExplSyntax commands, it didn't work. It turns out that expl3 is required by tabularray.sty which is needed by \RenewEnviron to renew the definition of the +bmatrix environment. I was lucky.
While still 'young' I began to seek consolation in the thought that then if ever, i.e. now, the true words at last, from the mind in ruins.
– Samuel Beckett
Post edited by ZettelDistraction on
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Well -- Since this forum is indexed by search engines, now there's a chance someone searching for Zettlr Pandoc LaTeX macro thingies will find the results of your tinkering!
The result looks very polished to me. Almost like technical documentation, but not in a bad sense. I wouldn't mind reading this on a Zettlr LaTeX Pandoc Workflow blog. So @Sascha's heuristic that every Zettel should be a blog post is satisfied. 👍
I hope your 2 problem descriptions help in the future, too. If you run into similar situations, it might turn out that these problem descriptions could be their own Zettel eventually when handling macros comes up more often.
It was necessary to try various libraries, driven by forces beyond human agency, until the subconscious (I have faith in this) led me to something that worked. Life is intrinsically wasteful. Most of our energy is wasted as heat. This process is no different.
That's just how things work with code, right?
I too don't document all I did during the 4 hours of failing to figure out what a given problem actually is until it takes all but 2 minutes to implement a fix. I make sure to take note of the happy path that led to the solution + whatever I need to consider to apply this solution in another situation later. That requires contextual information like what I tried, alternatives I considered but didn't get to try, the observations I made along the way for reference, the behavior of the software that was deemed buggy, etc.
In a year's time, I can only use actionable advice to quicken the path towards a fix, so that takes center stage. (Or: trying what I believe might lead to that.) Didn't mean to imply that this is all there is. Of course there is experimentation. That's why I suggested telling your future self what happened with xparse, and what your 'learnings' were so that when you search your ZK for "xparse" some day, you maybe have a bit of a clue
Comments
0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[0.2d.0.22.0407]] More header includes and Asterism
[[0.2e.0.22.0407]] Mike Shulman's twocell code
[[5.4.1.1.0.22.0410]] Replacing macros with styles
#latex #latex-environ #latex-tabularray
The macro environment was adapted from the definition of
+bmatrixin tabularray.sty This is my first ever modification of a macro from a LaTeX package.\usepackage{expl3} \usepackage{environ} \ExplSyntaxOn \RenewEnviron{ +bmatrix }[1][]{ \left[\begin{+array}[expand = \BODY]{ column{1} = {leftsep = 0pt}, column{Z} = {rightsep = 0pt}, rowsep={1pt},row{1}={#1},row{2}={#1}, cells = {c}, ##1 } \BODY \end{+array}\right] \ignorespacesafterend } \ExplSyntaxOffNOTE: The LaTeX package
expl3and the\ExplSyntaxOnand\ExplSyntaxOffcommands cannot be eliminated! Wilfing through tex.stackexchange.com and related LaTeX sites eventually led toexpl3andenviron(not to mention libraries I ended up not needing, such asxparse). TikZ styles don't seem to work with the Pandoc command line setting-latex_macros[[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr.I learned something. I hate when that happens.™
What if we omit the optional argument? Is there an error? No, but perhaps the code should check.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
@ZettelDistraction This is above my payroll.
I am a Zettler
Oh well. I'll subject it to severe criticism myself then:
Aside from the
\RenewEnviron, which replaces original\NewEnvironintabularray.sty, the modified line wasrowsep={1pt},row{1}={#1},row{2}={#1},The application was to color entries in a matrix, for a math project that is under wraps for the moment. Coloring rows and columns of matrices in LaTeX documents can be done in several ways. However, two methods I used (setting a style with
tikzsetand calling a macro within optional square bracket arguments in a LaTeX environment) stopped working after I modified the Pandoc reader configuration in the Zettlr configuration filesexport.pdf.yamlandexport.latex.yaml. The modification was to prevent Pandoc from expanding LaTeX macros while converting from markdown to latex. What you ordinarily want is for LaTeX to expand its macros, not for Pandoc to preprocess these. The setting to stop Pandoc from expanding macros before LaTeX gets the opportunity isreader: markdown-latex_macros.However, that meant that macro expansion within square bracket arguments never occurred during LaTeX processing. This is a general problem that people tend to encounter. I had a macro that set a few row and column colors that no longer expanded when called from the optional argument in+bmatrix, for example:begin{+bmatrix}[\myspec{lightskyblue}]. I set out to understand the more general problem and solve it for this case.tikzsetto set styles with tikz graphics?No, but uh, I wanted to get Mike Shulman's twocell code to work, and for that I needed to turn off Pandoc LaTeX expansion anyway.
OK you're right, that's probably unrelated. I might need to go down the
tikzsetversustikzstylerabbit hole. Dig the following, like acquiesce in it, man: https://tex.stackexchange.com/questions/52372/should-tikzset-or-tikzstyle-be-used-to-define-tikz-stylesFrom going down that rabbit hole? I'm not using
tikzpictureso there is no local environment in which to set styles, and I'm not sure that I want to set global styles withtikzset. How do you know this issue is germane enough to warrant your hectoring! Give me a break!GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Hold my luke warm tea!
Not being able to look at these, I don't know why you list them. Is this a replacement for the "twocell code"? Is it using that? You call the target by name, but don't explain what the relation is.
👍
That's a good summary of the "what". I'd be missing an intro paragraph about the "why". Like: `I tried to display tables with 20 short columns with package X but they looked ugly. Here's how I adapted a better-looking package Y to behave like X." -- Or whatever actually happens
Code is code, that's good enough. Maybe you can inline the "NOTE"s below as warnings into the code if you believe you need that warning when you want to use the code:
I'd argue that if you take for granted that the code in your notes is as small as possible, then this wouldn't be necessary. But:
I would personally think that whatever the code is listing is necessary. (Why else include it?) So what is the note/warning here intending to tell you?
(I'd scrap that as noise; in a blog post I likely would add this myself, but not in a Zettel)
Add links to the 2 packages you used? Document why
xparsewas a candidate but then not needed, if you mention it at all? (You likely also didn't need 99.99% of the rest of the libs that exist on CTAN, but you did decide to callxparseby name, why?)This sounds quite important. Put it onto its own line so it stands out from the backstory as an actionable warning? Is it an important deviation from your regular workflow that it might be better off before the code?
To me, it's not clear if the link is about the warning before the
[[...]], or if it's just aboutCustomizing Pandoc LaTeX in Zettlr. I adhere to the same conventions as @Sascha explains in https://zettelkasten.de/posts/embrace-dark-folgezettel-power/, so I would expect the link at this point to be aboutPandoc command line setting \-latex_macros``, i.e. a note about the pandoc setting. Which sounds like a useful reference note to have. Otherwise I'd write this as:Again, blog post or forum material, but not Zettel
Is this a TODO for you? As a note, I'd document this in the code, actually; not sure about TeX conventions, but maybe like this:
... \ExplSyntaxOn %% Omitting the optional argument does not error-out but is undefined behavior. \RenewEnviron{ +bmatrix }[1][]{ \left[\begin{+array}[expand = \BODY]{ ...The "NOTES" paragraph at the bottom, I'd consider splitting up into a list for easier parsing, and put it at the top to explain dependencies, so the code copypasta comes last:
Author at Zettelkasten.de • https://christiantietze.de/
@ctietze Thank for your extensive comments. I'm going to follow some of your suggestions. Perhaps I could take "feeting" notes. I tend to work interactively. That could be my downfall.
My notes aren't self-contained. Their function is to jog my memory. Also, my attitude to code differs from yours. Here I was performing surgery on a macro. Whatever understanding I have is local, situational. My attitude is that code is not code, that is, there is no reason to expect that everything present is necessary and sufficient. I came to this not knowing what was needed. I tried several approaches over three days. I happened to be playing with expl3, xparse, environ and several other things in a scattered effort to grasp at what might work. My reason for trying them? I felt like lt. They seemed to touch on possibly relevant aspects of macro evaluation in TeX and LaTeX, a system that has led TeX and LaTeX package designers to invent alternatives.
Perhaps this haphazard approach isn't methodical or efficient. I don't expect to be. I rely on unconscious processes beyond my control, on blindsight. It helps to cultivate blindsight. I don't insist on having reasons. The 2500 year history of normative philosophy is the history of disagreement among philosophers over what constitutes a reason. We are discussing programs, though, and these are supposed to have preconditions that can be stated, computational complexity notwithstanding. I grant you that. As for what led me to try one thing or another? I don't know. I was driven to try them.
Why not mention the things that didn't work, or that I tried or thought about or tried to think about before some clarity emerged from the mind shrouded in tenebrous vapors? Nothing that I had read contained the solution to this problem. There were hints about possible approaches here and there.
There are two kinds of programmers in the world: those who think code is there for a reason and should be enough, and those who don't think "should“ is a term of computer science. OK the other group includes Literate Programming, after D. E. Knuth. Avoid both of them. I'm in the second camp.
It was sheer luck that I happened to have enclosed the ‘\RenewEnviron’ macro in ‘\ExplSyntaxOn’ and ‘\ExplSyntaxOff’. The renewed macro definition worked with them present, and refused to compile without them. Perhaps expl3 is a prerequisite in tabularray.tex. I didn't see it in tabularray.sty, which contains the original macro definition. In any case, I had been attempting to understand expl3 previously, when I decided that I needed to rewrite the macro whose behavior changed once I prevented Pandoc from expanding LaTeX macros while converting from markdown to LaTeX. In retrospect that would have been the correct initial approach. But then I would have had to have known that ExplSyntaxOn and ExplSyntaxOff were somehow needed. Unless this is an inadvertently cached dependency left behind in some auxilliary intermediate compilation file after some long abandoned experiment. The approach is operational.
The macro has one optional argument, indicated by the presence of square brackets. LaTeX won't evaluate macros in square bracketed arguments to other macros, but it will recognize constants. This is a common problem if you need to set a series of comma-separated key-value pairs within optional arguments. I didn't want to have to copy the key-value pairs only to change the value of two keys to the same constant. The constant varied depending on the term of an equation in which it appeared.
Instead I defined a macro. It worked until I told Pandoc not to do LaTeX's job of evaluating macros. I needed to do this for Mike Shulman's macros to work. That was more important than my macro. Ordinarly you want LaTeX to compile LaTeX--it's nonstandard to rely on Pandoc for part of the compilation. Publishers won't accept documents that won't compile. But then my macro was left unevaluated.
Anyway, my preference is to telegraph a few things to reconstruct. Perhaps I could err closer in the direction of self-containment. But that is already Zettel Distraction from the mathematical project that led down this primrose path...
And as for omitting jokes, digressions and noise from Zettels, I'll never have any fun.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Funny.
No, the reason for listing Mike Shulman's code is that to get that code to work, the
latex_macrosextension to Pandoc has to be turned off in the invocation to the markdown reader, like so:And the other reason is that the Pandoc manual recommends that you turn off
latex_macros. It's left on by default in Zettlr. See https://disco.uv.es/disco_docs/wikibase/doc/cas/pandoc_manual_2.7.3.wiki?121The idea is that Pandoc's macro processing is useful for targeting formats other than LaTeX. But as the documentation says, "[w]hen latex_macros is disabled, the raw LaTeX and math will not have macros applied. This is usually a better approach when you are targeting LaTeX or PDF."
I don't want non-standard macro expansion behavior when creating LaTeX and PDF documents. But when Pandoc lets LaTeX parse and expand its own macros, a macro that I had written and that I had unwittingly assumed would continue to work stopped working. My macro was
\my_macro[1]I was using it to create colored matrices.
a & b \\ c & d \end{+bmatrix}The optional square bracket arguments in LaTeX macros lead to trouble. If the argument is a macro, in this case
\my_macro(not its real name), it won't expand. This is the expected behavior in LaTeX. So now I was wading into the icy waters of LaTeX macro expansion.In the end, I needed to replace a macro with a constant. Macros just won't expand inside square brackets, as far as I know (or need to know at this point).
I did one thing right.
I'll show you what happens.
I disagree. Code isn't code, unless you are a machine. I err on the side of literate programming. Programming Masters of the Art of Binary can log into a BBS by screeching into a phone in lieu of an acoustic modem. But my gifts are somewhat more limited. I often rely on more human (or in my case mutant) readable presentations...
I never make that assumption. Never. There are two kinds of programmers in the world: those who take it for granted that their code is as small as possible, and those who don't. As Schopenhauer said, "Avoid both of them."
But:
Because I felt like it. I never assume anything in code is necessary or sufficient just because I listed it. I tried removing them. The code wouldn't compile. I need to be reminded in no uncertain terms not to do that again. In neon lights.
Imagine my surprise when I was playing around with expl3 and left the enclosing
\ExplSyntaxOnand\ExplSyntaxOffwhen I decided it was time to use\RenewEnvironto rewrite the+bmatrixenvironment. It worked! I didn't know at the time that the ExplSyntax macros were needed, and removed them. Big mistake. By accident I had done the right thing. I wish to preserve the awe and mystery that reaches from the inner mind to The Outer Limits.It turns out that
tabularry.styrequiresexpl3.sty. Presumably this would have been included along withtabularry.sty, which is included in my markdown and in the Pandoc-transformed LaTeX output, but this wasn't the case for the macro definition in the markdown itself. In any case there is a dependency.It happened. It will get me in the mood. Sorry it's not working for you. I am hinting at the scattershot learning process (if it rises to the level of a process) of a LaTeX macro-expansion novitiate.
Good idea.
Something I toyed with, among those 99.999% I never touched or thought of during the exercise. I might return to it one day. Why not mention it? Is not mentioning what you ended up not using a theorem? Why only include what is necessary? What about the human element? Is it really necessary to create the misleading impression of linear progress? That would be an unconscionable lie. It's a disgrace! But there you are.
Why not hint at paths not taken (not all of them, but the ones under consideration)? Why not suggest things picked up through the mysterious mechanism of blindsight while investigating something else? Isn't the cultivation of such connections among the virtues of Zettelkasten? No, they have to be suppressed! Stamped out! Erased! Stricken from the record!
Nothing I looked at online had the solution I needed, though several discussions involving such libraries related to LaTeX3 touched on aspects of the problem I was attempting to solve.
It was necessary to try various libraries, driven by forces beyond human agency, until the subconscious (I have faith in this) led me to something that worked. Life is intrinsically wasteful. Most of our energy is wasted as heat. This process is no different.
That probably should be its own Zettel. In fact, I have such a Zettel. TikZ styles aren't really relevant here for a couple of reasons. First, I'm not using tikzpicture, so there is no style to define locally. Second, the alternative is to define global styles, but this would be pointless because I need to change the colors of the matrices individually, not globally for all matrices. I'm cutting it.
I guess I'm using reverse Polish notation, or something. I'm going to cut the reference to Tikz styles. That really isn't relevant (in my opinion).
Killjoy! I'll never have one iota of fun with this.
Actually the omission is in the call, not the code. There seems to be no side effect. There are conditional tests for no value...
Required by tabularray for the definition of +bmatrix and RenewEnviron, below
needed for \RenewEnviron, to redefine the +bmatrix environment so that it accepts an xcolor constant.
I felt like trying a lot of things....until I gained enough understanding...
This is very helpful. I'll rewrite. Thank you.
PS. Your lukewarm tea is now cold. I'm making more and will pour it into my microphone, so position your teacup next to your speakers.
GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
This is more than I would want to spend on an incidental technical problem related to a larger project, but you were gracious enough to offer sound criticism of my execrable Zettel. Here it is revised.
0.2f.0.2022.0411 Modifying tabularray +bmatrix with environ
CONTEXT [[0.2c.0.21.0617.1059]] Customizing Pandoc LaTeX in Zettlr
[[2.0a.0.22.0412]] LaTeX
#latex #latex-environ #latex-tabularray
The redefinition of the
+bmatrixenvironment given here was motivated by two problems below. It all started with an attempt to get Mike Shulman'stwocellmacro [[0.2e.0.22.0407]] to compile in LaTeX within an exported markdown document written in Zettlr.Problem 1. Zettlr calls Pandoc with
latex_macrosenabledThe Pandoc extension
latex_macroscauses Pandoc to expand LaTeX macro definitions as it converts to LaTeX. See Pandoc extension: latex_macros.The Pandoc manual recommends disabling the
latex_macrosextension when converting to LaTeX and PDF.Zettlr's default Pandoc configuration turns
latex_macroson. This is non-standard, it can make re-using some example macros difficult, and it complicates the already complicated handling of macro expansion in LaTeX.LaTeX programmers have been dissatisfied with latex macro handling and have written macro expansion packages with improved interfaces. One such library is
expl3(see below).Among other side-effects, the default Zettlr Pandoc configuration caused Mike Shulman's
twocellmacro [[0.2e.0.22.0407]] to fail. Disablinglatex_macrosin Zettlr's LaTeX and PDF export configuration files enables thetwocellmacro to compile to valid LaTeX. See [[0.2d.0.22.0407]] More header includes and Asterism for details on modifying markdown reader behavior in Zettlr, and on fencing LaTeX code in the YAML document header variablemore-header-includes:for injecting LaTeX code in the preamble of the exported LaTeX. (These should be split out.)Fixing Problem 1 led to Problem 2.
Problem 2. macros relying on Pandoc's
latex_macrosextension could fail.With Zettlr and Pandoc leaving macro LaTeX macro expansion to LaTeX, other problems surfaced. I had written the following macro to set the color of the rows and columns of a matrix. A minimum working example is shown below.
\newcommand\my_macro[1]{rowsep={1pt},row{1}={#1},row{2}={#1}}The code was used in the optional argument of the
+bmatrixenvironment defined intabularray.styas follows. Again this is a minimum working example.{\begin{+bmatrix}[\my_macro{apricot}] a & b \\ c & d \end{+bmatrix}}The macro
\my_macro{apricot}is invoked within the matching square bracketed optional argument of the+bmatrixenvironment. With the Pandoc extensionlatex_macrosdisabled, LaTex no longer expands\my_macro, and the rows and columns of the matrix are no longer colored.Non-expansion of LaTeX macros within optional square bracketed arguments of other LaTeX macros and commands is a relatively common problem. See, for example: tikz pgf - Can't use macro in square brackets - TeX - LaTeX Stack Exchange.
Solution: renew the environment
The macro environment definition below was adapted from the definition of
+bmatrixin tabularray.sty This is my first ever modification of a macro from a LaTeX package.\usepackage{expl3} % tabularray.sty requires expl3 \usepackage{environ} % Needed for \RenewEnviron \ExplSyntaxOn % Defined in expl3 and needed for the RenewEnviron below \RenewEnviron{ +bmatrix }[1][]{ % +bmatrix is defined in tabularray.sty (loaded in template.tex) \left[\begin{+array}[expand = \BODY]{ column{1} = {leftsep = 0pt}, column{Z} = {rightsep = 0pt}, rowsep={1pt},row{1}={#1},row{2}={#1}, % This line replaces the macro expansion cells = {c}, ##1 } \BODY \end{+array}\right] \ignorespacesafterend } \ExplSyntaxOff % Defined in expl3The
expl3package is available at CTAN: Package expl3.The
environpackage is available at CTAN: Package environ.I found out about them through the definitive online encyclopaedia of software documentation bar none (excluding the great D. E. Knuth himself), StackExchange.
The neat happy ending is unmitigated bullshit
I'm presenting this solution as if pulling a rabbit out of a hat. The solution came after three days of confused efforts to understand the LaTeX macro system and to get a feeling for various packages written to improve upon macro handling in LaTeX. Nothing that I had read contained the solution to this problem. There were hints about possible approaches here and there.
I had been experimenting with
expl3and absent-mindedly left behind the pair of\ExplSyntaxOnand\ExplSyntaxOffmacros when it occurred to me to renew the+bmatrixenvironment with the\RenewEnvironcommand of theenvironpackage. This worked. When I removed theExplSyntaxcommands, it didn't work. It turns out thatexpl3is required bytabularray.stywhich is needed by\RenewEnvironto renew the definition of the+bmatrixenvironment. I was lucky.GitHub. Erdős #2. Problems worthy of attack / prove their worth by hitting back. -- Piet Hein. Alter ego: Erel Dogg (not the first). CC BY-SA 4.0.
Well -- Since this forum is indexed by search engines, now there's a chance someone searching for Zettlr Pandoc LaTeX macro thingies will find the results of your tinkering!
The result looks very polished to me. Almost like technical documentation, but not in a bad sense. I wouldn't mind reading this on a Zettlr LaTeX Pandoc Workflow blog. So @Sascha's heuristic that every Zettel should be a blog post is satisfied. 👍
I hope your 2 problem descriptions help in the future, too. If you run into similar situations, it might turn out that these problem descriptions could be their own Zettel eventually when handling macros comes up more often.
That's just how things work with code, right?
I too don't document all I did during the 4 hours of failing to figure out what a given problem actually is until it takes all but 2 minutes to implement a fix. I make sure to take note of the happy path that led to the solution + whatever I need to consider to apply this solution in another situation later. That requires contextual information like what I tried, alternatives I considered but didn't get to try, the observations I made along the way for reference, the behavior of the software that was deemed buggy, etc.
In a year's time, I can only use actionable advice to quicken the path towards a fix, so that takes center stage. (Or: trying what I believe might lead to that.) Didn't mean to imply that this is all there is. Of course there is experimentation. That's why I suggested telling your future self what happened with
xparse, and what your 'learnings' were so that when you search your ZK for "xparse" some day, you maybe have a bit of a clueAuthor at Zettelkasten.de • https://christiantietze.de/