Context-sensitive APIs in LaTeX-commands
programming add tag
3 years ago निरंजन

I am trying to re-implement a package with a command \foo which I am guessing has 2 APIs (I am not very sure, maybe there are other ways of achieving it). The user-side API of the command is like this:

Case 1

This is the default behavior. The same behavior is repeated with the following, but with a value abc to a key named foo. We can interpret that the initial value of the key foo is abc.

Case 2

Case 3

I have coded the simple-most case as follows:

Screenshot_2022-06-26_12-01-55.png

For developing keys, I will use my favorite pair of packages expkv-opt,expkv-def. The following should give us case 2.

Screenshot_2022-06-26_17-32-04.png

So I think in order to equate the output of case 3 with the rest of the two; I will have to have an alternate argument structure for \foo. Is there any other way to get it?

Top Answer
3 years ago frougon

The input syntax is very weird. As I wrote in the comments, please first think if it really isn’t possible to support an input syntax that is more common in the LaTeX world.

I provide the following code to make it clear how the special parsing can be triggered when we detect the special case foo=def in the options of \foo. Consider it as code golf!

Note: I used l3keys because I am familiar with it and not with expkv; I have nothing particular against the latter. 😃

image.png

Regarding this part:

I used \NewExpandableDocumentCommand in case you want to use \useFooKey in expansion-only contexts (inside \edef, \write, \csname, \numexpr… similarly, expandable material works fine inside the argument of siunitx macros like \num, and there are many more places like that). But this is not a choice I can make because I don’t know what kind of tokens \l_mypkg_fookey_tl will legitimately contain. All I know is that \tl_use:N is expandable, thus works in expansion-only contexts (and can even be omitted because tl vars are simple non-\protected macros).

I can’t know if the material resulting from the expansion of \l_mypkg_fookey_tl is designed to be used in expansion-only contexts: only you, the API designer, can know.

For instance, if \l_mypkg_fookey_tl is supposed to contain non-expandable code fragments (e.g., containing \def, \setcounter or \addtocounter calls, markup like \section{...}…), probably the \NewExpandableDocumentCommand should be replaced with \NewDocumentCommand in order to prevent the expansion of \useFooKey in certain kinds of expansion-only contexts (inside \edef, \xdef, \expanded, \message, \errmessage, \special, \mark, \marks; when writing the token list for \write to a file; when looking ahead in an alignment for \noalign or \omit; plus everything that is based on one of these mechanisms of the TeX engine).

Also, I hesitated between “purity” and efficiency here: the \NewExpandableDocumentCommand clearly marks \useFooKey as belonging to the document-level layer, however we don’t need the (formerly from xparse, now documented in usrguide3.pdf) heavy \NewExpandableDocumentCommand machinery here, since \useFooKey is a trivial macro that takes no argument. The simple fact of using \NewExpandableDocumentCommand probably makes \useFooKey a bit slow.

So, an alternative which I didn’t put in the code because I wanted to keep it short and readable, but is at least as good in my opinion, is:

or, more generically:

(in which case you would input \useKey{foo} instead of \useFooKey).

Again, the \cs_new:Npn used here was chosen under the assumption that \useFooKey or \useKey{foo} should expand in expansion-only contexts; if \useFooKey or \useKey{foo} should be expanded only when TeX performs its full expansion + execution interleaved process (typically, when typesetting), use \cs_new_protected:Npn or \cs_new_protected:Nn instead.

comments
transcript
3 years
Skillmon — Monday, 27th Jun 2022 13:42

(though one would need to first define the \exp_args:... variant for this, so visiting the expl3-layer at least once)

Skillmon replying to निरंजन — Monday, 27th Jun 2022 13:41

I’d agree with @frougon here, \ExpandArgs{Nne} should do (note: e-type instead of x-type expansion to get the same behaviour regarding # as a simple \edef)

3 hours
निरंजन — Monday, 27th Jun 2022 11:09

Hmm, got it.

frougon replying to निरंजन — Monday, 27th Jun 2022 11:05

The \noindent could also be before the group (think of the effect of hooks like \everypar if they perform local assignments) but since \par is inside, I think it should be okay.

निरंजन replying to frougon — Monday, 27th Jun 2022 11:02

Right, in addition to that there are some conditionals too.

frougon replying to निरंजन — Monday, 27th Jun 2022 11:00

The \begingroup... \endgroup is useful in any MWE that sets keys (things like \keys_set:nn only affect the current group by design). Similarly, when you use \setkeys{Gin}{...}, \sisetup or \hypersetup, etc., this normally only affects the current group.

निरंजन replying to frougon — Monday, 27th Jun 2022 10:58

Right, it’s not the case AFAIK, but even if it is, I will use {\noindent ##1\par} now 😃

frougon replying to निरंजन — Monday, 27th Jun 2022 10:51

I really mean the braces in your \noindent{##1}\par. In case ##1 can legitimately contain declarations like \sffamily (you can say, I can’t), the braces do make sense.

निरंजन replying to frougon — Monday, 27th Jun 2022 10:47

Yes, you are right. \begingroup, \endgroup still has a reason to exist in my larger code (not in the MWE), but the braces were absolutely unnecessary.

निरंजन replying to frougon — Monday, 27th Jun 2022 10:44

Ah, thanks.

frougon replying to निरंजन — Monday, 27th Jun 2022 10:44

The braces (or \begingroup... \endgroup) do make sense if there is going to be stuff like \sffamily in the argument… so as not to contaminate everything that follows. 😃

निरंजन replying to frougon — Monday, 27th Jun 2022 10:43

😃

निरंजन replying to frougon — Monday, 27th Jun 2022 10:43

I’ll check.

निरंजन replying to frougon — Monday, 27th Jun 2022 10:42

Okay, thanks.

निरंजन — Monday, 27th Jun 2022 10:42

I thought you were talking about \begingroup and \endgroup.

निरंजन replying to frougon — Monday, 27th Jun 2022 10:42

Oh, those braces were just to protect my argument from I don’t know what 😂 I will remove them. I know \noindent takes no argument, even though it looks like that; it wasn’t intended.

frougon replying to निरंजन — Monday, 27th Jun 2022 10:42

Not directly to my knowledge, but the new \ExpandArgs could do (see usrguide3.pdf); however, this kind of thing would maybe be better done in expl3, I think (where there are tons of ways).

निरंजन replying to Skillmon — Monday, 27th Jun 2022 10:39

Thanks, is there anything similar \edef in xparse which expands everything from its definition?

an hour
frougon replying to निरंजन — Monday, 27th Jun 2022 09:50

Yes, the l3keys doc has this example:

(see interface3.pdf) plus, you can add a generic handler for unknown values. Also possible is to say foo .code:n { ... } inside the \keys_define:nn and do the comparisons using macros in ... (not my preferred choice a priori).

Orthogonal to the use of l3keys: \tl_case:NnTF (\str_case:NnTF if you manipulate an str var) is very very convenient when there are several special values each with a corresponding handler, and possibly a generic handler for all other cases.

15 minutes
frougon replying to निरंजन — Monday, 27th Jun 2022 09:35

I believe your discussion with Skillmon clarified most of your question. If not, say what isn’t clear (one part of the question wasn’t). Regarding this bit:

I added a few comments at the end of my answer.

30 minutes
frougon replying to निरंजन — Monday, 27th Jun 2022 09:05

Okay, everyone makes little mistakes like this, no need to be so sorry.

frougon replying to निरंजन — Monday, 27th Jun 2022 09:04

You did! 😃 Your code contains 3 times \noindent{##1}\par where the braces definitely form a group because \noindent takes no argument (you know this, right?).

Skillmon replying to निरंजन — Monday, 27th Jun 2022 08:59

so it is “expand \baz and then expand the replacement text of \baz, until you don’t find anything that can be expanded”

Skillmon replying to निरंजन — Monday, 27th Jun 2022 08:58

\edef is basically expand everything that’s expandable and set the replacement text to the result.

निरंजन — Monday, 27th Jun 2022 08:57

So can I say \edef\foo{\baz} is basically "expand \baz and define \foo"

Skillmon replying to निरंजन — Monday, 27th Jun 2022 08:57

yes, now you got it 😛

निरंजन replying to Skillmon — Monday, 27th Jun 2022 08:55

Ah, got it! So if I define \NewDocumentCommand for defining \foo & \baz they won’t expand inside an \edef, but if I use, \def or\NewExpandableDocumentCommand they will, right?

Skillmon replying to निरंजन — Monday, 27th Jun 2022 08:49

\NewExpandableDocumentCommand is not \protected, that’s the whole point of its existence compared to \NewDocumentCommand.

2 hours
निरंजन — Monday, 27th Jun 2022 06:36

If I understand correctly, the following code:

produces foobaz and pqrxyz, but if \foo and \baz are \protected, it results in pqrxyz twice, exactly like we are getting when we define \foo and \baz with \NewExpandableDocumentCommand instead of \def as all xparse macros are \protected. I understood this much, but now the question that I have is if commands defined with \NewExpandableDocumentCommand don’t expand like \edef what is the difference between these two expansions? What is exactly expandable in them? This question came to my mind because I saw the following in your code and I was wondering why do we specifically need \NewExpandableDocumentCommand here and why simple \NewDocumentCommand won’t work?

29 minutes
निरंजन — Monday, 27th Jun 2022 06:07

and the extra ghi was just a mistake. It’s quite embarrassing now. Thanks for still considering the question worth putting efforts into.

निरंजन — Monday, 27th Jun 2022 06:06

Was there a simple way of testing if the value of \fookey is the string abc or def without defining macros?

निरंजन replying to frougon — Monday, 27th Jun 2022 06:05

I am extremely sorry. This inconsistency arose because of my stupid labeling. Initially I chose abc and def, because well it looked natural to me. I wrote the half of the question where you actually see foo=def, but while writing the \ifx code I realized I need two macros for comparing. Hence the \def\abc{abc} part, but here I wrote \def\def{def} and boom! 🔥 Hence decided to change it to pqr instead of def, but I missed to change it in the earlier question. Sorry once again.

29 minutes
निरंजन replying to frougon — Monday, 27th Jun 2022 05:36

Thanks for this information, but haven’t I done that? I have 3 \pars in the code and all of them are before the \endgroup which is defined in \oof.

निरंजन — Monday, 27th Jun 2022 05:34

I am just trying to provide a new interface along with a lot of additional features and a well-documented implementation.

निरंजन replying to frougon — Monday, 27th Jun 2022 05:32

I would have definitely avoided this weird syntax and rather have used a common one in one of my packages, but as this is a released & widely used old package, I don’t have the liberty to change the API, sorry.

10 hours
frougon — Sunday, 26th Jun 2022 19:32

I simplified the definition of \__mypkg_parse_special:w because the special rule regarding catcode 10 characters encountered in state M given on p. 47 of the TeXbook makes it so that expl3's ~ is tokenized as a normal space token (char code 32, catcode 10), not as ~-with-catcode-10 as one could expect.

3 hours
Ulrike Fischer — Sunday, 26th Jun 2022 16:15

You shouldn’t define commands inside other commands unless it is really necessary. And the u argument type is deprecated and should not be used anymore.

44 minutes
frougon — Sunday, 26th Jun 2022 15:31

Finally, either there are inconsistencies between your various instances of case 3, or I don’t understand it (foo=def or foo=pqr? ghi repeated by mistake in the “second part” of case 3?).

27 minutes
frougon — Sunday, 26th Jun 2022 15:04

Thirdly, assuming I understand it correctly that abc and def are special values for the foo key and that case 3 should produce the same output as the other two, except for “The current value of foo is: …”, you don’t need to have the xparse code behind \foo to interpret the abc[ghi/mno] def[ghi/jkl/pqr] input.

More precisely, \foo can be a macro accepting an optional argument and nothing more. \baz, \foobaz et al. could a priori have one definition at top-level (no need to nest). Finally, when \foo finds that foo=def has been supplied in its optional argument, it can leave, e.g., \__mypkg_parse_special:w in the input stream, that will parse what immediately follows it. We don’t know the precise syntax rules, though, but if they are known and not too baroque, this shouldn’t be very difficult.

frougon — Sunday, 26th Jun 2022 14:56

Secondly: in general, \par should come before you close the group (otherwise, local settings like font-dependent \baselineskip vanish before the paragraph is typeset).

frougon — Sunday, 26th Jun 2022 14:54

First things first: is it necessary to support all these kinds of input? IOW, can’t you use an input syntax that is more regular for LaTeX (using control sequence tokens, brace groups, comma lists [which includes keyval lists], etc.)?

Enter question or answer id or url (and optionally further answer ids/urls from the same question) from

Separate each id/url with a space. No need to list your own answers; they will be imported automatically.