This returns errors.
xxxxxxxxxx
\documentclass{article}
\usepackage{pgffor}
\usepackage{expl3}
\begin{document}
\foreach \i in {0} {
\ExplSyntaxOn
\the\catcode`_, \the\catcode`:, \the\catcode`\ .
\prg_do_nothing:
\ExplSyntaxOff
}
\end{document}
Here it is very interesting:
- Space has category code 9 (ignored) (as expected after
\ExplSyntaxOn
). However the output still has spaces. I expect it to be “11,11,9.” and not “11,␣11,␣9.”, because no~
is provided. _
and:
have category code 11 (letter) (still as expected after\ExplSyntaxOn
). So\prg_do_nothing:
should be a valid control sequence and it should work. It didn’t, and the error messages implies_
is still considered “math subscript” character (catcode 8).
It is even more interesting when I put the whole \foreach
inside explosive syntax:
xxxxxxxxxx
\documentclass{article}
\usepackage{pgffor}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\foreach \i in {0} {
\the\catcode`_, \the\catcode`:, \the\catcode`\ .
\prg_do_nothing:
}
\ExplSyntaxOff
\end{document}
It works perfectly! Also bye-bye the spaces between the numbers.
This means either expl3
or pgffor
has a bug. Or I really misunderstood something trivial here.
What is happening?
No bug, it’s all working correctly.
In your first example, with \ExplSyntaxOn
inside \foreach
, the tokens in the loop body have their catcodes frozen, so spaces are tokenised as spaces, and \prg_do_nothing:
is tokenised as \prg _do_nothing:
(control sequence \prg
+ two subscript tokens and a bunch of letters).
What seems to have thrown you off tracks is the output of \the\catcode`\_
: the expansion of \the\catcode
will return the current catcode of that character, and not the catcode of that token. Maybe this helps you see more clearly:
xxxxxxxxxx
\def\x{cc: \the\catcode`\_\par \meaning_\par}
\tt
\catcode`\_=8 \x
\catcode`\_=11 \x
\catcode`\_=0 \x
\bye
The example above shows that \the\catcode`\_
outputs the current catcode of \_
, but \meaning_
always outputs subscript character _
because that token is frozen.
In your code, even if \ExplSyntaxOn
changes the catcode inside the loop body, the control sequence \prg_do_nothing:
is already tokenised incorrectly (not as a control sequence).
Your second example then works correctly because the catcodes are changed before TeX sees the tokens, so \prg_do_nothing:
is tokenised correctly as a single control sequence. In fact, you could even do:
xxxxxxxxxx
\ExplSyntaxOn
\foreach \i in {0} {
\ExplSyntaxOff
\the\catcode`_, \the\catcode`:, \the\catcode`\ .
\prg_do_nothing:
}
\ExplSyntaxOff
and the output would be 8,12,10.