joulev
This returns errors.


\documentclass{article}
\usepackage{pgffor}
\usepackage{expl3}
\begin{document}
\foreach \i in {0} {
\ExplSyntaxOn
\the\catcode_, \the\catcode:, \the\catcode\ .
\prg_do_nothing:
\ExplSyntaxOff
}
\end{document}



![image.png](/image?hash=03cd02a78b835c714d8650de0ccb346bcb313b133409e2f8541a0bf30d803b21)

log
/home/joulev/dev/tmp/tmp.tex:10: Undefined control sequence.
\pgffor@body ...tcode :, \the \catcode \ . \prg
_do_nothing: \ExplSyntaxOff
l.10 }

/home/joulev/dev/tmp/tmp.tex:10: Missing $inserted. <inserted text>$
l.10 }

/home/joulev/dev/tmp/tmp.tex:10: Missing $inserted. <inserted text>$
l.10 }


Here it is very interesting:

* Space has category code 9 (ignored) (as expected after \ExplSyntaxOn). However the output still has spaces. I expect it to be "11,11,9." and not "11,␣11,␣9.", because no ~ is provided.
* _ and : have category code 11 (letter) (still as expected after \ExplSyntaxOn). So \prg_do_nothing: should be a valid control sequence and it should work. It didn't, and the error messages implies _ is still considered "math subscript" character (catcode 8).

It is even more interesting when I put the whole \foreach inside explosive syntax:


\documentclass{article}
\usepackage{pgffor}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\foreach \i in {0} {
\the\catcode_, \the\catcode:, \the\catcode\ .
\prg_do_nothing:
}
\ExplSyntaxOff
\end{document}


![image.png](/image?hash=1105abf8125466efb2509814b9a671c782d0cb7a6ac60110966e14b77223d515)

It works perfectly! Also bye-bye the spaces between the numbers.

This means either expl3 or pgffor has a bug. Or I really misunderstood something trivial here.

What is happening?
Phelype Oleinik
No bug, it's all working correctly.

In your first example, with \ExplSyntaxOn inside \foreach, the tokens in the loop body have their catcodes frozen, so spaces are tokenised as spaces, and \prg_do_nothing: is tokenised as \prg _do_nothing: (control sequence \prg + two subscript tokens and a bunch of letters).

What seems to have thrown you off tracks is the output of \the\catcode\_:  the expansion of \the\catcode will return the _current_ catcode of that character, and not the catcode of that _token_.  Maybe this helps you see more clearly:
tex
\def\x{cc: \the\catcode\_\par \meaning_\par}
\tt
\catcode\_=8  \x
\catcode\_=11 \x
\catcode\_=0  \x
\bye


The example above shows that \the\catcode\_ outputs the current catcode of \_, but \meaning_ always outputs subscript character _ because that token is frozen.

In your code, even if \ExplSyntaxOn changes the catcode inside the loop body, the control sequence \prg_do_nothing: is already tokenised incorrectly (not as a control sequence).

Your second example then works correctly because the catcodes are changed _before_ TeX sees the tokens, so \prg_do_nothing: is tokenised correctly as a single control sequence.  In fact, you could even do:
latex
\ExplSyntaxOn
\foreach \i in {0} {
\ExplSyntaxOff
\the\catcode_, \the\catcode:, \the\catcode\ .
\prg_do_nothing:
}
\ExplSyntaxOff

and the output would be 8,12,10.`

Enter question or answer id or url (and optionally further answer ids/urls from the same question) from

Separate each id/url with a space. No need to list your own answers; they will be imported automatically.