निरंजन
I am working on a project in which I want to use the pgfparser. Almost everything is working as expected, but there is only one problem. The parser ignores spaces. See the following example.


\documentclass{article}
\usepackage{pgfparser}
\pgfparserdef{myparser}{initial}%
{the letter x}{a}
\pgfparserdef{myparser}{initial}%
{the letter y}{b}
\pgfparserdef{myparser}{initial}%
{the letter z}{c}%
\pgfparserdef{myparser}{initial}%
{the character ;}{\pgfparserswitch{final}}%
\NewDocumentCommand{\mycmd}{ m }{%
\expandafter\pgfparserparse\expandafter{%
\expandafter myparser\expandafter
}#1;%
}

\begin{document}
\mycmd{x y z}
\end{document}


It currently produces abc, whereas I want a b c. Is it possible with the current implementation of the parser?
Skillmon
This answer only shows how one could implement a pgfparser that acts similar to a switch. It requires knowledge about two pgfparser-internals that need to be restored at some point (if we don't do this we might get an undefined control sequence error or undefined behaviour if another parser was run before).

I tried to give some comments in the code, if you don't understand something feel free to ask!


\documentclass{article}

\usepackage{pgfparser}

\makeatletter
\newcommand*\anothercmd{\pgfparserparse{anothercmd}}
% the macro \anothercmd@initiated must be undefined outside of the parser!
\pgfparserdefunknown{anothercmd}{initial}
{%
\let\anothercmd@initiated\@empty
\pgfparserswitch{initiated}%
\pgfparserreinsert
}
\pgfparserset{anothercmd/silent=true}
% throw errors for unknown symbols even if we're silent
\pgfparserdefunknown{anothercmd}{all}
{\PackageError{anothercmd}{Unknown token \meaning\pgfparsertoken'}}
% macros containing brace tricks instead of the normal \bgroup/\egroup
\newcommand*\anothercmd@bgroup{\expandafter{\iffalse}\fi}
\newcommand*\anothercmd@egroup{\iffalse{\fi}}
% group handling, group-levels should be correct inside the parser, this is
% required for the check using \anothercmd@initiated
\pgfparserdef{anothercmd}{all}\begingroup{\begingroup}
\pgfparserdef{anothercmd}{all}\endgroup{\anothercmd@checkend\endgroup}
\pgfparserdef{anothercmd}{all}{\meaning\bgroup}{\anothercmd@bgroup}
\pgfparserdef{anothercmd}{all}{\meaning\egroup}
{\anothercmd@checkend\anothercmd@egroup}
% auxiliary
\newcommand\anothercmd@savedefinition[1]
{\unexpanded{\def#1}{\unexpanded\expandafter{#1}}}
\newcommand*\anothercmd@checkend[1]
{%
% we need to restore the internal state of the parser, because this is
% only stored locally by pgfparser and the group end that should also
% end \anothercmd will remove these definitions (and then we'll get an
% undefined cs error from the final action).
\expanded
{%
\unexpanded{#1}%
\unexpanded{\ifdefined\anothercmd@initiated}%
% use this if you need the parser to remain in the same state across
% group-borders (as long as it's not the final group)
\anothercmd@savedefinition\pgfparserstate
\unexpanded{\else}%
\pgfparserswitch{final}%
\anothercmd@savedefinition\pgfparser@current   % current parser name
\anothercmd@savedefinition\pgfparser@usersname % name the user sees
\unexpanded{\fi}%
}%
}
\pgfparserdef{anothercmd}{initiated}a{x}
\pgfparserdef{anothercmd}{initiated}b{y}
\pgfparserdef{anothercmd}{initiated}c{z}
\pgfparserdef{anothercmd}{initiated}{blank space}{ }
\makeatother

\begin{document}
{\anothercmd abc cba}

\begingroup\anothercmd abc cba\endgroup
\end{document}


As asked for, here is a version that combines the technique shown above and delayed output (I've also changed a few \newcommands to \protected\long\defs -- mostly because I can't remember the LaTeX2e interface to define protected macros, and \NewDocumentCommand isn't meant to be used to define internals).


\documentclass{article}

\usepackage{pgfparser}

\makeatletter
\NewDocumentCommand\textnir{m}{\begingroup\nirshape#1\endgroup}
\NewDocumentCommand\nirshape{}
{%
\ifdefined\nirshape@initiated
\PackageError{nirshape}{Nested use not allowed}%
{%
You somehow achieved that \string\nirshape\space was expanded inside
the parser. This isn't supported.%
}%
\expandafter\@gobbletwo
\else
\global\let\nirshape@outputcontainer\@empty
\fi
\pgfparserparse{nirshape}%
}
\protected\long\def\nirshape@output#1%
{%
\xdef\nirshape@outputcontainer
{\unexpanded\expandafter{\nirshape@outputcontainer\unexpanded{#1}}}%
}
\protected\long\def\nirshape@outputexpanded#1%
{%
\xdef\nirshape@outputcontainer
{\unexpanded\expandafter{\nirshape@outputcontainer#1}}%
}
\providecommand\@gobbletwo[2]{}
% the macro \nirshape@initiated must be undefined outside of the parser!
\pgfparserdefunknown{nirshape}{initial}
{%
\let\nirshape@initiated\@empty
\pgfparserswitch{initiated}%
\pgfparserreinsert
}
\pgfparserset{nirshape/silent=true}
% throw errors for unknown symbols even if we're silent
\pgfparserdefunknown{nirshape}{all}
{\expandafter\nirshape@output\expandafter{\pgfparserletter}}
% macros containing brace tricks instead of the normal \bgroup/\egroup
\newcommand*\nirshape@bgroup{\expandafter{\iffalse}\fi}
\newcommand*\nirshape@egroup{\iffalse{\fi}}
% group handling, group-levels should be correct inside the parser, this is
% required for the check using \nirshape@initiated
\pgfparserdef{nirshape}{all}\begingroup
{\begingroup\nirshape@outputexpanded\begingroup}
\pgfparserdef{nirshape}{all}\endgroup{\nirshape@checkend\endgroup}
\pgfparserdef{nirshape}{all}{\meaning\bgroup}
{\nirshape@bgroup\nirshape@outputexpanded\nirshape@bgroup}
\pgfparserdef{nirshape}{all}{\meaning\egroup}
{\nirshape@checkend\nirshape@egroup}
\newcommand\nirshape@savedefinition[1]
{\unexpanded{\def#1}{\unexpanded\expandafter{#1}}}
\protected\def\nirshape@checkend#1%
{%
% we need to restore the internal state of the parser, because this is
% only stored locally by pgfparser and the group end that should also
% end \nirshape will remove these definitions (and then we'll get an
% undefined cs error from the final action).
\expanded
{%
\unexpanded{#1}%
\unexpanded{\ifdefined\nirshape@initiated}%
% use this if you need the parser to remain in the same state across
% group-borders (as long as it's not the final group)
\nirshape@savedefinition\pgfparserstate
\unexpanded{\nirshape@outputexpanded#1}%
\unexpanded{\else}%
\unexpanded{\pgfparserswitch{final}}%
\nirshape@savedefinition\pgfparser@current   % current parser name
\nirshape@savedefinition\pgfparser@usersname % name the user sees
\unexpanded{\fi}%
}%
}
% the things here don't expand any further, so are safe in @outputexpanded
\pgfparserdef{nirshape}{initiated}a{\nirshape@outputexpanded{x}}
\pgfparserdef{nirshape}{initiated}b{\nirshape@outputexpanded{y}}
\pgfparserdef{nirshape}{initiated}c{\nirshape@outputexpanded{z}}
\pgfparserdef{nirshape}{initiated}{blank space}{\nirshape@outputexpanded{ }}
\pgfparserdeffinal{nirshape}{\expanded{\nirshape@outputcontainer}}
\makeatother

\begin{document}
{\nirshape abc cba}

\begingroup\nirshape abc cba \textbf{abc cba}\endgroup

\textnir{abc cba \textbf{abc cba} anything undefined just passes through}
\end{document}

frougon
I haven't used pgfparser before, but I see two problems in your code:

1. I'm afraid all these uses of \expandafter make no sense given the way you invoke \pgfparserparse.

2. Your parser doesn't specify what to do when a space token is encountered.

Regarding 1), in particular, it is very clear that \expandafter myparser is not something that makes sense: the \expandafter tries to expand the y character token which follows m. Since y is normally not an active character, it is unexpandable; thus, there isn't much point trying to expand it. You can remove the \expandafter calls in your code.

Point 2) is obviously the cause of your trouble. You simply need to tell your parser what to do when a blank space is encountered. :-)


\documentclass{article}
\usepackage{pgfparser}

\pgfparserdef{myparser}{initial}{the letter x}{a}
\pgfparserdef{myparser}{initial}{the letter y}{b}
\pgfparserdef{myparser}{initial}{the letter z}{c}
\pgfparserdef{myparser}{initial}{blank space}{ }
\pgfparserdef{myparser}{initial}{the character ;}{\pgfparserswitch{final}}

\NewDocumentCommand{\mycmd}{ m }{%
\pgfparserparse{myparser}#1;%
}

\begin{document}
\mycmd{x y z}
\end{document}


![image.png](/image?hash=29e181294e96873a3fcf7d4a0655005ac28c4223d7170761d6dbd3c0f11bbe97)

By the way, you can put \expandafter to good use in order to find the magic blank space string:


\documentclass{article}

\begin{document}
X\expandafter\meaning\space X
\end{document}


![image.png](/image?hash=8bcb16c66720c4b6565ba0a5c789d1c9c740f9e2ab9a9fdeb18201b506b6a02a)

Yes, there are **two spaces** after blank space in the expansion of this \meaning command. pgfparser seems to strip them, maybe for user-friendliness of \pgfparserdef.

Why did I mention the *expansion* of \meaning? We can retrieve it using \edef!


\documentclass{article}

\begin{document}
\edef\tmp{\expandafter\meaning\space}
\show\tmp
\end{document}


which produces the following output on the terminal, where you can easily count the two spaces:


> \tmp=macro:
->blank space  .
l.5 \show\tmp


Yet another way to find this blank space  :


\documentclass{article}
\begin{document}

\ExplSyntaxOn
\tl_analysis_show:n { ~ }
\ExplSyntaxOff

\end{document}


This prints the following text to the terminal, where you can again see the two spaces:


The token list contains the tokens:
>    (blank space  ).
`