or
marmot
latex3
I am a bit mystified by all the \ExplSyntaxOn ... \ExplSyntaxOff codes that appear all over the place, and have a couple of questions related to this.

1. Where can one get a basic introduction in the concepts, syntax and purpose?
2. What is the ultimate purpose of this? If this is to make the usual syntax obsolete, why?
3. How is the performance of this compared to more basic commands?
4. Are some of the things settled by now? What I mean is are there some parts that did not change in the last couple of years, and are guaranteed not to change in the future. In other words, are there packages and/or methods that can be used for publications that go to the arXiv with the guarantee that this can get compiled there, and if someone downloads this in 10 years from now, it will then still run through on an up-to-date LaTeX installation?
5. Are there any concerns that this development may eventually hurt LaTeX in the sense that users who use LaTeX but are not really into coding may no longer be able to fix their own codes, and that these users may switch to other typesetting systems?

Skillmon
I don't really consider myself an expert in this field, but at least I did some
expl3 coding myself in some of my packages, so I might be able to get you
started.

1. The document texdoc expl3 (expl3.pdf) gives an introduction to LaTeX3 on
16 pages. Described are:

- The layers of LaTeX3 (some of which might not be fully implemented). The
basic concept is that there should be different languages to achieve
different things. Programming is done in the programmer layer which is the
expl3-syntax. Atop that programmer layer there should be a designer
layer, which is implemented in expl3, but the language used there will
most likely not be expl3 (to be seen, as far as I know). And the
user-facing layer is the document mark-up.

- The naming conventions/scheme used in expl3's programmer's layer.

- The LaTeX3 approach to expansion. There shouldn't be many \expandafters
(which's equivalent in expl3 is \exp_after:wN) but instead we use
special function variants that expand their argument. Of course, this is
internally implemented using \exp_after:wN, but this makes code much

\exp_after:wN \tl_set:Nn \exp_after:wN \l_tmpa_tl \exp_after:wN { \l_tmpb_tl }
\tl_set:No \l_tmpa_tl { \l_tmpb_tl }

Both lines do the same, but the second is much easier to read.

- The distributions components and a bit about stability (I'll get back to
that later)

- Notes for people writing code for LaTeX2ε using the currently available
LaTeX3 facilities.

- Loadtime options of the expl3 package

- Usage of expl3 in other formats than LaTeX2ε

- Requirements

2. This might be best suited to be answered by Frank Mittelbach or Joseph
Wright, but here are my 2 cents: The ultimate purpose is to create a new
format, which will be called LaTeX3. LaTeX3 aims to be split up in more parts
than LaTeX2ε with a more sensible distinction between different tasks. The
result will be a clean programming language for the internal stuff, a design
language to define how document elements will be formatted, and a language to
mark-up documents, which might not differ from LaTeX2ε's document mark-up at
all or just in some aspects, but (to my knowledge) this isn't completely
worked out yet. So this will make the usual programming syntax of LaTeX2ε and
its internal macros obsolete, but most likely not the document mark-up
syntax.

3. The code focuses on stability over performance, so the performance might not
always be top-notch. That being said, LaTeX3 still has the fastest
space-stripping code I know of and most likely in other parts is pretty
optimized as well. There are some aspects of it that can be made much faster,
and means to do so are evaluated by the team, but speed gain shouldn't affect
stability and predictability (they want document-level code to have
predictable outcome in the code-layer, e.g., macros created with xparse's
\NewDocumentCommand always expands to the same structure, even if it might
be faster to just expand to the code if they don't grab arguments -- but they
have a branch that shortcuts in cases like this to make the overhead
smaller). All in all the performance isn't too bad, compared to what the code
does, after all they implemented something coming close to a general purpose
language in TeX, imho.

4. The code loaded by \usepackage{expl3} is pretty stable. There might be
internal changes in the future, but functions documented in
texdoc interface3 (interface3.pdf) can be considered stable and should
work in years, too. There are experimental functions and modules, but those
are contained in l3experimental (l3benchmark.sty is one of those packages
for example). To give you an idea, \ExplSyntaxOn and \ExplSyntaxOff were
last changed in 2011, some functions were not changed in ages, some were
added kind of recently. In interface3.pdf functions added recently (meaning
in this millennium) or changed are marked containing the date of the last
change. Citing from expl3.pdf:

> Wile expl3 is still experimental, the bundle is now regarded as broadly
> stable. The syntax conventions and functions provided are now ready for
> wider use. There may still be changes to some functions, but these will be
> minor when compared to the scope of expl3.
>
> New modules will be added to the distributed version of expl3 as they
> reach maturity. At present, the expl3 bundle consists of a number of
> modules, most of which are loaded by including the line:
>
> \RequirePackage{expl3}
>
> in a LaTeX2ε package, class or other file. The expl3 modules regarded as
> stable, and therefore suitable for basing real code on, are as follows:
>
> l3basics, l3box, l3clist, l3coffins, l3expan, l3int, l3keys,
> l3msg, l3names, l3prg, l3prop, l3quark, l3seq, l3skip,
> l3tl, l3token

I've changed the layout of that list to remove the scope of the modules.

5. The assumption that users can fix their own codes I'd consider bold.
Nevertheless I'd say no. The aim of all this is to get a stable and solid
platform on which designers and document authors can build. The average user
can't code in LaTeX2ε's internal level, and the internals of LaTeX2ε are
pretty obfuscated by \v@w@lr@pl@c@m@nt (vowel replacement) and the
inconsistent usage of @. That's going to turn a lot better with expl3,
but users need to learn the new conventions if they want to grasp the meaning
of code. But I'd personally say that these conventions are easy to understand
and understanding expl3 code is much easier than LaTeX2ε code once you're
accustomed to it (this is coming from someone who codes more in (La)TeX than
he writes documents with it, so I think I know what I'm saying here). Note
that with expl3 code I don't mean the inner workings of expl3 functions,
but the outcome of the used functions.

There is a not so short video on youtube from TUG conference 2011 of a speech given by Frank Mittelbach about the architecture of LaTeX3: https://www.youtube.com/watch?v=-lr6KEPGLDs

L3 basics
Raaja
arxiv is good ;) but the former part is one that I really hate :D
I have rather similar experiences. We do joint editing of documents (grant proposals etc.) and some are really pushing towards using overleaf. It is all-too clear where this will lead to, if they win eventually we will be using Google Docs. I am really concerned about all this and see arXiv as the one institution which slows down all these unfortunate developments.
It is not about *me* not liking the developments but the huge crowd of LaTeX users who are using it for decades and who have learned just enough to create the documents in the way they want. They have something that they are familiar with and which works. Now it is going to be changed to something that is not at all familiar and where developers publish stuff way before they have thoroughly tested it, and, more importantly, before they have documented it. You need to have quite some background in LaTeX before even the first page of interface3.pdf starts to make sense.
The argument that they spend years already doesn't mean that therefore they must be trustworthy, but you can take that into consideration when you assess the developments of L3. Even if you don't like the developments it is a strong hint that they think it is the right thing to do for LaTeX.
What I meant with that is that publishers earn money based on the results of (La)TeX. And arguably Overleaf doesn't monetize LaTeX but their editor, similar to the way you can spend loads of money for a C or C++ IDE. But Overleaf really isn't the thing we should be discussing here.
This is now becoming too emotional to me. I honestly do not think that the time spent by someone is the only criterion, And I do not know of any other major company that is selling LaTeX, something that is available for free. All I am saying is that these things may cause irreparable damage to LaTeX.
How come you think that people who invested over 20 years of their life to develop and fix all those issues in LaTeX2 now don't put interests of the users first by developing l3 (which is in the making since before LaTeX2ε came out, many of the fixes of LaTeX2ε over LaTeX2.09 were based on ideas from LaTeX3, iirc). You see, L3 is developed by the very same people who developed L2ε. And LaTeX has been used by people to generate profit since the very beginning.
Raaja
But yeah the easiest is to from the beginning motivate the use of TEx in local distribution. I have a very hard time in working with people who use overleaf (esp., when writing a manuscript together, it gives me nightmare)
Raaja
However, often get the reply (now-a-days more frequently) that we have overleaf, why do we need to install. It gives me a nice GUI. Sometimes, I feel very compelled to tell that, then better use paid-documentation systems ;)
Raaja
I often try to emphasise the local installation of TEX distribution.
Raaja
@marmot the simplest way is to forget that overleaf exists. However, now I often see a trend that  -- in the name of collaboration -- people are going/pushed towards using overleaf (this includes universities as well, sadly).
What am I getting at? IMHO LaTeX has the big advantage of being very stable since basically it has been used over decades, and every conceivable issue has surfaced over time, and been fixed. Another big advantage is that one has almost perfect control over the document. These are IMHO the biggest advantages. Both of them get now sacrificed for commercial and other interests. I am not excited about this. Not at all. I wish that those who are running overleaf and the l3 development put the interests of the LaTeX users first. That's all.
Let's not take Overleaf and their rights and wrongs into account. There is not much we can do about that. Can you explain why LaTeX3/expl3 should make developers irreplaceable? Do you have the feeling that most users of Ti*k*Z can solve low-level issues of Ti*k*Z themselves? The situation we currently have with countless packages each using different internal structure and coding style, isn't really any different to the situation with expl3, imho, so I don't understand why a powerful uniform interface to program stuff in should be the "nail in the coffin". I personally consider it a huge development in the correct direction. Sorry, but I'm really trying to understand you here, I just don't really get it, it seems :)
No, this is certainly not the only problem I have. Absolutely not. Short version: I feel that overleaf and expl3 are very much nails in the coffin of LaTeX. They will essentially divide the community into users who have no clue and developers who make themselves irreplaceable. And all the ordinary users will eventually switch to something else.
So the only issue you have is the imperfect documentation, is that correct? Should there be an introduction/tutorial, similar to Ti*k*Z that presents the basic ideas? (the issue is, that once you're accustomed to it, interface3.pdf is close to ideal, I often find that the Ti*k*Z manual is great for absolute beginners, but somewhat over-boarded to quickly find what you need.
A *way* better manual that is written in such a way that a standard user can understand it.
I don't quite understand why you think the current state is a nightmare. What do you think has to be changed for it to be all-good?
At least no one can say that there were no warnings if  the number of TeX users drops significantly.
If everything goes well, one would never see weird syntax in error messages. If there is, it means that some code failed not gracefully in which case the l3 developers can't do anything about this and the people to contact would be the engine developers that they should parse error trace backs and provide such link (unlikely). Do you face many trace backs with expl3 syntax in them? I agree that the documentation could be better, and there currently are efforts to rewrite some of it to make it more exhaustive and understandable, as far as I know. But even if the documentation might be imperfect, I don't share your opinion on the current state.
Not my colleagues (and there are many). They may just drop LaTeX altogether. If you have a chance to communicate with the l3 developers, please tell them that they cannot leave it as is. At the very least whenever there is an error with this syntax, there should be a link to a resource where those who have been using LaTeX for decades can read this, and this resource needs to be well written. Otherwise we will lose *many* users. I am very serious about this. The current state is a nightmare, to say the least.
yes, there is a threshold, but I'd argue that once you got the (rather simple -- that is it's only a naming convention of ordinary TeX syntax with powerful functions) syntax, the actual code is easier to debug/edit/understand than in LaTeX2ε. Again, this means on the code level of expl3, not on the internals of expl3, don't touch that one if you want to keep your sanity.
Yes, it seems we agree on 2. I'd have to know an additional syntax to debug this. This is different from the traditional macros containing  @, where the syntax is the same. I am not implying that any user will understand every hack, but at least there is not the *additional* threshold of an additional syntax.
1. I meant that they focus on that the provided code works for almost every possible input, they always sanitize input (e.g., space-stripping from elements, etc.) and try hard to not accidentally remove braces from arguments which should stay braced. I could create macros which would be considerably faster, but don't handle all the edge cases. For example take a look at the fast \ifempty code I pointed you towards a few days ago. It  is blazingly fast, but not as stable as the expl3 version. expl3 tries to handle every edge case there is. Another good example would be https://tex.stackexchange.com/questions/506038/, the answer provided by Henri does work in the case of the OP, but with a little modification it'll break (I've left a comment). On the other hand my answer (using expl3) will not break because the expl3 implementation of \scantokens handles all the strange edge cases the average user doesn't know about.
Thanks for the answer! There are two points, though. 1. I do not understand what you mean by stability. Are there any stability issues in LaTeX? 2. Yes, I do think that users can fix errors even if they concern macros that have @ in because it is still the same machinery that defines them, i.e. \def or \newcommand.  So, yes, a user can fix those codes, but with expl3 it is hopeless unless you worked through its documentation.