L3 basics - TeX - TopAnswers

TopAnswers TeX

Top Answer

Skillmon

I don't really consider myself an expert in this field, but at least I did some
`expl3` coding myself in some of my packages, so I might be able to get you
started.

1. The document `texdoc expl3` (`expl3.pdf`) gives an introduction to LaTeX3 on
   16 pages. Described are:

    - The layers of LaTeX3 (some of which might not be fully implemented). The
      basic concept is that there should be different languages to achieve
      different things. Programming is done in the programmer layer which is the
      `expl3`-syntax. Atop that programmer layer there should be a designer
      layer, which is implemented in `expl3`, but the language used there will
      most likely not be `expl3` (to be seen, as far as I know). And the
      user-facing layer is the document mark-up.
    
    - The naming conventions/scheme used in `expl3`'s programmer's layer.
    
    - The LaTeX3 approach to expansion. There shouldn't be many `\expandafter`s
      (which's equivalent in `expl3` is `\exp_after:wN`) but instead we use
      special function variants that expand their argument. Of course, this is
      internally implemented using `\exp_after:wN`, but this makes code much
      more readable, compare:
        ```
        \exp_after:wN \tl_set:Nn \exp_after:wN \l_tmpa_tl \exp_after:wN { \l_tmpb_tl }
        \tl_set:No \l_tmpa_tl { \l_tmpb_tl }
        ```
        Both lines do the same, but the second is much easier to read.
        
    - The distributions components and a bit about stability (I'll get back to
      that later)
    
    - Notes for people writing code for LaTeX2ε using the currently available
      LaTeX3 facilities.
    
    - Loadtime options of the `expl3` package
    
    - Usage of `expl3` in other formats than LaTeX2ε

    - Requirements

2. This might be best suited to be answered by Frank Mittelbach or Joseph
   Wright, but here are my 2 cents: The ultimate purpose is to create a new
   format, which will be called LaTeX3. LaTeX3 aims to be split up in more parts
   than LaTeX2ε with a more sensible distinction between different tasks. The
   result will be a clean programming language for the internal stuff, a design
   language to define how document elements will be formatted, and a language to
   mark-up documents, which might not differ from LaTeX2ε's document mark-up at
   all or just in some aspects, but (to my knowledge) this isn't completely
   worked out yet. So this will make the usual programming syntax of LaTeX2ε and
   its internal macros obsolete, but most likely not the document mark-up
   syntax.

3. The code focuses on stability over performance, so the performance might not
   always be top-notch. That being said, LaTeX3 still has the fastest
   space-stripping code I know of and most likely in other parts is pretty
   optimized as well. There are some aspects of it that can be made much faster,
   and means to do so are evaluated by the team, but speed gain shouldn't affect
   stability and predictability (they want document-level code to have
   predictable outcome in the code-layer, e.g., macros created with `xparse`'s
   `\NewDocumentCommand` always expands to the same structure, even if it might
   be faster to just expand to the code if they don't grab arguments -- but they
   have a branch that shortcuts in cases like this to make the overhead
   smaller). All in all the performance isn't too bad, compared to what the code
   does, after all they implemented something coming close to a general purpose
   language in TeX, imho.

4. The code loaded by `\usepackage{expl3}` is pretty stable. There might be
   internal changes in the future, but functions documented in
   `texdoc interface3` (`interface3.pdf`) can be considered stable and should
   work in years, too. There are experimental functions and modules, but those
   are contained in `l3experimental` (`l3benchmark.sty` is one of those packages
   for example). To give you an idea, `\ExplSyntaxOn` and `\ExplSyntaxOff` were
   last changed in 2011, some functions were not changed in ages, some were
   added kind of recently. In `interface3.pdf` functions added recently (meaning
   in this millennium) or changed are marked containing the date of the last
   change. Citing from `expl3.pdf`:

   > Wile `expl3` is still experimental, the bundle is now regarded as broadly
   > stable. The syntax conventions and functions provided are now ready for
   > wider use. There may still be changes to some functions, but these will be
   > minor when compared to the scope of `expl3`.
   >
   > New modules will be added to the distributed version of `expl3` as they
   > reach maturity. At present, the `expl3` bundle consists of a number of
   > modules, most of which are loaded by including the line:
   >
   > `\RequirePackage{expl3}`
   >
   > in a LaTeX2ε package, class or other file. The `expl3` modules regarded as
   > stable, and therefore suitable for basing real code on, are as follows:
   >
   > `l3basics`, `l3box`, `l3clist`, `l3coffins`, `l3expan`, `l3int`, `l3keys`,
   > `l3msg`, `l3names`, `l3prg`, `l3prop`, `l3quark`, `l3seq`, `l3skip`,
   > `l3tl`, `l3token`

   I've changed the layout of that list to remove the scope of the modules.

5. The assumption that users can fix their own codes I'd consider bold.
   Nevertheless I'd say no. The aim of all this is to get a stable and solid
   platform on which designers and document authors can build. The average user
   can't code in LaTeX2ε's internal level, and the internals of LaTeX2ε are
   pretty obfuscated by `\v@w@lr@pl@c@m@nt` (vowel replacement) and the
   inconsistent usage of `@`. That's going to turn a lot better with `expl3`,
   but users need to learn the new conventions if they want to grasp the meaning
   of code. But I'd personally say that these conventions are easy to understand
   and understanding `expl3` code is much easier than LaTeX2ε code once you're
   accustomed to it (this is coming from someone who codes more in (La)TeX than
   he writes documents with it, so I think I know what I'm saying here). Note
   that with `expl3` code I don't mean the inner workings of `expl3` functions,
   but the outcome of the used functions.

There is a not so short video on youtube from TUG conference 2011 of a speech given by Frank Mittelbach about the architecture of LaTeX3: https://www.youtube.com/watch?v=-lr6KEPGLDs

I have rather similar experiences. We do joint editing of documents (grant proposals etc.) and some are really pushing towards using overleaf. It is all-too clear where this will lead to, if they win eventually we will be using Google Docs. I am really concerned about all this and see arXiv as the one institution which slows down all these unfortunate developments.

It is not about *me* not liking the developments but the huge crowd of LaTeX users who are using it for decades and who have learned just enough to create the documents in the way they want. They have something that they are familiar with and which works. Now it is going to be changed to something that is not at all familiar and where developers publish stuff way before they have thoroughly tested it, and, more importantly, before they have documented it. You need to have quite some background in LaTeX before even the first page of interface3.pdf starts to make sense.

What I meant with that is that publishers earn money based on the results of (La)TeX. And arguably Overleaf doesn't monetize LaTeX but their editor, similar to the way you can spend loads of money for a C or C++ IDE. But Overleaf really isn't the thing we should be discussing here.

This is now becoming too emotional to me. I honestly do not think that the time spent by someone is the only criterion, And I do not know of any other major company that is selling LaTeX, something that is available for free. All I am saying is that these things may cause irreparable damage to LaTeX.

How come you think that people who invested over 20 years of their life to develop and fix all those issues in LaTeX2 now don't put interests of the users first by developing l3 (which is in the making since before LaTeX2ε came out, many of the fixes of LaTeX2ε over LaTeX2.09 were based on ideas from LaTeX3, iirc). You see, L3 is developed by the very same people who developed L2ε. And LaTeX has been used by people to generate profit since the very beginning.

But yeah the easiest is to from the beginning motivate the use of TEx in local distribution. I have a very hard time in working with people who use overleaf (esp., when writing a manuscript together, it gives me nightmare)

However, often get the reply (now-a-days more frequently) that we have overleaf, why do we need to install. It gives me a nice GUI. Sometimes, I feel very compelled to tell that, then better use paid-documentation systems ;)

@marmot the simplest way is to forget that overleaf exists. However, now I often see a trend that -- in the name of collaboration -- people are going/pushed towards using overleaf (this includes universities as well, sadly).

What am I getting at? IMHO LaTeX has the big advantage of being very stable since basically it has been used over decades, and every conceivable issue has surfaced over time, and been fixed. Another big advantage is that one has almost perfect control over the document. These are IMHO the biggest advantages. Both of them get now sacrificed for commercial and other interests. I am not excited about this. Not at all. I wish that those who are running overleaf and the l3 development put the interests of the LaTeX users first. That's all.

Let's not take Overleaf and their rights and wrongs into account. There is not much we can do about that. Can you explain why LaTeX3/`expl3` should make developers irreplaceable? Do you have the feeling that most users of Ti*k*Z can solve low-level issues of Ti*k*Z themselves? The situation we currently have with countless packages each using different internal structure and coding style, isn't really any different to the situation with `expl3`, imho, so I don't understand why a powerful uniform interface to program stuff in should be the "nail in the coffin". I personally consider it a huge development in the correct direction. Sorry, but I'm really trying to understand you here, I just don't really get it, it seems :)

No, this is certainly not the only problem I have. Absolutely not. Short version: I feel that overleaf and expl3 are very much nails in the coffin of LaTeX. They will essentially divide the community into users who have no clue and developers who make themselves irreplaceable. And all the ordinary users will eventually switch to something else.

So the only issue you have is the imperfect documentation, is that correct? Should there be an introduction/tutorial, similar to Ti*k*Z that presents the basic ideas? (the issue is, that once you're accustomed to it, `interface3.pdf` is close to ideal, I often find that the Ti*k*Z manual is great for absolute beginners, but somewhat over-boarded to quickly find what you need.

If everything goes well, one would never see weird syntax in error messages. If there is, it means that some code failed not gracefully in which case the l3 developers can't do anything about this and the people to contact would be the engine developers that they should parse error trace backs and provide such link (unlikely). Do you face many trace backs with `expl3` syntax in them? I agree that the documentation could be better, and there currently are efforts to rewrite some of it to make it more exhaustive and understandable, as far as I know. But even if the documentation might be imperfect, I don't share your opinion on the current state.

Not my colleagues (and there are many). They may just drop LaTeX altogether. If you have a chance to communicate with the l3 developers, please tell them that they cannot leave it as is. At the very least whenever there is an error with this syntax, there should be a link to a resource where those who have been using LaTeX for decades can read this, and this resource needs to be well written. Otherwise we will lose *many* users. I am very serious about this. The current state is a nightmare, to say the least.

yes, there is a threshold, but I'd argue that once you got the (rather simple -- that is it's only a naming convention of ordinary TeX syntax with powerful functions) syntax, the actual code is easier to debug/edit/understand than in LaTeX2ε. Again, this means on the code level of `expl3`, not on the internals of `expl3`, don't touch that one if you want to keep your sanity.

Yes, it seems we agree on 2. I'd have to know an additional syntax to debug this. This is different from the traditional macros containing `@`, where the syntax is the same. I am not implying that any user will understand every hack, but at least there is not the *additional* threshold of an additional syntax.

1. I meant that they focus on that the provided code works for almost every possible input, they always sanitize input (e.g., space-stripping from elements, etc.) and try hard to not accidentally remove braces from arguments which should stay braced. I could create macros which would be considerably faster, but don't handle all the edge cases. For example take a look at the fast `\ifempty` code I pointed you towards a few days ago. It is blazingly fast, but not as stable as the `expl3` version. `expl3` tries to handle every edge case there is. Another good example would be https://tex.stackexchange.com/questions/506038/, the answer provided by Henri does work in the case of the OP, but with a little modification it'll break (I've left a comment). On the other hand my answer (using `expl3`) will not break because the `expl3` implementation of `\scantokens` handles all the strange edge cases the average user doesn't know about. 2. `expl3` code is defined by a similar abstraction to `\newcommand` namely with `\cs_new:Nn` or `\cs_new:Npn`, `\cs_new:Nn` is just like `\newcommand`, but instead of specifying a number of arguments `\cs_new:Nn` parses the macro name and knows how much arguments it'll need (e.g., `\cs_new:Nn \marmot_hybernate: { ... }` will not take any arguments, there are no `n` or `N` after the colon, while `\cs_new:Nn \marmot_eat:n { ... }` will take one argument). and `\cs_new:Npn` is just like `\def` with checks whether the macro already exists or not (so we'd use `\cs_new:Npn \marmot_eat:n #1 { ... }`). And I think that guessing what a macro does based on the names used in `expl3` is much easier than guessing based on the names in LaTeX2ε.

Thanks for the answer! There are two points, though. 1. I do not understand what you mean by stability. Are there any stability issues in LaTeX? 2. Yes, I do think that users can fix errors even if they concern macros that have @ in because it is still the same machinery that defines them, i.e. `\def` or `\newcommand`. So, yes, a user can fix those codes, but with expl3 it is hopeless unless you worked through its documentation.

1 Answer