This package appears to be old and unmaintained, and it gets pulled into the
go.sum of every single package importing Chroma. Also fnmatch doesnt seem to do
any error checking on the pattern, while the standard package does. Also, the
standard package path/filepath is already imported, so it appears not to cost
anything to change it.
JSON spec doesn't allow comments, but many implementation support them anyway. Having comments is very convenient, for example, when showing code in blog posts and articles.
Huge hardcoded character lists have a cost. In some current lexers, such
as Haskell and JavaScript, this bloats lexer init time to 60-80ms on
some current systems, as opposed to sub-ms for many others.
Replacing them with character classes such as `\p{L}` seems to
eliminate this cost, reducing lexer init time to the norm (around 1ms).
In addition, this significantly reduces and simplifies the code.
The current tests pass, but there may be inaccuracies not covered by
tests. This requires a review.
This change is likely to cause edge case regressions, as the sets of
characters considered "letters" vary between languages. However, Chroma
lexers don't aim to be perfectly accurate. Performance should be just as
much a goal as accuracy. I believe this tradeoff to be justified.
This commit leaves at least two lexers unfixed: Julia and Kotlin.
Judging by the code, they might have the same issue, and should also be
addressed.
This adds support for the Fennel programming language: https://fennel-lang.org
I couldn't find much explanation for what the different lexer rules
meant, so I based it off of Clojure's lexer since the two languages
share a very similar syntax.
I also included a program to generate a list of keywords from Fennel's
own listing, which will make it easier to update in the future.
For named groups that are not given, an Error will be emitted anyway.
This also handles the case when an Emitter for group `0` is provided
or not. Since numbers can also be used for names.
But it might be over-doing, because why would anyone use ByGroupNames
if they wanted to assign a token to the whole match?!
