Huge hardcoded character lists have a cost. In some current lexers, such
as Haskell and JavaScript, this bloats lexer init time to 60-80ms on
some current systems, as opposed to sub-ms for many others.
Replacing them with character classes such as `\p{L}` seems to
eliminate this cost, reducing lexer init time to the norm (around 1ms).
In addition, this significantly reduces and simplifies the code.
The current tests pass, but there may be inaccuracies not covered by
tests. This requires a review.
This change is likely to cause edge case regressions, as the sets of
characters considered "letters" vary between languages. However, Chroma
lexers don't aim to be perfectly accurate. Performance should be just as
much a goal as accuracy. I believe this tradeoff to be justified.
This commit leaves at least two lexers unfixed: Julia and Kotlin.
Judging by the code, they might have the same issue, and should also be
addressed.
This adds support for the Fennel programming language: https://fennel-lang.org
I couldn't find much explanation for what the different lexer rules
meant, so I based it off of Clojure's lexer since the two languages
share a very similar syntax.
I also included a program to generate a list of keywords from Fennel's
own listing, which will make it easier to update in the future.
For named groups that are not given, an Error will be emitted anyway.
This also handles the case when an Emitter for group `0` is provided
or not. Since numbers can also be used for names.
But it might be over-doing, because why would anyone use ByGroupNames
if they wanted to assign a token to the whole match?!
Green and red are the de facto colors used to represent inserted and
deleted lines. This changes the red that was already used for deleted to
use the official Dracula red, getting rid of an unofficial color, and
changes the inserted color from foreground white to green, to make
insertions stand out from headings and such.
- Commands are syntax highlighted as functions
- Command options are highlighted as attributes
- Added more built-ins and operators
- Added Hashbang
- Added and improved syntax highlighting for function, variable and
keywords
- Added test files
