a56e228d32
https://github.com/alecthomas/chroma/issues/425 shows that generics are not correctly identified in general, but they are rather being treated as JSX elements. I proposed a simple solution in the comments by adding a space between `<` and the word next to it, but I believe most people will either not find the solution or some of them will find it rather unappealing. For this reason, I made the JSX rules recursive and added a `","` `Punctuation` token inside so that there can be a number of generics used, as well as allowing nested generics. While I am not really fond of this hack, given that generics are already treated as JSX elements, I think this is a fair and easy enough solution for most cases. #### Before <img width="359" alt="Screenshot 2024-09-20 at 9 28 05 PM" src="https://github.com/user-attachments/assets/b03c2c8a-3278-438b-8803-00eb62cc4a17"> #### With spacing solution <img width="392" alt="Screenshot 2024-09-20 at 9 30 13 PM" src="https://github.com/user-attachments/assets/89289476-c92a-41df-b893-5ab289fa96aa"> #### With recursive JSX and `","` `Punctuation` token <img width="362" alt="Screenshot 2024-09-20 at 9 55 11 PM" src="https://github.com/user-attachments/assets/d77d892e-667d-4fb4-93cf-8227d5bd4b17">
Chroma lexers
All lexers in Chroma should now be defined in XML unless they require custom code.
Lexer tests
The tests in this directory feed a known input testdata/<name>.actual into the parser for <name> and check
that its output matches <name>.expected.
It is also possible to perform several tests on a same parser <name>, by placing know inputs *.actual into a
directory testdata/<name>/.
Running the tests
Run the tests as normal:
go test ./lexers
Update existing tests
When you add a new test data file (*.actual), you need to regenerate all tests. That's how Chroma creates the *.expected test file based on the corresponding lexer.
To regenerate all tests, type in your terminal:
RECORD=true go test ./lexers
This first sets the RECORD environment variable to true. Then it runs go test on the ./lexers directory of the Chroma project.
(That environment variable tells Chroma it needs to output test data. After running go test ./lexers you can remove or reset that variable.)
Windows users
Windows users will find that the RECORD=true go test ./lexers command fails in both the standard command prompt terminal and in PowerShell.
Instead we have to perform both steps separately:
- Set the
RECORDenvironment variable totrue.- In the regular command prompt window, the
setcommand sets an environment variable for the current session:set RECORD=true. See this page for more. - In PowerShell, you can use the
$env:RECORD = 'true'command for that. See this article for more. - You can also make a persistent environment variable by hand in the Windows computer settings. See this article for how.
- In the regular command prompt window, the
- When the environment variable is set, run
go test ./lexers.
Chroma will now regenerate the test files and print its results to the console window.