Hi !
Here's a lexer for IBM RPG !
You can learn about this language here :
- https://en.wikipedia.org/wiki/IBM_RPG
- https://www.ibm.com/docs/en/i/7.6.0?topic=languages-rpg
For legacy purposes, the language can be written in two different forms,
the fixed format (looks like a punched card, expects specific characters
to be at certain columns in source code) and the free format (gets rid
of fixed format, and looks like modern languages as Python).
This lexer only handles the free format, I may open a pull request later
to implement the fixed format syntax.
There are some RPG repositories available on GitHub, you can search
using "language:RPGLE" to see them.
The language is old but still maintained and even continues to evolve
nowadays.
The lexer is based on the latest version at this day, RPG 7.6.
Are the MIME types in the XML files important ? I don't think this
language has one (and I didn't know there were MIME types for
programming languages), so I made one which looks like what I saw in
other lexers XML files.
And the language can have embedded SQL statements in the source code, so
I reused the SQL lexer. Please tell me if there's a better way to do it.
The language is rather tied to IBM environment, so if you're interested,
here's how to try it :
- https://pub400.com/ provides free accounts to an IBM machine to try
and discover it
-
https://www.nicklitten.com/use-a-5250-green-screen-terminal-in-visual-studio/
to get a terminal to access pub400 machine and run code
-
https://www.nicklitten.com/connecting-to-pub400-with-vs-code-for-ibm-i/
shows how to code RPG in VSCode and compile it
I noticed that the current syntax highlighting generator uses an older
color palette that doesn’t reflect GitHub’s latest light theme, which
now features lighter, pastel tones for improved readability and
accessibility. I’ve updated the color scheme to match the newer colors.
I was wondering if it makes sense to adopt these changes or, if
maintaining the current color scheme is essential, whether it would be
better to publish this as a separate variant under a different name.
## Example
### Before
### After
Right now there is no value for NameProperty making .chroma .py to be empty.
Because of that there are cases in which the color is wrongly used because it's not overwritten (for eg: when trying to use a light theme as the main one, and Nord under prefers-color-scheme: dark .
This sets the color to match the one from the theme preview that is applied for the methods
Right now there is no value for NameOther making .chroma .nx to be empty.
Because of that there are cases in which the color is wrongly used because it's not overwritten (for eg: when trying to use a light theme as the main one, and Nord under prefers-color-scheme: dark .
This MR sets the color to match the one from the theme preview.
This cleans up the API in general, removing a bunch of deprecated stuff,
cleaning up circular imports, etc.
But the biggest change is switching to an optional XML format for the
regex lexer.
Having lexers defined only in Go is not ideal for a couple of reasons.
Firstly, it impedes a significant portion of contributors who use Chroma
in Hugo, but don't know Go. Secondly, it bloats the binary size of any
project that imports Chroma.
Why XML? YAML is an abomination and JSON is not human editable. XML
also compresses very well (eg. Go template lexer XML compresses from
3239 bytes to 718).
Why a new syntax format? All major existing formats rely on the
Oniguruma regex engine, which is extremely complex and for which there
is no Go port.
Why not earlier? Prior to the existence of fs.FS this was not a viable
option.
Benchmarks:
$ hyperfine --warmup 3 \
'./chroma.master --version' \
'./chroma.xml-pre-opt --version' \
'./chroma.xml --version'
Benchmark 1: ./chroma.master --version
Time (mean ± σ): 5.3 ms ± 0.5 ms [User: 3.6 ms, System: 1.4 ms]
Range (min … max): 4.2 ms … 6.6 ms 233 runs
Benchmark 2: ./chroma.xml-pre-opt --version
Time (mean ± σ): 50.6 ms ± 0.5 ms [User: 52.4 ms, System: 3.6 ms]
Range (min … max): 49.2 ms … 51.5 ms 51 runs
Benchmark 3: ./chroma.xml --version
Time (mean ± σ): 6.9 ms ± 1.1 ms [User: 5.1 ms, System: 1.5 ms]
Range (min … max): 5.7 ms … 19.9 ms 196 runs
Summary
'./chroma.master --version' ran
1.30 ± 0.23 times faster than './chroma.xml --version'
9.56 ± 0.83 times faster than './chroma.xml-pre-opt --version'
A slight increase in init time, but I think this is okay given the
increase in flexibility.
And binary size difference:
$ du -h lexers.test*
$ du -sh chroma* 951371ms
8.8M chroma.master
7.8M chroma.xml
7.8M chroma.xml-pre-opt
Benchmarks:
$ hyperfine --warmup 3 \
'./chroma.master --version' \
'./chroma.xml-pre-opt --version' \
'./chroma.xml --version'
Benchmark 1: ./chroma.master --version
Time (mean ± σ): 5.3 ms ± 0.5 ms [User: 3.6 ms, System: 1.4 ms]
Range (min … max): 4.2 ms … 6.6 ms 233 runs
Benchmark 2: ./chroma.xml-pre-opt --version
Time (mean ± σ): 50.6 ms ± 0.5 ms [User: 52.4 ms, System: 3.6 ms]
Range (min … max): 49.2 ms … 51.5 ms 51 runs
Benchmark 3: ./chroma.xml --version
Time (mean ± σ): 6.9 ms ± 1.1 ms [User: 5.1 ms, System: 1.5 ms]
Range (min … max): 5.7 ms … 19.9 ms 196 runs
Summary
'./chroma.master --version' ran
1.30 ± 0.23 times faster than './chroma.xml --version'
9.56 ± 0.83 times faster than './chroma.xml-pre-opt --version'
Incompatible changes:
- (*RegexLexer).SetAnalyser: changed from func(func(text string) float32) *RegexLexer to func(func(text string) float32) Lexer
- (*TokenType).UnmarshalJSON: removed
- Lexer.AnalyseText: added
- Lexer.SetAnalyser: added
- Lexer.SetRegistry: added
- MustNewLazyLexer: removed
- MustNewLexer: changed from func(*Config, Rules) *RegexLexer to func(*Config, func() Rules) *RegexLexer
- Mutators: changed from func(...Mutator) MutatorFunc to func(...Mutator) Mutator
- NewLazyLexer: removed
- NewLexer: changed from func(*Config, Rules) (*RegexLexer, error) to func(*Config, func() Rules) (*RegexLexer, error)
- Pop: changed from func(int) MutatorFunc to func(int) Mutator
- Push: changed from func(...string) MutatorFunc to func(...string) Mutator
- TokenType.MarshalJSON: removed
- Using: changed from func(Lexer) Emitter to func(string) Emitter
- UsingByGroup: changed from func(func(string) Lexer, int, int, ...Emitter) Emitter to func(int, int, ...Emitter) Emitter
Green and red are the de facto colors used to represent inserted and
deleted lines. This changes the red that was already used for deleted to
use the official Dracula red, getting rid of an unofficial color, and
changes the inserted color from foreground white to green, to make
insertions stand out from headings and such.
'l' in gl is for under(l)ine, as the "gu" class is taken by
GenericSubheading.
- Rules for GenericUnderline are added to all the styles
- Make "Underline" style insert "text-decoration: underline" in CSS.
Fixes https://github.com/alecthomas/chroma/issues/159.
