The official playground quite often provides a timeout error as in
https://github.com/google/comprehensive-rust/issues/2549
This is mocking/overriding the response from the playground for success
and failure messages to avoid the server provided timeouts. As far as I
see most responses from the playground are delivered below 10 seconds
(but varies a lot...)
There still might be timeouts and I know of the following knobs that can
be tuned
- wdio waits for 10 seconds until it aborts the current attemp (e.g. to
wait for the text to be as expected)
347de61d13/tests/wdio.conf.ts (L98)
- another timeout is the 15 second timeout for fetching the result from
the playground - this is not overriden by the mock (currently, pending
https://github.com/webdriverio/webdriverio/issues/14090 which would
allow for an immediate mock response without waiting for the remote
endpoint)
347de61d13/theme/book.js (L31)
Implement two tests for the rust playground:
- successful run of the hello world example with the hello world message
in stdout and a hidden stderr
- on purpose compilation error is shown in stderr and "No output" in
stdout
This is a first draft for a test framework for testing the JS part of
the book that is discussed in #2462
It is using [webdriverIO](https://webdriver.io/) and the webdriverIO
[Expect API](https://webdriver.io/docs/api/expect-webdriverio/) in
combination with [Mocha](https://mochajs.org/). WebdriverIO is taking
care of accessing the webpage with a real browser and can access the
state of the page so behavior can be asserted.
Currently only a small test for the speaker-notes implementation demos
the functionality.
The [Static Server
Service](https://webdriver.io/docs/static-server-service/) is used to
serve the book in a way that the test runner can access it.
A CI integration can look like https://webdriver.io/docs/githubactions/
and is implemented with a headless setup. In CI it uses the language
variable to set environment variable that configures where the built
book should be mounted from
---------
Co-authored-by: Martin Geisler <martin@geisler.net>
