diff options
Diffstat (limited to 'test/netsurf/monkey-test-plans.mdwn')
| -rw-r--r-- | test/netsurf/monkey-test-plans.mdwn | 217 |
1 files changed, 217 insertions, 0 deletions
diff --git a/test/netsurf/monkey-test-plans.mdwn b/test/netsurf/monkey-test-plans.mdwn new file mode 100644 index 0000000..ae16a83 --- /dev/null +++ b/test/netsurf/monkey-test-plans.mdwn @@ -0,0 +1,217 @@ +Test plans for Monkey based NetSurf testing +=========================================== + +Schema +------ + +Top level test mapping must have: + +- Test `title` string. +- Test `group` string. +- Test `steps` sequence (of sequence step mappings). + +Top level test mapping might optionally have: + +- Test `expected-result` string. When not supplied, default is "pass". + +The sequence step mappings must have: + +- Step `action` string. + +Depending on the `action`, there may be other keys in the mapping for that +specific action type. + +### Step action types + +#### `launch` + +Starts NetSurf. + +Optional parameters: + +`args`: sequence of strings (command line arguments). + +#### `quit` + +Exits NetSurf. + + +Behaviour +--------- + +Standard modes of failure (unless `expected-result: fail`): + +- Test exits with non-zero return code. +- NetSurf logs something at ERROR or above. +- Leaked LWC strings on exit. +- TODO: Maybe fail if there are any Valgrind or sanitizer errors at any point? + +Tests are composed of a sequence of actions, which can optionally specify +failure conditions. + + +Examples +-------- + +A test will fail if NetSurf exits with a non-zero return code. + +```yaml +title: start and stop browser +group: basic +steps: +- action: launch +- action: quit +``` + +There are `launch` and `quit` actions. + +```yaml +title: start and stop browser without JS +group: basic +steps: +- action: launch + args: + - enable_javascript=0 +- action: quit +``` + +Use actions to get the browser to do things. + +```yaml +title: resource scheme +group: basic +steps: +- action: launch + language: en +- action: window-new + tag: win1 +- action: navigate + window: win1 + url: resource:does-not-exist +- action: block + conditions: + - window: win1 + status: complete +- action: plot-check + - text-contains: Not found + - text-contains: Error 404 +- action: navigate + window: win1 + url: resource:netsurf.png +- action: block + conditions: + - window: win1 + status: complete +- action: plot-check + - bitmap-count: 1 +- action: window-close + - window: win1 +- action: quit +``` + +Can make multiple windows fetch stuff at the same time, without block +actions. + + +```yaml +title: simultanious page fetches +group: real-world +steps: +- action: launch + language: en +- action: window-new + tag: win1 +- action: window-new + tag: win2 +- action: window-new + tag: win3 +- action: window-new + tag: win4 +- action: navigate + window: win1 + url: http://www.bbc.co.uk/news +- action: navigate + window: win2 + url: http://www.amazon.co.uk/ +- action: navigate + window: win3 + url: http://www.theregister.co.uk/ +- action: navigate + window: win4 + url: http://www.arstechnica.co.uk/ +- action: block + conditions: + - window: *all* + status: complete +- action: window-close + window: win1 +- action: window-close + window: win2 +- action: quit +``` + +Can quit during navigate, to check cleanup in intermediate states. +Note: this requires the test to run multiple times. + +```yaml +title: quitting mid-fetch +group: cleanup +steps: +- action: repeat + min: 0 + step: 50 + name: sleepytimer + actions: + - action: launch + - action: window-new + tag: win1 + - action: navigate + window: win1 + url: http://www.bbc.co.uk/news + - action: sleep-ms + time: sleepytimer + conditions: + - window: win1 + status: complete + breaks: sleepytimer + - action: quit +``` + +Can start and stop timers. + +```yaml +title: cache test +group: performance +steps: +- action: launch + language: en +- action: timer-start + tag: timer1 +- action: window-new + tag: win1 +- action: navigate + window: win1 + url: http://www.bbc.co.uk/news +- action: block + conditions: + - window: win1 + status: complete +- action: timer-stop + timer: timer1 +- action: timer-start + tag: timer2 +- action: window-new + tag: win2 +- action: navigate + window: win2 + url: http://www.bbc.co.uk/news +- action: block + conditions: + - window: win2 + status: complete +- action: timer-stop + timer: timer2 +- action: timer-check + condition: timer2 < timer1 +- action: quit +``` |