blob: 7c54c768a83999190b36205930f6e635bb59bd85 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
|
Test plans for Monkey based NetSurf testing
===========================================
Schema
------
Top level test mapping must have:
- Test `title` string.
- Test `group` string.
- Test `steps` sequence (of sequence step mappings).
Top level test mapping might optionally have:
- Test `expected-result` string. When not supplied, default is "pass".
The sequence step mappings must have:
- Step `action` string.
Depending on the `action`, there may be other keys in the mapping for that
specific action type.
### Step action types
#### `launch`
Starts NetSurf.
Optional parameters:
`args`: sequence of strings (command line arguments).
#### `quit`
Exits NetSurf.
Behaviour
---------
Standard modes of failure (unless `expected-result: fail`):
- Test exits with non-zero return code.
- NetSurf logs something at ERROR or above.
- Leaked LWC strings on exit.
- TODO: Maybe fail if there are any Valgrind or sanitizer errors at any point?
Tests are composed of a sequence of actions, which can optionally specify
failure conditions.
Examples
--------
A test will fail if NetSurf exits with a non-zero return code.
[[!format yaml """
title: start and stop browser
group: basic
steps:
- action: launch
- action: quit
"""]]
There are `launch` and `quit` actions.
[[!format yaml """
title: start and stop browser without JS
group: basic
steps:
- action: launch
args:
- enable_javascript=0
- action: quit
"""]]
Use actions to get the browser to do things.
[[!format yaml """
title: resource scheme
group: basic
steps:
- action: launch
language: en
- action: window-new
tag: win1
- action: navigate
window: win1
url: resource:does-not-exist
- action: block
conditions:
- window: win1
status: complete
- action: plot-check
- text-contains: Not found
- text-contains: Error 404
- action: navigate
window: win1
url: resource:netsurf.png
- action: block
conditions:
- window: win1
status: complete
- action: plot-check
- bitmap-count: 1
- action: window-close
- window: win1
- action: quit
"""]]
Can make multiple windows fetch stuff at the same time, without block
actions.
[[!format yaml """
title: simultanious page fetches
group: real-world
steps:
- action: launch
language: en
- action: window-new
tag: win1
- action: window-new
tag: win2
- action: window-new
tag: win3
- action: window-new
tag: win4
- action: navigate
window: win1
url: http://www.bbc.co.uk/news
- action: navigate
window: win2
url: http://www.amazon.co.uk/
- action: navigate
window: win3
url: http://www.theregister.co.uk/
- action: navigate
window: win4
url: http://www.arstechnica.co.uk/
- action: block
conditions:
- window: *all*
status: complete
- action: window-close
window: win1
- action: window-close
window: win2
- action: quit
"""]]
Can quit during navigate, to check cleanup in intermediate states.
Note: this requires the test to run multiple times.
[[!format yaml """
title: quitting mid-fetch
group: cleanup
steps:
- action: repeat
min: 0
step: 50
name: sleepytimer
actions:
- action: launch
- action: window-new
tag: win1
- action: navigate
window: win1
url: http://www.bbc.co.uk/news
- action: sleep-ms
time: sleepytimer
conditions:
- window: win1
status: complete
breaks: sleepytimer
- action: quit
"""]]
Can start and stop timers.
[[!format yaml """
title: cache test
group: performance
steps:
- action: launch
language: en
- action: timer-start
tag: timer1
- action: window-new
tag: win1
- action: navigate
window: win1
url: http://www.bbc.co.uk/news
- action: block
conditions:
- window: win1
status: complete
- action: timer-stop
timer: timer1
- action: timer-start
tag: timer2
- action: window-new
tag: win2
- action: navigate
window: win2
url: http://www.bbc.co.uk/news
- action: block
conditions:
- window: win2
status: complete
- action: timer-stop
timer: timer2
- action: timer-check
condition: timer2 < timer1
- action: quit
"""]]
|
