Compliance Criteria
The current defintion of compliance for 1.0 Candidate Release is defined below. One of the formal criteria for moving from Candidate Release to Full Release is that a full compliance suite must be available and have been approved by the BONDI steering comittee.
BONDI 1.0 compliance is applicable to webruntimes only. Specifically BONDI enhanced browsers and/or BONDI enhanced widget runtimes. Where BONDI enhancement means:
- Must support the BONDI defined developer facing APIs
- Must support the BONDI defined security model
BONDI compliance is measured against:
- Support of BONDI requirements
- Adherence to BONDI defined specifications
- Satisfaction of BONDI defined test and sample scripts
Requirements
A BONDI compliant runtime must be able to claim compliance against each requirement scoped for the 1.0 CR release defined in the API document and the Architecture and Security documents.
Specifications
The developer facing APIs supported by a BONDI compliant runtime must precisely conform to the specification which may be found in the Doxygen generated Javascript specification.
A BONDI runtime must be able to parse and correctly interpret an XML policy document which conforms to the BONDI security policy DTD.
Test and Sample scripts
A BONDI compliant runtime must be able to correctly run the following test scripts:
A BONDI compliant widget runtime must be able to render the following sample widgets:
A BONDI compliant runtime must be able to correctly interpret and police the following example security policies:
Browser and Widget runtime differences
The following essential differences apply to browser runtimes, as opposed to widget runtimes:
- A browser runtime does not have to support certificate identity models in the security framework
- A browser runtime does not have to support the UI API
BONDI Test Framework Tutorial
The test framework comprises several JavaScript files:
- runner.js - Test framework core
- assert.js - Various assert methods
- format_div.js - Formats output in DIVs. The framework is extendible to support formatting output to any resource (e.g. file, server etc).
- util.js - Utilities to determine JavaScript types
- bondi_loader.js - Loads WebVM into the bondi namespace. Defines the bondi.load API.
Background
The structure of the test framework consists of a test
suite that contains one or
more test cases that consist of one or more tests.
suite - 1:n - test case - 1:m - test
Thus, executing a test suite means that n * m tests will run.
Getting quickly started
Run and study the example.
Notice how the test references the example test suite.
You need a subversion login to the BONDI subversion
testFramework. Here
you edit and commit test cases which are automatically published to
http://tests.bondi.omtp.org/ with a subversion commit hook.
Positive tests
To write positive tests, use the assert helpers in assert.js to verify that
the test results are as you expect. Any exceptions that occur in the course of
running the test function will result in the test failing as they are not
caught.
Negative tests
To write negative tests, use assertError to verify exceptions like in this example.
- A test will pass if no exceptions are thrown out of the test function
- To fail a test, allow (or throw) an exception
- To create a negative test, catch expected exceptions and explicitly throw an exception if a test 'fails'
- You can use the assert methods in assert.js to help document the tests and throw exceptions
Testing Asynchronous Methods
The test framework supports testing asynchronous functions.
When testing asynchronous methods it is necessary to 'wrap' the asynchronous
callback method(s) in a call to the test framework method runTest.
Please see example 4.
Note, when testing asynchronous tests you have to indicate the end of a test
case in your asynchronous callbacks through calling the endAsync() framework
function. If you call this after running the runTestCase function, the test
case might not have exited already, because the asynchronous callbacks did not
happened yet.
Testing policies and permission checks
A special test build of WebVM has been made
availible in order to
verify policy definitions.
Here is an
example
that uses the test framework's assertPolicy API.
Configuration file
config.js is used typically for loading respective BONDI
APIs. It could also be used for sharing
configuration across several tests.
Running tests
To run your tests, execute the test HTML in the
Pocket IE browser on your Windows Mobile device with the latest Bondi reference
implementation installed. In future one HTML start file running all the test
cases will be more useful.
In cases where an API is dependent on the Widget runtime, such as
ui or
appconfig,
a simple widget start file can be constructed. See the test widget
template
for an example.
General guidance
- Coverage: every API member/method and property should be exercised by the tests
- JSlint your Javascript to help avoid errors.
- Input parameters: have we tested the invalid inputs, boundary cases, and unusual parameter combinations ?
- Errors and exceptions: have we tested that exceptions/errors are generated in the expected places
- API specific security policing: where the API is responsible for policing access to functions, this must be tested
- Keep consistency! Notice how tasklist.js maps to tasklist class API.
- If you find a bug, comment that in the code!
- Keep test cases indepedent and not reliant on previous test cases.
- In Pocket IE Javascript runtime, commas matter. Be very strict with commas, which JSlint might ignore