We used a set of documents each capturing a single component of structure e.g. a list, an equation array, an image or a table. The equations used were chosen to cover a range of one and two dimensional layouts e.g. symbols mixed with relations, sub- and super-scripts, matrices, roots or fractions. We transformed each document with each technology and evaluated the results.
We next used Representative sample documents provided by survey respondents as complex inputs to test the identified transform technologies. Often no output was produced; the cause of this was recorded as a constraint and removed to produce a new input document. If output could be produced this was evaluated. We discarded document transform technologies if they were too constrictive or unstable for our purposes or if the results difficult to realistically evaluate. In some cases we identified problems which we felt could be overcome. These were recorded as barriers but proof of concept work rounds were produced and evaluated.
Some of these experiences are recorded in section 5. A full account of the decisions made is outside the scope of an overview document!
We produced integrated test documents formed of structural components which met the recorded constraints then extended these to include a range of mathematical symbols and commands (those available in LaTeX with limited packages on the one hand and in Word on the other). The remaining transform methods were applied simultaneously to these and each output evaluated. We iteratively adjusted the symbol, command, style and working process constraints until all output formats were correct within certain tolerances. These tolerances were recorded as remaining barriers.
Finally, the constraints and methods were documented and a member of staff not involved in the above trialled the process.