Diagrams are an effective means of conveying a wide variety of different sources of information. The automatic generation of diagrams is essential for tasks such as the presentation of multiple views of large scale data sets (e.g. ontology analysis benefits from viewing information revolving around particular relationships). Enabling multiple significant relationships to be clearly presented greatly enhances the utility of visualisations (e.g. consider social network information presented as a diagram with graph edges used to depict the friends' relationship and containing curves to simultaneously indicate overlapping interest groups). The project aims to develop a unified framework for automatic diagram generation, allowing a mixture of features from different diagram types and enhancing control over layout; these theoretical advances are an essential precursor to use in practice. The project will provide significant academic impact (advances in the field, plus interaction between researchers in distinct fields) and industrial impact (improved support for ontology analysts, thereby reducing costs of product development cycles), and societal impact (enhanced communication between researchers and the public via improved visualisation capabilities). The involvement of prominent academic advisors from related fields and the substantial commitment from project partner Nokia will help to ensure the success of the project and assist in achieving long term impact. A major problem is the Generation Problem (GP): given an abstract specification, decide if there is a suitable drawing satisfying the specification and if so, then to automatically produce one. The drawing conventions (constraints on the diagram syntax that must be satisfied) and drawing rules (constraints that are desirable to be satisfied) may vary according to application domain requirements or user preferences. There is a substantial amount of work on the graph-based GP, and the GP for the region-based Euler diagrams has recently been addressed, but there is a serious gap concerning diagrams that intrinsically contain both graph and region based features. Generation techniques for such mixed diagram types are desirable to enhance views of complex information (thereby assisting analysts) by enabling the visualisation of the grouping of items for emphasis, alongside a graph based visualisation of other ontological relationships. Other application areas include visual languages (e.g. diagrammatic logic proof presentation), information visualisation (e.g. geographic information systems, network visualisation), graph drawing and knot theory. Thus use of knot theoretic codes within the Euler diagram setting to enhance generation and layout techniques is a novel approach, taken in conjunction with a graph based generation methodology. The automatic generation of diagrams has considerable potential for use as a means of efficiently and effectively displaying results or data in multiple scientific fields. Providing a single, formal basis will make the generation of diagrams easily accessible to researchers, without the need to reinvent techniques developed in other application areas. In general, communication between different disciplines is often difficult due to subject-specific terminologies and therefore research risks repetition. The development of a framework for automatically generating diagrams, which is applicable in different research areas, is important because it has the potential to stimulate communication and collaboration in different disciplines, as well as between researchers and industrial collaborators, by providing a common language. We will provide the theoretical grounding for software tools, enhancing industrial applicability as well as dissemination possibilities to the wider public and within the scientific community. The long term vision is that the project will set the groundwork for the establishment of a new field of Diagram Generation.

Ontologies are a way of reasoning about data in an efficient manner. Ontologies are increasingly prevalent in a range of applications, including the Semantic Web, medicine and law. The development and maintenance of ontologies are skilled tasks requiring knowledge of logical reasoning and symbolic notations. However, the wide variety of stakeholders for each ontology may not have the necessary skill set to perform ontology engineering effectively. Given the critical systems in which ontologies are used, it is of paramount importance that the ontologies encode exactly the information intended. Ontologies containing errors, called incoherent ontologies, undergo debugging or repair by an ontology engineer. Extant ontology reasoners provide a justification for the incoherence. However, interpreting the justification is a non-trivial and difficult task. Even if the engineer understands the domain of the ontology, for example medicine, and the symbolic notation in which the justification is represented, they could still struggle to debug the ontology. It is especially difficult to debug the ontology without unintentionally removing intended behaviour. This project will use concept diagrams to visualise justifications to reduce the burden on the ontology engineer. Using concept diagrams will help both the understanding of the problem and suggest appropriate repairs to the ontology. The project will provide a number of different visualisations of common bugs in ontologies and empirically test the effectiveness of each. Through this process, the project team will be able to develop effective visual justifications. Using real-world examples of ontologies for data privacy supplied by the project partner HERE (a Nokia company) the visual justifications will then be tested against equivalent symbolic and natural language justifications.

Most PDAs and smart phones have sophisticated graphical interfaces and commonly use small keyboards or styli for input. The range of applications and services for such devices is growing all the time. However, there are problems which make interaction difficult when a user is on the move. Much visual attention is needed to operate many of the applications, which may not be available in mobile contexts. Oulasvirta et al. [29] showed that attention can become very fragmented for users on the move as it must shift between navigating the environment and the device, making interaction hard. Our own research has shown that performance may drop by more than 20% when users are mobile [4]. Another important issue is that most devices require hands to operate many of the applications. These may not be available if the user is carrying bags, holding on to children or operating machinery, for example. The novel aspect of this proposal is to reduce the reliance on graphical displays and hands by investigating gesture input from other locations on the body com-bined with three-dimensional sound for output.Little work has gone into making input and control hands-free for mobile users. Speech recognition is still problematic in such settings due to its high processing requirements and the dynamic audio environments in which devices are used. Much of the research on ges-ture input still uses hands for making the gestures. There is some work on head-based input, often for users with disabilities [26], but little of this has been used in mobile settings. Our own previous work has begun to examine head pointing and showed that it might be a useful way to point and select on the move [3].Many other body locations could be useful for subtle and discreet input whilst mobile (e.g., users walking or sitting on a bumpy train). For example, wrist rotation has potential for controlling a radial menu as the wrist can be rotated to move a pointer across the menu. It is unobtrusive and could be tracked using the same sensor used for hand pointing gestures (in a watch for exam-ple). Small changes in gait are also a possibility for in-teraction. In previous work [12] we extracted gait in-formation from an accelerometer on a PDA to look at usability errors. We can adapt this technique so that users could slightly change the timing of a step to make input. There has been no systematic study of the differ-ent input possibilities across the body. We will develop a novel testing methodology using a Fitts' law analysis along with more subjective measures to find out which body locations are most useful for input on the move.Output is also a problem due to the load on visual atten-tion when users are mobile. We and others have begun to look at the use of spatialised audio cues for output when mobile as an alternative or complement to graph-ics [1, 6] [19, 32]. Many of these use very simple 3D audio displays, but, with careful design, spatial audio could provide a much richer display space. Our Audio-Clouds project built some foundations for 3D audio interactions, investigating basic pointing behaviour, target size and separation [1,3]. We need to now take this work forward and develop more sophisticated inter-actions. Key aspects here are to develop the use of ego-centric (fixed to the user) and exocentric (fixed to the world) displays, and how they can be combined to cre-ate a rich 3D display space for interaction.The final key part of this project is to create compelling applications which combine the best of the audio and gestures. We can then test these with users in more realistic settings over longer time periods to fine-tune how these interactions work in the real world.

The proposed research will develop the first techniques for recognizing sketches of Euler diagrams drawn by users. The proposal is for Dr Plimmer to travel to the UK team for 1 month as a Visiting Researcher and for Dr Stapleton to travel to New Zealand for one week. The research visits will facilitate the exchange of knowledge between experts in automated Euler diagram drawing (the UK team) and sketch recognition (Dr Plimmer, Auckland, NZ). This will allow an important exchange of information about relevant drawing and sketching techniques and result in further, significant, collaboration.A natural creation method for Euler diagrams is using a pen but no intelligent tool support exists for this mode of entry. Euler diagrams are a popular and frequently used visualization technique; in part, this popularity serves to motivate our selection of them for the proposed research, since the results are likely to have significant impact. Moreover, Euler diagrams form the basis of more expressive notations, built by augmenting them with graphs or arrows (or both), for instance. Example notations include spider diagrams, Euler/Venn diagrams, Venn-II diagrams, constraint diagrams, and ontology diagrams. The research results of the project provide a basis for developing sketch recognition tools for these more expressive notations. Extending the research results to these notations will be the subject of the future collaborations, between Dr Plimmer and the UK team, that will build on this project.

Ontologies are a way of reasoning about data in an efficient manner. Ontologies are increasingly prevalent in a range of applications, including the Semantic Web, medicine and law. The development and maintenance of ontologies are skilled tasks requiring knowledge of logical reasoning and symbolic notations. However, the wide variety of stakeholders for each ontology may not have the necessary skill set to perform ontology engineering effectively. Given the critical systems in which ontologies are used, it is of paramount importance that the ontologies encode exactly the information intended. Ontologies containing errors, called incoherent ontologies, undergo debugging or repair by an ontology engineer. Extant ontology reasoners provide a justification for the incoherence. However, interpreting the justification is a non-trivial and difficult task. Even if the engineer understands the domain of the ontology, for example medicine, and the symbolic notation in which the justification is represented, they could still struggle to debug the ontology. It is especially difficult to debug the ontology without unintentionally removing intended behaviour. This project will use concept diagrams to visualise justifications to reduce the burden on the ontology engineer. Using concept diagrams will help both the understanding of the problem and suggest appropriate repairs to the ontology. The project will provide a number of different visualisations of common bugs in ontologies and empirically test the effectiveness of each. Through this process, the project team will be able to develop effective visual justifications. Using real-world examples of ontologies for data privacy supplied by the project partner HERE (a Nokia company) the visual justifications will then be tested against equivalent symbolic and natural language justifications.