Jian Zhao, Ricardo Jota, Daniel Wigdor, Ravin Balakrishnan. Augmenting Mobile Phone Interaction with Face-Engaged Gestures. arXiv:1610.00214, 2016.

Abstract: The movement of a user's face, easily detected by a smartphone's front camera, is an underexploited input modality for mobile interactions. We introduce three sets of face-engaged interaction techniques for augmenting the traditional mobile inputs, which leverages the combination of the head movements with touch gestures and device motions, all sensed via the phone's built-in sensors. We systemically present the space of design considerations for mobile interactions using one or more of the three input modalities (i.e., touch, motion, and head). The additional affordances of the proposed techniques expand the mobile interaction vocabulary, and can facilitate unique usage scenarios such as one-hand or touch-free interaction. An initial evaluation was conducted and users had positive reactions to the new techniques, indicating the promise of an intuitive and convenient user experience.

Ji Wang, Jian Zhao, Sheng Guo, Chris North. Clustered Layout Word Cloud for User Generated Review. Yelp Dataset Challenge (Grand Prize Winner), 2013.

Jian Zhao. A Particle Filter Based Approach of Visualizing Time-varying Volume. IEEE Symposium on Large-Scale Data Analysis and Visualization, 2012.

Abstract: Extracting and presenting essential information of time-varying volumetric data is critical in many fields of sciences. This paper introduces a novel approach of identifying important aspects of the dataset under the particle filter framework in computer vision. With the view of time-varying volumes as dynamic voxels moving along time, an algorithm for computing the 3D voxel transition curves is derived. Based on the curves which characterize the local data temporal behavior, this paper also introduces several post-processing techniques to visualize important features such as curve clusters by k-means and curve variations computed from curve gradients.

Jian Zhao, R. William Soukoreff, Ravin Balakrishnan. A Model of Multi-touch Manipulation. Proceedings of the 2nd annual Grand Conference, 2011.

Abstract: As touch-sensitive devices become increasingly popular, fundamentally understanding the human performances of multi-touch gestures is critical. However, there is currently no mathematical model for interpreting such gestures. In this paper, a novel model of multi-touch interaction is derived by combining the Mahalanobis distance metric and Fitts' law. The model describes the time required to complete an object manipulation task that includes translocation, rotation, and scaling. Empirical data is reported that validates the new model (R2>0.9). Linear relationship between the difficulty and time elapsed is revealed indicating that the model can provide guidelines for interface designers for empirically comparing gestures and devices.

Jian Zhao. Interactive Visual Data Exploration: A Multi-Focus Approach. Department of Computer Science, University of Toronto, 2015.

Abstract: Recently, the amount of digital information available in the world has been growing at a tremendous rate. This huge, heterogeneous, and complicated data that we are continuously generating could be an incredible resource for us to seek insights and make informed decisions. For this knowledge extraction to be efficient, visual exploration of data is demanded in addition to fully automatic methods, because visual exploration can integrate the creativity, flexibility, and general experience of the human user into the sense-making process through interaction and visualization techniques.

Due to the scale and complexity of data, robust conclusions are usually formed by coordinating many sub-regions in an information space, which leads to the approach of multi-focus visual exploration that allows browsing different data segments with multiple views and perspectives simultaneously. While prior research has proposed a myriad of information visualization techniques, there still lacks comprehensive understanding about how visual exploration can be facilitated by multi-focus interactive visualizations. This dissertation investigates issues and techniques of multi-focus visual exploration through five design studies, touching various types of data in a range of application domains.

The first two design studies address the exploration of numerical data values. KronoMiner presents a multi-purpose visual tool for exploring time-series based on a dynamic radial hierarchy; and the ChronoLenses system supports exploratory visual analysis of time-series by allowing users to progressively construct advanced analytical pipelines. The third design study focuses on the exploration of logical data structures, and presents DAViewer that facilitates computational linguistics researchers to explore and compare rhetorical trees. The last two design studies consider the exploration of heterogeneous data attributes (or facets). TimeSlice facilitates the browsing of multi-faceted events timelines by organizing visual queries in a tree structure; and PivotSlice aids the mining of relationships in multi-attributed networks through a dynamic subdivision of data with customized semantics.

This dissertation ends with critical reflections and generalizations of the experiences obtained from the case studies. High-level design considerations, conceptual models, and visualization theories are distilled to inform researchers and practitioners in information visualization for devising effective multi-focus visual interfaces.

Jian Zhao, Michael Glueck, Azam Khan, Simon Breslay. Techniques For Mixed-Initiative Visualization of Data. Filed in 2017.

Jian Zhao, Michael Glueck, Azam Khan. Node Centric Analysis of Dynamic Networks. Filed in 2016.

Mira Dontcheva, Jian Zhao, Aaron Hertzmann, Allan Wilson, Zhicheng Liu. Providing Visualizations of Event Sequence Data. Filed in 2015.

Liang Gou, Fei Wang, Jian Zhao, Michelle Zhou. Personal Emotion State Monitoring from Social Media. US Patent 20150213002 A1, 2015.

Jian Zhao, Steven Drucker, Danyel Fisher, Donald Brinkman. Relational Rendering of Multi-Faceted Data. US Patent 20130194294 A1, 2013.