We performed our analysis on a dataset from a previous study for which acceleration data were available [16]. The study evaluated a novel autocorrection technique combining touch and language models. 10 participants transcribed phrases from the Enron Mobile Email dataset [20] on a custom soft keyboard, implemented on a Galaxy S3 Mini smartphone running Android 4.0.
Participants entered text while sitting, standing and walking. Error rates in the walking condition were significantly higher than either of the static conditions. However, although accelerometer data were captured in this study, the correction technique did not make use of these. The goal of our gait phase analysis is to identify factors contributing to the observed differences in error rate when walking. Note that we discarded the data for one subject because accelerometer data were incomplete.
Error rates
In the original study, error rates were reported using Character Error Rate (CER), defined as the number of substitutions, transpositions, insertions and deletions needed to transform the transcribed text into the stimulus, divided by the length of the stimulus. Here, however, we are interested primarily in the touch dynamics, rather than the corrections made possible by the language model. Only substitution errors (e.g. ‘thr’ instead of ‘the’) can be directly corrected by a touch model. We handle other errors in the following way:
-
Transpositions (e.g. ‘taht’ instead of ‘that’) are not counted as errors since the correct keys were hit, just in the wrong order.
-
Insertions (e.g. ‘thaat’ instead of ‘that’) are not considered errors if the inserted character(s) are repeats of the previous correct character. In other cases (e.g. ‘thast’) the inserted characters are considered as additional substitution errors.
-
Deletions (e.g ‘tht’ instead of ‘that’) are not considered errors since all successful touches hit the correct keys.
Some potential causes for these errors include: 1) hand-leg synchronization interference with two thumb typing which could cause small timing disturbances which could in turn lead to transpose letters or additional added characters; 2) feedback related issues caused by user movement which could interfere with screen visibility and/or haptic sensations; and 3) cognitive related issues. Additional testing, under more controlled conditions, is needed to positively identify error source(s).
The end result of the above discussion is that we use an alternative definition of error rate in our analysis: we manually mark the intended key for all touches and compute error as the number of substitution errors divided by the length of the stimulus. Our reported error rates are therefore lower than those in the original study.
Typing accuracy
Figure 3 shows the variation of baseline error rates in the three conditions. One-way repeated measures ANOVA showed there is no statistically significant effect between conditions (p = 0.0914, F = 2.79, df = 2). Mean values were 9.24 % (±2.92 %), 6.78 % (±2.12 %) and 9.39 % (±4.59 %) for sitting, standing and walking conditions, respectively. Note that the inclusion of all error types in the original study did lead to statistically significant differences — walking lead to about 5 % more errors overall compared to sitting and standing. This is a potentially interesting result as it might suggest that it is primarily non-substitution errors that increase in frequency when walking, rather than the substitution errors studied here. This might be explained by the divide in attention between the phone and surroundings while walking or one of the other sources described in the previous subsection.
Figure 4 gives an impression of the accuracy over the full keyboard in the three experimental conditions, for Subject 7. The touch variability is aggregated to a standard character key in Fig. 5 across all test subjects. The walking condition appears the most diffuse, followed by sitting. Interestingly the standing touches appear to be the most accurate. This reflects the earlier findings in [16]. It remains unclear why this should be the case.
Gait phase analysis
Figure 2 shows the mean number of taps as a function of inferred x,y,z phase angles, averaged over all subjects. z corresponds to vertical motion, y is the direction of travel for the user, and x is lateral (left/right) movement. Looking at individual subjects tapping distribution against the y-axis inferred phase angle, all subjects apart from 4 and 9 have statistically significant deviations (at the α=0.001 level) from mean tapping distributions, based on a multinomial significance test, with correction for multiple testing performed using the False Discovery Rate (FDR) control approach [21].
The variation in key error rates by phase was not statistically significant, because of the relatively small number of errors. We made a further comparison by tightening the typing accuracy, looking at error rates for a virtual key of the same shape as the original keys but with side lengths reduces to 50 % of the original. At this level error counts were such that Subjects 1, 3, 6 and 7 showed statistically significant gait phase related error rate variation at α=0.05 after compensating for multiple comparisons using the FDR approach.
Inspection of the data suggested that the y axis acceleration was most reliable for inference of the gait phase angle. Figure 6 shows the raw acceleration values in all three axes over a three second window for a single subject, and the gait phase angle as extracted from the y acceleration. More detailed depiction of y axis acceleration and extracted gait phase angle is presented in Fig. 7. The peak in tapping density in bins 3–4 for y axis in Fig. 2 corresponds to −112 to −36 degrees. Additional insight into gait phase dependency of the interaction can be obtained if the number of taps and error rates are examined across the gait phase bins, as depicted in Fig. 8.