HP Hewlett Packard Gesture Keyboard User Manual

Gesture Keyboard - User centered design of a unique input device  
for Indic Scripts♦  
Ashish Krishna1, Rahul Ajmera1, Sandesh Halarnkar, Prashant Pandit  
HP Laboratories India  
HPL-2005-56  
March 18, 2005*  
devnagari text  
entry, gesture  
keyboard (GKB),  
matras, modifier,  
syllable, IMEs  
Indic text input presents a unique challenge in the field of keyboard  
design because of the number of standalone characters, conjuncts, matras,  
and symbols. It poses a huge challenge to the HCI practitioners to  
accommodate the humongous array of Devnagari characters in a  
keyboard. Solutions for input in Devnagari scripts using a physical  
keyboard have been around for a while but none of them have emerged as  
a standard mechanism due to several usability related concerns. In this  
paper we have described the design of a new gesture based keyboard,  
which has a dual input mode 1. Gesturing 2. Tapping. The "gesture  
keyboard" concept is based on partial hand writing recognition as well as  
touch-typing paradigms. Conceptually, it is a stylus sensitive keypad that  
supports tapping for getting the base consonants and some symbols,  
while it also recognizes handwritten matras as per handwriting  
recognition methods. This paper also reports the findings of the user  
study, which was carried out to find the optimum sizes of the keys and  
layouts.  
* Internal Accession Date Only  
HCI International 2005, 22-27 July 2005, Las Vegas, Nevada, USA  
1Human Factors International, 4th Floor, Chemtex House, Hiranandani Gardens, Mumbai – 400 076, India  
Approved for External Publication  
© Copyright 2005 Hewlett-Packard Development Company, L.P.  
Figure 1: typical computer setup with a gesture keyboard  
2.1 Need for a Devnagari input device  
Computer usage in India is almost entirely in English and is restricted to the English-speaking, upper and upper-  
middle class people. This stands in direct contrast with the demand for communication in Indian languages. Census  
data for the year 1991shows that English is the first language for only 0.02% of Indians and second and third  
language for 8% and 3% respectively. Hindi TV programs claim all the top 10 slots in television program ratings.  
Circulation of daily English newspapers had a market share of 15% in the year 2002. (Chand, Ganu, Joshi, Mathur  
& Parmar, 2004) 2001 census indicates that almost 70% of the Indian population specks 5 major languages i.e.  
Hindi, Bengali, Telugu, Marathi and Tamil with Hindi being the most spoken language at 40.22%.  
2.2 Difference with roman script  
Using a standard keyboard for Devnagari scripts creates usability issues, as the Devnagari Script possesses some  
conceptual differences from the Roman script.  
Some of these are summarized below:  
The concept of matras (character modifiers) has no parallels in the Roman script.  
Each character has a shiro rekha (the top bounding line on Devnagari characters) on top of it.  
Character modifiers, basically matras and other special characters, can occur before, on top, below or  
after the main character that they modify. (Deb and Deshwal, 2003)  
Since in the written form there is no clear specification of the sequence in which modifiers should be  
added to the consonant, sequential entry modes like a keyboard would not fit the users mental models  
More than one modifier can be attached to one consonant  
2.3 Issues with using an Inscript Keyboard  
Current Indian language typing solutions have a steep learning curve. According to Arjun Mahanto, the Hindi  
Officer in IIT Bombay, learning to type requires approximately fifty hours of training and practice for a person to  
reach speeds of 25 words per minute. This is too much of a barrier for ordinary people and only professional typists  
are willing to make this investment. (Chand et al., 2004)  
Figure 2: Inscript keyboard (with multiple characters on a single key)  
Commercially the demand for Devnagari input has been patched by the Inscript keyboard. This is essentially a  
QWERTY keyboard with Devnagari characters mapped onto the keys. The minimum number of characters that are  
required even representing Devnagari is 54, base consonants (34), vowels (19), and a halant key. This excludes  
Devnagari numerals and various Vedic symbols and modifiers i.e. Ref, Ru, Half Ra. This issue manifests itself on  
this keyboard, which is essentially meant for entering just 26 alphabets. The aftereffect is that each key has multiple  
characters (two to three characters per key) mapped onto them. These additional symbols on the keys are inputted  
using combinations of toggle keys like shift, alt and control. This keyboard has a very high learning curve due to the  
multiple shift modes as is suited for professional typewriters who undergo rigorous training to learn this keyboard.  
3
Introduction to the GKB  
Computing devices demand text input schemes that can be quickly learnt and retained to achieve a fair speed and  
easy to use rather than “hunt and peck”. Roman keyboards are not particularly amenable to accommodate the  
phonetic non-alphabetic script like Indic. Many alternate layouts exist which are mapped on the Roman alphabets of  
QWERTY keyboards. However, no such known work exists for Indic scripts. (Joshi & Rathod, 2002)  
Keyboards support a ten fingered typing and also have two entry modes namely expert and novice. While the novice  
uses “hunting and pecking” to type, experts do a head up touch-typing. So, there is always a trade off between the  
learning and typing speed achieved in the keyboard.  
There is an analogy between keyboards and styli. Keyboards can be used with no training: the letters can be tapped  
out one-by-one using hunt-and-peck. This is similar to what is currently done with styli. No new training is required,  
and letters are printed one-by-one. However, unlike styli, keyboards have a “growth path.” With practice, hunt-and-  
peck with two fingers can become faster than handwriting. If even higher speeds are desired, then keyboard users  
can learn touch-typing. Touch-typing not only achieves high speeds, it also enables “eyes-free” operation, that is, the  
ability to type without having to look at your hands. This suggests that the solution to the problem of stylus text  
entry requires developing an analogue of touch-typing. An electronic pen (or stylus) would be more attractive as an  
input device if it supported expert users with some analogue of touch-typing. (Goldberg & Richardson, 1993)  
Working on this analogy we tried to develop a gesture-based keyboard which had gestures enabled on each key. So  
that user neither has to write fully as in the case of pen based interfaces instead they can just make gestures to get a  
matra along with the base consonant nor touch type fully to get a syllable. In this case, the design of gesture became  
an important consideration for us. They have to be very intuitive and also very easy to learn.  
3.1 Gestures  
Few major criterions for designing gesture (matras) were, 1.easy to learn, 2.fast to write and 3.easy to retain matras.  
Instead of developing new gestures for each Devnagari matra (as in the case of graffiti and unistrokes), we  
implemented these matras as they appear in the Varnmala. The only difference being the breakage of multi stroke  
matras into a combination of different single stroke matras. Also for a half character we designed a horizontal  
stroke, which was derived from the mental model of cutting a character in to half.  
All these aforementioned criterions for matras were satisfied as there was no learning involved; users had to write  
them they way they normally write on paper. Also, the retention was easy as the users had to learn few matras  
namely horizontal strokes and oblique scoring for getting secondary glyphs from the keypad.  
3.2 Spatial mapping of gestures  
In order to limit the technological complexity of recognition and thereby trying to reduce instances of errors, we  
have mapped the gestures on the coordinates of the writing space. This means that the number of elements that  
needs to be recognized has been reduced. The same gestures made in different areas of the pad produce different  
combinations wherein the consonant is selected using the spatial mapping and the modifiers are assigned after  
recognition of the gestures.  
3.3 Layout  
The Varnmala structure was useful for people to locate keys and to reduce the cognitive load. Even users who had  
‘forgotten’ the alphabetical order could remember the local sequence of the letters they were searching for. For  
example, if they were looking for the (n) key, they would mutter an entire line of consonants ‘(t) (th) (d) (dh) (n)’  
before locating the key. (Chand et al. 2004)  
Figure 3: Varnmala layout  
We tried to design the gesture keypad as cognitively easy for a novice user as possible, so that with minute training  
and instructions they are able to achieve a decent writing speed. We kept the layout very basic, as it appears in the  
Devnagari varnmala books in the block.  
3.4 Feedback  
Performance data with keyboards, where the auditory feedback can be switched on and off suggest that typing is  
significantly faster and more accurate with auditory feedback on than with off. (Birdwell, Monty & Snyder, 1983;  
Blake, Muto & Roe, 1984) Based on these prior studies we added a click sound on every pen down. This can be  
again toggled on and off based on user’s preference. The timings of the feedback and the frequency of sound are still  
to be investigated.  
In addition visual feedback has been provided so that the user can see his handwritten input on the tablet real-time.  
We believe that expert users of the system will not use this functionality much but it should be provided to help the  
novice users. With some errors always built in with this keyboard due to handwriting recognition and pen tilt errors  
users may be able to catch and correct their errors when they see their handwriting on the computer screen.  
3.5 Deleting using the GKB  
The Devnagari script is very differently from roman scripts because of the conjuncts it forms when Consonant-  
Consonant-Vowel modifier, Consonant-Vowel-Vowel combination happens. So, in the case of normal ‘backspace’  
where 1 unicode is deleted at a time and it becomes very irrational to display the left over characters from conjuncts,  
which the user never actually wrote.  
Figure 4: Editing problems eradicated using a horizontal score on the backspace  
To overcome this problem we added one more gesture along with a normal ‘backspace’ which was horizontal  
scoring on the backspace key. It deleted the whole syllable in a go, so that user can start afresh.  
3.6 Technology used  
The technology behind GKB can be broadly categorized into three major components:  
Graphics tablet with electronic pen (hardware): This device is used to capture handwritten gestures as  
digital ink  
Handwritten gesture recognition (software): This algorithm recognizes the gestures and maps them to  
corresponding matras  
GKB controller (software): This software component maps the pen position to corresponding base  
character (consonant or vowel)  
The tablet is used to provide a keypad layout on which user makes gestures using the pen. The digital ink is then  
processed & recognized. Position of the pen and recognizer output is used to form the final syllable, which is then  
transmitted to the application in focus.  
3.7 A scalable solution for multi lingual environment  
One of the biggest advantages of the solution is its ability to accommodate inputs in other scripts. The tablet does  
not have labeled keys instead it had a layout print stuck on the pad surface, this layout can be changed easily thus  
enabling flexibility of multi lingual input.  
4
Finalizing keys and layout size  
Prior research (e.g. Clare, 1976) tells us about the design and size of keys which are used for touch-typing using  
fingers. We found out it’s worth investigating the natural writing sizes and style for Devnagari before coming up  
with an optimal key size and finally layout size. This study was crucial in terms of knowing the average size and  
styles in which user writes the matras and base consonants. The size has to be taken in account to make the keyboard  
much more robust and comfortable to write for a user.  
We had to come up with a key size and in turn layout size, which has the following characteristics:  
The users should be able to see the key label  
The key top size should be optimized for gesturing as well as tapping using a pen  
They keys should prevent the erroneous mapping of the alphabets  
As the key size and the layout size are directly proportionate, it should minimize the hand movement on  
the keypad  
4.1 User study  
4.1.1 Objective  
The objective of this exploration was to find the average size users take to write consonants and modifiers (matras)  
and to find relationship between handwriting styles with different pen grip diameter and tip thickness.  
4.1.2 Method  
Each user was given a Hindi paragraph (76 words) and they had to copy this Hindi corpus in their own handwriting  
on the provided paper. They were given basic instructions explaining the task and the goal of the experiment.  
Participants were provided with two different size blank papers (A4 = 21 cm x 29.8 cm and A5 = 14.8 cm x 21 cm).  
They were first asked to write on the A4 sized paper and then on A5 paper with 0.5mm pen and 9 mm finger grip  
diameter pen. Same activity was then repeated with 0.1 mm tip and 12mm finger grip diameter pen.  
4.1.3 Subjects  
21 voluntary subjects took part in this study. All of them, 1 left-handed and 7 right-handed subjects had normal  
vision. All subjects were experienced computer users.  
7 HP labs Employee (who know Hindi and occasionally write in Hindi)  
11 Students from 10th standard, from a local high school in Bangalore (who have Hindi as second  
language and write Hindi regularly)  
3 Teachers from the same high school-Bangalore (who write in Hindi almost once everyday)  
Figure 5: Participants for user study  
4.1.4 Apparatus  
4.1.4.1 Pen sizes:  
0.5 tip and 9mm diameter grip pen  
0.1 tip and 12mm diameter pen  
4.1.4.2 Paper sizes:  
A4 = 21 cms x 29.8 cms  
A5 = 11.5 cms x 29.8 cms  
Figure 6: Pens used for study (middle one is the stylus)  
4.1.4.3 Hindi corpus used:  
The Hindi paragraph used for this user study was taken from the BBC’s Hindi news portal  
nuances of Hindi matras in the corpus.  
Figure 7: Hindi Corpus  
4.1.5 Evaluation method  
Subjects’ hands written samples were collected and analyzed by measuring the base consonants and the modifiers  
with help of digital calipers. Each subject’s consonants and Modifiers size were noted down and divided according  
to the core strip, bottom strip and top strip.  
Figure 8: Character study  
4.1.6 Analysis  
Out of the four data sets that were collected, the most appropriate pen size and paper size that resembled the Gesture  
pad was 0.1 tip and 12mm diameter pen and A5 sized paper. So we analyzed this dataset and came out with the  
recommendation for the key size. We took the final data set and divided it in three groups, namely Top, Core and  
Bottom strip. Since, the dataset was very distributed, we adhered to the 90th percentile data so that almost all of the  
population could be accommodated.  
sizes in mms  
Top Strip Size Plot  
90th percentile data  
4.5  
4
4.19  
3.5  
3
2.5  
2
1.5  
1
top strip  
0.5  
0
0
5
10  
15  
20  
25  
Participants  
sizes in mms  
90th percentile data  
Core Strip Size Plot  
6
5
4
3
2
1
0
4.63  
0
5
10  
15  
20  
25  
Participants  
sizes in mms  
Bottom Strip Size Plot  
90th percentile data  
4
3
2
1
0
3
0
5
10  
15  
20  
25  
Participants  
Figure 9: Graphs showing cluster data for handwriting samples (X axis shows the size in mm and Y axis denotes  
the no of participants)  
4.1.7 Results  
The final gesturable keypad sizes, which is analogous to handwriting was 11.82 mm. The core strip was 4.63 mm,  
space required for top strip was 4.13 mm, and the space required for the bottom strip was 3 mm. We didn’t take the  
left and right side into consideration as there are only few matras i.e. (aa, ee) which are added to the left and right of  
the core strip in the form a straight line and could easily be accommodated in the square top keys.  
Figure 10: Final layout  
5
Conclusions  
This keyboard has been accepted quite well with the novice users who have never used Devnagari keyboard before.  
In our small sessions of user studies they all seem to be satisfied and were able to achieve a speed of 10-12 syllables  
per minute. Also after a dedicated usage of an hour everyday for 2-3 days they were able to go up to 25-30 syllables  
per minute.  
The keyboard has a high satisfaction factor associated with it because writing with pen on paper (in our case the  
keyboard) is very intuitive, powerful and efficient. It aptly exhibits the same characteristics of simplicity,  
naturalness and straightforwardness like handwriting.  
6
Future Directions and discussions  
We are currently planning to do a constrained usability study to measure the efficiency of the gesture keyboard  
against the other “standard” Devnagari input mechanism namely Inscript keyboard with novice as well as expert  
users. The aim of this proposed study is to see whether the satisfaction ratings change with these different user  
groups and also to compare the words per minute that we are able to achieve with this keyboard.  
We are also studying various layouts (Varnmala, frequency and common letter-pair) using paper mock-ups and  
analyzing which one suits the mental model of the users the best. User studies are also underway to determine which  
layout is the fastest to learn and reduces the pen travel while writing.  
7
Acknowledgements  
The authors would like to thank Dr. Shekhar Borgaonkar, Dr. Sriganesh Madhvanath, and Dr. Girish Prabhu for  
their valuable inputs and support during the project.  
8
References  
Chand Aditya, Ganu Ashish, Joshi Anirudha, Mathur Gaurav & Parmar Vikram (2004). “Keylekh: A Keyboard for  
Text Entry in Indic Scripts”, CHI 2004, April 24–29, 2004, Vienna, Austria. ACM  
Clare C.R. (1976). “Human Factors: a most important ingredient in keyboard designs”, Handbook of Human  
computer Interaction 2nd edition, page no pp 1296-1297  
Goldberg David & Richardson Cate (1993). “Touch-Typing With a Stylus” InterCHI 1993. ACM  
Deb Dr. Kalyanmoy & Deshwal Priyendra Singh (2003). “An Optimal Character-Key Mapping for the Devnagari”  
Joshi Anirudha & Rathod Amit (2002). “A Dynamic Text Input scheme for phonetic scripts like Devanagari”  
Development by Design, Bangalore 2002  
Monty, Snyder and Birdwell, (1983) & Roe, Muto and Blake, (1984). Handbook of Human computer Interaction  
2nd edition, page no pp 1299-1301  

Hotpoint Convection Oven SH83CX S User Manual
Gateway FHD2400H User Manual
Eizo FLEXSCAN F67 User Manual
DVS DRL 100 User Manual
Dimplex DS5629 User Manual
Daewoo DVD VCR Combo DRVT 43 User Manual
Asus Xonar Ds 7 1 Channels Sound Card XONARDS User Manual
Asus Motherboard M2R32 MVP User Manual
Agilent Technologies Computer Hardware 66000A User Manual
Acer B203W User Manual