Mouse Gesture
   HOME

TheInfoList



OR:

In
computing Computing is any goal-oriented activity requiring, benefiting from, or creating computer, computing machinery. It includes the study and experimentation of algorithmic processes, and the development of both computer hardware, hardware and softw ...
, a pointing device gesture or mouse gesture (or simply gesture) is a way of combining
pointing device A pointing device is a human interface device that allows a User (computing)#End-user, user to input Three-dimensional space, spatial (i.e., continuous and multi-dimensional) data to a computer. Graphical user interfaces (GUI) and Computer- ...
or finger movements and clicks that the
software Software consists of computer programs that instruct the Execution (computing), execution of a computer. Software also includes design documents and specifications. The history of software is closely tied to the development of digital comput ...
recognizes as a specific computer event and responds to accordingly. They can be useful for people who have difficulties typing on a keyboard. For example, in a
web browser A web browser, often shortened to browser, is an application for accessing websites. When a user requests a web page from a particular website, the browser retrieves its files from a web server and then displays the page on the user's scr ...
, a user can navigate to the previously viewed page by pressing the right pointing device button, moving the pointing device briefly to the left, then releasing the button.


History

The first pointing device gesture, the " drag", was introduced by
Apple An apple is a round, edible fruit produced by an apple tree (''Malus'' spp.). Fruit trees of the orchard or domestic apple (''Malus domestica''), the most widely grown in the genus, are agriculture, cultivated worldwide. The tree originated ...
to replace a dedicated "move" button on mice shipped with its Macintosh and Lisa computers. Dragging involves holding down a pointing device button while moving the pointing device; the software interprets this as an action distinct from separate clicking and moving behaviors. Unlike most pointing device gestures, it does not involve the tracing of any particular shape. Although the "drag" behavior has been adopted in a huge variety of software packages, few other gestures have been as successful.


Current use

, most programs do not support gestures other than the drag operation. Each program that recognizes pointing device gestures does so in its own way, sometimes allowing for very short mouse movement distances to be recognized as gestures, and sometimes requiring very precise emulation of a certain movement pattern (e.g. circle). Some implementations allow users to customize these factors. Some
video game A video game or computer game is an electronic game that involves interaction with a user interface or input device (such as a joystick, game controller, controller, computer keyboard, keyboard, or motion sensing device) to generate visual fe ...
s have used gestures. For example, in the ''
Myth Myth is a genre of folklore consisting primarily of narratives that play a fundamental role in a society. For scholars, this is very different from the vernacular usage of the term "myth" that refers to a belief that is not true. Instead, the ...
''
real-time tactics Real-time tactics (RTT)(Article at IGN discussing their perception of RTS and related genres as of 2006. RTT is discussed as a new and not yet established genre from the publisher's perspective, so currently all RTT possible titles are still con ...
series, originally created by Bungie, players use them to order battlefield units to face in a desired direction. Another game using gestures is Lionhead's '' Black & White''. The game ''
Arx Fatalis ''Arx Fatalis'' is a 2002 action role-playing game developed by Arkane Studios and released for Windows and Xbox. The game is played from a first-person perspective and is set on a world whose sun has failed, forcing the above-ground creatures t ...
'' uses mouse gestures for drawing runes in the air to cast spells. Several
Nintendo is a Japanese Multinational corporation, multinational video game company headquartered in Kyoto. It develops, publishes, and releases both video games and video game consoles. The history of Nintendo began when craftsman Fusajiro Yamauchi ...
Wii The Wii ( ) is a home video game console developed and marketed by Nintendo. It was released on November 19, 2006, in North America, and in December 2006 for most other regions of the world. It is Nintendo's fifth major home game console, f ...
games take advantage of such a system. ''
ÅŒkami is a 2006 action-adventure game developed by Clover Studio and published by Capcom. It was released for PlayStation 2 in 2006 in Japan and North America, and in 2007 in Europe and Australia. After the closure of Clover Studio a few months afte ...
'' uses a system similar to mouse gestures; the player can enter a drawing mode in which the shape they create (circle, lightning bolt, line, etc.) performs a function in the game such as creating a bomb or changing the time from night to day. Other examples of computer games that use mouse gestures are '' Die by the Sword'' and ''
Silver Silver is a chemical element; it has Symbol (chemistry), symbol Ag () and atomic number 47. A soft, whitish-gray, lustrous transition metal, it exhibits the highest electrical conductivity, thermal conductivity, and reflectivity of any metal. ...
'' where basic mouse gestures actually map attack moves and such in real-time combat, along with '' MX vs. ATV: Reflex'', which has a control scheme that implements its titular rider "reflex" system with mouse gestures. The upcoming
Nintendo Switch 2 The is a hybrid video game console developed by Nintendo, released in most regions on June5, 2025. Like the original Nintendo Switch, Switch, it can be used as a Handheld game console, handheld, as a Tablet computer, tablet, or connected via ...
's Joy-Con 2 controllers, which also double as mice, can also support mouse gestures, as evidenced in the forthcoming online sports game '' Drag x Drive''. The
Opera web browser Opera is a multi-platform web browser developed by its namesake company Opera. The current edition of the browser is based on Chromium. Opera is available on Windows, macOS, Linux, Android, and iOS (Safari WebKit engine). Opera offers two mo ...
has recognized gestures since version 5.10 (April 2001) but this feature was disabled by default. However, this feature was introduced in Vivaldi Browser, a web browser meant to be a solution for the users of the former Opera, a more powerful browser than the current one. Opera browser also supports
mouse chording Mouse chording is the capability of performing actions when multiple mouse buttons are held down, much like a chorded keyboard and similar to mouse gestures. One common application of mouse chording, called ''rocker navigation'', is found in Op ...
which serves a similar function but doesn't necessitate mouse movement. The first browser that used advanced mouse gestures (in 2002) was
Maxthon Maxthon (, formerly named ''MyIE2'') is a freeware web browser, created by JeffChen in Singapore. It is available for Microsoft Windows, Windows, macOS, Linux, and as ''Maxthon Mobile'' for Android (operating system), Android, iOS, and Windows P ...
, in which a highly customizable interface allowed the assignment of almost every action to one of 52 mouse gestures and few mouse chords. Several mouse gesture extensions are also available for the
Mozilla Firefox Mozilla Firefox, or simply Firefox, is a free and open-source web browser developed by the Mozilla Foundation and its subsidiary, the Mozilla Corporation. It uses the Gecko rendering engine to display web pages, which implements curren ...
browser. These extensions use almost identical gestures as Opera. Some tools provide mouse gestures support in any application for
Microsoft Windows Windows is a Product lining, product line of Proprietary software, proprietary graphical user interface, graphical operating systems developed and marketed by Microsoft. It is grouped into families and subfamilies that cater to particular sec ...
. K Desktop Environment 3 includes universal mouse gesture support since version 3.2.
Windows Aero Windows Aero (a backronym for ''Authentic, Energetic, Reflective, and Open'') is the design language introduced in the Microsoft Windows Vista operating system in 2006. The changes introduced by Windows Aero encompassed many elements of the Windo ...
provides three mouse gestures called Aero Peek, Aero Shake and Aero Snap. See the corresponding article for a description.


Touchpad and touchscreen gestures

Touchscreen A touchscreen (or touch screen) is a type of electronic visual display, display that can detect touch input from a user. It consists of both an input device (a touch panel) and an output device (a visual display). The touch panel is typically l ...
s of tablet-type devices, such as the
iPad The iPad is a brand of tablet computers developed and marketed by Apple Inc., Apple that run the company's mobile operating systems iOS and later iPadOS. The IPad (1st generation), first-generation iPad was introduced on January 27, 2010. ...
, utilize
multi-touch In computing, multi-touch is technology that enables a surface (a touchpad or touchscreen) to recognize the presence of more than one somatosensory system, point of contact with the surface at the same time. The origins of multitouch began at CE ...
technology, with gestures acting as the main form of user interface. Many touchpads, which in laptops replace the traditional mouse, have similar gesture support. For example, a common gesture is to use two fingers in a downwards or upwards motion to scroll the currently active page. The rising popularity of touchscreen interfaces has led to gestures becoming a more standard feature in computing.
Windows 7 Windows 7 is a major release of the Windows NT operating system developed by Microsoft. It was Software release life cycle#Release to manufacturing (RTM), released to manufacturing on July 22, 2009, and became generally available on October 22, ...
introduced touchscreen support and touchpad gestures. Its successor,
Windows 8 Windows 8 is a major release of the Windows NT operating system developed by Microsoft. It was Software release life cycle#Release to manufacturing (RTM), released to manufacturing on August 1, 2012, made available for download via Microsoft ...
is designed to run both on traditional desktops and mobile devices and hence gestures are now enabled by default where the hardware allows it. Related to gestures are touchpad hotspots, where a particular region of the touchpad has additional functionality. For example, a common hotspot feature is the far right side of the touchpad, which will scroll the active page if a finger is dragged down or up it. Multi-touch touchscreen gestures are predefined motions used to interact with multi-touch devices. An increasing number of products like smartphones, tablets, laptops or desktop computers have functions that are triggered by multi-touch gestures. Common touchscreen gestures include: Other gestures including more than 2 fingers on screen have also been developed such as Sticky Tools. These techniques are often developed for 3D applications and are not considered standard.


Drawbacks

A major drawback of current gesture interaction solutions is the lack of support for two necessary user interface design principles, feedback and visibility (or
affordance In psychology, affordance is what the environment offers the individual. In design, affordance has a narrower meaning; it refers to possible actions that an actor can readily perceive. American psychologist James J. Gibson coined the term ...
). Feedback notification is required to indicate whether the gesture has been entered correctly by indicating the gesture recognized and the corresponding command activated, although Sensiva does approach this to some extent in providing voice notification. The other principle is visibility of gestures, providing the user some means of learning the necessary gestures and the contexts they can be used in. Both Mouse Gestures for
Internet Explorer Internet Explorer (formerly Microsoft Internet Explorer and Windows Internet Explorer, commonly abbreviated as IE or MSIE) is a deprecation, retired series of graphical user interface, graphical web browsers developed by Microsoft that were u ...
and ALToolbar Mouse Gestures display colored tracers that indicate the current motion that the user is taking to facilitate visual clues for the user.
Pie menu In user interface design, a pie menu or radial menu is a circular context menu where selection depends on direction. It is a graphical control element. A pie menu is made of several "pie slices" around an inactive center and works best with styl ...
s and marking menus have been proposed as solutions for both problems, since they support learning of the available options but can also be used with quick gestures. Most recent versions of Opera (11 and above) uses an on-screen pie menu to simply and instructively display which mouse gestures are available and how to activate them, providing feedback and visibility. One limitation with gesture interaction is the scope context in which the gestures can be used. For example, each gesture has only one corresponding command for each application window. Holding down buttons while moving the mouse can be awkward and requires some practice, since the downwards action increases friction for the horizontal motion. An optical mouse would be less susceptible to changes in behavior than a ball mouse with increased friction because the sensor does not rely on mechanical contact to sense movement; a
touchpad A touchpad or trackpad is a type of pointing device. Its largest component is a tactile sensor: an electronic device with a flat surface, that detects the motion and position of a user's fingers, and translates them to 2D motion, to control a Cu ...
provides no added friction with all its buttons held down with a thumb. However, it was also argued that muscular tension resulting from holding down buttons could be exploited in
user interface design User interface (UI) design or user interface engineering is the design of user interfaces for machines and software, such as computers, home appliances, mobile devices, and other electronic devices, with the focus on maximizing usability and the ...
as it gives constant feedback that the user is in a temporary state, or
mode Mode ( meaning "manner, tune, measure, due measure, rhythm, melody") may refer to: Arts and entertainment * MO''D''E (magazine), a defunct U.S. women's fashion magazine * ''Mode'' magazine, a fictional fashion magazine which is the setting fo ...
(Buxton, 1995).


See also

*
Mouse chording Mouse chording is the capability of performing actions when multiple mouse buttons are held down, much like a chorded keyboard and similar to mouse gestures. One common application of mouse chording, called ''rocker navigation'', is found in Op ...
*
Computer accessibility Computer accessibility refers to the accessibility of a computer system to all people, regardless of disability type or severity of impairment. The term ''accessibility'' is most often used in reference to specialized hardware or software, or a co ...
*
Drag and drop In computer graphical user interfaces, drag and drop is a pointing device gesture in which the user (computing), user selects a virtual object by "grabbing" it and dragging it to a different location or onto another virtual object. In general, i ...
*
Pie menu In user interface design, a pie menu or radial menu is a circular context menu where selection depends on direction. It is a graphical control element. A pie menu is made of several "pie slices" around an inactive center and works best with styl ...
*
Pen computing Pen computing refers to any computer user-interface using a digital pen or Stylus (computing), stylus and Graphics tablet, tablet, over input devices such as a keyboard or a mouse. Historically, pen computing (defined as a computer system employin ...
*
Gesture recognition Gesture recognition is an area of research and development in computer science and language technology concerned with the recognition and interpretation of human gestures. A subdiscipline of computer vision, it employs mathematical algorithms to ...


References

* Buxton, W. A. (1995)
"Chunking and phrasing and the design of human-computer dialogues"
in ''Human-Computer interaction: Toward the Year 2000'', R. M. Baecker, J. Grudin, W. A. Buxton, and S. Greenberg, Eds. Morgan Kaufmann Publishers, San Francisco, CA, 494-499.

contains a history of pen computing, including touch and gesture technology, from approximately 1917 to 1992.

* L. K. Welbourn and R. J. Whitrow. 1988. ttp://portal.acm.org/citation.cfm?id=54696 A gesture based text editor In Proceedings of the Fourth Conference of the British Computer Society on People and computers IV, D. M. Jones and R. Winder (Eds.). Cambridge University Press, New York, NY, USA, 363-371. * Brad A. Myers.
A Brief History of Human Computer Interaction Technology
". ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44–54.
Notes on the History of Pen-based Computing (YouTube)


External links



* * ttp://ruetersward.com/pens/penhist.html Notes on the (relatively unknown) History of Pen-based Computing {{DEFAULTSORT:Pointing Device Gesture User interface techniques Articles containing video clips ja:マルãƒã‚¿ãƒƒãƒ#マルãƒã‚¿ãƒƒãƒã‚¸ã‚§ã‚¹ãƒãƒ£ãƒ¼