Facebook Patent | Tracking Positions Of Portions Of A Device Based On Detection Of Magnetic Fields By Magnetic Field Sensors Having Predetermined Positions
Publication Number: 20200272235
Publication Date: 20200827
Applicants: Facebook
Abstract
A wearable device includes a plurality of magnetic field generators positioned at different points to be tracked. The magnetic field generators emit magnetic fields that are sensed by a plurality of magnetic field sensors having known positions relative to each other. The wearable device may have magnetic field generators placed at tracked points corresponding to portions of the fingers and the palm of the device, and magnetic field sensors located at predetermined positions. A position analyzer determines the spatial position of the tracked points on the palm as well as the tracked points on the fingers from the output signals of the magnetic field sensors. From the determined spatial positions of the tracked points, a gesture identification system determines a gesture corresponding to the spatial positions of the points of the wearable device where the magnetic field generators are positioned.
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62/808,679, filed on Feb. 21, 2019, which is incorporated by reference herein in its entirety.
BACKGROUND
[0002] Use of wearable devices as well as virtual reality (VR) or augmented reality (AR) devices has become more commonplace. Conventional wearable VR or AR devices commonly receive inputs by capturing audio data (e.g., voice comments), capturing gestures, receiving user interactions with one or more of a limited number of buttons, or receiving user interaction via a limited touch area on the wearable VR or AR device. However, many of these conventional input mechanisms are inconvenient or awkward for users to operate in various contexts.
[0003] When used in conjunction with a head mounted display device, such as those used in VR or AR systems, conventional input mechanisms have additional complications. For example, when interacting with a VR system, users perform gestures to simulate interaction with objects. To identify these gestures, conventional virtual reality systems use computer vision techniques to identify relative and absolute locations of portions of a user’s body. However, computer vision techniques are limited to identifying gestures from portions of a user’s body that are within a field of view of an image capture device. This decreases the effectiveness of computer vision detection and identification of gestures when a portion of a user’s body is occluded from the field of view of the image capture device. Some virtual reality systems use inertial measurement units positioned on portions of a user’s body to track gestures, but the inertial measurement units suffer from drift error requiring frequent sensor calibration to compensate for the drift error.
SUMMARY
[0004] A wearable device (such as a glove or other control adapted to be worn on a portion of a user’s body) includes a gesture identification system and a hand position system based on determining spatial locations of points being tracked (i.e., “tracked points”) on the wearable device. In embodiments where the wearable device is a glove, the tracked points used by the gesture identification system are the fingertip locations of the wearable glove as determined when worn by the user. Similarly, when the wearable device is a glove, the tracked points used by the hand position system are one or more points on a palm of the wearable glove as determined when worn by the user. The spatial positions of the tracked points on the wearable device are determined based on magnetic fields generated by a magnetic field generators that are located at the tracked points and detected by magnetic field sensors at known positions. For example, a plurality of magnetic field sensors with known positions relative to each other detect generated magnetic fields. The magnetic field sensors may be located on the wearable device or near the wearable device. Thus, the magnetic field generators (e.g., active transmitters, permanent magnets, or electromagnets) act as proxy locations for the tracked points.
[0005] In one embodiment, the wearable device is a glove having magnetic field generators on one or more fingers of the glove. The magnetic field generators generate magnetic fields that are sensed by the plurality of magnetic field sensors that located at a predefined location on or with respect to the glove. For example, at least two magnetic field sensors are located at a known fixed distance from each other on a palm of the glove or on a dorsal aspect (i.e., back side to the palm) of the glove, while an additional magnetic field sensor is located at the wrist of the glove, such as a portion of the glove configured to be worn at the wrist of the user’s hand. Alternatively, or additionally, the magnetic field sensors are separate from the wearable device, in predetermined or known locations. For example, the magnetic field sensors are located on a table top or on a headset worn by the user. In the preceding example, the tracked points correspond to the portions of the figures of the glove where the magnetic field generators are located. The magnetic field sensors on the palm at the known locations relative to each other sense magnetic field strength and direction of the magnetic fields generated by each of the magnetic field generators. Using the known locations of the magnetic field sensors, a spatial position of each of the tracked locations is determined with respect to a known coordinate system. In various embodiments, different magnetic field generators emit magnetic fields with different frequencies or emit magnetic fields at different times, allowing identification of magnetic fields emitted by different magnetic field generators. The spatial position of each of the tracked points is then combined to generate a representative state descriptor of the combined positions of at least a set of the tracked points. This representative state descriptor is subsequently used either with a stored mapping of positions of points, or via real time modeling, to identify a gesture made by the user using the wearable device.
[0006] In some embodiments, locating the magnetic field sensors at the palm of the glove (or on a dorsal side of the glove contacting a back of the user’s hand) and at the wrist of the glove, allows determination of positions of the tracked points on the glove (i.e., the points where the magnetic field generators are located) relative to the palm (or to the dorsal side of the glove), and relative to the wrist. In some embodiments, two magnetic field sensors may be both located on the palm of the hand or be both located on a dorsal side of the glove contacting a back of the user’s hand, so that the two magnetic field sensors are in a fixed position relative to each other with a known fixed distance from each other, and each tracked point’s position may be determined relative to the magnetic field sensors. In other embodiments, locating the magnetic sensors at a head mounted display (HMD) or at a stationary location enables the tracked points on the glove (i.e., the points where the magnetic field generators are located) to be located relative to the HMD or relative to the stationary location. In other embodiments, magnetic sensors are located at the palm (or back) of the glove, at the wrist of the glove, and also at a location external to the glove (e.g., a known stationary location or on a HMD), allowing of positions of tracked points on the glove (i.e., the points where the magnetic field generators are located) to be determined in a relative coordinate system with respect to a magnetic field sensor or with respect to an absolute coordinate system using the magnetic field generator at the known stationary location. The positions of the tracked points on the glove (e.g., positions on fingers of the glove) are used by the gesture identification system. In some embodiments, an additional magnetic field generator may be located on the palm of the hand (or on a dorsal side of the glove contacting a back of the user’s hand) to act as a proxy location for the point on the palm (or a point on the dorsal aspect of the palm). One or more magnetic field sensors located at the wrist may be used to determine the position of the tracked point on the palm with respect to the one or more wrist magnetic field sensors by the hand position system.
DESCRIPTION OF THE DRAWINGS
[0007] FIGS. 1A-1D illustrate frameworks within which spatial locations of magnetic field generators and magnetic field sensors are represented, in accordance with one or more embodiments.
[0008] FIG. 2 illustrates locations of magnetic field sensors and magnetic field generators in relation to a wearable glove device on a hand of a user, where a plurality of magnetic field generators are positioned on different fingers, and are coupled to magnetic field sensors at known locations, in accordance with one or more embodiments.
[0009] FIGS. 3A and 3B illustrate the positioning of the magnetic field generators to act as proxy locations for fingertip and finger section locations being tracked via magnetic field sensors, in accordance with one or more embodiments.
[0010] FIGS. 4A-4C are example gestures that may be identified, in accordance with one or more embodiments.
[0011] FIG. 4D is one example of a mapping of fingertip locations to gestures, in accordance with one or more embodiments.
[0012] FIG. 5 is a block diagram of a system for determining hand gestures and hand positions made by a user using a wearable glove device, in accordance with one or more embodiments.
[0013] FIG. 6 is a flowchart of a method for determining a gesture from locations of magnetic field generators determined from magnetic fields sensed by magnetic field sensors, in accordance with one or more embodiments.
DETAILED DESCRIPTION
Overview
[0014] A wearable device (such as a glove or other control adapted to be worn on a portion of a body of a user) includes a position sensing apparatus that determines spatial positions of tracked points on the wearable device. Magnetic field generators are located at the tracked points on the wearable device and emit magnetic fields that are detected by a plurality of magnetic fields sensors that have known positions relative to each other. The spatial positions of the tracked points on the wearable device are used to determine gestures made by a user wearing the wearable device. Gestures may be viewed as combinations of determined spatial positions for different tracked points on the wearable device. For example, a gesture is a combination of spatial positions determined for portions of the wearable device corresponding to different portions of a user’s body (e.g., different fingers or a user’s hand). As used herein, the term “magnetic field generator” refers to a device that generates one or more magnetic fields. Example magnetic field generators include an actively driven transmitter using alternating current signals (e.g., at a range of 1 KHz to 10 MHz) to generate the magnetic fields, an electromagnet, a permanent magnet, a programmable magnet, and a polymagnet. The term “magnetic field sensor,” as used herein, refers to devices that sense one or more magnetic fields and generate output signals based on the detected one or more magnetic fields.
[0015] For an arbitrary location in the space, the strength of a magnetic field H, generated by a magnetic field generator (e.g., a magnet) and sensed by a magnetic field sensor that is located at a particular point, is inversely proportional to the cube of the distance r (i.e., the distance between the sensing point and the magnetic field generator) and .theta. (i.e., the angle between the magnetic field generator’s north and the sensor). The magnetic field H may be decomposed into two orthogonal vectors (e.g., in radial space), H.sub.r and H.sub..theta.. These two vectors are the basis in this 2D magnetic-field space and can be mathematically represented in terms of r and .theta..
H.sub.r=M cos .theta./2.pi.r.sup.3 (1)
H.sub..theta.=M sin .theta./4.pi.r.sup.3 (2)
where M is a magnetic moment. When the magnetic field generator is an electromagnet, the value of M is related to the permeability of core material, current, and area of the electromagnet. Given an actively driven transmitter using a constant alternating current driving a circular current loop, the magnetic moment, M, may be assumed to be a constant, and equations (1) and (2) above represent the generated magnetic fields at locations, r, greater than threshold distance from the center of the current loop.
[0016] FIG. 1A depicts a magnetic field generator MG generating a magnetic field 100. In the example shown in FIG. 1A, a magnetic field sensor MS senses the magnetic field generated by the magnetic field generator MG. The sensed magnetic field is depicted by the magnetic field vector H that may be decomposed into orthogonal components in radial space as (H.sub.r, H.sub..theta.).
[0017] From equations (1) and (2) above, the magnetic field strength is the same for points having a common distance r from the magnetic field generator and having a common angle .theta. relative to a specific axis of the magnetic field generator MG (e.g., a magnetic axis along a north pole of the magnetic field generator MG). FIG. 1B shows that locations L1, L2, L3, L4, on a circumference of a circle having a center that is a magnetic axis of the magnetic field generator MG, have a common magnetic field strength, as locations L1, L2, L3, L4 are a common distance from the magnetic field generator MG and have a common angle relative to the magnetic axis of the magnetic field generator MG.
[0018] Embodiments described herein determine the position of the magnetic field generator, given a known location of the magnetic field sensor with respect to a chosen coordinate system. From equations (1) and (2) above, the distance (r) and orientation (.theta.) may be calculated from the magnetic field (H.sub.r and H.sub..theta.). However, a spatial ambiguity exists in the 3D space (such as illustrated by points L1, L2, L3, L4 in FIG. 1B) as, from a top view of the magnetic field generator MG, points located on a circle concentric with the axis of the magnetic field generator MG have a common distance and angle from the magnetic field generator MG, so the magnetic field strength is the same at each point along the circle concentric with the axis of the magnetic field generator MG. The magnetic field sensor MS detects the same measurement for points along the circle concentric with the axis of the magnetic field generator MG, resulting in spatial ambiguity of the location of the magnetic field generator MG relative to the magnetic field sensor MS.
[0019] According to one or more embodiments, to resolve the spatial ambiguity described above, a total magnetic field strength is calculated based on equation (3):
.parallel.H.parallel..sup.2=(H.sub.r).sup.2+(H.sub..theta.).sup.2=K*r.su- p.-6*(3 cos.sup.2 .theta.+1) (3)
where .parallel.H.parallel. is the norm-2 of the sensor vector, r and .theta. are the distance between sensors and electromagnet (as in equations 1 and 2 above), and K is a constant. The constant K is a factor of constant M (e.g., K=M.sup.2/16.pi..sup.2).
[0020] The configuration described above may be used to define a “beacon system” in which the received signal strength (i.e., the total magnetic field strength .parallel.H.parallel.) relates to the distance from the signal source (i.e., r) and angle at which the signal is received (i.e., .theta.). However, equation 3 is under-constrained since there are two unknown variables (r, .theta.) and a single equation (e.g., from .parallel.H.parallel.). Hence, the next step is to obtain an over-constrained system by remodeling the two unknown variables (r, .theta.).
[0021] To obtain an over-constrained system, a fixed or known layout of the magnetic field sensor MS is used to define a coordinate system 130, as shown in FIG. 1C. In FIG. 1C, the magnetic field sensor MS includes multiple sensing elements S1, S2, S3, S4. The multiple sensing elements sense the magnetic field along the particular axes of the coordinate system, such as the coordinate system 130 of FIG. 1C. Thus, for example, sensing element S1 is used as the origin, and coordinates of other sensing elements S2, S3, S4 are ascertained with respect to the coordinates of sensing element S1. In the example of FIG. 1C, sensing elements S1, S2, and S3 are coplanar, while sensing element S4 is positioned above sensing element S1 along a direction orthogonal to the plane in which sensing elements S1, S2, and S3 are positioned. Alternatively, the coordinate system 130 of FIG. 1C is determined from multiple magnetic field sensors MS having placements relative to each other paralleling the placement of the sensing elements S1, S2, S3, S4 shown in FIG. 1C.
[0022] FIG. 1C illustrates a coordinate system 130 for trilateration and for determination of spatial locations of magnetic field generators and magnetic field sensors, according to one or more embodiments. In some embodiments, the coordinate system 130 of FIG. 1C illustrates a single magnetic field sensor MS that includes a plurality of constituent sensing elements S1, S2, S3, S4 (e.g., magnetometers). Alternatively, each sensing element S1, S2, S3, S4 of the coordinate system 130 shown in FIG. 1C may correspond to a distinct magnetic field sensor MS.
[0023] The two unknown variables (r, .theta.) of equations (1)-(3) are replaced with the 3D position (x, y, z) of the magnetic field generator MG in the coordinate system of FIG. 1D. Thus, given the coordinate system 130, the two variables (r, .theta.) can be represented in terms of the 3D position (x, y, z) of the magnetic field generator MG using equations (4) and (5):
r.sub.1=(x.sup.2+y.sup.2+z.sup.2).sup.1/2 (4)
cos .theta..sub.1=z/r.sub.1 (5)
Similarly, these three shared variables (x, y, z) can be substituted into the equation (3) for the other three sensing elements of a magnetic field sensor MS, resulting in equations (6)-(11):
r.sub.2=((x+1).sup.2+(y-1).sup.2+z.sup.2).sup.1/2 (6)
cos .theta..sub.2=z/r.sub.2 (7)
r.sub.3=((x-1).sup.2+(y-1).sup.2+z.sup.2).sup.1/2 (8)
cos .theta..sub.3=z/r.sub.3 (9)
r.sub.4=(x.sup.2+y.sup.2+(z-1).sup.2).sup.1/2 (10)
cos .theta..sub.4=z/r.sub.4 (11)
[0024] The resulting system of equations is over-constrained and has four equations (of equation 3) from each of four sensing elements S1, S2, S3, S4 of the magnetic field sensor MS to be solved for three shared variables (x, y, z). In some embodiments, the magnetic field generator is an actively driven transmitter using alternating current signals (e.g., at a range of 1 KHz to 10 MHz) to generate the magnetic fields. In some embodiments, the magnetic field generator MG comprises a programmable magnet (e.g., a polymagnet) and magnetic field properties of the magnetic field generator MG are programmable, controllable, and/or reconfigurable. In some embodiments, the programmable, controllable, and/or reconfigurable magnetic field properties include a number of magnetic poles, a density of magnetic poles (e.g., number and distribution of magnetic poles over a given surface area of the magnetic field generator MG), a spatial orientation/configuration/layout of magnetic poles, magnetic field strength, a variation of magnetic field strength as a function of spatial coordinates (e.g., distance from the magnetic field generator MG), focal points of the magnetic field, mechanical forces (e.g., attraction, repulsion, holding, alignment forces) between poles of the same polymagnet or between polymagnets, and so on. For example, a spatial flux density or flux orientation mapping can be programmed or configured (e.g., to be distinct from the relationship described in equations (1) and (2)) to uniquely or distinctly encode position and/or orientation of different magnetic field generators MG or different magnetic field sensors MS. In some embodiments, polymagnets and programmable magnetic generators can be programmed to provide stronger magnetic fields, or fields concentrated within shorter ranges to improve resolution and accuracy of position sensing within the shorter sensing ranges. In some embodiments, magnetic field generators are implemented by winding a coil around a cylindrical or flat ferrite core, printing winding on thin, flexible magnetic core material, performing 3-D printing or embraiding as part of the thread around a glove finger (with or without magnetic core material as liner), and using air coils for limited range requirements).
[0025] In some embodiments, when the magnetic field generator MG is a programmable magnet or an electromagnet, the signals applied to the magnetic field generator MG to generate the magnetic field may be optionally pre-processed or normalized by a function of or corresponding to 1/r.sup.3 corresponding to the effect of spatial distance on the magnetic field components in a radial coordinate space. For example, a “non-affine” transformation may be applied to the signals applied to the magnetic field generator MG to transform the input signals based on a normalization function corresponding to 1/(r.sup.3) in radial coordinate (r, .theta.) space.
[0026] FIG. 1D illustrates an alternative 3-dimensional framework 140 for determining spatial position vectors in three-dimensional (3D) Cartesian space for a configuration with a single magnetic flux sensor (MS) 110 and single magnetic generator (MG) 120. In some embodiments, a spatial position is expressed as a vector with multiple components representing spatial coordinates (distance and/or orientation) in a multi-dimensional space. For example, in a three-dimensional (3D) coordinate system, the vector components of a spatial position vector include Cartesian distances along three orthogonal Cartesian coordinate axes (X, Y, Z) or angular orientation (angles .alpha., .phi., .psi.) defined with respect to three mutually perpendicular Cartesian axes (X, Y, Z) or defined with respect to mutually perpendicular Cartesian planes (YZ, XZ, and XY). In some embodiments, the spatial position vectors may include Cartesian distances along three orthogonal Cartesian coordinate axes (X, Y, Z), but not along the angular orientations (angles .alpha., .phi., .psi.).
[0027] In one or more embodiments, each magnetic field generator (MG) 120 includes one or more magnets; each magnet of a given magnetic generator (MG) 120 may be configured to generate a corresponding magnetic field oriented along a distinct direction (e.g., a distinct coordinate axis) from other magnets of that magnetic field generator 120. In some embodiments, a magnetic field generator 120 includes three magnets that generate three orthogonal magnetic fields along three orthogonal Cartesian coordinate axes (Hx, Hy, and Hz, as illustrated for MG 120 in FIG. 1D). Similarly, each magnetic flux sensor (MS) 110 includes one or more constituent sensing elements (e.g., one or more magnetometers), each sensing element (magnetometer) configured to generate a signal responsive to a detected magnetic field that is oriented along a distinct direction (e.g., a distinct coordinate axis). For example, a magnetic field sensor (e.g., MS 110 of FIG. 1D) may include three sensing elements (such as Hall-effect sensors) configured to generate (output) corresponding signals (e.g., current outputs) that are proportional to and responsive to magnetic fields along the three different orthogonal axes (X, Y, and Z) of a three-dimensional spatial coordinate system.
[0028] In such embodiments, a spatial position vector (e.g., vector V, as illustrated in FIG. 1D) may be defined for each pairing of magnetic field generator 120 and magnetic field sensor 110 to represent Cartesian distances along three orthogonal Cartesian coordinate axes (X, Y, Z) between the magnetic field generator 120 and the magnetic field sensor 110 included in the pairing. The spatial position vector may also include angular orientations represented as angles (.alpha., .phi., .psi.) between the magnetic field axes of the magnetic field generator 120 (e.g., Hx, Hy, and Hz) and the sensing axes of the magnetic field sensor 110 (e.g., X, Y, and Z). The angles may alternatively be computed with respect to the three mutually perpendicular Cartesian planes (YZ, XZ, and XY) that are defined for the magnetic field sensor 110 or for the magnetic field generator 120.
[0029] In some embodiments, magnetic fields generated by multiple different magnetic generators 120 are distinguishable from one another, allowing a magnetic field sensor 110 to identify magnetic fields from the different magnetic generators, which allows separate determination of positions of the different magnetic field generators 120. The magnetic fields generated by different magnetic field generators 120 are distinguished from each other using time division multiplexing (TDM) or frequency division multiplexing (FDM) in various embodiments. For example, a magnetic field generator 120, such as an electromagnet, generates a magnetic field by applying a driving current to magnetic field emitting coils of the magnetic field generator 120. Using TDM, the driving current is applied sequentially to the magnetic field emitting coils of different magnetic field generators 120 at different times. Thus, when a magnetic field is generated by a particular magnetic field generator 120 at a particular time interval, the other magnetic field generators 120 do not emit magnetic fields during the particular time interval. Hence, the magnetic field sensed by the magnetic field sensor 110 during the particular time interval is unambiguously associated with the particular magnetic field generator 120. Using FDM, driving currents having different distinct frequencies are concurrently applied to each of the multiple magnetic field generators 120. The corresponding magnetic fields generated by the multiple magnetic field generators 120 are distinguishable from each other because of the different frequencies of the driving currents applied to different magnetic field generators 120. In a FDM implementation, the magnetic field detected by a magnetic field sensor 110 includes multiple distinct frequency elements, each frequency element unambiguously associated a different emitting magnetic field generator 120.
[0030] As illustrated in FIG. 1D for magnetic field generator (MG) 120 with reference to magnetic field sensor (MS) 110, a spatial position vector (V) including the Cartesian distances (x, y, z) and angular orientations (.alpha., .phi., .psi.), can be computed based on the signals detected by the magnetic field sensor 110 responsive to the magnetic fields (Hx, Hy, Hz) generated by magnetic field generator 120.
[0031] In some embodiments, in a 3D coordinate system, spatial ambiguity in positions in the 3D sensor space (further explained above in conjunction with FIG. 1B) is resolved by performing 2D projections from the 3D space to a 2D magnetic field space. This 2D projection involves three unknown rotation angles and can be mathematically indicated as below:
T R , P , Y H = T R , P , Y ( H x H y H z ) = ( H r H 0 0 ) = ( M cos .theta. / 2 .pi. r 3 M sin .theta. / 4 .pi. r 3 0 ) ( 12 ) ##EQU00001##
where H is a sensor vector and T.sub.R,P,Y is a rotation matrix with three unknown variables R (Raw), P (Pitch) and Y (Yaw) corresponding to angular orientations (.alpha., .phi., .psi.), to project the 3D sensor space to the 2D magnetic-field space. As equation (12) is an under-constrained system, there are three equations (H.sub.x, H.sub.y, H.sub.z) for determining five unknown variables (R, P, Y, r, .theta.). In some embodiments, a searching process that determines a global optimal solution is used to solve for the unknown variables (e.g., R, P, Y).
[0032] FIG. 2 illustrates locations of multiple magnetic field generators (MG) 220-1, 220-2, 220-3, 220-4, 220-5 on different fingers of a user’s hand, and a three-axis magnetic field sensor 210-1 located on the palm of the user’s hand, and a three-axis magnetic field sensor MS 210-2 located on the palm of the user’s hand at a known distance, d, from magnetic field sensor MS 210-1, as well as another three-axis magnetic field sensor MS 210-3 that is located near the wrist of the user’s hand. In this configuration, the three-axis magnetic field sensor MS 210-1 and the other three-axis magnetic field sensor MS 210-2 have a known distance and angle relative to each other and do not experience rotational or translation movement relative to each other, allowing the three axis magnetic field sensor MS 210-1 and the other three-axis magnetic field sensor MS 210-2 to determine positions of magnetic field generators MG 220-1, 220-2, 220-3, 220-4, 220-5 as proxy locations of tracked points using triangulation. Thus, the described configuration allows determination of the distance and angle of each of the magnetic field generators MG 220-1, 220-2, 220-3, 220-4, 220-5 with respect to either or both of the magnetic field sensors MS 210-1 and MS 210-2. In some embodiments, the three-axis magnetic field sensor MS 210-1 and the other three-axis magnetic field sensor MS 210-2 are positioned on a user’s hand so one of the three-axis magnetic field sensors MS 210-1 is located on the palm of the user’s hand at less than a threshold distance from the fingers on the palm of the user’s hand, while the other three-axis magnetic field sensor MS 210-2 is located on the palm of the user’s hand at less than a threshold distance from the user’s wrist. Such a configuration of the magnetic field sensors MS 210-1 and 210-2 provides an improved r signal to noise ratio when calculating positions of the magnetic field generators MG 220-1, 220-2, 220-3, 220-4, 220-5 using triangulation. In some embodiments, the two magnetic field sensors MS 210-1 and MS 210-2 are poisoned diagonally across the palm of the user’s hand from each other, as shown in FIG. 2. In some embodiments, a transceiver comprising a three-axis magnetic field sensor and a single axis magnetic field generator is used in place of one or more of magnetic field sensor MS 210-1, 210-2. In these embodiments, the three-axis magnetic field sensor MS 210-3 that is located at the wrist of the user’s hand may be used to determine the position of the tracked location of the transceiver positioned at a location of magnetic field sensor 210-1 or of magnetic field sensor 210-2, providing information about the relative palm position, also termed the “hand position,” relative to the user’s wrist. In some embodiments, the magnetic field generators located at the fingers 220-1, 220-2, 220-3, 220-4, and 220-5 may also be used to determine the “hand position” in a configuration where two three-axis magnetic field sensors are located at the user’s wrist. In some embodiments, the magnetic field sensors MS 210-1, MS 220-2, and MS 210-3, and the magnetic field generators 220-1, 220-2, 220-3, 220-4, and 220-5, as shown in FIG. 2, are coupled to or included within a wearable glove to be worn on the user’s hand.
[0033] In some embodiments, as further described above in conjunction with FIGS. 1A-1D, the magnetic field sensors MS 210-1, MS 210-2, and MS 210-3 include one or more sensing elements. For example, the magnetic field sensors may include a plurality of sensing elements having predefined, known, or solvable relative positions, as further described above with reference to sensing elements S1, S2, S3, and S4 of the coordinate system 130 of FIG. 1C.
[0034] In some embodiments, each of the magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5, as shown in FIG. 2 are single axis magnetic field generators. These magnetic field generators may each be an actively driven transmitter using alternating current signals to generate the magnetic fields, such as an electromagnet. In some embodiments, when the magnetic field generator is located in combination with the magnetic field sensor as part of a transceiver, the magnetic field generator may be an actively driven single axis transmitter. In alternative embodiments, a single magnetic field generator may include one or more programmable magnets (e.g., polymagnets) with programmable, controllable, and/or reconfigurable magnetic properties. In some embodiments, each magnetic field generator, 220-1, 220-2, 220-3, 220-4, 220-5, includes one or more electromagnets that can be independently actuated based on current provided through a current carrying coil to generate directional magnetic field components (H1x, H1y, H1z; … , H5x, H5y, H5z;). The magnetic fields for each of the magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5 can also be expressed as vector components (Hr and H.theta.) as described above in conjunction with FIGS. 1A-1B.
[0035] Furthermore, in embodiments where the magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5 are actively driven transmitters, such as electromagnets, as noted previously, the magnetic fields generated by each electromagnet may be distinguished from magnetic fields generated by the other electromagnets by controlling one or more attributes (e.g., frequency, timing, modulation codes/patterns) of the stimulating current provided to different electromagnets. For example, each electromagnet may be stimulated at a different frequency for disambiguation (based on frequency division multiplexing) from other electromagnets. Accordingly, signals detected by the magnetic field sensor 210 are uniquely associated with a particular electromagnet based on a detected frequency of the signals. Alternatively, each electromagnet may be stimulated at a different time for disambiguation (e.g., time division multiplexing with a common frequency) from other electromagnets. Accordingly, signals detected by the magnetic field sensor MS 210-1 and the other magnetic field sensor MS 210-2 may be uniquely associated with a particular electromagnet based on a time interval when the magnetic field sensor MS 210-1 or the other magnetic field sensor MS 210-2 detected the signals.
[0036] In the example of FIG. 2, one or more magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5 are positioned on tracked points at one or more fingertips of a hand, magnetic field sensors MS 210-1 and MS 210-2 are at predefined locations (e.g., at the palm of the hand as shown, at a fixed distance from each other), and another magnetic field sensor 210-3 is positioned at another predefined location (e.g., at the wrist of the hand). The magnetic field sensor MS 210-1 outputs a signal (or combination of signals) responsive to magnetic fields generated by each magnetic field generator 220-1, 220-2, 220-3, 220-4, 220-5 detected by the magnetic field sensor MS 210-1. Spatial position vectors V1, V2, … , V5 are computed, as described with reference to FIGS. 1C and 1D above, from the signal output by the magnetic field sensor MS 210-1, in response to magnetic fields generated by various magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5. For example, spatial position vector V1 corresponds to an output from the magnetic field sensor MS 210-1 based on a magnetic field generated by magnetic field generator 220-1; similarly, spatial position vectors V2, V3, V4, V5 correspond to outputs from the magnetic field sensor MS 210-1 based on a magnetic field generated by magnetic field generators 220-2, 220-3, 220-4, and 220-5, respectively. The computed spatial position vectors V1, V2, … , V5 are used to determine the position of each of the corresponding magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5, relative to the location of the magnetic field sensor MS 220-1. These positions are used as proxy positions for each of the corresponding different fingertips, with reference to the (known) position of the magnetic field sensor 210-1. Similarly, the other magnetic field sensor MS 210-2 outputs a signal (or combination of signals) responsive to magnetic fields generated by each magnetic field generator 220-1, 220-2, 220-3, 220-4, 220-5 detected by the other magnetic field sensor MS 210-2. Spatial position vectors V1’, V2’, … , V5’ (not shown) are computed, as described with reference to FIGS. 1C and 1D above, from the signal output by the other magnetic field sensor MS 210-2 in response to magnetic fields generated by various magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5. The computed spatial position vectors V1’, V2’, … , V5’ are used to determine the positions of each of the corresponding magnetic field generators 220-1, 220-2, 220-3, 220-4, 220-5, relative to the position of the other magnetic field sensor MS 210-2. These positions are used as proxy positions of each of the corresponding different fingertips, with reference to the (known) position of the other magnetic field sensor 210-2. Thus, having two magnetic field sensors located at the palm of the hand at a known fixed distance from each other enables determination of the positions of the tracked points using triangulation. Furthermore, if a magnetic field sensor is located at an HMD or at a nearby stationary known location, the locations of the tracked points may be determined relative to the HMD or the stationary location. In embodiments where the magnetic field sensor 210-3 is used to sense the magnetic field generated by a transceiver co-located on the palm with the magnetic field sensor 210-1 or with the other magnetic field sensor 210-2, the position of the tracked point on the palm may be determined with respect to the magnetic field sensor 210-3. In the embodiments described herein, the magnetic field sensors located on the palm, wrist, and the stationary locations, or otherwise, may be one-, two-, or three-axis sensors.
Hand Gesture and Hand Position Recognition
[0037] FIGS. 3A and FIG. 3B illustrate embodiments comprising multiple magnetic field generators and multiple magnetic field sensors. As shown in FIG. 3A, magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5 are located near the fingertips of a hand, so determined positions for magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5 act as proxies for positions of the fingertips near which magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5 are located. In some embodiments, the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5 are single-axis magnetic field generators, generating magnetic fields along a single axis of a known coordinate system. In some embodiments, each of the magnetic field generators may employ FDM, using stimulating currents of different frequencies. In some embodiments, the magnetic field generators may employ TDM, using stimulating currents of a single frequency applied sequentially across the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5.
[0038] Magnetic field sensor 310-1 senses the magnetic fields transmitted by the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5. In some embodiments, the magnetic field sensor 310-1 is a three-axis receiver. Similarly, magnetic field sensor 310-2 senses the magnetic fields transmitted by the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5, and in some embodiments, may be a three-axis receiver. As shown in the example of FIG. 3A, the magnetic field sensor 310-1 is located at or near the palm of the hand in a predetermined position; however, in other embodiments, the magnetic field sensor 310-1 may be located at any suitable predetermined location. Magnetic field sensor 310-2 is located at a fixed known position relation to magnetic field sensor 310-1. In various embodiments, the three-axis magnetic field sensor MS 310-1 and the other three-axis magnetic field sensor MS 310-2 are positioned so one of the three-axis magnetic field sensors MS 310-1 is located on the palm of the user’s hand at less than a threshold distance from the fingers on the palm of the user’s hand, while the other three-axis magnetic field sensor MS 310-2 is located on the palm of the user’s hand at less than a threshold distance from the user’s wrist. Such positioning of the magnetic field sensors MS 310-1 and 310-2 improves a signal to noise ratio when calculating positions of the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5 using triangulation. As depicted in FIG. 3A, the magnetic field sensor 310-3 is located at or near the wrist of the hand in a predetermined position; however, in other embodiments, the magnetic field sensor 310-3 may be located at any suitable predetermined position. In some embodiments, the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5, and the magnetic field sensors 310-1, 310-2, and 310-3 are coupled to a wearable glove device configured to be worn around the hand of a user.
[0039] In the example of FIG. 3A, in alternative embodiments, additional field generators 325 may be additionally located at or near a middle of the lower section of the fingers on a user’s hand, such as shown in FIG. 3A. This placement of the additional magnetic field generators 325 allows determination of information describing bending of the individual fingers relative to the magnetic field sensors 310-1, 310-2. Similar to the magnetic field generators 320-1, 320-2, 320-3, 320-4, 320-5, the additional field generators 325 may also be coupled to a wearable glove device configured to be worn around the user’s hand. In some embodiments, magnetic field generators may be located at various locations of a user’s body corresponding to points being tracked.
……
……
……
