Ultraleap Patent | Algorithm enhancements for haptic-based phased-array solutions

where x is a complex-valued coefficient that fully describes the sinusoidal signal to drive each transducer with.

An illustrative example system involving an array of seven transducers detailing physical transducer activations and in-flight waves over fifteen time steps of the solution method described are shown in FIGS. 16A, 16B, 16C, 16D, 16E.

FIGS. 16A, 16B, 16C, 16D, 16E show a series of fifteen snapshots at each time step of the behavior of an illustrative system and the state of the waves in the air. The boxes across the center represent the seven transducing elements in the system. There are two incomplete control trajectories described, the longer of which moves through the transducer array and into a virtual space behind. Each sub-figure describes the actuation of transducing elements resulting from solver calculations through time and state of the physical system during each small time window.

In further detail, this simulation models a line of seven transducers and the steps that they will take to actuate four real control points over 3 time intervals: point “a” at t=0, points “b” and “c” at t=1, and point “e” and t=2. The transducers propagate waves towards the top of the frame. The transducers will also be actuated to produce virtual control point “d” at time t=2 and virtual control point “f” at time t=3.

As shown by the key in each of the FIGS. 16A, 16B, 16C, 16D, 16E, the six different types of shading shown for the transducers, propagating waves and boxes at the control point correspond to actions related to the actuation of each of the six control points at their respective times. At some intervals, there is an overlap of two control point-related actions, which produces a cross-hatched pattern comprised of two overlapping shading types.

Beginning at t=−5 1401, a basis function including the leftmost transducer is generated to add to the “a” point, with the leftmost transducer being consequently actuated.

At t=−4 1402, a wave begins propagating towards the “a” point from the originated location at the leftmost transducer, which is also now used in a second basis function for point “b”. The rightmost transducer starts being included in both a third basis function for point “c” and the fourth basis function for the point “e”. These four basis functions are then solved for to find coefficients describing how to create the extra output at the point “a”, and the new output at points “b”, “C” and “e”.

At t=−3 1403, different transducers are used to build the four basis functions for the “a” point at t=0 and the “b” and “c” points at t=1 and the “e” point at t=2.

At t=−2 1404, all four basis functions are being solved for and waves from previous time steps are in flight to all four points “a”, “b”, “c” and “e”.

At t=−1 1405, no more transducers are able to contribute to the “a” point, and so this is no longer included in the basis set, reducing the number of basis function in the solutions to three for points “b”, “c” and “e”.

At t=0 1406, the waves in flight during the previous ticks coalesce to a focus at point “a” and the only point that may still be affected is the point “b”, for which three transducers are actuated in a single basis function.

At t=1 1407, the two t=1 points “b” and “c” are actuated using waves that are already in flight.

At t=2 1408, the last real point “e” is actuated at the same time as a virtual point “d” behind the array, which is only notionally ‘actuated’ as no physical waves or transducers are involved.

At t=3 1409, the outgoing waves from the virtual t=2 point break across the surface of the transducer array in the form of a negative time-of-flight basis function. The virtual point “f” is also ‘actuated’.

At t=4 1410, the t=2 “d” point continues to generate waves across the surface of the array, while notionally virtual wavefronts are moving towards the surface of the array from the t=3 “f” point, but no real waves from the t=3 “f” point exist yet.

At t=5 1411, both basis functions for both virtual points “d” and “f” are solved for in reverse time to ensure the phase accuracy and consistency of their respective signals.

At t=6 1412 and t=7 1413, these two basis functions continue to be solved for as subsequent transducers generate more of the simulated spherical wavefronts for both of the virtual points.

At t=8 1414, the last leftmost transducer is included in the final spherical wavefront and at t=9 1415, no more wavefronts can be produced, the array having successfully rendered all the points “a”, “b”, “c”, “d”, “e” and “f”.

The system described above is illustrative. Similar system using a different number and layout of transducers and a different number and layout of control points may be created.

K. Applications

By using a system that accounts for the differences in time-of-flight between transducers and specified spatial positions along a trajectory (that may be a single point), a series of such virtual phase-corrected point sources may be generated (which may be moving in time). In the limit, this enables any form of spatial speaker arrangement to be recreated using parametric audio, which potentially may be used to recreate any speaker setup with high fidelity.

This system is not possible with audio speakers as the reconstruction of point sources is prevented in these cases by the wavelength of the sound, without consideration of the phase adjustment. Parametric audio on the other hand is limited primarily by the ultrasonic wavelength, and thus is capable of creating sound sources that are significantly more point-like and precise than any producible by means utilizing audible sound only.

However, there is a further corollary, as these sound sources do not have physical presence and can be moved and placed arbitrarily. This allows a scene of sound to be decomposed as a set of point sources and created not as channel of audio played on speakers, but as audio streams that are emitted by and correspond to any number of point virtual sources that may move, diverge and regroup freely in three-dimensional space.

These points may use any of the existing modulation schemes for the reproduction of audio using an ultrasonic carrier as generated by a phased array system, as the particular phase and amplitude characteristics that give rise to the parametric audio are perfectly synchronized at these points. Further, as parametric audio is capable of reproducing higher audible frequencies, but has significantly more difficulty reproducing lower frequencies, this may be combined with other more traditional lower frequency generation hardware. In this case, it is beneficial that due to the size of the human head generally only frequencies that can be substantially reproduced by parametric audio hardware carry sufficient phase cues that can be interpreted by humans to infer spatial positions. In this case, taking the weighted signal of the low frequencies required across each of the virtual sources and using a different method of low frequency reproduction such as a traditional speaker may be beneficial.

To find points in space that may be reproduced as point sources, a source separation algorithm may be applied to the signal resulting from a microphone array. By alternating between cross-correlation and other methods to identify time of flight for individual signals, and source separation algorithms such as principal component analysis (PCA) or non-negative matrix factorization, both a suitable position for the point source and the desired audio stream for the point source to emit may be found. By constructing an objective function whose aim is to recover the original sound field or at least the measurements of the field produced by a microphone array, these techniques may be iterated towards a solution. It may be necessary to consider the diffraction of the audio signals as they are slowly reconstructed through the action of the ultrasonic demodulation in space to achieve this, although the effects of the diffraction of these signals is expected to be small. On taking the entirety of the field, in the limit of many point sources, the original sound scene may then be reconstructed perfectly in front, behind or generally in the space of the ultrasonic phased array. This approach is like the way in which an illumination map of a three-dimensional scene in the ray-tracing of scene geometry in computer graphics can be reduced in complexity through discretization. By taking a continuous field and discretizing it into point sources that generate and recreate the field with somewhat reduced fidelity, but optimizing towards the perfect recreation of the field in the limit, the number of discrete calculations required to reach an arbitrary level of fidelity may be reduced.

VIII. Additional Illustrative Examples

The following numbered clauses show further illustrative examples only:

1. A method comprising:

b) at each of the plurality of vertices, evaluating in virtual space a level set function by:

i. determining an evaluation of the level set function at a point in virtual space corresponding to a point associated with a region with respect to an elements of the virtual scene having a haptic presence; and

ii. determining a zero value of the level set function at the point in virtual space;

c) sampling the level set function in space across a virtual representation of a surface of a human body part; and

d) mapping the level set function to a corresponding haptic effect to be generated in mid-air onto the corresponding position on the human body part.

2. The method as in claim 1, wherein when the level set function is negative, the level set function applies to points outside the region; and wherein when the level set function is positive, the level set function applies to points inside the region.

3. The method as in claim 2, wherein the region corresponds to the virtual scene.

4. The method as in claim 1, wherein when the level set function is positive, the level set function applies to points outside the region; and wherein when the level set function is negative, the level set function applies to points inside the region.

5. The method as in claim 4, wherein the region corresponds to the virtual scene.

6. The method of claim 1 wherein the gradients of the level set function are included in the parameters of the mapping the level set function to a corresponding haptic effect to be generated onto the corresponding position on the human body part.

7. The method of claim 1, wherein user-defined scalar fields are used to substantially provide extra haptic properties for the elements of the virtual scene having a haptic presence.

8. The method of claim 1, wherein user-defined vector field are used to substantially provide directional haptic properties for the elements of the virtual scene having a haptic presence.

9. The method of claim 1, wherein the one or more sampled parameters in the virtual space are combined into one or more curve segments which are then mapped to corresponding haptic effects.

10. The method of claim 1, wherein at least one sampled parameters in the virtual space are combined into at least one surface segment which are then mapped to corresponding haptic effects.

11. The method of claim 1, wherein the corresponding haptic effect is produced substantially using modulated ultrasound.

12. The method of claim 1, wherein the mesh function uses an octree-generated grid.

13. A method comprising:

ii) defining a plurality of control points to be experienced by at least one user, wherein each of the plurality of control points has a known spatial relationship relative to the transducer array;

iii) activating at least one of the plurality of transducers to produce the plurality of control points based on the location of at least one user.

14. The method as in claim 13 further comprising: using a k-means algorithm to divide the plurality of transducers into at least two transducer groups.

15. The method as in claim 13, moving at least one of the plurality of transducers to a spatial power-average location of the plurality of control points.

16. The method as in claim 13, wherein the plurality of control points comprise a control region, and wherein the location of the control region is determined in part by the location of the at least one user.

17. The method as in claim 16, wherein the location of the at least one user includes information related to a location of a body part of the at least one user.

18. The method as in claim 13, further comprising:

19. The method as in claim 18, wherein, the step of using a microphone to update the location of at least one user includes the use of a timestamp and a feedback control mechanism.

Further, improved algorithm techniques may be used for superior operation of haptic-based systems. An eigensystem may be used to determine for a given spatial distribution of control points with specified output the set of wave phases that are the most efficiently realizable. Reconstructing a modulated pressure field may use emitters firing at different frequencies. An acoustic phased-array device uses a comprehensive reflexive simulation technique. There may be an exchange of information between the users and the transducer control processors having the ability to use that information for optimal haptic generation shadows and the like. Applying mid-air haptic sensations to objects of arbitrary 3D geometry requires that sensation of the object on the user's hand is as close as possible to a realistic depiction of that object. Ultrasonic haptics with multiple and/or large aperture arrays have high-frequency update rates required by the spatio-temporal modulation. More efficient haptic systems require the prevention of a channel of audio unintentionally encoding phase information that may distort its perception.

IX. Grouping and Optimization of Phased Ultrasonic Transducers for Multi-Field Solutions

A. Transducer Improvement

An environment containing many transducers and potentially multiple users can create a challenging problem for producing mid-air haptic feedback. The system managing the content needs to determine, given one or multiple user relative positions and locations and the scene the users are involved in, a number of haptic control fields for haptic feedback and then how to activate the available transducers to achieve these fields. This could be associated with a number of virtual or holographic objects. Another possibility is a haptic effect generated by the interaction of two users, such as remote touch or induced effect (virtual ‘magic spells’ or etc.). By modulating these fields in space or time, a haptic effect can be generated on the user. This results in an immersive environment utilizing the sense of touch.

A system with multiple transducer arrays and multiple requested fields can become computationally challenging if every field is to be contributed to by every transducer in the system. Also, in a room-scale situation, users can interfere with fields by shadowing the fields produced by some transducers and not others. What is needed is a way to reduce the problem from one large system to many smaller, easily solved, systems and to attempt to optimize their relationships so that they minimally-interfere with each other.

Described herein is an algorithm which attempts to divide a large group of transducers into smaller groups and assign a small number of fields (ideally one) to each of them. Then the contribution of each group to every other field is calculated and optimized.

A phased array of ultrasonic transducers can produce an acoustic field by adjusting the phase and amplitude of the ultrasound emitted at each transducer. It is convenient to use complex notation where the emitted signal is the real part of a complex number given by X=Λ e^iϕ. We can modify the signal my multiplying by another complex number we will refer to as an activation coefficient. This can arbitrarily modify the phase or amplitude of the emitted signal. An acoustic field is defined by an activation vector X where each entry represents the complex activation for each transducer in an array.

A transducer has a defined acoustic radiation pattern p(x). This can be generated through measurement, theory, or some combination of the two. With this, we can define an activation matrix:

$A=\left[\begin{array}{ccc}{p}_1\left(x_1\right)& \dots & p_n\left(x_1\right)\\ ⋮& \ddots & ⋮\\ p_1\left(x_m\right)& \dots & p_n\left(x_m\right)\end{array}\right],$

where p_n is the pressure from transducer n and when multiplied by the activation vector X gives the pressure values at various points x₁, . . . , x_m. At this point an activation vector can be generated through the pseudo-inverse of A multiplied by a vector composed of the desired complex pressures at each point. Since the solution is not necessarily unique, there are various ways to select phases in order to generate optimal fields which is covered in other patents. In the context of large numbers of arrays and many points, however, it is not always computationally efficient to consider every transducer and every acoustic field simultaneously. Take, for instance, two arrays spaced far apart (˜meters) each with a nearby desired field. Since the fields and arrays are unlikely to interact or contribute to each other, a computationally efficient solution would assign each field to its closest array and ignore the other. This way, each array only needs to calculate an activation solution for its nearby field which may not need a matrix inversion, saving computation time. It is not difficult to evaluate the possible interaction between arrays and minimize it (described below). If the fields or arrays change in such a way that the minimum-interaction solution is no longer sufficient, the fields and arrays can be grouped together and solved using methods presented below.

A particular field of interest is that of a single focus point. The activation is readily calculated using p(x) by,

$X_{\mathrm{fp}}\left(x\right)=\left[\begin{array}{c}{p}_1^{*}\left(x\right)\\ ⋮\\ p_n^{*}\left(x\right)\end{array}\right]\frac{b}{{\sum }_k{❘"\backslash [LeftBracketingBar]"p_k\left(x\right)❘"\backslash [RightBracketingBar]"}^2},$

where p*(x) is the complex conjugate of the pressure activation and b is the desired complex pressure at the focal point x. This solution can be quickly calculated with an efficient pressure function. This represents one particular field which is commonly used for mid-air haptics. Other fields such as beams or vortices can be generated by other activation vectors and are similarly efficient to calculate.

The first step in creating an efficient activation coefficient generator for multiple fields from many transducers is to try and assign groups of transducers each to their own field. One assignment method is given by the following algorithm: First, assign each control field a characteristic point which when a specified amount of acoustic pressure is delivered, can be distributed with a specific activation solution into the desired field. This could be the spatial power average of the field or similar weighting. For some fields, a single point my pose a compromise between various structures within the field. This is acceptable as the field quality can be refined by iterative methods below. Second, step through each characteristic point and assign it the transducer capable of delivering the largest absolute pressure to the point. In many cases this will be the physically closest transducer (modified by an angle function if necessary). Cycle through the list in order and assign the closest unassigned transducer to each point. After each cycle, update the max pressure possible at each characteristic point ( ). Once the desired pressure at a point is achievable, stop assigning transducers to that point, but continue to cycle through the remaining points. When all points have enough transducers assigned to them to achieve the desired pressure at each point, then start assigning to all points again until all transducers have been allocated.

Some potential modifications to the assignment algorithm include:

2. Using a different function other largest absolute pressure for selecting transducers. For example, distance, acoustic intensity, or even potential particle velocity and direction could be used.

3. Lump transducers into groups by array or k-means or similar algorithm, then assign each point to the closest group or only consider transducers in the closest group until they are all assigned, then consider others.

4. Choosing a new transducer in each loop by the total shape of the sub-array its creating. For instance, only choosing new transducers that are physically touching the boundaries of the sub-array, and/or insisting on an approximate boundary function (circle, square, etc.). Some more-important points could take priority and swallow already assigned transducers from less-important points in order to maintain shape.

5. Including an “importance factor” where a given point could be given more than one transducer per assignment loop (specified by this factor).

6. Including another factor where after pressure-satisfying assignment is done the factor describes how many to assign per “leftover” step.

7. For leftover step, simply grow each subarray by a scaling factor until the edges run out of adjacent transducers

8. Defining division planes between two nearby points (say normal and equidistant on the line in between the two points) where each point is only given transducers on its side of the plane.

9. Defining a maximum distance/angle for assignment and disabling transducers completely which are unassigned due to this restriction.

10. A post-assignment scan which looks for undesired patterns (checker patterns, etc.) and reassigns transducers to remedy problems.

11. A post-assignment scan which looks for efficient trades such as a gain in pressure (closer distance) for one point and for an equal-pressure (or less loss than the gain) for the other (near-same distance). This can include swapping entire clusters of transducers which would be reassigned in groups.

12. Using a shadow or reflection function (described below)

After assignment a new matrix may be generated:

$R=\left[\begin{array}{ccc}{A}_1\left(x_1\right)X_1& \dots & A_l\left(x_l\right)X_l\\ ⋮& \ddots & ⋮\\ A_1\left(x_m\right)X_1& \dots & A_l\left(x_m\right)X_l\end{array}\right],$

where A_l(x_m)=|p₁(x_m), . . . , p_n(x_m)| is the pressure vector for group l (with n transducers) at investigation point x_m, and X_l is the activation vector for group l (determined by its assigned field associated with its characteristic point). When multiplied by vector ϕ which is the relative phases between investigation groups gives the actual realized complex pressure at each point. With careful selection of the investigation points, this can be used to evaluate the quality of the entire field.

The investigation points need not be the characteristic points and can be composed of multiple points for each field. For instance, for a field with both high and low regions (such as a tight beam), investigation points could be selected within the high-pressure region with a goal of evaluating to high-pressure and more points in the low-pressure region with the goal of evaluating to low-pressure. At this point the realized pressure can be evaluated for quality (desired pressure bounds, for instance) and if it passes, the activation solutions X₁ . . . X_l are accepted and sent to hardware to be realized. If the quality inspection does not pass, relative phases can be adjusted as a first step to improving the field.

One method to adjust phases is to power iterate on an R matrix using the original characteristic points as the investigation points, R_c. Power iteration with R_cϕ=λϕ, can be accomplished by dividing each entry in ϕ by corresponding diagonal entry in R_c on each iteration, this assures that ϕ represents the difference from baseline. If the characteristic point is a small pressure value, then care needs to be taken to assure stability (avoiding zeros for instance). After sufficient number of iterations (predetermined or adaptive), the solution then gives an optimized relative phase solution ϕ′=ϕ/λ. We then modify each individual group's activation solution by the associated entry in ϕ′ to give a final optimized solution. If a true Eigen value is not generated, the largest scaling on a per-entry basis of R_cϕ can be used. The final pressure values at each investigation point can be evaluated with Rϕ′. If still insufficient, grouping can be done with methods presented below.

Using the investigation points as basis for power iteration can be done with some modification to R. Any transducer group which has multiple investigation points needs to be padded to form,

$R^{\prime }=\left[\begin{array}{cccccccc}{A}_1\left(x_1\right)X_1& \dots & A_y\left(x_1\right)X_y& 0& \dots & 0& \dots & A_l\left(x_l\right)X_l\\ ⋮& \ddots & ⋮& \ddots & \ddots & ⋮& \ddots & ⋮\\ ⋮& \dots & A_y\left(x_{y1}\right)X_y& 0& \dots & 0& \dots & ⋮\\ ⋮& \dots & 0& A_y\left(x_{y2}\right)X_y& \dots & ⋮& \dots & ⋮\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮& \ddots & ⋮\\ ⋮& \dots & 0& \dots & \dots & A_y\left(x_{yz}\right)X_y& \ddots & ⋮\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮& \ddots & ⋮\\ A_1\left(x_m\right)X_1& \dots & A_y\left(x_m\right)X_y& 0& \dots & 0& \ddots & A_l\left(x_m\right)X_l\end{array}\right],$

where in this case, the transducer group y has multiple investigation points x_y1 through x_yz. The first column of the pad includes the group's contribution to all other investigation points (except those within group y), but the remaining columns are zero except for the given investigation point. Note that the rows outside the padding are not zeroed. This prevents ‘double counting’ the pressure supplied to investigation points from group y but still monitors the contribution of other groups to each of group y's investigation points. After each round of power iteration the entries of ϕ for control points x_y1 through x_yz will need to be adjusted to match each other as their phase relationship is locked through X_y. Various methods of performing this adjustment include setting it to the normalized sum of the entries, a weighted average, or a duplicate of the one with the maximum change in direction or magnitude. Just as above, after sufficient number of iterations, ϕ/λ is used to modify the each X_l. This includes using the adjusted value for any groups with multiple investigation points.

Power iteration can be done quickly, and the new solution can be evaluated by just one more multiplication. This phase adjustment step is computationally efficient and will likely improve the accuracy of pressure at each of the control points. In some cases, a specific phase might be necessary for one group. In this case, it can be adjusted back every iteration and the method should still converge to a reasonable solution.

If the resulting fields are out of user-desired pressure limits, we can generate a new solution by grouping problem points and their associated transducer group with adjacent points and their associated transducer group. By forming new, larger transducer groups and forcing a more sophisticated field solution will increase computational load but will also increase the quality of the solution. Grouping can be done via closest investigation points, closest characteristic point, some combination, or some other method. With the new group formed, a multi-field solution can be generated and optimized with any number of traditional methods including least-squares or power iteration. These solutions will generate activation vectors for the new groups which can then be applied to the overall field solutions with the methods above. The previous investigation points or characteristic points can be used to create a new R or new investigation points can be generated.

This process of evolution and grouping can be repeated if necessary and is detailed in FIG. 17. In step 1710, the system identifies n characteristic points and m investigation points. In step 1720, the system generates a grouping of transducers In step 1730, the system generates field activation vectors, pressure vectors and constructs R. In step 1740, the system pads R (if necessary) and uses the power iteration to generate ϕ′. In step 1750, the system evaluates the resulting investigation point pressures. In step 1760, the system determines if all fields are within pressure limits. If yes, the process is complete (step 1794). If no, the system proceeds to determine if more than one group exists (step 1770). If no, the process is complete (step 1794). If yes, in step 1780 the system proceeds to group problem points and transducers with their nearest neighbors. In step 1790, the system generates multi-field activation vector solutions for each new grouping. In step 1792, the system generates a new set of investigation points (if necessary) and activation and pressure vectors (if necessary) and generates a new matrix R.

Further, groups and relative phases do not need to be refreshed every cycle of the field update. Typically, a system will manipulate one field within a small region for a period of time before removing it or creating another distinct field. Grouping and phasing can remain fixed until some predetermined time limit, new fields are requested, or some field quality metric is reached. Grouping and phasing also need not be refreshed together. In one arrangement, power iteration on R (or R′) for relative phasing could be updated often (every cycle, for instance) using the previous solution to start the iteration. This maintains quality in a computationally efficient method. To prevent large jumps in case they occur, filtering or cross-fading could be implemented on ϕ′. Rϕ′ can be evaluated to check for solution quality. When investigation points are approaching limits or predicted to cross soon, a new grouping solution can be generated and merged into the system. The merging process would need to take the form of a cross-fader where one solution is decreased in amplitude as the new solution is increased. This would prevent any abrupt changes and associated problems. Alternatively, the system could fade out completely before bringing up the new solution. While safer, this could potentially cause a noticeable gap in the acoustic field. Splitting groups apart should be resisted as much as possible and require higher thresholds. It is advantageous to keep points grouped as multipoint solutions are more effective at creating the overall field and updating a multi-field solution is efficient compared to initial generation. When fields are no longer necessary, transducer groups which created those fields can be recycled in multiple ways. If the field was being generated by its own group, that group could be ramped to off and remain off until a re-grouping is requested. Alternatively, the unused group could be partitioned into its nearest neighbors. Following this the new groups formed would have to generate a new solution which would cross-fade in to include the new transducers. If the to-be-removed field was part of a multi-field group, its solution could ramp to zero and the entire group would then be dedicated to the remaining fields in that group until a re-group. In another arrangement, the one-field-less multi-field group could perform a re-grouping just within its own fields with the available transducers. If far more transducers are available than are necessary, they could then be assigned to adjacent groups which might be in greater need.

Acoustic prediction accuracy and therefore solution effectiveness and efficiency can be refined by tracking information. For instance, if the desired haptic is meant for a user's palm, which is facing down, transducers placed above the user will not be able to contribute to the palm control region (blocked by the top of the hand). We can use this information to refine both the grouping algorithm (do not assign any transducers above the hand to this field, regardless of distance), the investigation point matrix R (change contribution to this point to zero from groups with transducers all, or mostly, above the hand), and even individual pressure vectors (zero the pressure contribution from transducers above the hand). All 3 adjustments need not be applied simultaneously, as each can increasingly improve quality with increasing computation. Palm location and normal could be considered a first-order approach to the problem and is reasonably efficient for fields directed at the hand. Other information that could be shared includes body location and direction to preclude transducers behind the user which will likely be occluded.

A further refinement can be achieved by calculating detailed shadowing information. Assuming adequate location information, a shadow-map can be created with each characteristic point or investigation point considered a light source. At its most basic level, a shadow would count transducers which are occluded by part of a user from a direct-line path to a control point. Calculating this shadow-map can be hardware accelerated in the same manner as shadows are calculated in 3D-graphics including stencil buffers and projection approximations. As above, shadowed transducer information could be used to refine grouping by not including shadowed transducers in the group of the characteristic point whose “light” created the shadow. We can refine the R by zeroing (or proportionally reducing) the groups contribution to the characteristic point or investigation point based upon the shadowed percentage of that group. Lastly, we can refine the various pressure vectors by zeroing the contribution to a given investigation point from shadowed transducers. As above, not all refinements are necessary but each functions to improve quality of the solution. Further refinements could be applied to the shadow information including adding diffraction and scattering information and the contribution to R or A could model the actual response of the transducer to the region, including possible phase modifications. For complex control regions, this would factor into the complex acoustic contribution of the given transducer.

Certain stationary or movable objects in the area of the transducers could be used as reflective surfaces if they are sufficiently smooth and hard. These reflecting surfaces could, for instance, help the sound to get around a corner or could be mounted on a user to direct a field onto a dead zone at that user (such as a palm facing the users face where the reflective surface is located, say on a HMD). To calculate and develop groups, activation, and pressure vectors, the entire area of transducers will need to be duplicated and distorted about the reflected surface. A flat surface, for instance, will include a mirror of the transducer locations about the plane of the surface. Curvature can be included by including a displacement function based upon the distance from the reflective surface. Increasing orders of refinement can be included as the reflected surface gains complexity. Arbitrary surfaces can be simulated by straight-line ray-tracing from each transducer off the reflecting surface, placing the mirror-transducer at an equal distance from the average ray reflection. Even more sophisticated wave front simulations can be used to model the reflection if the computational overhead is available. A distance-based selection criterion could also ignore transducers if they are far enough from the reflector. Once a virtual set of mirror-transducers has been generated they are treated in the above algorithms as available transducers. Since they are not actually independent, when a mirror-transducer is assigned to a particular characteristic point, the real transducer is also assigned to that point. Likewise, when a real transducer is assigned, the mirror-transducer is also assigned. Care must be taken to only include transducers for points and any created pressure on the “real” side of the reflection. The pressure on the “virtual” side of the reflection is merely a construct to satisfy the boundary condition at the reflection. Pressure vectors are calculated for both mirror and real transducers independently but the final activation coefficient solution needs to hold the two fixed as they are actually only one transducer. Mirror and real transducers can also be shadowed independently. A ‘reflectiveness’ coefficient could modify the mirror-transducers pressure to model the lack of perfect reflection from the reflector. This improves and can enable alternate solutions with reflectors in the system.

Transducers are not necessarily stationary and the relative locations of each in the area needs to be continually updated to the control system to keep the calculated fields accurate. Transducers mounted on the users can provide acoustic fields for any control region(s) in the scene. Tracking information must be updated for each round of processing by the control system. In another arrangement, transducers could be mounted on devices designed to move in the scene (robot arms, on wheels, drones, etc.). Moving the transducers closer to the spatial power-average location of all characteristic points in the scene will likely make those transducers more effective. Generating and evaluating Rϕ for a variety of displacements could help to judge the best direction of motion. Alternatively, moveable transducers that are not necessary to achieve all requested control regions could be displaced towards regions of minimal coverage to anticipate possible new fields. If shadow information is being calculated, this could be included in an optimal-displacement algorithm such as moving the transducers so as to minimize shadowing.

Another challenge in this environment is continually updating the relative positions and orientations between the transducers and users. Ultrasonic arrays are able to precisely target inaudible sound. Using this, one could address user-mounted microphones in order to calibrate and update location information. Arrays could also address other array-mounted microphones to calibrate the relative location off all transducers in the space. This could be in the form of an unfocused pulse with encoded location information. Alternatively, a high-pressure point can be created and rapidly translated in air near the mic. When the microphone picks up a specified ultrasound level, a timestamp can be reported to the array (via sound or other electronic method). Referencing its history, the array has a calibration of its location relative to the mic. If the microphone information can be streamed to the transducer control system, a control point could be maintained on the user microphone through a variety of feedback control mechanisms, and a relative position continually updated. In another arrangement, multiple transducers could act as ranging devices and map the nearby environment. When a user is detected, the relative position could be communicated. In another arrangement, a pulse of structured ultrasound could be emitted periodically to broadcast the location of the array. The structured field would contain markers or tags unique to a given direction from the array and could also be configured to contain time-of-flight information. The receiving mic could decode this information to determine the relative location and orientation of the array.

B. Simulations

Simulations were performed using 2 separate focal point fields from a 256-transducer array divided in half at equal distances away from the array. R and ϕ′ were generated as described.

Shown in FIG. 1 is a graph 1810 showing a divided array with each half producing one focal point. Plotted is the pressure at one of those focal points denoted in the title. Goal pressure for each point was 1.8 kPa. Interference can be seen in the 3 deep troughs.

Shown in FIG. 18B is a graph 1820 showing the results of iterating on R and applying the solution. For both the solid-line curve and dashed-line curve, the interference-nulls are almost completely removed. For the dotted-dashed-line curve, the interference point improves from −800 Pa below goal to −500 Pa.

Shown in FIG. 18C is a graph 1830 showing the results of grouping the two points and using the entire array. This shows that for the dotted-line curve, if −500 Pa was not acceptable, the points could be grouped to generate an effective solution at z=0.22 m.

C. Additional Illustrative Examples

The following numbered clauses show further illustrative examples only:

1. A system for creating acoustic fields consisting of

b. Defining a plurality of fields having known relative positions to the transducers;

c. Assigning each field to a unique subset of the transducers;

d. Assigning a phase and amplitude to drive each transducer to produce the field it is assigned to.

2. A method as in paragraph 1 where the phase of each subset is adjusted in such a way that total field-to-field interference is minimized

3. A method as in paragraph 2 where the optimal phase is generated through a power iteration of a matrix consisting of the contribution of each group to every field.

4. A method as in paragraph 1 where each field is assigned a characteristic point which is used for assignment of each transducer

5. A method as in paragraph 4 where the optimal phase is generated through a power iteration of a matrix consisting of the contribution of each group to one or more characteristic points

6. A method as in paragraph 4 where the transducers are assigned to each field based upon the pressure that they are capable of generating at characteristic point

7. A method as in paragraph 1 where each field has a set of investigation points which are used to judge the quality of the field

8. A method as in paragraph 7 where the optimal phase is generated through a power iteration of a matrix consisting of the contribution of each group to one or more investigation points

9. A system for creating acoustic fields consisting of:

b. Defining an area of interest where acoustic fields are generated;

c. Defining one or more acoustic fields having known relative positions to the transducers;

d. Assigning a phase and amplitude to drive each transducer to produce the field(s);

e. Receiving the relative position, shape and orientations of object or users in the area of interest;

f. Using the field(s) as a virtual light source(s), collect shadow information from users and objects in the area;

g. Use the shadow information to modify phase and/or amplitude sent to one or more transducers.

10. A method as in paragraph 9 where the shadow information is used to modify calculations regarding pressure supplied by shadowed transducers to fields which supply the virtual lighting.

11. A method as in paragraph 9 where the shadow information is used to modify calculations regarding pressure supplied by shadowed groups to fields which supply the virtual lighting.

12. A method as in paragraph 9 where the shadow information is used to modify grouping of transducer to each field.

13. A system for creating acoustic fields consisting of:

b. Defining one or more acoustic fields having known relative positions to the transducers;

c. Modifying the position of one or more transducers to improve field quality

14. A method as in paragraph 13 where moving transducers is based upon the spatial power-average location of all characteristic points

15. A method as in paragraph 13 where moving transducers is based upon shadowing information

16. A method as in paragraph 13 where moving transducers is based upon acoustic field quality

17. A system for creating acoustic fields consisting of:

b. Defining one or more acoustic fields having known relative positions to the transducers;

c. Providing location, shape, and orientation information for one or more reflectors;

d. Defining a set of virtual transducers based upon the location, shape, and orientation of the reflector;

e. Using the position and orientation of the virtual transducers, assign a phase and amplitude to drive each transducer to produce the field(s);

18. A system of determining the position of a group of ultrasound transducers consisting of:

b. Providing a microphone which is sensitive to ultrasound transmitted from the transducers

c. Emitting a field from the transducers which when measured by the microphone can determine the position and orientation of the transducer array relative to the microphone

19. A method as in paragraph 18 where the field emitted from the transducers is a focus point located near the microphone

20. A method as in paragraph 18 where the field emitted from the transducers contains encoded information about the emitted direction.

21. A method as in paragraph 18 where the field emitted from the transducers contains encoded information about the emission time.

22. A method as in paragraph 19 where the focus point is translated in response to movement by the microphone.

X. Conclusion

While the foregoing descriptions disclose specific values of voltage, capacitance and current, any other specific values may be used to achieve similar results. Further, the various features of the foregoing embodiments may be selected and combined to produce numerous variations of improved haptic systems.

In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings.

Moreover, in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.

The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.